Mesh TensorFlow: Model Parallelism Made Easier
翻译 - 网格TensorFlow:简化模型并行化
The C++ Standard Library for Parallelism and Concurrency
翻译 - C ++并行和并发标准库
Pipeline Parallelism for PyTorch
Minimalistic large language model 3D-parallelism training
An API for data parallelism in JavaScript
Examples of training models with hybrid parallelism using ColossalAI
Testing memory-level parallelism
A baseline repository of Auto-Parallelism in Training Neural Networks
Enhancements to tile cutter for parallelism and image format support
A Rust program that plots the Mandelbrot set, using various kinds of parallelism.
Official code for "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient"
Quickly build large-scale ElasticSearch indices by using the fault tolerance and parallelism of Hadoop