数据并行（Data Parallelism）
参考文献： Team D, Majumder R, President V, et al. DeepSpeed: Extreme-scale model training for everyone \[J\]. Microsoft, 2020. Rajbhandari S, Rasley J, Ruwase O, et al. ZeRO: Memory optimizations Toward Training Trillion Parameter Models \[C\], 2020. IEEE, 2020-01-01. Ren J, Rajbhandari S, Aminabadi R Y, et al. ZeRO-Offload: Democratizing Billion-Scale Model Training \[J\]. arXiv.org, 2021. Rajbhandari S, Ruwase O, Rasley J, et al. ZeRO-Infinity: Breaking the GPU MemoryWall for Extreme Scale Deep Learning: SC’ 21, November 14–19, 2021, St. Louis, MO, USA ...