Summary: ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
Summary for paper ‘ZeRO: Memory Optimizations Toward Training Trillion Parameter Models’
TVM: 1D convolution CPU Optimization
This blog demonstrates optimization techniques for 1D convolution using TVM, including parallelization, loop tiling, vectorization, and unrolling.
Summary: Communication-Efficient Learning of Deep Networks from Decentralized Data
Summary for paper ‘Communication-Efficient Learning of Deep Networks from Decentralized Data’
2025 Technical Notes(2)
Technical notes during 2025.
Summary: Large Scale Distributed Deep Networks
Summary for paper ‘Large Scale Distributed Deep Networks’
Summary: TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
Summary for paper ‘TVM: An Automated End-to-End Optimizing Compiler for Deep Learning’
Summary: TinyML
Summary for paper ‘TinyML: Current Progress, Research Challenges, and Future Roadmap’
Summary: NAS with RL
Summary for paper ‘Neural Architecture Search with Reinforcement Learning’
Summary: Learning both Weights and Connections for NNs
Summary for paper ‘Learning both Weights and Connections for Efficient Neural Networks’
Summary: Quantization and Training of Neural Networks
Summary for paper ‘Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference’