All Posts - Wentao's Blog

All Posts

2025

Summary: Training Compute-Optimal Large Language Models 06-22

Summary: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-Of-Experts Layer 06-15

Summary: DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale 06-08

Summary: FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning 05-28

2025 Technical Notes（3） 05-23

Summary: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness 05-22

Summary: DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving 05-17

Summary: Fast Inference from Transformers via Speculative Decoding 05-11

Summary: MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generation 04-29

Summary: Infinite Retrieval: Attention Enhanced LLMs in Long-Context Processing 04-27

1
2
3
7