Summary: DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Summary for paper ‘DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model’
Summary: DeepSeek-V3 Technical Report
Summary for paper ‘DeepSeek-V3 Technical Report’
Summary: FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
Summary for paper ‘FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision’
Summary: FLASHINFER: EFFICIENT AND CUSTOMIZABLE ATTENTION ENGINE FOR LLM INFERENCE SERVING
Summary for paper ‘FLASHINFER: EFFICIENT AND CUSTOMIZABLE ATTENTION ENGINE FOR LLM INFERENCE SERVING’
Summary: Training Compute-Optimal Large Language Models
Summary for paper ‘Training Compute-Optimal Large Language Models’
Summary: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-Of-Experts Layer
Summary for paper ‘Outrageously Large Neural Networks: The Sparsely-Gated Mixture-Of-Experts Layer’
Summary: DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale
Summary for paper ‘DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale’
Summary: FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
Summary for paper ‘FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning’
2025 Technical Notes（3）
Technical notes during 2025 (3).
Summary: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
Summary for paper ‘FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness’