Summary: SGLang: Efficient Execution of Structured Language Model Programs
Summary for paper ‘SGLang: Efficient Execution of Structured Language Model Programs’
Bi-weekly Journal: Contributions to vLLM
My bi-weekly journal for contributions to vllm.
Summary: MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models
Summary for paper ‘MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models’
Summary: EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
Summary for paper ‘EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test’
Summary: DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Summary for paper ‘DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model’
Summary: DeepSeek-V3 Technical Report
Summary for paper ‘DeepSeek-V3 Technical Report’
Summary: FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
Summary for paper ‘FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision’
Summary: FLASHINFER: EFFICIENT AND CUSTOMIZABLE ATTENTION ENGINE FOR LLM INFERENCE SERVING
Summary for paper ‘FLASHINFER: EFFICIENT AND CUSTOMIZABLE ATTENTION ENGINE FOR LLM INFERENCE SERVING’
Summary: Training Compute-Optimal Large Language Models
Summary for paper ‘Training Compute-Optimal Large Language Models’
Summary: Outrageously Large Neural Networks: The Sparsely-Gated Mixture-Of-Experts Layer
Summary for paper ‘Outrageously Large Neural Networks: The Sparsely-Gated Mixture-Of-Experts Layer’