All Posts - Wentao's Blog

All Posts

2025

Summary: Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve 08-24

Summary: SARATHI: Efficient LLM Inference by Piggybacking Decodes with Chunked Prefills 08-17

Summary: SGLang: Efficient Execution of Structured Language Model Programs 08-09

Bi-weekly Journal: Contributions to vLLM 08-09

Summary: MARLIN: Mixed-Precision Auto-Regressive Parallel Inference on Large Language Models 08-03

Summary: EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test 07-26

Summary: DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model 07-20

Summary: DeepSeek-V3 Technical Report 07-12

Summary: FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision 07-05

Summary: FLASHINFER: EFFICIENT AND CUSTOMIZABLE ATTENTION ENGINE FOR LLM INFERENCE SERVING 06-29

1
2
3
8