Premium Content

Multi-Token Prediction (MTP)

Why next-token prediction under-uses the forward pass, DeepSeek's sequential MTP, and speculative decoding at inference.

This chapter requires a subscription to access.

What you'll unlock:

  • 1. The Single-Token Prediction Bottleneck
  • 2. Naive Parallel MTP and Its Failure Mode
  • 3. Sequential Causal MTP: DeepSeek's Implementation
  • 4. MTP Training Objective and Ablations
  • 5. MTP as Speculative Decoding at Inference
Subscribe to Unlock

Already have an account? Sign in