Premium Content

Long-Context Extension

Why extending context is non-trivial, and the YaRN technique used by DeepSeek V3 to reach 128K tokens.

This chapter requires a subscription to access.

What you'll unlock:

  • 1. Why Context Extension Is Hard
  • 2. NTK-Aware Scaling
  • 3. YaRN: Frequency-Domain Interpolation
  • 4. Evaluation at Long Context
Subscribe to Unlock

Already have an account? Sign in