Premium Content

Auxiliary-Loss-Free Load Balancing

Why MoE load imbalance is catastrophic, why auxiliary loss hurts quality, and how DeepSeek's bias-term solution resolves the tension.

This chapter requires a subscription to access.

What you'll unlock:

  • 1. The Routing Collapse Problem
  • 2. Auxiliary Loss Approaches and Their Cost
  • 3. Bias-Term Load Balancing: The DeepSeek Solution
  • 4. Sequence-Level Balance Loss
Subscribe to Unlock

Already have an account? Sign in