Skip to content
Premium Content

Sparse Attention — BigBird

BigBird's local/global/random graph and MoBA's query-dependent block routing

This chapter requires a subscription to access.

What you'll unlock:

  • 1. Sparse Attention — BigBird
  • 2. MoBA: Mixture of Block Attention
Subscribe to Unlock

Already have an account? Sign in