Skip to content
Premium Content

Multi-Query Attention (MQA)

Shazeer 2019 — shared K/V across all heads for fast inference

This chapter requires a subscription to access.

What you'll unlock:

  • 1. Multi-Query Attention (MQA)
Subscribe to Unlock

Already have an account? Sign in