Premium Content

Information Theoretic Foundations

Entropy, cross-entropy, KL divergence, and mutual information

This chapter requires a subscription to access.

What you'll unlock:

  • 1. Shannon Entropy
  • 2. Cross-Entropy
  • 3. KL Divergence
  • 4. Mutual Information
  • 5. Information Theory in ML Loss Functions
  • 6. Maximum Entropy Principle
Subscribe to Unlock

Already have an account? Sign in