Skip to content
Premium Content

Automatic Differentiation - How Machines Compute Derivatives

The algorithm behind PyTorch and JAX: exact derivatives at machine precision from the computational graph, via forward and reverse mode

This chapter requires a subscription to access.

What you'll unlock:

  • 1. Three Ways to Get a Derivative: Numerical, Symbolic, and Automatic
  • 2. Dual Numbers: Forward-Mode Autodiff in One Idea
  • 3. The Computational Graph: Every Program Is a Chain of Simple Steps
  • 4. Forward Mode: Propagating Tangents Through the Graph
  • 5. Reverse Mode: Backpropagation as Adjoints
  • 6. Building a Tiny Autodiff Engine From Scratch
  • 7. Autodiff in Practice: Jacobians, Checkpointing, and Pitfalls
Subscribe to Unlock

Already have an account? Sign in