How Neural Networks Work

Part I · Chapter 00 · 7 sections · 139 min

How computers store data and how neural networks learn meaning from numbers

1. 0.1The Computer Only Sees Numbers24m
2. 0.2Objects in a Space, and the Rule That Moves Them22m
3. 0.3Four Objects and One Campus17m
4. 0.4How Real Data Becomes Arrays18m
5. 0.5The Network Input and Its Transformation17m
6. 0.6From Raw Arrays to Features23m
7. 0.7Arrays in Python and PyTorch18m

Part II · Chapter 01 · 3 sections · 37 min

The big picture of neural networks

1. 1.1Biological Inspiration and History12m
2. 1.2The Artificial Neuron15m
3. 1.3Types of Neural Networks Overview10m

Part II · Chapter 02 · 5 sections · 78 min

Setting up your toolkit for neural network development

1. 2.1Python Refresher for Neural Networks15m
2. 2.2Setting Up and Running the Code10m
3. 2.3Introduction to PyTorch18m
4. 2.4Tensors and Operations20m
5. 2.5Autograd Basics15m

Part II · Chapter 03 · 4 sections · 68 min

The essential math you need and nothing more

1. 3.1Vectors, Matrices, and Operations18m
2. 3.2Derivatives and Gradients20m
3. 3.3Probability Basics15m
4. 3.4The Chain Rule15m

Part III · Chapter 04 · 3 sections · 45 min

The simplest neural network

1. 4.1Single Neuron Model15m
2. 4.2Learning Algorithm18m
3. 4.3Limitations and the XOR Problem12m

Part III · Chapter 05 · 3 sections · 40 min

The non-linearity that makes networks powerful

1. 5.1Sigmoid, Tanh, and ReLU18m
2. 5.2Choosing the Right Activation12m
3. 5.3Implementing Activations in PyTorch10m

Part III · Chapter 06 · 3 sections · 45 min

How networks measure their mistakes

1. 6.1Regression Losses15m
2. 6.2Classification Losses18m
3. 6.3Custom Loss Functions in PyTorch12m

Part IV · Chapter 07 · 9 sections · 164 min

How data flows through a network

1. 7.1Sixteen Tiny Images and Your First Network20m
2. 7.2Bias, the Bend, and the Equation18m
3. 7.3The Forward Pass by Hand, Then Animated16m
4. 7.4Code, Run, Look Inside: A Saved Function22m
5. 7.5A Second Hidden Layer: 4 → 6 → 5 → 418m
6. 7.6Two Rooms, and What Ten Layers Do20m
7. 7.7From Input to Output18m
8. 7.8Matrix Multiplication in Networks16m
9. 7.9Building Forward Pass in PyTorch16m

Part IV · Chapter 08 · 7 sections · 138 min

The algorithm that makes learning possible

1. 8.1Backpropagation by Hand: 4 → 6 → 424m
2. 8.2The Same Block Backward, and 1, 10, 100, 1000 Steps20m
3. 8.3The Graph Behind Backpropagation26m
4. 8.4The Gradient Descent Idea15m
5. 8.5Backpropagation Algorithm20m
6. 8.6One SGD Step: Reasoning About Updates15m
7. 8.7Autograd and Gradient Checking18m

Part IV · Chapter 09 · 3 sections · 45 min

Smarter ways to update weights

1. 9.1SGD and Momentum18m
2. 9.2Adam and Its Variants15m
3. 9.3Learning Rate Scheduling12m

Part V · Chapter 10 · 3 sections · 45 min

Stacking layers for more power

1. 10.1Network Architecture Design15m
2. 10.2Building MLPs in PyTorch18m
3. 10.3Universal Approximation Theorem12m

Part V · Chapter 11 · 4 sections · 62 min

Everything you need for real training

1. 11.1Data Loading and Batching15m
2. 11.2The Training Loop20m
3. 11.3Validation and Testing15m
4. 11.4Monitoring with Metrics12m

Part V · Chapter 12 · 5 sections · 99 min

Reducing overfitting

1. 12.1Diagnosing Underfitting and Overfitting17m
2. 12.2Dropout and Weight Decay18m
3. 12.3Early Stopping and Data Augmentation18m
4. 12.4Upgrading the Tiny Network: What Each Fix Buys28m
5. 12.5Scaling to MNIST: the Embedding Space You Built18m

Part VI · Chapter 13 · 3 sections · 70 min

How networks see images

1. 13.1From Pixels to Features: Why CNNs?18m
2. 13.2The Convolution Operation Explained22m
3. 13.3Stride, Padding, Pooling & Receptive Fields30m

Part VI · Chapter 14 · 3 sections · 55 min

Classic and modern CNN designs

1. 14.1Building a CNN from Scratch22m
2. 14.2Classic Architectures: LeNet to ResNet18m
3. 14.3Transfer Learning with Pretrained Models15m

Part VI · Chapter 15 · 3 sections · 50 min

Build a complete image classifier

1. 15.1Dataset Preparation15m
2. 15.2Building and Training the Model20m
3. 15.3Evaluation and Interpretation15m

Part VII · Chapter 16 · 3 sections · 50 min

Networks that remember

1. 16.1Sequential Data and RNNs15m
2. 16.2Training RNNs: Backpropagation Through Time20m
3. 16.3The Vanishing Gradient Problem15m

Part VII · Chapter 17 · 4 sections · 95 min

Gated memory for longer sequences

1. 17.1LSTM Architecture35m
2. 17.2GRU Architecture22m
3. 17.3Implementing Sequence Models in PyTorch23m
4. 17.4From Recurrence to Attention15m

Part VII · Chapter 18 · 3 sections · 55 min

Build a sentiment analyzer

1. 18.1Text Preprocessing and Embeddings20m
2. 18.2Building an LSTM Classifier20m
3. 18.3Training and Evaluation15m

Part VIII · Chapter 19 · 3 sections · 42 min

Tips and tricks from the pros

1. 19.1Batch and Layer Normalization18m
2. 19.2Mixed Precision Training12m
3. 19.3Gradient Clipping and Accumulation12m

Part VIII · Chapter 20 · 4 sections · 109 min

When things go wrong and how to fix them

1. 20.1Common Training Problems28m
2. 20.2Visualization Techniques35m
3. 20.3Performance Optimization Tips32m
4. 20.4Where This Goes Next14m