Skip to content
AI Book
← All books
Book · Intermediate · 50+ hours

Probability & Statistics for AI/ML

From Foundations to Advanced Statistical Inference

Master probability distributions and mathematical statistics from foundations to advanced inference. Learn Bayesian methods, information theory, and statistical learning for data science and machine learning.

24Chapters
135Sections
58hReading
13Parts
Start chapter 00Browse curriculum
Part IPrerequisites6Part IIProbability Basics12Part IIIDistributions16Part IVMultivariate11Part VLimit Theorems14Part VIEstimation21Part VIITesting12Part VIIIBayesian16Part IXInformation Theory6Part XMultivariate Analysis7Part XIRegression6Part XIIAdvanced ML8Part 13Applications0
Curriculum

24 chapters— in publication order.

Part I

Prerequisites

Mathematical foundations

  1. 00Prerequisites & Mathematical Foundations6 sections · 120m

    Review essential mathematics: set theory, combinatorics, calculus, and linear algebra

    0.1–0.6
    0.1Mode of Convergence and Stat Distributions0.2Set Theory Essentials0.3Combinatorics Review0.4Calculus Refresher0.5Linear Algebra Essentials0.6Python and NumPy Setup

Part II

Probability Basics

Core probability theory

  1. 02Random Variables6 sections · 120m

    Discrete and continuous random variables, PMF, PDF, and CDF

    2.1–2.6
    2.1Discrete Random Variables2.2Probability Mass Functions2.3Continuous Random Variables2.4Probability Density Functions2.5Cumulative Distribution Functions2.6Mixed Random Variables
  2. 03Expectation and Moments6 sections · 145m

    Expected value, variance, moment generating functions, and probability inequalities

    3.1–3.6
    3.1Expected Value - Definition and Properties3.2Variance and Standard Deviation3.3Higher Moments: Skewness and Kurtosis3.4Moment Generating Functions3.5Characteristic Functions3.6Probability Inequalities

Part III

Distributions

Probability distributions

  1. 04Discrete Distributions6 sections · 130m

    Bernoulli, Binomial, Poisson, Geometric, and other discrete distributions

    4.1–4.6
    4.1Bernoulli and Binomial Distribution4.2Geometric and Negative Binomial4.3Poisson Distribution and Process4.4Hypergeometric Distribution4.5Multinomial Distribution4.6Discrete Uniform Distribution
  2. 05Continuous Distributions10 sections · 225m

    Normal, Exponential, Gamma, Beta, and other continuous distributions

    5.1–5.10
    5.1Uniform Distribution5.2Normal Distribution - Deep Dive5.3Exponential Distribution5.4Gamma Distribution5.5Beta Distribution5.6Chi-Square Distribution5.7Student t-Distribution5.8F-Distribution5.9Log-Normal Distribution5.10Weibull Distribution

Part IV

Multivariate

Joint distributions and transformations

  1. 07Multivariate Distributions6 sections · 165m

    Joint distributions, covariance, correlation, and multivariate normal

    7.1–7.6
    7.1Joint PMF and PDF7.2Marginal and Conditional Distributions7.3Covariance and Correlation7.4Covariance Matrix - Deep Dive7.5Multivariate Normal Distribution7.6Conditional Distributions of MVN
  2. 08Transformations of Random Variables5 sections · 125m

    Functions of random variables, Jacobian method, and order statistics

    8.1–8.5
    8.1Functions of Random Variables8.2Jacobian Transformation Method8.3Sum of Random Variables8.4Order Statistics8.5Convolutions

Part V

Limit Theorems

Convergence and fundamental theorems

  1. 09Convergence Concepts7 sections · 155m

    Modes of convergence: in probability, almost surely, in distribution, and mean square

    9.1–9.7
    9.1Convergence in Probability9.2Almost Sure Convergence9.3Convergence in Distribution9.4Convergence in Mean Square9.5L¹ Convergence and Uniform Integrability9.6Relationships Between Convergence Modes9.7Glivenko-Cantelli Theorem
  2. 10Fundamental Theorems7 sections · 180m

    Law of Large Numbers, Central Limit Theorem, and related theorems

    10.1–10.7
    10.1Law of Large Numbers10.2Central Limit Theorem - Proof and Intuition10.3CLT Variants10.4Delta Method10.5Berry-Esseen Theorem10.6Slutsky Theorem10.7Concentration Inequalities

Part VI

Estimation

Point and interval estimation

  1. 11Point Estimation7 sections · 190m

    Properties of estimators: bias, variance, consistency, sufficiency, and completeness

    11.1–11.7
    11.1Decision Theory and Prediction11.2Estimators and Their Properties11.3Bias, Variance, and MSE11.4Consistency and Efficiency11.5Sufficiency and Minimal Sufficiency11.6Completeness and Ancillarity11.7Multivariate Estimation
  2. 12Methods of Estimation7 sections · 215m

    Method of Moments, Maximum Likelihood, Fisher Information, and EM Algorithm

    12.1–12.7
    12.1Method of Moments12.2Maximum Likelihood Estimation12.3Properties of MLE12.4Fisher Information and CRLB12.5Rao-Blackwell Theorem12.6EM Algorithm12.7Regularized MLE
  3. 13Interval Estimation7 sections · 180m

    Confidence intervals, bootstrap methods, and Bayesian credible intervals

    13.1–13.7
    13.1Confidence Intervals - Concepts13.2CI for Normal Parameters13.3Large Sample Confidence Intervals13.4Bootstrap Confidence Intervals13.5Credible Intervals13.6Prediction Intervals13.7Simultaneous Confidence Intervals

Part VII

Testing

Hypothesis testing

  1. 14Fundamentals of Testing5 sections · 125m

    Hypothesis testing framework, Type I/II errors, power, and p-values

    14.1–14.5
    14.1Hypothesis Testing Framework14.2Type I and Type II Errors14.3Power of a Test14.4P-Values - Proper Interpretation14.5Neyman-Pearson Lemma
  2. 15Common Statistical Tests6 sections · 150m

    Z-tests, t-tests, Chi-square tests, F-tests, and likelihood ratio tests

    15.1–15.6
    15.1Z-Tests and T-Tests15.2Chi-Square Tests15.3F-Tests15.4Likelihood Ratio Tests15.5Wald and Score Tests15.6Permutation Tests
  3. 16Multiple Testing and Modern Issues1 section · 30m

    Multiple comparisons, FDR, A/B testing, and sequential testing

Part VIII

Bayesian

Bayesian statistics

  1. 17Bayesian Foundations5 sections · 140m

    Bayesian paradigm, prior and posterior distributions, and conjugate priors

    17.1–17.5
    17.1Bayesian vs Frequentist Paradigm17.2Prior Distributions17.3Posterior Distributions17.4Conjugate Priors17.5Non-informative and Jeffreys Priors
  2. 18Bayesian Inference5 sections · 125m

    Bayesian point estimation, MAP, credible intervals, and model comparison

    18.1–18.5
    18.1Bayesian Point Estimation18.2Maximum A Posteriori18.3Bayesian Credible Intervals18.4Bayes Factors and Model Comparison18.5Empirical Bayes
  3. 19Computational Bayesian Methods6 sections · 185m

    Monte Carlo, MCMC, Metropolis-Hastings, Gibbs sampling, and variational inference

    19.1–19.6
    19.1Monte Carlo Integration19.2Markov Chain Monte Carlo19.3Metropolis-Hastings Algorithm19.4Gibbs Sampling19.5Variational Inference19.6Hamiltonian Monte Carlo

Part IX

Information Theory

Entropy and divergences

  1. 20Information Theoretic Foundations6 sections · 160m

    Entropy, cross-entropy, KL divergence, and mutual information

    20.1–20.6
    20.1Shannon Entropy20.2Cross-Entropy20.3KL Divergence20.4Mutual Information20.5Information Theory in ML Loss Functions20.6Maximum Entropy Principle

Part X

Multivariate Analysis

PCA and dimensionality reduction

  1. 21Multivariate Statistical Methods5 sections · 145m

    PCA, Factor Analysis, LDA, CCA, and ICA

    21.1–21.5
    21.1Principal Component Analysis21.2Factor Analysis21.3Linear Discriminant Analysis21.4Canonical Correlation Analysis21.5Independent Component Analysis
  2. 22Dimensionality Reduction Deep Dive2 sections · 55m

    Eigendecomposition, SVD, t-SNE, UMAP, and random projections

    22.1Eigenvalue Decomposition for Statistics22.2SVD and Statistical Applications

Part XI

Regression

Linear models and GLMs

  1. 23Linear Regression6 sections · 165m

    OLS theory, Gauss-Markov theorem, diagnostics, and regularization

    23.1–23.6
    23.1Simple Linear Regression23.2Multiple Linear Regression23.3OLS Theory23.4Gauss-Markov Theorem23.5Regression Diagnostics23.6Regularization: Ridge and Lasso

Part XII

Advanced ML

Stochastic processes and PGMs

  1. 25Stochastic Processes4 sections · 120m

    Markov chains, HMMs, Gaussian processes, and Poisson processes

    25.1Markov Chains25.2Hidden Markov Models25.3Gaussian Processes25.4Poisson Processes
  2. 26Probabilistic Graphical Models4 sections · 110m

    Bayesian networks, Markov random fields, and inference in graphical models

    26.1Bayesian Networks26.2Markov Random Fields26.3Inference in Graphical Models26.4Learning Graphical Model Structure

135 sections. Begin with one.

Chapter 0 — Prerequisites & Mathematical Foundations — is where every reader starts.

In progress — 135 of 175 lessons published

40 more sections are still being written and are not part of this count.

Start chapter 00All books