Probability & Statistics for AI/ML
From Foundations to Advanced Statistical Inference
Master probability distributions and mathematical statistics from foundations to advanced inference. Learn Bayesian methods, information theory, and statistical learning for data science and machine learning.
Prerequisites— Mathematical foundations.
Prerequisites & Mathematical Foundations
Review essential mathematics: set theory, combinatorics, calculus, and linear algebra
Probability Basics— Core probability theory.
Probability Fundamentals
Core probability concepts: sample spaces, events, conditional probability, and Bayes theorem
Random Variables
Discrete and continuous random variables, PMF, PDF, and CDF
Expectation and Moments
Expected value, variance, moment generating functions, and probability inequalities
Distributions— Probability distributions.
Discrete Distributions
Bernoulli, Binomial, Poisson, Geometric, and other discrete distributions
Continuous Distributions
Normal, Exponential, Gamma, Beta, and other continuous distributions
Special Distributions for ML
Dirichlet, Wishart, Laplace, mixture distributions, and exponential family
Multivariate— Joint distributions and transformations.
Multivariate Distributions
Joint distributions, covariance, correlation, and multivariate normal
Transformations of Random Variables
Functions of random variables, Jacobian method, and order statistics
Limit Theorems— Convergence and fundamental theorems.
Convergence Concepts
Modes of convergence: in probability, almost surely, in distribution, and mean square
Fundamental Theorems
Law of Large Numbers, Central Limit Theorem, and related theorems
Estimation— Point and interval estimation.
Point Estimation
Properties of estimators: bias, variance, consistency, sufficiency, and completeness
Methods of Estimation
Method of Moments, Maximum Likelihood, Fisher Information, and EM Algorithm
Interval Estimation
Confidence intervals, bootstrap methods, and Bayesian credible intervals
Testing— Hypothesis testing.
Fundamentals of Testing
Hypothesis testing framework, Type I/II errors, power, and p-values
Common Statistical Tests
Z-tests, t-tests, Chi-square tests, F-tests, and likelihood ratio tests
Multiple Testing and Modern Issues
Multiple comparisons, FDR, A/B testing, and sequential testing
Bayesian— Bayesian statistics.
Bayesian Foundations
Bayesian paradigm, prior and posterior distributions, and conjugate priors
Bayesian Inference
Bayesian point estimation, MAP, credible intervals, and model comparison
Computational Bayesian Methods
Monte Carlo, MCMC, Metropolis-Hastings, Gibbs sampling, and variational inference
Information Theory— Entropy and divergences.
Information Theoretic Foundations
Entropy, cross-entropy, KL divergence, and mutual information
Multivariate Analysis— PCA and dimensionality reduction.
Multivariate Statistical Methods
PCA, Factor Analysis, LDA, CCA, and ICA
Dimensionality Reduction Deep Dive
Eigendecomposition, SVD, t-SNE, UMAP, and random projections
Regression— Linear models and GLMs.
Linear Regression
OLS theory, Gauss-Markov theorem, diagnostics, and regularization
Generalized Linear Models
GLM framework, logistic regression, Poisson regression, and model selection
Advanced ML— Stochastic processes and PGMs.
Stochastic Processes
Markov chains, HMMs, Gaussian processes, and Poisson processes
Probabilistic Graphical Models
Bayesian networks, Markov random fields, and inference in graphical models
Statistical Decision Theory
Loss functions, risk, admissibility, and minimax decision rules
Applications— Deep learning and causal inference.
Statistics in Deep Learning
Weight initialization, batch normalization, dropout, and uncertainty quantification
Causal Inference
Potential outcomes, propensity scores, instrumental variables, and causal graphs
Putting It All Together
Statistical thinking for ML, common pitfalls, and further resources
Where the book lands in practice.
Statistics in Deep Learning
Weight initialization, batch normalization, dropout, and uncertainty quantification
Open chapterCausal Inference
Potential outcomes, propensity scores, instrumental variables, and causal graphs
Open chapterPutting It All Together
Statistical thinking for ML, common pitfalls, and further resources
Open chapter175 sections. Begin with one.
Chapter 0 — Prerequisites & Mathematical Foundations — is where every reader starts.