Uniform Distribution

Learning Objectives

By the end of this section, you will be able to:

1. Define the continuous uniform distribution $U(a, b)$ and understand its parameters
2. Interpret the uniform distribution as "equal probability everywhere" within bounds
3. Calculate probabilities using the PDF and CDF for any interval
4. Apply inverse transform sampling to generate samples from ANY distribution using uniform
5. Recognize uniform as the maximum entropy distribution for bounded support
6. Use uniform distribution in Monte Carlo simulation and numerical integration
7. Implement uniform distribution operations in Python

Deep Intuition: The Fairness Distribution

Think of it as "I have no reason to favor any value over another."

When you know a value lies somewhere in a range but have absolutely no information about where it's more likely to be, the uniform distribution is the only logical choice. It treats every point in the range with equal respect.

The Historical Principle: Insufficient Reason

In 1814, Pierre-Simon Laplace formalized the Principle of Insufficient Reason:

"When we have no information to distinguish between possibilities, we should assign them equal probabilities."

This principle led directly to the uniform distribution. It's not just a mathematical convenience - it's a statement about rational belief in the absence of information.

Why Do We Need the Uniform Distribution?

The uniform distribution serves three critical roles in probability and computing:

These three roles make uniform distribution appear everywhere:

What Data Can We Model?

What Does the Distribution Tell Us?

Let $X \sim U(a, b)$. Here's what each quantity means:

The PDF: A Perfect Rectangle

Interpretation: The probability density is constant everywhere within [a, b]. The height is 1/(b-a) because:

The CDF: A Linear Ramp

Interpretation: The CDF increases linearly from 0 to 1 across the interval. At any point, it tells you what fraction of the interval is below that point.

Exploring the Distribution

Use this interactive visualizer to explore how the uniform distribution behaves. Adjust the bounds a and b, and see how the PDF and CDF change:

What Do You Notice?

• Wider range → Lower PDF height: The probability gets "spread thinner" over a larger area
• PDF is always a rectangle: Height adjusts to keep area = 1
• CDF is always linear: Equal probability accumulation rate everywhere
• Mean is always centered: Exactly at (a+b)/2

Mathematical Derivation

Let's derive everything from first principles, understanding why each formula must be what it is.

Deriving the PDF from Fairness

Start with the requirement: equal probability density everywhere. This means f(x) = c (constant) for all x ∈ [a, b].

The total probability must be 1:

Therefore:

Deriving the CDF from the PDF

The CDF is the integral of the PDF from -∞ to x:

Deriving the Mean

The mean is exactly the midpoint! This makes perfect sense - the distribution is symmetric around the center.

Deriving the Variance

First, find $E[X^2]$:

Then use $\text{Var}(X) = E[X^2] - (E[X])^2$:

Key Properties

The Standard Uniform: U(0, 1)

The standard uniform U(0, 1) is special because:

The Inverse Transform Method

This is perhaps the most important application of the uniform distribution. It answers: How can we generate random samples from ANY distribution?

The Fundamental Theorem: If U ~ U(0, 1) and F is any CDF with inverse F⁻¹, then X = F⁻¹(U) has CDF F.

Why Does This Work?

Let X = F⁻¹(U) where U ~ U(0, 1). We want to prove X has CDF F:

The last step uses the fact that for U ~ U(0, 1), P(U ≤ p) = p!

Use this interactive demo to see the inverse transform method in action:

Examples of Inverse Transform

Monte Carlo Integration

One of the most powerful applications of uniform distribution is Monte Carlo integration - estimating integrals using random samples.

The Core Idea

Want to compute $\int_a^b g(x) \, dx$? Here's the trick:

So we can estimate the integral by:

1. Generate n samples U₁, U₂, ..., Uₙ ~ U(a, b)
2. Compute the average: $\bar{g} = \frac{1}{n}\sum_{i=1}^{n} g(U_i)$
3. Multiply by width: $\hat{I} = (b-a) \cdot \bar{g}$

By the Law of Large Numbers, this converges to the true integral as n → ∞!

Try this interactive demo that estimates π using Monte Carlo:

Why Monte Carlo Matters

• High dimensions: Traditional integration fails in high-D, but Monte Carlo keeps working
• Complex regions: Irregular integration domains are no problem
• Error rate: Error decreases as 1/√n regardless of dimension!

Real-World Applications

1. Random Number Generation

1import numpy as np
2
3# This is what the RNG actually produces
4u = np.random.random()  # U(0, 1) sample
5
6# Everything else is derived from it
7normal = np.random.randn()    # Uses Box-Muller on uniform
8exponential = np.random.exponential()  # Uses -ln(U)

2. Cryptography

3. A/B Testing

1def assign_variant(user_id):
2    # Hash gives uniform distribution
3    u = hash(user_id) / MAX_HASH
4    if u < 0.5:
5        return "control"
6    else:
7        return "treatment"

4. Simulation

AI/ML Applications

Uniform distribution is ubiquitous in machine learning, often working behind the scenes:

1. Weight Initialization

1import torch.nn as nn
2
3# Xavier uniform initialization
4nn.init.xavier_uniform_(layer.weight)
5
6# Equivalent to:
7# limit = sqrt(6 / (fan_in + fan_out))
8# W ~ U(-limit, limit)

2. Dropout

1def dropout(x, p):
2    mask = np.random.uniform(0, 1, x.shape) > p
3    return x * mask / (1 - p)

3. Data Augmentation

4. Hyperparameter Search

1# Learning rate: log-uniform (uniform in log space)
2log_lr = np.random.uniform(np.log(1e-5), np.log(1e-1))
3lr = np.exp(log_lr)
4
5# Dropout: uniform
6dropout = np.random.uniform(0.1, 0.5)
7
8# Hidden units: discrete uniform
9hidden = np.random.randint(64, 512)

5. Variational Inference

Python Implementation

Basic Operations

1import numpy as np
2from scipy import stats
3
4# Create uniform distribution U(2, 8)
5a, b = 2, 8
6uniform_dist = stats.uniform(loc=a, scale=b-a)  # NOTE: scale = b - a
7
8# PDF
9x = 5
10pdf_value = uniform_dist.pdf(x)
11print(f"f({x}) = {pdf_value:.4f}")  # 0.1667 = 1/(8-2)
12
13# CDF
14cdf_value = uniform_dist.cdf(x)
15print(f"F({x}) = {cdf_value:.4f}")  # 0.5 = (5-2)/(8-2)
16
17# Probability of interval
18prob = uniform_dist.cdf(6) - uniform_dist.cdf(4)
19print(f"P(4 ≤ X ≤ 6) = {prob:.4f}")  # 0.3333 = 2/6
20
21# Mean and variance
22print(f"Mean = {uniform_dist.mean():.4f}")  # 5.0
23print(f"Var = {uniform_dist.var():.4f}")   # 3.0
24
25# Generate samples
26samples = uniform_dist.rvs(size=10000)
27print(f"Sample mean: {samples.mean():.4f}")
28print(f"Sample var: {samples.var():.4f}")

Inverse Transform Sampling

1import numpy as np
2from scipy import stats
3import matplotlib.pyplot as plt
4
5# Generate standard uniform samples
6n = 10000
7u = np.random.uniform(0, 1, n)
8
9# Transform to exponential using inverse CDF
10# F(x) = 1 - e^(-λx), so F^(-1)(u) = -ln(1-u)/λ
11lambda_rate = 2.0
12exponential_samples = -np.log(1 - u) / lambda_rate
13
14# Verify: compare with scipy
15true_exponential = stats.expon(scale=1/lambda_rate).rvs(n)
16
17# Plot comparison
18fig, axes = plt.subplots(1, 2, figsize=(12, 4))
19
20axes[0].hist(exponential_samples, bins=50, density=True, alpha=0.7,
21             label='Inverse Transform')
22x = np.linspace(0, 5, 100)
23axes[0].plot(x, lambda_rate * np.exp(-lambda_rate * x), 'r-',
24             label='True PDF', linewidth=2)
25axes[0].legend()
26axes[0].set_title('Exponential from Uniform')
27
28axes[1].hist(true_exponential, bins=50, density=True, alpha=0.7,
29             label='scipy.stats')
30axes[1].plot(x, lambda_rate * np.exp(-lambda_rate * x), 'r-',
31             label='True PDF', linewidth=2)
32axes[1].legend()
33axes[1].set_title('Direct scipy Generation')
34
35plt.tight_layout()
36plt.show()

Monte Carlo Integration

1import numpy as np
2
3def monte_carlo_integrate(f, a, b, n=10000):
4    """Estimate integral of f from a to b using Monte Carlo."""
5    u = np.random.uniform(a, b, n)
6    return (b - a) * np.mean(f(u))
7
8# Example 1: Integral of sin(x) from 0 to π
9# True value: -cos(π) + cos(0) = 2
10estimate = monte_carlo_integrate(np.sin, 0, np.pi, n=100000)
11print(f"∫sin(x)dx from 0 to π: {estimate:.6f} (true: 2.0)")
12
13# Example 2: Estimate π using quarter circle
14# Area of quarter circle = π/4, so π = 4 * (fraction of points in circle)
15n = 100000
16x = np.random.uniform(0, 1, n)
17y = np.random.uniform(0, 1, n)
18in_circle = (x**2 + y**2) <= 1
19pi_estimate = 4 * np.mean(in_circle)
20print(f"π estimate: {pi_estimate:.6f} (true: {np.pi:.6f})")
21
22# Example 3: Higher-dimensional integral
23# ∫∫∫ e^(-(x² + y² + z²)) dx dy dz over [-1, 1]³
24def integrand(xyz):
25    return np.exp(-np.sum(xyz**2, axis=1))
26
27n = 100000
28samples = np.random.uniform(-1, 1, (n, 3))
29volume = 2**3  # volume of [-1, 1]³
30estimate = volume * np.mean(integrand(samples))
31print(f"3D Gaussian integral estimate: {estimate:.6f}")

Common Pitfalls

1from scipy import stats
2
3# For U(2, 8):
4correct = stats.uniform(loc=2, scale=6)   # ✓ scale = 8 - 2 = 6
5wrong = stats.uniform(2, 8)               # ✗ This gives U(2, 10)!
6
7# Always verify:
8print(correct.mean())  # Should be 5.0 for U(2, 8)

1import numpy as np
2from scipy import stats
3
4# NumPy: uses (low, high) directly
5np.random.uniform(2, 8)  # U(2, 8) ✓
6
7# SciPy: uses (loc, scale)
8stats.uniform(loc=2, scale=6)  # U(2, 8) ✓
9stats.uniform(2, 8)  # U(2, 10) ✗ - NOT what you expect!

Test Your Understanding

Check your understanding of the uniform distribution with these practice problems:

Summary

The uniform distribution is deceptively simple yet profoundly important. It is the foundation upon which all random number generation is built.

Key Formulas

Key Takeaways

1. Uniform distribution models "equal probability everywhere" within bounded support
2. Every computer RNG produces uniform first - it's the universal generator
3. Inverse transform sampling: U(0,1) + inverse CDF = any distribution
4. Monte Carlo integration uses uniform samples to estimate integrals
5. Uniform maximizes entropy for bounded support - the "least informative" distribution
6. In ML: weight initialization, dropout, data augmentation all use uniform

Coming Next: In the next section, we'll explore the Normal Distribution - the famous bell curve that arises from the Central Limit Theorem and dominates natural phenomena. You'll see why it's often called "the most important distribution in all of statistics."