Boo-AI — Master Artificial Intelligence by Building from Scratch

Learning Objectives

By the end of this section, you will be able to:

State the formal definition of the definite integral as a limit of Riemann sums
Explain what it means for the Riemann sum to converge as the partition becomes infinitely fine
Interpret the integral notation $\int_a^b f(x)\,dx$ in terms of limits
Visualize how rectangles "fill in" the area under a curve as $n \to \infty$
Recognize when a function is integrable and when it is not
Apply the limit definition to compute simple integrals
Connect this definition to numerical integration methods used in scientific computing and machine learning

The Big Picture: Exact Area from Approximation

"The definite integral is the limit of a sum — infinitely many infinitely thin rectangles that together capture the exact area under a curve."

In previous sections, we saw that Riemann sums approximate the area under a curve using rectangles. The more rectangles we use, the better our approximation becomes. But there's something much deeper going on here: as the number of rectangles approaches infinity, the approximation becomes exact.

The Central Question

If $L_n$ represents a Riemann sum with $n$ rectangles, and this approximation gets better as $n$ increases, does the sequence $L_1, L_2, L_3, \ldots$ converge to a specific number?

If so, that limiting value is what we call the definite integral.

The definite integral $\int_a^b f(x)\,dx$ is not just "the area under the curve." It is precisely defined as a limit — the value that Riemann sums approach as the partition of $[a,b]$ becomes infinitely fine.

Historical Context: From Archimedes to Riemann

The quest to compute areas precisely goes back over 2,000 years. The journey from intuition to rigor spans many brilliant minds.

Archimedes (~250 BCE)

Used the method of exhaustion to find areas by inscribing polygons with more and more sides. He essentially used limits before limits were formally defined!

Leibniz & Newton (1680s)

Developed calculus and recognized that integration is the inverse of differentiation. They used infinitesimals — infinitely small quantities.

Bernhard Riemann (1854)

Put integration on rigorous footing using limits. His definition replaced vague infinitesimals with precise limit-based reasoning.

Why Riemann?

Riemann's contribution was to define exactly when a function is "integrable" — when the limit of Riemann sums exists and is unique. His framework handles functions that earlier methods could not, and it provides the logical foundation for all of modern analysis.

From Sums to Limits: Building the Definition

Let's carefully construct the definite integral, step by step.

Step 1: Partition the Interval

Given an interval $[a, b]$ , we divide it into $n$ subintervals:

a = x_0 < x_1 < x_2 < \cdots < x_{n-1} < x_n = b

For equal subintervals, each has width $\Delta x = \frac{b - a}{n}$ .

Step 2: Choose Sample Points

In each subinterval $[x_{i-1}, x_i]$ , we choose a sample point $x_i^*$ where we evaluate the function:

Method	Sample Point	Formula
Left endpoint	x*ᵢ = xᵢ₋₁	f(a + (i-1)Δx)
Right endpoint	x*ᵢ = xᵢ	f(a + i·Δx)
Midpoint	x*ᵢ = (xᵢ₋₁ + xᵢ)/2	f(a + (i - ½)Δx)

Step 3: Form the Riemann Sum

The Riemann sum is the sum of all rectangle areas:

S_n = \sum_{i=1}^{n} f(x_i^*) \cdot \Delta x

Each term $f(x_i^*) \cdot \Delta x$ is the area of one rectangle — height times width.

Step 4: Take the Limit

The definite integral is defined as the limit of these sums as $n \to \infty$ :

\int_a^b f(x)\,dx = \lim_{n \to \infty} \sum_{i=1}^{n} f(x_i^*) \cdot \Delta x

As $n \to \infty$ , $\Delta x = \frac{b-a}{n} \to 0$ . The rectangles become infinitely thin and infinitely numerous, perfectly capturing the area.

The Formal Definition: The Definite Integral

Definition: The Definite Integral

Let $f$ be a function defined on $[a, b]$ . The definite integral of $f$ from $a$ to $b$ is:

\int_a^b f(x)\,dx = \lim_{\|\mathcal{P}\| \to 0} \sum_{i=1}^{n} f(x_i^*) \Delta x_i

provided this limit exists and is the same for all choices of sample points $x_i^*$ . Here, $\|\mathcal{P}\|$ denotes the mesh (or norm) of the partition — the width of the largest subinterval.

When this limit exists, we say $f$ is Riemann integrable on $[a, b]$ .

Understanding the Notation

The integral symbol $\int$ is an elongated "S" for "sum" (Leibniz's design). Each part of the notation has meaning:

Symbol	Name	Meaning
∫	Integral sign	Indicates a limiting sum (from Latin 'summa')
a	Lower limit	The left endpoint of the interval
b	Upper limit	The right endpoint of the interval
f(x)	Integrand	The function being integrated
dx	Differential	Represents Δx → 0; indicates variable of integration

Reading Integral Notation

Read $\int_0^3 x^2\,dx$ as "the integral of x squared from 0 to 3 with respect to x." This equals the exact area under the parabola $y = x^2$ between $x = 0$ and $x = 3$ .

Interactive Exploration: Riemann Sums in Action

Experiment with different functions, numbers of rectangles, and sampling methods. Watch how the Riemann sum converges to the exact integral as you increase $n$ :

Interactive Riemann Sum Explorer

Watch how rectangles approximate the area under a curve

Function f(x)

Rectangle Type

Number of Rectangles (n)

n = 6Δx = 0.3333

Riemann Sum (left)

2.037037

Exact Integral

2.666667

Approximation Error

0.629630

Mathematical Formula

L_n = Σ f(x_i-1) · Δx = Σ f(a + (i-1)Δx) · Δx

where Δx = (b - a) / n = (2 - 0) / 6 = 0.3333

Key Insight: The Limit Definition

As n → ∞ (and Δx → 0), all three types of Riemann sums converge to the same value: the definite integral. This is what we mean when we write ∫_a^b f(x) dx = lim_n→∞ Σ f(x_i*) Δx.

Watching the Limit Converge

The visualization below shows how the Riemann sum approaches the exact integral value as we increase the number of rectangles. Notice how:

All sum types converge — left, right, and midpoint all approach the same limit
Error decreases as $n$ increases
Convergence is orderly — the values settle toward the exact integral systematically

The Definite Integral as a Limit

Watch the Riemann sum converge to the exact integral as n → ∞

n =

Number of Rectangles

n = 8

Width of Each Rectangle

Δx = 0.2500

L_n = Σ f(x_i) Δx

2.187500

∫₀² x² dx (exact)

2.666667

Approximation Error

|L_n - I| = 0.47916717.97% error

Convergence: L_n → ∫₀² x² dx as n → ∞

n	Δx = 2/n	L_n	Error	% Error
2	1.000000	1.000000	1.666667	62.500%
4	0.500000	1.750000	0.916667	34.375%
8	0.250000	2.187500	0.479167	17.969%
16	0.125000	2.421875	0.244792	9.180%
32	0.062500	2.542969	0.123698	4.639%
64	0.031250	2.604492	0.062174	2.332%
128	0.015625	2.635498	0.031169	1.169%
256	0.007813	2.651062	0.015605	0.585%
∞	→ 0	2.666667	0	0%

The Limit Definition of the Definite Integral

∫_a^b f(x) dx = lim_n→∞ Σ_i=1ⁿ f(x_i*) Δx

The definite integral is defined as the limit of Riemann sums as the partition becomes infinitely fine (n → ∞, Δx → 0). This limit exists and equals the same value regardless of which sample points x_i* we choose (left, right, midpoint, or any other choice), as long as f is integrable on [a, b].

Partition Refinement: Why the Limit Works

Why does the limit of Riemann sums give us the exact area? The key insight is that as the partition gets finer:

Rectangles fit the curve better. With thinner rectangles, the "error" between the rectangle tops and the actual curve shrinks.
Local variations get captured. Where the function changes rapidly, more rectangles mean we sample the function more often, catching its behavior.
Overestimates and underestimates balance. As $n \to \infty$ , left and right sums converge to the same value from opposite sides.

Mathematical Insight

For a continuous function on $[a,b]$ , the difference between the upper and lower Riemann sums is bounded by:

U_n - L_n \leq (b - a) \cdot \omega_f(\Delta x)

where $\omega_f(\Delta x)$ is the modulus of continuity — how much $f$ can vary over intervals of width $\Delta x$ . As $\Delta x \to 0$ , this bound goes to zero, forcing upper and lower sums together.

When Does the Limit Exist? Integrability

Not every function has a well-defined integral. For the limit of Riemann sums to exist, the function must be "nice enough." Here are the key conditions:

Sufficient Conditions for Integrability

Continuous Functions ✓

If $f$ is continuous on $[a, b]$ , then $f$ is integrable. This is the most common case.

Bounded with Finitely Many Discontinuities ✓

If $f$ is bounded and has only finitely many discontinuities (like step functions), it is still integrable.

Monotonic Functions ✓

Any monotonic (always increasing or always decreasing) function on $[a, b]$ is integrable, even if it has countably many jump discontinuities.

Non-Integrable Functions

The Dirichlet Function

The function $f(x) = 1$ if $x$ is rational and $f(x) = 0$ if $x$ is irrational is NOT Riemann integrable. Every interval contains both rational and irrational points, so upper and lower sums never converge to the same value.

However, the Lebesgue integral (a more general theory) can handle such functions. For most practical applications in calculus, physics, and engineering, Riemann integration is sufficient.

Real-World Applications

The limit definition of the integral is not just abstract mathematics — it underlies how we model and compute in the real world.

Physics: Work Done by a Variable Force

When a force $F(x)$ varies with position, the work done moving from $a$ to $b$ is:

W = \int_a^b F(x)\,dx = \lim_{n \to \infty} \sum_{i=1}^{n} F(x_i^*) \Delta x

Each term $F(x_i^*) \Delta x$ is the work done over a small displacement where force is approximately constant.

Engineering: Fluid Flow

The total volume of fluid flowing through a pipe with variable cross-section $A(x)$ and velocity $v(x)$ :

V = \int_0^L A(x) \cdot v(x)\,dx

Probability: Expected Values

For a continuous random variable with probability density function $f(x)$ :

E[X] = \int_{-\infty}^{\infty} x \cdot f(x)\,dx

This is the "weighted average" of all possible values, with weights given by probabilities.

Economics: Consumer Surplus

The area between the demand curve $D(q)$ and the market price $p^*$ gives consumer surplus:

CS = \int_0^{q^*} [D(q) - p^*]\,dq

Machine Learning Connection

The limit definition of integration is foundational to many machine learning techniques.

Monte Carlo Integration

When integrals are too complex to compute analytically, we can approximate them using random samples — this is Monte Carlo integration:

\int_a^b f(x)\,dx \approx \frac{b-a}{N} \sum_{i=1}^{N} f(x_i)

where $x_i$ are random samples from $[a, b]$ . This is essentially a Riemann sum with randomly chosen sample points!

Key ML Applications

Loss function computation: Many loss functions involve integrals over probability distributions
Bayesian inference: Computing posterior distributions requires integrating (marginalizing) over parameters
Variational autoencoders: The ELBO objective involves expectations (integrals) that are approximated using samples
Reinforcement learning: Value functions are expectations over trajectories — integrals in disguise
Gaussian processes: Predictions involve integrals over function space

The Law of Large Numbers

Monte Carlo integration works because of the Law of Large Numbers: as $N \to \infty$ , the sample mean converges to the true expected value (integral). The rate of convergence is $O(1/\sqrt{N})$ , independent of dimension — making Monte Carlo essential for high-dimensional integration.

Python Implementation

Computing Riemann Sums

The following code demonstrates how to compute Riemann sums and observe their convergence to the definite integral:

Riemann Sum Computation and Convergence

🐍riemann_integration.py

Explanation(7)

Code(76)

1Riemann Sum Function

This function computes the Riemann sum approximation to the definite integral. It implements the formula Σ f(xᵢ*) Δx where xᵢ* is the sample point in each subinterval.

17Computing Δx

Δx = (b - a) / n is the width of each subinterval. As n increases, Δx decreases, making our approximation more accurate.

19Left Riemann Sum

For left sums, we sample at xᵢ = a + i·Δx, the left endpoint of each subinterval [xᵢ, xᵢ₊₁].

22Right Riemann Sum

For right sums, we sample at xᵢ = a + (i+1)·Δx, the right endpoint of each subinterval.

25Midpoint Riemann Sum

For midpoint sums, we sample at xᵢ = a + (i + 0.5)·Δx, the center of each subinterval. This often gives better accuracy.

31The Sum Itself

The core computation: sum all f(sample_points) values and multiply by Δx. This is the discrete approximation to the integral.

35Convergence Demonstration

This function shows how all three types of Riemann sums converge to the same limit (the exact integral) as n → ∞.

69 lines without explanation

1import numpy as np
2import matplotlib.pyplot as plt
3
4def riemann_sum(f, a, b, n, method='left'):
5    """
6    Compute the Riemann sum of f over [a, b] with n subintervals.
7
8    Parameters:
9    -----------
10    f : callable - The function to integrate
11    a, b : float - Integration bounds
12    n : int - Number of subintervals
13    method : str - 'left', 'right', or 'midpoint'
14
15    Returns:
16    --------
17    float : The Riemann sum approximation
18    """
19    dx = (b - a) / n  # Width of each subinterval
20
21    if method == 'left':
22        # Sample at left endpoints: x_i = a + i * dx
23        sample_points = np.array([a + i * dx for i in range(n)])
24    elif method == 'right':
25        # Sample at right endpoints: x_i = a + (i+1) * dx
26        sample_points = np.array([a + (i + 1) * dx for i in range(n)])
27    elif method == 'midpoint':
28        # Sample at midpoints: x_i = a + (i + 0.5) * dx
29        sample_points = np.array([a + (i + 0.5) * dx for i in range(n)])
30    else:
31        raise ValueError("method must be 'left', 'right', or 'midpoint'")
32
33    # Riemann sum: Σ f(x_i*) * Δx
34    return np.sum(f(sample_points)) * dx
35
36def demonstrate_convergence(f, a, b, exact_value, f_name="f"):
37    """
38    Show how Riemann sums converge to the integral as n → ∞.
39    """
40    n_values = [2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
41
42    print(f"Convergence of Riemann Sums for ∫[{a},{b}] {f_name}(x) dx")
43    print(f"Exact value: {exact_value:.10f}")
44    print("-" * 70)
45    print(f"{'n':>8} | {'Left Sum':>14} | {'Right Sum':>14} | {'Midpoint':>14}")
46    print("-" * 70)
47
48    for n in n_values:
49        left = riemann_sum(f, a, b, n, 'left')
50        right = riemann_sum(f, a, b, n, 'right')
51        mid = riemann_sum(f, a, b, n, 'midpoint')
52        print(f"{n:>8} | {left:>14.10f} | {right:>14.10f} | {mid:>14.10f}")
53
54    print("-" * 70)
55    print(f"{'n → ∞':>8} | {exact_value:>14.10f} | {exact_value:>14.10f} | {exact_value:>14.10f}")
56
57# Example: ∫₀² x² dx = 8/3 ≈ 2.6667
58print("=" * 70)
59print("Example: f(x) = x²")
60print("=" * 70)
61demonstrate_convergence(
62    f=lambda x: x**2,
63    a=0, b=2,
64    exact_value=8/3,
65    f_name="x²"
66)
67
68print("\n" + "=" * 70)
69print("Example: f(x) = sin(x)")
70print("=" * 70)
71demonstrate_convergence(
72    f=np.sin,
73    a=0, b=np.pi,
74    exact_value=2.0,  # ∫₀^π sin(x) dx = 2
75    f_name="sin"
76)

Visualization Code

This code creates visualizations showing both the geometric meaning and numerical convergence of Riemann sums:

Visualizing Riemann Sum Convergence

🐍visualize_convergence.py

Explanation(5)

Code(75)

6Two-Panel Visualization

We create two side-by-side plots: one showing the geometric meaning of Riemann sums (rectangles approximating area) and one showing how the sums converge numerically.

15Exact Area Shading

The green shaded region shows the exact area under the curve — this is what the integral measures and what our Riemann sums approximate.

24Drawing Rectangle Patches

Each blue rectangle represents one term f(xᵢ)·Δx in the Riemann sum. The total area of all rectangles approximates the integral.

45Computing All Sum Types

We calculate left, right, and midpoint Riemann sums for each value of n. All three converge to the same limit as n → ∞.

49Logarithmic Scale

Using a log scale for n lets us see the convergence behavior clearly across many orders of magnitude. Notice how all curves approach the red dashed line (exact value).

70 lines without explanation

1import numpy as np
2import matplotlib.pyplot as plt
3from matplotlib.patches import Rectangle
4from matplotlib.animation import FuncAnimation
5
6def visualize_riemann_convergence(f, a, b, exact_value, title=""):
7    """
8    Create a visualization showing Riemann sum convergence.
9    """
10    fig, axes = plt.subplots(1, 2, figsize=(14, 5))
11
12    # Left plot: Riemann rectangles
13    ax1 = axes[0]
14    x_curve = np.linspace(a, b, 500)
15    y_curve = f(x_curve)
16
17    ax1.fill_between(x_curve, y_curve, alpha=0.3, color='green',
18                      label='Exact area')
19    ax1.plot(x_curve, y_curve, 'r-', linewidth=2.5, label='f(x)')
20
21    # Draw rectangles for n = 8 (left Riemann sum)
22    n = 8
23    dx = (b - a) / n
24    for i in range(n):
25        x_i = a + i * dx
26        height = f(x_i)
27        rect = Rectangle((x_i, 0), dx, height,
28                         facecolor='blue', edgecolor='darkblue',
29                         alpha=0.5, linewidth=1)
30        ax1.add_patch(rect)
31
32    ax1.set_xlim(a - 0.2, b + 0.2)
33    ax1.set_ylim(0, max(y_curve) * 1.2)
34    ax1.set_xlabel('x', fontsize=12)
35    ax1.set_ylabel('f(x)', fontsize=12)
36    ax1.set_title(f'Left Riemann Sum (n = {n})', fontsize=14)
37    ax1.legend()
38    ax1.grid(True, alpha=0.3)
39
40    # Right plot: Convergence graph
41    ax2 = axes[1]
42    n_values = [2, 4, 8, 16, 32, 64, 128, 256]
43    left_sums = []
44    right_sums = []
45    mid_sums = []
46
47    for n in n_values:
48        dx = (b - a) / n
49        left_sums.append(sum(f(a + i*dx) * dx for i in range(n)))
50        right_sums.append(sum(f(a + (i+1)*dx) * dx for i in range(n)))
51        mid_sums.append(sum(f(a + (i+0.5)*dx) * dx for i in range(n)))
52
53    ax2.semilogx(n_values, left_sums, 'b-o', label='Left sum', linewidth=2)
54    ax2.semilogx(n_values, right_sums, 'g-s', label='Right sum', linewidth=2)
55    ax2.semilogx(n_values, mid_sums, 'm-^', label='Midpoint', linewidth=2)
56    ax2.axhline(y=exact_value, color='r', linestyle='--',
57                linewidth=2, label=f'Exact = {exact_value:.4f}')
58
59    ax2.set_xlabel('n (number of rectangles)', fontsize=12)
60    ax2.set_ylabel('Riemann Sum Value', fontsize=12)
61    ax2.set_title('Convergence to Definite Integral', fontsize=14)
62    ax2.legend()
63    ax2.grid(True, alpha=0.3)
64
65    plt.suptitle(title, fontsize=14, y=1.02)
66    plt.tight_layout()
67    plt.show()
68
69# Visualize for f(x) = x² on [0, 2]
70visualize_riemann_convergence(
71    f=lambda x: x**2,
72    a=0, b=2,
73    exact_value=8/3,
74    title="The Definite Integral as a Limit: ∫₀² x² dx = 8/3"
75)

Common Mistakes to Avoid

Mistake 1: Confusing Area with Signed Area

Wrong: "The integral of sin(x) from 0 to 2π is the area, which must be positive."

Correct: The definite integral gives signed area. Regions below the x-axis contribute negative values. $\int_0^{2\pi} \sin x\,dx = 0$ because positive and negative areas cancel.

Mistake 2: Forgetting the Limit

Wrong: "The integral is just a sum of rectangles."

Correct: The integral is the limit of such sums as $n \to \infty$ . Any finite Riemann sum is only an approximation.

Mistake 3: Thinking Sample Point Choice Matters for the Final Value

Wrong: "Left Riemann sums give a different integral than right Riemann sums."

Correct: For integrable functions, ALL choices of sample points give the same limit. The choice only affects the rate of convergence, not the final value.

Mistake 4: Confusing dx with Δx

Wrong: "dx is just a small number."

Correct: In the definite integral, "dx" is part of the notation indicating the variable of integration. It represents the limiting behavior as $\Delta x \to 0$ , not a specific small value.

Test Your Understanding

Question 1 of 8

What does the definite integral ∫ₐᵇ f(x) dx represent geometrically?

Summary

The definite integral is defined as the limit of Riemann sums — this is the foundation of integral calculus.

The Central Definition

\int_a^b f(x)\,dx = \lim_{n \to \infty} \sum_{i=1}^{n} f(x_i^*) \Delta x

Key Concepts

Concept	Description
Partition	Dividing [a, b] into n subintervals of width Δx = (b-a)/n
Sample points x*ᵢ	Points in each subinterval where we evaluate f
Riemann sum	Σ f(x*ᵢ) Δx — sum of rectangle areas
Definite integral	The limit of Riemann sums as n → ∞
Integrable	When the limit exists and is independent of sample point choice

Key Takeaways

The definite integral $\int_a^b f(x)\,dx$ is defined as a limit of sums, not just "area under the curve"
As $n \to \infty$ , $\Delta x \to 0$ and rectangles fill the area exactly
For integrable functions, all sample point choices give the same limit
Continuous functions are always integrable; most functions we encounter in applications are integrable
The integral gives signed area — regions below the x-axis contribute negatively
Numerical integration methods (trapezoidal rule, Simpson's rule, Monte Carlo) are all built on this limit definition

The Philosophical Core:

"The integral tames infinity: infinitely many infinitely thin slices, summed together, yield a precise finite answer."

Coming Next: In the next section, we'll explore the properties of definite integrals — linearity, additivity over intervals, and comparison properties that make integration a powerful tool for analysis.

Learning Objectives

The Big Picture: Exact Area from Approximation

The Central Question

Historical Context: From Archimedes to Riemann

Why Riemann?

From Sums to Limits: Building the Definition

Step 1: Partition the Interval

Step 2: Choose Sample Points

Step 3: Form the Riemann Sum

Step 4: Take the Limit

The Formal Definition: The Definite Integral

Definition: The Definite Integral

Understanding the Notation

Reading Integral Notation

Interactive Exploration: Riemann Sums in Action

Interactive Riemann Sum Explorer

Key Insight: The Limit Definition

Watching the Limit Converge

The Definite Integral as a Limit

Convergence: Ln → ∫₀² x² dx as n → ∞

The Limit Definition of the Definite Integral

Partition Refinement: Why the Limit Works

When Does the Limit Exist? Integrability

Sufficient Conditions for Integrability

Non-Integrable Functions

The Dirichlet Function

Real-World Applications

Physics: Work Done by a Variable Force

Engineering: Fluid Flow

Probability: Expected Values

Economics: Consumer Surplus

Machine Learning Connection

Monte Carlo Integration

Key ML Applications

The Law of Large Numbers

Python Implementation

Computing Riemann Sums

Visualization Code

Common Mistakes to Avoid

Mistake 1: Confusing Area with Signed Area

Mistake 2: Forgetting the Limit

Mistake 3: Thinking Sample Point Choice Matters for the Final Value

Mistake 4: Confusing dx with Δx

Test Your Understanding

Test Your Understanding

Summary

The Central Definition

Key Concepts

Key Takeaways

Convergence: L_n → ∫₀² x² dx as n → ∞