Boo-AI — Master Artificial Intelligence by Building from Scratch

Learning Objectives

By the end of this section, you will be able to:

Explain the area problem that motivated the development of integral calculus
Approximate the area under a curve using left, right, and midpoint Riemann sums
Calculate the width $\Delta x$ and sample points for a partition
Express Riemann sums using sigma notation $\sum$
Analyze how increasing the number of rectangles improves the approximation
Connect Riemann sums to numerical integration in scientific computing
Apply the concepts to real-world problems in physics, economics, and machine learning

The Big Picture: Why We Need to Calculate Areas

"Nature speaks in the language of differential equations, but she writes her answers in integrals."— Adapted from Galileo

In the first half of calculus, we learned about derivatives — how to find instantaneous rates of change. Now we turn to the inverse problem: given a rate of change, how do we find the total accumulated quantity?

This question appears everywhere in science and engineering:

🚗 Physics

Velocity → Distance traveled
Force × Distance → Work done
Power × Time → Energy consumed
Current × Time → Total charge

📈 Economics

Marginal cost → Total cost
Revenue rate → Total revenue
Consumer surplus calculations
Present value of income streams

🧬 Biology

Growth rate → Population size
Drug absorption → Total dosage
Reaction rates → Product formed
Blood flow measurements

🤖 Machine Learning

PDF → CDF (probability calculations)
Expected value computations
Loss function optimization
Kernel methods and RBFs

The Core Question of Integration

Given a function $f(x)$ and an interval $[a, b]$ , how do we calculate the total area between the curve and the x-axis?

For simple shapes (rectangles, triangles), we have formulas. But what about curves like $y = x^2$ or $y = \sin(x)$ ? This is the area problem that integral calculus solves.

Historical Origins: From Archimedes to Riemann

The quest to find areas under curves has ancient roots, but the rigorous foundation we use today took millennia to develop.

Archimedes and the Method of Exhaustion (c. 250 BCE)

The Greek mathematician Archimedes of Syracuse developed the method of exhaustion to find the area of a parabolic segment. His key insight: approximate the curved region with increasingly fine polygons whose area we can calculate.

Archimedes showed that the area under a parabola from 0 to 1 is exactly $\\frac{1}{3}$ — a result we can verify using integral calculus: $\\int_0^1 x^2\\,dx = \\frac{1}{3}$ .

Newton and Leibniz (1670s)

Isaac Newton and Gottfried Wilhelm Leibniz independently discovered the Fundamental Theorem of Calculus, which connected differentiation and integration. This made calculating areas systematic rather than ad hoc.

Bernhard Riemann (1854)

The German mathematician Bernhard Riemann provided the rigorous foundation for integration in his 1854 thesis. He defined the integral as the limit of sums of rectangles — what we now call Riemann sums. His approach works for a broad class of functions and laid the groundwork for modern analysis.

Why Riemann's Approach Matters

Before Riemann, integration was defined only for "nice" functions. Riemann's definition precisely characterized which functions are integrable and provided a computational method (approximating with rectangles) that generalizes to numerical algorithms used in computers today.

The Area Problem: Setting Up the Challenge

Consider the function $f(x) = x^2$ on the interval $[0, 2]$ . We want to find the area of the region bounded by:

The curve $y = x^2$ above
The x-axis below ( $y = 0$ )
The vertical line $x = 0$ on the left
The vertical line $x = 2$ on the right

This region has a curved boundary, so we cannot use the simple formulas for rectangles ( $A = bh$ ) or triangles ( $A = \\frac{1}{2}bh$ ). We need a new approach.

The Key Insight: Approximate, Then Take a Limit

Riemann's brilliant idea: even though we cannot calculate the area of the curved region directly, we can calculate the area of rectangles. So:

Divide the interval $[a, b]$ into $n$ smaller subintervals
Construct a rectangle on each subinterval with height determined by the function
Sum the areas of all rectangles to get an approximation
Refine by increasing $n$ (more, thinner rectangles)
Take the limit as $n \\to \\infty$ to get the exact area

Approximating with Rectangles

Let's make this concrete. We want to approximate the area under $f(x) = x^2$ on $[0, 2]$ .

Step 1: Partition the Interval

Divide $[0, 2]$ into $n$ equal subintervals. Each subinterval has width:

\\Delta x = \\frac{b - a}{n} = \\frac{2 - 0}{n} = \\frac{2}{n}

The endpoints of the subintervals are:

x_0 = 0, \\quad x_1 = \\frac{2}{n}, \\quad x_2 = \\frac{4}{n}, \\quad \\ldots, \\quad x_n = 2

In general, $x_i = a + i \\cdot \\Delta x = \\frac{2i}{n}$ for $i = 0, 1, 2, \\ldots, n$ .

Step 2: Choose Sample Points

For each subinterval $[x_{i-1}, x_i]$ , we need to choose a point to evaluate $f$ . This determines the rectangle's height. Common choices:

Method	Sample Point x*ᵢ	Visual Effect
Left Riemann Sum	x*ᵢ = xᵢ₋₁ (left endpoint)	Rectangle height from left edge of subinterval
Right Riemann Sum	x*ᵢ = xᵢ (right endpoint)	Rectangle height from right edge of subinterval
Midpoint Rule	x*ᵢ = (xᵢ₋₁ + xᵢ)/2 (midpoint)	Rectangle height from center of subinterval

Step 3: Calculate Rectangle Areas and Sum

Each rectangle has width $\\Delta x$ and height $f(x^*_i)$ . The area of the $i$ -th rectangle is:

A_i = f(x^*_i) \\cdot \\Delta x

The total area of all $n$ rectangles is the Riemann sum:

R_n = \\sum_{i=1}^{n} f(x^*_i) \\cdot \\Delta x

Types of Riemann Sums

Left Riemann Sum ( $L_n$ )

Use the left endpoint of each subinterval:

L_n = \\sum_{i=1}^{n} f(x_{i-1}) \\cdot \\Delta x = \\sum_{i=0}^{n-1} f(x_i) \\cdot \\Delta x

For an increasing function, the left sum underestimates the area because each rectangle lies entirely below the curve.

Right Riemann Sum ( $R_n$ )

Use the right endpoint of each subinterval:

R_n = \\sum_{i=1}^{n} f(x_i) \\cdot \\Delta x

For an increasing function, the right sum overestimates the area because each rectangle extends above the curve.

Midpoint Riemann Sum ( $M_n$ )

Use the midpoint of each subinterval:

M_n = \\sum_{i=1}^{n} f\\left(\\frac{x_{i-1} + x_i}{2}\\right) \\cdot \\Delta x

The midpoint rule often gives a better approximation for the same number of rectangles because overestimates and underestimates tend to cancel.

Which Sum to Use?

All three methods converge to the same limit (the definite integral) as $n \\to \\infty$ . For practical computation with finite n:

Midpoint is usually most accurate (error decreases as $O(1/n^2)$ )
Left/Right are simpler but less accurate (error $O(1/n)$ )
Trapezoidal rule (average of left and right) is also $O(1/n^2)$

Interactive Riemann Sum Explorer

Use the interactive visualization below to explore how different types of Riemann sums approximate the area under various curves. Adjust the number of rectangles and watch the approximation improve!

📊Interactive Riemann Sum Explorer

Function

Sum Type

Rectangles: 4

Current Function

f(x) = x^2

Interval: [0, 3]

Riemann Sum Formula

R_n = \sum_{i=1}^{4} f(x_i^*) \cdot \Delta x

\Delta x = \frac{3 - 0}{4} = 0.7500

Metric	Value
Left Riemann Sum (n = 4)	5.906250
Exact Area (definite integral)	9.000000
Error	3.093750 (34.38%)

Key Insight

As n increases, the Riemann sum approaches the exact area under the curve. Try increasing n to 50+ and watch the error shrink.

Convergence: How Approximations Improve

As we increase the number of rectangles, our approximation gets better. But how quickly does it improve? Let's analyze the convergence.

📈Convergence: How Fast Do Approximations Improve?

Let's compute the area under $f(x) = x^2$ from $x = 0$ to $x = 2$ . The exact answer is $\frac{8}{3} \approx 2.667$ . Watch how different methods converge as we increase the number of rectangles.

Sigma Notation: The Language of Sums

To express Riemann sums compactly, we use sigma notation (summation notation), denoted by the Greek capital letter sigma: $\\Sigma$ .

The Structure of Sigma Notation

\\sum_{i=1}^{n} a_i = a_1 + a_2 + a_3 + \\cdots + a_n

Component	Meaning
Σ	Sum (add up all terms)
i = 1	Start index (lower limit)
n	End index (upper limit)
aᵢ	General term (formula for each addend)

Examples

\\sum_{i=1}^{5} i = 1 + 2 + 3 + 4 + 5 = 15

\\sum_{i=1}^{4} i^2 = 1^2 + 2^2 + 3^2 + 4^2 = 1 + 4 + 9 + 16 = 30

\\sum_{k=0}^{3} 2^k = 2^0 + 2^1 + 2^2 + 2^3 = 1 + 2 + 4 + 8 = 15

Useful Summation Formulas

Sum	Closed Form	Example
Σᵢ₌₁ⁿ 1	n	Σ₁⁵ 1 = 5
Σᵢ₌₁ⁿ i	n(n+1)/2	Σ₁⁵ i = 5(6)/2 = 15
Σᵢ₌₁ⁿ i²	n(n+1)(2n+1)/6	Σ₁⁵ i² = 5(6)(11)/6 = 55
Σᵢ₌₁ⁿ i³	[n(n+1)/2]²	Σ₁⁵ i³ = 15² = 225

These formulas are crucial for evaluating Riemann sums algebraically before taking the limit.

Worked Examples

Example 1: Left Riemann Sum for f(x) = x² on [0, 2] with n = 4

Step 1: Calculate $\\Delta x$

\\Delta x = \\frac{2 - 0}{4} = \\frac{1}{2} = 0.5

Step 2: Identify left endpoints

x_0 = 0, \\quad x_1 = 0.5, \\quad x_2 = 1, \\quad x_3 = 1.5

Step 3: Evaluate f at each left endpoint

f(0) = 0, \\quad f(0.5) = 0.25, \\quad f(1) = 1, \\quad f(1.5) = 2.25

Step 4: Calculate the sum

L_4 = (0 + 0.25 + 1 + 2.25) \\times 0.5 = 3.5 \\times 0.5 = 1.75

Result: $L_4 = 1.75$

(The exact area is $\\frac{8}{3} \\approx 2.667$ , so we underestimate by about 0.92)

Example 2: Using Sigma Notation and Formulas

Calculate the left Riemann sum for $f(x) = x^2$ on $[0, 2]$ with $n$ rectangles, then take the limit as $n \\to \\infty$ .

Setup:

\\Delta x = \\frac{2}{n}, \\quad x_i = \\frac{2i}{n}

Left endpoints: $x_{i-1} = \\frac{2(i-1)}{n}$

Height of i-th rectangle:

f(x_{i-1}) = \\left(\\frac{2(i-1)}{n}\\right)^2 = \\frac{4(i-1)^2}{n^2}

Riemann sum:

L_n = \\sum_{i=1}^{n} \\frac{4(i-1)^2}{n^2} \\cdot \\frac{2}{n} = \\frac{8}{n^3} \\sum_{i=1}^{n} (i-1)^2

Substitute j = i - 1:

= \\frac{8}{n^3} \\sum_{j=0}^{n-1} j^2 = \\frac{8}{n^3} \\cdot \\frac{(n-1)n(2n-1)}{6}

Simplify:

= \\frac{8(n-1)(2n-1)}{6n^2} = \\frac{4(n-1)(2n-1)}{3n^2}

Take the limit:

\\lim_{n \\to \\infty} \\frac{4(n-1)(2n-1)}{3n^2} = \\lim_{n \\to \\infty} \\frac{4(2n^2 - 3n + 1)}{3n^2} = \\frac{8}{3}

Result: The exact area is $\\frac{8}{3} \\approx 2.667$

Preview of the Definite Integral

What we just computed — the limit of Riemann sums — is the definite integral:

\\int_0^2 x^2\\,dx = \\lim_{n \\to \\infty} L_n = \\frac{8}{3}

In later sections, we'll learn the Fundamental Theorem of Calculus, which provides a much faster way to evaluate such integrals without computing limits!

Real-World Applications

Physics: Distance from Velocity

If a car's velocity is $v(t)$ meters per second, the distance traveled from time $t = a$ to $t = b$ is:

\\text{Distance} = \\int_a^b v(t)\\,dt

Riemann sums provide an intuitive interpretation: in each small time interval $\\Delta t$ , the car travels approximately $v(t_i) \\cdot \\Delta t$ meters. Summing over all intervals gives total distance.

Economics: Total Revenue from Marginal Revenue

If $MR(q)$ is the marginal revenue (additional revenue per unit), total revenue from selling $Q$ units is:

TR = \\int_0^Q MR(q)\\,dq

Biology: Total Growth from Growth Rate

If a population grows at rate $r(t)$ organisms per day, the total population change over $[a, b]$ days is:

\\Delta P = \\int_a^b r(t)\\,dt

Machine Learning Connections

Integration concepts appear throughout machine learning, often in surprising places.

Probability: PDF to CDF

For a continuous random variable with probability density function $f(x)$ , the cumulative distribution function is:

F(x) = P(X \\leq x) = \\int_{-\\infty}^{x} f(t)\\,dt

Numerical integration (Riemann sums) allows us to compute probabilities when the integral has no closed-form solution.

Expected Value

The expected value of a continuous random variable:

E[X] = \\int_{-\\infty}^{\\infty} x \\cdot f(x)\\,dx

Monte Carlo Integration in Training

Many machine learning algorithms use Monte Carlo integration — a randomized version of Riemann sums. Instead of systematic sample points, we use random samples:

\\int f(x)\\,dx \\approx \\frac{1}{n} \\sum_{i=1}^{n} f(x_i), \\quad x_i \\sim \\text{Uniform}

This is used in stochastic gradient descent (sampling mini-batches), reinforcement learning (policy gradients), and variational inference.

Why Numerical Integration Matters for ML

Modern neural networks require computing gradients of loss functions. The loss is often an expectation (an integral over data distribution). Since we can't integrate analytically, we use mini-batch sampling — essentially a Monte Carlo Riemann sum!

Python Implementation

Computing Riemann Sums

Let's implement Riemann sums in Python and see convergence in action:

Riemann Sum Implementation

🐍riemann_sums.py

Explanation(6)

Code(59)

3Riemann Sum Function

This function implements all three types of Riemann sums: left endpoint, right endpoint, and midpoint. Each method samples the function at different points within each subinterval.

15Calculating Δx

The width of each rectangle is Δx = (b - a) / n. This divides the interval [a, b] into n equal parts. As n increases, Δx shrinks, and our approximation improves.

EXAMPLE

For [0, 2] with n = 4: Δx = 2/4 = 0.5

18Left Endpoint Samples

For left sums, we evaluate f at the left edge of each subinterval: x₀ = a, x₁ = a + Δx, ..., xₙ₋₁ = b - Δx. The left endpoints systematically underestimate the area for increasing functions.

21Right Endpoint Samples

For right sums, we evaluate f at the right edge: x₁ = a + Δx, x₂ = a + 2Δx, ..., xₙ = b. Right endpoints overestimate for increasing functions.

24Midpoint Samples

For midpoint sums, we evaluate f at the center of each subinterval. This tends to balance over- and underestimation, converging faster (O(Δx²) vs O(Δx)).

30Computing the Sum

The Riemann sum is Σf(xᵢ*)·Δx — the sum of rectangle areas. NumPy's vectorized operations make this efficient: evaluate f at all sample points, then sum the products.

53 lines without explanation

1import numpy as np
2import matplotlib.pyplot as plt
3
4def riemann_sum(f, a, b, n, method='left'):
5    """
6    Compute the Riemann sum for function f on [a, b] with n rectangles.
7
8    Parameters:
9    - f: The function to integrate
10    - a, b: Interval endpoints
11    - n: Number of rectangles (subintervals)
12    - method: 'left', 'right', or 'midpoint'
13
14    Returns: The Riemann sum approximation of the integral
15    """
16    # Width of each rectangle
17    delta_x = (b - a) / n
18
19    # Generate the sample points based on method
20    if method == 'left':
21        # Left endpoints: a, a+Δx, a+2Δx, ..., a+(n-1)Δx
22        x_samples = np.linspace(a, b - delta_x, n)
23    elif method == 'right':
24        # Right endpoints: a+Δx, a+2Δx, ..., b
25        x_samples = np.linspace(a + delta_x, b, n)
26    elif method == 'midpoint':
27        # Midpoints: a+Δx/2, a+3Δx/2, ..., b-Δx/2
28        x_samples = np.linspace(a + delta_x/2, b - delta_x/2, n)
29    else:
30        raise ValueError("method must be 'left', 'right', or 'midpoint'")
31
32    # Sum up the rectangle areas: f(x_i) * Δx
33    heights = f(x_samples)
34    total_area = np.sum(heights * delta_x)
35
36    return total_area
37
38# Define our function: f(x) = x^2
39def f(x):
40    return x ** 2
41
42# Interval [0, 2], exact integral = (2^3)/3 - (0^3)/3 = 8/3 ≈ 2.667
43a, b = 0, 2
44exact_integral = 8/3
45
46print("Approximating ∫₀² x² dx = 8/3 ≈ 2.6667")
47print()
48print(f"{'n':>6} {'Left Sum':>12} {'Right Sum':>12} {'Midpoint':>12} {'Exact':>12}")
49print("-" * 60)
50
51for n in [2, 4, 8, 16, 32, 64, 128]:
52    left = riemann_sum(f, a, b, n, 'left')
53    right = riemann_sum(f, a, b, n, 'right')
54    mid = riemann_sum(f, a, b, n, 'midpoint')
55    print(f"{n:>6} {left:>12.6f} {right:>12.6f} {mid:>12.6f} {exact_integral:>12.6f}")
56
57print()
58print("Notice how all methods converge to 8/3 ≈ 2.6667 as n increases!")
59print("The midpoint rule converges faster (quadratic vs linear error decay).")

Visualizing Riemann Sums

Here's how to create visualizations like the interactive explorer above:

Riemann Sum Visualization

🐍riemann_visualization.py

Explanation(5)

Code(68)

3Visualization Function

This function creates a visual representation of a Riemann sum, showing both the curve and the approximating rectangles. Visualization helps build geometric intuition.

14Plot the True Curve

We first plot the actual function f(x) using many points (200) to get a smooth curve. This red curve is what we are trying to measure the area under.

27Sample Point Selection

For each rectangle, we determine where to evaluate f. The sample point determines the height of the rectangle. Different choices (left, right, midpoint) give different approximations.

37Drawing Rectangles

Matplotlib's Rectangle patch creates each approximating rectangle. The alpha parameter (0.4) makes them semi-transparent so we can see overlapping regions and the curve behind.

55Comparing Methods

Creating side-by-side plots of left, right, and midpoint sums helps visualize how different sampling strategies affect the approximation. Notice how left underestimates and right overestimates for f(x) = x².

63 lines without explanation

1import numpy as np
2import matplotlib.pyplot as plt
3
4def visualize_riemann_sum(f, a, b, n, method='left', ax=None):
5    """
6    Visualize a Riemann sum with rectangles.
7    """
8    if ax is None:
9        fig, ax = plt.subplots(figsize=(10, 6))
10
11    delta_x = (b - a) / n
12
13    # Plot the actual curve
14    x_curve = np.linspace(a, b, 200)
15    y_curve = f(x_curve)
16    ax.plot(x_curve, y_curve, 'r-', linewidth=2, label='f(x) = x²')
17
18    # Colors for different methods
19    colors = {'left': 'blue', 'right': 'green', 'midpoint': 'purple'}
20    color = colors.get(method, 'blue')
21
22    total_area = 0
23
24    # Draw rectangles
25    for i in range(n):
26        # Rectangle left and right edges
27        left_edge = a + i * delta_x
28        right_edge = left_edge + delta_x
29
30        # Sample point depends on method
31        if method == 'left':
32            sample_x = left_edge
33        elif method == 'right':
34            sample_x = right_edge
35        else:  # midpoint
36            sample_x = (left_edge + right_edge) / 2
37
38        height = f(sample_x)
39        total_area += height * delta_x
40
41        # Draw rectangle
42        rect = plt.Rectangle((left_edge, 0), delta_x, height,
43                            facecolor=color, edgecolor='black',
44                            alpha=0.4, linewidth=1)
45        ax.add_patch(rect)
46
47        # Mark sample point
48        ax.plot(sample_x, height, 'ko', markersize=6)
49
50    ax.set_xlabel('x')
51    ax.set_ylabel('f(x)')
52    ax.set_title(f'{method.capitalize()} Riemann Sum (n={n}): Area ≈ {total_area:.4f}')
53    ax.legend()
54    ax.grid(True, alpha=0.3)
55    ax.set_xlim(a - 0.1, b + 0.1)
56    ax.set_ylim(0, max(y_curve) * 1.1)
57
58    return total_area
59
60# Create visualization comparing all three methods
61fig, axes = plt.subplots(1, 3, figsize=(15, 5))
62
63for ax, method in zip(axes, ['left', 'right', 'midpoint']):
64    visualize_riemann_sum(lambda x: x**2, 0, 2, n=6, method=method, ax=ax)
65
66plt.tight_layout()
67plt.savefig('riemann_comparison.png', dpi=150)
68plt.show()

Common Pitfalls

Pitfall 1: Confusing n (number of rectangles) with Δx (width)

They are inversely related: $\\Delta x = (b-a)/n$ . More rectangles means smaller widths. When n increases, Δx decreases.

Pitfall 2: Off-by-one errors with indices

For a left sum, you use endpoints $x_0, x_1, \\ldots, x_{n-1}$ (not including $x_n$ ). For a right sum, you use $x_1, x_2, \\ldots, x_n$ (not including $x_0$ ). Be careful with summation limits!

Pitfall 3: Assuming Riemann sums always converge

For the limit to exist (function to be Riemann integrable), the function must be "reasonably nice" — continuous, or at least bounded with only finitely many discontinuities. Pathological functions may not have well-defined integrals.

Numerical Precision

When computing Riemann sums with very large n on a computer, floating-point roundoff errors can accumulate. For production-quality numerical integration, use adaptive algorithms from libraries like scipy.integrate.

Test Your Understanding

📝Test Your UnderstandingScore: 0/8

Question 1 of 813% Complete

In a left Riemann sum, where is the height of each rectangle determined?

Summary

The area problem — finding the area under a curve — is the motivating question for integral calculus. We solve it by approximating with rectangles and taking a limit.

Key Concepts

Concept	Description
Area Problem	Find area under a curve between two vertical lines
Partition	Divide [a, b] into n subintervals of width Δx = (b-a)/n
Left Riemann Sum Lₙ	Use left endpoints: Σf(xᵢ₋₁)·Δx
Right Riemann Sum Rₙ	Use right endpoints: Σf(xᵢ)·Δx
Midpoint Mₙ	Use midpoints: Σf((xᵢ₋₁+xᵢ)/2)·Δx
Convergence	As n → ∞, all Riemann sums approach the exact area
Definite Integral	∫ₐᵇf(x)dx = lim(n→∞) Σf(x*ᵢ)·Δx

Key Takeaways

Approximation strategy: Use rectangles to approximate curved regions, then refine by using more rectangles
Riemann sums are the sum of rectangle areas: $\\sum f(x^*_i) \\cdot \\Delta x$
Three common types: left, right, and midpoint, differing only in sample point choice
Convergence: all methods approach the same limit — the definite integral
Sigma notation $\\Sigma$ compactly expresses Riemann sums
Practical importance: numerical integration in computers uses these ideas
Connection to ML: Monte Carlo sampling is a randomized Riemann sum

The Essence of Integration:

"The definite integral captures total accumulation by summing infinitely many infinitesimally small contributions — the limit of a process that starts with humble rectangles."

Coming Next: In the next section, we'll explore Left, Right, and Midpoint Rules in more detail, comparing their accuracy and understanding why the midpoint rule converges faster.