Basic Integration Rules

Learning Objectives

By the end of this section, you will be able to:

1. Understand integration as the reverse process of differentiation
2. Apply the Power Rule for Integration to find antiderivatives of polynomial terms
3. Use the Constant Multiple, Sum, and Difference Rules to integrate complex expressions
4. Recognize and integrate basic exponential, logarithmic, and trigonometric functions
5. Build a working table of fundamental integrals for reference
6. Connect integration rules to their applications in probability, optimization, and machine learning

The Big Picture: The Toolkit for Finding Antiderivatives

"Just as multiplication has its inverse in division, differentiation has its inverse in integration. The rules of integration are simply the derivative rules read backwards."

In the previous section, we introduced the concept of the antiderivative and the indefinite integral. We saw that finding $\int f(x) \, dx$ means finding a function $F(x)$ such that $F'(x) = f(x)$.

But how do we actually find antiderivatives? Do we need to guess and check each time? Fortunately, no. Just as differentiation has systematic rules (power rule, product rule, chain rule), integration has corresponding rules that make finding antiderivatives systematic.

This section develops the fundamental integration rules that form the building blocks for all antiderivative calculations. These rules are not just mathematical abstractions — they appear throughout:

• Physics: Finding position from velocity, work from force
• Probability: Computing expected values, normalizing densities
• Economics: Total cost from marginal cost, consumer surplus
• Machine Learning: Loss function normalization, computing gradients of integrals

Historical Context: From Art to Science

Before Newton and Leibniz formalized calculus in the 17th century, finding areas under curves was an ad hoc process. Each problem required clever geometric insights specific to that curve. Archimedes (287–212 BC) famously computed the area under a parabola using the "method of exhaustion," but this required immense ingenuity.

The revolutionary insight of Newton and Leibniz was recognizing that finding areas (integration) is the inverse of finding slopes (differentiation). This connection — the Fundamental Theorem of Calculus — transformed integration from an art into a science.

Leibniz's Contribution: Systematic Rules

Gottfried Wilhelm Leibniz developed the notation $\int f(x) \, dx$ and systematically catalogued integration rules. His approach was to think of integration as summation of infinitesimals (hence the elongated S symbol $\int$ for "sum").

Leibniz recognized that if you know how to differentiate a function, you can reverse the process to integrate. His systematic tables of derivatives and antiderivatives formed the basis of what we teach today.

Integration as Reverse Differentiation

The key to understanding integration rules is to think of them as derivative rules read backwards. Consider this simple principle:

Let's see how this works with a simple example:

The Power Rule for Integration

The most fundamental integration rule corresponds to reversing the power rule for derivatives. Recall that:

To reverse this, we need to find a function whose derivative is $x^n$. We can see that:

Why n ≠ -1?

When $n = -1$, the formula gives $\frac{x^0}{0}$, which is undefined. The integral of $x^{-1} = \frac{1}{x}$ is a special case:

This follows because $\frac{d}{dx}[\ln|x|] = \frac{1}{x}$.

Examples of the Power Rule

Interactive: Power Rule Explorer

Explore how the power rule works by adjusting the exponent. Watch how the antiderivative changes:

The Constant Multiple Rule

Just as constants "pass through" derivatives, they also pass through integrals. This is because differentiation and integration are linear operations.

Why This Works

If $F'(x) = f(x)$, then by the constant multiple rule for derivatives:

Therefore, $k \cdot F(x)$ is an antiderivative of $k \cdot f(x)$.

Examples

Sum and Difference Rules

Integration distributes over addition and subtraction, just like differentiation does.

Why These Work

If $F'(x) = f(x)$ and $G'(x) = g(x)$, then:

So $F(x) + G(x)$ is an antiderivative of $f(x) + g(x)$.

Example: Integrating a Polynomial

Exponential Integrals

Exponential functions have particularly elegant integration rules because they are closely related to their own derivatives.

The Natural Exponential

Recall that $e^x$ is its own derivative: $\frac{d}{dx}[e^x] = e^x$. Reversing this:

This is one of the most beautiful results in calculus: $e^x$ is its own antiderivative!

General Exponential Base

For a general base $a > 0, a \neq 1$:

Reversing this (and dividing both sides by ln(a)):

Examples

Trigonometric Integrals

The basic trigonometric integrals follow directly from reversing the derivative rules for sine, cosine, and their related functions.

Understanding the Signs

The negative signs in some of these formulas come from the derivative rules. For example:

• $\frac{d}{dx}[\cos(x)] = -\sin(x)$, so $\frac{d}{dx}[-\cos(x)] = \sin(x)$
• $\frac{d}{dx}[\cot(x)] = -\csc^2(x)$, so $\frac{d}{dx}[-\cot(x)] = \csc^2(x)$

Complete Integration Table

Here is a comprehensive reference table of the basic integration rules covered in this section. This table is your toolkit for finding antiderivatives.

Worked Examples

Example 1: Combined Rules

Example 2: Rewriting Before Integrating

Example 3: Fractional Exponents

Machine Learning Connections

The basic integration rules appear throughout machine learning, often in ways that might not be immediately obvious. Understanding these connections helps you see why calculus is essential for ML.

Probability Density Normalization

For a probability density function (PDF) to be valid, it must integrate to 1 over its domain:

The famous Gaussian normalization constant $\frac{1}{\sqrt{2\pi}\sigma}$ comes from solving:

While this particular integral requires advanced techniques (polar coordinates), the concept relies on the exponential integration rules.

Loss Functions and Regularization

Many ML loss functions involve integrals over probability distributions:

• Cross-Entropy: $H(p, q) = -\int p(x) \log q(x) \, dx$
• KL Divergence: $D_{KL}(p || q) = \int p(x) \log\frac{p(x)}{q(x)} \, dx$
• Expected Loss: $\mathbb{E}[L] = \int L(y, \hat{y}) p(y) \, dy$

The integration rules help us understand and sometimes analytically compute these quantities.

L2 Regularization

L2 regularization adds a penalty term $\lambda \sum w_i^2$. In the continuous limit (for function learning), this becomes:

The power rule for integration ($\int x^2 dx$) underlies the theory connecting weight decay to smoothness regularization.

Softmax and the Log-Sum-Exp

The softmax function normalizes exponentials:

In continuous settings, the denominator becomes an integral $\int e^{f(x)} dx$, requiring exponential integration. This appears in energy-based models and continuous normalizing flows.

Python Implementation

Demonstrating Integration Rules

Let's use Python to verify and explore the basic integration rules:

1import numpy as np
2from scipy import integrate
3import sympy as sp
4
5def demonstrate_integration_rules():
6    """
7    Demonstrate basic integration rules using symbolic and numerical methods.
8    These rules are the foundation of antidifferentiation.
9    """
10    x = sp.Symbol('x')
11
12    # Power Rule: ∫ x^n dx = x^(n+1)/(n+1) + C
13    print("=== Power Rule ===")
14    n_values = [0, 1, 2, 3, -2, 0.5]
15    for n in n_values:
16        integrand = x**n
17        antiderivative = sp.integrate(integrand, x)
18        print(f"∫ x^{n} dx = {antiderivative} + C")
19
20    # Constant Multiple Rule: ∫ k·f(x) dx = k·∫ f(x) dx
21    print("\n=== Constant Multiple Rule ===")
22    k = 5
23    f = x**2
24    integral_kf = sp.integrate(k * f, x)
25    k_times_integral_f = k * sp.integrate(f, x)
26    print(f"∫ {k}·x² dx = {integral_kf} + C")
27    print(f"Verification: {k}·∫ x² dx = {k_times_integral_f} + C")
28
29    # Sum/Difference Rule: ∫ [f(x) ± g(x)] dx = ∫ f(x) dx ± ∫ g(x) dx
30    print("\n=== Sum Rule ===")
31    f, g = x**3, sp.sin(x)
32    integral_sum = sp.integrate(f + g, x)
33    sum_of_integrals = sp.integrate(f, x) + sp.integrate(g, x)
34    print(f"∫ (x³ + sin(x)) dx = {integral_sum} + C")
35
36    # Exponential Integrals
37    print("\n=== Exponential Integrals ===")
38    print(f"∫ e^x dx = {sp.integrate(sp.exp(x), x)} + C")
39    print(f"∫ 2^x dx = {sp.integrate(2**x, x)} + C")
40
41    # Trigonometric Integrals
42    print("\n=== Trigonometric Integrals ===")
43    trig_functions = [sp.sin(x), sp.cos(x), sp.sec(x)**2, sp.csc(x)**2]
44    for func in trig_functions:
45        print(f"∫ {func} dx = {sp.integrate(func, x)} + C")
46
47demonstrate_integration_rules()

Integration in Machine Learning

Here's how integration rules appear in ML contexts:

1import numpy as np
2from scipy import integrate
3from scipy.optimize import minimize
4import matplotlib.pyplot as plt
5
6def integration_in_ml():
7    """
8    Integration rules appear throughout machine learning,
9    from loss function normalization to probability densities.
10    """
11
12    # 1. Normalizing Probability Densities
13    # For a valid PDF, ∫ f(x) dx = 1
14    print("=== Probability Density Normalization ===")
15
16    def unnormalized_gaussian(x, mu=0, sigma=1):
17        """Unnormalized Gaussian: exp(-x²/(2σ²))"""
18        return np.exp(-((x - mu)**2) / (2 * sigma**2))
19
20    # Using power rule and properties: ∫ exp(-x²/2) dx = √(2π)
21    sigma = 1.0
22    normalization_constant = np.sqrt(2 * np.pi) * sigma
23    print(f"Normalization constant Z = √(2π)σ = {normalization_constant:.4f}")
24
25    # 2. Expected Value Computation
26    # E[X] = ∫ x·f(x) dx - uses power rule and constant multiple
27    print("\n=== Expected Value (uses integration rules) ===")
28
29    def pdf(x, mu=2, sigma=1):
30        return (1 / (np.sqrt(2*np.pi) * sigma)) * \
31               np.exp(-((x - mu)**2) / (2 * sigma**2))
32
33    # E[X] = ∫ x·f(x) dx
34    expected_value, _ = integrate.quad(
35        lambda x: x * pdf(x, mu=2, sigma=1), -10, 10
36    )
37    print(f"E[X] for N(2, 1) = {expected_value:.4f} (should be 2)")
38
39    # 3. Loss Function - Cross-Entropy
40    # Cross-entropy uses logarithm integral rule
41    print("\n=== Cross-Entropy Loss (uses log integral) ===")
42
43    def cross_entropy_continuous(p, q):
44        """H(p, q) = -∫ p(x) log(q(x)) dx"""
45        def integrand(x):
46            p_val = pdf(x, mu=0, sigma=1)
47            q_val = pdf(x, mu=0.5, sigma=1.2)
48            if q_val > 1e-10:
49                return -p_val * np.log(q_val)
50            return 0
51        result, _ = integrate.quad(integrand, -10, 10)
52        return result
53
54    ce = cross_entropy_continuous(None, None)
55    print(f"Cross-entropy between two Gaussians: {ce:.4f}")
56
57    # 4. Regularization Terms (from integration)
58    print("\n=== L2 Regularization (power rule) ===")
59
60    # L2 penalty: λ·∫ ||w||² dP ≈ λ·Σw²
61    # The sum is a Riemann sum approximation to an integral
62    weights = np.array([0.5, -0.3, 0.8, -0.2, 0.1])
63    lambda_reg = 0.01
64
65    # L2 regularization uses x² integration: ∫ x² dx = x³/3
66    l2_penalty = lambda_reg * np.sum(weights**2)
67    print(f"L2 penalty for weights: {l2_penalty:.4f}")
68
69    # 5. Softmax Normalization
70    print("\n=== Softmax (exponential integral) ===")
71
72    logits = np.array([2.0, 1.0, 0.1])
73    # Softmax: exp(x_i) / Σexp(x_j)
74    # Continuous version: exp(f(x)) / ∫ exp(f(t)) dt
75    exp_logits = np.exp(logits)
76    Z = np.sum(exp_logits)  # Partition function (discrete integral)
77    softmax_probs = exp_logits / Z
78    print(f"Softmax probabilities: {softmax_probs}")
79    print(f"Sum (should be 1): {np.sum(softmax_probs):.4f}")
80
81integration_in_ml()

Common Mistakes to Avoid

Test Your Understanding

Summary

The basic integration rules are the foundation for finding antiderivatives. They emerge directly from reversing differentiation rules, giving us a systematic approach to integration.

Key Formulas

Key Takeaways

1. Integration is reverse differentiation: Every derivative rule can be reversed to create an integration rule.
2. The power rule is fundamental: $\int x^n dx = \frac{x^{n+1}}{n+1} + C$ works for all $n \neq -1$.
3. Integration is linear: Constants factor out, and sums integrate term by term.
4. Always include +C: Indefinite integrals represent families of functions differing by a constant.
5. Verify by differentiating: You can always check your antiderivative by taking its derivative.
6. ML connections are everywhere: These rules underlie probability normalization, loss functions, and regularization.

Coming Next: In Integration by Substitution (u-substitution), we'll learn how to integrate more complex functions by reversing the chain rule — the most powerful technique for transforming integrals into simpler forms.