What are Partial Differential Equations?

Learning Objectives

By the end of this section, you will be able to:

1. Define what a partial differential equation is and explain how it differs from an ordinary differential equation
2. Identify the key components: dependent variable, independent variables, partial derivatives, and order
3. Classify second-order linear PDEs as elliptic, parabolic, or hyperbolic based on their discriminant
4. Interpret the physical meaning of the heat equation and wave equation
5. Understand why PDEs require both initial conditions and boundary conditions
6. Recognize PDEs in applications from physics, engineering, finance, and machine learning
7. Implement basic finite difference solvers for PDEs in Python

The Big Picture: Why PDEs Matter

"The deep study of nature is the most fruitful source of mathematical discoveries." — Joseph Fourier

While ordinary differential equations describe how quantities change with respect to one variable (usually time), the physical world is far richer. Temperature varies across both space and time. Waves propagate through two or three dimensions. Electromagnetic fields permeate all of space. To describe these phenomena, we need equations that involve derivatives with respect to multiple independent variables.

PDEs: The Language of Continuous Phenomena

PDEs are the mathematical language for describing:

Historical Origins

The study of PDEs emerged from the great scientific questions of the 18th and 19th centuries: How does heat flow? How do waves propagate? What determines the shape of a vibrating membrane?

Key Historical Developments

What is a Partial Differential Equation?

A partial differential equation is an equation that contains partial derivatives of an unknown function of two or more independent variables.

Anatomy of a PDE

Consider the heat equation:

Order of a PDE

The order of a PDE is the order of the highest partial derivative appearing in the equation.

ODE vs PDE: The Key Distinction

Understanding the difference between ODEs and PDEs is fundamental. The distinction goes beyond notation to the very nature of the problems.

Notation and Terminology

PDEs use several notational conventions. Familiarity with these is essential for reading the literature.

Partial Derivative Notation

Differential Operators

Classification of Second-Order PDEs

Second-order linear PDEs are classified into three types based on a discriminant condition. This classification determines the fundamental behavior of solutions.

Consider the general second-order linear PDE in two variables:

The discriminant $D = B^2 - 4AC$ determines the type:

The Heat Equation: Diffusion in Action

The heat equation is the prototypical parabolic PDE. It describes how temperature (or any diffusing quantity) evolves over time.

Physical Interpretation

The heat equation says: Temperature tends to equalize.

• If a point is hotter than its neighbors (negative curvature), $\nabla^2 u < 0$, so $\partial u/\partial t < 0$: it cools down.
• If a point is colder than its neighbors (positive curvature), $\nabla^2 u > 0$, so $\partial u/\partial t > 0$: it warms up.
• Sharp peaks and valleys smooth out over time — this is diffusion.

The Wave Equation: Propagation Without Diffusion

The wave equation is the prototypical hyperbolic PDE. It describes how disturbances propagate through a medium at a finite speed.

Physical Interpretation

The wave equation says: Curvature creates restoring forces.

• If a point is displaced above its neighbors (negative curvature), the restoring force pulls it down — acceleration is negative.
• If a point is displaced below its neighbors (positive curvature), the restoring force pushes it up — acceleration is positive.
• This leads to oscillatory motion that propagates at speed c.

Key Difference from Heat Equation

Real-World Applications

PDEs are everywhere in science and engineering. The same mathematical structures describe phenomena from quantum mechanics to option pricing.

Machine Learning Connections

PDEs have become central to modern machine learning, appearing in both theoretical foundations and practical algorithms.

1. Diffusion Models (Stable Diffusion, DALL-E)

State-of-the-art image generation uses score-based diffusion, which is fundamentally connected to the heat equation:

The model learns to reverse a diffusion process, gradually converting noise into coherent images by following the score (gradient of log probability).

2. Physics-Informed Neural Networks (PINNs)

PINNs embed PDEs directly into the loss function:

where $\mathcal{L}_{physics}$ penalizes violations of the governing PDE. This allows neural networks to:

• Learn from sparse data by encoding physical constraints
• Solve forward and inverse PDE problems
• Discover unknown parameters in physical laws

3. Neural Operators (Fourier Neural Operator)

Instead of learning point-wise mappings, neural operators learn solution operators for entire families of PDEs:

This enables real-time prediction of PDE solutions for new initial/boundary conditions without re-solving.

4. Graph Neural Networks and the Laplacian

Many GNN architectures are based on the discrete Laplacian:

This is a discrete heat equation! Information diffuses across the graph, analogous to heat spreading through a material.

Python Implementation

Solving the Heat Equation Numerically

1import numpy as np
2import matplotlib.pyplot as plt
3from matplotlib import animation
4
5def solve_heat_equation_1d(L=1.0, T=0.5, Nx=50, Nt=1000, alpha=0.01):
6    """
7    Solve the 1D heat equation using finite differences.
8
9    ∂u/∂t = α ∂²u/∂x²
10
11    with boundary conditions u(0,t) = u(L,t) = 0
12    and initial condition u(x,0) = sin(πx/L)
13    """
14    # Discretization
15    dx = L / (Nx - 1)
16    dt = T / Nt
17    x = np.linspace(0, L, Nx)
18
19    # Stability condition (CFL): dt ≤ dx²/(2α)
20    stability_ratio = alpha * dt / dx**2
21    print(f"Stability ratio r = {stability_ratio:.4f}")
22    if stability_ratio > 0.5:
23        print("WARNING: Solution may be unstable! Reduce dt or increase dx.")
24
25    # Initialize solution array
26    u = np.zeros((Nt + 1, Nx))
27
28    # Initial condition: u(x,0) = sin(πx/L)
29    u[0, :] = np.sin(np.pi * x / L)
30
31    # Time-stepping (explicit finite difference)
32    r = alpha * dt / dx**2
33    for n in range(Nt):
34        # Interior points
35        u[n + 1, 1:-1] = u[n, 1:-1] + r * (
36            u[n, 2:] - 2 * u[n, 1:-1] + u[n, :-2]
37        )
38        # Boundary conditions (already zero)
39        u[n + 1, 0] = 0
40        u[n + 1, -1] = 0
41
42    return x, u, dt
43
44# Solve the heat equation
45x, u_history, dt = solve_heat_equation_1d()
46
47# Compare with analytical solution: u(x,t) = sin(πx)·e^(-α·π²·t)
48alpha = 0.01
49t_final = 0.5
50analytical = np.sin(np.pi * x) * np.exp(-alpha * np.pi**2 * t_final)
51
52# Plot comparison at final time
53plt.figure(figsize=(10, 5))
54plt.plot(x, u_history[-1], 'b-', linewidth=2, label='Numerical')
55plt.plot(x, analytical, 'r--', linewidth=2, label='Analytical')
56plt.xlabel('x')
57plt.ylabel('u(x, t_final)')
58plt.title('Heat Equation: Numerical vs Analytical Solution')
59plt.legend()
60plt.grid(True, alpha=0.3)
61plt.show()
62
63print(f"Max error: {np.max(np.abs(u_history[-1] - analytical)):.6f}")

Solving the Wave Equation Numerically

1import numpy as np
2import matplotlib.pyplot as plt
3
4def solve_wave_equation_1d(L=1.0, T=2.0, Nx=100, Nt=500, c=1.0):
5    """
6    Solve the 1D wave equation using finite differences.
7
8    ∂²u/∂t² = c² ∂²u/∂x²
9
10    with fixed boundary conditions u(0,t) = u(L,t) = 0
11    and initial conditions:
12        u(x,0) = initial displacement (plucked string)
13        ∂u/∂t(x,0) = 0 (starts from rest)
14    """
15    dx = L / (Nx - 1)
16    dt = T / Nt
17    x = np.linspace(0, L, Nx)
18
19    # CFL stability condition for wave equation: c·dt/dx ≤ 1
20    cfl = c * dt / dx
21    print(f"CFL number = {cfl:.4f}")
22    if cfl > 1:
23        print("WARNING: CFL > 1, solution will be unstable!")
24
25    # Initialize arrays: u_prev, u_curr, u_next
26    u_prev = np.zeros(Nx)
27    u_curr = np.zeros(Nx)
28    u_next = np.zeros(Nx)
29
30    # Initial condition: plucked string (triangle)
31    for i in range(Nx):
32        if x[i] < 0.3:
33            u_curr[i] = x[i] / 0.3
34        else:
35            u_curr[i] = (L - x[i]) / 0.7
36    u_curr *= 0.5  # Amplitude
37
38    # First time step (using ∂u/∂t = 0 at t=0)
39    r = (c * dt / dx)**2
40    for i in range(1, Nx - 1):
41        u_prev[i] = u_curr[i] + 0.5 * r * (
42            u_curr[i + 1] - 2 * u_curr[i] + u_curr[i - 1]
43        )
44
45    # Store solution history
46    history = [u_curr.copy()]
47
48    # Time-stepping (leapfrog scheme)
49    for n in range(Nt):
50        for i in range(1, Nx - 1):
51            u_next[i] = (
52                2 * u_curr[i] - u_prev[i]
53                + r * (u_curr[i + 1] - 2 * u_curr[i] + u_curr[i - 1])
54            )
55        # Boundary conditions
56        u_next[0] = 0
57        u_next[-1] = 0
58
59        # Rotate arrays
60        u_prev[:] = u_curr
61        u_curr[:] = u_next
62
63        if n % 50 == 0:
64            history.append(u_curr.copy())
65
66    return x, history
67
68# Solve and visualize
69x, history = solve_wave_equation_1d()
70
71# Plot several time snapshots
72plt.figure(figsize=(12, 6))
73for i, u in enumerate(history[::2]):
74    alpha = 0.3 + 0.7 * i / len(history[::2])
75    plt.plot(x, u, alpha=alpha, label=f't = {i*0.4:.1f}')
76plt.xlabel('x')
77plt.ylabel('u(x,t)')
78plt.title('Wave Equation: Propagation of a Plucked String')
79plt.legend(loc='upper right')
80plt.grid(True, alpha=0.3)
81plt.show()

Common Pitfalls

Test Your Understanding

Summary

Partial differential equations extend the power of calculus to describe phenomena that vary in multiple dimensions. They are the mathematical foundation for physics, engineering, and increasingly, machine learning.

Key Concepts

Key Takeaways

1. PDEs involve partial derivatives with respect to multiple independent variables, unlike ODEs which have only one
2. Second-order linear PDEs are classified as elliptic, parabolic, or hyperbolic based on the discriminant B² - 4AC
3. The heat equation (parabolic) describes diffusion — sharp features smooth out over time
4. The wave equation (hyperbolic) describes propagation — information travels at finite speed
5. PDEs require boundary conditions in addition to initial conditions to be well-posed
6. Modern ML applications include diffusion models, PINNs, and neural operators
7. Numerical solutions must respect stability conditions (CFL) to avoid blow-up

Coming Next: In the next section, we'll explore the Classification of PDEs in more depth. You'll learn how to determine the type of a PDE from its coefficients and understand the physical implications of each classification.