Boo-AI — Master Artificial Intelligence by Building from Scratch

Learning Objectives

By the end of this section, you will be able to:

Derive the steady-state equation from the heat equation by setting $\partial u/\partial t = 0$
Solve the one-dimensional Laplace equation $u''(x) = 0$ with non-homogeneous Dirichlet boundary conditions
Decompose the full heat equation solution into steady-state and transient components
Interpret the physical meaning of thermal equilibrium and constant heat flux
Calculate the time required to approach steady state within a given tolerance
Connect steady-state solutions to equilibrium distributions in machine learning and statistical mechanics

The Big Picture: Equilibrium and Eternity

"Nature abhors a gradient." — An informal principle of thermodynamics

In the previous sections, we solved the heat equation using Fourier series and watched how temperature distributions evolve over time. We observed that all the transient modes decay exponentially, with higher modes vanishing faster. But what happens as $t \to \infty$ ?

The answer is profound: the system approaches a steady state — an equilibrium temperature distribution that no longer changes with time. This final state is determined entirely by the boundary conditions, independent of the initial temperature distribution.

The Central Question

When we hold the ends of a rod at fixed temperatures $T_0$ and $T_L$ , what is the final temperature distribution after all transients have decayed?

The answer: a linear temperature profile connecting the two boundary temperatures.

u_{ss}(x) = T_0 + \frac{T_L - T_0}{L} x

Why Steady States Matter

Engineering Design

Most thermal systems are designed to operate at steady state. Heat sinks, insulation, and cooling systems are all analyzed using steady-state heat transfer equations.

Mathematical Simplification

Setting $\partial u/\partial t = 0$ reduces the PDE to an ODE, making the problem much simpler to solve analytically.

Reference Frame

The transient solution is most naturally expressed as departures from steady state, simplifying the boundary conditions to homogeneous ones.

Machine Learning

Steady states correspond to equilibrium distributions in diffusion models and stationary distributions of Markov processes—concepts central to modern generative AI.

The Steady-State Equation

The heat equation describes how temperature changes over time:

\frac{\partial u}{\partial t} = \alpha \frac{\partial^2 u}{\partial x^2}

At steady state, the temperature at every point stops changing: $\partial u/\partial t = 0$ . This means the right-hand side must also be zero:

The Steady-State Heat Equation (1D)

\frac{d^2 u_{ss}}{dx^2} = 0

This is Laplace's equation in one dimension. It says that the second spatial derivative of temperature is zero—the temperature profile has no curvature.

Solving the Steady-State Equation

The equation $u''_{ss}(x) = 0$ is one of the simplest differential equations. Integrating once:

u'_{ss}(x) = A \quad \text{(constant)}

Integrating again:

u_{ss}(x) = Ax + B

The steady-state temperature is a linear function of position! The constants $A$ and $B$ are determined by boundary conditions.

Why Linear?

A linear temperature profile is the only shape with zero curvature that connects two fixed boundary values. Physically, it represents constant heat flux through every cross-section of the rod.

Non-Homogeneous Boundary Conditions

In real applications, the boundary temperatures are rarely zero. Consider a rod with:

Non-Homogeneous Dirichlet Conditions

u(0, t) = T_0

— Left end at temperature

T_0

u(L, t) = T_L

— Right end at temperature

T_L

Finding the Steady-State Solution

We have $u_{ss}(x) = Ax + B$ . Applying boundary conditions:

Condition	Equation	Result
u_ss(0) = T₀	A·0 + B = T₀	B = T₀
u_ss(L) = T_L	A·L + T₀ = T_L	A = (T_L - T₀)/L

Therefore, the steady-state solution is:

Steady-State Temperature Distribution

u_{ss}(x) = T_0 + \frac{T_L - T_0}{L} x

A straight line from $(0, T_0)$ to $(L, T_L)$

Steady-State Temperature Explorer

Left Temperature T₀20°C

Right Temperature T_L80°C

Rod Length L1.0 m

Temperature Gradient

dT/dx = 60.00 °C/m

Heat Flux (q = -k dT/dx)

q = 60.00 W/m² (→)

Center Temperature

T(L/2) = 50.0 °C

Steady-State Solution

u_ss(x) = 20 + 60.00x = 20 + 60x/1.0

Physical Meaning of Steady State

The linear temperature profile has deep physical significance. Let's examine what it tells us about heat flow.

Constant Heat Flux

Fourier's law states that heat flux (power per unit area) is proportional to the temperature gradient:

q = -k \frac{du}{dx}

For our steady-state solution:

q = -k \cdot \frac{T_L - T_0}{L} = \frac{k(T_0 - T_L)}{L}

This is constant throughout the rod. Heat enters at the hot end and exits at the cold end at the same rate—no accumulation anywhere.

Hot End (x = 0)

Heat enters at rate $q$

Interior

Same flux $q$ passes through

Cold End (x = L)

Heat exits at rate $q$

Thermal Resistance

By analogy with Ohm's law ( $V = IR$ ), we can write:

\Delta T = q \cdot R_{th}

where the thermal resistance is:

R_{th} = \frac{L}{kA}

The Electrical Analogy

Temperature difference ↔ Voltage difference, Heat flow ↔ Current, Thermal resistance ↔ Electrical resistance. This analogy makes thermal circuit analysis intuitive for anyone familiar with electronics.

Combining Transient and Steady-State Solutions

The real power of understanding steady-state solutions emerges when we tackle the full problem with non-homogeneous boundary conditions.

The Decomposition Strategy

We decompose the solution into two parts:

Solution Decomposition

u(x, t) = u_{ss}(x) + v(x, t)

Steady-state

u_{ss}(x)

Satisfies $u''_{ss} = 0$
Satisfies non-homogeneous BCs
Time-independent

Transient

v(x, t)

Satisfies the heat equation
Has homogeneous BCs: v(0,t) = v(L,t) = 0
Decays to zero as t → ∞

Why This Works

Substituting $u = u_{ss} + v$ into the heat equation:

\frac{\partial(u_{ss} + v)}{\partial t} = \alpha \frac{\partial^2(u_{ss} + v)}{\partial x^2}

Since $u_{ss}$ is time-independent and $u''_{ss} = 0$ :

\frac{\partial v}{\partial t} = \alpha \frac{\partial^2 v}{\partial x^2}

The transient part $v$ satisfies the heat equation with homogeneous boundary conditions—exactly the problem we solved using Fourier series!

Initial Condition Transformation

If the original initial condition is $u(x, 0) = f(x)$ , then:

v(x, 0) = f(x) - u_{ss}(x)

We compute the Fourier coefficients for $v(x, 0)$ , and the full solution is:

u(x, t) = u_{ss}(x) + \sum_{n=1}^{\infty} B_n \sin\left(\frac{n\pi x}{L}\right) e^{-\alpha(n\pi/L)^2 t}

Approach to Steady State

Timet = 0.00s (0.0% to steady state)

Transient Magnitude

|v|_max = 72.20°C

Characteristic Time

τ = L²/(απ²) = 5.07s

Time Elapsed

t = 0.00τ

Watch how the temperature distribution evolves from the initial condition toward the steady-state linear profile. All transient modes decay exponentially, with higher modes vanishing faster.

Physical Interpretation

The approach to steady state reveals fundamental physics about heat diffusion and equilibrium.

Time Scale to Reach Equilibrium

The slowest-decaying mode is $n = 1$ , with decay rate $\alpha(\pi/L)^2$ . The characteristic time scale is:

Characteristic Relaxation Time

\tau = \frac{L^2}{\alpha \pi^2}

After time $5\tau$ , the transient has decayed to about 1% of its initial magnitude.

Property	Effect on τ	Physical Reason
Larger L	τ ∝ L²	Heat must travel farther to escape
Smaller α	τ ∝ 1/α	Slower diffusion means slower equilibration
Higher modes	Decay n² times faster	Sharp features smooth out quickly

Memory and Forgetting

A remarkable feature of the heat equation: the system forgets its initial condition as $t \to \infty$ . No matter what the starting temperature distribution, the final state is determined solely by the boundary conditions.

The Forgetting Property: All Paths Lead to Equilibrium

t = 0.00s | Max deviation: 82.0°C

The Forgetting Property: No matter the initial temperature distribution, all solutions converge to the same steady state—determined only by boundary conditions.

This irreversible loss of information about initial conditions connects to entropy increase and the arrow of time.

The Forgetting Property

Heat diffusion is an information-destroying process. The initial temperature distribution contains information (sharp gradients, complex patterns), but steady state is a simple linear profile. This irreversible loss of information connects to:

Entropy increase (Second Law of Thermodynamics)
The arrow of time in physics
The forward process in diffusion models (noise destroys information)

Engineering Applications

Steady-state heat transfer analysis is fundamental to engineering design across many fields.

1. Building Insulation

In winter, a wall separates indoor temperature $T_{in}$ from outdoor $T_{out}$ . At steady state:

q = \frac{k A (T_{in} - T_{out})}{L} = \frac{T_{in} - T_{out}}{R_{th}}

Higher $R_{th}$ (thicker walls, lower thermal conductivity) reduces heat loss.

2. Electronic Cooling

Heat sinks remove power $P$ from processors. The junction temperature is:

T_{junction} = T_{ambient} + P \cdot R_{th}

Keeping the junction cool requires minimizing thermal resistance (better thermal paste, larger heat sinks, active cooling).

3. Composite Walls and Multilayer Systems

For multiple layers in series (like wall + insulation + siding), thermal resistances add:

R_{total} = R_1 + R_2 + R_3 + \cdots

Within each layer, the temperature profile is linear with slope proportional to the heat flux.

Multilayer Wall Heat Transfer

Indoor Temperature22°C

Outdoor Temperature-5°C

Drywall1.5 cm

k = 0.16 W/(m·K), R = 0.094 m²K/W

Insulation10 cm

k = 0.04 W/(m·K), R = 2.500 m²K/W

Brick10 cm

k = 0.8 W/(m·K), R = 0.125 m²K/W

Total R-Value

2.719 m²K/W

Heat Flux

9.9 W/m²

Wall Thickness

21.5 cm

ΔT

27.0°C

At steady state, the temperature profile is piecewise linear, with different slopes in each layer depending on thermal conductivity. The heat flux q is constant throughout.

Machine Learning Connections

The concept of steady state connects deeply to machine learning, particularly in the context of diffusion models and Markov processes.

1. Stationary Distributions in Diffusion Models

In score-based diffusion models (like DALL-E and Stable Diffusion), the forward process adds noise to data according to:

dx = -\frac{1}{2}\beta(t) x \, dt + \sqrt{\beta(t)} \, dW

As $t \to \infty$ , the distribution approaches a stationary distribution—analogous to the steady state of the heat equation. For the Ornstein-Uhlenbeck process, this is a Gaussian: $\mathcal{N}(0, I)$ .

Heat Equation Analogy

Initial condition → Data distribution
Steady state → Standard Gaussian
Mode decay → Information destruction
τ (time constant) → Noise schedule

Diffusion Model Training

Forward: Data → Noise (like heat diffusion)
Reverse: Noise → Data (learned)
Boundary: Gaussian at t = T
Score: ∇ log p(x, t)

2. MCMC and Equilibrium Sampling

Markov Chain Monte Carlo methods rely on reaching a stationary distribution. The chain must "burn in" (reach equilibrium) before samples are valid—analogous to waiting for transients to decay.

3. Regularization as Diffusion

In neural networks, L2 regularization (weight decay) can be viewed as adding a diffusion term that pushes weights toward zero. The trained network represents a balance between fitting the data and the "steady state" of small weights.

The Equilibrium Perspective

Many machine learning algorithms can be understood as finding equilibrium states: gradient descent finds local minima, MCMC samples from stationary distributions, and diffusion models learn to reverse equilibration. The heat equation provides the simplest mathematical model of this equilibration process.

Python Implementation

Let's implement the steady-state solution and visualize how the temperature approaches equilibrium.

Steady-State and Transient Heat Equation Solutions

🐍steady_state_heat.py

Explanation(6)

Code(146)

4Steady-State Solution Formula

When ∂u/∂t = 0, the heat equation reduces to u''(x) = 0. Integrating twice gives u(x) = Ax + B. The boundary conditions determine A and B, yielding a linear temperature profile.

15Transient + Steady-State Decomposition

The key insight: we decompose u(x,t) = u_ss(x) + v(x,t) where u_ss satisfies the non-homogeneous BCs and v satisfies homogeneous BCs (v = 0 at boundaries). This lets us use Fourier series for v.

27Initial Condition Transformation

If the original initial condition is f(x), then v(x,0) = f(x) - u_ss(x). This ensures the total solution u(x,0) = u_ss(x) + v(x,0) = f(x) as required.

50Fourier Coefficients for Transient Part

We compute the Fourier sine coefficients of v(x,0) = f(x) - u_ss(x). The factor 2/L comes from orthogonality of sine functions on [0, L].

88Time to Reach Steady State

The fundamental mode (n=1) has the slowest decay rate. The characteristic time τ = L²/(απ²) tells us how quickly the system approaches equilibrium. After ~5τ, we're at 99% of steady state.

95Energy Conservation Verification

In steady state, the heat flux q = -k(du/dx) must be constant throughout the rod. A linear temperature profile has constant slope, hence constant flux—no energy accumulates anywhere.

140 lines without explanation

1import numpy as np
2import matplotlib.pyplot as plt
3from matplotlib.animation import FuncAnimation
4
5def steady_state_temperature(x, T0, TL, L=1.0):
6    """
7    Compute the steady-state temperature distribution.
8
9    When ∂u/∂t = 0, the heat equation becomes u''(x) = 0.
10    With boundary conditions u(0) = T0 and u(L) = TL,
11    the solution is linear: u_ss(x) = T0 + (TL - T0) * x / L
12    """
13    return T0 + (TL - T0) * x / L
14
15
16def transient_plus_steady_state(x, t, T0, TL, L=1.0, alpha=0.01,
17                                  n_modes=50, initial_condition='zero'):
18    """
19    Full solution: u(x,t) = u_ss(x) + v(x,t)
20
21    where u_ss(x) is the steady-state solution and v(x,t) is the
22    transient solution that satisfies v(0,t) = v(L,t) = 0.
23    """
24    # Steady-state part
25    u_ss = steady_state_temperature(x, T0, TL, L)
26
27    # Initial condition for v: v(x,0) = f(x) - u_ss(x)
28    if initial_condition == 'zero':
29        # f(x) = 0 initially, so v(x,0) = -u_ss(x)
30        f_minus_uss = lambda xi: -steady_state_temperature(xi, T0, TL, L)
31    elif initial_condition == 'uniform':
32        # f(x) = (T0 + TL)/2 uniformly
33        T_avg = (T0 + TL) / 2
34        f_minus_uss = lambda xi: T_avg - steady_state_temperature(xi, T0, TL, L)
35    elif initial_condition == 'hot_center':
36        # f(x) = T_max at center, linear decay to boundaries
37        T_max = max(T0, TL) + 50
38        f = lambda xi: T_max * (1 - np.abs(2*xi/L - 1))
39        f_minus_uss = lambda xi: f(xi) - steady_state_temperature(xi, T0, TL, L)
40    else:
41        f_minus_uss = lambda xi: np.zeros_like(xi)
42
43    # Compute Fourier coefficients for v(x,0) = f(x) - u_ss(x)
44    n_points = 1000
45    x_int = np.linspace(0, L, n_points)
46    dx = L / (n_points - 1)
47
48    # Transient part using Fourier series
49    v = np.zeros_like(x)
50    for n in range(1, n_modes + 1):
51        # Compute Fourier sine coefficient
52        integrand = f_minus_uss(x_int) * np.sin(n * np.pi * x_int / L)
53        b_n = (2 / L) * np.trapz(integrand, dx=dx)
54
55        if abs(b_n) < 1e-12:
56            continue
57
58        # Decay factor for this mode
59        lambda_n = (n * np.pi / L) ** 2
60        decay = np.exp(-alpha * lambda_n * t)
61
62        # Add contribution of this mode
63        v += b_n * np.sin(n * np.pi * x / L) * decay
64
65    return u_ss + v
66
67
68def visualize_approach_to_steady_state(T0=0, TL=100, L=1.0, alpha=0.01):
69    """
70    Visualize how the temperature approaches steady state over time.
71    """
72    x = np.linspace(0, L, 200)
73    times = [0, 0.5, 1.0, 2.0, 5.0, 10.0, 50.0]
74
75    plt.figure(figsize=(12, 6))
76
77    # Plot steady state (dashed line)
78    u_ss = steady_state_temperature(x, T0, TL, L)
79    plt.plot(x, u_ss, 'k--', linewidth=2.5, label='Steady State', zorder=10)
80
81    # Plot temperature at different times
82    colors = plt.cm.plasma(np.linspace(0.1, 0.9, len(times)))
83    for t, color in zip(times, colors):
84        u = transient_plus_steady_state(x, t, T0, TL, L, alpha,
85                                         initial_condition='hot_center')
86        alpha_val = 0.3 + 0.7 * (1 - t / max(times)) if t > 0 else 1.0
87        plt.plot(x, u, color=color, linewidth=2, alpha=alpha_val,
88                 label=f't = {t}')
89
90    plt.xlabel('Position x', fontsize=12)
91    plt.ylabel('Temperature u(x,t)', fontsize=12)
92    plt.title('Approach to Steady-State Temperature Distribution', fontsize=14)
93    plt.legend(loc='best')
94    plt.grid(True, alpha=0.3)
95    plt.xlim(0, L)
96    plt.show()
97
98    # Calculate time to reach steady state (99% of final value)
99    print("\n--- Time to Steady State Analysis ---")
100    tau = L**2 / (alpha * np.pi**2)  # Time constant for mode 1
101    print(f"Characteristic time (mode 1): τ = L²/(απ²) = {tau:.4f}")
102    print(f"Time to 99% of steady state: ~5τ = {5*tau:.4f}")
103    print(f"Time to 99.9% of steady state: ~7τ = {7*tau:.4f}")
104
105
106def energy_conservation_check(T0, TL, L=1.0):
107    """
108    Verify that the steady-state satisfies energy conservation.
109
110    In steady state, the heat flux q = -k du/dx must be constant
111    (no accumulation or depletion of energy at any point).
112    """
113    x = np.linspace(0, L, 100)
114    u_ss = steady_state_temperature(x, T0, TL, L)
115
116    # Numerical derivative (should be constant)
117    du_dx = np.gradient(u_ss, x)
118
119    print("--- Energy Conservation Check ---")
120    print(f"du/dx at x=0:   {du_dx[0]:.6f}")
121    print(f"du/dx at x=L/2: {du_dx[len(x)//2]:.6f}")
122    print(f"du/dx at x=L:   {du_dx[-1]:.6f}")
123    print(f"Analytical value: {(TL - T0)/L:.6f}")
124    print("\n✓ Constant gradient = constant heat flux = steady state!")
125
126
127# Demonstration
128print("=== Steady-State Temperature Distribution ===\n")
129
130# Set boundary conditions
131T0, TL = 20, 80  # Left = 20°C, Right = 80°C
132L = 1.0
133
134# Compute steady state
135x = np.linspace(0, L, 100)
136u_ss = steady_state_temperature(x, T0, TL, L)
137
138print(f"Boundary conditions: T(0) = {T0}°C, T(L) = {TL}°C")
139print(f"\nSteady-state is linear: u_ss(x) = {T0} + {TL-T0}*x/{L}")
140print(f"Temperature at center: u_ss(L/2) = {steady_state_temperature(L/2, T0, TL, L):.1f}°C")
141
142# Verify energy conservation
143energy_conservation_check(T0, TL, L)
144
145# Visualize approach to steady state
146visualize_approach_to_steady_state(T0, TL, L, alpha=0.02)

Common Pitfalls

Confusing Steady State with Equilibrium Temperature

Steady state means $\partial u/\partial t = 0$ , not that the temperature is uniform. With non-homogeneous boundary conditions, there's a temperature gradient even at steady state—heat is constantly flowing, but the temperature at each point is constant.

Ignoring Transient Time Scales

The time to reach steady state scales as $L^2/\alpha$ . For large systems or low diffusivity, this can be very long. Always verify that transients have decayed before treating a system as being at steady state.

Applying to Wrong Boundary Conditions

The linear profile $u_{ss} = T_0 + (T_L - T_0)x/L$ only applies to Dirichlet conditions (fixed temperatures). For Neumann conditions (fixed flux), the steady state is different. For insulated ends, the steady state is uniform temperature.

When Does Steady State Not Exist?

Some systems never reach steady state:

Unbounded domains (heat escapes to infinity)
Time-varying boundary conditions
Internal heat sources that vary with time
Convection-dominated flows with oscillating conditions

Test Your Understanding

Score: 0/8 | Question 1 of 8

What condition defines steady state in the heat equation?

Summary

Steady-state solutions represent the long-time limit of heat diffusion, where all transients have decayed and the temperature no longer changes with time.

Key Equations

Concept	Equation
Steady-state condition	∂u/∂t = 0 → u''(x) = 0
Steady-state solution	u_ss(x) = T₀ + (T_L - T₀)x/L
Heat flux	q = k(T₀ - T_L)/L (constant)
Solution decomposition	u(x,t) = u_ss(x) + v(x,t)
Relaxation time	τ = L²/(απ²)
Time to 99% steady state	t ≈ 5τ

Key Takeaways

At steady state, the heat equation reduces to Laplace's equation $u''(x) = 0$ , giving a linear temperature profile
The steady-state temperature is determined solely by boundary conditions, independent of initial state
Decomposition $u = u_{ss} + v$ separates the problem into steady-state (simple) and transient (Fourier series) parts
The characteristic time $\tau = L^2/(\alpha\pi^2)$ tells us how quickly the system approaches equilibrium
At steady state, heat flux is constant throughout the material—energy flows through without accumulating
The system forgets its initial condition—an irreversible process connected to entropy and the arrow of time
Machine learning connections: stationary distributions in diffusion models, MCMC equilibrium, and regularization as diffusion toward a prior

The Essence of Steady State:

"Eventually, all that remains is the equilibrium—a linear bridge between boundary conditions, through which heat flows endlessly without changing the temperature."

Coming Next: We have now completed our exploration of the heat equation! In the next chapter, we'll explore the Wave Equation—a hyperbolic PDE that describes vibrations, sound, and electromagnetic waves, where energy propagates rather than diffuses.