Linear First-Order Differential Equations

Learning Objectives

By the end of this section, you will be able to:

1. Identify first-order linear differential equations in standard form $y' + P(x)y = Q(x)$
2. Apply the integrating factor method to solve any first-order linear ODE
3. Interpret direction fields and solution curves geometrically
4. Understand the structure of solutions: homogeneous + particular
5. Model real-world phenomena: RC circuits, mixing problems, population dynamics
6. Connect linear ODEs to gradient descent and machine learning optimization
7. Implement numerical solvers in Python using scipy

The Big Picture: Why Linear ODEs Matter

"Differential equations are the language in which the laws of nature are expressed." — Richard Feynman

First-order linear differential equations are among the most important equations in all of applied mathematics. They describe systems where the rate of change of a quantity is proportional to the quantity itself, possibly with an external input.

Where Linear First-Order ODEs Appear

Historical Context

The study of differential equations began in the late 17th century with Newton and Leibniz's development of calculus. First-order linear equations were among the first to be solved systematically.

Key Historical Developments

Mathematical Definition

A first-order linear differential equation is an equation of the form:

What Each Part Means

Why "Linear"?

The equation is called linear because:

1. The dependent variable $y$ and its derivative $dy/dx$ appear only to the first power
2. There are no products like $y \cdot dy/dx$ or terms like $y^2$
3. If $y_1$ and $y_2$ are solutions to the homogeneous equation, so is $c_1 y_1 + c_2 y_2$

Direction Fields & Solution Curves

Before diving into algebraic solutions, let's develop geometric intuition. At every point $(x, y)$ in the plane, the ODE tells us the slope of the solution passing through that point:

By drawing small line segments with these slopes, we create a direction field (or slope field) that shows how solutions flow.

Key Observations from the Direction Field

• Solution curves never cross: Each point has exactly one slope, so solutions are unique
• Equilibrium solutions: Where $dy/dx = 0$, the slope is horizontal — these are constant solutions
• Asymptotic behavior: Solutions approach or diverge from equilibria depending on the sign of P(x)
• Stability: If P(x) > 0, the equilibrium is stable (attracting); if P(x) < 0, it's unstable (repelling)

The Integrating Factor Method

The integrating factor method is the systematic technique for solving any first-order linear ODE. It's one of the most elegant methods in differential equations.

The General Formula

Structure of the General Solution

The general solution to a first-order linear ODE has a beautiful additive structure:

Physical Interpretation

• Homogeneous solution (y_h): The system's natural behavior without external input — typically exponential decay or growth
• Particular solution (y_p): The forced response to external input — the steady-state or driven behavior
• Transient vs. Steady-State: Often y_h decays to zero (transient), leaving y_p as the long-term (steady-state) behavior

Application: RC Circuits

One of the most important applications of first-order linear ODEs is in electrical circuits. Consider a resistor-capacitor (RC) circuit:

The Governing Equation

By Kirchhoff's voltage law, the voltage across the resistor plus the voltage across the capacitor equals the source voltage:

Since $i = C \frac{dV_C}{dt}$ (the current through a capacitor), we get:

Dividing by RC gives the standard form:

Here, $P = 1/(RC)$ and $Q = V_s/(RC)$. The time constant is $\tau = RC$.

The Solutions

Application: Battery Charging

Have you noticed your phone charges quickly to 80% but then slows down dramatically? This is the same ODE as capacitor charging — batteries use constant-voltage charging which follows first-order linear kinetics.

Application: Mixing Problems

Another classic application is the tank mixing problem: a well-mixed tank has fluid flowing in and out, and we track the amount of a dissolved substance (like salt).

Setting Up the Model

Let y(t) = amount of salt in the tank at time t. Then:

• Rate in: (flow rate) × (inflow concentration) = r × c_in
• Rate out: (flow rate) × (tank concentration) = r × (y/V)

This gives the first-order linear ODE:

Application: Coffee Cooling

Newton's Law of Cooling states that the rate of temperature change is proportional to the temperature difference from the environment. This leads to a first-order linear ODE with a surprising application: when should you add cream to keep your coffee warmest?

Application: Drug Dosing

Why do medicine labels say "take every 8 hours"? Drug concentration in the body follows first-order elimination kinetics — the same exponential decay we've been studying. The dosing interval is carefully chosen to keep drug levels in the therapeutic window.

Application: Disease Spread

During the early phase of an epidemic, when nearly everyone is susceptible, the spread of infection follows a first-order linear ODE. This explains why diseases grow exponentially at first and why R₀ (the basic reproduction number) is so important.

More Real-World Applications

Newton's Law of Cooling

The rate of temperature change is proportional to the temperature difference from the environment:

Rewriting: $dT/dt + kT = kT_{env}$. The solution approaches $T_{env}$ exponentially.

Population with Harvesting

where r is growth rate and H is constant harvesting. This is $P' - rP = -H$, a linear ODE with equilibrium at $P = H/r$.

Drug Pharmacokinetics

where C is drug concentration, D(t) is dosing rate, V is volume of distribution, and k is elimination rate constant.

Machine Learning Connections

Remarkably, first-order linear ODEs appear at the heart of gradient descent optimization, the backbone of modern machine learning.

Gradient Flow

Consider minimizing a loss function L(θ). Gradient descent updates:

In the limit of small step sizes, this becomes the gradient flow ODE:

Quadratic Objectives

For a quadratic loss $L(\theta) = \frac{1}{2}(\theta - \theta^*)^2$:

This is exactly a first-order linear ODE with $P = 1$ and $Q = \theta^*$! The solution is:

Python Implementation

Numerical ODE Solver

1import numpy as np
2from scipy.integrate import odeint
3import matplotlib.pyplot as plt
4
5def linear_ode_solver(P, Q, y0, t_span, num_points=200):
6    """
7    Solve the first-order linear ODE: dy/dt + P(t)*y = Q(t)
8
9    Parameters:
10    - P: function P(t)
11    - Q: function Q(t)
12    - y0: initial condition y(t0)
13    - t_span: tuple (t0, t_final)
14    - num_points: number of time points
15
16    Returns:
17    - t: array of time values
18    - y: array of solution values
19    """
20    # Define the ODE in standard form: dy/dt = f(y, t)
21    def dydt(y, t):
22        return Q(t) - P(t) * y
23
24    # Create time array
25    t = np.linspace(t_span[0], t_span[1], num_points)
26
27    # Solve using scipy's odeint
28    y = odeint(dydt, y0, t).flatten()
29
30    return t, y
31
32# Example 1: Constant coefficients y' + 2y = 6
33P1 = lambda t: 2
34Q1 = lambda t: 6
35t1, y1 = linear_ode_solver(P1, Q1, y0=0, t_span=(0, 3))
36
37# Example 2: Variable coefficient y' + (2/t)y = t^3
38P2 = lambda t: 2/t if t > 0 else 0
39Q2 = lambda t: t**3
40t2, y2 = linear_ode_solver(P2, Q2, y0=0.1, t_span=(0.1, 3))
41
42# Plot both solutions
43fig, axes = plt.subplots(1, 2, figsize=(12, 4))
44
45axes[0].plot(t1, y1, 'b-', linewidth=2, label='Numerical')
46axes[0].axhline(y=3, color='g', linestyle='--', label='Equilibrium y=3')
47axes[0].set_xlabel('t')
48axes[0].set_ylabel('y')
49axes[0].set_title("y' + 2y = 6")
50axes[0].legend()
51axes[0].grid(True, alpha=0.3)
52
53axes[1].plot(t2, y2, 'b-', linewidth=2)
54axes[1].set_xlabel('t')
55axes[1].set_ylabel('y')
56axes[1].set_title("y' + (2/t)y = t³")
57axes[1].grid(True, alpha=0.3)
58
59plt.tight_layout()
60plt.show()

Gradient Descent as Gradient Flow

1import numpy as np
2
3def gradient_descent_as_ode(loss_func, grad_func, theta_init,
4                            lr=0.1, num_steps=100):
5    """
6    Gradient descent viewed as solving: dθ/dt = -∇L(θ)
7
8    The discrete update θ_{n+1} = θ_n - η∇L(θ_n) approximates
9    the continuous gradient flow ODE.
10    """
11    theta = theta_init.copy()
12    history = [theta.copy()]
13    loss_history = [loss_func(theta)]
14
15    for _ in range(num_steps):
16        grad = grad_func(theta)
17        theta = theta - lr * grad  # Euler's method!
18        history.append(theta.copy())
19        loss_history.append(loss_func(theta))
20
21    return np.array(history), np.array(loss_history)
22
23# Example: Quadratic loss L(θ) = 0.5 * (θ - θ*)^2
24# This gives gradient flow: dθ/dt = -(θ - θ*)
25# Solution: θ(t) = θ* + (θ_0 - θ*) * e^{-t}
26
27theta_star = 3.0
28loss = lambda theta: 0.5 * (theta - theta_star)**2
29grad = lambda theta: theta - theta_star
30
31# Run gradient descent from θ_0 = 10
32history, losses = gradient_descent_as_ode(
33    loss, grad, np.array([10.0]), lr=0.1, num_steps=50
34)
35
36# Compare with analytical solution of the ODE
37t_continuous = np.linspace(0, 5, 100)
38theta_analytical = theta_star + (10 - theta_star) * np.exp(-t_continuous)
39
40print("Gradient descent is discretized gradient flow!")
41print(f"Final θ: {history[-1][0]:.4f}, Target: {theta_star}")

Common Pitfalls

Test Your Understanding

Summary

First-order linear differential equations are foundational tools for modeling systems with exponential behavior. The integrating factor method provides a complete, systematic solution procedure.

Key Formulas

Key Takeaways

1. First-order linear ODEs have the form y' + P(x)y = Q(x) with P and Q as known functions
2. The integrating factor method always works: multiply by $\mu = e^{\int P dx}$ to make the left side an exact derivative
3. Solutions have the structure y = y_h + y_p (homogeneous + particular)
4. The direction field provides geometric insight into solution behavior
5. Applications include RC circuits, mixing problems, population dynamics, cooling, and more
6. Gradient descent is discretized gradient flow — the connection to ODEs explains exponential convergence for quadratic objectives

Coming Next: In the next section, we'll explore Exact Equations — another class of first-order ODEs that can be solved by recognizing them as total differentials. This generalizes the integrating factor idea in a beautiful way.