Derivative of e^x: The Special Exponential

Learning Objectives

By the end of this section, you will be able to:

1. State and prove the rule $\dfrac{d}{dx}\,e^x = e^x$
2. Explain the magical limit $\displaystyle\lim_{h \to 0} \dfrac{e^h - 1}{h} = 1$ and why it defines the number $e$
3. Compute the slope of the tangent line to $y = e^x$ at any point without using a calculator beyond evaluating $e^x$
4. Recognize why $a = e$ is the unique base for which $a^x$ is its own derivative
5. Apply the chain-rule version $\dfrac{d}{dx}\, e^{kx} = k\, e^{kx}$ to growth and decay problems
6. Verify the derivative numerically in plain Python and via PyTorch autograd

The Big Picture: A Function Equal to Its Own Slope

“Of all functions known to mathematics, only one — up to a constant multiplier — is exactly equal to its own derivative. That function is $e^x$.”

Stop and let that land. Take any other function you can think of — a polynomial, a sine, a logarithm — and its derivative is some different function. The derivative of $x^2$ is $2x$, a different shape entirely. The derivative of $\sin x$ is $\cos x$, a shifted wave. But the derivative of $e^x$ is itself:

This single fact is the reason $e$ shows up everywhere: compound interest, radioactive decay, population growth, RC-circuit charging, Bayesian priors, neural-network softmax outputs, the Schrödinger equation, the normal distribution. Every time a quantity grows at a rate proportional to itself, the answer is dressed in $e$.

Intuition: Money and Bacteria

The bank account that compounds continuously

Imagine a savings account that pays interest continuously at rate $100\%$ per year. After time $t$ years your balance is $B(t) = e^t$ (starting from $B(0) = 1$).

The rate at which money is added to the account at instant $t$ is $B'(t)$ — interest dollars per year. Common sense says:

So $B'(t) = B(t)$. The bigger the balance, the faster it grows. The function and its derivative are the same. That is the differential equation $y' = y$, and its solution is $y = e^t$.

A colony of bacteria

Now replace dollars with bacteria. Each bacterium splits at a constant per-capita rate. If there are $N(t)$ bacteria, the population produces new bacteria at rate proportional to $N(t)$ itself — twice as many parents, twice as many babies per minute. Same equation, $N'(t) = N(t)$, same solution $N(t) = e^t$.

Numerical Discovery

Let's set the theory aside for a moment and measure the slope of $y = e^x$ by hand at three different x values. We will use the secant-line slope with a small step $h = 0.001$ as a stand-in for the tangent slope:

Look at the last column. The ratio is the same number at every $x$! And as $h$ shrinks toward zero that ratio approaches 1. The slope and the height aren't just proportional; they're equal.

That is the entire empirical content of this section. The rest is just turning the observation into a proof.

Derivation from First Principles

Apply the limit definition of the derivative to $f(x) = e^x$:

Use the most useful property of the exponential — $e^{a+b} = e^a \cdot e^b$ — to split $e^{x+h}$ as $e^x \cdot e^h$:

The factor $e^x$ does not depend on $h$, so it survives the limit as a constant. Pull it outside:

Now the entire question reduces to a single number: what is $\displaystyle\lim_{h \to 0} \dfrac{e^h - 1}{h}$?

The Magical Limit (e^h − 1)/h → 1

Drag the slider below and watch the orange dot fall onto the green dashed line at $y = 1$. The orange curve has a hole at $h = 0$ (we'd divide by zero) — but the limiting value is unambiguously 1.

Why is the limit 1 and not some other number?

The number $e$ can be defined as the unique positive real number for which this limit equals 1. From chapter 1 we also have the equivalent definitions:

1. $\displaystyle e = \lim_{n \to \infty} \left(1 + \tfrac{1}{n}\right)^n$ — compound interest as the compounding period vanishes.
2. $\displaystyle e = \sum_{k=0}^{\infty} \tfrac{1}{k!} = 1 + 1 + \tfrac{1}{2} + \tfrac{1}{6} + \tfrac{1}{24} + \cdots$ — Euler's power series.
3. $e^x = \displaystyle\sum_{k=0}^{\infty} \tfrac{x^k}{k!}$ — the Taylor series for the exponential.

Use definition (3) and plug in $e^h - 1$:

Divide by $h$:

Now let $h \to 0$. Every term except the leading $1$ vanishes. The limit is exactly 1.

Why e Is the Unique Base

Slide the base $a$ in the visualizer below. The solid blue curve is $a^x$ and the red dashed curve is its derivative. They are always proportional. The proportionality constant is exactly $\ln(a)$. Slide all the way to $a = e \approx 2.71828$ and watch the two curves snap together.

Where ln(a) comes from

Run the same derivation we just did but with a general base $a$:

The remaining limit defines a number that depends only on $a$. Call it $k(a)$:

Using $a^h = e^{h\ln a}$ and the special limit we just proved:

(We substituted $u = h \ln a$, which also tends to zero.) Therefore:

And the “the function is its own derivative” property happens precisely when $\ln(a) = 1$, i.e. when $a = e$. That is the definition of natural exponential.

Geometric Meaning: Slope = Height

Pick any point $(a, e^a)$ on the curve. The tangent line at that point has equation:

Two consequences worth absorbing:

1. The tangent at $x = a$ always has slope $e^a$, which is the very height of the point of tangency.
2. That tangent line crosses the x-axis at $x = a - 1$ — exactly one unit to the left of the point of tangency, no matter where on the curve you are. (Try it: substitute $y = 0$ in the tangent equation and solve.) This is a striking self-similarity property unique to $e^x$.

Worked Example: Compute the Tangent by Hand

Find the tangent line to $y = e^x$ at $x = 2$. We will compute the slope, write the tangent equation, and check the “one unit to the left” property above.

Tangent Explorer: See It All in One Picture

Drag the purple dot along the curve below and watch the green tangent line stay glued to the curve. The reported slope is always equal to the height — the central property of $e^x$. Then shrink the step $h$ to see the orange secant collapse onto the tangent.

Chain Rule Preview: e^(kx) and Half-Life

The most common exponential you'll meet in physics, biology, and finance isn't $e^x$ with growth rate 1 per unit time — it's $e^{kx}$ with some non-unit rate $k$. By the chain rule (proved in detail in section 4.7):

So $e^{kx}$ is $k$ times its own derivative. The constant $k$ is precisely the per-unit-time growth rate.

Real-World Applications

Plain Python Implementation

Let's convert everything we proved into code. The script does three things:

1. Defines a numerical derivative routine using the limit definition.
2. Compares the numerical derivative of $e^x$ against $e^x$ itself at six points.
3. Watches $(e^h - 1)/h$ march toward 1 as $h$ shrinks.

At x = 0, h = 1e-6: (e^(1e-6) − 1)/1e-6 ≈ 1.0000005000017

1import math
2
3# 1. Numerical derivative of e^x via the limit definition.
4#    d/dx e^x = lim h->0  (e^(x+h) - e^x) / h
5#             = e^x * lim h->0  (e^h - 1) / h
6#             = e^x * 1
7#             = e^x   <-- the function is its own derivative
8
9def numerical_derivative_exp(x, h=1e-6):
10    return (math.exp(x + h) - math.exp(x)) / h
11
12# 2. Verify by comparing to e^x itself across a few points.
13print(f"{'x':>6}  {'e^x':>16}  {'approx d/dx e^x':>20}  {'|error|':>12}")
14for x in [-1.0, 0.0, 0.5, 1.0, 2.0, 3.0]:
15    true_val = math.exp(x)
16    approx   = numerical_derivative_exp(x)
17    error    = abs(approx - true_val)
18    print(f"{x:>6.2f}  {true_val:>16.10f}  {approx:>20.10f}  {error:>12.2e}")
19
20# 3. Inspect the magical limit (e^h - 1) / h directly.
21#    Watch the column converge to 1.
22print()
23print(f"{'h':>14}  {'(e^h - 1)/h':>20}")
24for h in [1, 0.5, 0.1, 0.01, 0.001, 0.0001, 1e-5, 1e-6]:
25    value = (math.exp(h) - 1) / h
26    print(f"{h:>14.6f}  {value:>20.10f}")

PyTorch Verification

Plain Python gave us numerical confirmation. Let's now ask PyTorch's autograd engine — designed to handle the messiest neural-network gradients — to compute $d/dx\; e^x$ for us. We expect the gradient tensor to be bit-for-bit equal to $e^x$.

[0.3679, 1.0, 1.6487, 2.7183, 7.3891, 20.0855]

matches? : True

1import torch
2
3# Build a tensor x with several test points and ask PyTorch to track gradients.
4x = torch.tensor([-1.0, 0.0, 0.5, 1.0, 2.0, 3.0], requires_grad=True)
5
6# Compute y = e^x element-wise.
7y = torch.exp(x)
8
9# Sum so we have a scalar to backpropagate through.
10# For a sum, d(y_sum)/dx_i = d(e^{x_i})/dx_i = e^{x_i}.
11y_sum = y.sum()
12
13# Trigger autograd. After this call, x.grad[i] holds d(y_sum)/dx_i.
14y_sum.backward()
15
16# Compare PyTorch's autograd to the analytic answer e^x.
17print("x        :", x.detach().tolist())
18print("e^x      :", y.detach().tolist())
19print("x.grad   :", x.grad.tolist())
20print("matches? :", torch.allclose(x.grad, torch.exp(x.detach())))

The final line prints $\text{matches?} \,:\, \text{True}$. Two independent computations — a hand derivation following the limit definition, and a general-purpose automatic-differentiation engine — agree on every digit.

Common Mistakes

Summary

In the next section we generalize: what is the derivative of a general exponential $a^x$, and how does $\ln(a)$ enter the formula? Spoiler — we already derived it above. We'll just put it to work.