Impulse Functions and the Delta Function

Learning Objectives

By the end of this section, you will be able to:

1. Construct the Dirac delta $\delta(t-a)$ as a careful limit of unit-area spikes, and explain why the limit must be interpreted as a distribution rather than a classical function.
2. Use the sifting property $\int_{-\infty}^{\infty} f(t)\,\delta(t-a)\,dt = f(a)$ to evaluate integrals and recognize when a delta is doing the work.
3. Compute the Laplace transform $\mathcal{L}\{\delta(t-a)\} = e^{-as}$ and contrast it with the step-function transform.
4. Solve initial-value problems of the form $y'' + by' + cy = A\,\delta(t - t_0)$ and interpret the impulse as an instantaneous change in momentum.
5. Connect the impulse response to convolution, linear systems, and the role of identity elements in signal processing and neural networks.

The Big Picture: Modeling an Instantaneous Kick

"A hammer strikes a tuning fork. The contact lasts almost no time at all, but it deposits a definite amount of momentum. How do we write down a force that is zero everywhere — except for one instant where it is infinite, yet has a finite total effect?"

The step function let us model a switch that turns on. But many of the most interesting events in physics and engineering are even more violent than a switch: they happen essentially at a single instant, and yet they have a definite measurable consequence.

• A bat hits a baseball — contact time is <1 ms, ball gains 40 m/s of velocity.
• Lightning strikes — a current pulse so brief and intense it cooks the air into plasma.
• A neuron fires — a millisecond-wide voltage spike triggers the next neuron in a chain.
• A gradient update in deep learning — a single backward pass injects a kick into every weight in the network.

None of these is a finite-amplitude continuous function. They all look like arbitrarily tall, arbitrarily narrow spikes that nonetheless carry a definite total "area" — total momentum, total charge, total update. To describe them mathematically we need a new object: the Dirac delta function $\delta(t-a)$.

Historical Context: Paul Dirac's Function That Was Not a Function

The delta function is named after Paul Adrien Maurice Dirac (1902–1984), the British theoretical physicist who introduced it in 1930 in his book The Principles of Quantum Mechanics. Dirac needed an object to represent a particle perfectly localized at one point — its wave function is zero everywhere except at that point, but it must still integrate to a finite probability.

"The most general function of x which is zero except at one value of x, where it is infinite, will be denoted by $\delta(x)$." — P. A. M. Dirac, 1930

Mathematicians of the day were horrified. There simply is no function in the classical sense with these properties. For nearly twenty years the delta function was treated as a useful fiction by physicists and engineers, despite reservations from pure mathematicians.

From Heresy to Rigor: Schwartz's Distributions

In 1944 the French mathematician Laurent Schwartz built the theory of distributions (generalized functions) — a rigorous framework in which the delta is a perfectly valid object. The key idea: stop thinking of $\delta$ as a thing with values at each point, and start thinking of it as something that acts on test functions:

Schwartz earned the Fields Medal in 1950 for this work. Today the delta function is a routine member of every applied mathematician's toolkit and is rigorously justified.

Building the Delta from a Limit

The cleanest way to understand $\delta(t)$ is to construct it. Start with a family of perfectly ordinary functions, each one a slightly narrower and taller spike, all with the same area underneath. Then watch what happens.

The Unit-Area Rectangle

Define the rectangular pulse:

The width is $\varepsilon$ and the height is $1/\varepsilon$, so the area is $\varepsilon \cdot (1/\varepsilon) = 1$ — no matter how tiny we make $\varepsilon$. As $\varepsilon \to 0^+$:

• The support shrinks to a single point at $t = 0$.
• The peak height blows up to $+\infty$.
• The total area stays pinned at $1$.

The Smooth Gaussian Approximation

We could just as well use a smooth bell-shaped curve:

Same story: area equals 1 for every $\varepsilon > 0$, and as $\varepsilon \to 0$ the bell collapses into the same spike. The shape of the approximation does not matter — only the limit does.

Interactive: Watch the Nascent Delta Collapse

Drag the $\varepsilon$ slider toward zero. Notice how the peak shoots up while the shaded area below stays at exactly 1. Swap between Rectangle / Gaussian / Lorentzian to confirm the limit does not depend on shape.

Defining the Delta Function

Formally, the Dirac delta is not a function in the classical sense. It is a distribution — an object defined entirely by how it acts inside an integral. The defining properties are:

For the shifted version $\delta(t-a)$, the spike sits at $t = a$ instead of the origin:

Delta as the Derivative of the Step Function

Recall the Heaviside step function from the previous section:

Classically, the derivative of $u(t)$ is zero everywhere except at $t = 0$, where it is undefined — there is an instantaneous jump of size 1. But what if we insist on a derivative anyway?

Approximate the step with a smooth ramp that climbs from 0 to 1 over a width of $\varepsilon$. Its derivative is a rectangle of width $\varepsilon$ and height $1/\varepsilon$ — exactly our nascent delta. In the limit $\varepsilon \to 0$:

Equivalently:

The step is the running integral of the delta. This pair (impulse, step) mirrors the pair (velocity, position) for an object that gets kicked: the position jumps in finite time only if the velocity contains a delta.

The Sifting Property

The single most important fact about the delta function is the sifting property. It is what makes the delta useful: sliding a delta across an integral "sifts out" a single value of the function it meets.

Why It Works (Informally)

Replace $\delta(t-a)$ with the nascent rectangle $R_\varepsilon(t-a)$. Then:

That right-hand side is the average value of f over a width-ε window around a. As $\varepsilon \to 0$ and $f$ is continuous at $a$, the average converges to $f(a)$. That is the sifting property.

What You Can Do with Sifting

Interactive: Sifting in Action

Pick a test function and drag the impulse location $a$. Watch the yellow dot ride along $f(t)$ — its height is always exactly $f(a)$, the result of the sifting integral.

Laplace Transform of the Delta Function

Now we can compute the Laplace transform of $\delta(t-a)$ directly from the definition — and the sifting property does all the work.

Compare with the Step Function

The factor of $s$ between them is no coincidence: differentiating in the time domain corresponds to multiplying by $s$ in the Laplace domain. So:

Two derivations, same answer. The delta really is the derivative of the step — even in the Laplace domain.

Solving Initial Value Problems with an Impulsive Forcing

Once we can transform the delta, solving differential equations with impulsive forcing becomes a routine algebraic exercise. The method is exactly the one you already know — only the right-hand side changes.

Template Problem

Solve $y'' + 4y = \delta(t - 2)$ with $y(0) = 0,\; y'(0) = 0$.

Physical interpretation. A unit-mass on a spring with natural frequency $\omega_0 = 2$, initially at rest, struck by an impulsive force at $t = 2$.

The solution is exactly zero for $t < 2$, then jumps into a pure sinusoidal oscillation the moment the impulse hits. Position is continuous at the impulse — it is velocity that jumps.

Interactive: Impulse Response of a Second-Order System

Play with the natural frequency $\omega_0$, damping $\gamma$, impulse time $t_0$, and amplitude $A$. The system stays silent until the impulse arrives, then rings down according to its own intrinsic dynamics. Try the underdamped, critically damped, and overdamped regimes.

What an Impulse Really Means: Momentum, Not Force

Looking at the solution above raises a puzzle. The forcing on the right-hand side is infinite at one instant. Why doesn't the position jump to infinity too?

Because the right object to track is not the force itself, but the force's integral: the impulse = total momentum delivered.

Newton's second law for a unit mass is $\dfrac{dv}{dt} = F(t)$. Integrate across the instant $t = t_0$:

The left side is $v(t_0^+) - v(t_0^-)$; the right side is $A$ (by sifting). So:

The velocity jumps by $A$ at the impulse. The position $x(t)$ is the integral of velocity and therefore stays continuous — no instantaneous jump in position, because the velocity is only delta-singular for an infinitesimal moment.

Worked Example: Hammer Strike on a Damped Spring

Machine Learning Connections

Impulses and deltas show up in ML in several places — usually disguised, but always doing the same job: concentrating information at a single point.

One-Hot Encoding = Discrete Delta

The one-hot vector $(0, 0, \ldots, 1, \ldots, 0)$ is just a Kronecker delta $\delta_{i,k}$ — a discrete impulse at class $k$. Cross-entropy against a one-hot label is exactly the sifting property in disguise:

Convolution Has δ as Its Identity

For any signal $f$ and impulse $\delta$:

In a CNN, a kernel that is zero everywhere except a single 1 at the center is the identity map: it copies the input feature through unchanged. This is what makes residual connections ($y = f(x) + x$) so well-suited to deep networks — the second term $x$ is literally the output of a delta-kernel convolution.

Derivative of ReLU Is a Step; Second Derivative Is a Delta

ReLU is $\text{ReLU}(x) = x \cdot u(x)$. Differentiating:

The second derivative is a delta at the kink. In practice deep-learning frameworks define $u'(0)$ as a subgradient (anywhere in $[0,1]$), which is exactly how distribution theory tells you to handle a delta inside a numerical gradient pipeline.

Point Source of a Green's Function

In physics-informed neural networks and PDE solvers, the response to a delta forcing is called the Green's function:

Once you have $G$, the response to any forcing is just a convolution with $G$. The whole field of linear-system analysis is built on this one idea.

Python Implementation

Building the Delta from a Limit, Numerically

First, let's watch the nascent delta collapse and verify that the sifting property emerges from a finite-width rectangle:

with eps=0.3:  peak = 1/(0.3·√π) ≈ 1.881

eps = 0.500  rect peak =     2.00  rect area =  1.0000  gauss area =  1.0000
eps = 0.100  rect peak =    10.00  rect area =  1.0000  gauss area =  1.0000
eps = 0.020  rect peak =    50.00  rect area =  1.0000  gauss area =  1.0000

eps = 0.500  sifted value =  -0.9583  (target = -1.0, error = +0.0417)
eps = 0.100  sifted value =  -0.9992  (target = -1.0, error = +0.0008)
eps = 0.020  sifted value =  -1.0000  (target = -1.0, error = +0.00003)

1import numpy as np
2import matplotlib.pyplot as plt
3
4# A "nascent delta": a rectangle of width epsilon and height 1/epsilon.
5# By construction its area is exactly 1, no matter how small epsilon gets.
6def rect_delta(t, eps):
7    return np.where(np.abs(t) <= eps / 2, 1.0 / eps, 0.0)
8
9# A Gaussian nascent delta: smooth, same unit-area property.
10def gauss_delta(t, eps):
11    return (1.0 / (eps * np.sqrt(np.pi))) * np.exp(-(t ** 2) / (eps ** 2))
12
13# Sweep through three values of epsilon and confirm the area stays 1.
14t = np.linspace(-2, 2, 4001)
15dt = t[1] - t[0]
16
17for eps in [0.5, 0.1, 0.02]:
18    rect_area = np.sum(rect_delta(t, eps)) * dt
19    gauss_area = np.sum(gauss_delta(t, eps)) * dt
20    rect_peak = 1.0 / eps
21    print(f"eps = {eps:5.3f}  "
22          f"rect peak = {rect_peak:8.2f}  rect area = {rect_area:7.4f}  "
23          f"gauss area = {gauss_area:7.4f}")
24
25# Now verify the SIFTING property numerically:
26#     integral of f(t) * delta(t - a) dt  ==  f(a)
27# We use f(t) = t**2 - 3*t + 1 and a = 2, so the answer should be f(2) = -1.
28def f(t):
29    return t ** 2 - 3 * t + 1
30
31a = 2.0
32target = f(a)                       # expected value: 4 - 6 + 1 = -1
33
34for eps in [0.5, 0.1, 0.02]:
35    approx = np.sum(f(t) * rect_delta(t - a, eps)) * dt
36    error = approx - target
37    print(f"eps = {eps:5.3f}  sifted value = {approx:8.4f}  "
38          f"(target = {target}, error = {error:+8.4f})")

Symbolic Laplace of δ and a Full IVP

Now we let SymPy do the symbolic algebra — and check that the answer matches our hand calculation:

L{u(t-a)} = exp(-a*s)/s   (step)
L{δ(t-a)} = exp(-a*s)      (impulse)

y(t) = sin(2*t - 4)*Heaviside(t - 2)/2
Meaning: 0 before t=2, then ½ sin(2(t−2)) afterward.

y(t) = (1/2) sin(2(t − 2)) · u(t − 2)

1from sympy import (symbols, DiracDelta, Heaviside, exp, sin, cos,
2                   laplace_transform, inverse_laplace_transform, Function,
3                   dsolve, Eq, Derivative)
4
5t, s = symbols('t s', positive=True, real=True)
6a = symbols('a', positive=True)
7
8# 1. Laplace transform of the unit impulse at t = a
9F = laplace_transform(DiracDelta(t - a), t, s, noconds=True)
10print("L{delta(t - a)} =", F)        # expected: exp(-a*s)
11
12# 2. Laplace transform at a = 0 (the canonical delta at the origin)
13F0 = laplace_transform(DiracDelta(t), t, s, noconds=True)
14print("L{delta(t)}     =", F0)       # expected: 1
15
16# 3. Solve y'' + 4 y = delta(t - 2),  y(0) = 0,  y'(0) = 0
17y = Function('y')
18ode = Eq(Derivative(y(t), t, t) + 4 * y(t), DiracDelta(t - 2))
19sol = dsolve(ode, y(t), ics={y(0): 0, Derivative(y(t), t).subs(t, 0): 0})
20print("\nIVP solution:")
21print("y(t) =", sol.rhs)
22
23# 4. Inverse-transform check, the hand-method way:
24#    Y(s)(s^2 + 4) = exp(-2s)  =>  Y(s) = exp(-2s) / (s^2 + 4)
25#    L^{-1}{1/(s^2+4)} = (1/2) sin(2t)
26#    Then the e^{-2s} factor shifts the inverse by 2 (second shifting theorem).
27Y = exp(-2 * s) / (s ** 2 + 4)
28y_hand = inverse_laplace_transform(Y, s, t)
29print("\nInverse-Laplace y(t) =", y_hand)
30
31# 5. Evaluate the solution numerically at a few points
32import numpy as np
33def y_num(tv):
34    return np.where(tv < 2, 0.0, 0.5 * np.sin(2 * (tv - 2)))
35
36for tv in [1.0, 2.0, 2.5, 3.0, 2 + np.pi / 4]:
37    print(f"y({tv:5.3f}) = {y_num(tv):+.6f}")

The Discrete Delta in PyTorch

Finally, the digital cousin of the delta function — the Kronecker impulse — and its role as the identity element of convolution and the probe for any linear filter:

delta = [0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0]

filter h = [1.0, 2.0, 3.0]

h         = [1, 2, 3]
h.flip(0) = [3, 2, 1]

h * delta = [0.0, 1.0, 2.0, 3.0, 0.0, 0.0, 0.0, 0.0]
             ^ entries 1,2,3 are exactly h, starting one position before the spike.

x                 = [4.0, -1.0,  3.0,  0.5,  2.0,  7.0,  1.0,  0.0]
x * delta_shifted = [-1.0,  3.0,  0.5,  2.0,  7.0,  1.0,  0.0,  0.0]
                     ^ same numbers, shifted left by 1 (kernel offset).

pad[2:]   = x[n]
pad[1:-1] = x[n-1]
pad[:-2]  = x[n-2]
y[n] = 0.5·x[n] + 0.3·x[n-1] + 0.2·x[n-2]

impulse response = [0.5, 0.3, 0.2, 0.0, 0.0, 0.0]

1import torch
2import torch.nn.functional as F
3
4# A discrete impulse: zero everywhere except a single 1 at index 2.
5# This is the digital cousin of delta(t - 2).
6delta = torch.zeros(8)
7delta[2] = 1.0
8print("delta             =", delta.tolist())
9
10# A simple 3-tap filter — could be a smoothing kernel, an FIR filter, etc.
11h = torch.tensor([1.0, 2.0, 3.0])
12print("filter h          =", h.tolist())
13
14# Convolution treats delta as the IDENTITY element:
15#   (h * delta)[n] = h[n - k0]
16# We use F.conv1d, which expects shape (batch, channels, length).
17delta_b = delta.view(1, 1, -1)
18h_kernel = h.flip(0).view(1, 1, -1)   # PyTorch conv1d does cross-correlation
19out = F.conv1d(delta_b, h_kernel, padding=2).view(-1)
20print("h * delta         =", out.tolist())
21
22# Same idea: convolve an arbitrary signal x with a delta and get x back,
23# shifted by the impulse position.
24x = torch.tensor([4.0, -1.0, 3.0, 0.5, 2.0, 7.0, 1.0, 0.0])
25shift_delta = torch.zeros(5)
26shift_delta[3] = 1.0
27x_b = x.view(1, 1, -1)
28d_kernel = shift_delta.flip(0).view(1, 1, -1)
29shifted = F.conv1d(x_b, d_kernel, padding=2).view(-1)
30print("x * delta_shifted =", shifted.tolist())
31
32# Impulse response of a digital filter: feed delta in, output IS the filter.
33# This is the discrete echo of "the inverse Laplace transform of 1/D(s)".
34def my_filter(signal):
35    # y[n] = 0.5 * x[n] + 0.3 * x[n-1] + 0.2 * x[n-2]
36    pad = torch.cat([torch.zeros(2), signal])
37    return 0.5 * pad[2:] + 0.3 * pad[1:-1] + 0.2 * pad[:-2]
38
39impulse_in = torch.zeros(6)
40impulse_in[0] = 1.0
41impulse_response = my_filter(impulse_in)
42print("impulse response  =", impulse_response.tolist())

Common Mistakes

Test Your Understanding

Summary

The Dirac delta is the ideal limit of an arbitrarily narrow, arbitrarily tall, unit-area spike. It is not a classical function but a distribution — defined by how it acts inside integrals. Combined with the Laplace transform, it gives us the cleanest possible way to model instantaneous events: collisions, switch closures, lightning strikes, neural spikes, and gradient updates.

Key Formulas

Key Takeaways

1. Construct, don't evaluate. Think of δ as the limit of unit-area spikes, not as a function with pointwise values.
2. Sifting is the workhorse. Almost every manipulation of δ reduces to $\int f(t)\, \delta(t-a)\, dt = f(a)$.
3. Differentiation in time = multiplication by s. Because δ is the derivative of the step, its Laplace transform is $s \cdot \tfrac{1}{s} = 1$.
4. An impulse delivers momentum, not displacement. Velocity jumps; position stays continuous.
5. Impulse response = system identity. Feed δ in, and the output completely characterizes any linear time-invariant system. Every other response is a convolution with this one.
6. ML is full of deltas. One-hot labels, residual connections, Green's functions for PDE solvers, ReLU's second derivative — all are deltas in disguise.

Coming Next: In Convolution we will build on the "δ is the identity" idea to show how any input signal can be written as a continuous sum of shifted impulses, each weighted by the input's value at that instant. Convolving with the impulse response then gives the system's output for arbitrary forcing.