One-Sided Limits and Jump Discontinuities

Learning Objectives

By the end of this section you will be able to:

1. Describe what a one-sided limit is, in both the approach-from-the-left form $\lim_{x \to a^-} f(x)$ and the approach-from-the-right form $\lim_{x \to a^+} f(x)$.
2. State the agreement theorem: $\lim_{x \to a} f(x)$ exists if and only if the two one-sided limits exist AND agree.
3. Identify a jump discontinuity by computing both one-sided limits.
4. Compute one-sided limits numerically by probing closer and closer values.
5. Connect this idea to machine learning, where ReLU creates a jump in the derivative at the origin.

Why One-Sided? A Physical Story

In the previous section we watched a function sneak up on a value from both sides at once. Real phenomena are not always so well-behaved. Flip a light switch at $t = 0$. An instant before, the current is $0$. An instant after, the current is $I_0$. If you stood at the exact moment the switch closed and asked “what is the current”, you would get a different answer depending on which direction in time you came from.

The motivating problem: how do we talk about a limit when the function disagrees with itself at a point?

The fix is almost embarrassingly simple: split the definition. We do not talk about ONE limit — we talk about TWO, one for each direction of approach. That single move unlocks the language we need to describe switches, step functions, digital signals, price ladders, the ReLU non-linearity, the floor function, and every real-world process that jumps.

The Two-Doors Intuition

Picture the point $x = a$ as a room with two doors. The LEFT door is the only entrance from values $x < a$. The RIGHT door is the only entrance from values $x > a$. If you walk toward the room through the left door you will see one value of $f$ getting nearer and nearer. Walk in through the right door and you may see something completely different.

Formal Definition

Let $f$ be defined on an open interval containing $a$ (except possibly at $a$ itself).

Interactive: Approaching From Each Side

Pick a piecewise function below. Drag the blue slider and watch the blue dot crawl along the LEFT branch toward the vertical purple line. Drag the orange slider for the RIGHT branch. The two one-sided limits are the heights the dots are heading to.

The Agreement Theorem

Here is the statement that ties one-sided limits back to the ordinary two-sided limit:

This gives us a clean failure taxonomy for limits:

What Is a Jump Discontinuity?

A jump discontinuity at $x = a$ is the specific failure mode where:

1. The left-hand limit $L^-$ exists (is a finite real number);
2. The right-hand limit $L^+$ exists (also finite);
3. But $L^- \neq L^+$.

The number $L^+ - L^-$ is called the jump, or sometimes the saltus (Latin for “leap”). It measures, in one signed number, exactly how much the function lies to itself at the discontinuity.

Interactive: Discontinuity Gallery

Four famous jumpy functions. Click through them and drag the point of interest to see how the jump size stays constant (for floor, ceiling, square wave) or is fixed by the definition (sign).

Worked Example (Collapsible)

Try the problem yourself before opening the walkthrough. Given

$f(x) = \begin{cases} x^2 + 1 & x < 2 \\ x - 1 & x \geq 2 \end{cases}$

Compute $\lim_{x \to 2^-} f(x)$, $\lim_{x \to 2^+} f(x)$, $f(2)$, and decide whether $\lim_{x \to 2} f(x)$ exists.

Python: Watch the Limit Form

The intuition we just walked through is something you can see the computer do. The script below is the same worked example, but automated: it evaluates $f$ at a few points crowding toward $2$ from each side and prints the results. Click any line in the code on the right — the left panel shows the full execution trace for that line, including every value that passes through.

1# numerical_limit_probe.py  —  watch a one-sided limit form itself
2
3def f(x: float) -> float:
4    """Piecewise function with a jump at x = 2."""
5    if x < 2:
6        return x**2 + 1        # left branch
7    else:
8        return x - 1           # right branch
9
10# Probe from the LEFT: x -> 2^-
11left_xs = [1.5, 1.9, 1.99, 1.999, 1.9999]
12print("Approaching 2 from the LEFT (x < 2)")
13for x in left_xs:
14    print(f"  f({x:<6}) = {f(x):.6f}")
15
16# Probe from the RIGHT: x -> 2^+
17right_xs = [2.5, 2.1, 2.01, 2.001, 2.0001]
18print("\nApproaching 2 from the RIGHT (x >= 2)")
19for x in right_xs:
20    print(f"  f({x:<6}) = {f(x):.6f}")
21
22print("\nConclusion:")
23print("  L^- = 5.0   (left limit)")
24print("  L^+ = 1.0   (right limit)")
25print("  L^- != L^+  ->  the two-sided limit does NOT exist")

PyTorch: ReLU's Jumpy Derivative

One-sided limits are not just textbook curiosities — they sit at the mathematical heart of modern deep learning. The ReLU activation $\operatorname{ReLU}(x) = \max(0, x)$ is continuous (its two one-sided limits agree at $x = 0$, both equal to 0), but its derivative has a JUMP at $x = 0$:

$\lim_{x \to 0^-} \frac{d}{dx}\operatorname{ReLU}(x) = 0 \quad \text{but} \quad \lim_{x \to 0^+} \frac{d}{dx}\operatorname{ReLU}(x) = 1.$

Every backward pass through a ReLU neuron hits this jump. PyTorch has to pick some value at exactly $x = 0$ — let's probe what choice it makes.

1# relu_one_sided.py  —  a real ML function with a jumpy derivative
2import torch
3import torch.nn.functional as F
4
5def relu_grad_at(x_value: float) -> float:
6    """Compute d/dx of ReLU(x) at a single point using autograd."""
7    x = torch.tensor(x_value, requires_grad=True)
8    y = F.relu(x)
9    y.backward()
10    return x.grad.item()
11
12# Probe the derivative near x = 0 from both sides
13left_probes  = [-0.1, -1e-3, -1e-6, -1e-9]
14right_probes = [+1e-9, +1e-6, +1e-3, +0.1]
15
16print("ReLU derivative from the LEFT (x -> 0^-):")
17for x in left_probes:
18    print(f"  dReLU/dx at x={x:+.1e}  =  {relu_grad_at(x):.1f}")
19
20print("\nReLU derivative from the RIGHT (x -> 0^+):")
21for x in right_probes:
22    print(f"  dReLU/dx at x={x:+.1e}  =  {relu_grad_at(x):.1f}")
23
24# And at the non-differentiable point itself:
25print(f"\nPyTorch's convention at x = 0: {relu_grad_at(0.0):.1f}")
26# -> 0.0 (PyTorch picks the left-side subgradient)

Common Pitfalls

• Assuming $f(a)$ equals one of the one-sided limits. Not required. The sign function has $L^- = -1$, $L^+ = 1$, but $\operatorname{sgn}(0) = 0$ — a value that neither side approaches.
• Forgetting that “exists” has a precise meaning. The two-sided limit EXISTS only when both one-sided limits exist AND equal each other. If either one-sided limit blows up or oscillates, the two-sided limit does not exist.
• Reading $a^-$ as “a minus something”. The little dash is a direction marker, not subtraction. It says “approach $a$ through values smaller than $a$”.
• Picking too few probe points. Three values is usually not enough to be confident the limit exists. Use a decreasing sequence like $0.1, 0.01, 0.001, 0.0001$ so each probe is ten times closer than the one before.

Summary

• The left-hand limit $\lim_{x \to a^-} f(x)$ watches $f$ as $x$ approaches $a$ through values $x < a$.
• The right-hand limit $\lim_{x \to a^+} f(x)$ is the mirror image: $x > a$.
• Agreement theorem: the two-sided limit exists iff both one-sided limits exist AND are equal.
• A jump discontinuity is the specific failure where both one-sided limits exist but differ. The jump is $L^+ - L^-$.
• Every real-world switching, rounding, or step phenomenon — from digital logic to ReLU activations — is precisely a jump discontinuity, and one-sided limits are the language we use to talk about it.

Next section we flip direction: what if $x$ itself is running away — $x \to \infty$? Limits at infinity, and the horizontal asymptotes they reveal.