The Intuition of Limits: Getting Arbitrarily Close

Learning Objectives

After this section you will be able to:

1. Explain what problem limits solve — why plugging in a number is sometimes forbidden but an answer still exists.
2. Read the notation $\displaystyle \lim_{x \to a} f(x) = L$ in plain English.
3. Approximate a limit from a numerical table and from an interactive graph.
4. Detect when a limit fails to exist (jump, oscillation, blow-up).
5. Connect limits to the derivative by watching a secant slope converge.
6. Compute a simple limit in Python and in PyTorch.

The Problem That Birthed Calculus

How fast are you going right now? Not between two moments — right now, at a single instant.

Speed is distance divided by time. That is the definition from primary school: $\text{avg speed} = \dfrac{\Delta x}{\Delta t}$. But what happens when $\Delta t$ is zero? Distance travelled in zero time is zero. We are asked to compute $0/0$. Our arithmetic cannot do this.

And yet a speedometer reads something — a definite number — at every instant. Cars and planets move with an instantaneous speed even though the formula $\Delta x / \Delta t$ breaks down. Physics insists the number exists. Algebra insists the computation is illegal. Who is right?

The answer calculus invented is called a limit. It is a new kind of “value” — not the output of a formula at a point, but the target the output approaches as we slide toward that point. Every derivative, every integral, every differential equation in this book rests on this one idea. Let us build it carefully.

The Hole in the Graph

To make the idea concrete, throw away the motion story for a moment and just stare at a formula:

Plug in any number you like: $x = 2$ gives $3/1 = 3$, $x = 0$ gives $-1/-1 = 1$, $x = 1.1$ gives $0.21 / 0.1 = 2.1$. Every value works — except $x = 1$. There both the numerator and denominator vanish, and we are handed $0/0$ — a meaningless expression.

The graph of this function looks exactly like the straight line $y = x + 1$ — except for a tiny infinitesimal puncture at the single point $(1, 2)$. The line is whole everywhere else; only this one point is missing.

So here is the puzzle. The graph is a straight line with one missing point. Every neighbour of $x = 1$ gives an output you can almost predict — values just below 1 produce outputs just below 2, values just above 1 produce outputs just above 2. The function is pointing at the value 2 from both sides. It just never arrives.

The limit is our formal name for “the value the function is pointing at.” Let us see it live.

Zoom Into the Hole — Interactive

Drag the slider. The window around $x = 1$ narrows by a factor of ten every decade. The function is plotted in indigo; the orange/teal dots are probes moving with the zoom. The white circle at $(1, 2)$ marks the hole — it is genuinely empty, no matter how far you zoom in.

The Numerical Table Approach

Before the formal definition, Leibniz and the Bernoullis convinced themselves that limits made sense by computing. Given a function and a target input, they built a table of outputs at nearby inputs and watched for convergence. We do the same here:

Left column: outputs are climbing toward 2 from below. Right column: outputs are descending toward 2 from above. The function does not attain the value 2 (because the inputs never attain 1). But both columns leave no doubt about the target.

Calculus captures this observation with a single piece of notation:

Read aloud: “the limit, as $x$ approaches 1, of $(x^{2} - 1)/(x - 1)$, equals 2.” The arrow is the important symbol — it means approaching but never reaching. The equation is not claiming $f(1) = 2$ (that is false — $f(1)$ is undefined). It is claiming something subtler and stronger: that the outputs line up arbitrarily close to 2 whenever the inputs line up arbitrarily close to 1.

The Intuitive Definition of a Limit

Three pieces of this sentence deserve emphasis.

• “As close as we please”. No tolerance is too small. Name any positive number $\varepsilon > 0$ — a billionth, a trillionth, $10^{-100}$ — and we must be able to place $f(x)$ within $\varepsilon$ of $L$.
• “By choosing x close enough”. For every such tolerance, there is a distance $\delta > 0$ around $a$ that does the job. Small $\varepsilon$ typically demands small $\delta$.
• “But not equal to a”. The value of $f$ at $a$ is irrelevant. It might be undefined, or defined but wrong (think of a graph with a single filled dot floating above the line). The limit only cares about neighbours of $a$.

This is the language that will become the formal $\varepsilon$-$\delta$ definition in §2.5. For now, hold onto the metaphor: you (the challenger) pick a tolerance around $L$; the function (the responder) must guarantee a matching window around $a$. If it can always respond, the limit equals $L$.

Worked Example — Factoring the 0/0

Before we trust any software, let us compute the same limit by hand. The trick is to break the spell of $0/0$ by factoring.

When the Limit Does Not Exist

The limit of our factoring example worked because both one-sided “targets” agreed. What if they don't? Four things can go wrong, and it is worth recognising each one.

1. Jump. The two sides approach different numbers. For the sign function, $\lim_{x \to 1^{-}} \mathrm{sgn}(x-1) = -1$ while $\lim_{x \to 1^{+}} \mathrm{sgn}(x-1) = +1$. No single $L$ can satisfy the definition.
2. Blow-up. The values grow without bound near $a$. For $f(x) = 1/(x-1)^{2}$, both one-sided outputs head off toward $+\infty$. No finite $L$ exists, though we write $\lim f = +\infty$ as shorthand.
3. Wild oscillation. The function bounces arbitrarily fast near $a$. The canonical example is $f(x) = \sin\!\left(\tfrac{1}{x-1}\right)$; every neighbourhood of 1 contains values arbitrarily close to $+1$ and $-1$. There is no single target.
4. Undefined on one side. e.g., $f(x) = \sqrt{x}$ near $x = 0$ has no left-sided limit (no real square root for negative inputs).

Left vs. Right — Interactive

Four scenarios, one probe on each side. Use the buttons to switch and the slider to shrink the probe distance $h$. Watch when the two sides agree and when they do not.

Python: Computing the Limit by Brute Force

The table we stared at earlier was hand-computed. Let us do the same thing in Python, so the student can paste this into a REPL or notebook and watch the convergence live. Click any line of the code to see the exact values flowing through it.

1def f(x):
2    """f(x) = (x^2 - 1) / (x - 1). Undefined at x = 1 (0/0)."""
3    return (x**2 - 1) / (x - 1)
4
5# Approach x = 1 from both sides. Shrink |h| each step.
6print(f"{'x':>10}  {'f(x)':>12}  side")
7print("-" * 32)
8
9for h in [0.1, 0.01, 0.001, 0.0001, 1e-5]:
10    x_r = 1 + h                                   # approach from the right
11    x_l = 1 - h                                   # approach from the left
12    print(f"{x_r:>10.5f}  {f(x_r):>12.6f}  right")
13    print(f"{x_l:>10.5f}  {f(x_l):>12.6f}  left")
14
15# Now try x = 1 exactly -- the forbidden point.
16try:
17    f(1)
18except ZeroDivisionError as e:
19    print(f"\nAt x = 1: ZeroDivisionError -- {e}")

The output prints a table that squeezes onto 2 from both sides, and the last three lines confirm that $f(1)$ really does raise ZeroDivisionError. The limit exists; the function value does not. These are two different things, and Python made the distinction painfully obvious.

PyTorch: Limits as Vectorised Evaluation

Pure Python required a for-loop. PyTorch lets us evaluate $f$ at many probe points in a single tensor operation. More importantly, we get a first glimpse of autograd — PyTorch's automatic-differentiation engine, which is nothing more than a limit machinery implemented in C++.

1import torch
2
3# Vectorised limit: evaluate f at many x-values close to 1 all at once.
4# Tensor broadcasting runs the arithmetic on all elements in parallel.
5x = torch.tensor([0.9, 0.99, 0.999, 1.001, 1.01, 1.1])
6y = (x**2 - 1) / (x - 1)
7
8print(f"{'x':>8}   {'f(x)':>8}")
9for xi, yi in zip(x.tolist(), y.tolist()):
10    print(f"{xi:>8.3f}   {yi:>8.4f}")
11
12# Autograd: the derivative IS a limit, computed symbolically.
13# f(x) = (x^2 - 1)/(x - 1) simplifies to x + 1, so f'(1) = 1.
14x1 = torch.tensor(1.0, requires_grad=True)
15(x1 + 1).backward()
16print(f"\nAutograd: d/dx (x + 1) at x = 1 = {x1.grad.item()}")

Seeding the Derivative — Interactive

We started with a physics question we could not answer: what is the instantaneous speed of a moving object? Let us close the loop and answer it — at least for one specific curve.

Consider the parabola $y = x^{2}$ at $x = 1$. Its slope there is well-defined in principle but there is no “rise-over-run” formula that works at a single point — the run would be zero. The trick is the same trick we just invented: approach the point with a secant line and take the limit of its slope.

The slope of the secant through $(1, 1)$ and $(1+h, (1+h)^{2})$ is, after a tiny algebra check,

and so $\displaystyle \lim_{h \to 0} \frac{(1+h)^{2} - 1}{h} = \lim_{h \to 0}(2 + h) = 2$. The number 2 is the tangent slope at $(1, 1)$ — our first derivative, computed with nothing more than the limit idea.

Drag $h$ all the way down. The secant (in orange) rotates onto the tangent (the dashed grey line) and the displayed slope converges to 2 from above. This is the picture behind every derivative in Chapter 4 — and, by extension, every gradient in gradient descent.

Common Pitfalls

• Confusing $\displaystyle \lim_{x \to a} f(x)$ with $f(a)$. They are independent. The limit can exist when $f(a)$ does not; they can also disagree (think of a graph with a single misplaced dot). The limit is about trend, not evaluation.
• Treating 0/0 as 0 or as 1. $0/0$ is an indeterminate form. It is a flag that says “do more work” — factor, rationalise, apply L'Hôpital — not a shortcut to an answer.
• Using a finite sample as proof. Our numerical table suggests $L = 2$ but does not prove it. Floating-point arithmetic can mislead: tables for $\sin(1/x)$ near 0 can look like they converge to any value you like, depending on where you sample. The formal definition (next sections) is what forbids that trap.
• Forgetting that both sides must agree. A left-only approach is not a limit. When a function has a natural boundary — a square root at zero, a log at a negative input — the two-sided limit simply does not exist there.

Summary

• The limit formalises the intuition of “the value a function is heading toward,” even when the function is not defined at the target.
• $\displaystyle \lim_{x \to a} f(x) = L$ means: for any tolerance $\varepsilon > 0$ there is a neighbourhood of $a$ on which $f$ stays within $\varepsilon$ of $L$.
• The two-sided limit exists iff the left and right limits exist and agree.
• Limits fail to exist through jumps, blow-up, wild oscillation, or one-sided domains.
• The derivative is a limit (of a secant slope). Autograd evaluates it for us symbolically; the underlying mathematics is exactly what we practiced here.