Boo-AI — Master Artificial Intelligence by Building from Scratch

Learning Objectives

By the end of this section, you will be able to:

Define vector-valued functions and understand how they differ from scalar functions
Interpret vector functions geometrically as curves in 2D or 3D space
Determine the domain and range of vector-valued functions
Compute limits of vector functions using component-wise analysis
Identify when a vector function is continuous
Recognize common space curves: helices, twisted cubics, and knots
Connect vector functions to trajectories in physics and optimization paths in machine learning

The Big Picture: Functions That Output Vectors

"To describe motion through space, we need more than a single number — we need a direction at every instant."

So far in calculus, we have studied functions of the form $y = f(x)$ , where a single input produces a single output. But many real-world phenomena require tracking multiple quantities simultaneously:

🚀 Physics: Particle Motion

A particle's position in space is described by three coordinates $(x(t), y(t), z(t))$ that all change with time. The position is a vector that evolves as a function of $t$ .

🧬 Biology: DNA Helix

The backbone of DNA traces a double helix through space. Each strand follows a spiral path that can be precisely described by a vector function.

💻 Graphics: Animation Paths

In computer animation, objects follow paths through 3D space. These trajectories are vector functions that determine position at each frame.

🤖 ML: Optimization Paths

When training neural networks, parameters follow a path through high-dimensional space as they converge to optimal values. This trajectory is a vector-valued function.

A vector-valued function (or vector function) takes a scalar input and produces a vector output. Instead of mapping numbers to numbers, it maps numbers to points in space.

Historical Context: The Path to Space Curves

The study of curves in space has deep roots in both mathematics and physics. Key developments include:

From Plane Curves to Space Curves

Leonhard Euler (1707-1783) was among the first to systematically study curves in three dimensions. He recognized that a curve in space could be described by giving each coordinate as a function of a single parameter.

Joseph-Louis Lagrange (1736-1813) developed the foundations of analytical mechanics, where particle trajectories became the central objects of study. His formulation of mechanics naturally led to thinking of motion as vector-valued functions of time.

The Development of Vector Analysis

The modern language of vectors emerged in the 19th century through the work of William Rowan Hamilton (quaternions), Hermann Grassmann (exterior algebra), and Josiah Willard Gibbs (vector analysis). Gibbs, in particular, synthesized these ideas into the vector notation we use today.

From Physics to Pure Mathematics

While vector functions arose from physical problems — describing planetary orbits, projectile motion, and electromagnetic fields — they became fundamental objects in differential geometry, where curves and surfaces are studied for their intrinsic mathematical properties.

Vector-Valued Functions: The Definition

Definition: Vector-Valued Function

A vector-valued function (or vector function) is a function whose input is a scalar and whose output is a vector.

In three dimensions:

\mathbf{r}(t) = \langle f(t), g(t), h(t) \rangle = f(t)\mathbf{i} + g(t)\mathbf{j} + h(t)\mathbf{k}

where $f$ , $g$ , and $h$ are real-valued functions called the component functions.

Understanding the Notation

Symbol	Meaning	Example
r(t)	The vector function itself	r(t) = (cos t, sin t, t)
t	The parameter (input)	Often represents time
f(t), g(t), h(t)	Component functions	cos t, sin t, t
i, j, k	Standard basis vectors	(1,0,0), (0,1,0), (0,0,1)

Example: Evaluating a Vector Function

Consider the vector function $\mathbf{r}(t) = \langle \cos t, \sin t, t \rangle$ . Let's evaluate it at several points:

At t = 0: $\mathbf{r}(0) = \langle \cos 0, \sin 0, 0 \rangle = \langle 1, 0, 0 \rangle$

At t = π/2: $\mathbf{r}(\pi/2) = \langle 0, 1, \pi/2 \rangle$

At t = π: $\mathbf{r}(\pi) = \langle -1, 0, \pi \rangle$

At t = 2π: $\mathbf{r}(2\pi) = \langle 1, 0, 2\pi \rangle$

Notice that the $x$ and $y$ components trace a circle while the $z$ component increases linearly. This creates a helix.

Space Curves: The Geometric Interpretation

The set of all points $(f(t), g(t), h(t))$ traced out as $t$ varies over its domain is called a space curve (or simply a curve).

Key Insight: Vector as Position

For each value of $t$ , the vector $\mathbf{r}(t)$ is a position vector — an arrow from the origin to the point $(f(t), g(t), h(t))$ on the curve.

As $t$ changes, the tip of this arrow traces out the curve in space.

Interactive: Explore Vector Functions

Use the visualization below to see how different vector functions trace curves through 3D space. Watch how the position vector $\mathbf{r}(t)$ sweeps through space as $t$ varies.

Loading 3D visualization...

Visualizing the Parameter

Think of $t$ as time. The curve represents the path of a moving particle, and $\mathbf{r}(t)$ gives the particle's position at time $t$ .

Domain and Range

Domain of a Vector Function

The domain of $\mathbf{r}(t)$ consists of all values of $t$ for which $\mathbf{r}(t)$ is defined. This is the intersection of the domains of all component functions.

Domain Rule

\text{Domain of } \mathbf{r} = \text{Domain of } f \cap \text{Domain of } g \cap \text{Domain of } h

Example: Finding the Domain

Find the domain of $\mathbf{r}(t) = \left\langle \sqrt{t-1}, \ln t, \frac{1}{t-3} \right\rangle$ .

Solution:

$\sqrt{t-1}$ requires $t \geq 1$
$\ln t$ requires $t > 0$
$\frac{1}{t-3}$ requires $t \neq 3$

Domain: $[1, 3) \cup (3, \infty)$ or equivalently $t \geq 1, t \neq 3$ .

Range of a Vector Function

The range is the set of all vectors (or equivalently, all points) $\mathbf{r}(t)$ as $t$ varies over the domain. Geometrically, this is the curve itself.

Limits and Continuity

Limits of Vector Functions

The limit of a vector function is computed component by component:

Limit Definition

\lim_{t \to a} \mathbf{r}(t) = \left\langle \lim_{t \to a} f(t), \lim_{t \to a} g(t), \lim_{t \to a} h(t) \right\rangle

provided all three component limits exist.

Example: Computing a Limit

Find $\lim_{t \to 0} \mathbf{r}(t)$ where $\mathbf{r}(t) = \left\langle \frac{\sin t}{t}, \frac{1-\cos t}{t}, e^t \right\rangle$ .

Solution: Compute each component limit:

$\lim_{t \to 0} \frac{\sin t}{t} = 1$ (famous limit)
$\lim_{t \to 0} \frac{1-\cos t}{t} = 0$ (L'Hôpital or Taylor series)
$\lim_{t \to 0} e^t = 1$

Answer: $\lim_{t \to 0} \mathbf{r}(t) = \langle 1, 0, 1 \rangle$

Continuity of Vector Functions

Definition of Continuity

A vector function $\mathbf{r}(t)$ is continuous at t = a if:

\lim_{t \to a} \mathbf{r}(t) = \mathbf{r}(a)

Equivalently, $\mathbf{r}$ is continuous at $a$ if and only if all three component functions are continuous at $a$ .

Continuity Means No Breaks

Geometrically, continuity means the curve has no breaks or jumps. A continuous curve can be drawn without lifting the pen.

Interactive: Limits and Continuity

Explore continuous and discontinuous vector functions. See how discontinuities in component functions create breaks in space curves.

Loading visualization...

Common Space Curves

Several space curves appear frequently in mathematics, physics, and engineering. Understanding these examples builds intuition for more complex curves.

1. The Circular Helix

Definition: $\mathbf{r}(t) = \langle a\cos t, a\sin t, bt \rangle$

where $a$ is the radius and $b$ controls the vertical pitch.

Properties:

Projects to a circle of radius $a$ in the xy-plane
Rises at constant rate in the z-direction
One complete turn occurs over interval $[0, 2\pi]$

2. The Twisted Cubic

Definition: $\mathbf{r}(t) = \langle t, t^2, t^3 \rangle$

Properties:

The simplest space curve not contained in any plane
Fundamental example in algebraic geometry
Intersection of parabolic cylinders: $y = x^2$ and $z = x^3$

3. The Trefoil Knot

Definition: $\mathbf{r}(t) = \langle \sin t + 2\sin 2t, \cos t - 2\cos 2t, -\sin 3t \rangle$

Properties:

The simplest nontrivial knot
Cannot be untangled without cutting
Closed curve: $\mathbf{r}(0) = \mathbf{r}(2\pi)$

Interactive: Build Your Own Helix

Adjust the parameters of a helix and observe how the curve changes. See the projections onto coordinate planes.

Loading 3D visualization...

Different Parameterizations, Same Curve

An important concept: the same curve can be described by different parameterizations. The parameter is not intrinsic to the curve itself.

Example: The Unit Circle

All of these vector functions trace the unit circle in the xy-plane:

Parameterization	Domain	Description
r(t) = (cos t, sin t, 0)	[0, 2π]	Standard, one revolution
r(t) = (cos 2t, sin 2t, 0)	[0, π]	Twice as fast
r(t) = (sin t, cos t, 0)	[0, 2π]	Starts at (0,1), clockwise
r(t) = (cos(-t), sin(-t), 0)	[0, 2π]	Clockwise direction

Orientation Matters

The orientation of a curve (the direction of travel as $t$ increases) depends on the parameterization. Some applications, like computing work done by a force, are sensitive to orientation.

Real-World Applications

1. Particle Kinematics

In physics, the position of a moving particle is naturally described by a vector function:

\mathbf{r}(t) = \langle x(t), y(t), z(t) \rangle

The velocity and acceleration are obtained by differentiation (covered in the next section):

Velocity: $\mathbf{v}(t) = \mathbf{r}'(t)$
Acceleration: $\mathbf{a}(t) = \mathbf{r}''(t)$

2. Planetary Orbits

Planets follow elliptical orbits that can be parameterized as vector functions. Kepler's laws describe these trajectories mathematically.

3. Roller Coaster Design

The track of a roller coaster is designed as a space curve. Engineers carefully control curvature and torsion to create exciting but safe experiences.

4. Robot Path Planning

In robotics, the end-effector of a robot arm traces a path through 3D space. Planning smooth, efficient trajectories requires understanding space curves.

Machine Learning Applications

Vector-valued functions appear naturally in machine learning, even though we often work in much higher dimensions than three.

1. Gradient Descent Trajectories

When training a neural network with parameters $\boldsymbol{\theta} = (\theta_1, \theta_2, \ldots, \theta_n)$ , each gradient descent step updates the parameters:

\boldsymbol{\theta}_{k+1} = \boldsymbol{\theta}_k - \eta \nabla L(\boldsymbol{\theta}_k)

The sequence $\boldsymbol{\theta}_0, \boldsymbol{\theta}_1, \ldots$ traces a path through parameter space. In the continuous limit, this becomes a vector-valued function $\boldsymbol{\theta}(t)$ governed by a differential equation.

2. Embedding Trajectories

Word embeddings like Word2Vec map words to vectors. During training, each word's embedding follows a trajectory through the embedding space.

3. Loss Landscape Exploration

Understanding how optimization algorithms navigate loss landscapes involves analyzing paths through high-dimensional spaces — fundamentally the study of vector-valued functions.

High Dimensions

While we visualize 3D curves, ML often involves thousands or millions of dimensions. The mathematical principles remain the same: component-wise analysis, limits, and continuity all generalize naturally.

Python Implementation

Creating and Visualizing Space Curves

Space Curves in Python

🐍space_curves.py

Explanation(3)

Code(86)

4Helix Definition

The helix is defined parametrically with radius and pitch parameters. As t increases, the curve spirals around the z-axis while moving upward.

EXAMPLE

At t = 0: r(0) = (1, 0, 0). At t = π: r(π) = (-1, 0, π/2).

22Twisted Cubic

The twisted cubic (t, t², t³) is the simplest example of a space curve that cannot lie in any single plane. It twists through 3D space.

32Trefoil Knot

The trefoil knot is a closed curve that loops around itself and cannot be untied. It's the simplest nontrivial mathematical knot.

83 lines without explanation

1import numpy as np
2import matplotlib.pyplot as plt
3from mpl_toolkits.mplot3d import Axes3D
4
5def helix(t, radius=1, pitch=0.5):
6    """
7    Parametric representation of a circular helix.
8
9    The helix is a fundamental space curve defined by:
10    r(t) = (r*cos(t), r*sin(t), c*t)
11
12    where r is the radius and c is the vertical pitch.
13
14    This curve appears in:
15    - DNA double helix structure
16    - Screw threads and springs
17    - Circular staircases
18    - Particle paths in magnetic fields
19    """
20    x = radius * np.cos(t)
21    y = radius * np.sin(t)
22    z = pitch * t
23    return np.array([x, y, z])
24
25def twisted_cubic(t):
26    """
27    The twisted cubic: r(t) = (t, t², t³)
28
29    This is the simplest space curve that is not contained
30    in any plane. It's a fundamental example in algebraic
31    geometry and differential geometry.
32    """
33    return np.array([t, t**2, t**3])
34
35def trefoil_knot(t):
36    """
37    The trefoil knot: the simplest nontrivial knot.
38
39    r(t) = (sin(t) + 2sin(2t), cos(t) - 2cos(2t), -sin(3t))
40
41    This curve forms a closed loop that cannot be
42    untangled without cutting - a fundamental object
43    in knot theory with applications in molecular biology.
44    """
45    x = np.sin(t) + 2 * np.sin(2 * t)
46    y = np.cos(t) - 2 * np.cos(2 * t)
47    z = -np.sin(3 * t)
48    return np.array([x, y, z])
49
50# Generate and visualize multiple space curves
51fig = plt.figure(figsize=(15, 5))
52
53# Helix
54ax1 = fig.add_subplot(131, projection='3d')
55t_helix = np.linspace(0, 4*np.pi, 500)
56curve = np.array([helix(t) for t in t_helix]).T
57ax1.plot(curve[0], curve[1], curve[2], 'c-', linewidth=2)
58ax1.set_title('Helix: r(t) = (cos t, sin t, t/2)')
59ax1.set_xlabel('x'); ax1.set_ylabel('y'); ax1.set_zlabel('z')
60
61# Twisted Cubic
62ax2 = fig.add_subplot(132, projection='3d')
63t_cubic = np.linspace(-1.5, 1.5, 500)
64curve = np.array([twisted_cubic(t) for t in t_cubic]).T
65ax2.plot(curve[0], curve[1], curve[2], 'm-', linewidth=2)
66ax2.set_title('Twisted Cubic: r(t) = (t, t², t³)')
67ax2.set_xlabel('x'); ax2.set_ylabel('y'); ax2.set_zlabel('z')
68
69# Trefoil Knot
70ax3 = fig.add_subplot(133, projection='3d')
71t_knot = np.linspace(0, 2*np.pi, 500)
72curve = np.array([trefoil_knot(t) for t in t_knot]).T
73ax3.plot(curve[0], curve[1], curve[2], 'g-', linewidth=2)
74ax3.set_title('Trefoil Knot')
75ax3.set_xlabel('x'); ax3.set_ylabel('y'); ax3.set_zlabel('z')
76
77plt.tight_layout()
78plt.show()
79
80# Demonstrate vector function evaluation
81print("\n=== Vector Function Evaluation ===")
82print("\nHelix at t = π:")
83print(f"  r(π) = {helix(np.pi)}")
84
85print("\nTwisted cubic at t = 2:")
86print(f"  r(2) = {twisted_cubic(2)}")

Computing Limits and Checking Continuity

Limits of Vector Functions

🐍vector_limits.py

Explanation(3)

Code(68)

3Component-wise Limits

The limit of a vector function is computed by taking the limit of each component function separately. All components must have limits for the vector limit to exist.

24Continuity Check

A vector function is continuous at a point if: (1) the function is defined there, (2) the limit exists, and (3) the limit equals the function value.

43Classical Limit

The limit lim(t→0) sin(t)/t = 1 is one of the most important limits in calculus. We handle the indeterminate form by defining the value at t=0 to make the function continuous.

65 lines without explanation

1import numpy as np
2
3def vector_limit(r, a, epsilon=1e-10):
4    """
5    Estimate the limit of vector function r(t) as t approaches a.
6
7    For a vector function r(t) = (f(t), g(t), h(t)), the limit
8    is computed component-wise:
9
10    lim r(t) = (lim f(t), lim g(t), lim h(t))
11    t→a        t→a       t→a       t→a
12
13    This is a numerical approximation using values close to a.
14    """
15    # Approach from both sides
16    left = r(a - epsilon)
17    right = r(a + epsilon)
18
19    # Check if limits agree (within tolerance)
20    if np.allclose(left, right, rtol=1e-6):
21        return (left + right) / 2
22    else:
23        return None  # Limit doesn't exist
24
25def is_continuous(r, a, epsilon=1e-10):
26    """
27    Check if r(t) is continuous at t = a.
28
29    A vector function is continuous at a if:
30    1. r(a) is defined
31    2. lim r(t) exists as t→a
32    3. lim r(t) = r(a)
33    """
34    try:
35        value_at_a = r(a)
36        limit = vector_limit(r, a, epsilon)
37
38        if limit is None:
39            return False, "Limit does not exist"
40
41        if np.allclose(value_at_a, limit, rtol=1e-6):
42            return True, "Continuous"
43        else:
44            return False, "Limit ≠ function value"
45    except:
46        return False, "Function undefined at a"
47
48# Example: Check continuity of sin(t)/t as t→0
49def sinc_vector(t):
50    """r(t) = (sin(t)/t, (1-cos(t))/t, t) for t ≠ 0"""
51    if abs(t) < 1e-10:
52        return np.array([1.0, 0.0, 0.0])  # Limit values
53    return np.array([
54        np.sin(t) / t,
55        (1 - np.cos(t)) / t,
56        t
57    ])
58
59print("=== Limit and Continuity Analysis ===")
60print("\nFor r(t) = (sin(t)/t, (1-cos(t))/t, t):")
61
62limit = vector_limit(sinc_vector, 0)
63print(f"lim r(t) as t→0: {limit}")
64
65# Famous limits verified
66print("\nComponent limits (verified):")
67print(f"  lim sin(t)/t = {np.sin(1e-10)/1e-10:.10f} ≈ 1")
68print(f"  lim (1-cos(t))/t = {(1-np.cos(1e-10))/1e-10:.10f} ≈ 0")

Optimization as a Space Curve

Gradient Descent Trajectory

🐍optimization_trajectory.py

Explanation(3)

Code(64)

4Optimization as Space Curve

Gradient descent traces a path through parameter space. Each step updates the position, creating a discrete approximation of a continuous trajectory r(t).

26Loss Surface

The loss function defines a surface in the parameter space. Gradient descent follows the steepest descent direction on this surface.

38Trajectory Analysis

By viewing the optimization path as a space curve, we can analyze convergence, oscillations, and the geometry of how the algorithm explores the loss landscape.

61 lines without explanation

1import numpy as np
2import matplotlib.pyplot as plt
3
4def gradient_descent_trajectory(loss_fn, grad_fn, x0, lr=0.1, steps=100):
5    """
6    Track the trajectory of gradient descent as a space curve.
7
8    In machine learning, optimization algorithms trace paths through
9    high-dimensional parameter spaces. These paths are naturally
10    described as vector-valued functions of the iteration step.
11
12    r(t) = (θ₁(t), θ₂(t), ..., θₙ(t))
13
14    where t represents the optimization step and θᵢ are parameters.
15    """
16    trajectory = [x0.copy()]
17    x = x0.copy()
18
19    for _ in range(steps):
20        grad = grad_fn(x)
21        x = x - lr * grad
22        trajectory.append(x.copy())
23
24    return np.array(trajectory)
25
26# Example: Minimize a 3D quadratic function
27def loss_3d(x):
28    """Bowl-shaped loss: L(x,y,z) = x² + 2y² + 3z²"""
29    return x[0]**2 + 2*x[1]**2 + 3*x[2]**2
30
31def grad_3d(x):
32    """Gradient: ∇L = (2x, 4y, 6z)"""
33    return np.array([2*x[0], 4*x[1], 6*x[2]])
34
35# Track optimization trajectory
36x0 = np.array([3.0, 2.0, 1.5])
37trajectory = gradient_descent_trajectory(loss_3d, grad_3d, x0, lr=0.1)
38
39print("=== Gradient Descent as a Space Curve ===")
40print(f"\nStarting point: {x0}")
41print(f"Final point: {trajectory[-1]}")
42print(f"\nTrajectory is a vector function r(t) where t = step number")
43print(f"r(0) = {trajectory[0]}")
44print(f"r(50) = {trajectory[50]}")
45print(f"r(100) = {trajectory[100]}")
46
47# The trajectory approaches the minimum at origin
48print(f"\nMinimum (expected): [0, 0, 0]")
49print(f"Distance from minimum: {np.linalg.norm(trajectory[-1]):.6f}")
50
51# Visualize
52fig = plt.figure(figsize=(10, 8))
53ax = fig.add_subplot(111, projection='3d')
54
55ax.plot(trajectory[:, 0], trajectory[:, 1], trajectory[:, 2],
56        'b-', linewidth=2, label='Optimization path')
57ax.scatter(*x0, color='red', s=100, label='Start')
58ax.scatter(*trajectory[-1], color='green', s=100, label='End')
59ax.scatter(0, 0, 0, color='gold', s=100, marker='*', label='Minimum')
60
61ax.set_xlabel('θ₁'); ax.set_ylabel('θ₂'); ax.set_zlabel('θ₃')
62ax.set_title('Gradient Descent Trajectory in Parameter Space')
63ax.legend()
64plt.show()

Test Your Understanding

Summary

Vector-valued functions are the natural language for describing motion and paths through space. They extend single-variable calculus to handle the richness of multidimensional geometry.

Key Concepts

Concept	Description
Vector Function	r(t) = (f(t), g(t), h(t)) maps scalars to vectors
Space Curve	The set of points traced by r(t) as t varies
Domain	Intersection of component function domains
Limit	Computed component-wise
Continuity	Requires all components to be continuous
Parameterization	Same curve can have different parameterizations

Key Takeaways

A vector function $\mathbf{r}(t)$ takes a scalar input and produces a vector output
The space curve is the geometric object traced by the tip of the position vector
Limits and continuity are determined component by component
Common curves like helices and twisted cubics appear throughout mathematics and physics
The same curve can have multiple parameterizations, each with different orientation or speed
Vector functions describe optimization trajectories in machine learning

The Essence of Vector Functions:

"A vector function

\mathbf{r}(t)

is a journey through space — the position at each moment completely determined by a single parameter."

Coming Next: In the next section, we'll explore derivatives of vector functions. Just as the derivative of a scalar function gives the rate of change, the derivative of a vector function gives the velocity vector — capturing both how fast and in what direction the curve is traversed.