Derivatives of Vector Functions

Learning Objectives

By the end of this section, you will be able to:

1. Define the derivative of a vector-valued function using limits
2. Compute derivatives by differentiating each component separately
3. Apply differentiation rules: sum, product, chain rule for vector functions
4. Interpret the derivative geometrically as the tangent vector to a curve
5. Calculate velocity, speed, and acceleration for motion problems
6. Find unit tangent vectors and understand their significance
7. Connect vector derivatives to gradients in machine learning

The Big Picture: Why Differentiate Vectors?

"The derivative tells us how things change — and in the vector world, change means not just 'how fast' but also 'in which direction.'"

In single-variable calculus, the derivative $f'(x)$ tells us the instantaneous rate of change of a function — how fast and in what sense (increasing or decreasing) the output changes as we nudge the input. With vector-valued functions, we face a richer question: when a point moves along a curve in space, how does its position vector change?

The answer is the derivative of a vector function, which gives us a new vector — the tangent vector to the curve. This tangent vector captures:

• Direction: Which way is the point moving at this instant?
• Speed: How fast is it moving (the magnitude of the tangent)?
• Velocity: The complete picture — direction and speed together

Where Vector Derivatives Appear

Historical Origins

The calculus of vector functions developed alongside classical mechanics in the 17th-19th centuries, as mathematicians sought precise ways to describe motion in space.

Newton and Leibniz: The Foundations

Isaac Newton (1643–1727) essentially invented vector calculus to solve physics problems. His "method of fluxions" treated velocity as the rate of change of position — exactly our modern concept of the derivative of a position vector. His laws of motion require computing derivatives of vector quantities.

Gottfried Leibniz (1646–1716) developed the notation we still use today. The symbols $\frac{d\mathbf{r}}{dt}$ and $\mathbf{r}'(t)$ both trace back to his systematic approach to infinitesimal calculus.

The 19th Century Formalization

The formal treatment of vector derivatives emerged with the work of William Rowan Hamilton, Josiah Willard Gibbs, and Oliver Heaviside in the 1800s. They established that:

• Vector functions can be differentiated component-wise
• The derivative rules (product, chain) extend naturally to vectors
• The geometric meaning is the tangent vector to the curve

The Definition: Derivative of a Vector Function

This definition is identical in form to the scalar derivative — we're taking the limit of a difference quotient. The key insight is that subtracting vectors and dividing by a scalar still yields a vector.

Component-Wise Differentiation

The remarkable practical consequence is that we can differentiate component by component:

This follows directly from limit laws: the limit of a sum is the sum of limits, and limits can be taken component by component.

Example: Circular Motion

Consider a particle moving on a unit circle: $\mathbf{r}(t) = \langle \cos(t), \sin(t) \rangle$

Visualizing the Limit: Secant to Tangent

Just as the derivative in single-variable calculus arises from the limit of secant lines approaching a tangent line, the vector derivative arises from secant vectors approaching the tangent vector.

The secant vector from $\mathbf{r}(t_0)$ to $\mathbf{r}(t_0 + \Delta t)$ is:

As $\Delta t \to 0$, this secant vector rotates and stretches/shrinks to become the tangent vector $\mathbf{r}'(t_0)$.

Use the interactive visualization below to watch the limit process in action:

Differentiation Rules for Vector Functions

All the familiar differentiation rules extend to vector functions. Here are the key rules:

Basic Rules

Product Rules

There are three important product rules for vectors:

Chain Rule

If $\mathbf{r}(t)$ is a vector function and $t = g(s)$ is a scalar function, then:

Geometric Interpretation: The Tangent Vector

The derivative $\mathbf{r}'(t)$ has a beautiful geometric meaning: it is the tangent vector to the curve at the point $\mathbf{r}(t)$.

The Tangent Line

The parametric equation of the tangent line at $t = t_0$ is:

Here, $s$ is a parameter ranging over all real numbers. When $s = 0$, we're at the point of tangency.

Interactive Exploration

Explore how the tangent vector changes as you move along different curves. Notice how the tangent always points in the direction of motion.

Velocity and Speed: The Physical Interpretation

When $\mathbf{r}(t)$ represents the position of a particle at time $t$, the derivative has a direct physical meaning:

This distinction is crucial: velocity tells you how fast and in what direction something is moving, while speed tells you only how fast.

Explore this distinction interactively below. Watch how the velocity vector changes direction around the ellipse while the speed (its magnitude) varies:

Unit Tangent Vector

Sometimes we want just the direction of motion, without the speed information. This is captured by the unit tangent vector:

The unit tangent vector $\mathbf{T}(t)$ points in the direction of motion with a standardized length of 1. This is useful for:

• Describing the direction of a curve independently of parameterization
• Computing curvature (how fast the direction changes)
• Building the Frenet-Serret frame (T, N, B) for curve analysis
• Normalizing directions in computer graphics

Higher-Order Derivatives

Just as with scalar functions, we can take multiple derivatives of vector functions:

Each level of derivative tells us about the rate of change of the previous quantity.

Applications in Science and Engineering

1. Projectile Motion

A projectile launched with initial velocity $\mathbf{v}_0$ from position $\mathbf{r}_0$ under gravity follows:

Taking derivatives:

• Velocity: $\mathbf{v}(t) = \mathbf{v}_0 + \mathbf{g}t$
• Acceleration: $\mathbf{a}(t) = \mathbf{g}$ (constant)

2. Circular Motion

For uniform circular motion with radius $R$ and angular velocity $\omega$:

The acceleration points toward the center (centripetal) with magnitude $R\omega^2$.

3. Robotics: End Effector Velocity

In robotics, the Jacobian relates joint velocities to end effector velocity. If joint angles are $\mathbf{q}(t)$ and the end effector position is $\mathbf{r}(\mathbf{q})$, then:

This is the chain rule for vectors, relating joint velocities $\dot{\mathbf{q}}$ to workspace velocity $\mathbf{v}_{end}$.

Machine Learning Applications

Vector derivatives are the heart of machine learning optimization. Every time you train a neural network, you're computing vector derivatives.

The Gradient: Derivative of a Scalar Function

Given a loss function $L(\mathbf{w})$ that depends on a weight vector $\mathbf{w} = (w_1, w_2, ..., w_n)$, the gradient is:

This gradient vector points in the direction of steepest increase of $L$. To minimize the loss, we move in the opposite direction:

where $\eta$ is the learning rate.

Backpropagation: Chain Rule in Action

Neural networks are compositions of functions: $L = L_n \circ L_{n-1} \circ ... \circ L_1$. The chain rule gives us:

This is exactly the vector chain rule applied repeatedly! Each term is a Jacobian matrix, and backpropagation efficiently computes this product.

Python Implementation

Vector Derivatives in NumPy

Here's how to work with vector function derivatives in Python:

1import numpy as np
2import matplotlib.pyplot as plt
3from mpl_toolkits.mplot3d import Axes3D
4
5# ============================================
6# DERIVATIVES OF VECTOR FUNCTIONS
7# ============================================
8
9def r(t):
10    """Position vector: r(t) = ⟨cos(t), sin(t), t/2⟩ (helix)"""
11    return np.array([np.cos(t), np.sin(t), t/2])
12
13def r_prime(t):
14    """Velocity vector (derivative): r'(t) = ⟨-sin(t), cos(t), 0.5⟩"""
15    return np.array([-np.sin(t), np.cos(t), 0.5])
16
17def r_double_prime(t):
18    """Acceleration vector (second derivative): r''(t) = ⟨-cos(t), -sin(t), 0⟩"""
19    return np.array([-np.cos(t), -np.sin(t), 0])
20
21# Evaluate at a specific time
22t0 = np.pi / 4
23position = r(t0)
24velocity = r_prime(t0)
25acceleration = r_double_prime(t0)
26
27print(f"At t = π/4:")
28print(f"  Position r(t):     {position}")
29print(f"  Velocity r'(t):    {velocity}")
30print(f"  Acceleration r''(t): {acceleration}")
31
32# Speed (magnitude of velocity)
33speed = np.linalg.norm(velocity)
34print(f"  Speed |r'(t)|:     {speed:.4f}")
35
36# Unit tangent vector
37T = velocity / speed
38print(f"  Unit tangent T(t): {T}")
39
40# ============================================
41# NUMERICAL DIFFERENTIATION
42# ============================================
43
44def numerical_derivative(r_func, t, h=1e-5):
45    """Approximate derivative using central differences."""
46    return (r_func(t + h) - r_func(t - h)) / (2 * h)
47
48# Compare numerical vs. analytical
49numerical_vel = numerical_derivative(r, t0)
50analytical_vel = r_prime(t0)
51
52print(f"\n--- Numerical vs Analytical ---")
53print(f"Numerical r'(t):  {numerical_vel}")
54print(f"Analytical r'(t): {analytical_vel}")
55print(f"Error: {np.linalg.norm(numerical_vel - analytical_vel):.2e}")
56
57# ============================================
58# VISUALIZATION
59# ============================================
60
61fig = plt.figure(figsize=(14, 5))
62
63# 3D helix with tangent and acceleration vectors
64ax1 = fig.add_subplot(131, projection='3d')
65
66# Draw the helix
67t_vals = np.linspace(0, 4*np.pi, 200)
68x = np.cos(t_vals)
69y = np.sin(t_vals)
70z = t_vals / 2
71
72ax1.plot(x, y, z, 'b-', linewidth=2, label='r(t) helix')
73
74# Draw vectors at t0
75pos = r(t0)
76vel = r_prime(t0) * 0.5  # Scale for visibility
77acc = r_double_prime(t0) * 0.5
78
79ax1.quiver(*pos, *vel, color='orange', linewidth=2,
80           label="r'(t) velocity", arrow_length_ratio=0.2)
81ax1.quiver(*pos, *acc, color='red', linewidth=2,
82           label="r''(t) acceleration", arrow_length_ratio=0.2)
83ax1.scatter(*pos, color='green', s=100, zorder=5)
84
85ax1.set_xlabel('X')
86ax1.set_ylabel('Y')
87ax1.set_zlabel('Z')
88ax1.legend(loc='upper left')
89ax1.set_title('Helix with Velocity and Acceleration')
90
91# Speed over time
92ax2 = fig.add_subplot(132)
93speeds = [np.linalg.norm(r_prime(t)) for t in t_vals]
94ax2.plot(t_vals, speeds, 'g-', linewidth=2)
95ax2.axhline(y=np.sqrt(1.25), color='r', linestyle='--',
96            label=f'Constant speed = √1.25')
97ax2.set_xlabel('Time t')
98ax2.set_ylabel('Speed |r\'(t)|')
99ax2.set_title('Speed vs Time')
100ax2.legend()
101
102# Tangent vector components
103ax3 = fig.add_subplot(133)
104T_x = [-np.sin(t)/np.sqrt(1.25) for t in t_vals]
105T_y = [np.cos(t)/np.sqrt(1.25) for t in t_vals]
106T_z = [0.5/np.sqrt(1.25) for t in t_vals]
107
108ax3.plot(t_vals, T_x, 'r-', label='T_x')
109ax3.plot(t_vals, T_y, 'g-', label='T_y')
110ax3.plot(t_vals, T_z, 'b-', label='T_z')
111ax3.set_xlabel('Time t')
112ax3.set_ylabel('Component')
113ax3.set_title('Unit Tangent Vector Components')
114ax3.legend()
115
116plt.tight_layout()
117plt.show()

Gradients in Machine Learning

Here's how vector derivatives appear in ML optimization:

w_new = w_old - learning_rate × ∂L/∂w

1import numpy as np
2
3# ============================================
4# VECTOR DERIVATIVES IN MACHINE LEARNING
5# ============================================
6
7# In ML, we differentiate loss functions with respect to weight vectors
8# The gradient is a vector of partial derivatives
9
10def loss_function(w, X, y):
11    """Mean squared error loss: L = (1/n) * ||Xw - y||²"""
12    predictions = X @ w
13    residuals = predictions - y
14    return np.mean(residuals ** 2)
15
16def gradient(w, X, y):
17    """Gradient: ∂L/∂w = (2/n) * X.T @ (Xw - y)"""
18    n = len(y)
19    predictions = X @ w
20    residuals = predictions - y
21    return (2/n) * X.T @ residuals
22
23# Example data: 3 features, 5 samples
24np.random.seed(42)
25X = np.random.randn(5, 3)  # 5 samples, 3 features
26y = np.random.randn(5)      # 5 target values
27w = np.array([1.0, -0.5, 0.2])  # Initial weights
28
29print("--- Gradient Computation ---")
30print(f"Weight vector w: {w}")
31print(f"Loss L(w): {loss_function(w, X, y):.4f}")
32print(f"Gradient ∂L/∂w: {gradient(w, X, y)}")
33
34# ============================================
35# GRADIENT DESCENT: Following the Negative Gradient
36# ============================================
37
38def gradient_descent(X, y, learning_rate=0.1, iterations=100):
39    """Minimize loss by moving opposite to the gradient."""
40    w = np.zeros(X.shape[1])  # Initialize at origin
41    history = []
42
43    for i in range(iterations):
44        loss = loss_function(w, X, y)
45        grad = gradient(w, X, y)
46        history.append({'iter': i, 'loss': loss, 'w': w.copy(), 'grad': grad.copy()})
47
48        # Key step: update in NEGATIVE gradient direction
49        w = w - learning_rate * grad
50
51        if i < 5:
52            print(f"Iter {i}: loss = {loss:.4f}, |grad| = {np.linalg.norm(grad):.4f}")
53
54    return w, history
55
56print("\n--- Gradient Descent ---")
57w_optimal, history = gradient_descent(X, y)
58print(f"\nOptimal weights: {w_optimal}")
59print(f"Final loss: {loss_function(w_optimal, X, y):.6f}")
60
61# ============================================
62# PATH AS A VECTOR-VALUED FUNCTION
63# ============================================
64
65# The optimization path w(t) is like a parametric curve!
66# Each iteration t gives a new weight vector w(t)
67
68print("\n--- Optimization Path as Vector Function ---")
69print("Think of w(iteration) as a vector-valued function:")
70print("  w(0) → w(1) → w(2) → ... → w_optimal")
71print("\nThe 'velocity' is approximately -grad (direction of update)")
72print("The path curves through weight space toward the minimum!")
73
74# Path length (total distance traveled)
75total_distance = 0
76for i in range(1, len(history)):
77    step = np.linalg.norm(history[i]['w'] - history[i-1]['w'])
78    total_distance += step
79print(f"\nTotal path length: {total_distance:.4f}")

Common Pitfalls

Test Your Understanding

Summary

The derivative of a vector function extends the fundamental concept of instantaneous rate of change to curves in space. By differentiating component-wise, we obtain the tangent vector — a powerful tool for analyzing motion, geometry, and optimization.

Key Concepts

Key Takeaways

1. The derivative of a vector function is computed component by component
2. $\mathbf{r}'(t)$ is the tangent vector to the curve — it points in the direction of motion
3. Velocity is a vector (direction + speed); speed is a scalar (just magnitude)
4. All differentiation rules (sum, product, chain) extend naturally to vectors
5. The unit tangent vector $\mathbf{T}(t)$ captures direction without speed
6. In machine learning, gradients are vector derivatives used for optimization
7. Backpropagation is the chain rule for vectors applied through neural network layers

Coming Next: In the next section, we'll explore Arc Length and Curvature — using the tangent vector to measure how long a curve is and how sharply it bends. These concepts complete our toolkit for analyzing space curves.