Vectors in 2D and 3D

Learning Objectives

By the end of this section, you will be able to:

1. Define vectors as objects with both magnitude and direction
2. Represent vectors using component form in 2D and 3D coordinate systems
3. Compute vector magnitude (length) using the Pythagorean theorem
4. Perform vector addition, subtraction, and scalar multiplication
5. Find unit vectors and understand their significance
6. Express vectors using standard basis vectors $\mathbf{i}$, $\mathbf{j}$, $\mathbf{k}$
7. Apply vectors to physics, engineering, and machine learning problems

The Big Picture: Why Vectors Matter

"Vectors are the language of physics and the foundation of multivariable calculus." — Richard Feynman

Imagine giving directions to a friend. Saying "walk 100 meters" isn't enough — they need to know which direction. "Walk 100 meters north" provides the complete picture. This combination of magnitude (100 meters) and direction (north) is exactly what vectors capture.

Vectors are everywhere in the physical world. When you throw a ball, it has a velocity — not just a speed, but a speed in a particular direction. Forces, accelerations, electric fields, gravitational pulls — all are vector quantities that combine magnitude with direction.

Where Vectors Appear

Historical Origins: From Arrows to Algebra

The concept of vectors evolved over centuries, with contributions from mathematicians and physicists working on seemingly different problems.

Early Developments

The idea of combining magnitude and direction appears in ancient mechanics, but the formal mathematical treatment began in the 19th century. Sir William Rowan Hamilton (1805–1865) developed quaternions in 1843, a precursor to modern vector algebra.

Hermann Grassmann (1809–1877) independently created a comprehensive theory of what he called "extension theory" — essentially vector spaces — in 1844, though his work was largely ignored during his lifetime.

The Modern Vector Concept

The modern notation and approach came from Josiah Willard Gibbs (1839–1903) at Yale and Oliver Heaviside (1850–1925) in England. Working independently in the 1880s, both extracted the useful parts of Hamilton's quaternions to create the vector analysis we use today.

What is a Vector?

Key Terminology

Two vectors are considered equal if and only if they have the same magnitude and the same direction. Their position in space doesn't matter — only their length and orientation.

Vector Notation

Vectors are denoted in several ways, depending on context:

The zero vector, denoted $\mathbf{0}$ or $\vec{0}$, has magnitude 0. It's the only vector with no defined direction.

Vectors in 2D: The Cartesian Plane

In two dimensions, we describe vectors using components along the x-axis and y-axis.

Position Vectors

If we have a point $P = (a, b)$ in the plane, the position vector from the origin $O$ to $P$ is:

Vector Between Two Points

For points $A = (a_1, a_2)$ and $B = (b_1, b_2)$, the vector from $A$ to $B$ is:

This is found by "terminal point minus initial point" — subtract coordinates component-wise.

Interactive 2D Vector Visualization

Explore vectors in 2D. Adjust the components of each vector and observe how addition, subtraction, and scalar multiplication work. Notice how the resultant vector (sum) follows the parallelogram law.

Vectors in 3D: Three-Dimensional Space

In three dimensions, we add a third component along the z-axis, giving vectors the ability to point anywhere in space.

Right-Hand Coordinate System

We use the right-hand coordinate system: if you point your right hand's fingers along the positive x-axis and curl them toward the positive y-axis, your thumb points in the positive z-direction.

Interactive 3D Vector Visualization

Explore vectors in 3D space. Click and drag to rotate the view. Adjust the x, y, and z components of each vector and observe how they combine. Toggle the sum to see vector addition in three dimensions.

Magnitude and Direction

Magnitude (Length)

The magnitude (or length or norm) of a vector $\mathbf{v}$ is denoted $|\mathbf{v}|$ or $\|\mathbf{v}\|$.

These formulas come from the Pythagorean theorem. In 2D, the vector forms the hypotenuse of a right triangle with legs $v_x$ and $v_y$.

Direction

In 2D, direction is often specified by an angle $\theta$ measured counterclockwise from the positive x-axis:

In 3D, direction is specified by either:

• Direction cosines: The cosines of angles with each axis
• Direction angles: The angles $\alpha, \beta, \gamma$ with the x, y, and z axes
• Unit vector: The normalized vector pointing in the same direction

Unit Vectors

Finding a unit vector is called normalizing the vector. This process is essential when you need direction without magnitude — for example, when specifying the direction of movement regardless of speed.

Interactive Unit Vector Demonstration

Explore the relationship between a vector and its unit vector. The unit vector (shown dashed in green) always lies on the unit circle and points in the same direction as the original vector.

Vector Operations

Vector Addition

Vectors add component-wise:

Geometric interpretation:

• Parallelogram law: Place both vectors at the same starting point; the sum is the diagonal of the parallelogram they form
• Head-to-tail: Place the tail of $\mathbf{b}$ at the head of $\mathbf{a}$; the sum goes from the tail of $\mathbf{a}$ to the head of $\mathbf{b}$

Vector Subtraction

Vector subtraction is also component-wise:

Geometric interpretation: $\mathbf{a} - \mathbf{b}$ points from the tip of $\mathbf{b}$ to the tip of $\mathbf{a}$ when both start at the origin.

Scalar Multiplication

Multiplying a vector by a scalar $k$ scales each component:

Geometric interpretation:

• If $k > 1$: The vector is stretched (lengthened)
• If $0 < k < 1$: The vector is shrunk
• If $k < 0$: The vector is reversed and scaled
• If $k = 0$: The result is the zero vector

Properties of Vector Operations

Vector operations satisfy the same algebraic properties as regular number arithmetic:

Standard Basis Vectors

The standard basis vectors are unit vectors along each coordinate axis:

Any vector can be written as a linear combination of basis vectors:

For example, $\langle 3, -2, 5 \rangle = 3\mathbf{i} - 2\mathbf{j} + 5\mathbf{k}$.

Applications in Science and Engineering

Physics: Forces and Motion

When multiple forces act on an object, we find the net force by vector addition:

Computer Graphics: Transformations

In 3D graphics, vectors represent:

• Positions: Vertex locations in 3D models
• Normals: Surface directions for lighting calculations
• Directions: Camera viewing directions, light rays

Navigation: GPS and Aviation

Pilots and navigators use vectors constantly. Wind correction, course plotting, and ground speed calculations all involve vector addition to account for wind vectors affecting aircraft velocity vectors.

Machine Learning Applications

Vectors are the fundamental data structure in machine learning. Understanding vector operations is essential for every ML practitioner.

Feature Vectors

In ML, each data point is represented as a feature vector — a vector in high-dimensional space where each dimension corresponds to a feature.

Word Embeddings

Modern NLP represents words as vectors in a high-dimensional space (typically 100-300 dimensions). Similar words have similar vectors, enabling semantic operations:

Gradient Vectors

In optimization, the gradient is a vector pointing in the direction of steepest increase of a function. For a loss function $L(\mathbf{w})$ with parameters $\mathbf{w} = (w_1, w_2, ..., w_n)$:

Gradient descent updates parameters by moving in the opposite direction of the gradient to minimize loss.

Python Implementation

Vector Basics with NumPy

NumPy is the standard library for vector operations in Python. Here's how to work with vectors:

This is the position vector pointing 3 units right and 4 units up.

1import numpy as np
2import matplotlib.pyplot as plt
3from mpl_toolkits.mplot3d import Axes3D
4
5# ============================================
6# VECTORS IN 2D
7# ============================================
8
9# Creating 2D vectors using NumPy arrays
10a = np.array([3, 4])
11b = np.array([1, 2])
12
13print("Vector a:", a)
14print("Vector b:", b)
15
16# Vector operations
17print("\n--- Vector Operations ---")
18print("a + b =", a + b)           # Addition
19print("a - b =", a - b)           # Subtraction
20print("2 * a =", 2 * a)           # Scalar multiplication
21print("-b =", -b)                  # Negation
22
23# Magnitude (length) of a vector
24magnitude_a = np.linalg.norm(a)
25print("\n|a| =", magnitude_a)
26
27# Unit vector (direction without magnitude)
28unit_a = a / np.linalg.norm(a)
29print("Unit vector of a:", unit_a)
30print("Magnitude of unit vector:", np.linalg.norm(unit_a))
31
32# ============================================
33# VECTORS IN 3D
34# ============================================
35
36# Creating 3D vectors
37v = np.array([1, 2, 3])
38w = np.array([4, 5, 6])
39
40print("\n--- 3D Vectors ---")
41print("v =", v)
42print("w =", w)
43print("v + w =", v + w)
44print("|v| =", np.linalg.norm(v))
45
46# Unit vector in 3D
47unit_v = v / np.linalg.norm(v)
48print("Unit vector of v:", unit_v)
49
50# ============================================
51# VISUALIZATION
52# ============================================
53
54# 2D vector plot
55fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 6))
56
57# Plot 2D vectors
58origin = np.array([0, 0])
59ax1.quiver(*origin, *a, angles='xy', scale_units='xy', scale=1,
60           color='blue', label='a = (3, 4)')
61ax1.quiver(*origin, *b, angles='xy', scale_units='xy', scale=1,
62           color='red', label='b = (1, 2)')
63ax1.quiver(*origin, *(a+b), angles='xy', scale_units='xy', scale=1,
64           color='green', label='a + b')
65
66ax1.set_xlim(-1, 6)
67ax1.set_ylim(-1, 7)
68ax1.grid(True, alpha=0.3)
69ax1.set_aspect('equal')
70ax1.legend()
71ax1.set_title('2D Vectors and Addition')
72ax1.set_xlabel('x')
73ax1.set_ylabel('y')
74
75# Show head-to-tail addition
76ax1.quiver(*a, *b, angles='xy', scale_units='xy', scale=1,
77           color='red', alpha=0.5, linestyle='--')
78
79# 3D vector plot
80ax2 = fig.add_subplot(122, projection='3d')
81ax2.quiver(0, 0, 0, *v, color='blue', arrow_length_ratio=0.1,
82           label='v = (1, 2, 3)')
83ax2.quiver(0, 0, 0, *w, color='red', arrow_length_ratio=0.1,
84           label='w = (4, 5, 6)')
85ax2.quiver(0, 0, 0, *(v+w), color='green', arrow_length_ratio=0.1,
86           label='v + w')
87
88ax2.set_xlim([0, 7])
89ax2.set_ylim([0, 8])
90ax2.set_zlim([0, 10])
91ax2.set_xlabel('X')
92ax2.set_ylabel('Y')
93ax2.set_zlabel('Z')
94ax2.legend()
95ax2.set_title('3D Vectors')
96
97plt.tight_layout()
98plt.show()

Vectors in Machine Learning

Here's how vectors appear in practical ML applications:

price = 0.1*1500 + 50000*3 + 25000*2 + (-1000)*10 + 100000

1import numpy as np
2
3# ============================================
4# VECTORS IN MACHINE LEARNING
5# ============================================
6
7# Feature vectors: Each sample is a vector
8# Example: House features [area_sqft, bedrooms, bathrooms, age]
9house_1 = np.array([1500, 3, 2, 10])
10house_2 = np.array([2000, 4, 3, 5])
11house_3 = np.array([1200, 2, 1, 25])
12
13# Stack into a matrix (each row is a sample)
14X = np.array([house_1, house_2, house_3])
15print("Feature matrix X shape:", X.shape)  # (3 samples, 4 features)
16
17# Weight vector for linear regression
18# y = w · x + b  (dot product!)
19weights = np.array([0.1, 50000, 25000, -1000])
20bias = 100000
21
22# Predict prices using dot product
23predictions = X @ weights + bias  # Matrix-vector multiplication
24print("\nPredicted prices:", predictions)
25
26# ============================================
27# WORD EMBEDDINGS: Words as Vectors
28# ============================================
29
30# Simplified word embeddings (real ones have 300+ dimensions)
31embeddings = {
32    'king':   np.array([0.9, 0.8, 0.1, 0.2]),
33    'queen':  np.array([0.9, 0.2, 0.9, 0.2]),
34    'man':    np.array([0.5, 0.9, 0.1, 0.1]),
35    'woman':  np.array([0.5, 0.1, 0.9, 0.1]),
36}
37
38# Famous analogy: king - man + woman ≈ queen
39result = embeddings['king'] - embeddings['man'] + embeddings['woman']
40print("\n--- Word Vector Arithmetic ---")
41print("king - man + woman =", result)
42print("Actual queen vector =", embeddings['queen'])
43
44# Find closest word using cosine similarity
45def cosine_similarity(a, b):
46    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
47
48print("\nSimilarity of result to 'queen':",
49      cosine_similarity(result, embeddings['queen']))
50
51# ============================================
52# GRADIENT VECTORS IN OPTIMIZATION
53# ============================================
54
55def loss_function(w):
56    """MSE loss for simple linear model y = wx on data."""
57    X = np.array([1, 2, 3, 4, 5])
58    y = np.array([2, 4, 5, 4, 5])
59    predictions = w * X
60    return np.mean((predictions - y) ** 2)
61
62def gradient(w):
63    """Gradient of MSE loss with respect to w."""
64    X = np.array([1, 2, 3, 4, 5])
65    y = np.array([2, 4, 5, 4, 5])
66    return 2 * np.mean(X * (w * X - y))
67
68# Gradient descent
69w = 0.0  # Initial weight
70learning_rate = 0.1
71
72print("\n--- Gradient Descent ---")
73for i in range(10):
74    grad = gradient(w)
75    w = w - learning_rate * grad  # Update in negative gradient direction
76    if i < 5:
77        print(f"Step {i+1}: w = {w:.4f}, gradient = {grad:.4f}")
78
79print(f"Final weight: {w:.4f}")

Common Pitfalls

Test Your Understanding

Summary

Vectors are mathematical objects that capture both magnitude and direction, making them essential for describing physical quantities like velocity, force, and displacement.

Key Concepts

Key Takeaways

1. Vectors combine magnitude (how much) and direction (which way)
2. In coordinates, vectors are represented by ordered lists of components
3. Vector magnitude is computed using the Pythagorean theorem
4. Unit vectors have magnitude 1 and represent pure direction
5. Addition and scalar multiplication are performed component-wise
6. The standard basis vectors i, j, k allow any vector to be expressed as a linear combination
7. Vectors are fundamental in physics, graphics, and machine learning

Coming Next: In the next section, we'll explore the Dot Product — a powerful operation that multiplies two vectors to produce a scalar. The dot product reveals the angle between vectors and enables projections, work calculations, and similarity measurements in machine learning.