Basic Derivative Rules: Power, Sum, Constant

Learning Objectives

By the end of this section, you will be able to:

1. Apply the Power Rule to differentiate any function of the form $x^n$
2. Use the Constant Rule to recognize that derivatives of constants are zero
3. Apply the Constant Multiple Rule to pull constants out of derivatives
4. Combine the Sum and Difference Rules to differentiate polynomial functions
5. Extend the Power Rule to negative and fractional exponents
6. Connect these rules to gradient computation in machine learning

The Big Picture: From Definition to Efficiency

"The derivative rules are shortcuts — they replace tedious limit calculations with simple algebraic operations."

In the previous sections, we learned that the derivative is defined as a limit: $f'(x) = \lim_{h \to 0} \frac{f(x+h) - f(x)}{h}$. While this definition is fundamental, computing derivatives using limits every time would be incredibly tedious. Imagine having to expand $(x+h)^{10}$ just to find the derivative of $x^{10}$!

The derivative rules we learn in this section are powerful shortcuts derived from the limit definition. Once proven, they allow us to differentiate most functions instantly, without ever writing a limit.

The rules we'll learn:

Historical Context: Newton and Leibniz

Both Isaac Newton (1643–1727) and Gottfried Wilhelm Leibniz (1646–1716) independently discovered calculus in the late 17th century. They both recognized that certain patterns emerged when differentiating polynomial functions:

• The derivative of $x^2$ is $2x$
• The derivative of $x^3$ is $3x^2$
• The derivative of $x^4$ is $4x^3$

The pattern was clear: the exponent comes down as a coefficient, and the exponent decreases by one. This became the Power Rule, one of the most frequently used rules in all of calculus.

The Constant Rule

The simplest derivative rule states that the derivative of any constant is zero:

Why Does This Make Sense?

The derivative measures the rate of change. A constant, by definition, doesn't change. If $f(x) = 5$, then no matter what $x$ is, the output is always 5. The function is perfectly flat — its slope is zero everywhere.

Proof Using the Limit Definition

The Power Rule

The Power Rule is perhaps the most frequently used differentiation formula. It tells us how to differentiate any power of $x$:

In words: bring down the exponent as a coefficient, then reduce the exponent by 1.

Deriving the Power Rule

Let's prove the Power Rule for positive integers using the limit definition and the Binomial Theorem:

All terms with $h$ vanish as $h \to 0$, leaving only $nx^{n-1}$.

Interactive Power Rule Explorer

Use the visualizer below to explore how the Power Rule works. Adjust the exponent $n$ and observe how both the function and its derivative change:

The Constant Multiple Rule

Constants can be "pulled out" of derivatives:

Proof

Example: Find the derivative of $f(x) = 5x^3$.

Sum and Difference Rules

The derivative of a sum is the sum of the derivatives:

Proof of Sum Rule

Interactive Sum Rule Demo

Explore how the derivative of a sum equals the sum of derivatives:

Combining the Rules: Polynomial Differentiation

With the Power Rule, Constant Multiple Rule, and Sum Rule, we can differentiate any polynomial:

Example 1: Find $\frac{d}{dx}(3x^4 - 5x^2 + 7x - 2)$

Example 2: Find $\frac{d}{dx}(x^6 + 2x^5 - x^3 + 4x - 9)$

Negative and Fractional Exponents

The Power Rule works for all real exponents, not just positive integers. This greatly extends its usefulness.

Negative Exponents

Recall that $x^{-n} = \frac{1}{x^n}$. The Power Rule still applies:

Example: Find $\frac{d}{dx}\left(\frac{3}{x^2}\right)$

Fractional Exponents (Roots)

Roots can be written as fractional exponents: $\sqrt{x} = x^{1/2}$, $\sqrt[3]{x} = x^{1/3}$, etc.

Example: Find $\frac{d}{dx}(\sqrt{x})$

Real-World Applications

Physics: Motion

If position is given by a polynomial function of time, velocity and acceleration are found by differentiation:

Economics: Marginal Analysis

In economics, the derivative represents "marginal" quantities — the rate of change of one quantity with respect to another:

Biology: Population Growth

Population models often involve derivatives to understand growth rates:

Machine Learning Connection

The derivative rules are the foundation of gradient-based optimization, which powers virtually all modern machine learning.

Polynomial Regression

In polynomial regression, we fit a model of the form: $y = w_0 + w_1 x + w_2 x^2 + \ldots + w_n x^n$

To minimize the loss function, we need derivatives with respect to each weight $w_i$. The Power Rule tells us exactly how polynomial features contribute to the gradient.

Gradient Descent

The update rule for gradient descent is: $w_{new} = w_{old} - \alpha \cdot \frac{\partial L}{\partial w}$

Computing $\frac{\partial L}{\partial w}$ requires the derivative rules. For polynomial models:

• Power Rule: Differentiates each polynomial feature
• Sum Rule: Combines gradients from multiple terms
• Constant Rule: Handles bias terms (their gradient is simpler)

Python Implementation

Computing Polynomial Derivatives

Let's implement the derivative rules in Python:

1import numpy as np
2
3def power_rule(coefficient, exponent, x):
4    """
5    Apply the power rule to find derivative of c * x^n at point x.
6
7    Power Rule: d/dx(c * x^n) = c * n * x^(n-1)
8
9    Args:
10        coefficient: The constant multiplier c
11        exponent: The power n
12        x: The point at which to evaluate
13
14    Returns:
15        The derivative value at x
16    """
17    if exponent == 0:
18        return 0  # Derivative of constant is 0
19
20    new_coefficient = coefficient * exponent
21    new_exponent = exponent - 1
22    return new_coefficient * (x ** new_exponent)
23
24# Examples
25print("Power Rule Examples:")
26print(f"d/dx(x^2) at x=3: {power_rule(1, 2, 3)}")        # 2*3 = 6
27print(f"d/dx(5x^3) at x=2: {power_rule(5, 3, 2)}")      # 15*4 = 60
28print(f"d/dx(x^(-1)) at x=2: {power_rule(1, -1, 2)}")   # -1/4 = -0.25
29
30def polynomial_derivative(coefficients, x):
31    """
32    Compute derivative of a polynomial at point x.
33
34    coefficients[i] is the coefficient of x^i
35    Uses Sum Rule: d/dx(f + g) = f' + g'
36
37    Example: [3, 2, 1] represents 3 + 2x + x^2
38    """
39    derivative = 0
40    for power, coef in enumerate(coefficients):
41        derivative += power_rule(coef, power, x)
42    return derivative
43
44# Example: f(x) = 3 + 2x + x^2, f'(x) = 2 + 2x
45coeffs = [3, 2, 1]  # 3 + 2x + x^2
46print(f"\nPolynomial 3 + 2x + x^2:")
47print(f"  f'(1) = {polynomial_derivative(coeffs, 1)}")  # 2 + 2 = 4
48print(f"  f'(2) = {polynomial_derivative(coeffs, 2)}")  # 2 + 4 = 6

Application to Machine Learning

Here's how these rules appear in gradient descent for polynomial regression:

1import numpy as np
2
3# In machine learning, we often need gradients of polynomial features
4
5def compute_polynomial_features_gradient(x, weights, degree):
6    """
7    Compute gradient of polynomial regression loss w.r.t. weights.
8
9    Model: y_pred = w_0 + w_1*x + w_2*x^2 + ... + w_d*x^d
10    Loss: L = (y_true - y_pred)^2
11
12    For SGD, we need dL/dw_i for each weight.
13
14    Using chain rule and power rule:
15    dL/dw_i = -2 * (y_true - y_pred) * x^i
16    """
17    # Create polynomial features: [1, x, x^2, ..., x^d]
18    features = np.array([x ** i for i in range(degree + 1)])
19
20    # Prediction
21    y_pred = np.dot(weights, features)
22
23    return features, y_pred
24
25# Simple gradient descent step
26def gradient_step(x, y_true, weights, learning_rate, degree):
27    """
28    One step of gradient descent for polynomial regression.
29
30    The derivative rules (power, sum, constant) appear here:
31    - Power rule: d/dx(x^n) = n*x^(n-1) in feature computation
32    - Sum rule: total gradient is sum of individual gradients
33    - Constant rule: bias term gradient is just the error signal
34    """
35    features, y_pred = compute_polynomial_features_gradient(x, weights, degree)
36    error = y_true - y_pred
37
38    # Gradient for each weight
39    gradient = -2 * error * features
40
41    # Update weights
42    new_weights = weights - learning_rate * gradient
43
44    return new_weights, error ** 2
45
46# Example: Fit y = x^2 with polynomial regression
47np.random.seed(42)
48weights = np.random.randn(3) * 0.1  # degree 2: [w_0, w_1, w_2]
49learning_rate = 0.01
50
51print("Training polynomial regression:")
52for step in range(5):
53    x = np.random.uniform(-2, 2)
54    y_true = x ** 2  # True function
55    weights, loss = gradient_step(x, y_true, weights, learning_rate, 2)
56    print(f"Step {step+1}: weights = {weights.round(3)}, loss = {loss:.4f}")
57
58print(f"\nFinal weights (should approach [0, 0, 1]): {weights.round(3)}")

Common Mistakes to Avoid

Test Your Understanding

Summary

The derivative rules provide powerful shortcuts for computing derivatives without using the limit definition every time.

The Core Rules

Key Takeaways

1. The Power Rule is the workhorse: $\frac{d}{dx}(x^n) = nx^{n-1}$ for any real $n$
2. Constants have zero derivative because they don't change
3. The Sum Rule allows term-by-term differentiation of polynomials
4. The Power Rule works for negative and fractional exponents too
5. These rules are building blocks for the Product, Quotient, and Chain Rules
6. Machine learning uses these rules constantly in gradient computation

Coming Next: In the next section, we'll learn the Product Rule — how to differentiate products of functions, which is essential when our functions can't be written as simple polynomials.