RUL Loss Component: Weighted MSE

Learning Objectives

By the end of this section, you will:

1. Understand why standard MSE is suboptimal for RUL
2. Design the weight function that emphasizes critical samples
3. Derive the weighted MSE formula
4. Analyze gradient behavior with sample weighting
5. Implement weighted MSE in PyTorch

Why This Matters: Not all RUL prediction errors are equal. An error of 10 cycles when true RUL is 100 is less critical than an error of 10 cycles when true RUL is 20. Weighted MSE ensures the model focuses on the samples that matter most—those approaching failure.

Why Weighted MSE?

Standard MSE treats all samples equally, but RUL prediction has asymmetric importance.

The Problem with Standard MSE

Standard MSE weights all prediction errors equally, regardless of the true RUL value.

Real-World Importance

Errors at low RUL have far greater practical consequences than errors at high RUL.

The Solution

Weight samples inversely to their RUL: low-RUL samples get higher weights.

The Weight Function

We use a linear decay weight function based on true RUL.

Linear Decay Weights

Where:

• $y_i$: True RUL for sample i
• $R_{\max} = 125$: Maximum RUL (cap)
• $w_i \in [1, 2]$: Weight range

Weight Distribution

1Weight vs. True RUL:
2
3Weight
4  2.0 ─┤●
5      │ ╲
6  1.8 ─┤  ╲
7      │   ╲
8  1.6 ─┤    ╲
9      │     ╲
10  1.4 ─┤      ╲
11      │       ╲
12  1.2 ─┤        ╲
13      │         ╲
14  1.0 ─┤          ╲────────────●
15      └──┬──┬──┬──┬──┬──┬──┬──┬
16         0  15 30 45 60 75 90 105 125
17                  True RUL
18
19Key points:
20  RUL = 0:   w = 2.0 (maximum weight)
21  RUL = 62:  w = 1.5 (midpoint)
22  RUL = 125: w = 1.0 (minimum weight)

Why Linear Decay?

Mathematical Formulation

The complete weighted MSE loss for RUL prediction.

Full Formula

With weight function:

Implementation

# BEFORE clamp: target with outliers
target = tensor([100.0, 50.0, 20.0, 150.0, 200.0])

# AFTER torch.clamp(target, max=125.0):
capped_target = tensor([100.0, 50.0, 20.0, 125.0, 125.0])
#                                          ↑       ↑
#                                       capped at 125

# With capped_target = [100.0, 50.0, 20.0, 125.0]
weights = 2.0 - capped_target / 125.0

# Step by step:
# 2.0 - 100/125 = 2.0 - 0.80 = 1.20
# 2.0 - 50/125  = 2.0 - 0.40 = 1.60
# 2.0 - 20/125  = 2.0 - 0.16 = 1.84
# 2.0 - 125/125 = 2.0 - 1.00 = 1.00

weights = tensor([1.20, 1.60, 1.84, 1.00])
# Critical samples (low RUL) → higher weight

# Predictions and targets:
pred   = tensor([95.0, 55.0, 30.0, 120.0])
target = tensor([100.0, 50.0, 20.0, 125.0])

# Squared errors:
squared_errors = (pred - target) ** 2
               = ([-5, 5, 10, -5]) ** 2
               = tensor([25.0, 25.0, 100.0, 25.0])

# Weights and squared errors:
weights = tensor([1.20, 1.60, 1.84, 1.00])
squared_errors = tensor([25.0, 25.0, 100.0, 25.0])

# Weighted errors:
weighted_errors = weights * squared_errors
                = tensor([30.0, 40.0, 184.0, 25.0])
#                              ↑
#                    Low RUL sample dominates!

# Weighted errors and weights:
weighted_errors = tensor([30.0, 40.0, 184.0, 25.0])
weights = tensor([1.20, 1.60, 1.84, 1.00])

# Normalized loss:
loss = weighted_errors.sum() / weights.sum()
     = 279.0 / 5.64
     = 49.5  # scalar

# Compare to simple mean: 279.0 / 4 = 69.75
# Weight normalization adjusts for weight magnitudes

1def weighted_mse_loss(
2    pred: torch.Tensor,
3    target: torch.Tensor,
4    r_max: float = 125.0
5) -> torch.Tensor:
6    """
7    Weighted MSE loss with linear decay weights.
8
9    Args:
10        pred: Predicted RUL (batch,)
11        target: True RUL (batch,)
12        r_max: Maximum RUL for weight computation
13
14    Returns:
15        Weighted MSE loss (scalar)
16    """
17    # Compute weights: w = 2 - y/R_max
18    capped_target = torch.clamp(target, max=r_max)
19    weights = 2.0 - capped_target / r_max
20
21    # Weighted squared errors
22    squared_errors = (pred - target) ** 2
23    weighted_errors = weights * squared_errors
24
25    # Normalize by sum of weights
26    loss = weighted_errors.sum() / weights.sum()
27
28    return loss

Numerical Example

Gradient Analysis

Understanding the gradient helps us see how weighting affects learning.

Gradient Derivation

The gradient of weighted MSE with respect to prediction:

Key observation:

• Gradient magnitude is proportional to weight $w_i$
• Low-RUL samples (high weight) produce larger gradients
• Model receives stronger learning signal from critical cases

Gradient Comparison

For the same prediction error (e.g., 10 cycles off):

Low-RUL samples contribute up to 2× larger gradients, ensuring the model prioritizes learning to predict accurately near failure.

Summary

In this section, we designed the weighted MSE loss for RUL:

1. Motivation: Low-RUL errors are more critical than high-RUL errors
2. Weight function: $w = 2 - y/R_{\max}$
3. Weight range: [1.0, 2.0] (maximum 2× emphasis)
4. Effect: Larger gradients for critical samples
5. Normalization: Divide by sum of weights

Looking Ahead: With the RUL loss component defined, the next section describes the health classification loss—the cross-entropy component that provides regularization for the RUL prediction task.

Having defined the RUL loss, we now examine the health classification component.