Fixed Equal Weighting (0.5 / 0.5)

The Simplest Multi-Task Loss

A small restaurant kitchen runs an unremarkable rule: every cook spends equal time on every dish. Vegetable prep, sauces, plating — same minutes per cook per task. The rule is suboptimal in any individual moment (some dishes need more attention than others on any given evening), but it is bulletproof. No coordination overhead. No misprioritisation. No specialist becoming a single point of failure.

Fixed equal weighting is the same idea applied to a multi-task loss. Two tasks — RUL regression and health classification — with one knob set once at 0.5 each, then never touched. No controller, no learnable scalars, no per-batch correction. Surprisingly, this rule is both the simplest MTL strategy AND the empirical winner of the AMNL paper's exhaustive ablation on FD002 + FD004 — beating the heuristic 0.75/0.25 split, any unbalanced choice, and (on chapter 23 §1 evidence) keeping pace with adaptive controllers when the gradient ratio is moderate.

The Equation In One Line

For two task losses $\mathcal{L}_{\text{rul}}$ and $\mathcal{L}_{\text{health}}$, the fixed-weight composition is

$\mathcal{L}(\theta) = \lambda_{\text{rul}} \, \mathcal{L}_{\text{rul}}(\theta) + \lambda_{\text{health}} \, \mathcal{L}_{\text{health}}(\theta), \quad \lambda_{\text{rul}} + \lambda_{\text{health}} = 1.$

Three properties to internalise:

• Linearity. The gradient is $\nabla\mathcal{L} = \lambda_{\text{rul}}\nabla\mathcal{L}_{\text{rul}} + \lambda_{\text{health}}\nabla\mathcal{L}_{\text{health}}$. A weighted sum of the per-task gradients — no interaction terms, no per-batch state.
• Statelessness. The two weights are PYTHON FLOATS, not learnable parameters. They never appear in the optimiser's parameter list and never receive a gradient. Compare with Uncertainty (chapter 24 §2) which makes them learnable.
• Sum-to-one is not enforced. The paper's FixedWeightLoss stores both weights independently; the user is responsible for the convex-combination convention. You CAN set 0.7/0.7 and it will still run (with an uninterpretable scale).

Why 0.5 / 0.5 — A Counter-Intuitive Finding

The natural prior is $\lambda_{\text{rul}} \approx 1$: RUL regression is the primary task, so why give half the gradient budget to a 3-class auxiliary classifier? The AMNL paper's exhaustive ablation answers: because health classification REGULARISES the shared backbone toward condition-invariant features, and that regularisation buys more accuracy than the lost RUL focus.

The four data rows trace a U-shape with the minimum at 0.5: too little health weight loses regularisation; too much loses RUL accuracy. The shape and minimum are robust across both multi-condition subsets — the 0.5/0.5 rule is a real empirical optimum on this benchmark, not a coincidence.

Interactive: λ_rul Sensitivity Sweep

Drag the slider to interpolate between the four AMNL ablation anchor points. The two curves are FD002 (blue) and FD004 (amber) RMSE. Both bottom out near $\lambda_{\text{rul}} = 0.5$ and rise steeply as the weight approaches 1.0 (single-task).

Try the slider. Push λ_rul above 0.85 and the log-scale RMSE shoots up sharply — that is the regularisation cliff. Auxiliary task weight matters more than intuition suggests, because the backbone is shared. Below ~0.45 RMSE rises again as the backbone is dragged toward a health-specialist regime.

AMNL-Fixed: Magnitude-Normalised 0.5 / 0.5

The paper's repo includes a second fixed-weight variant, AMNLFixedLoss (baselines.py:55). It also fixes the weights at 0.5/0.5 but FIRST normalises each per-task loss by its own magnitude:

$\mathcal{L}_{\text{AMNL-fix}} = 0.5 \cdot \frac{\mathcal{L}_{\text{rul}}}{\max(L_{\text{rul}},\ s_{\text{rul}})} + 0.5 \cdot \frac{\mathcal{L}_{\text{health}}}{\max(L_{\text{health}},\ s_{\text{health}})}$

with floor scales $s_{\text{rul}} = 1.0$ and $s_{\text{health}} = 0.1$ to prevent division by tiny early-training values. The intent: scale the two losses to comparable magnitudes BEFORE combining, so the equal-weight rule is meaningful. In practice the unnormalised FixedWeightLoss is what the paper reports as ‘Baseline’ and ‘AMNL fixed (with WMSE)’ in chapter 23 §1; the AMNL-fixed variant is included as an ablation but not the main result.

Python: FixedWeightLoss From Scratch

Reproduce the four ablation rows with one function. No PyTorch needed — just NumPy + a softmax CE helper. The for-loop at the bottom prints the L_total values for the four configurations at the same input losses.

      h    d    c
A   0.1  0.2  0.7    ← critical predicted (correct, target=2)
B   0.5  0.3  0.2    ← healthy predicted (target=1)
C   0.4  0.5  0.6    ← critical predicted (correct, target=2)
D   0.8  0.1  0.1    ← healthy predicted (correct, target=0)

L_rul    = 25.7500    L_health = 0.8999
config                    λ_rul    λ_h   L_total
-------------------------------------------------------
AMNL 0.5/0.5               0.50   0.50    13.3249
AMNL 0.6/0.4               0.60   0.40    15.8100
V7 baseline 0.75/0.25      0.75   0.25    19.5375
AMNL 0.9/0.1 (heavy RUL)   0.90   0.10    23.2650

1"""Fixed equal weighting — the simplest MTL loss.
2
3L = lam_rul * L_rul + (1 - lam_rul) * L_health.
4
5The two task losses are kept on the same scale by using compatible
6units (cycles² for MSE and bits for cross-entropy). At lam_rul = 0.5,
7the two losses contribute equally to the gradient, regardless of their
8absolute magnitudes — the OUTER axis is held fixed, no controller.
9"""
10
11import numpy as np
12
13
14def softmax_ce(logits, target):
15    """3-class cross-entropy on logits (B, 3) with int target (B,)."""
16    z = logits - logits.max(axis=-1, keepdims=True)
17    p = np.exp(z); p /= p.sum(axis=-1, keepdims=True)
18    return -np.log(p[np.arange(len(target)), target] + 1e-12).mean()
19
20
21def fixed_weight_loss(rul_loss, health_loss, lam_rul=0.5):
22    """L = lam_rul * L_rul + (1 - lam_rul) * L_health.
23
24    The whole MTL composition in one line. No state, no controller.
25    """
26    return lam_rul * rul_loss + (1.0 - lam_rul) * health_loss
27
28
29# ---------- Tiny mini-batch (4 samples) ----------
30y_rul     = np.array([20., 80.,  5., 100.])
31y_pred    = np.array([25., 75., 12.,  98.])
32hp_target = np.array([2, 1, 2, 0])
33hp_logits = np.array([[0.1, 0.2, 0.7],
34                      [0.5, 0.3, 0.2],
35                      [0.4, 0.5, 0.6],
36                      [0.8, 0.1, 0.1]])
37
38L_rul    = ((y_pred - y_rul) ** 2).mean()
39L_health = softmax_ce(hp_logits, hp_target)
40
41
42# ---------- Compare four fixed weightings ----------
43configs = [(0.50, "AMNL 0.5/0.5"),
44           (0.60, "AMNL 0.6/0.4"),
45           (0.75, "V7 baseline 0.75/0.25"),
46           (0.90, "AMNL 0.9/0.1 (heavy RUL)")]
47
48print(f"L_rul    = {L_rul:.4f}    L_health = {L_health:.4f}")
49print(f"{'config':<24} {'λ_rul':>6} {'λ_h':>6} {'L_total':>10}")
50print("-" * 55)
51for lam, name in configs:
52    L = fixed_weight_loss(L_rul, L_health, lam_rul=lam)
53    print(f"{name:<24} {lam:>6.2f} {1-lam:>6.2f} {L:>10.4f}")

PyTorch: The Paper's FixedWeightLoss

Verbatim from grace/core/baselines.py:34-49. 16 lines of class definition; 3 method implementations; no learnable parameters; no internal state. The simplest entry in the loss registry — and the baseline every other method in this chapter compares against.

L_total      = 13.3249
weights      = {'rul_weight': 0.5, 'health_weight': 0.5}
name         = Fixed(0.50/0.50)
dL/dL_rul    = 0.5000
dL/dL_health = 0.5000

1"""Paper's FixedWeightLoss class — verbatim.
2
3Source: paper_ieee_tii/grace/core/baselines.py:34-49.
4The simplest entry in the loss registry. Wraps the fixed weighted sum
5in the same BaseMTLLoss interface as GABA, GradNorm, Uncertainty, etc.,
6so it plugs into UnifiedTrainer without special-casing.
7"""
8
9import torch
10import torch.nn as nn
11from typing import Dict
12
13
14class BaseMTLLoss(nn.Module):
15    """Abstract base. forward() returns a scalar combined loss."""
16    def forward(self, rul_loss, health_loss, **kwargs): ...
17    def get_weights(self) -> Dict[str, float]: ...
18    def get_name(self) -> str: ...
19
20
21class FixedWeightLoss(BaseMTLLoss):
22    """Static task weighting."""
23
24    def __init__(self, rul_weight: float = 0.5,
25                 health_weight: float = 0.5) -> None:
26        super().__init__()
27        self.rul_weight    = rul_weight
28        self.health_weight = health_weight
29
30    def forward(self, rul_loss, health_loss, **kwargs):
31        return self.rul_weight * rul_loss + self.health_weight * health_loss
32
33    def get_weights(self):
34        return {"rul_weight":    self.rul_weight,
35                "health_weight": self.health_weight}
36
37    def get_name(self):
38        return f"Fixed({self.rul_weight:.2f}/{self.health_weight:.2f})"
39
40
41# ---------- Drop-in usage in the trainer ----------
42if __name__ == "__main__":
43    # Equivalent to get_loss("fixed_050") from the loss_registry.
44    loss = FixedWeightLoss(rul_weight=0.5, health_weight=0.5)
45
46    # Synthetic per-task losses
47    L_rul    = torch.tensor(25.75, requires_grad=True)
48    L_health = torch.tensor( 0.90, requires_grad=True)
49
50    L = loss(L_rul, L_health)
51    print(f"L_total      = {L.item():.4f}")
52    print(f"weights      = {loss.get_weights()}")
53    print(f"name         = {loss.get_name()}")
54    L.backward()
55    print(f"dL/dL_rul    = {L_rul.grad.item():.4f}")
56    print(f"dL/dL_health = {L_health.grad.item():.4f}")

Fixed Weights In Other Domains

The lesson: fixed weights are a valid first stop, but the right ratio is domain-specific. Run the AMNL ablation (4–5 fixed splits) on a small validation set before committing. If the U-shape minimum is at 0.5, you have the AMNL pattern; if it's elsewhere, the loss scales differ materially and you may benefit from magnitude-normalisation (AMNL-fixed) or learnable weighting (Uncertainty).

Pitfalls With Fixed Weights

Pitfall 1: weights that don't sum to 1

FixedWeightLoss does not enforce $\lambda_{\text{rul}} + \lambda_{\text{health}} = 1$. Setting (0.7, 0.7) scales the combined loss by 1.4 — same gradient direction, larger magnitude. Effectively the same as running with a 1.4× learning rate on a (0.5, 0.5) loss, which often blows up training. Always verify the sum.

Pitfall 2: assuming 0.5 / 0.5 transfers

The 0.5 / 0.5 result is C-MAPSS-specific. On datasets where the per-task losses have different magnitudes (e.g. raw MSE vs cross-entropy with very different ranges), 0.5/0.5 will be dominated by the larger-magnitude loss. Either rescale the losses to comparable magnitudes (AMNL-fixed pattern) or learn the weights (Uncertainty — chapter 24 §2).

Pitfall 3: forgetting that 0.9/0.1 is nearly single-task

At $\lambda_{\text{rul}} = 0.9$ the health task contributes only 10% of the gradient signal — the backbone has effectively no auxiliary supervision. RMSE on FD002 climbs from 6.33 to 8.59. Single-task (λ_rul=1.0) is even worse: 26.92. The continuum from MTL to single-task has a sharp cliff.

Pitfall 4: comparing fixed vs adaptive without controlling for inner loss

Fixed 0.5/0.5 with weighted MSE = AMNL. Fixed 0.5/0.5 with standard MSE = Baseline. GABA + standard MSE = adaptive version of Baseline. GABA + weighted MSE = GRACE. The four cells of the chapter 21 §1 grid. Confounding fixed-vs-adaptive with which inner loss is used produces apples-to-oranges comparisons.

Pitfall 5: starting fixed and never re-checking

A fixed 0.5/0.5 model trained successfully on one dataset version may break on a refresh of the data. The optimal split depends on the relative scales of the per-task losses, which change with data preprocessing. Re-run the 4-point sweep when your training data changes meaningfully.

Takeaway

• Fixed weighting is the simplest MTL loss: $\mathcal{L} = \lambda_{\text{rul}}\mathcal{L}_{\text{rul}} + (1-\lambda_{\text{rul}})\mathcal{L}_{\text{health}}$. No state, no controller, no learnable parameters.
• On C-MAPSS multi-condition the AMNL ablation finds $\lambda_{\text{rul}} = 0.5$ the optimal split: FD002 RMSE 6.33, FD004 RMSE 7.30, beating the intuitive 0.75/0.25 by 0.91 and 0.84 cycles.
• The U-shape RMSE-vs-λ_rul curve has a regularisation cliff near λ_rul → 1 (RMSE jumps to 26.92 at single-task) — the auxiliary task is doing real work.
• The production class is FixedWeightLoss at grace/core/baselines.py:34-49. Inherits BaseMTLLoss for uniform interface with GABA/Uncertainty/GradNorm/DWA. Weights stored as Python floats — explicitly NOT learnable.
• Use fixed weights as a baseline. Run a 4–5-point ablation on a validation set; if the U-minimum is at 0.5, you have the AMNL pattern. If not, escalate to magnitude normalisation (AMNL-fixed), learnable weights (Uncertainty), or gradient-magnitude adaptation (GABA / GRACE).