RUL Regression Head

NumPy is Python's numerical computing library. It provides ndarray (N-dimensional arrays) and a fast, vectorised math runtime in C. We need it here for the matrix multiply (@), element-wise ReLU (np.maximum), Kaiming init (np.random.randn, np.sqrt) and bias zeros (np.zeros). The 'np' alias is the universal convention.

Pure-NumPy reference implementation of the RUL regression head. Maps the 32-D shared feature vector through a four-layer MLP (32 → 64 → 32 → 16 → 1) with ReLU between every hidden layer and a final ReLU that clamps the output to [0, ∞). The non-negativity guarantee is the whole point of this function — RUL is 'cycles to failure' and cannot be negative.

States the input/output contract: takes a batch of 32-D shared features, returns one non-negative scalar per row.

Aliases the running activation tensor h to the input. This is the standard MLP idiom — h gets overwritten layer by layer in the loop below. No copy is made; h and shared point to the same ndarray until the first reassignment.

Iterates the four layers. zip(weights, biases) pairs each weight matrix with its bias vector; enumerate adds the index i so we can detect the last layer (i == 3 for a 4-layer net) and skip the ReLU there.

Affine layer: matrix-multiply the activations by the weight matrix, then add the bias. This is one fully-connected layer's forward pass. Broadcasting handles the bias automatically — b has shape (out,) and is added to every row of h @ W which has shape (B, out).

Skip the per-layer ReLU on the FINAL layer. For a 4-layer net (len(weights) = 4), this is True for i ∈ {0, 1, 2} and False for i = 3. The final ReLU is applied separately on line 11 — first to enforce non-negativity, second so we can squeeze the trailing size-1 dim cleanly.

Element-wise ReLU: each element becomes max(value, 0). This is THE non-linearity that makes the MLP an MLP — without it, four stacked Linears collapse mathematically into a single Linear.

After the loop, h has shape (B, 1) — the linear output of the final layer (i=3) WITHOUT a ReLU. We squeeze the trailing size-1 dim to get (B,), then apply the final ReLU. This is the line that guarantees the head's contract: one non-negative float per engine.

Seed NumPy's global random number generator for reproducibility. Every call to np.random.* (randn, choice, ...) below will produce the same values across runs. Critical for the smoke test to print stable numbers.

Four layer shapes for the 32 → 64 → 32 → 16 → 1 funnel. Each tuple is (in_features, out_features). Note the expand-then-contract pattern: bump up to 64 first, then squeeze down through 32 → 16 → 1.

List comprehension that builds 4 weight matrices using Kaiming (He) initialisation — the standard init for ReLU networks. Without proper init, deep ReLU nets either blow up (variance grows) or die (variance shrinks to 0).

Build 4 bias vectors, all initialised to zero. Standard practice for ReLU networks — non-zero biases would risk pushing activations into a saturated regime before training begins.

Build a fake batch of 2 shared feature vectors (each 32-D) for the smoke test. In production this comes from the shared backbone in §11.1; here we synthesise random N(0,1) values just to verify shapes and the non-negativity invariant.

Run the full forward pass: 32-D input → 64 → 32 → 16 → 1 → squeeze → final ReLU. End-to-end shape transformation: (2, 32) → (2,).

Confirms the input shape. Two engines, 32-D feature vectors each.

Confirms the output shape. Note (2,) NOT (2, 1) — the .squeeze(-1) on line 11 removed the trailing dim. This shape matches what MSE loss expects against a (B,) RUL target.

Print the actual scalar predictions. Both ≥ 0 by construction (final ReLU). For this seed both happen to be strictly positive, so the ReLU is not active — but the guarantee always holds.

Count total parameters in the head. Confirms the 4,737 figure from the table above.

PyTorch's top-level package. Provides torch.Tensor (the GPU-aware n-dim array), autograd, the random-number generator (torch.manual_seed, torch.randn), and the dtype/device system. Required even when the heavy lifting lives in torch.nn.

PyTorch's neural network sub-package. Provides nn.Module (the base class for all learnable models), nn.Linear (fully-connected layer), nn.ReLU (the non-linearity), and nn.Sequential (the container we use to wire layers in order). The 'nn' alias is convention.

Production wrapper for the regression head. Subclassing nn.Module makes RULHead recognisable to the optimizer, serialisable via state_dict, and movable to GPU with .to('cuda'). Same logic as the NumPy version above, but with proper parameter management.

States the input/output contract for callers and docs.

Constructor. Builds the layer stack and registers all parameters. Called once when you do head = RULHead(in_dim=32).

Call the nn.Module base-class constructor. This sets up internal bookkeeping dicts (_parameters, _modules, _buffers) that nn.Module needs before we can assign any layers to self. Forgetting this line raises 'cannot assign module before Module.__init__() call'.

Wrap the entire MLP in a single nn.Sequential container. Sequential stores child modules in order and runs them sequentially in its forward(): output of module k is input to module k+1. Equivalent to writing forward() manually with four lines, but cleaner and auto-registers all sub-modules' parameters.

Layer 1: project 32 → 64 with a fully-connected layer, then apply ReLU.

Layer 2: contract 64 → 32. The descent down the funnel begins. Same pattern as line 9 with new shapes.

Layer 3: 32 → 16. Halving again — funneling toward the scalar output.

Final layer: collapse 16 → 1, then ReLU to enforce non-negativity. THIS is the line that turns a generic MLP into a 'cycles to failure' regressor — the ReLU is a structural prior on the physics (RUL ≥ 0).

Close the nn.Sequential. self.net is now ready and all 4,737 parameters are registered.

The forward pass. nn.Module.__call__ dispatches to this method when you call head(shared). Never call .forward() directly — head(x) is preferred because it runs the registered hooks.

Run the input through the Sequential stack — this internally calls each child in order: Linear → ReLU → Linear → ReLU → Linear → ReLU → Linear → ReLU. The result has shape (B, 1); we squeeze the trailing size-1 dim to get (B,).

Seed PyTorch's CPU random number generator for reproducibility. Affects torch.randn, default-init weights inside nn.Linear, dropout masks, etc. The CUDA RNG is separate (torch.cuda.manual_seed_all) but not needed here since the demo runs on CPU.

Instantiate the head. Triggers __init__: builds nn.Sequential, allocates all 4 Linear layers' weight + bias tensors, registers them as parameters. After this line, head.parameters() yields 4,737 trainable scalars.

Build a fake batch of 2 shared feature vectors. Same role as the NumPy version above — verify shapes and the non-negativity invariant without needing the real backbone.

Forward pass. head(shared) is sugar for head.__call__(shared) which runs hooks then forward(). Returns a (2,) tensor of non-negative RUL predictions.

Print the input shape. tuple(...) converts torch.Size into a plain Python tuple for cleaner printing.

Confirms the output is (2,) — squeezed, ready for MSE against a (B,) target.

The crucial invariant check. Builds a boolean tensor with element-wise (rul >= 0), reduces with .all() to a single 0-D tensor, then .item() pulls the Python bool out.

Count every trainable scalar in the head. Confirms 4,737 parameters — tiny compared with the shared backbone (typically tens of thousands).