Why Bidirectional Beats Unidirectional

Live Closed Captions vs Subtitles

A live closed-captioner working off a microphone has no idea what word is coming next - she has to commit to her transcription word-by-word, with only past context. A film subtitler working from a finished movie can rewind, fast-forward, look at what comes both before AND after a confusing line, then write a subtitle that reads naturally in context. Same task; very different output quality, because one of them has access to the future.

That is exactly what a bidirectional LSTM gives a model. A unidirectional LSTM at cycle $t$ only sees $x_1, \ldots, x_t$. A bidirectional LSTM sees the entire window $x_1, \ldots, x_T$ from BOTH directions and lets the model decide what each cycle means in light of everything that surrounds it.

From CNN Features to Temporal Modelling

The CNN frontend (Chapter 8) produced a (B, 30, 64) tensor - 30 timesteps, 64 learned local-pattern channels. The BiLSTM's job is to integrate those LOCAL features into a sequence-aware representation that carries TEMPORAL structure: are the local spikes accelerating? Are oscillations growing in amplitude? Is a slow upward drift consistent across the window?

What a Unidirectional LSTM Cannot See

A unidirectional LSTM at cycle $t$ has only seen $x_1, \ldots, x_t$. For a spike at cycle 23 the model cannot tell whether the spike is the START of a failure cascade or just a transient anomaly that resolves at cycle 25 - because cycle 24 and cycle 25 do not exist yet from the unidirectional point of view.

Bidirectional LSTM solves this by running a SECOND LSTM right- to-left over the same input, then concatenating the two hidden states. At cycle 23, the model now also knows what cycles 24-30 look like - decisive context for distinguishing transient from terminal events.

The BiLSTM Mechanism

Two independent LSTM passes on the same sequence, concatenated:

$\overrightarrow{h}_t = \text{LSTM}_{\rightarrow}(x_t,\, \overrightarrow{h}_{t-1}), \quad \overleftarrow{h}_t = \text{LSTM}_{\leftarrow}(x_t,\, \overleftarrow{h}_{t+1}).$

The output at each timestep is the concatenation:

$h_t = [\overrightarrow{h}_t \,;\, \overleftarrow{h}_t] \in \mathbb{R}^{2H}.$

Two separate parameter sets - one for each direction - learn different things. The forward LSTM tends to learn “what came before” features; the backward LSTM learns “what comes after”. The concat lets the downstream layer use both.

Interactive: Watch the Two Passes Combine

The animation below runs a 6-cycle BiLSTM step by step. Toggle between “forward only”, “backward only”, and “both” to see how the concatenated output is only ready when BOTH directions have processed the relevant timestep.

Python: A Bidirectional Pass From Scratch

Twenty lines that take the scalar LSTM cell from §3.3 and run it twice: forward, then backward, on the SAME input sequence. The concatenation glues the two hidden-state lists into one (T, 2H) output.

1import numpy as np
2
3# A scalar bidirectional pass — same machinery as nn.LSTM(bidirectional=True)
4# but written so every line is visible.
5
6def lstm_step(x, h_prev, c_prev, params):
7    """One scalar LSTM step. Returns (h_new, c_new)."""
8    sig = lambda z: 1.0 / (1.0 + np.exp(-z))
9    W_ix, W_ih, b_i = params["i"]
10    W_fx, W_fh, b_f = params["f"]
11    W_gx, W_gh, b_g = params["g"]
12    W_ox, W_oh, b_o = params["o"]
13    i = sig (W_ix * x + W_ih * h_prev + b_i)
14    f = sig (W_fx * x + W_fh * h_prev + b_f)
15    g = np.tanh(W_gx * x + W_gh * h_prev + b_g)
16    o = sig (W_ox * x + W_oh * h_prev + b_o)
17    c = f * c_prev + i * g
18    h = o * np.tanh(c)
19    return h, c
20
21
22def bidirectional_lstm(seq, params_fwd, params_bwd):
23    """seq: list of scalars (length T). Returns (T, 2) — concat[fwd, bwd]."""
24    T = len(seq)
25    h, c = 0.0, 0.0
26    h_fwd = []
27    for t in range(T):                              # left → right
28        h, c = lstm_step(seq[t], h, c, params_fwd)
29        h_fwd.append(h)
30
31    h, c = 0.0, 0.0
32    h_bwd = [None] * T
33    for t in range(T - 1, -1, -1):                  # right → left
34        h, c = lstm_step(seq[t], h, c, params_bwd)
35        h_bwd[t] = h
36
37    return np.stack([np.stack([f, b]) for f, b in zip(h_fwd, h_bwd)])
38
39
40# ----- Run on a 6-step input -----
41seq = [0.0, 0.0, 1.0, 1.0, 0.0, 0.0]
42shared = {
43    "i": (0.5, 0.0, 0.0),
44    "f": (0.0, 0.0, 1.0),       # forget bias = 1, like §3.3
45    "g": (0.8, 0.0, 0.0),
46    "o": (1.0, 0.0, 0.0),
47}
48out = bidirectional_lstm(seq, shared, shared)
49
50print("seq           :", seq)
51print("h_fwd         :", out[:, 0].round(3).tolist())
52print("h_bwd         :", out[:, 1].round(3).tolist())
53print("concat shape  :", out.shape)             # (6, 2)

PyTorch: nn.LSTM(bidirectional=True)

1import torch
2import torch.nn as nn
3
4# ----- Production-grade bidirectional LSTM -----
5torch.manual_seed(0)
6B, T, F = 2, 30, 64                 # Comes from §8.4: CNN frontend output
7
8bilstm = nn.LSTM(
9    input_size=64,
10    hidden_size=256,
11    num_layers=2,
12    bidirectional=True,             # ← the magic word
13    batch_first=True,
14    dropout=0.3,                    # only between stacked layers
15)
16
17x = torch.randn(B, T, F)
18out, (h_n, c_n) = bilstm(x)
19
20print("input shape  :", tuple(x.shape))
21print("output shape :", tuple(out.shape))           # (2, 30, 512) — 2 * hidden_size
22print("h_n shape    :", tuple(h_n.shape))           # (4, 2, 256)  — 2 layers * 2 dirs
23print("# params     :", sum(p.numel() for p in bilstm.parameters()))
24# # params : 2,140,160

Bidirectional vs Streaming in Other Domains

Three Bidirectional Pitfalls

The point. Two LSTM passes on the same input, concatenated. Doubles the hidden dimension; doubles the parameters; gives the model both past and future context at every cycle. Required for windowed RUL; unsuitable for streaming.

Takeaway

• Bidirectional = forward LSTM + backward LSTM, concatenated.
• Output dim = 2 × hidden_size. 256 × 2 = 512 for our backbone.
• Two independent parameter sets. They learn different things during training.
• Cannot stream. Needs the full window.
• Used by every model in this book. AMNL / GABA / GRACE all use the same BiLSTM block; only the loss changes.