1D Convolutions for Time Series

Learning Objectives

By the end of this section, you will:

1. Understand the CNN-first architecture and why we extract local features before temporal modeling
2. Master 1D convolution mathematics for time series data
3. Apply convolution to multivariate sequences with multiple input channels
4. Identify local patterns that CNNs learn to detect in sensor data
5. Appreciate the CNN-LSTM synergy that makes our architecture powerful

Why This Matters: Our AMNL model uses a CNN as its first processing stage, before the BiLSTM. The CNN extracts local patterns from sensor readings—sudden spikes, gradual trends, noise artifacts—that the LSTM then integrates over time. Understanding this division of labor is key to understanding the architecture.

Why CNN Before LSTM?

A natural question: why not feed raw sensor data directly to the LSTM? The CNN-first approach offers several advantages.

Advantage 1: Local Feature Extraction

Sensor readings contain local patterns spanning a few timesteps:

• Spikes: Sudden temperature increases (2-5 cycles)
• Gradients: Rate of pressure change (3-7 cycles)
• Oscillations: Cyclic patterns in vibration (5-10 cycles)

CNNs excel at detecting these patterns through learned filters.

Advantage 2: Dimensionality Reduction

The CNN compresses 17 raw features into richer representations:

Where $T' \leq T$ (sequence may shorten with strided convolutions). The LSTM then processes fewer, more informative features.

Advantage 3: Translation Invariance

A spike at cycle 5 and a spike at cycle 25 should both be recognized as "spike patterns." Convolution provides this naturally—the same filter slides across all positions.

Architecture Overview

1Input: (batch, 30, 17)  ← Window of 30 timesteps, 17 sensors
2          ↓
3       CNN Block 1
4          ↓
5       CNN Block 2
6          ↓
7       CNN Block 3
8          ↓
9Output: (batch, 30, 64)  ← Same timesteps, 64 learned features

1D Convolution Operation

1D convolution slides a kernel across the temporal dimension, computing weighted sums at each position.

Mathematical Definition

For input $\mathbf{x} \in \mathbb{R}^T$ and kernel $\mathbf{w} \in \mathbb{R}^K$:

Where:

• $K$: Kernel size (receptive field)
• $w_k$: Learned kernel weights
• $b$: Bias term
• $t$: Output position

Interactive Visualization

Use the interactive visualization below to see how 1D convolution works step by step with real NASA C-MAPSS sensor data. Watch the kernel slide across the input, observe the calculations at each position, and see how the output is built up.

Output Dimension Formula

Where:

With padding P = 1 and kernel K = 3, we get $T_{\text{out}} = (30 + 2 - 3)/1 + 1 = 30$—the sequence length is preserved.

Convolution for Multivariate Series

Our sensor data has 17 channels. The convolution extends naturally to multiple input channels.

Multi-Channel Input

For input $\mathbf{X} \in \mathbb{R}^{T \times C_{\text{in}}}$, each output channel uses a separate 2D kernel:

Where:

• $j$: Output channel index (1 to $C_{\text{out}}$)
• $c$: Input channel index (1 to 17)
• $w_{c,k}^{(j)}$: Weight for input channel c, position k, output channel j

Dimension Transformation

For our first CNN layer:

Each of the 64 output channels is computed by a kernel of shape $(17, 3)$ that spans all 17 input channels and 3 timesteps.

Interactive Multi-Channel Visualization

The visualization below demonstrates how multi-channel convolution works with a stacked architecture (8 → 16 → 8 channels). Click on Conv1 or Conv2 to see the detailed step-by-step computation showing how each output channel sums contributions from all input channels.

What CNNs Learn

Through training, CNN kernels learn to detect patterns relevant for RUL prediction.

Examples of Learned Patterns

Hierarchical Feature Extraction

Multiple CNN layers build increasingly abstract features:

1. Layer 1: Detects simple patterns (edges, gradients) in raw sensor data
2. Layer 2: Combines simple patterns into compound features (e.g., "spike followed by decay")
3. Layer 3: Creates high-level representations (degradation signatures)

This hierarchical structure mirrors how degradation manifests: low-level sensor anomalies combine into recognizable degradation patterns.

CNN-LSTM Synergy

The CNN and LSTM have complementary strengths:

Division of Labor

In our architecture:

• CNN: "What patterns exist?" — Detects local features independent of when they occur
• LSTM: "How do patterns evolve?" — Models the temporal sequence of detected patterns
• Attention: "Which timesteps matter?" — Focuses on relevant moments for prediction

Design Principle: Let each component do what it does best. CNNs excel at pattern detection; LSTMs excel at sequence modeling. Combining them gives the best of both worlds.

Summary

In this section, we introduced 1D convolutions for time series:

1. CNN-first architecture: Extract local features before temporal modeling
2. 1D convolution: $y_t = \sum_k w_k \cdot x_{t+k} + b$
3. Multi-channel: Each output channel combines all input channels
4. Learned patterns: Edges, gradients, complex degradation signatures
5. CNN-LSTM synergy: Pattern detection + temporal dynamics

Looking Ahead: A single convolution layer has limited capacity. Our model uses three stacked CNN layers with increasing channel counts (17 → 64 → 128 → 64). The next section details this architecture and explains the channel progression.

With 1D convolution understood, we are ready to design the complete three-layer CNN architecture.