11Personal research

Manufacturing Failure Prediction

Author, public repo

I built models that predict which production batches will fail from 590 sensor signals (UCI SECOM, 6.6% failure rate). Random cross-validation leaks future data here, so every result I stand behind uses walk-forward splits. I tested 16 explicit hypotheses with LightGBM and, on a simulated process chain, traced late defects back to the stage that caused them.

590: sensor signals per batch
0.118 vs 0.052: PR-AUC: random CV vs walk-forward
16: hypotheses tested

Python / LightGBM / scikit-learn

Source on GitHub

Problem

The UCI SECOM dataset has 1,567 production runs, 590 sensor signals and a 6.64% failure rate over 86 time-ordered days. With so few failures and data ordered in time, it is easy to report a model that scores well in testing and would not hold up on future batches.

Approach

I compared random cross-validation with walk-forward splits and showed that random CV leaks future information, so every reported result uses walk-forward expanding splits.
Preprocessing drops constant signals and imputes inside each fold, so no statistics cross the split.
I tested 16 explicit hypotheses with LightGBM, including threshold tuning, missing-value indicators, SMOTE versus class weights, PSI drift monitoring, retraining windows and time-cycle features.
On a simulated deposition, etch and inspection tool chain, a root-cause step attributes late defects to a stage, and deposition gets the top vote.

Architecture

Sensor data
Fold-safe preprocessing
Walk-forward splits
LightGBM
Drift & thresholds
Root-cause stage

Outcome

Random CV reports a PR-AUC of 0.118; walk-forward gives 0.052, less than half. I report the walk-forward number. The repo ships a dashboard, JSON results and a reproducibility manifest.

What I learnedOn time-ordered manufacturing data, choose the validation split before the model. Here the split alone changed the headline metric by more than 2×.

Building something like this?

Tell me about it