Skip to main content

Public code · Repository (opens in new tab)

Real-time credit-card fraud detection

A LightGBM fraud model, evaluated on transactions it never saw in training, with 1.4 ms p95 in-process model inference.

Role
Sole engineer
Period
Jun 2026 to Nov 2026
Stack
Python, LightGBM, Optuna, SHAP, scikit-learn, FastAPI, Redis, PostgreSQL, Docker

Problem#Link to this section

Card fraud is rare and expensive. In this dataset it's 0.17% of payments, so a model that calls everything legitimate is 99.8% accurate and useless. The job is to catch as much fraud as possible, with few enough false alarms that someone will still read them, quickly enough to decide before the payment clears.

Constraints#Link to this section

  • Time. A score has to come back within the 10 ms the project set for itself, so a slower configuration loses even if it's more accurate.
  • No peeking at the future. Fraud patterns change over time. Training on a random shuffle would let the model learn from transactions that happen after the ones it's tested on.
  • Few positives. 284,807 transactions hold only 492 frauds, so every metric on the test set rests on a few dozen cases.

Approach#Link to this section

How one transaction is scoredA transaction enters with its time, 28 PCA components and amount. Window features are computed from the previous ten transactions. A LightGBM model trained with focal loss produces a raw score, which becomes a probability through a sigmoid with the training init score, and is compared with the tuned threshold to flag or allow. SHAP contributions explain the score, and the service runs behind FastAPI with Redis and PostgreSQL in Docker.TransactionTime, V1 to V28 (PCA), AmountWindow featuresbuilt from the previous 10 transactionsLightGBM, focal loss79 inputs, tuned with OptunaProbabilitysigmoid(raw score + init score)Decisionflag or allow at the tuned thresholdExplanation: SHAP contributions from the same model.Service: FastAPI, Redis and PostgreSQL in Docker.Latency: 1.4 ms p95 for the model call alone.
One transaction's path through the model. The window features are why a single row can't be scored on its own.

I split the data by time, 70/15/15: training on the earliest transactions, tuning and calibrating on the next block, and holding out the last 42,722 for the final test. All preprocessing was fitted on the training block only.

Beyond the 28 PCA components and the amount, the model sees 50 engineered features. Most describe how a transaction compares with the card's recent history: rolling means, spreads and z-scores of the amount over the previous 3, 5 and 10 transactions, the time since the last one, and cumulative spending.

The model is LightGBM trained with focal loss, which concentrates learning on hard, rare examples. SMOTE plus undersampling rebalanced the training block. Optuna searched the hyperparameters and threw out any configuration too slow to serve. Isotonic calibration on the validation block made the probabilities usable as probabilities.

Evaluation#Link to this section

Every number below comes from the time-ordered holdout, which the model never saw during training or tuning. The decision threshold is the one stored in the repository with the model.

Measured results with the data each was measured on and its caveat
MetricValueMeasured onCaveat
F10.80Time-ordered holdout, 42,722 transactionsBootstrap 95% CI 0.71 to 0.88. The project's own target was 0.85, and this misses it.
Precision0.87Time-ordered holdout, at the tuned thresholdAbout 13 in every 100 flags are false alarms at this threshold.
Recall0.75Time-ordered holdout, at the tuned thresholdBootstrap 95% CI 0.63 to 0.86. The 0.85 recall gate failed.
PR-AUC0.77Time-ordered holdout, after isotonic calibrationThe stricter score for rare fraud. The pre-calibration value isn't recorded in the repo.
ROC-AUC0.98Time-ordered holdoutLooks high on data that is 0.17% fraud; read PR-AUC first.
Model latency, p951.4 msOne in-process predict call per row, on precomputed featuresNot API latency. Feature engineering and the network add to it.

Bootstrap resampling of the holdout gives the intervals in the caveats. They're wide because the holdout holds few frauds, and they're the honest reading of the result.

Results#Link to this section

The model catches three in four frauds, and 87 of every 100 flags it raises are real fraud. It missed both targets the project set itself: F1 0.85 and recall 0.85. Recall's upper interval just reaches 0.86, so the gate isn't hopeless, but on this evidence it isn't met.

Latency is comfortably inside budget. The model call takes 1.4 ms at the 95th percentile, measured on a single row with its features already computed.

Security#Link to this section

The scoring API checks an API key on every route except health and docs, and rate-limits requests. It loads pickled model files, which run code when opened; that's acceptable only because the files come from this repository's own build. The live demo on this site loads the LightGBM text model and plain JSON instead, so it never unpickles anything.

Limits#Link to this section

  • The data is a 2013 European card dataset with PCA-anonymised features. It says nothing about Kenyan card traffic.
  • These numbers come from the repository's own evaluation script. A rerun from a clean clone, with only tracked files, is still to do.
  • Tuning reached F1 0.96 on its validation objective, far above the 0.80 on the holdout. The validation setup flattered the model.
  • The service is shaped for production but has never carried real traffic.

What I'd do next#Link to this section

  • Rerun the evaluation from a clean clone and replace these numbers with that run.
  • Measure end-to-end API latency under load, not just the model call.
  • Move the tuning objective onto a time-ordered validation block, so tuning stops overestimating the model.
  • Pick the threshold from an explicit cost of a missed fraud against a false alarm, rather than from a recall target.

Evidence