05 / Machine Learning / Anomaly Detection
Fraud Detection
A two-stage approach combining autoencoder representation/anomaly learning with a supervised Random Forest stage to identify fraud.
Overview
A fraud-detection experiment combining autoencoder representation/anomaly learning with a supervised Random Forest stage.
Problem
Fraud is a minority-class detection task. A useful evaluation needs to examine how many minority-class examples are detected, rather than relying on overall accuracy.
Architecture
An autoencoder-based learning stage is combined with a supervised Random Forest classifier.
- Input dataClassification inputs
- AutoencoderRepresentation / anomaly learning
- Random ForestSupervised classification
Implementation
- Use Python and an autoencoder for representation/anomaly learning.
- Combine that stage with supervised Random Forest classification using Scikit-learn.
- Examine minority-class recall as a relevant detection measure.
Challenges
Higher recall can come with more false positives. Precision, thresholds and the evaluation distribution are needed to understand that trade-off.
Technology
- Python
- Autoencoder
- Random Forest
- Scikit-learn
Outcome
- Minority-class recall
- ≈0.86
Reported minority-class recall is around 0.86. Precision, false-positive rate and operational suitability are not established by that metric alone.
Dataset details, split design, precision and threshold selection still need to be documented.
What I learned
For imbalanced problems, evaluation should reflect the minority-class behavior and the cost of errors. Recall is a useful measure, but cannot stand alone.