Back To Projects

Comparing Machine Learning and Soft-Voting Ensemble Models for Credit Card Fraud Detection in Imbalanced Transaction Data

This study evaluates multiple machine-learning models for detecting highly imbalanced credit card fraud, finding that the configured XGBoost model achieved the best overall performance with an AP of 0.9084 and F1 score of 0.8534. However, the results are specific to the synthetic dataset and experimental setup, with several limitations preventing broader conclusions about XGBoost’s superiority.


Credit card fraud detection is a highly imbalanced classification problem in which headline accuracy can be misleading. In the held-out Kaggle test file used in this study, only 2,145 of 555,719 transactions (0.386%) were fraudulent; a classifier that labeled every transaction legitimate would therefore achieve approximately 99.61% accuracy while detecting no fraud. The analysis evaluated four configured machine-learning pipelines - Decision Tree, Random Forest, XGBoost, and a Multilayer Perceptron (MLP) - together with equal and validation-AP-weighted soft-voting ensembles. Average Precision (AP) was the primary ranking metric, along with precision, recall, F1 score, PR AUC, false positives (FP), and false negatives (FN) used to characterize operating tradeoffs. The selected XGBoost configuration produced the highest observed test AP (0.9084) and F1 score (0.8534) on this synthetic dataset. The soft-voting ensembles reduced some false positives but did not exceed XGBoost in AP. These findings apply to the specific configured pipelines, feature engineering choices, and recurring synthetic customer/merchant population studied here; they do not establish that XGBoost is generally the best fraud-detection method. The study also identifies limitations related to repeated validation use, unequal imbalance treatments, synthetic-data artifacts, probability calibration, and the absence of uncertainty estimates or entity-disjoint evaluation.

Explore More!

Published Paper
Christian B.
Eric Bradford
Electrical Engineering and Computer Science Masters from MIT, Technical PM at Apple

Related Projects

Predicting International Commodity Prices, Production, and Land Usage Towards Reducing Agricultural Emissions

Our long-term goal for this research project was to analyze food insecurity and determine if we could predict the emissions and pricing of different food commodities.
Sahir G.
Mentored by Landon Butler
Using Neural Networks to Predict U.S. Corporate Profits on Electronic Goods

The goal of this project is to train two neural network AI models: a Multi-Layer Perceptron (MLP) neural network and a Long Short-Term Memory (LSTM) neural network, to predict U.S. corporate profits on electronic goods into the future.
Will K.
Mentored by Ana Sofia Muñoz Valadez
Backtesting and Machine Learning for a Self-Developed Futures Trading Strategy

This study investigates whether the poor backtesting performance of a discretionary futures trading strategy stems from important subconscious rules that were never formally documented. Testing ten years of Nasdaq-100 and S&P 500 futures data showed that adding an previously unwritten market-structure rule substantially improved performance, while machine learning found short-term price direction largely unpredictable but volatility expansion moderately predictable.
Anant S.
Mentored by Odysseas Drosis