Learned Classifier Strategy (LCS)

Updated 14 July 2026

Learned Classifier Strategy (LCS) is a data-driven approach that empirically selects the optimal classifier, sampler, and dimensionality reduction pipeline based on dataset characteristics.
The methodology integrates preprocessing, stratified validation, and imbalance-aware metrics (F1, G-mean) to robustly detect rare events like fraud in heterogeneous datasets.
Empirical findings show that intelligent undersampling and models like Random Forest achieve strong performance, though the approach is computationally intensive and dataset-specific.

Learned Classifier Strategy (LCS) denotes a data-driven classifier selection strategy in which the final model is chosen by empirically comparing multiple combinations of dimensionality reduction, class balancing, and classification on the target dataset itself rather than by assuming a universal best classifier in advance. In the cited fraud-detection formulation, the strategy is designed for massively imbalanced, potentially PCA-encoded, and heterogeneous datasets, and it selects the model that best matches the dataset’s characteristics and evaluation objective under stratified validation with balancing applied only to the training data (Kulatilleke, 2022).

1. Definition and conceptual scope

In the fraud-detection formulation, LCS is a practical empirical selection procedure rather than a formal meta-learning algorithm. Its core idea is to use the data itself to determine the final classifier. The procedure therefore evaluates classifier + sampler + dimensionality combinations and chooses the best-performing configuration for that specific dataset. The “learned” aspect lies in empirical comparison on the target dataset, not in a separate meta-learner trained across many datasets (Kulatilleke, 2022).

This definition is tightly linked to the paper’s problem setting: fraud datasets are described as highly heterogeneous and massively imbalanced, and the paper argues that no single classifier is universally best across such datasets. A model effective on one dataset may fail on another because datasets differ in feature definitions, dimensionality, encoding, fraud ratio, size, and statistical characteristics. This motivates selection after measurement rather than selection by prior preference.

A common misconception is to treat LCS here as a synonym for a fixed “best fraud model.” The paper rejects that premise explicitly. Another misconception is to read the acronym as referring to Learning Classifier Systems, a distinct family of rule-based evolutionary methods. In the fraud-detection paper, LCS refers instead to a dataset-adaptive classifier selection strategy.

2. Problem setting and motivation

The primary motivation for LCS is the interaction of rarity, heterogeneity, and imbalance in credit card fraud detection. On the primary dataset, only 492 of 284,807 transactions are frauds, i.e. 0.172%. The paper further emphasizes that false negatives are very undesirable, fraud behavior may drift over time, and fraud datasets vary widely in representation and prevalence (Kulatilleke, 2022).

In this setting, standard accuracy is insufficient. A classifier can achieve high accuracy by mostly predicting the majority class, yet miss frauds. The paper therefore treats balancing methods as a necessary pre-processing mechanism and prefers imbalance-aware metrics, especially F1 score and G-mean. This makes LCS not merely a model-selection heuristic, but a protocol for aligning classifier choice with the minority-detection objective.

The broader implication is methodological. Because fraud data may be PCA encoded, temporally unstable, and distributionally non-uniform, the model-selection problem becomes inseparable from preprocessing. This suggests that LCS is best understood as a pipeline-selection framework rather than as a classifier-only ranking exercise.

3. Pipeline structure and candidate components

The strategy is organized as a three-stage pipeline: Node A for dimensionality reduction / feature selection, Node B for sampling / balancing, and Node C for classifier (Kulatilleke, 2022).

Stage	Role	Reported options
Node A	Dimensionality reduction / feature selection	PCA / feature selection
Node B	Sampling / balancing	Random majority under-sampling with replacement, Instance Hardness Threshold, Random minority oversampling with replacement, SMOTE, ADASYN
Node C	Classifier	DummyClassifier, Logistic Regression, Random Forest, Gaussian Naive Bayes, Linear SVC, MLPClassifier, RidgeClassifier, Decision Tree, SGDClassifier, Passive Aggressive Classifier, Perceptron, KNeighborsClassifier, AdaBoostClassifier (Real), AdaBoostClassifier (Discrete), Gradient Boosting Classifier, Quadratic Discriminant Analysis

The primary dataset already has 28 PCA components plus 2 raw features (date, amount). For the secondary dataset, the raw data are PCA-encoded to match the experimental style of the primary dataset, producing 23 principal components using full SVD. The paper also studies dimensionality sensitivity and reports that performance peaks around the first ~15 principal components and then slightly degrades (Kulatilleke, 2022).

The selection workflow is explicit: take the unknown imbalanced fraud dataset; optionally reduce dimensions using PCA / feature selection; apply a balancing method only to the training split; train multiple classifiers; score each candidate model using one or more metrics; select the best-performing model for that dataset; and optionally use the top-k classifiers to build a voting ensemble and test whether it improves performance. This makes LCS a compositional search over pipeline variants rather than a single-model benchmark.

4. Validation protocol and decision criteria

The validation protocol is designed to avoid information leakage and to preserve the real fraud distribution in evaluation. Each model uses stratified cross-validation, preserving class proportions across folds, with a 20:80 test:train split, and balancing performed only on the training set (Kulatilleke, 2022).

The paper reports multiple evaluation criteria: Accuracy, Precision, Recall, F1, G-mean, AUROC, Cohen’s kappa, Matthews correlation coefficient, Hamming loss, and training time. In practice, the paper uses the top scoring models by F1 and inspects corresponding G-mean and runtime. The best classifier is therefore defined as the one with the best balance of minority detection and overall classification quality, not simply the one with the highest accuracy.

The metric emphasis is central to the meaning of LCS. Precision is defined as TP / (TP + FP), Recall / TPR as TP / (TP + FN), F1 as the harmonic mean of precision and recall, G-mean as the geometric mean of class-wise performance, FPR as FP / (FP + TN), and TNR as TN / (TN + FP). The paper especially favors F1 and G-mean because they are more informative than accuracy in highly imbalanced settings.

The paper also notes that both oversampling and undersampling can introduce sample selection bias, since they alter the training distribution away from the true prior. It references importance weighting and ensemble classifiers as general remedies, but its own empirical conclusion is that intelligent undersampling works best in this setting.

5. Empirical findings and operating conditions

The empirical results are presented on two datasets. The primary benchmark is the ULB / Pozzolo credit card fraud dataset with 284,807 transactions, 492 frauds, and 30 features, consisting of 28 PCA components and 2 raw features. The secondary dataset is the UCI credit card default dataset with 30,000 instances and 6,634 frauds, with 23 raw features subsequently PCA-encoded (Kulatilleke, 2022).

Several findings define the operational profile of LCS. First, undersampling generally beats oversampling on both datasets. Second, random undersampling is not good enough; the best results come from intelligent undersampling, especially Instance Hardness Threshold. Third, Random Forest is the strongest overall performer in the primary experiments: on the primary dataset, Random Forest with Instance Hardness Threshold undersampling achieves the best overall performance among the tested configurations and reaches F1 = 0.8466. Fourth, KNeighborsClassifier can give the best raw F1 in the unbalanced setting, but it is much slower. Fifth, ensembles are not automatically better; soft voting underperformed hard voting in this setting.

The paper argues that the strategy outperforms peer models because it uses the true data distribution, tests multiple balancing methods, avoids naive random resampling, selects models based on imbalance-aware metrics, exploits the fact that different datasets favor different classifier/sampler combinations, and uses PCA carefully rather than assuming more dimensions are always better. It also states that some published comparisons used artificially altered class distributions, which makes results less realistic than the paper’s evaluation.

The framework is nevertheless constrained by explicit limitations. Temporal drift is not modeled; the selection strategy is computationally expensive; full dataset runs can take 5–6 hours; ensemble gains are not consistent; and the framework is not optimized for speed. The paper also states that it does not present a formal meta-learning algorithm, but a practical empirical selection protocol.

The acronym LCS is overloaded across adjacent literatures, and disambiguation is necessary. In many machine-learning papers, LCS means Learning Classifier Systems: rule-based machine learning methods that balance prediction accuracy with interpretability and that evolve populations of IF:THEN rules (Zhang et al., 2021). That usage is unrelated to Learned Classifier Strategy in fraud detection, even though both concern classifier design.

A second adjacent usage appears in Learned Bloom Filters, where classifier selection is treated as a complexity-aware and budget-aware design criterion. That work argues that the classifier is the main design choice determining whether a learned filter improves on a standard Bloom Filter, and recommends a classifier selection criterion based on dataset classification complexity, memory budget, and filter architecture. It identifies linear SVMs for the easiest datasets and small FFNNs for moderately more complex but still “easy” datasets, while emphasizing that the “best” classifier depends on the dataset’s hardness and on whether the designer prioritizes FPR, memory, or latency (Malchiodi et al., 2022). This is not the same pipeline protocol as the fraud-detection LCS, but it shares the principle that classifier choice should be data- and objective-dependent rather than universal.

A third, conceptually different line of work concerns strategic classification. In repeated strategic classification with linear classifiers, Stackelberg regret and external regret are shown to be strongly incompatible, and the proposed solution is a strategy-aware algorithm that learns under the fact that agents react strategically to the deployed classifier (Chen et al., 2019). A later extension argues that classifier deployment can induce a market for features, so costs become endogenous rather than exogenous: the classifier creates demand, competition gives rise to prices, and learning must account for the induced market (Sommer et al., 27 Feb 2025). These works use “strategy-aware” in the sense of anticipating strategic responses, not in the sense of empirically selecting among classifier pipelines.

Taken together, these distinctions locate Learned Classifier Strategy as a specific dataset-adaptive empirical selection procedure. Its defining properties are exhaustive or broad comparative evaluation on the target dataset, explicit treatment of preprocessing and balancing as part of model choice, and selection by imbalance-aware performance criteria rather than by a presumed universal best classifier.