Detect-Classify-Compare Framework

• Detect-Classify-Compare is a modular framework that stages detection, classification, and comparison to differentiate candidate instances using domain-specific metrics.
• It adapts its phases to various domains, including classifier disagreement analysis, static code vulnerability detection, and CLIP-based visual comparisons.
• The framework provides actionable insights through feature profiling and ensembling, emphasizing operational comparison over mere evaluation.

The detect-classify-compare framework denotes a family of staged analytical workflows in which candidate instances are first isolated or flagged, then assigned labels or routed through an auxiliary classifier, and finally compared through fingerprints, discriminative descriptors, or explanation-oriented profiles. In the supplied literature, this pattern appears in visually comparing binary classifiers through learning from disagreement, in static source-code vulnerability analysis with MARFCAT, and in CLIP-based visual classification with comparative descriptors; the sources show that the ordering and semantics of the three stages are domain dependent rather than canonical (Wang et al., 2022, Mokhov, 2010, Lee et al., 2024).

1. Scope and domain-specific meanings

Across the cited work, “detect,” “classify,” and “compare” do not denote a single fixed algorithm. Instead, the same stage labels are instantiated with different objects: disagreement regions between classifiers, distances to vulnerability fingerprints, or pairwise discriminative text cues among visually similar classes. The sources therefore support reading the framework as a modular pattern rather than as a standardized protocol.

Context	Detect	Classify / Compare
Learning-from-disagreement	Identify instances where two classifiers disagree after score-based thresholding	Train a discriminator on disagreement instances; compare models via SHAP and feature profiling
MARFCAT	Flag a file when the minimal distance to stored fingerprints falls below a threshold	Assign CVE/CWE type by nearest fingerprint; compare via distance or similarity to class centroids
Comparative descriptors for CLIP	Comparative descriptor module is specified for incorporation into a pipeline	Classify by averaging image-text similarities over descriptor sets; compare a target class against nearest semantic neighbors

A recurrent structural property is that comparison is not merely an evaluation afterthought. In the LFD setting, comparison is itself the main analytical target. In MARFCAT, comparison is operationalized as fingerprint matching. In comparative descriptors, comparison is encoded directly into the text prompts by conditioning on semantically nearest classes.

2. Learning-from-disagreement as a detect-classify-compare pipeline

The LFD framework starts from two pre-trained binary classifiers, $f_1 : X \to [0,1]$ and $f_2 : X \to [0,1]$, each returning a real-valued score for input $x \in X$. The detection step is score-based. For a user-selected cutoff $\alpha$, thresholds $\tau_1,\tau_2$ are defined as the $\alpha$-quantiles of the score lists $S_1=\{f_1(x):x\in X\}$ and $S_2=\{f_2(x):x\in X\}$. Binary capture indicators are then defined by $y_1(x)=I[f_1(x)\ge \tau_1]$ and $y_2(x)=I[f_2(x)\ge \tau_2]$. The disagreement set is

$f_2 : X \to [0,1]$0

It is split into $f_2 : X \to [0,1]$1, captured by $f_2 : X \to [0,1]$2 only, and $f_2 : X \to [0,1]$3, captured by $f_2 : X \to [0,1]$4 only (Wang et al., 2022).

The classification step trains a discriminator on disagreement instances. Each $f_2 : X \to [0,1]$5 is mapped to a meta-feature vector $f_2 : X \to [0,1]$6, where the meta-features are user-defined probes of the two base models’ behaviors. The paper uses a SHAP-compatible binary classifier, specifically XGBoost, to approximate $f_2 : X \to [0,1]$7, with label $f_2 : X \to [0,1]$8 indicating membership in $f_2 : X \to [0,1]$9. The training objective is empirical cross-entropy:

$x \in X$0

where $x \in X$1.

The meta-features are domain-specific. In the merchant case study, they include time-series counts and rates over a fixed historical window, such as nonzero_numApprTrans, nonzero_amtDeclTrans, mean_avgAmtAppr, mean_rateTxnAppr, and mean_numApprTrans, with 70 meta-features in total for the merchant dataset. In the GNN case, graph-based merchant affinities include n_connection and entropy_neighbors. In the Avazu CTR case, each original categorical feature yields four probes: n_impressions_f, n_clicks_f, ctr_f, and n_active_hours_f, producing 168 meta-features from 42 original features.

The comparison step is then built on the trained discriminator rather than directly on the base models. This design allows comparative interpretation even when the compared classifiers’ training features are not available. A plausible implication is that LFD treats disagreement regions as a compressed representation of model difference: only the captured-only cells are retained, and the comparison is delegated to a meta-feature space chosen to probe hypothesized behavioral distinctions.

3. Comparative interpretation, feature profiling, and empirical behavior in LFD

LFD interprets the discriminator with Tree-SHAP. Because $x \in X$2 is a tree ensemble, each disagreement instance receives SHAP values $x \in X$3 satisfying

$x \in X$4

where $x \in X$5 is the expected base-score. The standard global importance measure is

$x \in X$6

The framework then adds four complementary metrics: Consistency, Contrast, Correlation, and Overall. Consistency is based on the inverse of a bin-weighted Shannon-entropy over the empirical distribution of feature values within SHAP-value bins. Contrast is the Jensen-Shannon divergence between feature-value histograms on $x \in X$7 and $x \in X$8. Correlation is the absolute Pearson correlation between $x \in X$9 and $\alpha$0. Overall is the linear combination

$\alpha$1

All five metrics are exposed for user-driven ordering and comparison (Wang et al., 2022).

The visual analytics system provides two coordinated views. The Disagreement Distribution View overlays the captured sets $\alpha$2 and $\alpha$3, shows the two non-overlap disagreement cells $\alpha$4 and $\alpha$5, and allows users to sweep $\alpha$6 from $\alpha$7 to $\alpha$8 while dynamically recomputing cell sizes. The Feature View offers an “Overview + Details” design with two vertical lists of area-plots, one for the TP discriminator and one for the FP discriminator; features may be ordered by any of Magnitude, Consistency, Contrast, Corr, or Overall. On demand, a bubble-plot shows a 2D histogram of SHAP versus $\alpha$9, and a transfer-function widget remaps feature-value to color.

The framework is evaluated in two industrial case studies. In merchant category verification, the dataset contains approximately 3.8 million merchants and 2.5 years of daily transaction logs, with the true-category binary label “restaurant” versus “non-restaurant.” Base models include Tree, CNN, RNN, and GNN. The discriminator achieves AUC $\tau_1,\tau_2$0 on the TP side and AUC $\tau_1,\tau_2$1 on the FP side. Reported findings include nonzero_numApprTrans high $\tau_1,\tau_2$2 RNN correctly captures active merchants where Tree misses, and neighbor_entropy high $\tau_1,\tau_2$3 GNN captures merchants with diverse affinities better than RNN.

In CTR prediction on Avazu, the dataset contains 40 million ad-impressions over 10 days and 21 anonymized categorical fields. Day 1 is used to build historical meta-features, Days 2–8 for training, Day 9 for test/comparison, and Day 10 is held out for ensembling. Base models are LR using FTRL-proximal and Fi-GNN. The discriminator AUC is $\tau_1,\tau_2$4 on Day 9 TP and $\tau_1,\tau_2$5 on FP. The top TP-side meta-features by Overall are ctr_site_id, ctr_site_app_id, and ctr_c14; high ctr_site_id indicates that GNN catches click-instances better than LR.

The same paper uses the top-15 meta-features, ranked by each metric, in Feature-Weighted Linear Stacking of LR+GNN. On Day 10, all FWLS ensembles beat the individual base models. The finding reported in the paper is that ranking by Overall yields the strongest Day 9 AUC, while Contrast slightly wins Day 10. The ablation study further reports that, with the 5 highest-Overall features alone, AUC is approximately $\tau_1,\tau_2$6, whereas random 5 features give AUC approximately $\tau_1,\tau_2$7. This suggests that the compare stage in LFD is not limited to explanation; it can also guide feature-wise ensembling and meta-feature selection.

4. MARFCAT: fingerprint comparison, vulnerability detection, and weakness classification

MARFCAT realizes a detect-classify-compare framework for static source-code vulnerability analysis using the open-source MARF framework and the MARFCAT application. Its Compare stage is centered on a transformation $\tau_1,\tau_2$8 that maps raw source code into a fixed-length numerical fingerprint. Two parallel pipelines are provided. In the signal-processing pipeline, source text or binary is treated as a 1-D signal of overlapping n-grams, with experiments using $\tau_1,\tau_2$9. After optional preprocessing such as silence removal or band-pass filtering, the short-time Fourier transform or direct FFT is computed, and the magnitude spectrum is aggregated into a $\alpha$0-dimensional feature vector

$\alpha$1

In the NLP pipeline, code is viewed as a sequence of tokens or characters, unigrams or n-grams are extracted, and empirical counts or smoothed probability vectors are recorded (Mokhov, 2010).

Training files annotated with CVE or CWE labels are used to build a fingerprint store by clustering feature vectors by class. In the preliminary release, one centroid per CVE/CWE is computed:

$\alpha$2

A test file $\alpha$3 is compared against a stored fingerprint $\alpha$4 by distance or similarity measures including Euclidean distance,

$\alpha$5

cosine similarity converted to a distance,

$\alpha$6

as well as Chebyshev, Minkowski, and Diff distances.

The Detection stage uses the minimum distance over all trained classes:

$\alpha$7

If $\alpha$8 falls below a predefined threshold $\alpha$9, the file is flagged for further analysis. The paper notes that MARFCAT’s current reference implementation uses nearest-centroid and k-NN classifiers, but that the same detection stage could be backed by any supervised model; a linear SVM hinge-loss formulation is given as an example.

The Classification stage assigns a suspicious file to a specific vulnerability or weakness type. In the nearest-centroid scheme,

$S_1=\{f_1(x):x\in X\}$0

A probabilistic alternative using multinomial logistic regression or a neural-network softmax layer is also specified.

The evaluation uses CVE-selected vulnerable/fixed pairs for C/C++, including Wireshark 1.2.0 versus 1.2.9 and Chrome 5.0.375.54 versus 5.0.375.70, CVE-selected Java with Tomcat 5.5.13 versus 5.5.29, and non-CVE cases Dovecot 2.0β6 and Pebble 2.5-M2. Metrics are Precision, Recall, and F1. For the SATE2010 top configuration with the signal pipeline and first guess, reported results are approximately 92.7% precision and 100% recall for Wireshark CVEs, approximately 83.7% precision and approximately 93% recall for Tomcat CVEs, and approximately 90.9% precision and 100% recall for Chrome CVEs. For Dovecot 2.0β6 and Pebble 2.5-M2, the paper reports no false negatives on non-CVE data under tuned thresholds. The unigram-plus-add-delta NLP pipeline yields slightly lower precision but remains competitive, with Wireshark CVEs at 83.3%.

A common misunderstanding would be to assume that “compare” must occur after detection and classification. MARFCAT shows the opposite ordering explicitly: the fingerprint comparison machinery is foundational, and detection and classification are derived from it.

5. Comparative descriptors in CLIP-based visual classification

In the CLIP setting, the relevant contribution is a comparative descriptor module suitable for direct incorporation into a detect-classify-compare pipeline. A comparative descriptor is defined as a short text phrase that highlights features by which a target class $S_1=\{f_1(x):x\in X\}$1 can be distinguished from one of its semantically nearest neighbors $S_1=\{f_1(x):x\in X\}$2. Instead of describing a class in isolation, the method asks an LLM for cues specific to the pair $S_1=\{f_1(x):x\in X\}$3, for example: “What are useful visual features for distinguishing a $S_1=\{f_1(x):x\in X\}$4 from a $S_1=\{f_1(x):x\in X\}$5 in a photo?” The stated purpose is to focus on subtle inter-class differences, reduce ambiguity, and avoid overly generic or spurious attributes (Lee et al., 2024).

Let $S_1=\{f_1(x):x\in X\}$6 be the set of candidate classes. For each class $S_1=\{f_1(x):x\in X\}$7, the standard CLIP-style prompt is $S_1=\{f_1(x):x\in X\}$8 “A photo of a $S_1=\{f_1(x):x\in X\}$9,” with text embedding $S_2=\{f_2(x):x\in X\}$0. Semantic proximity is measured entirely in CLIP’s text space by cosine similarity:

$S_2=\{f_2(x):x\in X\}$1

For each target class $S_2=\{f_2(x):x\in X\}$2, the method selects the top-$S_2=\{f_2(x):x\in X\}$3 nearest neighbors $S_2=\{f_2(x):x\in X\}$4. For each pair $S_2=\{f_2(x):x\in X\}$5 with $S_2=\{f_2(x):x\in X\}$6, an LLM generates a short phrase $S_2=\{f_2(x):x\in X\}$7, whose CLIP text embedding is $S_2=\{f_2(x):x\in X\}$8. The full descriptor set is $S_2=\{f_2(x):x\in X\}$9.

Classification averages similarities between the image embedding $y_1(x)=I[f_1(x)\ge \tau_1]$0 and the comparative descriptor embeddings:

$y_1(x)=I[f_1(x)\ge \tau_1]$1

where $y_1(x)=I[f_1(x)\ge \tau_1]$2 or cosine similarity if both vectors are L2-normalized. The baseline zero-shot classifier predicts $y_1(x)=I[f_1(x)\ge \tau_1]$3, while the comparative-descriptor classifier predicts $y_1(x)=I[f_1(x)\ge \tau_1]$4. The paper also allows fusion of the original prompt with the comparative descriptors by setting $y_1(x)=I[f_1(x)\ge \tau_1]$5 and averaging over the union.

A filtering process removes descriptors that are irrelevant to CLIP’s vision-language space. For class $y_1(x)=I[f_1(x)\ge \tau_1]$6, a small set of $y_1(x)=I[f_1(x)\ge \tau_1]$7 representative images is gathered, and the mean image feature is

$y_1(x)=I[f_1(x)\ge \tau_1]$8

The lower-bound threshold is

$y_1(x)=I[f_1(x)\ge \tau_1]$9

with $y_2(x)=I[f_2(x)\ge \tau_2]$0. Each raw descriptor embedding $y_2(x)=I[f_2(x)\ge \tau_2]$1 is scored by

$y_2(x)=I[f_2(x)\ge \tau_2]$2

descriptors with $y_2(x)=I[f_2(x)\ge \tau_2]$3 are discarded, and the top-$y_2(x)=I[f_2(x)\ge \tau_2]$4 survivors are retained. If none survive, the method falls back to $y_2(x)=I[f_2(x)\ge \tau_2]$5.

The experimental summary uses CLIP ViT-B/32 in zero-shot mode on 13 datasets: ImageNet, IN1k-V2, Caltech256, CIFAR-100, CUB200, EuroSAT, Places365, Food101, Pets, DTD, Flowers102, Aircraft, and Cars. The average top-1 accuracy is 57.6% for the CLIP baseline, 60.0% with comparative descriptors and no filtering, and 62.4% with filtering for few-shot $y_2(x)=I[f_2(x)\ge \tau_2]$6–64. Reported per-dataset gains include CUB200 from 51.4 to 54.1 to 56.4, DTD from 43.2 to 46.5 to 53.8, and Flowers102 from 63.0 to 67.1 to 73.4. Descriptor-only classification accuracy using only $y_2(x)=I[f_2(x)\ge \tau_2]$7 as text is 34.7% versus 23.7% for DCLIP on the 13 benchmarks. The paper also reports qualitative gains such as avoiding ambiguous descriptors and enabling per-descriptor explainability by plotting $y_2(x)=I[f_2(x)\ge \tau_2]$8 for each descriptor.

6. Conceptual synthesis, misconceptions, and methodological implications

The three sources collectively show that the detect-classify-compare framework is best treated as a domain-specific composition of modules rather than a single canonical recipe. In LFD, detection isolates disagreement regions by thresholded capture indicators, classification is delegated to a discriminator over meta-features, and comparison is explanation-driven through Tree-SHAP and feature profiling. In MARFCAT, comparison is the primary act of fingerprint matching, detection is a threshold on the minimum class distance, and classification is nearest-class assignment. In comparative descriptors, the compare operation is built into the prompt-generation process itself by conditioning descriptor construction on a target class and its nearest semantic neighbors (Wang et al., 2022, Mokhov, 2010, Lee et al., 2024).

One potential misconception is that “classification” always refers to the original task label. The LFD formulation contradicts that assumption: the trained classifier predicts whether a disagreement instance belongs to $y_2(x)=I[f_2(x)\ge \tau_2]$9 or $f_2 : X \to [0,1]$00, that is, which base model captured it. Another potential misconception is that “detection” must mean anomaly detection or vulnerability flagging. In LFD, detection is the extraction of disagreement instances; in MARFCAT, it is threshold-based suspicion; in the comparative-descriptor setting, the provided specification focuses on comparison and classification, with the descriptor module intended for incorporation into a broader pipeline.

The sources also suggest that representation design is decisive. LFD depends on user-defined meta-features proposed on the basis of hypotheses of the classifiers, MARFCAT depends on whether source code is represented as signal spectra or n-gram statistics, and the CLIP framework depends on whether generated descriptors remain close to image embeddings in CLIP space. A plausible implication is that the effectiveness of detect-classify-compare workflows is often governed less by the stage labels themselves than by the intermediate representation chosen for the compare operation.

Finally, the surveyed work places comparison in an operational rather than merely evaluative role. In LFD, comparison yields actionable insights and supports better ensembling. In MARFCAT, comparison drives the entire detection-and-classification decision path. In comparative descriptors, comparison sharpens semantic focus by encoding subtle inter-class differences directly into the text side of the model. Taken together, these formulations establish detect-classify-compare as a recurring design pattern for systems that must move from candidate isolation to discriminative labeling while preserving an explicit account of what distinguishes one decision route, one fingerprint, or one class from another.