Dummy-Masking in molFTP Evaluation

• Dummy masking is a fold-specific leakage-control mechanism for molFTP, preventing feature leakage by zeroing out keys absent in the training fold and downweighting shared ones.
• It efficiently approximates leave-one-out behavior without full per-fold recomputation, maintaining near full-data key discovery while correcting for bias.
• Empirical results show robust performance with minimal shifts in fragment scores and competitive AUPRC and AUROC compared to key-LOO methods.

Dummy-masking procedure is a fold-specific leakage-control mechanism introduced for molecular fragment-target prevalence (molFTP), a representation in which fragment-level features are computed from supervised fragment–target statistics rather than from raw molecular descriptors. Because those prevalence statistics are estimated from labeled data, ordinary cross-validation becomes biased unless the held-out molecules are prevented from influencing the feature map itself. Dummy masking addresses that problem by removing information about fragments present in the held-out molecules: keys absent from the training fold are zeroed out, and keys that are seen in both training and held-out molecules are downweighted so that held-out occurrences do not influence fold-specific feature construction (Godin, 7 Oct 2025).

1. Leakage problem addressed by dummy masking

The procedure is motivated by a specific form of feature leakage. In molFTP, the feature value assigned to a fragment key is determined by how often that fragment appears in positive versus negative molecules across the labeled corpus. If those prevalence statistics are computed once on the full dataset and then reused inside cross-validation, the validation or test fold influences the representation used for both training and evaluation. Molecules do not cross folds, so this is not sample leakage in the usual sense; the leakage arises because the supervised feature map already contains information derived from held-out molecules (Godin, 7 Oct 2025).

For a validation molecule $x_i$, what leaks is the held-out molecules’ fragment membership and labels as summarized in prevalence statistics. The leaked information includes whether a fragment $K$ occurs in held-out positives or negatives, whether a key is globally frequent or rare, and how test-fold occurrences shift the fragment’s odds-ratio-like score. In consequence, the feature vector of $x_i$ can indirectly contain information derived from $y_i$ itself and from other held-out molecules in the same fold.

Dummy masking is therefore a safeguard for supervised feature construction rather than for train/test partitioning in the ordinary sense. Its purpose is to ensure that, within each cross-validation fold, the molFTP representation is built only from information justified by the training molecules of that fold.

2. molFTP representation and the protected object

The object protected by dummy masking is the molFTP representation. At the 1D single-fragment level, for a fragment key $K$, with binary fragment indicator $X_{iK}\in\{0,1\}$ and binary label $y_i\in\{0,1\}$, the contingency counts are

$$a=\sum_i \mathbf{1}\{y_i=1, X_{iK}=1\}, \quad b=\sum_i \mathbf{1}\{y_i=0, X_{iK}=1\},$$

$$c=\sum_i \mathbf{1}\{y_i=1, X_{iK}=0\}, \quad d=\sum_i \mathbf{1}\{y_i=0, X_{iK}=0\},$$

with $N=a+b+c+d$. Using Haldane smoothing, the 1D prevalence weight is the smoothed log-odds ratio

$K$0

The paper describes this as “Fisher’s formulation of $K$1.” It also derives a Fisher-style significance score from $K$2, using

$K$3

$K$4

and the signed score

$K$5

At vector-construction time, fragment-level prevalence maps are pooled into a fixed-length molecular vector. The supplementary formulation gives

$K$6

$K$7

and, for each radius $K$8,

$K$9

Thus,

$x_i$0

The 2D and 3D views use analogous prevalence statistics over fragment pairs and triplets, and the final representation concatenates the 1D, 2D, and 3D vectors. For $x_i$1, each order contributes $x_i$2 dimensions, so the final molFTP vector has $x_i$3 dimensions (Godin, 7 Oct 2025).

Dummy masking operates on the fold-specific lookup map from fragment keys to prevalence-derived scores. It does not change the global vocabulary of keys, but it changes which keys contribute and by how much within each fold.

3. Fold-specific masking and correction mechanism

Within a cross-validation fold, let $x_i$4 denote the training set and $x_i$5 the held-out set. Dummy masking modifies the full-corpus key map into a fold-specific map $x_i$6. The procedure has two components (Godin, 7 Oct 2025).

First, if a fragment key $x_i$7 appears in a held-out molecule but is absent from the training fold, then it is treated as unseen in training for that fold and its contribution is masked: $x_i$8 where $x_i$9 is the number of training molecules containing $y_i$0. The paper describes this as “zero[ing] out contributions from fragments not observed in the training fold” and, for a test molecule, “zero[ing] the contribution of any FTP key not observed in the training fold (unseen keys).”

Second, if a key is observed in both the full dataset and the training fold, but some of its occurrences lie in held-out molecules, then using the global prevalence score would still leak information from those held-out occurrences. To avoid full per-fold recomputation of contingency tables, the paper applies a factor correction. The intended correction is described as

$y_i$1

where $y_i$2 is the number of molecules in the full dataset containing $y_i$3. The paper states that the FTP score is adjusted by the ratio of training-fold support to full-dataset support, so that test-set occurrences do not influence the training-fold statistic.

Operationally, the procedure is fold-specific and key-specific rather than molecule-specific. For a given fold, all molecules in that fold—both training and held-out—are represented using the same fold-specific map $y_i$4. A fragment present in a held-out molecule is therefore handled in one of two ways: if it never appears in training, its contribution becomes $y_i$5; if it appears in training, its contribution is retained but downweighted to discount held-out support.

4. Integration with cross-validation and relation to leave-one-out

Dummy masking is designed as a middle ground between two extremes. One extreme is leakage-prone full-data feature construction, where the prevalence map is built once on the full corpus and reused in every fold. The other is the classical train-only solution, in which all prevalence statistics are recomputed separately inside each fold using only training molecules. Train-only recomputation is leakage-free, but it uses less data per fold and is computationally less attractive when all prevalence tables must be rebuilt repeatedly (Godin, 7 Oct 2025).

Dummy masking preserves a fixed global key inventory and nearly full-data key discovery, but it changes the effective score map across folds by suppressing unseen keys and discounting seen ones. The feature mapping therefore changes across folds even though the vocabulary does not. This is the source of its efficiency gain: it approximates train-only or leave-one-out behavior without full per-fold recomputation.

The paper places dummy masking alongside two related evaluation strategies. True leave-one-out (LOO) recomputes each molecule’s feature statistics with that molecule removed; it is the most exact and the most expensive. Key-LOO is a fold-agnostic first-order approximation that removes singleton or rare-key influence and rescales to mimic LOO. The abstract states that key-LOO “closely approximates true molecule-level leave-one-out (LOO), with deviation below 8% on our datasets,” and the supplement reports that on BBBP_free and Odorless, 92–94% of features lie within the theoretical first-order LOO bound while fewer than 8% exceed it. The paper also notes that with $y_i$6 molecules and $y_i$7 folds, dummy masking and key-LOO are “identical by construction” (Godin, 7 Oct 2025).

This establishes dummy masking as a practical approximation to leakage-free evaluation when the held-out fold is known in advance, while key-LOO serves a related role when pre-specified folds are not required.

5. Empirical behavior, performance, and computational profile

The paper reports that dummy masking alters fragment-target prevalence only marginally, with mean absolute shift $y_i$8. Figure 1 compares full-data keys, dummy masking, and key-LOO for 1D, 2D, and 3D vectors, and argues that the masked or corrected distributions remain close to the originals while removing leakage (Godin, 7 Oct 2025).

Performance is reported to be close to key-LOO across datasets, with key-LOO often slightly better but not uniformly so. On BBBP4094, the following results are reported.

Configuration	AUPRC	AUROC
MolFTP-key-LOO + Random Forest	$y_i$9	$K$0
MolFTP-dummy-mask + Random Forest	$K$1	$K$2
MolFTP-key-LOO + XGBoost	$K$3	$K$4
MolFTP-dummy-mask + XGBoost	$K$5	$K$6

Figure 2 reportedly shows that dummy masking and key-LOO are very similar across aggregation functions, with especially small gaps on the harder Odorless dataset.

Several practical details are explicit. Circular fragments are generated with RDKit for radii $K$7, with default $K$8 in most experiments. Pair and triplet keys are filtered by a similarity threshold $K$9, with default $X_{iK}\in\{0,1\}$0 in the main experiments. The method can use binary presence or counts, and the paper reports little difference between them under either dummy masking or key-LOO. Runtime is favorable: about 2,000 molFTP vectors/s, with supplementary timing suggesting per-fold feature generation around $X_{iK}\in\{0,1\}$1–$X_{iK}\in\{0,1\}$2 seconds on the stated hardware (Godin, 7 Oct 2025).

6. Interpretation, scope, and limitations

The intuitive rationale is that a fragment-target prevalence score is a compressed supervised summary of how the dataset “judges” a fragment relative to the target. If a held-out molecule contains fragment $X_{iK}\in\{0,1\}$3, and the score for $X_{iK}\in\{0,1\}$4 was partly estimated using that molecule or its held-out neighbors, then the representation has already incorporated evaluation-fold information. Dummy masking blocks exactly that route. Zeroing an unseen key means that supervised knowledge for that key is unavailable in the training fold; downweighting a seen key means that only the portion of support justified by the training fold is retained (Godin, 7 Oct 2025).

A common misconception is to treat this as per-molecule recalculation. The paper is explicit that dummy masking is not molecule-specific leave-one-out recomputation. It is fold-specific key masking/correction applied to the feature map used to construct molFTP vectors. Another potential misconception is to treat it as a generic masking method. In contrast to image-space masking procedures that generate masked views or suppress intermediate activations (Doloriel, 8 Dec 2025, Balasubramanian et al., 2022), dummy masking in molFTP acts on a supervised fragment-score map during feature construction.

The paper also notes several textual ambiguities. The extracted text contains minor inconsistencies and OCR damage in a few formulas, especially the exact smoothing constant, the exact pooling definition of margin in the main versus supplementary text, and the detailed expression of the key-LOO bound. These points require confirmation from the clean main text or appendix. The mechanics of dummy masking itself, however, are clear: within each cross-validation fold, build a fold-specific fragment-score map by zeroing keys absent from training and downweighting keys whose global prevalence partly depends on held-out molecules; then compute molFTP vectors for both train and validation molecules using only that fold-specific map (Godin, 7 Oct 2025).