Caprese Method: Causal Inference & LLM Acceleration

• Caprese Method is a dual framework that infers cancer progression trees from genetic data and accelerates LLM math reasoning by applying low-rank residual distillation.
• For cancer progression inference, it leverages probability-raising and occurrence-frequency principles to construct biologically plausible causal trees with superior noise robustness.
• In LLM applications, the method compensates feed-forward sparsification losses by learning efficient low-rank corrections, restoring math accuracy while maintaining overall language performance.

The term Caprese Method refers to two distinct methodologies in computational science: one for inferring causal tree models of cancer progression from cross-sectional genetic data, and another for accelerating LLM math reasoning via low-rank residual distillation after feed-forward sparsification. Both approaches are characterized by methodological innovations grounded in principled statistical frameworks and empirical performance validation.

1. Probabilistic Causation and the Cancer Progression CAPRESE Method

The original CAPRESE (CAncer PRogression Extraction with Single Edges) framework is a method for reconstructing tree-structured models of accumulative processes, primarily cancer progression, from cross-sectional data. It operates under a probability-raising conception of causality, as formulated by Suppes, which posits that a cause raises the probability of its effect. In the absence of time-of-occurrence information common in biological datasets, the CAPRESE method leverages this notion to infer likely sequences of somatic events.

Let $c$ and $e$ be two events (mutations or genomic aberrations). Suppes' criterion for prima facie causation is

$P(e|c) > P(e|\neg c),$

where $P(e|c)$ denotes the conditional probability of $e$ given $c$. The algorithm retains only the probability-raising condition due to lack of temporal data. Moreover, CAPRESE imposes occurrence-frequency temporal priority: in a progression tree, ancestors must be more frequent than their descendants ($P(a) > P(b)$ whenever $a$ is an ancestor of $b$).

2. The CAPRESE Statistical Estimator and Algorithm

The CAPRESE estimator computes, for every event pair $(a,b)$:

• A raw estimator (normalized PR-score),

$e$0

where $e$1.

• A correlation correction factor,

$e$2

with $e$3.

A shrinkage-like estimator linearly combines these: $e$4 Parameter $e$5 trades off between precision ($e$6) and robustness ($e$7); cross-validation suggests $e$8 for noise-free data and $e$9 for moderate noise.

The algorithm takes an $P(e|c) > P(e|\neg c),$0 binary matrix of samples-by-events, computes all pairwise probabilities, then assigns to each event $P(e|c) > P(e|\neg c),$1 its unique parent as the event (or special root) maximizing $P(e|c) > P(e|\neg c),$2 (subject to $P(e|c) > P(e|\neg c),$3 and $P(e|c) > P(e|\neg c),$4). An additional filter replaces weakly supported parent assignments with direct roots, allowing for multiple independent progressions.

The computational complexity is $P(e|c) > P(e|\neg c),$5, tractable for hundreds of events and samples.

3. Statistical Guarantees, Noise Robustness, and Performance

CAPRESE provably converges to the true underlying progression tree as the sample size increases and the shrinkage parameter $P(e|c) > P(e|\neg c),$6, provided the data are generated from a tree model. With synthetic data, CAPRESE consistently outperforms Desper’s oncotrees in terms of Tree-Edit-Distance (TED), requires fewer samples for exact reconstruction, and shows superior robustness to uniform random noise, outperforming also hidden Conjunctive Bayesian Networks (h-CBN) under comparable measurement error rates.

Under uniform noise (false positive/negative rate $P(e|c) > P(e|\neg c),$7), consistency is maintained for $P(e|c) > P(e|\neg c),$8, where $P(e|c) > P(e|\neg c),$9 is the least marginal probability of any event.

Case studies demonstrate that CAPRESE identifies biologically plausible progression paths, provides higher bootstrap edge confidence than oncotrees, and is capable of resolving independent or convergent evolutionary trajectories in cancer subtypes (Loohuis et al., 2013).

4. Practical Considerations for Cancer Progression Inference

Recommended practice begins with the selection of a panel of candidate driver alterations, binarization of per-sample event presence, and optional filtering of rare or ubiquitous events. The shrinkage parameter $P(e|c)$0 should be tuned via cross-validation on synthetic data with matched noise/sample statistics. Non-parametric bootstrap is used to assess edge confidence.

The output is a rooted progression tree (or forest) elucidating likely temporal orderings. Strongly supported edges may warrant biological validation. Multiple tree roots reflect hypothesized parallel or independent progression paths, indicating possible disease subtypes.

5. The Caprese Method for Efficient LLM Math Reasoning

A separate method, also denominated Caprese ("CAPability REcovery with Scalable Efficiency"), addresses the degradation of LLM math-reasoning performance due to aggressive feed-forward (FF) sparsification or structured pruning (e.g., CATS, GRIFFIN). The standard approach achieves substantial speedups in decoding but at the cost of drastic performance drops in math tasks, despite minimal impact on language benchmarks.

Caprese remedies this by freezing sparse FF weights and learning lightweight low-rank corrections ($P(e|c)$1) that directly minimize the mean-squared error (MSE) of the residual between the original and pruned FF block outputs. At inference, the FF block computes

$P(e|c)$2

where $P(e|c)$3, $P(e|c)$4, with $P(e|c)$5 (typically $P(e|c)$6). No extra sequential depth is introduced; computations are parallelized within the kernel (Dong et al., 8 May 2025).

6. Training, Integration, and Evaluation of Caprese for LLMs

Caprese utilizes a two-stage training scheme:

1. Layerwise regression: Each FF block learns its $P(e|c)$7 via MSE minimization between the full and sparse output on a synthetic math problem set (20,000 samples), with FF weights frozen.
2. End-to-end fine-tuning: All low-rank parameters are jointly refined via the overall model MSE.

Typical configurations inject under 1% additional parameters into models with 8–14B parameters, yet recover most, and sometimes all, lost math accuracy. For example, under CATS/GRIFFIN with 50% FF sparsity, Caprese+GRIFFIN restores GSM8K accuracy from 30–40% back up to 40–51%, and math performance from ≈10% back up to within 1–2% of the dense model (Llama 3.1 8B, Gemma 2 9B). Language-task performance remains within ±1% of the original.

Caprese yields >11% net latency reduction on 2,048-token outputs, even after correction overheads, via active parameter reduction (example: ≈2B fewer FF parameters in Gemma 2 9B and Llama 3.1 8B). Empirical ablations corroborate that raising low-rank $P(e|c)$8 monotonically enhances math accuracy under fixed sparsity, and that end-to-end joint optimization delivers further gains. Neuron-reselection during decoding further closes the gap with dense models on long chain-of-thought tasks.

7. Limitations and Prospective Directions

Both Caprese methods rely on strong assumptions: in cancer modeling, that all relevant events are observed (closed-world), and in LLM acceleration, that the sparse FF baseline preserves basic language function. The low-rank LLM variant has been trained chiefly on synthetic math; its transferability to domains such as code or logic remains untested. Input-dependent or layerwise-adaptive ranks may further optimize parameter efficiency. The LLM variant, by design, only compensates the approximation error of FF blocks; it cannot address core model biases or pathologies.

In summary, the Caprese methods exemplify statistically grounded, computationally efficient frameworks for causal inference in cancer and capability restoration in sparse LLMs, each with provable convergence and demonstrated empirical superiority over prior methods in their respective domains (Loohuis et al., 2013, Dong et al., 8 May 2025).