xCompress: Reader-Aware Compression

• xCompress is an inference-time control framework for retrieval-augmented generation that optimizes summaries for better compatibility with reader LLMs using SPS.
• It adaptively generates and ranks multiple candidate compressions to select the one most aligned with the reader’s representational geometry.
• Empirical results show improved QA metrics on benchmarks, highlighting the benefits of reader-aware compression and adaptive sampling.

xCompress is an inference-time control framework for retrieval-augmented generation (RAG) whose central goal is not to improve retrieval in isolation, but to improve the compatibility between compressed retrieved evidence and the downstream reader LLM. It is formulated for the retriever–compressor–reader pipeline, where a summary is treated as useful only if it is represented in a way that the reader can actually exploit. The framework generates multiple candidate compressions at inference time and selects the candidate that best aligns with the reader according to the Spectrum Projection Score (SPS), a lightweight, supervision-free metric defined on the reader’s hidden representation space (Hu et al., 8 Aug 2025).

1. Reader-aware compression in the retriever–compressor–reader pipeline

xCompress is motivated by a specific asymmetry in modern RAG pipelines with compressed retrieval. In standard RAG, a retriever returns passages, a compressor condenses them, and the reader answers the question from the compressed context. The method argues that this pipeline is unidirectional: the compressor produces summaries without considering the representational geometry of the reader. A factually reasonable summary may therefore be suboptimal if its salient content lies in directions that the reader does not readily encode or use for generation (Hu et al., 8 Aug 2025).

The framework is positioned against three classes of evaluation practice. First, end-to-end QA performance conflates retriever, compressor, and reader behavior, making it difficult to isolate the value of the retrieval summary itself. Second, perplexity-based measures assess how typical a sequence is under the reader, not whether its semantics align with the reader’s useful latent structure. Third, mean-pooled embedding heuristics collapse a sequence into a centroid and underweight rare but semantically important boundary tokens. xCompress addresses this gap by operating between retrieval/compression and answer generation, and by selecting compressed summaries according to reader-side alignment rather than generic fluency or holistic task reward.

This design makes xCompress distinct from passage rerankers and from generic context compression. It is not ranking raw passages by query relevance alone, and it does not merely shorten context. Instead, it adaptively samples alternative compressions and uses a reader-aware criterion to choose among them. In the reported setup, the number of retrieved documents is unchanged; retrieval remains top-30, and the gains come from choosing better compressions of the same retrieved set.

2. Spectrum Projection Score

SPS is the formal core of xCompress. The reader representation space is defined by a matrix

$W\in\mathbb{R}^{D\times M},$

where $W$ can be the input embedding matrix or a bank of hidden states from the reader, $D$ is the representation dimension, and $M$ indexes columns, features, or states. The reader’s principal directions are extracted with singular value decomposition,

$W=U\Sigma V^\top.$

The method retains the top $95\%$ eigenvalues in $\Sigma$, denoted $\Sigma_p$, and reconstructs a projection object

$P=U\Sigma_p V^\top.$

Given a retrieval summary, the summary is run through the reader, token representations are extracted from the penultimate layer, and elementwise max pooling across the token dimension produces a salient summary vector

$\mathbf{x}\in\mathbb{R}^{D}.$

The Spectrum Projection Score is then defined as

$W$0

where $W$1 is the identity matrix, $W$2 is the component captured by the reader’s principal subspace, and $W$3 is the residual component orthogonal to that subspace. A smaller SPS means the summary vector lies more fully inside the reader’s principal representational geometry, and xCompress therefore selects the candidate with the lowest SPS (Hu et al., 8 Aug 2025).

The score is framed as an “area-based” alternative to token-likelihood metrics. Its appendix-level rationale uses the notion of a bounder vector: for a sequence of token embeddings $W$4, a vector $W$5 is a bounder vector if for every token $W$6 and dimension $W$7, $W$8. In practice, max pooling gives such a bounder vector. The appendix further defines a hyper-rectangle order: for two sequences with bounder vectors $W$9, one writes $D$0 if $D$1 for all coordinates $D$2. This is used to justify the claim that the max-pooled vector approximates the support or envelope of the token distribution and therefore preserves salient extent information that mean pooling misses. The appendix also states that if samples come from the same distribution, their bounder vectors converge in probability as sample size grows. This suggests that max pooling is being used as a stable proxy for semantic extent before projection into the reader spectrum.

3. Inference-time controller design

The xCompress pipeline begins with a query $D$3 and corpus $D$4. A retriever returns relevant passages $D$5; in experiments, the retriever is Contriever through BEIR, and the top 30 documents are retrieved. These top-$D$6 passages are then compressed into summaries. The framework supports both text-to-text compression and text-to-embedding compression. After an initial summary is produced, xCompress embeds it with the reader’s penultimate-layer states and computes latent-space statistics to decide whether additional sampling is worthwhile. If not, the initial summary is accepted. If so, the framework generates additional candidate compressions, computes SPS for each candidate, ranks them by SPS, and passes the lowest-SPS candidate to the reader for final answer generation (Hu et al., 8 Aug 2025).

In the text-to-text setting, xCompress uses a compressor LLM to summarize the retrieved passages. The prompt instructs the model to generate a summary under 200 words, avoid pronouns, and summarize only rather than answer. Instead of deterministic decoding such as greedy search or beam search, the method uses stochastic decoding to generate $D$7 diverse summary candidates per query. In the implementation, temperature is set to 1.0, repetition penalty to 1.2, and five summaries are generated per question. Each candidate is processed by the reader, penultimate-layer token representations are max-pooled into $D$8, SPS is computed, and the lowest-SPS candidate is selected.

In the text-to-embedding setting, xCompress builds on xRAG-style compression, where retrieved passages and the query are deterministically mapped to a summary embedding via a trained projector. Because the embedding compressor is deterministic, xCompress introduces stochasticity by probe injection. It samples $D$9 small Gaussian probe vectors $M$0, appends each to the summary–query embedding, and runs the fused representation through the reader. For each probe, it extracts the penultimate-layer hidden state at the probe position, denoted $M$1, and computes a diversity score. The printed formula is

$M$2

where $M$3 is the gap between the $M$4-th and $M$5-th largest elements of $M$6. The notation appears slightly malformed typographically, but the intended meaning is that $M$7 sums squared sorted gaps. Smaller $M$8 is said to indicate stronger semantic deviation from the existing summary–query signal. The framework keeps the $M$9 probes with the smallest scores and forms $W=U\Sigma V^\top.$0 candidate embedding summaries: the original one plus $W=U\Sigma V^\top.$1 probed variants. These candidates are then SPS-ranked exactly as in text-to-text mode.

Architecturally, the controller is intentionally training-free. SPS itself is supervision-free and requires no finetuning. In text-to-text mode, xCompress is layered on top of an existing summarization compressor and a reader LLM. In text-to-embedding mode, it sits on top of a preexisting embedding compressor like xRAG and perturbs that representation with probes. The reported setup uses CompAct as the text-to-text baseline compressor, xRAG as the text-to-embedding baseline, and readers based on LLaMA-3.1-8B-Instruct, Gemma3-12B-Instruct, Qwen3-8B, and Mistral-7B. Reader answer generation is done with greedy decoding at temperature 0.0 to remove answer-side randomness.

4. Adaptive sampling and theoretical rationale

A practical component of xCompress is its adaptive sampling mechanism, called norm-guided filtering. Because sampling multiple candidates for every query is inference-time expensive, the method first estimates whether a query is likely to benefit from SPS-guided exploration. For the initial summary, it computes the L2 norm of the mean-pooled representation, denoted $W=U\Sigma V^\top.$2, and the L1 norm of the max-pooled representation, denoted $W=U\Sigma V^\top.$3. Their ratio,

$W=U\Sigma V^\top.$4

acts as a concentration or stability signal. Higher values indicate a more skewed or concentrated information distribution and suggest that the summary is less likely to benefit from additional sampling, whereas lower values indicate a sparser distribution where further exploration may help. The threshold is estimated on a validation set by full sampling; in implementation, it is set to the top-30% value on the validation set (Hu et al., 8 Aug 2025).

At inference time, if the ratio exceeds the threshold, xCompress accepts the initial summary and skips additional sampling. Otherwise, it performs candidate generation and SPS-based selection. This adaptive gate is the main control logic by which the framework reduces its own overhead. The computational tradeoff is therefore explicit: answer quality is improved by doing more work at inference time, but the filter is introduced to skip that extra work when it is likely to be unnecessary.

The appendix gives a theoretical rationale for this filter. It introduces the ratio

$W=U\Sigma V^\top.$5

though the typesetting is slightly broken; the intended expression is the norm of the mean vector divided by the norm of the bounder vector. Under a Gaussian assumption on token embeddings, the sample mean converges to the true mean while the expected sample maximum grows approximately like $W=U\Sigma V^\top.$6, so the ratio shrinks toward zero as sample size increases:

$W=U\Sigma V^\top.$7

and

$W=U\Sigma V^\top.$8

hence

$W=U\Sigma V^\top.$9

The intended interpretation is that sparse, high-variance token distributions have larger boundary vectors relative to their means, making xCompress more useful there. A plausible implication is that norm-guided filtering functions as a heuristic for deciding when the boundary-based SPS view is likely to carry incremental information beyond a one-shot compression.

5. Empirical behavior

The experimental evaluation covers five QA benchmarks—HotpotQA, 2WikiMulti-hopQA, Natural Questions, TriviaQA, and Musique—and is designed both to validate SPS as a metric and to test xCompress as a downstream controller. To evaluate SPS directly, the method generates ten distinct summaries per query, scores each summary with PPL, LongPPL, and SPS, ranks summaries by each metric, bins them, and then measures downstream EM and F1. On LLaMA-3.1-8B-Instruct, SPS shows markedly stronger Pearson correlation and AUROC than PPL or LongPPL. On HotpotQA, for example, PCC with EM is $95\%$0 for SPS versus $95\%$1 for PPL and $95\%$2 for LongPPL, while PCC with F1 is $95\%$3 for SPS versus $95\%$4 and $95\%$5. On 2Wiki, PCC(EM) is $95\%$6 for SPS versus $95\%$7 and $95\%$8; on Natural Questions, PCC(EM) is $95\%$9 for SPS versus $\Sigma$0 and $\Sigma$1. AUROC is also consistently best for SPS, including $\Sigma$2 on HotpotQA and $\Sigma$3 on 2Wiki (Hu et al., 8 Aug 2025).

As a downstream system, xCompress improves QA performance over standard compression baselines. In text-to-text mode with LLaMA-3.1-8B-Instruct, CompAct obtains 34.0/43.17 EM/F1 on HotpotQA, while xCompress+SPS reaches 37.6/47.87; on Natural Questions, 35.2/47.49 rises to 39.4/51.18; on TriviaQA, 62.4/71.25 rises to 65.4/73.11. Similar gains appear for Gemma and often for Qwen. For Gemma3-12B-Instruct on HotpotQA, CompAct reaches 19.2/29.69 while xCompress+SPS reaches 25.2/35.66. In text-to-embedding mode with Mistral-7B, xRAG on TriviaQA gives 16.0/40.09, while xCompress+SPS gives 29.2/46.76. Across the large results table, xCompress+SPS is generally the best-performing variant relative to retrieval-direct, baseline compression, and xCompress variants that rank by PPL or LongPPL instead of SPS.

Several ablations isolate design choices. A pooling ablation shows that max pooling is important: with LLaMA on HotpotQA, max pooling gives 37.6/47.87, mean pooling 36.2/47.65, and last-token pooling 33.6/43.37. Layer analysis finds that the penultimate layer gives the best embeddings for SPS, while earlier layers lack semantic abstraction and final-layer states are too specialized to next-token prediction. The PCA retained variance ratio matters: performance peaks when the retained variance ratio is 0.95; lower ratios lose too much information, while 0.99 includes more noise. The number of generated summaries also shows diminishing returns, with performance saturating around five candidate summaries, which explains the implementation choice of five.

6. Limits, failure cases, and naming ambiguities

xCompress has several explicit limitations. It depends on access to the reader’s internal hidden states or embedding matrix, which makes it suitable for open-source or otherwise inspectable LLMs rather than arbitrary black-box APIs. It adds inference cost because it may generate or evaluate multiple candidates. Its adaptive filter threshold is estimated from a validation set, so the method is training-free but not entirely hyperparameter-free. The theoretical motivation around “area,” convex hulls, and bounder vectors is only partially translated into the final SPS implementation; operationally, the score is a residual norm after projection, and the projection notation in the paper is somewhat loose (Hu et al., 8 Aug 2025).

The method also does not alter the top-$\Sigma$4 retrieval depth in the reported experiments. Its gains come from choosing better compressions of the same retrieved set, not from finding new evidence. If the retriever misses the necessary evidence entirely, xCompress cannot recover it. This suggests that xCompress is best understood as a reader-aware control layer rather than a replacement for retrieval.

A reported robustness limit appears with Qwen3 on HotpotQA and Natural Questions, where xCompress+PPL can slightly exceed xCompress+SPS. The authors attribute this to model overconfidence and probable memorization or contamination: Qwen has much lower answer entropy than LLaMA on these datasets, with average entropy 0.46 vs. 1.49 on HotpotQA and 0.38 vs. 1.16 on Natural Questions. In such cases, perplexity may accidentally reward familiar memorized contexts rather than genuinely better alignment. The exception is therefore presented as a caution about dataset-specific memorization artifacts rather than as a general failure of reader-space alignment.

The name should also not be conflated with visually similar systems in other domains. In particular, "Comp-X" (Gao et al., 21 Aug 2025) denotes an interactive learned image compression framework with an LLM-based coding agent, not the RAG controller described here. The two names are close in form, but the methods address different technical problems, use different architectures, and are not identified as the same system in the cited work.