Knowledge Composition Sampling (KCS)

• Knowledge Composition Sampling is a framework that selects alternative multi-hop evidence chains from long contexts to diversify question generation.
• It separates evidence selection and question realization using a BERT-based encoder and a Transformer sequence model to enhance generation quality.
• Empirical results demonstrate that KCS improves QA performance and logical coherence while offering a cost-effective alternative to graph-based methods.

Knowledge Composition Sampling (KCS) is a framework for conditional generation in which diversity is introduced by sampling a structured intermediate representation of content before surface realization. In its most specific formulation, KCS denotes a method for diversifying multi-hop question generation by sampling alternative sentence-level knowledge compositions from a long context and conditioning question generation on those sampled evidence chains (Wang et al., 28 Aug 2025). The same plan-first intuition has a direct antecedent in Composition Sampling for summarization and question generation, where the intermediate composition is an entity chain rather than a sentence sequence (Narayan et al., 2022). A separate educational-technology literature uses the term “knowledge components” for fine-grained skills linked to assessment items; that usage is related only in the broad sense of automatically generating and organizing knowledge units, not in the sense of the multi-hop question generation framework (Moore et al., 2024).

1. Definition and scope

In the multi-hop question generation setting, KCS treats a knowledge composition $c = [f_1, \dots, f_K]$ as a subset of sentences drawn from a long context $D = [s_1, \dots, s_M]$, where the selected sentences together form a multi-hop evidence chain. Rather than assuming that each example has a single fixed evidence composition, KCS explicitly models and samples multiple compositions $C = \{c_1, \dots, c_{N_q}\}$ from the same context and answer, and then uses those compositions to generate multiple questions (Wang et al., 28 Aug 2025).

The motivation is tied to two properties of multi-hop QA data. First, datasets such as HotpotQA and 2WikiMultihopQA typically annotate only one gold chain of supporting facts per question even though the underlying context may contain many plausible multi-hop chains that lead to the same answer. Second, prior work on diverse question generation often concentrates on token-level content planning or expression-level variation while treating the relevant context as already fixed. KCS repositions diversity at the knowledge level: which sentences are selected, and in what sequence, becomes the primary locus of variation.

This emphasis matters because KCS is not merely a stochastic paraphrasing method. It is a sentence-level conditional prediction framework over evidence chains. The generated questions differ because they are grounded in different sampled knowledge compositions, not only because the decoder chooses different surface forms.

2. Formalization of knowledge compositions

The standard multi-hop question generation formulation often models $p(q \mid c, a)$, where $q$ is the question, $c$ is a fixed context, and $a$ is the answer. KCS instead treats the composition itself as a random variable over subsets of sentences in the full long context $D$, and writes the problem as

$$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$

Here, $D = [s_1, \dots, s_M]$ is the long context, $D = [s_1, \dots, s_M]$0 is the answer, and $D = [s_1, \dots, s_M]$1 is a sequence of $D = [s_1, \dots, s_M]$2 selected sentences. The framework therefore separates knowledge composition selection from question realization (Wang et al., 28 Aug 2025).

KCS models composition selection as sentence-level conditional sequence prediction with an autoregressive structure,

$D = [s_1, \dots, s_M]$3

where $D = [s_1, \dots, s_M]$4 is the answer treated as the zeroth piece of knowledge, and $D = [s_1, \dots, s_M]$5 denotes per-sentence question-worthiness labels. This factorization turns multi-hop evidence selection into a sequence model over sentence indices rather than tokens.

The sentence representations are produced by a BERT-based encoder $D = [s_1, \dots, s_M]$6, with each sentence $D = [s_1, \dots, s_M]$7 mapped to a latent vector $D = [s_1, \dots, s_M]$8. For the context, this yields

$D = [s_1, \dots, s_M]$9

A Transformer-based sentence-level encoder–decoder $C = \{c_1, \dots, c_{N_q}\}$0 then computes contextualized sentence states $C = \{c_1, \dots, c_{N_q}\}$1, predicts sentence-level question-worthiness probabilities $C = \{c_1, \dots, c_{N_q}\}$2, and produces a decoder-side latent state $C = \{c_1, \dots, c_{N_q}\}$3 for the next-hop prediction. The classification signal is injected into cross-attention through modified keys

$C = \{c_1, \dots, c_{N_q}\}$4

where $C = \{c_1, \dots, c_{N_q}\}$5 and $C = \{c_1, \dots, c_{N_q}\}$6. This biases attention toward sentences classified as knowledge-worthy. Conceptually, the next-sentence distribution $C = \{c_1, \dots, c_{N_q}\}$7 is a softmax-like distribution over candidate sentences in $C = \{c_1, \dots, c_{N_q}\}$8.

3. Architecture and learning objectives

The KCS framework has three components: a sentence encoder $C = \{c_1, \dots, c_{N_q}\}$9, a sentence-level sequence model $p(q \mid c, a)$0, and a multi-hop question generation model. The sentence encoder is BERT-based, with BERT-base, BERT-large, and RoBERTa-large variants explored. The sequence model is a Transformer encoder–decoder operating at sentence level. The downstream question generator is a vanilla MixQG-base Transformer (Salesforce/mixqg-base) (Wang et al., 28 Aug 2025).

Training data for composition selection is constructed from gold supporting facts in HotpotQA and 2WikiMultihopQA. The paper defines a heuristic order called “document” arrangement: documents containing the answer are prioritized, sentence order is maintained within a document, and the sentence containing the answer is used to split and reorder sequences. This heuristic determines the training-time order of gold knowledge compositions.

The training objective combines a sentence-level classification loss $p(q \mid c, a)$1 with a probabilistic contrastive sequence loss $p(q \mid c, a)$2,

$p(q \mid c, a)$3

with $p(q \mid c, a)$4 in the experiments. The contrastive term is inspired by Contrastive Predictive Coding and is designed to learn temporal coherence across hops by maximizing the mutual information between the decoder prediction state $p(q \mid c, a)$5 and the true next sentence representation $p(q \mid c, a)$6 relative to negatives from the rest of the context. In implementation, the mutual information function is instantiated as cosine similarity. The sequence loss is written as

$p(q \mid c, a)$7

This objective gives KCS two distinct inductive biases. The classification branch estimates which sentences are individually question-worthy, while the sequence branch learns how those sentences chain together across hops. The paper’s ablations show that both contributions matter: removing the decoder and selecting purely from classification probabilities sharply reduces performance, and omitting the integration of classification probabilities into decoder cross-attention also lowers F1 (Wang et al., 28 Aug 2025).

Knowledge composition selection is optimized with AdamW using learning rate $p(q \mid c, a)$8 and linear warmup ratio $p(q \mid c, a)$9. The question generator is fine-tuned with standard cross-entropy loss. For downstream QA experiments, Llama3.1-8B and Qwen2.5-7B are fine-tuned with LoRA using learning rate $q$0, cosine warmup $q$1, LoRA rank $q$2, and LoRA alpha $q$3.

4. Sampling, decoding, and the interface to question generation

At inference time, KCS applies stochastic decoding not to output tokens but to the sentence-level composition. Starting from an empty composition $q$4, the model predicts $q$5 over sentences at each hop $q$6. It then constructs the top-$q$7 nucleus $q$8, the smallest set of sentences whose cumulative probability mass reaches $q$9, rescales probabilities over that set, and samples the next sentence from the renormalized distribution. In the experiments, $c$0 (Wang et al., 28 Aug 2025).

This procedure is summarized as Algorithm 1, “Diversify Multi-hop QG.” Its purpose is to balance correctness and variation. Greedy decoding repeatedly selects the same high-probability chain and therefore has low diversity. Unrestricted sampling exposes the model to the unreliable tail of the sentence distribution. Nucleus sampling truncates that tail while retaining stochasticity among plausible next-hop sentences.

Once a composition $c$1 has been sampled, the question generator receives $c$2 and produces a question $c$3. For data augmentation, the paper samples $c$4 compositions per example with $c$5, producing five generated questions from the same context–answer pair. The question generation model itself is unchanged; KCS functions as a pre-step that provides structured, knowledge-level inputs to the MHQG model.

This design differentiates KCS from graph-based multi-hop QG systems. Previous methods often rely on pre-constructed knowledge graphs or entity graphs and determine compositions through graph structure or graph paths. KCS instead operates directly on unstructured text, selecting coherent sentence-level chains without graph construction or entity extraction. It also differs from purely expression-level diversification methods: the principal variation is in the sampled evidence chain, not merely in wording.

5. Empirical results and analytical findings

The central quantitative claim is that KCS improves the overall accuracy of knowledge composition selection by 3.9% relative to competitive baselines (Wang et al., 28 Aug 2025). On HotpotQA, the BERT-base classification baseline achieves F1@2 $c$6 and F1@3 $c$7, while KCS with BERT-base reaches F1@2 $c$8 and F1@3 $c$9. On 2WikiMultihopQA, the same comparison is larger: CLS achieves F1@2 $a$0 and F1@3 $a$1, whereas KCS reaches F1@2 $a$2 and F1@3 $a$3. The paper emphasizes that performance improves when $a$4 increases from $a$5 to $a$6, which it interprets as evidence that the sequence model has learned coherent multi-hop transitions.

The downstream QA results are stronger still. On Llama3.1-8B for HotpotQA, the original training set gives F1 $a$7, augmentation with Composition gives F1 $a$8, and augmentation with KCS gives F1 $a$9 with EM $D$0. On 2WikiMultihopQA, KCS reaches F1 $D$1, compared with $D$2 for Composition and $D$3 for RAST. On Qwen2.5-7B, HotpotQA improves from F1 $D$4 with the original data and $D$5 with Composition to $D$6 with KCS; on 2WikiMultihopQA, the corresponding numbers are $D$7, $D$8, and $D$9 (Wang et al., 28 Aug 2025).

The diversity analysis shows that KCS does not maximize diversity in the same way as the most stochastic generators, but it improves materially over entity-chain Composition while remaining useful for QA augmentation. On HotpotQA, Composition has Pairwise-BLEU $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$0, LLM-Diversity $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$1, and Consistency $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$2; RAST has $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$3, $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$4, and $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$5; GPT-4 has $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$6, $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$7, and $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$8; and KCS has $$p(q \mid D, a) = \mathbb{E}_{c}\left[p(q \mid c, a)\times p(c \mid D, a)\right].$$9, $D = [s_1, \dots, s_M]$0, and $D = [s_1, \dots, s_M]$1. The paper characterizes KCS as occupying a middle regime: much higher diversity than Composition, lower cost than GPT-4, and substantially stronger downstream QA performance than the other augmentation baselines.

Several ablations clarify the mechanism. A decoder-only variant reaches F1@1 $D = [s_1, \dots, s_M]$2, F1@2 $D = [s_1, \dots, s_M]$3, and F1@3 $D = [s_1, \dots, s_M]$4 on HotpotQA, while the encoder–decoder with classification reaches $D = [s_1, \dots, s_M]$5, $D = [s_1, \dots, s_M]$6, and $D = [s_1, \dots, s_M]$7. A purely encoder-based classifier without the decoder falls to F1@1 $D = [s_1, \dots, s_M]$8, F1@2 $D = [s_1, \dots, s_M]$9, and F1@3 $D = [s_1, \dots, s_M]$00, indicating that sequence modeling is essential. The default “document” ordering of training compositions performs best; “original,” “shuffle,” “sorted,” “cluster,” and “cropping” all reduce performance, with up to approximately 17-point degradation in F1@1. Post-encoder concatenation of the answer representation slightly improves performance, increasing Transformer layers and attention heads does not help, halving $D = [s_1, \dots, s_M]$01 reduces performance, and removing classification-score modulation from the decoder’s keys lowers F1 by about $D = [s_1, \dots, s_M]$02–$D = [s_1, \dots, s_M]$03 points (Wang et al., 28 Aug 2025).

The logical coherence study reports that KCS compositions achieve approximately 91.55% and 93.32% of ground-truth LLM scores on HotpotQA and 2WikiMultihopQA, respectively, while providing five compositions per example. The reported error categories are illogical compositions, superficially similar but distinct facts, and pronoun-induced ambiguity. A notable case study shows that some failures arise not from the selected knowledge chain itself but from the vanilla MHQG model’s inability to realize the selected chain consistently.

6. Relation to earlier composition sampling and adjacent uses of “KC”

The conceptual precursor to KCS is Composition Sampling for diverse conditional generation. That method factors conditional generation through a latent composition $D = [s_1, \dots, s_M]$04, written as

$D = [s_1, \dots, s_M]$05

samples a composition using nucleus sampling, and then generates the final text via beam search,

$D = [s_1, \dots, s_M]$06

In the 2022 formulation, the composition is an entity chain extracted from the target summary or question, optionally segmented by sentence with ||| markers and prefixed by [content], while the realized text is prefixed by [summary]. The model is built on the plan-based neural generation framework of Narayan et al. (2021, “FROST”), and the planner and generator are two phases of the same decoder rather than separate models (Narayan et al., 2022).

Composition Sampling is relevant because it established the broader plan-first principle later specialized by KCS. On XSum and CNN/DailyMail, direct nucleus sampling produces large ROUGE-L drops relative to beam search, whereas Composition Sampling produces smaller drops: on XSum, Frost ROUGE-L declines from $D = [s_1, \dots, s_M]$07 to $D = [s_1, \dots, s_M]$08 under Nucleus(Frost) but only to $D = [s_1, \dots, s_M]$09 under Composition(Frost); on CNN/DailyMail, the decline is from $D = [s_1, \dots, s_M]$10 to $D = [s_1, \dots, s_M]$11 for Nucleus(Frost) and to $D = [s_1, \dots, s_M]$12 for Composition(Frost). Faithfulness also improves under plan-level sampling, especially with Frost$D = [s_1, \dots, s_M]$13, which constrains entity chains to entities present in the input. The proposed combined metric Edna reaches $D = [s_1, \dots, s_M]$14 on XSum and $D = [s_1, \dots, s_M]$15 on CNN/DailyMail for Composition(Frost$D = [s_1, \dots, s_M]$16), the best values in the diverse block. On SQuAD question generation, Composition(Frost$D = [s_1, \dots, s_M]$17) obtains Top-1 $D = [s_1, \dots, s_M]$18, Top-5 $D = [s_1, \dots, s_M]$19, and Self-BLEU $D = [s_1, \dots, s_M]$20, outperforming previous diverse decoding baselines on Top-1 and Top-5 accuracy (Narayan et al., 2022).

The relation between the two frameworks is therefore precise. Composition Sampling diversifies conditional generation by sampling entity-level semantic plans; KCS diversifies multi-hop question generation by sampling sentence-level evidence chains. Both separate a knowledge-selection stage from a realization stage, and both use nucleus sampling to explore plausible intermediate structures while avoiding low-probability surface-level degeneration.

There is also a terminological ambiguity around the abbreviation KC. In educational measurement, KC means knowledge component, a fine-grained skill or competency associated with assessment items. GPT-4 has been used to generate such KC labels from multiple-choice questions and to induce ontologies that cluster questions by underlying KC. In that work, the best LLM strategy matches human KC labels for $D = [s_1, \dots, s_M]$21 of Chemistry MCQs and $D = [s_1, \dots, s_M]$22 of E-Learning MCQs, with higher top-five success, and human evaluators prefer LLM-generated KCs over human-assigned ones in about two-thirds of mismatched cases (Moore et al., 2024). This usage is related in spirit to automatic knowledge-unit discovery, but it is not the same method as KCS for multi-hop question generation.

Formulation	Intermediate structure	Primary use
Composition Sampling	Entity chain	Diverse summarization and question generation
KCS	Sentence-level knowledge composition	Diverse multi-hop question generation and QA data augmentation
KC generation for assessment	Textual KC labels and induced ontology	Assessment tagging and clustering

Within the broader literature, KCS is positioned against several neighboring lines of work. It extends question-worthy sentence selection beyond single-hop classification, it offers an alternative to graph-based multi-hop QG methods associated with Pan et al., Fei et al., and Kumar et al., it complements expression-level diversification such as RAST by Gou et al., and it imports a CPC-style contrastive learning perspective from Oord et al. into evidence-chain selection (Wang et al., 28 Aug 2025).

7. Limitations, assumptions, and prospective extensions

The current KCS formulation is constrained by its data assumptions. It is evaluated on HotpotQA and 2WikiMultihopQA, both of which provide relatively clean sentence segmentation and annotated supporting facts. The model relies on those supporting facts to construct gold training compositions, so it may inherit annotation biases even as it attempts to reduce spurious patterns by exploring alternative chains. Sentence-level encoding, encoder–decoder sequence modeling, and contrastive loss over all sentences are also computationally heavy on very long contexts, and sampling multiple compositions per example multiplies generation cost (Wang et al., 28 Aug 2025).

The model design also leaves known headroom. Only cosine similarity is used for the mutual-information function in the contrastive loss. The paper explicitly points to other MI estimators, different scoring functions, and architectural modifications such as MoE-LoRA as in GraphMoE as possible future directions. It also notes that the vanilla MHQG model limits consistency: some inconsistent generated questions are attributable to the realization model rather than to the selected knowledge composition itself.

The earlier Composition Sampling framework has a different set of limitations. Because its compositions are entity chains extracted with spaCy, performance depends on entity extraction quality. Mis-tagged or missing entities can distort the plan. On abstractive datasets such as XSum, Frost$D = [s_1, \dots, s_M]$23 can diverge from references because it restricts compositions to entities present in the input, which may lower ROUGE while improving factual faithfulness. The 2022 work also explores only one type of composition—entity chains—leaving discourse skeletons, keyphrase chains, topic sequences, and structured knowledge graphs as open alternatives (Narayan et al., 2022).

These limitations help clarify the broader significance of KCS. The strongest shared insight across the relevant papers is that conditional generation can often benefit from an explicit separation between knowledge selection and surface realization. In the 2022 work, that separation appears as entity-chain sampling followed by beam-search realization. In the 2025 work, it appears as sentence-level knowledge-composition sampling followed by multi-hop question generation. This suggests a general design pattern for tasks in which multiple coherent latent plans can support the same output, including evidence selection for explainable QA, multi-hop answer explanation, data-to-text generation, dialogue, story generation, machine translation with content plans, and other high-level representation prediction problems (Wang et al., 28 Aug 2025).