Highlight & Summarize (H&S) Pattern

• Highlight & Summarize (H&S) is a design pattern that separates the retrieval process by using a highlighter to extract trusted passages and a summarizer to generate answers without user query exposure.
• The pipeline partitions the task by first using various highlighter implementations—ranging from zero-shot LLMs to fine-tuned extractive models—to securely and accurately mark relevant text from the knowledge base.
• Empirical evaluations show that H&S improves answer correctness and robustness by limiting the attack surface, ensuring that the summarizer only processes pre-validated highlights.

Highlight & Summarize (H&S) is a design pattern for retrieval-augmented generation (RAG) systems where the answer generation task is partitioned into two components: a highlighter, which extracts relevant passages from retrieved documents based on the user’s query, and a summarizer, which fuses these highlights into a final answer—without ever exposing the user’s question to the generative LLM. This architecture is motivated by the need to neutralize jailbreaking and model hijacking attacks, wherein adversaries craft malicious prompts that manipulate the LLM into producing undesirable or unauthorized output. H&S delivers not only improved correctness but also strong built-in security guarantees by construction, as demonstrated through empirical evaluation against standard RAG systems (Cherubin et al., 4 Aug 2025).

1. H&S Pipeline: Concept and Motivation

The central innovation is the decoupling of the RAG pipeline into separate highlighting and summarization stages. In standard RAG, the LLM receives a concatenated prompt containing both the user’s query and retrieved textual evidence, making the system vulnerable to prompt injection: adversarial queries can directly manipulate generative behavior. H&S interrupts this channel by ensuring that the generative summarizer is only provided with pre-extracted, trusted passages ("highlights") and never the original user input. This invariant means the summarizer cannot be coerced to generate undesirable content via adversarial user instructions.

This design pattern ensures that:

• The highlighter is solely responsible for examining user intent and marking relevant document spans.
• The summarizer synthesizes an answer constrained by and rooted in the pre-highlighted, knowledge-base-derived content.
• The only "attack surface" is the trusted knowledge base and the highlighter, which can be tightly controlled and post-processed.

2. Implementation Details and H&S Instantiations

Multiple highlighter implementations are detailed, ranging from LLM-based prompt extractors to fine-tuned extractive QA models:

• H Baseline: Uses a zero-shot LLM to select relevant text from the retrieved documents. Since the LLM’s output may paraphrase or reorder the source, fuzzy string matching (e.g., via RapidFuzz with a threshold of 95) is done to align extractions with contiguous substrings from the sources.
• H Structured: Requests the LLM highlighter to output a structured JSON object with fields for "answer" and "text_extracts", enforced using Azure OpenAI’s structured output features. This enhances traceability and reliability of passage extraction.
• H DeBERTaV3: Applies an extractive QA model (DeBERTaV3) fine-tuned on SQuAD2 or the RepliQA dataset. The RepliQA variant is trained to select longer, contiguous gold-standard passages, yielding comprehensive highlights.

The generic H&S workflow is as follows:

1. Retrieve documents $D$ using the user's question $Q$.
2. Compute highlights $P = H(D, Q)$ using the chosen highlighter $H$.
3. Summarization $A = S(P)$, where the summarizer $S$ never observes $Q$.
4. The answer $A$ is returned, with provenance rooted in $D$ but with complete isolation from $Q$ at the generation stage.

This architecture—when instantiated with LLM-based highlighters and trusted document retrieval—constrains the generator to faithful, in-domain responses.

3. Security and Adversarial Robustness

H&S fundamentally prevents prompt injection and jailbreaking because the generative LLM is never exposed to adversarially controlled queries. Experiments on the LLMail-Inject jailbreak dataset show that while a standard RAG pipeline and even the standalone highlighter might occasionally "break," the full H&S pipeline is robust: tool calls or unauthorized action triggers through the generative step are eliminated. The sparse, audit-able surface (the knowledge base and highlight selection) further allows system designers to:

• Use fuzzy matching to guarantee all summarized content is a strict substring of trusted retrieved text, preventing the introduction of adversarial snippets not originally present.
• Restrict highlighter outputs to answer only the original, information-seeking intent, as structured outputs can be validated for integrity before entering the summarization stage.

The model reduces the problem of detecting adversarial prompt patterns—a hard, open problem in natural language attacks—to a manageable string search problem over the knowledge base.

4. Response Quality and Evaluation

Empirical evaluations on benchmark QA tasks (RepliQA, BioASQ, and others) demonstrate that, contrary to expectations, the majority of H&S responses—especially using LLM-based structured highlighters—are rated superior to those from standard RAG pipelines. Key evaluation components include:

• Token-based recall: The proportion of reference answer tokens recovered in the model’s output.
• K-Precision: The proportion of generated tokens also present in the “gold passage” from documents.
• LLM-as-a-judge metrics: Including Multihop Correctness and relevance/quality scales (MTBench, Reliable CI Relevance).
• Direct comparison: Pairwise head-to-head (Elo-rated) comparisons across RAG baselines, revealing that H Structured responses are more frequently correct, helpful, and relevant.

Trade-offs are observed: LLM-based highlighters (particularly those using structured outputs or longer passage selection) offer improved security and response interpretability but incur higher latency ($\sim$3s per question vs. $<$1s for standard RAG).

5. Coverage, Limitations, and Attack Surface

H&S ensures that model-generated answers are grounded in retrievable, trusted information. However, limitations remain:

• Recall dependency on highlighter: If the highlighter fails to extract all semantically necessary information, the summarizer cannot compensate—leading to incomplete answers.
• Adversarial highlighting and incomplete coverage: Attackers might attempt to game the highlighter (if they control the knowledge base) by constructing misleading or partial evidence snippets. A plausible implication is that minimum span length requirements and context-enriched extraction may improve defense.
• Guessed question validation: Since the summarizer can be asked to output a “guessed question” (as a form of intent reverse-engineering), future systems may compare this with the true question for further security or refuse to answer if the gap is too large.

6. Broader Implications and Future Work

The H&S design pattern has ramifications for secure LLM deployment and QA system construction:

• Directly addressing model hijacking attack vectors by permanently separating the adversary-controlled signal (user question) from the generative model input.
• Simplifying security auditing via string matching on a finite set of trusted highlights, compared to the open-ended problem of adversarial prompt defense in traditional architectures.
• Mitigating hallucination by ensuring that generation is restricted to information present in the trusted documents; although empirical confirmation is pending, this suggests reduced propensity for unsupported outputs.

The authors identify further research directions:

• Automating adversarial chunk detection in the knowledge base.
• Extending the H module for new query types (yes/no, multi-part), enforcing minimum highlight granularity.
• Comparing guessed question and original question as a defense filter.
• Enhancing highlighter recall and summarizer grounding fidelity.

7. Comparative Summary

Aspect	H&S Pipeline	Standard RAG Pipeline	Security Implication
User prompt to LLM	No	Yes	Prevents prompt injection/jailbreaking
Source grounding	Enforced via highlights	May be indirect or omitted	Increased answer faithfulness
Adversarial attack	Knowledge-base only	Open input and prompt space	Attack surface reduced and easily auditable
Empirical response	Improved correctness/relevance	Baseline	Most H&S responses rated superior

In conclusion, H&S provides a principled redesign of RAG architectures for security, interpretability, and improved answer quality, offering an effective approach for building robust LLM-based QA systems in adversarial and high-stakes settings (Cherubin et al., 4 Aug 2025).