Frequency Framed Prompting (REFER)

Updated 12 July 2026

Frequency Framed Prompting (REFER) is a framework that replaces abstract fairness instructions with explicit count-based reasoning over opinions in a defined reference class.
It employs a two-stage process—explicit counting via reference class specification, numerical anchoring, and sequential deliberation followed by balanced summarisation—to ensure fairness.
Empirical findings show that REFER improves second-order fairness metrics in review and tweet summarisation, especially when used with larger models and structured reasoning prompts.

Frequency Framed Prompting, abbreviated REFER, is a prompting framework that elicits LLM reasoning using frequency-based information—counts of opinions over an explicit reference class—rather than abstract probability language or generic instructions to be “fair” or “balanced.” In its canonical formulation, the model is asked first to determine how many input documents express each social value or stance, and only then to generate a summary that reflects those frequencies (Huang et al., 19 Sep 2025). Within opinion summarisation, REFER is intended to mitigate systematic bias in how summaries represent majority and minority viewpoints; more broadly, it belongs to a family of prompting methods that treat prompt structure, explicit counting, and staged reasoning as first-class control variables rather than incidental phrasing choices (Huang et al., 19 Sep 2025).

1. Definition and conceptual basis

REFER is defined as Fre*requency Framed Pr*ompting, a framework that replaces abstract fairness instructions with explicit reasoning over counts such as “how many reviews out of $N$ are positive versus negative,” where $N$ is the actual number of source documents in the input collection (Huang et al., 19 Sep 2025). The core representation is frequency-based rather than probability-based. If social values are denoted $v_1, \ldots, v_k$ , REFER operates over counts $n_i$ in a reference class of size $N$ , with the paper giving the conceptual mapping

$p_i \approx \frac{n_i}{N}, \qquad \sum_{i=1}^k n_i = N.$

The prompting emphasis is not on supplying the model with ground-truth $p_i$ , but on instructing it to infer and use $n_i$ directly from the input (Huang et al., 19 Sep 2025).

The framework is grounded in cognitive science results on natural frequencies. The motivating claim is that frequency formats make reference classes explicit, reduce cognitive load, and encourage sequential analysis of counts before qualitative judgment. In the REFER formulation, these effects are translated into prompt design through three mechanisms named explicitly in the source: Reference Class Specification, Numerical Anchoring, and Sequential Deliberation (Huang et al., 19 Sep 2025). The first fixes the denominator; the second introduces concrete count tokens; the third enforces a two-stage reasoning path in which counting precedes summarisation.

REFER also presupposes a specific notion of fairness in opinion summarisation. A fair summary should represent both majority and minority stances and preserve the distribution of opinions in the source rather than amplifying one side. The motivating problem is not merely omission, but systematic second-order bias: deviations from the source distribution can be patterned rather than random, such as consistently down-weighting one political side or one sentiment polarity (Huang et al., 19 Sep 2025).

2. Prompting framework and formal variants

The baseline prompt in the REFER study is a standard direct summarisation instruction with no distributional guidance: reviews or tweets are presented, separated by delimiters, and the model is asked to produce a short summary (Huang et al., 19 Sep 2025). REFER modifies this by inserting a frequency-analysis stage before summarisation. Its simplest form is:

“Let's first determine how many reviews out of {number of input in the source documents} are {social value 1, social value 2}. Then, let's generate a balanced summary that accurately reflects the frequency of these opinions. {direct prompt}” (Huang et al., 19 Sep 2025)

This formulation introduces three structural constraints. First, it specifies the reference class size using the actual input cardinality. Second, it names the target opinion categories. Third, it imposes an explicit temporal order: first count, then summarise (Huang et al., 19 Sep 2025).

The paper evaluates five prompt families that integrate this counting stage with different reasoning or control signals.

Variant	Characteristic instruction
Plain REFER	Count opinions out of $N$ , then summarise
Prefix-Instruct-R	Add “Be fair with your output” before counting
Prefix-Role-R	Add “You are an unbiased person” before counting
CoT-R	Add “Let’s think step by step” before counting
Agent-R	Use summarisation, frequency, judge, and senior editor agents

In Prefix-Instruct-R, fairness language is explicitly prepended to the count-then-summarise structure. In Prefix-Role-R, the same structure is combined with persona prompting. In CoT-R, the instruction “Let’s think step by step” is used to reinforce sequential deliberation. In Agent-R, the framework is expanded into a four-role workflow consisting of a Summarisation Agent-REFER, a Frequency Agent, a Judge Agent, and a Senior Editor Agent, with the frequency agent producing counts in a fixed format such as “{social value 1 #number, social value 2 #number}” (Huang et al., 19 Sep 2025).

An oracle variant is also included. In that condition, the prompt supplies the actual ground-truth frequencies and asks for a summary reflecting them. This serves as a theoretical upper bound on what happens when exact counts are available in the prompt (Huang et al., 19 Sep 2025).

A broader methodological interpretation follows from work on prompt reframing. Reframing studies show that itemized instructions, explicit sequential steps, and constrained output formats can materially improve instruction following across models and tasks, including cases where reframed prompts outperform raw prompts despite far fewer examples (Mishra et al., 2021). This suggests that REFER’s effectiveness is not only semantic—reasoning about counts—but also structural: it reframes the summarisation task into a pattern of explicit substeps that better matches the instruction-following preferences of current LLMs.

3. Experimental setting and fairness metrics

The REFER study evaluates two summarisation tasks. The first is review summarisation, based on Amazon Reviews 2023, where each test collection contains 8 reviews of 30–120 words. The second is political tweet summarisation, based on FairSumm and a curated political tweets dataset, where each test collection contains 30 tweets labelled by political stance (Huang et al., 19 Sep 2025).

For both tasks, the experimental design uses 900 input collections per dataset: 300 balanced, and 300 skewed with 75/25 proportions in each direction. This setup is intended to test whether summaries preserve the input stance distribution under both balanced and asymmetric source mixtures (Huang et al., 19 Sep 2025). The evaluated models are GPT-3.5-Turbo, GPT-4o-mini, GPT-4o, Llama-3.1-8B-Instruct, and Llama-3.1-70B-Instruct, all in zero-shot prompting mode with fixed hyperparameters: max new tokens $=256$ , temperature $N$ 0, repetition penalty $N$ 1 (Huang et al., 19 Sep 2025).

Fairness is operationalised distributionally. The central question is how closely the stance distribution of the summary matches the stance distribution of the source. The paper distinguishes first-order fairness—whether groups are represented at all and whether minority opinions appear—from second-order fairness, which concerns whether the relative differences between groups are preserved (Huang et al., 19 Sep 2025).

Four metrics are used:

Metric	Intended measurement
SPD	Second-order fairness mismatch
BUR	Proportion of summaries judged unfair
UER	Under-representation error for minority opinions
SOF	Variance of unfairness across social values

The paper states that Second-Order SPD (SPD) is taken from Huang et al. 2024 and compares summary and source distributions over social values, with lower values indicating better preservation of second-order fairness. BUR, UER, and SOF are taken from Zhang et al. 2023, with lower values again indicating better fairness (Huang et al., 19 Sep 2025). Because summaries can contain compound sentences expressing multiple viewpoints, they are first decomposed into simple propositions using GPT-4o-mini before classification and metric computation (Huang et al., 19 Sep 2025).

This metric design is significant for understanding REFER. The framework is not evaluated primarily as a style intervention or a coherence intervention, but as a mechanism for controlling distributional faithfulness in generated summaries. That emphasis distinguishes it from prompt designs that optimize summary informativeness without explicit regard to opinion proportions.

4. Empirical findings and model dependence

The central empirical result is that REFER improves fairness metrics in opinion summarisation, with especially strong gains for larger models and when combined with stronger reasoning instructions (Huang et al., 19 Sep 2025). In the review summarisation setting, the paper reports, for GPT-3.5-Turbo under direct prompting, $N$ 2, $N$ 3, $N$ 4, and $N$ 5. Relative to this baseline, plain REFER improves SPD by $N$ 6 and SOF by $N$ 7, while BUR and UER slightly worsen by $N$ 8 and $N$ 9, respectively. By contrast, Prefix-Instruct-R improves all four metrics relative to Prefix-Instruct, and CoT-R yields across-the-board improvements relative to CoT, including SPD $v_1, \ldots, v_k$ 0, BUR $v_1, \ldots, v_k$ 1, UER $v_1, \ldots, v_k$ 2, and SOF $v_1, \ldots, v_k$ 3 (Huang et al., 19 Sep 2025).

The broader pattern is that CoT-R, Prefix-Instruct-R, and Prefix-Role-R generally reduce SPD, BUR, UER, and SOF for GPT-3.5-Turbo, GPT-4o-mini, and GPT-4o in review summarisation (Huang et al., 19 Sep 2025). In political tweet summarisation, the task is explicitly described as harder: absolute SPD, BUR, UER, and SOF values are larger, meaning bias is more severe overall. REFER still yields notable improvements, especially in the second-order metrics SPD and SOF, but the gains are smaller and can be accompanied by slight regressions in BUR or UER (Huang et al., 19 Sep 2025).

The paper further reports that the oracle condition with actual frequencies produces the lowest SPD and SOF, confirming that exact count information is a strong upper bound (Huang et al., 19 Sep 2025). Figures and appendix tables show SPD reductions often in the 5–20% range and SOF reductions of 5–10% relative to base prompts for larger models in review summarisation; improvements also occur for political tweets, though less uniformly (Huang et al., 19 Sep 2025).

Model size and instruction-following ability are decisive. Larger models such as GPT-4o and Llama-3.1-70B-Instruct are described as better able to follow complex instructions and therefore benefit more from REFER. Smaller models such as Llama-3.1-8B-Instruct often ignore the counting step and show high variability in summary length and structure, which reduces REFER’s effectiveness (Huang et al., 19 Sep 2025). This dependence on instruction-following strength aligns with a separate line of work showing that prompt topology matters independently of content: when identical content is reordered into non-sequential “jumping” structures, instruction-following accuracy in multistep prompts can collapse by up to 72% relative to baseline, with current architectures behaving as though instruction following were a sequential pattern rather than a reasoning skill (Jaffe et al., 26 Jan 2026). A plausible implication is that REFER works best when its count-then-summarise stages remain simple, linear, and explicitly ordered.

5. Mechanisms, interpretation, and scope conditions

The REFER paper attributes its improvements to the same three cognitive mechanisms that motivate the framework. Reference class specification directs the model to think in terms of a concrete denominator such as 8 reviews or 30 tweets rather than an abstract population. Numerical anchoring provides count tokens that can guide generation. Sequential deliberation enforces the order “first determine the counts, then summarise,” which is especially pronounced in CoT-R (Huang et al., 19 Sep 2025).

The mechanism is presented as an analogy rather than a literal cognitive equivalence. The paper explicitly notes that LLM internal mechanisms differ from human brains and that the observed improvements may arise from computational reasons other than human-like cognition (Huang et al., 19 Sep 2025). Nonetheless, the empirical pattern supports the claim that frequency-style prompts can activate learned text patterns that encourage more faithful handling of opinion distributions.

There are also clear limits. REFER is tested only on English product reviews and political tweets, and only for binary social values consisting of two opposing stances (Huang et al., 19 Sep 2025). The prompts are manually crafted rather than automatically optimized. The method addresses model fairness in summarisation outputs, not bias in the underlying training data. The evaluation itself depends on labelled datasets and classification pipelines, so flawed labels could distort fairness measurement (Huang et al., 19 Sep 2025).

The results also indicate trade-offs. In some settings, SPD and SOF improve while BUR or UER slightly worsen. The paper interprets this as evidence that improving second-order fairness—preserving proportional relationships between groups—need not improve every first-order fairness criterion simultaneously (Huang et al., 19 Sep 2025). This is an important corrective to a common misconception that “fairness” in summarisation is a single scalar property. In the REFER formulation, fairness is multidimensional and metric-dependent.

A second misconception is that frequency framing merely restates probabilities in another notation. The REFER formulation instead treats counts and denominators as computationally salient prompt tokens. This difference matters because the intervention does not rely on the model being given gold percentages; it relies on the model being asked to construct a count-based intermediate representation from the source documents and then condition the summary on that representation (Huang et al., 19 Sep 2025).

The term “frequency” appears in several adjacent prompt-design literatures, but with different semantics. In cross-domain few-shot classification, “Frequency-Aware Prompting” denotes a training-time mechanism based on Discrete Wavelet Transform and Inverse DWT, where high-frequency image components are zeroed or randomized and mutual attention is used to align features across original and frequency-modified views (Zhang et al., 2024). That work is not about opinion summarisation, yet it establishes a general design principle: robustness can improve when models are forced to rely less on brittle signals and more on stable structure-preserving cues (Zhang et al., 2024). A plausible analogy is that REFER treats explicit counts as stable anchors for summarisation in the same way that low-frequency visual structure is treated as a stable anchor under domain shift.

A different but complementary connection comes from Reflective Prompt Tuning, which optimizes prompts by evaluating a target model over an optimization set, clustering recurring failure modes, and revising prompts using structured diagnostic reports and memory across iterations (Bayat et al., 20 May 2026). Because that framework explicitly aggregates recurring failures into topics and uses their prevalence to guide edits, it provides a natural route for extending REFER beyond hand-written templates. This suggests a future REFER variant in which error frequencies such as under-representation of minority propositions or systematic miscounting are tracked over a development set and used to revise prompt structure automatically (Bayat et al., 20 May 2026).

RIFT adds a structural warning to such extensions. It shows that even when semantic content is held constant, non-sequential prompt topologies can sharply degrade multistep instruction following, with failures often attributable to instruction-order violations and semantic drift (Jaffe et al., 26 Jan 2026). For REFER, the implication is that count-based reasoning should be scaffolded in a linear prompt regime rather than embedded in elaborate non-linear control flow. The empirical success of CoT-R over more complex Agent-R variants in many settings is consistent with that interpretation, though the paper does not frame it in those terms (Huang et al., 19 Sep 2025).

Taken together, these adjacent results place REFER within a broader research trend: prompt performance depends not only on semantic content, but on how instructions expose intermediate structure, constrain outputs, and align with the sequential processing biases of present-day models (Mishra et al., 2021). In that sense, Frequency Framed Prompting is both a specific fairness intervention for opinion summarisation and a case study in how explicit counting, reference-class binding, and staged reasoning can reshape model behavior without retraining (Huang et al., 19 Sep 2025).