OpinionQA Dataset Benchmark

• OpinionQA is a benchmark dataset designed for evaluating demographic-specific opinion alignment in large language models using survey-based questions.
• It comprises 1,498 survey questions and approximately 91,000 unique question-demographic pairs, facilitating analysis of political attitudes and social issues.
• The dataset supports multiple modeling approaches, with RLVR demonstrating superior performance in aligning model outputs with observed survey responses.

OpinionQA is a benchmark dataset constructed to evaluate and advance steerable pluralistic alignment in LLMs, particularly regarding the generation of perspective-specific outputs for opinion-rich, demographic-conditioned survey questions. The dataset is designed around the alignment of model predictions with nuanced human perspectives, as required in applications involving social science surveys, policy analysis, and user-facing systems where demographic-specific reasoning is critical.

1. Dataset Composition and Construction

OpinionQA comprises 1,498 multiple-choice survey questions sourced from Pew Research’s American Trends Panel. Each question is annotated with response distributions across U.S. demographics, producing a total of approximately 91,000 unique $\langle$question, demographic$\rangle$ pairs. For each pair, the dataset records the most commonly selected answer for the specified demographic.

The survey questions span a range of topics relevant to political attitudes, social issues, and consumer sentiment. Demographic information includes categorical attributes such as age group, gender, race/ethnicity, education, and geographic region—thus supporting a wide spectrum of pluralistic opinion modeling.

Component	Size/Count	Description
Survey Questions	1,498	Multiple-choice, drawn from Pew Research’s survey pool
Demographic-Conditioned Pairs	≈91,000	$\langle$question, demographic$\rangle$ combinations
Answer Distribution Annotation	Full U.S. Panel	Most common response per demographic

2. Task Formulation and Quantitative Objectives

OpinionQA's core task is defined as follows: given a survey question $s$ and a target demographic $d$, the model must select the most commonly chosen answer $a_i$ for that demographic. The formal objective is:

$a_i = \arg\max_{a_i \in A} \; p(a_i \mid d, s)$

where $A$ is the finite set of multiple-choice options for each question, $d$ encodes demographic attributes, and $s$ denotes the input survey question. This formulation isolates the problem of demographic-specific opinion alignment and facilitates direct quantitative comparison between model outputs and observed survey results.

3. Methodological Approaches for Steerable Alignment

Multiple methods were evaluated using the OpinionQA dataset for steering model outputs toward targeted pluralistic viewpoints:

• Zero-Shot Chain-of-Thought (CoT) Prompting: Directly prompts LLMs to reason stepwise, without supervised signal.
• Supervised Fine-Tuning (SFT): Trains models to predict the answer using input pairs (question, demographic) and correct label.
• Synthetic Chain-of-Thought: Employs generated rationales as intermediate training targets, following the STaR prompting technique.
• Reinforcement Learning with Verifiable Rewards (RLVR): Optimizes both the correctness of the final answer and the inclusion of reasoning steps that expose pluralistic perspectives.

Due to the lack of human-authored CoT justifications in OpinionQA, approaches relying on human-generated rationales were not applied; only SFT, synthetic CoT, and RLVR were used.

4. Evaluation Protocols and Empirical Findings

The evaluation employed the following metrics:

• Accuracy (Acc): Percentage of correct majority-answer predictions for demographic-conditioned pairs.
• Class-Balanced Accuracy (BAcc): Accuracy averaged across all answer classes to control for imbalances.
• Macro F1 (MaF): Harmonic mean of per-class precision and recall.

Standard splits were utilized (approximately 77,000 training and 9,000 testing examples). Models evaluated included Llama 3 8B and Qwen2.5 7B.

Model (Llama 3 8B)	Accuracy (%)	BAcc (%)	Macro F1 (%)
RLVR	72.3	74.5	68.4
SFT	Lower	Lower	Lower
Synthetic CoT	Lower	Lower	Lower

RLVR consistently outperformed other methods, marking an improvement in steering model outputs to reflect the target demographic’s majority opinion. Similar results were observed across Qwen2.5 7B variants.

5. Role and Diagnosis of Chain-of-Thought Reasoning

Chain-of-Thought (CoT) reasoning enables interpretable modeling of how demographic context informs answer selection. Within OpinionQA:

• CoT traces are used to analyze the fidelity (faithfulness) of generated rationales: evaluators receive a question and the model’s CoT chain, and are tasked to select an answer based solely on it.
• A visual diagnostic (see Figure~\ref{fig:vk-opinionqa} in (Zhang et al., 5 Oct 2025)) displays the stepwise deliberative reasoning colored by perspective, highlighting how a model justifies its choice given demographic context.

This dual evaluation of both outcomes and reasoning faithfulness provides evidence of the model’s degree of pluralistic alignment and exposes trade-offs between reasoning richness and strict answer fidelity.

6. Contextual Significance and Future Directions

By enabling explicit assessment of demographic-specific opinion alignment, OpinionQA advances evaluation standards for steerable pluralistic systems. Its formulation is distinguished from aspect-based opinion mining datasets (e.g., ReviewQA (Grail et al., 2018)), as the emphasis is on aligning outputs with observed collective human judgments rather than extracting aspect-level textual evidence.

A plausible implication is that RLVR, by linking answer correctness to the generation of pluralistic reasoning traces, provides higher sample efficiency and performance in pluralistic tasks, albeit with occasional decreases in reasoning faithfulness. This trade-off highlights a key area for methodological refinement, particularly as future research seeks to balance pluralistic justification and answer accuracy.

OpinionQA thus serves as a benchmark for diagnosing and comparing alignment strategies in LLMs, informing the development of systems engineered for nuanced, contextually-informed opinion reasoning across diverse user segments.