Concern Alignment in AI

• Concern alignment is a framework that specifies how AI systems integrate operator objectives and social welfare concerns through measurable properties.
• Methodologies include property testing, conformal risk-control, and explicit modeling of evolving concerns with actionable thresholds.
• It also examines participatory governance and diagnostic auditing to enhance system accountability and mitigate misalignment risks.

Concern alignment denotes a family of alignment concepts in which the operative unit is not only an aggregate objective, benchmark score, or final verdict, but the concrete concerns that a system ought to recognize, prioritize, satisfy, surface, or remain accountable to in context. In the available literature, this idea appears in several related senses: as post hoc alignment of a predictor to a user-specified property set $\mathcal{P}$, as distinction between direct and social alignment, as concern-level auditing of AI peer review, as explicit modeling of continuously rising clinical concern, as structured extraction of concern types and moral dimensions from text, and as participatory or governance-centered management of plural stakeholder concerns (Overman et al., 2024, Korinek et al., 2022, Jin, 21 Apr 2026, Subaharan et al., 30 Apr 2026, Mather et al., 2022, Arzberger et al., 22 Jan 2026).

1. Conceptual scope and principal meanings

The literature does not treat concern alignment as a single uniform doctrine. One line defines alignment through desired model properties: a model $f$ is aligned when it belongs to a subset $\mathcal{P}$ of functions exhibiting specified behaviors such as monotonicity or concavity, and misalignment is measured by the distance $\delta_D(f,\mathcal{P})$ under a data distribution $D$ (Overman et al., 2024). A second line distinguishes direct alignment, where an AI system pursues goals consistent with the goals of its operator, from social alignment, where the system must instead pursue goals consistent with the broader welfare of all stakeholders and internalize externalities (Korinek et al., 2022). A third line uses the term at the level of evaluation artifacts: in AI peer review, concern alignment asks whether a model identified the same concerns as the official review process, assigned comparable severity, and treated them as blocking only when the human process did (Jin, 21 Apr 2026).

A broader structural interpretation treats concern alignment as a governance question rather than an engineering property alone. On this view, misalignment arises along three interacting axes—objectives, information, and principals—and alignment must be judged relative to who counts as a principal, which concerns are represented, and what institutional processes adjudicate trade-offs (LaCroix, 22 Apr 2026). Participatory work pushes this further by treating alignment as an interactional practice co-constructed during human–AI interaction rather than a one-off optimization performed at training time (Arzberger et al., 22 Jan 2026).

Sense	Core object	Representative source
Property alignment	Function subset $\mathcal{P}$ and distance $\delta_D(f,\mathcal{P})$	(Overman et al., 2024)
Direct/social alignment	Operator goals versus stakeholder welfare	(Korinek et al., 2022)
Concern-level diagnostics	Match graph between official and AI concerns	(Jin, 21 Apr 2026)
Structural value alignment	Objectives, information, principals	(LaCroix, 22 Apr 2026)
Situated co-construction	Runtime negotiation of misalignment	(Arzberger et al., 22 Jan 2026)
Concern representation	$\langle \text{concern\_type}(P), D\rangle$	(Mather et al., 2022)

This multiplicity matters. A plausible implication is that concern alignment is best understood as a layered vocabulary for alignment problems that are irreducible to raw accuracy, reward maximization, or verdict agreement.

2. Property-centered formulations and conformal post-processing

A precise formalization appears in work that interprets model alignment through property testing. Here the user specifies a property set $\mathcal{P}$, a subset of measurable functions $f:\mathcal{X}\to\mathcal{Y}$ encoding desired behavior. The distance to alignment is defined by first fixing a data distribution $f$0 and then taking $f$1, where $f$2 is the disagreement probability under $f$3. A model is $f$4-far from $f$5 if $f$6 (Overman et al., 2024).

The central technical move is to convert a proximity-oblivious tester for $f$7 into a family of non-increasing loss functions over a conformal band around a pretrained model. In the regression setting the band is written as

$f$8

with calibration losses $f$9 indicating whether some prediction inside the band would pass the tester on calibration point $\mathcal{P}$0. The conformal risk-control procedure selects

$\mathcal{P}$1

where $\mathcal{P}$2 and $\mathcal{P}$3 in the presented setting. Under exchangeability, boundedness, and monotonicity, the resulting band satisfies an expected loss guarantee on a fresh point; the main theorem states that with probability at least $\mathcal{P}$4 over calibration data, there exists a function $\mathcal{P}$5 such that $\mathcal{P}$6 (Overman et al., 2024).

The paper instantiates this framework for shape constraints, especially monotonicity and concavity, on supervised learning datasets with XGBoost backbones. With train/calibration/test splits of 70/15/15, empirical risk on the test set was reported as approximately $\mathcal{P}$7 for $\mathcal{P}$8. For moderate $\mathcal{P}$9 values, conformal MSE closely tracked or even improved on dedicated monotonic-training models; at $\delta_D(f,\mathcal{P})$0 the band widened and MSE worsened while risk fell to approximately zero. The same work also proves that pretrained models will always require alignment techniques even as model sizes or training data increase, provided the training data contain even small biases (Overman et al., 2024).

This property-centered formulation differs from reward-centric alignment. It aligns outputs to an auditable concern set without retraining the underlying predictor, and it treats acceptable misalignment risk $\delta_D(f,\mathcal{P})$1 as the primitive control variable.

3. Training-time tensions, safety alignment, and robustness

Concern alignment is often discussed against the background of modern LLM safety tuning. One empirical result is that alignment examples embedded directly in supervised fine-tuning can behave like a contaminant. In a setup where the full SFT dataset $\delta_D(f,\mathcal{P})$2 is the union of reasoning examples $\delta_D(f,\mathcal{P})$3 and alignment examples $\delta_D(f,\mathcal{P})$4, the alignment portion consisted of refusals or “safe” but uninformative answers such as “Sorry, I can’t help with that.” In the merged GoatAssistant∪Guanaco dataset, roughly one-third of examples were alignment refusals. Removing these examples improved a 7B LLaMA 2 model from 45.63% to 49.31% on MMLU, from 34.28% to 35.69% on BBH, from 9.15% to 12.20% pass@1 on HumanEval, and from 22.61% to 28.10% F1 on DROP, corresponding to percentage drops from 4.1% to 33.3% when alignment data were retained. No confidence intervals or significance tests were reported (Bekbayev et al., 2023).

The stated explanation is that refusal-style alignment responses shift next-token distributions toward short, template-style outputs and away from rich, multi-sentence reasoning, creating something akin to mild mode collapse on safe refusals. The proposed mitigation is to separate content-safety alignment from core reasoning SFT, perform SFT on informative data, and inject refusals in a later phase through RLHF or rejection-sampling (Bekbayev et al., 2023).

This concern is sharpened by broader arguments that current alignment strategies remain structurally vulnerable because in-context learning can be hijacked by adversarial context. On that account, system prompts, instruction tuning, and RLHF are not completely robust defenses: the same mechanism that makes LLMs versatile across tasks also allows “virtual fine-tuning” within the prompt, creating a persistent capability–alignment tension (Millière, 2023).

Several technical responses operate without full retraining. RA-LLM wraps an aligned LLM with a robust alignment checking function based on random token dropping and Monte Carlo estimation. In reported experiments on open-source models, the method reduced attack success rates from nearly 100% to around 10% or less while only slightly lowering benign answering rate (Cao et al., 2023). By contrast, the evaluation scheme called Multilingual Blending shows that safety alignment can be compromised by mixed-language prompts: peak bypass rates reached 67.23% on GPT-3.5 and 40.34% on GPT-4o, with mixed morphology and mixed-family language combinations especially effective (Song et al., 2024).

A separate line seeks cheaper preference alignment rather than stronger refusal filtering. CycleAlign builds a bidirectional loop between a black-box aligned LLM and a white-box model: the black-box model ranks candidate responses under in-context demonstrations, the white-box model self-ranks by log-probability, and agreement rankings become pseudo-labels for later cycles. On HH-RLHF, CycleAlign$\delta_D(f,\mathcal{P})$5 achieved an average total reward-model score of 68.4, compared with 67.4 for PRO and 63.1 for RRHF, using zero human-labelled preference pairs and a single-GPU fine-tuning regime (Hong et al., 2023).

Taken together, these results show that “alignment” is not a monolithic intervention. Some alignment data can degrade reasoning, some alignment layers are brittle under adversarial prompting, and some low-resource procedures can improve preference alignment without PPO or large human annotation pipelines.

4. Concern-level diagnostics and behavioral alignment metrics

Concern alignment has also become an evaluation framework. In AI peer review, the core object is the match graph, a bipartite graph between official concerns $\delta_D(f,\mathcal{P})$6 and agentic concerns $\delta_D(f,\mathcal{P})$7 for paper $\delta_D(f,\mathcal{P})$8, with edges labeled by match type—exact, partial, or related—and nodes labeled by severity, decisive flag, post-rebuttal Area-Chair treatment, and whether a resolved issue was addressed in the PDF (Jin, 21 Apr 2026). From this graph the evaluation ladder proceeds from Level 0 verdict agreement, to Level 1 concern recall and phantom rate, to Level 2 verdict-stratified behavior, to Level 3 decision-aware calibration, and finally to Level 4 rebuttal-aware decomposition.

The pilot study covered four public AI review systems in six configurations over 48 ML-aligned papers with 670 official concerns and 79 decisive blockers. The central result is not merely that systems miss concerns, but that calibration is often the binding constraint. Most systems detected non-trivial fractions of official concerns, yet most marked 25--55% of concerns on accepted papers as decisive even though, under the operationalization used, no official concern on accepted papers was treated as a decisive blocker. The study also found that identical overall verdict accuracy can conceal reject-heavy behavior versus low-recall profiles, and that verdict inference can be unstable when systems do not emit native accept/reject labels: accepted-paper accuracy varied by 46–96 percentage points depending on rater and inference rule (Jin, 21 Apr 2026).

A related diagnostic move in decision-making systems is to compare not only which instances two systems get wrong, but which wrong labels they choose. Misclassification Agreement (MA) applies multiclass Cohen’s $\delta_D(f,\mathcal{P})$9 to the joint-error set, while Class-Level Error Similarity (CLES) compares per-class confusion distributions using Jensen–Shannon divergence. Across image and video domains, these behavioral alignment metrics showed strong Spearman correlation with representational alignment measures such as CKA on most datasets, typically in the range of approximately 0.6–0.9, while also exposing cases where error consistency alone is misleading. The stated exception was Epic-Kitchen, where CLES fell to 0.08 because of class imbalance (Xu et al., 2024).

These diagnostic frameworks share a common thesis: aggregate success measures hide structure. Concern alignment, in this evaluative sense, is the recovery of that hidden structure—what was noticed, what was missed, what was escalated, and whether error patterns reflect the same distinctions that matter to humans.

5. Domain-specific concern representations and temporal trajectories

In NLP, concern alignment can refer to a structured representation of topical issues and their moral valence. One formulation defines a concern for proposition $D$0 as

$D$1

where $D$2 is drawn from a small domain-specific taxonomy and $D$3 assigns endorsement scores to moral dimensions such as care/harm, fairness/cheating, loyalty/betrayal, authority/subversion, and purity/degradation. The method combines semi-automatic induction of concern-type lexicons, requiring approximately 2–4 hours of human labor per domain, with automatic moral-dimension extraction via semantic similarity and lexical expansion. On a 50-tweet ground truth, the expanded Moral4 lexicon improved recall by 231% relative to the baseline, with a 10% loss in precision; F1 improved by 66%, and concern-type detection reached 97.8% of human performance (Mather et al., 2022).

A different operationalization appears in clinical LLM agents. Here the concern signal is not a symbolic concern type but a continuous state variable built from a memoryless risk encoder and temporal integration dynamics. At hour $D$4, structured patient data are mapped to a three-dimensional vector $D$5 for stability, urgency, and control margin, and then integrated into a latent state $D$6 using either stateless updates, a first-order leaky integrator with $D$7, or second-order hysteretic dynamics with asymmetric smoothing and momentum. Escalation pressure is then computed as

$D$8

with escalation triggered when $D$9 (Subaharan et al., 30 Apr 2026).

Across 100 synthetic ward trajectories, stateless agents showed sharp escalation cliffs, whereas second-order dynamics produced smooth, anticipatory concern trajectories despite similar escalation timing. The reported metrics were Unease Lead Time, Unease Area, and Escalation Jerk. Stateless dynamics yielded ULT $\mathcal{P}$0 h, UA $\mathcal{P}$1, EJ $\mathcal{P}$2; first-order dynamics yielded ULT $\mathcal{P}$3 h, UA $\mathcal{P}$4, EJ $\mathcal{P}$5; second-order dynamics yielded ULT $\mathcal{P}$6 h, UA $\mathcal{P}$7, EJ $\mathcal{P}$8. The paper highlights that second-order dynamics reduce abruptness relative to stateless behavior by 42% in EJ while preserving adaptive variability in lead time (Subaharan et al., 30 Apr 2026).

These two strands illustrate a common pattern. Concern alignment can be representational, in the sense of extracting explicit concern objects from language, or dynamical, in the sense of exposing how concern accumulates over time before a threshold decision.

6. Pluralistic, participatory, and institutional dimensions

A recurrent theme is that concern alignment cannot be reduced to matching a single operator’s objective. The direct/social distinction formalizes this: direct alignment asks whether the system’s reward or objective matches the operator’s goals, while social alignment asks whether it internalizes externalities and advances a social welfare benchmark or, in practice, avoids choosing policies that society prefers less along dimensions where a social preference exists (Korinek et al., 2022). On this view, governance mechanisms—laws, regulations, taxes, subsidies, rights-based constraints, runtime enforcement, and multi-level norms at group, country, and world scales—are integral to alignment rather than external add-ons (Korinek et al., 2022).

The structural governance account makes this pluralism explicit by decomposing misalignment into objectives, information, and principals. Even if objective misspecification is corrected, information asymmetries and conflicts among multiple principals remain. Relative principals include shareholders, developers, users, regulators, bystanders, and civil society; absent a social-choice or deliberative mechanism, systems reflect the concerns of the most powerful principal set (LaCroix, 22 Apr 2026). This suggests that concern alignment is inherently context-dependent and contestable.

Participatory HCI research supplies an interaction-level counterpart. In workshops with researchers using LLMs as research assistants, misalignments were recorded as diary tuples $\mathcal{P}$9 of prompt, misaligned response, intuitive intervention, and final outcome. Participants described misalignments less as abstract ethical violations than as unexpected responses, and task or social breakdowns. They also articulated several roles: adjusting model behavior through prompt refinement, interpreting model assumptions, exercising self-restraint, withdrawing from interaction, and proposing collective infrastructures such as shared prompt repositories and environmental impact dashboards (Arzberger et al., 22 Jan 2026).

Human forecasting platforms provide an institutional analogy. Alignment failures there were classified as reward-specification problems and principal–agent problems. The cited examples include an improper participation-weighted Brier score on Good Judgment Open and Metaculus-style tournament incentives that discourage information sharing and encourage probability distortion to maximize the chance of finishing in the top few positions (Sempere et al., 2021). Proposed remedies—probabilistic rewards, collaborative scoring, forcing forecasts on all questions, anonymizing winners, and Shapley-style contribution allocation—show that alignment of concerns is often an incentive-design problem before it is a model-design problem (Sempere et al., 2021).

The major misconception challenged across these works is that alignment is a single technical property solved by better optimization. The literature instead treats concern alignment as a composite task: specify what counts as a concern, measure whether systems detect and weight those concerns appropriately, expose how concern develops over time, and build institutional processes through which affected parties can contest or reshape the governing objectives.