MATCHA: Answer-Reasoning Robustness in LLMs

• MATCHA is an adversarial evaluation framework designed to expose answer–reasoning misalignment in LLMs by perturbing input tokens and embeddings.
• The framework employs token-level and embedding-level perturbation techniques, using gradient updates and semantic judges to isolate failures in the reasoning process.
• Empirical evaluations on benchmarks like GSM8K reveal an accuracy–robustness trade-off, showing that high answer accuracy does not guarantee consistent internal justification.

MATCHA is an adversarial evaluation framework for LLMs that measures the robustness of Chain-of-Thought reasoning under small perturbations to the input while deliberately preserving the final answer. In this context, MATCHA denotes Misaligned Answer and Thought CHAin, and its central objective is to construct cases in which a model still gives the same, correct answer, but the accompanying reasoning becomes wrong, inconsistent, or nonsensical. The framework is motivated by settings in which the explanation itself is safety-critical or trust-critical, including education and healthcare, where a correct answer paired with faulty justification can still be misleading (2505.17406).

1. Conceptual basis and motivation

MATCHA is designed around a specific failure mode: answer–reasoning misalignment. Rather than evaluating whether a perturbation changes the output answer, it searches for perturbations that leave the answer unchanged and correct while degrading the Chain-of-Thought. This shifts evaluation from answer accuracy alone to the consistency between an answer and the reasoning used to justify it (2505.17406).

The framework is grounded in the observation that LLM decision-making is opaque and that Chain-of-Thought is often used as an explanation technique. The motivating cases are explicitly post-hoc: a student may ask an LLM to explain a known answer from an answer key, or a patient may ask for an explanation of an already provided diagnosis. In such situations, the answer is fixed, and the primary question is whether the explanation is robust, sensible, and aligned with that answer. MATCHA treats explanation fragility under semantically minor perturbations as a direct challenge to trustworthiness.

This suggests a broader evaluative distinction between answer robustness and reasoning robustness. A plausible implication is that a model can appear reliable under conventional accuracy metrics while still being vulnerable at the level of justification, especially when explanations are consumed as evidence of competence rather than as auxiliary text.

2. Formal formulation

MATCHA is defined in an in-context learning plus Chain-of-Thought setting. The prompt includes examples $(x_i, y_i, r_i)_{i=1}^{k}$, where $x_i \in \mathcal{X}$ is a question, $y_i \in \mathcal{Y}$ is an answer, and $r_i \in \mathcal{R}$ is the Chain-of-Thought. For a new question $x_p$, an LLM $f_\theta$ generates an answer $y_p$ and reasoning $r_p$ according to

$$\argmax_{(y_p, r_p) \in \mathcal{Y} \times \mathcal{R}} P_{\theta}\!\left((y_p, r_p)\mid (x_i, y_i, r_i)_{i=1}^{k}, x_p\right).$$

A defining design choice is the answer-first setup: the model is prompted to output “The answer is X.” first, and only then the Chain-of-Thought. This anchors the answer and isolates failures in the reasoning channel.

For a full prompt sequence $x_{1:n}$, the framework marks token positions $x_i \in \mathcal{X}$0 through $x_i \in \mathcal{X}$1: the start of in-context examples, the current question, the current answer, and the Chain-of-Thought span. The model maps tokens to logits over the vocabulary,

$x_i \in \mathcal{X}$2

MATCHA separates logits for the answer and reasoning segments:

$x_i \in \mathcal{X}$3

$x_i \in \mathcal{X}$4

where $x_i \in \mathcal{X}$5 is the original input and $x_i \in \mathcal{X}$6 is a perturbed input. It then defines cross-entropy losses

$x_i \in \mathcal{X}$7

Here, $x_i \in \mathcal{X}$8 measures how much the reasoning distribution changes, whereas $x_i \in \mathcal{X}$9 measures how much the answer distribution changes. To balance the two, MATCHA introduces

$y_i \in \mathcal{Y}$0

and optimizes

$y_i \in \mathcal{Y}$1

The optimization therefore seeks perturbations that increase divergence in the Chain-of-Thought while minimizing divergence in the answer. This objective is the formal core of the framework.

3. Perturbation mechanisms and misalignment criteria

MATCHA instantiates its objective through two attack families: token-level perturbations and embedding-level perturbations (2505.17406).

Token-level perturbation proceeds in two stages. The first stage inserts random tokens into the question span $y_i \in \mathcal{Y}$2 at random positions, with insertion ratio $y_i \in \mathcal{Y}$3, producing a slightly noisy question $y_i \in \mathcal{Y}$4. The second stage refines the inserted tokens using gradients of $y_i \in \mathcal{Y}$5. After generating the original answer $y_i \in \mathcal{Y}$6 and Chain-of-Thought $y_i \in \mathcal{Y}$7, the method computes gradients with respect to one-hot token indicators $y_i \in \mathcal{Y}$8,

$y_i \in \mathcal{Y}$9

selects influential positions via

$r_i \in \mathcal{R}$0

and chooses replacement vocabulary items according to

$r_i \in \mathcal{R}$1

Candidate replacements are then filtered by an LLM semantic-preservation judge, which rejects edits that change numbers, names, or core verbs. The final perturbation is successful only if the answer remains correct and the reasoning is judged to be incorrect, inconsistent, or nonsensical.

Embedding-level perturbation changes continuous input embeddings while leaving the token sequence unchanged. Let $r_i \in \mathcal{R}$2 and $r_i \in \mathcal{R}$3; the perturbation is applied only to the question segment. MATCHA performs projected FGSM-style updates with step size $r_i \in \mathcal{R}$4 and bound $r_i \in \mathcal{R}$5:

$r_i \in \mathcal{R}$6

followed by projection into the $r_i \in \mathcal{R}$7 box

$r_i \in \mathcal{R}$8

After $r_i \in \mathcal{R}$9 iterations, the perturbed embedding sequence is decoded and evaluated under the same “correct answer, wrong reasoning” criterion.

Reasoning is treated as misaligned if calculations are wrong, logical steps contradict the question or answer, fabricated or false facts are used, or long irrelevant or hallucinated content breaks coherence. The paper groups successful attack outputs into four error types: wrong calculation, wrong reasoning, wrong information, and unrelated information.

4. Evaluation protocol, metrics, and judge architecture

MATCHA evaluates only examples for which the unperturbed answer is correct. The protocol first generates baseline answer-reasoning pairs under CoT-style in-context learning, then applies either token-level or embedding-level perturbations, rejects semantically altered prompts, and finally classifies the result into one of three outcome categories (2505.17406).

If the perturbed answer becomes wrong, the example is counted as WR (“wrong after attack”). If the answer remains correct, the original and perturbed Chains-of-Thought are sent to a reasoning judge. If the judge finds the perturbed reasoning incorrect, the attack is counted as SR (“success rate”); otherwise it is UR (“unattackable rate”). The paper defines

$x_p$0

The metric suite is organized around clean accuracy and robustness under attack.

Metric	Meaning
ACC	Accuracy on clean inputs
SR	Correct answer, wrong reasoning after perturbation
UR	Correct answer and correct reasoning after perturbation
WR	Wrong answer after perturbation

Two LLM-as-judge components are used. The reasoning correctness judge $x_p$1 is GPT-3.5-turbo. It receives “Response 0” and “Response 1” and evaluates logicality, deviation from reference reasoning, correctness of calculations, and factuality, while ignoring harmless repetition. The semantic preservation judge $x_p$2 is DeepSeek-R1-Distill-Qwen-7B and decides whether the original and perturbed questions preserve the same semantic meaning, especially with respect to numbers, names, and important verbs. The framework also uses hard constraints such as arithmetic checkers where applicable.

MATCHA is evaluated on three benchmarks: SingleEq, a single-step algebra word-problem task from MAWPS; GSM8K, a multi-step grade-school math benchmark; and StrategyQA, a commonsense QA benchmark. The tested open-source models are Llama-3-8B, Mistral-7B, Zephyr-7B-beta, Qwen2.5-7B, and DeepSeek-R1-Distill-Qwen-7B.

5. Empirical findings and transferability

The central empirical result is that reasoning robustness varies substantially across task types and perturbation modes. Under token-level perturbations, Llama-3-8B records UR values of 68.9% on SingleEq, 39.0% on GSM8K, and 60.3% on StrategyQA, with SR values of 10.2%, 14.7%, and 16.2% respectively; its average UR is 56.1% and average SR is 13.7%. Qwen2.5-7B shows average UR 54.1% and average SR 3.7%, whereas DeepSeek-R1-7B shows average UR 42.2% and average SR 11.1%. Under embedding-level perturbations, the pattern shifts: Llama-3-8B has average UR 29.3% and average SR 15.2%, Mistral-7B has average UR 77.2% and average SR 7.1%, and Qwen2.5-7B has average UR 22.3% and average SR 8.8% (2505.17406).

These results support three recurring findings. First, multi-step and commonsense reasoning are more fragile than simple logical reasoning: UR is consistently lower on GSM8K and StrategyQA than on SingleEq, especially under token-level perturbations. Second, correct-answer but wrong-reasoning cases are common, including wrong calculations that still lead to the correct answer, wrong intermediate quantities that cancel out, fabricated facts used to justify a correct answer, and long irrelevant sequences appended to an otherwise correct response. Third, the paper identifies an accuracy–robustness trade-off: high-accuracy models such as Qwen2.5 and DeepSeek-R1 often have lower UR under embedding attacks, suggesting that answer accuracy does not imply stable internal reasoning.

MATCHA also studies black-box transfer. Successful token-level examples crafted on open-source models are evaluated against GPT-3.5-turbo and GPT-4o, and the paper reports non-trivial transfer rates. Transfer is lower on SingleEq, higher on GSM8K, and moderate on StrategyQA. This suggests that answer–reasoning misalignment is not purely a model-specific artifact but reflects vulnerabilities that can persist across architectures and training pipelines.

6. Trustworthiness, relation to prior work, and limitations

MATCHA advances the evaluation of LLM reasoning by explicitly targeting the consistency between answers and their justifications. Its results challenge the assumption that Chain-of-Thought reliably reflects the model’s internal reasoning process. When small, semantically minor perturbations can leave the answer intact while corrupting the explanation, CoT becomes a fragile basis for trust, especially in domains where users rely on the rationale rather than the label alone (2505.17406).

Within the broader literature, MATCHA connects several research strands. It belongs to work on the stability and faithfulness of explanations, where prior NLP and vision research showed that explanations can be manipulated while predictions remain fixed. It is orthogonal to adversarial LLM attacks that primarily seek to change outputs; MATCHA instead keeps the answer fixed and attacks the reasoning. It also operationalizes earlier observations that CoT can improve performance even when intermediate reasoning is invalid, by introducing a concrete optimization objective and quantitative robustness metrics. Its distinctive elements are the objective

$x_p$3

the dual use of token-level and embedding-level perturbations, the ACC/UR/SR/WR metric set, and the demonstration of transferability to closed-source models.

The framework has several limitations. It relies on LLM judges—GPT-3.5-turbo for reasoning correctness and DeepSeek-R1 for semantic preservation—and errors or inconsistencies in those judges propagate to robustness estimates. Token-level attacks are computationally expensive, requiring many forward and backward passes as well as judge calls, with runtime per dataset-model pair in the range of several thousand seconds. The evaluation covers only three datasets and does not address domains such as formal logic, code reasoning, or theorem proving. Despite the semantic-preservation filter, some token edits may still subtly alter meaning.

The paper therefore points toward several future directions: stronger non-LLM evaluators for reasoning correctness and semantic equivalence, broader task coverage, more natural perturbation schemes such as paraphrases or causal interventions, and training objectives that directly enforce answer–reasoning consistency across paraphrases or adversarial inputs. A plausible implication is that future robust reasoning systems may require tighter coupling between answer generation and verifiable intermediate derivation, rather than treating Chain-of-Thought as an unconstrained post-hoc explanation.