Quantum Field Theory Benchmark

• QFT-Bench is a benchmark for evaluating large language models on quantum field theory problems, featuring 75 challenges across eight subareas that require symbolic derivation, physical reasoning and strict formatting.
• The benchmark focuses on modules including classical field theory, quantization, QED, and supersymmetric field theories, with a notable emphasis on SUSY and CFT making up 44% of problems. Problems emphasize the need for physically meaningful results and adherence to conventions.
• Evaluations combine Symbolic verification with an LLM-based judge, ensuring compliance and correctness. Models can achieve high accuracy on these benchmarks, although faithfulness in derivations varies with some models showing errors in intermediate steps leading to potentially nervous derivations.

QFT-Bench is a benchmark for evaluating whether LLMs can solve graduate- to research-level quantum field theory problems requiring symbolic derivation, physical reasoning, and strict adherence to conventions and requested output formats. Introduced in “LLMs Can Tackle High Energy Theoretical Physics, yet Gaps Remain: Part I, Quantum Field Theory” (Liu et al., 23 Oct 2026), it comprises 75 problems spanning eight QFT subareas. Each problem has a well-defined target and a closed-form reference answer, and evaluation combines symbolic verification with LLM-based judging. The benchmark is designed to measure not only whether a model produces a correct final expression, but also whether it follows the stated conventions, derives the requested quantity, and presents the result in the prescribed form.

1. Scope and composition

QFT-Bench targets capabilities extending beyond formula retrieval. Its problems are described as ranging from hard graduate level to easy research level and were selected to require nontrivial reasoning or calculation while admitting closed-form answers suitable for verification. The benchmark emphasizes several characteristic difficulties of QFT: convention dependence, sign and normalization sensitivity, long derivations, and the need for physically meaningful results rather than merely well-formed symbolic expressions (Liu et al., 23 Oct 2026).

The 75 questions are distributed across eight subareas:

Subarea	Problems	Percentage
Classical field theory	7	9.3%
Quantization of scalar, spinor, and vector fields	8	10.7%
Quantum electrodynamics	7	9.3%
Regularization and renormalization	6	8.0%
Non-Abelian gauge theory and QCD	6	8.0%
Electroweak theory and the Higgs mechanism	8	10.7%
Supersymmetric field theories	18	24.0%
Conformal field theory	15	20.0%

Supersymmetric field theory and conformal field theory together account for 44% of the benchmark. This distribution distinguishes QFT-Bench from conventional graduate-course evaluations, which more often emphasize classical fields, canonical quantization, perturbative QED, and renormalization while giving less weight to SUSY and CFT.

The benchmark is distributed through the Hugging Face dataset LLMsHEP/QFT-Bench and an associated project webpage. Its questions were drawn from standard textbooks and public lecture notes but substantially rewritten to reduce direct formula retrieval and to make the tasks self-contained.

2. Dataset construction and problem formats

QFT-Bench was constructed through original problem design or adaptation, iterative refinement, and reference-solution development. Preliminary evaluations, particularly with Sonnet-5, were used to identify ambiguities and clarify assumptions, conventions, analytic-continuation paths, bases, and requested quantities. Each problem received a step-by-step reference chain of thought, a closed-form answer, and consistency checks for notation and conventions (Liu et al., 23 Oct 2026).

The questions are predominantly symbolic derivation tasks. Requested outputs include scattering amplitudes, spin-averaged squared amplitudes, propagators, beta functions, topological currents, decay widths, monodromy matrices, OPE coefficients, and central charges. Models were instructed to reason step by step and place the final result inside \boxed{...}.

Representative benchmark targets include:

• For a four-fermion theory, deriving the spin-averaged scattering result

$$\boxed{|\overline{\mathcal M}|^2=8\lambda^2(t^2+u^2)}.$$

• For a Proca field with metric $\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$, obtaining

$$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$

• For massless Bhabha scattering, deriving

$$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$

• For a massless Yukawa theory with $N$ Dirac fermions, obtaining

$$\boxed{ \beta_g(\lambda,g) =\frac{(2N+3)g^3}{(4\pi)^2} }.$$

• For a Chern–Simons current,

$$\boxed{ K^\mu= 2\epsilon^{\mu\nu\lambda\rho} \left( A_\nu^aF_{\lambda\rho}^a -\frac{g}{3}f^{abc}A_\nu^aA_\lambda^bA_\rho^c \right) }.$$

• For a Higgs decay into a fermion pair,

$$\boxed{ \Gamma(h\to f\bar f) = \frac{N_cg^2m_f^2m_h}{32\pi M_W^2} \left(1-\frac{4m_f^2}{m_h^2}\right)^{3/2} }.$$

• For a scalar conformal OPE, determining descendant coefficients

$$\boxed{ (\alpha,\beta,\gamma) = \left( \frac12,\, \frac{\Delta+2}{8(\Delta+1)},\, -\frac{\Delta} {16\left(\Delta-\frac{d-2}{2}\right)(\Delta+1)} \right) }.$$

The output specification is part of the task. A response can be physically correct in substance but receive a zero if it omits the required boxed answer, provides the wrong requested quantity, or gives several mutually incompatible boxed expressions.

3. Evaluation methodology

QFT-Bench uses a two-stage correctness pipeline. The first stage performs symbolic verification, primarily through a SymPy-based grader. This stage tests algebraic equivalence for many closed-form expressions and can accept unsimplified but mathematically equivalent forms.

The second stage sends responses for which symbolic verification is inconclusive to an LLM judge, Claude Opus-5. The judge compares the question, reference solution, reference answer, model derivation, and extracted answer. It assesses whether the boxed result is mathematically and physically equivalent to the reference in the specified convention and form (Liu et al., 23 Oct 2026).

The judging rules are strict. Examples of failure include:

• giving $m^2$ when the requested output is $\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$0;
• giving $\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$1 instead of $\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$2;
• replacing an explicitly specified normalization with a conventional alternative;
• presenting a correct derivation without the required boxed answer;
• supplying multiple incompatible boxed answers.

Equivalent rewritings, unsimplified expressions, relabeled indices, and harmless $\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$3 conventions are accepted.

For model $\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$4, the main accuracy measure is

$\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$5

where $\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$6 indicates whether the model correctly answered problem $\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$7. The main evaluation uses one inference per problem and does not report accuracy@$\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$8. Models were run with thinking enabled where available, with a maximum of 65,536 tokens; other sampling parameters were left at platform defaults.

Human validation used 60 responses sampled from 15 models, with four responses per model and three QFT experts comparing them with LLM judgments. Agreement with expert correctness labels was 98% for the Opus-5 judge and 92% for the GPT-5.6 judge. Agreement for faithfulness judgments was lower: 73% for Opus-5 and 67% for GPT-5.6. The benchmark therefore provides stronger validation for correctness judgments than for reasoning-faithfulness judgments.

4. Models and reported performance

The benchmark evaluates 39 models: 22 proprietary models from Claude, GPT, Gemini, and Muse, and 17 open-source models from Kimi, DeepSeek, Llama, GLM, and Qwen. All models were evaluated with a 64k-token limit.

Reported overall accuracies range from 41.3% to 100%:

Model	Accuracy
GPT-6.1-Sol	100.0%
Opus-5.5	98.7%
GPT-5.6-Sol	98.7%
Gemini-3.8-Flash	98.7%
Gemini-3-Flash-Preview	98.7%
Muse-Spark-1.3	98.7%
Sonnet-5.5	97.3%
Opus-5	97.3%
GPT-6-Sol	97.3%
GPT-5.6-Terra	96.0%
Gemini-3.6-Flash	96.0%
Qwen3.8-Max	96.0%
GPT-5.6-Luna	94.7%
Kimi-K3	94.7%
GPT-5	92.0%
DeepSeek-v4.1-Flash	92.0%
Qwen3.5-397B-A17B	92.0%
Muse-Spark-1.1	89.3%
Opus-4.6	85.3%
GLM-5	85.3%
GLM-5.3	84.0%
Sonnet-5	82.7%
Qwen3.5-35B-A3B	81.3%
Gemini-3.5-Flash-Lite	80.0%
Qwen3.8-27B	80.0%
Qwen3-235B-A22B-Thinking	80.0%
GPT-5.2	78.7%
DeepSeek-v3.1	77.3%
Llama-4-Maverick	74.7%
GPT-4.1	72.0%
Gemini-3.1-Flash-Lite	65.3%
GPT-4o	57.3%
DeepSeek-v3	53.3%
Haiku-4.5	50.7%
Llama-3.3-Instruct	49.3%
Qwen3-8B	46.7%
Llama-4-Scout	41.3%
Llama-3.1-Instruct	41.3%

Twelve models exceed 95% accuracy: Opus-5.5, Sonnet-5.5, Opus-5, GPT-6.1-Sol, GPT-6-Sol, GPT-5.6-Sol, GPT-5.6-Terra, Gemini-3.8-Flash, Gemini-3.6-Flash, Gemini-3-Flash-Preview, Muse-Spark-1.3, and Qwen3.8-Max.

The strongest models frequently achieve perfect scores in classical field theory, electroweak theory, non-Abelian gauge theory, quantization, SUSY, or CFT while still losing points in QED or renormalization. Renormalization is particularly difficult for several models: GPT-4o scores 33.3%, Llama-4-Scout 33.3%, Qwen3-8B 16.7%, and GLM-5.3 33.3%. Symbolic-verification scores are often lower than final judged accuracy, indicating that the LLM judge accepts mathematically equivalent expressions that SymPy does not directly establish.

Within model families, newer systems generally outperform earlier versions. Several open-source systems—including Qwen3.8-Max, Kimi-K3, DeepSeek-v4.1-Flash, and Qwen3.5-397B—are competitive with several proprietary systems, although proprietary models are generally stronger in the reported comparison.

5. Failure modes and reasoning faithfulness

Incorrect responses are classified into four categories: conceptual error, calculation error, strong-prior error, and instruction-following error. The reported percentages are percentages of failures within each model group, not percentages of all responses.

Failure type	Top half of models	Other models
Conceptual	$\eta_{\mu\nu}=\operatorname{diag}(-1,+1,+1,+1)$9	$$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$0
Calculation	$$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$1	$$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$2
Strong prior	$$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$3	$$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$4
Instruction following	$$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$5	$$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$6

Conceptual errors involve selecting the wrong physical relation or method. In one divergence-counting example, a model uses $$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$7 instead of the correct relation $$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$8, counting each internal vector line only once.

Calculation errors arise when the correct method is chosen but algebra, signs, factors, or field normalizations are mishandled. In an Abelian Higgs example, a model obtains $$\boxed{ \widetilde\Delta_F^{\nu\rho}(p) =-\frac{i}{p^2+m^2} \left(\eta^{\nu\rho}+\frac{p^\nu p^\rho}{m^2}\right) }.$$9 instead of the conventionally correct $$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$0.

Strong-prior errors occur when a familiar formula or normalization overrides explicit instructions. In a non-Abelian Noether-charge problem, a model initially derives the reference result

$$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$1

but then rescales it to

$$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$2

despite the problem fixing the gauge-transformation normalization.

Instruction-following errors occur when the physics is essentially correct but the requested format is not satisfied. A model may correctly solve

$$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$3

and obtain

$$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$4

but fail to place the result in the required box.

QFT-Bench separately evaluates whether displayed reasoning supports the final answer. The unfaithfulness rate is defined as

$$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$5

where $$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$6 is the number of correct final answers and $$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$7 is the number of those answers whose derivations are judged unfaithful.

A derivation is considered unfaithful when the final result is correct but the reasoning contains an unrepaired intermediate error, silently reverses a sign or factor, derives one quantity while boxing another, or quotes a familiar result after an incompatible calculation. Across the 39 models, the mean unfaithfulness rate is 7.2%. Llama models exhibit approximately 20–40% unfaithfulness, whereas Qwen models are generally lower. Muse-Spark, GLM, and Kimi are reported to produce comparatively faithful traces. Faithfulness increases approximately linearly with model capability within families, with the strongest models approaching near-perfect faithfulness.

One example involves a CFT Gram-matrix calculation in which a model computes

$$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$8

but subsequently inserts

$$\boxed{ \frac{d\sigma}{d\cos\theta} =\frac{\pi\alpha^2}{s} \left[ \frac{t^2}{s^2} +\frac{s^2}{t^2} +u^2\left(\frac1s+\frac1t\right)^2 \right] }.$$9

into the matrix while nevertheless obtaining the correct determinant

$N$0

Another model obtains the correct $N$1-system central charge,

$N$2

but presents an intermediate expression whose algebra would imply a $N$3-independent result.

The faithfulness metric concerns whether the visible derivation logically supports the answer. It does not establish that the displayed reasoning is the model’s complete internal causal process.

6. Interpretation, limitations, and research significance

QFT-Bench provides evidence that frontier LLMs can solve a substantial fraction of nontrivial graduate-level QFT calculations. The strongest evaluated model reaches 100% on the 75-question set, and twelve models exceed 95%. The results also show that correct final answers do not guarantee reliable derivations: models remain vulnerable to convention mismatches, algebraic slips, strong prior knowledge, and output-format requirements (Liu et al., 23 Oct 2026).

The benchmark has several limitations.

Closed-form bias: only problems with closed-form, verifiable answers are included. Open-ended theory construction, numerical research calculations, conceptual discovery, and tasks whose correctness depends heavily on physical interpretation are excluded.

Limited size and uneven coverage: the benchmark contains 75 problems, with 44% concentrated in SUSY and CFT. Its scores therefore should not be interpreted as a uniform measure of all QFT capabilities.

Reference-answer dependence: alternative conventions or physically equivalent formulations may be rejected when symbolic verification cannot establish equivalence, although the LLM judge is intended to mitigate this problem.

LLM-judge dependence: correctness and especially faithfulness partly depend on LLM judgments. Human agreement is high for correctness but lower for faithfulness.

Formatting sensitivity: the requirement for a boxed answer makes instruction following part of the benchmark score. This is useful for testing compliance but means that benchmark accuracy is not identical to mathematical competence.

Sampling and provider effects: reported results depend on model snapshots, prompting, API behavior, and default sampling parameters.

Contamination concerns: although problems were rewritten to reduce direct retrieval, they draw on standard QFT topics, public textbooks, and familiar formulas, so related material may have appeared in training data.

Limited human validation: the human-evaluation sample contains 60 responses from 15 models.

Single-inference evaluation: the main results use one inference per problem and do not report accuracy@$N$4.

The benchmark’s practical significance is consequently twofold. It establishes a quantitative test of symbolic and derivational QFT competence, and it identifies the difference between answer correctness and reasoning reliability. Models may already function as useful assistants for QFT calculations, symbolic manipulation, and derivation, particularly when their output is independently checked. The observed conceptual, calculational, conventional, and faithfulness failures nevertheless preclude treating high benchmark accuracy as a substitute for expert verification or as evidence of autonomous physical understanding and theory discovery.