European Qualifying Examination (EQE)

• European Qualifying Examination (EQE) is a professional exam for aspiring patent attorneys, assessing the application of the EPC, PCT, and other legal frameworks in hands-on scenarios.
• The exam structure combines legal questions and multimodal claim evaluation to test issue-spotting, deadline computation, and the integration of text with diagrams.
• Recent studies reveal that while current AI models achieve moderate accuracy, none surpass the ~0.90 threshold needed for full professional competence.

Searching arXiv for papers on the European Qualifying Examination and related patent-claim evaluation. The European Qualifying Examination (EQE) is the professional examination for future European patent attorneys, and in the cited literature it is framed as a hands-on professional examination for candidates seeking to practice as a professional representative before the European Patent Office (EPO). In the recent computational-law literature, the EQE functions not only as a credentialing mechanism but also as a demanding benchmark for legal reasoning systems, because it tests the application of the European Patent Convention (EPC), PCT, Paris Convention, Boards of Appeal case law, and relevant national laws to practical, fact-dependent scenarios rather than merely testing legal recall (Khera et al., 11 Jul 2025). Research using the EQE as an evaluation substrate emphasizes that the examination probes subsumption, procedural issue-spotting, claim interpretation, and multimodal integration, thereby making it a particularly stringent proxy for real European patent practice (Khera et al., 11 Jul 2025).

1. Institutional role and professional function

The EQE is presented in the literature as an examination directed toward future European patent attorneys, with the pre-examination specifically designed to test whether a candidate is fit to practice as a professional representative before the EPO (Khera et al., 11 Jul 2025). In that framing, the examination is not reducible to abstract doctrinal questioning. It is intended to assess whether candidates can work through regulated, practical patent-law problems resembling day-to-day European patent practice, including advising on whether rights exist or can be maintained, determining what prior art counts and for what purpose, calculating procedural deadlines, understanding priority effects, and interpreting claims against descriptions and drawings (Khera et al., 11 Jul 2025).

This practical orientation is central to why the EQE has attracted attention in AI and computational legal studies. The examination is treated as a benchmark for whether a system can function like a junior European patent attorney rather than merely generate legally flavored text. The emphasis on application over recitation implies that performance on the EQE reflects competence in structured legal reasoning under uncertainty, where procedural consequences, doctrinal distinctions, and fact sensitivity must be integrated correctly (Khera et al., 11 Jul 2025).

A plausible implication is that the EQE occupies a dual role in contemporary discourse: first, as a gatekeeping examination within the European patent profession, and second, as a high-fidelity testbed for evaluating legal-reasoning systems in a domain where mistakes in doctrine, deadlines, or claim interpretation have concrete professional significance.

2. Exam structure and competencies assessed

The paper evaluating LLMs on the EQE focuses on the EQE pre-examination, using papers from 2012–2024, excluding 2020 because it was cancelled due to COVID-19 (Khera et al., 11 Jul 2025). Within that study, the pre-examination is split into two broad portions. Questions 1–10 form the legal part, while Questions 11–20 form a claims and multimodal part involving analysis and interpretation of patent claims based on provided descriptions and prior art; these may include figures, diagrams, drawings, and mixed text-image content (Khera et al., 11 Jul 2025).

Each pre-exam problem contains four statements, and candidates must decide whether each statement is True or False, with a written explanation (Khera et al., 11 Jul 2025). The dataset derived from past pre-examinations contains 480 questions (120 problems with four questions each), corresponding to the legal Questions 1–10 across the usable exam years (Khera et al., 11 Jul 2025). For text-only experiments, the prompts included statutory references, the four True/False statements, and the instruction to answer with justification (Khera et al., 11 Jul 2025).

The competencies implicated by this structure are broad but technically specific. The examination tests proficiency in the EPC, PCT, Paris Convention, Boards of Appeal case law, and relevant national laws (Khera et al., 11 Jul 2025). It also tests the application of patentability rules such as novelty and inventive step, the handling of priority, the treatment of unpublished European applications and their effects on novelty and inventive step, the calculation of deadlines and time limits, claim interpretation in view of descriptions and prior art, and the procedural consequences of filings, dates, and status changes (Khera et al., 11 Jul 2025).

The multimodal portion adds another layer. Because claims questions may involve figures, drawings, and mixed text-image content, success requires not only legal analysis but also integration of textual and graphical materials (Khera et al., 11 Jul 2025). This supports the characterization of the EQE as a practical legal-technical examination rather than a purely doctrinal one.

3. Doctrinal and procedural dimensions

The literature emphasizes several doctrinal and procedural distinctions as central to EQE performance. The most frequently discussed is the distinction between novelty and inventive step/nonobviousness. Many evaluated models could define these concepts but failed to apply them correctly, at times reasoning that if something is obvious it is therefore not novel, or vice versa (Khera et al., 11 Jul 2025). The cited work stresses that novelty requires that no single prior-art disclosure contains all features together, whereas inventive-step reasoning can involve combinations and obviousness analysis (Khera et al., 11 Jul 2025). This distinction is treated as a core patent-law issue and as a litmus test for practical competence.

A second core area is the handling of Article 54(3) EPC, priority, and unpublished applications. The paper highlights repeated failures involving unpublished European patent applications within the 18-month publication period, the interaction between Article 54(3) EPC and Article 56 EPC, and situations in which some claims in an application validly enjoy earlier priority while others do not (Khera et al., 11 Jul 2025). The ability to determine how priority alters prior-art status is presented as a distinctly European and practically consequential reasoning task.

A third dimension is deadline and date calculation. The study reports recurring errors in calculating deadlines, choosing the correct triggering date, accounting for weekends and public holidays including Easter, and drawing the proper procedural consequence from the resulting date (Khera et al., 11 Jul 2025). In this respect, the EQE tests procedural precision as much as substantive doctrine.

The examination also assesses citation discipline and legal-source control. The cited paper reports wrong EPC articles, wrong PCT provisions, wrong subparagraphs, fabricated references, and references from the correct source with incorrect content attached (Khera et al., 11 Jul 2025). This suggests that EQE competence includes not only legal outcome determination but also the correct invocation of operative legal authorities.

These doctrinal and procedural demands underscore why the EQE is described as testing not simply whether a candidate knows patent law, but whether the candidate can apply it correctly to concrete facts. The paper explicitly distinguishes knowing or reciting legal definitions from applying them correctly to fact patterns, and concludes that many models show partial legal-text familiarity without reliable legal application competence (Khera et al., 11 Jul 2025). That same distinction is intrinsic to the examination’s professional function.

4. Claim interpretation, drafting discipline, and internal claim quality

Although the paper on automated claim evaluation is not directly about the EQE, its account of claim-quality dimensions is highly relevant to the examination’s claim-analysis and claim-review aspects. Patent claims are described there as the “most legally significant section” of a patent document because they define the scope of protection and establish the legal boundaries of an invention (Jiang et al., 16 May 2025). That proposition aligns closely with the EQE’s practical orientation toward claim interpretation, amendment, and structured patent drafting.

The paper identifies five expert evaluation criteria for patent claims: feature completeness, conceptual clarity, terminology consistency, logical linkage, and overall quality (Jiang et al., 16 May 2025). These criteria were introduced in the context of Patent-CE, described as the first comprehensive benchmark for evaluating patent claims, and used to compare two candidate claim sets against a reference claim set (Jiang et al., 16 May 2025). Even though this benchmark is aimed at generated patent claims, the same dimensions map directly onto the skills required when EQE candidates assess whether claims are fit for purpose.

The practical significance of each criterion is explicit. Feature completeness concerns “the extent to which the generated claims encapsulate all critical aspects of the invention” (Jiang et al., 16 May 2025). Conceptual clarity concerns “the clarity and unambiguity of the language used in the claims” (Jiang et al., 16 May 2025). Terminology consistency concerns “the uniformity in the use of terms throughout the claims,” and logical linkage concerns “the accuracy with which features are interconnected and related” (Jiang et al., 16 May 2025). Together, these dimensions formalize the proposition that claims are not merely semantic paraphrases of technical disclosure; they are structured legal-technical instruments requiring disciplined wording and internally coherent feature relationships (Jiang et al., 16 May 2025).

This has direct significance for EQE-style work. Candidates are expected not only to identify relevant law but also to scrutinize whether a claim captures the invention’s essential technical features, is clearly expressed, uses terms consistently, and preserves structurally sound dependencies and feature relationships. The claim-evaluation paper’s qualitative example—where a claim is judged better because it uses “an annular edge” rather than “a annular edge,” uses “further comprising,” and employs “wherein” to preserve structural relationships—illustrates the sort of drafting criticism that is germane to professional patent work (Jiang et al., 16 May 2025).

A plausible implication is that the EQE’s patent-drafting and claim-interpretation dimensions can be usefully decomposed into criterion-specific subskills rather than treated as a single undifferentiated capacity. The literature suggests that expert assessment of claims is multidimensional, and that structured review along those dimensions is more reliable than generic fluency or similarity judgments (Jiang et al., 16 May 2025).

5. The EQE as a benchmark for LLMs

The most explicit recent research treatment of the EQE evaluates several open-source and proprietary LLMs on parts of the pre-examination (Khera et al., 11 Jul 2025). The benchmark includes models from the GPT series, Anthropic, Deepseek, and Llama-3 variants, among others (Khera et al., 11 Jul 2025). The central quantitative finding is that OpenAI o1 led with 0.82 accuracy and 0.81 F1 score in the abstract, while AWS Llama 3.1 8B lagged at 0.50 accuracy, and a Python-deployed Llama 3.1 8B scored 0.55 (Khera et al., 11 Jul 2025). The main text reports a minor internal discrepancy, giving o1 0.82 accuracy and 0.84 F1 (Khera et al., 11 Jul 2025).

The study’s principal conclusion is that none of the evaluated models could have passed the examination fully, because accuracy never exceeded the approximately 0.90 average threshold the authors treat as necessary for professional-level standards (Khera et al., 11 Jul 2025). The paper further states that some weaker systems were within the range of mere guessing for the forced-choice design, and notes that random performance is approximately 0.5 in this binary setting (Khera et al., 11 Jul 2025).

The experimental design comprised five experiments: legal-question answering; multimodal full-PDF processing; temperature effects; prompting effects; and platform differences (Khera et al., 11 Jul 2025). For multimodal testing, GPT-4o and Claude 3 Opus were fed complete EQE pre-exam PDFs for 2012–2022, including text, tables, figures, drawings, and full exam formatting (Khera et al., 11 Jul 2025). The paper states that the complete Parts 1–3, totaling 20 questions plus claim drawings, were passed in one shot, with the OCR result for the full paper amounting to about 14,000 tokens (Khera et al., 11 Jul 2025).

The following table condenses several reported model-performance reference points.

Model / setting	Reported result	Context
OpenAI o1	0.82 accuracy; 0.81 F1 in abstract	Legal-part benchmark
OpenAI o1	0.82 accuracy; 0.84 F1 in main text	Legal-part benchmark
AWS Llama 3.1 8B	0.50 accuracy	Legal-part benchmark
Python Llama 3.1 8B	0.55 accuracy	Legal-part benchmark
GPT-4o multimodal	0.70 accuracy	Full-PDF multimodal summary
Claude 3 Opus multimodal	0.67 accuracy	Full-PDF multimodal summary

Beyond overall ranking, the paper reports that DeepSeek-v3 achieved the highest recall = 0.87, though with lower precision due to overpredicting True, while the Python-deployed Llama 3.1 8B had recall = 0.25 because it answered False too often (Khera et al., 11 Jul 2025). The AWS-hosted Llama 3.1 8B produced recall = 0.44 and F1 = 0.44, with a more balanced True/False distribution (Khera et al., 11 Jul 2025). The authors therefore argue that raw accuracy alone is insufficient in this setting (Khera et al., 11 Jul 2025).

Year-specific performance also varied. Across the 12 usable pre-exam years, panel-wide mean accuracy rose from 0.59 in 2012, peaked at 0.77 in 2013, then fluctuated between 0.58 and 0.69, with median = 0.63 (Khera et al., 11 Jul 2025). OpenAI o1 was the yearly benchmark leader in 10 of 12 papers, with average $0.82 \pm 0.07$ (Khera et al., 11 Jul 2025). This suggests both that recent models can solve a substantial portion of EQE-style legal tasks and that the benchmark remains difficult enough to discriminate sharply between systems.

6. Expert judgment, failure modes, and robustness

A major contribution of the EQE study lies in its comparison between automatic scoring and human patent-professional evaluation. Human evaluators scored model-generated justifications from 1 to 10, later normalized to 0–1, using criteria of completeness, conceptual clarity, consistency in terminology, technical correctness, and overall quality (Khera et al., 11 Jul 2025). The evaluators were blind to model identity, and the sampled years for this analysis were 2012, 2014, and 2017 (Khera et al., 11 Jul 2025).

The main qualitative finding is that automatic metrics and expert legal judgment are misaligned (Khera et al., 11 Jul 2025). The experts valued clarity, legal rationale, coherent application of law to facts, references and citations, and reasoning quality, whereas automatic metrics could reward binary correctness or textual overlap even when the reasoning was legally unsound (Khera et al., 11 Jul 2025). The paper explicitly reports negative correlations between correctness and human judgment for some models: Sonnet 3.5: $-0.07$ and Python Llama 3.1 8B: $-0.16$ (Khera et al., 11 Jul 2025). In effect, a correct True/False label could coincide with weak or confused legal reasoning.

The recurring failure modes are doctrinally specific. Models often confused novelty with inventive step, mishandled Article 54(3) EPC, priority, and unpublished applications, failed at deadline and date computation, and produced wrong or hallucinated legal citations (Khera et al., 11 Jul 2025). The paper repeatedly emphasizes the danger of plausible but legally incorrect explanations, including outputs that imitate structured legal analysis without actually performing correct subsumption (Khera et al., 11 Jul 2025).

Robustness is another major concern. The paper reports substantial sensitivity to temperature, prompt wording, formatting conditions, and deployment platform (Khera et al., 11 Jul 2025). In temperature experiments, lower temperature generally yielded more stable outputs, with $T = 0.3$ described as the best “sweet spot” for Llama 3.1 405B and Claude Sonnet 3.5, while intermediate temperatures produced answer drift (Khera et al., 11 Jul 2025). The study describes a verbal prediction reliability measure based on the proportion of times the modal answer recurs across repeated samples, with 1.0 denoting completely deterministic behavior and values below about 0.8 treated as too variable for safe high-stakes use (Khera et al., 11 Jul 2025).

Prompt effects were especially notable for smaller models. In the random-question prompt/no-prompt experiment, Llama 3.1 8B improved by about +12 percentage points, from 0.45 to 0.57, and Mistral 7B improved by about +10 percentage points, from 0.55 to 0.65 (Khera et al., 11 Jul 2025). Platform dependence was also demonstrated by the same Llama 3.1 8B checkpoint behaving differently in AWS-hosted and local Python/Hugging Face deployment settings, which the authors attribute to context-length handling, KV-cache management, bfloat16 storage, and cache retention or eviction policies (Khera et al., 11 Jul 2025).

These results position the EQE not merely as an exam that current systems fail to pass fully, but as a benchmark that exposes where apparently strong legal-language performance breaks down under fact-sensitive, procedural, and multimodal stress.

7. Scope, misconceptions, and contemporary significance

A common misconception would be to treat the EQE as a test of rote legal memory. The cited research contradicts that interpretation. The examination is used precisely because it tests practical legal application: issue-spotting, doctrinal distinction, priority-sensitive prior-art analysis, deadline calculation, and interpretation of claims in light of descriptions, prior art, and drawings (Khera et al., 11 Jul 2025). The benchmark’s difficulty arises less from vocabulary than from structured application of law to facts.

A second misconception would be to assume that strong LLM performance on some legal tasks implies near-professional competence on the EQE. The study explicitly rejects that inference. Despite recent models’ strong performance relative to earlier expectations, no model reached the roughly 0.90 average standard the authors treat as necessary for passing at professional level, and the field “has a long way to go to develop a virtual patent attorney” (Khera et al., 11 Jul 2025). The paper further warns that the general public may overestimate current model performance (Khera et al., 11 Jul 2025).

A third misconception would be to collapse the EQE’s claim-related components into general language evaluation. The claim-evaluation literature shows that patent claims require patent-specific assessment dimensions—feature completeness, conceptual clarity, terminology consistency, and logical linkage—that ordinary semantic or fluency metrics do not capture well (Jiang et al., 16 May 2025). This suggests that any serious understanding of EQE-relevant claim work must distinguish between general textual well-formedness and legally disciplined claim drafting.

The contemporary significance of the EQE therefore extends beyond professional certification. In recent scholarship, it has become an unusually informative benchmark for legal AI because it lies at the intersection of legal doctrine, procedural computation, technical-document interpretation, and domain-specific drafting discipline. It tests whether a system can handle European patent practice as an integrated activity rather than as isolated tasks. At the same time, the literature reinforces that success on component tasks such as claim-quality evaluation does not amount to success on the full legal examination, because the latter also implicates novelty, inventive step, added subject-matter, support, sufficiency, and broader EPC compliance—matters that the claim-evaluation framework does not address (Jiang et al., 16 May 2025).

Taken together, the current literature portrays the EQE as a rigorous professional examination whose value lies in testing precisely those capacities that remain difficult for both humans in training and advanced AI systems: legally precise reasoning, procedural exactness, disciplined claim analysis, and the ability to apply complex patent-law rules to concrete and often multimodal factual records (Khera et al., 11 Jul 2025).