DeepAssert: Automated Assertion Generation

• DeepAssert is a framework that automatically generates executable assertions from program context or design intent in both Java unit tests and RTL verification.
• It leverages transformer (BART) and LLM-based methodologies to improve test coverage, accuracy, and bug localization compared to traditional approaches.
• Evaluations show notable gains, with increased exact-match accuracy and enriched coverage metrics, demonstrating practical advantages over baseline assertion generation methods.

DeepAssert denotes automated assertion-generation research in which learned models synthesize executable assertions from program context or design intent. In the literature considered here, the name refers both to a BART-based method for generating JUnit assert statements from a test method and its focal method in Java unit testing (Tufano et al., 2020) and to an LLM-aided verification framework that extracts module-level specifications and generates fine-grained SystemVerilog deep assertions for internal RTL modules in assertion-based verification (ABV) (Wang et al., 18 Sep 2025). Across these usages, the unifying objective is to produce assertions that encode expected behavior more effectively than coverage-oriented test scaffolding, top-level-only checks, or direct one-shot generation.

1. Scope and research context

Assertions function as test or verification oracles: in unit testing they specify the expected behavior of the unit under test, while in ABV they formalize design intent as SystemVerilog Assertions (SVAs). The broader literature treats weak or incorrect assertions as a central bottleneck. In software testing, automatically generated asserts have been criticized as weak, incomplete, or overly simple, and one empirical study on LLM-generated HumanEval assertions reported that over 62% of generated assertions were incorrect, either because they did not compile or because they failed on the ground-truth solution (Yu et al., 6 Jan 2025). In RTL verification, methods based only on top-level specifications are described as unable to generate deep assertions for internal module functionality, while RTL-based methods may inherit errors from buggy implementations and may require a golden RTL model (Wang et al., 18 Sep 2025).

Within this context, the term “DeepAssert” is used for two technically distinct systems.

Usage of DeepAssert	Primary setting	Core output
BART-based DeepAssert	Java unit testing	JUnit assert statement
LLM-aided DeepAssert	RTL ABV	Module-level deep SVA

The software-testing variant treats assertion generation as a sequence-to-sequence prediction problem over a test method with a placeholder and its focal method. The hardware-verification variant treats deep assertion generation as a staged inference problem over design specifications, RTL hierarchy, port relations, and extracted module-level specifications. A plausible implication is that the common label marks an assertion-centric research agenda rather than a single canonical architecture.

2. DeepAssert for unit-test assertion generation

In the software-testing line, DeepAssert is a transformer-based approach for generating Java unit-test assertions from a Test-Assert Pair: $TAP_i = \{ tm'_i, fm_i, a_i \}$ where $tm'_i$ is the test method with the assert replaced by <AssertPlaceholder>, $fm_i$ is the focal method under test, and $a_i$ is the developer-written assert. The source sequence is $s_i = tm'_i + fm_i$, the target sequence is $t_i = a_i$, and the model learns $P(a_i \mid tm'_i + fm_i)$ with cross-entropy loss (Tufano et al., 2020).

The architecture is based on BART, specifically a large model with a 12-layer encoder and 12-layer decoder, GeLU activations, and denoising pretraining. The training pipeline has three stages. First, the model is pretrained on 160GB of English text for 40 epochs with a corruption process that masks 30% of tokens, uses masked spans whose lengths follow a Poisson distribution with $\lambda = 3$, and permutes all sentences. Second, it is pretrained on 25GB of Java code from 26,000 repositories for 10 epochs, using noise that deletes 20% of tokens independently and rotates half of all documents. Third, it is fine-tuned for assert generation on a dataset mined from more than 9,000 GitHub projects, using methods annotated with @Test, retaining only cases with exactly one assert statement and identifying the focal method with a heuristic based on the last method invocation before or within the assert (Tufano et al., 2020).

Four variants are evaluated: BART_Scratch, BART_English, BART_Code, and BART_English+Code. The paper attributes the gains primarily to transfer learning: English pretraining contributes strong semantic priors over identifier and method names, source-code pretraining improves Java-specific structure, and the focal method provides essential semantic context. The reported comparison without the focal method states that including the focal method improves accuracy by about 10%, indicating that the task is not reducible to local test-method completion (Tufano et al., 2020).

The work also positions generated assertions as practical augmentations to automated test generation. In an EvoSuite-based experiment on Defects4J using 18 unique focal methods of NumberUtils, adding one predicted assert increased line and/or condition coverage for 13 out of 18 methods, with improvements of 1–3 lines and 1–4 conditions in the reported cases (Tufano et al., 2020).

3. DeepAssert for module-level deep assertions in RTL verification

In the RTL-verification line, DeepAssert is an LLM-aided assertion-generation framework for ABV whose stated target is the generation of deep assertions over internal module signals rather than only top-level architectural signals. The problem statement emphasizes two limitations in prior work: top-level specification methods such as AssertLLM and Spec2Assertion do not naturally generate deep assertions for internal module functionality, and RTL-based methods may align assertions with buggy implementations and often rely on a golden RTL model (Wang et al., 18 Sep 2025).

DeepAssert is defined as a four-step workflow.

First, LLM₁: Relationship Extractor analyzes module invocation relationships, module port information, and signal propagation changes across interfaces: $$RE(\mathcal{R}) \rightarrow \{\mathcal{L}^r, \mathcal{L}^p, \mathcal{L}^s\}$$ with $$\mathcal{L} := (\mathcal{L}^r, \mathcal{L}^p, \mathcal{L}^s)$$. The framework assumes that module input/output ports are correct in practical verification settings (Wang et al., 18 Sep 2025).

Second, LLM₂: Specification Extractor infers a dedicated specification for each module from the original design specification $tm'_i$0, the extracted relationship information $tm'_i$1, and a module $tm'_i$2: $tm'_i$3 The key methodological choice is that module-level meaning is inferred from design intent plus hierarchy and port context, rather than from internal RTL implementation details (Wang et al., 18 Sep 2025).

Third, LLM₃: Verification Item Extractor converts each extracted module specification into atomic verification items: $tm'_i$4 The paper describes these as concise condition-result statements. One reported example is: if any command like start, stop, read, or write is asserted, then go should be high and cmd_ack low (Wang et al., 18 Sep 2025).

Fourth, LLM₄: Deep Assertion Generator produces module-level deep assertions: $tm'_i$5 The inputs include the original specification, RTL, extracted relationships, module specification, and verification items. To reduce hallucination, the generation step uses a standard template

$tm'_i$6

for signal references (Wang et al., 18 Sep 2025).

The resulting assertions are characterized as module-level, signal-level, fine-grained, and deep. Their practical significance is tied to bug proximity: the paper argues that deep assertions are closer to the actual bug location, trigger earlier, and simplify debugging. It explicitly contrasts them with weak forced deep assertions such as $t_i = a_i$0 which it criticizes as primarily checking bit-width rather than meaningful internal behavior (Wang et al., 18 Sep 2025).

4. Quantitative evaluations and reported behavior

For the unit-test DeepAssert, the headline result is 62.47% top-1 exact-match accuracy for BART_English+Code, with top-$tm'_i$7 results of 77.94% at top-5, 80.79% at top-10, 83.12% at top-25, and 84.74% at top-50. The paper compares this with ATLAS values of 26.40%, 41.76%, 46.83%, 51.89%, and 54.89%, respectively, and summarizes the top-1 gain as 80% relative improvement over ATLAS. Pretraining effects are also explicit: top-1 accuracy is 37.77% for BART_Scratch, 59.44% for BART_Code, 60.75% for BART_English, and 62.47% for BART_English+Code. Syntax Top-1 is reported between 99.54% and 99.58% across variants, and validation loss decreases from 0.67 for BART_Scratch to 0.13 for BART_English+Code (Tufano et al., 2020).

The hardware-verification DeepAssert is evaluated on 20 IWLS 2005 designs using Cadence JasperGold 21.12.002 and GPT-4o, with detailed results highlighted for I2C, Pairing, ECG, and SHA3. For deep assertion generation alone, the reported results are: I2C $tm'_i$8, $tm'_i$9, $fm_i$0, $fm_i$1, $fm_i$2; Pairing $fm_i$3, $fm_i$4, $fm_i$5, $fm_i$6, $fm_i$7; ECG $fm_i$8, $fm_i$9, $a_i$0, $a_i$1, $a_i$2; and SHA3 $a_i$3, $a_i$4, $a_i$5, $a_i$6, $a_i$7. The paper states that prior methods perform poorly when forced to generate deep assertions directly, with related coverage metrics near 0% in many cases (Wang et al., 18 Sep 2025).

DeepAssert is also evaluated as an enhancement layer for existing RTL assertion generators. For AssertLLM + DeepAssert, the paper reports, for example, I2C $a_i$8 80.23% → 82.84%, $a_i$9 82.26% → 83.87%, $s_i = tm'_i + fm_i$0 78.93% → 80.61%; SHA3 $s_i = tm'_i + fm_i$1 92% → 100%, $s_i = tm'_i + fm_i$2 90.24% → 95.12%. For Spec2Assertion + DeepAssert, examples include I2C $s_i = tm'_i + fm_i$3 87.87% → 89.51%, $s_i = tm'_i + fm_i$4 89.44% → 91.05%; Pairing $s_i = tm'_i + fm_i$5 66.67% → 80.66%; and SHA3 $s_i = tm'_i + fm_i$6 92.24% → 97.83%, $s_i = tm'_i + fm_i$7 90.89% → 96% (Wang et al., 18 Sep 2025).

A further reported behavior concerns debugging latency. In the SHA3 padder module, a top-level assertion reportedly took 10 clock cycles to trigger and only indicated that an error exists, while a module-level deep assertion triggered in 2 clock cycles and identified the error signal directly. The paper uses this example to support the claim that deep assertions improve both error detection latency and localization (Wang et al., 18 Sep 2025).

5. Relation to adjacent assertion-generation research

The unit-test DeepAssert emerged in a sequence of work on learned test oracles. ATLAS framed assert generation as neural machine translation over a test method and focal method, achieving 31.42% top-1 exact match and 49.69% top-5 on its abstract representation, with the central claim that meaningful assert generation is complementary to test generation rather than a replacement for it (Watson et al., 2020). SAGA extended this direction by adding developer-written summarization of the focal method to the input and reported 53.1% accuracy, 75.56 BLEU-4, and 85.96 ROUGE-L on CAPS$s_i = tm'_i + fm_i$8, together with statistically significant gains from summarization for neural models (Zhang et al., 2023).

Retrieval-augmented approaches then shifted the field from pure generation toward prototype reuse and semantic adaptation. EditAS argued that prior integration methods succeeded mainly because retrieval often found a nearly correct assertion; it reported 53.46 accuracy and 80.77 BLEU on Data_old, and 44.36 accuracy and 63.46 BLEU on Data_new (Sun et al., 2023). RetriGen combined hybrid lexical and embedding-based retrieval with a CodeT5 generator, reporting an average 57.66% accuracy and 73.24% CodeBLEU across two Java datasets (Zhang et al., 22 Feb 2025). AG-RAG introduced a dense CodeT5-based retriever jointly trained with the generator and reported 64.59% accuracy and 80.86% CodeBLEU on old, 56.33% accuracy and 68.11% CodeBLEU on new, and detection of 41 Defects4J bugs versus 21 for EditAS (Zhang et al., 15 Feb 2025).

In RTL ABV, the most immediate adjacent work is CoverAssert, which extends LLM-based SVA generation with coverage-driven iteration. Its six-module pipeline combines semantic feature extraction, AST-based structural features, clustering, specification splitting into Sub-SPECs, functional point extraction, assertion-to-spec mapping, and a coverage-driven feedback loop that iterates until a threshold $s_i = tm'_i + fm_i$9 is reached. When integrated with AssertLLM and Spec2Assertion, it reports average improvements of 9.57 % in branch coverage, 9.64 % in statement coverage, and 15.69 % in toggle coverage (Wang et al., 8 Apr 2026).

These results arise from different tasks, datasets, and evaluation protocols. This suggests that DeepAssert is best situated within a broader methodological progression: direct seq2seq assertion generation, retrieval-augmented editing, summarization-guided conditioning, and, in hardware, module-specification extraction plus coverage-driven iterative refinement.

6. Limitations, evaluation caveats, and significance

Several evaluation caveats recur across DeepAssert-related work. In unit-test assertion generation, exact-match metrics systematically undercount semantically correct outputs. The software DeepAssert paper gives examples such as assertTrue(status == 0) versus assertEquals(0, status) and assertTrue(ps1 == ps2) versus assertSame(ps1, ps2), which are treated as incorrect by exact-match evaluation despite encoding the same logic. It also notes one benchmark method, createBigDecimal(String), for which no correct assert was present in the top-10 predictions (Tufano et al., 2020). Closely related work makes the same point, reporting that many predictions marked incorrect are semantically equivalent or differ only in assertion form, message strings, or equivalent null checks (Zhang et al., 2023).

The RTL DeepAssert framework introduces a different set of assumptions. It explicitly assumes that module input/output ports are correct. The framework is also LLM-driven, so its behavior depends on prompt quality, model capability, and resistance to hallucination. The paper further notes a practical scalability constraint: mutation testing on Pairing was excluded from one experiment because it would require over 120 hours (Wang et al., 18 Sep 2025).

The significance of DeepAssert therefore lies less in a single definitive metric than in a shift of emphasis. In software testing, DeepAssert demonstrates that assertion generation benefits from large-scale transfer learning and from conditioning on the focal method rather than only the test prefix. In RTL verification, DeepAssert demonstrates that deep assertions can be generated by extracting module-level meaning from design intent and structural context, thereby moving verification from coarse top-level checking toward module-local, signal-level properties. Taken together, these lines of work define DeepAssert as an assertion-centric approach to learned verification, spanning both software test oracles and hardware ABV.