Methods2Test Corpus Overview

• Methods2Test Corpus is a large-scale, metadata-rich dataset that maps unit tests to their focal methods in Java and Python.
• It employs heuristic mapping, combining name matching and AST-based extraction, to reliably trace test cases to specific implementation units.
• The dataset enables advanced test generation research by providing detailed contextual, structural, and tokenized representations for machine learning models.

The Methods2Test corpus is a large-scale, metadata-rich supervised dataset mapping unit test cases to the focal methods they exercise within source code. Designed to drive the development and evaluation of machine learning models for automated unit test generation, it provides explicit traceability between test cases and underlying implementation units, with rich contextual and structural information. Methods2Test initially targets Java (JUnit settings) (Tufano et al., 2022), with pyMethods2Test as an analogous resource for Python (pytest and unittest) (Abdelmadjid et al., 7 Feb 2025). These resources supply not only precise pairings but complex contextual inputs, facilitating advanced sequence-to-sequence and retrieval-based test synthesis research.

1. Construction Methodology and Mapping Heuristics

Java (Methods2Test)

The Methods2Test corpus derives its 780,944 unique test-to-focal-method mappings from 91,385 Java projects, selecting only repositories with open-source licenses permitting redistribution and recent maintenance activity. The key challenge is reliably associating each JUnit test (@Test-annotated) with a single focal method in the corresponding production class.

The heuristic-based mapping proceeds in two stages:

• Class-level mapping: The focal class $C^f$ for a test class $C^t$ is identified through a sequence of heuristics—(1) path matching (mirroring src/test/java to src/main/java), and (2) name matching (removing Test prefixes/suffixes).
• Method-level mapping: For test method $t$, let $M^f$ be the set of methods in $C^f$, $\text{name}(\cdot)$ the identifier, and $\text{stripTest}(\cdot)$ the removal of "Test" from names. Two heuristics are applied:
  • $H_1$ (Name Matching): $$H_1(t) = \{ m \in M^f \mid \text{name}(m) = \text{stripTest}(\text{name}(t)) \}$$. If $|H_1(t)| = 1$, that method is selected.
  • $H_2$ (Unique Method Call): $H_2(t) = \text{Inv}(t) \cap M^f$ (methods called by $t$ and defined in $C^f$); if $|H_2(t)| = 1$, choose it.
  • If neither heuristic yields a unique method, the mapping is discarded.

Python (pyMethods2Test)

pyMethods2Test encompasses 2,198,378 mappings from 88,846 open-source Python repositories. Test file identification leverages AST-parsed import statements and filename patterns associated with pytest (test_*.py/*_test.py) or unittest. Test-method extraction looks for:

• Top-level def nodes beginning with test in pytest files,
• unittest.TestCase subclasses with methods whose names start with test.

The mapping process involves:

• Focal-file resolution: Preferring explicit local imports, filename-based matching, or fallback to fuzzy string similarity (Levenshtein distance) between test and candidate source files.
• Focal-class resolution: Matching explicit class references or using AST ancestor-walking and method-name overlap.
• Focal-method resolution: Assembling $Invoked(t)$—the set of called project methods from within the test. Candidates $C$ are those methods where the name matches the test (suffix or fuzzy similarity ≥50). The best match is chosen by maximum similarity or omitted if no suitable candidate is found.

A summary of the mapping workflow for both Java and Python is given below:

Language	Test ID Heuristics	Focal Class/File Heuristics	Focal Method Heuristics
Java	@Test annotation	Path & class-name matching	Name, then unique call, match
Python	Import+AST+filename	Import match, name match, fuzzy filename	Suffix/fuzzy name match among invoked methods

2. Dataset Scale, Coverage, and Distribution

Methods2Test (Java) deduplicates from an initial 887,646 candidate mappings to 780,944 one-to-one test-focal pairs. Python’s pyMethods2Test identifies 22,662,037 test methods, with 2,198,378 valid focal method mappings (approximately 9.7% of test methods).

Dataset splits are engineered by repository to prevent data leakage:

Set	#Repositories	#Mapped Pairs (Java)
Training	72,188	624,022
Validation	9,104	78,534
Test	10,093	78,388
Total	91,385	780,944

Coverage for pyMethods2Test reveals:

• 1,289,630 test files (pytest or unittest) out of 18,517,737 .py files,
• 22,662,037 test methods from 222,020,293 total methods,
• Two-thirds of test files use pytest, one-third unittest,
• All intermediate AST data are distributed for transparency.

An independent manual assessment of Java mappings yields a correctness estimate of 90.72% (95% confidence, ±10%) (Tufano et al., 2022).

3. Metadata Schema and Context Extraction

Java (Methods2Test)

Every mapping is encoded in a per-pair JSON, partitioned into three tiers:

• Repository-level: id, url, language, is_fork, fork_count, stargazer_count.
• Class-level (focal/test): identifier, superclass, interfaces, fields, methods, source file.
• Method-level (focal/test): identifier, parameters, signature, body, testcase flag, constructor flag, invoked methods.

Example excerpt:

{ "repo": {"id": 12345, "url": "...", ...}, "focal_class": {"identifier": "Calculator", ...}, "test_class": {"identifier": "CalculatorTest", ...}, "focal_method": {"identifier": "add", ...}, "test_method": {"identifier": "testAdd", ...} }

Python (pyMethods2Test)

Repository-level focal mappings are stored as single JSONs keyed by test file path. Entries include test method boundaries (line numbers, indentation), focal class path, and focal method position within the production file.

Example (abridged):

{ "tests/unit/metrics/test_ffwd.py": { "focal_file": "gordon/metrics/ffwd.py", "methods": { "test_ffwd_protocol_connection_made": { "line": 23, "line_end": 32, "indent": 0, "focal_class": "gordon.metrics.ffwd.UDPClientProtocol", "focal_method": { "name": "connection_made", "line": 59, "line_end": 67, "indent": 4 } } } } }

4. Context Representation and Tokenization

To facilitate neural modeling, both datasets support extraction of “focal context” at varying degrees of granularity.

Java (Methods2Test): Five hierarchical encodings, each as raw source text and Byte-Level BPE tokenized forms:

1. fm: Focal method body only.
2. fm+fc: Prepend focal class name.
3. fm+fc+c: Prepend constructor signatures.
4. fm+fc+c+m: Prepend other public method signatures.
5. fm+fc+c+m+f: Prepend public field declarations.

Sequences $S_0, \ldots, S_4$ are tokenized and truncated to ≤1024 tokens, then binarized via fairseq. This organization is designed for transformer architectures (e.g., BART, T5, AthenaTest).

Python (pyMethods2Test): A script emits context snippets per mapped test:

1. Focal class declaration,
2. Focal method body,
3. __init__ method (if present),
4. Signatures of other class methods,
5. Class attributes and observed instance attributes.

This context is intended as an LLM prompt input, for training, evaluation, or benchmarking of automated test-generation capabilities.

5. Quality Assurance, Limitations, and Use Cases

Quality and Noise

For Java, mapped pairs achieve 90.72% correctness upon manual inspection. Python mapping quality is limited by AST-based extraction (files with syntax errors are dropped), incomplete import resolution (dynamic imports/aliasing cause both noise and omissions), and imperfect fuzzy matching (e.g., test/method name similarity errors).

Limitations

• Language/framework coverage: pyMethods2Test covers only pytest and unittest; other frameworks (nose, doctest) and metaprogrammed or dynamically generated tests are not included.
• Mapping specificity: Tests invoking multiple methods are assigned only a single best match.
• Naming conventions: Non-standard or factory-generated test methods may be missed.
• AST-based extraction: Non-syntactic files and complex code constructs may be omitted.

Research and Engineering Applications

• Training and fine-tuning LLMs for test generation (Java and Python),
• Empirical studies of naming conventions, coverage, framework adoption,
• Mining for test smells, refactoring, and API usage patterns,
• Automated tools for test recommendation or skeleton generation,
• Educational resources: corpus of real-world, context-linked test cases.

For Java, AthenaTest provides a baseline transformer model; comparative evaluations with search-based approaches (EvoSuite, Randoop) or structural neural models are recommended (Tufano et al., 2022).

6. Distribution, Licensing, and Access

Methods2Test is publicly available at https://github.com/microsoft/methods2test with Zenodo archival (DOI here), and includes both raw/tokenized corpora and fairseq binaries for rapid research iteration (Tufano et al., 2022).

pyMethods2Test is available at Zenodo, with all focal mappings, intermediate AST extractions (~42 GB), and context generation scripts included (Abdelmadjid et al., 7 Feb 2025).

Licensing for both datasets restricts included repositories to those with explicit open-source redistribution compatibility, ensuring compliance and reproducibility.

7. Impact and Comparative Perspective

Methods2Test and its Python analogue, pyMethods2Test, address a critical bottleneck in data-driven automated unit test generation: scale-coupled traceability and contextual richness. They enable systematic pretraining and evaluation of LLMs and transformer-based architectures for code-to-test translation across major programming ecosystems. Comparative resources for Python are uniquely scarce, making pyMethods2Test notable for bridging this gap. This design facilitates empirical, reproducible, and scalable research at the intersection of program analysis, machine learning, and software engineering (Tufano et al., 2022, Abdelmadjid et al., 7 Feb 2025).