---
title: 'TestMigrationsInPy: unittest to pytest'
url: https://www.emergentmind.com/topics/testmigrationsinpy
type: topic
---

# TestMigrationsInPy: unittest to pytest

TestMigrationsInPy is a curated dataset of real-world Python test migrations from `unittest` to `pytest`, designed to support research on automated framework migration and test-refactoring tools [2602.05122]. It addresses the problem of how to build, evaluate, and compare automated tools that migrate Python tests from `unittest` to `pytest` using real developer changes as ground truth. The dataset contains 923 real-world migrations performed by developers, collected from 100 highly popular Python software systems, found in 37 projects, and derived from 690 migration commits. Its distinctive features are before/after code pairs, migration-type labels, and manual curation of isolated migrations, which together make it suitable for evaluating migration systems ranging from simpler assertion migrations to more complex fixture migrations [2602.05122].

## 1. Research context and motivation

The motivation for TestMigrationsInPy is that `unittest` and `pytest` are the most popular testing frameworks in Python, while migration from `unittest` to `pytest` is often time-consuming, incremental, and error-prone [2602.05122]. `pytest` is presented as attractive because it offers less verbose tests, simpler assertions, fixture support, and interoperability with existing `unittest` tests, allowing gradual migration. In the formulation used for the dataset, `unittest` tests are usually written inside classes that inherit from `TestCase`, whereas `pytest` tests can be plain functions; `unittest` uses specialized methods such as `self.assertEqual` and `self.assertIn`, whereas `pytest` allows ordinary Python `assert` statements; and `pytest` fixtures often replace `setUp`, `tearDown`, and `setUpClass` [2602.05122].

The ecosystem significance of this migration pattern is reinforced by prior evidence that 27% of the top-100 most popular Python projects had migrated or were migrating to `pytest` [2602.05122]. Related work on large-scale migration in Python also shows that migration is rarely a trivial substitution problem. In the LSST modernization effort, migration testing exposed issues involving global state, randomness, shared filesystem paths, and process-local behavior, illustrating that framework migration is not only about language syntax changes [1712.00461]. This suggests that a ground-truth dataset of real test migrations is useful not merely for rewriting assertions, but for studying broader structural and behavioral transformations.

A common misconception is that migration from `unittest` to `pytest` is just syntax replacement. The source material explicitly rejects that view. Test framework migration can involve converting class-based tests to function-based tests, replacing `self.assert*` methods with plain `assert`, and moving `setUp` and `tearDown` logic into `@pytest.fixture` functions; the last of these is described as much harder because there is no direct one-to-one mapping between unittest setup/teardown and Pytest fixtures [2602.02964].

## 2. Dataset composition and curation model

The headline structure of TestMigrationsInPy is:

| Item | Value |
|---|---|
| Real-world migrations | 923 |
| Highly popular Python software systems | 100 |
| Projects | 37 |
| Migration commits | 690 |

The relationship is summarized as $100\ \text{projects} \rightarrow 690\ \text{migration commits} \rightarrow 923\ \text{isolated migrations}$ [2602.05122]. The 100 systems were selected from the authors’ earlier empirical study on Python test migration.

The dataset was built in two major steps. First, the authors automatically detected migration commits by reusing the migration-detection tool from Barbosa and Hora. This tool mines the version history using PyDriller, scans added and removed lines per commit, and identifies commits as migration commits if they contain at least one migration type from the dataset’s taxonomy [2602.05122]. Second, the authors manually identified isolated migrations. Because a migration commit may contain multiple migrations plus unrelated changes, they manually inspected commits and extracted isolated migrations, defined as migrations that simply replace `unittest` by `pytest` and nothing else unrelated. Commits modifying more than 5 test files were excluded in order to reduce noise [2602.05122].

This curation procedure is central to the dataset’s use as ground truth. The reported rationale is that verified inputs and outputs are necessary for automated migration evaluation, and manually isolating clean transformations makes the examples more suitable for benchmarking than tangled repository diffs [2602.05122]. A plausible implication is that the dataset is oriented toward evaluating transformation correctness under controlled conditions rather than reconstructing the full socio-technical process of repository-wide migration.

## 3. Migration taxonomy and transformation granularity

TestMigrationsInPy explicitly defines five migration types used by the detection tool and the dataset [2602.05122].

| Migration type | Definition |
|---|---|
| Assert migration | removes `unittest` `self.assert*`; adds `pytest` `assert` statements |
| Fixture migration | removes `unittest` setups/teardowns; adds `pytest` fixtures |
| Import migration | removes `import unittest`; adds `import pytest` |
| Skip migration | removes `unittest` skip usage; adds `pytest` skip usage |
| Expected failure migration | removes `unittest` expected-failure usage; adds `pytest` expected-failure usage |

These labels provide what the source describes as a taxonomy of migration difficulty. Assert migrations are often simpler and more local, while fixture migrations are harder because there is no direct one-to-one mapping between `unittest` lifecycle methods and `pytest` fixtures [2602.05122]. Import, skip, and expected-failure migrations capture other framework-specific constructs. The dataset is therefore not limited to assertion rewriting; it also includes test lifecycle and control-flow conventions that are framework-specific.

This taxonomy aligns with a broader empirical view of migration complexity in Python. The study on Python library migrations introduces PyMigTax, a taxonomy in which migration-related code changes may involve function call, function reference, type, exception, attribute, decorator, and import, together with properties such as argument transformation and output transformation [2207.01124]. Although PyMigTax concerns Python library migrations rather than test framework migration, it provides relevant context: real migrations frequently combine several kinds of edits, and many cannot be reduced to one-to-one function-call replacement. TestMigrationsInPy adopts a narrower but operationally useful taxonomy tailored to `unittest`-to-`pytest` transformation [2602.05122].

An important interpretive point is that fixture migration is treated as structurally harder than assertion migration. That distinction is not merely descriptive; it supports benchmarking tools at different difficulty levels, from local assertion replacement to transformations involving `setUp`, `tearDown`, fixture injection, and test restructuring [2602.05122].

## 4. Repository organization and representative examples

TestMigrationsInPy is organized as a GitHub repository. For each project, there is a list of migration commits; each migration commit contains a list of isolated migrations; and each isolated migration includes the before code in `unittest` form and the after code in `pytest` form [2602.05122]. The repository structure includes `diff/`, which stores the migration code with before and after test files, and `output.info`, a summary file containing commit hash, number of changed files, number of migrations in the commit, and migration type [2602.05122].

The paper reports representative examples that illustrate the dataset’s range. A simple assertion migration comes from Saleor: one migration commit containing four migrations. The example includes changes such as `self.assertEquals(func, oauth_callback)` becoming `assert func is oauth_callback`, `self.assertIn(a, b)` becoming `assert a in b`, and `self.assertIsInstance(a, b)` becoming `assert isinstance(a, b)` [2602.05122]. These examples show that even assertion migration may require expression restructuring rather than method-name substitution alone.

A more complex example comes from Dash: four migration commits, with the first commit containing nine migrations. That example includes removal of a `TestCase` class, replacement of a `setUp` method with a `pytest` fixture, conversion of a test method into a plain test function, and use of fixture injection by function parameters [2602.05122]. The paper also highlights a case from pyvim in which a `setUp` method was split into four separate pytest fixtures: `prompt_buffer`, `editor_buffer`, `window`, and `tab_page`. This demonstrates that fixture migration may require structural decomposition [2602.05122].

These examples help clarify what the dataset captures as “isolated migrations.” The emphasis is on transformations where `unittest` is replaced by `pytest` without unrelated code changes. That makes the before/after pairs well suited to supervised learning, rule inference, or direct evaluation of automated migration systems.

## 5. Use in automated migration research

The primary intended role of TestMigrationsInPy is as a ground-truth resource for future research proposing novel solutions to migrate frameworks in Python [2602.05122]. Because it provides real developer-produced migrations, before/after pairs, migration-type labels, and manual curation to isolate clean transformations, it is suitable for training or evaluating machine learning and LLM-based migration tools, studying which migration types are easier or harder, and measuring accuracy on assertion-only migrations versus more complex fixture migrations [2602.05122].

The paper “Testing Framework Migration with Large Language Models” operationalizes this use case directly. It introduces a curated dataset of real-world migrations extracted from the top 100 Python open-source projects and then executes LLM-generated test migrations in their respective test suites [2602.02964]. From the 923 real-world migrations, additional filtering yielded 40 migrations suitable for execution-based evaluation, drawn from seven projects and classified into 30 simple and 10 complex cases [2602.02964]. Across 480 LLM migration requests, 233 passed and 247 failed, corresponding to 48.5% passed and 51.5% failed [2602.02964]. The reported outcome is that LLMs can accelerate migration, but human review is still essential.

The same study also found that coverage remained exactly the same for all successful migrations [2602.02964]. This is significant because it suggests that, when execution succeeds, the migrated tests usually preserve intended test behavior rather than merely producing syntactically acceptable code. The reported failure categories—AssertionError, Structural Mismatch, Missing Fixtures, TypeError, Signature Drift, and SyntaxError—indicate that errors are concentrated in dependency handling, fixture adaptation, and setup inconsistencies [2602.02964]. These findings are consistent with the taxonomy in TestMigrationsInPy, especially the distinction between simpler assert migration and harder fixture migration.

The dataset paper also reports an initial LLM-based usage: GPT-4o was used with TestMigrationsInPy to migrate tests from `unittest` to `pytest`, and the initial results suggest GPT-4o can accelerate migration, but it may produce minor incorrect updates, especially for fixture migrations [2602.05122]. This suggests that the dataset is already functioning both as a benchmark and as a diagnostic instrument for error analysis.

## 6. Relation to broader migration benchmarks and methodologies

TestMigrationsInPy belongs to a broader class of migration benchmarks, but its scope is distinctive. CODEMENV benchmarks large language models on code migration across software environments, with 922 examples spanning 19 Python and Java packages and three core tasks: identifying functions incompatible with specific versions, detecting changes in function definitions, and adapting code to target environments [2506.00894]. Its logic is organized around detection, diagnosis, and repair. By contrast, TestMigrationsInPy focuses specifically on framework migration from `unittest` to `pytest`, grounded in real developer-performed transformations rather than version-specific API evolution [2602.05122].

This contrast is methodologically important. CODEMENV evaluates migration with exact-match function localization, structured change classification, and unit-test-backed migration validation [2506.00894]. TestMigrationsInPy instead provides before/after test code and migration-type labels, enabling evaluation of framework-specific refactorings such as assertion conversion, fixture introduction, and skip or expected-failure replacement [2602.05122]. A plausible implication is that TestMigrationsInPy is especially suitable for studying source-to-source refactoring and test modernization, whereas CODEMENV addresses version-aware environment migration.

Another relevant comparison is IntentTester, an intent-driven framework for cross-library and cross-language test reuse. IntentTester abstracts tests into a language-agnostic Test Description Language, aligns them with semantically related entities in a repository graph, and synthesizes executable tests through LLM-guided reasoning and iterative validation [2606.25588]. That work concerns cross-library and cross-language migration, including Python, and reports 2,776 syntactically correct tests with 85% correctness and 74% effectiveness [2606.25588]. TestMigrationsInPy is narrower in scope but more tightly grounded in one concrete migration axis: `unittest` to `pytest`. This suggests a complementary relationship: TestMigrationsInPy provides a focused corpus of framework transformations, while broader systems such as IntentTester explore semantic transfer beyond framework boundaries.

Historical migration experience in LSST provides further context. There, modernization included a move from exit-status-based script execution to `pytest`, together with JUnit-compatible reporting, `pytest-xdist`, and `pytest-flake8` [1712.00461]. That example shows that adoption of `pytest` in large systems can be part of a broader modernization pass rather than an isolated framework switch. TestMigrationsInPy captures the code-level manifestations of such shifts in a form usable for comparative research.

## 7. Limitations, validity considerations, and prospective extensions

The limitations discussed for TestMigrationsInPy are explicit. The dataset is drawn from 100 highly popular projects, which do not represent the entire Python ecosystem, and less popular projects may have different migration practices [2602.05122]. This introduces popularity bias. The authors therefore suggest expanding the dataset to include more projects for better coverage [2602.05122].

A second limitation concerns taxonomy granularity. The current classification could be made more specific, for example by distinguishing `setUp`, `setUpClass`, and `tearDown` on the `unittest` side, and pytest features such as parametrized tests on the `pytest` side [2602.05122]. This indicates that the current five-type taxonomy is useful but not exhaustive. A plausible implication is that future versions could support more fine-grained evaluation of migration difficulty and idiomatic `pytest` adoption.

A third concern is possible residual noise. Although the dataset filters tangled changes and manually curates isolated migrations, any manual curation process can still miss edge cases [2602.05122]. The construction also relies on a migration-detection tool that reportedly has 100% precision and recall for detecting migrations [2602.05122]. Within the confines of the reported methodology, this supports confidence in the mining stage, but it does not eliminate all downstream curation risks.

More broadly, empirical work on Python migration warns against overly narrow assumptions. The characterization of Python library migrations shows that real migrations often involve non-function program elements, higher-cardinality mappings, and combinations of properties such as argument transformation and output transformation [2207.01124]. While that result concerns library migration rather than test framework migration, it strengthens the view that migration benchmarks should evolve toward richer transformation taxonomies. In the case of TestMigrationsInPy, future extensions that distinguish finer-grained fixture patterns, parametrization, and other `pytest` idioms would be consistent with that broader empirical lesson.

In summary, TestMigrationsInPy is significant because it combines a real-world corpus, explicit migration-type labels, manual isolation of clean transformations, and code-level before/after pairs [2602.05122]. Its main limitation is representativeness: it currently reflects a subset of the ecosystem, primarily popular projects. Even so, it provides a concrete and reusable benchmark for research on automated migration, LLM-based test refactoring, and comparative evaluation of framework-migration tools.

Source: https://www.emergentmind.com/topics/testmigrationsinpy