Predicting Metamorphic Relations (PMR)

Updated 10 June 2026

PMR is a research area that automates the discovery of metamorphic relations through techniques like machine learning, search-based optimization, and template synthesis.
Empirical results show that methods such as graph-kernel SVMs and label propagation achieve high accuracy (AUC ≥ 0.9) and effective fault detection.
Key challenges include cross-language transferability, ensuring adequate diversity of generated relations, and scaling solutions to large, industrial codebases.

Predicting Metamorphic Relations (PMR) is a research area at the intersection of software testing and automated analysis, centered on the automated discovery or prediction of metamorphic relations (MRs) for programs lacking reliable test oracles. MRs are necessary properties relating several executions of a program via systematic transformations of inputs and the corresponding relation on outputs. PMR methods enable scalable metamorphic testing by automating or semi-automating the identification of such relations, thus alleviating manual effort and enhancing fault-detection effectiveness.

1. Formalization of Metamorphic Relations in Predictive Frameworks

A metamorphic relation for a program $f: \mathcal{I} \to \mathcal{O}$ is a property specified as a pair $(T,R)$ , where $T: \mathcal{I} \to \mathcal{I}$ is an input transformation and $R \subseteq \mathcal{O}\times \mathcal{O}$ is a binary relation on outputs. The MR holds if

$\forall\,x \in \mathcal{I},\quad \bigl( f(T(x)), f(x) \bigr) \in R.$

Typical MR forms include $f(T(x))=R(f(x))$ for equality relations, or more general predicates on pairs or tuples of outputs. More complex systems may require expressing MRs as $R(f(x_1), f(x_2), ..., f(x_k)) = \mathit{true}$ , involving multiple source and transformed inputs (Li et al., 2024).

The PMR problem is to automatically predict, for a target program $f$ or a component $h$ , which members of a set of predefined MR templates are valid, or to synthesize new MRs by mining code, specifications, or empirical data (Li et al., 2024, Hardin et al., 2018).

2. Empirical Approaches and Machine Learning for PMR

The canonical PMR workflow relies on supervised or semi-supervised classifiers trained on program representations annotated with known MRs. The most influential approach, due to Kanewala et al., consists of extracting features from a program's control-flow graph (CFG) and using those as inputs to multi-label classifiers for MR prediction (Duque-Torres et al., 2022, Duque-Torres et al., 2022). Key feature types include:

Node features: $(\mathrm{label}(v), \mathrm{indegree}, \mathrm{outdegree})$ for each node $(T,R)$ 0 in the CFG.
Path features: label sequences for shortest paths from entry to exit nodes or between CFG nodes.

Classification algorithms include linear or RBF-kernel SVMs, random forests, decision trees, and, in recent work, label propagation for semi-supervised learning when label scarcity is a constraint (Hardin et al., 2018, Duque-Torres et al., 2022). Graph kernels such as the random-walk kernel and graphlet kernel encode control-flow similarity directly within a kernel-SVM framework (Rahman et al., 2018).

Evaluation metrics are standard: accuracy, precision, recall, F1-score, and area under the ROC curve (AUC). In replicated studies, SVMs with random-walk kernels achieve AUC $(T,R)$ 1 on multiple MRs, with best results on permutation and additive/multiplicative relations for scientific numeric methods (Duque-Torres et al., 2022, Duque-Torres et al., 2022).

The table summarizes representative MR classifiers for Java methods (Duque-Torres et al., 2022):

MR	Best Classifier	AUC-ROC	Precision
ADD	Gaussian NB	0.875	0.914
EXC	Random Forest	0.833	0.866
INC	Random Forest	0.717	0.888
MUL	Logistic Reg.	0.833	0.875
PER	Decision Tree	0.857	0.914
INV	Linear SVM	0.857	0.844

CFG-based graph kernels generally outperform simpler source-metric feature models except for invertive (INV) MRs where simpler features suffice (Duque-Torres et al., 2022).

3. Generalizability and Domain-Specific Adaptation

PMR frameworks are effective within data- and language-matched training regimes but demonstrate limited cross-language transferability (Duque-Torres et al., 2022). Rebuilding the classification pipeline and retraining on the target language and domain artifacts are required for accuracy, even when program logic is functionally equivalent (e.g., Java $(T,R)$ 2 Python or C++).

Adaptation to new domains is facilitated by leveraging schema-matching (e.g., for model transformations), domain ontologies, pre/postcondition mining, or natural-language resource mining (e.g., code comments with MeMo or LLM-generated candidates) (Li et al., 2024).

4. Algorithmic Methods Beyond Supervised Learning

Beyond classical supervised classifiers, several alternative paradigms are prominent:

Semi-supervised learning: Label propagation on feature-graph representations, exploiting unlabeled methods to propagate MR applicability, yields significant gains for MRs such as permutation and inversion (accuracy improvement up to 0.10; p-values $(T,R)$ 3) (Hardin et al., 2018).
Search- and Optimization-Based Methods: For numerical programs admitting parametric transformations (e.g., physical simulation code), evolutionary or genetic/Monte Carlo algorithms discover affine or polynomial MRs by minimizing cost functions encoding MR validity and diversity. The cost function typically penalizes trivial solutions and redundancy among discovered transformations (Hiremath et al., 2020, Hiremath et al., 2021).
Pattern- and Template-Based Synthesis: For model transformation programs, generic parameterized MR templates such as AddElement, RemoveElement, UpdateAttribute, and CloneSubgraph are automatically instantiated by analyzing transformation rules and metamodel mappings. Empirical results for this approach demonstrate an MR detection rate of approximately 80% for injected faults and fast, scalable instantiation (Troya et al., 2018).

5. Taxonomy of Predictive and Generative PMR Techniques

State-of-the-art analyses classify PMR techniques into several families (Li et al., 2024):

Composition-Based: Compose new MRs from existing ones if their test chains align.
AI/ML-Based: Use learned models from code/feature datasets.
Pattern/Template-Based: Instantiate abstract MR patterns in context.
Specification/Category-Choice: Generate MRs from structured specifications.
Genetic/Evolutionary/Search-Based: Evolve or search parametric MRs by fitness-driven optimization.
Hybrid and Miscellaneous: Combine static, dynamic, natural-language, and mutation-based data sources; use LLMs and forum mining.

Each approach leverages different data and knowledge bases, from static code artifacts and formal specifications to runtime traces and human-generated documentation.

6. Evaluation, Effectiveness, and Limitations

Empirical evidence demonstrates that PMR frameworks can match or exceed mutation-killing rates compared to manually crafted MRs in scientific code (70–85% mutant-killing by SVM approaches (Li et al., 2024)). Search- or optimization-based PMR on simulation code (e.g., ocean models) rediscovers physical symmetries and exposes faults in boundary condition handling (Hiremath et al., 2020, Hiremath et al., 2021). Graph-kernel SVMs outperform other classifiers for function-level PMR on matrices (AUC: 0.81 for permutative, 0.78 additive, 0.72 multiplicative MRs (Rahman et al., 2018)).

Noteworthy limitations include:

Non-transferability of learned classifiers across programming languages without retraining (Duque-Torres et al., 2022).
Current template-based approaches cover only a subset of MR types (structural rather than behavioral) (Troya et al., 2018).
Small corpus sizes and MR catalogues limit coverage and generalizability (Li et al., 2024, Hardin et al., 2018).
Automated approaches may generate redundant or trivial MRs, necessitating diversity or adequacy measures (Li et al., 2024).
Scalability of combinatorial instantiation for large codebases or metamodels remains challenging (Troya et al., 2018).

7. Open Challenges and Future Directions

Ongoing and emerging challenges in PMR research include:

Adequacy and Diversity Metrics: Formally quantifying whether a set of predicted MRs suffices to exercise program functionality and covers distinct behaviors.
Unification of Hybrid Methods: Integrating ML predictors, search-based optimization, and LLM-driven MR suggestion with traditional static analysis or model-based approaches.
Domain-Specific MR Languages: Designing domain-aligned MR specification notations, especially for emerging areas (e.g., concurrency, AI, cyber-physical systems).
Automation and Scalability: Minimizing manual intervention in both MR template definition and candidate validation, and scaling instantiation to industrial codebases.
MRs Beyond Fault Detection: Leveraging automatically generated MRs for debugging, regression assessment, system understanding, and procurement (Li et al., 2024).

A significant direction is synthesizing domain knowledge, static and dynamic analysis, and data-driven learning in unified PMR frameworks capable of rapidly and reliably generating high-utility MRs for new, complex systems with minimal manual effort.

In sum, Predicting Metamorphic Relations encapsulates a suite of techniques for automating one of the most challenging aspects of metamorphic testing: the identification of non-redundant, high-quality relations that enable systematic test generation and fault detection without explicit test oracles. The spectrum of PMR research spans formal template instantiation, machine and deep learning over code features and graphs, search-based schema, and automation via evolutionary computation and LLMs, with cross-validation against both theoretical criteria and empirical fault-detection benchmarks (Li et al., 2024, Rahman et al., 2018, Troya et al., 2018, Duque-Torres et al., 2022, Duque-Torres et al., 2022, Hardin et al., 2018, Hiremath et al., 2021, Hiremath et al., 2020).