Semantically Equivalent Metamorphic Relation

• SE-MR is a metamorphic relation that converts a source test case into a follow-up case while preserving semantics, ensuring outputs match either exactly or under defined equivalence.
• It is applied in multiple domains such as compiler testing, deep-learning-based code search, and bias evaluation to validate system robustness and semantic fidelity.
• Implementation involves customized test generation, semantic comparators, and automation pipelines that address functional correctness and overcome challenges in semantic validation.

Semantically Equivalent Metamorphic Relation (SE-MR) denotes a metamorphic relation in which a source test case is transformed into a follow-up case that is intended to preserve the relevant semantics of the artifact under test—such as a program, query, prompt, interaction trace, or relational expression—so that the expected relation between outputs is exact equality, semantic equivalence, or another stability notion defined over the same underlying intent. The exact label appears explicitly in data-poisoning detection for deep-learning-based code search, where “the DL-based code search model should return consistent results for two semantically equivalent queries,” but closely related formulations also appear as “semantically equivalent but syntactically different” programs, “equivalent behaviours,” identity-preserving rewrites, and semantically equivalent code perturbations in other metamorphic-testing settings (Chen et al., 15 Jul 2025, Alzahrani et al., 2022, Duque-Torres et al., 2023, Li et al., 17 May 2026, Xue et al., 2024).

1. Conceptual scope and definitional boundaries

Metamorphic testing treats correctness relationally rather than pointwise. Its core object is the metamorphic relation (MR), a property linking multiple inputs with their corresponding outputs. In that sense, an SE-MR is not a single-run invariant but a relation across at least a source execution, a follow-up execution, and an output comparator. The literature is explicit that “MRs are properties of the function under test,” that they “relate multiple inputs to their expected outputs,” and that “any property” is not automatically an MR; the relation must connect multiple executions in a way that constrains expected behavior under transformation (Alzahrani et al., 2022).

The term is broader than exact-output invariance. In some domains, semantic preservation means literal output equality; in others it means equality under an abstraction function or domain-specific equivalence predicate. The binary-search-tree example in property-based metamorphic testing makes this distinction explicit: two output trees may need only contain the same keys and values to count as “semantically equivalent,” even when raw structure differs. Review work on MR generation places the closest general neighbors of SE-MR in equality/equivalence output patterns and in symmetry patterns where the system “appears the same” from different viewpoints, which situates SE-MR as a semantics-preserving subclass within the wider MR landscape (Alzahrani et al., 2022, Li et al., 2024).

Historically, the label is not standardized. Some papers introduce SE-MR explicitly, while others instantiate the same idea without naming it. The result is a concept with stable semantics but heterogeneous vocabulary: semantic equivalence, semantic fidelity, meaning-preserving transformations, equivalent behaviours, relational equivalence, and output-preserving invariance all play overlapping roles depending on the domain (Chen et al., 15 Jul 2025, Akhond et al., 23 Nov 2025, Duque-Torres et al., 2023, Li et al., 17 May 2026).

2. Formal structure

The generic metamorphic-testing pattern can be written as a source-to-follow-up transformation

$x' = T(x)$

followed by an expected relation on outputs

$R(f(x), f(x')).$

For SE-MRs, the most direct specializations are

$f(x) \equiv f(x') \quad\text{or}\quad f(x)=f(x'),$

depending on whether the oracle is semantic equivalence or exact equality. This is the pattern reconstructed in property-based metamorphic testing, including compiler testing with a “semantically equivalent but syntactically different” program and BST insert-order relations whose intended comparator is semantic equivalence over represented key-value content (Alzahrani et al., 2022).

A closely related formulation writes metamorphic testing as

$f(g(X)) = h(f(X)).$

When the output transformation is restricted to the identity map, the SE-MR case becomes

$h=\mathrm{id} \Rightarrow f(g(X)) = f(X),$

which is exactly the output-preserving form studied in ocean-model metamorphic testing. There, the target is not arbitrary equality but invariance under physically motivated symmetries of the modeled system (Hiremath et al., 2020).

Automated MR-generation work also uses the pairwise implication form

$R_i(x_1,x_2) \Rightarrow R_o(f(x_1),f(x_2)),$

which is broad enough to include both exact-equivalence and non-equivalence relations. A canonical SE-MR instance is the square-function example

$(x_1 = -x_2) \Rightarrow (f(x_1)=f(x_2)),$

where distinct inputs are semantically equivalent with respect to the output property under test (Ayerdi et al., 2023).

In relational domains, the strongest formal analogue of SE-MR is denotational equality under identity-preserving rewrite rules:

$$\forall D.\ \mathrm{eval}(E,D)=_{\mathrm{bag}}\mathrm{eval}(E',D).$$

This is the exact pattern used for relational query optimizers in the relational-equivalence block of NOETHER, where source and follow-up expressions differ syntactically but are required to agree under all valid evaluation contexts of the underlying semiring (Li et al., 17 May 2026).

3. Canonical SE-MR patterns across domains

The concept appears in multiple technical settings with different transformed artifacts and different equality notions.

Domain	Transformation	Expected relation
Compiler and BST testing (Alzahrani et al., 2022)	Semantically equivalent program rewrite; operation reordering	Same output or semantically equivalent tree
Ocean modeling (Hiremath et al., 2020)	Physical symmetry $g$ with $h=\mathrm{id}$	$R(f(x), f(x')).$0
Code search poisoning detection (Chen et al., 15 Jul 2025)	Semantically equivalent follow-up queries	Consistent source and follow-up rankings
LLM-assisted coding (Akhond et al., 23 Nov 2025)	Semantically equivalent specification and test mutations	Semantic fidelity and cross-variant consistency
LAPR robustness (Xue et al., 2024)	Semantically equivalent buggy-code perturbations	Repair behavior should remain stable
Relational query optimization (Li et al., 17 May 2026)	Identity-preserving query rewrite	Exact bag-semantic equality
Bias testing in LLMs (Salimian et al., 29 Nov 2025)	Semantically aligned prompt variants	Consistent fairness behavior
Accountable socio-legal software (Tizpaz-Niari et al., 2024)	Syntactically different but semantically equivalent cases	Same decision or legally constrained relation

The strongest explicit SE-MR in conventional software testing is the compiler example: a source program is modified into a “semantically equivalent but syntactically different” program, there are “two paths that are expected to lead to the same output,” and the compiler is faulty if the outputs are “not exactly the same” (Alzahrani et al., 2022). The BST case sharpens the notion of semantic equivalence by distinguishing structural identity from equality of represented mappings: insertion orders may differ syntactically while the resulting trees remain equivalent as maps (Alzahrani et al., 2022).

In scientific computing, SE-MR-like reasoning often takes the form of invariance under symmetry. Ocean-model testing explicitly restricts attention to the case $R(f(x), f(x')).$1, so the transformed input is supposed to preserve the output exactly. Sign changes, coordinate transformations, scaling of $R(f(x), f(x')).$2 and $R(f(x), f(x')).$3 that preserve the ratio $R(f(x), f(x')).$4, and translations under cyclic boundary conditions all instantiate output-preserving semantic transformations at the level of the modeled physical quantity (Hiremath et al., 2020).

In machine-learning systems, SE-MRs frequently shift from input-object semantics to intent semantics. MT4DP treats natural-language queries as semantically equivalent when they preserve retrieval intent, and regards large discrepancies in code ranking as evidence of backdoor poisoning rather than normal retrieval variation (Chen et al., 15 Jul 2025). LLM-assisted coding uses MRs as “semantic operators” that create semantically equivalent variants of task descriptions before generation and semantically constrained test cases afterward, effectively moving SE-MR from post-hoc validation into prompt/specification mutation (Akhond et al., 23 Nov 2025). Robustness testing for LLM-powered automated program repair applies semantically equivalent code perturbations—renaming, expression rewriting, loop transformation, comment insertion, and related edits—to buggy programs and treats repair instability across those variants as robustness failure (Xue et al., 2024).

In security and accountability settings, SE-MR becomes more nuanced because preserving intent need not imply identical outputs. Web-security metamorphic testing includes cases where added attack syntax should be sanitized away so that source and follow-up outputs remain equal or safely rejected; those are strong SE-MR instances. By contrast, same-resource access under different credentials preserves action syntax but not security subject, so it is better regarded as a near-equivalent transformation with policy-governed output divergence (Chaleshtari et al., 2022). Socio-legal debugging makes the distinction explicit by separating “similar cases should yield similar rulings” from legally relevant counterfactuals that should induce ordered output changes rather than equality (Tizpaz-Niari et al., 2024).

4. Construction, representation, and automation

A standard implementation workflow for SE-MRs in property-based metamorphic testing is: “Specify an MR property,” “Customize the test case generator,” “Customize the test case shrinker,” and “Run the checker.” The generator must produce valid source cases in the semantic domain of the relation, while the shrinker must preserve invariants rather than collapsing counterexamples into invalid artifacts. The paper is explicit that validity preconditions matter and that semantic comparators often need to replace raw structural equality (Alzahrani et al., 2022).

For Web systems, MST-wi provides a domain-specific language, SMRL, in which inputs are interaction sequences and follow-up cases are typically created by copying a valid source input and then applying targeted modifications. The DSL includes declarative operators such as CREATE, EQUAL, and IMPLIES, along with Web-specific utilities such as changeCredentials, copyActionTo, setChannel, setSession, and EncodeUrl. Source inputs are collected automatically from Crawljax-generated interaction graphs, and the MRs are transformed into executable Java code and run as JUnit tests (Chaleshtari et al., 2022).

In LLM-centered workflows, automation is often split into mutation, validation, generation, and evaluation. CodeMetaAgent uses four modules—Mutator, Reviewer, Generator, and Evaluator—to construct semantically equivalent task descriptions $R(f(x), f(x')).$5 or transformed test cases $R(f(x), f(x')).$6, filter prompt/specification variants with a BERT/Sentence-BERT similarity threshold of $R(f(x), f(x')).$7, regenerate inconsistent variants for at most three iterations, and then perform cross-variant consistency checks on generated outputs (Akhond et al., 23 Nov 2025). The “complete metamorphic testing pipeline” agenda extends this idea by proposing machine-readable MR representation, a DSL bridge for MR databases, equivalence-oriented MR mining from Javadoc through MeMo, and explicit constraint handling for MR applicability (Duque-Torres et al., 2023).

At the most formal end of the spectrum, NOETHER treats MR discovery as a deductive downstream step from an upstream operator algebra. Its Translate operator converts block invariants into executable MRs, while CONSTRUCT-MP quotients them into MetaPatterns. For SE-MR-like cases, the relevant source is the relational-equivalence block $R(f(x), f(x')).$8, whose identity-preserving rewrite rules mechanically induce exact denotational-equality MRs (Li et al., 17 May 2026).

5. Empirical performance and adequacy evidence

The first explicit SE-MR paper in the supplied corpus is MT4DP. Its SE-MR states that a code-search model “should return consistent results for two semantically equivalent queries,” and operationalizes inconsistency by generating two follow-up queries—one synonym-based, one mask-based—re-ranking the source top-50 list, and measuring divergence with Hybrid Similarity Variation (HSV). On CodeBERT, MT4DP improves over the best baseline by about 191% average $R(f(x), f(x')).$9 and 265% average precision; the best HSV weighting is

$f(x) \equiv f(x') \quad\text{or}\quad f(x)=f(x'),$0

which gives more weight to rank-position change than to raw similarity-score change (Chen et al., 15 Jul 2025).

Bias testing for black-box LLMs uses six semantically aligned MRs to transform direct bias-inducing questions into harder variants and then checks whether fairness behavior remains stable. The framework reports up to 14\% more hidden biases than existing tools and, when the same MR-generated data are reused for mitigation, increases safe response rates from 54.7\% to over 88.9\% across fine-tuned models. Semantic filtering is itself imperfect: in a manual audit of $f(x) \equiv f(x') \quad\text{or}\quad f(x)=f(x'),$1 MR-transformed questions, 41 were filtered out, and most failures came from the Attribute Flip MR (Salimian et al., 29 Nov 2025).

MT-LAPR demonstrates the same phenomenon for code. Nine semantically equivalent perturbations at token, statement, and block levels reveal that 34.4\% - 48.5\% of transformed test cases expose instability in LLM-powered automated program repair on average. The paper also shows that training a CodeT5-based readability editor on MR-generated equivalent code pairs can increase robustness by 49.32\% at most, which connects SE-MR not only to testing but also to robustness lifting (Xue et al., 2024).

Web-security work provides large-scale evidence that MR catalogs can scale operationally. MST-wi reports a catalog of 76 system-agnostic MRs, covers 39\% of the OWASP security testing activities not automated by state-of-the-art techniques, and can automatically discover 102 different types of vulnerabilities, corresponding to 45\% of the vulnerabilities due to violations of security design principles according to MITRE CWE. An earlier Web-systems security-testing catalog defined 22 system-agnostic MRs and detected 10 out of 12 vulnerabilities across two systems (Chaleshtari et al., 2022, Mai et al., 2019).

A distinct question is not whether an SE-MR finds bugs, but whether an MR set is semantically adequate. The Semantic Mutation Score (SMS) addresses this by replacing classical AST-level mutation adequacy with a semantics-aware mutant space:

$f(x) \equiv f(x') \quad\text{or}\quad f(x)=f(x'),$2

Equivalence is judged conservatively through $f(x) \equiv f(x') \quad\text{or}\quad f(x)=f(x'),$3: AVP coherence under the MR set and output equivalence over $f(x) \equiv f(x') \quad\text{or}\quad f(x)=f(x'),$4 samples within tolerance $f(x) \equiv f(x') \quad\text{or}\quad f(x)=f(x'),$5. SMS degenerates almost everywhere to classical Mutation Score in a characterized limit, but the paper’s main empirical point is that semantic mutant classes such as Hyperparameter, Structural Injection, and Trajectory Flip are unreachable under default first-order syntactic mutation, so semantics-aware adequacy can diverge substantially from syntactic adequacy (Li et al., 17 May 2026).

6. Limitations, misconceptions, and open technical issues

The first limitation is terminological. SE-MR is not yet a universal taxonomy item. A recent survey on MR generation does not list “semantic-equivalence MR” as a formal category, even though it repeatedly points to equality/equivalence output patterns and symmetry-based patterns that function as its closest conceptual neighbors (Li et al., 2024). The result is a literature in which the concept is common but the label is uneven.

A second issue is that semantic equivalence is often assumed rather than proved. The LLM-assisted coding framework validates specification mutations using a similarity threshold of $f(x) \equiv f(x') \quad\text{or}\quad f(x)=f(x'),$6 and oracle execution, but explicitly provides no formal semantics of equivalence (Akhond et al., 23 Nov 2025). Bias testing relies on a Claude 3 Opus semantic filter and finds that 7.7\% of audited mutants fail equivalence screening, with Attribute Flip causing most failures (Salimian et al., 29 Nov 2025). MT-LAPR presents several code rewrites as semantically equivalent, but some are only conditionally so in languages with nontrivial typing and arithmetic semantics (Xue et al., 2024). These are practical equivalence approximations, not theorem-proved identities.

A third issue is applicability. The BST commutation formulas in property-based metamorphic testing illustrate the general problem: relations written as equalities may only hold under side conditions such as key distinctness or abstract-map comparison rather than exact tree-shape equality (Alzahrani et al., 2022). “Bug or not Bug?” turns this into an explicit research problem by using association-rule mining to refine overgeneral MRs into conditional ones, distinguishing likely software faults from MR/data mismatches and deriving support, confidence, and lift for candidate applicability conditions (Duque-Torres et al., 2023).

A fourth issue is the persistent need for human judgment. The ChatGPT experience report on MR generation shows that GPT-4 can produce correct MRs and even some novel ones for previously untested systems, but also that the majority of candidates are often vague, repetitive, incorrect, or unjustifiable; human experts remain necessary to justify, rectify, and refine them (Luu et al., 2023). The broader MR-generation review makes the same point structurally: full automation remains difficult because MR validity is grounded in software semantics and domain knowledge rather than syntax alone (Li et al., 2024).

Finally, adequacy itself is still unsettled. SMS introduces a semantics-aware mutant space and a three-layer attribution classifier for separating true semantic failures from tolerance, OOD, statistical, and artifact categories, but the paper also reports that its pre-registered large-effect threshold for Cliff’s delta is not met and that many cells in the experimental design have SMS $f(x) \equiv f(x') \quad\text{or}\quad f(x)=f(x'),$7. This suggests that SE-MR research has progressed from intuition to operationalization, but not yet to a settled theory of semantic coverage or completeness (Li et al., 17 May 2026).