IBM watsonx Code Assistant for Z

• WCA4Z is a modernization platform that segments mainframe transformation into distinct phases of analysis, refactoring, translation, and validation.
• Its multi-stage workflow employs a Class Designer for mapping COBOL to Java alongside a Refactoring Assistant for precise API signature inference.
• Automated functional validation and quality evaluation leverage symbolic execution and LLM-based insights to build trust through measurable, evidence-driven metrics.

IBM watsonx Code Assistant for Z (WCA4Z) is an end-to-end modernization platform for mainframe applications that is described as analyzing source code and dependencies using tooling such as ADDI, supporting refactoring and service extraction from monolithic COBOL/PL/I systems, and offering a two-phase transformation path into Java. Across the available research literature, WCA4Z is presented not as a single translation model but as a modernization stack spanning discovery, refactoring, paragraph-level COBOL-to-Java transformation, automated validation, and longitudinal quality evaluation. A recurring theme is that modernization trust is established through evidence-producing workflows rather than through a proof of semantic equivalence, which the authors explicitly state is infeasible in general (Froimovich et al., 31 Jul 2025, Kanvar et al., 2024, Hans et al., 14 Apr 2025).

1. Platform scope and modernization model

WCA4Z is framed as a modernization platform for IBM Z applications in which analysis, refactoring, transformation, and validation are tightly related but technically distinct stages. In the modernization path described for COBOL-to-Java conversion, the first phase, Class Designer, analyzes the full COBOL program and proposes a Java class design and mappings; the second phase translates paragraph-level logic into Java methods. The evaluation literature is explicit that the paragraph-to-method stage is driven by a prompt containing COBOL plus class-designer-produced auxiliary mappings, sent to a fine-tuned LLM, with the output then postprocessed into a Java method (Froimovich et al., 31 Jul 2025).

This decomposition matters because the surrounding papers treat WCA4Z as a system of cooperating components rather than a monolithic model. The testing framework paper describes a Class Generator that analyzes COBOL record structures, paragraphs, variables, and control units, generates Java classes, member variables, and method signatures, and produces a structural mapping called CJMap. The Refactoring Assistant paper, by contrast, concentrates on identifying refactorable slices and computing API signatures for extracted functionality rather than on Java generation itself. Taken together, these accounts suggest an architecture in which discovery, interface inference, translation, and validation are separate but composable stages.

A common misconception is that WCA4Z is presented as a fully autonomous converter from legacy source to production-ready modern services. The papers do not make that claim. Instead, they repeatedly describe semantically guided, human-in-the-loop workflows, explicit dependence on mappings and benchmark datasets, and the continued role of developers in reviewing outputs, ambiguous interface fields, and production hardening.

2. Refactoring Assistant and API enablement

One important WCA4Z capability concerns “APIfication” of legacy mainframe applications: identifying candidate functionality inside a monolithic z/OS application, computing request and response fields, refactoring the exposed code and its callers, and enabling exposure through tools such as z/OS Connect. The artifact sources used to identify candidate APIs are stated as transactions and jobs, screens, control-flow blocks, procedures, inter-microservice calls, business rules, data access points, and client requirements / externally driven integration needs. The implementation of API signature computation is explicitly said to be part of IBM watsonx Code Assistant for Z Refactoring Assistant (Kanvar et al., 2024).

The signature computation is defined over a selected code region, abstractly lines $p$ to $q$. A field $f$ is a request field if there exists a path from $p$ to $q$ with an r-value occurrence of $f$ not preceded by an l-value occurrence of $f$. A field $f$ is a response field if there exists a path from $p$ to $q$ with an l-value occurrence of $q$0, and there exists a path from $q$1 to application exit that later contains an r-value occurrence of $q$2. The paper compares flow-insensitive, flow-sensitive, and path-sensitive formulations conceptually, although path-sensitive analysis was not implemented because it is too expensive for large software. The formal flow-sensitive request analysis is given as

$q$3

$q$4

with the flow-insensitive block-level approximation

$q$5

and

$q$6

The paper also introduces an optional surety flag. If a field is only read along all paths from $q$7 to $q$8, it is marked as request with optional = false; if it is only written along all paths, it is marked as response with optional = false; if it is both read and written along some path, optional = true is used and final judgment is left to the user. This reflects the paper’s broader position that static analysis can reduce manual effort without eliminating expert review.

The empirical case studies emphasize why this matters in practice. GENAPP is reported as having 36 programs, 5 copybooks, 8 DB2 tables, and 7,566 variables; the telecom bill-proration application has 34 programs, 134 copybooks, 123 DB2 tables, and 11,543 variables; CBSA has 46 programs, 9 CICS transactions, 3 DB2 tables, and 31 copybooks. In GENAPP, two copybooks, LGCMAREA and SSMAP, total 593 fields, whereas inferred API signatures for the studied APIs used far fewer, with the minimum number of linkage section request/response fields reported as 2 and the maximum as 91. The automated method found 15 transaction-based APIs, 21 procedure-based APIs with no data access, and 21 data-accessing APIs in GENAPP, matching all manually found transaction APIs and achieving 100% coverage for GENAPP transaction APIs. For the qualitative study, 9 practitioners with average ~15 years of experience reported that manual endpoint and signature work would take at least one week for GENAPP and several weeks for the private industry application, whereas the automated approach was believed to reduce such tasks to only a few minutes.

The strongest caution in this line of work is that only API signature computation is explicitly claimed as part of WCA4Z Refactoring Assistant. The full candidate-identification-plus-refactoring-plus-exposure pipeline is described as a framework and demonstrated with tooling and z/OS Connect, but not as a fully integrated push-button capability.

3. COBOL-to-Java transformation workflow

The COBOL-to-Java path described for WCA4Z is deliberately staged. Class Designer analyzes the full COBOL program, proposes the Java class design, and produces mappings that are then supplied as context for paragraph-level translation. The transformation target in the testing literature is a selected COBOL paragraph and its translated Java counterpart, with CJMap encoding correspondences among COBOL records and Java classes, COBOL paragraphs and Java methods or functions, and COBOL variables and Java variables, fields, or parameters (Hans et al., 14 Apr 2025).

This structure is consequential for both generation and downstream validation. Because the LLM receives the paragraph together with design-time mappings, translation is not modeled as unconstrained source-to-source conversion. Instead, it is conditioned on a prior structural decomposition. A plausible implication is that translation quality depends not only on the model itself but also on the quality of Class Designer output and associated mappings; this is consistent with the evaluation paper’s claim that the reporting stack helps teams determine whether a failure originates in the LLM, the Class Designer mappings, middleware handling, or other infrastructure.

The same staging also defines the boundary of what is being evaluated. The automated quality-evaluation paper is explicit that its main focus is the paragraph-to-method translation stage rather than proof of whole-program equivalence. The automated testing paper narrows the unit further to paragraph or method level for symbolic execution, mock generation, and JUnit synthesis. These boundaries are significant because they explain why the system can scale to many benchmark datapoints while still leaving full-program execution and production hardening as open engineering problems.

4. Automated functional validation

The testing framework around WCA4Z is described as automating the process of testing the functional equivalence of the translated Java code against the original COBOL programs in an industry context. Its end-to-end architecture consists of Class Generator, Model, COBOL Test Generator, COBOL Runner, Resource Mapper, JUnit Generator, JUnit Runner, and Fault Localization. The framework uses static symbolic execution to generate unit tests for COBOL, mocking external calls and transforming them into JUnit tests to validate semantic equivalence with translated Java, while treating COBOL as the behavioral oracle (Hans et al., 14 Apr 2025).

Static symbolic execution is used to determine execution paths under a branch coverage criterion, construct path constraints, and solve them with Z3. The generated test artifacts include program inputs, resource inputs, program outputs, and resource outputs. The design is notable for byte-level modeling of COBOL data semantics: every data item is represented as a sequence of bytes, each byte is an 8-bit BitVector in Z3, and type constraints encode COBOL representations such as integers, decimals, Zoned Decimal, Packed Decimal, REDEFINES, and PICTURE clauses. The authors report an Intermediate Representation with data types aligned to COBOL and 55 statement types, as well as catalogs of use-def behavior for enterprise-specific constructs such as CICS and IMS.

Execution of generated COBOL tests uses debugger-based control on IBM Z. The debugger sets breakpoints, injects program inputs and resource-input values at runtime, captures resource outputs at designated statements, and records program outputs at path end. The JUnit Generator then uses COBOL inputs, COBOL outputs, CJMap, and CJResourceMap to synthesize Java tests with Mockito-based mocking for database interactions, file system calls, platform-specific libraries such as CICS and JZOS, and generated mock implementations of Z libraries for use on non-Z platforms. When tests fail, the system compares expected Java statements to executed Java statements and uses llama-3-70b-instruct for candidate-fault explanation.

The reported benchmark results are extensive. For a Genapp + IMS Bank subset, the paper reports 33 COBOL programs, 105 paragraphs, and 51 paragraphs with multiple execution paths; 43 paragraphs (84%) achieved full branch coverage, and 46 paragraphs (90%) exceeded 80% branch coverage. On the IBM Compiler Benchmark dataset, consisting of 54,284 COBOL programs and 94,524 paragraphs, 87,725 paragraphs (92.8%) achieved full branch coverage; for 85,657 paragraphs, test generation took less than 1 second. On the StarCoder dataset, covering 936 COBOL programs and 2,871 paragraphs, 2,707 paragraphs achieved full branch coverage, mean branch coverage was 97.19%, mean time was 1.28 seconds, median was 0.14 second, and the maximum was 176 seconds. Representative end-to-end JUnit validation results include LGACDB01, where 4 COBOL test cases yielded 4 JUnit tests and all passed, and CHANN11, where 2 JUnit tests both failed and exposed an incorrect loop-condition translation. Productivity estimates for Genapp validation scenarios include LGAPDB01 from 55 min manually to 10 min automated, LGACVS01 from 35 min to 8.5 min, LGIPVS01 from 45 min to 8 min, and LGUCDB01 from 20 min to 8 min.

The papers are careful not to overstate these results. Passing tests increase confidence but do not prove equivalence; path-space and data-space remain limited, and exact transfer of COBOL control semantics to Java is sometimes impossible, especially where COBOL branches on exact SQLCODE values and Java exposes generic exceptions instead.

5. Automated quality evaluation and reporting

A separate line of work presents an automated evaluation system for assessing COBOL-to-Java code translation within WCA4Z. This evaluation framework is intentionally decoupled from the product itself and is organized as a data-centric ETL pipeline built around a relational database. A dedicated testing driver substitutes for the IDE, runs benchmark translations, gathers outputs and metadata, and emits them into a .jsonl file. A file listener monitors a shared Git repository, retrieves new files, validates them, extracts evaluation points, loads them into the database, and then triggers configured evaluators. The resulting metrics, errors, and coverage metadata feed analysis modules and Grafana dashboards (Froimovich et al., 31 Jul 2025).

The benchmark model is static and coverage-oriented. Datasets correspond either to applications such as GenApp or to language-feature slices such as “basic COBOL” or “CICS.” Selected paragraphs are the datapoints. Coverage is defined hierarchically over the input COBOL: top-level categories such as basic COBOL or SQL, subcategories such as individual statements or CICS operation types, and finer sub-subcategories such as IF-with-ELSE, nested IF, or complex-condition IF. This permits versioned comparison of evaluation sets under transformation configurations such as model version and backend version.

Methodologically, the evaluation framework combines three evaluator families: static analytic checkers, dynamic compilation/execution checks, and LLM-as-a-judge (LaaJ). The syntactic analytic layer checks for empty output, endlessly repeated text, parseability under the Tree-sitter Java parser, and the presence of at least one executable statement. The semantic analytic layer traverses an IBM ADDI-derived COBOL control-flow graph and the Java parse tree to check variable access matching, procedure invocation matching, and middleware call matching, with hallucination detection derived from these checks. Middleware handling is especially distinctive: the system extracts ordered sequences of EXEC SQL, EXEC CICS, and some IMS calls from COBOL and Java, aligns them with the Needleman–Wunsch dynamic programming algorithm, and uses the alignment to detect correct translations, missing translations, hallucinated Java middleware calls, and parameter mismatches. Dynamic testing is implemented through a framework that can compile and run both COBOL and Java on Z platforms, generating jobs and scheduling them through Galasa on z/OS, but the paper states that this family currently contributes mainly as supplemental signals around compilability and design consistency.

The most elaborate subjective layer is LaaJ. The judge prompt asks specifically for “correctness of the control flow in the Java code translation” and uses a seven-point ordinal scale developed with domain experts. Score 1 denotes no attempt at translation; 2 denotes an attempted translation with entirely fabricated classes or methods; 3 denotes partial elements of correct translation but major errors or hallucinations; 4 denotes mostly correct translation with moderate errors or hallucinations; 5 denotes mostly accurate translation with minor errors or hallucinations; 6 denotes functionally equivalent translation with verbosity, non-idiomatic constructs, or harmless hallucinations; and 7 denotes a fully accurate, functionally equivalent, concise, and idiomatic translation. Judge prompts include domain-specific instructions, for example to ignore COBOL EXIT statements and Java TODO comments, and the authors validate the judge against human expert evaluations and “partial order benchmarks” in which translation variants $q$9, $f$0, and $f$1 are expected to satisfy an ordinal quality relation.

Reporting is stakeholder-specific. Project managers are given high-level dashboards with aggregate metric comparisons across model versions and benchmarks; technical leads receive an “all samples view” over every evaluation point in a benchmark; engineers receive a “single sample debug view” containing the original COBOL, translated Java, mappings, checker scores, reported errors, and side-by-side comparison of two translations. Coverage-based reports include a heatmap of weighted average LaaJ score by COBOL statement and hierarchical coverage frequencies for benchmark-authoring teams. The system is explicitly described as supporting continuous integration workflows and large-scale, frequently executed regression tests.

6. Significance, failure modes, and limitations

Across the three papers, WCA4Z emerges as a modernization environment whose core problem is not merely code generation but controlled reduction of modernization risk. The Refactoring Assistant work addresses the problem of determining safe API boundaries and signatures in legacy applications with weak naming conventions, large shared copybooks, monolithic COBOL programs, mixed record layouts, and terminal-oriented interaction models. The testing work addresses the trust problem for translated Java through symbolic execution, debugger-based mocking, and cross-language test transfer. The evaluation work adds a trust layer for modernization by enabling scalable benchmarking, regression testing, historical comparison across versions, and reduction of dependence on scarce COBOL SMEs (Kanvar et al., 2024, Hans et al., 14 Apr 2025, Froimovich et al., 31 Jul 2025).

The principal failure modes discussed in the literature are correspondingly broad. At the syntactic level they include empty responses, repeated-text degeneration, unparsable code, and code with no executable substance. At the semantic level they include omitted variable reads and writes, incorrect data binding against provided mappings, missing or malformed Java method calls for COBOL PERFORMs, missing middleware operations, extra hallucinated middleware operations, parameter mismatches on middleware calls, incorrect exception-type selection in Java for CICS semantics, and hardcoded values replacing intended inputs. In testing, additional failure modes include assertion failure, path divergence, semantic mismatches around SQLCODE-specific handling, and loss of assertability when a COBOL-observed variable becomes a Java local temporary.

The limitations are explicit and consequential. Full semantic equivalence cannot be established. The quality-evaluation paper rejects any single “silver bullet,” including formal equivalence checking, full compile-and-execute testing, reference translation matching, manual SME review, or pure LLM-as-a-judge. The testing paper states that performance and security are out of scope and repeatedly acknowledges no full functional proof, limited path-space and data-space exploration, and scalability stress from large arrays and loops. The Refactoring Assistant paper notes that path-sensitive analysis was not implemented, that strongest evidence comes from field counts and qualitative time savings rather than randomized controlled task studies, and that the “without call chain” variant works best largely because the studied systems were organized so requests appear before and responses after calls. It also states that correctness validation of exposed APIs was manual and done on a subset of APIs.

For that reason, WCA4Z should not be understood as a formal verifier or as a fully autonomous modernization pipeline. The research record supports a narrower but technically substantial characterization: WCA4Z combines discovery and dependency analysis, code slicing and API-signature inference, paragraph-level COBOL-to-Java transformation, automated differential testing, analytic and LLM-based evaluation, and dashboard-driven regression analysis. This suggests a modernization strategy in which trust is accumulated through partial semantic checks, controlled execution evidence, versioned benchmarking, and human review of the remaining ambiguities rather than through any single definitive correctness guarantee.