CLARA: Automated Repair for Intro Programming

• CLARA is a semantic-driven automated repair system for introductory programming assignments that clusters correct solutions to compare against incorrect submissions.
• The system employs internal representations and flexible control flow alignment to overcome structural differences in student code, enabling precise minimum repairs.
• Empirical evaluations demonstrate that extended CLARA significantly increases repair success rates while highlighting challenges in source-level readability for novices.

CLARA is a data-driven automated program repair and feedback system for introductory programming assignments. Later descriptions of the framework characterize it as clustering correct solutions by semantic equivalence or execution status and comparing an incorrect submission with representative or nearest correct programs to derive a repair; the system has consequently been used both as a research baseline for automated feedback and as a deployable service inside notebook-based programming environments (Lu et al., 2021, Kurniawan et al., 2023). A 2024 extension further adapted CLARA to real-world introductory programs by adding flexible control flow graph alignment and broader abstract syntax tree support, reporting a substantially higher percentage of successful repairs than baseline CLARA on a Codeforces benchmark (Chowdhury et al., 2024).

1. Problem setting and role in introductory programming

CLARA addresses a recurrent problem in introductory programming education: large courses can provide immediate compiler or interpreter diagnostics and can run predefined test cases, but these mechanisms usually indicate that a program is wrong without adequately localizing the logical error or suggesting how to proceed. In the feedback taxonomy used by later educational work, CLARA goes beyond simple “knowledge about mistakes” and aims to provide “knowledge on how to proceed” in the form of a suggested repair (Kurniawan et al., 2023).

This framing places CLARA between conventional autograders and manual tutoring. It does not replace syntax diagnostics or test-based feedback; rather, later classroom deployments treat it as an additional layer that students can invoke when they are stuck. A plausible implication is that CLARA is best understood as an automated semantic-debugging aid for novice code, rather than as a general-purpose program-repair engine for arbitrary software systems.

2. Core repair model and internal representations

Later comparative work describes CLARA as a semantics-driven repair system. Correct solutions are first clustered based on semantic equivalence or equivalent execution status, and an incorrect program is then compared with representative programs from those clusters to produce a minimum repair (Lu et al., 2021). A separate deployment study summarizes the same mechanism at a higher level: correct solutions are reduced into clusters, each cluster “intuitively representing a different solution approach,” and an incorrect submission is matched to the nearest cluster representation to generate feedback (Kurniawan et al., 2023).

The same comparison paper further states that CLARA parses source code into an internal representation, interprets that representation, records intermediate execution status such as variable values, and uses those recorded program states to determine whether two programs implement the same semantics (Lu et al., 2021). This semantics-first design is one of CLARA’s main distinguishing characteristics. It allows comparison across programs that may be syntactically different but semantically equivalent, which later work explicitly identifies as a strength.

The cost of that design is that repairs are naturally expressed at the level of the internal representation rather than the student’s original source code. Educational deployments observed feedback containing internal identifiers such as iter#0 and ind#0, which made the messages harder for novices to interpret (Kurniawan et al., 2023). This tension between semantic flexibility and source-level readability recurs throughout subsequent research on CLARA.

3. Structural assumptions and the flexible alignment extension

The 2024 extension of CLARA targets a specific limitation: existing automated-repair approaches, including baseline CLARA, are limited because they only compare programs with similar control flow and order of statements (Chowdhury et al., 2024). In introductory programming, however, students often reach the same task semantics through materially different statement orderings, branch structures, or loop organizations. The extension is explicitly motivated by the set of repair feedback that is lost when those structural differences block comparison.

The extension introduces several modifications to CLARA to deal with real-world introductory programs. First, it extends CLARA’s abstract syntax tree processor to handle common introductory programming constructs. Second, it proposes a flexible alignment algorithm over control flow graphs in which nodes are enriched with semantic annotations extracted from programs using operations and calls. Third, once an alignment has been found, the incorrect program’s control flow graph is modified to match the correct programs so that CLARA’s original repair process can be applied on top of the transformed graph (Chowdhury et al., 2024).

This extension preserves CLARA’s original repair pipeline while relaxing one of its most consequential structural assumptions. In effect, the repair system shifts from requiring near-isomorphic control flow toward an alignment-based comparison of semantically annotated control flow graphs. The implementation is reported as open source (Chowdhury et al., 2024).

4. Language support and deployment as CLARA-S

A later educational deployment reworked CLARA as CLARA-S, a service-oriented version intended for classroom use with Python notebooks. The original system is described there as supporting only a limited subset of Python and running in a Linux terminal. The authors extended the parser to support a larger subset of the language, specifically adding import statements, lambda functions, and some object-oriented programming features (Kurniawan et al., 2023).

The same work embedded CLARA into the programming environment rather than leaving it as an external command-line tool. It exposed CLARA through FastAPI as a RESTful web service and implemented a Jupyter Notebook extension so that students could request feedback directly from notebook exercises. The operational workflow was simple: students worked in an online notebook containing problem statements, starter code, and test cases; when they wanted help, they selected the relevant function name and clicked a help button; the extension sent the code to the CLARA-S service; and the resulting feedback was inserted into a new notebook cell (Kurniawan et al., 2023).

This deployment also makes explicit an operational dependency already implicit in CLARA’s original design: the system requires a repository of correct solutions for each task. That repository can be populated from prior student submissions or instructor-authored solutions. In the reported study, the repositories produced three clusters for one problem and four for another, again reinforcing the view that CLARA models tasks through a small set of canonical solution approaches rather than through hand-authored bug rules (Kurniawan et al., 2023).

5. Empirical results and comparative evaluations

The flexible-alignment extension reports its main benchmark on the twenty most popular programming problems in Codeforces. On that benchmark, flexible alignment achieved a 46% successful-repair rate, compared with 5% for baseline CLARA (Chowdhury et al., 2024). Within the scope of that study, the result is a direct empirical argument that relaxing strict control-flow matching materially changes repair coverage.

A different comparative study, focused on C programs, positioned CLARA as a state-of-the-art baseline for introductory programming repair and reported that its coverage across five C benchmarks ranged from 57.1% to 92.0% (Lu et al., 2021). The same paper emphasized the computational cost of CLARA’s semantics-driven analysis: it reported a total of 62 hours of preprocessing to cluster correct C solutions across the five benchmarks and an average of 31.2 seconds to suggest each repair. It also noted implementation constraints in that setting, including a self-defined C parser and interpreter that could not process all syntactic structures or library function calls, and the absence of C++ support in the evaluated implementation (Lu et al., 2021).

These two evaluations illuminate complementary aspects of CLARA. The first shows that structural flexibility can substantially expand the range of incorrect programs that are repairable. The second shows that semantic clustering and dynamic execution analysis provide expressive matching power, but at a nontrivial runtime and implementation cost. Together they explain why subsequent work has alternated between extending CLARA’s semantic reach and criticizing its scalability or source-level usability.

6. Pedagogical interpretation, usability, and criticism

The most detailed study of CLARA’s educational use is a preliminary think-aloud deployment with 20 students from two Python-based courses. That study found that novices often struggled to understand the proposed repairs, in ways resembling the long-standing difficulty novices have with compiler and interpreter messages. Reported causes included internal variable naming such as iter#0 and ind#0, repair suggestions that followed a different solution structure from the student’s own program, and a more general mismatch between cluster-derived feedback and student intent (Kurniawan et al., 2023).

At the same time, the study did not conclude that CLARA was pedagogically useless. Students often treated it as an additional support channel when they had exhausted other options, and the strongest positive signal concerned localization rather than full correction: students valued being shown where a fix was needed, even when the full suggested repair was confusing. The authors therefore argued that, pedagogically, “less may be more,” and proposed an incremental hinting design in which the first hint identifies the location of the logical error, later hints gradually reveal more information, and the full repair is provided only as a last step (Kurniawan et al., 2023).

The same study also observed that perceived usefulness depended on problem difficulty and student background. Intermediate students tended to evaluate CLARA-S more positively than novices, and the tool was judged most useful on the harder of the two study tasks. This suggests that CLARA’s educational value is not identical to its repair accuracy: a technically correct repair may still be educationally ineffective if its representation is too far from the student’s source-level understanding.

In this respect, CLARA occupies an analytically interesting position in the literature. It is simultaneously a successful data-driven repair framework, a baseline against which newer systems are measured, and a case study in the distinction between generating correct repairs and delivering feedback that novices can actually use.