Neatest with Branch Coverage (NBC) Overview

• NBC is a branch-coverage-guided extension of NEATEST that uses neuroevolutionary test agents and branch distance metrics to systematically explore Scratch game logic.
• The method replaces statement-oriented guidance with branch-oriented metrics, achieving higher branch coverage (86.31% vs 84.30%) and fewer false positives.
• Its dynamic evolution loop and robust mutation analysis ensure repeated branch execution under randomized conditions for reliable fault detection.

Searching arXiv for the directly relevant NBC/NEATEST papers and one branch-coverage context paper. Neatest with Branch Coverage, denoted $N_{BC}$ and often abbreviated NBC, is a branch-coverage-guided extension of NEATEST for automated testing of Scratch games. It preserves NEATEST’s use of neuroevolutionary test agents implemented as neural networks in the Whisker framework, but replaces statement-oriented guidance with branch-oriented guidance based on control dependence, approach level, and branch distance. In the reported evaluation on 25 Scratch games, $N_{BC}$ achieved mean branch coverage of 86.31% versus 84.30% for Neatest with Statement Coverage ($N_{SC}$), and its mutation-analysis pipeline reported a lower mean false positive count, 20.65 versus 22.40 (Sohail et al., 12 Jul 2025). The term also belongs to a broader branch-oriented evolution of NEATEST: an earlier paper had already reformulated NEATEST for branch coverage through many-objective optimization, although it did not explicitly name that reformulation “NBC” (Feldmeier et al., 14 Jan 2025).

1. Nomenclature and research lineage

Within Scratch game testing, NBC refers specifically to Neatest with Branch Coverage, the variant introduced as $N_{BC}$ in direct comparison with $N_{SC}$, or Neatest with Statement Coverage (Sohail et al., 12 Jul 2025). This naming matters because the acronym “NBC” is also used elsewhere in the literature for Neuron Boundary Coverage in deep neural network testing, where it denotes a metric over neuron activations rather than a game-testing method (Usman et al., 2022). In the present context, NBC is neither a neural-network coverage metric nor a generic branch-coverage tool; it is a NEATEST-based game-testing method specialized to Scratch.

NBC emerged from the limitations of the original NEATEST workflow. NEATEST was designed as a neuroevolution-based test generator for games, using dynamic neural-network controllers rather than static input sequences. In its original form it was single-objective at any moment: it selected one coverage goal, evolved networks toward that goal, stopped when the goal was robustly covered or the budget was exhausted, and then moved to the next goal. That design could get stuck on hard or unreachable goals, miss useful intermediate individuals that were promising for other objectives, and require sequential decisions about goal order and budget allocation (Feldmeier et al., 14 Jan 2025). The 2025 NBC paper retained the NEATEST architecture but changed the coverage target from statements to branches, whereas the 2025 many-objective paper changed the optimization formulation itself by introducing MIO-Neatest, MOSA-Neatest, and NEWS/D-Neatest as branch-coverage-oriented variants (Feldmeier et al., 14 Jan 2025).

2. Representation, execution model, and Scratch-specific testing context

NBC inherits NEATEST’s core representation: a test case is a neural network encoded as genes for neurons and connections, and evolved in both weights and topology. These controllers act as dynamic test agents during bounded gameplay episodes rather than as fixed input scripts (Feldmeier et al., 14 Jan 2025). That representation is central for games because game testing is complicated by non-determinism, event-driven execution, ongoing loops, and the need for meaningful gameplay to reach deep program states. A dynamic controller can react to the current game state and therefore better handle randomized execution than a fixed input sequence (Feldmeier et al., 14 Jan 2025).

The target domain is Scratch game-like programs executed through Whisker. The 2025 NBC evaluation used a filtered benchmark of 25 Scratch games selected from an original collection of 187 Scratch programs. Selection required that a program accept user inputs via Whisker, exhibit dynamic behavior, and not be trivially coverable by arbitrary input. The benchmark included titles such as FlappyParrot, Frogger, EndlessRunner, WhackAMole, SpaceOdyssey, and DieZauberlehrlinge, with mean values of 5.5 sprites, 16.6 scripts, and 118.3 statements (Sohail et al., 12 Jul 2025).

This setting makes branch coverage materially different from merely executing code broadly. The NBC paper’s motivation is that statement coverage can execute all statements at least once while still missing alternative outcomes of control decisions, a weakness that is especially consequential in game logic built from collision checks, score thresholds, conditional updates, and other control-dependent mechanics (Sohail et al., 12 Jul 2025).

3. Branch targets, control dependence, and the NBC fitness function

NBC changes NEATEST’s objective from executing uncovered statements to executing uncovered branches. Target selection remains control-dependence-driven: uncovered branches are selected through the Control Dependence Graph (CDG), and fitness is computed from the structural distance to the target branch and the closeness of the relevant predicate evaluation (Sohail et al., 12 Jul 2025).

The paper defines the normalization function

$\eta(x) = \frac{x}{1 + x}$

and the branch-oriented fitness function

$f = \eta \Big(\alpha + \eta(\beta) \Big),$

where $\alpha$ is the approach level and $\beta$ is the branch distance (Sohail et al., 12 Jul 2025). The intended interpretation is explicit: $\alpha = 0$ means the target control statement has been reached, $N_{BC}$0 means the target branch itself has been executed, and $N_{BC}$1 means the target branch has been successfully executed. NBC is therefore a minimization-based SBST objective in which lower values indicate closer progress toward the target branch (Sohail et al., 12 Jul 2025).

Branch distance is attached either to the target branch’s predicate or, if the target predicate was not reached, to the nearest unsatisfied control dependency. The paper illustrates both cases. For the false branch of If (Y = 0), when the target control statement is reached with $N_{BC}$2, the branch distance is computed as

$N_{BC}$3

When the execution instead fails an earlier dependency such as If (X > 0) under $N_{BC}$4, the approach level becomes $N_{BC}$5 and the branch distance is computed on that nearer unsatisfied dependency: $N_{BC}$6 The paper then works these values through the fitness function, although the numeric examples are somewhat inconsistent with the qualitative statement that lower is better (Sohail et al., 12 Jul 2025).

NBC also introduces ControlFilter, a Scratch-specific analysis component that scans Scratch blocks, especially opcodes beginning with control_, and classifies structures into branching blocks, no-false-branch blocks, and execution-halting blocks. Reported examples of branching blocks include control_if, control_if_else, control_repeat, control_repeat_until, control_forever, and control_wait_until; execution-halting and timing-related examples include control_stop, control_wait, wait, say, and play sound (Sohail et al., 12 Jul 2025). The practical role of ControlFilter is to identify branch-relevant control structures, maintain the set of uncovered branches, and guide branch selection using execution history and structural dependencies.

4. Search procedure and robustness validation

NBC preserves the NEAT-style neuroevolutionary loop while changing the target representation and fitness guidance. The paper describes the workflow as follows: initialize a flag requireNextBranch; when a new target is needed, select the next uncovered branch with selectTargetBranch(); generate a population of neural networks; execute each network on the game; compute branch-based fitness; and, if a network covers the target branch, perform a robustness check. If the candidate is sufficiently robust, add it to the dynamic test suite, mark the branch as covered, and move to the next uncovered branch; otherwise continue evolving the population through the NEAT process (Sohail et al., 12 Jul 2025).

The emphasis on robustness is inherited from NEATEST’s game-testing setting. A branch is not treated as reliably covered merely because it is reached once; the successful network must survive repeated executions under randomized conditions before it is admitted to the final test suite (Sohail et al., 12 Jul 2025). Earlier branch-oriented NEATEST work made this robustness criterion explicit by requiring a branch to be reached in ten randomized program executions before it counted as covered (Feldmeier et al., 14 Jan 2025). This suggests that NBC should be understood as a method for robust branch execution in randomized games, not just one-off branch hits.

The published NBC evaluation used a population size of 150, species size 5, a non-improving generation limit of 5, a search duration of 5 hours, and a maximum playthrough duration of 5 seconds, reported as equivalent to 100 seconds because Whisker’s acceleration factor was set to $N_{BC}$7. Each game-method pair was run for 30 independent repetitions on a 9-node cluster with AMD EPYC 7443P CPUs at 2.8 GHz (Sohail et al., 12 Jul 2025).

5. Empirical results on Scratch games

Across the 25-game benchmark, $N_{BC}$8 achieved mean branch coverage of 86.31%, compared with 84.30% for $N_{BC}$9. The paper’s headline statement is that NBC achieved higher branch coverage than NSC in 13 out of 25 games, particularly in programs with complex conditional structures (Sohail et al., 12 Jul 2025). Representative gains included LineUp from 90.47 to 100.00, BrainGame from 98.12 to 100.00, EndlessRunner from 59.26 to 66.66, CatchTheGifts from 96.94 to 100.00, SnowballFight from 75.94 to 83.04, and DieZauberlehrlinge from 36.77 to 45.16 (Sohail et al., 12 Jul 2025).

The same study reported a small increase in statement coverage as a side effect of branch-oriented search: mean statement coverage rose from 90.67% under $N_{SC}$0 to 91.11% under $N_{SC}$1 (Sohail et al., 12 Jul 2025). The paper attributes the largest NBC benefits to games with complex conditional logic, deep control dependence, and branch-gated gameplay behavior, and it notes that in simpler games the extra branch-targeting overhead may offer less benefit. Significant NSC wins were reported for Frogger and Dodgeball, while several games showed equal coverage or non-significant differences (Sohail et al., 12 Jul 2025).

The mutation analysis used 205,545 mutants across eight operators: Key Replacement Mutation, Single Block Deletion, Script Deletion Mutation, Arithmetic Operator Replacement, Logical Operator Replacement, Relational Operator Replacement, Negate Conditional Mutation, and Variable Replacement Mutation. To control the experimental budget, each repetition used 50 randomly selected mutants per mutation operator. Each extracted test suite was then evaluated across 10 different random seeds, yielding 100 executions per test suite per game (Sohail et al., 12 Jul 2025). A mutant was considered killed if neural activation values exceeded a threshold of 30, based on Likelihood-based Surprise Adequacy with Kernel Density Estimation, or if the network structure changed significantly during execution (Sohail et al., 12 Jul 2025).

On this protocol, mean mutation score was 72.57% for $N_{SC}$2 and 72.15% for $N_{SC}$3, while mean false positives were 22.40 for $N_{SC}$4 and 20.65 for $N_{SC}$5 (Sohail et al., 12 Jul 2025). The paper therefore frames NBC’s mutation result less as a pure kill-score improvement than as a precision result: NBC produced a lower false positive rate and is described as a more reliable approach for identifying faulty behavior in game programs. Reported operator-level observations were that NBC was better on decision-related operators, especially Negate Conditional Mutation and Relational Operator Replacement, and also stronger on Single Block Deletion and Script Deletion Mutation, while both methods reached 100% on Arithmetic Operator Replacement (Sohail et al., 12 Jul 2025).

6. Relation to many-objective branch coverage, adequacy, and future directions

NBC belongs to a larger attempt to make NEATEST more branch-aware. The earlier many-objective study did not name a method “NBC,” but it directly reformulated branch coverage as a many-objective optimization problem over NEATEST-style dynamic tests and introduced MIO-Neatest, MOSA-Neatest, and NEWS/D-Neatest. On 20 Scratch programs, that reformulation increased average branch coverage from 75.88% to 81.33% while reducing required search time by 93.28% (Feldmeier et al., 14 Jan 2025). Relative to that work, the 2025 NBC paper is best understood as a branch-fitness extension of the sequential NEATEST pipeline rather than a many-objective redesign.

At the same time, NBC inherits the general limitations of branch coverage as a testing adequacy criterion. Industrial evidence on Java systems shows that branch coverage is materially weaker than mutation coverage because branch coverage checks reachability, whereas mutation coverage additionally reflects infection, propagation, and revealability; the same study reported concrete cases of high branch coverage with much lower mutation coverage, including an example with 88% branch coverage and 41% mutation coverage (Parsai et al., 2021). In that sense, NBC’s reported gains are best read as gains in structural exploration of Scratch game logic, together with lower false positives under the paper’s mutation-analysis protocol, rather than as a standalone guarantee of fault-detection adequacy.

A further branch-coverage issue concerns loops. Standard branch coverage usually treats a loop as a conditional executed zero or one time, and Meyer formalizes loop unrolling as a way to create progressively stronger finite approximations $N_{SC}$6 that expose additional traces, branch obligations, and bugs (Meyer, 2024). This suggests a possible extension of branch-based game testing: an NBC-style method could, in principle, define branch targets over unrolled control structures rather than over the default zero-or-one loop abstraction. That implication is not part of the published NBC implementation, but it is consistent with the broader trajectory of branch-coverage research.