Representability vs. Overfitting for ARC Tasks in the Presented NCA Architecture

Determine whether, for ARC tasks where the Neural Cellular Automata model fits the training examples but fails to generalize, the correct general transformation rule is representable within the specific Neural Cellular Automata architecture used in this work (10 one‑hot color channels with additional hidden channels and a learned local convolutional update applied with stochastic asynchrony), or whether the apparent success on training examples reflects overfitting caused by limited supervision.

Background

The paper applies Neural Cellular Automata (NCA) to ARC-AGI tasks, training local update rules via gradient descent to transform input grids to outputs over multiple steps. The architecture uses one-hot color channels plus hidden channels, learned convolutional updates, layer normalization, and stochastic asynchronous updates, with loss applied at each step.

In the failure analysis, the authors note that many tasks are solved on training examples but fail to generalize to test inputs. Due to the extremely limited number of training examples per ARC task, it is ambiguous whether such failures indicate that the correct general rule cannot be represented by the given NCA architecture or are merely the result of overfitting driven by insufficient supervision. Disentangling these two possibilities is explicitly identified as unclear in the text.