Ambiguity in ARC task semantics: “touching” vs “going through”

Determine the intended transformation rule for an ARC task involving blue lines and red rectangles: specifically, ascertain whether a red rectangle should be turned blue when a blue line touches the rectangle without going through it, given demonstration examples that admit both interpretations (“only if the line goes through” versus “if the line merely touches”).

Background

The Abstraction and Reasoning Corpus (ARC) comprises puzzle-like tasks defined by few input–output demonstrations, where the underlying rule must be inferred and applied to new inputs. Despite most tasks being designed to be unambiguous, the paper highlights that some tasks can admit multiple plausible rules based on the limited demonstrations.

In the ambiguous example discussed, blue lines and red rectangles appear in demonstrations such that the correct transformation rule is unclear: whether a red rectangle should be turned blue only when a blue line passes through it, or also when a blue line merely touches it. The authors show their model producing both plausible interpretations, revealing that the dataset’s intended semantics for this specific scenario are uncertain.

Clarifying this ambiguity is important both for consistent evaluation and for guiding model behavior, as different but reasonable interpretations can yield distinct outputs under identical demonstrations. Establishing the intended rule would disambiguate training and inference for this class of tasks.