A Neurodiversity-Inspired Solver for the Abstraction \& Reasoning Corpus (ARC) Using Visual Imagery and Program Synthesis (2302.09425v3)

Abstract: Core knowledge about physical objects -- e.g., their permanency, spatial transformations, and interactions -- is one of the most fundamental building blocks of biological intelligence across humans and non-human animals. While AI techniques in certain domains (e.g. vision, NLP) have advanced dramatically in recent years, no current AI systems can yet match human abilities in flexibly applying core knowledge to solve novel tasks. We propose a new AI approach to core knowledge that combines 1) visual representations of core knowledge inspired by human mental imagery abilities, especially as observed in studies of neurodivergent individuals; with 2) tree-search-based program synthesis for flexibly combining core knowledge to form new reasoning strategies on the fly. We demonstrate our system's performance on the very difficult Abstraction & Reasoning Corpus (ARC) challenge, and we share experimental results from publicly available ARC items as well as from our 4th-place finish on the private test set during the 2022 global ARCathon challenge.

• The paper introduces a novel VIMRL that combines visual imagery and program synthesis to solve abstract reasoning tasks on the ARC benchmark.
• It leverages neurodivergence-inspired cognitive strategies to mimic human-like problem solving, achieving notable performance in ARCathon challenges.
• The study highlights future opportunities to refine image-based operations and program synthesis for more flexible and general AI reasoning.

Advanced Visual Imagery and Program Synthesis for Reasoning in AI

Introduction to Core Knowledge in AI

The paper introduces a novel approach for visually representing and reasoning about core knowledge in artificial intelligence systems. Core knowledge, encompassing basic understanding of physical objects and their interactions, is foundational in biological intelligence and has posed challenges for AI systems in terms of flexibility and applicability to novel tasks. This research proposes the Visual Imagery Reasoning Language (VIMRL), inspired by cognitive studies on mental imagery, particularly in neurodivergent individuals. Coupled with tree-search-based program synthesis, the method aims at enhancing AI's capability to solve abstract reasoning tasks, demonstrated through its application to the Abstraction Content Reasoning Corpus (ARC) challenge.

The Abstraction Content Reasoning Corpus (ARC)

ARC emerges as a challenging benchmark that tests general intelligence through core knowledge priors. Tasks within ARC necessitate understanding object segmentation, permanence, interaction, and abstract processes by presenting puzzles that artificial systems must solve from scratch. The ARC's design, lacking direct transferability between training and evaluation sets, underlines its difficulty and utility in evaluating AI systems' abstraction and reasoning capabilities.

Core Knowledge, Visual Imagery, and Autism

The paper draws a significant parallel between visual-imagery-based reasoning observed in neurodivergent individuals and the proposed AI system's approach. Studies indicating atypical yet effective problem-solving strategies in individuals on the autism spectrum inspire the utilization of visual mental imagery in computational models for core knowledge tasks. VIMRL capitalizes on this inspiration, representing core knowledge as image transformation operations directly applicable to perceptual inputs.

The Visual Imagery Reasoning Language (VIMRL) and Program Synthesis

The essence of the proposed system combines the imperative style VIMRL, designed around imagery operations for ARC tasks, with a program synthesis solver that generates VIMRL programs to tackle specific puzzles. VIMRL's architecture supports a range of operation types, from low-level functions addressing basic imaging transformations to high-level operations conducting analyses and inferences based on the task at hand. The reasoning process employs iterative program synthesis to find solutions that adhere to specified accuracy thresholds against the ARC challenge training problems.

Experimental Results and Implications

Performance on the ARC challenge underscored the potential of the proposed approach, with the system finishing in 4th place on the private test set of the 2022 global ARCathon challenge. This achievement highlights the method's utility in generating abstract reasoning capabilities in AI, addressing tasks that require a deep understanding of core knowledge priors. Such results lay foundational work for future developments in general AI reasoning, hinting at the scalability and flexibility of using visual imagery and high-level cognitive processes as a basis for problem-solving in artificial systems.

Future Directions

The paper speculates on expanding the corpus of ground truth programs and refining operations within VIMRL to balance specificity and generality. Enhancing the existing dataset and streamlining the reasoning process represent crucial steps toward a more robust system capable of tackling varied and complex abstract reasoning tasks. Furthermore, ongoing research aims to improve the correlation between task visuals and function selection, potentially narrowing the search space for solutions and optimizing the problem-solving approach.

Conclusion

In conclusion, this paper presents a significant advancement in the application of visual imagery and program synthesis in AI, targeting the core knowledge domain. By drawing inspiration from human cognitive processes, particularly those observed in neurodivergent individuals, the research offers a fresh perspective on imbuing artificial systems with flexible and generalizable reasoning abilities. The promising results from the ARC challenge illuminate the path forward, suggesting that deeper explorations into visual mental imagery and tailored program synthesis could bridge the gap between AI systems and human-like reasoning capabilities.