MARIC: Multi-Agent Reasoning for Image Classification

• MARIC is a multi-agent framework that divides image classification into specialized reasoning stages, enhancing interpretability and accuracy.
• The system employs an Outliner, multiple Aspect Agents, and a Reasoning Agent to extract and integrate diverse visual evidence.
• Experimental results on datasets such as CIFAR-10 and Skin Cancer demonstrate statistically significant improvements over baseline models.

Multi Agent based Reasoning for Image Classification (MARIC) denotes a paradigm in which the classification process is deliberately decomposed into specialized reasoning stages handled by independent agents. Instead of monolithic inference or single-pass representations, MARIC frameworks employ a sequence of agent operations—each capturing distinct aspects of image content, integrating complementary evidence, and executing a reflective synthesis—thereby increasing accuracy, transparency, and robustness. The approach explicitly addresses traditional limitations in parameter-heavy, fine-tuned models and the context blindness of vision LLMs (VLMs) by reformulating classification as collaborative visual reasoning.

1. Conceptual Architecture and Agent Roles

MARIC formalizes image classification as a multi-agent process involving three agent categories:

• Outliner Agent: Operates globally to analyze the image and generate a suite of structured prompts that succinctly capture the overall theme. These prompts partition the visual content into orthogonal descriptive cues, preventing overlap and guiding subsequent extraction.
• Aspect Agents (n=3 by default): Each agent receives a dedicated prompt and specializes in extracting fine-grained information along a distinct visual dimension—such as color, texture, shape, or background context. Responses take the form $d_i = G_{asp}(I, S_{asp} | p_i)$, producing textual descriptions that are diverse and nonredundant.
• Reasoning Agent: Synthesizes the set of aspect descriptions $D = \{d_1, d_2, ..., d_n\}$ through an integrated reflection step. Instead of standard vector embeddings, classification is approximated as $C_{\theta}(I) \simeq G_{rea}(I, S_{rea} | D)$, yielding a structured output $$\langle reasoning\rangle r \langle /reasoning\rangle \langle answer\rangle \hat{y} \langle /answer\rangle$$ that includes both an explanatory reasoning trace $r$ and the final prediction $\hat{y}$.

2. Structured Prompt Generation and Fine-Grained Evidence Extraction

The Outliner Agent frames the image with prompts of the form [prefix — visual region/attribute] [postfix — descriptive objective]. This template ensures that each subsequent Aspect Agent focuses on one visual aspect and produces a targeted, complementary description. For instance, one agent may extract color attributes while another focuses on texture, ensuring the accumulation of a rich, varied evidence base. By assigning non-overlapping tasks to each agent, the MARIC framework enforces diversity and comprehensiveness in the extracted evidence, directly addressing gaps common in monolithic and single-pass VLM-based approaches.

3. Reasoning Integration via Reflective Synthesis

A foundational innovation in MARIC is the reflective synthesis step performed by the Reasoning Agent. Rather than simply concatenating evidence, the agent revisits each aspect description, critiques potential inconsistencies, and reinforces salient cues. This iterative process leads to a reasoning trace that is explicitly logical and coherent, overcoming the typical opacity of black-box classifiers. The agent’s output is not just a class label but a human-readable explanation detailing the decision path, enhancing both interpretability and robustness.

4. Experimental Results and Quantitative Assessment

The MARIC method was evaluated across four diverse image classification datasets: CIFAR-10, OOD-CV, Weather, and Skin Cancer. Performance results demonstrate statistically significant improvements over baselines:

Model/Method	CIFAR-10	OOD-CV	Weather	Skin Cancer
MARIC (LLaVA 1.5-13B)	93.5%	89.9%	85.2%	56.3%
Chain-of-Thought	83.5%	—	—	—
Direct Generation	lower	lower	lower	lower
SAVR Baseline	lower	lower	lower	lower

The improvements are attributed to the explicit multi-stage reasoning: the Outliner Agent’s prompts lead to more comprehensive aspect descriptions and the Reasoning Agent’s synthesis ensures logical consistency. Qualitative human studies further indicate that the framework’s explanations are superior in relevance and descriptive accuracy. t-SNE visualizations of reasoning traces reveal well-separated clusters for different classes, supporting claims of enhanced semantic disentanglement.

5. Methodological Implications and Interpretability

MARIC strategically decomposes visual evidence extraction and reasoning, contrasting with traditional parameter-heavy models and single-pass VLMs. The multi-agent structure ensures that classifiers benefit from complementary, nonredundant evidence and explicit logical synthesis. This model architecture confers enhanced interpretability—each agent’s reasoning trace is auditable—and improves robustness by critiquing and cross-validating evidence prior to decision. The approach’s inherent modularity allows adaptation to other domains or classification tasks requiring fine-grained and reflective reasoning.

6. Limitations and Comparative Perspective

While MARIC achieves high classification accuracy and interpretable outputs, its dependency on well-structured prompts presents potential sensitivities to prompt engineering quality. The qualitative property of reflection—agent critique and synthesis—may also be influenced by the underlying VLM’s reasoning capacity. Nevertheless, empirical evidence indicates that the explicit decomposition and collaborative integration of agent outputs outperforms direct generation and single-agent reasoning approaches on standard benchmarks.

7. Summary and Future Directions

Multi Agent based Reasoning for Image Classification (MARIC) represents a substantive shift in the deployment of collaborative, explainable reasoning systems for visual tasks. By using an Outliner Agent (global theme extraction and prompt generation), multiple Aspect Agents (fine-grained, orthogonal evidence extraction), and a Reasoning Agent (integrated reflection and synthesis), MARIC delivers interpretable, robust, and accurate classification. Experimental validations across varied datasets confirm both statistical and interpretive benefits, suggesting that future image classification systems may increasingly adopt agent-based modular reasoning architectures to enhance both accuracy and explanatory transparency (Seo et al., 18 Sep 2025).