AstroVLM: Expert Multi-agent Collaborative Reasoning for Astronomical Imaging Quality Diagnosis

Abstract: Vision LLMs (VLMs) have been applied to several specific domains and have shown strong problem-solving capabilities. However, astronomical imaging, a quite complex problem involving multidisciplinary knowledge and several subtasks, has not been adequately studied. Due to the complexity of the astronomical imaging process, both world-class astronomical organizations, such as NASA, and expert enthusiasts devote a great deal of time and effort. This is because the processes in astronomical imaging have complex underlying correlations that significantly influence one another, making the quality diagnosis and error localization of astronomical images challenging. To address this problem, we propose AstroVLM, a collaborative multi-agent system for diagnosing the quality of astronomical images. Experiment results show that AstroVLM outperforms all baselines on real-world astronomical imaging quality diagnosis tasks, providing a reference for LLMs to handle complicated multi-process tasks.

• The paper introduces AstroVLM, a multi-agent system integrating agent-specific RAG and tree-based backtracking for diagnosing astronomical imaging errors.
• It leverages structured knowledge graph partitioning and correlation factor calculation to minimize noise and enhance diagnostic precision.
• Quantitative evaluation shows significant improvements in rationality, accuracy, and diversity compared to existing imaging diagnostic methods.

AstroVLM: Multi-agent Collaborative Reasoning for Astronomical Imaging Diagnosis

Problem Setting: Complexity and Interdependencies in Astronomical Imaging

Astronomical imaging quality diagnosis is characterized by high complexity due to its multi-stage pipeline—encompassing preparation, shooting, and post-processing—where errors may propagate or manifest across stages. Each stage consists of numerous sub-processes, with latent cross-stage and intra-stage dependencies, complicating the identification and attribution of image degradation phenomena. Manual diagnosis is time-intensive and demands multidisciplinary expertise in astronomy, optics, instrumentation, and computational imaging, making automated, expert-level diagnosis highly desirable.

Figure 1: The astronomical imaging pipeline involves preparation, shooting, and post-processing, each comprised of critical, interdependent sub-processes whose interactions confound failure diagnosis.

System Architecture: AstroVLM and its Collaborative Framework

To address the intrinsic complexity of astronomical imaging quality assessment, the paper introduces AstroVLM—a multi-agent, VLM-based system leveraging expert reasoning and structured knowledge augmentation tailored to this domain.

The core architecture, AstroSight, instantiates twelve specialized agents mapped to key imaging processes, coordinated by a central agent. Problem decomposition and responsibility assignment are explicitly modeled, and the agents are integrated within a procedural pipeline, invoking external analytical tools where appropriate. The central coordinator orchestrates the collaborative diagnosis process, implements backtracking protocols, and integrates sub-agent outputs to produce comprehensive quality assessments.

Figure 2: Macro-level depiction of the AstroVLM architecture, where collaborative agents, each supplied with tailored knowledge, are orchestrated to diagnose imaging quality through iterative, tool-integrated reasoning.

Agent-Specific Knowledge RAG: Decomposition and Augmentation

Direct application of vanilla RAG to this domain induces excessive irrelevant retrieval, with high hallucination risk due to discipline-spanning knowledge and ambiguous context boundaries. To address this, AstroVLM employs Agent-Specific Knowledge RAG (ASK-RAG), which partitions a unified knowledge graph into highly targeted, role-specific subgraphs for each agent.

The ASK-RAG pipeline comprises three stages:

1. Relevant Wordlist Construction: Automated extraction and synthesis of hierarchically structured keyword lists for each process using KeyBert and LLM-driven synthesis ensures coverage and ordering from general to specific.

   Figure 3: Generation of structured, agent-specific keyword lists ensures decomposed, context-aware knowledge retrieval for each diagnosis component.

2. Knowledge Graph Partitioning and Aggregation: A flow-based path retrieval algorithm dynamically partitions the master knowledge graph, incorporating semantic similarity weighting and graph structure. Cosine similarity between GCN-derived node embeddings drives aggregation, and LLM-driven edge inference resolves knowledge gaps across otherwise disconnected subgraphs.

   Figure 4: The knowledge graph is partitioned and aggregated according to agent keywords, ensuring subgraphs are both specific and sufficiently interconnected for cross-process reasoning.

3. Correlation Factor Calculation: A parametrized similarity metric balances the semantic affinity versus hierarchy depth, dynamically triggering partitioning or aggregation operations to optimize subgraph scope per agent.

This design results in precise, minimal-noise reference graphs per agent, addressing the known pitfalls of unfiltered RAG approaches in multi-domain, multi-step reasoning.

Reasoning with Backtracking: Tree-based Error Attribution

Systemic error attribution in the presence of process interdependencies is managed by a tailored "Reasoning with Backtracking" (RwB) mechanism, underpinned by Chain-of-Backtracking (CoB). CoB constructs a Collaborative Reasoning Tree (CRT): leaf nodes represent observable errors, internal nodes capture agent-processes that may have contributed, and edge weights quantify confidence that faults in one process caused downstream failures.

The central coordinator guides recursive error tracing, invoking re-examination across upstream agents with response/confidence gating. This enables attribution to either single or multiple causal nodes, with robust conflict resolution based on cross-agent confidence pooling.

Figure 5: AstroVLM constrains the diagnostic search space via structured, agent-driven reasoning, avoiding the combinatorial irrelevance typical of unstructured VLM prompting.

This mechanism stands in contrast to conventional multi-agent debate/reasoning strategies, demonstrating improved tractability and higher-fidelity causal inference in this context.

Quantitative Evaluation and Ablation

Empirical evaluation on diverse real-world datasets (galaxies, nebulas, star clusters) benchmarks AstroVLM against state-of-the-art VLMs, RAG systems, and multi-agent reasoning baselines. Agents are implemented using Qwen2.5-VL (7B), with Qwen3-VL (30B) as coordinator. Metrics include rationality, accuracy, and diversity, evaluated independently and in tandem using LLM-based meta-evaluators (GPT-4o).

AstroVLM yields a significant improvement in diagnostic performance, with average gains over the best baseline (Claude Sonnet 4) of +5.9% rationality, +11.8% accuracy, and +6.3% diversity. ASK-RAG demonstrates 18.4% higher effectiveness over the best alternative graph-based RAG (GraphRAG). RwB surpasses MAD by 37.9% in accuracy and CMD by 41.9% in diversity.

Ablation experiments reveal sharp performance degradation when omitting ASK-RAG (accuracy −16.3%) or RwB (accuracy −24.7%), establishing the necessity and non-redundancy of these components. Hyperparameter studies on knowledge graph partition parameters (decay factor $\mu$, balance factor $\gamma$) empirically confirm the importance of constrained aggregation in optimizing agent-specific information scope.

Figure 6: AstroVLM’s RwB process outperforms alternatives in accuracy and diversity, establishing effective error chain reasoning.

Figure 7: Analysis of hyperparameters $\mu$ and $\gamma$, illustrating impact on rationality, accuracy, and diversity.

Case Studies and Diagnostic Interpretability

Case studies illustrate AstroVLM’s interpretability and explanatory fidelity. Unlike single-stage or naively prompted VLMs, which provide superficial factual commentary, AstroVLM traces error causality deeply through collaborative, multi-agent chains, elucidating latent or multi-factorial faults.

Figure 8: An AstroVLM-assisted case study, showcasing multi-agent reasoning for non-obvious error source identification in real astronomical imaging data.

Implications and Future Prospects

AstroVLM operationalizes the combination of targeted knowledge retrieval and structured, multi-agent causal inference for highly interdependent, domain-expert tasks. The system architecture generalizes to multi-step diagnosis contexts where cross-process or cross-disciplinary reasoning is required. Further research may extend this paradigm to scientific discovery pipelines, complex engineering diagnostics, or real-time process monitoring.

Notably, the interaction between agent-specific subgraph construction and tree-based error reasoning could inform new classes of domain-adapted LLM-based collaborative systems, especially where end-to-end fine-tuning is infeasible or cost-prohibitive.

Conclusion

AstroVLM advances automated astronomical image quality diagnosis through an overview of agent-specific, graph-augmented retrieval and collaborative, backtracking-enabled reasoning. The architecture demonstrates strong empirical gains over previous VLMs, RAG, and multi-agent reasoning baselines, with significant improvements in the interpretability, robustness, and comprehensiveness of error attribution. The framework provides a substantive basis for the extension of large-scale, agent-augmented VLM reasoning in other knowledge-intensive, multi-process scientific domains.

Reference: "AstroVLM: Expert Multi-agent Collaborative Reasoning for Astronomical Imaging Quality Diagnosis" (2604.16024)