InsightX Agent: LMM Framework for X-ray NDT
- InsightX Agent is an LMM-based agentic framework designed for X-ray NDT, emphasizing interactivity, interpretability, and reliability.
- It orchestrates specialized tools—SDMSD for defect localization and EGR for evidence-grounded reflection—to validate and explain defect hypotheses.
- Empirical evaluations on industrial datasets show that the framework enhances precision by systematically reducing false positives and supporting operator decision-making.
Searching arXiv for the primary paper and closely related agentic systems to ground the article. InsightX Agent is an LMM-based agentic framework for reliable, interpretable, and interactive X-ray non-destructive testing (NDT) analysis. It was proposed to address a specific limitation of existing deep-learning-based X-ray inspection systems: they often lack interactivity, interpretability, and the capacity for critical self-assessment, which constrains reliability and operator trust. Rather than using a typical sequential pipeline, the framework places a Large Multimodal Model (LMM) at the center of the system and has it coordinate two specialized tools—the Sparse Deformable Multi-Scale Detector (SDMSD) and the Evidence-Grounded Reflection (EGR) tool—to analyze radiographic images, validate defect hypotheses, and produce explanations grounded in observable evidence (Liu et al., 20 Jul 2025).
1. Industrial setting and design rationale
Non-destructive testing, particularly X-ray inspection, is described as vital for industrial quality assurance, and InsightX Agent is explicitly framed as a response to the reliability constraints of conventional automated inspection pipelines (Liu et al., 20 Jul 2025). The motivating critique is not only that prior systems are imperfect at detection, but that they remain insufficiently interactive and insufficiently self-critical. In industrial inspection, these properties matter because an operator must often review ambiguous findings, differentiate true defects from artifacts, and understand why a system reached a conclusion.
The framework is therefore designed around three target properties: reliability, interpretability, and interactivity. Reliability is pursued by combining a specialized perception module with an explicit review stage. Interpretability is pursued by requiring diagnoses to be grounded in referenced evidence and by leaving an explicit reasoning trace. Interactivity is pursued by allowing the LMM to recognize user intent, decide when to invoke tools, and engage in dialogue about findings, severity, and recommendations (Liu et al., 20 Jul 2025).
A common misconception is to treat InsightX Agent as a detector with a language wrapper. The system is described differently: SDMSD outputs candidate defect regions as hypotheses rather than final decisions, and those hypotheses are then subjected to evidence-grounded review before confirmation or rejection. This design suggests that the framework should be understood as an agentic inspection system rather than a simple perception stack.
2. Core architecture and orchestration logic
The architectural distinction of InsightX Agent is the placement of the LMM as a central orchestrator rather than as a passive post-processor (Liu et al., 20 Jul 2025). The LMM performs intent recognition, decides when and how to invoke SDMSD and EGR, synthesizes their outputs in light of domain knowledge and context, and produces explanations at multiple levels of detail. To ensure industrial X-ray and NDT specificity, the LMM is adapted using Low-Rank Adaptation (LoRA), with the update written as
The paper describes three principal components.
| Component | Primary function | Output role |
|---|---|---|
| LMM | Orchestration, intent recognition, synthesis, dialogue | Structured interpretation and reporting |
| SDMSD | Defect localization in radiographic imagery | Candidate defect regions with confidence scores |
| EGR | Systematic review and refinement of detections | Confirmed, uncertain, and rejected findings |
The LMM is also trained with template-based alignment and regularization. The stated purpose of this alignment is to ensure outputs that ground diagnoses in observable, referenced evidence, provide quantitative and precise descriptions, and offer actionable recommendations (Liu et al., 20 Jul 2025). This makes the LMM not merely a natural-language interface, but the mechanism that enforces report structure and ties the perception and reflection stages into a coherent inspection workflow.
This orchestration pattern places InsightX Agent within a broader class of agentic visual systems, but with a notably stronger emphasis on tool-mediated validation. For comparison, "InsightSee" organizes a description agent, two reasoning agents, and a decision agent for complex visual understanding, whereas InsightX Agent couples an LMM to an industrial detector and a specialized reflection tool for X-ray NDT (Zhang et al., 2024).
3. Sparse Deformable Multi-Scale Detector
SDMSD is the framework’s perception module, specialized for localizing defects in radiographic X-ray imagery, especially under conditions involving small, dense, or subtle anomalies (Liu et al., 20 Jul 2025). Its design combines dense proposal generation, Non-Maximum Suppression (NMS) sparsification, and multi-scale deformable attention.
The detector begins by generating an exhaustive set of region proposals from multi-scale CNN feature maps, prioritizing recall. Those proposals are then sparsified through NMS so that highly redundant boxes are removed while retaining salient candidates. The remaining candidates are processed by a transformer encoder-decoder with multi-scale deformable attention, which concentrates computation on spatially relevant locations across multiple feature resolutions. In the paper’s formulation, this design is intended to optimize detection of small, dense targets while maintaining computational efficiency (Liu et al., 20 Jul 2025).
The training objective combines classification, bounding-box regression, and generalized IoU terms. Although the detailed loss expression is part of the method specification, the more important systems point is that SDMSD is not treated as a complete end-point solution. Its coordinates and confidence scores are explicitly interpreted as initial hypotheses. That framing is consequential: it enables the downstream reflective stage to revise, reject, or recalibrate those proposals rather than simply verbalizing detector output.
The paper’s ablation results reinforce this interpretation. SDMSD alone already achieves a strong F1-score, but the full framework is still better, indicating that the added value of InsightX Agent lies not only in detection quality but in reflective filtering and quality assurance (Liu et al., 20 Jul 2025).
4. Evidence-Grounded Reflection and defect validation
EGR is the framework’s systematic validation module. Given SDMSD detections, it conducts a chain-of-thought-inspired review process that incorporates context assessment, individual defect analysis, false positive elimination, confidence recalibration, and quality assurance (Liu et al., 20 Jul 2025). In the paper’s description, the EGR workflow has six stages.
First, context assessment evaluates image quality, environment, and pre-existing patterns. Second, individual detection analysis performs visual verification and bounding-box assessment for each proposal. Third, false positive elimination rejects candidates according to logical criteria and applies a conservative decision rule that only confirms findings with strong evidence. Fourth, confidence recalibration adjusts scores based on evidence, consistency, and plausibility. Fifth, quality assurance checks overall consistency, completeness, and reasonableness. Sixth, reporting produces structured outputs partitioned into confirmed, uncertain, and rejected findings (Liu et al., 20 Jul 2025).
Two features of EGR are especially important. One is that thresholds and parameters are adaptively set by the LMM based on context and domain knowledge rather than hard-coded globally. The other is that each step leaves an explicit reasoning trace. Together, these choices convert the review phase into an auditable process rather than a hidden heuristic layer.
The paper presents EGR as the principal mechanism for suppressing false positives. That is a particularly relevant objective in industrial NDT, where unnecessary alarms can degrade trust and burden human inspectors, while missed defects carry obvious safety and quality risks. The framework’s stance is therefore conservative: it does not simply maximize raw detector acceptance, but tries to preserve high recall while improving precision through evidence-backed rejection (Liu et al., 20 Jul 2025).
5. End-to-end workflow and interactive use
The operational workflow begins with X-ray images and may also include user queries (Liu et al., 20 Jul 2025). The LMM first performs intent recognition, determining whether the task is defect localization, assessment, summarization, or clarification. If perceptual analysis is needed, it invokes SDMSD to generate candidate defect regions. It then orchestrates EGR to review those candidates, refine confidence, and decide which detections should be confirmed, rejected, or left uncertain.
The reporting stage produces structured outputs containing supporting evidence, defect locations, severity, recommendations, and explanations for every decision. This is one of the framework’s central departures from ordinary deep-learning inspection systems: the output is meant to be a usable inspection narrative rather than only a set of boxes and scores (Liu et al., 20 Jul 2025).
The paper also emphasizes optional interactive dialogue. Operators can query the system for clarification, request deeper analysis, or ask for contextual explanations of ambiguous findings. The LMM then uses its domain-adapted knowledge and, if needed, the EGR tool to provide further validated answers. This suggests a shift from static inference toward interactive inspection assistance.
A plausible implication is that the system is designed not only for autonomous detection performance but also for operator-facing workflow integration. The framework’s emphasis on user intent recognition, explanation granularity, and explicit justification aligns with settings in which a human inspector remains part of the decision loop.
6. Empirical performance, ablation, and significance
Experimental evaluation is reported on the GDXray+ casting inspection images, described as comprising 2,700 images, with 714 annotated after preprocessing (Liu et al., 20 Jul 2025). On this benchmark, InsightX Agent achieves an object detection F1-score of 96.35%. The paper also reports an ablation in which an LMM-only system reaches 2.06% F1-score, SDMSD only reaches 95.57%, and the full InsightX Agent reaches 96.35%. This isolates the incremental value of the agentic reflection layer over a high-performing detector baseline.
The reported mechanism of improvement is precise: EGR increases precision by 2.17% with modest recall reduction, and the paper highlights systematic false positive suppression together with detailed justifications for each rejection (Liu et al., 20 Jul 2025). This is consistent with the framework’s stated reliability objective. The performance comparison provided in the paper also places the system above Faster R-CNN at 89.8%, YOLOX-s at 95.8%, DINO at 94.0%, Deformable DETR at 86.7%, and PVTv2+RetinaNet at 93.1%.
The empirical discussion is not restricted to scalar metrics. Case studies are said to show that classic methods output raw numbers, direct LMMs hallucinate, and InsightX Agent instead produces narrative, stepwise, and justified outputs. The paper also states that the framework retains high recall for small, medium, and large objects, which is consistent with the role of the multi-scale detector (Liu et al., 20 Jul 2025).
Within the broader agent literature, InsightX Agent exemplifies a pattern in which LMMs serve as orchestrators over specialized tools rather than replacing them. Comparable design tendencies appear in agentic observability and safety systems that reconstruct causal chains or bridge intent and action, such as AgentSight and ADR, though those works operate in software-agent settings rather than industrial radiography (Zheng et al., 2 Aug 2025, Li et al., 17 May 2026). In the case of InsightX Agent, the same general principle is applied to X-ray NDT: tool outputs are treated as evidence to be reviewed, synthesized, and explained, rather than as final answers.
The resulting framework is therefore best understood as a hybrid of specialized perception and explicit reflective validation. Its significance lies not only in achieving a reported F1-score of 96.35%, but in making inspection outputs reviewable, interactive, and grounded in an evidence-centered reasoning process suited to industrial quality assurance (Liu et al., 20 Jul 2025).
