IndusAgent: Multimodal Industrial Anomaly Detection

• IndusAgent is an 8B-parameter, tool-augmented MLLM agent designed for open-vocabulary industrial anomaly detection across unseen object categories and defect types.
• It employs an active multi-round inspection loop that dynamically invokes specialized tools like Crop, Enhance, Prior, and Measure to enhance diagnostic precision.
• The framework integrates structured supervision with reinforcement learning to achieve state-of-the-art performance on benchmarks such as MVTec-AD and DTD.

Searching arXiv for papers relevant to "IndusAgent" and adjacent industrial-agent uses so the article can be grounded in the current literature. IndusAgent is an 8B-parameter, tool-augmented multimodal LLM (MLLM) agent for zero-shot, open-vocabulary industrial anomaly detection (IAD). It is designed for settings in which anomalies must be detected in previously unseen object categories and unseen defect types from a single query image and an instruction, without paired normal references. The framework addresses domain-misaligned reasoning, hallucinated structural inferences, perceptual dilution, and scale-blindness by combining structured industrial supervision in the form of Indus-CoT with an active, multi-round inspection loop that can crop suspicious regions, enhance high-frequency cues, retrieve normalcy priors, and measure geometry before issuing a final anomaly decision, localization, and type description (Tan et al., 20 May 2026).

1. Problem setting and diagnostic scope

Open-vocabulary IAD in IndusAgent comprises three tasks: image-level anomaly classification, localization, and open-vocabulary anomaly type reasoning. The defining constraint is category and defect novelty: the system must operate on previously unseen object categories and unseen defect types, and it must do so from a single query image and an instruction rather than from paired normal references. This setting differs from reconstruction-based, diffusion-based, feature-embedding, and passive vision-language approaches because those paradigms either assume category-specific normal data, assume closed sets, or remain single-pass and therefore less capable of isolating micro-defects and validating subtle structural evidence (Tan et al., 20 May 2026).

The motivating failure modes are explicitly industrial. General MLLMs are described as struggling with domain-misaligned reasoning because they are pretrained for open-ended conversation rather than industrial diagnostic protocols. They also exhibit hallucinated structural inferences, such as explaining away high-frequency textures or specularities as false defects, or asserting defects without localized evidence. A further difficulty is perceptual dilution: small defects are diluted in global features, while subtle surface variations such as scratches and pinholes are suppressed by global encoders. IndusAgent is therefore framed not as a generic visual question-answering model, but as an inspection policy that conditionally augments its own perceptual context before deciding (Tan et al., 20 May 2026).

2. Architecture and agentic inspection loop

The base policy $\pi_\theta$ is instantiated with Qwen3-VL-8B-Instruct, while teacher synthesis uses Qwen3-VL-Max. Multimodal fusion follows the model’s native visual-token and text-token processing; no extra encoders are introduced beyond Qwen3-VL’s vision-language stack. The paper formulates unified agentic inference as

$$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$

where $I$ is the query image, $Q$ is the instruction, $F$ denotes visual feedbacks such as local crops and enhanced maps, $E$ denotes semantic or quantitative feedbacks such as priors and measurements, and $\mathcal{T}$ is the tool set (Tan et al., 20 May 2026).

Operationally, the inference loop has five stages: observe, route, execute, verify, and output. The agent first reads the global image and task and produces an initial >. If evidence is insufficient, it emits <call_tool> specifying any subset of the available tools. Those tools return observations, after which the agent performs a second reasoning step that fuses the original image with returned feedback. The final output is structured: if the image is anomalous, the agent emits <location>, <type>, and <answer>Yes</answer>; otherwise it emits <answer>No</answer>. Tool selection is learned rather than hard-coded: there is no hard-coded gating at inference, and judicious invocation emerges from the reinforcement objective (Tan et al., 20 May 2026).

The external tools are specialized for distinct industrial ambiguities. T_crop performs dynamic high-resolution cropping of suspicious regions to recover details that global encoders dilute. T_enhance applies deterministic, lightweight high-frequency enhancement such as CLAHE and edge extraction. T_prior retrieves textual normalcy priors that anchor reasoning about what a component should look like. T_measure returns quantitative distances and angles for misalignment, spacing, or deformation checks. Together they convert passive inspection into an active evidence-acquisition loop (Tan et al., 20 May 2026).

3. Indus-CoT and supervision design

IndusAgent is trained with Indus-CoT, a structured dataset of approximately 3,000 tool-integrated reasoning trajectories generated from Real-IAD images and curated with self-correction and LLM-as-a-judge validation. The trajectories are roughly balanced between normal and anomalous cases. The dataset is category-disjoint from all evaluation benchmarks through explicit filtering and semantic normalization, including removal of terms such as “toothbrush,” “zipper,” “pcb,” and “transistor,” so that evaluation preserves the intended open-vocabulary regime (Tan et al., 20 May 2026).

Each trajectory contains three phases. Phase 1 covers global perception and tool routing. Phase 2 covers tool execution and contextual observation, including high-resolution crops, enhanced texture maps, textual priors, and geometric measurements. Phase 3 covers final diagnostic verification with grounding and type reasoning. The representation is explicitly structured through tags such as <think>, <call_tool>, <observation>, <location>, <type>, and <answer>. The fine-tuning stage uses selective masking over generated reasoning tokens so that the model learns the diagnostic trajectory rather than memorizing context (Tan et al., 20 May 2026).

Supervised fine-tuning is followed by GRPO-based reinforcement learning. The reported training regime uses one epoch of SFT on Indus-CoT and one epoch of RL. The default GRPO group size is $G=4$, although sensitivity analysis reports that performance peaks around 6 at higher cost. The best reward-weight setting is $\alpha=0.8$, $\beta=0.6$, $$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$0, while a uniform setting of $$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$1 is slightly worse. Training uses 4× A100 80GB, bfloat16, gradient checkpointing, and DeepSpeed ZeRO-3; SFT takes approximately 21.6 minutes and RL approximately 23.4 hours (Tan et al., 20 May 2026).

4. Accuracy-gated reinforcement learning and hallucination control

The central reinforcement design is an accuracy-gated reward in which localization, anomaly type reasoning, and tool-use credit are multiplicatively conditioned on correct final classification. In prose, the classification gate takes values in $$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$2; if it is 0, the policy receives no reward for localization, type reasoning, or tool usage. Localization reward is based on $$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$3, type reasoning is graded by semantic distance to the ground-truth anomaly taxonomy, and malformed outputs are penalized by a format-compliance term. This structure directly targets a common pathology in MLLM reasoning: plausible but ungrounded explanations (Tan et al., 20 May 2026).

Tool utility is explicitly tied to confidence improvement and invocation cost. The paper gives

$$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$4

with $$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$5 and $$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$6, where $$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$7 is the set of invoked tools. Confidence improvement can be estimated through the margin

$$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$8

so that

$$O \sim \pi_\theta(\cdot \mid I \oplus F, Q \oplus E; \mathcal{T}),$$9

The implication is precise: a tool call is rewarded only if it increases decision confidence and the final diagnosis is correct, while redundant calls are penalized (Tan et al., 20 May 2026).

This gating is also reflected in measured tool-use behavior. During zero-shot inference, average calls remain near or below 1.0 per sample on the reported benchmarks. On MVTec, Crop is used 62.4% of the time, Enhance 21.3%, Prior 18.7%, Measure 9.8%, with Avg Calls 1.12 and Success >98%. On VisA, the corresponding figures are 54.8%, 18.9%, 24.6%, 7.5%, Avg Calls 1.06, Success >98%. On DTD, Enhance becomes dominant at 34.7%, followed by Crop 28.5%, Prior 6.2%, Measure 2.1%, Avg Calls 0.72, Success >99%. These statistics are consistent with the claim that the reward encourages dataset-adaptive sparseness rather than indiscriminate tool use (Tan et al., 20 May 2026).

5. Empirical performance and ablations

The evaluation is zero-shot on MVTec-AD, VisA, MPDD, DTD, and SDD under a strictly category-disjoint protocol and without paired normal reference at inference. The paper emphasizes balanced accuracy as the main metric across datasets and also reports recall and F1 on shared sets; AUROC and PRO are not reported (Tan et al., 20 May 2026).

Model	MVTec	MPDD	VisA	DTD	SDD	Average
IndusAgent (8B)	83.6	72.7	76.8	95.6	88.9	83.4
Anomaly-OV 7B	74.3	70.3	74.3	90.7	88.7	79.6

IndusAgent is reported to achieve state-of-the-art zero-shot performance among all existing methods. The best open-source baseline listed in the paper is Anomaly-OV 7B with average 79.6, while commercial GPT-4.1 attains average 77.5. The paper also states that IndusAgent improves MVTec by +9.3 points over the best baseline while remaining strong across both objects and textures with an 8B backbone. On anomaly recall, IndusAgent reports DTD 94.1%, MPDD 95.4%, MVTec 85.5%, SDD 83.3%, VisA 73.4%, and average 86.3%. On F1, it reports DTD 96.1, MPDD 78.2, MVTec 89.8, and VisA 78.2 (Tan et al., 20 May 2026).

Ablation results attribute gains to all three major ingredients: SFT, RL, and tools. Relative to the full system, removing RL reduces balanced accuracy to 72.3 on MVTec, 57.6 on VisA, and 74.1 on DTD; removing SFT yields 69.5, 55.5, and 72.8; removing tools yields 78.1, 67.5, and 87.9. Tool-specific ablations show differentiated utility: removing Crop causes the largest drop on VisA, while removing Enhance causes the largest drop on DTD. Reward-design ablations further show that removing the tool-utility term lowers performance relative to the gated design, removing format reward causes a significant drop, and removing localization or type terms degrades grounding and open-vocabulary semantics (Tan et al., 20 May 2026).

6. Broader industrial-agent context, limitations, and outlook

Within the supplied literature, IndusAgent also sits in a broader industrial-agent trajectory. "SceneGenAgent" is described as a strong core for an “IndusAgent” in industrial applications because it couples a planner, verifier, code generator, executor, and refinement loop to generate precise industrial scenes in Siemens Tecnomatix Process Simulate (Xia et al., 2024). "IASelect" addresses a different layer of the industrial-agent stack by implementing IEEE P2660.1 practice selection through a Neo4j property graph that ranks interfacing practices for Industrial Cyber-Physical Systems according to domain, function, and ISO/IEC 25010-derived qualities (Sharma et al., 2021). This suggests that “IndusAgent” can denote both a specific anomaly-detection framework and, more broadly, an emerging pattern of industrial agents that combine structured domain knowledge, explicit tool use, and decision support grounded in industrial constraints.

The limitations reported for IndusAgent itself are concrete. Tool calls introduce additional inference overhead even though the reward discourages unnecessary invocation. Tool reliability remains a dependency: poor crops, noisy enhancements, or weak priors can mislead the policy. The evaluation scope focuses on image-level decision, with future work proposed for pixel-level localization and efficiency analyses. Indus-CoT is teacher- and template-dependent, which may introduce bias. Future directions include scaling to temporal or multi-view streams, edge deployment under tighter compute budgets, and stronger robustness to tool noise (Tan et al., 20 May 2026).

A plausible implication is that the architectural pattern validated by IndusAgent is not limited to anomaly detection. The accompanying literature shows closely related industrial-agent forms for standards-driven practice selection and simulator-grounded scene generation. Taken together, these works point toward industrial agents that do not merely describe or classify, but actively acquire evidence, invoke domain tools, and operate within structured industrial workflows (Xia et al., 2024, Sharma et al., 2021).