IndustryAssetEQA: Industrial Embodied QA

• IndustryAssetEQA is a neurosymbolic system that combines episodic telemetry with a Failure Mode Effects Analysis Knowledge Graph to enable context-aware industrial maintenance.
• It reframes maintenance queries as time-situated, decision-centric interactions by processing structured episodes and enforcing deterministic provenance checks.
• The system employs neural-symbolic fusion, counterfactual simulation, and rigorous verifier checks to provide actionable, safe, and auditable intervention recommendations.

Searching arXiv for the specified paper and closely related embodied QA work in industrial settings. arXiv search query: (Shyalika et al., 25 Apr 2026) IndustryAssetEQA embodied question answering industrial maintenance IndustryAssetEQA is a neurosymbolic operational intelligence system for industrial maintenance that combines episodic telemetry representations with a Failure Mode Effects Analysis Knowledge Graph (FMEA-KG) to enable Embodied Question Answering (EQA) over industrial assets. It reframes maintenance question answering as an embodied, decision-centric problem in which an operator asks questions about a concrete episode defined by an asset identifier and a time window, and the system must perceive telemetry and context, reason over domain knowledge, predict intervention effects via an explicit risk simulator, and return an answer that is both grounded, with provenance, and verifiable via deterministic checks. The system is motivated by persistent shortcomings of LLM-only assistants in industrial contexts: generic explanations disconnected from the asset’s episode and sensors, omitted verifiable provenance, and non-testable counterfactuals and action suggestions without an explicit risk model, all of which undermine trust in safety- and cost-critical settings (Shyalika et al., 25 Apr 2026).

1. Industrial maintenance as embodied question answering

IndustryAssetEQA defines EQA for industrial assets as a time-situated interaction loop centered on a specific episode rather than on a generic asset class. The episode is defined by an asset identifier and a time window, and the query space is organized into five compact types aligned with the perception → reasoning → prediction → decision loop: descriptive, temporal, diagnostic, counterfactual, and action-oriented. This formulation makes the maintenance assistant answerable not only for what it says, but also for what telemetry window, sensor features, maintenance records, and failure semantics support the answer (Shyalika et al., 25 Apr 2026).

The interaction model is explicit and structured. The system retrieves the relevant episode, constructs structured evidence, augments it with FMEA knowledge, optionally simulates interventions, and produces a JSON-constrained answer with direct_answer, reasoning_answer, provenance, and confidence. A Verifier and Safety Gate then validate structure, evidence references, and, when applicable, counterfactual consistency before recommendations are surfaced or routed to human review. Although the formulation is not cast as a full MDP/POMDP, the embodied loop is stated directly as observations (telemetry episodes and alerts) → state abstraction (episode-level features aligned to a knowledge graph) → reasoning and prediction (diagnosis and risk simulation) → decision (action recommendation) → verification and safety gate. In this sense, IndustryAssetEQA operationalizes embodied interaction through telemetry, symbolic maintenance knowledge, and deterministic post hoc checks rather than through physical navigation.

This design matters because the target questions are inherently intervention-sensitive. A diagnostic answer that names a failure mode, a counterfactual answer that predicts what would happen if maintenance were performed, and an action-oriented answer that recommends whether to open a work order all have operational consequences. The system therefore treats answer generation as only one component of a broader advisory pipeline.

2. Episodic telemetry representation and structured memory

The perceptual substrate of IndustryAssetEQA is the episodic telemetry representation produced by the Fact Extractor. For each failure at time $t_f$ and machine $m$, the extractor builds a history window $[t_f - \Delta, t_f]$ and computes per-sensor summary descriptors

$$F_s = \{\mu_s, \sigma_s, \min_s, \max_s, \text{trend}_s\},$$

where trend is the slope of a least-squares fit over the window. It also derives error_count_last_window, distinct_error_types_last_window, hours_since_last_maint_<component>, hours_since_last_maint_any, machine_age, and model one-hots such as model_<name>. Healthy episodes are sampled at times with no failure in $[t, t+H]$ and use $$[t-\Delta, t]`. Each episode is labeled, enriched with KG context, and emitted as a JSONL record containing full provenance, including source files, time ranges, and row indices (<a href="/papers/2604.23446" title="" rel="nofollow" data-turbo="false" class="assistant-link" x-data x-tooltip.raw="">Shyalika et al., 25 Apr 2026</a>).</p> <p>The extracted facts are persisted in an Episodic Store implemented as a SQLite-backed store with a <code>facts</code> table and a <code>features</code> table. The <code>facts</code> table stores full JSON episodes with metadata and time bounds, whereas the <code>features</code> table explodes numeric features into atomic name-value rows for efficient numeric queries. This supports deterministic retrieval by asset, label, and time, feature-threshold search, and verifier-side checks of the form$$\{ f_i \mid x_{i,j} \,\bowtie\, \tau \}$. The store therefore functions as structured memory that bridges raw telemetry and higher-level reasoning.

A representative example in the paper is the fact pdm_m56_comp3_2015-01-02T03, a failure_window for machine_56 labeled comp3 over the interval 2015-01-01 03:00:00 to 2015-01-02 03:00:00, with features such as volt_mean=169.0608, volt_std=17.8556, volt_min=139.2351, hours_since_last_maint_comp3=477.0, and machine_age=10.0. Its provenance explicitly cites PdM_telemetry.csv, PdM_failures.csv, PdM_errors.csv, PdM_maint.csv, and PdM_machines.csv. The system’s claim to groundedness depends on this episode-level serialization: answers are expected to refer back to concrete records rather than to abstract domain priors.

3. Neurosymbolic architecture and FMEA-KG grounding

The architecture integrates neural and symbolic components through a series of fixed fusion points. The symbolic core is the FMEA-KG, a domain-level Failure Mode and Effects Analysis Knowledge Graph constructed from ISO-style specifications and expert-curated documentation using EMPWR. It contains 210 entities and 1004 relations, including 63 failure modes mapped to nine asset categories: drilling, marine, electrical, mechanical, rotating machinery, safety and control, subsea, well completion, and well intervention. Nodes represent asset classes, components, failure modes, sensor types or abstractions, and maintenance actions, while key relations include affects, component_of, indicated_by, and mitigated_by. Failure modes carry severity and occurrence metadata, associated sensors, typical indicators, and recommended actions. Triple-level verification and SME review indicate 96% validity across 1004 candidate triples, although structurally weaker relations such as sample/example are reported as noisier (Shyalika et al., 25 Apr 2026).

The neural components are the LLMs and the multinomial logistic regression risk model $P(y \mid x)$. The LLM never freewrites from generic memory; Prompt Builder constructs a compact evidence block from the Episodic Store and the FMEA-KG, adds task-scoped instructions, and enforces a strict JSON output contract. Neurosymbolic fusion occurs during fact extraction, prompt building, QA construction, and verifier checks. There is no learned path scoring over the graph; instead, graph reasoning is implemented through retrieval of relevant nodes and relations plus rule-like checks in the Verifier.

The resulting architecture includes the Fact Extractor, Episodic Store, FMEA-KG, Causal Simulator, EQA Builder, Prompt Builder, Verifier, and Safety Gate. Runtime orchestration follows the sequence operator query → classification by task type → episodic retrieval and KG context → optional simulator call → structured generation → verification and safety gate → action integration, such as CMMS work order creation, or human escalation. This split is central to the system’s design philosophy: linguistic fluency is retained, but validity and auditability are imposed by structured evidence, symbolic constraints, and deterministic policy checks.

4. Counterfactual reasoning, action recommendation, and auditability

IndustryAssetEQA formalizes counterfactual intervention by a parametric “do” operator on episode features. The risk model is a multinomial logistic regression over episode features $x$, and interventions are implemented as explicit substitutions $x \rightarrow x^{\mathrm{do}}$. The simulator defines

$$r_{\mathrm{before}} = 1 - P(y=\mathrm{healthy}\mid \mathbf{x}), \qquad r_{\mathrm{after}} = 1 - P(y=\mathrm{healthy}\mid \mathbf{x}^{\mathrm{do}}),$$

with signed risk change

$m$0

It returns pre- and post-intervention class probabilities for all labels and uses a confidence heuristic

$m$1

For counterfactual and action tasks, the prompt contract requires a numeric counterfactual object with risk_before, risk_after, and a direction consistent with the numbers (Shyalika et al., 25 Apr 2026).

The paper’s worked counterfactual example resets hours_since_last_maint_comp3 from 477.0 to 0.0 for machine_56, yielding a direct answer that “The risk of failure would decrease,” with risk_before: 1.0, risk_after: 9.256e-06, delta_risk: -0.99999, and direction: "decrease". The action-recommendation example then combines high surrogate risk, severe KG failure modes, and KG-recommended interventions to conclude that “A maintenance work order should be opened now.”

Auditability is treated as a first-class property. Answers must include a provenance object citing fact_id, referenced features, files, row indices, telemetry window, and KG nodes or IDs when used. The Verifier cross-checks all citations against the Episodic Store and the KG; Struct.OK and Prov.OK are gatekeeping criteria. The Safety Gate logs failures and decisions, enforces admissibility, and routes low-confidence or simulator-inconsistent outputs to human review. A common misconception is that the “do” operator implies identified causal effects. The paper explicitly cautions that this intervention mechanism is a surrogate risk estimator implemented as feature substitution over a discriminative $m$2 and that the outputs are not proven causal effects.

5. Evaluation protocol, datasets, and empirical results

The evaluation uses black-box API instantiations of GPT-4o-mini and Claude Sonnet 4 under five configurations: LLM-only; LLM+Episodic; LLM+Episodic+KG; Provenance-Enforced (no simulator); and Full IndustryAssetEQA. All models receive only structured episodic evidence and, where applicable, KG context and simulator outputs, and must return machine-parseable JSON with provenance. Metrics are computed per instance and aggregated: Structural Validity (Struct.OK), Provenance Accuracy (Prov.OK), Label Consistency (Label Cons.), Temporal/Counting Accuracy, Counterfactual Direction Accuracy (CF Acc.), Entailment Pass (Entail.Pass) using FacebookAI/roberta-large-mnli with threshold 0.80, Claim Precision (Claim Prec.), and Full Pass Rate (Full_pass). McNemar’s test on the Microsoft PdM dataset shows statistically significant differences (p < 0.05) between model variants for descriptive, diagnostic, and counterfactual questions, and non-significant differences for temporal and action-oriented questions (p = 0.14 and 0.93) (Shyalika et al., 25 Apr 2026).

Industrial setting	Episodes	QA instances
Microsoft Azure PdM (rotating machinery)	5716	Descriptive 5716; Temporal 5716; Diagnostic 5716; Counterfactual 761; Action 902
NASA C-MAPSS (turbofan engines)	4842	Descriptive 4842; Temporal 4842; Diagnostic 4842
Genesis CPS (cyber–physical production systems)	478	Descriptive 120; Temporal 478; Diagnostic 214; Counterfactual 210; Action 23
Hydraulic test rig (hydraulic systems)	2205	Descriptive 2205; Temporal 2205; Diagnostic 50; Counterfactual 2184; Action 2205

Across the four settings, the total is 13,241 episodes, with 12,883 descriptive QAs, 13,241 temporal QAs, 10,822 diagnostic QAs, 3155 counterfactual QAs, and 3130 action QAs. The headline result is that, compared to LLM-only baselines, improvements reach up to +0.51 for structural validity, +0.47 for counterfactual accuracy, and +0.64 for explanation entailment, while severe expert-rated overclaims fall from 28% to 2%, an approximately 93% reduction. For GPT-4o-mini, Struct.OK improves from 0.42 to 0.88, Prov.OK from 0.47 to 0.89, Label Cons. from 0.62 to 0.94, CF Acc. from 0.45 to 0.88, Entail.Pass from 0.08 to 0.72, and Claim Prec. from 0.12 to 0.67. For Claude Sonnet 4, the corresponding changes are 0.39 → 0.90, 0.44 → 0.89, 0.59 → 0.95, 0.44 → 0.91, 0.10 → 0.78, and 0.12 → 0.74. The expert study on 22 QA pairs with 10 experts reports answerability 97% versus 46% for LLM-only, data grounding 4.5 ± 0.6 versus 3.0 ± 0.9 with paired t-test p < 0.001, severe overclaims 2% versus 28%, and inter-annotator agreement Fleiss’ κ = 0.63.

Ablations show complementarity among the system’s components. Removing the simulator reduces CF Acc. from 0.88 to 0.49, Entail.Pass from 0.72 to 0.59, and Full Pass from 0.89 to 0.72. Removing provenance enforcement reduces Entail.Pass from 0.72 to 0.42 and Full Pass from 0.89 to 0.19. Removing FMEA-KG reduces Entail.Pass from 0.72 to 0.59, Full Pass from 0.89 to 0.35, and CF Acc. from 0.88 to 0.61. Removing episodic memory reduces Entail.Pass from 0.72 to 0.27, Full Pass from 0.89 to 0.36, and CF Acc. from 0.88 to 0.34. The paper’s conclusion is therefore specific: episodic grounding, KG grounding, provenance enforcement, and simulator integration are complementary and jointly necessary for deployable embodied QA.

6. Position within industrial EQA research and stated limitations

Within the broader literature, Embodied Question Answering is often defined as the evaluation of an agent that must perceive an environment over time and answer a question grounded in its experience. IndustryAssetEQA specializes that notion to maintenance episodes defined by telemetry windows and asset identifiers, replacing navigation-centric embodiment with time-situated operational perception, symbolic failure semantics, and intervention-aware verification (Chen et al., 12 Jun 2025). A separate but related industrial line is the warehouse benchmark "IndustryEQA," which provides safety-critical episodic memory videos with industrial assets, dynamic human agents, and hazards in warehouse scenarios. In that work, “IndustryAssetEQA” is explicitly described as “an asset-centric slice of IndustryEQA” and “not a separate dataset,” which distinguishes that benchmark usage from the maintenance system described here (Li et al., 27 May 2025).

The limitations reported for IndustryAssetEQA are operational rather than rhetorical. Counterfactual modeling is based on feature substitution over a discriminative classifier and is therefore a surrogate risk estimator rather than an identified structural causal model. The FMEA-KG captures core semantics but exhibits variability across relation types, with weaker structural relations carrying more noise. Episodes use fixed windows, so long-horizon or multi-scale precursors may be missed. Evaluation is offline on four benchmarks, with a controlled pilot reserved for measuring operational impact and calibrating counterfactual estimates. The full pipeline also introduces additional engineering and runtime costs relative to LLM-only systems. These limitations do not negate the system’s contribution; they specify the boundary conditions under which its claims hold.

A plausible implication is that IndustryAssetEQA marks a shift from maintenance assistants that generate plausible language to maintenance systems that are expected to satisfy structural validity, provenance accuracy, simulator-aligned counterfactual consistency, and safety gating. In that narrower but operationally stronger sense, it defines industrial EQA as an auditable decision-support problem rather than as a free-form conversational interface.