---
title: Embodied Questions Answering (EQsA)
url: https://www.emergentmind.com/topics/embodied-questions-answering-eqsa
type: topic
---

# Embodied Questions Answering (EQsA)

Searching arXiv for recent papers on Embodied Questions Answering (EQsA) and closely related Embodied Question Answering work.
Embodied Questions Answering (EQsA) is a formulation in which an embodied agent answers multiple questions that may arrive asynchronously while operating in a 3D environment, rather than solving a single isolated query per episode. In this setting, questions may carry different urgencies, may share overlapping spatial information, and may include dependencies, so correctness alone is insufficient: the system must also schedule, reuse memory, and respond efficiently. The formulation is introduced explicitly in “ParaEQsA: Parallel and Asynchronous Embodied Questions Scheduling and Answering” [2509.11663], and it is best understood as a realistic extension of Embodied Question Answering (EQA), a broader family of tasks in which an agent must explore, remember, reason, and answer grounded questions from egocentric observations [2509.11663].

## 1. From EQA to EQsA

Embodied Question Answering (EQA) is the task of answering natural-language questions by grounding them in visual evidence from an environment the agent observes from an embodied, often egocentric, perspective [2505.20640]. In the classic formulation, the agent may need to navigate, gather evidence, remember what it saw, and reason about the scene before answering [2505.20640]. Standard EQA is typically formulated as answering one single question by actively exploring a 3D environment [2509.11663].

A central limitation of this classical formulation is that it assumes a single, isolated query and sequential completion [2509.11663]. Real deployments, however, often demand handling multiple questions that may arrive asynchronously and carry different urgencies [2509.11663]. EQsA addresses this mismatch by modeling a situation where the agent receives an initial set of questions, additional follow-up questions arrive asynchronously during execution, each question may have a different urgency, and questions may have dependencies and overlapping spatial information [2509.11663].

This shifts the objective from question answering alone to workload management under embodied constraints. A plausible implication is that EQsA imports scheduling, latency, and shared-memory considerations into embodied QA, making it not merely a larger instance of EQA but a distinct systems-level problem.

## 2. Formalization and task structure

EQsA is formalized as a scenario
\[
\xi := (e, T, g^0, \mathcal{Q}_{init}, \mathcal{Q}_{follow})
\]
where \(e\) is the 3D environment/scene, \(T\) is maximum number of time steps, \(g^0\) is the initial pose, \(\mathcal{Q}_{init}\) are initial questions at time 0, and \(\mathcal{Q}_{follow}\) are follow-up questions arriving asynchronously [2509.11663]. Each question is represented as
\[
q_i = (s_i, u_i, y_i, t_i)
\]
where \(s_i\) is question text, \(u_i \in (0,1)\) is urgency, \(y_i\) is the ground-truth answer, and \(t_i \ge 0\) is arrival time [2509.11663]. The paper uses four-way multiple choice answers \(\mathcal{Y}=\{A,B,C,D\}\) [2509.11663].

This formulation differs from prior EQA variants along several axes. In standard EQA, an agent is typically given one question per episode and must actively explore a 3D environment to answer it [2509.11663]. Other extensions broaden the reasoning demand without introducing asynchronous multi-question scheduling. “Multi-Target Embodied Question Answering” [1904.04686] generalizes EQA to questions with multiple targets in them, such as “Is the dresser in the bedroom bigger than the oven in the kitchen?”, requiring navigation to multiple locations and comparative reasoning [1904.04686]. “Knowledge-based Embodied Question Answering” [2109.07872] further generalizes EQA by requiring external knowledge, logical reasoning, and multi-turn memory [2109.07872]. These formulations increase semantic or compositional complexity, but they do not redefine the task around urgency-aware asynchronous question handling.

The distinction is important because EQsA introduces performance criteria that do not arise in one-question EQA. A system may answer all questions correctly yet still be inadequate if it delays urgent questions, duplicates exploration, or fails to exploit cross-question overlap. This suggests that EQsA is partly a scheduling problem over embodied evidence acquisition.

## 3. Core architectural ideas in EQsA systems

The canonical system proposed for EQsA is ParaEQsA, a framework for parallel, urgency-aware scheduling and answering [2509.11663]. Its overall idea is to process multiple embodied questions in a shared, parallelized pipeline instead of answering each question in isolation [2509.11663]. The system is designed as a microservice-style asynchronous architecture with several modules communicating through a Redis stream queue [2509.11663].

The main modules are: Generator, Parser, Finishing Module, Question Pool, Planner, Stopping Module, Answering Module, and Group Memory [2509.11663]. The Generator loads question groups and scenes and emits initial and follow-up questions according to the temporal schedule [2509.11663]. The Parser uses an LLM to extract semantic metadata from each question, specifically urgency and scope, computed once per question [2509.11663]. The Finishing Module checks whether the question can be answered directly using Group Memory and computes a confidence score from retrieved evidence; if confidence is enough, the answer is produced immediately, otherwise the question goes to the Question Pool [2509.11663]. The Planner picks the highest-priority question and performs targeted exploration to gather information for that question [2509.11663]. The Stopping Module periodically decides whether exploration should stop, and the Answering Module uses a VLM plus memory context to produce the final answer [2509.11663].

The defining architectural contribution is Group Memory. It stores observations gathered during exploration, is shared across all questions in the same scenario, and supports retrieval when a new question arrives or when the system attempts a direct answer [2509.11663]. In classical sequential EQA, each question is often solved independently, causing repeated exploration of the same areas [2509.11663]. ParaEQsA avoids this by reusing accumulated knowledge across questions, enabling direct answering of some questions without exploration and reduced redundant navigation [2509.11663].

This shared-memory design connects EQsA to a broader line of memory-augmented embodied QA. MEIA introduces a Multimodal Environment Memory (MEM) that combines object IDs and coordinates as environmental language memory and visual observation pictures as environmental image memory, plus an environmental floor plan derived from accumulated 3D observations [2402.00290]. GraphEQA uses real-time 3D metric-semantic scene graphs and task relevant images as multi-modal memory [2412.14480]. Long-term Active EQA introduces a structured memory system for robots in the form of a Robotic Mind Palace \(\mathcal{M} = [G_0, G_1, \dots, G_N]\), where each \(G_i\) is a scene-graph world instance corresponding to one episode in time [2507.12846]. EQsA differs in that its memory is explicitly shared among concurrently pending questions, but it inherits the same premise: embodied QA requires persistent, structured external memory rather than frame-local reasoning alone.

## 4. Scheduling, urgency, and dependency reasoning

The priority-planning mechanism in ParaEQsA is implemented through the Question Pool and its Updater [2509.11663]. Instead of FIFO order, questions are assigned a priority:
\[
P(q_i) = w_u \cdot Urgency(q_i) + w_s \cdot Scope(q_i) + w_r \cdot Reward(q_i) + w_d \cdot Dependency(q_i)
\]
where \(w_u, w_s, w_r, w_d\) are weights for the four components [2509.11663].

Urgency is modeled nonlinearly:
\[
Urgency(q_i) = -\ln(1-u_i)
\]
so that as \(u_i\) increases toward 1, urgency grows sharply [2509.11663]. Scope is defined as
\[
Scope(q_i)= \begin{cases} 1, & \text{if scope is local} \\ 0, & \text{if scope is global} \end{cases}
\]
favoring local questions because they can often be answered with limited exploration [2509.11663]. Reward is based on how many other questions refer to the same or nearby locations or objects, encouraging exploration of information-rich places that may help multiple questions [2509.11663]. Dependency is defined as
\[
Dependency(q_i)= \begin{cases} 1, & \text{if status is ready} \\ 0, & \text{if status is pending} \end{cases}
\]
and dependencies are maintained in a DAG [2509.11663].

This machinery formalizes a major conceptual shift. Classical EQA asks where to navigate next for the current question; EQsA asks which question should control the next navigation decision. A plausible implication is that urgency-aware scheduling becomes as fundamental as target selection, because poor question ordering directly degrades both efficiency and responsiveness.

The exploration strategy remains grounded in the EQA tradition. ParaEQsA uses targeted exploration aligned with Explore-EQA style frontier-based navigation [2509.11663]. Explore-EQA itself builds a semantic map of the scene from depth information and visual prompting of a VLM, then uses conformal prediction to calibrate the VLM’s question answering confidence for stopping [2403.15941]. GraphEQA performs hierarchical planning over object and frontier nodes in a real-time 3D semantic scene graph [2412.14480]. FAST-EQA combines question-conditioned target localization, global and local relevance-guided navigation, bounded visual memory, and LLM-based reasoning [2602.15813]. These methods focus on efficient evidence acquisition for one question; EQsA reuses such exploration policies but places them under a higher-level scheduler.

## 5. Benchmarks and evaluation

The benchmark introduced for EQsA is PAEQs, the Parallel Asynchronous Embodied Questions benchmark [2509.11663]. It is built on HM3D indoor 3D scenes and contains 40 high-quality residential scenes, each with 5 questions, for 200 total questions [2509.11663]. Each scenario has 3 initial questions and 2 follow-up questions, with follow-up questions introduced 120 seconds after the previous question [2509.11663]. All 200 questions were manually written and reviewed, and ground-truth answers were verified carefully [2509.11663].

The benchmark covers five semantic types: existence, counting, state, identification, and location [2509.11663]. Each question also has an urgency score \(u \in [0,1]\), generated using gpt-oss-20b with few-shot prompting and temperature 0 [2509.11663]. Urgency is grouped into low, medium, and high categories [2509.11663]. The agent does not know these labels at test time; they are only for benchmarking and evaluation [2509.11663].

EQsA requires metrics beyond answer accuracy. ParaEQsA defines:
\[
\text{Acc} = \frac{1}{|\mathcal{Q}|} \sum_{q_i \in \mathcal{Q}} \mathbb{I}(\hat{y}_i = y_i)
\]
for accuracy [2509.11663]. It then introduces Direct Answer Rate (DAR),
\[
\text{DAR} = \frac{|\mathcal{Q}_{direct}|}{|\mathcal{Q}|}
\]
where \(\mathcal{Q}_{direct}\) is the subset of questions answered directly from memory with zero exploration steps [2509.11663]. It also defines normalized steps,
\[
\text{NS} = \frac{1}{|\mathcal{Q}|} \sum_{q_i \in \mathcal{Q}} \frac{\text{used\_steps}_i}{\text{max\_steps}_i}
\]
to measure exploration efficiency [2509.11663]. The key responsiveness metric is Normalized Urgency-Weighted Latency (NUWL), intended to penalize questions that wait longer before being addressed, with more urgent questions contributing more heavily [2509.11663].

These metrics are distinctive because they disentangle three aspects of performance: correctness, memory reuse, and urgency-sensitive delay. A plausible implication is that DAR operationalizes a form of embodied amortization: a good EQsA system converts past exploration into immediate future answers.

## 6. Relation to neighboring EQA problem settings

EQsA emerges against a rapidly broadening EQA landscape. Several nearby task variants clarify what EQsA includes and what it does not.

Long-term Active Embodied Question Answering (LA-EQA) requires an agent to combine persistent long-term memory with active exploration in a changing environment [2507.12846]. It is defined as the tuple
\[
(Q, M, E, x_0, A^*)
\]
where \(Q\) is the question, \(M=[m_1,\dots,m_N]\) is a set of episodic memories, \(E\) is the current environment, \(x_0\) is the initial pose, and \(A^*\) is the ground-truth answer [2507.12846]. Its action space is explicitly three-way: retrieve a past observation from memory, explore the current environment, or answer and terminate [2507.12846]. LA-EQA studies recall–exploration trade-offs over long timescales, whereas EQsA studies scheduling–reuse trade-offs across simultaneous or asynchronous questions.

NoisyEQA benchmarks whether EQA agents can remain reliable when humans ask noisy questions containing mistaken assumptions, memory errors, perceptual confusions, or semantic substitutions [2412.10726]. Its four noise types are Latent Hallucination Noise, Memory Noise, Perception Noise, and Semantic Noise [2412.10726]. It adds a Self-Correction mechanism with NAP and NACoT prompting, and evaluates detection and correction through DR and CR metrics [2412.10726]. This makes question reliability the central issue, rather than multi-question scheduling.

AbstainEQA asks whether an embodied agent should answer at all. It introduces five abstention categories: actionability limitation, referential underspecification, preference dependence, information unavailability, and false presupposition [2512.04597]. The paper argues that abstention is a prerequisite for reliable embodied interaction and a necessary basis for effective clarification [2512.04597]. EQsA does not center abstention, but in realistic deployment the two concerns intersect: urgency-aware scheduling is valuable only if the agent can also identify when a question should not be answered.

Other extensions widen the reasoning target. S-EQA introduces situational queries such as “Is the house ready for sleeptime?”, where the answer depends on consensus over multiple object-states [2405.04732]. StreamEQA moves EQA into streaming video understanding over egocentric video streams with embodied and temporal dimensions [2512.04451]. EQA-Decision extends embodied QA from perception to decision-making with four reasoning dimensions—static scene construction, spatial understanding, task dynamics reasoning, and instant decision [2605.25813]. These works broaden embodied QA along semantics, time, and action; EQsA broadens it along concurrency and responsiveness.

## 7. Empirical findings, significance, and open issues

ParaEQsA is evaluated against two sequential baselines adapted from recent EQA systems: Explore-EQA and Memory-EQA [2509.11663]. The reported results are:

| Method | Acc | DAR | NS | NUWL |
|---|---:|---:|---:|---:|
| ParaEQsA | 0.65 | 0.09 | 0.321 | 0.204 |
| Explore-EQA | 0.62 | 0.00 | 0.472 | 0.551 |
| Memory-EQA | 0.64 | 0.00 | 0.410 | 0.474 |

These results show slightly better accuracy, substantially reduced normalized steps, greatly lower NUWL, and nonzero direct answer rate only for ParaEQsA [2509.11663]. The paper reports about a 57% reduction in NUWL versus Memory-EQA and 63% versus Explore-EQA [2509.11663]. Ablations show that removing priority altogether causes the biggest degradation, urgency modeling is especially important for lowering NUWL, and reward and dependency both help efficiency and timeliness [2509.11663].

The central insight is that embodied agents in realistic environments should not solve questions one by one in arrival order [2509.11663]. Instead, they should exploit shared information across questions, prioritize urgent and ready questions, exploit local or high-reward regions first, and delay low-urgency or blocked questions when appropriate [2509.11663]. This leads to less redundant exploration, more direct answers from memory, faster responses to critical questions, and better overall workload handling [2509.11663].

At the same time, EQsA inherits unresolved limitations from the wider EQA literature. Memory quality remains structurally important: MEIA’s ablations show that removing multimodal memory lowers embodied QA accuracy from 70.0% to 56.7% [2402.00290], while GraphEQA’s ablations show that scene-graph-only or image-only memory underperforms the combined representation [2412.14480]. Robustness to noisy or ambiguous questions remains weak in current models [2412.10726; 2512.04597]. Efficient stopping remains an open issue, addressed differently by conformal prediction in Explore-EQA [2403.15941], outlier-based stopping in EfficientEQA [2410.20263], GPT-4o-based stopping in FAST-EQA [2602.15813], and value-of-information-based stopping in LA-EQA [2507.12846]. This suggests that a mature EQsA system will likely need to combine asynchronous scheduling with calibrated stopping, memory retrieval, abstention, and noise handling.

In summary, EQsA designates a multi-question, asynchronous, urgency-aware extension of embodied question answering in which shared memory and question scheduling are first-class components [2509.11663]. Its significance lies in recasting embodied QA from a single-query benchmark into a persistent interactive workload, where the agent must decide not only where to look and when to stop, but also which question deserves attention next.

Source: https://www.emergentmind.com/topics/embodied-questions-answering-eqsa