COMMET: Conflict Resolution in Domestic Robotics

• The paper introduces COMMET, a hybrid system that combines multimodal retrieval with LLM inference to detect and resolve human-induced conflicts in household robotics.
• It employs a two-stage detection strategy, first using separate speech and visual embeddings with optimized thresholds, then escalating to deep inference for ambiguous cases.
• Preference-aware conflict resolution integrates user annotations and GPT-4o-guided analysis to select socially appropriate actions during domestic mobile manipulation tasks.

COMMET, short for COnflicts in Mobile Manipulation of Everyday Tasks, is a system for detecting and resolving human-induced conflicts that arise when robots execute domestic mobile manipulation tasks in everyday environments. It is motivated by the observation that dynamic and unpredictable human activities can directly or indirectly conflict with robot actions, and that, because such conflicts have social attributes, solutions are not always unique and depend highly on the user’s personal preferences. The system therefore combines conflict detection with preference-aware action selection. Its operation is divided into an offline stage for data collection, labeling, and construction of retrieval and fine-tuning resources, and an online stage for real-time detection, escalation to model inference in ambiguous cases, and preference-aware resolution selection (Li et al., 5 Sep 2025).

1. Problem formulation and operational scope

COMMET is designed for household robotics, specifically for mobile manipulation tasks executed in domestic settings. The central problem is not merely physical feasibility, but interference produced by humans who may block a path, engage the robot verbally, occupy relevant spaces or objects, or alter the state of task-relevant objects. The paper frames these as human-induced conflicts, emphasizing that they may arise both directly and indirectly during task execution (Li et al., 5 Sep 2025).

A defining property of the setting is that conflict resolution is socially contingent. The same physical situation may admit multiple acceptable responses, and the preferred response can vary across users. The system therefore does not treat conflict handling as a single fixed policy. Instead, it couples detection with a mechanism for retrieving prior user annotations and synthesizing a preference-aligned response through a LLM (Li et al., 5 Sep 2025).

The offline/online decomposition is central to this design. Offline processing includes data collection, scenario/trajectory labeling, user annotation, construction of retrieval buffers and fine-tuning datasets. Online processing includes real-time multi-modal conflict detection, escalation to LLM model inference in ambiguous scenarios, and preference-aware resolution selection. This partitioning suggests a deliberate separation between user-specific supervision and low-latency deployment behavior.

2. Input representation and conflict taxonomy

COMMET ingests four real-time input modalities. These are visual observation, consisting of camera images from the robot’s first-person view; final user task, expressed as a high-level goal such as “put apple in sink”; current step, representing the present sub-task; and background speech, obtained from transcribed ambient speech (Li et al., 5 Sep 2025).

The system organizes human-induced conflicts into four categories:

Conflict type	Description
Goal Absence Conflict	Target object absent from scene
Human Interaction Conflict	Another person tries to engage or command the robot
Human Occupancy Conflict	A human passively obstructs the robot
Object State Conflict	Object state prevents continued task progress

This taxonomy is used for fine-grained detection and response. A Goal Absence Conflict corresponds to cases in which the target object is not present in the scene. A Human Interaction Conflict is triggered when another person attempts to engage or command the robot. A Human Occupancy Conflict covers passive obstruction, such as blocking passage or occupying relevant space or objects. An Object State Conflict captures situations in which task progress is prevented by the state of an object, such as a full container or a closed door (Li et al., 5 Sep 2025).

Because the taxonomy is explicit, downstream preference modeling can be conditioned on conflict type. This suggests a modular division between conflict recognition and user-specific response synthesis.

3. Hybrid detection architecture

COMMET uses a hybrid conflict detection approach. The first stage is multi-modal retrieval, and the second stage is fine-tuned LLM model inference for low-confidence cases (Li et al., 5 Sep 2025).

In the retrieval stage, the system maintains two embedding buffers: $B_s$, a speech buffer containing text embeddings for speech, and $B_m$, a multi-modal buffer containing both image and task attribute embeddings. For efficiency, it uses gte-large-en-v1.5 for text embeddings (Zhang et al., 2024) and CLIP for image embeddings (Radford et al., 2021). Retrieval is performed separately for speech and task attributes, interfacing with their respective buffers (Li et al., 5 Sep 2025).

The similarity calculations are:

$$S_s = \max_{E_i \in B_s} \left( \cos(E_s, E_i) \right)$$

and

$$S_t = \max_{E_i \in B_m} \left( w \cdot \cos(E_i^{p}, E_p) + (1 - w) \cdot \cos(E_i^{obs}, E_{obs}) \right)$$

where $w$ is the weight for prompt versus observation similarity. The reported optimal value is $w=0.87$, obtained by grid search to maximize accuracy (Li et al., 5 Sep 2025).

The decision logic uses thresholds $\tau_s$ and $\tau_t$. If a similarity exceeds the corresponding threshold, the system detects the corresponding conflict; speech conflicts, for example, map to Human Interaction. If $S_t < \tau_t$, the system escalates to the second stage (Li et al., 5 Sep 2025).

The second stage employs fine-tuned multimodal LLMs, specifically Qwen 2.5VL-3B and Qwen 2.5VL-7B, for cases that are not well represented in the buffers. These models take the same inputs as retrieval but use what the paper describes as deep reasoning to detect conflicts. The architecture is therefore explicitly organized around a tradeoff: retrieval is fast and low compute, whereas LLM inference offers higher accuracy for novel or ambiguous situations but is slower (Li et al., 5 Sep 2025).

A notable empirical design choice is the use of separate retrieval rather than unified retrieval. The reported results show that separate retrieval is more accurate, which gives the retrieval stage a more structured role than simple nearest-neighbor fallback (Li et al., 5 Sep 2025).

4. Preference-aware conflict resolution

Conflict detection in COMMET is followed by a preference-aware resolution procedure. During offline data collection, users annotate conflict scenarios by selecting a preferred action from a set of solutions for each conflict type and by assigning an emergency level on a 3-level scale (Li et al., 5 Sep 2025).

When a conflict is detected online, the system retrieves user-annotated cases with the same conflict type. GPT-4o is then prompted to summarize the user’s options and emergency levels so as to infer user preferences. The model is instructed, through tailored prompts, to produce a synthesized, preference-aligned choice for the current conflict (Li et al., 5 Sep 2025).

The paper characterizes this stage as preference-aware user resolution with GPT-4o. Its inputs are not only the current conflict type but also the user’s previously expressed preferences and urgency assessments. This makes the resolution stage case-based and user-conditioned rather than globally rule-based.

Two interfaces support this component. A Preference Collection UI allows users to select options and emergency levels for sample conflicts. A Prediction Rating UI allows users to rate the quality of the system’s action selection in post-hoc review (Li et al., 5 Sep 2025). In the reported user study, GPT-4o receives user data for the corresponding conflict type and predicts preferred actions, after which users rate the predictions for continual feedback (Li et al., 5 Sep 2025).

A plausible implication is that COMMET treats preference acquisition as an ongoing process rather than as a one-time configuration step.

5. Empirical evaluation

The reported detection evaluation uses 224 test samples, with the remaining data used for buffer construction and model training. The comparison includes GPT-4o, retrieval-only baselines, fine-tuned Qwen models, and the hybrid COMMET variants (Li et al., 5 Sep 2025).

Model	Accuracy (Total)	Anomaly Acc.	Time (s)
GPT-4o	73.58%	50.00%	5.01
Retrieval (unified)	65.57%	52.17%	0.07
Retrieval (separate)	75.00%	55.43%	0.06
Qwen2.5-VL-3B (ft)	83.96%	65.22%	1.55
Qwen2.5-VL-7B (ft)	87.26%	77.17%	2.01
COMMET (3B)	84.43%	66.30%	1.34
COMMET (7B)	87.26%	77.17%	1.81

The paper’s stated takeaways are that COMMET outperforms GPT-4o in total and anomalous case accuracy, with significantly lower latency, that separate retrieval is more accurate than unified, and that fine-tuned smaller LLMs plus retrieval hybrid beats standalone large LLMs in this task (Li et al., 5 Sep 2025). The abstract also states that, in preliminary studies, the detection module shows better accuracy and latency compared with GPT models (Li et al., 5 Sep 2025).

Hyperparameter tuning is reported for the prompt-versus-visual weight and the escalation thresholds. $w$ is selected by grid search, with the optimal value reported as 0.87. The thresholds $B_m$0 and $B_m$1 are chosen to optimize the tradeoff between retrieval speed and accuracy and the need for LLM inference (Li et al., 5 Sep 2025).

The preference-prediction component is evaluated through a user study in which users provide preferences on presented scenarios, along with emergency ratings. GPT-4o then predicts preferred actions for test cases using the user data corresponding to the relevant conflict type, and users subsequently rate the predictions (Li et al., 5 Sep 2025). The paper does not report summary accuracy numbers for this study in the provided description, so the main quantitative evaluation remains concentrated on detection.

6. Data pipeline, deployment workflow, and research significance

Each COMMET data record contains an observation image, task, current step, background speech, and conflict type. The dataset includes both static scene-based samples and dynamic trajectory samples. The collection protocol emphasizes detection in immediate, realistic time windows, including cases in which humans are within an approximately 1 m “safety” zone (Li et al., 5 Sep 2025).

The real-world deployment workflow is described as follows. First, the robot receives a task and begins execution. During execution, it collects real-time images and speech, performs hybrid detection through fast retrieval with escalation to LLM inference when ambiguous, and, upon detecting a conflict, determines the conflict type and retrieves user preferences. GPT-4o then summarizes preferences and selects a solution, after which the robot executes or suggests the recommended action. The system can also be combined with a skill library for execution primitives, enabling seamless task completion (Li et al., 5 Sep 2025).

The paper provides an illustrative scenario in which the robot is instructed to put a bowl in the sink while a human blocks the way. COMMET detects a Human Occupancy Conflict, consults user preferences, and may select wait if the emergency level is low (Li et al., 5 Sep 2025). This example captures the system’s combination of perceptual conflict detection, conflict typing, and socially conditioned response selection.

Within the literature described in the paper, COMMET is positioned as a practical system for household robots that integrates hybrid architecture, empirical optimization, user-centric preference integration, and a user-friendly interface for data collection and deployment (Li et al., 5 Sep 2025). This suggests an orientation toward real-world robotic operation rather than benchmark-only evaluation. At the same time, the paper characterizes several findings as preliminary studies, so the current evidence is best understood as establishing a system design and an initial empirical workflow for future research rather than a closed solution to human-robot conflict management.