---
title: 'COMMET: Conflict Resolution in Domestic Robotics'
url: https://www.emergentmind.com/topics/commet
type: topic
---

# COMMET: Conflict Resolution in Domestic Robotics

COMMET, short for **COnflicts in Mobile Manipulation of Everyday Tasks**, is a system for detecting and resolving human-induced conflicts that arise when robots execute domestic mobile manipulation tasks in everyday environments. It is motivated by the observation that dynamic and unpredictable human activities can directly or indirectly conflict with robot actions, and that, because such conflicts have social attributes, solutions are not always unique and depend highly on the user’s personal preferences. The system therefore combines conflict detection with preference-aware action selection. Its operation is divided into an **offline** stage for data collection, labeling, and construction of retrieval and fine-tuning resources, and an **online** stage for real-time detection, escalation to model inference in ambiguous cases, and preference-aware resolution selection [2509.04836].

## 1. Problem formulation and operational scope

COMMET is designed for household robotics, specifically for mobile manipulation tasks executed in domestic settings. The central problem is not merely physical feasibility, but interference produced by humans who may block a path, engage the robot verbally, occupy relevant spaces or objects, or alter the state of task-relevant objects. The paper frames these as **human-induced conflicts**, emphasizing that they may arise both directly and indirectly during task execution [2509.04836].

A defining property of the setting is that conflict resolution is socially contingent. The same physical situation may admit multiple acceptable responses, and the preferred response can vary across users. The system therefore does not treat conflict handling as a single fixed policy. Instead, it couples detection with a mechanism for retrieving prior user annotations and synthesizing a preference-aligned response through a language model [2509.04836].

The offline/online decomposition is central to this design. Offline processing includes **data collection, scenario/trajectory labeling, user annotation, construction of retrieval buffers and fine-tuning datasets**. Online processing includes **real-time multi-modal conflict detection, escalation to LLM model inference in ambiguous scenarios, and preference-aware resolution selection**. This partitioning suggests a deliberate separation between user-specific supervision and low-latency deployment behavior.

## 2. Input representation and conflict taxonomy

COMMET ingests four real-time input modalities. These are **visual observation**, consisting of camera images from the robot’s first-person view; **final user task**, expressed as a high-level goal such as “put apple in sink”; **current step**, representing the present sub-task; and **background speech**, obtained from transcribed ambient speech [2509.04836].

The system organizes human-induced conflicts into four categories:

| Conflict type | Description |
|---|---|
| Goal Absence Conflict | Target object absent from scene |
| Human Interaction Conflict | Another person tries to engage or command the robot |
| Human Occupancy Conflict | A human passively obstructs the robot |
| Object State Conflict | Object state prevents continued task progress |

This taxonomy is used for **fine-grained detection and response**. A **Goal Absence Conflict** corresponds to cases in which the target object is not present in the scene. A **Human Interaction Conflict** is triggered when another person attempts to engage or command the robot. A **Human Occupancy Conflict** covers passive obstruction, such as blocking passage or occupying relevant space or objects. An **Object State Conflict** captures situations in which task progress is prevented by the state of an object, such as a full container or a closed door [2509.04836].

Because the taxonomy is explicit, downstream preference modeling can be conditioned on conflict type. This suggests a modular division between conflict recognition and user-specific response synthesis.

## 3. Hybrid detection architecture

COMMET uses a **hybrid conflict detection approach**. The first stage is **multi-modal retrieval**, and the second stage is **fine-tuned LLM model inference** for low-confidence cases [2509.04836].

In the retrieval stage, the system maintains two embedding buffers: **$B_s$**, a speech buffer containing text embeddings for speech, and **$B_m$**, a multi-modal buffer containing both image and task attribute embeddings. For efficiency, it uses **gte-large-en-v1.5** for text embeddings [2407.19669] and **CLIP** for image embeddings [2103.00020]. Retrieval is performed separately for speech and task attributes, interfacing with their respective buffers [2509.04836].

The similarity calculations are:

$$
S_s = \max_{E_i \in B_s} \left( \cos(E_s, E_i) \right)
$$

and

$$
S_t = \max_{E_i \in B_m} \left( w \cdot \cos(E_i^{p}, E_p) + (1 - w) \cdot \cos(E_i^{obs}, E_{obs}) \right)
$$

where $w$ is the weight for prompt versus observation similarity. The reported optimal value is **$w=0.87$**, obtained by grid search to maximize accuracy [2509.04836].

The decision logic uses thresholds **$\tau_s$** and **$\tau_t$**. If a similarity exceeds the corresponding threshold, the system detects the corresponding conflict; speech conflicts, for example, map to **Human Interaction**. If **$S_t < \tau_t$**, the system escalates to the second stage [2509.04836].

The second stage employs fine-tuned multimodal LLMs, specifically **Qwen 2.5VL-3B** and **Qwen 2.5VL-7B**, for cases that are not well represented in the buffers. These models take the same inputs as retrieval but use what the paper describes as **deep reasoning** to detect conflicts. The architecture is therefore explicitly organized around a tradeoff: retrieval is **fast** and **low compute**, whereas LLM inference offers **higher accuracy for novel or ambiguous situations but is slower** [2509.04836].

A notable empirical design choice is the use of **separate retrieval** rather than **unified retrieval**. The reported results show that separate retrieval is more accurate, which gives the retrieval stage a more structured role than simple nearest-neighbor fallback [2509.04836].

## 4. Preference-aware conflict resolution

Conflict detection in COMMET is followed by a preference-aware resolution procedure. During offline data collection, users annotate conflict scenarios by selecting a **preferred action** from a set of solutions for each conflict type and by assigning an **emergency level** on a **3-level scale** [2509.04836].

When a conflict is detected online, the system retrieves user-annotated cases with the same conflict type. **GPT-4o** is then prompted to summarize the user’s options and emergency levels so as to infer **user preferences**. The model is instructed, through tailored prompts, to produce a synthesized, preference-aligned choice for the current conflict [2509.04836].

The paper characterizes this stage as **preference-aware user resolution with GPT-4o**. Its inputs are not only the current conflict type but also the user’s previously expressed preferences and urgency assessments. This makes the resolution stage case-based and user-conditioned rather than globally rule-based.

Two interfaces support this component. A **Preference Collection UI** allows users to select options and emergency levels for sample conflicts. A **Prediction Rating UI** allows users to rate the quality of the system’s action selection in post-hoc review [2509.04836]. In the reported user study, GPT-4o receives user data for the corresponding conflict type and predicts preferred actions, after which users rate the predictions for continual feedback [2509.04836].

A plausible implication is that COMMET treats preference acquisition as an ongoing process rather than as a one-time configuration step.

## 5. Empirical evaluation

The reported detection evaluation uses **224 test samples**, with the remaining data used for buffer construction and model training. The comparison includes GPT-4o, retrieval-only baselines, fine-tuned Qwen models, and the hybrid COMMET variants [2509.04836].

| Model | Accuracy (Total) | Anomaly Acc. | Time (s) |
|---|---:|---:|---:|
| GPT-4o | 73.58% | 50.00% | 5.01 |
| Retrieval (unified) | 65.57% | 52.17% | 0.07 |
| Retrieval (separate) | 75.00% | 55.43% | 0.06 |
| Qwen2.5-VL-3B (ft) | 83.96% | 65.22% | 1.55 |
| Qwen2.5-VL-7B (ft) | 87.26% | 77.17% | 2.01 |
| COMMET (3B) | 84.43% | 66.30% | 1.34 |
| COMMET (7B) | 87.26% | 77.17% | 1.81 |

The paper’s stated takeaways are that **COMMET outperforms GPT-4o in total and anomalous case accuracy, with significantly lower latency**, that **separate retrieval is more accurate than unified**, and that **fine-tuned smaller LLMs plus retrieval hybrid beats standalone large LLMs in this task** [2509.04836]. The abstract also states that, in preliminary studies, the detection module shows **better accuracy and latency compared with GPT models** [2509.04836].

Hyperparameter tuning is reported for the prompt-versus-visual weight and the escalation thresholds. **$w$** is selected by grid search, with the optimal value reported as **0.87**. The thresholds **$\tau_s$** and **$\tau_t$** are chosen to optimize the tradeoff between retrieval speed and accuracy and the need for LLM inference [2509.04836].

The preference-prediction component is evaluated through a user study in which users provide preferences on presented scenarios, along with emergency ratings. GPT-4o then predicts preferred actions for test cases using the user data corresponding to the relevant conflict type, and users subsequently rate the predictions [2509.04836]. The paper does not report summary accuracy numbers for this study in the provided description, so the main quantitative evaluation remains concentrated on detection.

## 6. Data pipeline, deployment workflow, and research significance

Each COMMET data record contains an **observation image, task, current step, background speech, and conflict type**. The dataset includes both **static** scene-based samples and **dynamic** trajectory samples. The collection protocol emphasizes detection in immediate, realistic time windows, including cases in which humans are within an approximately **1 m “safety” zone** [2509.04836].

The real-world deployment workflow is described as follows. First, the robot receives a task and begins execution. During execution, it collects real-time images and speech, performs hybrid detection through fast retrieval with escalation to LLM inference when ambiguous, and, upon detecting a conflict, determines the conflict type and retrieves user preferences. GPT-4o then summarizes preferences and selects a solution, after which the robot executes or suggests the recommended action. The system can also be combined with a **skill library** for execution primitives, enabling seamless task completion [2509.04836].

The paper provides an illustrative scenario in which the robot is instructed to put a bowl in the sink while a human blocks the way. COMMET detects a **Human Occupancy Conflict**, consults user preferences, and may select **wait** if the emergency level is low [2509.04836]. This example captures the system’s combination of perceptual conflict detection, conflict typing, and socially conditioned response selection.

Within the literature described in the paper, COMMET is positioned as a practical system for household robots that integrates **hybrid architecture**, **empirical optimization**, **user-centric preference integration**, and a **user-friendly interface** for data collection and deployment [2509.04836]. This suggests an orientation toward real-world robotic operation rather than benchmark-only evaluation. At the same time, the paper characterizes several findings as **preliminary studies**, so the current evidence is best understood as establishing a system design and an initial empirical workflow for future research rather than a closed solution to human-robot conflict management.

Source: https://www.emergentmind.com/topics/commet