MiCU: End-to-End Smart Home Command Understanding with Large Language Model

Abstract: Command understanding systems in smart home ecosystems can automate device control and substantially improve user experience. However, while they perform well on precise utterances (e.g., "turn on the bedroom light"), they struggle with ambiguous or misaligned commands (e.g., "make the bedroom cozy"). LLMs generalize well across various domains and can outperform traditional rule-based systems on such tasks, but their effectiveness is often constrained by scarce domain-specific data, insufficient task-specific adaptation, and high computational costs. In this paper, we propose an automated training data synthesis workflow using user logs and LLMs; then we build MiCU, a domain-specific LLM that excels at command understanding. Specifically, we employ curriculum learning to inject domain knowledge into the base LLM, then we enhance its reasoning ability via cold-start training combined with reinforcement learning (RL) guided by domain-specific thinking rules. Additionally, we introduce a token compression technique that condenses device description into a single special token, substantially reducing inference overhead and enabling \model-fast, an efficient variant optimized for long inputs. Extensive experiments show that MiCU significantly outperforms baselines, with an average accuracy gain of 20.01% across all device categories. We have deployed MiCU in the Xiaomi Home app, receiving approximately 1.7 million page views per day. Production evaluations show that MiCU reduces user correction rate by 1.57% and increases human audited accuracy by 32.05%. Our data and code are available at https://github.com/xiaomi-research/iot_spec_llm

• The paper introduces MiCU, an LLM-based system that achieves significant accuracy improvements in smart home command understanding.
• It employs a novel methodology combining domain-specific data synthesis, curriculum learning, chain-of-thought enhancement, and token compression.
• Experimental results show a 94.61% accuracy, lower latency, and reduced user correction rates, confirming its real-world utility.

MiCU: An LLM-Based System for End-to-End Smart Home Command Understanding

Introduction and Motivation

End-to-end command understanding in the smart home domain presents unique technical challenges, largely due to the diversity and ambiguity of user utterances, the necessity for device- and action-specific reasoning, and the real-world requirement for scalability and efficiency. Traditional rule-based systems, while interpretable, have limited robustness to linguistic variation and fail to generalize from a combinatorial explosion of device-action pairs. LLMs, by virtue of pretraining on extensive corpora, offer a promising alternative but are constrained by limited domain-specific data, insufficient adaptation to embodied device contexts, and nontrivial computational overhead when handling multi-device, long-context queries.

The paper introduces MiCU, an LLM-based system targeting smart home command understanding in the Xiaomi ecosystem, which encompasses over a billion IoT devices and extensive daily active users. MiCU is instantiated as a specialized LLM capable of both high-accuracy device and action identification from natural language commands—even for ambiguous or misaligned utterances—while maintaining computational feasibility suitable for large-scale deployment.

System Architecture and Methodology

The MiCU framework is comprised of three main components: automated domain-specific data synthesis, staged model adaptation with curriculum learning and chain-of-thought (CoT) enhancement, and a novel token compression strategy for prompt length reduction.

Figure 1: Illustration of the LLM-based end-to-end command understanding process in Xiaomi's smart home ecosystem.

MiCU's command understanding pipeline leverages LLM reasoning to map an arbitrary free-form user instruction to an actionable device-control tuple, implicitly inferring intent, device, and action parameters under device and context constraints.

Data Synthesis: DevCmd

The scarcity of high-quality, annotated training data is addressed by the automated workflow for synthesizing the DevCmd dataset. This uses a dual approach: straightforward template population for easy samples and advanced LLM-assisted annotation (DeepSeek with RAG and SelfRefine) for hard, ambiguous cases.

Figure 2: Dataset synthesis workflow. The top shows direct template-based generation for easy data, while the bottom shows advanced annotation for hard data using DeepSeek.

DevCmd ultimately consists of 50K samples spanning 28 device categories, with strong reliability via iterative LLM annotation, self-reflection, and human validation, ensuring accurate mapping from context-rich natural language commands to device-action targets.

Curriculum Learning

The curriculum learning phase progressively adapts the base LLM (Qwen3-4B-instruct) to the smart home command domain. The model is first continued-pretrained (CPT) on large general-domain corpora (e.g., WuDao, Firefly, LongAlign), then fine-tuned on "easy" template samples to encode foundational device knowledge, and finally fine-tuned on "hard" user-derived samples to embed command ambiguity resolution competence.

Figure 3: Curriculum learning stage. It begins with continual pre-training (CPT) of the base LLM on general datasets (e.g., WuDao), followed by progressive supervised fine-tuning (SFT) on synthesized easy-to-hard samples.

Ablation studies establish that an easy-to-hard curriculum outperforms alternative orderings or mixtures, preventing instability due to the long-tail of complex examples and securing optimal device and action prediction accuracy.

Chain-of-Thought (CoT) Enhancement

MiCU incorporates domain-guided CoT reasoning via SFT on machine-generated thought processes, bootstrapped by DeepSeek-R1 under explicit smart home reasoning prompts. This is further refined by DAPO reinforcement learning, which rewards accurate, well-structured, and syntactically validated outputs.

Figure 4: CoT enhancement stage. We first cold start model via SFT on domain-guided DeepSeek-generated CoT processes, and then refine it using DAPO reinforcement learning.

CoT enhancement demonstrably boosts complex intent inference and action mapping abilities, with further gains achieved through RL reward-shaping for device, action, and structured JSON format correctness.

Token Compression

To address inference efficiency and mitigate prompt length explosion due to device specification redundancy (94% of prompt tokens in Xiaomi Home's production setting), the authors propose a token compression technique: compressing each device category specification into a unique special token.

Figure 5: Token compression technique. The model undergoes a coarse-to-fine adaptation, progressing from the bi-directional mapping of description-special token pairs to task-specific QA fine-tuning.

A bi-directional mapping task followed by fine-grained Q&A ensures the LLM internalizes the mapping between each special token and its functional space. This enables MiCU-fast, a variant with 31.9% fewer tokens and 13.5% higher QPS, while retaining high semantic fidelity with minimal accuracy trade-off.

Experimental Results

Extensive evaluation benchmarks MiCU and MiCU-fast against rule-based systems and state-of-the-art general and proprietary LLMs (GPT-4o, DeepSeek-R1, Llama-3, Qwen3), across both device identification and action prediction tasks.

• MiCU achieves an overall accuracy of 94.61%, improving by 20.01% over the strongest LLM baseline (DeepSeek-V3) and by 28.29% over traditional rule-based approaches.
• MiCU-fast retains 94.01% overall accuracy with dramatically reduced token counts and latency suitable for high-throughput deployment.
• Online deployment over 1.7 million daily PVs yields a 1.57% reduction in user correction rate and a 32.05% gain in human-audited accuracy compared to production heuristics.
• Online A/B testing confirmed a 37.37% reduction in user-initiated corrections, demonstrating robust real-world utility.

  Figure 6: Efficiency study. The average token counts and corresponding accuracy for MiCU and MiCU-fast are shown.

The model saturates performance quickly as a function of parameter and data scale; the 4B architecture is empirically sufficient for the target domain, with only marginal gains observed beyond this size.

Figure 7: Scalability study. Figure (a) shows performance across base model (Qwen3) size, while figure (b) shows performance across training data size using Qwen3-4B.

Failure analysis highlights that the main error cases (especially for lighting devices) involve device priority conflicts and residual contextual ambiguity, indicating clear future directions for incorporating longer-term context and more compositional user modeling.

Figure 8: Failure analysis for the main category "Light".

Theoretical and Practical Implications

This work demonstrates that with a carefully engineered dataset synthesis pipeline, progressive supervised adaptation, domain-guided structured reasoning, and prompt compression, it is possible to build small-scale, highly effective LLMs for complex, real-world command understanding tasks. The mixture of curriculum learning and reinforcement learning for CoT is validated as essential for domain-specific reasoning induction. The findings also illustrate that token-level adaptation—compressing context that the model has been explicitly trained to internalize—is practical for high-QPS, context-rich applications, overcoming computational constraints of generic LLM deployment.

From a broader AI perspective, MiCU exemplifies how post-pretraining, domain-specific pipelines using both synthetic and real user data, combined with novel supervision and optimization techniques, overcome classical LLM limitations in structured, context-dependent prediction. Future directions include incorporating multi-turn dialogue history, rich personalization, and uncertainty-aware inference for better intent disambiguation.

Conclusion

MiCU establishes a new paradigm for applying LLMs to smart home command understanding, combining automated dataset synthesis, curriculum-based and CoT-based adaptation, and innovative prompt compression for real-world deployment. These contributions, validated by significant offline and online gains, effectively close the gap between general LLMs and domain-specialized reasoning agents, setting a foundation for further work in scalable, personalized, and privacy-preserving natural command interfaces in IoT-enabled environments.