HomeLLaMA: Local, Private Smart Home LLM

• HomeLLaMA is a local-first smart home assistant that uses a tailored small language model to preserve privacy and enable personalized control.
• It employs a two-stage design with offline dataset synthesis and dynamic, user-mediated cloud fallback via PrivShield for secure plan generation.
• The system excels in device relevance mapping and long-term personalization by integrating local SLM enhancement and structured user profiling.

Searching arXiv for HomeLLaMA and closely related smart-home LLM papers to ground the article in current literature. HomeLLaMA is an on-device assistant for privacy-preserving and personalized smart home serving built around a tailored small LLM (SLM). It is designed to preserve the natural-language flexibility associated with cloud LLM assistants while avoiding routine transmission of commands, user profiles, and home device states to remote servers. The system combines local SLM enhancement, user-in-the-loop local–cloud collaboration, and dynamic user preference learning, and supplements local inference with an optional privacy-preserving cloud fallback called PrivShield. In adjacent literature, related systems are sometimes described as “HomeLLaMA-style,” but the term HomeLLaMA properly refers to the specific local-first architecture introduced in “Towards Privacy-Preserving and Personalized Smart Homes via Tailored Small LLMs” (Huang et al., 10 Jul 2025).

1. Problem formulation and design objectives

HomeLLaMA addresses a tension identified in LLM-based smart-home assistance. Cloud LLM assistants can understand free-form, under-specified commands and generate high-quality action plans, but they require sending commands, user profiles, and home device states to remote servers. Local SLM assistants preserve privacy because they run on-device, but they usually underperform because small models lack the reasoning and world knowledge of cloud LLMs. HomeLLaMA is proposed as a middle ground: an on-device smart-home assistant that aims to be privacy-preserving, personalized, and still high quality enough for daily home assistance (Huang et al., 10 Jul 2025).

The system is organized around two stages. In the offline stage, the local model is enhanced using a synthesized smart-home dataset generated with the help of a cloud LLM. In the online stage, HomeLLaMA responds locally by default, updates user profiles continuously, and only consults the cloud when the user explicitly approves it after dissatisfaction with local output. This local-first design is central to the project’s privacy model. A common misconception is that HomeLLaMA is simply a compressed cloud assistant running locally. The paper instead presents it as a composite architecture with explicit mechanisms for device relevance identification, personalized planning, user-mediated escalation, and privacy-aware cloud interaction.

2. System architecture and local-first operation

HomeLLaMA has three main technical modules: Local SLM Enhancement, Multi-party Interaction, and User Preference Learning. These modules are arranged so that local execution is the default path and cloud access is an exception rather than a prerequisite (Huang et al., 10 Jul 2025).

Module	Core role	Principal mechanism
Local SLM Enhancement	Improve local smart-home reasoning	GPT-4/GPT-4-Turbo synthesis and QLoRA fine-tuning
Multi-party Interaction	Coordinate user, assistant, and cloud	local confirmation loop and optional PrivShield-mediated escalation
User Preference Learning	Maintain long-term personalization	structured profiles, FAISS storage, top-3 retrieval

Operationally, the local assistant receives a command, generates an action plan, and asks for confirmation. The user can Accept, Advice, or Reject. Only when the user explicitly accepts a plan is it translated into executable smart-home control actions. If the response is rejected, HomeLLaMA can ask permission to consult a cloud LLM for a better answer; this step is optional and is mediated by PrivShield. The paper emphasizes that once deployed, HomeLLaMA facilitates proactive interactions by continuously updating local SLMs and user profiles.

This architecture is not merely a deployment choice. It encodes a service model in which privacy, plan quality, and personalization are balanced through staged reasoning: local generation first, user correction second, and remote consultation only under explicit consent. That sequencing distinguishes HomeLLaMA from cloud-dominant smart-home assistants and from purely local assistants that lack a recovery path when the local model fails.

3. Tailored SLM enhancement and device relevance modeling

The paper reports that vanilla SLMs such as LLaMA3-8B struggle most with device relevance: they fail to identify the right smart-home devices from under-specified commands. A simple fine-tune on existing crowd-sourced command-action data produced only small gains because the data did not sufficiently cover command diversity, scenario diversity, and device diversity. HomeLLaMA therefore uses a cloud LLM as a teacher to synthesize a better training set (Huang et al., 10 Jul 2025).

The augmentation pipeline begins from 90 seed commands manually selected from IFTTT / crowd-sourced smart-home commands, with 10 commands each across 9 scenarios. It then performs vertical synthesis, in which GPT-4 generates new commands for different scenarios from the seed commands, and horizontal synthesis, in which GPT-4 rewrites commands into different expression styles while preserving meaning. A ROUGE-L-based inspector removes redundant commands according to

$$\text{Retain } s_{\text{new}} \iff \max_{i \in \{1, 2, \dots, n\}} R(s_{\text{new}}, s_i) < \alpha$$

where $R(s,s')$ is ROUGE-L similarity and the default $\alpha = 0.7$. GPT-4 then labels each synthesized command with the relevant device set from a comprehensive device universe of 39 devices, yielding

$$\mathcal{D}_a = \{s_i \rightarrow G(s_i, \boldsymbol{D})\}, \ \forall s_i\in \mathcal{S}_a .$$

Fine-tuning is performed with QLoRA on LLaMA3-8B, with rank $r=64$, $lora\_alpha=128$, learning rate $3\times10^{-5}$, dropout 0.1, 3 epochs, and an 80/20 train-test split. The resulting adapter is trained to behave as a device identifier and to improve the SLM’s smart-home planning ability without full-model retraining. HomeLLaMA also adopts a tailored two-step inference paradigm: first generate a comprehensive relevant device set for a command, assuming a large home with many devices; then match this set with the actual devices available in the target home,

$$\boldsymbol{D}_{f}^i = \boldsymbol{D}_{l} \cap \boldsymbol{D}_{i},$$

where $\boldsymbol{D}_{l}$ is the broader candidate set, $\boldsymbol{D}_{i}$ is the device set in home $R(s,s')$0, and $R(s,s')$1 is the final matched set.

This two-step formulation is significant because it separates semantic understanding from deployment-specific actuation. The local model does not need to memorize every possible home configuration. Instead, it learns a broader device relevance mapping and then intersects that mapping with the actual installed device set. A plausible implication is that this reduces brittleness when device inventories vary across households.

4. PrivShield and privacy-preserving cloud collaboration

PrivShield is the paper’s privacy layer for cloud assistance. It is described as a lightweight obfuscation module operating as an SLM-in-the-middle between the user and the cloud LLM. The paper motivates PrivShield by noting that even if only the user command is sent to the cloud, raw commands can still reveal personal identity details, routines, inferred activity patterns, and household state. The example given is that a sequence of commands could reveal that the user is watching TV and then going to bed (Huang et al., 10 Jul 2025).

PrivShield has three stages. First, the local SLM rewrites the user command by filtering sensitive personal information and paraphrasing the command to reduce redundancy and make it less traceable. Second, given the rewritten command, PrivShield asks the local SLM to generate $R(s,s')$2 adversarial commands from unrelated scenarios; these are assigned unique IDs, shuffled, and mixed with the real command, while only the ID of the real command is tracked locally. Third, the cloud returns plans for all mixed commands, and PrivShield extracts the response belonging to the original command, feeds it back to the local assistant as advice, and the local assistant produces the final user-facing plan.

PrivShield is invoked only when the user rejects the local response and explicitly agrees to consult the cloud. Users can choose the number of adversarial commands $R(s,s')$3: higher $R(s,s')$4 improves privacy, but increases latency and cloud cost. The paper further characterizes privacy in terms of activity-monitoring attacks on mixed queries and gives the overall attack success relation as

$R(s,s')$5

where $R(s,s')$6 is the success rate when PrivShield is used and $R(s,s')$7 is the frequency of PrivShield use. As the user profile database matures, $R(s,s')$8 decreases, so privacy protection improves over time.

Two misconceptions are explicitly countered by this design. The first is that “local by default” is sufficient to solve privacy leakage; the paper argues that raw-command exposure remains a risk whenever cloud escalation occurs. The second is that privacy preservation and plan quality are mutually exclusive; HomeLLaMA instead treats privacy as a controllable tradeoff layered on top of selective cloud consultation.

5. User preference learning and long-term personalization

Because local SLMs have limited context length, HomeLLaMA does not simply retain and replay full conversation history. It instead maintains a compact, dynamically updated user-profile database. After each interaction, the system summarizes the conversation into a structured profile containing topics, preferences, the current command, and the final approved plan. These profiles are embedded and stored in a vector database $R(s,s')$9 using FAISS (Huang et al., 10 Jul 2025).

When a new profile $\alpha = 0.7$0 is created, it is compared with existing profiles using cosine similarity. The insertion rule is

$\alpha = 0.7$1

where $\alpha = 0.7$2 is cosine similarity and the default $\alpha = 0.7$3. If a new profile is too similar to an existing one, the system merges them rather than storing duplicates. At inference time, HomeLLaMA retrieves the top-3 most similar profiles, decodes them back into text, combines them with the current query and current home configuration, and generates a personalized plan:

$\alpha = 0.7$4

where $\alpha = 0.7$5 are decoded profile texts, $\alpha = 0.7$6 is the current home configuration, $\alpha = 0.7$7 is the query, and $\alpha = 0.7$8 is the local SLM planner.

This profile mechanism is central to HomeLLaMA’s personalization claim. The system is not merely adapting a generic model to a smart-home domain; it is updating a household-specific memory substrate that can influence future plan generation without requiring full retraining after each interaction. The paper’s description suggests a memory architecture optimized for locality, compactness, and retrieval quality rather than full conversational replay.

6. Evaluation, benchmark, deployment, and research context

HomeLLaMA is implemented locally on a server using PyTorch, LangChain, FAISS, QLoRA, and LLaMA3-8B. The reported training pipeline uses GPT-4-Turbo for augmentation, 14K action-command pairs, and 9 smart-home scenarios, with fine-tuning taking about 8 hours on a single RTX 4090. The system is presented as deployable on consumer-grade local hardware, with memory requirements in the reported experiments staying below roughly 16 GB depending on quantization (Huang et al., 10 Jul 2025).

Its principal benchmark is DevFinder, built from 100 test commands selected from the IFTTT dataset, with human-annotated device labels spanning common smart-home scenarios such as environmental control, atmosphere adjustment, power management, lighting, and security. The main metric is Device Relevance Score (DRS),

$\alpha = 0.7$9

where $$\mathcal{D}_a = \{s_i \rightarrow G(s_i, \boldsymbol{D})\}, \ \forall s_i\in \mathcal{S}_a .$$0 is the ground-truth device set and $$\mathcal{D}_a = \{s_i \rightarrow G(s_i, \boldsymbol{D})\}, \ \forall s_i\in \mathcal{S}_a .$$1 is the device set generated by the model. The paper compares HomeLLaMA against HomeGPT, Sasha, SAGE, TT-Gemma, and TT-Phi-2. It reports that HomeLLaMA substantially outperforms other local assistants, approaches the performance of GPT-3.5-level cloud systems on DRS, and still lags behind the strongest cloud LLMs, though by a much smaller margin than ordinary local SLMs.

The user study includes 100 survey participants and 10 onsite interview participants over 25 days. Survey ratings cover general plan satisfaction, user profile correctness, personalization fit, and privacy assurance. HomeLLaMA achieved high service satisfaction, close to cloud-based SAGE; it achieved the best personalization scores; and it received strong privacy ratings, approaching the fully local baseline and surpassing cloud systems. In the long-term onsite interview, participants interacted with HomeLLaMA over 50 conversation turns, and personalization ratings increased steadily over time, ease of use improved over time, and PrivShield activation decreased steadily and approached zero by the end.

Within the broader smart-home LLM literature, HomeLLaMA occupies a specific niche. Unlike DESAMO, which is an embedded, fully on-device smart home system built around an Audio LLM and targeted at elder-friendly voice intent classification and emergency event detection from ambient audio (Choi et al., 26 Aug 2025), HomeLLaMA centers on text-based smart-home serving, device relevance, and privacy-aware personalization. Unlike LLM interfaces for Home Energy Management Systems that focus on extracting well-formatted parameters for optimization (Michelon et al., 14 Jan 2025), HomeLLaMA emphasizes action-plan generation for general smart-home serving. It also differs from architectures that use preferences and retrieval-augmented prompting to choose a best next action in a simulated house state (Rey-Jouanchicot et al., 2024), because HomeLLaMA combines local SLM enhancement, user-profile memory, and explicit user-consented cloud escalation. A plausible implication is that HomeLLaMA represents a local-first branch of smart-home LLM research: one in which privacy, personalization, and graceful degradation are treated as first-class system properties rather than secondary deployment concerns.