RobotValues: Evaluating Household Robots When Human Values Conflict

Abstract: While household robots are often evaluated based on task completion, everyday domestic environments involve value-conflicting situations in which robots are expected to choose actions that prioritize other values than task success, such as human autonomy, efficiency, or social appropriateness. Yet, there are no benchmarks for evaluating robots' value preferences in such scenarios. We introduce RobotValues, a benchmark to evaluate household robot planners in 10K value-conflict scenarios. Each instance consists of a realistic household image with multiple plausible robot actions that prioritize different human values. We construct RobotValues through LLM-assisted scenario generation, stakeholder-grounded value extraction, image generation and automatic quality control. Using RobotValues we evaluate VLMs used in robotics and find that models exhibit default value preferences, including safety and accommodation, while underselecting privacy-prioritizing actions. When the models are instructed to prioritize specific values that conflict with their own preferences, they often fail to override their default actions, choosing incorrect actions for 80% of the time. These findings suggest that household robot evaluation should measure not only task completion or safety compliance, but also whether robots can choose among plausible actions when human values conflict.

• The paper introduces RobotValues, a large-scale benchmark to assess how household robots resolve conflicting human values.
• It employs synthesized high-quality household scenarios with image and context inputs enhanced by stakeholder-grounded value annotations.
• Evaluation reveals that current VLMs favor safety over privacy, highlighting challenges in overcoming default value biases.

Evaluating Household Robots in Value-Conflict Scenarios with RobotValues

Introduction and Motivation

Contemporary household robot evaluation typically focuses on metrics such as task success, safety, and social scene understanding. However, these criteria fail to capture the decision complexity encountered in domestic environments, where household robots frequently face situations requiring judgment across conflicting human values: autonomy, privacy, safety, social appropriateness, and more. The absence of dedicated benchmarks for evaluating robot planners in value-conflict contexts has limited progress toward genuinely value-sensitive and trustworthy robot behavior. The "RobotValues: Evaluating Household Robots When Human Values Conflict" paper (2606.03312) addresses this deficit by introducing RobotValues—a large-scale, multimodal benchmark specifically designed to probe the ability of robot decision systems to navigate value-laden, ambiguous household scenarios.

Dataset Construction and Methodology

RobotValues comprises 10,073 high-quality, image-grounded household scenarios, each representing a decision point where a robot must select from multiple plausible actions that promote divergent human values.

Figure 1: Diverse household images from RobotValues. Each image depicts a realistic household decision point in which a robot must choose between candidate actions that prioritize different human values.

Scenario diversity is achieved by leveraging persona seeds and context conditioning: persona attributes (age, household composition, country, etc.) are drawn from the World Values Survey Wave 7 (WVS7), enhancing demographic diversity and ecological validity. Context seeds (room type, time of day) further diversify generated settings. Scenarios are crafted by LLMs, followed by action generation, stakeholder reaction synthesis, and stakeholder-grounded value extraction. Notably, rather than mapping actions to abstract value taxonomies based solely on text, RobotValues annotates candidate actions through first-person stakeholder reactions, ensuring that value labels reflect authentic perspectives grounded in concrete decision contexts.

Figure 2: Data generation pipeline of RobotValues.

High-fidelity, photorealistic images are produced for each scenario using generative models (GPT Image 2), with strict filtering at every stage. Images are egocentric, exclude visible robot embodiment, and undergo programmatic and human-in-the-loop quality control to enforce physical realism, scenario relevance, plausible viewpoints, and artifact-free rendering.

Examples of both accepted and filtered images illustrate the qualitative spectrum and the effectiveness of the filtering process:

Figure 3: Example image of RobotValues (high-quality, accepted sample).

Figure 4: Example image of RobotValues (high-quality, accepted sample).

Figure 5: Example image rejected during image-quality filtering (e.g., physical implausibility or artifact).

Figure 6: Example image rejected during image-quality filtering (e.g., failed rendering of human anatomy).

Fine-grained value annotations extracted from stakeholder reactions are subsequently mapped to established HRI and psychological taxonomies (e.g., Schwartz’s basic human values) to facilitate structured analysis.

Evaluation Protocol and Experimental Framework

The benchmark formulates the evaluation as an action-selection task for VLM-based robot planners. The model receives a scenario image, a textual context omitting explicit visible-state details, and multiple annotated candidate actions. Evaluation is performed under two regimes: (1) Default-choice (unconditioned), where the model selects the action it considers most appropriate; (2) Value-conditioned, where the model is provided a target value and must intentionally choose the action prioritizing that value—even when at odds with its intrinsic or default tendencies.

Metrics include Bradley-Terry (BT) scores for quantifying default value preferences across value categories (via pairwise comparisons) and value-conditioned accuracy for measuring the model’s ability to follow explicit value instructions, especially under norm-value mismatches.

Results and Empirical Analysis

Analysis of default preferences reveals a consistent pattern across state-of-the-art robotics-oriented VLMs: actions prioritizing Safety and Accommodation exhibit the highest selection rates, while those promoting Privacy and Security (protection of sensitive information) are robustly underselected.

Strong numerical findings:

• In value-conditioned settings, models exhibit up to a 40.9 percentage point drop in accuracy when asked to prioritize a value that conflicts with their default preference.
• Even with explicit value-targeting prompts, models select the value-aligned action only 6.9–16.8% of the time in these scenarios (compared to 40.2–51.3% when the target matches the default).

This demonstrates a pronounced default-value lock-in: VLMs struggle to override implicit learned preferences through instruction alone.

Additionally, while VLMs demonstrate relatively higher accuracy (~44–60%) in identifying which value an action is aligned to (action-value matching), they fail to use this recognition effectively for value-conditioned action selection under conflict.

Ablation studies discarding either image or context input confirm that the default bias toward Safety and against Privacy persists regardless of input modality, although the exact BT scores and secondary value orderings exhibit some dependency on context granularity.

Implications and Future Directions

The empirical results expose critical deficits in the value-sensitivity and steerability of current VLM-based robot planners. The persistent underweighting of privacy, even in situations where stakeholder perspectives clearly elevate privacy, is particularly alarming for deployment in real-world, relationship-intensive household environments. These findings align with previous human studies indicating high user sensitivity to privacy in domestic HRI [10.3389/frobt.(2021.62795)8], [10.1145/361(0977.36349)46].

The release of RobotValues thus enables the systematic diagnosis of value biases in VLMs and provides a foundation for research into:

• Pluralistic alignment: Developing models that can represent, learn, and flexibly prioritize diverse stakeholder values as first-class decision inputs.
• Robust value conditioning: Training and adapting VLMs to not only recognize, but to act on explicit value instructions, even in the presence of strong learned priors.
• Grounded policy learning: Integrating stakeholder-grounded annotation pipelines into downstream policy adaptation to ensure alignment with situated values in deployment environments.

The design methodology, combining synthetic yet high-diversity scenario creation with stringent quality control and grounded annotation, can be extended to other value-sensitive embodied AI domains (eldercare, public spaces, etc.).

Conclusion

RobotValues (2606.03312) establishes a comprehensive, high-scale benchmark targeting the essential but previously unmeasured capability of VLM-based household robots: value-sensitive action selection under conflict and ambiguity. Experimental results demonstrate that current VLMs not only have strong unintended default value alignments but also are unable to consistently follow explicit value-instruction when it conflicts with those defaults. The benchmark's release will facilitate research in robust, user-controllable value alignment for embodied AI and highlights the need for evaluation metrics that center human values rather than only safety or task proficiency.