Kitchen-R: A Benchmark for Kitchen Robotics
- Kitchen-R is a simulator-based benchmark for digital-twin kitchens that integrates language-guided high-level planning with low-level robotic control.
- It employs the Mobile ALOHA robot in NVIDIA Isaac Sim to execute over 500 complex language instructions, evaluating both planning and physical execution.
- The benchmark also supports research in realistic kitchen perception, symbolic recipe reasoning, and integrated evaluation methods for multi-step manipulation tasks.
Kitchen-R most specifically denotes a simulator-based benchmark for language-guided mobile manipulation in a digital-twin kitchen, designed to evaluate high-level task planning and low-level physical control within a single framework. In its explicit benchmark form, it is built in NVIDIA Isaac Sim, uses a Mobile ALOHA robot, provides more than 500 complex language instructions, baseline methods for both planning and control, and a trajectory collection system (Kachaev et al., 21 Aug 2025). Related work also uses the label more loosely for kitchen-centered robotics and reasoning problems, including egocentric perception, recipe formalization, robot-centric kitchen design, multimodal activity recognition, and dynamic scene understanding in realistic kitchens (Younes et al., 2024).
1. Scope, nomenclature, and adjacent uses
Within embodied AI, Kitchen-R names a benchmark that addresses a stated gap between language instruction-following benchmarks that assume perfect execution and low-level control benchmarks that reduce language to simple one-step commands (Kachaev et al., 21 Aug 2025). In adjacent kitchen-robotics literature, the same label is used informally to denote kitchen-centered reasoning and robotics systems, especially when authors discuss perception benchmarks, recipe representations, or assistive kitchen AI that must combine semantics, sensing, and action (Kumbhakern et al., 4 Sep 2025).
This broader usage is heterogeneous rather than strictly standardized. Real-world kitchen perception work presents dataset and benchmark components that are explicitly described as useful for “Kitchen-R” scenarios, especially where a robot must estimate 6D poses from an egocentric viewpoint in cluttered appliances, drawers, sinks, or shelves (Younes et al., 2024). Recipe-centric work uses the label to motivate internal symbolic representations for planning and execution of cooking procedures, including directed action graphs and temporally structured ontologies (Kumbhakern et al., 4 Sep 2025). Wearable and multimodal activity-recognition systems likewise situate kitchen activity understanding as a foundation for behavior modeling, assistance, and health monitoring (Liu et al., 2022).
A distinct naming collision also exists. In quantum machine learning, “Kitchen-R” can refer to Rahimi and Recht’s random kitchen sinks framework, and the adiabatic quantum variant extends that interpretation rather than any kitchen-robotics meaning (Noori et al., 2019). In the robotics and cooking literature, however, Kitchen-R refers to kitchen environments, kitchen reasoning, or kitchen robotics.
| Layer | Representative works | Function |
|---|---|---|
| Joint benchmark | Kitchen-R (Kachaev et al., 21 Aug 2025) | unified planning-control evaluation |
| Real-world perception | KITchen (Younes et al., 2024), multiview RGB-D kitchens (Georgakis et al., 2016), relocation tracking (Esfahani et al., 3 Mar 2025) | pose, detection, scene change |
| Symbolic reasoning | action-graph ontology (Kumbhakern et al., 4 Sep 2025), semantic parsing (Ventirozos et al., 2023), quantity prediction (Choi et al., 2023), abort planning (Has et al., 2024) | recipe structure and safe replanning |
| Integrated robotic kitchens | YORI (Noh et al., 2024), batter stirring and pouring (Luo et al., 2024), computational kitchen design (Zhi et al., 2023) | manipulation, scheduling, kitchen layout |
| Human activity understanding | Smart-Badge (Liu et al., 2022), wearable edge system (Liu et al., 2024), multimodal recipe inference (Ishat, 21 Aug 2025) | activity and procedure recognition |
2. Kitchen-R as a joint benchmark for planning and control
The benchmark formulation in Kitchen-R centers on language-guided mobile manipulation with a Mobile ALOHA robot in a digital twin of a real kitchen (Kachaev et al., 21 Aug 2025). It provides 563 natural-language instructions derived from 58 templates, with plans ranging from 4 to 8 steps and built from three primitive actions: move to, pick, and place. The planning input consists of a complex instruction plus a top-down scene view with colored placement or taking zones. The control side operates in continuous space: the low-level policy predicts 16 future steps of a 10-dimensional action vector
where base velocities, end-effector translation, end-effector quaternion, and gripper opening are all explicitly represented (Kachaev et al., 21 Aug 2025).
The benchmark defines three evaluation modes: independent planning assessment, independent control assessment, and integrated end-to-end evaluation. Planning quality is measured by an Exact Match metric over textual plan steps. Low-level control is evaluated offline with mean squared error against expert trajectories. The benchmark also defines composite metrics that combine plan correctness and control error, and an online metric that combines plan exactness with subtask success rate, where navigation succeeds when the robot base center is within 10 cm of the target and manipulation succeeds when the object’s geometric center is within 5 cm of the target (Kachaev et al., 21 Aug 2025).
Baseline planning uses OmniFusion with in-context examples and constrained decoding. The reported Exact Match scores show the importance of this prompt structure: Vanilla OmniFusion yields 0.000, adding valid instructions in context remains 0.000, adding in-context examples of plans raises performance to 0.612, and adding constrained generation yields 0.632 (Kachaev et al., 21 Aug 2025). The low-level baseline is a diffusion-policy controller conditioned on two onboard RGB views and robot state, with cross-attention replacing FiLM-style conditioning. The benchmark also ships a trajectory collection system and reports approximately 2,700 mobile-manipulation trajectories, making Kitchen-R both an evaluation protocol and a data-generation framework (Kachaev et al., 21 Aug 2025).
A common misconception is that Kitchen-R is only a language-planning benchmark. Its defining claim is the opposite: it is intended to bridge the disconnection between symbolic task planning and realistic execution by evaluating both modules independently and jointly within the same kitchen environment (Kachaev et al., 21 Aug 2025).
3. Perception in realistic kitchen environments
Kitchen-R-style perception research is motivated by the mismatch between tabletop benchmarks and actual kitchen manipulation. The KITchen benchmark explicitly targets that gap with around 205k real-world RGBD images, 111 kitchen objects, two distinct real kitchens, and a humanoid ARMAR-6 egocentric viewpoint (Younes et al., 2024). The scenes are densely cluttered, with 10–50 annotated objects per image, and evaluation follows BOP metrics—Visible Surface Discrepancy, Maximum Symmetry-Aware Surface Distance, and Maximum Symmetry-Aware Projection Distance—under constraints intended for real-time robotics: a single model must handle all objects, and submitted methods must run at ≥ 5 fps (Younes et al., 2024). The benchmark’s emphasis on off-center objects, higher shelves, sinks, dishwashers, fridges, and transparent or reflective backgrounds is a direct rejection of the fixed-camera tabletop assumption.
Earlier multiview RGB-D kitchen data established related difficulties from a different angle. A multiview RGB-D dataset of nine kitchen scenes provides hundreds of frames per scene, dense 3D reconstructions with millions of points, and instance-level 3D point annotations that back-project to 2D bounding boxes (Georgakis et al., 2016). Its multiview 3D proposal generator reaches 85.8% average recall at IoU 0.5, or 90.8% when excluding the transparent coke bottle that fails under Kinect v2 depth sensing, and a CNN Scene Folds baseline reports 41.7 mAP. The same work shows that kitchen scenes are substantially more difficult than WRGB-D, largely because of clutter, multiple support surfaces, and occlusion (Georgakis et al., 2016).
Dynamic kitchens add another layer: relocation tracking. A YOLOv5-based system trained on more than 9,000 AI2-THOR kitchen images defines a visibility score
$2(1-D) + 10WH + (1-C) + F$
over normalized depth, bounding-box size, centrality, and detector confidence, then detects relocation by comparing the best-associated frame before and after scene change (Esfahani et al., 3 Mar 2025). On 614 objects across 10 scenes, it reports 97.72% accuracy, 95.83% precision, and 96.84% recall. This makes Kitchen-R-style perception not only a matter of static localization, but also of detecting when objects have moved in a cluttered household setting (Esfahani et al., 3 Mar 2025).
4. Symbolic recipe representation and task reasoning
A second major interpretation of Kitchen-R concerns explicit recipe reasoning. One line of work introduces a Recipe Action-Graph DSL in which recipes are represented as directed acyclic graphs
with typed action nodes—Process, Transfer, and Plate—and typed entities including Ingredients, Environments, and PPCs (Partially Processed Components) (Kumbhakern et al., 4 Sep 2025). The formalism models material flow, temporal precedence, concurrency, environment persistence, and compositional reuse. Process nodes carry parameters such as technique, tool, temperature, time, termination conditions, and modifiers; Transfer nodes change environment; Plate performs final assembly. Manual encoding of a full English breakfast is used to demonstrate that the DSL can represent pan reuse, interjected steps, and concurrent branches (Kumbhakern et al., 4 Sep 2025).
Another line addresses semantic parsing of recipe instructions for IoT programming. Here a device-related instruction is mapped into a tuple
where Where denotes the device, What a property, Why the purpose or role, and How the value or duration (Ventirozos et al., 2023). An annotated corpus derived from RecipeQA supports CRF and neural sequence-labeling baselines. Results show that parsing is feasible but that many natural-language instructions are incomplete: device references, properties, and precise values are often implicit. This incompleteness is central for Kitchen-R, because recipe text is not yet an executable program unless missing information is inferred or requested (Ventirozos et al., 2023).
KitchenScale extends recipe reasoning to ingredient numeracy. It predicts measurement type, unit, and normalized quantity from recipe context using fine-tuned BERT models and a Discrete Latent Exponent method that decomposes quantity into exponent and mantissa to handle high-variance scales (Choi et al., 2023). The constructed dataset contains 98,725 instances and 14 SI-consistent units. Best results include 0.9233 accuracy for measurement type, 0.7335 for unit classification, and, for quantity regression, 0.7365 exponent accuracy with 0.3333 log mean absolute error (Choi et al., 2023). In Kitchen-R terms, this gives a numeracy module for recipe completion, scaling, or recommendation.
Task reasoning also includes interruption and recovery. Planning-based abort handling for household kitchen robots uses PDDL plus DBpedia-derived semantic typing to infer fallback safe states after cancellation (Has et al., 2024). Derived predicates such as safe-perishable and safe-utensil let the system compute cleanup plans rather than merely stopping. Reported scenarios include putting down a held bowl and, in a cutting-onion task, placing the knife on the counter and moving the onion to the fridge (Has et al., 2024). This shows that Kitchen-R reasoning extends beyond nominal task execution to contingency-aware kitchen behavior.
5. Manipulation systems, kitchen infrastructure, and digital environments
At the system level, YORI presents a fully integrated autonomous cooking cell built around a modular robotic kitchen and a dual-arm 11-DoF manipulator with quasi-direct-drive proprioceptive actuators (Noh et al., 2024). The cell includes a storage shelf, rotating mixer, convection oven, salamander broiler, deep fryer, pasta cooker, spice dispenser, and induction cooktop. Layout is optimized by minimizing the distance from key operational points to the manipulator workspace subject to reachability and oriented bounding-box non-overlap constraints. Recipe execution is scheduled as a Job-Shop Scheduling Problem solved with Google OR-Tools CP-SAT, with precedence, no-overlap, concurrency, and deadline constraints. YORI demonstrates repeated public preparation of steak frites and explicitly argues for robot-centric rather than human-centric kitchen design (Noh et al., 2024).
Fine-grained liquid handling exposes another dimension of Kitchen-R. A UR5e-based system for pancake batter stirring and precise pouring combines a WSG-50 gripper, a 6-axis F/T sensor sampled at 1000 Hz, and an overhead RGB camera to estimate bowl geometry, batter uniformity, liquid level, and water–flour ratio, then pour arbitrary shapes (Luo et al., 2024). Liquid level is inferred from the intersection of torque–depth fits for batter and air regimes; water–flour ratio is estimated by matching torque–immersion curves. Reported errors are 1.37 mm and 2.33 mm for liquid level in small and large bowls, 0.033 and 0.048 for water–flour ratio, 1.95 mm mean absolute line-width error for shaped pouring, and 5.25 cm² area error for round pancakes (Luo et al., 2024). These results show that Kitchen-R manipulation includes haptic perception and controlled interaction with viscous, non-Newtonian food materials.
Kitchen layout itself can be optimized for collaboration. Computational kitchen design formulates a joint objective
that combines layout quality and path quality for a human–robot team, evaluated through decentralized multi-agent motion planning and simulated annealing (Zhi et al., 2023). The method optimizes counter positions and orientations under kitchen design rules, and reports noticeable performance improvement for human–robot collaboration. In parallel, digital kitchen remodeling from a single panorama contributes a different form of kitchen digital twin: paired indoor/outdoor HDR capture, automatic kitchen layout generation with new components, and an editable global-illumination rendering pipeline, together with a Pano-Pano HDR dataset of 141 paired panoramas (Ji et al., 5 Feb 2025). Although this work is not a manipulation benchmark, it is directly relevant to Kitchen-R as a representation and environment-construction technology.
6. Multimodal activity understanding and wearable kitchen intelligence
Kitchen-R also includes systems that infer kitchen activity from multimodal sensory streams rather than robot manipulation alone. Smart-Badge is a wearable badge with six sensor modalities—an infrared array, IMU, optical spectrum sensor, gas sensor, barometric pressure sensor, and time-of-flight sensor—for kitchen activity recognition in a realistic unmodified kitchen (Liu et al., 2022). Using a multi-channel CNN with data and feature fusion, it classifies 14 activities performed by 10 volunteers with 92.44% average accuracy and an F1 score of 88.27%. The activity set includes posture transitions, walking, appliance interaction, washing hands, cutting food, and beverage-related actions, making it directly relevant to behavior modeling and diet-related monitoring (Liu et al., 2022).
A later wearable edge-computing system pushes this line fully onto microcontrollers (Liu et al., 2024). It uses the same six sensing modalities, two microcontrollers, and end-to-end local inference for 15 activities including a null class. The deployed compact model is 184.5 kbytes, reaches 87.83% average accuracy, and runs in 25.26 ms on the microcontroller; comparative measurements show the power–latency trade-off across nRF52840, MIMXRT1062, STM32L4S5, and STM32F767 boards (Liu et al., 2024). This gives Kitchen-R a practical, privacy-preserving, on-device activity-recognition profile.
Vision-and-language kitchen understanding appears in a third form. A multimodal pipeline combines YOLOv8 segmentation, MediaPipe keypoints with an LSTM action recognizer, Whisper-base automatic speech recognition, and TinyLLaMA to infer recipes and generate step-by-step guides from cooking videos (Ishat, 21 Aug 2025). The custom object dataset expands to more than 17,000 images across 16 kitchen-related classes; the action dataset contains 1,000 three-second clips for eight hand-action classes. Reported headline results include 71% mAP50 in object segmentation and 87.5% accuracy with 86.2% macro F1 for action recognition (Ishat, 21 Aug 2025). This work is not a benchmark, but it exemplifies Kitchen-R as a multimodal bridge from low-level observations to high-level recipe semantics.
7. Limitations and research directions
Across these lines of work, several recurrent limitations define the current frontier of Kitchen-R. The benchmark literature identifies the persistent disconnect between symbolic language planning and realistic physical execution, and Kitchen-R itself was proposed to bridge that disconnect rather than assume it away (Kachaev et al., 21 Aug 2025). Real-world perception datasets emphasize that tabletop assumptions fail in kitchens with transparent shelves, reflective sinks, dishwashers, drawers, higher shelves, and severe clutter (Younes et al., 2024). Dynamic-scene methods still miss small relocations when viewpoint-index changes fall below threshold, even when gross relocation detection is accurate (Esfahani et al., 3 Mar 2025).
Symbolic recipe work remains incomplete in different ways. Recipe instructions are often underspecified for device programming, so semantic parsing alone does not fully solve execution (Ventirozos et al., 2023). Action-graph ontologies still rely on manual encoding in their current form, and transfer semantics for residues, partial quantities, and concurrent subrecipes remain open questions (Kumbhakern et al., 4 Sep 2025). Abort handling with PDDL demonstrates graceful cleanup, but the reported formulation lacks capacity constraints and a notion of best storage location, so “safe” does not yet mean fully optimized household reasoning (Has et al., 2024).
Integrated robotic kitchens also remain narrow in scope. YORI demonstrates multi-dish cooking, but its menu coverage is limited and perception still relies on markers and structured appliances (Noh et al., 2024). The batter-stirring system attains precise control over one class of viscous liquid, but sharp corners in poured shapes and transfer to other containers or rheologies remain difficult (Luo et al., 2024). Wearable activity-recognition systems achieve strong results, yet battery life, larger activity vocabularies, and generalization across kitchens and users remain unresolved (Liu et al., 2024).
Taken together, these works indicate that Kitchen-R is best understood not as a single completed system but as a layered research program. Its explicit benchmark formulation supplies joint evaluation of planning and control; its surrounding ecosystem supplies realistic perception, symbolic recipe representations, robot-centric kitchen environments, manipulation of deformable food media, and multimodal activity understanding. The open challenge is their integration into a single kitchen intelligence stack that remains realistic, data-efficient, safe, and executable.