ROSA: A Robotics Foundation Model Serving System for Robot Factories

Abstract: Robotics foundation models (RFMs) are making general-purpose robots increasingly practical for factory deployments. While RFM serving systems are central to this vision, existing systems are largely shaped by a single-robot, single-model assumption: inference is treated as an edge-computing problem handled by an on-robot or dedicated nearby GPU, and the serving objective is to minimize the latency of a single action model. In this paper, we propose ROSA, an RFM serving system for robot factories designed around three key principles. First, ROSA adopts shared GPU-pool serving, allowing a fleet of robots to access powerful server-class GPUs over the network in order to improve inference performance, battery duration, and GPU utilization. Second, ROSA provides a robotics-aware programming abstraction and system design that supports multi-model pipelines, per-task performance requirements, and failure handling. Third, ROSA uses factory-objective-driven scheduling to maximize SLO-qualified factory productivity rather than minimizing individual request latency. We implement ROSA on top of Ray Serve for distributed orchestration, with vLLM, PyTorch, and JAX as model-serving backends, and evaluate it on both real robots and synthetic large-scale workloads. The results show that ROSA improves factory productivity by up to 12.06x over conventional dedicated serving systems.

• The paper introduces ROSA, which centralizes robotics foundation model serving to enhance GPU utilization and throughput in heterogeneous robot fleets.
• It leverages a robotics-aware API and advanced scheduling algorithms to balance safety, performance, and economic priorities in factory environments.
• ROSA demonstrates up to 12.06× productivity gains and 99.96% SLO compliance through profiling-guided batching and optimized resource allocation.

ROSA: A Robotics Foundation Model Serving System for Robot Factories

Motivation and Problem Context

The growing sophistication of robotics foundation models (RFMs)—notably vision-language-action (VLA), planning, safety, and monitoring models—has catalyzed the practical deployment of general-purpose robots in industrial settings, such as manufacturing and warehousing. However, the traditional paradigm for model inference in robotics treats serving as a per-robot edge problem, tightly coupling each robot to a dedicated accelerator. This leads to suboptimal GPU utilization, limited scalability, and higher operational costs, especially when robot fleets are confronted with heterogeneous, multi-model inference pipelines and stringent per-task service-level objectives (SLOs).

The paper presents ROSA (Robotics Oriented Serving Architecture), a system designed to address these challenges by reformulating RFM serving as a centralized, factory-scale problem. ROSA rejects the conventional focus on minimizing latency for individual robotic actions, instead optimizing for factory-level productivity—action throughput weighted by task value—subject to diverse SLOs and resilience to failures.

Figure 1: Robots working on various tasks in a factory.

System Architecture and Design Principles

ROSA is constructed around three critical principles:

1. Shared GPU-Pool Serving Infrastructure: Instead of one-to-one robot-to-GPU mapping, ROSA consolidates compute in centralized GPU clusters, accessible via high-speed networking. This enables large-scale inter-robot batching, improved GPU utilization, and facilitates serving RFMs that exceed the capacity of traditional edge compute.
2. Robotics-Aware Programming Abstraction: The declarative configuration API captures heterogeneous task profiles, specifying the composition of model components (action, reasoning, safety, monitor), invocation frequencies, latency targets, and explicit fallback and retry behaviors in case of SLO or safety violations.
3. Factory-Objective-Driven Scheduling: The scheduler's objective is weighted factory action throughput, not individual latency minimization. It combines heuristics and integer linear programming (ILP) to maximize the number of SLO-compliant robot actions across the fleet, balancing throughput among tasks of varying economic priority and hardware requirements.

   Figure 2: ROSA system overview.

Multi-Component Model Pipelines

ROSA is engineered for multi-model inference pipelines, typical in modern robotics deployments:

• System 1 (Low-Level Action Model): Fast, reactive control for action generation based on robot observation and instruction.
• System 2 (Reasoning/Planning Model): Deliberative, lower-frequency planning for decomposing long-horizon tasks.
• Safety Models: Periodically invoked to semantically vet action plans and trajectories for human or machine safety constraints.
• Monitor Models: Assess real-time task progress, support fault detection, and trigger escalation or recovery policies.

  Figure 3: An example multi-model RFM inference pipeline.

The configuration abstraction enables explicit specification of differing frequencies and SLOs for each component, capturing the requirements for highly orchestrated, robust robotic operations.

Scheduling Algorithms

The core scheduling challenge is to partition servers, determine batching strategies, and route requests so as to maximize SLO-qualified action throughput for both homogeneous and heterogeneous fleets. Key features include:

• Obligation vs. Goal-Coupled Models: The scheduler dedicates minimal required resources to "obligation" models (safety/monitor), then uses the remaining for "goal-coupled" models (action/reasoning), maximizing controllable action rates.
• Profiling-Guided Batching: Empirical latency/throughput curves for each model/configuration guide optimal batch size selections, trading off delay and hardware utilization.
• Frontier Search for Heterogeneous Workloads: Rather than exponential grid search, a greedy adaptive strategy incrementally explores the action-rate space for fleets with mixed task requirements.

  Figure 4: Adaptive frequency search over heterogeneous tasks.

Experimental Evaluation

ROSA is evaluated using both real Franka Panda robot deployment and large-scale synthetic workloads, stressing the system with up to 64 virtual robots across 8 NVIDIA H200 GPUs.

Factory Productivity

ROSA achieves strong, robust improvements over dedicated and naively-shared baselines:

• Up to 12.06× productivity improvement over per-robot dedicated serving hardware, primarily due to batching and more flexible server allocation.
• 2.44× improvement over static shared-server architectures without intelligent scheduling.

  Figure 5: Single-task end-to-end performance measured as SLO-qualified action throughput.

  

  Figure 6: ROSA versus dedicated serving baselines.

• Baselines saturate much earlier due to lack of rate controls, batching, and optimal allocation, resulting in rapid SLO violation and collapse in qualified throughput as robot count or pipeline complexity increases.
• ROSA maintains high SLO-compliance (up to 99.96%) even under heavy load, where other approaches fail.

Heterogeneous Task Scheduling

For scenarios with mixed task pipelines, ROSA's scheduling further outpaces baselines due to superior cross-task resource allocation.

Figure 7: Heterogeneous-task end-to-end performance.

Ablation and Scheduling Insights

Experimental ablation underscores the importance of each scheduling component:

• Request-Rate Control: Uncapped clients frequently collapse under high load; rate-capping guided by scheduling extends SLO-compliant throughput substantially.
• Resource Allocation Across Models: Suboptimal static allocation leads to infeasibility or resource wastage as pipelines diversify.
• Batch Size Optimization: Profiling-guided decisions for System 1 batching yield 2.46× speedup over fixed, conservative batch sizing under representative load.

  Figure 8: System 1 latency and SLO qualification on P4.

  

Figure 9: Effect of request-rate control on P4.

Figure 10: Performance of different resource allocations on P4.

Figure 11: System 1 performance under different batch sizes.

Figure 12: System 1 batching decision: the optimal batch sizes in various configurations and their speedup over batch size one.

Real-Robot Validation

ROSA is demonstrated in real-world operation, orchestrating a Franka Panda arm through complex pick-and-place tasks with active progress monitoring and safety detection via VLM subcomponents.

Figure 13: Real-robot execution trace: the action model drives a Franka Panda arm to place tools into a bucket.

Figure 14: VLM judges for monitoring task completion (left) and detecting nearby humans for safe operation (right).

Practical and Theoretical Implications

ROSA's approach imposes a new systemic abstraction for embodied AI deployment in industrial robotics: shifting the serving bottleneck from individual inference latency to global resource coordination in a pooled architecture. This unlocks practical scalability, improved economic efficiency (via shared hardware), and extensibility toward future, rapidly-evolving RFM architectures. The formulation of serving as an SLO-qualified throughput maximization under constraints is general and can be directly ported to other real-time, multi-agent embodied AI domains.

Theoretically, ROSA demonstrates that judicious orchestration of model composition, batching, and request rate—grounded in accurate, profiling-driven scheduling—enables resource pools to serve large robot fleets with complex, heterogeneous task structures at far higher efficiency than previously believed achievable under the one-robot/one-model/one-GPU conventions.

Future directions include more dynamic online adaptation to unpredictable workloads, tighter coupling between robot local control stacks and server-side inference for emergent behaviors, and feedback-driven learning to improve fallback and failure handling in closed-loop operation.

Conclusion

ROSA augments robot-factory deployments with a model-agnostic, server-scale RFM serving substrate, underpinned by robotics-aware abstractions and factory-level scheduling. This enables substantial productivity gains for large, heterogeneous robot fleets while robustly satisfying safety and latency constraints. As embodied AI continues to advance, the design principles and architectural strategies instantiated by ROSA offer a generalizable foundation for next-generation RFM serving solutions in complex, multi-agent, real-world environments.