Hardware-Agnostic Policy Representation

• Hardware-Agnostic Policy Representation is a framework that abstracts control policies from specific hardware, enabling effective zero-shot transfer and cross-domain adaptability.
• Graph-based representations use modular design and message passing to mirror physical system topologies, supporting scalable policy instantiation on unseen platforms.
• Integrating hardware descriptors with sensorimotor embeddings allows rapid adaptation and robust performance in diverse settings, from robotics to dynamic scheduling.

A hardware-agnostic policy representation is a parametrization of control or decision policies that enables effective transfer, adaptation, or zero-shot generalization across a wide class of systems with varying hardware configurations. Such representations are critical for scalable robotics, multi-agent control, scheduling with heterogeneous resources, and cross-platform RL, where hardware-specific policies prohibit efficient reuse or adaptation.

1. Formal Foundations and Definitions

A hardware-agnostic policy seeks to decouple policy structure from specific hardware dependencies, so that a single policy class (possibly conditioned on compact hardware descriptors) can generalize to systems unseen in training. Canonical formalizations include:

• Input-Conditioned Universal Policies: Policies $\pi(a|s, v_h)$ that are conditioned on a hardware descriptor $v_h$ encoding kinematics, dynamics, or system properties (Chen et al., 2018).
• Structural Policies via Design/Policy Graphs: Representations where the physical system is encoded as a typed graph $\mathcal{G} = (\mathcal{V}, \mathcal{E})$, with policy architecture instantiated to mirror $\mathcal{G}$ and type-sharing across modules (Whitman et al., 2021).
• Condition-Based Policies for Scheduling: Policies $\pi_\theta(a|s)$ that choose actions based on discretized "condition bins" of system features, ensuring invariance to system size or structure (Lee, 2022).
• Sensorimotor Abstractions: Use of intermediate representations (e.g., optic flow) as action surrogates, permitting pretraining and policy transfer between morphologies (Wang et al., 17 Jul 2025).

The explicit goal is for a single policy representation $\pi$ or a parameter set $\theta$ to generalize over a set of MDPs $\{M^k\}$ or morphologies $\{\mathcal{G}^i\}$, achieving high performance with no or minimal retraining.

2. Graph-Based Policy Representations

For modular and reconfigurable robotic systems, the design is naturally captured as a labeled graph $\mathcal{G} = (\mathcal{V}, \mathcal{E})$, where nodes are modules (e.g., leg, wheel, body) and edges indicate physical connections (Whitman et al., 2021). The policy architecture is then constructed as a policy graph $\mathcal{G}_p = (\mathcal{V}_p, \mathcal{E}_p)$ sharing the same topology as $\mathcal{G}$ with the following key properties:

• Per-Module Shared-Parameter Policies: Each node $\nu \in \mathcal{V}_p$ hosts a local policy $\pi_\nu$, parameter-shared across module types:

$\pi_\nu(a_\nu | s_\nu; \theta_{\text{type}(\nu)})$

• Global Policy Factorization:

$$\pi(a|s, \mathcal{G}; \Theta) = \prod_{\nu \in \mathcal{V}_p} \pi_\nu(a_\nu | s_\nu; \theta_{\text{type}(\nu)})$$

• GNN-Based Message Passing: Coordination employs a Graph Neural Network, where message passing between modules follows the hardware connectivity:
  • Input embedding, message computation for each port, message aggregation, hidden state update iterated for a fixed number of message-passing steps.
  • Module outputs are computed via shared MLPs or LSTMs.

Component	Mathematical Form	Shared Across Modules?
Node policy	$\pi_\nu(a_\nu | s_\nu; \theta_t)$	Yes, per module type
Message passing encoder	$F_{\rm in}(o_\nu)$	Yes, per module type
Hidden state update	$F_{\rm up}(h,\ m_{\rm in})$	Yes, per module type

This structure enables instantiation of a policy for an unseen robot simply by supplying its design graph, yielding, in practice, strong zero-shot transfer to new hardware configurations (Whitman et al., 2021).

3. Hardware-Conditioned and Input-Augmented Policies

A complementary approach encodes hardware variability explicitly via a hardware descriptor $v_h$ incorporated into policy inputs (Chen et al., 2018):

• Explicit Kinematic Encoding (HCP-E): The kinematic chain is encoded as relative joint displacements and orientations; all policies receive this as fixed-length input, allowing zero-shot generalization across e.g., manipulators differing in DOF or link lengths.
• Implicit Embedding (HCP-I): Where dynamics are crucial (e.g., legged locomotion), a jointly-learned embedding $v_h = f_{\rm hw}(p_h)$ of system parameters allows the policy to adapt to variations in unobserved or complex dynamics.

Training proceeds over a pool of robots of varying types, with RL losses applied to policy and critic networks that always concatenate $v_h$ to the state $s_t$. Zero-shot generalization and rapid fine-tuning are observed for both real and simulated robots with previously unseen morphologies (Chen et al., 2018).

4. Policy Representations for System-Agnostic Scheduling

In domains such as dynamic scheduling, hardware-agnosticism reduces to system-agnosticism: learning a single "scheduling principle" $\theta$ that operates invariant to the number and type of resources (Lee, 2022). Critical features include:

• Descriptive Policy Representation:
  • State is mapped to a "descriptive state" $\bar{s}$ indexed by bins of item features.
  • Actions correspond to selecting condition cells $h$ (multi-index over feature partitions), plus auxiliary parameters $m$.
  • Parameters are shared over all bins, independent of system size $N$:

  $$\pi_\theta(a|s) = \operatorname{softmax}_{h,m} \{ f_\theta(x_{h,m}) \}$$

  where $x_{h,m}$ encodes the bin and auxiliary action.

• Meta-Optimization Across Heterogeneous Systems: $\theta$ is optimized over a meta-objective aggregating returns across a corpus of source MDPs, corresponding to different hardware/resource settings.

Zero-shot transfer and minimal fine-tuning are achieved when deploying to unseen numbers (or types) of resources, with empirical reward degradation $\leq$3% relative to tailored policies (Lee, 2022). This approach extends to hardware scheduling (e.g., CPUs/GPUs) by binning hardware features and sharing policy parameters across all configurations.

5. Embodiment-Agnostic Sensorimotor Policy Representations

Recent advances utilize embodiment-agnostic action/proprioceptive representations to learn shared world models and policies from heterogeneous datasets:

• Optic Flow as Action Surrogate: Optical flow between image frames is used as a universal "action" representation that is independent of underlying embodiment, enabling the pretraining of a single world model on data from varied robots or human hands (Wang et al., 17 Jul 2025).

• World Model Pretraining and Latent Policy Steering (LPS):

  • Pretrain an RSSM on multi-embodiment data using optic flow as action input.
  • Fine-tune on limited target robot demonstrations using true actions.
  • At inference, perform latent policy steering by sampling candidate action sequences from a base policy, unrolling them in the world model, and selecting trajectories maximizing a learned value head.
• Empirical Transfer: Pretraining on human play or multi-robot datasets enables rapid adaptation to new robots; e.g., 50 Franka demonstrations with Open-X or play pretraining yield 78–80% task success versus 63% for standard behavior cloning (Wang et al., 17 Jul 2025).

The core technical advance is the formal decoupling of "actions" from embodiment, achieved by leveraging visual or flow-based representations robust to kinematic and morphological changes.

6. Training Paradigms and Practical Outcomes

Across approaches, key training methodologies include:

• Model-Based RL with Graph/Dynamics Priors: Alternating phases of model fitting, trajectory optimization (e.g., constrained DDP, MPC), and behavioral cloning onto a shared policy (Whitman et al., 2021).
• Multi-Task/Meta-RL Across Hardware/MDPs: Policies are trained via RL or supervised learning jointly across a diverse suite of systems, either with meta-objectives (Lee, 2022) or multi-robot rollouts (Chen et al., 2018).
• Fine-Tuning and Zero-Shot Evaluation: Policies conditioned on hardware or built with GNN/topology priors demonstrate substantial zero-shot transfer; fine-tuning with sparse data achieves rapid adaptation, often $3\times$–$5\times$ faster than from scratch training (Chen et al., 2018, Wang et al., 17 Jul 2025).

Transfer Setting	Zero-Shot Success	Few-Shot Speedup	Representative Work
New kinematic type	$75$–$93\%$	$3\times$	(Chen et al., 2018, Whitman et al., 2021)
New scheduling system	$0.95$–$0.98\times$ opt	Near-optimal in 50 steps	(Lee, 2022)
New robot embodiment	+20–50% rel. w/ pretraining	–	(Wang et al., 17 Jul 2025)

7. Limitations and Open Questions

• Extreme Morphological Shifts: For radical hardware changes (wheeled vs legged, drastically different morphologies), fixed-dimensional embeddings or simple GNNs may be insufficient, suggesting a need for hierarchical or more expressive structural representations (Chen et al., 2018).
• Sensorimotor Gaps: Embodiment-agnostic sensorimotor representations (e.g., optic flow) often assume static, fixed-view cameras; egocentric or moving-camera scenarios require new abstractions (Wang et al., 17 Jul 2025).
• Dataset and Training Diversity: Generalization is contingent on diversity and coverage in training (module types, system variations, morphology). Insufficient coverage leads to performance drops in zero-shot settings (Whitman et al., 2021).
• Interpretability of Encodings: Implicit embeddings $v_h$ may lack physical interpretability, requiring new methods for validation and debugging of policy adaptation (Chen et al., 2018).

A plausible implication is that future research must address the seamless combination of explicit structural priors, learned hardware abstractions, and universal sensorimotor representations to achieve scalable, reliable hardware-agnostic policy transfer.

• (Whitman et al., 2021): "Learning Modular Robot Control Policies"
• (Chen et al., 2018): "Hardware Conditioned Policies for Multi-Robot Transfer Learning"
• (Lee, 2022): "System-Agnostic Meta-Learning for MDP-based Dynamic Scheduling via Descriptive Policy"
• (Wang et al., 17 Jul 2025): "Latent Policy Steering with Embodiment-Agnostic Pretrained World Models"