Friction-Aware Reinforcement Learning

• Friction-aware reinforcement learning is a framework that explicitly integrates friction phenomena (e.g., static, sliding, rolling) into the decision-making process.
• It employs physics-inspired reward shaping, high-fidelity simulation, and online friction estimation to tailor policies for tasks like drag reduction, navigation, and in-hand manipulation.
• Empirical results across domains—such as up to 37% drag reduction and near 100% success in ballbot control—demonstrate improved robustness and transferability when friction is modeled as a first-class component.

to=arxiv_search.search 微信的天天中彩票്സոն code _string="all:(\"friction-aware reinforcement learning\" OR \"skin friction drag\" OR \"static friction\" OR \"sliding manipulation\" OR \"terrain-aware dynamics\" )", "max_results": 10} to=arxiv_search.search 大发快三计划 code 天天中彩票腾讯json='{"query":"all:(\"friction-aware reinforcement learning\" OR \"skin friction drag\" OR \"static friction\" OR \"sliding manipulation\" OR \"terrain-aware dynamics\")","max_results":10}' to=arxiv_search.search code ажәларjson='{"query": "all:(\"friction-aware reinforcement learning\" OR \"skin friction drag\" OR \"static friction\" OR \"sliding manipulation\" OR \"terrain-aware dynamics\")", "max_results": 10}' to=arxiv_search.search code արկjson={"query":"all:(\"friction-aware reinforcement learning\" OR \"skin friction drag\" OR \"static friction\" OR \"sliding manipulation\" OR \"terrain-aware dynamics\")","max_results":10} to=arxiv_search.search 天天中彩票如何json {"query":"(Sonoda et al., 2022) OR (Wan et al., 27 Apr 2026) OR (Guttikonda et al., 2023) OR (Riley et al., 15 Jan 2025) OR (Song et al., 2022) OR (Raei et al., 24 Feb 2025) OR (Hu et al., 3 Mar 2025) OR (Peng et al., 2024) OR (Duclusaud et al., 2024) OR (Ozmen et al., 16 Apr 2025)","max_results":10} Friction-aware reinforcement learning denotes a family of reinforcement learning formulations in which friction, contact dissipation, or execution frictions are modeled as central elements of the control or decision problem rather than treated as unstructured disturbance. In recent arXiv work, this perspective appears in turbulent drag reduction through wall blowing and suction (Sonoda et al., 2022), underactuated ballbot control dominated by coupled rolling, lateral, and torsional friction (Wan et al., 27 Apr 2026), terrain-aware navigation over spatially varying friction maps (Guttikonda et al., 2023), adaptive time integration for Coulomb-friction fault dynamics (Riley et al., 15 Jan 2025), robotic rearrangement and sliding manipulation under variable friction (Song et al., 2022, Raei et al., 24 Feb 2025), friction-conditioned locomotion and sim-to-real transfer (Hu et al., 3 Mar 2025, Peng et al., 2024), tactile contact-rich dexterous manipulation (Kim et al., 22 Sep 2025), and option hedging under transaction-cost frictions (Hu et al., 1 Feb 2026). Taken together, these works suggest that friction-aware RL is best understood as a modeling stance: the policy, simulator, reward, or evaluation metric is explicitly structured around frictional mechanisms that materially govern behavior.

1. Scope and defining characteristics

Across the literature, friction-aware RL spans several distinct physical and operational meanings of “friction.” In fluid mechanics, the objective is the reduction of skin friction drag in a fully developed turbulent channel flow (Sonoda et al., 2022). In mobile robotics and manipulation, friction appears as rolling, lateral, torsional, static, dynamic, and torsional friction, or as contact pressure acting as a proxy for frictional coupling (Wan et al., 27 Apr 2026, Raei et al., 24 Feb 2025, Kim et al., 22 Sep 2025). In navigation, friction is a map-dependent field that alters dissipation and path cost (Guttikonda et al., 2023). In numerical mechanics, it is a set-valued Coulomb law that generates stick-slip transitions and solver failures (Riley et al., 15 Jan 2025). In derivatives hedging, friction is realized as proportional transaction costs and spread effects (Hu et al., 1 Feb 2026).

These differences matter because they prevent a narrow definition tied to any single algorithm. Some papers place friction directly in the environment dynamics, some in the observation space, some in reward shaping, and some in the simulator or model class used for training. A plausible implication is that friction-aware RL is less a subfield with a fixed benchmark suite than a recurrent design pattern for problems in which dissipation, breakaway thresholds, and contact discontinuities dominate policy quality.

Domain	Friction-aware element	Representative paper
Turbulent flow control	Skin-friction drag, wall blowing/suction	(Sonoda et al., 2022)
Ballbot control	Rolling, lateral, torsional friction; roller mechanics	(Wan et al., 27 Apr 2026)
Navigation and planning	Spatially varying terrain friction map	(Guttikonda et al., 2023)
Nonsmooth mechanics	Coulomb friction and stick-slip	(Riley et al., 15 Jan 2025)
Manipulation and locomotion	Static/dynamic/torsional/contact friction	(Song et al., 2022, Raei et al., 24 Feb 2025, Kim et al., 22 Sep 2025, Hu et al., 3 Mar 2025, Peng et al., 2024)
Finance	Transaction-cost frictions	(Hu et al., 1 Feb 2026)

A common misconception is that friction-aware RL is merely domain randomization over a coefficient of friction. The surveyed papers show a broader picture: friction may be inferred from vision, estimated online from motion, encoded in tactile reward shaping, built into a probabilistic dynamics model, or represented through solver-level contact mechanics (Guttikonda et al., 2023, Raei et al., 24 Feb 2025, Peng et al., 2024, Kim et al., 22 Sep 2025).

2. Formalization in state, action, reward, and transition models

A defining property of friction-aware RL is that friction changes the formal RL problem rather than only the physical interpretation of the task. In channel-flow drag reduction, the control objective is written through the instantaneous reward $r(t)=-C_f(t)$, with an extended form $r=-C_f-d\frac{(\phi^+)^2}{2}$ when control effort is penalized; the action is the wall-normal blowing/suction input $\phi(x,z,t)$, and the state can include the local fluctuations $u'$ and $v'$ at $y_a^+=15$ (Sonoda et al., 2022). In the linear setting, the policy reduces to $\phi(x,z,t)=a\,v'(x,y_a,z,t)+B+N$, and the learned slope converges to approximately $a=-1.0$, reproducing opposition control (Sonoda et al., 2022).

In terrain-aware navigation, friction enters the transition model itself. TRADYN specializes the environment dynamics to

$$\bm{x}_{n+1}=f(\bm{x}_n,u_n,\beta,\tau(\bm{x}_n))+\epsilon_n,$$

so rollout prediction depends jointly on a latent robot-specific context $\beta$ and terrain observations $r=-C_f-d\frac{(\phi^+)^2}{2}$0 queried from a spatial map (Guttikonda et al., 2023). This makes terrain friction queryable during planning rather than latent and unmodeled.

In sliding manipulation, the state explicitly contains a friction estimate $r=-C_f-d\frac{(\phi^+)^2}{2}$1 together with desired remaining displacement, previous actions, and previous induced displacements, while the action is the continuous tuple $r=-C_f-d\frac{(\phi^+)^2}{2}$2 controlling initial acceleration, maximum acceleration, and maneuver duration (Raei et al., 24 Feb 2025). In friction-aware safety locomotion for a wheeled inverted pendulum, the observation augments proprioception with an estimated friction coefficient $r=-C_f-d\frac{(\phi^+)^2}{2}$3 obtained by a Friction-From-Vision module, and the policy outputs a desired wheel velocity (Peng et al., 2024). In force-aware pushing, friction and unsafe contact are not represented by an explicit coefficient, but by force and touch indicators appended to the observation and converted into reward terms $r=-C_f-d\frac{(\phi^+)^2}{2}$4 and $r=-C_f-d\frac{(\phi^+)^2}{2}$5 (Lin et al., 30 Oct 2025).

The result is that friction-awareness can be implemented at different levels of abstraction. Some methods learn a direct reactive controller from local measurements; others learn a context-conditional transition model; still others treat friction as a latent variable that is estimated online and then fed back into the policy. This suggests that friction-aware RL is structurally heterogeneous but formally unified by the explicit insertion of friction-sensitive variables into the MDP or POMDP specification.

3. Mechanisms for making policies friction-aware

One major mechanism is physics-inspired reward shaping. In risk-aware rearrangement, the per-step “virtual physical work” cost $r=-C_f-d\frac{(\phi^+)^2}{2}$6 depends on $r=-C_f-d\frac{(\phi^+)^2}{2}$7, $r=-C_f-d\frac{(\phi^+)^2}{2}$8, $r=-C_f-d\frac{(\phi^+)^2}{2}$9, object geometry, displacement, and rotation; the normalized cost $\phi(x,z,t)$0 is then used in the reward so that the agent prefers physically cheaper pushing trajectories (Song et al., 2022). In the variable friction pushing task, the environment contains a two-band floor with friction coefficients $\phi(x,z,t)$1 and $\phi(x,z,t)$2, and the reward term for pushing is modulated by step physical cost (Song et al., 2022). In Tac2Motion, tactile sensing is not merely observed; it is also used to define contact pressure, contact release, and rotation rewards, combined as

$\phi(x,z,t)$3

so that firm grasping, controlled release, and smooth finger gaiting are learned jointly in a torsional-friction-sensitive task (Kim et al., 22 Sep 2025). In force-based safe control, touch rewards appropriate contact while force/torque terms penalize excessive loads and collision-prone states (Lin et al., 30 Oct 2025).

A second mechanism is simulator fidelity and contact modeling. The ballbot work argues that the main obstacle is the tribology of the wheel–sphere–ground system rather than the RL algorithm itself, and it therefore introduces a MuJoCo model with three ETH-type omni-wheels, each built as a ring of 12 free-spinning rollers, plus solver settings for discontinuous stick-slip and six-dimensional ball–ground contact including tangential, torsional spin, and rolling resistance (Wan et al., 27 Apr 2026). The same emphasis appears in work on servo actuators and transferable friction models, where simplified Coulomb-Viscous heuristics are treated as a source of simulation mismatch for RL and control workflows (Duclusaud et al., 2024, Ozmen et al., 16 Apr 2025).

A third mechanism is friction estimation or conditioning. Sliding manipulation estimates friction analytically or with an LSTM after each action and feeds the updated estimate back to the actor (Raei et al., 24 Feb 2025). FSL-LVLM predicts a ground friction coefficient from vision before the robot reaches the surface and then conditions the policy on that estimate (Peng et al., 2024). TRADYN uses terrain lookup during rollout prediction so planning can bend around high-friction regions and prefer lower-friction corridors (Guttikonda et al., 2023).

A fourth mechanism is cost-aware objective design in settings where “friction” is operational rather than mechanical. In shortfall-aware hedging, transaction costs are modeled explicitly as $\phi(x,z,t)$4, and the relevant risk measures are shortfall probability and Expected Shortfall rather than only static pricing fit (Hu et al., 1 Feb 2026). This widens the concept of friction-aware RL to include execution frictions that materially distort realized control performance.

4. Algorithms, architectures, and representative learning workflows

The algorithmic choices are varied and task-specific. Turbulent channel-flow control uses Deep Deterministic Policy Gradient (DDPG) with an actor mapping state to wall actuation and a critic trained by a Bellman residual with $\phi(x,z,t)$5; episodes have length $\phi(x,z,t)$6, policy updates occur every $\phi(x,z,t)$7, Adam learning rates are $\phi(x,z,t)$8 for the actor and $\phi(x,z,t)$9 for the critic, the replay buffer is $u'$0, and the batch size is 64 (Sonoda et al., 2022). The nonlinear actor is a one-hidden-layer network with 8 nodes and tested activations ReLU, sigmoid, leaky ReLU, and tanh (Sonoda et al., 2022).

Adaptive time integration for nonsmooth Coulomb-friction fault dynamics uses Truncated Quantile Critics (TQC) for continuous step-size selection, with an observation space $u'$1, action $u'$2, actor hidden layers of 64 and critic hidden layers of 64, five critic networks, learning rate $u'$3, buffer size $u'$4, batch size 256, $u'$5, and Polyak coefficient $u'$6 (Riley et al., 15 Jan 2025). The reward jointly encourages large steps, low runtime, low local error, and convergence of the nonlinear solve (Riley et al., 15 Jan 2025).

TRADYN is implemented as a GRU-based latent dynamics model trained in a Neural-Process-style meta-learning framework, with a permutation-invariant context encoder and an ELBO objective over latent robot context $u'$7 (Guttikonda et al., 2023). Rearrangement with physics-inspired risk awareness uses PPO in Bullet/OpenRooms/iGibson with a discrete action space for a two-wheeled Fetch robot (Song et al., 2022). Tac2Motion also uses PPO with actor-critic structure and asymmetric learning in IsaacGym, with an action $u'$8 filtered by an EMA update (Kim et al., 22 Sep 2025). The sliding manipulation framework uses DDPG because the action space is continuous and the task requires deterministic, smooth control (Raei et al., 24 Feb 2025). Friction-aware sim-to-real locomotion on Saturn Lite uses RMA, comparing conventional domain randomization, Actuator Net, and static friction-aware domain randomization (Hu et al., 3 Mar 2025).

The diversity of algorithms indicates that friction-awareness is largely orthogonal to the choice of RL backbone. Actor-critic, off-policy deterministic control, latent-model planning, and adaptation-based locomotion all appear. What changes across papers is not the generic optimizer class but the way friction is represented, sensed, randomized, or embedded into the environment model.

5. Empirical behavior and cross-domain results

In turbulent channel flow, the simplest linear actor recovers opposition control, with learned slope $u'$9, while nonlinear policies based on $v'$0 and $v'$1 produce abrupt switching between strong wall blowing and suction (Sonoda et al., 2022). In the full channel, the reported drag-reduction rates are 31% for R18, 35% for S18, 35% for LR18, 27% for T18, and 23% for conventional opposition control, while the abstract highlights a best-case drag reduction of 37% (Sonoda et al., 2022). The paper further reports that wave-like patterns generated by the learned feedback policy do not by themselves explain performance, because fixed open-loop versions produce weak drag reduction (Sonoda et al., 2022).

In the ballbot setting, the friction-aware policy is reported to close the Sim2Real gap through high-fidelity contact modeling, friction and sensor randomization, and a low-dimensional state/action formulation centered on body dynamics (Wan et al., 27 Apr 2026). Under nominal settings both RL and LQR do well, but under random torsional friction, random initial tilt, and random arm configurations, LQR drops to 56% success for velocity tracking under random orientation and to 0% station-keeping success in the same condition, whereas RL remains at or near 100% success across most conditions (Wan et al., 27 Apr 2026). On hardware, the robot successfully drives on indoor carpet, ceramic tile, a yoga mat, and a hard anti-slip floor, traverses a small aluminum bump and a 5-degree slope, stays within about 5 cm on ceramic tile and 3 cm on a yoga mat during station keeping, and exhibits hardware velocity-tracking error about 0.05 m/s (Wan et al., 27 Apr 2026).

In terrain-aware planning, the full TRADYN model with terrain lookup and calibration achieves median final distance to goal $v'$2 mm for $v'$3T, $v'$4C, compared to $v'$5 mm for $v'$6T, $v'$7C and $v'$8 mm for $v'$9T, $y_a^+=15$0C, with failure rate $y_a^+=15$1 for the calibrated variants (Guttikonda et al., 2023). In non-prehensile sliding manipulation, the learned policy generalizes across distances from 0.02 m to 0.2 m and friction coefficients from 0.05 to 0.45, transfers zero-shot to hardware, and on a surface with true $y_a^+=15$2, the LSTM estimator updates $y_a^+=15$3 from 0.13 to 0.2106 while the analytical method updates it to 0.255 (Raei et al., 24 Feb 2025). In friction-aware locomotion for a wheeled inverted pendulum, the proposed method achieves simulation tracking error $y_a^+=15$4, compared to $y_a^+=15$5 for LQR, $y_a^+=15$6 for the student, $y_a^+=15$7 for PPO DR, and $y_a^+=15$8 for the variant without translation joint input; only the proposed method successfully transfers across all tested surfaces in real-world experiments (Peng et al., 2024).

In nonsmooth mechanics, the adaptive integrator achieves average speed-up about 1.3× for the 3 km fault and about 2.26× for the unseen 5 km fault, with the abstract highlighting up to a fourfold speed-up (Riley et al., 15 Jan 2025). In option hedging, RLOP achieves the lowest shortfall probability in 6 of 8 slices in the main $y_a^+=15$9 analysis, and an RL method achieves the best $\phi(x,z,t)=a\,v'(x,y_a,z,t)+B+N$0 in 5 of 8 slices, with the clearest tail-risk gains in XOP 2020Q1 (Hu et al., 1 Feb 2026). These results broaden the empirical meaning of friction-aware RL: the central measurable effect is often not only raw task success, but improved behavior under dissipation, discontinuity, or trading cost.

6. Limitations, misconceptions, and open problems

The literature is explicit that friction-awareness does not remove the hard parts of RL. In the ballbot work, training still depends on a virtual ball joint, Coulomb friction in the drivetrain creates a persistent dead zone and residual jitter during station keeping, and the system relies on Apple’s proprietary VIO pipeline (Wan et al., 27 Apr 2026). In static-friction-aware locomotion, direct inclusion of friction made training difficult, the iterative fine-tuning approach failed, and the successful “deception method” worked by significantly expanding the randomization range rather than by exact friction matching (Hu et al., 3 Mar 2025). In adaptive time integration, the initial setup produced seven failing scenarios, requiring transfer learning on a 30-by-30 discretization scenario to recover convergence (Riley et al., 15 Jan 2025).

Another misconception is that better friction modeling is always sufficient for transfer. Work on servo actuators and physics-informed LuGre identification argues instead that friction modeling is a critical enabling layer for RL and control, but not itself an RL policy (Duclusaud et al., 2024, Ozmen et al., 16 Apr 2025). The actuator paper shows that more expressive friction models can reduce trajectory MAE by factors of 2.93 for MX-64, 2.02 for MX-106, 2.34 for eRob80:100, and 1.51 for eRob80:50 relative to the Coulomb-Viscous baseline, yet this remains an infrastructure contribution rather than an end-to-end RL result (Duclusaud et al., 2024). The PINN-based LuGre paper similarly motivates RL relevance strongly but reports no explicit RL benchmark (Ozmen et al., 16 Apr 2025).

Several papers also delimit the current scope of evaluation. The risk-aware rearrangement method is demonstrated in simulation rather than on physical hardware (Song et al., 2022). TRADYN assumes a clean simulated setting with deterministic dynamics and perfect access to a terrain map, with future work explicitly framed around partial observability and noisy terrain estimation (Guttikonda et al., 2023). FSL-LVLM notes that FFV is slow mainly because of GPT-4 server latency, which limits real-time updating (Peng et al., 2024). Tac2Motion demonstrates successful real-robot transfer, but the method is specialized to contact-rich in-hand manipulation and uses a virtual torque approximation rather than full patch-contact friction simulation (Kim et al., 22 Sep 2025).

A broader synthesis is that friction-aware RL is most mature when it treats friction as a first-class modeling object throughout the pipeline: simulator, state, reward, randomization, and evaluation. The open problem is not only to estimate friction more accurately, but to learn policies that remain stable when friction is discontinuous, partially observed, morphology-dependent, or operationally realized as transaction cost.