---
title: Friction-Aware Reinforcement Learning
url: https://www.emergentmind.com/topics/friction-aware-reinforcement-learning
type: topic
---

# Friction-Aware Reinforcement Learning

to=arxiv_search.search  微信的天天中彩票്സոն code _string="all:(\"friction-aware reinforcement learning\" OR \"skin friction drag\" OR \"static friction\" OR \"sliding manipulation\" OR \"terrain-aware dynamics\" )", "max_results": 10}
to=arxiv_search.search  大发快三计划 code  天天中彩票腾讯json='{"query":"all:(\\"friction-aware reinforcement learning\\" OR \\"skin friction drag\\" OR \\"static friction\\" OR \\"sliding manipulation\\" OR \\"terrain-aware dynamics\\")","max_results":10}'
to=arxiv_search.search code  ажәларjson='{"query": "all:(\"friction-aware reinforcement learning\" OR \"skin friction drag\" OR \"static friction\" OR \"sliding manipulation\" OR \"terrain-aware dynamics\")", "max_results": 10}'
to=arxiv_search.search code արկjson={"query":"all:(\"friction-aware reinforcement learning\" OR \"skin friction drag\" OR \"static friction\" OR \"sliding manipulation\" OR \"terrain-aware dynamics\")","max_results":10}
to=arxiv_search.search  天天中彩票如何json
{"query":"2206.15355 OR 2604.24916 OR 2307.09206 OR 2501.08934 OR 2206.12784 OR 2502.17221 OR 2503.01255 OR 2409.09845 OR 2410.08650 OR 2504.12441","max_results":10}
Friction-aware reinforcement learning denotes a family of reinforcement learning formulations in which friction, contact dissipation, or execution frictions are modeled as central elements of the control or decision problem rather than treated as unstructured disturbance. In recent arXiv work, this perspective appears in turbulent drag reduction through wall blowing and suction [2206.15355], underactuated ballbot control dominated by coupled rolling, lateral, and torsional friction [2604.24916], terrain-aware navigation over spatially varying friction maps [2307.09206], adaptive time integration for Coulomb-friction fault dynamics [2501.08934], robotic rearrangement and sliding manipulation under variable friction [2206.12784, 2502.17221], friction-conditioned locomotion and sim-to-real transfer [2503.01255, 2409.09845], tactile contact-rich dexterous manipulation [2509.17812], and option hedging under transaction-cost frictions [2603.06587]. Taken together, these works suggest that friction-aware RL is best understood as a modeling stance: the policy, simulator, reward, or evaluation metric is explicitly structured around frictional mechanisms that materially govern behavior.

## 1. Scope and defining characteristics

Across the literature, friction-aware RL spans several distinct physical and operational meanings of “friction.” In fluid mechanics, the objective is the reduction of **skin friction drag** in a fully developed turbulent channel flow [2206.15355]. In mobile robotics and manipulation, friction appears as **rolling, lateral, torsional, static, dynamic, and torsional friction**, or as **contact pressure** acting as a proxy for frictional coupling [2604.24916, 2502.17221, 2509.17812]. In navigation, friction is a **map-dependent field** that alters dissipation and path cost [2307.09206]. In numerical mechanics, it is a **set-valued Coulomb law** that generates stick-slip transitions and solver failures [2501.08934]. In derivatives hedging, friction is realized as **proportional transaction costs** and spread effects [2603.06587].

These differences matter because they prevent a narrow definition tied to any single algorithm. Some papers place friction directly in the environment dynamics, some in the observation space, some in reward shaping, and some in the simulator or model class used for training. A plausible implication is that friction-aware RL is less a subfield with a fixed benchmark suite than a recurrent design pattern for problems in which dissipation, breakaway thresholds, and contact discontinuities dominate policy quality.

| Domain | Friction-aware element | Representative paper |
|---|---|---|
| Turbulent flow control | Skin-friction drag, wall blowing/suction | [2206.15355] |
| Ballbot control | Rolling, lateral, torsional friction; roller mechanics | [2604.24916] |
| Navigation and planning | Spatially varying terrain friction map | [2307.09206] |
| Nonsmooth mechanics | Coulomb friction and stick-slip | [2501.08934] |
| Manipulation and locomotion | Static/dynamic/torsional/contact friction | [2206.12784], [2502.17221], [2509.17812], [2503.01255], [2409.09845] |
| Finance | Transaction-cost frictions | [2603.06587] |

A common misconception is that friction-aware RL is merely domain randomization over a coefficient of friction. The surveyed papers show a broader picture: friction may be inferred from vision, estimated online from motion, encoded in tactile reward shaping, built into a probabilistic dynamics model, or represented through solver-level contact mechanics [2307.09206, 2502.17221, 2409.09845, 2509.17812].

## 2. Formalization in state, action, reward, and transition models

A defining property of friction-aware RL is that friction changes the formal RL problem rather than only the physical interpretation of the task. In channel-flow drag reduction, the control objective is written through the instantaneous reward \(r(t)=-C_f(t)\), with an extended form \(r=-C_f-d\frac{(\phi^+)^2}{2}\) when control effort is penalized; the action is the wall-normal blowing/suction input \(\phi(x,z,t)\), and the state can include the local fluctuations \(u'\) and \(v'\) at \(y_a^+=15\) [2206.15355]. In the linear setting, the policy reduces to \(\phi(x,z,t)=a\,v'(x,y_a,z,t)+B+N\), and the learned slope converges to approximately \(a=-1.0\), reproducing opposition control [2206.15355].

In terrain-aware navigation, friction enters the transition model itself. TRADYN specializes the environment dynamics to
\[
\bm{x}_{n+1}=f(\bm{x}_n,u_n,\beta,\tau(\bm{x}_n))+\epsilon_n,
\]
so rollout prediction depends jointly on a latent robot-specific context \(\beta\) and terrain observations \(\tau(\bm{x}_n)\) queried from a spatial map [2307.09206]. This makes terrain friction queryable during planning rather than latent and unmodeled.

In sliding manipulation, the state explicitly contains a friction estimate \(\mu_e\) together with desired remaining displacement, previous actions, and previous induced displacements, while the action is the continuous tuple \((a_i,a_m,t_m)\) controlling initial acceleration, maximum acceleration, and maneuver duration [2502.17221]. In friction-aware safety locomotion for a wheeled inverted pendulum, the observation augments proprioception with an estimated friction coefficient \(p\) obtained by a Friction-From-Vision module, and the policy outputs a desired wheel velocity [2409.09845]. In force-aware pushing, friction and unsafe contact are not represented by an explicit coefficient, but by force and touch indicators appended to the observation and converted into reward terms \(r_{ft}(s)=c_{ft}I_{ft}(s)\) and \(r_{touch}(s)=c_{touch}I_{touch}(s)\) [2512.02022].

The result is that friction-awareness can be implemented at different levels of abstraction. Some methods learn a direct reactive controller from local measurements; others learn a context-conditional transition model; still others treat friction as a latent variable that is estimated online and then fed back into the policy. This suggests that friction-aware RL is structurally heterogeneous but formally unified by the explicit insertion of friction-sensitive variables into the MDP or POMDP specification.

## 3. Mechanisms for making policies friction-aware

One major mechanism is **physics-inspired reward shaping**. In risk-aware rearrangement, the per-step “virtual physical work” cost \(E(t)\) depends on \(\mu\), \(m\), \(g\), object geometry, displacement, and rotation; the normalized cost \(c(t)\) is then used in the reward so that the agent prefers physically cheaper pushing trajectories [2206.12784]. In the variable friction pushing task, the environment contains a two-band floor with friction coefficients \(0.2\) and \(0.8\), and the reward term for pushing is modulated by step physical cost [2206.12784]. In Tac2Motion, tactile sensing is not merely observed; it is also used to define contact pressure, contact release, and rotation rewards, combined as
\[
r=\lambda_{\text{cpr}}r_{\text{cpr}}+\lambda_{\text{crr}}r_{\text{crr}}+\lambda_{\text{rr}}r_{\text{rr}},
\]
so that firm grasping, controlled release, and smooth finger gaiting are learned jointly in a torsional-friction-sensitive task [2509.17812]. In force-based safe control, touch rewards appropriate contact while force/torque terms penalize excessive loads and collision-prone states [2512.02022].

A second mechanism is **simulator fidelity and contact modeling**. The ballbot work argues that the main obstacle is the **tribology** of the wheel–sphere–ground system rather than the RL algorithm itself, and it therefore introduces a MuJoCo model with three ETH-type omni-wheels, each built as a ring of 12 free-spinning rollers, plus solver settings for discontinuous stick-slip and six-dimensional ball–ground contact including tangential, torsional spin, and rolling resistance [2604.24916]. The same emphasis appears in work on servo actuators and transferable friction models, where simplified Coulomb-Viscous heuristics are treated as a source of simulation mismatch for RL and control workflows [2410.08650, 2504.12441].

A third mechanism is **friction estimation or conditioning**. Sliding manipulation estimates friction analytically or with an LSTM after each action and feeds the updated estimate back to the actor [2502.17221]. FSL-LVLM predicts a ground friction coefficient from vision before the robot reaches the surface and then conditions the policy on that estimate [2409.09845]. TRADYN uses terrain lookup during rollout prediction so planning can bend around high-friction regions and prefer lower-friction corridors [2307.09206].

A fourth mechanism is **cost-aware objective design** in settings where “friction” is operational rather than mechanical. In shortfall-aware hedging, transaction costs are modeled explicitly as \(\mathrm{TC}(\Delta u,S)=\epsilon |\Delta u|S\), and the relevant risk measures are shortfall probability and Expected Shortfall rather than only static pricing fit [2603.06587]. This widens the concept of friction-aware RL to include execution frictions that materially distort realized control performance.

## 4. Algorithms, architectures, and representative learning workflows

The algorithmic choices are varied and task-specific. Turbulent channel-flow control uses **Deep Deterministic Policy Gradient (DDPG)** with an actor mapping state to wall actuation and a critic trained by a Bellman residual with \(\gamma=0.99\); episodes have length \(T^+=600\), policy updates occur every \(\Delta t_{\text{update}}^+=0.6\), Adam learning rates are \(0.001\) for the actor and \(0.002\) for the critic, the replay buffer is \(5{,}000{,}000\), and the batch size is 64 [2206.15355]. The nonlinear actor is a one-hidden-layer network with 8 nodes and tested activations ReLU, sigmoid, leaky ReLU, and tanh [2206.15355].

Adaptive time integration for nonsmooth Coulomb-friction fault dynamics uses **Truncated Quantile Critics (TQC)** for continuous step-size selection, with an observation space \(\mathcal{O}=\{\hat U,\hat v,E_{iter}/4,E_r,\hat h,I\}\), action \(h\in[0.001,t_{max}/4]\ \mathrm{s}\), actor hidden layers of 64 and critic hidden layers of 64, five critic networks, learning rate \(3\times 10^{-4}\), buffer size \(10^6\), batch size 256, \(\gamma=0.99\), and Polyak coefficient \(\tau=0.005\) [2501.08934]. The reward jointly encourages large steps, low runtime, low local error, and convergence of the nonlinear solve [2501.08934].

TRADYN is implemented as a **GRU-based latent dynamics model** trained in a Neural-Process-style meta-learning framework, with a permutation-invariant context encoder and an ELBO objective over latent robot context \(\beta\) [2307.09206]. Rearrangement with physics-inspired risk awareness uses **PPO** in Bullet/OpenRooms/iGibson with a discrete action space for a two-wheeled Fetch robot [2206.12784]. Tac2Motion also uses **PPO** with actor-critic structure and asymmetric learning in IsaacGym, with an action \(\bm{a}_t\in\mathbb{R}^{22}\) filtered by an EMA update [2509.17812]. The sliding manipulation framework uses **DDPG** because the action space is continuous and the task requires deterministic, smooth control [2502.17221]. Friction-aware sim-to-real locomotion on Saturn Lite uses **RMA**, comparing conventional domain randomization, Actuator Net, and static friction-aware domain randomization [2503.01255].

The diversity of algorithms indicates that friction-awareness is largely orthogonal to the choice of RL backbone. Actor-critic, off-policy deterministic control, latent-model planning, and adaptation-based locomotion all appear. What changes across papers is not the generic optimizer class but the way friction is represented, sensed, randomized, or embedded into the environment model.

## 5. Empirical behavior and cross-domain results

In turbulent channel flow, the simplest linear actor recovers opposition control, with learned slope \(a\approx -1\), while nonlinear policies based on \(u'\) and \(v'\) produce abrupt switching between strong wall blowing and suction [2206.15355]. In the full channel, the reported drag-reduction rates are 31% for R18, 35% for S18, 35% for LR18, 27% for T18, and 23% for conventional opposition control, while the abstract highlights a best-case drag reduction of 37% [2206.15355]. The paper further reports that wave-like patterns generated by the learned feedback policy do not by themselves explain performance, because fixed open-loop versions produce weak drag reduction [2206.15355].

In the ballbot setting, the friction-aware policy is reported to close the Sim2Real gap through high-fidelity contact modeling, friction and sensor randomization, and a low-dimensional state/action formulation centered on body dynamics [2604.24916]. Under nominal settings both RL and LQR do well, but under random torsional friction, random initial tilt, and random arm configurations, LQR drops to 56% success for velocity tracking under random orientation and to 0% station-keeping success in the same condition, whereas RL remains at or near 100% success across most conditions [2604.24916]. On hardware, the robot successfully drives on indoor carpet, ceramic tile, a yoga mat, and a hard anti-slip floor, traverses a small aluminum bump and a 5-degree slope, stays within about 5 cm on ceramic tile and 3 cm on a yoga mat during station keeping, and exhibits hardware velocity-tracking error about 0.05 m/s [2604.24916].

In terrain-aware planning, the full TRADYN model with terrain lookup and calibration achieves median final distance to goal \(3.85\) mm for \(+\)T, \(+\)C, compared to \(5.19\) mm for \(-\)T, \(-\)C and \(4.22\) mm for \(+\)T, \(-\)C, with failure rate \(0/750\) for the calibrated variants [2307.09206]. In non-prehensile sliding manipulation, the learned policy generalizes across distances from 0.02 m to 0.2 m and friction coefficients from 0.05 to 0.45, transfers zero-shot to hardware, and on a surface with true \(\mu_k=0.244\), the LSTM estimator updates \(\mu_e\) from 0.13 to 0.2106 while the analytical method updates it to 0.255 [2502.17221]. In friction-aware locomotion for a wheeled inverted pendulum, the proposed method achieves simulation tracking error \(0.0042 \pm 0.0096\), compared to \(0.0096 \pm 0.0055\) for LQR, \(0.0275 \pm 0.0149\) for the student, \(0.0297 \pm 0.0214\) for PPO DR, and \(0.0125 \pm 0.0776\) for the variant without translation joint input; only the proposed method successfully transfers across all tested surfaces in real-world experiments [2409.09845].

In nonsmooth mechanics, the adaptive integrator achieves average speed-up about 1.3× for the 3 km fault and about 2.26× for the unseen 5 km fault, with the abstract highlighting up to a fourfold speed-up [2501.08934]. In option hedging, RLOP achieves the lowest shortfall probability in 6 of 8 slices in the main \(\tau=28\) analysis, and an RL method achieves the best \(\mathrm{ES}_{5\%}\) in 5 of 8 slices, with the clearest tail-risk gains in XOP 2020Q1 [2603.06587]. These results broaden the empirical meaning of friction-aware RL: the central measurable effect is often not only raw task success, but improved behavior under dissipation, discontinuity, or trading cost.

## 6. Limitations, misconceptions, and open problems

The literature is explicit that friction-awareness does not remove the hard parts of RL. In the ballbot work, training still depends on a **virtual ball joint**, Coulomb friction in the drivetrain creates a persistent dead zone and residual jitter during station keeping, and the system relies on Apple’s proprietary VIO pipeline [2604.24916]. In static-friction-aware locomotion, direct inclusion of friction made training difficult, the iterative fine-tuning approach failed, and the successful “deception method” worked by significantly expanding the randomization range rather than by exact friction matching [2503.01255]. In adaptive time integration, the initial setup produced seven failing scenarios, requiring transfer learning on a 30-by-30 discretization scenario to recover convergence [2501.08934].

Another misconception is that better friction modeling is always sufficient for transfer. Work on servo actuators and physics-informed LuGre identification argues instead that friction modeling is a critical enabling layer for RL and control, but not itself an RL policy [2410.08650, 2504.12441]. The actuator paper shows that more expressive friction models can reduce trajectory MAE by factors of 2.93 for MX-64, 2.02 for MX-106, 2.34 for eRob80:100, and 1.51 for eRob80:50 relative to the Coulomb-Viscous baseline, yet this remains an infrastructure contribution rather than an end-to-end RL result [2410.08650]. The PINN-based LuGre paper similarly motivates RL relevance strongly but reports no explicit RL benchmark [2504.12441].

Several papers also delimit the current scope of evaluation. The risk-aware rearrangement method is demonstrated in simulation rather than on physical hardware [2206.12784]. TRADYN assumes a clean simulated setting with deterministic dynamics and perfect access to a terrain map, with future work explicitly framed around partial observability and noisy terrain estimation [2307.09206]. FSL-LVLM notes that FFV is slow mainly because of GPT-4 server latency, which limits real-time updating [2409.09845]. Tac2Motion demonstrates successful real-robot transfer, but the method is specialized to contact-rich in-hand manipulation and uses a virtual torque approximation rather than full patch-contact friction simulation [2509.17812].

A broader synthesis is that friction-aware RL is most mature when it treats friction as a first-class modeling object throughout the pipeline: simulator, state, reward, randomization, and evaluation. The open problem is not only to estimate friction more accurately, but to learn policies that remain stable when friction is discontinuous, partially observed, morphology-dependent, or operationally realized as transaction cost.

Source: https://www.emergentmind.com/topics/friction-aware-reinforcement-learning