---
title: 'RACCT: Recurrent Action-Confidence Transformer'
url: https://www.emergentmind.com/topics/recurrent-action-confidence-chunking-with-transformer-racct
type: topic
---

# RACCT: Recurrent Action-Confidence Transformer

Recurrent Action-Confidence Chunking with Transformer (RACCT) is a transformer-based imitation-learning model for autonomous nasotracheal intubation (NTI), introduced as the central learning component of a robotic system designed to perform low-contact tube insertion under partial visual observability [2508.01808]. The method extends Action Chunking with Transformer (ACT) by adding a recurrent decoder mechanism and a learned action-confidence output, with the specific aim of improving temporal consistency and reducing the influence of unreliable chunk predictions during contact-rich insertion. In the reported phantom experiments, RACCT achieves the same 100% success rate as the doctor and experienced teleoperators, while reducing the most hazardous peak-force channels relative to ACT-based baselines [2508.01808].

## 1. Clinical problem and technical motivation

NTI is a standard airway-management procedure in anesthesia, critical care, and emergency medicine, in which a tube is inserted through the nose into the airway. The RACCT paper frames NTI as difficult to automate for three linked reasons: the nasotracheal tube is larger in diameter and more rigid than a standard endoscope, the procedure takes place in narrow anatomy where tube-tissue interaction can cause mucosal injury, and the internal state of the tube is only partially observable during insertion [2508.01808].

The paper treats low-contact insertion as a primary safety objective rather than a secondary performance criterion. To quantify sustained excessive loading, it defines a force-impulse metric
$$
I = \int \max\left(F - F_{\text{threshold}}, 0\right)\,\mathrm{d}t,
$$
which penalizes prolonged contact above threshold rather than only instantaneous peaks. This is significant because the injury mechanism is not limited to brief spikes; continuous pressure is also undesirable. The work also emphasizes infection-risk reduction, arguing that autonomous or remote robotic intubation can reduce clinician exposure to aerosols and contamination [2508.01808].

The authors present the overall system as, to their knowledge, the first autonomous system for NTI. That claim is carefully bounded in the paper itself. More specifically, the novelty is not only robotic insertion, but autonomous NTI with low-contact control, force-instrumented data collection, and a chunked transformer policy tailored to partial observability and safety-sensitive contact [2508.01808].

## 2. Robotic platform, sensing, and perception front end

The autonomous NTI platform combines a KUKA iiwa robot with 7 degrees of freedom, joint position and torque sensing, a 3D force sensor at the end effector, two cameras, a nasotracheal tube, a PC, a teleoperation controller, and a custom prosthesis with embedded force sensors. During data collection, the robot end-effector motion is constrained to the $x$-$z$ plane with 2D translation and rotation about the $y$-axis, giving 3 controlled DoFs [2508.01808].

A major hardware contribution is the force-sensing prosthesis. It is built by 3D scanning a commercial tracheal intubation training model, post-processing the geometry, biomechanically simplifying three critical luminal regions, integrating sensors, and using multi-material 3D printing for fabrication. The prosthesis contains one 3D force sensor near the nostril measuring $[F_x, F_y, F_z]$ and two 1D force sensors, $F_1$ and $F_2$, near the sphenoid or nasal-cavity region and the pharynx or throat region, respectively. The soft nostril region is printed using hybrid soft materials with Shore hardness 85A, specifically Agilus30Clear and VeroClear, while rigid PLA is used for the nasal-cavity regions [2508.01808].

The perception stack is deliberately asymmetric. Camera 1 observes the portion of the tube outside the prosthesis and supplies the model’s visual input. Camera 2 records the procedure for post hoc analysis only and is explicitly excluded from policy input in order to mimic realistic surgical partial observability. The phantom is also partially occluded by a white box so that the internal tube configuration cannot be directly seen [2508.01808].

Because the tube is a thin deformable linear object with a transparent reflective surface, the paper introduces a task-specific segmentation pipeline. It first uses RepViT-SAM for coarse segmentation, then applies connected-component extraction, area filtering, skeletonization, geometric filtering, and selection of the component with the largest mean $x$-coordinate. The resulting skeleton is fit with a quadratic
$$
ax^2 + bx + c = y,
$$
from which curvature is derived as
$$
\kappa = \frac{2|a|}{\sqrt{(1+2ax+b)^3}}, \qquad
s_\kappa = \frac{1}{1+\frac{1}{|P|}\sum_{i\in P}\kappa_i}.
$$
This front end is intended to preserve ex vivo tube shape information while suppressing irrelevant background structure [2508.01808].

A notable design choice is that the prosthesis contact sensors are not policy inputs during autonomous inference. The paper states that such sensing is unavailable in routine clinical settings. Instead, those sensors serve for real-time feedback during teleoperation, safety assessment, data filtering, and evaluation. The model receives only information available on the robot side: visual observations from Camera 1, robot end-effector pose, and the robot’s own 3D end-effector force signal [2508.01808].

## 3. RACCT architecture

RACCT is explicitly described as an extension of ACT. The paper identifies three modifications relative to ACT: a tube-segmentation module, an action-confidence pair sequence output structure, and a recurrent decoder architecture [2508.01808]. The resulting model is intended to preserve the long-horizon robustness of action chunking while making chunk execution more history-aware and reliability-aware.

The encoder side retains the ACT-style observation processing. RACCT takes as input the segmented visual observation from Camera 1, the robot end-effector pose, the robot end-effector 3D force, and the ACT style variable $Z$, which the authors describe as containing distribution information from training data. The model does not use prosthesis contact forces as inputs. Each model has about 80M parameters [2508.01808].

As in ACT, the decoder predicts a chunk of future actions of length $k$. In the reported experiments, the chunking size is 80. RACCT departs from ACT by outputting not only an action sequence $A_t[i]$ but also a corresponding confidence sequence $C_t[i]$. The confidence head is followed by a sigmoid. At execution time, the current control command is obtained through a confidence-weighted temporal fusion:
$$
p_t = \sum_i \left(e^{-mi} C_t[i] A_t[i]\right)\Big/\sum_i \left(e^{-mi} C_t[i]\right),
$$
where $m$ is a positive constant. This replaces ACT’s fixed temporal-ensemble weighting with a recency term modulated by learned confidence. The intended effect is that poor predictions need not dominate execution for as long as they would under a fixed ensemble [2508.01808].

The “recurrent” element is not an RNN or LSTM cell. Instead, RACCT uses what the paper describes as a shift-recurrent decoder. The previous predicted sequence is shifted by one step, the leftmost token is discarded, a CLS token is appended on the right, and position embeddings are added before the next decoding step. The paper expresses the intended temporal consistency as
$$
s_k^{t-1} = s_{k-1}^{t},
$$
meaning that predictions for the same future event should agree across consecutive replanning times. The recurrence is thus implemented structurally through token reuse across chunk predictions rather than through an explicit hidden-state update [2508.01808].

This architecture can be summarized as a two-part modification of ACT. The first part is reliability estimation at the action-token level through $C_t[i]$. The second part is temporal plan carryover through the shifted previous chunk. In the paper’s interpretation, confidence mitigates unreliable chunk influence, while recurrence compensates for hidden tube state under partial observability [2508.01808].

## 4. Demonstration pipeline, filtering, and optimization

The learning paradigm is supervised imitation from teleoperated demonstrations. Operators manipulate the robot through a controller whose handle inputs are mapped to end-effector pose increments in the restricted 3-DoF action space. During collection, operators see the force displays and can adapt insertion strategy accordingly. Recorded data include end-effector 6D pose, end-effector 3D force, the Camera 1 image, and prosthesis force-sensor signals [2508.01808].

The paper places unusual emphasis on demonstration filtering. Thresholds are derived from a professional otolaryngology doctor’s manual intubation data and clinical suggestion:
- intubation time $t < 20\,\text{s}$,
- peak force $F_{\text{peak}} < 5\,\text{N}$,
- $\ln(I) < 1\,\text{N*s}$ with $F_{\text{threshold}} = 1.5\,\text{N}$.

Only demonstrations with metrics below 70% of those safety thresholds are retained for training. This means the policy is intentionally trained on especially gentle demonstrations rather than on the full teleoperation distribution [2508.01808].

Training uses 50 episodes of high-quality data, 20,000 optimization steps, batch size 8, chunking size 80, learning rate $1\times10^{-5}$, and hyperparameters $m=0.95$, $\epsilon=0.2$, and $\lambda=0.1$. Training is reported to take about 1 hour on an RTX A6000 GPU [2508.01808].

The paper gives the RACCT loss, with slightly imperfect typesetting in the manuscript, in a form whose intended structure is
$$
Loss=\sum_{i=t}^{t+k-1}\frac{c_i|\hat{a_i}-a_i|}{k(\epsilon+1-c_i)}-\lambda \log\left(\sum_{i=t}^{t+k-1}\frac{c_i}{k}\right).
$$
Here $k$ is chunk size, $a_i$ is the model-predicted action, $\hat{a_i}$ is the ground-truth action, and $c_i$ is the predicted confidence. The first term weights prediction error by confidence while penalizing overconfident mistakes more strongly as $c_i \to 1$. The second term prevents trivial collapse of confidence toward zero. The paper explicitly notes that without this regularizer, both confidence and the first loss term could collapse [2508.01808].

At deployment, RACCT runs in closed loop at 5 Hz, or about 200 ms per frame, on a workstation with Intel Core i9-14900K CPU, 48 GB RAM, and an RTX 3060 GPU. The execution loop is: capture Camera 1, segment the external tube, encode visual and robot-side state, decode a future action-confidence sequence using the shifted previous chunk, fuse the chunk into the current command $p_t$, and send that command to the robot [2508.01808].

## 5. Experimental evaluation

The evaluation compares three human modes—manual proximal intubation, manual distal intubation, and robotic teleoperation—with four autonomous ACT-family models: ACT, ACCT, RACT, and RACCT. Human participants include one professional otolaryngology doctor, five novices, and two experienced operators who each had at least 200 successful intubation experiences and who also collected the training data. Each participant repeated each assigned method 20 times. Autonomous evaluation uses success rate, intubation time, peak force in channels $F_x, F_y, F_z, F_1, F_2$, and logarithmic impulse in those channels. The paper notes that force and time metrics exclude failed episodes, so success rate is relatively more important [2508.01808].

| Model | Modification relative to ACT | Success rate |
|---|---|---|
| ACT | Baseline ACT | 80% |
| ACCT | ACT + action-confidence output | 85% |
| RACT | ACT + recurrent architecture | 90% |
| RACCT | Confidence + recurrent architecture | 100% |

The ablation indicates that both modifications help, and that recurrence contributes more than confidence alone. RACCT reaches the same 100% success rate as the doctor and experienced teleoperators, while the baseline ACT remains at 80% [2508.01808].

Manual operation remains faster. Doctor proximal intubation averages 5.03 s, whereas RACCT averages 9.13 s. The paper attributes this gap to deliberately conservative robot speed and emphasizes that RACCT still remains well below the 20 s safety threshold [2508.01808].

The force analysis is more nuanced than the headline success rate. The paper highlights $F_x$ and $F_2$ as the most injury-relevant channels. Selected peak-force values are:
- Doctor: $F_x = 2.75\,\text{N}$, $F_2 = 2.66\,\text{N}$,
- ACT: $F_x = 2.41\,\text{N}$, $F_2 = 2.36\,\text{N}$,
- ACCT: $F_x = 1.57\,\text{N}$, $F_2 = 2.05\,\text{N}$,
- RACT: $F_x = 2.00\,\text{N}$, $F_2 = 1.99\,\text{N}$,
- RACCT: $F_x = 1.56\,\text{N}$, $F_2 = 1.81\,\text{N}$.

RACCT is therefore best on the two highest-risk channels, although the doctor remains best on $F_z$ and $F_1$, and experienced teleoperation is best on $F_y$ [2508.01808].

The paper also reports impulse behavior rather than only peak loading. RACCT has the lowest impulse in $F_z$ and $F_1$, and the second-best impulse in $F_x$ behind ACCT, whereas the doctor performs better in $F_y$ and $F_2$. This distinction matters because a lower peak does not always imply lower sustained contact. The paper explicitly notes that the doctor’s $F_2$ impulse is lower than RACCT’s despite the doctor having a larger $F_2$ peak, suggesting briefer contact duration by the human operator [2508.01808].

One interpretive issue in the paper concerns the reported “66% reduction.” The abstract states that RACCT “achieves a 66% reduction in average peak insertion force compared to manual operations,” but the detailed results section supports a narrower statement: RACCT’s largest peak force is $1.81\,\text{N}$, the doctor’s largest peak force is $2.75\,\text{N}$, and $1.81/2.75 \approx 0.66$. This supports the claim that RACCT’s maximum peak force is 66% of the doctor’s maximum peak force, or about 34% lower, rather than a literal 66% reduction [2508.01808].

## 6. Position within the action-chunking literature

RACCT belongs to the action-chunking transformer family but occupies a specific niche: safety-critical deformable insertion under partial observability with learned per-action confidence. The closest architectural ancestor in the paper is ACT, which it modifies directly. Related work in adjacent domains clarifies what RACCT adds and what it does not [2508.01808].

Bi-ACT transfers ACT-style chunk prediction to bilateral-control imitation learning with force or torque channels and leader-follower asymmetry, but the available description explicitly notes that it does not introduce recurrent hidden-state updates, confidence scores, chunk ranking, or adaptive chunk horizons [2401.17698]. Relative to that line, RACCT adds a learned confidence head and a shift-recurrent decoder, but it does so in a different application regime: tube insertion rather than bilateral telemanipulation.

Real-Time Chunking (RTC) addresses the asynchronous execution problem for chunked policies under inference delay by freezing guaranteed actions and inpainting the overlap region, yet it does not define explicit confidence scores or a transformer-specific recurrent architecture [2506.07339]. RACCT addresses a different issue. Its recurrence is internal to the chunk decoder, and its confidence operates at the action-token level during temporal fusion rather than as an overlap-conditioning mechanism for delayed asynchronous generation.

Asynchronous Action Chunk Correction (A2C2) adds a lightweight per-step residual corrector on top of a frozen chunking policy, using the latest observation and the current base action to restore responsiveness, but it includes no explicit confidence head and no recurrent hidden state [2509.23224]. This suggests a complementary relationship rather than a direct alternative: RACCT internalizes reliability weighting inside the chunk predictor, whereas A2C2 preserves the base predictor and corrects execution externally.

Adaptive Action Chunking (AAC) is especially close in spirit because it uses action entropy as an uncertainty cue to adapt chunk size at inference time, repeatedly deciding how many future actions to execute before replanning [2604.04161]. AAC, however, uses sample-based entropy from a diffusion or flow VLA and a knee-point heuristic over entropy growth, whereas RACCT uses a learned confidence head over action tokens inside a fixed-size chunk. This suggests that RACCT and AAC instantiate two different notions of reliability: token-level learned confidence within a chunk versus chunk-length selection from predictive entropy.

## 7. Limitations and open directions

The RACCT study is confined to a force-instrumented phantom rather than living tissue. The prosthesis is anatomically inspired and sensorized, but the paper acknowledges by implication that it cannot reproduce bleeding, lubrication variability, secretions, patient motion, reflexes, pathology, or full anatomical diversity [2508.01808]. Any interpretation beyond phantom validation therefore remains provisional.

The training dataset is also small by current imitation-learning standards: 50 high-quality episodes after aggressive filtering. That design strengthens the low-contact bias of the training set but raises unanswered questions about generalization, failure recovery, and robustness to broader insertion styles [2508.01808].

The system depends on a specific external-view perception setup and a custom segmentation pipeline. Because Camera 2 is withheld and the phantom is occluded, partial observability is built into the benchmark. However, the clinical operating room would add additional variability in illumination, occlusion, contamination, and tube appearance. The paper does not report robustness studies for such factors, nor does it provide a failure taxonomy isolating segmentation errors, hidden-state inference errors, or contact-accumulation failures [2508.01808].

Methodologically, the paper does not report formal statistical tests, does not provide large-scale participant numbers, and does not describe deployment-side safety interlocks beyond the trained policy and evaluation thresholds. It also does not use direct in vivo tissue-force sensing, because such sensing is unavailable clinically; the policy must instead infer hidden contact state from end-effector force and external tube shape [2508.01808].

These limitations also indicate the most plausible next steps. The paper itself points toward animal experiments and eventual human use. In research terms, the design suggests at least three broader directions: integrating richer confidence or uncertainty models, combining recurrent chunk prediction with inference-time correction or adaptive chunk sizing, and extending validation beyond phantom environments. Those directions are not implemented in RACCT, but the reported results indicate why they are natural continuations of the method [2508.01808].

Source: https://www.emergentmind.com/topics/recurrent-action-confidence-chunking-with-transformer-racct