ASkDAgger: Active Skill-Level Data Aggregation

• The paper introduces ASkDAgger, an interactive imitation learning framework that reduces human teaching effort by selectively querying teacher feedback on skill-level actions.
• It employs three components—S-Aware Gating, Foresight Interactive Experience Replay, and Prioritized Interactive Experience Replay—to manage uncertainty, data aggregation, and replay prioritization.
• Experiments on robotics benchmarks demonstrate improved generalization and a reduced annotation burden under distribution shift using both plan validation and relabeling.

Searching arXiv for the specified papers and closely related work on ASkDAgger. Active Skill-level Data Aggregation (ASkDAgger) is an interactive imitation learning framework for robotics that reduces human teaching effort by querying a teacher selectively while reusing the novice policy’s own skill-level plans as training signal. In its canonical formulation, a novice policy $\pi_\mathrm{N}$ maps observations and goals to mid-/high-level actions, an uncertainty operator $U$ estimates confidence in the proposed plan, and teacher feedback is used not only to correct errors but also to validate or relabel novice plans as demonstrations (Luijkx et al., 7 Aug 2025). The framework is organized around three components—S-Aware Gating (SAG), Foresight Interactive Experience Replay (FIER), and Prioritized Interactive Experience Replay (PIER)—and is designed for settings where actions are parameterized skills rather than low-level torques (Luijkx et al., 7 Aug 2025). A distinct but related use of the ASkDAgger lens appears in humanoid whole-body control, where rollout-based aggregation of skill-induced command trajectories is used to maintain a shared low-level controller under distribution shift during long-horizon skill composition (Kim et al., 14 Feb 2026).

1. Formal problem setting and core idea

ASkDAgger is proposed for interactive imitation learning in robotics, with the explicit goal of reducing human teaching effort while improving safety and generalization (Luijkx et al., 7 Aug 2025). The novice policy is defined as

$$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$

where at episode $k$ and time $t$ it receives observation $o_t^k \in \mathcal{O}$ and goal $g^k \in \mathcal{G}$ and outputs action $a_t^k \in \mathcal{A}$ (Luijkx et al., 7 Aug 2025). The teacher, modeled as a human or simulation oracle with policy $\pi_\mathrm{T}$, can provide annotation demonstrations, validate a novice’s proposed action as acceptable, or relabel a failed novice action as successful for another goal (Luijkx et al., 7 Aug 2025).

The environment is described as a dynamical system with hidden state $s_t^k \in \mathcal{S}$ and observations given by $U$0, while goals define success sets $U$1 (Luijkx et al., 7 Aug 2025). A transition $U$2 is successful if $U$3 (Luijkx et al., 7 Aug 2025). The novice learns from a demonstration dataset $U$4 of trajectories

$U$5

where the scalar teacher reward records the outcome of a query: $U$6 The learning objective is behavioral cloning on the aggregated dataset,

$U$7

with a supervised loss between novice and teacher actions over $U$8 (Luijkx et al., 7 Aug 2025).

The distinctive protocol is the novice’s ability to present a plan together with uncertainty: “I plan to do this, but I am uncertain” (Luijkx et al., 7 Aug 2025). Formally, ASkDAgger assumes an uncertainty operator

$U$9

where larger $$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$0 indicates lower confidence in the novice’s planned skill-level action (Luijkx et al., 7 Aug 2025). This shifts the role of a query from merely requesting a corrective label to presenting a candidate action that can itself become part of the training corpus if validated or relabeled.

2. Skill-level abstraction and relation to DAgger

ASkDAgger operates at the level of parameterized skills rather than low-level motor commands (Luijkx et al., 7 Aug 2025). A skill-level action is described as a vector of parameters for a primitive such as grasping, walking, pushing, door opening, or insertion (Luijkx et al., 7 Aug 2025). In the CLIPort setting used in the paper, a pick-and-place primitive is parameterized by pick pixel location and orientation together with place pixel location and orientation (Luijkx et al., 7 Aug 2025). Because one decision corresponds to a complete skill execution rather than a single control cycle, queries are less frequent and more interpretable to a human teacher (Luijkx et al., 7 Aug 2025).

This abstraction distinguishes ASkDAgger from step-level active imitation learning. Classical DAgger executes the novice and queries the expert for labels at all visited states, addressing covariate shift by collecting expert supervision on the novice’s own state distribution (Luijkx et al., 7 Aug 2025). Active DAgger variants reduce the label burden by querying only in uncertain, risky, or novel states, but at queried states they typically keep only the expert action and discard the novice proposal (Luijkx et al., 7 Aug 2025). ASkDAgger differs in three stated respects: plan-aware data usage, statistically adaptive gating, and priority-based updates (Luijkx et al., 7 Aug 2025).

The framework is therefore an active DAgger variant at skill level with performance-aware gating and replay, and with explicit use of novice plans as training signal (Luijkx et al., 7 Aug 2025). This suggests a conceptual reinterpretation of covariate shift: the key issue is not only that the expert must label states visited by the novice, but also that the novice’s own plan distribution contains information about capability, uncertainty, and alternative goal satisfaction.

3. Architecture: SAG, FIER, and PIER

The ASkDAgger framework is decomposed into three components: S-Aware Gating (SAG), Foresight Interactive Experience Replay (FIER), and Prioritized Interactive Experience Replay (PIER) (Luijkx et al., 7 Aug 2025). The main loop initializes an optional behavioral cloning dataset $$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$1 and a pre-trained novice $$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$2, sets $$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$3, and maintains arrays for update index per tuple, uncertainty per tuple, and reward per tuple (Luijkx et al., 7 Aug 2025). For each episode, the novice plans an action, computes uncertainty, obtains a threshold from SAG, and then either queries through FIER or executes autonomously; the resulting tuple is appended, PIER computes replay priorities, and the policy is updated from the aggregated dataset (Luijkx et al., 7 Aug 2025).

SAG dynamically maintains a query threshold on uncertainty so as to track one of three user-selected metrics: sensitivity, specificity, or minimum system success rate (Luijkx et al., 7 Aug 2025). FIER governs what happens when the novice queries the teacher: the teacher may validate the plan, reject it and provide an annotation, and optionally relabel the novice plan as a success for a different goal (Luijkx et al., 7 Aug 2025). PIER defines how tuples in the aggregated dataset are sampled during training, prioritizing them on the basis of uncertainty, novice success, and demonstration age (Luijkx et al., 7 Aug 2025).

The division of labor among the three components is central. SAG decides when the teacher is asked, FIER decides how a queried interaction becomes data, and PIER decides how strongly that data influences subsequent optimization (Luijkx et al., 7 Aug 2025). This modularity also clarifies the framework’s relationship to prior active imitation learning methods: thresholding alone is not ASkDAgger; the defining feature is the joint use of adaptive gating, plan-aware reuse of novice proposals, and replay prioritization.

4. S-Aware Gating (SAG)

SAG is described as an adaptive gating strategy that controls whether to query the teacher, with the explicit goal of maintaining a desired sensitivity, specificity, or minimum overall system success rate (Luijkx et al., 7 Aug 2025). It casts each time step as a binary classification problem in which “positive” means a query is made and “negative” means the novice acts autonomously, while the latent ground truth is whether the novice’s action is valid or invalid (Luijkx et al., 7 Aug 2025). In this view, a false positive wastes teacher time by querying on a valid novice action, whereas a false negative corresponds to an unqueried invalid action and thus system failure (Luijkx et al., 7 Aug 2025).

SAG keeps rolling histories of uncertainty, reward, and episode index: $$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$4 It uses a window of recent entries

$$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$5

where the definition of relevant labels depends on the selected mode (Luijkx et al., 7 Aug 2025). Because uncertainty tends to decrease as the policy improves, older uncertainty values are normalized using linear regression fitted on the current window: $$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$6 followed by shifting each uncertainty to its expected value at the current episode $$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$7: $$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$8 This compensates for temporal drift in the uncertainty distribution (Luijkx et al., 7 Aug 2025).

SAG then defines failure labels $$\pi_\mathrm{N}: \mathcal{O} \times \mathcal{G} \rightarrow \mathcal{A},$$9, so that teacher rejections correspond to novice failures and teacher validations correspond to novice successes (Luijkx et al., 7 Aug 2025). A logistic regression is fitted,

$k$0

to estimate

$k$1

and unlabeled points with $k$2 receive pseudo-labels by sampling from this inferred Bernoulli model (Luijkx et al., 7 Aug 2025). Repeating the pseudo-labeling process $k$3 times yields a distribution of candidate thresholds, and the final gating threshold is taken as the median (Luijkx et al., 7 Aug 2025).

The method explicitly accounts for random querying. In sensitivity-aware mode, the expected total sensitivity under active thresholding plus random queries with probability $k$4 is

$k$5

where $k$6 is the sensitivity under active gating alone (Luijkx et al., 7 Aug 2025). In specificity-aware mode,

$k$7

and in success-aware mode,

$k$8

under the assumption that the expert is always correct (Luijkx et al., 7 Aug 2025). By construction, increasing the desired sensitivity or desired success rate lowers the threshold and produces more queries, whereas increasing the desired specificity raises the threshold and favors autonomy (Luijkx et al., 7 Aug 2025).

5. Foresight Interactive Experience Replay (FIER) and Prioritized Interactive Experience Replay (PIER)

FIER governs the data transformation performed at query time (Luijkx et al., 7 Aug 2025). Given observation $k$9, novice action $t$0, teacher policy $t$1, current trajectory $t$2, and goal $t$3, the teacher responds to the novice’s plan by either validating it, rejecting it and supplying an annotation, or optionally relabeling the novice plan as successful for another goal $t$4 (Luijkx et al., 7 Aug 2025). When the teacher validates the plan, FIER appends

$t$5

to the trajectory and executes the novice plan (Luijkx et al., 7 Aug 2025). When the teacher rejects the plan, it computes $t$6 and appends the annotation tuple

$t$7

and if a relabeled goal is provided it also appends

$t$8

as a relabeled tuple (Luijkx et al., 7 Aug 2025).

The stated interpretation is that validated novice actions become expert-quality demonstrations, while novice failures may still become positive examples for a different goal via relabeling (Luijkx et al., 7 Aug 2025). The paper explicitly compares the latter mechanism to Hindsight Experience Replay, because the transition can be reinterpreted as successful with respect to another goal if the resulting state belongs to the corresponding goal-success set (Luijkx et al., 7 Aug 2025). The significant departure from standard interactive imitation learning is that many useful demonstrations arise from the novice’s own plan and from teacher foresight, not only from executed expert corrections (Luijkx et al., 7 Aug 2025).

PIER determines sampling probabilities for tuples in $t$9 during training (Luijkx et al., 7 Aug 2025). It defines an age feature

$o_t^k \in \mathcal{O}$0

a prioritization exponent

$o_t^k \in \mathcal{O}$1

and a priority score

$o_t^k \in \mathcal{O}$2

which corresponds to Eq. (4) in the paper (Luijkx et al., 7 Aug 2025). The resulting sampling distribution is

$o_t^k \in \mathcal{O}$3

and importance weights are

$o_t^k \in \mathcal{O}$4

to correct bias from non-uniform replay (Luijkx et al., 7 Aug 2025).

The stated training effect is that recent failures at low uncertainty receive the highest priority, since they indicate confident mistakes that should be corrected quickly, whereas old low-uncertainty successes receive low priority because they correspond to already mastered behavior (Luijkx et al., 7 Aug 2025). Relabeled and offline demonstrations remain at priority $o_t^k \in \mathcal{O}$5, which prevents them from being ignored or overemphasized (Luijkx et al., 7 Aug 2025).

6. Uncertainty estimation, experiments, and empirical findings

ASkDAgger is agnostic to the particular uncertainty estimator, requiring only an operator $o_t^k \in \mathcal{O}$6 (Luijkx et al., 7 Aug 2025). In the MNIST experiments, uncertainty is computed with Monte Carlo Dropout using $o_t^k \in \mathcal{O}$7 stochastic forward passes: $o_t^k \in \mathcal{O}$8 with dropout rate $o_t^k \in \mathcal{O}$9, batch size $g^k \in \mathcal{G}$0, and $g^k \in \mathcal{G}$1 time steps per run, querying when uncertainty exceeds the SAG threshold or with probability $g^k \in \mathcal{G}$2 (Luijkx et al., 7 Aug 2025). In CLIPort experiments, uncertainty is the prediction entropy over pixel-wise Q-maps, while in the affordance-learning comparison a least-confidence measure is used: $g^k \in \mathcal{G}$3 (Luijkx et al., 7 Aug 2025).

The principal robotic simulation benchmark consists of four CLIPort language-conditioned manipulation tasks: packing-google-objects-seq, packing-google-objects-group, packing-shapes, and put-blocks-in-bowls (Luijkx et al., 7 Aug 2025). Training uses $g^k \in \mathcal{G}$4 runs per method and $g^k \in \mathcal{G}$5 interactive demonstrations, with checkpoints evaluated every $g^k \in \mathcal{G}$6 demonstrations (Luijkx et al., 7 Aug 2025). In these experiments, SAG is configured with mode $g^k \in \mathcal{G}$7 sensitivity, $g^k \in \mathcal{G}$8, $g^k \in \mathcal{G}$9, and $a_t^k \in \mathcal{A}$0, while PIER uses $a_t^k \in \mathcal{A}$1, $a_t^k \in \mathcal{A}$2, $a_t^k \in \mathcal{A}$3, and $a_t^k \in \mathcal{A}$4 (Luijkx et al., 7 Aug 2025). The compared methods are ASkDAgger, an Active DAgger baseline defined as ASkDAgger without FIER and PIER, SafeDAgger, ThriftyDAgger, and several ablations (Luijkx et al., 7 Aug 2025).

The experimental findings are organized around three claims. First, SAG tracks desired operating characteristics: in MNIST, for $a_t^k \in \mathcal{A}$5, it tracks sensitivity and specificity accurately, and in success-aware mode the overall system success rate follows the desired minimum (Luijkx et al., 7 Aug 2025). In CLIPort tasks and real assembly, sensitivity is maintained near $a_t^k \in \mathcal{A}$6 throughout training (Luijkx et al., 7 Aug 2025). Second, FIER reduces the need for annotation demonstrations while improving generalization: ASkDAgger collects fewer annotation demonstrations than Active DAgger and the reported baselines, and on unseen tasks it significantly outperforms them (Luijkx et al., 7 Aug 2025). The paper reports that with relabeling, ASkDAgger achieves on unseen CLIPort tasks an average evaluation reward improvement of about $a_t^k \in \mathcal{A}$7 over its no-relabeling variant (Luijkx et al., 7 Aug 2025). Third, PIER improves adaptation under domain shift, particularly as the demonstration dataset grows (Luijkx et al., 7 Aug 2025).

The real-world experiments use a Franka Panda for engine assembly and a Boston Dynamics Spot for object sorting (Luijkx et al., 7 Aug 2025). In the assembly setting, the task is to insert colored 3D-printed bolts into specified locations on a 3D-printed engine block using an in-hand RealSense D405 RGB-D camera processed into a top-down projection and commands of the form “Insert the [color] bolt at location number [n]” (Luijkx et al., 7 Aug 2025). A Gradio GUI presents plans to the human, who validates, relabels, or rejects them (Luijkx et al., 7 Aug 2025). With $a_t^k \in \mathcal{A}$8 interactive demonstrations, the system shows the same qualitative pattern seen in simulation: a growing fraction of demonstrations become validations and relabelings while maintaining high system success rate (Luijkx et al., 7 Aug 2025). The reported approximate user interaction times are $a_t^k \in \mathcal{A}$9 s for validation, $\pi_\mathrm{T}$0 s for annotation, and $\pi_\mathrm{T}$1 s for relabeling; with environment overhead, validation and annotation demonstrations take roughly $\pi_\mathrm{T}$2 s and $\pi_\mathrm{T}$3 s respectively, while relabeling can often be provided during an annotation interaction without additional execution (Luijkx et al., 7 Aug 2025).

7. Extensions, limitations, and the ASkDAgger lens in shared low-level control

ASkDAgger is explicitly designed for moderate query frequencies and high-level skills, not for raw torque control or very high-frequency decisions (Luijkx et al., 7 Aug 2025). The framework assumes that the teacher can interpret proposed skill plans, judge validity, and supply relabelings without excessive cognitive burden, and that uncertainty is sufficiently calibrated to correlate with failure (Luijkx et al., 7 Aug 2025). In success-aware mode, the expert is assumed always correct (Luijkx et al., 7 Aug 2025). Relabeling is straightforward in structured manipulation tasks where alternative goal satisfaction is visually apparent, but the paper notes that it becomes harder in more abstract settings (Luijkx et al., 7 Aug 2025).

Within the broader literature, ASkDAgger is positioned at the intersection of active imitation learning, affordance learning, and RLHF-like interactive feedback, though it does not learn a reward model and instead performs behavior cloning at skill level with intelligent query and replay mechanisms (Luijkx et al., 7 Aug 2025). The authors describe possible extensions to long-horizon and non-sparse tasks, integration with foundation models for plan visualization or synthetic demonstrations, hierarchical control architectures, and settings with heterogeneous or imperfect teachers (Luijkx et al., 7 Aug 2025).

A distinct but informative extension of the ASkDAgger perspective is provided by "Humanoid Hanoi: Investigating Shared Whole-Body Control for Skill-Based Box Rearrangement" (Kim et al., 14 Feb 2026). That work studies a skill-based hierarchy for long-horizon humanoid box rearrangement in which all skills share a single task-agnostic whole-body controller (WBC), and observes that naïve reuse of the same pretrained WBC causes distribution shift as new skills and their compositions generate new state and command distributions (Kim et al., 14 Feb 2026). The proposed remedy is a rollout-based data aggregation procedure: execute the hierarchical system in closed loop, record the directive trajectories sent from skill policies to the WBC during successful rollouts,

$\pi_\mathrm{T}$4

expand the reference set by

$\pi_\mathrm{T}$5

and continue PPO training on the enlarged directive distribution using the same task-agnostic objective with domain randomization and disturbance injection (Kim et al., 14 Feb 2026).

That paper does not present the human-interactive ASkDAgger algorithm of SAG, FIER, and PIER; however, it explicitly frames its procedure as “DAgger for the low-level shared controller” and as a concrete instance of Active Skill-level Data Aggregation for a shared low-level controller (Kim et al., 14 Feb 2026). The aggregation occurs at WBC command level rather than at the level of teacher-labeled novice plans, and there is no external expert because the corrective signal is implicit in the original tracking-and-stability objective (Kim et al., 14 Feb 2026). The reported results show that this coverage-expansion approach improves long-horizon robustness in Humanoid Hanoi, with the “Extended” shared WBC achieving approximately $\pi_\mathrm{T}$6 final success without dynamics randomization and $\pi_\mathrm{T}$7 with dynamics randomization, outperforming base, residual, and per-skill finetuned alternatives in the long-horizon setting (Kim et al., 14 Feb 2026). A plausible implication is that the term “ASkDAgger” now names both a specific interactive imitation learning framework centered on teacher feedback at the skill level and a broader methodological viewpoint in which skill execution actively drives data aggregation to mitigate distribution shift in hierarchical control systems.