---
title: Action Discovery in AI & Robotics
url: https://www.emergentmind.com/topics/action-discovery
type: topic
---

# Action Discovery in AI & Robotics

Action discovery is the problem of inferring useful action representations, action sets, or action structure from data rather than assuming a fixed action vocabulary. In contemporary research, the discovered entity varies by domain: it may be a symbolic manipulation primitive in robotics, a high-level action abstraction or macro-action in reinforcement learning, a subset of causal action dimensions in continuous control, an unknown action class in partially annotated video, an action concept grounded in image–sentence corpora, a stochastic action-order graph in process mining, or a STRIPS action schema in classical planning [2505.18248][2410.15184][2201.00354][2508.05529][1604.04784][2312.05433][2001.11457]. Across these settings, the common objective is to replace hand-specified action vocabularies with discovered structures that improve prediction, exploration, planning, interpretability, or transfer.

## 1. Scope and problem formulations

Action discovery is not a single task but a family of problems defined by what counts as an “action” and what information is available during discovery. In robotic manipulation, the target is often an executable primitive grounded in continuous control parameters. One recent formulation learns an encoder–decoder effect predictor over object features \(o \in \mathbb{R}^4\), continuous actions \(a \in \mathbb{R}^{12}\), and effects \(e \in \mathbb{R}^3\), then discretizes the action latent space into symbolic action primitives for planning [2505.18248]. In amortized sampling and long-horizon RL, the target can instead be a high-level action obtained by chunking repeated subsequences in sampled trajectories and adding them back to the action space [2410.15184]. In Meta-RL, a macro-action may be a latent variable \(z\) that summarizes a multi-step transition from \(s_t\) to \(s_{t+M}\) [2412.11930].

In other settings, the action is not a temporally extended control primitive but a structural variable. State-Wise Action Refined learning treats action discovery as a state-dependent selection problem over action coordinates, using a binary mask \(G(a \mid s)\in\{0,1\}^L\) to identify task-relevant dimensions and suppress nuisance dimensions [2201.00354]. In agentic LLMs with very large tool sets, action discovery is formulated as recovery of a small support set \(S^\star\subseteq\mathcal{A}\) of relevant actions under a block-sparse contextual linear reward model [2601.08280].

Observation-driven formulations are equally diverse. Unsupervised semantic action discovery from instructional videos seeks recurring latent activity steps shared across videos, with a Beta Process Hidden Markov Model deciding which steps are present, where they occur, and how they transition [1605.03324]. Video novel category discovery addresses partially labeled action spaces in which known actions are labeled and unknown actions must be clustered as disjoint novel classes [2304.07354]. Temporal Action Segmentation has also adopted an “Action Discovery” setting in which unknown actions are labeled as background during training and must later be segmented and assigned semantic labels [2508.05529].

Planning and process modeling introduce still different formulations. Process discovery from event logs infers observable actions and directly-follows relations, represented as nodes and arcs in a Stochastic Directed Action Graph with stochastic-language semantics [2312.05433]. STRIPS action discovery infers action signatures, preconditions, and add/delete effects from traces containing only initial and goal states, with no observed actions or intermediate states [2001.11457].

| Setting | What is discovered | Representative approach |
|---|---|---|
| Robotics and manipulation | Symbolic primitives, affordances | Latent binarization, uncertainty-guided exploration |
| RL and Meta-RL | Action abstractions, macro-actions, causal action dimensions | Chunking, VAE-style latent actions, state-wise masks |
| Vision and video | Semantic steps, unknown action classes, action concepts | BP-HMM, clustering, partial-label discovery |
| Planning and process modeling | Action schemas, action-order graphs | Compilation to planning, grammatical inference |
| Large-action agents | Relevant action subset | Block-sparse recovery |

This diversity suggests that action discovery is best understood as a representational problem coupled to an execution or inference problem. The discovered object must not only partition data; it must support downstream use.

## 2. Representation learning and abstraction mechanisms

A central design choice is the level at which actions are abstracted. In manipulation, one approach learns separate object and action encoders, concatenates their latent embeddings, and predicts a Gaussian effect model
\[
p(e \mid a,o)=\mathcal{N}(e;\mu(a,o),\sigma^2(a,o)).
\]
Action symbols emerge by binarizing the action encoder output,
\[
z_a=b(\phi_a(a))\in\{0,1\}^k,
\]
so continuous motor parameters are converted into discrete bit vectors. With \(k=3\), the learned action space admits up to \(2^3=8\) symbols [2505.18248]. This mechanism treats action discovery as latent discretization constrained by effect prediction.

A different abstraction mechanism appears in ActionPiece, where actions are discovered online from trajectories by Byte Pair Encoding over action sequences. Frequent adjacent primitive pairs are merged into composite tokens, and the policy’s action space is augmented with these chunks. The policy handles the changing action set through action embeddings rather than a fixed output layer, with logits computed by similarity between a query vector and learned action embeddings; the action encoder itself is an LSTM over the primitive-action sequence comprising a chunk [2410.15184]. Here, discovery is explicitly compositional: abstractions are induced from repeated subsequences rather than latent binarization.

HiMeta uses a modified VAE-style latent-variable model to discover a macro-action \(z_t\) from the current state \(s_t\), task representation \(y_t\), and a future goal state. The encoder samples
\[
z_t \sim \tanh\big(\mathcal{N}(\psi_{z_\mu}(y_t,s_t),\psi_{z_\sigma}(y_t,s_t))\big),
\]
and the decoder predicts a future ego-state \(s^{ego}_{t+M}\). Because the decoder is trained only on \(\mathcal{S}_{ego}\), not the full state \(\mathcal{S}\), the discovered macro-actions are intended to be task-agnostic and reusable across tasks [2412.11930]. This suggests a distinct criterion for action abstraction: invariance to task-specific state components.

In continuous control, abstraction can take the form of dimension selection rather than temporal extension. SWAR defines a refined action
\[
a^{(G(a\mid s))}=G(a\mid s)\odot a
\]
and learns the selector \(G\) to preserve TD or dynamics prediction while minimizing the number of selected coordinates. The discovered “action” is therefore a state-wise causal subset of the full action vector, not a new primitive in the conventional sense [2201.00354].

Observation-driven work also relies on latent structures. Instructional video discovery represents each frame as a binary multimodal vector over language atoms and visual atoms, then uses a Beta Process Hidden Markov Model with latent step indicators \(f_k^{(i)}\), state assignments \(z_t^{(i)}\), and Bernoulli emissions to discover shared activity steps without fixed cardinality [1605.03324]. In still images, ACD extracts verb–object pairs, filters them by visualness with a linear SVM on VGG-16 features, and clusters the survivors using a multimodal embedding that concatenates visual and linguistic representations,
\[
V_{concept}=\alpha V_{visual}\oplus(1-\alpha)V_{linguistic},
\]
with cosine similarity for cluster formation [1604.04784].

These methods differ in supervision, modality, and inductive bias, but all impose a structural bottleneck between raw observations and the final action representation. That bottleneck may be binary, sequential, variational, or clustering-based; in each case it is the locus of discovery.

## 3. Discovery through interaction, uncertainty, and exploration

A substantial branch of the literature treats action discovery as an active process in which the system must choose interactions that make useful actions observable. In symbolic manipulation, curiosity is defined directly from predictive uncertainty. For each sampled candidate action, the system evaluates the entropy of the predicted Gaussian effect distribution,
\[
H_j(a,o)=\frac{1}{2}\log\!\bigl(2\pi e\,\sigma_j^2(a,o)\bigr), \qquad
\bar H(a,o)=\frac{1}{3}\sum_{j=1}^{3}H_j(a,o),
\]
and executes the action with maximal mean entropy. On a UR10 robot in PyBullet simulation, exploration runs for 10,000 steps; 2,000 candidate actions are sampled at each step; retraining occurs every 512 steps for 10 epochs; and the learned primitives support 82% single-object planning success and 59% double-object planning success, compared with 56% and 38% for active learning and 13% and 9% for random exploration [2505.18248]. The reported result is not merely better prediction error; it is discovery of a richer set of executable primitives.

Information-Driven Affordance Discovery uses a closely related but distinct quantity: expected information gain over model parameters. With an ensemble of affordance predictors, the information gain of a context–action pair is
\[
I(x,a)=H\!\left(\mathbb{E}_{\theta\sim\Theta}[p(b\mid x,a,\theta)]\right)-\mathbb{E}_{\theta\sim\Theta}[H(p(b\mid x,a,\theta))],
\]
which is the Jensen–Shannon divergence of the ensemble predictive distributions. Actions are chosen by a UCB-style rule
\[
\arg\max_a \big[\hat r(x,a)+c_{\mathrm{expl}}\,I(x,a)\big].
\]
The method discovers visual affordances for grasping, stacking, and opening, improves data efficiency in ManiSkill2 simulation, and reaches up to 90% grasp success on a UFACTORY XArm 6 real-world setup [2405.03865].

Interactive perception introduces another interpretation of discovery: selecting actions that reveal latent structure. Structure from Action predicts informative 3D hold–push interactions from the current RGB point cloud and a history volume \(\mathcal{H}_t\), then uses the resulting motion to discover object parts and joints. The interaction policy prefers holds on no-flow regions and pushes on points with large predicted flow, so action choice is explicitly tied to part isolation and articulation inference [2207.08997]. The discovered object structure depends on action selection; action discovery and structure discovery are therefore coupled.

Sparse action discovery in agentic LLMs replaces uncertainty-guided exploration with statistical recovery guarantees. Under a contextual linear reward model \(r_t=\langle W^\star_{a_t},z_t\rangle+\varepsilon_t\) and row sparsity \(W^\star_a=0\) for all \(a\notin S^\star\), Contextual Block-OMP iteratively selects the action block most correlated with the residual and refits least squares. Exact support recovery holds with high probability when \(T\gtrsim kd\log M\), under coverage, block incoherence, and minimum signal-strength conditions [2601.08280]. This formulation is theoretical rather than interactive in the robotic sense, but it shares the same premise: only a small subset of actions is informative.

A common misconception is that action discovery is equivalent to passive clustering of behavioral data. The literature includes a large class of methods in which discovery is inseparable from targeted intervention, uncertainty estimation, or support recovery.

## 4. Discovery from observation, partial supervision, and traces

Observation-driven action discovery addresses settings in which the system cannot intervene directly, or where the relevant actions are only partially labeled. In instructional videos, the objective is to recover recurring semantic steps from large video collections without step labels, object labels, or temporal boundaries. The multimodal BP-HMM of “Unsupervised Semantic Action Discovery from Video Collections” discovers a latent inventory of activity steps and their temporal alignments, and its full model reports \(IOU_{csm}=52.36\), \(mAP_{csm}=44.09\), and \(mAP_{sem}=39.01\), outperforming HMM and kernel temporal segmentation baselines [1605.03324]. The discovered storyline is a sequence of objective steps rather than low-level motion primitives.

Partially labeled video recognition introduces a different challenge: the unlabeled set contains unknown-but-disjoint action classes. NEV-NCD addresses this with supervised cross-entropy on labeled classes, negative learning on unlabeled clips,
\[
\mathcal{L}_{nl}=-\mathbb{E}_{x_i\in\mathcal{D}^u}\Big[\sum_{j=0}^{L-1}\log(1-\hat y_{ij})\Big],
\]
entropy minimization, and variance regularization to enforce equipartition among novel categories. On UCF101, NEV-NCD reports 92.4% labeled top-1 accuracy and 82.7% clustering accuracy on unlabeled data [2304.07354]. The method does not discover temporal segments; it discovers novel action categories in an open-world video recognition setting.

Temporal Action Segmentation extends partial labeling to frame-wise discovery. In the Action Discovery setting, only known actions are annotated during training; unknown actions are labeled as UNK or background. The proposed pipeline first trains an MS-TCN backbone on known classes plus UNK, then applies the Granularity-Guided Segmentation Module to refine unknown intervals by matching the granularity of known annotations, and finally applies Unknown Action Segment Assignment to cluster segment means and estimate the number of unknown classes by BIC. On DesktopAssembly, the combined GGSM+UASA method reports unknown-action MoF 51.3, Edit 52.8, and F1@10/25/50 of 62.5/58.0/45.4, versus 39.9, 17.4, and 25.6/25.6/14.2 for the baseline [2508.05529].

Weakly supervised action discovery from image–sentence corpora operates at the concept level rather than the temporal level. ACD extracts candidate verb–object pairs, removes terms with frequency below \(k=30\), verifies visualness by cross-validated SVM AP, retains 171 action concepts from 327 candidates, and clusters them into about 84 multimodal concept clusters. On PASCAL VOC 2012 action classification, ACD-A reaches 54.3 mAP, compared with 48.3 for ACD and 44.2 for a keyword-based weak baseline [1604.04784].

Process mining offers a trace-centric analogue. Starting from a finite multiset of positive traces, stochastic directly-follows process discovery infers a Stochastic Directed Action Graph
\[
(N,\actions,\beta,\gamma,q,i,o),
\]
where nodes are labeled by observable actions, arcs encode execution order, and outgoing probabilities are normalized. The model has stochastic-language semantics and can represent trace likelihoods as sums of execution probabilities [2312.05433]. Here, action discovery means recovering action-order structure and frequency-aware semantics from logs rather than labeling perceptual data.

These observation-driven formulations show that action discovery can target classes, segments, concepts, or trace languages. The shared element is latent structure induced from incomplete or weak supervision.

## 5. Planning, execution, and downstream use

Discovered actions are valuable only to the extent that they support downstream reasoning or execution. In symbolic manipulation, the action symbols learned from binarized latent codes are used in breadth-first search over sequences of discrete primitives. Because execution still requires continuous controls, the paper introduces parameter distillation: the action encoder is frozen and continuous action parameters are optimized by SGD so that the encoder output matches a target binary symbol [2505.18248]. This closes the loop from continuous actions to symbols and back to executable motor commands.

ActionPiece uses discovered chunks as actual actions during training. When a chunk is selected, its constituent primitive actions are executed in order, unless masked by invalidity at the current state. The framework reports improved mode discovery and density estimation for GFlowNets, including an increase from about 43.24 to 48.67 discovered modes on L14_RNA1 for GFlowNet with ActionPiece-Increment + BPE, and a graph-task ELBO gap improvement from 0.72 to 0.25 for Atomic versus ActionPiece-Increment [2410.15184]. The action discovery mechanism is therefore embedded directly in the sampler’s optimization loop.

HiMeta conditions the primitive policy on both task representation \(y_t\) and macro-action \(z_t\),
\[
a_t=\pi_\theta(y_t,z_t,s_t),
\]
and adds an intrinsic reward that encourages sign agreement between macro-action and primitive action. In MetaWorld ML10, the reported average success is \(0.64\pm0.07\) on train and \(0.13\pm0.11\) on test for HiMeta-CD, versus \(0.42\pm0.09\) and \(0.05\pm0.06\) for SD [2412.11930]. The discovered macro-actions thus function as guidance variables for low-level control and cross-task adaptation.

In planning, STRIPS action discovery compiles model synthesis into a classical planning problem. The learner searches over the number of action schemas \(k\), maximum arity \(r\), and trace set size \(m\), edits partially defined actions by removing preconditions and adding effects, then validates the resulting model across all traces. The compilation is sound and complete, and the learned models achieve 30/30 coverage on held-out validation instances in hanoi, blocks, and visitall [2001.11457]. The discovered action theory is useful precisely because it generalizes to unseen planning problems.

Process mining makes a similar downstream claim for frequency-aware action models. Because SDAGs are obtained from SDFAs with preserved stochastic-language semantics, they support reasoning about likely future actions, simulation, and trace frequencies, not just visualization of directly-follows relations [2312.05433]. Sparse action discovery for agentic LLMs adds a decision-theoretic version of the same idea: once the support \(\widehat S\) is recovered, the plug-in policy optimizes only over discovered relevant actions and is near-optimal on new latent states, with decision loss bounded by parameter estimation error [2601.08280].

A plausible implication is that action discovery is increasingly evaluated by its operational consequences rather than by intrinsic cluster quality alone. Executability, compositionality, support recovery, and planning performance have become central criteria.

## 6. Assumptions, limitations, and open directions

The literature is explicit that discovered actions are shaped by modeling assumptions. Symbolic manipulation work assumes known object features, a fixed 12D action parameterization, a single-target object effect signal during learning, and simulation-based evaluation; the breadth-first planner is workable on small benchmarks but may become slow as the number of objects or task complexity grows [2505.18248]. Information-driven affordance discovery assumes hand-designed primitive families, binary success labels, invalid-action masking, and short-horizon interaction [2405.03865]. Structure from Action assumes that one joint is activated per interaction step and does not estimate other physical parameters such as friction [2207.08997].

In RL, action abstractions can improve exploration but introduce tradeoffs. ActionPiece improves GFlowNet mode discovery across all tested tasks, but for RL methods chunking can increase reward while decreasing diversity, suggesting over-specialized chunks and a negative feedback loop toward a small region of the state space [2410.15184]. SWAR identifies task-related action dimensions, but TD-SWAR is less stable in higher-dimensional tasks because TD targets move during training, while Dyn-SWAR models only the dynamics subset rather than the full union of reward- and dynamics-relevant actions [2201.00354]. HiMeta mitigates tri-level instability with independently tailored training schemes, but its effectiveness depends on the ego-state decomposition and the chosen goal-state construction [2412.11930].

Observation-driven discovery inherits limitations from supervision and representation quality. Unsupervised instructional-video discovery assumes a collection of videos from the same broad category with recurring latent steps, and some steps that always co-occur may be merged [1605.03324]. NEV-NCD assumes a known number of novel classes and benefits from large batch sizes for the variance term [2304.07354]. Temporal Action Discovery depends on the quality of learned embeddings and on hyperparameters such as \(\lambda\), \(\alpha\), and \(\tau\); the authors also note a tradeoff in which GGSM can reduce known-action performance while improving unknown discovery [2508.05529]. ACD depends on descriptive captions, verb–object extraction, and heuristic thresholds such as the AP \(>70\%\) visualness criterion and the frequency cutoff \(k=30\) [1604.04784].

Symbolic and trace-based discovery methods face their own constraints. STRIPS Action Discovery assumes a noise-free, offline setting with known predicates and objects, but no action observations or intermediate states [2001.11457]. Stochastic directly-follows discovery relies on positive examples only and uses a multi-objective genetic algorithm to tune inference parameters toward small, frequency-faithful models [2312.05433]. Sparse action discovery for agentic LLMs is established under incoherence, coverage, and minimum signal-strength assumptions; the theory also shows that without sparsity or sufficient coverage, sample complexity must grow at least linearly in the number of actions in simplified settings [2601.08280].

A common misconception is that action discovery always aims to recover a single, human-readable action vocabulary. The surveyed work instead supports multiple end states: executable primitives, reusable chunks, latent macro-actions, causal action subsets, unknown semantic classes, stochastic action graphs, and planning schemas. This suggests that future work will continue to treat “action” as a domain-relative construct, with discovery methods judged by how well the resulting construct supports the surrounding system’s objectives.

Source: https://www.emergentmind.com/topics/action-discovery