---
title: Watch-Try-Learn (WTL) Meta-Learning
url: https://www.emergentmind.com/topics/watch-try-learn-wtl
type: topic
---

# Watch-Try-Learn (WTL) Meta-Learning

Searching arXiv for the cited WTL-related papers to ground the article with current references.
Searching arXiv for "Watch, Try, Learn: Meta-Learning from Demonstrations and Reward".
Searching arXiv for "Tree based Progressive Regression Model for Watch-Time Prediction in Short-video Recommendation".
Searching arXiv for "Explicit Uncertainty Modeling for Video Watch Time Prediction".
Searching arXiv for "Watch an AI Weather Model Learn (and Unlearn) Tropical Cyclones".
Searching arXiv for "Watch and Learn: Learning to Use Computers from Online Videos".
Watch-Try-Learn (WTL) denotes, in its original and most formal sense, a meta-learning framework for few-shot control in which an agent first **watches** one or a few demonstrations, then **tries** the task in the environment, and finally **learns** from the combination of demonstrations and sparse reward feedback to improve its policy [1906.03352]. In later work, the same tripartite logic has also been used as an interpretive lens rather than a fixed algorithm: recommendation systems treat watch behavior as a learnable signal, AI weather models are analyzed by watching checkpoint-wise skill acquisition and loss, and computer-use agents learn executable trajectories by watching human tutorial videos and inferring actions between screen states [2306.03392] [2504.07575] [2603.20541] [2510.04673].

## 1. Definition and conceptual scope

WTL was introduced for the setting in which a robot or agent must learn a new task from very little supervision, but demonstrations alone are not enough [1906.03352]. The motivating scenario is a few-shot control problem in which a human provides one or a few demonstrations, the task may remain ambiguous or partially specified, the agent is allowed one or more trial episodes in the environment, and the only trial feedback may be sparse binary reward. The central claim is that imitation alone can fail when the demonstration does not fully identify the task or the correct action under new conditions, whereas reinforcement learning alone is too exploration-heavy, especially in vision-based manipulation [1906.03352].

In the original formulation, the essential semantic decomposition is literal. “Watch” refers to conditioning on demonstrations. “Try” refers to executing the task and observing the outcome. “Learn” refers to using both the demonstrations and the trial trajectories, together with reward, to infer a better task-conditioned policy [1906.03352]. This differs from meta-imitation learning, which conditions only on demonstrations, and from meta-reinforcement learning, which depends on reward-driven exploration without the strong prior supplied by demonstrations [1906.03352].

A broader contemporary usage has emerged around the same structure. In short-video recommendation, WTL-style reasoning treats watch time as a key behavioral signal from which preference models should learn, even when the method is not explicitly branded as WTL [2306.03392] [2504.07575]. In AI weather, WTL describes checkpoint-wise observation of how a model learns and unlearns tropical cyclone structure and intensity [2603.20541]. In computer-use agents, “Watch & Learn” is described as a WTL-style pipeline that watches online tutorial videos, tries to infer intermediate actions through inverse dynamics, and then learns from the resulting executable trajectories [2510.04673]. This suggests that WTL has evolved from a specific robotics method into a more general research pattern for exploiting observational data, interventions, and downstream adaptation.

## 2. Original WTL formulation in meta-learning

The original WTL paper models each task \(T_i\) as an MDP
\[
\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},
\]
with continuous state space \(\mathcal{S}\), continuous action space \(\mathcal{A}\), reward function \(r_i\), unknown dynamics \(P_i(s_{t+1}\mid s_t,a_t)\), and horizon \(H\) [1906.03352]. Tasks are drawn from a distribution \(p(T)\), and the objective is rapid adaptation to a new unseen task using demonstrations plus one or a few trials.

A defining design choice is the use of two separate context-conditioned policies rather than a single policy updated in place. The first-stage demo-conditioned policy is
\[
\pi_\theta(a \mid s, \{d_{i,k}\}),
\]
and the second-stage re-trial policy is
\[
\pi_\phi(a \mid s, \{d_{i,k}\}, \{\tau_{i,\ell}\}),
\]
where \(d_{i,k}\) are demonstrations for task \(T_i\) and \(\tau_{i,\ell}\) are trial trajectories produced by the first policy [1906.03352]. The separation is important because it keeps the trial-data distribution stationary during training of the second stage.

The first-stage loss is a supervised imitation objective on held-out demonstration trajectories:
\[
L^{1}(\theta, \mathcal{D}_i^*) =
\mathbb{E}_{\{d_{i,k}\}\sim D_i^*}
\mathbb{E}_{d_i^\text{test} \sim D_i^*\setminus \{d_{i,k}\}}
\mathbb{E}_{(s_t, a_t)\sim d_i^\text{test}}
\left[-\log \pi_\theta(a_t \mid s_t,\{d_{i,k}\})\right],
\]
with the meta-objective
\[
\min_\theta \frac{1}{|\{T_i\}|}\sum_{T_i \in \{T_i\}} L^{1}(\theta, D_i^*).
\]
The second-stage loss conditions additionally on trial episodes:
\[
L^{2}(\phi, D_i, D_i^*) =
\mathbb{E}_{(\{d_{i,k}\}, \{\tau_{i,\ell}\}) \sim \mathcal{D}_i}
\mathbb{E}_{d_i^\text{test} \sim D_i^* \setminus \{d_{i,k}\}}
\mathbb{E}_{(s_t, a_t)\sim d_i^\text{test}}
\left[-\log \pi_\phi(a_t \mid s_t,\{d_{i,k}\},\{\tau_{i,\ell}\})\right],
\]
with
\[
\min_\phi \frac{1}{|\{T_i\}|}\sum_{T_i \in \{T_i\}} L^{2}(\phi, D_i, D_i^*).
\]
Because the loss is evaluated on held-out demonstration trajectories rather than simply replaying successful trials, the second policy is trained to generalize from trial outcomes rather than memorize them [1906.03352].

Architecturally, the vision-based gripper setting uses a 4-layer CNN with ReLU and layer norm, followed by a spatial softmax to extract 2D keypoints; these visual features are concatenated with gripper pose, gripper velocity, and a context embedding [1906.03352]. The actor predicts a Gaussian mixture model over actions comprising end-effector position, axis-angle orientation, and finger angle [1906.03352]. For the context embedding, the model applies a vision network to each frame in the demonstration and trial trajectories, concatenates the demo and trial feature matrices, concatenates trial rewards as additional features, applies a 1D convolution along time, flattens, and feeds the result through an MLP [1906.03352]. The first policy uses the same structure without trial features and trial rewards.

The training procedure is explicitly decoupled. First, \(\theta\) is trained from demonstrations only. Then \(\theta\) is frozen and rolled out to collect trial data for each meta-training task. Finally, \(\phi\) is trained on the demo-trial pairs [1906.03352]. This decoupling avoids instability that would arise if both policies were trained jointly while the trial-data distribution was changing.

## 3. Ambiguity, reward, and empirical results in robotics

The original motivation for WTL is that demonstrations may be ambiguous or insufficiently informative. The paper explicitly highlights settings with ambiguous grasp points, unclear force or trajectory requirements, unknown dynamics, and partial observability of task intent from one demonstration [1906.03352]. WTL addresses this by using the first policy to explore according to a demo-conditioned hypothesis, then using environment response and sparse reward to refine that hypothesis via the second policy. Failed trials are treated as informative negative evidence rather than discarded noise.

The reacher environment is designed to isolate this logic. It is a 2D reaching task with two target objects, 2-DOF arm control, and randomized dynamics in which each joint may have reversed orientation with 50% probability [1906.03352]. The demonstrations indicate which object to reach, but not necessarily how the action mapping changes under the new dynamics. The setup uses 2 demonstrations per task, 10,000 meta-training tasks, and 1,000 meta-test tasks [1906.03352]. The paper reports that WTL learns to imitate the expert successfully after one demo and one trial, whereas methods without trial feedback struggle [1906.03352].

The more demanding evaluation is a 3D gripper manipulation environment with four task families—button pressing, grasping, pushing/sliding, and pick and place—each with two task variants [1906.03352]. The action space is 7-D, rewards are sparse and binary, maximum episode length is 5 seconds or 50 steps, and both vision-based and state-based policies are evaluated [1906.03352]. The dataset contains 1,536 demonstrations, 768 distinct tasks, 96 kitchenware object sets, and 40 held-out meta-validation tasks plus 40 held-out meta-test tasks [1906.03352].

The main reported state-space success rates are \(0.09 \pm 0.01\) for behavior cloning, \(0.30 \pm 0.02\) for meta-imitation learning, and \(0.42 \pm 0.02\) for WTL with 1 trial [1906.03352]. In comparison with BC+SAC, the paper reports that RL fine-tuning requires thousands of trials per task to approach WTL’s performance, with example values of \(0.11 \pm 0.07\) at 1500 trials, \(0.29 \pm 0.10\) at 2000 trials, and \(0.39 \pm 0.11\) at 2500 trials [1906.03352]. The practical implication is that one demonstration plus one trial episode can outperform methods that require orders of magnitude more interaction.

The paper also notes limitations. Failed trial trajectories may be insufficiently informative if the first policy misses the correct object entirely; the exploration heuristic is fairly simple and described as posterior/Thompson-sampling-like; evaluation is on held-out objects within task families rather than maximally novel task families; and the method still requires a demonstration plus at least one trial [1906.03352]. A common misconception is therefore that WTL eliminates the need for exploration. The original evidence does not support that reading: it reduces the exploration burden by conditioning it on demonstrations.

## 4. Later reinterpretations across research areas

Subsequent work uses the watch-try-learn logic in ways that are conceptually related but algorithmically distinct. Some papers instantiate the triplet directly; others are better understood as WTL-style or WTL-compatible rather than formal descendants of the original robotics framework.

| Work | Domain | WTL interpretation |
|---|---|---|
| [2306.03392] | Short-video recommendation | Watch behavior is modeled more faithfully so the system can learn better recommendation signals |
| [2504.07575] | Video watch-time prediction | Explicit uncertainty modeling upgrades the watch-time learning pipeline |
| [2603.20541] | AI weather | Training checkpoints are watched to diagnose when storm-specific skills are learned or unlearned |
| [2510.04673] | Computer-use agents | Tutorial videos are watched and converted into executable UI trajectories through inverse dynamics |

This diversification changes the meaning of each component. In recommendation, “watch” refers to user engagement signals, especially watch time; “learn” refers to the prediction model and ranking pipeline [2306.03392] [2504.07575]. In AI weather, “watch” refers to observing checkpoint-wise evolution of task-specific performance; “try” is implicit in evaluating each checkpoint on storms; “learn” refers to diagnosing what the model acquires, stabilizes, or loses over training [2603.20541]. In computer-use agents, “watch” refers to online videos, “try” is operationalized as inverse-dynamics inference over adjacent screen states, and “learn” refers to using extracted trajectories as in-context demonstrations or supervised training data [2510.04673].

This expansion also clarifies what WTL is not. It is not a single universally shared architecture across these domains. Rather, the common structure is methodological: observe an informative signal, perform a task- or environment-grounded inference step, and use the result to improve prediction, control, or understanding. Where the later papers do not explicitly define WTL, the WTL reading should be treated as an interpretation rather than a formal naming claim [2306.03392] [2504.07575] [2510.04673].

## 5. WTL-style watch-time learning in recommendation systems

In short-video recommendation, watch time is treated as a central preference signal, and later work argues that modeling it as a simple point-regression target is inadequate. The tree-based progressive regression model (TPM) begins from four requirements for a watch-time prediction framework: preserve ordinal relations, model conditional dependencies across watch-time levels, represent uncertainty, and reduce bias amplification [2306.03392]. The paper’s motivating example is that two predictions can have the same absolute error while implying different ranking consequences; consequently, ordinal structure matters for recommendation, not only regression fidelity.

TPM quantizes watch time into ordinal ranks or intervals
\[
\{\gamma_0 \le \gamma_1 \le \cdots \le \gamma_m\},
\]
with each leaf corresponding to an interval \(l_k : [\gamma_k, \gamma_{k+1}]\) and each internal node corresponding to a union of consecutive ranks [2306.03392]. The model decomposes prediction into a tree of binary classification tasks, and the probability of reaching leaf \(l_k\) is expressed as a product of conditional probabilities along the root-to-leaf path:
\[
p(T\in l_k \mid X,\mathcal{T}) =
\prod_{1\le i\le d(l_k)}
p\Big(T\in \hat n_{\phi_{l_k}(i)} \mid X,\mathcal{T},T\in \hat n_{\phi_{l_k}(i-1)}\Big).
\]
The expected watch time is then computed as a weighted sum over leaf intervals, using the midpoint of each interval as the leaf-level conditional expectation [2306.03392]. The model also explicitly computes predictive variance,
\[
Var(T\mid X,\mathcal{T}) = E(T^2\mid X,\mathcal{T}) - E(T\mid X,\mathcal{T})^2,
\]
and includes a variance penalty in the objective:
\[
\max \mathcal{L} =
\alpha_1 \log p(\hat T\in l_k(T)\mid X,\mathcal{T})
-\alpha_2 Var(\hat T\mid X,\mathcal{T})^{0.5}
-\alpha_3 \|E(\hat T)-T\|_2.
\]
Backdoor adjustment is incorporated through conditioning on a confounder \(D\), with
\[
E(T\mid do(X)) = \sum_d P(D=d)\,E(T\mid X,D=d),
\]
to mitigate bias amplification, particularly duration bias [2306.03392].

Offline, TPM is evaluated on Kuaishou and CIKM16 against WLR, D2Q, and ordinal regression, using MAE and XAUC [2306.03392]. On Kuaishou, TPM reports MAE \(4.741\) and XAUC \(0.599\), compared with WLR \(6.047/0.525\), D2Q \(5.426/0.565\), and OR \(5.321/0.558\) [2306.03392]. On CIKM16, TPM reports MAE \(0.884\) and XAUC \(0.676\), compared with WLR \(0.998/0.672\), D2Q \(0.899/0.661\), and OR \(0.918/0.664\) [2306.03392]. Online deployment on Kuaishou uses 10 traffic buckets over 4 days, with Watch Time as a positive metric, Forward as a positive constraint metric, and Short View as a negative metric [2306.03392]. The reported relative improvements over baseline are Watch Time \(+0.246\%\), \(+0.210\%\), \(+0.234\%\), and \(+0.265\%\) across the four days, with corresponding Short View reductions of \(-0.312\%\), \(-0.139\%\), \(-0.110\%\), and \(-0.220\%\) [2306.03392]. The system is reported as deployed in Kuaishou with over 300 million DAUs [2306.03392]. The source material explicitly states that this paper is not framed in WTL terminology but strongly supports and complements WTL-style recommendation.

A complementary development is EXUM, an **EXplict Uncertainty Model** that wraps an existing watch-time backbone and makes uncertainty explicit rather than leaving it implicitly absorbed into the prediction head [2504.07575]. For each sample \(i\), the backbone prediction \(p_i\) is combined with a learned confidence \(c_i \in [0,1]\) to produce
\[
p_i' = c_i p_i + (1-c_i) y_i
\]
for quantile prediction, and an analogous confidence-modulated output for ordinal regression [2504.07575]. To prevent the trivial shortcut \(c_i \to 0\), which would make the model collapse toward the ground truth during training, EXUM adds an adversarial confidence maximization term
\[
\mathcal{L}_c = -\sum_i \log c_i,
\]
leading to
\[
\mathcal{L}_\text{QP+EXUM} = \mathcal{L}_\text{QP} + \lambda \mathcal{L}_c
\quad\text{and}\quad
\mathcal{L}_\text{OR+EXUM} = \mathcal{L}_\text{OR} + \lambda \mathcal{L}_c.
\]
The framework is deployed in the reranking stage of a large industrial short-video recommendation platform serving 300M+ daily active users, with the confidence head used only during training and no extra inference cost online [2504.07575]. In a 20% traffic, 5-day A/B test, the treatment group uses CREAD + EXUM and reports a watch-time improvement of about \(0.31\%\), with daily lifts of \(0.303\%\), \(0.310\%\), \(0.306\%\), \(0.298\%\), and \(0.325\%\), while like/follow/comment changes are small and within variance [2504.07575]. Offline, the framework consistently improves D2Q and CREAD backbones on WeChat and KuaiRand, with especially large gains for CREAD-EXUM on KuaiRand [2504.07575]. In the source material, EXUM is presented as the “Learn” mechanism that upgrades a WTL-style watch-time pipeline.

## 6. Checkpoint watching, unlearning, and learning from videos

In AI weather modeling, WTL is instantiated as checkpoint-level analysis of how a model acquires or loses task-specific skill. The study uses a Spherical Fourier Neural Operator with scale factor 3 and embedding size 384, built from the open-source SFNO v0.1.0 implementation in NVIDIA’s modulus-makani repository, trained on ERA5 reanalysis at \(0.25^\circ\) resolution using 74 variables and 13 pressure levels plus solar zenith angle, orography, and land-sea mask [2603.20541]. Training runs for 90 epochs total—70 epochs with cosine annealing starting at \(10^{-3}\), followed by 20 epochs of 2-step fine-tuning with cosine annealing starting at \(10^{-4}\)—and the model weights are saved after each epoch, producing 90 checkpoints [2603.20541]. Every tropical cyclone is forecast with each checkpoint, so each storm has a 90-step learning trajectory.

The analysis defines a storm-relative domain as “a 1200km radius circle around the minimum MSLP value in the ERA5 valid time,” and evaluates intensity error as the difference between the forecasted minimum MSLP value and the ERA5 minimum MSLP, plus location error as the distance between the forecasted minimum MSLP and the true minimum MSLP [2603.20541]. A key result is that learning is not monotonic. For Hurricane Larry, location forecast improves steadily, but intensity behaves differently: at checkpoint 30 the forecast minimum MSLP error is only \(0.76\) hPa, whereas by checkpoints 70 and 89 the intensity error exceeds 18 hPa [2603.20541]. Across 106 storms, k-means clustering identifies three intensity-learning patterns: 30 storms with consistent underprediction, 53 storms with more conventional improvement, and 23 storms—including Larry—that improve early and then degrade, constituting the reported unlearning phenomenon [2603.20541]. Cluster 3 is associated with anomalously moist environments represented by 700 hPa relative humidity, and the paper cautiously suggests that the SFNO may be unlearning the relationship between moisture and tropical cyclone intensity rather than forgetting the moisture pattern itself [2603.20541]. A common misconception addressed by this result is that the best overall validation checkpoint must also be best for a specific high-impact downstream skill; the paper explicitly notes that this need not hold.

A distinct but related direction appears in computer-use agents through “Watch and Learn,” also described in the source material as a Watch-Try-Learn-style pipeline [2510.04673]. The central problem is the scarcity of large-scale, high-quality computer-use trajectories. The proposed solution is to convert raw human tutorial videos from the Internet into executable UI trajectories by framing the labeling problem as inverse dynamics on screen transitions:
\[
(O_t, a_t, O_{t+1}), \qquad
a_t \sim p(a \mid O_t, O_{t+1}).
\]
The inverse dynamics model is vision-only, using a SigLIP-2 vision encoder followed by 4 Transformer layers, with three heads: an action classification head over five primitives—click, scroll, type, wait, move—a coordinate head that predicts normalized \((x,y)\) coordinates discretized into integers from 0 to 1000, and a GPT-2 small decoder for typed strings [2510.04673]. It is trained on roughly 500k synthetic transitions plus 132k human-annotated transitions from Mind2Web, for over 630k triples [2510.04673].

On the Mind2Web test split, the reported action accuracies are 72.8% for Gemini 2.5 Flash, 82.7% for TongUI, and 91.6% for the W inverse dynamics model; action-type accuracies are 81.4%, 88.9%, and 96.4%, respectively [2510.04673]. After task-aware retrieval and filtering of tutorial videos, the pipeline produces 53,125 high-quality trajectories covering 69 applications across 7 categories, with category counts of 8,691 for Productivity, 12,829 for Programming, 7,948 for Design, 7,808 for Screen Editing, 5,206 for Audio Production, 4,601 for System Utilities, and 6,042 for Science/Data [2510.04673]. These trajectories are used both as in-context exemplars and as supervised fine-tuning data. On OSWorld, adding W trajectories improves Gemini 2.5 Flash from 19.0 to 22.0, OpenAI o3 from 21.8 to 24.3, Claude 4 Sonnet from 43.9 to 45.5, and Jedi from 50.6 to 52.8 [2510.04673]. For supervised fine-tuning, Qwen 2.5-VL 7B improves from 1.9 to 13.0 with W labels, while UI-TARS-7B improves from 27.3 to 31.1; TongUI labels help Qwen to 5.4 but degrade UI-TARS to 23.8 [2510.04673]. The paper’s stated limitations include a limited action space, weak scroll data, task granularity mismatch in retrieval, and difficulty in text-heavy or rare-action domains [2510.04673].

Taken together, these later works show that WTL has become a useful organizing concept for problems in which observation alone is insufficient, aggregate metrics are potentially misleading, and the learning system benefits from explicitly modeling the transition from passive signal to active inference. In the original robotics paper, this structure is formalized as a two-stage meta-learning procedure [1906.03352]. In recommendation, weather, and computer-use learning, the same structure reappears as a way to improve watch-time prediction, diagnose hidden training dynamics, or recover actionable trajectories from videos [2306.03392] [2504.07575] [2603.20541] [2510.04673].

Source: https://www.emergentmind.com/topics/watch-try-learn-wtl