Watch-Try-Learn (WTL) Meta-Learning

• Watch-Try-Learn (WTL) is a meta-learning framework that integrates demonstration observation, trial execution, and sparse reward feedback to address few-shot control challenges.
• In robotics, one demonstration plus one trial enables WTL to achieve a 42% success rate versus 9% for behavior cloning in ambiguous task settings.
• WTL’s methodology has been adapted to video recommendation, AI weather forecasting, and computer-use agents, illustrating its broad applicability.

Searching arXiv for the cited WTL-related papers to ground the article with current references. Searching arXiv for "Watch, Try, Learn: Meta-Learning from Demonstrations and Reward". Searching arXiv for "Tree based Progressive Regression Model for Watch-Time Prediction in Short-video Recommendation". Searching arXiv for "Explicit Uncertainty Modeling for Video Watch Time Prediction". Searching arXiv for "Watch an AI Weather Model Learn (and Unlearn) Tropical Cyclones". Searching arXiv for "Watch and Learn: Learning to Use Computers from Online Videos". Watch-Try-Learn (WTL) denotes, in its original and most formal sense, a meta-learning framework for few-shot control in which an agent first watches one or a few demonstrations, then tries the task in the environment, and finally learns from the combination of demonstrations and sparse reward feedback to improve its policy (Zhou et al., 2019). In later work, the same tripartite logic has also been used as an interpretive lens rather than a fixed algorithm: recommendation systems treat watch behavior as a learnable signal, AI weather models are analyzed by watching checkpoint-wise skill acquisition and loss, and computer-use agents learn executable trajectories by watching human tutorial videos and inferring actions between screen states (Lin et al., 2023, Wu et al., 10 Apr 2025, Baiman et al., 20 Mar 2026, Song et al., 6 Oct 2025).

1. Definition and conceptual scope

WTL was introduced for the setting in which a robot or agent must learn a new task from very little supervision, but demonstrations alone are not enough (Zhou et al., 2019). The motivating scenario is a few-shot control problem in which a human provides one or a few demonstrations, the task may remain ambiguous or partially specified, the agent is allowed one or more trial episodes in the environment, and the only trial feedback may be sparse binary reward. The central claim is that imitation alone can fail when the demonstration does not fully identify the task or the correct action under new conditions, whereas reinforcement learning alone is too exploration-heavy, especially in vision-based manipulation (Zhou et al., 2019).

In the original formulation, the essential semantic decomposition is literal. “Watch” refers to conditioning on demonstrations. “Try” refers to executing the task and observing the outcome. “Learn” refers to using both the demonstrations and the trial trajectories, together with reward, to infer a better task-conditioned policy (Zhou et al., 2019). This differs from meta-imitation learning, which conditions only on demonstrations, and from meta-reinforcement learning, which depends on reward-driven exploration without the strong prior supplied by demonstrations (Zhou et al., 2019).

A broader contemporary usage has emerged around the same structure. In short-video recommendation, WTL-style reasoning treats watch time as a key behavioral signal from which preference models should learn, even when the method is not explicitly branded as WTL (Lin et al., 2023, Wu et al., 10 Apr 2025). In AI weather, WTL describes checkpoint-wise observation of how a model learns and unlearns tropical cyclone structure and intensity (Baiman et al., 20 Mar 2026). In computer-use agents, “Watch & Learn” is described as a WTL-style pipeline that watches online tutorial videos, tries to infer intermediate actions through inverse dynamics, and then learns from the resulting executable trajectories (Song et al., 6 Oct 2025). This suggests that WTL has evolved from a specific robotics method into a more general research pattern for exploiting observational data, interventions, and downstream adaptation.

2. Original WTL formulation in meta-learning

The original WTL paper models each task $T_i$ as an MDP

$\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$

with continuous state space $\mathcal{S}$, continuous action space $\mathcal{A}$, reward function $r_i$, unknown dynamics $P_i(s_{t+1}\mid s_t,a_t)$, and horizon $H$ (Zhou et al., 2019). Tasks are drawn from a distribution $p(T)$, and the objective is rapid adaptation to a new unseen task using demonstrations plus one or a few trials.

A defining design choice is the use of two separate context-conditioned policies rather than a single policy updated in place. The first-stage demo-conditioned policy is

$\pi_\theta(a \mid s, \{d_{i,k}\}),$

and the second-stage re-trial policy is

$$\pi_\phi(a \mid s, \{d_{i,k}\}, \{\tau_{i,\ell}\}),$$

where $\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$0 are demonstrations for task $\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$1 and $\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$2 are trial trajectories produced by the first policy (Zhou et al., 2019). The separation is important because it keeps the trial-data distribution stationary during training of the second stage.

The first-stage loss is a supervised imitation objective on held-out demonstration trajectories: $\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$3 with the meta-objective

$\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$4

The second-stage loss conditions additionally on trial episodes: $\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$5 with

$\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$6

Because the loss is evaluated on held-out demonstration trajectories rather than simply replaying successful trials, the second policy is trained to generalize from trial outcomes rather than memorize them (Zhou et al., 2019).

Architecturally, the vision-based gripper setting uses a 4-layer CNN with ReLU and layer norm, followed by a spatial softmax to extract 2D keypoints; these visual features are concatenated with gripper pose, gripper velocity, and a context embedding (Zhou et al., 2019). The actor predicts a Gaussian mixture model over actions comprising end-effector position, axis-angle orientation, and finger angle (Zhou et al., 2019). For the context embedding, the model applies a vision network to each frame in the demonstration and trial trajectories, concatenates the demo and trial feature matrices, concatenates trial rewards as additional features, applies a 1D convolution along time, flattens, and feeds the result through an MLP (Zhou et al., 2019). The first policy uses the same structure without trial features and trial rewards.

The training procedure is explicitly decoupled. First, $\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$7 is trained from demonstrations only. Then $\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$8 is frozen and rolled out to collect trial data for each meta-training task. Finally, $\{\mathcal{S}, \mathcal{A}, r_i, P_i, H\},$9 is trained on the demo-trial pairs (Zhou et al., 2019). This decoupling avoids instability that would arise if both policies were trained jointly while the trial-data distribution was changing.

3. Ambiguity, reward, and empirical results in robotics

The original motivation for WTL is that demonstrations may be ambiguous or insufficiently informative. The paper explicitly highlights settings with ambiguous grasp points, unclear force or trajectory requirements, unknown dynamics, and partial observability of task intent from one demonstration (Zhou et al., 2019). WTL addresses this by using the first policy to explore according to a demo-conditioned hypothesis, then using environment response and sparse reward to refine that hypothesis via the second policy. Failed trials are treated as informative negative evidence rather than discarded noise.

The reacher environment is designed to isolate this logic. It is a 2D reaching task with two target objects, 2-DOF arm control, and randomized dynamics in which each joint may have reversed orientation with 50% probability (Zhou et al., 2019). The demonstrations indicate which object to reach, but not necessarily how the action mapping changes under the new dynamics. The setup uses 2 demonstrations per task, 10,000 meta-training tasks, and 1,000 meta-test tasks (Zhou et al., 2019). The paper reports that WTL learns to imitate the expert successfully after one demo and one trial, whereas methods without trial feedback struggle (Zhou et al., 2019).

The more demanding evaluation is a 3D gripper manipulation environment with four task families—button pressing, grasping, pushing/sliding, and pick and place—each with two task variants (Zhou et al., 2019). The action space is 7-D, rewards are sparse and binary, maximum episode length is 5 seconds or 50 steps, and both vision-based and state-based policies are evaluated (Zhou et al., 2019). The dataset contains 1,536 demonstrations, 768 distinct tasks, 96 kitchenware object sets, and 40 held-out meta-validation tasks plus 40 held-out meta-test tasks (Zhou et al., 2019).

The main reported state-space success rates are $\mathcal{S}$0 for behavior cloning, $\mathcal{S}$1 for meta-imitation learning, and $\mathcal{S}$2 for WTL with 1 trial (Zhou et al., 2019). In comparison with BC+SAC, the paper reports that RL fine-tuning requires thousands of trials per task to approach WTL’s performance, with example values of $\mathcal{S}$3 at 1500 trials, $\mathcal{S}$4 at 2000 trials, and $\mathcal{S}$5 at 2500 trials (Zhou et al., 2019). The practical implication is that one demonstration plus one trial episode can outperform methods that require orders of magnitude more interaction.

The paper also notes limitations. Failed trial trajectories may be insufficiently informative if the first policy misses the correct object entirely; the exploration heuristic is fairly simple and described as posterior/Thompson-sampling-like; evaluation is on held-out objects within task families rather than maximally novel task families; and the method still requires a demonstration plus at least one trial (Zhou et al., 2019). A common misconception is therefore that WTL eliminates the need for exploration. The original evidence does not support that reading: it reduces the exploration burden by conditioning it on demonstrations.

4. Later reinterpretations across research areas

Subsequent work uses the watch-try-learn logic in ways that are conceptually related but algorithmically distinct. Some papers instantiate the triplet directly; others are better understood as WTL-style or WTL-compatible rather than formal descendants of the original robotics framework.

Work	Domain	WTL interpretation
(Lin et al., 2023)	Short-video recommendation	Watch behavior is modeled more faithfully so the system can learn better recommendation signals
(Wu et al., 10 Apr 2025)	Video watch-time prediction	Explicit uncertainty modeling upgrades the watch-time learning pipeline
(Baiman et al., 20 Mar 2026)	AI weather	Training checkpoints are watched to diagnose when storm-specific skills are learned or unlearned
(Song et al., 6 Oct 2025)	Computer-use agents	Tutorial videos are watched and converted into executable UI trajectories through inverse dynamics

This diversification changes the meaning of each component. In recommendation, “watch” refers to user engagement signals, especially watch time; “learn” refers to the prediction model and ranking pipeline (Lin et al., 2023, Wu et al., 10 Apr 2025). In AI weather, “watch” refers to observing checkpoint-wise evolution of task-specific performance; “try” is implicit in evaluating each checkpoint on storms; “learn” refers to diagnosing what the model acquires, stabilizes, or loses over training (Baiman et al., 20 Mar 2026). In computer-use agents, “watch” refers to online videos, “try” is operationalized as inverse-dynamics inference over adjacent screen states, and “learn” refers to using extracted trajectories as in-context demonstrations or supervised training data (Song et al., 6 Oct 2025).

This expansion also clarifies what WTL is not. It is not a single universally shared architecture across these domains. Rather, the common structure is methodological: observe an informative signal, perform a task- or environment-grounded inference step, and use the result to improve prediction, control, or understanding. Where the later papers do not explicitly define WTL, the WTL reading should be treated as an interpretation rather than a formal naming claim (Lin et al., 2023, Wu et al., 10 Apr 2025, Song et al., 6 Oct 2025).

5. WTL-style watch-time learning in recommendation systems

In short-video recommendation, watch time is treated as a central preference signal, and later work argues that modeling it as a simple point-regression target is inadequate. The tree-based progressive regression model (TPM) begins from four requirements for a watch-time prediction framework: preserve ordinal relations, model conditional dependencies across watch-time levels, represent uncertainty, and reduce bias amplification (Lin et al., 2023). The paper’s motivating example is that two predictions can have the same absolute error while implying different ranking consequences; consequently, ordinal structure matters for recommendation, not only regression fidelity.

TPM quantizes watch time into ordinal ranks or intervals

$\mathcal{S}$6

with each leaf corresponding to an interval $\mathcal{S}$7 and each internal node corresponding to a union of consecutive ranks (Lin et al., 2023). The model decomposes prediction into a tree of binary classification tasks, and the probability of reaching leaf $\mathcal{S}$8 is expressed as a product of conditional probabilities along the root-to-leaf path: $\mathcal{S}$9 The expected watch time is then computed as a weighted sum over leaf intervals, using the midpoint of each interval as the leaf-level conditional expectation (Lin et al., 2023). The model also explicitly computes predictive variance,

$\mathcal{A}$0

and includes a variance penalty in the objective: $\mathcal{A}$1 Backdoor adjustment is incorporated through conditioning on a confounder $\mathcal{A}$2, with

$\mathcal{A}$3

to mitigate bias amplification, particularly duration bias (Lin et al., 2023).

Offline, TPM is evaluated on Kuaishou and CIKM16 against WLR, D2Q, and ordinal regression, using MAE and XAUC (Lin et al., 2023). On Kuaishou, TPM reports MAE $\mathcal{A}$4 and XAUC $\mathcal{A}$5, compared with WLR $\mathcal{A}$6, D2Q $\mathcal{A}$7, and OR $\mathcal{A}$8 (Lin et al., 2023). On CIKM16, TPM reports MAE $\mathcal{A}$9 and XAUC $r_i$0, compared with WLR $r_i$1, D2Q $r_i$2, and OR $r_i$3 (Lin et al., 2023). Online deployment on Kuaishou uses 10 traffic buckets over 4 days, with Watch Time as a positive metric, Forward as a positive constraint metric, and Short View as a negative metric (Lin et al., 2023). The reported relative improvements over baseline are Watch Time $r_i$4, $r_i$5, $r_i$6, and $r_i$7 across the four days, with corresponding Short View reductions of $r_i$8, $r_i$9, $P_i(s_{t+1}\mid s_t,a_t)$0, and $P_i(s_{t+1}\mid s_t,a_t)$1 (Lin et al., 2023). The system is reported as deployed in Kuaishou with over 300 million DAUs (Lin et al., 2023). The source material explicitly states that this paper is not framed in WTL terminology but strongly supports and complements WTL-style recommendation.

A complementary development is EXUM, an EXplict Uncertainty Model that wraps an existing watch-time backbone and makes uncertainty explicit rather than leaving it implicitly absorbed into the prediction head (Wu et al., 10 Apr 2025). For each sample $P_i(s_{t+1}\mid s_t,a_t)$2, the backbone prediction $P_i(s_{t+1}\mid s_t,a_t)$3 is combined with a learned confidence $P_i(s_{t+1}\mid s_t,a_t)$4 to produce

$P_i(s_{t+1}\mid s_t,a_t)$5

for quantile prediction, and an analogous confidence-modulated output for ordinal regression (Wu et al., 10 Apr 2025). To prevent the trivial shortcut $P_i(s_{t+1}\mid s_t,a_t)$6, which would make the model collapse toward the ground truth during training, EXUM adds an adversarial confidence maximization term

$P_i(s_{t+1}\mid s_t,a_t)$7

leading to

$P_i(s_{t+1}\mid s_t,a_t)$8

The framework is deployed in the reranking stage of a large industrial short-video recommendation platform serving 300M+ daily active users, with the confidence head used only during training and no extra inference cost online (Wu et al., 10 Apr 2025). In a 20% traffic, 5-day A/B test, the treatment group uses CREAD + EXUM and reports a watch-time improvement of about $P_i(s_{t+1}\mid s_t,a_t)$9, with daily lifts of $H$0, $H$1, $H$2, $H$3, and $H$4, while like/follow/comment changes are small and within variance (Wu et al., 10 Apr 2025). Offline, the framework consistently improves D2Q and CREAD backbones on WeChat and KuaiRand, with especially large gains for CREAD-EXUM on KuaiRand (Wu et al., 10 Apr 2025). In the source material, EXUM is presented as the “Learn” mechanism that upgrades a WTL-style watch-time pipeline.

6. Checkpoint watching, unlearning, and learning from videos

In AI weather modeling, WTL is instantiated as checkpoint-level analysis of how a model acquires or loses task-specific skill. The study uses a Spherical Fourier Neural Operator with scale factor 3 and embedding size 384, built from the open-source SFNO v0.1.0 implementation in NVIDIA’s modulus-makani repository, trained on ERA5 reanalysis at $H$5 resolution using 74 variables and 13 pressure levels plus solar zenith angle, orography, and land-sea mask (Baiman et al., 20 Mar 2026). Training runs for 90 epochs total—70 epochs with cosine annealing starting at $H$6, followed by 20 epochs of 2-step fine-tuning with cosine annealing starting at $H$7—and the model weights are saved after each epoch, producing 90 checkpoints (Baiman et al., 20 Mar 2026). Every tropical cyclone is forecast with each checkpoint, so each storm has a 90-step learning trajectory.

The analysis defines a storm-relative domain as “a 1200km radius circle around the minimum MSLP value in the ERA5 valid time,” and evaluates intensity error as the difference between the forecasted minimum MSLP value and the ERA5 minimum MSLP, plus location error as the distance between the forecasted minimum MSLP and the true minimum MSLP (Baiman et al., 20 Mar 2026). A key result is that learning is not monotonic. For Hurricane Larry, location forecast improves steadily, but intensity behaves differently: at checkpoint 30 the forecast minimum MSLP error is only $H$8 hPa, whereas by checkpoints 70 and 89 the intensity error exceeds 18 hPa (Baiman et al., 20 Mar 2026). Across 106 storms, k-means clustering identifies three intensity-learning patterns: 30 storms with consistent underprediction, 53 storms with more conventional improvement, and 23 storms—including Larry—that improve early and then degrade, constituting the reported unlearning phenomenon (Baiman et al., 20 Mar 2026). Cluster 3 is associated with anomalously moist environments represented by 700 hPa relative humidity, and the paper cautiously suggests that the SFNO may be unlearning the relationship between moisture and tropical cyclone intensity rather than forgetting the moisture pattern itself (Baiman et al., 20 Mar 2026). A common misconception addressed by this result is that the best overall validation checkpoint must also be best for a specific high-impact downstream skill; the paper explicitly notes that this need not hold.

A distinct but related direction appears in computer-use agents through “Watch and Learn,” also described in the source material as a Watch-Try-Learn-style pipeline (Song et al., 6 Oct 2025). The central problem is the scarcity of large-scale, high-quality computer-use trajectories. The proposed solution is to convert raw human tutorial videos from the Internet into executable UI trajectories by framing the labeling problem as inverse dynamics on screen transitions: $H$9 The inverse dynamics model is vision-only, using a SigLIP-2 vision encoder followed by 4 Transformer layers, with three heads: an action classification head over five primitives—click, scroll, type, wait, move—a coordinate head that predicts normalized $p(T)$0 coordinates discretized into integers from 0 to 1000, and a GPT-2 small decoder for typed strings (Song et al., 6 Oct 2025). It is trained on roughly 500k synthetic transitions plus 132k human-annotated transitions from Mind2Web, for over 630k triples (Song et al., 6 Oct 2025).

On the Mind2Web test split, the reported action accuracies are 72.8% for Gemini 2.5 Flash, 82.7% for TongUI, and 91.6% for the W inverse dynamics model; action-type accuracies are 81.4%, 88.9%, and 96.4%, respectively (Song et al., 6 Oct 2025). After task-aware retrieval and filtering of tutorial videos, the pipeline produces 53,125 high-quality trajectories covering 69 applications across 7 categories, with category counts of 8,691 for Productivity, 12,829 for Programming, 7,948 for Design, 7,808 for Screen Editing, 5,206 for Audio Production, 4,601 for System Utilities, and 6,042 for Science/Data (Song et al., 6 Oct 2025). These trajectories are used both as in-context exemplars and as supervised fine-tuning data. On OSWorld, adding W trajectories improves Gemini 2.5 Flash from 19.0 to 22.0, OpenAI o3 from 21.8 to 24.3, Claude 4 Sonnet from 43.9 to 45.5, and Jedi from 50.6 to 52.8 (Song et al., 6 Oct 2025). For supervised fine-tuning, Qwen 2.5-VL 7B improves from 1.9 to 13.0 with W labels, while UI-TARS-7B improves from 27.3 to 31.1; TongUI labels help Qwen to 5.4 but degrade UI-TARS to 23.8 (Song et al., 6 Oct 2025). The paper’s stated limitations include a limited action space, weak scroll data, task granularity mismatch in retrieval, and difficulty in text-heavy or rare-action domains (Song et al., 6 Oct 2025).

Taken together, these later works show that WTL has become a useful organizing concept for problems in which observation alone is insufficient, aggregate metrics are potentially misleading, and the learning system benefits from explicitly modeling the transition from passive signal to active inference. In the original robotics paper, this structure is formalized as a two-stage meta-learning procedure (Zhou et al., 2019). In recommendation, weather, and computer-use learning, the same structure reappears as a way to improve watch-time prediction, diagnose hidden training dynamics, or recover actionable trajectories from videos (Lin et al., 2023, Wu et al., 10 Apr 2025, Baiman et al., 20 Mar 2026, Song et al., 6 Oct 2025).