ReWatch: Revisit Operations in Video Systems

• ReWatch is a family of selective revisit operations that enable re-immersion, replay analytics, and evidence reexamination in diverse video applications.
• It integrates methodologies such as character-oriented summarization, temporal evidence localization, and verification protocols to balance efficiency with accuracy.
• Applications range from narrative recap generation and affective tagging to continual learning and interactive video reasoning, emphasizing localized and verifiable revisits.

ReWatch is used in the literature to denote several related but distinct operations centered on revisiting. In viewer-facing media systems, it refers to re-immersion before resuming a complex serial narrative and to the desire to watch a clip again; in platform analytics, it denotes replay hotspots and repeat-consumption dynamics; in video question answering and multimodal reasoning, it denotes explicit re-entry into temporally or spatially localized evidence; and in interactive systems it denotes replay or verified reuse of prior trajectories, caches, or workflows (Bost et al., 2019, Duico et al., 2023, Rasheed et al., 28 Nov 2025, Li, 16 Jun 2026).

1. Terminological scope

The term is not standardized across subfields. In TV-series summarization, it denotes pre-season re-immersion through automatically generated character-centered recaps (Bost et al., 2019). In social and platform analytics, it denotes repeated consumption, either as revisits by the same user or as “Most Replayed” timeline hotspots exposed by a video platform (Figueiredo et al., 2014, Duico et al., 2023). In affect modeling, it is an explicit rating dimension defined as the desire to watch again (Hadar, 2017). In video reasoning, it denotes active revisiting of evidence, implemented by temporal clip retrieval, frame freezing, spatial zoom, or second-pass high-fidelity processing (Rasheed et al., 28 Nov 2025, Li et al., 23 Jun 2026). In computer-use and continual-learning settings, closely related phenomena appear under the terms replay, selective retrieval, record-and-replay, and verified workflow reuse (Hickok et al., 2024, Li, 16 Jun 2026, Feng et al., 2023).

Domain	Operational meaning	Representative paper
TV narratives	Character-oriented re-immersion summaries	(Bost et al., 2019)
Video platforms	“Most Replayed” timeline hotspots	(Duico et al., 2023)
Social-media popularity	Revisits by returning users	(Figueiredo et al., 2014)
Affective media tagging	Desire to watch again	(Hadar, 2017)
Video reasoning	Rewatch/refocus/verify evidence	(Rasheed et al., 28 Nov 2025)
Repeated-task execution	Verified replay of prior workflows	(Li, 16 Jun 2026)

A common thread is revisitation under constrained memory, attention, or compute. This suggests that ReWatch is best understood as a family of revisit operators rather than a single framework.

2. Re-immersion for long-form narrative media

A concrete viewer-facing formulation appears in "Remembering Winter Was Coming: Character-Oriented Video Summaries of TV Series" (Bost et al., 2019). The problem setting is modern serial television with continuous plots viewed under discontinuous conditions. The study documents a “cold-start” at new season launch, visible as an early-season dip in IMDb episode ratings that recovers as viewers re-immerse. Its survey reports that 41% watch a whole season in one week, 9% in 1–2 days, and 66% prefer serials over standalone series; nearly 60% feel the need to remember the plot before a new season, 49% discuss with friends, 48% read textual synopses, and 43% watch video recaps (Bost et al., 2019).

The proposed ReWatch mechanism is character-oriented summarization. The pipeline assumes the full set of episodes and partial annotations: manually inserted scene boundaries, subtitle lines manually labeled with speakers, and character identities inferred from those labels. Candidate units are Logical Story Units rather than isolated shots. Shot similarity is computed from 3D HSV histograms with block-based comparison, yielding similarity detection with F-score approximately 0.90 on annotated subsets. LSU boundaries are defined by

$S^{(k)} := \sum_{(i > k, j < k)} s_{i,j},$

with recursive update

$$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$

Elementary LSUs are then constrained between 5 and 15 seconds (Bost et al., 2019).

Plot structure is modeled as a dynamic social network built from speaker turns and scene boundaries using narrative smoothing. For a target character, the relationship vector $r_t$ contains weights in $[0,1]$ to all other characters at scene or time $t$. Storylines are partitioned into narrative episodes by an adapted set-covering problem with temporal contiguity, controlled by a user-set threshold $\tau$. Within each episode, one representative scene anchors the typical social configuration of that stage (Bost et al., 2019).

Candidate LSUs are scored by combining social relevance and filmmaking grammar. Social relevance is the cosine similarity between the episode representative $r_t$ and the LSU’s relationship vector. Two stylistic cues are added: shot size, estimated from the median height of detected faces in five sampled frames per shot, and musicality, estimated with MIRtoolbox and chroma-based speech-versus-music tracking. The combined score is

$$p_i = \lambda_1 \cdot sr_i + \lambda_2 \cdot ss_i + \lambda_3 \cdot m_i.$$

Selection is then posed as a quadratic knapsack:

$$\max f(x) = \sum_{i=1}^n p_i x_i + \sum_{i=1}^n \sum_{j=1}^n d_{ij} x_i x_j$$

subject to

$\sum_{i=1}^n w_i x_i \le T,\qquad x_i \in \{0,1\},$

where $$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$0 is LSU duration, $$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$1 is the per-episode time budget, and $$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$2 is the normalized Euclidean distance between LSU relationship vectors (Bost et al., 2019).

The evaluation was conducted a few weeks before the release of Game of Thrones Season 6 with 187 subjects. For each of five characters, participants ranked three summary types: full, style-only, and a semi-random baseline. Among viewers who had watched all five seasons, the full summary was most often preferred as recap at 42.3%, compared with 32.2% for style-only and 25.4% for baseline; for trailer-like appeal, style-only and full were nearly tied at 36.9% and 36.8% (Bost et al., 2019). Arya’s full summary was especially strong, with 70.9% for recap and 57.1% for trailer. The reported interpretation is that dynamic social-network plot modeling improves recap quality, whereas stylistic cues improve trailer-like appeal; relying only on style risks omitting short or late narrative episodes (Bost et al., 2019).

3. ReWatch as a behavioral, educational, and affective signal

A second major meaning of ReWatch is an observable behavioral trace. In "Can we predict the Most Replayed data of video streaming platforms?" (Duico et al., 2023), Most Replayed is treated as a normalized intensity curve of length 100 over the video timeline, with values in $$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$3. The YTMR500 benchmark contains 500 Creative Commons YouTube videos, 3 to 20 minutes long, each with MR annotations retrieved via the YouTube operational API and paired with pre-extracted I3D features. The paper frames prediction as ranking rather than regression, using MarginRankingLoss with pairwise targets $$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$4 and

$$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$5

On the 100-bin task, the best reported model, a PGL-SUM variant, achieved precision@15 of $$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$6 with interpolation and $$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$7 with bin-averaging, only modestly above random 15%, which the paper interprets as evidence that MR prediction is difficult even though deep models outperform random and human raters (Duico et al., 2023).

At the population-dynamics level, "Revisit Behavior in Social Media: The Phoenix-R Model and Discoveries" (Figueiredo et al., 2014) formalizes rewatching as revisits by returning users. Popularity is decomposed into audience and revisits, and revisits are modeled with a Poisson process of rate $$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$8 embedded in a multi-shock SIR-like system:

$$S^{(k)} = S^{(k-1)} - \sum_{j < k - 1} s_{k,j} + \sum_{i > k} s_{i,k-1}.$$9

The study reports median $r_t$0 of 1.70 for Twitter, 0.68 for MMTweet, and 25.39 for LastFM, and median $r_t$1 of 0.62, 0.40, and 0.96 respectively (Figueiredo et al., 2014). These measurements show that repeat consumption can dominate total popularity, especially in music. Phoenix-R further models multiple shocks and periodicity, and is reported to achieve lower RMSE than the compared baselines on several datasets (Figueiredo et al., 2014).

In educational video analytics, rewatching is detected directly from clickstreams. "Mining MOOC Clickstreams: On the Relationship Between Learner Behavior and Performance" (Brinton et al., 2015) distinguishes reflecting, defined as repeated play–pause cycles, from revising, defined as repeated local skip-backs interleaved with play. Several revising motifs were positively associated with Correct on First Attempt, while skimming motifs dominated by skip-forward were negatively associated. The position-based models that encode visited windows and transition types improved over a skewed-random baseline: on FMB, the discrete-position model reached accuracy $r_t$2 and F1 $r_t$3 versus $r_t$4 and $r_t$5 for the baseline; on NI, the same model reached $r_t$6 and $r_t$7 versus $r_t$8 and $r_t$9 (Brinton et al., 2015).

Affective media tagging introduces yet another formulation. In "Implicit Media Tagging and Affect Prediction from video of spontaneous facial expressions, recorded with depth camera" (Hadar, 2017), Rewatch is a 3-point scale capturing the desire to watch again. Across clips, it correlated with Likability at $[0,1]$0 and with Valence at $[0,1]$1, but not significantly with Arousal at $[0,1]$2. Inter-rater reliability was high, with ICC $[0,1]$3 and Cronbach’s alpha $[0,1]$4 (Hadar, 2017). The two-step regression pipeline achieved Rewatch Pearson correlation of 0.661 for implicit media tagging from a single viewer, 0.953 when aggregating multiple viewers, 0.574 for within-viewer affect prediction, and 0.410 for cross-viewer affect prediction (Hadar, 2017). In that setting, Rewatch is not a replay count but an explicit intention signal.

These studies make clear that ReWatch can denote at least three measurable targets: repeated access behavior, timeline-localized replay demand, and subjective desire to revisit.

4. ReWatch as a model-side video reasoning primitive

Long before contemporary multimodal LLMs, video question answering introduced explicit re-watching as an internal attention mechanism. "The Forgettable-Watcher Model for Video Question Answering" (Xue et al., 2017) processes each token of a concatenated question–answer sentence and, at every token, attends over all video frames. Its ReWatch equations are

$[0,1]$5

$[0,1]$6

$[0,1]$7

The final score is

$[0,1]$8

On TGIF-QA, the Re-Watcher reached test accuracy 0.8663, above the straightforward baseline at 0.8253, while the combined Forgettable-Watcher reached 0.8733 (Xue et al., 2017).

A second early formulation appears in "Watch It Twice: Video Captioning with a Refocused Video Encoder" (Shi et al., 2019). The model first encodes the video using a default middle-frame key, predicts a key frame, and then re-encodes the sequence centered on that predicted anchor with a key-frame-based bidirectional GRU. On MSR-VTT, the full VRE with temporal, spatial, and audio cues reached BLEU4 43.2, ROUGE-L 62.0, METEOR 28.0, and CIDEr 48.3; on MSVD, VRE with temporal and spatial cues reached BLEU4 51.7, ROUGE-L 71.9, METEOR 34.3, and CIDEr 86.7 (Shi et al., 2019). The paper’s rationale is that a second pass reduces contamination from irrelevant content at the beginning and end.

In current video-MLLM research, ReWatch becomes an explicit action policy. "Video-CoM: Interactive Video Reasoning via Chain of Manipulations" (Rasheed et al., 28 Nov 2025) introduces Interactive Video Reasoning, in which a controller based on Qwen2.5-VL-7B-Instruct alternates textual reasoning with visual actions executed by an action module $[0,1]$9. The available actions are Find-segment, Find-frame, and Spatial-zoom. The trajectory is formalized as

$t$0

with $t$1. Training adds reasoning-aware GRPO with step-level rewards. On nine benchmarks, the model improved average performance by 3.6% over recent state of the art while training on only 25K SFT and 3K GRPO samples (Rasheed et al., 28 Nov 2025). On VCoM-Bench, the progression SFT $t$2 GRPO $t$3 RA-GRPO was 64.0 $t$4 66.7 $t$5 68.7 (Rasheed et al., 28 Nov 2025).

"video-SALMONN-R$t$6: Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding" (Li et al., 23 Jun 2026) formulates ReWatch as a two-stage policy under fixed token budgets. A first pass on low-fidelity video produces an initial answer $t$7, a short reasoning trace, and a temporal localization $t$8. A second pass revisits only that interval at higher fidelity, re-injects the question as $t$9, and refines the answer to $\tau$0. The policy is trained end-to-end with DAPO and a reward

$\tau$1

The reported results show gains over both the base model and the QA-SFT baseline, including 76.3 on VideoMME versus 72.9 for QA-SFT, and 42.9 on LVOmniBench versus 40.6 (Li et al., 23 Jun 2026). An important ablation is that uniform re-watch with the same token budget underperformed targeted re-watch, which the paper interprets as evidence that gains come from localization rather than simply spending more tokens (Li et al., 23 Jun 2026).

At the data-and-training level, "ReWatch-R1: Boosting Complex Video Reasoning in Large Vision-LLMs through Agentic Data Synthesis" (Zhang et al., 28 Sep 2025) uses ReWatch to name a dataset and training recipe for difficult video reasoning. ReWatch includes 10k temporally dense captions, 170k difficult QAs, and 135k grounded CoTs synthesized by a Multi-Agent ReAct loop. During RLVR, it introduces an Observation & Reasoning reward,

$\tau$2

which scores final-answer correctness, observation alignment with detailed captions, answer recoverability from the extracted actions and observations, and format compliance (Zhang et al., 28 Sep 2025). On the five reasoning benchmarks at 192 frames, ReWatch-R1 + O&R reached an average of 35.51, above the 30.71 of Qwen2.5-VL-7B in non-thinking mode (Zhang et al., 28 Sep 2025).

Taken together, these papers show a shift from soft internal attention, to second-pass encoding, to explicit action policies, and finally to end-to-end agentic synthesis and RL verification.

5. Replay, cache reuse, and repeated-task execution

In some settings ReWatch is less about media consumption and more about efficient reuse of past computations or trajectories. "Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning" (Wang et al., 25 Jun 2026) studies visual revisiting during autoregressive multimodal decoding. The paper shows that directly appending historical visual key-value cache fails because the keys remain bound to stale positions; on M$\tau$3CoT, direct KV reuse achieved 23.50% accuracy with an 81.04% stuck rate (Wang et al., 25 Jun 2026). PRCR addresses this by storing raw pre-RoPE visual keys and values with spatial coordinates, reassigning position-compatible coordinates, rebinding the keys, and injecting reconstructed cache entries into the active decoder. On Qwen3-VL-8B with 32 selected visual tokens, replay costs 483.18G FLOPs per insertion, whereas PRCR costs 14.16M; on Qwen3-VL-32B with $\tau$4, replay costs 7.75T FLOPs versus 125.83M for PRCR (Wang et al., 25 Jun 2026). The reported result is replay-level or better accuracy with 0% stuck decoding under the PCR variant (Wang et al., 25 Jun 2026).

In continual learning, the exact term is replay and selective retrieval rather than ReWatch. "Watch Your Step: Optimal Retrieval for Continual Learning at Scale" (Hickok et al., 2024) explicitly states that “ReWatch” would map to rewatching or replaying stored pretraining samples during fine-tuning. The paper decomposes retrieval into class-selective and sample-selective primitives, studies deduplication, and evaluates them on a 15-dataset sequence with OWL-ViT. Its main recommendation is to use SWIL for class selection or GRASP for sample selection, enforce dataset-level deduplication, and maintain a 1:1 new-to-replay ratio; loss-adaptive reductions of replay harmed retention on O365, LVIS, and LVIS rare (Hickok et al., 2024).

In computer-using agents, ReWatch becomes verified replay of a prior workflow. "PreAct: Computer-Using Agents that Get Faster on Repeated Tasks" (Li, 16 Jun 2026) compiles a successful run into a finite-state machine with screen-check predicates and transitions that act only when the expected UI state is visible. Replay is guarded both at run time and at store time: a freshly compiled program is replayed from a clean state and kept only if an independent evaluator confirms that the task was actually solved. Across mobile, desktop, and web benchmarks, warm replays are reported as 8.5–13× faster, and the store-time verification gate improved repeated-run performance by 1.75–2.6 tasks depending on benchmark and setting (Li, 16 Jun 2026). The paper also reports that prompt wording, runtime guardrails, and whether selection used an LLM or an embedding retriever did not materially drive the outcome (Li, 16 Jun 2026).

A closely related engineering study is "Towards Efficient Record and Replay: A Case Study in WeChat" (Feng et al., 2023). There, the central problem is when to fire the next replay event. WeReplay uses adb screenshots, MobileNetV2, and a binary rendering-state classifier to decide whether the GUI is Fully Rendered or Partially Rendered. The detector achieved 92.1% precision, 93.3% recall, and 92.7% F1 on WeChat test images (Feng et al., 2023). On 23 same-device scenarios, WeReplay replayed all scenarios successfully with mean 18.45 seconds per scenario, compared with 39.1% success for SARA and 152.99 seconds for the industrial 10× wait baseline; on three different devices, WeReplay again achieved 100% success (Feng et al., 2023).

These systems make a useful distinction: revisiting is beneficial only when verification preserves correctness. Naive replay, stale-position cache reuse, duplicate replay, or fixed waits can all degrade performance.

6. Limits, misconceptions, and recurrent design constraints

One recurrent misconception is that ReWatch is a single mature paradigm. The literature instead documents multiple partially overlapping uses. Viewer-oriented recap generation depends on manual scene boundaries and speaker labels and does not process linguistic content directly; the authors explicitly leave speaker diarization and recognition to future work because of error rates under TV audio conditions (Bost et al., 2019). MR hotspot prediction remains difficult, with modest gains over random and strong dependence on visual features that may miss audio- or language-driven replay peaks (Duico et al., 2023). In social-media revisit modeling, audience–revisit decomposition is inferred rather than observed on platforms without user-level logs (Figueiredo et al., 2014). In affect prediction, cross-viewer generalization of Rewatch ratings is limited even though clip-level tagging from multiple viewers is strong (Hadar, 2017).

A second misconception is that more replay is automatically better. Several papers argue against this. Style-only recaps can miss short or late but content-critical episodes (Bost et al., 2019). Uniform re-watch under the same token budget underperforms targeted re-watch in video QA (Li et al., 23 Jun 2026). Direct KV reuse collapses because stale positional binding distorts attention (Wang et al., 25 Jun 2026). In continual learning, excessively strict or misallocated replay can hurt downstream performance, and even small reductions in replay for low-loss samples produced substantial forgetting on pretraining distributions (Hickok et al., 2024). In computer-use benchmarks, blind record-and-replay without verification accumulated faulty programs and degraded warm-run success (Li, 16 Jun 2026).

A third design constraint is the need for verifiability. Video-CoM adds step-level rewards for segment, frame, and spatial-zoom correctness because sparse answer rewards do not enforce grounded intermediate behavior (Rasheed et al., 28 Nov 2025). ReWatch-R1 introduces observation and reasoning rewards precisely to score whether intermediate observations align with the underlying video-derived caption evidence (Zhang et al., 28 Sep 2025). PreAct admits compiled programs to its store only if replay and an independent evaluator both succeed (Li, 16 Jun 2026). WeReplay schedules events from the GUI’s inferred rendering state rather than from recorded delays (Feng et al., 2023). This suggests that, across domains, revisit operations become reliable only when the revisited evidence remains auditable.

A final cross-cutting issue is cost. ReWatch is often introduced because uniform access is too expensive: TV viewers cannot manually reconstruct seasons from scratch, video-LLMs cannot process every frame at maximal fidelity, multimodal decoders cannot repeatedly replay visual tokens without large FLOP overhead, and computer-using agents cannot afford to invoke a model on every already-solved step (Bost et al., 2019, Li et al., 23 Jun 2026, Wang et al., 25 Jun 2026, Li, 16 Jun 2026). The research trajectory therefore couples revisitation with selective localization, compact intermediate representations, or verified replay.

Across these literatures, ReWatch is not merely repetition. It is a selective revisit operation whose value depends on four conditions repeatedly emphasized by the evidence: localization of what should be revisited, preservation of the relevant structure during revisiting, a scoring or verification mechanism that keeps the revisit grounded, and an efficiency gain large enough to justify the added machinery.