WayMoCo: Diverse Applications in Waymo Research

Updated 7 July 2026

WayMoCo is defined as an overloaded research label representing distinct Waymo-centered applications including interactive motion forecasting, VRU motion retrieval, and long-tail end-to-end driving evaluation.
In interactive motion forecasting, WayMoCo leverages mined driving scenarios with rich HD maps and joint metrics (minADE, MR, mAP) to benchmark multi-agent prediction performance.
The term also encompasses safety-performance analysis and even model collaboration research, illustrating its multifaceted role in shaping evaluation methods for autonomous driving.

“WayMoCo” is not a single canonical term in the arXiv literature. Instead, it functions as an overloaded label across several Waymo-centered research contexts: interactive motion forecasting built around the Waymo Open Motion Dataset, context-aware retrieval of vulnerable-road-user motion from Waymo data, shorthand for Waymo corner cases in end-to-end driving, and summaries of Waymo’s public collision/contact safety evidence. A further, non-automotive reuse appears in the model-collaboration literature around the MoCo library (Ettinger et al., 2021, Englmeier et al., 1 Aug 2025, Xu et al., 30 Oct 2025, Schwall et al., 2020, Feng et al., 29 Jan 2026).

1. Terminological scope

A common misconception is that WayMoCo names a single benchmark or dataset. The current literature does not support that reading. The same string is used for distinct objects, tasks, and evaluation regimes.

Usage	Referent	Primary source
Interactive forecasting	Waymo Open Motion Dataset and joint motion forecasting tasks	(Ettinger et al., 2021)
Motion-context retrieval	WOD extension for SMPL/video/text retrieval of VRU behavior	(Englmeier et al., 1 Aug 2025)
Corner-case E2E driving	WOD-E2E long-tail scenario benchmark and RFS metric	(Xu et al., 30 Oct 2025)
Safety-performance evidence	Waymo public collision/contact dataset and counterfactual analysis	(Schwall et al., 2020)
Model-collaboration mnemonic	Editorial reuse tied to the MoCo library	(Feng et al., 29 Jan 2026)

The first three senses are all directly related to Waymo autonomous-driving data, but they target different technical problems. The forecasting usage centers on predicting future trajectories of interacting agents. The retrieval usage centers on open-vocabulary search over human motion and scene context. The corner-case usage centers on planning in rare long-tail scenarios with human-preference-aligned evaluation. The safety-performance usage is observational and counterfactual rather than benchmark-oriented. The MoCo-library usage is semantically separate from Waymo driving research.

2. WayMoCo as interactive motion forecasting

In "Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset" (Ettinger et al., 2021), WayMoCo refers to the Waymo Open Motion Dataset (WOMD) and the motion-forecasting tasks built around it. The dataset is designed explicitly for both single-agent motion forecasting and joint multi-agent interaction forecasting, with emphasis on critical situations such as merges, lane changes, and unprotected turns. It contains over 100,000 scenes, concretely 104k 20-second segments at 10 Hz, totaling 574 hours over 1,750 km of unique roadways across six U.S. cities.

The dataset is interaction-centric by construction. Scenes are mined by defining semantic predicates over agent relationships in an SQL/relational framework, including events such as lane changes, path crossings, merges, unprotected turns, vehicle–pedestrian interactions, vehicle–cyclist interactions, close proximity, and high accelerations. Each selected segment contains an annotated pair of interacting objects, positioned near the 10-second mark within the 20-second clip so that 9.1-second evaluation windows centered around 4–6 seconds capture the interaction. In the interactive splits, only this mined pair is required to be predicted jointly (Ettinger et al., 2021).

The data representation is unusually rich for forecasting. Agent states at 10 Hz include 3D bounding-box parameters, velocity vector, and a valid flag for missing measurements. HD 3D maps are provided as polylines and polygons at 0.5 m resolution, including lane centers, typed boundaries, road edges, stop signs, crosswalks, speed bumps, traffic-signal states, and lane associations. The dataset covers vehicles, pedestrians, and cyclists; all scenes contain at least one vehicle, 57% contain at least one pedestrian, and 16% contain at least one cyclist.

The task formulation generalizes from single-agent to joint prediction. For each sample, a model outputs $K$ joint predictions, each consisting of a confidence $c_k$ and a joint trajectory over $A$ agents and $T$ future steps. The single-agent case is the marginal case $A=1$ , and the forecasting horizon is 8 seconds, i.e. 80 steps at 10 Hz. The paper defines joint minADE and minFDE as

$\mathrm{minADE}=\frac{1}{TA}\min_k \sum_a \sum_t \left\| \hat{s}_{a,t} - s_{a,t}^{k} \right\|_2$

and

$\mathrm{minFDE}=\frac{1}{A}\min_k \sum_a \left\| \hat{s}_{a,T} - s_{a,T}^{k} \right\|_2.$

Miss Rate is generalized through an IsMatch criterion with separate lateral and longitudinal thresholds, scaled by speed and prediction horizon; Overlap Rate counts box-overlap events from the highest-confidence joint prediction; and AP/mAP are defined through MR-based matching, with mAP averaged over eight trajectory-shape buckets: straight, straight-left, straight-right, left, right, left u-turn, right u-turn, and stationary (Ettinger et al., 2021).

The standardized data splits are 70% training, 15% validation, and 15% test, determined by hashing date and vehicle ID. From each subset, 9.1-second windows are extracted with different start times for training and evaluation. Standard prediction lists contain up to eight agents per window, whereas the interactive validation/test sets list only the mined interacting pair.

Baseline models establish the initial evaluation regime. The Constant Velocity baseline performs poorly at 8 seconds; for standard validation on vehicles it yields minADE = 11.0, MR = 0.95, mAP = 0.02. The best LSTM baseline with road-graph, traffic-signal, and high-order interaction features reaches minADE = 1.34, MR = 0.25, mAP = 0.23 on the same split. On interactive validation for vehicles under joint metrics, Constant Velocity yields minADE = 10.3, MR = 0.98, mAP = 0.00, while the LSTM with the same feature set yields minADE = 2.42, MR = 0.66, mAP = 0.08. A direct marginal-to-joint comparison on interactive validation shows a benefit to true joint modeling: at 8 seconds, minADE improves from 4.16 to 3.81 and mAP from 0.01 to 0.03 (Ettinger et al., 2021).

Within this usage, WayMoCo denotes a standardized technical backbone for interaction-aware forecasting: mined scenarios, high-quality offboard tracks, HD maps, joint metrics, and baseline architectures.

3. WayMoCo as context-aware motion retrieval

In "Context-based Motion Retrieval using Open Vocabulary Methods for Autonomous Driving" (Englmeier et al., 1 Aug 2025), WayMoCo denotes a different resource: an extension of the Waymo Open Dataset for context-aware retrieval of vulnerable-road-user behavior. Here the central problem is not trajectory forecasting but searchability of rare or unusual human behavior in large-scale driving data through open-vocabulary text queries.

The dataset pairs 2-second SMPL-based motion sequences with corresponding video frames and natural-language descriptions of motion and context. Each sequence contains 20 frames at 10 Hz and is centered on a single VRU track ID. The pipeline begins with WOD person bounding boxes and track IDs, enforces a minimum box size of 90×35 pixels, splits sequences when more than two frames are missing, discards invalid segments, and smooths root-orientation outliers with LOWESS. TokenHMR is then applied per frame to recover SMPL pose $\theta$ and shape $\beta$ , producing pseudo-ground-truth SMPL motion over time (Englmeier et al., 1 Aug 2025).

Annotation is multimodal and partly open-vocabulary. A retrained MotionCLIP annotator adapted to 10 Hz assigns BABEL-60 motion tags, after filtering out actions irrelevant to driving and removing “turn” because of false positives. Context is extracted from the middle frame through semantic segmentation: ONEFORMER trained on Mapillary Vistas handles ground-level context such as road, crosswalk, sidewalk, and pavement, while ODISE supplies open-vocabulary object categories and spatial relations such as “next to,” “in front of,” and “behind,” using fixed geometric regions around the person box. Each sequence is assigned both short tags and sentence-style descriptions, and synonym expansion substantially enlarges the textual surface form inventory.

The scale is materially smaller than WOMD but large for VRU-centric retrieval. WayMoCo contains 27,466 valid segments, split into 18,814 train, 3,585 validation, and 5,067 test sequences. Without synonyms, the total annotation counts are 32,133 motion labels, 56,747 context labels, 66,898 motion-context combinations, and 66,898 sentences. With synonym expansion, the corpus reaches 48,554 motion labels, 61,108 context labels, 108,413 combinations, and 600,252 sentences. The distribution is dominated by locomotion actions such as stepping forward, walking, and running, while context is dominated by sidewalk and crosswalk, with many relations involving car and building (Englmeier et al., 1 Aug 2025).

The accompanying retrieval model, ContextMotionCLIP, fuses three encoders: a retrained MotionCLIP backbone operating on XYZ joint coordinates from the SMPL sequence, a TC-CLIP video encoder applied to the 20-frame clip, and a frozen CLIP ViT-B/32 text encoder. Three fusion strategies are explored—concatenation, bilinear pooling, and self-attention—followed by layer normalization, dropout, and a 512-unit MLP to align the fused representation with CLIP text space. Retrieval is performed by cosine similarity between text and indexed motion-context embeddings, with Qdrant used as the vector database infrastructure (Englmeier et al., 1 Aug 2025).

The principal metric is Top- $K$ accuracy, equivalent to Recall@ $c_k$ 0 under single ground truth. On the test split with Motion+Context tags, ViFi-CLIP reaches Top-1 35.9% and Top-5 69.3%, while TC-CLIP reaches Top-1 37.8% and Top-5 69.8%. ContextMotionCLIP with bilinear fusion reaches Top-1 48.2%, Top-2 58.3%, Top-3 62.6%, and Top-5 69.9%, with the paper reporting up to 27.5% absolute improvement in Top-1 accuracy versus TC-CLIP. The red bounding-box prompt around the person of interest materially improves performance, raising TC-CLIP Top-1 from 32.4% to 37.8% and ContextMotionCLIP Top-1 from 43.8% to 48.2% (Englmeier et al., 1 Aug 2025).

In this usage, WayMoCo is a retrieval dataset and system for surfacing rare human-centered events—especially safety-critical VRU behavior—through compositional text queries such as “A person is walking on the crosswalk” or “A person is walking behind a police car.”

4. WayMoCo as Waymo corner cases in end-to-end driving

In "WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarios" (Xu et al., 30 Oct 2025), WayMoCo is used as shorthand for Waymo corner cases: rare, safety-critical situations that disproportionately challenge end-to-end driving systems. The benchmark is WOD-E2E, not WayMoCo itself, but the term names the scenario class that the dataset is designed to expose.

WOD-E2E contains 4,021 driving segments, approximately 12 hours in total, each 20 seconds long. The splits are 2,037 train, 479 validation, and 1,505 test segments. These segments are mined from millions of miles of logs to concentrate events that occur with frequency less than 0.03% in daily driving after human filtering. The benchmark organizes scenarios into 11 clusters: Construction, Intersection, Pedestrians, Cyclists, Multi-Lane Maneuvers, Single-Lane Maneuvers, Cut-ins, Foreign Object Debris, Special Vehicles, Spotlight, and Others. Representative examples include construction disruptions, unprotected intersection maneuvers with limited visibility, cyclists losing control, aggressive freeway cut-ins, animals or debris in lane, and emergency-vehicle pull-over protocols (Xu et al., 30 Oct 2025).

The modality set is planner-oriented. Each segment provides 8 synchronized cameras with 360-degree coverage at 10 Hz, camera intrinsics and extrinsics, past ego trajectory over 4 seconds at 4 Hz, aligned velocity and acceleration, and a high-level routing command in $c_k$ 1. Future ego trajectory over 5 seconds at 4 Hz is available for train and validation, while the test future is hidden.

The distinctive feature of WOD-E2E is its human-aligned evaluation. Validation scenes include three rater-preferred 5-second future trajectories at a critical moment, each scored from 0 to 10, with at least one trajectory above 6. Raters choose these from a set of sampled candidates after identifying the earliest critical frame where the event is visually apparent and the ego has begun reacting. Scoring dimensions are safety, legality, reaction time, braking necessity, and efficiency, with cumulative deductions from a base score of 10 (Xu et al., 30 Oct 2025).

The resulting Rater Feedback Score (RFS) is meant to replace or at least supplement conventional distance metrics such as ADE and FDE in long-tail evaluation. ADE and FDE are defined conventionally as

$c_k$ 2

RFS instead evaluates proximity to any of the rater trajectories at $c_k$ 3 seconds using rectangular trust regions. Base thresholds are $c_k$ 4, $c_k$ 5 at 3 seconds and $c_k$ 6, $c_k$ 7 at 5 seconds, scaled by initial speed. For lateral and longitudinal errors $c_k$ 8 and $c_k$ 9 relative to a rater trajectory, the contribution is

$A$ 0

At each evaluation time, the maximum over the three rater trajectories is taken; the two times are then averaged; and a floor of 4 is applied to reflect unacceptable behaviors far from all preferred futures (Xu et al., 30 Oct 2025).

The benchmark demonstrates systematic divergence between ADE and preference-aligned competence. The public leaderboard includes Swin-Trajectory at RFS 7.543 and ADE 2.814, DiffusionLTF at 7.717 and 2.977, UniPlan at 7.779 and 2.986, AutoVLA at 7.556 and 2.958, HMVLM at 7.736 and 3.071, and Poutine at 7.986 and 2.741. The paper explicitly notes that ADE and RFS diverge for many submissions, which is presented as evidence that conventional waypoint distance is insufficient on corner cases (Xu et al., 30 Oct 2025).

In this usage, WayMoCo refers not to a general motion benchmark but to the long-tail scenario regime against which end-to-end planners are stress-tested.

5. WayMoCo as collision and contact safety performance

A further usage appears in "Waymo Public Road Safety Performance Data" (Schwall et al., 2020), where WayMoCo denotes Waymo’s collision/contact safety performance. This is not a benchmark in the usual machine-learning sense; it is an empirical and counterfactual safety dataset covering Phoenix operations.

The scope is tightly defined. All data come from the East Valley operational design domain of the Phoenix metropolitan area on urban roads with speed limits at or below 45 mph, in day and night operation except inclement weather. The paper covers more than 6.1 million miles of automated driving: 6.1 million supervised miles from calendar year 2019 and 65,000 driverless miles from 2019 through September 2020 (Schwall et al., 2020).

The event set includes every actual collision and minor contact, plus every simulated predicted contact identified by counterfactual replay of supervised disengagements. In that replay, the recorded sensor data and the software version active at the time of disengagement are used to simulate the post-disengagement trajectory; deterministic short-term human collision-avoidance models are then applied if trajectory overlap with other agents is detected. In more than 99.9% of disengagements, no simulated contact occurs (Schwall et al., 2020).

Across the full corpus, the paper reports 47 contact events: 18 actual outcomes and 29 simulated predicted contacts. Of these, one occurred during driverless operation and 46 during supervised operation. Severity is summarized via ISO 26262 S0–S3 classes. There are 30 S0 events and 17 S1 events, the latter split into 9 without airbag deployment and 8 with actual or expected airbag deployment. There are no S2 or S3 events, and none of the 47 events would be expected to cause severe or life-threatening injuries (Schwall et al., 2020).

The collision typology is also notable. Road departure, fixed-object, and rollover events are entirely absent. Waymo striking a pedestrian or cyclist is also absent. Three events involve Waymo being struck by a pedestrian or cyclist, all S0 and all at low speed while the Waymo vehicle is stationary. Rear-end events are the most common category, with 16 total, overwhelmingly cases in which Waymo is the struck vehicle. Angled collisions total 15, largely involving other drivers failing to yield to a Waymo vehicle traveling straight with right-of-way. The most severe simulated event is an unprotected left turn by an opposing vehicle across the Waymo path (Schwall et al., 2020).

The paper also reports derived event rates. Using $A$ 1, the overall rate for all actual and simulated events is approximately 7.62 per million miles over 6,165,000 miles, while the actual-only rate is approximately 2.92 per million miles. Because the driverless-only mileage is only 65,000 miles with one event, the corresponding confidence interval is very wide (Schwall et al., 2020).

This usage frames WayMoCo as safety evidence rather than as a predictive benchmark. Its significance lies in the distribution of collision modes and severities, especially the absence of S2/S3 outcomes, road-departure/fixed-object events, and Waymo-striking-VRU events within the Phoenix ODD. The paper is explicit, however, that broader safety conclusions are limited by ODD specificity, the small driverless exposure, and the assumptions built into counterfactual simulation.

6. Cross-cutting themes and the status of the term

Across the Waymo-centered meanings, three technical motifs recur. The first is deliberate concentration of informative rarity. WOMD mines interactive segments from a larger corpus to enrich social behavior (Ettinger et al., 2021). The retrieval version of WayMoCo seeks rare VRU behavior in the long tail of WOD (Englmeier et al., 1 Aug 2025). WOD-E2E explicitly mines events that survive to roughly 0.03% of real-world miles after human filtering (Xu et al., 30 Oct 2025). The safety-performance paper, by contrast, records all observed or simulated contacts, but its analytic emphasis also falls on rare and consequential failure modes (Schwall et al., 2020).

The second motif is multimodality. In WOMD, multimodality appears as $A$ 2 future trajectory hypotheses and joint metrics over interacting agents. In the retrieval dataset, it appears as joint motion–vision–language embedding over SMPL sequences, video, and text. In WOD-E2E, it appears as multiple human-preferred futures rather than a single logged trajectory. This suggests a broad shift from deterministic path prediction toward representations that preserve behavioral plurality.

The third motif is increasing human alignment in evaluation. WOMD introduces joint forecasting metrics such as joint minADE, Miss Rate, Overlap Rate, and mAP over trajectory-shape buckets (Ettinger et al., 2021). The retrieval dataset evaluates whether textual descriptions recover semantically and contextually appropriate VRU behavior (Englmeier et al., 1 Aug 2025). WOD-E2E goes further by using rater preference labels and RFS to score safety, legality, and reaction quality rather than mere proximity to logs (Xu et al., 30 Oct 2025). The safety-performance paper grounds evaluation in realized and counterfactual contact outcomes and injury-severity expectations (Schwall et al., 2020). Taken together, these usages suggest a progression from behavior prediction, to behavior retrieval, to preference-aligned planning, and finally to operational safety accounting.

A separate and semantically unrelated reuse appears in "MoCo: A One-Stop Shop for Model Collaboration Research" (Feng et al., 29 Jan 2026). Its explanatory notes gloss “WayMoCo” as the way to do model collaboration in practice and summarize a Python library implementing 26 collaboration methods over 25 datasets, with collaboration outperforming non-collaborative baselines in 61.0% of $A$ 3 settings on average and the strongest methods improving by up to 25.8%. This is not a Waymo driving benchmark or dataset; it is an independent reuse of the string in the language-model collaboration literature (Feng et al., 29 Jan 2026).

The most accurate encyclopedic characterization, therefore, is that WayMoCo is an overloaded research label rather than a stable proper noun. In contemporary arXiv usage it most often denotes one of three Waymo-adjacent technical regimes—interactive motion forecasting, context-aware VRU motion retrieval, or long-tail corner-case end-to-end driving evaluation—while adjacent summaries also use it for collision/contact safety analysis and, in one unrelated case, for model collaboration.