Cousin Trajectory Generator for Autonomous Driving

• CTG is a trajectory augmentation module that diversifies expert driving trajectories by extending maneuvers and interpolating additional points.
• It counteracts the bias of straight-line data in behavior cloning by generating 'cousin' trajectories with lane changes, sharp turns, and denser motion details.
• CTG integrates feasibility and kinematic checks to ensure physically plausible maneuvers, enhancing the initialization for reinforcement learning.

The Cousin Trajectory Generator (CTG) is a trajectory augmentation module introduced in the ReconDreamer-RL framework to counter training-data bias toward simple straight-line driving and to alleviate cold-start limitations in behavior cloning before reinforcement learning. In this formulation, “cousin” trajectories are related to, but not identical to, expert ego trajectories: they are derived from expert demonstrations by extending them with additional maneuvers such as lane changes and sharp turns, and by densifying them through interpolation, while preserving scene context and feasibility checks. CTG is therefore not a standalone planner or generative model with an explicitly specified loss; rather, it is a data-diversification mechanism used to construct a more balanced offline pretraining corpus for autonomous driving policy learning (Ni et al., 11 Aug 2025).

1. Definition, motivation, and conceptual scope

CTG is motivated by the observation that the training data distribution is “mostly on simple straight-line movements.” Within ReconDreamer-RL, this bias is treated as a limitation for imitation learning, because a policy pretrained only on such data inherits a narrow action distribution and enters the reinforcement-learning stage with a weak initialization. CTG is introduced specifically “to enhance action diversity via trajectory extension and interpolation” and thereby create a more balanced dataset for policy pretraining (Ni et al., 11 Aug 2025).

The term “cousin” indicates trajectories that remain close relatives of the underlying expert demonstration rather than arbitrary synthetic rollouts. They preserve the scene context and feasibility constraints of the original clip, but alter the ego motion so that the learner observes a broader set of actions than the original recordings provide. The intended effect is not merely more data, but more varied behaviorally relevant data, especially for maneuvers underrepresented in the source corpus such as lane changes, sharp turns, and rare cases including U-turns.

This framing places CTG in the category of trajectory augmentation rather than closed-loop control. A plausible implication is that its primary value lies upstream of policy optimization: it shapes the support of the imitation-learning dataset so that subsequent reinforcement learning begins from a policy less dominated by straight-driving priors.

2. Trajectory construction by extension and interpolation

CTG has two stated mechanisms: trajectory extension and trajectory interpolation. Extension adds novel maneuvers to a base ego trajectory, while interpolation densifies trajectories, including rare scenarios, to provide more detailed driving information (Ni et al., 11 Aug 2025).

For extension, CTG follows the same functional form as the Dynamic Adversary Agent trajectory editing in Eq. (1):

$$T'_{\text{target}} = f(T_{\text{ego}}, T_{\text{target}}, \mathcal{B}).$$

The paper states that CTG applies a “similar process” to ego-trajectory generation, using a generation function $f$ conditioned on scene context and desired behavior. It does not provide a separate explicit formula for $T'_{\text{ego}}$. The stated examples of extended maneuvers include lane changes and sharp turns. The paper also mentions implementations such as text-to-trajectory and custom rules, but does not specify sampling distributions or spline or other parametric trajectory forms.

For interpolation, CTG performs linear interpolation between consecutive expert timestamps. Given expert trajectories $$\mathbf{X}_{\text{ego}} = \{ X_{\text{ego}}^{t_1}, X_{\text{ego}}^{t_2}, \dots, X_{\text{ego}}^{t_n} \}$$, the interpolated ego position at time $t$, where $t_i \le t \le t_{i+1}$, is defined as

$$X_{\text{ego}}^{t} = X_{\text{ego}}^{t_i} + \frac{t - t_i}{t_{i+1} - t_i} \left( X_{\text{ego}}^{t_{i+1}} - X_{\text{ego}}^{t_i} \right).$$

The interpolation times are parameterized by

$$t = t_i + k \cdot \Delta t, \quad k \in \{1, 2, \dots, m\},$$

with

$\Delta t = \frac{t_{i+1} - t_i}{m+1}.$

For each interpolated ego point, CTG “accordingly adjust[s] the positions of surrounding vehicles to ensure the interpolated trajectories maintain a realistic spatial relationship and interactions.” This is a substantive part of the method: interpolation is not limited to the ego trace, but is paired with coordinated scene updates so that the resulting data remain interaction-consistent.

The output of this stage is a diversified set of ego trajectories that are feasibility-checked and paired with rendered sensor observations. In aggregate, the resulting Cousin-nuScenes dataset contains “4× more non-straight-line driving maneuvers than the nuScenes dataset” (Ni et al., 11 Aug 2025).

3. Feasibility checks, kinematics, and scene rendering

CTG is constrained by feasibility and safety checks rather than by an explicit optimization objective. The paper states that generated trajectories must remain in drivable areas, avoid unwanted collisions, and satisfy kinematic constraints. The collision-distance condition is written as

$$\|T'_{\text{target}} - o_j\| \ge d_{\text{min}}, \quad \forall j \in \{1, \dots, M\}.$$

For CTG specifically, ego extensions are “checked to evaluate the validity of the generated trajectories, ensuring they remain physically plausible and adhere to the vehicle's operational limitations, including steering angle and velocity, and verifying collision avoidance from the BEV perspective.” Invalid trajectories are visibly marked and removed in the figure referenced by the paper.

The feasibility layer relies on ReconSimulator’s physical modeling. The simulator uses a kinematic bicycle model with pose updates

$f$0

$f$1

$f$2

The associated rotation matrix is

$f$3

The paper states that trajectory updates are checked “to ensure the trajectory updates remain physically plausible and within the vehicle's operational constraints, including the maximum steering angle and velocity,” but does not provide numerical bounds for those quantities.

Rendering is handled by ReconSimulator’s scene model, which separates static background and moving vehicles. Each vehicle’s local Gaussian representation is transformed into world coordinates according to

$f$4

where $f$5 and $f$6 are the rotation and translation describing object $f$7’s pose at time $f$8. CTG trajectories are rendered through this pipeline, which combines a video diffusion prior for appearance modeling with kinematic physical modeling (Ni et al., 11 Aug 2025).

4. Generation pipeline and system integration

CTG takes as input a base ego trajectory from expert data and its scene context, including HD map lanes, drivable regions, and 3D boxes; surrounding agent states from the reconstructed clip; and a desired maneuver set for extension, such as lane change or sharp turn. The method then proceeds through four stages: extension, feasibility checking, interpolation, and rendering into a new dataset (Ni et al., 11 Aug 2025).

In the extension stage, CTG generates new ego maneuvers using a process similar to Eq. (1), conditioned on scene context and desired behavior. The paper emphasizes that the method operates in the BEV frame for validity checks and uses the bicycle model for pose updates. In the next stage, trajectories that leave drivable areas, violate distance constraints, or exceed vehicle limits are rejected. The interpolation stage densifies expert trajectories by inserting linearly interpolated points and adjusting surrounding vehicles accordingly. Finally, ReconSimulator renders photorealistic sensor data along these cousin trajectories, and the resulting samples are compiled into the Cousin-nuScenes dataset.

Within ReconDreamer-RL, CTG is used in the imitation-learning stage to “enhance the diversity of sensor-collected data by synthesizing new trajectories from expert trajectories.” These rendered cousin trajectories feed behavior cloning and initialize the policy’s action distribution more robustly. During reinforcement learning, the policy interacts closed-loop in ReconSimulator, while the Dynamic Adversary Agent continues to generate corner cases. CTG’s main contribution is therefore the diversified offline data used for pretraining, not the online generation of adversarial interactions (Ni et al., 11 Aug 2025).

This division of labor is important. ReconSimulator supplies appearance and physical realism; DAA modifies surrounding vehicles to create corner-case interactions such as cut-ins; CTG broadens the ego action distribution. Together they constitute the modified two-stage pipeline that follows RAD’s overall structure, but CTG’s functional role remains concentrated in offline augmentation.

5. Empirical effects and distributional rebalancing

The primary quantitative claim about CTG’s dataset effect is that Cousin-nuScenes contains “4× more non-straight-line driving maneuvers than the nuScenes dataset.” The paper does not report entropy, KL divergence, bin coverage, or other formal distributional metrics for this shift. The evidence provided for distributional balancing is the maneuver-type count increase and the downstream ablation results (Ni et al., 11 Aug 2025).

In the ablation study, adding CTG to ReconSimulator improves Collision Ratio (CR) from $f$9 to $T'_{\text{ego}}$0 and Deviation Ratio (DR) from $T'_{\text{ego}}$1 to $T'_{\text{ego}}$2. The paper interprets this as evidence that “by addressing the data bias, CTG enhances the action distribution learned during the imitation learning phase.” These improvements are specific to the ablation setting in which CTG augments ReconSimulator’s training pipeline.

The full ReconDreamer-RL system achieves $T'_{\text{ego}}$3, but the paper distinguishes the roles of the constituent modules. Corner-case performance gains are attributed mainly to DAA and ReconSimulator, whereas CTG contributes via improved pretraining diversity. This is an important interpretive boundary: CTG is empirically beneficial, but the paper does not claim that all closed-loop improvements arise from CTG alone.

A plausible implication is that CTG functions as a distribution-support widening mechanism. By populating the pretraining corpus with feasible non-straight maneuvers, it reduces the mismatch between the imitation-learning policy’s initial action distribution and the broader maneuver repertoire demanded during later closed-loop reinforcement learning.

6. Misconceptions, nomenclature, and methodological limits

A common source of confusion is the acronym CTG itself. In the paper “Guided Conditional Diffusion for Controllable Traffic Simulation,” CTG stands for Controllable Traffic Generation, not Cousin Trajectory Generator. That work introduces a dynamics-enforced conditional diffusion model guided by signal temporal logic; it is a different method with a different acronym expansion and a different objective (Zhong et al., 2022). By contrast, the present topic refers specifically to the trajectory augmentation module inside ReconDreamer-RL (Ni et al., 11 Aug 2025).

Another potential misconception is to treat Cousin Trajectory Generator as a fully specified generative model. The paper does not define an explicit optimization objective, loss, or probabilistic model for CTG beyond feasibility checks and linear interpolation. Where a functional form is referenced, it points only to a general trajectory-generation function $T'_{\text{ego}}$4. The exact implementation details of that function are not elaborated with a formal cost function. Similarly, the paper does not specify the number of cousin trajectories generated per base trajectory, the exact bounds on curvature, acceleration, jerk, steering rates, or speed ranges, the value of $T'_{\text{ego}}$5 in interpolation, or the computational cost of CTG itself.

The safeguards that are documented are pragmatic rather than formal: BEV collision and drivable-area checks, kinematic constraint checks involving maximum steering angle and velocity, and qualitative filtering of invalid trajectories. The paper also notes potential failure cases indirectly by stating that unrealistic or unsafe trajectories can arise if the extension violates drivable regions or kinematic constraints, or if interpolation yields implausibly abrupt motion when surrounding agents are not coherently adjusted.

In this sense, CTG is best understood as a constrained augmentation procedure integrated into a simulator-backed data generation pipeline. Its contribution is neither a standalone control policy nor a fully parameterized stochastic trajectory model, but a method for producing feasible variants of expert ego behavior that rebalance a straight-biased dataset and improve imitation-learning initialization.