CAA-Policy: End-to-End Autonomous Parking

• CAA-Policy is an end-to-end imitation learning system that uses control-aided attention to ensure that visual focus aligns with control decision-making in parking scenarios.
• It fuses six surround-view RGB images with current vehicle state and target slot data to predict control commands, waypoints, and auxiliary perceptions.
• Empirical evaluations in CARLA show that combining target tokenization, waypoint prediction, and attention from control gradients outperforms both modular and traditional end-to-end systems.

Searching arXiv for the specified paper and closely related parking-policy work to ground the article. arXiv search query: (Chen et al., 14 Sep 2025) autonomous parking control-aided attention E2EParking CAA-Policy is an end-to-end imitation-learning system for autonomous parking that uses Control-Aided Attention (CAA) to make control signals guide the learning of visual attention, rather than relying on task-loss supervision alone. The method takes six surround-view RGB images, the current vehicle state, and the target parking slot in ego coordinates, and produces short-horizon control commands, short-horizon waypoints, a next-position estimate, and auxiliary perception predictions. Its central claim is that attention trained from gradients backpropagated from the control outputs focuses on regions that most influence action decisions, yielding a policy that is more robust, more temporally stable, and more interpretable in parking scenarios. The system is evaluated in CARLA against both end-to-end and modular baselines (Chen et al., 14 Sep 2025).

1. Motivation and problem setting

CAA-Policy is motivated by a specific weakness in end-to-end parking systems: perception features are often learned only from task-loss supervision, so the resulting attention can drift toward visually salient but control-irrelevant regions. The paper argues that this is especially problematic in autonomous parking, where small errors near the target slot can lead to collisions, wrong-slot parking, or timeouts (Chen et al., 14 Sep 2025).

The method is positioned against two existing tendencies. In prior end-to-end parking systems such as E2EParking, images are mapped directly to control, but the model is not explicitly forced to focus on control-sensitive structures such as parking-slot boundaries, nearby obstacle edges, free-space gaps, and the target-slot vicinity. In transformer/self-attention or CBAM-style variants learned only from task supervision, attention is described as spatially scattered, unstable across time, and weakly aligned with decision-critical regions. In contrast, classical modular parking stacks based on perception, mapping, Hybrid A* planning, and PID or bicycle-model control are interpretable and usually stable, but they suffer from error accumulation across modules, expensive planning, and poor handling of delayed or outdated perception (Chen et al., 14 Sep 2025).

Within this setting, CAA-Policy is defined as a parking policy that remains end-to-end in its control output while injecting structure through explicit goal encoding, short-horizon waypoint prediction, a learnable motion predictor, and control-guided attention. The paper’s framing is therefore not a rejection of end-to-end learning, but a reorganization of it around policy-relevant visual focus (Chen et al., 14 Sep 2025).

2. Architecture and information flow

CAA-Policy is organized into five main stages: perception backbone, feature fusion, learnable motion prediction module, Control-Aided Attention module, and prediction heads. Auxiliary heads for depth estimation and BEV semantic segmentation are also included (Chen et al., 14 Sep 2025).

The perception backbone uses a shared ResNet-18 to encode the six RGB views. These features are fused into a bird’s-eye-view representation through Lift-Splat-Shoot (LSS). The resulting top-down feature map is then combined with ego-state features and goal features. The fusion is written as

$$\mathbf{f}_{\text{BEV}} = \mathrm{CNN}(\mathbf{I}_{1:N}), \quad \mathbf{f}_{\text{ego}} = \mathrm{MLP}(\mathbf{e}), \quad \mathbf{f}_{\text{goal}} = \mathrm{MLP}(\mathbf{g}),$$

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$

Here, $\mathbf{g}$ denotes the target slot in the ego frame. The explicit tokenization of the target is termed the Target Tokenization Module (TTM) and is intended to make the parking objective more explicit than implicit goal conditioning (Chen et al., 14 Sep 2025).

The learnable motion prediction module is used mainly at inference time to track the vehicle’s current position and the target slot more robustly over time. It consumes the past four frames of control commands and vehicle states, using a CNN followed by an LSTM:

$$h_t = \mathrm{LSTM}\!\left(\{\mathrm{CNN}([c_k, s_k])\}_{k=t-4}^{t-1}\right).$$

Two heads then predict the mean and variance of a 2D displacement:

$$\boldsymbol{\mu}_t = W_{\mu}h_t + b_{\mu}, \qquad \log \boldsymbol{\sigma}_t^2 = W_{\sigma}h_t + b_{\sigma}.$$

The position update is

$$(x_t, y_t) = (x_{t-1}, y_{t-1}) + \boldsymbol{\mu}_t.$$

Training uses a Gaussian negative log-likelihood loss on the ground-truth displacement:

$$\mathcal{L}_{\text{dyn}} = -\log \mathcal{N}\!\left(\Delta \mathbf{x}_t^{\text{GT}} \mid \boldsymbol{\mu}_t, \boldsymbol{\sigma}_t^2 \right).$$

The paper notes that this module is not needed during training, because ground-truth poses are available then; it primarily serves inference-time tracking (Chen et al., 14 Sep 2025).

3. Control-Aided Attention

The CAA mechanism is the defining component of CAA-Policy. It predicts a BEV-shaped spatial attention map intended to represent control sensitivity at each location in the fused feature tensor. If a spatial region strongly influences the control output, the attention assigned to that region should be high (Chen et al., 14 Sep 2025).

Given fused features

$$\mathbf{f}_{\text{fused}} \in \mathbb{R}^{C_b \times X_b \times Y_b},$$

the model predicts an attention map

$$\widehat{\nabla}_f = \mathrm{MLP}(\mathbf{f}_{\text{fused}}),$$

with spatial size $X_b \times Y_b$. The attention is broadcast over channels and applied multiplicatively:

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$0

Operationally, the module acts as a spatial gate over BEV features (Chen et al., 14 Sep 2025).

What makes the mechanism “control-aided” is the supervision target. Instead of learning attention from manually labeled masks or from the ordinary training loss, the target attention is derived from the gradient of the control output with respect to the fused perception features:

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$1

The predicted attention map is trained to match this gradient-derived signal through

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$2

The paper characterizes this as a self-supervised training strategy because the attention target is obtained from the model’s own control gradients rather than from human-provided annotations (Chen et al., 14 Sep 2025).

The conceptual consequence is that the attention module is encouraged to focus on visual features that induce high variance in action outputs, not merely on features that minimize the task loss. The paper reports that, qualitatively, attention without CAA is scattered, whereas with CAA it concentrates around the target slot and nearby obstacles. This suggests a tighter alignment between the internal attention distribution and the downstream control decision (Chen et al., 14 Sep 2025).

4. Prediction heads and training objective

The refined feature map $$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$3 feeds two main prediction heads: a control head and a waypoint head. The control head follows the structure of E2EParking and predicts control tokens autoregressively. Ground-truth continuous commands

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$4

are quantized into discrete tokens, and the control objective over the next four steps is

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$5

This keeps the control prediction within a sequence-modeling framework while preserving short-horizon temporal structure (Chen et al., 14 Sep 2025).

The waypoint head predicts short-horizon 2D waypoints for the next four time steps:

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$6

These are also discretized and trained with cross-entropy:

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$7

The paper presents waypoint prediction as an auxiliary planning signal that improves trajectory smoothness and temporal consistency by giving the policy an explicit short-horizon geometric target (Chen et al., 14 Sep 2025).

Two auxiliary perception losses regularize the BEV representation. Semantic segmentation uses

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$8

and depth uses

$$\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).$$9

where $\mathbf{g}$0 denotes foreground pixels with valid depth. The full multi-task objective is

$\mathbf{g}$1

The paper’s interpretation is that direct control supervision, waypoint supervision, perception regularization, and control-guided attention are complementary rather than competing objectives (Chen et al., 14 Sep 2025).

5. Empirical evaluation

The experiments are conducted in CARLA 0.9.11 in a parking-lot scenario with 64 slots. The baselines are reproduced E2EParking, the original E2EParking results reported by its authors, and a modular Hybrid A* pipeline. The full CAA-Policy model achieves the best overall results among the tested methods (Chen et al., 14 Sep 2025).

Method	TSR	Other reported rates
CAA-Policy	87.5 ± 3.7%	TFR 2.3 ± 0.9%, NTSR 0.8 ± 0.6%, CR 3.5 ± 2.6%, TR 3.8 ± 2.2%
Hybrid A*	59.1%	CR 12.2%, TR 21.1%
Reproduced E2EParking	26.8%	CR 34.2%
Original E2EParking	74.7%	CR 12.0%

These figures are presented as evidence that the proposed combination of CAA, explicit goal tokenization, waypoint prediction, and motion tracking outperforms both a modular planner-controller stack and earlier end-to-end parking baselines (Chen et al., 14 Sep 2025).

The ablation study is central to the paper’s interpretation of the system. It reports that CAA alone is not sufficient and can even hurt performance if the backbone provides weak guidance. By contrast, TTM alone greatly improves TSR by making the target explicit, and waypoints alone also strongly improve TSR while reducing collision rate. The best result is obtained by combining TTM + CAA + Waypoints, which the paper describes as a genuine synergy among explicit goal representation, short-horizon planning supervision, and control-guided attention (Chen et al., 14 Sep 2025).

The appendix further compares target and ego tracking strategies. It finds that multi-frame dynamics outperforms segmentation-based tracking and single-frame dynamics in final displacement error and variance, supporting the design choice of using historical motion frames for more stable tracking (Chen et al., 14 Sep 2025).

6. Interpretation, relation to prior parking systems, and remaining issues

CAA-Policy occupies an intermediate position between purely modular parking stacks and earlier end-to-end controllers. Relative to modular systems, it removes online planning while retaining interpretability through attention maps, explicit goal tokenization, waypoint prediction, and a dedicated motion predictor. Relative to prior end-to-end parking approaches, it augments direct image-to-control learning with an explicit mechanism that ties perception to the control pathway itself (Chen et al., 14 Sep 2025).

The paper reports that the method is more robust in unseen parking slots and less sensitive to perturbations in target or speed. It also emphasizes interpretability: the learned attention maps concentrate around the target slot and nearby obstacles rather than scattering over irrelevant visual regions. This is presented as a practical benefit in parking, where high-level route selection is less difficult than accurate local maneuvering in cluttered geometry (Chen et al., 14 Sep 2025).

The remaining errors are described as relatively rare and mostly concentrated near the target slot. The paper interprets this as evidence that the policy is strong in the global-to-local approach phase but less certain in the final fine-positioning stage. A plausible implication is that, within the system’s current design, the largest unresolved difficulty is not coarse parking intent but precision in the terminal maneuver (Chen et al., 14 Sep 2025).

In that sense, CAA-Policy is best understood as a structured end-to-end parking policy: it preserves direct policy learning while reorganizing supervision so that visual attention is trained from control sensitivity, short-horizon geometry is modeled explicitly, and inference-time state tracking is stabilized through temporal motion prediction. Within the scope of the reported CARLA experiments, this design yields the highest parking success rate and the lowest collision and timeout rates among the compared systems (Chen et al., 14 Sep 2025).