---
title: 'CAA-Policy: End-to-End Autonomous Parking'
url: https://www.emergentmind.com/topics/caa-policy
type: topic
---

# CAA-Policy: End-to-End Autonomous Parking

Searching arXiv for the specified paper and closely related parking-policy work to ground the article.
arXiv search query: 2509.11090 autonomous parking control-aided attention E2EParking
CAA-Policy is an end-to-end imitation-learning system for autonomous parking that uses **Control-Aided Attention (CAA)** to make control signals guide the learning of visual attention, rather than relying on task-loss supervision alone. The method takes six surround-view RGB images, the current vehicle state, and the target parking slot in ego coordinates, and produces short-horizon control commands, short-horizon waypoints, a next-position estimate, and auxiliary perception predictions. Its central claim is that attention trained from gradients backpropagated from the control outputs focuses on regions that most influence action decisions, yielding a policy that is more robust, more temporally stable, and more interpretable in parking scenarios. The system is evaluated in CARLA against both end-to-end and modular baselines [2509.11090].

## 1. Motivation and problem setting

CAA-Policy is motivated by a specific weakness in end-to-end parking systems: perception features are often learned only from task-loss supervision, so the resulting attention can drift toward visually salient but control-irrelevant regions. The paper argues that this is especially problematic in autonomous parking, where small errors near the target slot can lead to collisions, wrong-slot parking, or timeouts [2509.11090].

The method is positioned against two existing tendencies. In prior end-to-end parking systems such as **E2EParking**, images are mapped directly to control, but the model is not explicitly forced to focus on control-sensitive structures such as parking-slot boundaries, nearby obstacle edges, free-space gaps, and the target-slot vicinity. In transformer/self-attention or CBAM-style variants learned only from task supervision, attention is described as spatially scattered, unstable across time, and weakly aligned with decision-critical regions. In contrast, classical modular parking stacks based on perception, mapping, Hybrid A* planning, and PID or bicycle-model control are interpretable and usually stable, but they suffer from error accumulation across modules, expensive planning, and poor handling of delayed or outdated perception [2509.11090].

Within this setting, CAA-Policy is defined as a parking policy that remains end-to-end in its control output while injecting structure through explicit goal encoding, short-horizon waypoint prediction, a learnable motion predictor, and control-guided attention. The paper’s framing is therefore not a rejection of end-to-end learning, but a reorganization of it around policy-relevant visual focus [2509.11090].

## 2. Architecture and information flow

CAA-Policy is organized into five main stages: **perception backbone**, **feature fusion**, **learnable motion prediction module**, **Control-Aided Attention module**, and **prediction heads**. Auxiliary heads for **depth estimation** and **BEV semantic segmentation** are also included [2509.11090].

The perception backbone uses a shared **ResNet-18** to encode the six RGB views. These features are fused into a bird’s-eye-view representation through **Lift-Splat-Shoot (LSS)**. The resulting top-down feature map is then combined with ego-state features and goal features. The fusion is written as
$$
\mathbf{f}_{\text{BEV}} = \mathrm{CNN}(\mathbf{I}_{1:N}), \quad
\mathbf{f}_{\text{ego}} = \mathrm{MLP}(\mathbf{e}), \quad
\mathbf{f}_{\text{goal}} = \mathrm{MLP}(\mathbf{g}),
$$
$$
\mathbf{f}_{\text{fused}} = \mathrm{Concat}(\mathbf{f}_{\text{BEV}}, \mathbf{f}_{\text{ego}}, \mathbf{f}_{\text{goal}}).
$$
Here, $\mathbf{g}$ denotes the target slot in the ego frame. The explicit tokenization of the target is termed the **Target Tokenization Module (TTM)** and is intended to make the parking objective more explicit than implicit goal conditioning [2509.11090].

The learnable motion prediction module is used mainly at inference time to track the vehicle’s current position and the target slot more robustly over time. It consumes the past four frames of control commands and vehicle states, using a CNN followed by an LSTM:
$$
h_t = \mathrm{LSTM}\!\left(\{\mathrm{CNN}([c_k, s_k])\}_{k=t-4}^{t-1}\right).
$$
Two heads then predict the mean and variance of a 2D displacement:
$$
\boldsymbol{\mu}_t = W_{\mu}h_t + b_{\mu}, \qquad
\log \boldsymbol{\sigma}_t^2 = W_{\sigma}h_t + b_{\sigma}.
$$
The position update is
$$
(x_t, y_t) = (x_{t-1}, y_{t-1}) + \boldsymbol{\mu}_t.
$$
Training uses a Gaussian negative log-likelihood loss on the ground-truth displacement:
$$
\mathcal{L}_{\text{dyn}} = -\log \mathcal{N}\!\left(\Delta \mathbf{x}_t^{\text{GT}} \mid \boldsymbol{\mu}_t, \boldsymbol{\sigma}_t^2 \right).
$$
The paper notes that this module is **not needed during training**, because ground-truth poses are available then; it primarily serves inference-time tracking [2509.11090].

## 3. Control-Aided Attention

The CAA mechanism is the defining component of CAA-Policy. It predicts a BEV-shaped spatial attention map intended to represent **control sensitivity** at each location in the fused feature tensor. If a spatial region strongly influences the control output, the attention assigned to that region should be high [2509.11090].

Given fused features
$$
\mathbf{f}_{\text{fused}} \in \mathbb{R}^{C_b \times X_b \times Y_b},
$$
the model predicts an attention map
$$
\widehat{\nabla}_f = \mathrm{MLP}(\mathbf{f}_{\text{fused}}),
$$
with spatial size $X_b \times Y_b$. The attention is broadcast over channels and applied multiplicatively:
$$
\mathbf{f}_{\text{CAA}}(c,i,j) = \mathbf{f}_{\text{fused}}(c,i,j)\cdot \widehat{\nabla}_f(i,j).
$$
Operationally, the module acts as a spatial gate over BEV features [2509.11090].

What makes the mechanism “control-aided” is the supervision target. Instead of learning attention from manually labeled masks or from the ordinary training loss, the target attention is derived from the gradient of the control output with respect to the fused perception features:
$$
\nabla_f^\ast = \frac{\partial \text{Control}}{\partial \mathbf{f}_{\text{fused}}}.
$$
The predicted attention map is trained to match this gradient-derived signal through
$$
\mathcal{L}_{\text{CAA}} = \left\| A - \nabla_f^\ast \right\|_1.
$$
The paper characterizes this as a **self-supervised** training strategy because the attention target is obtained from the model’s own control gradients rather than from human-provided annotations [2509.11090].

The conceptual consequence is that the attention module is encouraged to focus on visual features that induce high variance in action outputs, not merely on features that minimize the task loss. The paper reports that, qualitatively, attention without CAA is scattered, whereas with CAA it concentrates around the target slot and nearby obstacles. This suggests a tighter alignment between the internal attention distribution and the downstream control decision [2509.11090].

## 4. Prediction heads and training objective

The refined feature map $\mathbf{f}_{\text{CAA}}$ feeds two main prediction heads: a control head and a waypoint head. The control head follows the structure of E2EParking and predicts control tokens autoregressively. Ground-truth continuous commands
$$
\tilde{c}_t^{\text{GT}} = (\text{steering}_t,\text{throttle}_t,\text{brake}_t)
$$
are quantized into discrete tokens, and the control objective over the next four steps is
$$
\mathcal{L}_{\text{ctrl}} = \sum_{k=1}^{4} \text{CrossEntropy}(\hat{c}_{t+k}, c_{t+k}^{\text{GT}}).
$$
This keeps the control prediction within a sequence-modeling framework while preserving short-horizon temporal structure [2509.11090].

The waypoint head predicts short-horizon 2D waypoints for the next four time steps:
$$
\mathbf{w}_{t+1:t+4} = [(x_{t+1},y_{t+1}),\dots,(x_{t+4},y_{t+4})].
$$
These are also discretized and trained with cross-entropy:
$$
\mathcal{L}_{\text{wp}} = \sum_{k=1}^{4} \text{CrossEntropy}(\hat{w}_{t+k}, w_{t+k}^{\text{GT}}).
$$
The paper presents waypoint prediction as an auxiliary planning signal that improves trajectory smoothness and temporal consistency by giving the policy an explicit short-horizon geometric target [2509.11090].

Two auxiliary perception losses regularize the BEV representation. Semantic segmentation uses
$$
\mathcal{L}_{\text{seg}} = \text{CE}(\hat{M}, M),
$$
and depth uses
$$
\mathcal{L}_{\text{depth}} =
\frac{1}{N_{\mathcal{FG}}}\sum_{i \in \mathcal{FG}} \text{BCE}(\hat{D}_i, D_i),
$$
where $\mathcal{FG}$ denotes foreground pixels with valid depth. The full multi-task objective is
$$
\mathcal{L}_\text{total} =
\lambda_\text{ctrl}\mathcal{L}_\text{ctrl} +
\lambda_\text{wp}\mathcal{L}_\text{wp} +
\lambda_\text{seg}\mathcal{L}_\text{seg} +
\lambda_\text{depth}\mathcal{L}_\text{depth} +
\lambda_\text{CAA}\mathcal{L}_\text{CAA}.
$$
The paper’s interpretation is that direct control supervision, waypoint supervision, perception regularization, and control-guided attention are complementary rather than competing objectives [2509.11090].

## 5. Empirical evaluation

The experiments are conducted in **CARLA 0.9.11** in a parking-lot scenario with **64 slots**. The baselines are reproduced **E2EParking**, the original **E2EParking** results reported by its authors, and a modular **Hybrid A\*** pipeline. The full CAA-Policy model achieves the best overall results among the tested methods [2509.11090].

| Method | TSR | Other reported rates |
|---|---:|---|
| CAA-Policy | **87.5 ± 3.7%** | TFR **2.3 ± 0.9%**, NTSR **0.8 ± 0.6%**, CR **3.5 ± 2.6%**, TR **3.8 ± 2.2%** |
| Hybrid A* | 59.1% | CR 12.2%, TR 21.1% |
| Reproduced E2EParking | 26.8% | CR 34.2% |
| Original E2EParking | 74.7% | CR 12.0% |

These figures are presented as evidence that the proposed combination of CAA, explicit goal tokenization, waypoint prediction, and motion tracking outperforms both a modular planner-controller stack and earlier end-to-end parking baselines [2509.11090].

The ablation study is central to the paper’s interpretation of the system. It reports that **CAA alone is not sufficient and can even hurt performance if the backbone provides weak guidance**. By contrast, **TTM alone** greatly improves TSR by making the target explicit, and **waypoints alone** also strongly improve TSR while reducing collision rate. The best result is obtained by combining **TTM + CAA + Waypoints**, which the paper describes as a genuine synergy among explicit goal representation, short-horizon planning supervision, and control-guided attention [2509.11090].

The appendix further compares target and ego tracking strategies. It finds that **multi-frame dynamics** outperforms segmentation-based tracking and single-frame dynamics in final displacement error and variance, supporting the design choice of using historical motion frames for more stable tracking [2509.11090].

## 6. Interpretation, relation to prior parking systems, and remaining issues

CAA-Policy occupies an intermediate position between purely modular parking stacks and earlier end-to-end controllers. Relative to modular systems, it removes online planning while retaining interpretability through attention maps, explicit goal tokenization, waypoint prediction, and a dedicated motion predictor. Relative to prior end-to-end parking approaches, it augments direct image-to-control learning with an explicit mechanism that ties perception to the control pathway itself [2509.11090].

The paper reports that the method is more robust in unseen parking slots and less sensitive to perturbations in target or speed. It also emphasizes interpretability: the learned attention maps concentrate around the target slot and nearby obstacles rather than scattering over irrelevant visual regions. This is presented as a practical benefit in parking, where high-level route selection is less difficult than accurate local maneuvering in cluttered geometry [2509.11090].

The remaining errors are described as relatively rare and mostly concentrated near the target slot. The paper interprets this as evidence that the policy is strong in the global-to-local approach phase but less certain in the final fine-positioning stage. A plausible implication is that, within the system’s current design, the largest unresolved difficulty is not coarse parking intent but precision in the terminal maneuver [2509.11090].

In that sense, CAA-Policy is best understood as a structured end-to-end parking policy: it preserves direct policy learning while reorganizing supervision so that visual attention is trained from control sensitivity, short-horizon geometry is modeled explicitly, and inference-time state tracking is stabilized through temporal motion prediction. Within the scope of the reported CARLA experiments, this design yields the highest parking success rate and the lowest collision and timeout rates among the compared systems [2509.11090].

Source: https://www.emergentmind.com/topics/caa-policy