---
title: 'CREStE: Scalable Mapless Navigation'
url: https://www.emergentmind.com/topics/creste
type: topic
---

# CREStE: Scalable Mapless Navigation

Searching arXiv for the specified CREStE paper and a few closely related navigation/IRL works for context.
CREStE is a scalable learning-based mapless navigation framework for outdoor navigation in city, residential, and offroad environments. It is designed to address two coupled problems in open-world autonomy: learning perceptual representations that generalize to open-set factors such as novel semantic classes, terrains, and dynamic entities, and inferring expert-aligned navigation costs from limited demonstrations. The framework combines a visual foundation model distillation objective for structured bird’s-eye-view perception with counterfactual inverse reinforcement learning for reward inference, and it was evaluated on kilometer-scale mapless navigation, where it was reported to outperform state-of-the-art approaches with 70% fewer human interventions, including a 2-kilometer mission in an unseen environment with just 1 intervention [2503.03921].

## 1. Problem setting and design goals

CREStE addresses the open-world generalization and robustness challenges of outdoor urban navigation. The motivating failure modes are those of learning-based mapless systems that operate without HD-maps and without reliance on finely annotated semantic lists. In this formulation, the navigation stack must remain effective under distribution shift across cities, residential areas, parks, and offroad terrain, while also handling dynamic agents and rare edge cases.

The framework targets two design requirements. First, perception must transfer to open-set conditions rather than overfitting to a fixed ontology of classes or terrains. Second, reward learning must extract the cues that actually govern expert behavior from a limited number of demonstrations. The stated premise is that expert trajectories alone do not reveal which environmental factors were decisive, particularly when many plausible but undesirable alternatives exist. CREStE therefore treats perception and reward inference as jointly limiting factors in scalable mapless navigation.

This positioning places CREStE at the intersection of mapless navigation, structured representation learning, inverse reinforcement learning, and active learning. A plausible implication is that its contribution is not a single planner or policy module, but an interface between open-set scene understanding and expert-aligned cost inference.

## 2. System architecture and structured representations

At a systems level, CREStE is organized around a perceptual encoder, a structured bird’s-eye-view representation, and a reward-learning component. The encoder is trained to distill information from multiple visual foundation models, after which the resulting features are lifted into a local BEV grid and used for planning. From RGB-D input, high-dimensional image features are “lifted and splatted” into BEV grids over a 6-meter planning window, and an inpainting stage fills occlusions.

The BEV representation is explicitly structured. It contains static panoptic features, dynamic panoptic features, and elevation maps. Static channels encode elements such as sidewalks, buildings, and curb cuts; dynamic channels encode pedestrians and vehicles; elevation captures terrain geometry. This structure is central to CREStE’s claim of robustness, because each BEV pixel is tied to a spatially meaningful region of the robot’s surroundings rather than remaining in an image-centric latent space.

| Component | Function | Reported role |
|---|---|---|
| Perceptual encoder | Distills VFM knowledge | Open-set semantic, geometric, and entity-aware features |
| Structured BEV | Static, dynamic, elevation channels | Spatially grounded planning and interpretability |
| Counterfactual IRL | Learns navigation costs from expert and counterfactual trajectories | Expert alignment from limited demonstrations |

The representation is intended to support traversability reasoning in settings where novel terrain or uncommon semantic structure appears at test time. The paper’s qualitative examples emphasize curb cuts, crosswalks, grass versus concrete, and risky slopes, suggesting that CREStE is designed to reason about affordances rather than only object categories.

## 3. Visual foundation model distillation

A defining element of CREStE is its visual foundation model distillation objective. The perceptual encoder is trained to distill knowledge from multiple VFMs pretrained on large heterogeneous internet data. The paper identifies Dinov2 as the source for semantics and geometry and SegmentAnything/SAM2 as the source for panoptic, entity-aware understanding [2503.03921].

The resulting supervision is multi-channel. A semantic decoder head regresses to Dinov2 features, depth completion uses cross-entropy on discretized depth values, and BEV heads are trained on projected instance masks for static and dynamic content. The overall formulation is summarized as
$$
\mathcal{L}_{\mathrm{CREStE}}=\mathcal{L}_{\mathrm{rgbd}}+\mathcal{L}_{\mathrm{bev}}+\mathcal{L}_{\mathrm{IRL}}.
$$

The significance of this construction lies in how it operationalizes “internet-scale priors.” Rather than enumerating a closed class list for robotics data collection, CREStE distills broad semantic and geometric regularities from foundation models into a planning-oriented BEV representation. The paper presents this as a way to inherit invariances to lighting, weather, and viewpoint while preserving the spatial structure needed for navigation.

A common misconception is to equate this with generic image-feature transfer. CREStE does not stop at feature extraction in the image plane; it converts distilled semantics, entity structure, and geometry into a structured BEV representation intended for downstream cost inference and planning. In that sense, the distillation stage is coupled to navigation-specific spatial abstraction rather than used as a generic frozen backbone.

## 4. Counterfactual inverse reinforcement learning

The second principal contribution is counterfactual inverse reinforcement learning, presented as a novel active learning formulation for inferring navigation costs. Traditional imitation learning or conventional IRL is described as insufficient for identifying which map features are salient to humans, because expert demonstrations provide positive behavior but not the nearby alternatives that would clarify why a choice was made.

CREStE introduces counterfactual trajectory demonstrations: plausible alternate trajectories that violate human navigation preferences, such as cutting across grass or failing to use a crosswalk. Reward learning then compares expert trajectories, suboptimal counterfactual trajectories, and the current policy. The objective is written as
$$
\min_\pi \max_\phi \; \mathbb{E}_{\rho^E}[r_\phi(s,a,y_{BEV})]
-\left(\alpha \mathbb{E}_{\rho^S}[r_\phi(s,a,y_{BEV})] + (1-\alpha)\mathbb{E}_{\rho^\pi}[r_\phi(s,a,y_{BEV})]\right),
$$
where $\rho^E$ denotes expert trajectories, $\rho^S$ suboptimal or counterfactual trajectories, and $\rho^\pi$ the current policy distribution [2503.03921].

The associated active learning cycle is: warm-start on expert demonstrations, identify divergences between the learned policy and expert paths, generate alternate trajectories, obtain human judgments about which counterfactuals violate preferences, and retrain the reward. The paper’s interpretation is that counterfactuals concentrate supervision near the decision boundary. This suggests that CREStE is designed to learn sparse but decisive cues—such as curb gaps or crosswalk structure—that may be underdetermined by expert-only data.

The paper also points to derivations using Bradley-Terry preference models and connections to classic IRL. Within the framework’s logic, the reward is not merely fitted to path imitation; it is refined through explicit comparisons between acceptable and unacceptable local alternatives.

## 5. Empirical evaluation and ablations

CREStE was evaluated on real robot deployments across six urban and offroad locations, using short-horizon and long-horizon tasks. Reported baselines include state-of-the-art IRL, multi-factor methods such as PACER+G, end-to-end imitation learning with ViNT, and VLM-based approaches such as PIVOT/GPT-4o. The evaluation reports AST, %S, NIR, Dist. (m), and total interventions, where AST is average subgoal completion time, %S is percentage of subgoals completed, NIR is necessary interventions per 100 m, Dist. is total distance driven before failure, and total interventions counts human overrides [2503.03921].

The headline result is a 70% reduction in human interventions relative to prior state of the art. In Hemphill Park, described as an unseen environment, CREStE achieved $\mathrm{NIR}=0.0$, compared with PACER+G at $\mathrm{NIR}=9.47$ and ViNT at $\mathrm{NIR}=10.96$. In a long-horizon 2 km urban trail mission, CREStE completed the route with 1 intervention and $\mathrm{NIR}=0.05$, whereas PACER+G averaged 48 interventions with $\mathrm{NIR}=3.56$ and covered only 1.3 km before failure.

The ablation study is structurally important. Removing the BEV inpainting backbone increased interventions by 28%. Removing counterfactuals increased interventions by approximately 70%. The paper also reports that fusing LiDAR or stereo improves robustness to lighting relative to monocular RGB only. These results are presented as evidence that both structured BEV perception and counterfactual reward learning materially contribute to long-horizon robustness.

Qualitatively, CREStE is reported to infer traversability around curb cuts, across crosswalks, and over grass versus concrete, while avoiding risky slopes, including cases where those features do not appear in labeled training data. The intended interpretation is that the model leverages open-set priors and counterfactual guidance rather than memorizing a closed catalog of traversable patterns.

## 6. Interpretation, scope, and place in mapless navigation

CREStE is best understood as a framework for mapless navigation rather than a claim that maps are unnecessary in all autonomy regimes. Its contribution is specifically to scalable learning-based navigation without HD-maps, using structured perceptual priors and counterfactual reward learning to reduce hand-engineering. It also differs from direct end-to-end imitation learning, because its decision process is mediated by a structured BEV and an inferred reward rather than by direct policy cloning.

The paper frames CREStE against several limitations in prior work: geometric or multi-factor pipelines with curated semantic lists, end-to-end methods that overfit and require large expert datasets, and zero-shot LLM/VLM navigation that is not tuned for actual navigation. CREStE’s response to these limitations is, respectively, distillation of unbounded semantic/entity/geometric priors, data-efficient reward learning via counterfactuals, and active discovery of the cues that matter most for navigation. A plausible implication is that the framework aims to scale by shifting supervision from exhaustive annotation toward foundation-model transfer plus targeted human preference correction.

The structured BEV reward maps are also presented as interpretability tools. The paper argues that when failure occurs, it becomes possible to inspect whether the source lies in visual input, semantic reasoning, or reward mapping. For robotics researchers, this is significant because it positions CREStE not only as a performance-driven system but also as a diagnostic architecture.

Within mapless outdoor navigation, CREStE therefore represents a synthesis of three ideas: open-set perceptual transfer from visual foundation models, spatial grounding through structured BEV embeddings, and active counterfactual guidance for reward learning. Its reported performance on kilometer-scale missions, together with the ablation evidence, identifies those three elements as the central content of the method rather than ancillary engineering choices [2503.03921].

Source: https://www.emergentmind.com/topics/creste