Domain-Invariant NBV Planner

• Domain-invariant NBV planner is a reinforcement learning framework that uses robust, physically stable visual features to guide viewpoint transitions for active self-localization.
• It achieves efficient localization by selecting actions that maximize information gain while minimizing travel, leveraging descriptors like RRF, OLC, and ILC.
• Experimental protocols on datasets such as NCLT demonstrate improvements in mean reciprocal rank and cost efficiency compared to traditional single-view and heuristic approaches.

A domain-invariant next-best-view (NBV) planner is a reinforcement learning-based framework for autonomous visual place recognition (VPR) and robot self-localization, designed to generalize across varying environmental domains—such as seasons, illumination, and weather—without retraining. By focusing on visual cues or scene descriptors that are physically or statistically stable, domain-invariant NBV frameworks achieve robust, active localization by selecting actions (viewpoint transitions) that maximize information gain in unfamiliar or changing conditions.

1. Conceptual Foundations

The domain-invariant NBV planner formalizes active self-localization as a Markov Decision Process (MDP) $$\langle \mathcal S, \mathcal A, \mathcal O, T, R, \gamma \rangle$$. Here, states $\mathcal S$ are domain-invariant representations of the robot's current sensory input; actions $\mathcal A$ are motion primitives (e.g., forward steps of varying length); observations $\mathcal O$ are new sensor readings following each action; transitions $T$ are usually deterministic along the robot's path; and the reward function $R$ is formulated to promote informative viewpoint selection and penalize unnecessary travel.

Domain invariance is achieved by using scene representations that are robust to photometric, seasonal, and transient variation. Approaches vary in their construction of such representations but converge on the insight that only physically stable elements—such as pole-like landmarks or relative similarity rankings to a persistent prototype set—can achieve generalization without per-domain retraining (Kurauchi et al., 2022, Tanaka, 2021, Kanji, 2021).

2. Domain-Invariant Visual Representations

Different systems instantiate domain invariance through carefully constructed perceptual pipelines:

• Pole-Like Landmark Detection (PLD): A multi-scale, multi-encoder deep convolutional network detects vertical, pole-like structures, discarding color and texture (Tanaka, 2021). The pipeline produces a compact 4D feature vector per image via spatial pooling and vector quantization, summarizing only physically persistent geometric cues.
• Output and Intermediate Layer Cues (OLC, ILC): A convolutional neural network (CNN) trained for VPR exposes two key representations: OLC (the output layer's place-specific probabilistic distribution vector, or PDV) provides a world-centric belief about place class via a softmax and Bayes filter; ILC (intermediate activations) encodes ego-centric visual saliency maps, identifying the most informative image regions by meaningful perturbation methods. Each is transformed into a reciprocal-rank fusion (RRF) feature to ensure robustness to outliers and to decorrelate scale (Kurauchi et al., 2022).
• Reciprocal Rank Feature (RRF) Descriptor: The SIMBAD-based approach uses deep features (e.g., NetVLAD) to compute dissimilarity to a curated set of landmark prototypes, then represents each scene by the rank ordering of its $h$ closest landmarks, yielding a sparse, domain-stable vector (Kanji, 2021).

Representation	Dimensionality	Invariance Method	Source
PLD+SLA	4	Geometric + quantization	(Tanaka, 2021)
OLC	$|C|$ (classes)	Bayes+softmax fusion	(Kurauchi et al., 2022)
ILC	$|A|$ (actions)	Saliency+dim. reduction	(Kurauchi et al., 2022)
RRF (SIMBAD)	$r$ (landmarks)	Rank ordering	(Kanji, 2021)

3. State Construction and Policy Learning

The planner’s state vector is a concatenation or summary of domain-invariant features from the perception pipeline. For OLC+ILC fusion, the state at time $t$ is $$s_t = [\mathrm{RRF}(\text{OLC});\ \mathrm{RRF}(\text{ILC})] \in \mathbb{R}^{|C|+|A|}$$ (Kurauchi et al., 2022). For landmark-based systems, the state is simply the RRF descriptor of the current image, typically ultra-sparse (e.g., $h$-hot in $r=500$) (Kanji, 2021).

Policy learning is cast as deep reinforcement learning—usually Deep Q-Learning Network (DQN) or nearest-neighbor Q-learning (NNQL):

• DQN: States are fed to a multi-layer MLP, which outputs Q-values for each action. Off-policy updates are performed with a target network, experience replay, and periodic target synchronization. Losses are standard squared TD errors.
• NNQL: For high-dimensional or sparse RRF state spaces, Q-values are indexed by prototype scene descriptors, and the Q-value for a new state is approximated by averaging the values of its $k$ nearest neighbors in prototype space (Kanji, 2021). This structure allows efficient storage in an incremental inverted index.

4. Action Spaces, Reward Design, and Planning

Action spaces are tailored to realistic robot mobility constraints—typically discretized forward motions along a reference trajectory:

• $$A = \{1\ \mathrm{m}, 2\ \mathrm{m}, ..., 30\ \mathrm{m}\}$$ in OLC+ILC fusion (Kurauchi et al., 2022).
• $A = \{\text{short step},\ \text{long step}\}$ in PLD frameworks (Tanaka, 2021).
• $A = \{1, 2, ..., 10\}$ for SIMBAD’s landmark pipeline (Kanji, 2021).

Reward functions vary in timing and design:

• Terminal Reward (VPR success): Nonzero only at the final step; +1 if the top predicted place matches ground-truth class, −1 otherwise (Kurauchi et al., 2022).
• Reward Shaping (Landmark Detection): Per-step +1 if a landmark is detected, minus a travel penalty scaled by actual distance (Tanaka, 2021).
• Posterior Rank Reward: +100 if the robot’s location is in the top 10% of the VPR posterior after action; 0 otherwise (Kanji, 2021).

The planning process consists of observing the current state, selecting an action by $\epsilon$-greedy (DQN) or Boltzmann exploration (PLD), executing the motion, updating the belief or descriptor, and repeating for $T$ steps. Policies can generalize to new domains by virtue of invariant state representations.

5. Experimental Protocols and Comparative Results

All evaluated systems conduct experiments on the NCLT dataset, using distinct seasonal sessions to probe domain generalization. Common protocols include training solely on one domain (e.g., a summer traversal), then testing on multiple held-out domains (e.g., winter, spring, autumn, different illumination).

Performance is quantified by domain-agnostic localization accuracy and NBV efficiency metrics:

• Mean Reciprocal Rank (MRR): Assesses correct place identification; higher is better. The OLC+ILC (proposed) planner consistently attains higher MRR across five test sessions, e.g., $0.647$ (1/8), $0.493$ (1/15), compared to OLC-only, ILC-only, random, and single-view baselines (Kurauchi et al., 2022).
• Step Efficiency / Cost: Number of steps until localization confidence exceeds a predefined gap. The PLD-based "Learned" planner achieves lower steps-to-confidence (12.4) and higher accuracy (rank 2.1) than heuristics or constant-step baselines (Tanaka, 2021).
• Averaged Normalized Rank (ANR): Used in SIMBAD; the proposed NNQL NBV yields improvement over random and fixed-step across all domain splits (Kanji, 2021).

Method	MRR (Representative)	Cost	Rank
OLC+ILC (proposed)	0.647	—	—
OLC-only	0.619	—	—
ILC-only	0.625	—	—
PLD-DQN (Learned)	—	12.4	2.1
Heuristics	—	14.7	5.3
Random/Single-view	0.547	—	—

The combined OLC+ILC and RRF-based approaches provide robust cross-domain localization and efficient action policies with low computational and storage overhead.

6. Data Structures, Efficiency, and Practical Implementation

Domain-invariant NBV planners are engineered for high data and compute efficiency:

• Incremental Inverted Index (SIMBAD/RRF): Maps from landmark/rank pairs to image IDs, enabling rapid retrieval of relevant experiences with minimal memory (e.g., 1.4 MB for 26K images, $h=4$) (Kanji, 2021).
• Compact Feature Storage: RRF descriptors can be as short as 40 bits per scene (Kanji, 2021), while PLD encodings are 4-dimensional.
• Computational Costs: Dominated by deep feature extraction (e.g., NetVLAD), but real-time performance is attainable (e.g., 8.6 ms GPU, 495 ms CPU for VPR per viewpoint) (Kanji, 2021).
• REPRO-critical Hyperparameters: For OLC+ILC DQN: $\gamma=0.99$, learning rate $10^{-4}$, batch size 32, replay size $10^5$ (Kurauchi et al., 2022). For PLD-DQN: Adam optimizer, learning rate $10^{-3}$, $\gamma=0.99$, batch 64, replay $10^6$ (Tanaka, 2021). For NNQL: learning rate 0.1 (Kanji, 2021).

Implementation for new robotic platforms requires minimal adaptation: retrain or fine-tune the VPR CNN and saliency or PLD model on a single reference trajectory, adjust the motion model, and apply the same state extraction and NBV learning pipeline (Kurauchi et al., 2022).

7. Limitations, Ablations, and Future Extensions

Domain-invariant NBV planners achieve generalization by discarding transient visual features and focusing on geometrically or statistically persistent cues. Limitations include:

• Motion Dimensionality: Current systems primarily operate in 1D (forward-only); real-world navigation demands 2D or 3D NBV primitives (Tanaka, 2021).
• Manual Annotation: PLD-based approaches require one-time manual pole labeling; automating via self-supervised methods (e.g., pole-SLAM) is a research direction (Tanaka, 2021).
• Descriptor Compression: While RRF is already highly compact, further compression via binary hashing is under consideration (Kanji, 2021).
• Landmark Set Utility: Static landmark sets do not account for visibility or occlusion—a limitation for highly dynamic environments (Kanji, 2021).
• Computational Scaling: Dissimilarity computation (NetVLAD+L2) for RRF is a bottleneck during RL training, suggesting the need for approximate distances or caching (Kanji, 2021).

Future enhancements involve extending action sets to include yaw/pitch, employing prioritized replay or dueling DQN architectures, multi-modal perception, and optimized selection of landmarks for maximally informative NBV planning. The domain-invariant NBV planning paradigm establishes a foundation for lifelong robotic autonomy, with demonstrated robustness across domain shifts and explicit efficiency advantages over per-domain retraining (Kurauchi et al., 2022, Tanaka, 2021, Kanji, 2021).