LifelongPR: Continual PCPR Learning

• LifelongPR is a continual learning framework for point cloud place recognition that addresses catastrophic forgetting and domain shifts through replay and prompt learning.
• It integrates an information-theoretic, spatially aware replay mechanism with a lightweight prompt module and a two-stage adaptation strategy to enhance model stability and recall.
• Ablation studies demonstrate improvements of up to 7.96% in mR@1 and 8.95% reduction in forgetting compared to baseline methods in dynamic environments.

Searching arXiv for the specified paper and closely related point-cloud place recognition continual-learning work. LifelongPR is a continual learning framework for point cloud place recognition (PCPR) that is designed for the domain-incremental setting in which sequential datasets or domains arrive one at a time and only the current domain together with a small replay buffer are available during training. Introduced for large-scale PCPR in photogrammetry and robotics applications, it addresses two coupled failure modes of sequential PCPR adaptation: catastrophic forgetting of previously learned scenes and performance degradation under domain shifts induced by different cities or LiDAR sensors. Its central design combines an information-theoretic, spatially aware replay mechanism with a prompt learning-based continual-learning pipeline and a two-stage training strategy, with the stated objective of continuously acquiring, updating, and accumulating knowledge from sequential point-cloud data while maintaining performance on earlier domains (Zou et al., 14 Jul 2025).

1. Problem setting and formal objective

LifelongPR considers continual, specifically domain-incremental, learning for PCPR. Let $T$ denote the total number of sequential tasks or domains, and let $D_1,\dots,D_T$ denote the training sets arriving one at a time, where each $D_t$ contains $N_t$ submaps, each submap being a 3D point cloud. The PCPR network is denoted $F(\theta)$. At stage $t$, only $D_t$ and a small replay buffer $M_{1:t-1}$ drawn from previous datasets $D_{1:t-1}$ are accessible; storing all prior data is disallowed. The optimization target at stage $t$ is

$D_1,\dots,D_T$0

where $D_1,\dots,D_T$1 is the place-recognition loss, $D_1,\dots,D_T$2 is a knowledge-distillation term on replay samples, and $D_1,\dots,D_T$3 is the epoch index (Zou et al., 14 Jul 2025).

The notation used in the framework fixes $D_1,\dots,D_T$4, a fixed total replay capacity $D_1,\dots,D_T$5, and per-dataset replay allocations $D_1,\dots,D_T$6 satisfying $D_1,\dots,D_T$7. Performance is evaluated through $D_1,\dots,D_T$8, the $D_1,\dots,D_T$9 on test set $D_t$0 after training stage $D_t$1. This setup explicitly targets the case in which environments are sequential, heterogeneous, and non-stationary, rather than jointly accessible.

A common simplification is to treat lifelong PCPR as ordinary fine-tuning with a memory buffer. LifelongPR rejects that simplification in two ways. First, it makes replay allocation dataset-aware rather than uniform. Second, it treats domain adaptation as an architectural problem via stage-specific prompts, rather than as replay alone. The paper’s ablations indicate that both choices contribute materially to the final performance.

2. Replay allocation and replay sample selection

The replay subsystem has two distinct components: deciding how many samples to retain from each previously seen dataset and deciding which specific samples to retain. The first component is based on a quantity called $D_t$2, which estimates the information content of a dataset $D_t$3. Given $D_t$4 samples with current-model embeddings $D_t$5, LifelongPR forms a Gaussian-kernel matrix

$D_t$6

With $D_t$7, $D_t$8, its effective rank is defined as

$D_t$9

and the dataset information quantity is

$N_t$0

Across all seen datasets $N_t$1, LifelongPR allocates replay capacity by a temperature-softmax:

$N_t$2

This procedure gives higher replay budgets to datasets whose embedding structure has higher effective rank, rather than enforcing class- or domain-uniform replay (Zou et al., 14 Jul 2025).

The second component selects samples by spatial-feature diversity. For a dataset $N_t$3, the memory subset $N_t$4 of size $N_t$5 is chosen to maximize

$N_t$6

where for each $N_t$7,

$N_t$8

The score combines normalized Euclidean distance $N_t$9 and cosine dissimilarity in feature space. Because the exact optimization is NP-hard, the method uses a greedy algorithm: initialize $F(\theta)$0; repeatedly sample a candidate set $F(\theta)$1 of size $F(\theta)$2; select $F(\theta)$3; and update $F(\theta)$4. After selecting $F(\theta)$5 for the current dataset, LifelongPR also prunes each previous memory $F(\theta)$6 to its new allocation $F(\theta)$7 using the same greedy rule.

This replay design has a specific empirical role. The ablation study reports that random replay yields poor performance, greedy spatial replay improves $F(\theta)$8 by $F(\theta)$9 and reduces $t$0 by $t$1, and adding $t$2 allocation yields a further $t$3 $t$4. The intended interpretation is that replay efficiency depends not only on per-sample diversity but also on cross-dataset budget allocation.

3. Prompt-learning architecture and two-stage continual adaptation

LifelongPR augments a PCPR backbone $t$5 with a lightweight prompt module $t$6 at stage $t$7, producing a combined model $t$8. The prompt module consists of an $t$9, a learnable prompt matrix $D_t$0, $D_t$1 attention layers that use key/value pairs from $D_t$2 and queries from $D_t$3, and an $D_t$4. For a raw point cloud $D_t$5,

$D_t$6

$D_t$7

$D_t$8

The insertion mode is backbone-dependent. For sparse-convolution backbones such as MinkLoc3D, $D_t$9 is concatenated with raw points as extra channels. For PointNet-style backbones, $M_{1:t-1}$0 is added to intermediate features (Zou et al., 14 Jul 2025).

Training proceeds in two stages rather than as a joint optimization. In Stage 1, Prompt Warm-up, the backbone parameters $M_{1:t-1}$1 are frozen and only the prompt is trained on $M_{1:t-1}$2. In Stage 2, Backbone Adaptation, the prompt is frozen and $M_{1:t-1}$3 is fine-tuned on the same current-plus-replay data. Both stages use the same total loss,

$M_{1:t-1}$4

The place-recognition term is triplet loss, and the knowledge-distillation term keeps the current model’s outputs on replay samples close to those of the previous model. The framework is intended to achieve domain-specific feature adaptation with low additional parameter overhead while minimizing forgetting.

A recurrent misconception is that the prompt module merely acts as a small auxiliary encoder. The paper’s description is more specific: it is a domain-adaptive modulation mechanism whose effect depends on the insertion point and on the staged training schedule. The training-strategy ablation reports that, with PatchAugNet on Seq2, replacing two-stage training with one-stage joint training lowers $M_{1:t-1}$5 by $M_{1:t-1}$6 and increases $M_{1:t-1}$7 by $M_{1:t-1}$8. This suggests that optimization order, not only prompt capacity, is structurally important.

4. Experimental protocol, datasets, backbones, and metrics

The experimental study uses two continuous domain sequences. The first is

Sequence	Domains
Seq1	Oxford $M_{1:t-1}$9 DCC $D_{1:t-1}$0 Riverside $D_{1:t-1}$1 In-house
Seq2	Oxford $D_{1:t-1}$2 Hankou $D_{1:t-1}$3 WHU-Campus $D_{1:t-1}$4 In-house

After ground removal and normalization, each submap contains 4,096 points. The dataset details are: Oxford, captured by SICK LMS-151 in urban Oxford, with 21.7k training and 3.0k test submaps; DCC and Riverside, captured by Ouster OS1-64 on Seoul roads, with approximately 5.5k training and 15–18k test submaps each; In-house, captured by Velodyne HDL-64E in an urban environment, with 6.6k training and 1.7k test submaps; Hankou, captured by Livox Avia and Hi-Target in Wuhan, with 11.3k training and 2.5k test submaps; and WHU-Campus, captured by Livox and CHCNAV on a campus, with 4.0k training and 1.1k test submaps (Zou et al., 14 Jul 2025).

Three PCPR backbones are used: PointNetVLAD, PatchAugNet without the patch-augmentation branch, and MinkLoc3D. The continual-learning baselines are Fine-tuning (FT), InCloud (“InC”), which combines replay and distillation, and CCL, a contrastive continual-learning baseline. The metrics are $D_{1:t-1}$5, mean Recall, mean Incremental Recall, and Forgetting:

$D_{1:t-1}$6

$D_{1:t-1}$7

$D_{1:t-1}$8

Implementation details are fixed as follows: total replay budget $D_{1:t-1}$9, $t$0, $t$1, $t$2, $t$3, and $t$4. The prompt configuration uses $t$5, $t$6, $t$7, and an output dimension $t$8 matched to the backbone. Optimization uses Adam with learning rate $t$9 and 40 epochs per stage on an NVIDIA RTX 4080S.

5. Quantitative performance and ablation evidence

The central reported result is that LifelongPR outperforms the state of the art on continual PCPR. The abstract states improvements of $D_1,\dots,D_T$00 in $D_1,\dots,D_T$01, $D_1,\dots,D_T$02 in $D_1,\dots,D_T$03, and an $D_1,\dots,D_T$04 reduction in $D_1,\dots,D_T$05 relative to state-of-the-art methods. The detailed results on the harder Seq2 with MinkLoc3D specify the corresponding values as $D_1,\dots,D_T$06 $D_1,\dots,D_T$07, $D_1,\dots,D_T$08 $D_1,\dots,D_T$09, and $D_1,\dots,D_T$10 $D_1,\dots,D_T$11. Across all backbones and both sequences, LifelongPR is reported to outperform FT, InCloud, and CCL in average recall and forgetting (Zou et al., 14 Jul 2025).

The ablation study decomposes the contribution of each component. Random replay performs poorly. Replacing random replay with greedy spatial selection yields $D_1,\dots,D_T$12 $D_1,\dots,D_T$13 and $D_1,\dots,D_T$14 $D_1,\dots,D_T$15. Adding $D_1,\dots,D_T$16-based replay allocation contributes a further $D_1,\dots,D_T$17 $D_1,\dots,D_T$18. Adding the prompt module contributes $D_1,\dots,D_T$19 $D_1,\dots,D_T$20 and $D_1,\dots,D_T$21 $D_1,\dots,D_T$22. In the training-strategy ablation using PatchAugNet on Seq2, the two-stage strategy improves $D_1,\dots,D_T$23 by $D_1,\dots,D_T$24 and reduces $D_1,\dots,D_T$25 by $D_1,\dots,D_T$26 relative to one-stage joint optimization.

Qualitative evidence is also reported. t-SNE visualizations show that FT clusters old-domain features, InCloud and CCL partially preserve feature dispersion, and LifelongPR best maintains well-separated clusters for each domain. The paper also presents six street-scene query cases in which FT and CCL fail while LifelongPR retrieves correct matches.

These results support a specific reading of the method. LifelongPR is not solely a memory-selection method, nor solely a prompt-tuning method. The replay allocator, the spatially diverse greedy selector, the lightweight prompt module, and the two-stage optimization schedule are treated as jointly necessary for the reported stability-plasticity trade-off.

6. Significance, constraints, and future directions

Within the PCPR setting, LifelongPR frames continual place recognition as a knowledge-fusion problem under tight replay budgets. Its stated key takeaways are threefold: an information-theoretic and spatial-aware replay selection improves memory efficacy; a lightweight prompt module together with two-stage training yields domain-adaptive features without over-parameterization; and the resulting system yields up to $D_1,\dots,D_T$27 $D_1,\dots,D_T$28 and $D_1,\dots,D_T$29 $D_1,\dots,D_T$30 improvements over the state of the art (Zou et al., 14 Jul 2025).

The limitations are also explicit. Some degenerate or highly repetitive scenes, including T-junctions, remain challenging and may require multi-shot or temporal aggregation. The prompt module itself may undergo forgetting. Future work is proposed in online continual learning and stronger domain generalization, including contrastive or domain-generalization losses. The paper also provides code and pre-trained models, indicating an implementation-oriented research agenda rather than a purely conceptual proposal.

A plausible implication is that LifelongPR occupies a middle ground between classical replay-driven continual learning and parameter-efficient domain adaptation. It retains a fixed-capacity replay buffer, but it does not treat replay as sufficient; it uses prompts to absorb domain-specific variation and a staged optimizer to decouple prompt adaptation from backbone drift. In that sense, the framework is best understood as a continual PCPR system for sequential city- and sensor-level domain shifts, rather than as a generic memory replay baseline with auxiliary tuning.