---
title: 'LifelongPR: Continual PCPR Learning'
url: https://www.emergentmind.com/topics/lifelongpr
type: topic
---

# LifelongPR: Continual PCPR Learning

Searching arXiv for the specified paper and closely related point-cloud place recognition continual-learning work.
LifelongPR is a continual learning framework for point cloud place recognition (PCPR) that is designed for the domain-incremental setting in which sequential datasets or domains arrive one at a time and only the current domain together with a small replay buffer are available during training. Introduced for large-scale PCPR in photogrammetry and robotics applications, it addresses two coupled failure modes of sequential PCPR adaptation: catastrophic forgetting of previously learned scenes and performance degradation under domain shifts induced by different cities or LiDAR sensors. Its central design combines an information-theoretic, spatially aware replay mechanism with a prompt learning-based continual-learning pipeline and a two-stage training strategy, with the stated objective of continuously acquiring, updating, and accumulating knowledge from sequential point-cloud data while maintaining performance on earlier domains [2507.10034].

## 1. Problem setting and formal objective

LifelongPR considers continual, specifically domain-incremental, learning for PCPR. Let \(T\) denote the total number of sequential tasks or domains, and let \(D_1,\dots,D_T\) denote the training sets arriving one at a time, where each \(D_t\) contains \(N_t\) submaps, each submap being a 3D point cloud. The PCPR network is denoted \(F(\theta)\). At stage \(t\), only \(D_t\) and a small replay buffer \(M_{1:t-1}\) drawn from previous datasets \(D_{1:t-1}\) are accessible; storing all prior data is disallowed. The optimization target at stage \(t\) is

\[
L_{\text{total}}(\theta_t)=L_{\text{PR}}(\theta_t;D_t\cup M_{1:t-1})+\lambda^\gamma L_{\text{KD}}(\theta_t;M_{1:t-1}),
\]

where \(L_{\text{PR}}\) is the place-recognition loss, \(L_{\text{KD}}\) is a knowledge-distillation term on replay samples, and \(\gamma\) is the epoch index [2507.10034].

The notation used in the framework fixes \(M_{1:t}=\{M_1,\dots,M_t\}\), a fixed total replay capacity \(k_{\text{total}}=|M_{1:t}|\), and per-dataset replay allocations \(k_t\) satisfying \(\sum_i k_i=k_{\text{total}}\). Performance is evaluated through \(Recall_{t,i}\), the \(Recall@1\) on test set \(i\) after training stage \(t\). This setup explicitly targets the case in which environments are sequential, heterogeneous, and non-stationary, rather than jointly accessible.

A common simplification is to treat lifelong PCPR as ordinary fine-tuning with a memory buffer. LifelongPR rejects that simplification in two ways. First, it makes replay allocation dataset-aware rather than uniform. Second, it treats domain adaptation as an architectural problem via stage-specific prompts, rather than as replay alone. The paper’s ablations indicate that both choices contribute materially to the final performance.

## 2. Replay allocation and replay sample selection

The replay subsystem has two distinct components: deciding how many samples to retain from each previously seen dataset and deciding which specific samples to retain. The first component is based on a quantity called \(InfoQ\), which estimates the information content of a dataset \(D\). Given \(n\) samples with current-model embeddings \(\{f_i\}\), LifelongPR forms a Gaussian-kernel matrix

\[
A_{ij}=\exp(-\gamma \lVert f_i-f_j\rVert^2), \qquad i,j=1,\dots,n.
\]

With \(A=U\Sigma V^\top\), \(\Sigma=\mathrm{diag}(\sigma_1,\dots,\sigma_n)\), its effective rank is defined as

\[
Rank(A)=|\{i:\sigma_i\ge \epsilon \sigma_1\}|,
\]

and the dataset information quantity is

\[
InfoQ(D)=Rank(A)/n \in [0,1].
\]

Across all seen datasets \(D_1,\dots,D_t\), LifelongPR allocates replay capacity by a temperature-softmax:

\[
k_i=k_{\text{total}}\cdot
\frac{\exp(InfoQ(D_i)/\tau)}
{\sum_{j=1}^{t}\exp(InfoQ(D_j)/\tau)}, \qquad \tau>0.
\]

This procedure gives higher replay budgets to datasets whose embedding structure has higher effective rank, rather than enforcing class- or domain-uniform replay [2507.10034].

The second component selects samples by spatial-feature diversity. For a dataset \(D\), the memory subset \(M\subset D\) of size \(k\) is chosen to maximize

\[
M^*=\arg\max_{M\subset D,\ |M|=k} \sum_{i\in M} S^i,
\]

where for each \(i\in M\),

\[
S^i=\min_{j\in M,\ j\neq i}
\left[
\min(d_{ij}/d_{\text{thr}},1)+\frac{1-\cos(f_i,f_j)}{2}
\right].
\]

The score combines normalized Euclidean distance \(d_{ij}\) and cosine dissimilarity in feature space. Because the exact optimization is NP-hard, the method uses a greedy algorithm: initialize \(M=\varnothing\); repeatedly sample a candidate set \(R\subset (D\setminus M)\) of size \(|D|/k\cdot \alpha\); select \(m^*=\arg\max_{m\in R} g(M\cup\{m\})\); and update \(M\leftarrow M\cup\{m^*\}\). After selecting \(M_t\) for the current dataset, LifelongPR also prunes each previous memory \(M_i'\) to its new allocation \(k_i\) using the same greedy rule.

This replay design has a specific empirical role. The ablation study reports that random replay yields poor performance, greedy spatial replay improves \(mR@1\) by \(+1.36\%\) and reduces \(F\) by \(4.11\%\), and adding \(InfoQ\) allocation yields a further \(+0.60\%\) \(mR@1\). The intended interpretation is that replay efficiency depends not only on per-sample diversity but also on cross-dataset budget allocation.

## 3. Prompt-learning architecture and two-stage continual adaptation

LifelongPR augments a PCPR backbone \(F\) with a lightweight prompt module \(P_t\) at stage \(t\), producing a combined model \(H_t(x)=F(x,P_t(x))\). The prompt module consists of an \(MLP_{in}:\mathbb{R}^3\to\mathbb{R}^d\), a learnable prompt matrix \(Q\in\mathbb{R}^{k_q\times d}\), \(N\) attention layers that use key/value pairs from \(MLP_{in}(x)\) and queries from \(Q\), and an \(MLP_{out}:\mathbb{R}^d\to\mathbb{R}^{d'}\). For a raw point cloud \(x\in\mathbb{R}^{N_x\times 3}\),

\[
Z=MLP_{in}(x)\in\mathbb{R}^{N_x\times d},
\]
\[
A=Attn(Z,Q)\in\mathbb{R}^{k_q\times d},
\]
\[
P_t(x)=MLP_{out}(A)\in\mathbb{R}^{k_q\times d'}.
\]

The insertion mode is backbone-dependent. For sparse-convolution backbones such as MinkLoc3D, \(P(x)\) is concatenated with raw points as extra channels. For PointNet-style backbones, \(P(x)\) is added to intermediate features [2507.10034].

Training proceeds in two stages rather than as a joint optimization. In Stage 1, Prompt Warm-up, the backbone parameters \(\theta_F\) are frozen and only the prompt is trained on \(D_t\cup M_{1:t-1}\). In Stage 2, Backbone Adaptation, the prompt is frozen and \(\theta_F\) is fine-tuned on the same current-plus-replay data. Both stages use the same total loss,

\[
L_{\text{total}}=L_{\text{PR}}+\lambda^\gamma L_{\text{KD}}.
\]

The place-recognition term is triplet loss, and the knowledge-distillation term keeps the current model’s outputs on replay samples close to those of the previous model. The framework is intended to achieve domain-specific feature adaptation with low additional parameter overhead while minimizing forgetting.

A recurrent misconception is that the prompt module merely acts as a small auxiliary encoder. The paper’s description is more specific: it is a domain-adaptive modulation mechanism whose effect depends on the insertion point and on the staged training schedule. The training-strategy ablation reports that, with PatchAugNet on Seq2, replacing two-stage training with one-stage joint training lowers \(mR@1\) by \(6.22\%\) and increases \(F\) by \(7.83\%\). This suggests that optimization order, not only prompt capacity, is structurally important.

## 4. Experimental protocol, datasets, backbones, and metrics

The experimental study uses two continuous domain sequences. The first is

| Sequence | Domains |
|---|---|
| Seq1 | Oxford \(\rightarrow\) DCC \(\rightarrow\) Riverside \(\rightarrow\) In-house |
| Seq2 | Oxford \(\rightarrow\) Hankou \(\rightarrow\) WHU-Campus \(\rightarrow\) In-house |

After ground removal and normalization, each submap contains 4,096 points. The dataset details are: Oxford, captured by SICK LMS-151 in urban Oxford, with 21.7k training and 3.0k test submaps; DCC and Riverside, captured by Ouster OS1-64 on Seoul roads, with approximately 5.5k training and 15–18k test submaps each; In-house, captured by Velodyne HDL-64E in an urban environment, with 6.6k training and 1.7k test submaps; Hankou, captured by Livox Avia and Hi-Target in Wuhan, with 11.3k training and 2.5k test submaps; and WHU-Campus, captured by Livox and CHCNAV on a campus, with 4.0k training and 1.1k test submaps [2507.10034].

Three PCPR backbones are used: PointNetVLAD, PatchAugNet without the patch-augmentation branch, and MinkLoc3D. The continual-learning baselines are Fine-tuning (FT), InCloud (“InC”), which combines replay and distillation, and CCL, a contrastive continual-learning baseline. The metrics are \(Recall@1\), mean Recall, mean Incremental Recall, and Forgetting:

\[
mR@1_t = \frac{1}{t}\sum_{i=1}^{t} Recall_{t,i},
\]

\[
mIR@1_T = \frac{1}{T}\sum_{k=1}^{T} mR@1_k,
\]

\[
F = \frac{1}{T-1}\sum_{i=1}^{T-1}
\left[
\max_{l\le i} Recall_{l,i} - Recall_{T,i}
\right].
\]

Implementation details are fixed as follows: total replay budget \(k_{\text{total}}=256\), \(\alpha=0.4\), \(\tau=4.0\), \(\epsilon=10^{-6}\), \(\gamma=0.2\), and \(d_{\text{thr}}=10^3\). The prompt configuration uses \(k_q=64\), \(d=8\), \(N=2\), and an output dimension \(d'\) matched to the backbone. Optimization uses Adam with learning rate \(10^{-3}\) and 40 epochs per stage on an NVIDIA RTX 4080S.

## 5. Quantitative performance and ablation evidence

The central reported result is that LifelongPR outperforms the state of the art on continual PCPR. The abstract states improvements of \(6.50\%\) in \(mIR@1\), \(7.96\%\) in \(mR@1\), and an \(8.95\%\) reduction in \(F\) relative to state-of-the-art methods. The detailed results on the harder Seq2 with MinkLoc3D specify the corresponding values as \(mIR@1\) \(73.16\rightarrow 79.67\), \(mR@1\) \(67.12\rightarrow 75.08\), and \(F\) \(12.01\rightarrow 11.42\). Across all backbones and both sequences, LifelongPR is reported to outperform FT, InCloud, and CCL in average recall and forgetting [2507.10034].

The ablation study decomposes the contribution of each component. Random replay performs poorly. Replacing random replay with greedy spatial selection yields \(+1.36\%\) \(mR@1\) and \(-4.11\%\) \(F\). Adding \(InfoQ\)-based replay allocation contributes a further \(+0.60\%\) \(mR@1\). Adding the prompt module contributes \(+2.28\%\) \(mR@1\) and \(-4.17\%\) \(F\). In the training-strategy ablation using PatchAugNet on Seq2, the two-stage strategy improves \(mR@1\) by \(6.22\%\) and reduces \(F\) by \(7.83\%\) relative to one-stage joint optimization.

Qualitative evidence is also reported. t-SNE visualizations show that FT clusters old-domain features, InCloud and CCL partially preserve feature dispersion, and LifelongPR best maintains well-separated clusters for each domain. The paper also presents six street-scene query cases in which FT and CCL fail while LifelongPR retrieves correct matches.

These results support a specific reading of the method. LifelongPR is not solely a memory-selection method, nor solely a prompt-tuning method. The replay allocator, the spatially diverse greedy selector, the lightweight prompt module, and the two-stage optimization schedule are treated as jointly necessary for the reported stability-plasticity trade-off.

## 6. Significance, constraints, and future directions

Within the PCPR setting, LifelongPR frames continual place recognition as a knowledge-fusion problem under tight replay budgets. Its stated key takeaways are threefold: an information-theoretic and spatial-aware replay selection improves memory efficacy; a lightweight prompt module together with two-stage training yields domain-adaptive features without over-parameterization; and the resulting system yields up to \(+7.96\%\) \(mR@1\) and \(-8.95\%\) \(F\) improvements over the state of the art [2507.10034].

The limitations are also explicit. Some degenerate or highly repetitive scenes, including T-junctions, remain challenging and may require multi-shot or temporal aggregation. The prompt module itself may undergo forgetting. Future work is proposed in online continual learning and stronger domain generalization, including contrastive or domain-generalization losses. The paper also provides code and pre-trained models, indicating an implementation-oriented research agenda rather than a purely conceptual proposal.

A plausible implication is that LifelongPR occupies a middle ground between classical replay-driven continual learning and parameter-efficient domain adaptation. It retains a fixed-capacity replay buffer, but it does not treat replay as sufficient; it uses prompts to absorb domain-specific variation and a staged optimizer to decouple prompt adaptation from backbone drift. In that sense, the framework is best understood as a continual PCPR system for sequential city- and sensor-level domain shifts, rather than as a generic memory replay baseline with auxiliary tuning.

Source: https://www.emergentmind.com/topics/lifelongpr