---
title: 'ReLoc: Robust Tampering Localization'
url: https://www.emergentmind.com/topics/reloc
type: topic
---

# ReLoc: Robust Tampering Localization

ReLoc is a restoration-assisted framework for robust image tampering localization that addresses the degradation of forensic traces caused by post-processing, especially JPEG compression. It is formulated as a two-stage pipeline in which a restoration module recovers a high-quality counterpart of a distorted tampered image and a localization module predicts a pixel-wise tampering probability map from the restored output. The central idea is not merely to improve perceptual image quality, but to re-enhance tampering traces in a way that benefits downstream forensic inference; accordingly, the restoration stage is optimized with both conventional image-quality constraints and a forensics-oriented objective, and the two modules are trained alternately rather than jointly from scratch [2211.03930].

## 1. Problem setting and scope

ReLoc is situated in digital image forensics, specifically in tampering localization under post-processing. The motivating problem is that existing image tampering localization methods suffer from severe performance degradation when tampered images are subjected to post-processing, because the tampering traces are distorted by the post-processing operations. The framework is evaluated by fighting against JPEG compression, identified as the most commonly used post-processing.

The notation used in the framework distinguishes four objects: the plain tampered image \(\mathbf{I}^P\), the distorted image \(\mathbf{I}^D\), the restored image \(\mathbf{I}^R\), and the predicted tampering probability map \(\mathbf{P}\). At test time, the system is a simple cascade,
\[
\mathbf{I}^D \xrightarrow{\mathcal{M}_R} \mathbf{I}^R \xrightarrow{\mathcal{M}_L} \mathbf{P},
\]
where \(\mathcal{M}_R\) denotes the restoration module and \(\mathcal{M}_L\) denotes the localization module.

A common misunderstanding is to treat ReLoc as a generic restoration pipeline. The formulation instead targets forensic robustness: the restored image is meant to facilitate localization of tampered regions by recovering weakened forensic evidence rather than by optimizing visual fidelity alone. This is reflected directly in the loss design and training protocol.

## 2. Modular architecture

The framework consists of two modules. The restoration module \(\mathcal{M}_R\) takes a distorted tampered image \(\mathbf{I}^D\) as input and outputs a restored image \(\mathbf{I}^R \approx \mathbf{I}^P\). The localization module \(\mathcal{M}_L\) then processes \(\mathbf{I}^R\) and produces the pixel-wise tampering probability map \(\mathbf{P}\). During training, \(\mathcal{M}_L\) provides a forensics-oriented loss term back to \(\mathcal{M}_R\), so restoration is explicitly guided to improve localization performance rather than only image quality [2211.03930].

For the restoration stage, ReLoc uses SwinConv-UNet, described as a U-shaped encoder-decoder with residual swin-transformer blocks and convolutional blocks. The choice is motivated by state-of-the-art blind denoising and artifact removal capability. The localization stage is plug-and-play: the paper instantiates it with DFCN, SCSE-UNet, and MVSS-Net, all described as fully convolutional, encoder-decoder-style networks customized for binary tampering localization.

This modularity is significant because it separates trace recovery from mask prediction while still coupling them through training. A plausible implication is that ReLoc occupies an intermediate position between pure preprocessing and monolithic end-to-end forensics models: it preserves architectural flexibility at the localization stage, but constrains restoration with task-aware supervision.

## 3. Objective functions

The restoration module is optimized with three components: a conventional image-quality loss, an adversarial image-level loss, and a forensics-oriented loss derived from localization. The pixel-level term is an L1-type visual loss,
\[
\mathcal{L}_{\mathrm{MAE}}
= \frac{1}{mn}\sum_{i=1}^{m}\sum_{j=1}^{n}
\left|\mathbf{I}^{P}_{i,j}-\mathbf{I}^{R}_{i,j}\right|.
\]

The adversarial part uses a discriminator \(D\) to distinguish \(\mathbf{I}^P\) from \(\mathbf{I}^R\):
\[
\mathcal{L}_{D}
= -\log D(\mathbf{I}^P)-\log\bigl(1-D(\mathbf{I}^R)\bigr),
\]
\[
\mathcal{L}_{G}
= -\log D(\mathbf{I}^R).
\]

The localization loss is a weighted sum of pixel-wise cross-entropy and soft-Dice terms,
\[
\mathcal{L}_{\mathrm{CE}}
= -\frac{1}{mn}\sum_{i,j}
\Bigl[
\mathbf{G}_{i,j}\log \mathbf{P}_{i,j}
+
(1-\mathbf{G}_{i,j})\log(1-\mathbf{P}_{i,j})
\Bigr],
\]
\[
\mathcal{L}_{\mathrm{DICE}}
= 1-
\frac{2\sum_{i,j}\mathbf{P}_{i,j}\mathbf{G}_{i,j}}
{\sum_{i,j}\mathbf{P}_{i,j}^{2}+\sum_{i,j}\mathbf{G}_{i,j}^{2}+\epsilon},
\]
\[
\mathcal{L}_{L}
= \lambda_1 \mathcal{L}_{\mathrm{CE}}
+
(1-\lambda_1)\mathcal{L}_{\mathrm{DICE}}.
\]

The restoration objective combines all three terms,
\[
\mathcal{L}_{R}
= \lambda_2 \mathcal{L}_{\mathrm{MAE}}
+ \lambda_3 \mathcal{L}_{G}
+ \lambda_4 \mathcal{L}_{L}.
\]
At the localization stage, \(\mathcal{M}_L\) is trained again with \(\mathcal{L}_L\). The key design choice is that the localization loss is re-used as a restoration objective. That makes the restoration network explicitly sensitive to downstream forensic discriminability rather than solely to pixel fidelity [2211.03930].

## 4. Alternating optimization strategy

The paper reports that jointly optimizing \(\mathcal{M}_R\) and \(\mathcal{M}_L\) from scratch proved unstable because early poor restorations mislead localization. ReLoc therefore uses epoch-wise alternation. In odd epochs, the discriminator \(D\) is updated using \(\mathcal{L}_D\), and the restoration module \(\mathcal{M}_R\) is updated using \(\mathcal{L}_R\). In even epochs, \(\mathcal{M}_R\) and \(D\) are frozen, and the localization module \(\mathcal{M}_L\) is updated using \(\mathcal{L}_L\) [2211.03930].

The stated motivation for this two-step per-epoch scheme is that it ensures the restored images remain of sufficient quality before they are used to train the localization network, leading to faster convergence and higher final accuracy. This directly addresses a second common misunderstanding: ReLoc is not a straightforward jointly trained cascade. Its optimization schedule is part of the method, not an auxiliary implementation detail.

From a methodological perspective, this suggests that ReLoc treats restoration quality as a moving target that must be stabilized before localization can meaningfully exploit it. The framework therefore uses alternating training as a control mechanism for gradient coupling between the two modules.

## 5. Datasets, protocols, and quantitative results

ReLoc is evaluated on three datasets: Certificate PS, DEFACTO, and IMD2020. Certificate PS contains 4,840 originals with five manual tampering types and is saved as PNG. DEFACTO contains approximately \(100\) K automatically generated splicing, copy-move, and removal manipulations on MS-COCO. IMD2020 contains 2,010 real tampered images plus GAN and inpainting forensics. The train/test split is \(75\%/25\%\) per set. Plain tampered images are JPEG-compressed at various quality factors to produce distorted inputs. Fixed-QF experiments use \(QF=75\) for both training and testing, while multi-QF experiments use training QF uniformly sampled from \([70,100]\) and testing at \(QF=60,70,80\). Evaluation uses pixel-wise F1-score, IoU, and AUC of the ROC, with the threshold for binarizing probability maps fixed at \(0.5\) [2211.03930].

Representative fixed-\(QF=75\) and transferability results are summarized below.

| Setting | Baseline | ReLoc |
|---|---:|---:|
| Certificate PS, DFCN, F1 | 0.484 | 0.567 |
| Certificate PS, SCSE-UNet, F1 | 0.604 | 0.651 |
| DEFACTO, MVSS-Net, F1 | 0.507 | 0.515 |
| Certificate PS, compressed, SCSE-UNet\(^P\) | 0.031 | 0.441 |
| Certificate PS, compressed, DFCN\(^P\) | 0.062 | 0.404 |

On Certificate PS with DFCN at fixed \(QF=75\), the upper bound obtained by training and testing on plain images is \(F1=0.912\), \(IoU=0.858\), and \(AUC=0.993\). The same model without restoration on compressed data yields \(F1=0.062\). Fine-tuning the network on compressed data without restoration yields \(F1=0.484\). ReLoc improves this to \(F1=0.567\), corresponding to an increase of \(0.083\) over fine-tuning and \(0.326\) over no fine-tuning. Similar gains are reported for SCSE-UNet and MVSS-Net. On Certificate PS with SCSE-UNet at \(QF=75\), fine-tuned only gives \(F1=0.604\), while ReLoc gives \(F1=0.651\). On DEFACTO with MVSS-Net at \(QF=75\), fine-tuned only gives \(F1=0.507\), while ReLoc gives \(F1=0.515\) [2211.03930].

Under multi-QF training, Certificate PS with DFCN shows fine-tune-only F1 scores of \(0.443/0.466/0.494\) at \(QF=60/70/80\), whereas ReLoc yields \(0.479/0.513/0.584\), with average \(\Delta \approx +0.107\). The paper further states that consistent improvements of \(\Delta F1\) of \(0.03\)–\(0.06\) were observed across all three datasets and all three localization backbones.

These results support the central claim that restoration-assisted localization substantially improves robustness against JPEG compression. They also indicate that the performance gain is not tied to a single localization backbone.

## 6. Transferability, interpretation, and nomenclature

One of the paper’s most distinctive claims is that the restoration module in a well-trained ReLoc model is transferable. Once \(\mathcal{M}_R\) is trained in ReLoc with one \(\mathcal{M}_L\), it can be frozen and paired with a completely different localization model trained only on plain images. On Certificate PS at \(QF=75\), SCSE-UNet\(^P\) alone on compressed images yields \(F1=0.031\), but rises to \(F1=0.441\) when preceded by a DFCN-trained \(\mathcal{M}_R\). Likewise, DFCN\(^P\) alone on compressed images yields \(F1=0.062\), but rises to \(F1=0.404\) when paired with an SCSE-UNet-trained \(\mathcal{M}_R\). Similar large gains are reported on DEFACTO [2211.03930].

This transferability constrains how ReLoc should be interpreted. The restoration module is not merely overfitted to one downstream network; the paper characterizes it as a plug-and-play front end that can boost arbitrary tampering-localization networks without any further joint training. A plausible implication is that ReLoc separates JPEG-trace recovery from architecture-specific localization sufficiently well for the former to generalize across forensic backbones.

The designation “ReLoc” is not unique across arXiv. It also appears in visual relocalization and code-generation contexts, including LM-Reloc [2010.06323], ReLoc-PDR [2309.01646], and the local-search framework ReLoc for code generation [2508.07434]. In the present usage, however, ReLoc specifically denotes the restoration-assisted framework for robust image tampering localization under post-processing.

Source: https://www.emergentmind.com/topics/reloc