EfficientIML: High-Res Localization Framework

• EfficientIML is a high-resolution image manipulation localization framework that employs a lightweight three-stage EfficientRWKV backbone to target diffusion-based semantic inpainting forgeries.
• It integrates direct multi-scale decoding and hierarchical supervision to efficiently capture both global semantics and fine local details in large images.
• The introduced SIF dataset and benchmark validate its superior accuracy and efficiency, outperforming traditional ViT-based models, especially at higher resolutions.

EfficientIML is a high-resolution image manipulation localization framework designed for modern diffusion-generated semantic inpainting forgeries. In the paper that introduces it, the term denotes both a model and an experimental setting: a lightweight three-stage EfficientRWKV backbone with direct multi-scale decoding and hierarchical supervision, together with the SIF dataset, a curated benchmark of high-resolution semantic inpainting edits. The framework is positioned against a specific failure mode of prior image manipulation localization systems: many are trained mainly on traditional manipulations and lower or medium resolutions, whereas diffusion-based semantic inpainting produces visually coherent, semantically aligned edits in much larger images, making localization both computationally harder and forensically subtler (Li et al., 10 Sep 2025).

1. Problem formulation and scope

EfficientIML addresses image manipulation localization in the strict pixel-level sense: given an input image, it predicts a binary mask of manipulated regions. The paper focuses on high-resolution images containing diffusion-based semantic inpainting forgeries, rather than classical splicing, copy-move, or object removal. This distinction matters because semantic inpainting replaces an object-aligned region with newly generated content that is semantically plausible, rather than merely copying or deleting existing pixels, so the manipulated region can be realistic at both local and scene levels (Li et al., 10 Sep 2025).

The framework is also explicitly motivated by scaling behavior. The input image is written as

$X \in \mathbb{R}^{H \times W \times 3},$

and the paper argues that transformer-style localization backbones become increasingly expensive as resolution grows. A central design goal is therefore to retain global reasoning while avoiding the prohibitive computational behavior associated with quadratic self-attention on large images. The paper further argues that existing datasets such as CASIA, Columbia, Coverage, NIST16, DEFACTO, IMD, and CIMD emphasize traditional artifact-based manipulations or do not reflect current high-resolution semantic inpainting conditions, while even diffusion-oriented datasets such as CSI-IMD are described as too low-resolution for this setting. A plausible implication is that EfficientIML is intended as both a detector and a response to benchmark shift: it targets a manipulation regime that prior training corpora do not adequately expose.

2. SIF dataset and benchmark construction

The paper introduces SIF, “Semantic Inpainting Forgery,” presented as the first publicly released high-resolution semantic inpainting forgery dataset with images exceeding $1024\times 1024$. The final dataset contains 1,228 manipulated images, with source images selected from SAM9K. The generation pipeline is a modified version of Inpaint Anything: HIPPIE is used to obtain an object mask, Qwen-VL Plus generates a text prompt for the masked region, and the original image, mask, and prompt are fed to Stable Diffusion 2 for inpainting. About 5,000 manipulated images were generated, of which 1,228 were manually selected based on realism, semantic coherence, and diversity, with validation by the authors’ group and volunteers (Li et al., 10 Sep 2025).

SIF images are described as around $1800\times 1200$, “up to $1800\times 1200$” in the main text, and approximately $\sim 1800\times 1200$ in the appendix. For SIF experiments, the dataset is split with a 6:1 ratio. The paper does not state a separate validation split. A technically important property of SIF is that the masks trace actual scene elements rather than simple geometric regions, which the appendix contrasts with datasets such as NIST16. This makes SIF not only high-resolution but semantically structured: edited regions are object-aligned and therefore harder to localize. The dataset is consequently integral to the EfficientIML concept, because the model’s architectural choices are evaluated precisely in this high-resolution, semantic-inpainting regime.

3. EfficientRWKV backbone and multi-stage representation

EfficientIML uses a three-stage EfficientRWKV backbone. After patch embedding, the model produces a feature map of size

$\left(\frac{H}{16}, \frac{W}{16}, C_1\right),$

followed by three backbone outputs

$$\left(\frac{H}{16}, \frac{W}{16}, C_1\right), \quad \left(\frac{H}{32}, \frac{W}{32}, C_2\right), \quad \left(\frac{H}{64}, \frac{W}{64}, C_3\right),$$

with appendix values

$[C_1, C_2, C_3] = [200, 376, 448].$

Downsampling between stages is done through Patch Merging, and each stage contains residual FFN blocks. The backbone is designed to replace the more expensive ViT-style feature extractor while preserving both long-range context and local structure (Li et al., 10 Sep 2025).

The core unit is the EfficientRWKV Block. Its EfficientRWKV Module splits the input tensor along the channel dimension into three parallel branches: a global branch $x_g$ using an RWKV attention mechanism, a local branch $x_l$ using a depthwise separable convolution block, and an identity branch $1024\times 1024$0 that preserves raw features. Channel allocation is stage-dependent: Stage 1 uses $1024\times 1024$1 for the global branch, $1024\times 1024$2 for the local convolution branch, and 0 for the identity branch; Stage 2 uses $1024\times 1024$3, $1024\times 1024$4, and $1024\times 1024$5; Stage 3 uses $1024\times 1024$6, $1024\times 1024$7, and $1024\times 1024$8. This progression increases local modeling capacity in deeper stages while retaining global reasoning throughout.

For the global branch, the paper gives the Bi-WKV aggregation

$1024\times 1024$9

Here $1800\times 1200$0 is a learnable channel-wise decay vector and $1800\times 1200$1 is a learnable channel-wise bias vector. The paper states that this yields complexity

$1800\times 1200$2

which is the basis for the claim that EfficientIML scales more favorably than ViT-based localization systems on high-resolution inputs. The exact post-branch fusion equation is not given in the paper.

4. Decoder design, hierarchical supervision, and optimization

The three backbone outputs are sent directly to a multi-branch decoder head that produces intermediate predictions at each scale and a fused final manipulation mask. A central architectural claim is that using multi-stage backbone features directly avoids the additional Simple Feature Pyramid Network used in IML-ViT, thereby preserving fine spatial detail, improving gradient flow, and reducing overhead. Multi-scale supervision is applied at the three resolutions

$1800\times 1200$3

with each prediction supervised by a downsampled binary manipulation mask and a corresponding edge target (Li et al., 10 Sep 2025).

The paper states that the printed total loss is typeset incorrectly, but the surrounding explanation indicates an intended structure consisting of: a BCE loss on each intermediate prediction $1800\times 1200$4 against downsampled mask $1800\times 1200$5, an edge-aware BCE term on each intermediate prediction using edge target $1800\times 1200$6, a BCE loss on the final fused output $1800\times 1200$7 against full-resolution mask $1800\times 1200$8, and a final edge-aware BCE on $1800\times 1200$9 with edge target $1800\times 1200$0. The tested weighting schemes are reported in the appendix, and the final setup uses

$1800\times 1200$1

with fused output weight $1800\times 1200$2. The paper says this setup worked best, suggesting that deeper, coarser semantic features deserve more weight while still benefiting from hierarchical supervision.

Training uses an EfficientRWKV backbone initialized from an ImageNet-1K checkpoint pretrained for 1,000 epochs under teacher-student distillation, with TResNet-L as teacher following the DeiT distillation protocol. Fine-tuning uses AdamW with initial learning rate $1800\times 1200$3, a cosine decay schedule, and an effective batch size of 64. The paper does not explicitly report the number of fine-tuning epochs, augmentation strategy, crop strategy, preprocessing normalization details, or hardware. This omission is relevant because EfficientIML is presented as efficient in FLOPs and throughput, but exact training reproducibility remains partially underspecified.

5. Evaluation protocols, accuracy, and computational profile

The paper evaluates EfficientIML in two main settings. In the CASIAV2-training protocol, the model is trained on the entire CASIAV2 dataset and tested on CASIAV1, Columbia, Coverage, NIST16, IMD, and CIMD. In the SIF-training protocol, it is trained and tested on SIF with a 6:1 split, with experiments reported at $1800\times 1200$4 and $1800\times 1200$5. Metrics are Pixel-F1, AUC, IoU, and Pixel Accuracy. Under CASIAV2-training at $1800\times 1200$6, EfficientIML achieves average F1 42.6, average AUC 83.2, average IoU 38.0, and average Accuracy 93.0. The authors summarize this as gains of +4.3% F1 and +5.2% IoU over the best tested SOTA baseline in that setting (Li et al., 10 Sep 2025).

On SIF-training at $1800\times 1200$7, EfficientIML reports F1 68.1, AUC 97.3, IoU 56.2, and ACC 97.4. At $1800\times 1200$8, the same model reports F1 78.0, AUC 98.8, IoU 67.3, and ACC 98.2. The paper notes an approximately +13% F1 improvement over the best comparison model on the high-resolution SIF benchmark. This is significant because the accuracy gain increases rather than collapses at higher resolution, which suggests that the model can exploit high-resolution forensic detail rather than merely tolerate it.

Efficiency claims are quantitative. The model has 19.8M parameters. At $1800\times 1200$9, it uses 21.7 GFLOPs and runs at 58.5 img/s; at $\sim 1800\times 1200$0, it uses 86.7 GFLOPs and runs at 23.0 img/s. For comparison, ViT-S requires 128.2 GFLOPs and 37.8 img/s at $\sim 1800\times 1200$1, and 512.7 GFLOPs and 9.3 img/s at $\sim 1800\times 1200$2 with checkpointing. The paper treats this as one of EfficientIML’s strongest practical properties: localization performance is improved while computation remains under 100 GFLOPs even at $\sim 1800\times 1200$3, which is why the model is described as suitable for real-time forensic applications.

6. Ablation results, interpretation, and limitations

The ablations attribute most of the gain to three components: the EfficientRWKV backbone, removal of the separate SFPN in favor of direct multi-stage features, and multi-scale supervision. In the incremental ablation table, ViT-S + SFPN without multi-scale loss yields F1 23.8 and IoU 16.9; Effi-RWKV + SFPN without multi-scale loss yields F1 57.6 and IoU 47.0; Effi-RWKV without SFPN and without multi-scale loss yields F1 61.6 and IoU 50.0; and the full model without SFPN but with multi-scale loss yields F1 68.1 and IoU 56.2. Branch-specific ablations further show that freezing the global branch gives F1 57.3 and IoU 46.3, whereas freezing the local branch gives F1 64.2 and IoU 53.0, while the final setup reaches F1 68.1 and IoU 56.2. The paper interprets this as evidence that the global branch is especially critical, but best performance requires both global and local branches together (Li et al., 10 Sep 2025).

A related ablation concerns pretraining. Distillation-based pretraining improves F1 from 63.8 to 68.1, a contribution of about +4.3 F1. The weighting experiments for hierarchical supervision also support the final loss configuration, with the paper stating that it improves consistency across hierarchical predictions, stabilizes learning, and accelerates convergence. This suggests that EfficientIML’s efficiency is not reducible to a smaller backbone alone; it is tied to a coordinated design involving linear-complexity global aggregation, local convolutional refinement, direct multi-scale decoding, and hierarchical supervision.

The paper is also explicit about limitations. It does not provide some implementation details needed for exact reproduction, including fine-tuning epoch count, augmentation strategy, hardware, or explicit decoder fusion equations. The printed loss equation is malformed. SIF, although important, contains 1,228 final images and focuses specifically on semantic inpainting rather than a broader taxonomy of manipulations. The paper does not deeply analyze domain shift between synthetic diffusion edits and real-world edited content beyond the reported cross-dataset experiments, and failure cases are only indirectly described through ablations. A common misconception would therefore be to treat EfficientIML as a fully general manipulation detector or as a complete reproducibility package. The evidence supports a narrower and more precise conclusion: it is a high-resolution localization framework for diffusion-generated semantic inpainting forgeries, with a strong empirical trade-off between accuracy and computational cost, especially at $\sim 1800\times 1200$4.