EfficientRWKV: Lightweight Visual Backbone

• EfficientRWKV is a lightweight backbone that integrates RWKV-style global attention with depthwise convolution, achieving linear complexity for high-resolution forensic image analysis.
• It employs a three-stage architecture with progressive downsampling and multi-scale decoder coupling to enhance gradient flow and precise dense mask prediction.
• Empirical results demonstrate notable improvements with F1 of 78.0 and IoU of 67.3, efficiently processing images up to 2048×2048 on commodity GPUs.

EfficientRWKV is a lightweight RWKV-derived backbone for high-resolution image manipulation localization, introduced within EfficientIML as a three-stage architecture for large forensic images. It is inspired by and based on Vision-RWKV6, and it combines a hybrid state-space and attention network with depthwise convolutional processing so that global context and local details are captured in parallel. In the reported formulation, EfficientRWKV replaces the quadratic scaling typical of ViT-style backbones with linear complexity, enabling operation on images up to $2048 \times 2048+$ on commodity GPUs while serving a multi-scale decoder for dense mask prediction (Li et al., 10 Sep 2025).

1. Lineage and conceptual position

EfficientRWKV belongs to the broader RWKV family, whose defining objective is to combine the efficient parallelizable training of transformers with the efficient inference of RNNs. The original RWKV architecture was introduced as Receptance Weighted Key Value, with a linear attention mechanism and an RNN-form inference procedure that maintains constant computational and memory complexity during inference (Peng et al., 2023). A subsequent review characterized RWKV by its temporal mixing, channel mixing, and token shifting paradigm, and emphasized that the architecture had already been adapted beyond language to vision, 3D, and multimodal settings (Datta, 2024).

Within that lineage, EfficientRWKV is a task-specific visual backbone rather than a generic LLM. Its immediate design target is high-resolution image manipulation localization under computational constraints, especially in the presence of diffusion-generated manipulations. The EfficientIML work presents EfficientRWKV as the backbone that addresses the prohibitive computational complexities of existing methods while remaining lightweight enough for real-time or near-real-time deployment on large images (Li et al., 10 Sep 2025).

This suggests that EfficientRWKV should be understood less as an isolated architecture than as a specialization of RWKV-style linear-complexity modeling for dense visual forensics.

2. EfficientRWKV block and token mixing

The core EfficientRWKV block is organized as a three-branch token mixer. Each block splits channels into a Global Branch $x_g$, a Local Branch $x_l$, and an Identity Branch $x_i$. The global branch allocates $C_v$ channels to RWKV-style attention for efficient global context modeling; the local branch allocates $C_{con}$ channels to depthwise separable convolutions for fine-grained spatial patterns; and the identity branch passes the remaining channels through unchanged to preserve information flow and gradient propagation (Li et al., 10 Sep 2025).

The global branch is formalized by the following WKV operator:

$$\mathrm{wkv}_t = \frac{ \sum_{i=0,i\neq t}^{T-1} \exp\left(-\frac{|t-i|-1}{T}w + k_i\right)v_i + \exp(u + k_t)v_t }{ \sum_{i=0,i\neq t}^{T-1} \exp\left(-\frac{|t-i|-1}{T}w + k_i\right) + \exp(u + k_t) }$$

where $w$ and $u$ are learnable channel-wise parameters. The reported complexity of this branch is $O(T C_v)$, i.e., linear with sequence length (Li et al., 10 Sep 2025).

The local branch uses depthwise separable convolutions, with kernel size varying per stage, to capture detailed, edge-aware features. The identity branch is explicitly retained rather than omitted, which is important in the ablation evidence: freezing the global branch produces blurry outputs and loss of global context, whereas freezing the local branch yields sharper but less robust masks with more false positives (Li et al., 10 Sep 2025).

At the block level, the paper describes a fixed sequence of components: depthwise convolution residual modules, then the EfficientRWKV token mixer, then pointwise FFNs. This arrangement places the RWKV-style global pathway and the convolutional local pathway in parallel rather than in alternation, which is central to the backbone’s efficiency/accuracy trade-off (Li et al., 10 Sep 2025).

3. Stage-wise backbone and decoder coupling

EfficientRWKV is embedded in a three-stage backbone with progressive downsampling via Patch Merging and residual FFN blocks. The spatial resolutions are reported as Stage 1: $x_g$0, Stage 2: $x_g$1, and Stage 3: $x_g$2, with $x_g$3, $x_g$4, and $x_g$5 (Li et al., 10 Sep 2025).

Stage	Spatial size	Channel allocation
1	$x_g$6	$x_g$7
2	$x_g$8	$x_g$9
3	$x_l$0	$x_l$1

The decoder receives feature maps from all three stages directly and produces both a fused mask prediction and intermediate masks at each resolution. Supervision is applied at each scale with weighted BCE and edge-aware losses, and the authors attribute several training benefits to this scheme: better gradient flow, faster convergence, improved edge precision, and the elimination of a separate Feature Pyramid Network used in previous ViT-based methods (Li et al., 10 Sep 2025).

Ablation studies further report that the final set of loss weights leads to approximately $x_l$2 improvement in F1. In this formulation, EfficientRWKV is not only a backbone but part of a tightly coupled multi-scale localization system in which hierarchical features are exposed directly to the decoder instead of being funneled through a heavier pyramid-style neck (Li et al., 10 Sep 2025).

4. Computational profile and scaling behavior

The principal computational claim attached to EfficientRWKV is linear complexity. The paper contrasts this with ViT’s quadratic attention, arguing that EfficientRWKV therefore supports much larger images without window partitioning or cropping tricks. In the reported experiments, this enables operation on images up to $x_l$3 on commodity GPUs (Li et al., 10 Sep 2025).

At $x_l$4, EfficientRWKV requires 21.7 GFLOPs, compared with 95.6 for SHViT and 654.3 for MVSS, and runs at 58.5 img/s. At $x_l$5, it requires 86.7 GFLOPs, compared with 382.4 for SHViT and 2617.8 for MVSS. The reported parameter count is 19.8M, compared with 19.5M for SHViT and 142.8M for MVSS (Li et al., 10 Sep 2025).

For throughput at high resolution, the reported figure is 23 images/sec at $x_l$6, compared with 16.9 for SHViT, again at much lower FLOPs (Li et al., 10 Sep 2025). The combination of low FLOPs and sustained throughput is the operational meaning of “Efficient” in EfficientRWKV: the design attempts to preserve dense global context modeling without reintroducing the quadratic cost that usually accompanies global attention in visual backbones.

In broader RWKV research, analogous efficiency arguments recur across domains. AudioRWKV reports that an RWKV7-based audio model maintains $x_l$7 complexity and achieves up to a 13.3X speedup in long-form audio processing, while P-RWKV for point clouds reports that FLOPs grow linearly with input token number and that inference latency is lower than Transformer- and Mamba-based baselines (Xiong et al., 2 Sep 2025, Liu et al., 9 Jun 2026). This suggests that EfficientRWKV’s efficiency profile is consistent with a larger pattern in RWKV adaptation: the architecture family is repeatedly used where dense long-range modeling must be retained under strict compute budgets.

5. Empirical localization performance

On the SIF benchmark of diffusion-generated, high-resolution manipulations, evaluated at $x_l$8, EfficientRWKV achieves F1: 78.0, AUC: 98.8, IoU: 67.3, and Acc: 98.2. The next-best model reported in the summary, MVSS, reaches F1 65.0 and IoU 51.3 (Li et al., 10 Sep 2025).

On CASIAV2-based evaluation, EfficientRWKV is reported to lead in all performance metrics—F1, AUC, IoU, and Accuracy—relative to ViT-S, SHViT, MVSS, and PSCC (Li et al., 10 Sep 2025). The accompanying interpretation in the source is that the model’s joint handling of global and local information is especially important for the new diffusion-based manipulations that motivated the SIF dataset.

The ablation results identify the functional roles of the branches with unusual clarity. When the global branch is frozen, the output becomes blurry and loses global context. When the local branch is frozen, masks become sharper but less robust and exhibit more false positives. The paper therefore treats both branches as necessary for best performance, rather than viewing the local branch as a mere efficiency add-on to an RWKV core (Li et al., 10 Sep 2025).

The same ablation section reports that knowledge distillation during pre-training provides +4.3% F1, indicating that the final localization behavior is not attributable to backbone structure alone. EfficientRWKV’s empirical position is therefore inseparable from the broader EfficientIML training pipeline, especially multi-scale supervision and pre-training strategy (Li et al., 10 Sep 2025).

6. Relation to other RWKV adaptations

EfficientRWKV is one instance of a broader tendency to modify RWKV around domain-specific locality priors while preserving linear-complexity sequence modeling. In audio, AudioRWKV inherits the stable and efficient recurrent formulation of RWKV7, replaces the 1D token-shift operation with a 2D depthwise separable convolution, and adapts the original causal WKV kernel into a bidirectional WKV kernel (Bi-WKV) for full-sequence context modeling (Xiong et al., 2 Sep 2025). In 3D point clouds, P-RWKV augments the RWKV block with Local Perception Expansion (LPE) and Spatial Context Enhancement (SCE) to handle irregular geometry while preserving efficiency advantages (Liu et al., 9 Jun 2026).

These parallels are informative because EfficientRWKV likewise supplements RWKV-style global modeling with explicitly local operators rather than relying on a pure sequence formulation. The local depthwise separable convolution in EfficientRWKV plays an analogous role to ConvShift in AudioRWKV and to locality-enhancing modules in P-RWKV: each is a mechanism for injecting domain structure that vanilla sequential RWKV would not capture as directly (Xiong et al., 2 Sep 2025, Liu et al., 9 Jun 2026).

Other RWKV variants emphasize different efficiency targets. RWKV-X combines RWKV blocks with Top-$x_l$9 chunk sparse attention to preserve constant-time decoding while improving long-range recall in language modeling (Hou et al., 30 Apr 2025). Triplet-Block Diffusion RWKV introduces a triplet-block layout that allows a causal RWKV backbone to support parallel, bidirectional discrete-diffusion denoising without architectural change (Lin et al., 25 May 2026). Read together, these works suggest that “efficient RWKV” is less a single recipe than a family of architectural programs: linear-complexity recurrence is retained, while the surrounding operators are adapted to domain-specific requirements.

7. Applications, constraints, and open questions

The applications explicitly identified for EfficientRWKV are real-time image forgery detection and localization on modern, very high-res images, forensic tools deployed in field or edge devices with limited computing power, and any task requiring efficient, fine-grained, and contextual segmentation at high resolution, including medical imaging, document analysis, or industrial inspection (Li et al., 10 Sep 2025).

The limitations are also stated directly. The paper notes dataset dependency, and it says that the model’s potential limitation on generalizability to unseen manipulation types or non-diffusion-based forgeries is not deeply explored. It also states that, although SIF is a major step forward, the approach has yet to be validated on even larger, more diverse real-world datasets. Finally, it acknowledges some performance sacrifice for extreme parameter/compute reduction compared to heaviest SOTA models (Li et al., 10 Sep 2025).

These caveats matter because EfficientRWKV is optimized for a specific operating regime: dense localization on very large images under constrained compute. The current evidence supports that regime strongly, but a broader claim about universal high-resolution segmentation would require the larger and more diverse validation that the source itself says is still missing.