Unified ROI Encryption Benchmark

• Unified ROI Encryption Evaluation Benchmark is a quantitative framework that standardizes the assessment of selective encryption in H.265/HEVC through precise ROI localization and objective metrics.
• It employs dual evaluation modules—ROI-Fineness and ROI-Perturbation—that integrate measures like IoU, PSNR, SSIM, and NPCR to ensure balanced security and visual quality analysis.
• Its rigorous methodology fosters fair comparisons and repeatable experiments, supporting advances in privacy-preserving video technology and potential adaptations to other codecs.

Unified ROI Encryption Evaluation Benchmark provides a standardized, quantitative framework for assessing region-of-interest (ROI) selective encryption algorithms in H.265/HEVC video coding. It rigorously harmonizes ROI recognition, fine-grained encryption evaluation, and a comprehensive set of objective metrics. The benchmark enables fair comparison and repeatable experimentation, fostering robust development in privacy-preserving video technology.

1. Architecture and Design Objectives

At its core, the benchmark is structured as two cooperative evaluation modules: ROI-Fineness Evaluation and ROI-Perturbation Evaluation.

• ROI-Fineness Evaluation: This module inputs ground-truth ROI coordinates $G_i$ and the algorithmically encrypted pixel set $E_i$ for each frame, outputting the per-frame and average Intersection-over-Union (IoU) as a precise measure of how well encryption is localized to sensitive content.
• ROI-Perturbation Evaluation: Comprising subjective (side-by-side frame comparison) and objective (coding unit-level) submodules, this arm assesses the visual and cryptographic perturbation induced by selective encryption. It extracts the actual encrypted coding units, aggregates original and encrypted pixel sets ($P_{\rm ori}$, $P_{\rm enc}$), and computes a suite of seven quantitative indicators.

By mandating uniform ROI extraction and metric calculation procedures, the benchmark enforces methodological consistency across all evaluated ROI encryption schemes. This standardization is essential for meaningful comparison of security, coding efficiency, and visual protection.

2. Metric Suite and Mathematical Definitions

The unified benchmark adopts eight objective metrics, mathematically defined as follows (with $N=h \times w$ denoting ROI pixel count, $i$ indexing frames):

• ROI Encryption Fineness (IoU):

$\IoU_i = \frac{|E_i \cap G_i|}{|E_i \cup G_i|} \,,\qquad \overline{\IoU} = \frac{1}{N_f}\sum_{i=1}^{N_f}\IoU_i$

Quantifies spatial alignment between encrypted area and ground-truth ROI.

• Peak Signal-to-Noise Ratio (PSNR):

$$\mathrm{MSE} = \frac{1}{N}\sum_{x=1}^h\sum_{y=1}^w \bigl(I_{\rm ori}(x,y)-I_{\rm enc}(x,y)\bigr)^2$$

$$\mathrm{PSNR} = 20\log_{10}\Bigl(\frac{255}{\sqrt{\mathrm{MSE}}}\Bigr)$$

Measures fidelity loss induced by encryption.

• Structural SIMilarity (SSIM):

$$\mathrm{SSIM} = \frac{(2\mu_{\rm ori}\mu_{\rm enc} + C_1)\,(2\sigma_{\rm ori,enc} + C_2)} {(\mu_{\rm ori}^2 + \mu_{\rm enc}^2 + C_1)\,(\sigma_{\rm ori}^2 + \sigma_{\rm enc}^2 + C_2)}$$

Addresses visual perceived similarity.

• Edge Distortion Ratio (EDR):

$$\mathrm{EDR} = \frac{\sum_{x,y}|PE(x,y) - CE(x,y)|} {\sum_{x,y}|PE(x,y) + CE(x,y)|}$$

Quantifies preservation of edge structure in encrypted regions.

• Information Entropy:

$H(I) = -\sum_{j=0}^{2^L-1}p(I_j)\,\log_2 p(I_j)$

Assesses degree of information randomness; $H=8$ is ideal for $L=8$.

• Number of Pixel Change Rate (NPCR):

$$D(x,y)= \begin{cases} 1 & C_1(x,y)\neq C_2(x,y), \ 0 & \text{otherwise} \end{cases}$$

$$\mathrm{NPCR} = \frac{\sum_{x,y}D(x,y)}{N} \times 100\%$$

• Unified Average Changing Intensity (UACI):

$$\mathrm{UACI} = \frac{1}{255\,N} \sum_{x,y}|C_1(x,y)-C_2(x,y)| \times 100\%$$

• Bitrate Overhead:

$$\Delta\mathrm{BR}(\%) = \frac{\mathrm{BR}_{\rm enc} - \mathrm{BR}_{\rm ori}}{\mathrm{BR}_{\rm ori}}\times 100\%$$

Collectively, these metrics enable multidimensional characterization of both security properties and perceptual quality of ROI encryption approaches.

3. Experimental Protocol and Dataset Regimen

Experiments are conducted using HM 16.9 reference software on a Windows 10 PC (Intel i7-6700HQ, 16 GB RAM). Five standard YUV sequences—Akiyo, PartyScene, Johnny, Kimono, PeopleOnStreet—span resolutions from $352\times288$ to $2560\times1600$, encoded at 60 fps, Group-of-Pictures “IBBB,” with quantization parameters $QP\in\{8,24,40\}$.

ROI detection leverages a YOLOv8 network fine-tuned on the WiderFace dataset, producing per-frame bounding boxes. Each $32\times32$ tile is classified as ROI or non-ROI, with three encryption strategies applied within each ROI-tile:

• Basic: Encrypts bypass-mode syntax elements (MVD sign/value, residual sign/value, δQP-sign, merge index in bypass mode, reference-frame-index suffix). Zero bitrate overhead.
• Enhanced: Adds regular-mode elements (IPM, MVPIdx, δQP-value), incurring moderate bitrate overhead (~2–3%).
• Advanced: Additionally applies coefficient-scrambling to edge TUs, utilizing a key-driven logistic map permutation, resulting in larger bitrate increase (8–11%).

4. Comparative Performance Analysis

The benchmark systematically evaluates and compares its three strategies (Basic, Enhanced, Advanced) against prior schemes by Taha et al. [18] and Yu et al. [39], using a consistent detection/evaluation pipeline.

• ROI-Fineness (IoU): All three strategies achieve $\overline{\IoU}>0.92$ on high-resolution videos, exceeding Taha et al.'s 32×32-tile scheme (0.83–0.88). This improvement is attributed to the finer 16×16 tiling granularity.
• PSNR and SSIM: Advanced level produces the lowest PSNR and SSIM, reflecting maximal visual scrambling.
• EDR, Entropy, NPCR, UACI, and Bitrate Overhead: Advanced encryption attains the strongest perturbation, e.g., entropy near 8.0, NPCR $\approx$ 99.6%, UACI up to 33.4%, but with an 8–11% bitrate increase. Enhanced offers second-best disturbance with moderate overhead; Basic preserves bitrate at the expense of lower distortion.

Summary Table: ROI Encryption Levels and Key Metrics

Level	Bitrate Δ	Entropy	NPCR (%)
Basic	0%	7.12–7.46	99.35–99.48
Enhanced	2.4–3.5%	7.75–7.92	99.41–99.50
Advanced	8.1–10.6%	7.88–7.95	99.55–99.66

PSNR/SSIM/EDR/UACI results follow consistent trends, with stronger encryption yielding increased distortion and security.

5. Advantages and Limitations

Strengths:

• Enforces pixel-accurate, unified encryption fineness through IoU calculation.
• Extensible indicator set (PSNR, SSIM, EDR, entropy, NPCR, UACI), enabling thorough cross-criteria analysis.
• Ensures metrics are computed exclusively on ground-truth encrypted ROI, mitigating background-induced bias.

Limitations:

• Does not natively evaluate cryptographic measures such as key sensitivity, bitstream correlation, or resistance to ciphertext-only attacks.
• Ground-truth ROI determination is dependent on a single detector (YOLOv8); using alternative detectors may yield different IoU scores.
• Lacks formal user studies for subjective privacy assessment.

6. Prospects for Extension and Broader Impact

Potential directions for expansion include:

• Key-sensitivity and cryptanalysis benchmarks, encompassing key-space exploration and avalanche effect quantification.
• Standard-generalization, adapting the framework for alternate codecs (H.264/AVC, VVC/H.266, AV1) via tailored unit extraction and syntax handling.
• Automated perceptual privacy assessment, such as integrating face recognition success rates within decrypted ROIs as proxies for privacy leakage.
• User perception studies, appraising human visual recognition performance.

This suggests broader applicability beyond H.265, contingent on adapting coding unit extraction logic and metric alignment. A plausible implication is increased rigour and comparability in future research concerning secure, efficient, selectively encrypted video.

7. Conclusion

The unified ROI encryption evaluation benchmark establishes a rigorous, repeatable platform for the quantitative assessment of selective encryption in H.265/HEVC. By integrating standardized ROI detection, fine-grained spatial evaluation, and a comprehensive set of security and quality metrics, it underpins principled development and comparison of privacy-preserving video schemes. Its modular, extensible architecture provides a basis for ongoing methodological refinement and adaptation to evolving codec standards and use cases.