Cropping-Based Resampling Techniques

• Cropping-based resampling is a method that confines image or video content to a defined subregion and then resizes it using interpolation, pooling, or mixing, preserving key compositional features.
• It addresses challenges such as coordinate system changes, misalignment, interpolation artifacts, and distribution shifts across diverse tasks like adversarial watermarking and forensic detection.
• The technique is applied in various paradigms—including robust watermarking, aesthetic cropping, video contrastive learning, and multi-scale training—yielding measurable improvements in performance.

Cropping-based resampling denotes a family of operators in which an image or video is first restricted to a spatial or spatiotemporal subregion and then re-expressed through resizing, interpolation, pooling, mixing, or downstream scoring. In the recent literature, this basic pattern appears in at least six distinct roles: as an adversarial perturbation and defense in deep watermarking, as a composition-preserving view extraction mechanism for image cropping, as a hybrid content-preserving stage in image retargeting, as a learnable view generator for self-supervised video representation learning, as a multi-scale data resampling strategy for supervised and self-supervised training, and as a detector-side pre-processing step for high-resolution synthetic image detection (Ding et al., 2021, Lu et al., 2019, Valdez-Balderas et al., 2022, Qing et al., 2021, Han et al., 2022, Mishra et al., 2021, Hoorick et al., 2020, Konstantinidou et al., 2024). Across these settings, the central technical issue is not merely which pixels are retained, but how cropping changes coordinate systems, overlap statistics, interpolation artifacts, compositional alignment, and the effective input distribution seen by the model.

1. Formal operators and representational effects

A canonical formulation appears in crop-resize watermarking. Let $x \in [0,1]^{H\times W\times 3}$ be an image, and let a crop window be specified by upper-left corner $(u_0,v_0)$ and size $(w,h)$. The crop operator is

$C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$

and a resampling operator $R$ maps the cropped image back to the original resolution. In the bilinear case,

$$T(x)=(R\circ C)(x), \qquad T(x)(p,q)=\sum_{i,j} x'(i,j)\,k(\alpha-i)\,k(\beta-j),$$

with

$$\alpha=(p+0.5)\cdot (w/W)-0.5,\qquad \beta=(q+0.5)\cdot (h/H)-0.5,$$

and $k(t)=\max(0,1-|t|)$. This induces an affine coordinate change consisting of translation plus scale, followed by interpolation (Ding et al., 2021).

A related but distinct formulation appears in composition-sensitive image cropping, where the resampling target is not the original image but a fixed feature tensor for ranking candidate views. In "Listwise View Ranking for Image Cropping" (Lu et al., 2019), RoIRefine uses the sequence Interp + Crop + Interp + Pool: first bilinearly upsample the backbone feature map, then crop with float-accurate coordinates, then bilinearly resample the cropped region to a fixed $14\times 14$ grid, then pool for fully connected scoring. The coordinate map from image coordinates $(x,y)$ to the upsampled feature map is

$(u_0,v_0)$0

where $(u_0,v_0)$1 and $(u_0,v_0)$2 (Lu et al., 2019).

In video contrastive learning, cropping-based resampling becomes explicitly spatiotemporal. ParamCrop defines a differentiable 3D affine crop with

$(u_0,v_0)$3

mapping normalized crop coordinates to source-video coordinates before interpolation-based sampling. Here $(u_0,v_0)$4 and $(u_0,v_0)$5 are spatial and temporal scales, $(u_0,v_0)$6 is in-plane rotation, and $(u_0,v_0)$7 are offsets (Qing et al., 2021).

Other formulations are less geometric and more distributional. CropMix defines a family of multi-scale crop operators $(u_0,v_0)$8, partitions the crop-scale interval into disjoint sub-ranges, samples one crop from each sub-range, and mixes them into a single input

$(u_0,v_0)$9

while keeping the label unchanged, $(w,h)$0 (Han et al., 2022). TextureCrop, by contrast, treats crop selection as window scoring on grayscale standard deviation,

$(w,h)$1

retaining a window if $(w,h)$2 (Konstantinidou et al., 2024).

Taken together, these formulations show that cropping-based resampling is not a single primitive but a spectrum of coordinate-selection and representation-canonicalization procedures. A plausible implication is that the mathematical role of cropping is task-dependent: in some settings it is primarily a nuisance transform to be made invariant, while in others it is the mechanism by which supervision, ranking structure, or forensic evidence becomes accessible.

2. Failure modes: misalignment, deformation, shortcut signals, and view disparity

One recurring conclusion is that the principal damage caused by cropping is often not raw information loss. In CropDefender, the reported vulnerability mechanism is spatial misalignment: a decoder trained on full-frame coordinates expects watermark structure at fixed positions, but crop+resize translates and scales those features. The paper supports this with a framing experiment in which adding black borders covering up to 57.75% of pixels leaves StegaStamp’s decoding accuracy above 95%, whereas actual cropping of a much smaller fraction causes severe failures (Ding et al., 2021). This directly challenges the common assumption that edge-pixel removal is the dominant factor.

In composition learning, the dominant failure mode is geometric deformation and feature misalignment. LVRN argues that warping candidate views to fixed sizes damages composition-sensitive cues such as rule of thirds, golden ratio, and golden spiral, while pooling-based down-sampling makes features insensitive to fine shifts and scales. RoIPool further introduces integer quantization of RoI boundaries, and the paper’s ablations isolate substantial gains from more accurate resampling and alignment (Lu et al., 2019).

In self-supervised video learning, the issue is neither edge loss nor composition distortion but the distribution of positive-pair disparity. ParamCrop argues that common random crop operations keep the difference distribution between two views unchanged along training, so the hardness of the contrastive objective is stationary. Its learned crop generator instead adaptively increases center distance and reduces IoU over epochs, with early stopping to avoid no-overlap extremes (Qing et al., 2021). This suggests that cropping-based resampling can act as a curriculum mechanism rather than merely a stochastic augmentation.

For crop forensics, the relevant signals are different again. "Dissecting Image Crops" (Hoorick et al., 2020) studies subtle traces introduced by cropping, especially chromatic aberration, vignetting, and composition priors. Cropping changes the sampled region of the sensor field and thereby alters channel-dependent radial distortions and radial brightness falloff. The same paper also shows why resampling shortcuts must be controlled experimentally: multiple random resizes, random interpolation kernels, and patch jitter are used to prevent detectors from exploiting recoverable resize factors instead of lens traces or semantic composition (Hoorick et al., 2020).

These results collectively undermine any unitary view of cropping as simple spatial truncation. Depending on the task, cropping may primarily alter coordinate alignment, compositional geometry, contrastive difficulty, or optical trace statistics.

3. Major methodological families

The recent literature organizes cropping-based resampling into several method families with distinct objectives and downstream semantics.

Family	Core mechanism	Representative paper
Robust watermark decoding	Randomized crop+resize perturbation during training	CropDefender (Ding et al., 2021)
Composition-aware candidate ranking	Refined feature-map resampling with RoIRefine	LVRN (Lu et al., 2019)
Aspect-ratio retargeting	Distortion-limited warp followed by content-aware cropping	Fast Hybrid Image Retargeting (Valdez-Balderas et al., 2022)
Learnable SSL view generation	Differentiable 3D affine crop with adversarial training	ParamCrop (Qing et al., 2021)
Multi-scale training distribution enrichment	Distinct-scale crops mixed into one label-preserving sample	CropMix (Han et al., 2022)
Object-centric SSL positives	Proposal-based crop plus dilated proposal crop	Object-Aware Cropping (Mishra et al., 2021)
High-resolution forensic pre-processing	Texture-scored crop retention with prediction aggregation	TextureCrop (Konstantinidou et al., 2024)

In robust watermarking, crop-resize is modeled as an attack and then inserted explicitly into the forward path during training. CropDefender retains the basic encoder-decoder structure of StegaStamp but augments it with randomized crop area, aspect ratio, and translation before resizing back to $(w,h)$3 (Ding et al., 2021).

In image cropping as an aesthetic ranking problem, resampling occurs in feature space rather than directly at the pixel level. LVRN generates 1,745 candidate boxes per image, uses VGG16 truncated before the last max pooling layer, and scores each candidate with a three-layer fully connected head after RoIRefine sampling (Lu et al., 2019).

In retargeting, cropping is coordinated with a warping stage rather than replacing it. Fast Hybrid Image Retargeting first constructs an importance map using either Deeplabv3+ with MobileNetV2 backbone or a saliency detector, then computes axis-aligned deformations, then stops warping when the distortion measure

$(w,h)$4

would exceed the threshold $(w,h)$5, and finally removes the residual width or height by minimizing lost 1D importance under cropping (Valdez-Balderas et al., 2022).

In self-supervised learning, two distinct lines appear. ParamCrop makes the crop operator itself learnable and adversarial in video contrastive learning (Qing et al., 2021). Object-Aware Cropping, by contrast, fixes the operator class but changes the sampling distribution: one view is drawn from an unsupervised object proposal and the other from a dilated version of the same proposal, thereby encouraging both object and scene level semantics (Mishra et al., 2021).

CropMix represents a third line, in which cropping-based resampling is used to produce a richer input distribution rather than merely multiple positive views. It samples multiple crops from disjoint scale intervals and mixes them into one training sample, typically with Mixup or CutMix, while preserving the original label (Han et al., 2022).

TextureCrop is detector-side rather than training-side. It does not learn crop parameters; instead, it filters high-resolution windows by local texture, forwards only the retained crops to an existing fixed-size synthetic image detector, and aggregates the crop-level outputs (Konstantinidou et al., 2024).

4. Optimization objectives and architectural patterns

The optimization strategies used with cropping-based resampling are as heterogeneous as the applications.

CropDefender jointly optimizes residual energy, perceptual fidelity, and message fidelity:

$(w,h)$6

with uncertainty-based learnable weights

$(w,h)$7

The encoder output is constrained by

$(w,h)$8

and instance normalization is inserted before activations in both encoder and decoder, including before the decoder’s final sigmoid, to stabilize gradient flow (Ding et al., 2021).

LVRN formulates image cropping as a listwise ranking problem. Given candidate scores $(w,h)$9, it defines a Top-1 probability

$C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$0

and optimizes cross-entropy between predicted and target Top-1 distributions. This replaces pairwise comparison with a listwise objective that better matches the task of selecting the single best crop (Lu et al., 2019).

Fast Hybrid Image Retargeting separates warping energy from perceptual distortion control. The paper does not reprint the original AAD energy, but it specifies the scalar distortion functional $C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$1 and a thresholding strategy: if a candidate warp exceeds $C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$2, solve for the largest acceptable warp factor via 1D search or bisection, then invoke content-aware cropping to bridge the remaining gap (Valdez-Balderas et al., 2022). This is a thresholded coordination policy rather than a single monolithic loss.

ParamCrop uses a standard InfoNCE-style contrastive objective for the backbone, but the crop generator is trained adversarially:

$C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$3

implemented by gradient reversal on the cropping MLPs. An early-stopping detachment rule with $C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$4 prevents crop parameters from drifting to degenerate no-overlap configurations (Qing et al., 2021).

CropMix requires no new network objective for supervised classification. The mixed sample is trained with ordinary cross-entropy,

$C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$5

and unlike conventional MixUp or CutMix across different images, the label is not mixed because all views come from the same image (Han et al., 2022).

Object-Aware Cropping is built on existing self-supervised frameworks, especially MoCo-v2, but adds two auxiliary tasks. The final objective is

$C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$6

where $C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$7 is relative rotation prediction and $C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$8 is object localization. The method also uses two projection heads, one for the object-centric view and one for the dilated view (Mishra et al., 2021).

TextureCrop is exceptional in that it is training-free. Its “objective” is a selection rule at inference: keep crop $C_{u_0,v_0,w,h}(x)(i,j)=x(u_0+i,v_0+j),$9 if $R$0, evaluate the detector on each retained crop, and aggregate the scores by average, median, weighted average, majority vote, or max (Konstantinidou et al., 2024).

5. Empirical behavior across tasks

In deep watermarking, explicit crop-resize training substantially changes robustness. Under center crops removing 27.75% and 43.75% of the area, StegaStamp yields accuracies of approximately 80% and 50%, whereas CropDefender reaches $R$1 and approximately 88%, respectively. For framing, adding black borders of width up to 70 px covering 57.75% of pixels keeps StegaStamp accuracy at $R$2, while CropDefender remains high at approximately 90% despite being slightly reduced relative to StegaStamp (Ding et al., 2021).

In composition-aware image cropping, the combined effect of improved resampling and listwise optimization is large on FCDB. Without RoI-aware sampling, pairwise and listwise variants reach 0.6080 and 0.6204 IoU; with RoIRefine they reach 0.6882 and 0.7100. On the benchmark comparison, LVRN reports 0.7100 / 0.0735 on FCDB for Avg. IoU / Avg. Disp and 0.8434 Top-1 Max IoU on FLMS. The method also reports 125 FPS on an NVIDIA GeForce GTX 1080 GPU while evaluating 1,745 candidates, exceeding the compared FPS of VFN+SW, A2-RL, and VPN (Lu et al., 2019).

In video contrastive learning, ParamCrop consistently improves downstream action recognition relative to random cropping. For SimCLR with S3D-G, fully supervised fine-tuning improves from 56.0 to 59.9 on HMDB51 and from 85.3 to 86.9 on UCF101; linear evaluation improves from 33.5 to 37.3 on HMDB51 and from 57.9 to 59.3 on UCF101. The paper also reports that removing early stopping reduces linear accuracy from 37.3/59.3 to 32.6/55.8, and that disabling gradient reversal drops performance to or below random cropping (Qing et al., 2021).

In supervised and self-supervised training, CropMix reports broad gains from multi-scale crop mixing. On CIFAR-100, average top-1 error drops from 27.13% with RRC to 24.67% with CropMix; on CIFAR-10, from 5.69% to 4.77%. On ImageNet-1K with ResNet-50, recipe R1, top-1 improves from 76.59 to 77.60, FGSM from 21.01 to 23.94, IN-R from 35.30 to 39.12, and IN-A from 4.29 to 6.53. In MoCo v2 improved, CropMix raises linear probe performance from 65.8 to 67.8 and semi-supervised fine-tuning with 1% labels from 14.6 to 18.7 (Han et al., 2022).

Object-Aware Cropping shows that standard scene-level random cropping is particularly weak on multi-object data. On OpenImages Hard Multi-object Subset, MoCo-v2 with scene-scene crops reaches 49.8 mAP, while MoCo-v2 plus OAC reaches 58.6 mAP, an improvement of 8.8% mAP. On VOC detection with OHMS pretraining, the baseline 51.5 AP rises to 53.4 with OAC at $R$3, and to 54.6 with OAC plus both auxiliary losses (Mishra et al., 2021).

For crop forensics, "Dissecting Image Crops" reports joint crop detection accuracy of 86% on Flickr, compared with 79% for the global model, 77% for the patch model, and a 67% human baseline. In smooth images the task becomes much harder, with joint performance at 50% and patch performance at 55%. The same paper reports absolute patch localization accuracy of approximately 21% versus chance at approximately 6% (Hoorick et al., 2020).

In high-resolution synthetic image detection, TextureCrop improves performance over standard adaptation strategies. Averaged across detectors and datasets, it improves Balanced Accuracy by 3.13%, Average Precision by 5.17%, and AUC by 5.71% relative to center-crop; against resizing, the gains are 9.33%, 15.78%, and 13.96%, respectively. It also exceeds 10-crop by 1.85% BA, 2.11% AP, and 4.57% AUC (Konstantinidou et al., 2024).

These results do not imply a single universal advantage for cropping-based resampling. Rather, they show that the effect size depends strongly on whether the crop operator is aligned with the task’s latent invariances, evaluation resolution, and sampling biases.

6. Limitations, misconceptions, and unresolved questions

Several recurring misconceptions are explicitly challenged in this literature. One is that crop robustness is primarily about surviving removal of border pixels. CropDefender argues instead that the dominant mechanism is position movement, and its framing-versus-cropping experiment supports that interpretation (Ding et al., 2021). Another is that better crop prediction in aesthetic ranking is mainly a matter of better scoring functions; LVRN attributes a substantial part of the error to deformation and quantization introduced during view generation and feature extraction (Lu et al., 2019).

A further misconception is that random cropping is uniformly beneficial in self-supervised learning. Object-Aware Cropping shows that on non-iconic, multi-object datasets such as OpenImages and COCO, the assumptions underlying scene-level random crops break down, because randomly cropped views may miss objects or include only tiny fragments (Mishra et al., 2021). ParamCrop adds a related but different point: even when random cropping is useful, a fixed disparity distribution can be suboptimal because the contrastive task difficulty then remains stationary across training (Qing et al., 2021).

The literature also identifies concrete limits. CropDefender notes that extreme crops, strong scale changes, out-of-distribution non-uniform resampling, and severe rotations or perspective crops remain challenging, and the method was not evaluated for physical-world prints or camera captures (Ding et al., 2021). Fast Hybrid Image Retargeting acknowledges that densely distributed salient content, text, faces, and structured repetitive patterns make the trade-off between warp distortion and content loss intrinsically difficult, and that composition penalties beyond lost-importance minimization are only an inferred extension, not part of the paper’s specified method (Valdez-Balderas et al., 2022).

For distribution-enrichment methods, gains are not unconditional. CropMix reports that narrow scale ranges produce weaker gains or small drops, that benefits for detection and segmentation transfer from contrastive pretraining are not guaranteed, and that broader attention may be suboptimal for tasks requiring precise localization (Han et al., 2022). TextureCrop likewise reports that low-texture synthetic images or artifacts concentrated in low-frequency regions may not be prioritized by standard-deviation-based selection, and that DALL·E 3 is a notable case in which resizing performed well (Konstantinidou et al., 2024).

Finally, crop forensics reveals a methodological caution relevant beyond forensics. "Dissecting Image Crops" shows that models can exploit lens traces, chromatic aberration, vignetting, and composition priors as shortcut signals, and therefore random resizing, kernel randomization, and jitter may be necessary when the scientific goal is to study cropping itself rather than unintended correlations (Hoorick et al., 2020). A plausible implication is that future work on cropping-based resampling will continue to divide into two lines: one that seeks invariance to crop-induced artifacts, and another that deliberately exploits those artifacts when they are informative for the task.