Object-Preserving Perturbations

• Object-preserving perturbations are specialized modifications that target specific system behaviors while keeping designated invariant objects intact.
• They are implemented across domains using constraint-based optimization, masking techniques, and analytical constructions to balance modification and preservation.
• Empirical results show these methods effectively degrade unwanted performance metrics while maintaining essential features in adversarial, spectral, and dynamical applications.

Object-preserving perturbations are carefully designed modifications to systems—typically in images, dynamical maps, or structured matrices—that achieve a specific effect (e.g., altering detection, prediction, or spectral properties) while explicitly preserving specified "objects" or invariant structures. The definition of "object" is context-dependent: it may refer to detected bounding boxes in computer vision, invariant subspaces or Jordan chains in linear algebra, or strong invariant manifolds in differentiable dynamics. The core principle is to exert a targeted influence on a system without damaging or directly affecting designated objects, often through optimization constraints or specialized algorithmic constructions.

1. Definitions and Conceptual Foundations

The term "object-preserving perturbation" encompasses several technical classes:

• In computer vision, these are adversarial changes constrained to avoid true object regions, either to degrade detection performance globally without tampering with any object's pixels, or to target only specific object types for modification while leaving others unaffected.
• In dynamical systems, object-preserving refers to perturbations adjusting the derivative or dynamics along a periodic orbit such that strong stable/unstable manifolds (local invariant geometric objects) are preserved outside a neigborhood.
• In linear algebra, such perturbations change certain eigenvalues or invariant subspaces of a structured matrix while precisely preserving specified invariant pairs or Jordan chains.

Object-preservation is operationalized via problem-specific constraints, typically implemented via distance penalties, masking, projection onto constraint sets, or algebraic construction.

2. Object-Preserving Perturbations in Neural Networks and Computer Vision

Object-preserving adversarial attacks on object detectors and classifiers have attracted considerable research interest. A spectrum of methods exists:

Butterfly Effect Attacks

In "Butterfly Effect Attack: Tiny and Seemingly Unrelated Perturbations for Object Detection," object-preserving perturbations are constructed to be spatially disjoint from object regions (quantified by a minimum pixel distance), and are subject to an $L_p$-norm bound ("tininess"). Their optimization aims to maximize detection performance degradation while maintaining spatial unrelatedness and small perturbation magnitude. This is achieved through a multi-objective genetic algorithm (NSGA-II), with objectives

• Minimizing $\|\delta\|_p$ (perturbation intensity)
• Minimizing $\text{obj\_degrad}$ (maximum overlap IoU degradation relative to original detections)
• Maximizing $\text{obj\_dist}$ (mean distance of perturbed pixels to object centers, penalizing overlap)

Empirically, transformer-based detectors (e.g., DETR) are highly susceptible to small, distant perturbations, with performance (IoU) dropping to 0.4–0.6 even at $\|\delta\|_2\approx 5$–$10$, while single-stage CNNs (YOLOv5) retain higher robustness for comparably sized perturbations (Doan et al., 2022).

Type- and Box-Specific Label Attacks

"Pick-Object-Attack" introduces constrained adversarial examples where perturbations are confined to only the bounding boxes of a selected object type. All other object detections and regions are unmodified. The method masks gradients and updates to support $\delta$ only on pixels inside a union of designated bounding boxes, using projected gradient descent under an $\ell_2$ or $\ell_\infty$ constraint. Success rates are high (up to 98.4% for targeted class-flipping within boxes), while mean average precision (mAP) for non-targeted objects stays close to the unperturbed baseline (e.g., mAP $\approx 94$–95%) (Nezami et al., 2020).

Attribute-Preserving Adversarial Obfuscation

Attribute-preserving perturbations, as in "Maximal adversarial perturbations for obfuscation," are engineered via a multi-task loss: they maximally obscure a sensitive attribute (e.g., race) while preserving model performance on a public attribute (e.g., gender). The attack maximizes the loss for the hidden attribute classifier and minimizes loss for the preserved attribute, subject to a norm constraint. Both one-step (FGSM) and iterative (PGD) solvers are used. For instance, gender accuracy can be preserved at 73% (baseline: 97%) even when race accuracy is reduced to 0% (baseline: 87%) on the UTKFace dataset (Ilanchezian et al., 2019).

Frequency-Domain Object-Preservation

"Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations" demonstrates preservation of actual object signals in vision-LLMs by explicitly fusing both high- and low-frequency image domains in feature space. The perturbation, additive at the token level, complements original content rather than suppressing or overwriting it. Frequency-domain attenuation at inference time further reduces spurious responses (object hallucinations) while retaining true object representations (Li et al., 19 Mar 2025).

3. Object-Preserving Perturbations in Structured Linear Algebra

In the context of structured matrices (e.g., with symmetries defined by Jordan or Lie algebra constraints), object-preserving perturbations are designed to achieve specified eigenvalue, invariant subspace, or Jordan structure modifications without spillover to complementary invariant objects.

Central results (Ganai et al., 2020) give explicit formulas for structured perturbations:

• Perturbations $\|\delta\|_p$0 (the structure algebra) constructed so that $\|\delta\|_p$1 for a prescribed $\|\delta\|_p$2, preserving object $\|\delta\|_p$3 as an invariant subspace with new dynamics.
• No-spillover perturbations, via block-diagonal modifications in the basis of complementary invariant pairs, guarantee the preservation of designated subspaces and Jordan chains.
• Minimal-rank, structure-preserving updates can modify a prescribed set of eigenvalues while maintaining all other eigenvalues and their associated Jordan chains unchanged.

These constructions typically require full-rank conditions and mutual spectral disjointness of the moved and preserved parts under the algebra involution.

4. Object-Preserving Perturbations in Smooth Dynamical Systems

In differentiable dynamics, object-preserving perturbation techniques selectively modify local behavior (usually derivatives along periodic orbits) while maintaining semi-local invariant manifolds.

The Isotopic Perturbation Lemma (Gourmelon, 2012) refines Franks’ classical result: for a $\|\delta\|_p$4-diffeomorphism and a periodic orbit $\|\delta\|_p$5, any isotopy of derivatives along $\|\delta\|_p$6—subject to persistence of strong stable/unstable subspaces—can be realized by a $\|\delta\|_p$7-small perturbation supported near $\|\delta\|_p$8 that preserves the given strong invariant manifolds (up to a designated size) outside this support. This implies the ability to control nonlinear invariant structures while altering local linearization properties.

Related volume- and symplectic-preserving tools (Buzzi et al., 2016) provide Franks-type lemmas ensuring that prescribed linear data at a set (e.g., a periodic orbit) can be imposed by a $\|\delta\|_p$9-small perturbation, strictly preserving the measure or symplectic form. Applications include creation of new homoclinic intersections and fine adjustment of periodic orbit spectra, all while conserving key geometric objects.

5. Optimization Strategies and Algorithms

Optimization methods for object-preserving perturbations are problem-specific:

• In deep learning, they rely on multi-objective formulations balancing intensity, object separation, and performance degradation (Doan et al., 2022); masking and projection for region-specific attacks (Nezami et al., 2020); or joint adversarial-preservation loss constructions (Ilanchezian et al., 2019).
• In structured algebra, explicit analytic expressions (requiring projectors, pseudo-inverses, and block matrix manipulations) yield object-preserving solutions (Ganai et al., 2020).
• In dynamics, perturbations are constructed via local linearization, surgical pasting, and isotopic connections to maintain both linear and nonlinear invariant structures while realizing desired derivative modifications (Gourmelon, 2012, Buzzi et al., 2016).

A thematic pattern is the use of constraint projection or masking to enforce preservation, and Pareto or max-min balancing to optimize conflicting objectives (e.g., destruction vs. preservation).

6. Applications and Empirical Outcomes

Object-preserving perturbations have achieved several empirical effects:

Domain	Preserved Object	Effect Achieved	Source
Object detection	Non-touched boxes	Extensive detector breakdown with spatially distant, tiny perturbations	(Doan et al., 2022)
Object detection	Non-targeted types	Type-specific label flipping; non-target mAP maintained	(Nezami et al., 2020)
Attribute classification	Public attribute	Targeted attribute obfuscation; public attr. accuracy maintained	(Ilanchezian et al., 2019)
MLLM V+L	True object signals	Hallucination suppression without degrading true object grounding	(Li et al., 19 Mar 2025)
Structured matrices	Complementary subspaces/Jordan chains	Targeted spectral modification, rest of structure unchanged	(Ganai et al., 2020)
Diffeomorphisms	Strong stable/unstable manifolds	Periodic linear data manipulated; manifolds preserved	(Gourmelon, 2012, Buzzi et al., 2016)

Significance includes improved understanding of deep model vulnerabilities, adversarial robustness design, controlled synthesis of privacy-preserving data, precise matrix spectrum engineering, and advanced manipulation of smooth dynamical systems while preserving invariant geometry.

7. Limitations, Variations, and Extensions

Limitations and variants arise from model architecture specificity, potential visual artifacts (in maximal norm adversarial settings), and technical constraints (full-rank, spectral disjointness conditions in linear algebra). Extensions are feasible by adapting object-preservation notions to continuous attributes, richer regularizers for independence/preservation, frequency-decomposed cues (e.g., wavelets), and to broader modalities such as audio and video.

In summary, object-preserving perturbations constitute a unifying methodology across disciplines for inducing desired changes to complex systems with precise control over invariant or semantically critical structures. This enables targeted testing, privacy management, interpretability interventions, and fine-grained system design under explicit preservation guarantees.