Precise Within-Patch Localization for Semantic Correspondence

Characterize and achieve precise localization within a single image patch for semantic correspondence, addressing the inability of the FiLM-conditioned decoder's single, axis-aligned relative-offset modulation to represent within-patch position hierarchically and in a geometry-aware manner, with the goal of improving performance at the strictest PCK thresholds.

Background

ImCorr replaces grid-bound feature lookup with a continuous feature field that can be queried at arbitrary sub-pixel coordinates. However, the decoder represents a query location using a single FiLM modulation conditioned on its relative offset within one grid cell. The paper notes that this treats each cell as a flat, axis-aligned coordinate system and does not encode within-patch position hierarchically from coarse to fine.

The unresolved issue is particularly important for fine-grained semantic correspondence: although ImCorr substantially reduces quantization error through source-side continuous querying and target-side densification, its target search grid remains finite and its within-cell representation is not fully geometry-aware. Consequently, precise localization within one patch remains unresolved, and performance at the strictest threshold remains substantially lower than at the standard threshold.