Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait (2505.04616v1)

Abstract: We address the problem of whole-body person recognition in unconstrained environments. This problem arises in surveillance scenarios such as those in the IARPA Biometric Recognition and Identification at Altitude and Range (BRIAR) program, where biometric data is captured at long standoff distances, elevated viewing angles, and under adverse atmospheric conditions (e.g., turbulence and high wind velocity). To this end, we propose FarSight, a unified end-to-end system for person recognition that integrates complementary biometric cues across face, gait, and body shape modalities. FarSight incorporates novel algorithms across four core modules: multi-subject detection and tracking, recognition-aware video restoration, modality-specific biometric feature encoding, and quality-guided multi-modal fusion. These components are designed to work cohesively under degraded image conditions, large pose and scale variations, and cross-domain gaps. Extensive experiments on the BRIAR dataset, one of the most comprehensive benchmarks for long-range, multi-modal biometric recognition, demonstrate the effectiveness of FarSight. Compared to our preliminary system, this system achieves a 34.1% absolute gain in 1:1 verification accuracy ([email protected]% FAR), a 17.8% increase in closed-set identification (Rank-20), and a 34.3% reduction in open-set identification errors (FNIR@1% FPIR). Furthermore, FarSight was evaluated in the 2025 NIST RTE Face in Video Evaluation (FIVE), which conducts standardized face recognition testing on the BRIAR dataset. These results establish FarSight as a state-of-the-art solution for operational biometric recognition in challenging real-world conditions.

Overview of "Person Recognition at Altitude and Range: Fusion of Face, Body Shape, and Gait"

The paper "Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait" addresses a significant challenge in biometric recognition—person identification in unconstrained and adverse environments, particularly at long distances and elevated angles. This scenario is typical in surveillance systems like the IARPA Biometric Recognition and Identification at Altitude and Range (BRIAR) program, where biometric data acquisition is complicated by factors such as atmospheric turbulence and high wind speeds.

Key Contributions and Methodology

The authors propose FarSight, an end-to-end unified system designed to enhance person recognition by synergizing complementary biometric modalities including face, gait, and body shape. FarSight integrates novel algorithms across four central modules:

1. Multi-Subject Detection and Tracking: This module reliably identifies and tracks individuals in crowded scenes, accommodating variable distances and angles.
2. Recognition-Aware Video Restoration: The system enhances video clarity and image quality, addressing challenges generated by adverse conditions, making it conducive for subsequent recognition tasks.
3. Modality-Specific Biometric Feature Encoding: This module extracts distinct features across different modalities—each tailored to capture unique and complementary biometric information.
4. Quality-Guided Multi-Modal Fusion: The system merges features from various modalities through a quality-guided process, optimizing recognition performance even when individual modality data is of varying quality.

Experimental Results

FarSight's efficacy is demonstrated through extensive experimentation on the BRIAR dataset, a comprehensive benchmark for long-range, multi-modal biometric recognition. Key performance metrics reveal notable improvements over previous systems:

• A 34.1% increase in 1:1 verification accuracy ([email protected]% FAR).
• A 17.8% rise in closed-set identification metric (Rank-20).
• A 34.3% reduction in open-set identification errors (FNIR@1% FPIR).

Additional validation in the NIST 2025 RTE Face in Video Evaluation (FIVE) further substantiates FarSight’s advanced capabilities in challenging conditions.

Implications and Future Directions

FarSight establishes itself as a robust tool for biometric recognition in operational environments where traditional systems may falter. Its capacity to successfully integrate multiple biometric modalities underlines its potential for deployment in security and surveillance domains.

Looking forward, the major challenges to be addressed include further improving scalability, integrating an effective reject option for dubious classifications, and enhancing the system's generalization capabilities concerning ambient illumination, sensor variations, and different altitudes. Investigating these areas could refine FarSight's functionality and broaden its applicability across diverse real-world scenarios. Moreover, further developments in multi-modal fusion techniques may yield finer improvements in biometric system accuracy and reliability.

Overall, the paper presents significant advancements in the field of long-range biometric recognition, laying groundwork for future research and practical implementations in complex environmental settings.