PnLCalib: Sports Field Registration via Points and Lines Optimization

Abstract: Camera calibration in broadcast sports videos presents numerous challenges for accurate sports field registration due to multiple camera angles, varying camera parameters, and frequent occlusions of the field. Traditional search-based methods depend on initial camera pose estimates, which can struggle in non-standard positions and dynamic environments. In response, we propose an optimization-based calibration pipeline that leverages a 3D soccer field model and a predefined set of keypoints to overcome these limitations. Our method also introduces a novel refinement module that improves initial calibration by using detected field lines in a non-linear optimization process. This approach outperforms existing techniques in both multi-view and single-view 3D camera calibration tasks, while maintaining competitive performance in homography estimation. Extensive experimentation on real-world soccer datasets, including SoccerNet-Calibration, WorldCup 2014, and TS-WorldCup, highlights the robustness and accuracy of our method across diverse broadcast scenarios. Our approach offers significant improvements in camera calibration precision and reliability.

• The paper introduces a novel framework for sports field registration using geometric keypoints to achieve robust multi-view camera calibration.
• It employs an encoder-decoder CNN with HRNetv2 alongside RANSAC and DLT algorithms to generate accurate 2D-3D correspondences from soccer field features.
• Experimental evaluations on SoccerNet and WorldCup datasets demonstrate superior performance in terms of reprojection error and camera parameter estimation.

PnLCalib: Sports Field Registration via Points and Lines Optimization

Introduction

The paper introduces a novel framework for sports field registration with a focus on soccer field camera calibration using a 3D model. Traditional methods treated this problem as a homography estimation from a single camera view, whereas this work extends calibration to multi-view scenarios typical of broadcast videos. The method leverages the SoccerNet dataset annotations for keypoint extraction and employs classical camera calibration methods, achieving impressive results on real-world datasets like SoccerNet-Calibration and WorldCup datasets.

Proposed Method

The framework employs a geometry-based keypoint retrieval approach, leveraging the distinct geometric features of the soccer field for accurate 3D registration. The key contribution is a hierarchical pipeline structure for keypoint generation, allowing for robust 2D-3D correspondences without further refinement steps. A key innovation is the extraction of keypoints from intersecting field lines utilizing an encoder-decoder neural architecture, specifically the HRNetv2, for keypoint prediction. The approach uses RANSAC and DLT algorithms to compute camera projection matrices.

Figure 1: Overview of the proposed framework. The top diagram illustrates the training data generation pipeline starting with SoccerNet annotations for field line extraction.

The model uses a comprehensive set of keypoints categorized into several sets including line-line, line-ellipse intersections, and others. These keypoints are then used to generate homography matrices and camera parameters, allowing for accurate calibration across multiple camera views.

Key Components and Architecture

The methodological framework presented in this paper consists of the following primary components:

• Soccer Field Modeling: Defines lines and circle segments to serve as calibration objects. Keypoint sets are derived mathematically from these.
• Keypoint and Line Detection: Utilizes two CNNs for heatmap generation, estimating the positions of defined keypoints and field lines.
• Projection and Calibration: Calibration involves transforming 3D soccer field model elements into image coordinates, enhanced by known geometric relationships.

Figure 2: Definition of keypoint positions on a soccer field projected vertically to address analytical intersection of lines and curves.

The architecture ensures precise feature extraction using HRNetv2 as the backbone, which maintains high-resolution representations aiding in accurate homography and camera parameter estimation.

Experimental Evaluation

The proposed system was rigorously evaluated against several state-of-the-art methods across multiple datasets. Performance metrics primarily included reprojection error, projection error in world space, and IoU for both partial and whole field views. Experimental results demonstrated the method's superior performance in 3D camera calibration compared to existing techniques.

Figure 3: Qualitative results showing projection of soccer field lines correlated with estimated camera parameters, alongside annotations.

Quantitative results reflected the robust nature of this methodology across diverse baseline comparisons with alternative calibration strategies in multi-view settings.

Implications and Future Work

The implications of this work are significant in computer vision applications for sports analytics, where accurate field registration can enhance tasks such as automatic camera control and 3D ball tracking. The minimalist nature of the approach, relying solely on field geometry, simplifies implementation without sacrificing accuracy. Future work may explore temporal consistency integration, improving alignment across sequential broadcast frames, further expanding the method's applicability to dynamic sports environments.

Conclusion

The paper presents a novel sports field registration framework capitalizing on a simplified yet effective approach by exploiting geometric features for 3D calibration. The successful integration into multi-view setups positions it as a valuable tool for enhancing broadcast sports analysis, setting a benchmark in camera calibration methodologies for soccer fields. Future research could extend these capabilities by incorporating temporal analysis for real-time applications.