ReFineR: Single image to dense point cloud registration at pixel level with point splatting
Abstract: Cameras and LiDARs (Light Detection and Ranging) provide complementary observations of the same scene: cameras capture rich appearance cues while LiDAR sensors provide accurate geometric measurements. In this work, we address the problem of fine image-to-point cloud registration from a single image and a roughly estimated camera pose. On robotic platforms, residual reprojection errors of several pixels are commonly observed even after calibration due to temporal desynchronization, mechanical vibrations, or inaccuracies in point cloud reconstruction. Such errors can lead to decimeter-level geometric discrepancies at moderate distances and significantly affect multiple downstream tasks, such as dense cross-modal supervision, texture mapping, and semantic label transfer. To address this challenge, we introduce ReFineR: an offline framework to finely register individual images to pre-reconstructed point clouds. ReFineR builds upon differentiable point splatting rasterization and a smooth Gradient Orientation Measure (GOM) to iteratively refine the camera pose through analysis-by-synthesis optimization. We evaluated the method on a curated dataset of manually calibrated images collected from heterogeneous data sources. Furthermore, we show that an automatic point cloud rescale factor can be estimated prior to optimization, enabling robust refinement across a wide range of scenarios using a single scale-invariant formulation. The project page can be found at https://refiner-project.github.io/
Paper Prompts
Sign up for free to create and run prompts on this paper.
Explain it Like I'm 14
1. What is the paper about?
This paper introduces ReFineR, a computer-vision method that helps a camera image line up accurately with a 3D point cloud.
A point cloud is a collection of millions of points showing the shape of objects and places in 3D. LiDAR sensors create point clouds by sending out laser pulses and measuring how long they take to return. Cameras, meanwhile, record colorful 2D images.
These two sensors are useful together:
- A camera shows colors, patterns, and textures.
- LiDAR shows accurate distances and shapes.
The problem is that the camera image and the LiDAR point cloud may not line up perfectly. Even a small error of a few pixels can mean that an object is placed several centimetersโor even tens of centimetersโaway from its correct position in 3D.
ReFineR is designed to fix this alignment using one image, a pre-built point cloud, and an approximate starting position for the camera.
2. What questions does the research ask?
The researchers mainly wanted to find out:
- Can a cameraโs position and direction be improved using only one image and a 3D point cloud?
- Can this alignment be made accurate to just a few pixels?
- Can the same method work with many different types of point clouds, such as indoor scans, outdoor LiDAR scans, colored point clouds, and computer-generated scenes?
- Can the method automatically handle point clouds stored at different sizes or scales?
- Does combining ReFineR with another matching method make the results even better?
The method is meant for tasks such as:
- Adding camera colors to 3D maps
- Moving labels from images onto LiDAR points
- Building detailed maps for robots and self-driving cars
- Combining information from different sensors
3. How does ReFineR work?
ReFineR uses an idea called analysis by synthesis. This means it repeatedly creates a simulated image from the point cloud and compares that simulated image with the real camera image.
A simple analogy is adjusting a projector until its picture matches a photograph on a wall. If the pictures do not match, you move or rotate the projector. ReFineR does something similar with the virtual camera.
The process has several important steps.
Creating a view of the point cloud
First, the method places a virtual camera at the approximate starting position. It then projects the 3D points onto a 2D image.
Because ordinary point projections can leave empty spaces between points, ReFineR uses point splatting. Each point is treated like a tiny soft spot, similar to a small cloud of paint. These spots overlap to create a more complete image.
The spots are represented using small mathematical shapes called Gaussians. In everyday terms, each point is turned into a small blurry dot whose size depends on how close nearby points are.
Comparing edges
Next, ReFineR looks at the edges in both images. Edges are places where brightness or structure changes sharply, such as:
- The outline of a car
- The edge of a building
- The border between a road and a sidewalk
It uses a measure called the Gradient Orientation Measure, or GOM. This compares the direction of edges in the real image and the rendered point-cloud image.
For example, if a wall edge slopes upward in both images, that is a good sign that the camera is correctly positioned.
The method can compare several kinds of information, including:
- Image brightness
- Point-cloud surface directions, called normals
- LiDAR reflectivity
- Depth
- Color, when available
Improving the camera pose
The cameraโs pose means its position and direction. It has six main parts:
- Movement left and right
- Movement up and down
- Movement forward and backward
- Rotation around three different directions
ReFineR changes these values little by little. After each change, it renders a new point-cloud image and checks whether it matches the real image better.
This is done using gradient descent, a technique similar to walking downhill. The algorithm checks which direction makes the mismatch smaller and takes a step that way.
Because the entire rendering process is differentiable, the system can calculate how each small camera movement changes the final error. This is called differentiable rendering.
Automatically handling scale
Point clouds can have very different sizes. One might show a small room, while another might show an entire street.
If the system uses the same movement settings for both, it may move too far in one scene and not far enough in another. ReFineR therefore estimates a suitable scale automatically. This helps balance the effects of camera movement and camera rotation without manually adjusting settings for every scene.
4. How was the method tested?
The researchers created and collected a dataset containing 164 carefully aligned images from 29 different scenes. The data came from several sources:
- ETH3D
- CARLA, a driving simulator
- nuScenes
- Tanks and Temples
- RGBDScenes
- Livox LiDAR data
They deliberately added errors to the camera positions, usually creating an initial error of about 10 pixels. The algorithms then had to correct those errors.
ReFineR was compared with several other methods, including:
- Methods that compare image brightness and LiDAR intensity
- Methods that compare image and point-cloud edges
- MatchAnything, a system that finds matching features
- GS-SLAM, a method based on differentiable rendering
The researchers measured how many results ended up within a certain number of pixels of the correct alignment. For example, pix@2 means the percentage of cases with less than 2 pixels of error.
5. Main findings
ReFineR greatly improves alignment
When only the geometry of the point cloud was available, ReFineR often performed better than the older methods, especially on accurate datasets.
For example:
- On ETH3D, ReFineR produced about 41% of registrations with less than 1 pixel of error, compared with 8% for MatchAnything.
- On CARLA, it produced about 44% with less than 2 pixels of error, compared with 14% for MatchAnything.
This shows that ReFineR can make very small corrections successfully when the point cloud is reasonably clean.
Combining methods works best
ReFineR was especially effective when used after MatchAnything.
MatchAnything is good at making a rough alignment, while ReFineR is good at making the final tiny adjustments. Together, they worked better than either method alone.
In one experiment:
- On ETH3D, the combined method reached about 58% below 1 pixel of error.
- On CARLA, it reached about 68% below 2 pixels of error.
This suggests that the two methods have complementary strengths:
- MatchAnything finds a promising starting position.
- ReFineR carefully refines it.
Extra information helps
When the point cloud included color or reflectivity, registration generally became easier. These extra values provide patterns that can be compared with the camera image.
For example, road markings may create special reflectivity patterns in a LiDAR scan. Such patterns can help the system determine where the image belongs.
In some tests with colored or intensity-based point clouds, the combined method reached very strong results, including:
- About 86% below 1 pixel on ETH3D
- About 99% below 3 pixels on RGBDScenes
However, reflectivity is not always reliable because it can depend on the material, laser angle, and distance. It does not always correspond directly to what a camera sees.
The method has limits
ReFineR does not always work well when the point cloud is noisy, sparse, or poorly aligned at the start. It can also struggle when image brightness and point-cloud geometry have little relationship.
The method is designed mainly for offline processing, where accuracy matters more than speed. It usually takes around 10 to 40 seconds per image, depending on the scene and number of points. This makes it less suitable for immediate real-time use in a fast-moving robot, but useful for improving maps after data collection.
6. Why is this research important?
Small camera-LiDAR errors can cause large 3D mistakes. For example, an error of 10 pixels might place a road marking, building edge, or object dozens of centimeters away from its true location.
ReFineR offers a way to correct these errors without training a new neural network for every type of scene. It can work with many different point clouds and can automatically adjust to their scale.
The research could improve:
- Self-driving car maps, by making camera and LiDAR data line up more accurately
- Robot vision, by helping robots understand where objects are
- 3D maps, by placing colors and labels in the correct locations
- Training data, by creating more reliable links between images and 3D points
- Mixed-reality systems, where virtual objects must fit precisely into the real world
Overall, the paper shows that carefully comparing a real image with a computer-generated view of a point cloud can produce very accurate camera alignment. The strongest approach is to first obtain a rough match and then use ReFineR as a precise โfinal adjustmentโ tool.
Practical Applications
Immediate Applications
The paperโs main practical contribution is a training-free, differentiable method for refining the pose of a camera relative to a dense, preconstructed point cloud. The method is most suitable for offline or near-offline processing, rather than safety-critical real-time localization.
- HD map construction and maintenance โ autonomous driving and mapping
- Apply ReFineR to align each camera frame with an aggregated LiDAR map after conventional calibration and LiDAR odometry.
- Use the refined poses to produce more accurate road markings, lane boundaries, traffic-sign textures, building facades, and other camera-derived map attributes.
- A practical workflow is:
coarse calibration โ LiDAR reconstruction โ point-cloud rescaling โ ReFineR refinement โ texture/map attribute projection.- This is immediately deployable for batch processing of recorded vehicle fleets and mobile-mapping surveys.
- Dependencies: a sufficiently accurate initial pose, overlap between the image and point cloud, known camera intrinsics, and a relatively static scene. Dynamic objects, sparse regions, and large initial pose errors can reduce reliability.
- Dense semantic label transfer from images to LiDAR
- Transfer pixel-level semantic or instance labelsโsuch as cars, pedestrians, road surfaces, vegetation, and buildingsโfrom an RGB image to dense 3D points after refining the camera pose.
- This can improve the creation of multimodal training datasets for autonomous-driving perception, robotics, and urban-scene understanding.
- ReFineR is particularly relevant where a few-pixel error would otherwise assign labels to neighboring objects or create noisy 3D masks.
- Dependencies: the image segmentation model must produce reliable labels, and the point cloud must contain enough density and geometric structure in the relevant image regions. Occlusions and independently moving objects require additional filtering.
- CameraโLiDAR calibration quality control
- Use ReFineR as a diagnostic tool for detecting frame-specific calibration failures caused by vibration, rolling shutter, temporal desynchronization, SLAM drift, or mechanical movement.
- Instead of assuming one fixed extrinsic transform for an entire sequence, systems can estimate per-frame residual corrections and log them for maintenance or recalibration.
- A calibration dashboard could report reprojection error distributions, failure cases, and drift over time.
- Dependencies: the coarse calibration must place the correct scene in the camera frustum. ReFineR refines an existing pose; it is not a replacement for global sensor calibration or robust initialization.
- Texture mapping and photorealistic 3D reconstruction
- Refine camera poses before projecting RGB imagery onto LiDAR meshes or point clouds.
- This can reduce texture seams, ghosting, double edges, and incorrect color assignment in digital twins, cultural-heritage scans, construction surveys, and industrial facilities.
- The method can use geometric channels alone, or richer channels such as LiDAR reflectivity and embedded scanner color when available.
- Dependencies: the point cloud must represent the same scene state as the image, and point density must support stable splatting. Lighting changes and transient objects can weaken intensity-based alignment.
- Post-processing of mobile mapping and surveying data
- Apply ReFineR to images and dense point clouds collected by cars, handheld scanners, drones, or terrestrial LiDAR systems.
- It can correct residual alignment errors in workflows for road inspection, infrastructure inventories, faรงade documentation, and construction progress monitoring.
- The automatic scale-normalization procedure is useful when datasets combine indoor, outdoor, RGB-D, and LiDAR sources without manually retuning translation learning rates.
- Dependencies: the method is computationally variableโroughly dependent on the number of pointsโand may require GPU processing for large scenes. It is therefore better suited to batch pipelines than to embedded deployment.
- RGB-D and indoor robotics data fusion
- Refine the alignment of RGB frames with reconstructed indoor point clouds from depth cameras or RGB-D SLAM systems.
- Potential uses include room-scale semantic mapping, object annotation, manipulation-scene reconstruction, and inspection of warehouses or laboratories.
- Normal, depth, grayscale, and color channels can be selected according to sensor availability.
- Dependencies: indoor scenes may contain repetitive geometry, reflective surfaces, sparse depth, or insufficient texture. These conditions can create local minima or ambiguous alignments.
- Improved multimodal dataset creation and benchmarking โ academia and industry
- Use the released heterogeneous dataset and registration procedure to construct more accurately aligned imageโpoint-cloud datasets.
- Researchers can benchmark semantic transfer, cross-modal feature learning, 2Dโ3D matching, and cameraโLiDAR calibration under controlled pixel-error thresholds.
- The paperโs use of
pix@1,pix@2, andpix@5metrics provides a practical way to distinguish coarse alignment from true pixel-level registration. - Dependencies: manually calibrated ground truth remains expensive, and results may vary across point-cloud density, sensor type, scene dynamics, and image resolution.
- Hybrid coarse-to-fine registration tools
- Combine a learned or feature-based method such as MatchAnything with ReFineR:
- 1. generate a coarse imageโpoint-cloud alignment using feature matching;
- 2. render a dense splatted point-cloud view;
- 3. refine the pose using differentiable GOM optimization.
- The paper reports that this combination improves robustness over either method alone, especially when the initial error is several pixels.
- This could be implemented as an offline calibration module in robotics or mapping software.
- Dependencies: the coarse matcher must provide a pose within ReFineRโs effective convergence basin. The combined pipeline also increases software and computational complexity.
- Quality improvement for 3D annotation and simulation assets
- Use refined registration to align real images with reconstructed scenes used for annotation, synthetic-data validation, or simulator comparison.
- In autonomous-driving simulation, the method can validate whether rendered sensor views remain geometrically consistent with real-camera observations.
- Dependencies: simulator and real-world appearance differences may make grayscale or photometric channels unreliable. Geometry-based channels and robust initialization remain important.
Long-Term Applications
The following applications are plausible extensions but require additional research, engineering, or validation before broad deployment.
- Per-frame online cameraโLiDAR self-calibration
- Adapt ReFineR for operation during data acquisition so that a vehicle, robot, or drone continuously estimates small time-varying extrinsic corrections.
- This could compensate for thermal expansion, vibration, sensor mounting deformation, and synchronization errors.
- A future system might maintain a slowly varying calibration state while rejecting corrections caused by moving objects.
- Dependencies: the current method is explicitly designed for offline processing and prioritizes accuracy over speed. Online use would require faster rasterization, limited iterations, temporal priors, confidence estimation, and robust handling of dynamic scenes.
- Real-time multimodal localization
- Integrate differentiable point splatting into a localization stack that estimates a camera pose against a large prebuilt map.
- Potential sectors include warehouse robots, autonomous vehicles, augmented reality, and drone navigation.
- Dependencies: ReFineR currently assumes a coarse pose and can take approximately 10โ40 seconds depending on the scene and method configuration. Real-time deployment would require map tiling, aggressive frustum culling, GPU optimization, hierarchical search, and a reliable global relocalization front end.
- Joint optimization of pose, time offset, and rolling-shutter parameters
- Extend the differentiable formulation beyond six-degree-of-freedom pose to optimize:
- cameraโLiDAR temporal offset;
- rolling-shutter readout time;
- sensor trajectory during exposure;
- possibly camera intrinsics and distortion.
- This could directly address several error sources identified in the paper rather than treating them as residual pose errors.
- Dependencies: the scene must provide sufficient geometric and visual constraints. Motion must be modeled accurately, and additional parameters may make the optimization ill-conditioned or ambiguous.
- Dynamic-scene-aware registration
- Develop masks or robust losses that distinguish static geometry from moving vehicles, pedestrians, machinery, vegetation, or construction activity.
- A future workflow could use object detection and scene-flow estimation to exclude dynamic regions before GOM optimization.
- Dependencies: reliable segmentation or tracking is required. Removing too many regions may leave insufficient structure for stable registration.
- Automatic channel selection and confidence estimation
- Build a system that chooses among normals, depth, reflectivity, grayscale, RGB, and learned depth or normal channels based on scene and sensor quality.
- It could also output uncertainty estimates or a registration-validity score for downstream systems.
- Dependencies: channel usefulness varies substantially across datasets. Reflectivity may encode material or range rather than appearance, while normal-to-grayscale comparisons can be weak. Large-scale validation is needed to learn failure indicators.
- Differentiable alignment of heterogeneous 3D representations
- Extend the method from raw point clouds to meshes, surfels, Gaussian Splatting maps, neural implicit fields, and multi-resolution spatial maps.
- Such a unified alignment layer could support mixed mapping pipelines in which different parts of a facility or city are represented by different 3D technologies.
- Dependencies: each representation requires an appropriate differentiable renderer, visibility model, and treatment of missing or uncertain data. The current method is designed around pre-reconstructed point clouds.
- Automatic multimodal supervision for foundation models
- Use refined imageโpoint-cloud alignment to generate high-quality pixelโpoint correspondences for training 2Dโ3D foundation models, open-vocabulary perception systems, and cross-modal retrieval models.
- This could reduce label noise in large-scale datasets and improve learning of shared representations across images, depth, LiDAR, and reflectivity.
- Dependencies: registration quality must be verified at scale. Systematic failures in reflective, textureless, dynamic, or sparsely sampled environments could introduce biased training data.
- Robotic manipulation and embodied AI
- Align RGB observations with dense 3D reconstructions to improve object pose estimation, grasp planning, scene memory, and language-grounded spatial reasoning.
- For example, a robot could use a refined camera pose to transfer object masks or affordance predictions into a persistent 3D workspace model.
- Dependencies: manipulation requires low latency and high reliability. Occlusions, object motion, viewpoint changes, and the need for centimeter- or millimeter-level accuracy may exceed the current methodโs assumptions.
- Infrastructure digital twins and change detection
- Apply refined registration to repeated imageโLiDAR surveys of bridges, roads, factories, mines, or power infrastructure.
- More accurate alignment would allow temporal comparison of cracks, corrosion, deformation, missing components, and construction progress.
- Dependencies: change detection must distinguish true physical change from viewpoint, illumination, vegetation, and sensor differences. The point cloud map and current imagery must be registered to a stable reference frame.
- Policy and standards for multimodal spatial data
- The paper supports the case for reporting pixel-level calibration qualityโnot merely nominal sensor extrinsicsโin public datasets and procurement specifications.
- Future standards could require per-frame reprojection-error distributions, dynamic-scene exclusions, calibration drift reports, and thresholds appropriate for tasks such as semantic transfer or HD mapping.
- Dependencies: standardized evaluation protocols are needed across image resolution, point density, camera models, and scene type. A single pixel threshold is not sufficient for all operational contexts.
- Consumer and daily-life spatial-computing applications
- A scaled-down version could improve alignment between smartphone images, depth scans, and room-scale 3D models for augmented reality, home measurement, indoor navigation, and personal digital twins.
- Potential products include a scan-quality checker, automatic room-model texturing, or an AR calibration assistant.
- Dependencies: consumer devices have limited compute, variable camera intrinsics, sparse or noisy depth, and frequently changing scenes. Real-time optimization and robust failure detection would be essential.
Glossary
- Adam optimizer: An adaptive gradient-descent algorithm that adjusts learning rates for individual parameters. โGradient descent is handled by the Adam optimizer in PyTorch with learning rateโ
- analysis-by-synthesis: An optimization strategy that renders a model, compares the rendering with observed data, and updates the model or viewpoint iteratively. โanalysis-by-synthesis -- the principle of iteratively rendering the point cloud in artificial images and comparing each render with the real image in a loopโ
- backpropagation: The process of propagating loss gradients backward through a computational graph to update parameters. โThe camera pose is optimized by backpropagation through the differentiable imageโ
- bundle adjustment: A nonlinear optimization procedure that jointly refines camera parameters and 3D structure by minimizing reprojection error. โwhich is a major issue in numerical optimization~\cite{Triggs1999BundleA}โ
- camera extrinsics: Parameters describing a cameraโs position and orientation relative to a reference coordinate system. โa small perturbation of the camera extrinsics around $0$โ
- camera intrinsics: Internal camera parameters, such as focal lengths and principal-point coordinates, that define the projection from 3D space to the image plane. โwith , the camera intrinsicsโ
- correspondence: A presumed association between elements in two data modalities, such as a 3D point and a 2D pixel. โComparison between modalities is achieved by creating correspondences between point clouds and image channelsโ
- cross-modal supervision: Training or optimization guidance obtained by relating information from different sensor modalities. โdense cross-modal supervisionโ
- differentiable rendering: Rendering in which image values have computable derivatives with respect to scene and camera parameters. โDifferentiable rendering creates images whose pixel values are differentiable with respect to the 3D scene's parametersโ
- diffusion model: A generative model that learns to produce or transform data through a gradual denoising process. โadding generated channels for images using the Marigold diffusion modelโ
- eigenvalue: A scalar characterizing how a matrix scales a corresponding eigenvector; often used to analyze a transformationโs principal directions. โwith the respective eigenvalues of and โ
- extrinsic calibration: The estimation of the relative pose between sensors, including their rotation and translation. โassuming a fixed extrinsic calibration between sensors across an entire sequenceโ
- frustum culling: The removal of 3D points that lie outside the cameraโs visible viewing volume. โwhich in general doesn't exceed 10M (including frustum culling)โ
- Gaussian Splatting: A scene-representation and rendering technique that projects Gaussian primitives into an image and blends their contributions. โbased on Gaussian Splatting's~\cite{kerbl3DGaussianSplatting2023} frameworkโ
- Gauss-Newton optimization: An iterative method for solving nonlinear least-squares problems using a linearized approximation of the residuals. โminimizing the reprojection error with Gauss-Newton optimizationโ
- Gradient Orientation Measure (GOM): An image-comparison measure that evaluates the alignment of gradient directions between two images. โwe follow Taylor et al's work~\cite{Taylor2015MultiModalSC} and adapt the Gradient Orientation Measure (GOM)โ
- gradient descent: An optimization method that updates parameters in the direction that reduces an objective function. โOptimization is carried out using the Lie version of gradient descent, with a fixed learning rate.โ
- Hilbert-Schmidt norm: A matrix norm equal to the square root of the sum of the squared matrix entries, also called the Frobenius norm. โwe choose to equalize the Hilbert-Schmidt norms between and โ
- homogeneous modality: A data modality or source type that is consistent in form or sensing characteristics; in the paper, heterogeneous sources are contrasted with one another. โa curated dataset of manually calibrated images collected from heterogeneous data sourcesโ
- Histogram of Oriented Gradients (HOG): A feature descriptor that summarizes local image-gradient directions in spatial regions. โHistogram of Oriented Gradients (HOG) featuresโ
- hyperparameter: A configuration value selected externally to control an algorithmโs behavior rather than learned during optimization. โThe first two methods (Koide~et al and Yuan~et al) require hyperparameter tuning based on the scene type and scaleโ
- implicit registration: Registration performed by optimizing a comparison between rendered and observed data without explicitly estimating point-to-pixel correspondences first. โour approach (ReFineR) falls into the category of implicit registrationโ
- Lie algebra: A mathematical representation of infinitesimal transformations associated with a continuous transformation group. โminimized a line reprojection error over the Lie algebra of rigid transformsโ
- Lie gradient: A gradient expressed with respect to parameters in a Lie-algebra representation of geometric transformations. โBackpropagation is carried out with computation of Lie gradients with respect to the camera parameter โ
- LiDAR reflectivity: A sensor-return attribute related to the strength of light reflected by a surface. โLiDAR reflectivity ()โ
- Levenberg-Marquardt optimizer: A nonlinear least-squares optimization algorithm combining gradient-descent and Gauss-Newton behavior. โusing the Levenberg-Marquardt optimizerโ
- Mutual Information: A statistical measure of the dependence between two variables or distributions, used here to compare image modalities. โthanks to Mutual Information and Histogram of Oriented Gradients (HOG) featuresโ
- Nelder-Mead optimizer: A derivative-free optimization method that searches with a simplex in parameter space. โThis loss was then optimized using the non-differentiable Nelder-Mead optimizerโ
- parallax: The apparent displacement of a projected point caused by a viewpoint or camera-translation change. โThis is the well-known phenomenon of parallax.โ
- photometric error: A discrepancy between the intensity or color values of corresponding pixels in two images. โoften based on photometric error minimizationโ
- pinhole camera model: A geometric camera model that projects 3D points onto an image plane using perspective division. โderived from first-order pinhole camera model analysisโ
- point splatting: A rendering technique that projects spatially extended point primitives onto image pixels. โDifferentiable point splatting creates a view of the point cloudโ
- pointmap: A tensor assigning a 3D spatial position to each image pixel. โa Width Height 3 tensor representing the position in space of each pixelโ
- RANSAC: A robust estimation method that repeatedly fits a model to random subsets to reject outliers. โthe EPnP algorithm~\cite{lepetitEPnPAccurateSolution2008} combined with the RANSACโ
- reparameterization: The reformulation of a model using alternative variables to improve numerical behavior. โthis rescaling procedure is a form of reparameterization to obtain better variable conditioningโ
- reprojection error: The discrepancy between an observed image location and the location predicted by projecting a 3D point through a camera model. โPixel error (a.k.a. reprojection error)โ
- scale-invariant: Unaffected by a change in the overall scale of the input geometry. โReFineR is a scale-invariant approachโ
- Sobel filter: A convolutional image-processing operator used to estimate horizontal and vertical intensity gradients. โGradients are computed using a Sobel filter.โ
- Structure-from-Motion (SfM): A computer-vision technique that jointly estimates camera poses and 3D structure from multiple images. โStructure-from-Motion point clouds registered on a LiDAR point cloudโ
- surfel: A small surface element, commonly represented by a point with associated normal, size, and appearance properties. โGaussian/surfel splattingโ
- variable conditioning: The numerical sensitivity and scaling relationship among optimization variables, which affects optimization stability. โto obtain better variable conditioningโ
- volumetric rendering: Image synthesis by accumulating contributions along camera rays through a participating volume. โinvolve mechanisms like volumetric renderingโ
- 6 degrees of freedom (6-DoF): The three rotational and three translational parameters defining a rigid-body pose. โestimate the camera's 6 degrees of freedomโ

