Focal Plane Image Analysis

• Focal plane image analysis is the process of extracting and interpreting optical signals directly from the image plane, enabling applications like adaptive optics, computational photography, and sensor verification.
• Methods include wavefront sensing using phase diversity, deep learning for multi-focal reconstruction, and on-sensor processing that transforms focal-plane data into actionable control or calibration signals.
• The approach leverages the inherent physics of image formation to calibrate optical systems, measure detector performance, and achieve precision in applications ranging from astronomical instrumentation to high-speed event detection.

Searching arXiv for recent and foundational papers relevant to focal-plane image analysis to support the encyclopedia entry. {"query":"focal plane image analysis focal plane wavefront sensing adaptive optics bokeh rendering focal plane array arXiv", "max_results": 10} {"query":"focal plane image analysis arXiv", "max_results": 5} Focal plane image analysis denotes the extraction, calibration, reconstruction, or control of information directly from data formed at the image plane of an optical or electro-optical system. In the literature, the term spans focal-plane wavefront sensing for adaptive optics and coronagraphy, detector-plane metrology and verification for large astronomical instruments, event reconstruction in X-ray focal planes, multi-focal-plane reconstruction in microscopy, controllable computational photography, and on-sensor or compressive focal-plane processing (Korkiakoski et al., 2014, Hao et al., 2010, Ma et al., 2020, Chen et al., 2024, Güngör et al., 2023). Across these settings, the focal plane is not treated merely as the terminal recording surface; it is used as the primary inferential interface for phase, depth, detector geometry, spectral response, scene structure, or downstream control.

1. Conceptual scope and problem classes

A unifying feature of focal plane image analysis is that it relies on quantities measured where the focused field is actually recorded. In adaptive optics, this is the science camera, used because it contains the real residual wavefront error in the science path rather than an upstream proxy. In detector metrology, it is the mosaic or array whose image deformation, beam shape, or event morphology is itself the diagnostic. In computational imaging, the focal plane becomes the locus where multiplexed, refocused, or aperture-conditioned measurements are decoded into higher-level scene variables (Korkiakoski et al., 2014, Tapia et al., 2020, Cheng et al., 2019).

The main problem classes in the cited work are distinct but structurally related. One class estimates hidden optical states, especially pupil-plane phase or electric field, from focal-plane intensity patterns. A second class characterizes the focal plane as hardware: CCD flatness, beam uniformity, resonance yield, read noise, dark current, or spectral response. A third reconstructs missing spatial or axial information, as in multi-focal-plane microscopy, temporal-ghost-imaging-based depth recovery, or aperture-controlled bokeh synthesis. A fourth pushes computation onto the focal plane itself, as in focal-plane sensor-processor arrays and compressive focal plane array systems (Groff et al., 2012, Chen et al., 2021, Bo et al., 2018, Lisondra et al., 2024).

A recurrent misconception is that focal plane analysis is synonymous with post-processing of ordinary images. The cited literature shows a broader meaning. In some systems, focal-plane data are used inside closed-loop optical control; in others, the focal plane is the calibration target; in others still, the focal plane performs part of the computation before full readout (Nousiainen et al., 1 Apr 2026, Hao et al., 2010, Lisondra et al., 2024).

2. Physical observables and forward models

The fundamental observable in many focal-plane methods is the point-spread function or its intensity distribution. In focal-plane wavefront sensing, the monochromatic PSF is modeled by Fraunhofer diffraction as

$p = \left|F\{A \exp(i\phi)\}\right|^2,$

where $A$ is the pupil amplitude and $\phi$ is the pupil-plane phase (Korkiakoski et al., 2014). This makes the focal-plane image a nonlinear function of the pupil field, which is useful but also ambiguous: intensity alone does not uniquely determine phase.

In computational photography, the focal plane is tied to physically grounded depth-of-field. The bokeh-rendering work on customized focal-plane guidance explicitly uses the circle-of-confusion relation

$$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$

where $r$ is the circle-of-confusion diameter, $A$ is aperture diameter, $f$ is focal length, $D_o$ is object distance, and $D_f$ is focal plane distance. This formulation motivates coupling focal-plane control and aperture control rather than treating blur as an unstructured image effect (Chen et al., 2024).

In anisoplanatic adaptive optics imaging, off-axis focal-plane morphology is modeled through the optical transfer function

$$OTF(\lambda, C_n^2(h), \theta) = OTF_0(\lambda)\cdot ATF(\lambda, C_n^2(h), \theta),$$

so that the field dependence of the PSF becomes an estimator of the turbulence profile $A$0. The central claim of focal plane profiling is that the science image itself contains information about the vertical distribution of turbulence because anisoplanatism is a function of $A$1 (Beltramo-Martin et al., 2018).

In temporal ghost imaging for focal-plane 3D imaging, the relevant observable is neither a conventional PSF nor a directly time-resolved waveform. Instead, a slow integrating camera measures depth-dependent integrated returns, and the integration time per pixel is inferred through temporal correlation: $A$2 followed by

$A$3

The focal plane therefore becomes a depth sensor through correlation statistics rather than direct time stamping (Bo et al., 2018).

A further measurement model appears in LED-array multi-focal microscopy, where a multiplexed image is approximated as

$A$4

Here the focal-plane measurement is deliberately engineered so that one exposure carries information about several focal planes, which are then recovered by a neural network (Cheng et al., 2019).

3. Wavefront sensing and control at the focal plane

Focal-plane wavefront analysis is a major methodological family because it measures aberrations in the same optical path as the science image, avoiding non-common-path mismatch. The classical problem is to estimate the complex field or phase from intensity measurements, often with phase diversity introduced by known corrections or deformable-mirror probes (Korkiakoski et al., 2014, Groff et al., 2012).

The Fast & Furious lineage exploits weak-aberration structure. The original formulation expands the pupil field under a weak-phase approximation and separates odd and even phase components using pupil symmetries. In experiments with a spatial light modulator controlling the wavefront with a resolution of $A$5 pixels, the methods increased the Strehl ratio from $A$6 to $A$7-$A$8; the remaining wavefront rms error was estimated to be $A$9 rad with FF and $\phi$0 rad with FF-GS (Korkiakoski et al., 2014). For higher-order adaptive optics with more than $\phi$1 control elements, the comparative study of FF, FF-GS, Gerchberg-Saxton, and convex optimization concludes that algorithms similar to Fast & Furious are the easiest practical solution in the considered framework, whereas GS requires roughly $\phi$2 more computation than FF-GS and tends to amplify high-frequency speckles after convergence (Korkiakoski et al., 2014).

A central controversy in this area is phase uniqueness. Several papers state, in different forms, that a single focal-plane image is insufficient for unambiguous phase recovery. Pairwise probing, phase diversity, and temporal diversity are therefore not implementation details but observability mechanisms. The Kalman-filter-based focal-plane estimator makes this explicit by treating the electric field in the dark hole as a state, combining prior knowledge and new probe images. In the reported experiments, a 2-pair Kalman configuration achieved $\phi$3 after 30 iterations with 86 estimation images, while reducing exposure burden relative to conventional DM-diversity baselines (Groff et al., 2012). The broadband estimator on HiCAT extends pairwise probing to an incoherent sum of monochromatic intensities and reports about an order-of-magnitude contrast improvement, from around $\phi$4 to $\phi$5, across a 6% band centered at 640 nm (Redmond et al., 2021).

Recent focal-plane methods increasingly replace explicit inversion with learned control. The Photonic Lantern Wavefront Sensor places a photonic lantern at the science focal plane and reconstructs the first 9 non-piston Zernike modes from 19 single-mode output intensities; its best neural network achieved $\phi$6 radians RMSE, compared with $\phi$7 radians for a linear SVD reconstructor (Norris et al., 2020). PO4NCPA reformulates non-common-path-aberration correction as model-based reinforcement learning with sequential phase diversity. In static non-coronagraphic cases it achieves average Strehl $\phi$8, close to the fitting-error limit of $\phi$9; in the ELT pupil plus vector vortex coronagraph setting it improves contrast relative to open loop by up to $$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$0 at $$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$1 and about $$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$2 at $$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$3–$$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$4 (Nousiainen et al., 1 Apr 2026).

4. Detector metrology, instrument verification, and event reconstruction

A second major branch of focal plane image analysis treats the focal plane itself as the measurement target. For DECam, the problem is whether sixty-two $$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$5 and twelve $$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$6 fully depleted CCDs lie within a 60-micron envelope when mounted inside a high-vacuum dewar at 173 K. The proposed image-based solution projects a dot pattern, measures spacing changes across CCDs, and infers effective offsets and tilt from magnification changes. After fitting and subtracting a global plane, the method identified 7 CCDs outside specification. The paper explicitly notes that the resulting values are effective offsets inferred from image deformation rather than direct mechanical $$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$7-measurements (Hao et al., 2010).

For submillimetre instrumentation, beam maps across the focal plane are themselves the primary analysis product. The MUSCAT focal plane contains 1458 horn-coupled LEKIDs in six readout channels. Raster scanning of a 1500 K Interspectrum type-2 thermal IR source behind an adjustable aperture of about 2.7 mm produced detector-wise beam maps, which were fit with 2D Gaussian and elliptical Gaussian models. The typical deconvolved beam FWHM peaks at $$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$8 mm, close to the expected 3.1 mm, and the eccentricity distribution peaks around $$r = \frac{A}{2} \times \frac{f}{D_o} \times \frac{|D_o - D_f|}{|D_f - f|},$$9. The same analysis exposed a localized region suggesting slight tilt or deformation of the focal plane relative to the image plane. Downstream verification included responsivity $r$0, $r$1, and spectral response; the NEP histograms at 298.5 K peak at approximately $r$2, close to the theoretical $r$3, and the final yield after beam-quality and readout-bandwidth filtering is 961 usable detectors (Tapia et al., 2020).

High-energy focal planes add event-level reconstruction. The Arcus X-Ray Spectrograph uses two Detector Assemblies, each with an 8-CCD linear array, and processes the digitized stream with an Event Recognition Processor that bias-corrects pixels, identifies local maxima, and stores $r$4 event islands containing position, time, and pulse height. Ground tests report $r$5–$r$6 RMS read noise and $r$7 eV FWHM at 0.7 keV, meeting the requirement of $r$8 eV (Grant et al., 2024). A related soft-X-ray focal-plane detector based on the GSENSE6060BSI back-illuminated sCMOS sensor constructs a pixel-wise bias map from 100 dark frames, uses a $r$9 event threshold, classifies single-pixel and split-pixel events, and calibrates energy with

$A$0

It reports readout noise about 3.2 $A$1, dark current about $A$2 below $A$3, a low-energy threshold of about 186 eV, $A$4 eV FWHM at 8.04 keV, and integral nonlinearity $A$5 over 1.2 keV to 8.9 keV (Chen et al., 2021).

These studies also clarify a methodological point: focal-plane analysis in instrumentation is not restricted to image sharpness. It includes geometry, beam morphology, event grading, resonance validity, and spectral-response verification.

5. Multi-focal-plane reconstruction and computational photography

In microscopy and computational photography, focal plane image analysis often means inferring or synthesizing multiple focus conditions from limited measurements. MFPINet reconstructs a high-resolution multi-focal-plane stack from a single 2D low-resolution wide-field image. Using a GAN framework with post-sampling and all-plane refocusing in one pass, it reconstructs 11 focal planes from the middle layer of an 11-layer stack. On a single RTX 2080 Ti GPU, reconstructing a $A$6 volume takes 27.8 ms, whereas Deep-Z takes 61.3 ms for one layer and 673.8 ms for all 11 layers, making MFPINet approximately 24 times faster for the full volume. The model is also much smaller, with 2.50 million parameters versus Deep-Z’s 19.41 million (Ma et al., 2020).

A different single-shot strategy is deep learned optical multiplexing for multi-focal plane microscopy. In that system, a $A$7 LED array microscope uses the 69 centermost LEDs, and the image

$A$8

is jointly optimized with a neural network so that one multiplexed exposure can be decoded into 5 focal planes. The live-imaging demonstration on Dugesia japonica reports 192 ms exposure time and 5.2 frames/s (Cheng et al., 2019). Temporal ghost imaging offers another route: a standard framing camera records a single-shot 2D projection image, while repeated random temporal modulation and correlation recover per-pixel integration time and hence depth (Bo et al., 2018).

Computational photography extends these ideas to user-facing focal control. The variable-aperture bokeh method introduces explicit focal-plane customization: a pretrained Depth-Anything-V2 network predicts depth, a user-provided mask selects the region of interest, and the depth histogram is partitioned into $A$9 regions by maximizing between-class variance, with $f$0 set empirically. The renderer then fuses the all-in-focus image, depth map, focal plane, and aperture embedding through a Multiple Information Fusion Block and a Lens-Fusion Mamba Block. The method is trained with AdamW, L1 loss and SSIM loss, and the full VABM model has only 4.4M parameters and 9.9 GFLOPs. On the EBB! benchmark, it achieves PSNR 24.83, SSIM 0.8815, and LPIPS 0.2169 while remaining much lighter than mainstream computational bokeh models. The accompanying Variable Aperture Bokeh Dataset is captured with $f$1, $f$2, $f$3, and $f$4 settings and includes 535 training image groups and 200 test groups (Chen et al., 2024).

The same branch also includes optical systems that split the focal-plane image into regions for separate processing rather than splitting the entire beam. The double-beam optical image analyzer diverts only a selected region at the primary focal plane into an indirect path, permitting different magnification, different object distances, or different wavelength-band processing within a single secondary image plane (Popowicz et al., 2016). This suggests that focal-plane analysis can be optical as well as algorithmic.

6. On-sensor focal-plane processing, compressive imaging, and measurement limits

Some systems move analysis from downstream computation to the focal plane array itself. BIT-VIO is built on the SCAMP-5 focal-plane sensor-processor array, a $f$5 SIMD architecture in which each pixel has 7 analog registers, 13 digital registers, and an ALU. SCAMP-5 performs early-stage front-end processing directly on the image plane, detecting FAST corner features and binary edge features at 300 FPS; these sparse features are fused with a 400 Hz IMU in a loosely coupled iterated EKF. Reported absolute trajectory error improvements include trajectory A from 0.215732 m to 0.167631 m, trajectory B from 0.134617 m to 0.12071 m, and trajectory G from 0.132624 m to 0.10535 m (Lisondra et al., 2024). Here focal-plane image analysis is literal pixel-parallel computation on the sensor.

Compressive focal plane array imaging uses a different principle. CalibFPA forms multiple low-resolution multiplexed measurements with a fixed coded aperture moved by a piezo-stage, then applies online deep-learning calibration and plug-and-play ADMM reconstruction. The measurement model is

$f$6

with $f$7 assumed and the relay lens PSF modeled as an Airy disk of radius

$f$8

The online calibration network estimates corrected low-resolution measurements rather than the high-resolution image directly, after which reconstruction uses a pre-trained DnCNN denoiser in a PnP-ADMM framework. Reported gains include about 10.3 dB over the no-coding variant in the ablation study, about 8.4 dB pSNR over the closest competitor in calibration, and complexity reduction from roughly $f$9 to $D_o$0 for the corrected structured formulation (Güngör et al., 2023).

Measurement limits at the focal plane can also arise from the sensor’s microstructure. The study of focal plane array fill factor in digital image correlation synthesized 100%, 50%, and 25% fill-factor cases at $D_o$1 from rigid-body-translation experiments. It shows that fill factor can significantly affect displacement and strain errors, but that the effect depends strongly on speckle design. For high-contrast, smaller-dot speckles with about 3-pixel diameter, the reported increases relative to 100% fill factor are about 13.2% in displacement error and 19.8% in strain error. By contrast, when the dot diameter is about 7 pixels, fill factor has very minor effect on measurement error (Hijazi et al., 2021). This directly counters the simplified view that lower fill factor uniformly degrades all focal-plane measurements.

Taken together, these results define focal plane image analysis as a heterogeneous but coherent field. Its common premise is that the focal plane is not only where optical information arrives, but where physically meaningful inversion, calibration, control, and even computation can be anchored.