---
title: 'OmniFHT: Pose-Free 3D Tomographic Imaging'
url: https://www.emergentmind.com/topics/omnifht
type: topic
---

# OmniFHT: Pose-Free 3D Tomographic Imaging

OmniFHT is a pose-free 3D refractive-index reconstruction framework for high-throughput quantitative phase imaging of cells flowing through microfluidic channels. It was introduced for in-flow holographic tomography, where holographically measured projections are acquired while cells naturally rotate in flow, and it combines the Fourier diffraction theorem with implicit neural representations to jointly estimate unknown pose trajectories and volumetric structure under weak-scattering assumptions. The method is designed to support arbitrary cell geometries and multi-axis rotations, and its continuous representation is reported to produce high-fidelity reconstructions with as few as 10 views or only 120 degrees of angular range, enabling in situ, high-throughput tomographic imaging of entire flowing cell populations [2509.04848].

## 1. Problem setting and motivation

OmniFHT is situated in flow cytometry with digital holography, or in-flow holographic tomography. In the reported system, cells travel through a \(200\,\mu\text{m} \times 200\,\mu\text{m}\) microchannel, are illuminated by a \(532\ \text{nm}\) plane wave, and are recorded by a Mach–Zehnder off-axis interferometric microscope using a \(40\times\), NA 0.7 objective and an 80 fps camera. Hydrodynamic shear passively rotates the cells, so multiple views are acquired without mechanical rotation or beam scanning. The target output is a 3D refractive-index distribution that provides label-free morphometric and biophysical information, including cell shape, internal heterogeneity, vacuoles, concavities, and aggregate structure [2509.04848].

The methodological motivation is the pose bottleneck in flow-based tomography. Prior flow holographic tomography methods generally assumed uniform, single-axis rotation and known or recoverable rotation angle per frame, often by identifying a full rotation period and assigning equally spaced angles across that cycle. The reported limitations are explicit: irregularly shaped cells experience asymmetric hydrodynamic torques; cell aggregates rotate in more complex ways than a rigid uniaxial roll; pump pulsations, channel imperfections, blockages, and mechanical instabilities disturb flow; collisions and overtaking events induce abrupt pose changes; many cells leave the field of view before completing a full turn; and occlusions remove frames or angular sectors. The consequence is that many cells must be discarded, biasing analysis toward a subset of the population.

In this context, “pose-free” has a precise meaning. It does not mean that the cell has no pose; it means that the reconstruction does not require pose annotations, known angles, or a fixed rotation model beforehand. Instead, pose estimation and tomographic reconstruction are coupled and solved together.

## 2. Physical model and reconstruction variables

The measurement model begins with off-axis hologram formation. If \(O(x,y)\) and \(R(x,y)\) denote the object and reference waves, the recorded hologram is

$$
I_H(x,y)=|O(x,y)+R(x,y)|^2.
$$

The paper writes the hologram multiplied by the reference as

$$
R I_H = R (|O|^2 + |R|^2) + |R|^2 O + R^2 O^*.
$$

In an off-axis setup, the \(+1\) diffraction order \(|R|^2 O\) is separated in Fourier space, filtered, and propagated back to recover the complex object field \(O\). The recovered field is expressed through amplitude and phase,

$$
\phi_o(x,y)=\arg[O(x,y)], \qquad A_o(x,y)=|O(x,y)|.
$$

OmniFHT uses the Rytov approximation as its weak-scattering model. The complex field perturbation is

$$
\psi(x,y)=\ln\left[\frac{O(x,y)}{O_0(x,y)}\right],
$$

where \(O_0\) is the incident plane wave, taken as unit amplitude. This Rytov perturbation is the quantity used for tomography [2509.04848].

The core forward relation is the Fourier diffraction theorem with unknown pose. For each tracked frame, the pose consists of a rotation \(\mathbf{R}\in SO(3)\) and an in-plane translation \(\mathbf{t}\in \mathbb{R}^2\). If \(\hat\psi(K_x,K_y)\) is the 2D Fourier transform of \(\psi(x,y)\), then

$$
\hat{f}\bigl(\mathbf{R}^{-1}\mathbf{k}\bigr) = 2i\,k_z\,\hat{\psi}(K_x,K_y)\, \exp\bigl(-i\,\mathbf{k}\cdot\mathbf{t}\bigr) \exp\bigl[i\bigl(k_{0z}-k_z\bigr)z_0\bigr],
$$

with

$$
k_x = K_x + k_{0x},\quad k_y = K_y + k_{0y},\quad k_z = \sqrt{k_m^2 - k_x^2 - k_y^2},\quad k_x^2 + k_y^2 \le k_m^2.
$$

Here \(\hat f\) is the Fourier-domain 3D scattering potential, \(\hat\psi\) is the Fourier-domain Rytov perturbation, \(\mathbf{k}=(k_x,k_y,k_z)\) is the scattered wavevector, and \(\mathbf{k}_0=(k_{0x},k_{0y},k_{0z})\) is the incident wavevector. For the experimental setup, illumination is along \(z\), so \(k_{0x}=k_{0y}=0\) and \(k_{0z}=k_m\).

The reconstructed scattering potential is converted to refractive index through

$$
n(\mathbf{r}) = n_m\sqrt{\frac{f(\mathbf{r})}{k_m^2}+1},
$$

where

$$
k_m = \frac{2\pi n_m}{\lambda}.
$$

This establishes the link between the reconstructed 3D scattering potential and the final 3D refractive-index map.

## 3. Joint optimization and the meaning of “pose-free”

OmniFHT formulates reconstruction as a joint estimation problem over the volumetric scattering potential and the set of per-frame poses \(\{\omega_i\}\), with \(\omega_i=(\mathbf R_i,\mathbf t_i)\). The paper casts this as maximum-likelihood estimation of both the object and the poses. The novelty is therefore not merely a better tomographic prior, but the coupling of pose recovery and reconstruction in a single inverse problem [2509.04848].

The 3D scattering potential is represented by an implicit neural representation \(f_\theta\), rather than by a fixed voxel grid. The network maps a 3D Fourier coordinate to the scattering potential value, and the predicted Rytov perturbation for frame \(i\) is

$$
\hat\psi_{\mathrm{pred},i}(K_x,K_y) = \frac{f_\theta\bigl(\mathbf R_i^{-1}\mathbf k\bigr)\,\exp\bigl(i\,\mathbf k \cdot \mathbf t_i\bigr)}{2\,i\,k_z} \exp\bigl[-\,i\bigl(k_{0z} - k_z\bigr)\,z_0\bigr].
$$

The self-supervised data-consistency loss is

$$
\mathcal{L}_\theta \;=\; \sum_{i=1}^N \bigl\lVert \hat\psi_{\mathrm{pred},i} \;-\; \hat\psi_{\mathrm{obs},i}\bigr\rVert_2^2.
$$

The paper states that there is no explicit hand-crafted regularizer in the loss; the regularization comes largely from the compact continuous INR representation itself. Sparse-view and limited-angle robustness are therefore attributed to two elements: the physical Fourier diffraction model and the INR prior, which imposes continuity and smoothness and can interpolate missing spectrum better than voxelwise inversion.

A common misunderstanding is that a pose-free formulation must imply continuous differentiable pose-parameter optimization. OmniFHT does not do this. The paper states that it does not use quaternions, Lie algebra, splines, or explicit trajectory smoothness constraints. Instead, pose is recovered by coarse-to-fine discrete search over \(SO(3)\times\mathbb R^2\). The method is thus pose-free in the sense of requiring no known poses, but it is not a fully continuous differentiable pose-parameter optimization.

## 4. Algorithmic pipeline and implementation

The end-to-end workflow begins with off-axis hologram acquisition of flowing cells under natural rotation. Preprocessing converts raw hologram videos into projection sequences through refocusing, phase unwrapping, denoising, and automated cell tracking or ROI extraction. Reconstruction then begins from a random initialization of cellular poses. Given current pose estimates, the INR is trained by minimizing the data-consistency loss; given the updated volume estimate, each frame pose is refined by search over candidate rotations and translations; and the process alternates between volume update and pose update [2509.04848].

Pose estimation is hierarchical. The initial candidate grid is a uniform grid over \(SO(3)\) with \(30^\circ\) angular intervals and a translation grid over \([-0.5,0.5]^2\) with spacing 0.1. For each candidate \((\mathbf R,\mathbf t)\), the method synthesizes the corresponding predicted perturbation and computes complex cross-correlation between predicted and observed perturbations as a similarity score. The top 8 pose hypotheses are retained, a finer local grid is built around each hypothesis by bisecting rotational and translational resolutions, this refinement is repeated for five iterations, and the highest-scoring pose is selected.

The INR is an MLP with 3 hidden layers and 256 neurons per hidden layer. Its input is a normalized 3D Fourier coordinate \(\mathbf{k}=(k_x,k_y,k_z)\in[-0.5,0.5]^3\). Each scalar coordinate is embedded by positional encoding,

$$
\gamma(k) = \bigl[\sin(\alpha_1 k),\;\cos(\alpha_1 k),\;\sin(\alpha_2 k),\;\cos(\alpha_2 k),\;\dots,\;\sin(\alpha_{16} k),\;\cos(\alpha_{16} k)\bigr],
$$

where

$$
\alpha_m = 2^{m-1}\pi,\qquad m=1,2,\dots,16.
$$

Each axis therefore has a 32-dimensional encoding, and concatenation over the three axes gives a 96-dimensional input to the MLP. The paper states that the MLP outputs a single scalar value corresponding to the scattering potential at that Fourier coordinate, and also notes that, since the model works with complex fields, this point is slightly under-specified in the text.

The reported optimization details are: 200 epochs, SGD optimizer, learning rate 0.01, batch size 4, implementation in PyTorch, training on a single NVIDIA RTX 3090, and a memory footprint of approximately 17 GB. Pose estimation is run every five epochs. For simulation, diffraction patterns were generated by the beam propagation method. Experimental evaluation used SW780 bladder cancer cells, red blood cells, white blood cells, multicellular aggregates, and clinical ascites specimens containing mixed cell populations.

## 5. Experimental evaluation

The main baseline is the standard Rytov-based reconstruction method used in prior flow holographic tomography, which assumes or infers poses by conventional rolling-angle logic and then reconstructs using the Rytov approximation. Evaluation uses absolute error, line profiles, and cross-correlation in simulation, and Fourier shell correlation as a resolution metric: in simulation, FSC against ground truth with the \(1/2\) criterion; in experiments, half-set FSC with the \(1/7\) criterion; and, for sparse-view and limited-view comparisons to the full-view reconstruction, FSC with a 0.5 threshold as an approximate resolution indicator [2509.04848].

Under simulated multi-axis rotation, OmniFHT reduced the average absolute difference at the central slice \(z=0~\mu\text{m}\) from 0.0088 to 0.0031, a 64.77% reduction, and improved cross-correlation from 0.469 to 0.908, a 93.60% improvement. FSC resolution improved from \(2.78\ \mu\text{m}\) for the baseline to \(1.59\ \mu\text{m}\) for OmniFHT, which the paper describes as about a \(1.75\times\) resolution improvement. The simulation also showed recovery of the correct complex rotational trajectory, including sinusoidal pitch and yaw changes.

For experimental data, the reported representative FSC results are as follows:

| Setting | Baseline FSC | OmniFHT FSC |
|---|---:|---:|
| Simulated multi-axis rotation | \(2.78\ \mu\text{m}\) | \(1.59\ \mu\text{m}\) |
| SW780 cell, 123 frames | \(1.92\ \mu\text{m}\) | \(1.38\ \mu\text{m}\) |
| RBC, 106 projections | \(1.61\ \mu\text{m}\) | \(1.42\ \mu\text{m}\) |
| Two-cell SW780 aggregate, 116 frames | \(6.41\ \mu\text{m}\) | \(3.21\ \mu\text{m}\) |

The qualitative reconstructions are also specific. For a vacuolated SW780 cell with mostly single-axis motion, OmniFHT reconstructed the vacuoles clearly. For an RBC with pronounced multi-axis rotation, OmniFHT recovered a triangularly skewed biconcave elliptical morphology, whereas the baseline reconstructed an almost spherical object with artifacts; the baseline FSC was especially poor in low frequencies, consistent with pose-estimation artifacts. For a two-cell SW780 aggregate, the baseline failed due to incorrect pose estimation, whereas OmniFHT resolved cell boundaries and vacuoles. The paper also shows successful reconstructions for 3-cell and 4-cell aggregates, though not all have explicit numerical tables.

Sparse-view and limited-angle tests are central to the method’s claims. Using a 220-view SW780 reference dataset over full \(360^\circ\), subsampling to 20, 15, 10, and 5 views produced FSC-based approximate resolutions of \(1.37\), \(1.44\), \(1.89\), and \(3.14\ \mu\text{m}\), respectively. The reported interpretation is that with 10 views key vacuoles remained distinguishable, while with 5 views finer details blurred but boundary and protrusion remained recognizable. Restricting angular coverage to \(180^\circ\), \(150^\circ\), \(120^\circ\), and \(90^\circ\) yielded FSC resolutions of \(1.43\), \(1.48\), \(1.55\), and \(1.87\ \mu\text{m}\), respectively. The headline claim is that performance remains strong with only \(120^\circ\) angular range, corresponding to one-third of Fourier content missing.

A practically significant demonstration involved a clinical ascites video. After excluding out-of-focus cells and cells entering too late, 21 cells were retained: 1 ovarian cancer cell, 5 RBCs comprising 1 echinocyte and 4 discocytes, and 15 WBCs comprising 14 neutrophils and 1 lymphocyte. OmniFHT reconstructed all retained cells despite diverse shapes and rotation behaviors.

## 6. Assumptions, limitations, and significance

The principal physics assumption is the weak-scattering or Rytov approximation. OmniFHT also assumes static cell structure during acquisition, plane-wave illumination with known wavevector geometry, sufficient signal quality and tracking, and a rigid per-frame pose model consisting of rigid rotation plus in-plane translation. No non-rigid deformation model is included [2509.04848].

The limitations are explicit. Pose estimation may lack robustness under extremely limited-angle or otherwise highly challenging conditions. Cells near microfluidic channel edges suffer from poorer phase recovery due to optical distortions and lower SNR and are not yet handled reliably. Strong multiple scattering is not modeled, which may limit fidelity for thicker or higher-contrast specimens. Because the problem jointly optimizes unknown poses and unknown structure, there is inherent nonconvexity, and no formal identifiability or convergence guarantee is given. Imaging throughput is high, but inference remains an optimization pipeline requiring GPU computation, and real-time integration with microfluidics is described as future work. The paper also notes that some implementation details remain terse, including the exact output parameterization for the complex-valued Fourier scattering potential and the exact runtime per sample.

Several interpretive points follow directly from these constraints. First, “pose-free” should not be confused with “assumption-free”: the framework still depends on weak scattering, rigid motion, and accurate complex-field recovery. Second, its reported improvement over prior flow holographic tomography methods lies specifically in removing the single-axis pose-assumption bottleneck through joint pose and volume estimation. Third, the population-level significance is not only better per-cell reconstruction, but broader usable sample coverage. A plausible implication is that this reduction in sample rejection can improve the statistical representativeness of label-free flow cytometry analyses.

Relative to previous pose-aware or fixed-rotation methods, the reported novelties are: no requirement for known poses or a single-axis rotation assumption; joint estimation of pose trajectory and 3D refractive-index structure; use of a physics-informed INR in Fourier space rather than standard voxelized inversion; effectiveness with sparse views and restricted angular ranges; and reconstruction of entire heterogeneous flowing populations rather than only preselected compliant cells. In the paper’s own summary, OmniFHT is a pose-free, physics-informed, INR-based reconstruction framework for in-flow holographic tomography that addresses unknown, arbitrary, multi-axis rotation and broadens population-scale, label-free 3D refractive-index imaging.

Source: https://www.emergentmind.com/topics/omnifht