PPMStereo: Multifocal Stereoscopic Projection Mapping

• PPMStereo is a multifocal stereoscopic projection mapping system that synchronizes ETL modulation and shutter timing to deliver accurate depth cues on non-planar and moving surfaces.
• It integrates high-speed projection with electrically focus-tunable lenses to create virtual images at targeted depths, thereby mitigating the vergence–accommodation conflict.
• The system features precise timing, optical calibration, and a dual-pass rendering pipeline that together enhance depth realism and reduce viewer discomfort.

Searching arXiv for the cited PPMStereo-related papers to ground the article. PPMStereo most commonly denotes a multifocal stereoscopic projection mapping system introduced in “Multifocal Stereoscopic Projection Mapping” (Kimura et al., 2021). It couples high-speed projection with electrically focus-tunable lenses (ETLs) mounted on active shutter glasses to provide correct focus cues for stereoscopic projection mapping on arbitrary, non-planar, and even moving surfaces. The system is designed to mitigate the vergence–accommodation conflict (VAC) that arises when conventional stereoscopic projection mapping supplies binocular disparity but not the accommodative stimulus required for correct depth perception. In the cited work, PPMStereo achieves this by sweeping near-eye optical power through multiple focal states and gating projection and eye shutters at precise instants so that the virtual image of projected computer-generated content lies at the desired depth (Kimura et al., 2021).

1. Problem setting and conceptual basis

Stereoscopic projection mapping renders left- and right-eye views with binocular disparities and time-sequentially projects them onto physical surfaces; observers view the scene through active shutter glasses. This produces binocular 3D cues, but conventional stereoscopic projection mapping does not deliver correct focus cues. Vergence is driven by disparity, whereas accommodation is driven by the optical distance at which the eye must focus; the mismatch causes VAC, leading to discomfort, fatigue, and reduced depth realism (Kimura et al., 2021).

The problem is exacerbated when projected content spans a wide range of distances or is mapped onto non-planar or moving surfaces. In such cases, the accommodative stimulus remains tied to the physical surface depth or to fixed display optics, rather than to the intended virtual depth of the rendered object. PPMStereo addresses this by placing the virtual image of the projected content at the distance at which the eye is intended to accommodate, thereby matching vergence and accommodation (Kimura et al., 2021).

The optical basis is the ETL-formed virtual image of the projector-illuminated surface. With ETL–surface distance $s=d_p$ and virtual image distance $s' = d_v$, the thin-lens relation in diopters is

$v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$

where $v$ is the ETL optical power. Magnification along the ETL axis follows

$m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$

Because the eye is behind the ETL, changes in $d_v$ induce “lens breathing,” that is, apparent size variation on the retina during focal modulation (Kimura et al., 2021).

2. Optical and hardware architecture

The near-eye subsystem uses electrically focus-tunable lenses, specifically Optotune EL-16-40-TC lenses with optical aperture 16 mm and optical power range $-10$ D to $+10$ D. These polymer-membrane fluid lenses are actuated electromagnetically. In the prototype, each ETL is driven by a sinusoidal current at 60 Hz with offset 2.5 mV and amplitude 28 mV (Kimura et al., 2021).

Binocular separation is implemented with off-the-shelf liquid-crystal shutter glasses from RV-3DGDLP1, specified at field rate 96–144 Hz. The measured switching delays are open→closed 0.1 ms and closed→open 3.0 ms. The combined ETL-plus-shutter eyewear is integrated in a 3D-printed frame measuring 69×128×67 mm and weighing 200 g (Kimura et al., 2021).

Projection is provided by an Inrevium TB-UK-DYNAFLASH high-speed DLP-based engine with resolution 1024×768 and 8-bit grayscale output at up to 2000 fps. The measured trigger-to-photon latency is $\delta t_p = 0.15$ ms and is reported as stable across time; image transfer latency of approximately 3 ms is hidden via a multithreaded pipeline. The projector provides 330 ANSI lumen output and supports precise external trigger gating (Kimura et al., 2021).

Control and synchronization are handled by a workstation with an Intel Xeon E3-1225 v5 CPU and an NVIDIA Quadro K620 GPU. A NI USB-6211 D/A outputs the sinusoid, and a custom LM675T op-amp current driver supplies ETL current. An Arduino Uno receives the global trigger at $\phi=0$ and emits time-shifted projector and LC control signals using calibrated delays and a measured ETL phase-to-optical-power waveform $s' = d_v$0 (Kimura et al., 2021).

For scene geometry, static surface shape and pose are measured offline, while moving targets are tracked in real time with an OptiTrack setup using five Prime 17W cameras. In the moving-surface demonstration, this configuration tracks the 6-DoF pose of a hand-held planar target and updates the mapping per frame (Kimura et al., 2021).

3. Multifocal sweep, gating, and synchronization

PPMStereo drives each ETL with a sinusoidal current $s' = d_v$1 at 60 Hz. The ETL behaves approximately as an LTI system, and its output power $s' = d_v$2 is calibrated empirically over one cycle because the waveform is a non-ideal sinusoid. For a rendered fragment with desired virtual image distance $s' = d_v$3, projection is emitted only when $s' = d_v$4 is closest to $s' = d_v$5 (Kimura et al., 2021).

The system samples optical powers $s' = d_v$6 within the sweep interval $s' = d_v$7 and selects

$s' = d_v$8

Equivalently, gating is scheduled at phase offsets $s' = d_v$9 such that $v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$0. Emitting imagery only in short windows around these phases makes the virtual image appear at the desired depth (Kimura et al., 2021).

The timing schedule assigns the upward half-cycle, from $v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$1 to $v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$2, to the left eye and the downward half-cycle, from $v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$3 to $v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$4, to the right eye. LC shutters open for the corresponding half-cycle, and transitions are scheduled near waveform extrema, where $v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$5, in order to minimize blur during LC changeover (Kimura et al., 2021).

To cover target virtual-image extents around a surface at distance $v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$6, the sweep boundaries are defined by

$v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$7

The required sweep bandwidth increases as $v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$8 decreases. An example reported in the prototype uses a nearest surface at 0.5 m and target depths spanning $v(d_p,d_v)=\frac{1}{d_p}-\frac{1}{d_v},$9 m to $v$0, which yields $v$1 D (Kimura et al., 2021).

Synchronization explicitly compensates projector and LC-shutter delays. The microcontroller pre-triggers the projector by $v$2 in phase before the target $v$3, and the measured shutter delays $v$4 ms and $v$5 ms are incorporated into control timing. White illumination frames are injected only when $v$6 D so that real physical surfaces remain visible without focus modulation (Kimura et al., 2021).

4. Rendering pipeline for arbitrary surfaces and motion

For static scenes, PPMStereo uses a two-pass projection-mapping pipeline. First, off-screen rendering from the observer viewpoint produces the desired appearance and a depth map of the CG object and the physical surfaces. Second, the textured surfaces are reprojected to the projector viewpoint and rendered as the final projector image for each focal window (Kimura et al., 2021).

This pipeline operates on arbitrary mesh surfaces. The depth map of surfaces from the observer viewpoint is used to assign focal samples $v$7 per pixel via the nearest-sample rule, and a depth-filtering procedure distributes radiance across adjacent focal windows to preserve continuity (Kimura et al., 2021).

Because the projector can display only a finite number $v$8 of focal windows per sweep, ETL powers are sampled at intervals in diopters. To avoid stepwise discontinuities when the required optical power falls between adjacent samples, radiance is split proportionally in diopters across the two nearest focal windows. The paper recommends a sampling interval of at most 0.6 D for continuous focus cues; in practice, $v$9 was 3–5 planes depending on the scene and experiment (Kimura et al., 2021).

For moving targets, OptiTrack motion capture updates the planar target’s pose each frame before emission. The reported prototype required no additional prediction to preserve apparent size and position, and the high-speed projector with low-latency synchronization maintained registration during motion (Kimura et al., 2021). A plausible implication is that the architecture depends as much on timing determinism as on geometric calibration.

Per-eye imagery is rendered with standard stereoscopic projection mapping using observer-tracked viewpoints. Left- and right-eye off-screen renders provide binocular disparities, while the second pass generates projector-space imagery with spatial masks matched to the tracked surface (Kimura et al., 2021).

5. Lens-breathing compensation and image continuity

A technical issue unique to this configuration is “lens breathing.” As ETL focal power changes, the apparent retinal size of a projected region changes because the eye is positioned behind the ETL. When adjacent regions are rendered at different ETL powers, they can overlap or leave gaps on the retina, degrading texture continuity and geometric coherence (Kimura et al., 2021).

PPMStereo compensates for this by applying a uniform scale per subdivided region so that the virtual image preserves the visual angles of region boundaries as seen without the ETL. With ETL–surface distance $m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$0, ETL–eye distance $m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$1, and virtual image distance $m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$2 for the region, the scale factor is expressed as

$m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$3

This is intended to keep $m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$4 over the region, thereby removing breathing at boundaries (Kimura et al., 2021).

The implementation avoids explicit image resampling. Instead, it modifies the virtual projector’s field of view in the second rendering pass. If the far-edge visual angle is

$m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$5

the compensated angle $m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$6 is computed analogously from $m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$7, and the virtual projector’s FOV is multiplied by $m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$8 when emitting the region’s frame (Kimura et al., 2021).

Validation is reported using a checkerboard plane split across ETL powers $m=-\frac{s'}{s}=-\frac{d_v}{d_p}.$9 D. In that experiment, compensation eliminated overlaps and gaps and significantly improved texture continuity (Kimura et al., 2021).

6. Empirical evaluation and measured performance

The reported ETL sweep frequency is 60 Hz. Prototype sweep ranges include $d_v$0 D for static and multifocal demonstrations and $d_v$1 D for the user study. The projector operates at up to 2000 fps grayscale. The LC shutters close in 0.1 ms and open in 3.0 ms. Registration during motion preserved consistent apparent size and position of content across tracked pose changes, and all participants reported no flicker (Kimura et al., 2021).

The principal perceptual evaluation is a depth-matching study with 10 participants with normal or corrected vision. The virtual target was a 2.4° checker pattern projected on a screen at 2.5 m, with five target distances of 300, 364, 400, 444, and 500 mm from the ETLs. A physical pointer moved along the ETL axis was used to match perceived depth. The experiment was conducted in a dark room, with the LC-plus-ETL glasses mounted on a chin rest and interpupillary distance adjusted per participant (Kimura et al., 2021).

Two conditions were compared. In the proposed multifocal condition, ETL optical power was swept in $d_v$2 D and targets were gated at $d_v$3, $d_v$4, and $d_v$5 D, with intermediate distances produced by depth filtering. In the conventional condition, the ETL was fixed at 0 D for all targets and no focus-cue modulation was provided (Kimura et al., 2021).

The reported analyses show that regression slopes did not differ significantly between conditions, with $d_v$6, $d_v$7, whereas intercepts differed, with $d_v$8, $d_v$9. Mean error, with positive values indicating overestimation, was $-10$0 mm for conventional stereoscopic projection mapping and $-10$1 mm for the proposed method. A two-way ANOVA found a main effect of condition, $-10$2, $-10$3, but no main effect of target distance, $-10$4, $-10$5, and no interaction, $-10$6, $-10$7; Tukey HSD showed a significant pairwise difference, $-10$8 (Kimura et al., 2021).

These results support the paper’s conclusion that VAC in stereoscopic projection mapping caused depth overestimation and that multifocal PPMStereo significantly reduced bias and improved accuracy (Kimura et al., 2021).

7. Limitations, practical considerations, and nomenclature

Several limitations are identified. Because the ETLs move continuously, optical power changes within each emission window, so observers perceive a temporal integration that includes some undesired states. This suggests that waveform shaping to dwell longer at target powers could improve image quality. Increasing the number of focal planes reduces per-plane brightness and depth filtering divides energy further. Projector depth of field can also become a bottleneck: non-planar surfaces may fall outside the projector DOF even if the virtual image depth is correct. The paper proposes that a projector-side ETL with coordinated focal sweep could extend projector DOF (Kimura et al., 2021).

Additional practical constraints concern ETL aberrations, viewing angle, ambient lighting, and scalability. The ETL aperture limits field of view; current ETLs reduce angle of viewing; projection-mapping contrast degrades in bright environments; and multi-user support would require close viewpoints or user-specific projection and synchronization. The prototype used grayscale projection, though the text notes that full-color high-speed projectors are emerging (Kimura et al., 2021).

A nomenclatural complication is that “PPMStereo” has later been reused for unrelated methods. “PPMStereo: Pick-and-Play Memory Construction for Consistent Dynamic Stereo Matching” denotes a memory-based method for temporally consistent disparity estimation from stereo video (Wang et al., 23 Oct 2025). The term is also used in a derived sense for “Polarimetric PatchMatch Multi-View Stereo,” where PolarPMS is denoted “PPMStereo for brevity” in explanatory material (Zhao et al., 2023). By contrast, the 2021 usage refers specifically to multifocal stereoscopic projection mapping with ETLs and active shutter glasses (Kimura et al., 2021). A common misconception is therefore to treat PPMStereo as a uniquely identifying acronym across subfields; the available literature shows that it is not.

Within the projection-mapping context, PPMStereo occupies a specific position: it combines optics, synchronization, and computational projection mapping to present virtual images at correct depth while maintaining registration on arbitrary physical surfaces, including non-planar and moving targets (Kimura et al., 2021).