Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
129 tokens/sec
GPT-4o
28 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Stochastic Light Field Holography (2307.06277v1)

Published 12 Jul 2023 in cs.CV, cs.GR, eess.IV, and physics.optics

Abstract: The Visual Turing Test is the ultimate goal to evaluate the realism of holographic displays. Previous studies have focused on addressing challenges such as limited \'etendue and image quality over a large focal volume, but they have not investigated the effect of pupil sampling on the viewing experience in full 3D holograms. In this work, we tackle this problem with a novel hologram generation algorithm motivated by matching the projection operators of incoherent Light Field and coherent Wigner Function light transport. To this end, we supervise hologram computation using synthesized photographs, which are rendered on-the-fly using Light Field refocusing from stochastically sampled pupil states during optimization. The proposed method produces holograms with correct parallax and focus cues, which are important for passing the Visual Turing Test. We validate that our approach compares favorably to state-of-the-art CGH algorithms that use Light Field and Focal Stack supervision. Our experiments demonstrate that our algorithm significantly improves the realism of the viewing experience for a variety of different pupil states.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (68)
  1. M. Walton, P. Hao, M. Vermeulen, F. Willomitzer, and O. Cossairt, “Characterizing the immaterial. noninvasive imaging and analysis of stephen benton’s hologram engine no. 9,” arXiv, 2021.
  2. F. Zhong, A. Jindal, Ö. Yöntem, P. Hanji, S. Watt, and R. Mantiuk, “Reproducing reality with a high-dynamic-range multi-focal stereo display,” ACM Transactions on Graphics, vol. 40, no. 6, 2021.
  3. G. Kuo, L. Waller, R. Ng, and A. Maimone, “High resolution étendue expansion for holographic displays,” ACM Transactions on Graphics (TOG), vol. 39, no. 4, p. 66, 2020.
  4. S.-H. Baek, E. Tseng, A. Maimone, N. Matsuda, G. Kuo, Q. Fu, W. Heidrich, D. Lanman, and F. Heide, “Neural etendue expander for ultra-wide-angle high-fidelity holographic display,” arXiv, 2021.
  5. S. Monin, A. C. Sankaranarayanan, and A. Levin, “Exponentially-wide étendue displays using a tilting cascade,” in ICCP, 2022.
  6. Y. Jo, D. Yoo, D. Lee, M. Kim, and B. Lee, “Multi-illumination 3d holographic display using a binary mask,” Optics Letters, vol. 47, no. 10, pp. 2482–2485, 2022.
  7. S. Monin, A. C. Sankaranarayanan, and A. Levin, “Analyzing phase masks for wide étendue holographic displays,” in 2022 IEEE International Conference on Computational Photography (ICCP).   IEEE, 2022, pp. 1–12.
  8. P. Chakravarthula, S.-H. Baek, E. Tseng, A. Maimone, G. Kuo, F. Schiffers, N. Matsuda, O. Cossairt, D. Lanman, and F. Heide, “Pupil-aware holography,” arXiv preprint arXiv:2203.14939, 2022.
  9. R. Ng, M. Levoy, M. Brédif, G. Duval, M. Horowitz, and P. Hanrahan, “Light field photography with a hand-held plenoptic camera,” Ph.D. dissertation, Stanford University, 2005.
  10. S. Choi, M. Gopakumar, Y. Peng, J. Kim, and G. Wetzstein, “Neural 3d holography: Learning accurate wave propagation models for 3d holographic virtual and augmented reality displays,” ACM Transactions on Graphics (TOG), vol. 40, no. 6, pp. 1–12, 2021.
  11. K. Kavaklı, Y. Itoh, H. Urey, and K. Akşit, “Realistic defocus blur for multiplane computer-generated holography,” arXiv preprint arXiv:2205.07030, 2022.
  12. R. Ng, “Fourier slice photography,” Arxiv, pp. 735–744, 2005.
  13. L. Lesem, P. Hirsch, and J. Jordan, “The kinoform: a new wavefront reconstruction device,” IBM Journal of Research and Development, vol. 13, no. 2, pp. 150–155, 1969.
  14. R. W. Gerchberg, “A practical algorithm for the determination of the phase from image and diffraction plane pictures,” Optik, vol. 35, pp. 237–246, 1972.
  15. J. R. Fienup, “Phase retrieval algorithms: a comparison,” Applied optics, vol. 21, no. 15, pp. 2758–2769, 1982.
  16. H. H. Bauschke, P. L. Combettes, and D. R. Luke, “Hybrid projection–reflection method for phase retrieval,” JOSA A, 2003.
  17. R. Lane, “Phase retrieval using conjugate gradient minimization,” journal of Modern Optics, vol. 38, no. 9, pp. 1797–1813, 1991.
  18. Z. Wen, C. Yang, X. Liu, and S. Marchesini, “Alternating direction methods for classical and ptychographic phase retrieval,” Inverse Problems, vol. 28, no. 11, p. 115010, 2012.
  19. S. Marchesini, Y.-C. Tu, and H.-t. Wu, “Alternating projection, ptychographic imaging and phase synchronization,” Applied and Computational Harmonic Analysis, vol. 41, no. 3, pp. 815–851, 2016.
  20. J. Zhang, N. Pégard, J. Zhong, H. Adesnik, and L. Waller, “3d computer-generated holography by non-convex optimization,” Optica, vol. 4, no. 10, pp. 1306–1313, 2017.
  21. E. J. Candes, T. Strohmer, and V. Voroninski, “Phaselift: Exact and stable signal recovery from magnitude measurements via convex programming,” Communications on Pure and Applied Mathematics, vol. 66, no. 8, pp. 1241–1274, 2013.
  22. T. Goldstein and C. Studer, “Phasemax: Convex phase retrieval via basis pursuit,” IEEE Transactions on Information Theory, 2018.
  23. S. Bahmani and J. Romberg, “Phase retrieval meets statistical learning theory: A flexible convex relaxation,” in Artificial Intelligence and Statistics.   PMLR, 2017, pp. 252–260.
  24. Y. Peng, S. Choi, N. Padmanaban, and G. Wetzstein, “Neural holography with camera-in-the-loop training,” ACM Transactions on Graphics (TOG), vol. 39, no. 6, p. 185, 2020.
  25. P. Chakravarthula, Y. Peng, J. Kollin, H. Fuchs, and F. Heide, “Wirtinger holography for near-eye displays,” ACM Transactions on Graphics (TOG), vol. 38, no. 6, p. 213, 2019.
  26. H. Zhang, Y. Zhao, L. Cao, and G. Jin, “Layered holographic stereogram based on inverse fresnel diffraction,” Applied optics, vol. 55, no. 3, pp. A154–A159, 2016.
  27. Y. Zhao, L. Cao, H. Zhang, D. Kong, and G. Jin, “Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method,” Optics express, vol. 23, no. 20, pp. 25 440–25 449, 2015.
  28. M. H. Eybposh, N. W. Caira, M. Atisa, P. Chakravarthula, and N. C. Pégard, “Deepcgh: 3d computer-generated holography using deep learning,” Optics Express, vol. 28, no. 18, 2020.
  29. M. Makowski, M. Sypek, A. Kolodziejczyk, G. Mikula, and J. Suszek, “Iterative design of multiplane holograms: experiments and applications,” Optical Engineering, vol. 46, no. 4, pp. 1 – 6, 2007.
  30. D. Kim, S.-W. Nam, B. Lee, J.-M. Seo, and B. Lee, “Accommodative holography: improving accommodation response for perceptually realistic holographic displays,” ACM Transactions on Graphics (TOG), vol. 41, no. 4, pp. 1–15, 2022.
  31. D. Yoo, Y. Jo, S.-W. Nam, C. Chen, and B. Lee, “Optimization of computer-generated holograms featuring phase randomness control,” Optics Letters, vol. 46, no. 19, pp. 4769–4772, 2021.
  32. A. Maimone, A. Georgiou, and J. S. Kollin, “Holographic near-eye displays for virtual and augmented reality,” ACM Transactions on Graphics (TOG), vol. 36, no. 4, p. 85, 2017.
  33. L. Xiao, A. Kaplanyan, A. Fix, M. Chapman, and D. Lanman, “Deepfocus: Learned image synthesis for computational display,” in ACM SIGGRAPH 2018 Talks, 2018, pp. 1–2.
  34. H. Zhang, Y. Zhao, L. Cao, and G. Jin, “Fully computed holographic stereogram based algorithm for computer-generated holograms with accurate depth cues,” Optics express, vol. 23, 2015.
  35. M. Lucente and T. A. Galyean, “Rendering interactive holographic images,” in Proceedings of the 22nd annual conference on Computer graphics and interactive techniques.   ACM, 1995, pp. 387–394.
  36. M. Yamaguchi, H. Hoshino, T. Honda, and N. Ohyama, “Phase-added stereogram: calculation of hologram using computer graphics technique,” in Proc. SPIE, vol. 1914, 1993, pp. 25–31.
  37. Q. Y. Smithwick, J. Barabas, D. E. Smalley, and V. M. Bove, “Interactive holographic stereograms with accommodation cues,” in Practical Holography XXIV: Materials and Applications, vol. 7619.   International Society for Optics and Photonics, 2010, p. 761903.
  38. D. Lanman and D. Luebke, “Near-eye light field displays,” ACM Transactions on Graphics (TOG), vol. 32, no. 6, p. 220, 2013.
  39. N. Padmanaban, Y. Peng, and G. Wetzstein, “Holographic near-eye displays based on overlap-add stereograms,” ACM Transactions on Graphics (TOG), vol. 38, no. 6, p. 214, 2019.
  40. D. Blinder and P. Schelkens, “Accelerated computer generated holography using sparse bases in the stft domain,” Optics express, vol. 26, no. 2, pp. 1461–1473, 2018.
  41. J.-H. Park and M. Askari, “Non-hogel-based computer generated hologram from light field using complex field recovery technique from wigner distribution function,” Optics express, vol. 27, 2019.
  42. P. Chakravarthula, E. Tseng, T. Srivastava, H. Fuchs, and F. Heide, “Learned hardware-in-the-loop phase retrieval for holographic near-eye displays,” ACM TOG, vol. 39, no. 6, 2020.
  43. L. Shi, F.-C. Huang, W. Lopes, W. Matusik, and D. Luebke, “Near-eye light field holographic rendering with spherical waves for wide field of view interactive 3d computer graphics,” ACM Transactions on Graphics (TOG), vol. 36, no. 6, pp. 1–17, 2017.
  44. K. Min, D. Min, and J.-H. Park, “Wigner inverse transform based computer generated hologram for large object at far field from its perspective light field,” Optics Communications, vol. 532, p. 129229, 2023.
  45. Y. Rivenson, Y. Zhang, H. Günaydın, D. Teng, and A. Ozcan, “Phase recovery and holographic image reconstruction using deep learning in neural networks,” Light: Science & Applications, vol. 7, no. 2, p. 17141, 2018.
  46. R. Horisaki, R. Takagi, and J. Tanida, “Deep-learning-generated holography,” Applied optics, vol. 57, no. 14, pp. 3859–3863, 2018.
  47. L. Shi, B. Li, C. Kim, P. Kellnhofer, and W. Matusik, “Towards real-time photorealistic 3d holography with deep neural networks,” Nature, vol. 591, no. 7849, pp. 234–239, 2021.
  48. S. Choi, M. Gopakumar, Y. Peng, J. Kim, M. O’Toole, and G. Wetzstein, “Time-multiplexed neural holography: a flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators,” in ACM SIGGRAPH 2022 Conference Proceedings, 2022, pp. 1–9.
  49. P. Chakravarthula, E. Tseng, H. Fuchs, and F. Heide, “Hogel-free holography,” ACM Transactions on Graphics (TOG), 2022.
  50. R. Ziegler, S. Bucheli, L. Ahrenberg, M. Magnor, and M. Gross, “A bidirectional light field-hologram transform,” in Computer Graphics Forum, vol. 26.   Wiley Online Library, 2007, pp. 435–446.
  51. Z. Zhang and M. Levoy, “Wigner distributions and how they relate to the light field,” in 2009 IEEE International Conference on Computational Photography (ICCP).   IEEE, 2009, pp. 1–10.
  52. T. Cuypers, R. Horstmeyer, S. B. Oh, P. Bekaert, and R. Raskar, “Validity of wigner distribution function for ray-based imaging,” in IEEE ICCP.   IEEE, 2011, pp. 1–9.
  53. M. J. Bastiaans et al., “Wigner distribution in optics,” 2009.
  54. S. Hamann, L. Shi, O. Solgaard, and G. Wetzstein, “Time-multiplexed light field synthesis via factored wigner distribution function,” Optics letters, vol. 43, no. 3, pp. 599–602, 2018.
  55. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
  56. S. Moser, M. Ritsch-Marte, and G. Thalhammer, “Model-based compensation of pixel crosstalk in liquid crystal spatial light modulators,” Optics express, vol. 27, no. 18, pp. 25 046–25 063, 2019.
  57. P. Schroff, A. La Rooij, E. Haller, and S. Kuhr, “Accurate holographic light potentials using pixel crosstalk modelling,” Scientific Reports, vol. 13, no. 1, p. 3252, 2023.
  58. P. Shedligeri, F. Schiffers, S. Barutcu, P. Ruiz, A. K. Katsaggelos, and O. Cossairt, “Improving acquisition speed of x-ray ptychography through spatial undersampling and regularization,” in ICIP.   IEEE, 2021, pp. 2968–2972.
  59. Y. S. Nashed, T. Peterka, J. Deng, and C. Jacobsen, “Distributed automatic differentiation for ptychography,” Procedia Computer Science, vol. 108, pp. 404–414, 2017.
  60. Z. Xiao, Q. Wang, G. Zhou, and J. Yu, “Aliasing detection and reduction in plenoptic imaging,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 3326–3333.
  61. A. Yu, R. Li, M. Tancik, H. Li, R. Ng, and A. Kanazawa, “Plenoctrees for real-time rendering of neural radiance fields,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 5752–5761.
  62. P. Shedligeri, F. Schiffers, S. Ghosh, O. Cossairt, and K. Mitra, “Selfvi: Self-supervised light-field video reconstruction from stereo video,” in ICCV, 2021, pp. 2491–2501.
  63. C. Jang, K. Bang, M. Chae, B. Lee, and D. Lanman, “Waveguide holography: Towards true 3d holographic glasses,” arXiv preprint arXiv:2211.02784, 2022.
  64. Y. Peng, S. Choi, J. Kim, and G. Wetzstein, “Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration,” Science advances, vol. 7, no. 46, p. eabg5040, 2021.
  65. P. Chakravarthula, Z. Zhang, O. Tursun, P. Didyk, Q. Sun, and H. Fuchs, “Gaze-contingent retinal speckle suppression for perceptually-matched foveated holographic displays,” IEEE Transactions on Visualization and Computer Graphics, vol. 27, no. 11, 2021.
  66. E. Markley, N. Matsuda, F. Schiffers, O. Coissart, and G. Kuo, “Simultaneous color holography,” arXiv:2303.11287, 2023.
  67. K. Kavaklı, L. Shi, H. Ürey, W. Matusik, and K. Akşit, “Holohdr: Multi-color holograms improve dynamic range,” arXiv preprint arXiv:2301.09950, 2023.
  68. X. Xia, Y. Guan, A. State, P. Chakravarthula, T.-J. Cham, and H. Fuchs, “Towards eyeglass-style holographic near-eye displays with statically expanded eyebox,” in ISMAR.   IEEE, 2020.
Citations (3)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now