Papers
Topics
Authors
Recent
Search
2000 character limit reached

Thermal Chameleon: Task-Adaptive Tone-mapping for Radiometric Thermal-Infrared images

Published 24 Oct 2024 in cs.RO and cs.CV | (2410.18340v1)

Abstract: Thermal Infrared (TIR) imaging provides robust perception for navigating in challenging outdoor environments but faces issues with poor texture and low image contrast due to its 14/16-bit format. Conventional methods utilize various tone-mapping methods to enhance contrast and photometric consistency of TIR images, however, the choice of tone-mapping is largely dependent on knowing the task and temperature dependent priors to work well. In this paper, we present Thermal Chameleon Network (TCNet), a task-adaptive tone-mapping approach for RAW 14-bit TIR images. Given the same image, TCNet tone-maps different representations of TIR images tailored for each specific task, eliminating the heuristic image rescaling preprocessing and reliance on the extensive prior knowledge of the scene temperature or task-specific characteristics. TCNet exhibits improved generalization performance across object detection and monocular depth estimation, with minimal computational overhead and modular integration to existing architectures for various tasks. Project Page: https://github.com/donkeymouse/ThermalChameleon

Definition Search Book Streamline Icon: https://streamlinehq.com
References (24)
  1. Y.-S. Shin and A. Kim, “Sparse depth enhanced direct thermal-infrared slam beyond the visible spectrum,” IEEE Robot. and Automat. Lett., vol. 4, no. 3, pp. 2918–2925, 2019.
  2. U. Shin, K. Park, B.-U. Lee, K. Lee, and I. S. Kweon, “Self-supervised monocular depth estimation from thermal images via adversarial multi-spectral adaptation,” in Proc. IEEE Winter Conf. on Applications of Comput. Vision., 2023, pp. 5798–5807.
  3. F. Bao, X. Wang, S. H. Sureshbabu, G. Sreekumar, L. Yang, V. Aggarwal, V. N. Boddeti, and Z. Jacob, “Heat-assisted detection and ranging,” Nature, vol. 619, no. 7971, pp. 743–748, 2023.
  4. S. Khattak, C. Papachristos, and K. Alexis, “Keyframe-based thermal–inertial odometry,” J. of Field Robot., vol. 37, no. 4, pp. 552–579, 2020.
  5. M. P. Das, L. Matthies, and S. Daftry, “Online photometric calibration of automatic gain thermal infrared cameras,” IEEE Robot. and Automat. Lett., vol. 6, no. 2, pp. 2453–2460, 2021.
  6. H. Gil, M.-H. Jeon, and A. Kim, “Fieldscale: Locality-aware field-based adaptive rescaling for thermal infrared image,” IEEE Robot. and Automat. Lett., 2024.
  7. U. Shin, K. Lee, B.-U. Lee, and I. S. Kweon, “Maximizing self-supervision from thermal image for effective self-supervised learning of depth and ego-motion,” IEEE Robot. and Automat. Lett., vol. 7, no. 3, pp. 7771–7778, 2022.
  8. A. Gödrich, D. König, G. Eilertsen, and M. Teutsch, “Joint tone mapping and denoising of thermal infrared images via multi-scale retinex and multi-task learning,” in Infrared Tech. and Appli. XLIX, vol. 12534.   SPIE, 2023, pp. 275–291.
  9. R. Xu, C. Chen, J. Peng, C. Li, Y. Huang, F. Song, Y. Yan, and Z. Xiong, “Toward raw object detection: A new benchmark and a new model,” in Proc. IEEE Conf. on Comput. Vision and Pattern Recog., 2023, pp. 13 384–13 393.
  10. U. Shin, K. Lee, S. Lee, and I. S. Kweon, “Self-supervised depth and ego-motion estimation for monocular thermal video using multi-spectral consistency loss,” IEEE Robot. and Automat. Lett., vol. 7, no. 2, pp. 1103–1110, 2021.
  11. C. Herrmann, M. Ruf, and J. Beyerer, “Cnn-based thermal infrared person detection by domain adaptation,” in Autonomous Sys.: Sensors, Vehicles, Security, and the IoE, vol. 10643.   SPIE, 2018, pp. 38–43.
  12. M. R. U. Saputra, P. P. De Gusmao, C. X. Lu, Y. Almalioglu, S. Rosa, C. Chen, J. Wahlström, W. Wang, A. Markham, and N. Trigoni, “Deeptio: A deep thermal-inertial odometry with visual hallucination,” IEEE Robot. and Automat. Lett., vol. 5, no. 2, pp. 1672–1679, 2020.
  13. M. M. Gündoğan, T. Aksoy, A. Temizel, and U. Halici, “Ir reasoner: Real-time infrared object detection by visual reasoning,” in Proc. IEEE Conf. on Comput. Vision and Pattern Recog., 2023, pp. 422–430.
  14. J.Veitch-Michaelis, “flirpy documentation,” https://flirpy.readthedocs.io/en/latest/index.html, 2023.
  15. B. Mildenhall, P. Hedman, R. Martin-Brualla, P. P. Srinivasan, and J. T. Barron, “Nerf in the dark: High dynamic range view synthesis from noisy raw images,” in Proc. IEEE Conf. on Comput. Vision and Pattern Recog., 2022, pp. 16 190–16 199.
  16. T. FLIR, “Flir-adas v2,” https://www.flir.com/oem/adas/adas-dataset-form/, 2022.
  17. A. J. Lee, Y. Cho, Y.-s. Shin, A. Kim, and H. Myung, “Vivid++: Vision for visibility dataset,” IEEE Robot. and Automat. Lett., vol. 7, no. 3, pp. 6282–6289, 2022.
  18. S. Yun, M. Jung, J. Kim, S. Jung, Y. Cho, M.-H. Jeon, G. Kim, and A. Kim, “Sthereo: Stereo thermal dataset for research in odometry and mapping,” in Proc. IEEE/RSJ Intl. Conf. on Intell. Robots and Sys.   IEEE, 2022, pp. 3857–3864.
  19. U. Shin, J. Park, and I. S. Kweon, “Deep depth estimation from thermal image,” in Proc. IEEE Conf. on Comput. Vision and Pattern Recog., 2023, pp. 1043–1053.
  20. T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in Proc. IEEE Intl. Conf. on Comput. Vision, 2017, pp. 2980–2988.
  21. Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun, “Yolox: Exceeding yolo series in 2021,” arXiv preprint arXiv:2107.08430, 2021.
  22. P. Sun, R. Zhang, Y. Jiang, T. Kong, C. Xu, W. Zhan, M. Tomizuka, L. Li, Z. Yuan, C. Wang et al., “Sparse r-cnn: End-to-end object detection with learnable proposals,” in Proc. IEEE Conf. on Comput. Vision and Pattern Recog., 2021, pp. 14 454–14 463.
  23. A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in Proc. Intl. Conf. on Machine Learning.   PMLR, 2021, pp. 8748–8763.
  24. R. Girdhar, A. El-Nouby, Z. Liu, M. Singh, K. V. Alwala, A. Joulin, and I. Misra, “Imagebind: One embedding space to bind them all,” in Proc. IEEE Conf. on Comput. Vision and Pattern Recog., 2023, pp. 15 180–15 190.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up