Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detection (2402.17323v2)

Published 27 Feb 2024 in cs.CV

Abstract: In the field of class incremental learning (CIL), generative replay has become increasingly prominent as a method to mitigate the catastrophic forgetting, alongside the continuous improvements in generative models. However, its application in class incremental object detection (CIOD) has been significantly limited, primarily due to the complexities of scenes involving multiple labels. In this paper, we propose a novel approach called stable diffusion deep generative replay (SDDGR) for CIOD. Our method utilizes a diffusion-based generative model with pre-trained text-to-diffusion networks to generate realistic and diverse synthetic images. SDDGR incorporates an iterative refinement strategy to produce high-quality images encompassing old classes. Additionally, we adopt an L2 knowledge distillation technique to improve the retention of prior knowledge in synthetic images. Furthermore, our approach includes pseudo-labeling for old objects within new task images, preventing misclassification as background elements. Extensive experiments on the COCO 2017 dataset demonstrate that SDDGR significantly outperforms existing algorithms, achieving a new state-of-the-art in various CIOD scenarios. The source code will be made available to the public.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (67)
  1. Rodeo: Replay for online object detection. BMVC, 2020.
  2. Rainbow memory: Continual learning with a memory of diverse samples. In CVPR, 2021.
  3. End-to-end object detection with transformers. In ECCV, 2020.
  4. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In ECCV, 2018.
  5. Continual learning with tiny episodic memories. arXiv, 2019.
  6. Gan memory with no forgetting. NeurIPS, 2020.
  7. Continual prototype evolution: Learning online from non-stationary data streams. In ICCV, 2021.
  8. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv, 2018.
  9. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 2021.
  10. Overcoming catastrophic forgetting in incremental object detection via elastic response distillation. In CVPR, 2022.
  11. Ddgr: Continual learning with deep diffusion-based generative replay. ICML, 2023.
  12. Generative adversarial nets. NeurIPS, 2014.
  13. Improved schemes for episodic memory-based lifelong learning. In NIPS, 2020.
  14. Ow-detr: Open-world detection transformer. In CVPR, 2022.
  15. An end-to-end architecture for class-incremental object detection with knowledge distillation. In ICME, 2019.
  16. Exemplar-supported generative reproduction for class incremental learning. In BMVC, 2018.
  17. Deep residual learning for image recognition. In CVPR, 2016.
  18. Classifier-free diffusion guidance. arXiv, 2022.
  19. Denoising diffusion probabilistic models. NeurIPS, 2020.
  20. Composer: Creative and controllable image synthesis with composable conditions. arXiv preprint arXiv:2302.09778, 2023.
  21. Selective experience replay for lifelong learning. In AAAI, 2018.
  22. Auto-encoding variational bayes. ICLR, 2013.
  23. Online continual learning on class incremental blurry task configuration with anytime inference. arXiv, 2021.
  24. Rilod: Near real-time incremental learning for object detection at the edge. In SEC, 2019.
  25. Dn-detr: Accelerate detr training by introducing query denoising. In CVPR, 2022a.
  26. Grounded language-image pre-training. In CVPR, 2022b.
  27. Generalized focal loss: Learning qualified and distributed bounding boxes for dense object detection. NIPS, 2020.
  28. Gligen: Open-set grounded text-to-image generation. In CVPR, 2023.
  29. Learning without forgetting. ECCV, 2016.
  30. Microsoft coco: Common objects in context. In ECCV, 2014.
  31. Incdet: In defense of elastic weight consolidation for incremental object detection. TNNLS, 2020a.
  32. Multi-task incremental learning for object detection. arXiv, 2020b.
  33. Continual detection transformer for incremental object detection. In CVPR, 2023.
  34. Augmented geometric distillation for data-free incremental person reid. In CVPR, 2022.
  35. Nerf: Representing scenes as neural radiance fields for view synthesis. ECCV, 2021.
  36. Conditional generative adversarial nets. arXiv preprint arXiv, 2014.
  37. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. arXiv preprint arXiv:2302.08453, 2023.
  38. Im2text: Describing images using 1 million captioned photographs. NIPS, 2011.
  39. Sid: Incremental learning for anchor-free object detection via selective and inter-related distillation. CVIU, 2021.
  40. Gdumb: A simple approach that questions our progress in continual learning. In ECCV, 2020.
  41. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PMLR, 2021a.
  42. Learning transferable visual models from natural language supervision. In ICML, 2021b.
  43. icarl: Incremental classifier and representation learning. In CVPR, 2017.
  44. Faster r-cnn: Towards real-time object detection with region proposal networks. In NIPS, 2015.
  45. Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 1995.
  46. Experience replay for continual learning. In NIPS, 2019.
  47. High-resolution image synthesis with latent diffusion models. In CVPR, 2022.
  48. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015.
  49. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv, 2021.
  50. Laion-5b: An open large-scale dataset for training next generation image-text models. NeurIPS, 2022.
  51. Objects365: A large-scale, high-quality dataset for object detection. In ICCV, 2019.
  52. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In ACL, pages 2556–2565, 2018.
  53. Continual learning with deep generative replay. In NIPS, 2017a.
  54. Continual learning with deep generative replay. NIPS, 2017b.
  55. Diversity is definitely needed: Improving model-agnostic zero-shot classification via stable diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 769–778, 2023.
  56. Incremental learning of object detectors without catastrophic forgetting. In ICCV, 2017.
  57. On learning the geodesic path for incremental learning. In CVPR, 2021.
  58. Denoising diffusion implicit models. arXiv, 2020a.
  59. Score-based generative modeling through stochastic differential equations. arXiv, 2020b.
  60. Attention is all you need. Advances in neural information processing systems, 30, 2017.
  61. Sketch-guided text-to-image diffusion models. In SIGGRAPH, 2023.
  62. Memory replay gans: Learning to generate new categories without forgetting. NIPS, 31, 2018.
  63. Incremental learning using conditional adversarial networks. In NeurIPS, 2019.
  64. Boosting human-object interaction detection with text-to-image diffusion model. arXiv preprint arXiv:2305.12252, 2023a.
  65. Reco: Region-controlled text-to-image generation. In CVPR, 2023b.
  66. Adding conditional control to text-to-image diffusion models. In ICCV, 2023.
  67. Deformable detr: Deformable transformers for end-to-end object detection. ICLR, 2020.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Junsu Kim (16 papers)
  2. Hoseong Cho (4 papers)
  3. Jihyeon Kim (7 papers)
  4. Yihalem Yimolal Tiruneh (3 papers)
  5. Seungryul Baek (32 papers)
Citations (6)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
X Twitter Logo Streamline Icon: https://streamlinehq.com

Tweets