Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
167 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Leveraging Image Augmentation for Object Manipulation: Towards Interpretable Controllability in Object-Centric Learning (2310.08929v4)

Published 13 Oct 2023 in cs.CV

Abstract: The binding problem in artificial neural networks is actively explored with the goal of achieving human-level recognition skills through the comprehension of the world in terms of symbol-like entities. Especially in the field of computer vision, object-centric learning (OCL) is extensively researched to better understand complex scenes by acquiring object representations or slots. While recent studies in OCL have made strides with complex images or videos, the interpretability and interactivity over object representation remain largely uncharted, still holding promise in the field of OCL. In this paper, we introduce a novel method, Slot Attention with Image Augmentation (SlotAug), to explore the possibility of learning interpretable controllability over slots in a self-supervised manner by utilizing an image augmentation strategy. We also devise the concept of sustainability in controllable slots by introducing iterative and reversible controls over slots with two proposed submethods: Auxiliary Identity Manipulation and Slot Consistency Loss. Extensive empirical studies and theoretical validation confirm the effectiveness of our approach, offering a novel capability for interpretable and sustainable control of object representations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Baldi, P. Autoencoders, unsupervised learning, and deep architectures. In Proceedings of ICML workshop on unsupervised and transfer learning, pp.  37–49. JMLR Workshop and Conference Proceedings, 2012.
  2. Invariant slot attention: Object discovery with slot-centric reference frames. arXiv preprint arXiv:2302.04973, 2023.
  3. Causal contraction: spatial binding in the perception of collision events. Psychological Science, 21(1):44–48, 2010.
  4. Monet: Unsupervised scene decomposition and representation. arXiv preprint arXiv:1901.11390, 2019.
  5. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325, 2015.
  6. Unsupervised object discovery and localization in the wild: Part-based matching with bottom-up region proposals. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.  1201–1210, 2015.
  7. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.
  8. Savi++: Towards end-to-end object-centric learning from real-world videos. Conference on Neural Information Processing Systems (NeurIPS), 35:28940–28954, 2022.
  9. Genesis: Generative scene inference and sampling with object-centric latent representations. arXiv preprint arXiv:1907.13052, 2019.
  10. Genesis-v2: Inferring unordered object representations without iterative refinement. Conference on Neural Information Processing Systems (NeurIPS), 34:8085–8094, 2021.
  11. Feldman, J. The neural binding problem (s). Cognitive neurodynamics, 7:1–11, 2013.
  12. The representation and matching of pictorial structures. IEEE Transactions on computers, 100(1):67–92, 1973.
  13. Tagger: Deep unsupervised perceptual grouping. Conference on Neural Information Processing Systems (NeurIPS), 29, 2016.
  14. Multi-object representation learning with iterative variational inference. In International Conference on Machine Learning (ICML), pp.  2424–2433. PMLR, 2019.
  15. On the binding problem in artificial neural networks. arXiv preprint arXiv:2012.05208, 2020.
  16. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.  770–778, 2016.
  17. Masked autoencoders are scalable vision learners. In CVPR, 2022.
  18. Ptr: A benchmark for part-based conceptual, relational, and physical reasoning. NeurIPS, 2021.
  19. Denoising criterion for variational auto-encoding framework. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017.
  20. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.  2901–2910, 2017.
  21. Clevrtex: A texture-rich benchmark for unsupervised multi-object segmentation. NeurIPS, 2021.
  22. Shepherding slots to objects: Towards stable and robust object-centric learning. arXiv preprint arXiv:2303.17842, 2023.
  23. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.
  24. Conditional object-centric learning from video. arXiv preprint arXiv:2111.12594, 2021.
  25. Kuhn, H. W. The hungarian method for the assignment problem. Naval research logistics quarterly, 1955.
  26. Building machines that learn and think like people. Behavioral and brain sciences, 40:e253, 2017.
  27. Learning object-centric representations of multi-object scenes from multiple views. Conference on Neural Information Processing Systems (NeurIPS), 33:5656–5666, 2020.
  28. Object-centric learning with slot attention. Conference on Neural Information Processing Systems (NeurIPS), 33:11525–11538, 2020.
  29. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019.
  30. Adversarial autoencoders. arXiv preprint arXiv:1511.05644, 2015.
  31. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. arXiv preprint arXiv:1904.12584, 2019.
  32. Marr, D. Vision: A computational investigation into the human representation and processing of visual information. MIT press, 2010.
  33. Multi-object datasets. https://github.com/deepmind/multi_object_datasets/, 2019.
  34. Object scene representation transformer. Conference on Neural Information Processing Systems (NeurIPS), 35:9512–9524, 2022a.
  35. Scene representation transformer: Geometry-free novel view synthesis through set-latent scene representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.  6229–6238, 2022b.
  36. Bridging the gap to real-world object-centric learning. arXiv preprint arXiv:2209.14860, 2022.
  37. Illiterate dall-e learns to compose. arXiv preprint arXiv:2110.11405, 2021.
  38. Simple unsupervised object-centric learning for complex and naturalistic videos. Conference on Neural Information Processing Systems (NeurIPS), 35:18181–18196, 2022.
  39. Neural systematic binder. In International Conference on Learning Representations (ICLR), 2023.
  40. Unsupervised discovery and composition of object light fields. arXiv preprint arXiv:2205.03923, 2022.
  41. Core knowledge. Developmental science, 10(1):89–96, 2007.
  42. Treisman, A. The binding problem. Current opinion in neurobiology, 6(2):171–178, 1996.
  43. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
  44. Attention is all you need. Conference on Neural Information Processing Systems (NeurIPS), 30, 2017.
  45. The all-seeing project: Towards panoptic visual recognition and understanding of the open world. arXiv preprint arXiv:2308.01907, 2023a.
  46. Slot-vae: Object-centric scene generation with slot attention. In International Conference on Machine Learning (ICML), 2023b.
  47. Spatial broadcast decoder: A simple architecture for learning disentangled representations in vaes. arXiv preprint arXiv:1901.07017, 2019.
  48. Slotformer: Unsupervised visual dynamics simulation with object-centric models. arXiv preprint arXiv:2210.05861, 2022.
  49. Coat: Measuring object compositionality in emergent representations. In International Conference on Machine Learning (ICML), pp.  24388–24413. PMLR, 2022.

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now