Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
162 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Assessing SATNet's Ability to Solve the Symbol Grounding Problem (2312.11522v1)

Published 13 Dec 2023 in cs.AI and cs.LG

Abstract: SATNet is an award-winning MAXSAT solver that can be used to infer logical rules and integrated as a differentiable layer in a deep neural network. It had been shown to solve Sudoku puzzles visually from examples of puzzle digit images, and was heralded as an impressive achievement towards the longstanding AI goal of combining pattern recognition with logical reasoning. In this paper, we clarify SATNet's capabilities by showing that in the absence of intermediate labels that identify individual Sudoku digit images with their logical representations, SATNet completely fails at visual Sudoku (0% test accuracy). More generally, the failure can be pinpointed to its inability to learn to assign symbols to perceptual phenomena, also known as the symbol grounding problem, which has long been thought to be a prerequisite for intelligent agents to perform real-world logical reasoning. We propose an MNIST based test as an easy instance of the symbol grounding problem that can serve as a sanity check for differentiable symbolic solvers in general. Naive applications of SATNet on this test lead to performance worse than that of models without logical reasoning capabilities. We report on the causes of SATNet's failure and how to prevent them.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (51)
  1. Satnet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. arXiv preprint arXiv:1905.12149, 2019.
  2. Stevan Harnad. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1-3):335–346, 1990.
  3. Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3):211–252, 2015.
  4. The microsoft 2017 conversational speech recognition system [technical report]. Technical Report MSR-TR-2017-39, Microsoft, August 2017. URL https://www.microsoft.com/en-us/research/publication/microsoft-2017-conversational-speech-recognition-system/.
  5. Deep speech 2: End-to-end speech recognition in english and mandarin. In International conference on machine learning, pages 173–182, 2016.
  6. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484, 2016.
  7. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815, 2017.
  8. Agent57: Outperforming the atari human benchmark. arXiv preprint arXiv:2003.13350, 2020.
  9. First return then explore. arXiv preprint arXiv:2004.12919, 2020.
  10. Gary Marcus. The next decade in ai: four steps towards robust artificial intelligence. arXiv preprint arXiv:2002.06177, 2020.
  11. François Chollet. The measure of intelligence. arXiv preprint arXiv:1911.01547, 2019.
  12. Jonathan St BT Evans. Heuristic and analytic processes in reasoning. British Journal of Psychology, 75(4):451–468, 1984.
  13. Daniel Kahneman. Thinking, fast and slow. Macmillan, 2011.
  14. Yoshua Bengio. From system 1 deep learning to system 2 deep learning, 2019. URL https://slideslive.com/38922304/from-system-1-deep-learning-to-system-2-deep-learning. Conference on Neural Information Processing Systems.
  15. Solving rubik’s cube with a robot hand. arXiv preprint arXiv:1910.07113, 2019.
  16. Harnessing deep neural networks with logic rules. arXiv preprint arXiv:1603.06318, 2016.
  17. End-to-end differentiable proving. In Advances in Neural Information Processing Systems, pages 3788–3800, 2017.
  18. Deeplogic: Towards end-to-end differentiable logical reasoning. arXiv preprint arXiv:1805.07433, 2018.
  19. Learning explanatory rules from noisy data. Journal of Artificial Intelligence Research, 61:1–64, 2018.
  20. Logic tensor networks: Deep learning and logical reasoning from data and knowledge. arXiv preprint arXiv:1606.04422, 2016.
  21. Lifted relational neural networks. arXiv preprint arXiv:1508.05128, 2015.
  22. Analyzing differentiable fuzzy logic operators. arXiv preprint arXiv:2002.06100, 2020.
  23. Differentiation of blackbox combinatorial solvers. arXiv preprint arXiv:1912.02175, 2019.
  24. Deep graph matching via blackbox differentiation of combinatorial solvers. arXiv preprint arXiv:2003.11657, 2020.
  25. Differentiable submodular maximization. arXiv preprint arXiv:1803.01785, 2018.
  26. Fast differentiable sorting and ranking. arXiv preprint arXiv:2002.08871, 2020.
  27. Optnet: Differentiable optimization as a layer in neural networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 136–145. JMLR. org, 2017.
  28. A simple neural network module for relational reasoning. In Advances in neural information processing systems, pages 4967–4976, 2017.
  29. Recurrent relational networks. In Advances in Neural Information Processing Systems, pages 3368–3378, 2018.
  30. Learning a sat solver from single-bit supervision. arXiv preprint arXiv:1802.03685, 2018.
  31. Deepproblog: Neural probabilistic logic programming. In Advances in Neural Information Processing Systems, pages 3749–3759, 2018.
  32. Differentiable learning of logical rules for knowledge base reasoning. In Advances in Neural Information Processing Systems, pages 2319–2328, 2017.
  33. Turning 30: New ideas in inductive logic programming. arXiv preprint arXiv:2002.11002, 2020.
  34. William W Cohen. Tensorlog: A differentiable deductive database. arXiv preprint arXiv:1605.06523, 2016.
  35. Raven: A dataset for relational and analogical visual reasoning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5317–5327, 2019.
  36. Measuring abstract reasoning in neural networks. arXiv preprint arXiv:1807.04225, 2018.
  37. Hierarchical rule induction network for abstract visual reasoning. arXiv preprint arXiv:2002.06838, 2020.
  38. Peter Norvig. Solving every sudoku puzzle. http://norvig.com/sudoku.html, 2006.
  39. Kyubyong Park. Can convolutional neural networks crack sudoku puzzles? https://github.com/Kyubyong/sudoku, 2018.
  40. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
  41. Vikraman Arvind and NV Vinodchandran. The complexity of exactly learning algebraic concepts. In International Workshop on Algorithmic Learning Theory, pages 100–112. Springer, 1996.
  42. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
  43. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  44. Monya Baker. Reproducibility crisis? Nature, 533(26):353–66, 2016.
  45. John PA Ioannidis. Why most published research findings are false. PLos med, 2(8):e124, 2005.
  46. Daniele Fanelli. Opinion: Is science really facing a reproducibility crisis, and do we need it to? Proceedings of the National Academy of Sciences, 115(11):2628–2631, 2018.
  47. Reproducibility in machine learning for health. arXiv preprint arXiv:1907.01463, 2019.
  48. Towards reproducible neural architecture and hyperparameter search. Reproducibility in Machine Learning Workshop, 2018.
  49. A hitchhiker’s guide to statistical comparisons of reinforcement learning algorithms. arXiv preprint arXiv:1904.06979, 2019.
  50. A systems perspective to reproducibility in production machine learning domain. Reproducibility in Machine Learning Workshop, 2018.
  51. user265554 (https://puzzling.stackexchange.com/users/16477/user265554). Complete the sequence. Puzzling Stack Exchange, 2015. URL https://puzzling.stackexchange.com/questions/22495/complete-the-sequence. URL:https://puzzling.stackexchange.com/questions/22495/complete-the-sequence (version: 2020-05-08).
Citations (14)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now