Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Deciphering Raw Data in Neuro-Symbolic Learning with Provable Guarantees (2308.10487v2)

Published 21 Aug 2023 in cs.AI and cs.LG

Abstract: Neuro-symbolic hybrid systems are promising for integrating machine learning and symbolic reasoning, where perception models are facilitated with information inferred from a symbolic knowledge base through logical reasoning. Despite empirical evidence showing the ability of hybrid systems to learn accurate perception models, the theoretical understanding of learnability is still lacking. Hence, it remains unclear why a hybrid system succeeds for a specific task and when it may fail given a different knowledge base. In this paper, we introduce a novel way of characterising supervision signals from a knowledge base, and establish a criterion for determining the knowledge's efficacy in facilitating successful learning. This, for the first time, allows us to address the two questions above by inspecting the knowledge base under investigation. Our analysis suggests that many knowledge bases satisfy the criterion, thus enabling effective learning, while some fail to satisfy it, indicating potential failures. Comprehensive experiments confirm the utility of our criterion on benchmark tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. Abductive Learning with Ground Knowledge Base. In IJCAI, 1815–1821.
  2. Deep learning for classical japanese literature. arXiv preprint arXiv:1812.01718.
  3. EMNIST: Extending MNIST to handwritten letters. In IJCNN, 2921–2926.
  4. Tensorlog: A probabilistic database implemented using deep-learning infrastructure. Journal of Artificial Intelligence Research, 67: 285–325.
  5. Learning from partial labels. Journal of Machine Learning Research, 12: 1501–1536.
  6. Abductive knowledge induction from raw data. In IJCAI, 1845–1851.
  7. Bridging machine learning and logical reasoning by abductive learning. In NeurIPS, 2811–2822.
  8. Combining logical abduction and statistical induction: Discovering written primitives with human knowledge. In AAAI, 4392–4398.
  9. From statistical relational to neural-symbolic artificial intelligence. In IJCAI, 4943–4950.
  10. Statistical relational artificial intelligence: Logic, probability, and computation. Synthesis lectures on artificial intelligence and machine learning, 10(2): 1–189.
  11. Probabilistic (logic) programming concepts. Machine Learning, 100: 5–47.
  12. Solving the multiple instance problem with axis-parallel rectangles. Artificial intelligence, 89(1-2): 31–71.
  13. Logic tensor networks for semantic image interpretation. In IJCAI.
  14. Neural Logic Machines. In ICLR.
  15. Learning classifiers from only positive and unlabeled data. In KDD, 213–220.
  16. Making sense of raw input. Artificial Intelligence, 299: 103521.
  17. Provably consistent partial-label learning. In NeurIPS, 10948–10960.
  18. Gagniuc, P. A. 2017. Markov chains: from theory to implementation and experimentation. John Wiley & Sons.
  19. Neural-symbolic learning systems: foundations and applications. Springer Science & Business Media.
  20. Abductive reasoning in neural-symbolic systems. Topoi, 26: 37–49.
  21. Differentiable programs with neural libraries. In ICML, 1213–1222.
  22. Introduction to statistical relational learning. MIT press.
  23. Stochastic Optimization of Sorting Networks via Continuous Relaxations. In ICLR.
  24. Deep residual learning for image recognition. In CVPR, 770–778.
  25. Neuro-Symbolic Artificial Intelligence - The State of the Art. Frontiers in Artificial Intelligence and Applications. IOS Press.
  26. Fast abductive learning by similarity-based consistency optimization. In NeurIPS, 26574–26584.
  27. Enabling Knowledge Refinement upon New Concepts in Abductive Learning. In AAAI, 7928–7935.
  28. Semi-supervised abductive learning and its application to theft judicial sentencing. In ICDM, 1070–1075.
  29. Enabling abductive learning to exploit knowledge graph. In IJCAI, 3839–3847.
  30. Hull, J. J. 1994. A database for handwritten text recognition research. IEEE Transactions on pattern analysis and machine intelligence, 16(5): 550–554.
  31. Learning with multiple labels. In NeurIPS, 897–904.
  32. Abductive logic programming. Journal of logic and computation, 2(6): 719–770.
  33. Adam: A method for stochastic optimization. In ICLR.
  34. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11): 2278–2324.
  35. Closed loop neural-symbolic learning via integrating neural perception, grammar parsing, and symbolic reasoning. In ICML, 5884–5894.
  36. Softened Symbol Grounding for Neuro-symbolic Systems. In ICLR.
  37. Statistical Analysis With Missing Data. Wiley Series in Probability and Statistics. Wiley. ISBN 9780471802549.
  38. Out-of-Distribution Generalization by Neural-Symbolic Joint Training. In AAAI, 12252–12259.
  39. Learnability of the superset label learning problem. In ICML, 1629–1637.
  40. Deepproblog: Neural probabilistic logic programming. In NeurIPS, 3753–3763.
  41. Neuro Symbolic Continual Learning: Knowledge, Reasoning Shortcuts and Concept Rehearsal. In ICML, 23915–23936.
  42. Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcuts. In NeurIPS.
  43. Muggleton, S. H. 2023. Hypothesizing an algorithm from one example: the role of specificity. Philosophical Transactions of the Royal Society A, 381(2251): 20220046.
  44. Learning with noisy labels. In NeurIPS, 1196–1204.
  45. Pytorch: An imperative style, high-performance deep learning library. In NeurIPS, 8024–8035.
  46. Making deep neural networks robust to label noise: A loss correction approach. In CVPR, 1944–1952.
  47. Peirce, C. S. 1955. Abduction and induction. Philosophical Writings of Pierce, 150–56.
  48. Russell, S. 2015. Unifying logic and probability. Communications of the ACM, 58(7): 88–97.
  49. A simple neural network module for relational reasoning. In NeurIPS, 4967–4976.
  50. Human problem solving: The state of the theory in 1970. American psychologist, 26(2): 145.
  51. Thoma, M. 2017. The hasyv2 dataset. arXiv preprint arXiv:1701.08380.
  52. Knowledge-based artificial neural networks. Artificial intelligence, 70(1-2): 119–165.
  53. Neural arithmetic logic units. In NeurIPS, 8046––8055.
  54. Neural-symbolic integration: A compositional perspective. In AAAI, 5051–5060.
  55. Tac-valuer: Knowledge-based stroke evaluation in table tennis. In KDD, 3688–3696.
  56. On Learning Latent Models with Multi-Instance Weak Supervision. In NeurIPS.
  57. Satnet: Bridging deep learning and logical reasoning using a differentiable satisfiability solver. In ICML, 6545–6554.
  58. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747.
  59. A semantic loss function for deep learning with symbolic knowledge. In ICML, 5502–5511.
  60. NeurASP: embracing neural networks into answer set programming. In IJCAI, 1755–1762.
  61. Learning with biased complementary labels. In ECCV, 68–83.
  62. Learning from aggregate observations. In NeurIPS, 7993–8005.
  63. Zhou, Z. 2019. Abductive learning: towards bridging machine learning and logical reasoning. Science China Information Sciences, 62(7): 76101:1–76101:3.
  64. Zhou, Z.-H. 2018. A brief introduction to weakly supervised learning. National science review, 5(1): 44–53.
  65. Abductive Learning. In Neuro-Symbolic Artificial Intelligence: The State of the Art, 353–369. IOS Press.
  66. Multi-instance learning by treating instances as non-iid samples. In ICML, 1249–1256.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (4)
  1. Lue Tao (10 papers)
  2. Yu-Xuan Huang (1 paper)
  3. Wang-Zhou Dai (9 papers)
  4. Yuan Jiang (48 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now