Papers
Topics
Authors
Recent
Search
2000 character limit reached

Parallel Neurosymbolic Integration with Concordia

Published 1 Jun 2023 in cs.AI and cs.CL | (2306.00480v2)

Abstract: Parallel neurosymbolic architectures have been applied effectively in NLP by distilling knowledge from a logic theory into a deep model.However, prior art faces several limitations including supporting restricted forms of logic theories and relying on the assumption of independence between the logic and the deep network. We present Concordia, a framework overcoming the limitations of prior art. Concordia is agnostic both to the deep network and the logic theory offering support for a wide range of probabilistic theories. Our framework can support supervised training of both components and unsupervised training of the neural component. Concordia has been successfully applied to tasks beyond NLP and data classification, improving the accuracy of state-of-the-art on collective activity detection, entity linking and recommendation tasks.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. Integrating knowledge and reasoning in image understanding. In IJCAI, pp.  6252–6259, 2019.
  2. Hinge-loss markov random fields and probabilistic soft logic. Journal of Machine Learning Research, 18:109:1–109:67, 2017.
  3. Learning context for collective activity recognition. In CVPR, pp.  3273–3280. IEEE, 2011.
  4. Tensorlog: A probabilistic database implemented using deep-learning infrastructure. J. Artif. Intell. Res., 67:285–325, 2020. doi: 10.1613/jair.1.11944.
  5. Bridging Machine Learning and Logical Reasoning by Abductive Learning. In NeurIPS, pp.  2815–2826, 2019.
  6. Knowledge distillation as semiparametric inference. In ICLR, 2021.
  7. Neural-symbolic learning systems: foundations and applications. Perspectives in neural computing. Springer, 2002.
  8. Probabilistic (logic) programming concepts. Machine Learning, 100(1):5–47, 2015.
  9. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
  10. Logic tensor networks for semantic image interpretation. In IJCAI, pp.  1596–1602, 2017.
  11. Neural network matrix factorization. arXiv preprint arXiv:1511.06443, 2015.
  12. Learning explanatory rules from noisy data. J. Artif. Intell. Res., 61:1–64, 2018. doi: 10.1613/jair.5714.
  13. Principled and efficient motif finding for structure learning in lifted graphical models. arXiv preprint arXiv:2302.04599, 2023.
  14. Inference and learning in probabilistic logic programs using weighted boolean formulas. Theory and Practice of Logic Programming (TPLP), 15(3):358–401, 2015.
  15. Dl2: training and querying neural networks with logic. In International Conference on Machine Learning, pp. 1931–1941. PMLR, 2019.
  16. Posterior regularization for structured latent variable models. Journal of Machine Learning Research, 11:2001–2049, 2010.
  17. Parameter learning in probabilistic databases: A least squares approach. In Machine Learning and Knowledge Discovery in Databases, pp. 473–488, 2008.
  18. The movielens datasets: History and context. ACM Transactions on Interactive Intelligent Systems, 5(4), 2015.
  19. Mask R-CNN. In ICCV, pp.  2980–2988, 2017a.
  20. Neural collaborative filtering. In WWW, pp.  173–182, 2017b.
  21. Distilling the knowledge in a neural network. CoRR, abs/1503.02531, 2015.
  22. Harnessing deep neural networks with logic rules. In ACL, pp.  2410–2420, 2016a.
  23. Deep neural networks with massive learned knowledge. In EMNLP, pp.  1670–1679, 2016b.
  24. Hierarchical relational networks for group activity recognition and retrieval. In ECCV, pp.  742–758, 2018.
  25. Adaptive mixtures of local experts. Neural computation, 3(1):79–87, 1991.
  26. Gradient-based boosting for statistical relational learning: the markov logic network and missing data cases. Machine Learning, 100(1):75–100, 2015.
  27. Learning markov logic networks using structural motifs. In ICML, volume 10, pp.  551–558, 2010.
  28. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37, 2009.
  29. Hyper: A flexible and extensible probabilistic framework for hybrid recommender systems. In RecSys, pp.  99–106, 2015.
  30. Video understanding based on human action and group activity recognition. CoRR, abs/2010.12968, 2020.
  31. Retrieving actions in group contexts. In ECCV Workshops, pp.  181–194, 2012.
  32. Augmenting neural networks with first-order logic. arXiv preprint arXiv:1906.06298, 2019.
  33. Collective activity detection using hinge-loss markov random fields. In CVPR Workshops, pp.  566–571, 2013.
  34. Deepproblog: Neural probabilistic logic programming. In NeurIPS, pp.  3749–3759, 2018.
  35. Relational neural machines. In ECAI, volume 325, pp.  1340–1347, 2020.
  36. Towards neural theorem proving at scale. arXiv preprint arXiv:1807.08204, 2018.
  37. Self-distillation amplifies regularization in hilbert space. In NeurIPS, 2020.
  38. Distributional semantics resources for biomedical text processing. Proceedings of LBM, pp.  39–44, 2013.
  39. A Comprehensive Survey of Neighborhood-Based Recommendation Methods, pp.  37–76. Springer, 2015.
  40. Cross-sentence n-ary relation extraction with graph lstms. Transactions of the Association for Computational Linguistics, 5:101–115, 2017.
  41. stagnet: An attentive semantic RNN for group activity recognition. In ECCV, pp.  104–120, 2018.
  42. Probabilistic logic neural networks for reasoning. In NeurIPS, pp.  7710–7720, 2019.
  43. Attribute-aware non-linear co-embeddings of graph features. In Proceedings of the 13th ACM conference on recommender systems, pp.  314–321, 2019.
  44. Markov logic networks. Mach. Learn., 62(1-2):107–136, 2006.
  45. Bayesian probabilistic matrix factorization using markov chain monte carlo. In ICML, pp.  880–887, 2008.
  46. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.
  47. Deep adaptive semantic logic (dasl): Compiling declarative knowledge into deep neural networks. arXiv preprint arXiv:2003.07344, 2020.
  48. Neural-symbolic integration: A compositional perspective. In AAAI, 2021.
  49. Analyzing Differentiable Fuzzy Implications. In KR, pp.  893–903, 2020.
  50. Deep probabilistic logic: A unifying framework for indirect supervision. In EMNLP, pp.  1891–1902, 2018.
  51. Learning actor relation graphs for group activity recognition. In CVPR, pp.  9964–9974, 2019.
  52. Embedding symbolic knowledge into deep networks. In NeurIPS, pp.  4235–4245, 2019.
  53. NeurASP: Embracing neural networks into answer set programming. In IJCAI, pp.  1755–1762, 2020.
  54. Reasoning about object affordances in a knowledge base representation. In ECCV, volume 8690, pp.  408–424, 2014.
Citations (1)
View on Semantic Scholar

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now

Collections

Sign up for free to add this paper to one or more collections.

Sign Up