Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Math Word Problem Solving by Generating Linguistic Variants of Problem Statements (2306.13899v1)

Published 24 Jun 2023 in cs.CL and cs.AI

Abstract: The art of mathematical reasoning stands as a fundamental pillar of intellectual progress and is a central catalyst in cultivating human ingenuity. Researchers have recently published a plethora of works centered around the task of solving Math Word Problems (MWP) $-$ a crucial stride towards general AI. These existing models are susceptible to dependency on shallow heuristics and spurious correlations to derive the solution expressions. In order to ameliorate this issue, in this paper, we propose a framework for MWP solvers based on the generation of linguistic variants of the problem text. The approach involves solving each of the variant problems and electing the predicted expression with the majority of the votes. We use DeBERTa (Decoding-enhanced BERT with disentangled attention) as the encoder to leverage its rich textual representations and enhanced mask decoder to construct the solution expressions. Furthermore, we introduce a challenging dataset, $\mathrm{P\small{ARA}\normalsize{MAWPS}}$, consisting of paraphrased, adversarial, and inverse variants of selectively sampled MWPs from the benchmark $\mathrm{M\small{AWPS}}$ dataset. We extensively experiment on this dataset along with other benchmark datasets using some baseline MWP solver models. We show that training on linguistic variants of problem statements and voting on candidate predictions improve the mathematical reasoning and robustness of the model. We make our code and data publicly available.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (95)
  1. Yefim Bakman. 2007. Robust understanding of word problems with extraneous information. arXiv preprint math/0701393.
  2. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In Proceedings of the 43rd annual meeting of the Association for Computational Linguistics (ACL’05), pages 597–604.
  3. Regina Barzilay and Kathleen McKeown. 2001. Extracting paraphrases from a parallel corpus. In Proceedings of the 39th annual meeting of the Association for Computational Linguistics, pages 50–57.
  4. Daniel G Bobrow. 1964. Natural language input for a computer problem solving system.
  5. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  6. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712.
  7. Deng Cai and Wai Lam. 2020. Graph transformer for graph-to-sequence learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7464–7471.
  8. A bottom-up dag structure extraction model for math word problems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 39–46.
  9. A weakly supervised model for solving math word problems. arXiv preprint arXiv:2104.06722.
  10. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588.
  11. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.
  12. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805.
  13. Computers and thought. New York McGraw-Hill.
  14. Charles R Fletcher. 1985. Understanding and solving arithmetic word problems: A computer simulation. Behavior Research Methods, Instruments, & Computers, 17(5):565–571.
  15. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654.
  16. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2(7).
  17. Large language models are reasoning teachers. arXiv preprint arXiv:2212.10071.
  18. Learning by fixing: Solving math word problems with weak supervision. In AAAI Conference on Artificial Intelligence.
  19. Learning to solve arithmetic word problems with verb categorization. In EMNLP, volume 523533. Citeseer.
  20. Neural math word problem solver with reinforcement learning. In Proceedings of the 27th International Conference on Computational Linguistics, pages 213–223.
  21. Learning fine-grained expressions to solve math word problems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 805–814.
  22. How well do computers solve math word problems? large-scale dataset construction and evaluation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 887–896.
  23. Jie Huang and Kevin Chen-Chuan Chang. 2022. Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403.
  24. Recall and learn: A memory-augmented solver for math word problems. arXiv preprint arXiv:2109.13112.
  25. Learning to reason deductively: Math word problem solving as complex relation extraction. arXiv preprint arXiv:2203.10316.
  26. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
  27. Sheri Kingsdorf and Jennifer Krawec. 2016. A broad look at the literature on math word problem-solving interventions for third graders. Cogent Education, 3(1):1135770.
  28. Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907.
  29. Parsing algebraic word problems into equations. Transactions of the Association for Computational Linguistics, 3:585–597.
  30. Mawps: A math word problem repository. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1152–1157.
  31. Adversarial examples for evaluating math word problem solvers. arXiv preprint arXiv:2109.05925.
  32. Practice makes a solver perfect: Data augmentation for math word problem solvers. arXiv preprint arXiv:2205.00177.
  33. Learning to automatically solve algebra word problems. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 271–281.
  34. Mwptoolkit: An open-source framework for deep learning-based math word problem solvers. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 13188–13190.
  35. Deduplicating training data makes language models better. arXiv preprint arXiv:2107.06499.
  36. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461.
  37. Modeling intra-relation in math word problems with different functional multi-head attentions. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6162–6167.
  38. Graph-to-tree neural networks for learning structured input-output translation with applications to semantic parsing and math word problem. arXiv preprint arXiv:2004.13781.
  39. On the advance of making language models better reasoners. arXiv preprint arXiv:2206.02336.
  40. Seeking patterns, not just memorizing procedures: Contrastive learning for solving math word problems. arXiv preprint arXiv:2110.08464.
  41. A tag-based english math word problem solver with understanding, reasoning and explanation. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 67–71.
  42. A meaning-based English math word problem solver with understanding, reasoning and explanation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, pages 151–155, Osaka, Japan. The COLING 2016 Organizing Committee.
  43. Let gpt be a math tutor: Teaching math word problem solvers with customized exercise generation. arXiv preprint arXiv:2305.14386.
  44. Mwp-bert: A strong baseline for math word problems. arXiv preprint arXiv:2107.13435.
  45. Mwp-bert: A numeracy-augmented pre-trained encoder for math word problems.
  46. Hms: A hierarchical solver with dependency-enhanced understanding for math word problem. In Thirty-Fifth AAAI Conference on Artificial 2021, pages 4232–4240.
  47. Roda: reverse operation based data augmentation for solving math word problems. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:1–11.
  48. Tree-structured decoding for solving math word problems. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 2370–2379.
  49. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
  50. Nitin Madnani and Bonnie J Dorr. 2010. Generating phrasal and sentential paraphrases: A survey of data-driven methods. Computational Linguistics, 36(3):341–387.
  51. Paraphrasing revisited with neural machine translation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 881–893, Valencia, Spain. Association for Computational Linguistics.
  52. Kathleen R McKeown. 1980. Paraphrasing using given and new information in a question-answer system. Technical Reports (CIS), page 723.
  53. Marie Meteer and Varda Shaked. 1988. Strategies for effective paraphrasing. In Coling Budapest 1988 Volume 2: International Conference on Computational Linguistics.
  54. A diverse corpus for evaluating and developing english math word problem solvers. arXiv preprint arXiv:2106.15772.
  55. Equgener: A reasoning network for word problem solving by generating arithmetic equations. In Proceedings of the 32nd Pacific Asia Conference on Language, Information and Computation.
  56. Arindam Mitra and Chitta Baral. 2016. Learning to use formulas to solve simple arithmetic problems. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2144–2153.
  57. Are nlp models really able to solve simple math word problems? arXiv preprint arXiv:2103.07191.
  58. Neuropsychological performance, iq, personality, and grades in a longitudinal grade-school male sample. Individual Differences Research, 1:159–172.
  59. Reasoning like program executors. arXiv preprint arXiv:2201.11473.
  60. Jean Piaget. 2013. Child’s Conception of Number: Selected Works vol 2. Routledge.
  61. Neural paraphrase generation with stacked residual lstm networks. arXiv preprint arXiv:1610.03098.
  62. Neural-symbolic solver for math word problems with auxiliary tasks. arXiv preprint arXiv:2107.01431.
  63. Subhro Roy and Dan Roth. 2016. Solving general arithmetic word problems. arXiv preprint arXiv:1608.01413.
  64. Subhro Roy and Dan Roth. 2017. Unit dependency graph and its application to arithmetic word problem solving. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31.
  65. Equation parsing: Mapping sentences to grounded equations. arXiv preprint arXiv:1609.08824.
  66. Reasoning about quantities in natural language. Transactions of the Association for Computational Linguistics, 3:1–13.
  67. Timo Schick and Hinrich Schütze. 2021. Generating datasets with pretrained language models. arXiv preprint arXiv:2104.07540.
  68. Chaining simultaneous thoughts for numerical reasoning. arXiv preprint arXiv:2211.16482.
  69. Generate & rank: A multi-task framework for math word problems. arXiv preprint arXiv:2109.03034.
  70. Yibin Shen and Cheqing Jin. 2020. Solving math word problems with multi-encoders and multi-decoders. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2924–2934.
  71. Automatically solving number word problems by semantic parsing and reasoning. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1132–1142.
  72. Variational recurrent neural machine translation. In Proceedings of the AAAI conference on artificial intelligence, volume 32.
  73. Why are nlp models fumbling at elementary math? a survey of deep learning based word problem solvers. arXiv preprint arXiv:2205.15683.
  74. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.
  75. Learning from explicit and implicit supervision jointly for algebra word problems. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 297–306.
  76. Attention is all you need. Advances in neural information processing systems, 30.
  77. Translating a math word problem to an expression tree. arXiv preprint arXiv:1811.05632.
  78. Mathdqn: Solving arithmetic word problems via deep reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
  79. Template-based math word problem solvers with recursive neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 7144–7151.
  80. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  81. Deep neural solver for math word problems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 845–854.
  82. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903.
  83. Sam Witteveen and Martin Andrews. 2019. Paraphrasing with large language models. arXiv preprint arXiv:1911.09661.
  84. A knowledge-aware sequence-to-tree network for math word problem solving. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7137–7146.
  85. Zhipeng Xie and Shichao Sun. 2019. A goal-driven tree-structured neural model for math word problems. In IJCAI, pages 5299–5305.
  86. Self-consistent reasoning for solving math word problems. arXiv preprint arXiv:2210.15373.
  87. Harnessing the power of llms in practice: A survey on chatgpt and beyond. arXiv preprint arXiv:2304.13712.
  88. Improving math word problems with pre-trained knowledge and hierarchical reasoning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3384–3394.
  89. Frame-based calculus of solving arithmetic multi-step addition and subtraction word problems. In 2010 Second International Workshop on Education Technology and Computer Science, volume 2, pages 476–479. IEEE.
  90. Graph transformer networks. Advances in neural information processing systems, 32.
  91. The gap of semantic parsing: A survey on automatic math word problem solvers. IEEE transactions on pattern analysis and machine intelligence, 42(9):2287–2305.
  92. Teacher-student networks with multiple decoders for solving math word problem. IJCAI.
  93. Graph-to-tree learning for solving math word problems. Association for Computational Linguistics.
  94. Learn to solve algebra word problems using quadratic programming. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 817–822.
  95. Yanyan Zou and Wei Lu. 2019. Text2math: End-to-end parsing text into math expressions. arXiv preprint arXiv:1910.06571.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (6)
  1. Syed Rifat Raiyan (4 papers)
  2. Md. Nafis Faiyaz (1 paper)
  3. Shah Md. Jawad Kabir (1 paper)
  4. Mohsinul Kabir (11 papers)
  5. Hasan Mahmud (20 papers)
  6. Md Kamrul Hasan (71 papers)
Citations (8)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now
Github Logo Streamline Icon: https://streamlinehq.com

GitHub

  1. GitHub - Starscream-11813/Variational-Mathematical-Reasoning