Papers
Topics
Authors
Recent
Search
2000 character limit reached

Strahler Number of Natural Language Sentences in Comparison with Random Trees

Published 6 Jul 2023 in cs.CL and physics.data-an | (2307.02697v4)

Abstract: The Strahler number was originally proposed to characterize the complexity of river bifurcation and has found various applications. This article proposes computation of the Strahler number's upper and lower limits for natural language sentence tree structures. Through empirical measurements across grammatically annotated data, the Strahler number of natural language sentences is shown to be almost 3 or 4, similarly to the case of river bifurcation as reported by Strahler (1957). From the theory behind the number, we show that it is one kind of lower limit on the amount of memory required to process sentences. We consider the Strahler number to provide reasoning that explains reports showing that the number of required memory areas to process sentences is 3 to 4 for parsing (Schuler et al., 2010), and reports indicating a psychological "magical number" of 3 to 5 (Cowan, 2001). An analytical and empirical analysis shows that the Strahler number is not constant but grows logarithmically; therefore, the Strahler number of sentences derives from the range of sentence lengths. Furthermore, the Strahler number is not different for random trees, which could suggest that its origin is not specific to natural language.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (66)
  1. Memory requirements and local ambiguities of parsing strategies. Journal of Psycholinguistic Research, 20, 233–250.
  2. Beyond word frequency: Bursts, lulls, and scaling in the temporal distributions of words. PLoS One, 4(e7678).
  3. On the origin of long-range correlations in texts. Proceedings of the National Academy of Sciences, 109(29), 11582–11587.
  4. New Strahler Numbers for Rooted Plane Trees. In M. Drmota, P. Flajolet, D. Gardy, and B. Gittenberger, editors, Mathematics and Computer Science III, pages 203–215, Basel. Birkhäuser Basel.
  5. Horton’s Laws and the fractal nature of streams. Water Resources Research, 29(5), 1475–1487.
  6. Text Compression. Prentice Hall.
  7. Buchholz, S. N. (2002). Memory-Based Grammatical Relation Finding. Eigen beheer. Doctoral Thesis.
  8. Menzerah-altmann law for syntactic structures in ukrainian. https://arxiv.org/pdf/cs/ 0701194.pdf.
  9. Chomsky, N. (1956). Three models for the description of language. IRE Transactions on Information Theory, 2(3), 113–124.
  10. Elements of Information Theory. John Wiley & Sons, Inc.
  11. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 87 – 114.
  12. Universal Dependencies. Computational Linguistics, pages 1–52.
  13. Degiuli, E. (2019a). Emergence of order in random languages. 52. 504001.
  14. Degiuli, E. (2019b). Random language model. 122. 128301.
  15. Long-range correlations between letters and sentences in texts. Physica A, 215, 233–241.
  16. Entropy and long-range correlations in literary English. Europhysics Letters, 26(4), 241–246.
  17. On Etol Systems with Finite Tree-Rank. SIAM Journal on Computing, 10(1), 40–58.
  18. Ershov, A. P. (1958). On Programming of Arithmetic Operations. Commun. ACM, 1(8), 3–6.
  19. Faster shift-reduce constituent parsing with a non-binary, bottom-up strategy. Artificial Intelligence, 275, 559–574.
  20. Discontinuous grammar as a foreign language. Neurocomputing, 524, 43–58.
  21. Parsing as Reduction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing Volume 1 : Long Papers, pages 1523–1533. Association for Computational Linguistics.
  22. Tree balance indices: a comprehensive survey. https://arxiv/abs/2109.12281.
  23. The number of registers required for evaluating arithmetic expressions. Theoretical Computer Science, 9(1), 99–125.
  24. Forster, K. I. (1968). Sentence completion in left-and right-branching languages. Journal of Verbal Learning and Verbal Behavior, 7(2), 296–299.
  25. Friedrich, R. (2021). Complexity and entropy in legal language. Frontiers in Physics, 9. 671882.
  26. Gibson, E. (2000). The dependency locality theory: A distance-based theory of linguistic complexity. Image, Language, Brain: Papers from the First Mind Articulation Project Symposium, pages 94–126.
  27. Sentence-Incremental Neural Coreference Resolution. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 427–443.
  28. Horton, R. E. (1945). Erosional Development of Streams and Their Drainage Basins; Hydrophysical Approach to Quantitative Morphology. Geological Society of America Bulletin, 56(3), 275.
  29. A study on correlation between chinese sentence and constituting clauses based on the menzerath-altmann law. Journal of Quantitative Linguistics, pages 350–366.
  30. Kimball, J. P. (1975). Predictive Analysis and Over-the-Top Parsing. Brill, Leiden, The Netherlands.
  31. Taylor’s law for human linguistic sequences. Annual Conference of Association for Computational Lingusitics, pages 1138–1148.
  32. Transforming Dependencies into Phrase Structures. In Proceedings of the 2015 Annual Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, pages 788–798. Association for Computational Linguistics.
  33. Critial behavior in physics and probabilistic formal languages. Entropy, 19(7), 299.
  34. Liu, H. (2008). Dependency Distance as a Metric of Language Comprehension Difficulty. Journal of Cognitive Science, 9(2), 159–191.
  35. Dependency distance: A new perspective on syntactic patterns in natural languages. Physics of life reviews, 21, 171–193.
  36. The capacity of visual working memory for features and conjunctions. Nature, 390(6657), 279–281.
  37. Menzerath-altmann law in syntactic dependency structure. In Proceedings of the Fourth International Conference on Dependency Linguistics (Depling 2017), pages 100–107, Pisa,Italy. Linköping University Electronic Press.
  38. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological review, 63(2), 81.
  39. Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4034–4043, Marseille, France. European Language Resources Association.
  40. Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4034–4043, Marseille, France. European Language Resources Association.
  41. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4034–4043, Marseille, France. European Language Resources Association.
  42. Left-corner transitions on dependency parsing. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 2140–2150, Dublin, Ireland. Dublin City University and Association for Computational Linguistics.
  43. Universal Semantic Parsing. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 89–101, Copenhagen, Denmark. Association for Computational Linguistics.
  44. Sanada, H. (2016). The menzerath-altmann law and sentence structure. Journal of Quantitative Lingusitics, 23(3), 256–277.
  45. Broad-Coverage Parsing Using Human-like Memory Constraints. Computational Linguistics, 36(1), 1–30.
  46. The Generation of Optimal Code for Arithmetic Expressions. J. ACM, 17(4), 715–728.
  47. Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379–423.
  48. Sichel, H. S. (1974). On a Distribution Representing Sentence-length in written Prose. Journal of the Royal Statistical Society: Series A, 137(1), 25–34.
  49. Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs: General and Applied, 74(11), 1.
  50. Stanley, R. P. (2015). Catalan numbers. Cambridge University Press.
  51. Strahler, A. N. (1957). Quantitative analysis of watershed geomorphology. Eos, Transactions American Geophysical Union, 38(6), 913–920.
  52. Evaluating computational language models with scaling properties of natural language. Computational Lingusitics, 45(3), 1–33.
  53. Entropy rate estimates for natural language—a new extrapolation of compressed large-scale corpora. Entropy, 18(10), 364.
  54. Tanaka-Ishii, K. (2021). Menzerath’s law in the syntax of languages compared with random sentences. Entropy. e23060661. Selected as Editor’s Choice Articles of Entropy 2022.
  55. Computational constancy measures of texts – Yule’s K and Rényi’s entropy. Association for Computational Linguistics, 41(3), 481–502.
  56. Long-range memory in literary texts: On the universal clustering of the rare words. PLoS One, 11(11), e0164658.
  57. Taylor’s law for linguistic sequences and random walk models. Journal of Physics Communications, 2(11), 115024. 089401.
  58. Tesnière, L. (1959). Élements de syntaxe structurale. C. Klincksieck, Paris.
  59. Development of a Multilingual CCG Treebank via Universal Dependencies Conversion. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5220–5233, Marseille, France. European Language Resources Association.
  60. Convergence of Syntactic Complexity in Conversation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 443–448.
  61. Word Order Typology Interacts With Linguistic Complexity: A Cross-Linguistic Corpus Study. Cognitive Science, 44(4), e12822.
  62. Strongly Incremental Constituency Parsing with Graph Neural Networks. Advances in Neural Information Processing Systems, 33, 21687–21698.
  63. Yngve, V. H. (1960). A Model and an Hypothesis for Language Structure. Proceedings of the American Philosophical Society, 104(5), 444–466.
  64. Yule, U. G. (1968). The Statistical Study of Literary Vocabulary. Archon Books.
  65. Zhang, M. (2020). A survey of syntactic-semantic parsing based on constituent and dependency structures. Science China Technological Sciences, 63(10), 1898–1920.
  66. Zipf, G. K. (1949). Human Behavior and the Principle of Least Effort : An Introduction to Human Ecology. Addison-Wesley Press.

Summary

No one has generated a summary of this paper yet.

Sign Up to Summarize

Paper to Video (Beta)

No one has generated a video about this paper yet.

Sign Up to Generate All Videos Subscribe on YouTube

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Sign Up to Generate

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Generate Now

Continue Learning

We haven't generated follow-up questions for this paper yet.

Generate Now