Absence of Phase Transition in Random Language Model

Abstract: The Random LLM, proposed as a simple model of human languages, is defined by the averaged model of a probabilistic context-free grammar. This grammar expresses the process of sentence generation as a tree graph with nodes having symbols as variables. Previous studies proposed that a phase transition, which can be considered to represent the emergence of order in language, occurs in the random LLM. We discuss theoretically that the analysis of the "order parameter" introduced in previous studies can be reduced to solving the maximum eigenvector of the transition probability matrix determined by a grammar. This helps analyze the distribution of a quantity determining the behavior of the "order parameter" and reveals that no phase transition occurs. Our results suggest the need to study a more complex model such as a probabilistic context-sensitive grammar, in order for phase transitions to occur.