Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
125 tokens/sec
GPT-4o
53 tokens/sec
Gemini 2.5 Pro Pro
42 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Who Said What? An Automated Approach to Analyzing Speech in Preschool Classrooms (2401.07342v3)

Published 14 Jan 2024 in eess.AS and cs.LG

Abstract: Young children spend substantial portions of their waking hours in noisy preschool classrooms. In these environments, children's vocal interactions with teachers are critical contributors to their language outcomes, but manually transcribing these interactions is prohibitive. Using audio from child- and teacher-worn recorders, we propose an automated framework that uses open source software both to classify speakers (ALICE) and to transcribe their utterances (Whisper). We compare results from our framework to those from a human expert for 110 minutes of classroom recordings, including 85 minutes from child-word microphones (n=4 children) and 25 minutes from teacher-worn microphones (n=2 teachers). The overall proportion of agreement, that is, the proportion of correctly classified teacher and child utterances, was .76, with an error-corrected kappa of .50 and a weighted F1 of .76. The word error rate for both teacher and child transcriptions was .15, meaning that 15% of words would need to be deleted, added, or changed to equate the Whisper and expert transcriptions. Moreover, speech features such as the mean length of utterances in words, the proportion of teacher and child utterances that were questions, and the proportion of utterances that were responded to within 2.5 seconds were similar when calculated separately from expert and automated transcriptions. The results suggest substantial progress in analyzing classroom speech that may support children's language development. Future research using natural language processing is under way to improve speaker classification and to analyze results from the application of the automated framework to a larger dataset containing classroom recordings from 13 children and 3 teachers observed on 17 occasions over one year.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (32)
  1. Y. Gong, S. Khurana, L. Karlinsky, and J. Glass, “Whisper-at: Noise-robust automatic speech recognizers are also strong general audio event taggers,” arXiv preprint arXiv:2307.03183, 2023.
  2. O. Räsänen, S. Seshadri, M. Lavechin, A. Cristia, and M. Casillas, “Alice: An open-source tool for automatic measurement of phoneme, syllable, and word counts from child-centered daylong recordings,” Behavior Research Methods, vol. 53, pp. 818–835, 2021.
  3. N. C. f. E. Statistics, “Preprimary education enrollment,” https://nces.ed.gov/fastfacts/display.asp?id=516, 2023, (Accessed on 01/07/2024).
  4. E. M. Barnes, D. K. Dickinson, and J. F. Grifenhagen, “The role of teachers’ comments during book reading in children’s vocabulary growth,” The Journal of Educational Research, vol. 110, no. 5, pp. 515–527, 2017.
  5. E. F. Ferguson, A. S. Nahmias, S. Crabbe, T. Liu, D. S. Mandell, and J. Parish-Morris, “Social language opportunities for preschoolers with autism: Insights from audio recordings in urban classrooms,” Autism, vol. 24, no. 5, pp. 1232–1245, 2020.
  6. S. Houen, K. Thorpe, D. van Os, E. Westwood, D. Toon, and S. Staton, “Eliciting and responding to young children’s talk: A systematic review of educators’ interactional strategies that promote rich conversations with children aged 2–5 years,” Educational Research Review, vol. 37, p. 100473, 2022.
  7. D. K. Dickinson and M. V. Porche, “Relation between language experiences in preschool classrooms and children’s kindergarten and fourth-grade language and reading abilities,” Child development, vol. 82, no. 3, pp. 870–886, 2011.
  8. M. Burchinal, C. Howes, R. Pianta, D. Bryant, D. Early, R. Clifford, and O. Barbarin, “Predicting child outcomes at the end of kindergarten from the quality of pre-kindergarten teacher–child interactions and instruction,” Applied Development Science, vol. 12, no. 3, pp. 140–153, 2008.
  9. R. M. Fasano, S. G. Mitsven, S. A. Custode, D. Sarker, R. J. Bulotsky-Shearer, D. S. Messinger, and L. K. Perry, “Automated measures of vocal interactions and engagement in inclusive preschool classrooms,” Autism research, vol. 16, no. 8, pp. 1586–1599, 2023.
  10. R. M. Fasano, L. K. Perry, Y. Zhang, L. Vitale, J. Wang, C. Song, and D. S. Messinger, “A granular perspective on inclusion: Objectively measured interactions of preschoolers with and without autism,” Autism research, vol. 14, no. 8, pp. 1658–1669, 2021.
  11. S. G. Mitsven, L. K. Perry, Y. Tao, B. E. Elbaum, N. F. Johnson, and D. S. Messinger, “Objectively measured teacher and preschooler vocalizations: Phonemic diversity is associated with language abilities,” Developmental science, vol. 25, no. 2, p. e13177, 2022.
  12. E. B. Hadley, E. M. Barnes, and H. Hwang, “Purposes, places, and participants: A systematic review of teacher language practices and child oral language outcomes in early childhood classrooms,” Early Education and Development, vol. 34, no. 4, pp. 862–884, 2023.
  13. L. Girolametto and E. Weitzman, “Responsiveness of child care providers in interactions with toddlers and preschoolers,” 2002.
  14. L. M. Justice, H. Jiang, and K. Strasser, “Linguistic environment of preschool classrooms: What dimensions support children’s language growth?” Early Childhood Research Quarterly, vol. 42, pp. 79–92, 2018.
  15. S. Q. Cabell, L. M. Justice, A. S. McGinty, J. DeCoster, and L. D. Forston, “Teacher–child conversations in preschool classrooms: Contributions to children’s vocabulary development,” Early Childhood Research Quarterly, vol. 30, pp. 80–92, 2015.
  16. L. K. Perry, E. B. Prince, A. M. Valtierra, C. Rivero-Fernandez, M. A. Ullery, L. F. Katz, B. Laursen, and D. S. Messinger, “A year in words: The dynamics and consequences of language experiences in an intervention classroom,” PloS one, vol. 13, no. 7, p. e0199893, 2018.
  17. C. De Rivera, L. Girolametto, J. Greenberg, and E. Weitzman, “Children’s responses to educators’ questions in day care play groups,” 2005.
  18. S. L. Massey, K. L. Pence, L. M. Justice, and R. P. Bowles, “Educators’ use of cognitively challenging questions in economically disadvantaged preschool classroom contexts,” Early Education and Development, vol. 19, no. 2, pp. 340–360, 2008.
  19. B. Elbaum, L. K. Perry, and D. S. Messinger, “Investigating children’s interactions in preschool classrooms: An overview of research using automated sensing technologies,” Early childhood research quarterly, vol. 66, pp. 147–156, 2024.
  20. Y. Seven, D. W. Irvin, P. V. Kothalkar, S. Dutta, J. F. Buzhardt, B. Rous, and J. H. Hansen, “Capturing the quantity and location of adult wh-words in the preschool classroom using a sensing tool system,” Early Childhood Research Quarterly, vol. 66, pp. 168–177, 2024.
  21. S. Dutta, D. Irvin, J. Buzhardt, and J. H. Hansen, “Activity focused speech recognition of preschool children in early childhood classrooms,” in Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), 2022, pp. 92–100.
  22. R. Lileikyte, D. Irvin, and J. H. Hansen, “Assessing child communication engagement via speech recognition in naturalistic active learning spaces,” ISCA ODYSSEY-2020, 2020.
  23. J. Gilkerson, J. A. Richards, S. F. Warren, J. K. Montgomery, C. R. Greenwood, D. Kimbrough Oller, J. H. Hansen, and T. D. Paul, “Mapping the early language environment using all-day recordings and automated analysis,” American journal of speech-language pathology, vol. 26, no. 2, pp. 248–265, 2017.
  24. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz et al., “The kaldi speech recognition toolkit,” in IEEE 2011 workshop on automatic speech recognition and understanding, no. CONF.   IEEE Signal Processing Society, 2011.
  25. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in International Conference on Machine Learning.   PMLR, 2023, pp. 28 492–28 518.
  26. J. Li et al., “Recent advances in end-to-end automatic speech recognition,” APSIPA Transactions on Signal and Information Processing, vol. 11, no. 1, 2022.
  27. G. E. Bianchini, L. Zanotti, and C. Meléndez, “Using openai models as a new tool for text analysis in political leaders’ unstructured discourse,” 2023.
  28. N. D. Duran, A. Paxton, and R. Fusaroli, “Align: Analyzing linguistic interactions with generalizable techniques—a python library.” Psychological methods, vol. 24, no. 4, p. 419, 2019.
  29. R. Fusaroli, E. Weed, R. Rocca, D. Fein, and L. Naigles, “Caregiver linguistic alignment to autistic and typically developing children: A natural language processing approach illuminates the interactive components of language development,” Cognition, vol. 236, p. 105422, 2023.
  30. J. A. O’Sullivan, G. Bogaarts, M. Kosek, R. Ullmann, P. Schoenenberger, C. H. Chatham, D. Nobbs, L. Murtagh, M. Lindemann, J. Parish-Morris et al., “Automatic speech recognition for autism using the open-source whisper model from openai,” INSAR 2023, 2023.
  31. E. B. Hadley, E. M. Barnes, B. M. Wiernik, and M. Raghavan, “A meta-analysis of teacher language practices in early childhood classrooms,” Early Childhood Research Quarterly, vol. 59, pp. 186–202, 2022.
  32. R. L. Walsh and K. A. Hodge, “Are we asking the right questions? an analysis of research on the effect of teachers’ questioning on children’s language during shared book reading with young children,” Journal of Early Childhood Literacy, vol. 18, no. 2, pp. 264–294, 2018.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now