Automatic Proficiency Assessment in L2 English Learners (2505.02615v1)

Abstract: Second language proficiency (L2) in English is usually perceptually evaluated by English teachers or expert evaluators, with the inherent intra- and inter-rater variability. This paper explores deep learning techniques for comprehensive L2 proficiency assessment, addressing both the speech signal and its correspondent transcription. We analyze spoken proficiency classification prediction using diverse architectures, including 2D CNN, frequency-based CNN, ResNet, and a pretrained wav2vec 2.0 model. Additionally, we examine text-based proficiency assessment by fine-tuning a BERT LLM within resource constraints. Finally, we tackle the complex task of spontaneous dialogue assessment, managing long-form audio and speaker interactions through separate applications of wav2vec 2.0 and BERT models. Results from experiments on EFCamDat and ANGLISH datasets and a private dataset highlight the potential of deep learning, especially the pretrained wav2vec 2.0 model, for robust automated L2 proficiency evaluation.

Application of Deep Learning Models for L2 English Proficiency Assessment

The paper on "Automatic Proficiency Assessment in L2 English Learners" explores the deployment of deep learning methodologies for the evaluation of English language proficiency in second language (L2) learners. Traditionally, proficiency assessments have relied on human evaluators, whose subjective judgments can lead to variability in outcomes. Automated approaches offer scalable and objective alternatives, addressing the inherent challenges in dynamic spoken environments.

The paper introduces various neural architectures employed to assess language proficiency from both audio signals and text transcriptions. It encompasses two domains of analysis: speech-based and text-based proficiency assessment, utilizing models that capture different dimensions of language use. Among the architectures evaluated are the 2D CNNs, ResNets, and the self-supervised, pretrained model wav2vec 2.0 for audio-based evaluations. Additionally, BERT-based models are explored for text assessment tasks.

Speech-Based Proficiency Assessment

The paper demonstrates the application of multiple architectures on structured and spontaneous speech datasets. Notably, wav2vec 2.0 exhibited superior performance, achieving a classification accuracy of up to 75% on single-task operations and 80% in multi-task scenarios across the ANGLISH dataset. This highlights the capabilities of pretrained models in handling complex acoustic features and their generalizability across different speaker profiles—a crucial aspect in realistic assessment frameworks. In contrast, traditional models such as 2D CNNs presented lower accuracies, underscoring the benefits of utilizing highly contextualized transformer networks for audio data processing.

The results emphasize the importance of context length, as longer audio segments contribute richer linguistic cues, without significantly increasing computational complexity. Separate segmentation pipelines for dialogic audio highlighted the incorporation of interactional signals in improving model performance, particularly within spontaneous dialogues.

Text-Based Proficiency Assessment

For text-based evaluations, BERT models fine-tuned on learner transcriptions showed commendable results, with accuracy reaching 87.5% on the EFCamDat dataset. This performance improvement was facilitated by contextual embeddings that richly encode syntactic and semantic information, confirming the effectiveness of transformer-based models in linguistic analysis tasks. The investigation into token length revealed optimal sequence lengths around 60 tokens, balancing performance and computation.

Implications and Future Directions

The findings underscore the potential of deep learning models, particularly self-supervised and transformer architectures, in automating proficiency assessments. The superior performance of multimodal and pretrained models suggests significant implications for developing assessment tools in educational settings, providing scalable evaluations with robust accuracy.

Future research may further explore the integration of multimodal learning frameworks, optimizing the fusion of audio and text-based features to enhance the predictive accuracy of language proficiency assessments. This paper hints at the increasing necessity of specialized architectures for nuanced language evaluation, catering to both structured and spontaneous speech contexts. Addressing challenges, such as proficiency level misclassification in text-only models, and improving speaker-independent generalization remain fruitful research paths for advancing this domain. Overall, the potential for applying these methodologies extends beyond language proficiency assessments, paving the way for broader applications across various AI-driven linguistic evaluations.