Reheat Nachos for Dinner? Evaluating AI Support for Cross-Cultural Communication of Neologisms

Abstract: Neologisms and emerging slang are central to daily conversation, yet challenging for non-native speakers (NNS) to interpret and use appropriately in cross-cultural communication with native speakers (NS). NNS increasingly make use of AI tools to learn these words. We study the utility of such tools in mediating an informal communication scenario through a human-subjects study (N=234): NNS participants learn English neologisms with AI support, write messages using the learned word to an NS friend, and judge contextual appropriateness of the neologism in two provided writing samples. Using both NS evaluator-rated communicative competence of NNS-produced writing and NNS' contextual appropriateness judgments, we compare three AI-based support conditions: AI Definition, AI Rewrite into simpler English, AI Explanation of meaning and usage, and Non-AI Dictionary for comparison. We show that AI Explanation yields the largest gains over no support in NS-rated competence, while contextual appropriateness judgments show indifference across support. NNS participants' self-reported perceptions tend to overestimate NS ratings, revealing a mismatch between perceived and actual competence. We further observe a significant gap between NNS- and NS-produced writing, highlighting the limitations of current AI tools and informing design for future tools.

• The paper demonstrates that AI Explanation support significantly improves native speaker-rated communicative competence in the use of neologisms.
• It uses a controlled experiment comparing five support conditions, revealing that detailed, context-rich feedback outperforms minimalist definitions.
• Findings indicate a disparity between non-native speakers' self-assessments and actual pragmatic performance, urging enhanced AI interventions.

Evaluating AI Support for Cross-Cultural Communication of Neologisms

Introduction and Motivation

Cross-cultural informal communication presents unique lexical challenges for non-native speakers (NNS), especially given the rapid emergence of neologisms and slang terms in English online discourse. Traditional learning resources fail to provide timely or context-appropriate support for these expressions, leading users to turn to AI-powered tools. The study, "Reheat Nachos for Dinner? Evaluating AI Support for Cross-Cultural Communication of Neologisms" (2604.23842), conducts a large-scale human-subjects experiment to empirically assess the efficacy of current AI interventions—specifically those based on LLM-generated feedback—for NNS learning and usage of neologisms in informal written interactions with native speakers (NS).

Experimental Design

The research implements a controlled, between-subjects design involving 234 NNS and 144 NS evaluators. The primary task emulates realistic cross-cultural encounters in which NNS participants learn eight selected neologisms appearing in authentic social media posts and subsequently use them in writing messages addressed to a hypothetical NS acquaintance.

Figure 1: Overview of the study workflow, featuring Learning, Production, and Comprehension stages per neologism, and subsequent NS evaluation of NNS outputs.

Participants are randomly assigned one of five support conditions:

• Control: Access to the original post only.
• AI Definition: GPT-4.1-produced dictionary-style definition.
• AI Rewrite: GPT-4.1 paraphrased, simplified version of the original post.
• AI Explanation: GPT-4.1-generated explanation (3–5 sentences) of meaning and usage patterns, including pragmatics, connotations, and typical context.
• Non-AI Dictionary: Full Merriam-Webster dictionary entry.

The tasks—Learning, Production, and Comprehension—are designed to probe communicative competence (as rated by NS evaluators) and NNSs’ ability to judge contextual appropriateness. NS raters assess outputs across well-formedness, contextual appropriateness, and message understandability.

Main Findings

Effectiveness of Support Conditions

The results demonstrate that the AI Explanation condition produces statistically significant gains over both the no-support and other support baselines across every NS-rated metric (well-formedness, contextual appropriateness, understandability):

Figure 2: Comparison of NS-rated communicative competence across support conditions; AI Explanation yields consistently higher ratings.

Despite its comprehensive content, the Non-AI Dictionary condition only outperforms Control on certain dimensions, likely reflecting information overload and elevated cognitive demands. In contrast, basic AI Definition and AI Rewrite conditions do not yield statistically significant improvements over the Control in the primary NS-rated metrics.

NNS Comprehension and Perceptions

Measures of NNS comprehension competence (difference between NNS and NS ratings of contextual appropriateness) reveal that AI-based support does not reliably close the gap: even the best support (AI Explanation) does not lead to significantly smaller distances compared to Control.

Figure 3: Distribution of NNS comprehension distance across support types; no significant reduction with AI-based support.

NNS self-reported confidence and perceived helpfulness are generally inflated compared to actual NS evaluations, indicating that NNS overestimate their communicative success—particularly in conditions that are subjectively easier (such as AI Rewrite), irrespective of objective performance improvements.

Figure 4: NNS self-reported confidence and perceived helpfulness; subjective gains do not always align with actual communicative improvements.

Remaining Gaps and Error Analysis

A clear, statistically significant gap persists between NNS- and NS-produced writing, particularly in contextual appropriateness and pragmatic felicity, with AI support alone unable to bridge this fully. Qualitative analysis reveals frequent misapplication of literal meaning, especially where AI outputs are ambiguous or insufficiently contextualized—e.g., erroneous literal usage of “reheat nachos.”

Sample error annotation and rater interface:

Figure 5: NS annotation interface for error rate analysis in AI-generated support conditions.

Implications for AI-Driven Language Support

The empirical evidence supports the utility of rich, context-sensitive AI explanations over minimalist or purely definitional feedback. However, the analysis indicates that:

• AI-supported learning is sensitive to output quality: NNS lack robust mechanisms to detect or correct imperfect, ambiguous, or context-agnostic AI guidance, amplifying the risk of inappropriate word usage.
• Information density vs. cognitive load: Comprehensive dictionary-derived content may be less accessible in practice, as reflected in user feedback concerning mental burden.
• Self-assessment is unreliable for communicative competence: Overestimation is prevalent, suggesting that self-reported metrics are not suitable proxies for actual pragmatic adequacy.
• Societal and cultural considerations: Over-normalization of neologisms via AI mediation could dilute their role as social markers of in-group membership and shared identity.

From a design perspective, future AI support for L2 sociopragmatic learning should incorporate dynamic uncertainty signaling, provide richer, contrastive usage exemplars (highlighting both successful and failing pragmatic patterns), and combine automatic retrieval of up-to-date examples with user-facing guidance on discourse-level felicity.

Theoretical and Practical Impacts

This study rigorously demonstrates the current limitations and promise of LLM-driven sociopragmatic support tools for NNS. Practically, deploying AI Explanation functionality in educational and cross-cultural communication settings could enhance informal communicative competence in emergent lexical domains. Theoretically, the research clarifies the boundaries of single-turn AI feedback and the need for socioculturally-grounded, context-aware scaffolding—a direction aligned with broader trends in human-centered and adaptive NLP systems.

For future advances, integration of multimodal context retrieval, adaptive uncertainty expression, and community-authored usage examples could offer more reliable mediation of lexical innovation across cultures. The results also underscore the necessity of regular AI retraining to reflect rapidly evolving idiomatic phenomena.

Conclusion

Through a large-scale, ecologically valid experimental framework, this work establishes robust evidence that AI Explanation outperforms other support approaches in enhancing NNS communicative competence with English neologisms. Nonetheless, substantial challenges remain with regard to NNS self-calibration, the detection and correction of AI errors, and the deeper sociolinguistic functions of slang phenomena. These insights should inform ongoing development of AI language support tools and the methodologies for evaluating cross-cultural communicative efficacy in L2 learning contexts.