Survey of Cultural Awareness in Language Models: Text and Beyond (2411.00860v1)

Abstract: Large-scale deployment of LLMs in various applications, such as chatbots and virtual assistants, requires LLMs to be culturally sensitive to the user to ensure inclusivity. Culture has been widely studied in psychology and anthropology, and there has been a recent surge in research on making LLMs more culturally inclusive in LLMs that goes beyond multilinguality and builds on findings from psychology and anthropology. In this paper, we survey efforts towards incorporating cultural awareness into text-based and multimodal LLMs. We start by defining cultural awareness in LLMs, taking the definitions of culture from anthropology and psychology as a point of departure. We then examine methodologies adopted for creating cross-cultural datasets, strategies for cultural inclusion in downstream tasks, and methodologies that have been used for benchmarking cultural awareness in LLMs. Further, we discuss the ethical implications of cultural alignment, the role of Human-Computer Interaction in driving cultural inclusion in LLMs, and the role of cultural alignment in driving social science research. We finally provide pointers to future research based on our findings about gaps in the literature.

Survey of Cultural Awareness in LLMs: Text and Beyond

In the ongoing discourse concerning inclusivity and diversity in technology, cultural awareness in LLMs has become a focal point, especially as these models are deployed across diverse applications like chatbots and virtual assistants. The paper "Survey of Cultural Awareness in LLMs: Text and Beyond" by Pawar et al. provides an in-depth examination of this critical subject, scrutinizing how these models incorporate cultural nuances and sensitivities.

The paper embarks on its exploration by defining cultural awareness in LLMs, drawing insights from cultural understandings in psychology and anthropology. These disciplines offer a dual perspective: anthropology, focusing on the contextual understanding of human action, and psychology, emphasizing the sociocultural grounding of behavior. The authors suggest cultural awareness in LLMs should encapsulate the ability to understand diverse social contexts and variably interpret task elements across cultures, extending beyond spontaneous multilinguality.

In evaluating the methodologies utilized for integrating culture into LLMs, the paper distinguishes between data collection strategies and LLM adaptations. The methodologies leverage both automatic pipelines, such as large-scale web scraping from culturally marked sources, and manual data creation, which involves human annotators to ensure cultural precision. Noteworthy is the emergence of automatic and model-in-the-loop refinement techniques, which have increased the scale and specificity of cultural datasets. These include culture-specific corpora for languages like Korean, Arabic, and others, whose creation was previously hindered by resource constraints.

Cultural alignment in LLMs is achieved primarily through pre-training and fine-tuning approaches, encompassing both model training and prompting methods. Training methods include pre-training from scratch using culturally relevant data, while fine-tuning leverages instructions and specific datasets to align closely with cultural norms, practices, and shared values. On the other hand, prompting strategies, free from additional training, enable models to dynamically adjust to cultural cues in the input text, enhancing their adaptability across diverse cultural contexts.

The evaluation of these models is meticulously structured, utilizing benchmarks encompassing commonsense knowledge, social values, norms, biases, and emotional undertones across cultures. These benchmarks not only assess LLMs’ adaptability but also highlight discrepancies in model outputs across various cultural scenarios, necessitating more culturally aware data handling and bias reduction strategies.

The implications of this research paper are multifaceted. Practically, culturally aware LLMs hold the promise of reducing cultural biases and enhancing user interaction quality across global contexts. Theoretically, they pave the way for more equitable and inclusive AI systems that acknowledge and respect the rich tapestry of global cultural heritage. Future iterations of these models could potentially harness multilayered cultural data, allowing them to adapt dynamically to cultural shifts and more accurately align with the intricate web of human values, norms, and social expectations.

This survey positions itself at the unique intersection of NLP, multimodality (including vision and audio), and social sciences, emphasizing the intricate role human-computer interaction plays in evolving these systems. Given the rapid technological advancements and the expanding scope of AI applications, the paper argues for a paradigmatic shift toward culturally inclusive language technologies that do not merely incorporate multilinguality but also embed deep cultural competence. As researchers, this calls for a continued focus on dataset diversity, model adaptability, and ethical deployment strategies to fulfill the societal discourse of equitable technology development.