Accent-Based Personalization and Bias Evaluation

Investigate accent-based personalization and bias evaluation for speech large language models, which are not addressed by the present evaluation framework.

Background

The BASIS-Bench evaluation examines accent only within demographic robustness, specifically assessing disparities in speech processing, while excluding accent from its personalization and stereotypical-bias axes. The paper explains that defining appropriate personalization based on accent is difficult because doing so may require assumptions about a speaker’s linguistic background, preferences, or cultural identity.

The unresolved research direction is therefore to develop principled methods and datasets for evaluating both personalization conditioned on accent and bias associated with accent, without presupposing inappropriate assumptions about speakers or their identities. This would extend the benchmark beyond age and gender personalization and bias evaluation.