Emergence of nuanced social perception and behavior generation in training and fine-tuning

Ascertain how abilities in nuanced social perception and social behavior generation emerge in machine learning models during training or fine-tuning, and evaluate the capabilities and limitations of training objectives such as next-token prediction or masking and the influence of tokenization schemes on inducing nuanced social understanding.

Background

The paper notes that it is unclear whether and how fine-grained social understanding arises from common training paradigms. Understanding the emergence of these competencies is essential for designing models that can perceive subtle cues (e.g., timing, vocal nuances) and generate socially appropriate behavior.

The authors specifically call for investigation into the impact of training objectives and tokenization schemes on nuanced social understanding, suggesting that inductive biases may shape performance and that new datasets with annotated fine-grained social signals are needed.