Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering and Manipulating Human Perceptual Variability (2505.03641v1)

Abstract: Human decision-making in cognitive tasks and daily life exhibits considerable variability, shaped by factors such as task difficulty, individual preferences, and personal experiences. Understanding this variability across individuals is essential for uncovering the perceptual and decision-making mechanisms that humans rely on when faced with uncertainty and ambiguity. We present a computational framework BAM (Boundary Alignment & Manipulation framework) that combines perceptual boundary sampling in ANNs and human behavioral experiments to systematically investigate this phenomenon. Our perceptual boundary sampling algorithm generates stimuli along ANN decision boundaries that intrinsically induce significant perceptual variability. The efficacy of these stimuli is empirically validated through large-scale behavioral experiments involving 246 participants across 116,715 trials, culminating in the variMNIST dataset containing 19,943 systematically annotated images. Through personalized model alignment and adversarial generation, we establish a reliable method for simultaneously predicting and manipulating the divergent perceptual decisions of pairs of participants. This work bridges the gap between computational models and human individual difference research, providing new tools for personalized perception analysis.

Evaluating and Manipulating Human Perceptual Variability Through AI-Generated Images

This paper introduces the Boundary Alignment Manipulation (BAM) framework to assess and manipulate human perceptual variability through the generation of synthetic images using artificial neural networks (ANNs). The variability in human decision-making, influenced by factors such as task difficulty, is an essential aspect of cognitive science. While prior work often overlooks perceptual differences in simple tasks like digit classification, this paper aims to leverage ANN-generated stimuli to uncover insights into human perceptual variability.

Methodology

The research employs a perceptual boundary sampling approach, stressing the perceptual boundaries of ANNs—specifically, regions where the classification confidence of ANNs is at its lowest due to ambiguous stimuli. To explore these boundaries effectively, two novel guidance techniques were developed:

1. Uncertainty Guidance: This method targets ANNs to produce elements that drive classification uncertainty, forcing the ANN's confidence across categories to balance, thus enhancing ambiguity.
2. Controversial Guidance: This strategy involves utilizing paired ANNs to generate images that maximize divergences in classification results, thereby fostering dissimilarity in human responses.

These guidance methods were applied within a diffusion model, enriched by classifier guidance, to generate images that more closely adhere to the natural appearance of requested stimuli—mitigating the common pitfall of synthetic images appearing unnaturally.

To empirically support this conceptual model, the paper collected behavioral data from 246 participants across 116,715 trials, culminating in the variMNIST dataset, which included 19,943 images.

Findings and Implications

The BAM framework led to three primary findings:

1. Generation and Validation of High-Variability Stimuli: The stimuli generated from ANN perceptual boundaries via the BAM method successfully induced perceptual variability in human participants, signified by high entropy values in choice probabilities.
2. Predictive Models via Individualized Alignment: The paper demonstrated that fine-tuning ANNs with individual participant behavioral data effectively predicted perceptual variability at both group and individual levels, a notable advancement over baseline ANNs.
3. Effective Manipulation of Human Perception: The manipulative potential of synthesized stimuli was corroborated through the framework's ability to systematically shift perceptual judgments between individuals.

Speculation on AI Implications

This paper's methodologies provide a promising avenue for the felicitation of AI-human alignment, revealing the shared perceptual disturbances between human subjects and computational models. The introduced frameworks could advocate for improved precision in personalized AI deployments, fostering the evolution of AI systems capable of offering tailored solutions in human-centric fields such as cognitive neuroscience, behavioral predictions, and decision-making applications.

Future Directions

Potential future developments could involve exploring cross-cultural variations in perceptual responses, providing more generalized datasets beyond simple image classification (e.g., tasks like emotion recognition or attention studies), and refining metrics for assessing perceptual variability across diverse cognitive tasks. Moreover, integrating the BAM framework with optimal experimental designs could afford further improvements in AI-human alignment, minimizing experimental costs while maximizing perceptual accuracy in tailored settings.

In conclusion, the research bridges computational models and human variability by offering a systematic approach to studying and manipulating perceptual differences through artificial intelligence models, positing the basis for personalized perceptual analysis tools.