Privacy leakage and private-information extraction in reinforcement-learning split learning

Determine how to formally define the privacy leakage of reward models in reinforcement-learning-based split learning and how an adversary could effectively extract private information from those reward models.

Background

The paper studies gradient-matching attacks and defenses in label-shielded split learning for supervised fine-tuning of LLMs, where private labels can leak through gradients exchanged at the split interface. Its proposed Gradient Mirage defense relies on disrupting the correspondence between the training objective and the exposed gradient.

The authors note that reinforcement-learning-based split learning differs fundamentally from supervised learning because it does not rely on explicit token-level labels. Consequently, the appropriate notion of privacy leakage for reward models, and the means by which an adversary might recover private information from them, remain unresolved research questions beyond the paper’s evaluated setting.