Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence (2305.03010v1)

Abstract: Sentence-level representations are beneficial for various natural language processing tasks. It is commonly believed that vector representations can capture rich linguistic properties. Currently, LLMs (LMs) achieve state-of-the-art performance on sentence embedding. However, some recent works suggest that vector representations from LMs can cause information leakage. In this work, we further investigate the information leakage issue and propose a generative embedding inversion attack (GEIA) that aims to reconstruct input sequences based only on their sentence embeddings. Given the black-box access to a LLM, we treat sentence embeddings as initial tokens' representations and train or fine-tune a powerful decoder model to decode the whole sequences directly. We conduct extensive experiments to demonstrate that our generative inversion attack outperforms previous embedding inversion attacks in classification metrics and generates coherent and contextually similar sentences as the original inputs.

An Expert Review of "Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence"

The paper "Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence" by Haoran Li, Mingshi Xu, and Yangqiu Song presents a compelling investigation into the potential privacy vulnerabilities associated with sentence embeddings produced by large pre-trained LLMs (LMs). The research introduces a novel generative embedding inversion attack (GEIA) to reconstruct original sentences from their embeddings, challenging existing perceptions about the security of embedding models.

Summary of Core Contributions

The authors argue that current embedding inversion attacks, which primarily focus on reconstructing partial keywords or unordered sets of words, inadequately address the potential information leakage from sentence embeddings. The primary contribution of this paper is the proposal and development of GEIA, which treats embedding inversion as a sequence generation problem rather than a classification problem. This allows for the reconstruction of ordered, coherent sequences that maintain high contextual similarity with the original inputs.

GEIA is designed to be a flexible and adaptive attack that can be applied to a range of popular LM-based sentence embedding models, including Sentence-BERT, SimCSE, Sentence-T5, and MPNet. By leveraging powerful decoder models accessed in a black-box manner, GEIA effectively decodes entire sentences from their embeddings.

Key Findings and Results

The paper presents extensive experimental evaluations to compare the performance of GEIA against traditional multi-label classification (MLC) and multi-set prediction (MSP) methods. These evaluations span diverse datasets, such as PersonaChat and QNLI, offering insights into the effectiveness of the attacks across different domains and data types. The notable findings include:

• Superior Performance: GEIA consistently outperforms MLC and MSP in classification metrics, achieving higher precision, recall, and F1 scores.
• Recovery of Informative Tokens: Unlike prior techniques, which tend to recover mostly stop words, GEIA successfully retrieves significant informative content, including named entities, indicative of its potential to breach sensitive data.
• Generation Metrics: The generative approach showcased impressive results in terms of ROUGE, BLEU, and embedding similarity scores, which measure the syntactic and semantic fidelity of reconstructed sentences compared to original inputs.
• Privacy Implications: The paper underscores that state-of-the-art embedding models are not immune to information leakage, thereby necessitating reconsideration of their deployment in privacy-sensitive environments.

Implications and Future Directions

This paper contributes a crucial perspective to the discourse on privacy risks associated with sentence embeddings, emphasizing the need for robust privacy-preserving techniques in NLP systems. The demonstrated vulnerability points towards the financial and legal ramifications if sensitive information is inadvertently disclosed through embedding models, particularly in domains like legal, medical, and financial services.

Future research avenues should focus on developing effective methods to mitigate such vulnerabilities, potentially through more sophisticated privacy-preserving mechanisms or modifications in embedding strategies. Furthermore, expanding the generative inversion framework to encompass more varied models and exploring its adaptability in evolving AI environments could further elucidate the scope and depth of information leakage risks.

In conclusion, this paper provides a significant step forward in understanding and addressing privacy issues in NLP, highlighting the necessity for ongoing vigilance and innovation in safeguarding sensitive data within our expanding digital landscape.