Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychology (2402.15690v1)
Abstract: LLMs have gradually become the gateway for people to acquire new knowledge. However, attackers can break the model's security protection ("jail") to access restricted information, which is called "jailbreaking." Previous studies have shown the weakness of current LLMs when confronted with such jailbreaking attacks. Nevertheless, comprehension of the intrinsic decision-making mechanism within the LLMs upon receipt of jailbreak prompts is noticeably lacking. Our research provides a psychological explanation of the jailbreak prompts. Drawing on cognitive consistency theory, we argue that the key to jailbreak is guiding the LLM to achieve cognitive coordination in an erroneous direction. Further, we propose an automatic black-box jailbreaking method based on the Foot-in-the-Door (FITD) technique. This method progressively induces the model to answer harmful questions via multi-step incremental prompts. We instantiated a prototype system to evaluate the jailbreaking effectiveness on 8 advanced LLMs, yielding an average success rate of 83.9%. This study builds a psychological perspective on the explanatory insights into the intrinsic decision-making logic of LLMs.
- 2024. Claude 2. [Online; accessed 22. Jan. 2024].
- 2024a. THUDM/chatglm2-6b ⋅⋅\cdot⋅ Hugging Face. [Online; accessed 10. Feb. 2024].
- 2024b. THUDM/chatglm3-6b ⋅⋅\cdot⋅ Hugging Face. [Online; accessed 10. Feb. 2024].
- Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning.
- Playing repeated games with large language models. ArXiv, abs/2305.16867.
- Constitutional ai: Harmlessness from ai feedback. ArXiv, abs/2212.08073.
- Daryl J. Bem. 1967. Self-perception: An alternative interpretation of cognitive dissonance phenomena. Psychological review, 743:183–200.
- Jailbreaking black box large language models in twenty queries. ArXiv, abs/2310.08419.
- Masterkey: Automated jailbreak across multiple large language model chatbots. In Network and Distributed System Security.
- A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily. ArXiv, abs/2311.08268.
- Leon Festinger. 1957. A theory of cognitive dissonance.
- FlowGPT. 2023. Explore and Browse ChatGPT Prompts on FlowGPT. [Online; accessed 4. May 2023].
- Jonathan L. Freedman and Scott C. Fraser. 1966. Compliance without pressure: the foot-in-the-door technique. Journal of personality and social psychology, 42:195–202.
- Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. ArXiv, abs/2209.07858.
- Wes Gurnee and Max Tegmark. 2023. Language models represent space and time. ArXiv, abs/2310.02207.
- Thilo Hagendorff. 2023. Machine psychology: Investigating emergent capabilities and behavior in large language models using psychological methods. ArXiv, abs/2303.13988.
- Jailbreak. 2023. Jailbreak Chat.
- Wolfgang Köhler. 1943. Gestalt psychology. Psychologische Forschung, 31:XVIII–XXX.
- Open sesame! universal black box jailbreaking of large language models. ArXiv, abs/2309.01446.
- Large language models understand and can be enhanced by emotional stimuli.
- Emergent world representations: Exploring a sequence model trained on a synthetic task. ArXiv, abs/2210.13382.
- Deepinception: Hypnotize large language model to be jailbreaker. ArXiv, abs/2311.03191.
- Autodan: Generating stealthy jailbreak prompts on aligned large language models. ArXiv, abs/2310.04451.
- Jailbreaking chatgpt via prompt engineering: An empirical study. ArXiv, abs/2305.13860.
- Samuel Marks and Max Tegmark. 2023. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. ArXiv, abs/2310.06824.
- Locating and editing factual associations in gpt. In Neural Information Processing Systems.
- Meta. 2023. Llama-2-7b-chat-hf ⋅⋅\cdot⋅ Hugging Face.
- OpenAI. 2023a. Introducing ChatGPT.
- OpenAI. 2023b. OpenAI Moderation API.
- OpenAI. 2023c. Our approach to AI safety.
- Argumentation and persuasion in the cognitive coherence theory. In Comma.
- Machine behaviour. Nature, page 477–486.
- "do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. ArXiv, abs/2308.03825.
- Anirudh V. K. 2023. ChatGPT in Grandma Mode will Spill All Your Secrets. Analytics India Magazine.
- Jailbroken: How does llm safety training fail? ArXiv, abs/2307.02483.
- Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5:1486–1496.
- Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. ArXiv, abs/2309.10253.
- Jade: A linguistic-based safety evaluation platform for llm.
- Judging llm-as-a-judge with mt-bench and chatbot arena.
- Rethinking machine ethics - can llms perform moral reasoning through the lens of moral theories? ArXiv, abs/2308.15399.
- Representation engineering: A top-down approach to ai transparency. ArXiv, abs/2310.01405.
- Universal and transferable adversarial attacks on aligned language models. ArXiv, abs/2307.15043.
- Zhenhua Wang (74 papers)
- Wei Xie (151 papers)
- Baosheng Wang (4 papers)
- Enze Wang (4 papers)
- Zhiwen Gui (2 papers)
- Shuoyoucheng Ma (2 papers)
- Kai Chen (512 papers)