Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
119 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
43 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Causal Prompting: Debiasing Large Language Model Prompting based on Front-Door Adjustment (2403.02738v3)

Published 5 Mar 2024 in cs.CL

Abstract: Despite the notable advancements of existing prompting methods, such as In-Context Learning and Chain-of-Thought for LLMs, they still face challenges related to various biases. Traditional debiasing methods primarily focus on the model training stage, including approaches based on data augmentation and reweighting, yet they struggle with the complex biases inherent in LLMs. To address such limitations, the causal relationship behind the prompting methods is uncovered using a structural causal model, and a novel causal prompting method based on front-door adjustment is proposed to effectively mitigate LLMs biases. In specific, causal intervention is achieved by designing the prompts without accessing the parameters and logits of LLMs. The chain-of-thought generated by LLM is employed as the mediator variable and the causal effect between input prompts and output answers is calculated through front-door adjustment to mitigate model biases. Moreover, to accurately represent the chain-of-thoughts and estimate the causal effects, contrastive learning is used to fine-tune the encoder of chain-of-thought by aligning its space with that of the LLM. Experimental results show that the proposed causal prompting approach achieves excellent performance across seven natural language processing datasets on both open-source and closed-source LLMs.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (57)
  1. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.
  2. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
  3. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR.
  4. Causal intervention and counterfactual reasoning for multi-modal fake news detection. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 627–638.
  5. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416.
  6. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
  7. Active prompting with chain-of-thought for large language models. arXiv preprint arXiv:2302.12246.
  8. A survey for in-context learning. arXiv preprint arXiv:2301.00234.
  9. Causal inference in natural language processing: Estimation, prediction, interpretation and beyond. Transactions of the Association for Computational Linguistics, 10:1138–1158.
  10. Mitigating label biases for in-context learning. arXiv preprint arXiv:2305.19148.
  11. Demystifying prompts in language models via perplexity estimation. arXiv preprint arXiv:2212.04037.
  12. Counterfactual multihop qa: A cause-effect approach for reducing disconnected reasoning. arXiv preprint arXiv:2210.07138.
  13. Counterfactual multihop QA: A cause-effect approach for reducing disconnected reasoning. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4214–4226, Toronto, Canada. Association for Computational Linguistics.
  14. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199–22213.
  15. Can language models learn from explanations in context? arXiv preprint arXiv:2204.02329.
  16. Crossaug: A contrastive data augmentation method for debiasing fact verification models. In Proceedings of the 30th ACM International Conference on Information amp; Knowledge Management.
  17. On the advance of making language models better reasoners. arXiv preprint arXiv:2206.02336.
  18. What makes good in-context examples for gpt-3333? arXiv preprint arXiv:2101.06804.
  19. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35.
  20. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint arXiv:2104.08786.
  21. End-to-end bias mitigation by modelling biases in corpora. arXiv: Computation and Language,arXiv: Computation and Language.
  22. Active learning principles for in-context learning with large language models. arXiv preprint arXiv:2305.14264.
  23. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint arXiv:2202.12837.
  24. Counterfactual vqa: A cause-effect look at language bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12700–12710.
  25. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114.
  26. OpenAI. 2022. Introducing chatgpt. https://openai.com/blog/chatgpt. Accessed: 2024-02-06.
  27. Judea Pearl. 2019. The seven tools of causal inference, with reflections on machine learning. Communications of the ACM, 62(3):54–60.
  28. Judea Pearl. 2022. Direct and indirect effects. In Probabilistic and causal inference: the works of Judea Pearl, pages 373–392.
  29. Causal inference in statistics: A primer. John Wiley & Sons.
  30. Judea Pearl and Dana Mackenzie. 2018. The book of why: the new science of cause and effect. Basic books.
  31. Judea Pearl et al. 2000. Models, reasoning and inference. Cambridge, UK: CambridgeUniversityPress, 19(2):3.
  32. Semeval-2016 task 5: Aspect based sentiment analysis. In ProWorkshop on Semantic Evaluation (SemEval-2016), pages 19–30. Association for Computational Linguistics.
  33. Learning to retrieve prompts for in-context learning. arXiv preprint arXiv:2112.08633.
  34. Towards debiasing fact verification models. arXiv preprint arXiv:1908.05267.
  35. An information-theoretic approach to prompt engineering without ground truth labels. arXiv preprint arXiv:2203.11364.
  36. Fever: a large-scale dataset for fact extraction and verification. arXiv preprint arXiv:1803.05355.
  37. Debiasing nlu models via causal intervention and counterfactual reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, page 11376–11384.
  38. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.
  39. A causal view of entity bias in (large) language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 15173–15184, Singapore. Association for Computational Linguistics.
  40. Training data is more valuable than you think: A simple and effective method by retrieving from training data. arXiv preprint arXiv:2203.08773.
  41. Textflint: Unified multilingual robustness evaluation toolkit for natural language processing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pages 347–355.
  42. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171.
  43. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
  44. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837.
  45. Jason Wei and Kai Zou. 2019. Eda: Easy data augmentation techniques for boosting performance on text classification tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP).
  46. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426.
  47. Huggingface’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771.
  48. An explanation of in-context learning as implicit bayesian inference. arXiv preprint arXiv:2111.02080.
  49. Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning, pages 2048–2057. PMLR.
  50. Counterfactual debiasing for fact verification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6777–6789, Toronto, Canada. Association for Computational Linguistics.
  51. Deconfounded image captioning: A causal retrospect. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  52. Causal attention for vision-language tasks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9847–9857.
  53. A comprehensive capability analysis of gpt-3 and gpt-3.5 series models. arXiv preprint arXiv:2303.10420.
  54. Ground-truth labels matter: A deeper look into input-label demonstrations. arXiv preprint arXiv:2205.12685.
  55. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning, pages 12697–12706. PMLR.
  56. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625.
  57. Causal intervention for mitigating name bias in machine reading comprehension. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12837–12852.
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (5)
  1. Congzhi Zhang (5 papers)
  2. Linhai Zhang (12 papers)
  3. Deyu Zhou (42 papers)
  4. Jialong Wu (36 papers)
  5. Yulan He (113 papers)
Citations (11)
View on Semantic Scholar

Summary

Causal Prompting: A New Debiasing Method for LLM Prompts Using Front-Door Adjustment

Introduction

The influence of bias in LLMs has shown to be a significant challenge, impacting the reliability of outputs across various NLP tasks. Traditional efforts to debias LLMs during the model training phase via data augmentation or reweighting strategies have faced limitations, particularly in handling the complex, multifaceted nature of bias within these models. This paper proposes a novel approach named "Causal Prompting" that utilizes causal inference—specifically front-door adjustment—to mitigate bias by intervening in the prompt design process without requiring direct access to LLM parameters or output logits.

Debiasing Through Causal Inference

Causal inference offers a robust framework for understanding the relationships between variables within a system. Specifically, this approach leverages the concept of front-door adjustment, enabling the estimation of the causal effect between an input prompt (treatment) and the model's output (outcome) without necessitating the manipulation or direct measurement of confounding variables (unobservable biases in this context). By identifying and utilizing chain-of-thoughts (CoT) generated by LLMs as a mediator variable, Causal Prompting provides a structured method to estimate and mitigate the biasing effects of unobserved confounders.

Methodology

The Causal Prompting approach encompasses two critical stages to estimate and adjust for biased causal effects in prompting LLMs:

  1. Estimation of P(r∣do(X))P(r|do(X)): This phase aims to estimate the causal effect between the input prompt and the CoT. It employs self-consistency and a clustering algorithm where multiple CoTs generated by the LLM are clustered. The center of each cluster acts as the representative CoT with its probability estimated based on cluster size. This step effectively factors in the variations within the CoTs to select those most reflective of unbiased reasoning paths.
  2. Estimation of P(A∣do(r))P(A|do(r)): The second stage calculates the causal effect between the CoT and the final answer. By leveraging an NWGM (normalized weighted geometric mean) approximation, ICL (In-Context Learning) demonstrations are selected based on their relevance to the CoT, serving as a proxy for rigorous counterfactual analysis. This approximation seeks to represent the entire data distribution and, in turn, guide the LLM towards generating unbiased answers.

Furthermore, the methodology incorporates contrastive learning to fine-tune the encoder, aligning the representation space of samples with that of LLMs. This alignment is crucial for accurately estimating causal effects and enhancing the overall debiasing process.

Experimental Results

The efficacy of Causal Prompting was evaluated across three distinct NLP tasks (Aspect-based Sentiment Analysis, Natural Language Inference, and Fact Verification) using both open-source and closed-source LLMs. The approach not only showed significant improvements in performance across adversarial datasets but also demonstrated its applicability across different model architectures.

Implications and Future Directions

The introduction of Causal Prompting presents a scalable, model-agnostic strategy for debiasing LLMs, potentially revolutionizing the way biases are addressed in AI systems. The method's reliance on causal inference, particularly front-door adjustment, fills a critical gap in current debiasing practices, moving beyond the limitations of direct manipulation of training data or model parameters.

Future work may explore the application of Causal Prompting across a wider range of tasks, LLM architectures, and languages. Additionally, further refinement of the methodology, including optimization of the NWGM approximation and clustering mechanisms, could enhance its effectiveness and efficiency. The exploration of other causal inference techniques within the prompting context also presents an exciting avenue for research, potentially unveiling new strategies for mitigating bias in AI.

File Document Download Save Streamline Icon: https://streamlinehq.com PDF File Document Download Save Streamline Icon: https://streamlinehq.com Markdown