Defense Mechanisms for Securing LLM-based Evaluation Pipelines

Design defense mechanisms that secure LLM-as-a-Judge evaluation pipelines against adversarial manipulation throughout the assessment process.

Background

Even when adversarial examples are detected, end-to-end evaluation pipelines may remain exposed if downstream components or aggregation steps can still be influenced.

The paper emphasizes the need for comprehensive, pipeline-level defenses that address threats at multiple stages, ensuring secure and trustworthy automated evaluation.