Trading Inference-Time Compute for Adversarial Robustness (2501.18841v1)

Abstract: We conduct experiments on the impact of increasing inference-time compute in reasoning models (specifically OpenAI o1-preview and o1-mini) on their robustness to adversarial attacks. We find that across a variety of attacks, increased inference-time compute leads to improved robustness. In many cases (with important exceptions), the fraction of model samples where the attack succeeds tends to zero as the amount of test-time compute grows. We perform no adversarial training for the tasks we study, and we increase inference-time compute by simply allowing the models to spend more compute on reasoning, independently of the form of attack. Our results suggest that inference-time compute has the potential to improve adversarial robustness for LLMs. We also explore new attacks directed at reasoning models, as well as settings where inference-time compute does not improve reliability, and speculate on the reasons for these as well as ways to address them.

• The paper shows that increasing inference-time compute consistently reduces adversarial attack success rates across varied threat scenarios.
• It introduces novel paradigms like the 'Think Less' attack to reveal vulnerabilities in reasoning models under limited computational reasoning.
• The research provides practical insights on deploying resilient LLMs by balancing compute allocation between pre-training and inference phases.

Analyzing the Impact of Inference-Time Compute on Adversarial Robustness

The presented research investigates an intriguing aspect of adversarial robustness in the context of LLMs: the potential improvements afforded by scaling inference-time compute. The paper challenges the prevailing paradigm that pre-training efforts provide limited returns in enhancing adversarial robustness by focusing on inference-time adjustments. This work is significant as it offers a new understanding of how reasoning models, specifically variants of OpenAI's o1 family, might better handle adversarial perturbations when given more computational resources during inference.

Key Findings and Methodologies

The paper presents several key findings pertinent to the AI research community:

1. Inference-Time Compute and Robustness: Across various attack vectors, including many-shot attacks, prompt injections, and adversarial soft-token manipulations, increased inference-time compute consistently diminishes the success rate of adversarial engagements. The paper underscores that this robustness is achieved without tailored adversarial training, showcasing a potentially universal defense strategy applicable across different adversarial contexts.
2. Novel Attacks for Reasoning Models: The research introduces novel attack paradigms such as the "Think Less" attack, which manipulates the computational reasoning allowed to the model, demonstrating a new dimension of vulnerability specifically pertinent to reasoning models.
3. Comprehensive Assessment Across Contexts: The paper utilizes a robust array of benchmarks and tests, including mathematical problem solving, language policy adherence, and multimodal challenges with both adversarial and clean images. These contexts offer a well-rounded view of the efficacy of their proposed method.
4. LMP and Human Red-Teaming: The experiments include LLM program (LMP) attacks and human red-teaming to simulate real-world adversarial engagement, providing valuable insights into how models might perform against human ingenuity in exploiting LLM vulnerabilities.

Implications for AI Development

The implications of these results are multifaceted:

• Practical Deployment: For practitioners deploying LLMs in potentially adversarial environments or high-stakes applications, the ability to enhance robustness through inference-time scaling offers a viable pathway to mitigate risks without the preemptive need for exhaustive adversarial training datasets.
• Adversarial Strategy Framework: The research presents a valuable framework for understanding and categorizing adversarial strategies, particularly in reasoning models. This lays the groundwork for future work focused on expanding the taxonomy of attacks and defenses in LLMs.
• Refined Understanding of Compute Utilization: By demonstrating improved adversarial outcomes when increasing inference-time resources, this paper challenges the field to reconsider how computational investments are allocated between pre-training and inference phases.

Future Directions

The research opens several avenues for future exploration:

• Balancing Compute Allocation: Further studies could explore optimal balancing strategies between pre-training and inference compute, especially examining economic trade-offs and performance payoffs across different model architectures.
• Tuning for Robustness in Ambiguous Tasks: While inference-time compute showed promise in precise tasks, ambiguous contexts remain problematic. Future work could delve into specialized strategies to enhance robustness in such scenarios, potentially integrating policy awareness directly into inference calculations.
• Deep-Dive into Attack Innovation: The novel "Think Less" and potential "Nerd Sniping" attacks represent exciting frontier threats that could benefit from deeper exploration, especially in developing models resilient to subtle adversarial influences that exploit model processing preferences and cognitive biases.

Conclusion

This research provides compelling evidence that increased inference-time compute is a broadly effective strategy for enhancing adversarial robustness in reasoning models. By highlighting both the efficacy of this approach and identifying areas where challenges persist, the authors contribute significantly to our understanding of LLM defenses in adversarial settings. This paper not only enriches the academic discourse but also offers practical insights for deploying resilient AI systems in real-world applications where adversarial risks are non-trivial.