Many-Tier Instruction Hierarchy in LLM Agents

Abstract: LLM agents receive instructions from many sources-system messages, user prompts, tool outputs, other agents, and more-each carrying different levels of trust and authority. When these instructions conflict, agents must reliably follow the highest-privilege instruction to remain safe and effective. The dominant paradigm, instruction hierarchy (IH), assumes a fixed, small set of privilege levels (typically fewer than five) defined by rigid role labels (e.g., system > user). This is inadequate for real-world agentic settings, where conflicts can arise across far more sources and contexts. In this work, we propose Many-Tier Instruction Hierarchy (ManyIH), a paradigm for resolving instruction conflicts among instructions with arbitrarily many privilege levels. We introduce ManyIH-Bench, the first benchmark for ManyIH. ManyIH-Bench requires models to navigate up to 12 levels of conflicting instructions with varying privileges, comprising 853 agentic tasks (427 coding and 426 instruction-following). ManyIH-Bench composes constraints developed by LLMs and verified by humans to create realistic and difficult test cases spanning 46 real-world agents. Our experiments show that even the current frontier models perform poorly (~40% accuracy) when instruction conflict scales. This work underscores the urgent need for methods that explicitly target fine-grained, scalable instruction conflict resolution in agentic settings.

• The paper introduces ManyIH, a paradigm that enables arbitrarily granular privilege tiers in LLM agents to overcome fixed-tier limitations.
• It employs a Privilege Prompt Interface with ordinal and scalar representations to resolve intra-message conflicts dynamically.
• Benchmark results reveal significant accuracy drops in current LLMs with increased privilege tiers, highlighting critical safety and robustness issues.

Many-Tier Instruction Hierarchy: Fine-Grained Privilege Resolution in LLM Agents

Motivation and Limitations of Standard Instruction Hierarchy

LLM agents deployed in agentic systems routinely face instructions from multiple, heterogeneous sources—system prompts, users, tool responses, other agents, etc.—each with different trust levels and authority. The prevailing approach for resolving instruction conflicts is to employ an Instruction Hierarchy (IH): a fixed, discrete set of privilege levels (typically ≤5), represented via role labels and enforced in templates and training protocols. This design is effective in scenarios like standard chat, but is fundamentally inadequate for complex, real-world multi-agent and tool-augmented settings where the origin and priority of instructions can be arbitrarily granular and may dynamically shift at runtime.

In such cases, rigid, role-based IH induces a fixed- and few-tier bottleneck: it cannot discriminate between fine-grained trust differentials, e.g., among sub-agents with differing roles, tools of varying trustworthiness, or user messages with distinct organizational rank. This granularity constraint restricts both the expressivity and the safety of LLM deployments: agents cannot resolve conflicts when more than a handful of privilege sources exist, which undermines both alignment and robustness to compound prompt injection and privilege escalation attacks.

Many-Tier Instruction Hierarchy: Paradigm and Mechanism

To eliminate the limitations of fixed-tier IH, the paper proposes the Many-Tier Instruction Hierarchy (ManyIH) paradigm. Rather than encoding privilege via hardcoded role labels, ManyIH introduces a dedicated Privilege Prompt Interface (PPI) that allows the privilege level of every instruction to be specified explicitly and arbitrarily, and at inference time.

Figure 1: Overview of Many-Tier Instruction Hierarchy compared with existing IH, enabling definition of arbitrary privilege levels at inference through a Privilege Prompt Interface.

Two PPI variants are instantiated:

• Ordinal PPI: Instructions are bracketed by privilege numerals, with lower numbers denoting higher authority. Conflict resolution is then a comparison of ordinal values.
• Scalar PPI: Each instruction is tagged with a scalar privilege (e.g., $z = 82$), and conflicts are resolved by maximizing privilege value.

This decoupling of privilege from message role semantics enables ManyIH to support:

• Arbitrarily many privilege tiers—dynamically instantiated per prompt.
• Privilege defined at the level of any instruction, including intra-message conflicts.

Privilege values are assumed provided externally at inference, reflecting trust levels set collaboratively by developers and deployers, but ManyIH itself is agnostic to the provenance or assignment mechanism.

ManyIH-Bench: Benchmarking Scalable Privilege Resolution

To evaluate ManyIH, the authors develop ManyIH-Bench: the first benchmark designed for scalable IH with up to 12 privilege levels per instance, far exceeding prior benchmarks limited to 2–3 tiers. ManyIH-Bench comprises 853 tasks across two domains:

• Coding: Python programming tasks from MBPP, augmented with 12 conflicting style constraints (e.g., naming, indentation, quoting), each tagged with a distinct privilege. Models must both pass functional unit tests and comply with the highest privilege constraint within each style group.
• Instruction-Following: Agentic tasks from AgentIF, with up to 12 constraints per prompt, again with privilege-based conflicts and programmatic or LLM-judge verification.

The benchmark construction ensures: (1) non-adversarial but realistic prompts, (2) deterministic constraint verification, (3) controllable scaling of hierarchy depth independent of instruction difficulty, and (4) diverse, real-world agentic scenarios.

Experimental Evaluation and Results

Aggregate Results

Ten proprietary and open-source LLMs, including top-tier models (Gemini 3.1 Pro, GPT-5.4, Claude Opus 4.6), are evaluated on ManyIH-Bench under high-effort, deterministic conditions.

Figure 2: Left: overall accuracy on ManyIH-Bench; Right: accuracy by subset—frontier and open-source models uniformly underperform on ManyIH.

Key findings:

• The best model (Gemini 3.1 Pro) achieves only 42.7% accuracy on ManyIH-Bench.
• GPT-5.4, despite reporting >99% accuracy on standard two-tier IH tasks, reaches only 39.5% here.
• As the number of privilege tiers rises (from 6 to 12), accuracy drops monotonically; degradation per model can exceed 24%.

These results demonstrate that the transition from few-tier to scalable IH is nontrivial and that current models lack robust, generalized mechanisms for fine-grained privilege reasoning.

Sensitivity to Prompt Format and Privilege Encoding

Experiments show LLMs are highly sensitive to the privilege representation:

• Shifting from ordinal to scalar PPI reduces accuracy for GPT-5.4 and Opus 4.6 by over 8%.
• Small perturbations to privilege values, even when preserving relative orderings, produce up to 17% per-sample decision flips.
• Thus, privilege-based reasoning is not reliably invariant to superficial prompt changes, revealing a representation brittleness.

Reasoning Behavior: Chain-of-Thought and Effort

Figure 3: Distribution of CoT length (tokens) for different models, revealing a spectrum of reasoning verbosity and style.

• Some LLMs (Qwen 3.5-397B, Kimi K2.5) generate verbose reasoning traces (median 7K tokens), while others (Claude, GPT) are concise (~1K tokens).
• Length of CoT tracing does not correlate positively with ManyIH performance—conciseness is not a barrier to correct reasoning.
• Increasing explicit reasoning effort (e.g., higher CoT, more stepwise thinking) improves accuracy for GPT-5.4, but saturates well below ceiling.

Implications, Theoretical and Practical

The results reveal that current LLM architectures and training paradigms, optimized for fixed-template or limited-role IH, generalize poorly to complex, real-world agentic deployments where privilege conflicts are granular, dynamic, and context-specific.

For practical safety: Deployed LLM agents in enterprise, research, or multi-user environments cannot be reliably aligned to nuanced privilege structures solely via current role-based IH methods, exposing them to indirect prompt injection and privilege escalation beyond the “system vs. user” context.

On the theoretical front: The inability of state-of-the-art LLMs to parse and obey arbitrarily specified privilege dynamics, even with explicit formatting, highlights a critical architectural and training bottleneck in scalable alignment and controllable agentic reasoning.

The ManyIH paradigm thus sets a design requirement for future agentic LLMs:

• Models must be optimized for privilege-robustness and reasoning invariance to privilege encoding format.
• Scalable, dynamic privilege assignment requires new alignment curricula, possibly architectural modifications (e.g., privilege-segmented attention), and benchmarks targeting these skills.

The ManyIH-Bench benchmark provides an actionable, fine-grained suite for evaluating progress on scalable IH reasoning, decoupled from adversarial attack resistance.

Conclusion

Many-Tier Instruction Hierarchy in LLM Agents (2604.09443) establishes that contemporary instruction hierarchy approaches are inadequate for real-world, complex agent deployments requiring documentable, scalable, fine-grained privilege resolution. ManyIH and ManyIH-Bench present a challenging new frontier: top models perform at or below 40% with accuracy falling as instruction complexity increases, and models are surprisingly brittle to representation. Resolving these deficits necessitates research into privilege-robust inference, prompt-invariant alignment, and architectural innovation.