Delegated Authorization for Agents Constrained to Semantic Task-to-Scope Matching (2510.26702v1)

Abstract: Authorizing LLM driven agents to dynamically invoke tools and access protected resources introduces significant risks, since current methods for delegating authorization grant overly broad permissions and give access to tools allowing agents to operate beyond the intended task scope. We introduce and assess a delegated authorization model enabling authorization servers to semantically inspect access requests to protected resources, and issue access tokens constrained to the minimal set of scopes necessary for the agents' assigned tasks. Given the unavailability of datasets centered on delegated authorization flows, particularly including both semantically appropriate and inappropriate scope requests for a given task, we introduce ASTRA, a dataset and data generation pipeline for benchmarking semantic matching between tasks and scopes. Our experiments show both the potential and current limitations of model-based matching, particularly as the number of scopes needed for task completion increases. Our results highlight the need for further research into semantic matching techniques enabling intent-aware authorization for multi-agent and tool-augmented applications, including fine-grained control, such as Task-Based Access Control (TBAC).

• The paper introduces a delegated authorization architecture that uses a trusted proxy and NLP-based semantic matching to ensure agents access only the scopes justified by user intent.
• Evaluation shows that while LLM-ResM outperforms SemSimM on single-tool tasks, accuracy declines in multi-tool scenarios, highlighting a trade-off between over-scoping and under-scoping.
• The integration of semantic matching into OAuth flows enhances security and scalability, providing a pathway to implement fine-grained, intent-aware access control for LLM-driven systems.

Delegated Authorization for Agents Constrained to Semantic Task-to-Scope Matching

Introduction and Motivation

The proliferation of LLM-driven agents capable of dynamic tool invocation and resource access has introduced significant security and compliance risks in enterprise environments. Traditional delegated authorization mechanisms, such as OAuth 2.0/2.1, are ill-suited for these agentic workflows because they grant permissions based on static, preconfigured scopes, without visibility into the original user intent. This disconnect enables agents to request and receive access to resources beyond the scope of their assigned tasks, either inadvertently or maliciously, thereby expanding the attack surface and undermining least-privilege principles.

Figure 1: Delegated authorization with semantic task-to-scope matching counters against attempts to access additional protected resources.

The paper introduces a delegated authorization architecture that integrates semantic task-to-scope matching, enabling authorization servers to inspect the original natural language task and constrain access tokens to the minimal set of scopes required for task fulfillment. This approach is operationalized via a trusted proxy that captures the user's intent and mediates all authorization flows, ensuring that the agent's access requests are semantically aligned with the original task.

System Architecture and Delegated Authorization Flow

The proposed architecture extends the standard delegated authorization flow by introducing a trusted proxy and augmenting the authorization server with semantic inspection capabilities. The trusted proxy intercepts the user's natural language prompt, attaches contextual metadata, and relays it to both the agent and the authorization server. All subsequent authorization requests from the agent are routed through this proxy, ensuring that the original intent is available for semantic comparison at the point of access control.

Figure 2: Proxied delegated authorization enabling trusted semantic matching between task and scope requests.

The authorization server employs an NLP-based semantic matching module to compare the user's task description with the agent's requested scopes (i.e., tool or resource access). Only those scopes that are semantically justified by the original task are granted, and permissions are provisioned dynamically and revoked upon task completion. This just-in-time, least-privilege model mitigates the risk of privilege escalation and resource overreach.

The architecture is designed for scalability by segmenting the authorization infrastructure into lightweight, virtualized AuthZ servers, each dedicated to a specific agentic application. This segmentation reduces scope bloat, improves manageability, and constrains the blast radius of any given agent.

Figure 3: Delegated Authorization Constrained to Semantic Task-to-Scope Matching.

Semantic Task-to-Scope Matching Techniques

The core technical challenge is the semantic alignment of tasks and scopes. The paper evaluates two approaches for this matching:

1. Semantic Similarity Matcher (SemSimM): Utilizes LLM embeddings to compute the similarity between an LLM-generated ideal tool description (given the task) and the descriptions of available tools. If the similarity exceeds a threshold, the tool is deemed appropriate. This method is effective for single-tool tasks but does not scale to multi-tool workflows and is sensitive to the quality of tool descriptions.
2. LLM Reasoning Matcher (LLM-ResM): Leverages a LLM to directly reason about the appropriateness of a requested tool for a given task, based on the task context and tool metadata. This approach is scalable, as it evaluates each tool-task pair independently, and is better suited for complex, multi-tool scenarios.

   Figure 4: Semantic task-to-scope matching using SemSimM and LLM-ResM in the AuthZ server.

Dataset and Experimental Methodology

To address the lack of datasets for benchmarking semantic task-to-scope matching, the authors introduce ASTRA, a synthetic dataset and data generation pipeline. ASTRA comprises tasks generated from real-world MCP servers, with both correct and incorrect tool-scope matches, supporting tasks requiring one, two, or three tools. The dataset is constructed to ensure semantic realism and diversity, with explicit simulation of both wrong and null matches.

The evaluation also includes a processed subset of the Toucan dataset, which provides a large-scale, multi-tool agentic task corpus. The experiments assess matcher performance on both datasets, focusing on single-tool and multi-tool scenarios.

Experimental Results

Single-Tool Tasks

LLM-ResM consistently outperforms SemSimM across all datasets, achieving higher accuracy, recall, and F1 scores. SemSimM exhibits high precision but low recall, indicating a conservative bias that rejects many valid tool requests. On the ASTRA test set, LLM-ResM achieves 96% accuracy and 99% recall, while SemSimM achieves 77% accuracy and 55% recall.

Multi-Tool Tasks

For tasks requiring multiple tools, only LLM-ResM is evaluated. Performance degrades as task complexity increases: for two-tool tasks, LLM-ResM achieves 86% accuracy and 82% recall on ASTRA, but for three-tool tasks, accuracy drops to 72% and recall to 57%. On the Toucan dataset, recall for three-tool tasks is higher (86%), attributed to more explicit tool-task mappings in Toucan compared to the more implicit, realistic tasks in ASTRA.

Figure 5: Trade off between under-scoping and over-scoping for tasks requiring one, two or three tools across both our dataset and Toucan dataset.

The results highlight a fundamental trade-off: minimizing over-scoping (false positives) increases the risk of under-scoping (false negatives), especially as the number of required tools grows. For three-tool tasks, the false negative rate reaches 0.78 on ASTRA, indicating significant under-scoping risk in complex scenarios.

Implementation Considerations

• Integration: The architecture can be integrated with existing OAuth 2.0/2.1 flows by introducing a trusted proxy and augmenting the authorization server with an LLM-based semantic matching module.
• Model Selection: LLM-ResM requires access to a high-quality LLM (e.g., GPT-4o) for real-time inference. Embedding-based approaches (SemSimM) are less computationally intensive but less effective for multi-tool tasks.
• Scalability: The segmentation of AuthZ servers and the stateless nature of the semantic matching module support horizontal scaling. However, LLM inference latency and cost may be a bottleneck in high-throughput environments.
• Security: The trusted proxy must be secured against compromise, as it mediates all authorization flows and holds sensitive context.
• Dataset Availability: ASTRA is publicly available, enabling reproducible benchmarking and further research.

Practical and Theoretical Implications

The proposed architecture advances the state of delegated authorization by enabling intent-aware, fine-grained access control for agentic applications. It operationalizes Task-Based Access Control (TBAC) in dynamic, multi-agent environments, addressing the limitations of RBAC, ABAC, and ReBAC in the context of LLM-driven workflows.

The results demonstrate that LLM-based semantic matching is effective for constraining agent permissions, but accuracy degrades with task complexity and implicit tool requirements. This exposes a new axis of risk: under-scoping, where agents are denied access to necessary tools, potentially impairing task completion and user experience.

Theoretically, the work highlights the need for more robust semantic alignment techniques, especially for multi-turn, multi-agent, and adversarial scenarios. The combinatorial explosion of possible tool combinations in large organizations remains a challenge for both policy management and semantic matching accuracy.

Future Directions

• Multi-turn and Multi-agent Workflows: Extending semantic matching to conversational and chained agent scenarios, preserving context and enforcing scopes across delegation chains.
• Lightweight Models: Developing and fine-tuning smaller, open-source LLMs for task-tool matching to reduce inference cost and latency.
• Adversarial Robustness: Simulating and defending against adversarial agents that attempt to circumvent semantic matching.
• Protocol Standardization: Integrating semantic metadata and matching into widely adopted authorization protocols (e.g., OAuth, OIDC).
• Tool Description Quality: Improving the quality and granularity of tool metadata to enhance semantic matching performance.

Conclusion

This work presents a delegated authorization architecture for LLM-driven agents that constrains access to protected resources via semantic task-to-scope matching. By introducing a trusted proxy and augmenting the authorization server with LLM-based semantic inspection, the system enforces least-privilege, intent-aware access control. The ASTRA dataset and experimental results demonstrate the promise and current limitations of semantic matching, particularly as task complexity increases. Future research should focus on improving matcher robustness, supporting complex agentic workflows, and integrating these mechanisms into standard authorization protocols to enable secure, scalable deployment of tool-augmented AI agents.