SafeInstructTool: Pre-Execution Safety

• SafeInstructTool is a risk assessment framework that proactively evaluates tool-use safety by analyzing user instructions, tool properties, and their interaction.
• It uses a modular nine-dimension scoring system combining precomputed API scores with instruction analysis to preemptively block hazardous execution plans.
• The framework shows significant improvements across diverse models, marking a shift from reactive inspections to prospective, actionable risk mitigation.

SafeInstructTool is a risk assessment and mitigation framework for the prospective safety of tool utilization with LLMs. It is introduced together with SafeToolBench to shift evaluation from retrospective inspection after tool execution to pre-execution risk analysis, with the explicit aim of preventing harms such as financial loss, privacy leakage, or physical injury. Its central design principle is that tool-use risk should be modeled simultaneously from the perspectives of the user instruction, the tool itself, and the interaction between the instruction and the tool, yielding a nine-dimension scoring framework for blocking or escalating risky execution plans before they are run (Xia et al., 9 Sep 2025).

1. Conceptual setting and benchmark context

SafeInstructTool was proposed in a setting where LLMs had already shown strong performance in autonomously calling tools in external environments, but where existing safety evaluations largely remained retrospective. The motivating claim is that retrospective evaluation is insufficient when tool execution can produce irreversible consequences. SafeToolBench was therefore introduced as a benchmark for prospective evaluation, and SafeInstructTool as the corresponding mitigation framework (Xia et al., 9 Sep 2025).

SafeToolBench contains 1200 adversarial user instructions spanning 16 real-world application domains and four overarching risk categories. Within this benchmark setting, SafeInstructTool is not merely a detector of malicious phrasing. It is presented as a framework for analyzing the full risk structure of tool use before execution, including risks embedded in tool APIs and risks that emerge only when a particular instruction is paired with a particular tool call (Xia et al., 9 Sep 2025).

This framing is significant because it treats tool-use safety as a planning problem rather than only a content-moderation problem. A plausible implication is that the framework targets failure modes missed by approaches that only inspect the user query or only audit outputs after the tool has already run.

2. Three-perspective architecture and the nine dimensions

SafeInstructTool organizes pre-execution risk into three perspectives and nine detailed dimensions. The three perspectives are the User Instruction Perspective, the Tool Itself Perspective, and the Joint Instruction-Tool Perspective (Xia et al., 9 Sep 2025).

Perspective	Dimensions
User Instruction	Data Sensitivity; Harmfulness of the Instruction; Urgency of the Instruction; Frequency of Tool Utilization
Tool Itself	Key Sensitivity; Type of Operation; Impact Scope of the Operation
Joint Instruction-Tool	Alignment Between Instruction and Tool; Value Sensitivity

The User Instruction Perspective asks what risks are present in the instruction itself. Its four dimensions capture whether the instruction references sensitive data, whether it expresses harmful or malicious intent, whether it pressures immediate execution, and whether the rate of requested tool use is suspiciously high. The inclusion of urgency and frequency means the framework does not reduce instruction risk to semantic harmfulness alone (Xia et al., 9 Sep 2025).

The Tool Itself Perspective captures risks inherent to the tool or API. Key Sensitivity examines whether parameters require or handle sensitive information. Type of Operation distinguishes intrinsically safer operations from irreversible ones such as delete or transfer money. Impact Scope of the Operation measures how broadly misuse or error could propagate (Xia et al., 9 Sep 2025).

The Joint Instruction-Tool Perspective is the interaction layer. Alignment Between Instruction and Tool asks whether the requested use is consistent with the tool’s intended function or instead exploits it to bypass safeguards. Value Sensitivity addresses cases in which the specific tool output or action, under the provided parameters, may violate societal values, ethics, or laws. The paper reports that the largest performance gap is seen in handling risks that arise from this joint perspective, and that most errors are due to failures in the Joint Instruction-Tool perspective, indicating that cross-perspective interaction is the hardest part of the task (Xia et al., 9 Sep 2025).

3. Scoring procedure and execution-plan risk

SafeInstructTool operationalizes its nine dimensions with a modular, compositional scoring process. The procedure begins by precomputing tool or API risk scores with an LLM-powered judge and storing them in an API Safety Database. For the $m^{\text{th}}$ API in the $i^{\text{th}}$ app, the three tool-perspective scores are defined as

$$\{t^1_{im}, t^2_{im}, t^3_{im}\} = \mathcal{M}(a^m_i),$$

and aggregated as

$\mathcal{T}_{im} = \sum_{n=1}^{3} t^n_{im}.$

These stored scores enable real-time lookup during risk assessment (Xia et al., 9 Sep 2025).

Given an input instruction $q$, the framework computes four instruction-side scores:

$\{u^1, u^2, u^3, u^4\} = \mathcal{M}(q),$

with aggregate instruction risk

$\mathcal{U} = \sum_{n=1}^{4} u^n.$

This explicitly separates instruction analysis from tool analysis rather than collapsing them into a single prompt-level judgment (Xia et al., 9 Sep 2025).

For each anticipated tool call $l^m_i$ in the planned sequence $p$, the framework then computes the two joint scores:

$\{c^1_{im}, c^2_{im}\} = \mathcal{M}(l^m_i),$

and

$i^{\text{th}}$0

The overall execution-plan risk is finally defined as

$i^{\text{th}}$1

If $i^{\text{th}}$2 exceeds a calibrated threshold $i^{\text{th}}$3, experimentally set often to $i^{\text{th}}$4, the tool calls are flagged as high risk and are blocked or routed for further confirmation before execution (Xia et al., 9 Sep 2025).

This aggregation rule is notable because it combines a global instruction-level score with the maximum local tool-plus-joint risk in the planned sequence. A plausible implication is that one especially dangerous step can dominate the plan-level decision even if other steps appear benign.

4. Experimental protocol and reported gains

The reported evaluation compares SafeInstructTool with four alternative methods: None, Simple Prompt, CoT, and Self-Consistency. The metric is Safety score $i^{\text{th}}$5, defined as the percentage of risky instructions correctly identified or blocked. The experiments reported results for Llama3.1-8B-Instruct, Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, Qwen2.5-32B-Instruct, GPT-3.5, and GPT-4o (Xia et al., 9 Sep 2025).

Model	Best Baseline	SafeInstructTool	Improvement
Llama3.1-8B-Instruct	29.2%	44.0%	+14.8%
Qwen2.5-7B-Instruct	48.5%	54.3%	+5.8%
Qwen2.5-14B-Instruct	51.4%	60.2%	+8.8%
Qwen2.5-32B-Instruct	56.5%	71.2%	+14.7%
GPT-3.5	50.1%	56.7%	+6.6%
GPT-4o	71.7%	80.7%	+9.0%

The framework is reported to outperform all baselines across all listed models and across all four risk categories. The improvement is described as more pronounced for open-source models and for more complex, multi-app scenarios. The ablation findings further state that all three perspectives are crucial, and that dropping any one perspective leads to significant performance degradation, with especially strong effects from removing the User Instruction and Tool Itself perspectives (Xia et al., 9 Sep 2025).

The diagnostic analysis reports that 47% of errors are due to failures in the Joint Instruction-Tool perspective. Traditional high-risk domains such as finance and healthcare remain challenging, and some implicit risks, especially those requiring causal or context-based inference, remain difficult even for top LLMs (Xia et al., 9 Sep 2025).

5. Interpretation, strengths, and stated limitations

The paper characterizes SafeInstructTool as holistic, preemptive, modular and explainable, automatable, and scalable across domains/tools. Holistic refers to its coverage of all three sources of risk: the user, the tool, and their interaction. Preemptive refers to risk analysis before action execution. Modularity and explainability derive from the explicit fine-grained scores assigned to the nine dimensions. Automatable reflects the LLM-driven scoring and the API Safety Database. Scalability follows from precomputing tool risk and extending the database as new APIs appear (Xia et al., 9 Sep 2025).

The framework is also positioned as advancing tool-use safety by moving beyond approaches that inspect only surface cues in user instructions or that evaluate safety only after execution. In that sense, SafeInstructTool is a structured decision framework for execution planning rather than a purely reactive content filter (Xia et al., 9 Sep 2025).

The stated limitation is that the method does not currently model user-specific risks, described as personalization issues such as allergies. This matters because some high-stakes tool decisions depend not only on generic risk categories but also on individual constraints that are not encoded in the current nine dimensions (Xia et al., 9 Sep 2025).

6. Relation to adjacent research on safe tool use

SafeInstructTool belongs to a broader line of work that treats tool-use safety as a systems problem rather than only a response-alignment problem. Within that landscape, it is specifically a prospective scoring framework for pre-execution decision-making (Xia et al., 9 Sep 2025).

Other work addresses related goals with different operational abstractions. Task Shield enforces task alignment at test time by requiring each instruction and tool call to contribute to user-specified goals, reporting an attack success rate of 2.07% and utility under attack of 69.79% on GPT-4o in AgentDojo (Jia et al., 2024). ToolSafe focuses on step-level tool invocation safety through TS-Bench, TS-Guard, and TS-Flow, and reports a 65 percent average reduction in harmful tool invocations together with approximately 10 percent improvement in benign task completion under prompt injection attacks (Mou et al., 15 Jan 2026). AgentTrust places a runtime interception layer between agent and tool, returning verdicts of allow, warn, block, or review, and reports 95.0% verdict accuracy on a 300-scenario internal benchmark at low-millisecond latency (Yang, 6 May 2026).

A different branch of work emphasizes formal guarantees. The STPA- and MCP-based framework in “Towards Verifiably Safe Tool Use for LLM Agents” derives hazards, formalizes them as information-flow and temporal specifications, and argues that if labels and rules are correct and enforced at boundaries, violations are provably impossible within the scope of the model (Doshi et al., 12 Jan 2026). Multi-turn safety work such as ToolShield addresses longer-horizon attacks by self-exploration of tools and reports a 30% average reduction in attack success rate in multi-turn interactions (Li et al., 13 Feb 2026).

Against this backdrop, SafeInstructTool is distinguished by its explicit aggregation of instruction risk, tool-intrinsic risk, and joint instruction-tool risk into a single execution-plan score $i^{\text{th}}$6. That design places it between prompt-only safety assessment and full runtime or formally verified control layers: it is pre-execution and structured, but not presented as a formal verification system. This suggests that it is best understood as a prospective risk-analysis framework for deciding whether a planned tool-use sequence should proceed, be blocked, or require confirmation (Xia et al., 9 Sep 2025).