SLEEC-LLM: LLM Explanation for Normative Debugging

• The paper introduces SLEEC-LLM, a tool that uses LLMs to interpret model-checking counterexamples for conflicts in normative requirements.
• It integrates SLEEC DSL rules, tock CSP conversion, and FDR model checking to diagnose inconsistencies and offer repair suggestions in accessible language.
• Preliminary studies show about a 63% reduction in debugging time and iterations, enhancing efficiency in resolving normative rule conflicts.

SLEEC-LLM is a debugging and explanation tool for normative requirements that encode social, legal, ethical, empathetic, and cultural norms, abbreviated as SLEEC. Introduced as a tool that uses LLMs to provide natural-language interpretations for model-checking counterexamples corresponding to SLEEC rule inconsistencies, it is designed to make formal consistency analysis understandable to non-technical stakeholders such as lawyers, ethicists, psychologists, and regulators who often author or validate such requirements but may not be comfortable interpreting formal counterexamples produced by model checkers (Kleijwegt et al., 7 Jul 2025).

1. Place within SLEEC requirements engineering

SLEEC-LLM emerges from a line of work that treats human-centered norms as formalizable requirements rather than as informal guidance. In that literature, SLEEC requirements are specified as rules with triggers, responses, time constraints, and defeaters, and then subjected to formal analysis for conflicts, redundancy, and compliance checking (Yaman et al., 2023). A complementary satisfiability-based line of work extends this emphasis on well-formedness by diagnosing conflicts, redundancy, restrictiveness, and insufficiency in normative non-functional requirements, again with the aim of supporting multidisciplinary stakeholders (Feng et al., 2024).

Within the broader operationalisation of SLEEC norms for AI agents, the relevant lifecycle includes capability specification, SLEEC requirements elicitation, well-formedness checking, SLEEC-aware implementation, and verification of AI agent compliance (Calinescu et al., 12 Mar 2026). SLEEC-LLM is situated at the well-formedness and debugging stage. Its specific target is the explainability gap that remains after formal analysis has already identified an inconsistency. In other words, it does not replace the formal method; it intervenes at the point where the formal method has succeeded technically but failed communicatively.

The problem it addresses is practical and recurrent. Normative requirements are typically defined by stakeholders in the non-technical system with diverse expertise, and ensuring their consistency while managing elicitation is described as a complex and error-prone task. Even when domain-specific languages and model checkers are used, the resulting counterexamples are often inaccessible to the people who must decide whether a rule should be refined, merged, removed, or defended (Kleijwegt et al., 7 Jul 2025).

2. Normative requirements, SLEEC DSL, and formal analysis substrate

SLEEC-LLM operates over SLEEC DSL, a domain-specific language for specifying normative requirements. A SLEEC ruleset consists of two parts: definitions and rules. Definitions declare events, measures, and constants. Events are instantaneous actions; measures are system capabilities or observable values, such as Boolean, numeric, or scale values. Rules follow the basic syntax

$\texttt{when trigger then response}$

and can be extended with defeaters using unless and time bounds using within (Kleijwegt et al., 7 Jul 2025).

A representative rule is:

$$\texttt{when DetectUserFallen then CallEmergencySupport within 2 minutes unless emergencyLevel > L4}$$

This style is intended to be accessible to non-technical stakeholders, but the formal analysis pipeline remains more technical. SLEEC rules are translated by SLEECVAL into tock CSP, a timed process algebra, and then checked with the FDR model checker. If the rules are inconsistent, FDR produces counterexamples in trace form (Kleijwegt et al., 7 Jul 2025). The underlying SLEEC framework already established this tock-CSP semantics and the use of FDR for detecting conflicts and redundancy, as well as for verifying autonomous-agent compliance against validated rules (Yaman et al., 2023).

The interpretive difficulty of the trace is central to the motivation for SLEEC-LLM. A counterexample such as

<DetectUserFallen, emergencyLevel.E1, emergencyLevel.E1, tock, ..., tock>

is meaningful to a formal-methods expert, but not to most stakeholders. The trace exposes the problematic execution prefix, yet does not directly explain what conflicts, why it conflicts, what category of inconsistency is involved, or how the rules might be repaired (Kleijwegt et al., 7 Jul 2025).

3. Architecture, workflow, and prompting strategy

The architecture has three major parts: a SLEEC specification layer, a formal analysis layer, and an LLM explanation layer. The specification layer contains rules written in SLEEC DSL, including events, measures, constants, triggers, responses, defeaters, and time bounds. The formal analysis layer consists of SLEECVAL translation to tock CSP, FDR model checking, and generation of counterexamples for inconsistent rules. The LLM explanation layer performs prompt construction with formal and natural-language context, counterexample interpretation, output formatting via a JSON template, and generation of conflict explanations and repair suggestions (Kleijwegt et al., 7 Jul 2025).

The workflow is explicitly staged. First, a user provides SLEEC rules in SLEEC DSL. Second, SLEECVAL translates them to tock CSP. Third, FDR checks consistency. Fourth, if a conflict is found, counterexamples are generated. Fifth, these counterexamples, together with context, are fed to an LLM. Sixth, the LLM produces a natural-language explanation of the conflict, a categorization of the issue, and two resolution suggestions (Kleijwegt et al., 7 Jul 2025).

The prompt is treated as a key methodological component. It includes the SLEEC rules in DSL form, the formal CSP semantics, the counterexample traces, a natural-language description of the system, and a JSON-style output template that structures the response. The paper characterizes this as retrieval-augmented generation style prompting, because the LLM receives rich contextual material so its response is grounded in the formal artifacts (Kleijwegt et al., 7 Jul 2025).

The output template is designed to be actionable rather than merely descriptive:

"Conflicting Rules":{ "Error": { "Rule1": {name of Rule1}, "Rule2": {name of Rule2}, "Scenario": {a scenario that creates conflict in the rules}, "Category": {deadlock, divergence, naming}, "Justification": {your justification} } "Resolution": { add rule, combine rule, remove rule, modify rule "Suggestion1": {name of Rule1 A}, "Justification": {your justification} "Suggestion2": {name of Rule2}, "Justification": {your justification} } }

The prompt is also described as iterative and adaptable: it can evolve depending on the number of counterexamples, the available context, and the LLM used (Kleijwegt et al., 7 Jul 2025).

4. Counterexample interpretation and explanatory function

The core contribution of SLEEC-LLM is the conversion of low-level trace data into stakeholder-usable explanations. Instead of exposing only a sequence of event labels and time ticks, the tool identifies which rules are in conflict, gives a natural-language scenario describing the failure, assigns a conflict type, and proposes ways to resolve it (Kleijwegt et al., 7 Jul 2025).

The paper’s formal example uses two conflicting rules. The first rule is:

when DetectUserFallen then CallEmergencySupport within 2 minutes unless emergencyLevel > L4

The second is:

when DetectUserFallen and emergencyLevel < L2 then not CallEmergencySupport within 2 minutes

These are conflicting because, under some conditions, one rule requires a call while the other prohibits it. The model checker returns the trace shown above; SLEEC-LLM is intended to explain that this trace corresponds to a conflict in the system’s required response to an emergency event (Kleijwegt et al., 7 Jul 2025).

This explanatory role is demonstrated in two case studies. The first is ALMI, a home healthcare robot for assisting elderly users with tasks such as cooking and medication-taking. The second is ASPEN, a forest-monitoring drone domain. In ALMI, the ruleset includes intentionally contradictory rules about emergency response: one rule requires emergency support to be called within a deadline, while another says it must not be called under certain conditions. In both ALMI and ASPEN, the LLM output gave a scenario-based explanation of the contradiction, the specific rules involved, a conflict category, and suggested fixes (Kleijwegt et al., 7 Jul 2025).

A common misconception is that SLEEC-LLM performs the consistency check itself. The formal checking remains the responsibility of SLEECVAL and FDR; SLEEC-LLM sits on top of that pipeline and interprets the resulting counterexamples. Another possible misconception is that it is a runtime ethics enforcer. In the SLEEC ecosystem, runtime enforcement is instead associated with [email protected], which uses ASM formalism and a MAPE-K loop to enforce SLEEC rules during system operation (Sanctis et al., 4 Apr 2026).

5. Evaluation, stakeholder study, and observed limitations

The paper reports a preliminary control study with small multidisciplinary groups of participants. The participants worked on the ALMI and ASPEN case studies, were experts in fields such as law, ethics, and psychology, and performed debugging tasks on inconsistent SLEEC rulesets. The study design assigned one task with the LLM-assisted tool and one task without the LLM tool, then switched case studies; each task had a 25-minute limit. Participants were asked to identify and fix intentionally inconsistent rules (Kleijwegt et al., 7 Jul 2025).

The reported results are summarized below.

Task	Manual	LLM assisted
Study 1: ALMI	Time expired; 2 / 4 resolved; 6 iterations	12m 30s; 4 / 4 resolved; 4 iterations
Study 2: ASPEN	Time expired; 2 / 4 resolved; 8 iterations	25 m; 4 / 4 resolved; 6 iterations

The appendix summarizes these results as approximately a 63% reduction in average time and a 63% reduction in number of iterations when using the LLM tool (Kleijwegt et al., 7 Jul 2025). The reported interpretation is that SLEEC-LLM helps users resolve conflicts faster, require fewer revise-and-resubmit cycles, and successfully repair more of the ruleset within the time limit.

User feedback was positive overall. Participants found the explanations helpful, felt more confident when using the tool, and regarded the resolution suggestions as useful. The paper nevertheless records important limitations. Some users experienced syntax correctness problems in the LLM output; prompting will likely need refinement; and the work notes threats to validity from relying on LLM-generated suggestions. The tool therefore improves explainability, but the suggestions still need human judgment and syntactic checking (Kleijwegt et al., 7 Jul 2025).

6. Relation to adjacent approaches and broader significance

SLEEC-LLM’s novelty lies in its focus on the explainability gap between formal counterexamples and stakeholder understanding. Earlier SLEEC research concentrated on formal consistency validation, tock-CSP semantics, FDR-based conflict and redundancy detection, and verification of autonomous agents against SLEEC rules (Yaman et al., 2023). Other work such as N-Check broadened the well-formedness space to include restrictiveness and insufficiency, using FOL* encodings and the LEGOS satisfiability checker, while also emphasizing user-friendly diagnostics for non-technical stakeholders (Feng et al., 2024). SLEEC-LLM is narrower in formal scope but more specific in its use of LLMs to render model-checking artifacts intelligible.

The 2026 operationalisation framework explicitly places an LLM-guided debugging tool alongside process-algebraic and logic-based analyses in the well-formedness stage of SLEEC norm operationalisation, and presents the overall lifecycle as iterative from capability specification through deployment or cancellation (Calinescu et al., 12 Mar 2026). In that setting, SLEEC-LLM functions as an interface technology between formal methods and stakeholder deliberation rather than as a substitute for either. This suggests that its main significance lies in reducing the transaction cost of iterative elicitation: the same formal inconsistency can be re-presented as a scenario, a category, and a repair space that are meaningful to ethicists, lawyers, psychologists, and regulators.

The future direction identified for SLEEC-LLM is extension from consistency checking to verification of systems against SLEEC requirements, together with continued improvement of the prompt and the reliability of LLM-generated suggestions (Kleijwegt et al., 7 Jul 2025). More broadly, the tool illustrates a hybrid pattern that recurs across the SLEEC literature: formal models provide semantic precision and analyzability, while LLMs provide explanation and interaction. In that sense, SLEEC-LLM is best understood as an interpretive layer over formal normative analysis, intended to make the engineering of social, legal, ethical, empathetic, and cultural requirements more explainable, more collaborative, and more usable in practice.