MERLAN: Multimodal Requirements DSL

• MERLAN is a DSL for specifying multimodal interface requirements in AI-enhanced systems by unifying entity declarations and modality triggers.
• It uses a structured metamodel and ANTLR4 grammar to transform declarative requirements into executable agent behaviors over diverse input streams.
• The formal semantics and transformation chain enable clean separation between what to detect and how detection is implemented, facilitating modular system design.

Searching arXiv for the cited MERLAN and related MERLIN entries to ground the article in the current record. MERLAN, short for Multimodal Environment Requirements LANguage, is a Domain-Specific Language (DSL) for specifying requirements for multimodal interfaces in AI-enhanced systems that process multiple input types such as text, audio, images, gestures, and temperature (Gomez-Vazquez et al., 20 Aug 2025). It is presented as a response to an open challenge in requirements engineering: existing languages and methods do not offer a unified way to declare entities recognized across heterogeneous inputs, a precise notation for logical conditions over combined modalities, or an independent layer separating what must be detected from how detection is implemented (Gomez-Vazquez et al., 20 Aug 2025). MERLAN addresses these issues through a metamodel for entities and attributes, a compositional structure for simple and complex requirements, a textual syntax implemented as an ANTLR grammar, and a transformation chain that generates an executable agent from MERLAN scripts, for example on top of the BESSER framework (Gomez-Vazquez et al., 20 Aug 2025).

1. Problem Setting and Design Objectives

Multimodal User Interfaces (MUIs) are described as increasingly common in AI-enhanced systems, where software must interpret two or more input types and react in real time (Gomez-Vazquez et al., 20 Aug 2025). The motivating problem is not merely multimodal perception, but the formalization of requirements attached to multimodal user interactions, potentially involving more than one modality at the same time (Gomez-Vazquez et al., 20 Aug 2025).

The language is defined around three explicit deficiencies in prior requirements-engineering approaches. First, they do not provide a unified way to declare entities—objects, concepts, and abstractions—that may be recognized across heterogeneous inputs. Second, they do not provide a precise notation to express the logical conditions under which an agent should trigger actions when multiple modalities are combined or constrained. Third, they do not provide an independent layer that cleanly separates “what” to detect from “how” that detection is implemented through ML models, computer-vision pipelines, or dialog engines (Gomez-Vazquez et al., 20 Aug 2025).

MERLAN is therefore positioned as a requirements formalization layer rather than as a perception model. This distinction is central to its scope. A plausible implication is that MERLAN is intended to sit above modality-specific inference stacks and below executable agent behavior, acting as a bridge between declarative requirements and runtime logic.

2. Metamodel and Core Abstractions

The abstract syntax of MERLAN is given through an Ecore/UML-style metamodel centered on Entity and MultimodalRequirement (Gomez-Vazquez et al., 20 Aug 2025). Entity is abstract, has a name: String, and contains attributes: Attribute[*]. Two subclasses are defined: ConcreteEntity and AbstractEntity. The language also introduces ArbitraryEntity, extending (ConcreteEntity | AbstractEntity), and this is stated to be used to define domain-specific entities on the fly (Gomez-Vazquez et al., 20 Aug 2025).

Attributes are modeled by a class Attribute with key: String and value: Value?, where the value may be literal, "?", or omitted (Gomez-Vazquez et al., 20 Aug 2025). This allows entity descriptions to represent both known and unspecified properties.

Requirements are organized through an abstract MultimodalRequirement with name: String, specialized into SimpleRequirement and ComplexRequirement (Gomez-Vazquez et al., 20 Aug 2025). SimpleRequirement contains modality: ModalityType, [confidence](https://www.emergentmind.com/topics/confidence): Real, cardinality: Cardinality?—only for ConcreteRequirement—and entityRef: Entity. It is further refined into ConcreteRequirement and AbstractRequirement. ComplexRequirement contains operator: {AND, OR, NOT} and operands: MultimodalRequirement[+], with NOT constrained to exactly one operand (Gomez-Vazquez et al., 20 Aug 2025).

The following table summarizes the principal metamodel elements.

Construct	Declared role	Key fields or relations
Entity	Abstract entity definition	name, attributes
ConcreteEntity / AbstractEntity	Entity specializations	Extend Entity
Attribute	Entity property	key, value
SimpleRequirement	Atomic multimodal requirement	modality, confidence, cardinality, entityRef
ConcreteRequirement / AbstractRequirement	Requirement specializations	Extend SimpleRequirement
ComplexRequirement	Boolean composition of requirements	operator, operands
Cardinality	Multiplicity constraint	min, max
ModalityType	Modality enumeration	{IMAGE, AUDIO, TEXT, TEMPERATURE, …}

This metamodel gives MERLAN a typed representation of both recognition targets and logical trigger conditions. The distinction between concrete and abstract entities, and between simple and complex requirements, is especially important because the later semantics treat them differently (Gomez-Vazquez et al., 20 Aug 2025).

3. Textual Syntax and Grammar

MERLAN is implemented through an ANTLR4 grammar (Gomez-Vazquez et al., 20 Aug 2025). The textual syntax is organized around two optional top-level sections, ENTITIES: and REQUIREMENTS:, within a script production (Gomez-Vazquez et al., 20 Aug 2025). Entity declarations can be grouped into CONCRETE: and ABSTRACT: blocks, with each entity followed by zero or more attributes written as - ID: (STRING | NUMBER | '?') (Gomez-Vazquez et al., 20 Aug 2025).

Requirements are named definitions. Each requirement is introduced by an identifier and then specified either as a simple_requirement or a complex_requirement (Gomez-Vazquez et al., 20 Aug 2025). Simple requirements support either CONCRETE with optional cardinality or ABSTRACT followed by attributes. Complex requirements support boolean structure through AND, OR, and NOT (Gomez-Vazquez et al., 20 Aug 2025).

The grammar excerpt included in the specification is:

$\begin{aligned} \text{script} &::= \text{entities}?\, \text{requirements}? \,;\ \text{entities} &::= \text{'ENTITIES:'}\,\text{NEWLINE}\,\bigl(\text{concrete\_entities}\bigr)?\,\bigl(\text{abstract\_entities}\bigr)?;\ \text{concrete\_entities} &::= \text{'CONCRETE:'}\,\text{NEWLINE}\,\text{concrete\_entity}^*;\ \text{concrete\_entity}&::= \text{ID}\,\text{NEWLINE}\,\text{attribute}^*;\ \text{attribute} &::= '\-'~\text{ID}':'~(\text{STRING}\,|\,\text{NUMBER}\,|\,'?')\,\text{NEWLINE};\[6pt] \text{requirements} &::= \text{'REQUIREMENTS:'}\,\text{NEWLINE}\,\text{requirement}^+;\ \text{requirement} &::= \text{ID ':'}\,\text{NEWLINE}\, (\text{simple\_requirement}\,|\,\text{complex\_requirement});\ \text{simple\_requirement} &::= (\text{'CONCRETE'}\,\text{cardinality}?) |\,(\text{'ABSTRACT'})\,\text{NEWLINE}\,\text{attribute}^*;\ \text{complex\_requirement} &::= (\text{'AND'}\,|\,\text{'OR'})\,\text{NEWLINE}\,\text{requirement}^+ \;|\;\text{'NOT'}\,\text{NEWLINE}\,\text{requirement}; \end{aligned}$

An excerpt of the actual ANTLR grammar is stated to be available in the MERLAN GitHub repository (Gomez-Vazquez et al., 20 Aug 2025). The syntax is therefore not merely illustrative; it is tied to an implementation artifact intended for executable parsing.

4. Formal Semantics

MERLAN defines a small semantic domain with Env = current perceptual state (multimodal observations) and Bool = {true, false} (Gomez-Vazquez et al., 20 Aug 2025). Semantic functions are introduced for entities, simple requirements, complex requirements, and multimodal requirements as boolean test functions (Gomez-Vazquez et al., 20 Aug 2025). The top-level interpretation of a script is a map from names to predicate functions:

$\text{Name} \to (\text{Env} \to \text{Bool})$

The semantics of entity recognition are given as follows. For an Entity definition $E$ with attribute bindings $a_1,\dots,a_n$,

$$\llbracket E(a_1,\ldots,a_n)\rrbracket(env) = MLModel(E.modality).detect(E.name, \{a_1\ldots a_n\}, env) \ge confidence$$

For a ConcreteRequirement, the semantics depend on cardinality:

$$\llbracket R\rrbracket(env) = \text{let } c = R.cardinality \text{ in } count(\llbracket R.entityRef\rrbracket(env)) \in [c.min .. c.max]$$

For an AbstractRequirement, the requirement is satisfied when entity recognition reaches the required confidence:

$$\llbracket R\rrbracket(env) = \llbracket R.entityRef\rrbracket(env) \ge R.confidence$$

For complex requirements, conjunction and disjunction are interpreted over the operand set, and negation applies to the single operand:

$$\llbracket R\rrbracket(env) = \text{if operator=AND then } \forall r \in R.operands.\ \llbracket r\rrbracket(env) \text{ else } \exists r \in R.operands.\ \llbracket r\rrbracket(env)$$

$$\llbracket R\rrbracket(env) = \neg \llbracket R.operands[1]\rrbracket(env)$$

These semantics formalize MERLAN as a boolean trigger language over perceptual environments. The formalization is significant because it makes explicit that the runtime consumes named predicates rather than raw modality-specific signals. This suggests a clean separation between declarative requirement meaning and backend-specific realization.

5. Transformation Chain and Executable Realization

MERLAN includes a toolchain intended to close the gap between requirements and running code (Gomez-Vazquez et al., 20 Aug 2025). The transformation flow is described in four stages. First, the ANTLR grammar generates a parser that produces a parse tree. Second, a tree-visitor builds instances of the metamodel, either as EMF/Ecore objects or plain data classes. Third, a model-to-text phase uses a StringTemplate or Acceleo/XTEND template to emit Python code for each RequirementDefinition and Entity. Fourth, the generated agent skeleton wires these conditions into state-machine transitions or callback handlers (Gomez-Vazquez et al., 20 Aug 2025).

The code-generation stage emits, among other elements, concrete and abstract entity declarations, RequirementDefinition("name").set(…R expression…), and boolean combinators mapped to BESSER.AgenticFramework’s OR([]), AND([]), and NOT(r) calls (Gomez-Vazquez et al., 20 Aug 2025). An example transformation rule is given in pseudocode for each ConcreteRequirement:

for each ConcreteRequirement R: emit ”ConcreteRequirement(name=‘%s’, concrete_entity=%s, attributes=%s)” % (R.name, R.entityRef.name, R.allAttributes)

The generated agent skeleton is also described as wiring these conditions into state-machine transitions or callback handlers (Gomez-Vazquez et al., 20 Aug 2025). This is an important implementation detail because it shows that MERLAN specifications are not merely checked or analyzed statically; they are compiled into executable agents.

The integration claim is that MERLAN is technology-agnostic and that the underlying ML or computer-vision modules can be swapped by reconfiguring the agentic framework while keeping the requirements unchanged (Gomez-Vazquez et al., 20 Aug 2025). That statement is central to the language’s architectural intent, because it operationalizes the separation between requirement specification and modality-specific implementation.

6. Worked Example and Case-Study Evaluation

A full example in the specification combines speech and gesture recognition in a single trigger condition (Gomez-Vazquez et al., 20 Aug 2025). The entity declarations are:

$\text{Name} \to (\text{Env} \to \text{Bool})$0

The requirement definition is:

$\text{Name} \to (\text{Env} \to \text{Bool})$1

The explanation given for this example is exact: greetVisitor fires only if exactly one person is recognized in audio with at least 0.8 confidence, and the image pipeline detects a waving gesture with at least 0.6 confidence (Gomez-Vazquez et al., 20 Aug 2025). The generated Python snippet instantiates ConcreteEntity("person", {"gender": None}), AbstractEntity("waving", {"description":"hand raised gesture detected"}), constructs the RequirementDefinition("greetVisitor"), and hooks the condition into an agent transition through initial_state.when_requirement_matched_go_to(greetVisitor, welcome_state) (Gomez-Vazquez et al., 20 Aug 2025).

Evaluation is reported through a Case Study (House Agent) in which four sample rules—smoke alarm, fire+empty house, night+sound lights, and unrecognized car plate—were encoded in MERLAN, used to generate a BESSER-agent, and run on simulated multimodal streams (Gomez-Vazquez et al., 20 Aug 2025). The generated agent is reported to have correctly triggered alarm notifications, police calls, and light controls (Gomez-Vazquez et al., 20 Aug 2025). Toolchain performance is summarized as parsing plus transformation end-to-end taking < 200 ms for a 50-rule MERLAN script (Gomez-Vazquez et al., 20 Aug 2025).

Early feedback from two requirements engineers is also reported. The feedback highlighted that MERLAN’s separation of entity definitions and requirement logic greatly eases iteration when adding new modalities or refining confidence thresholds (Gomez-Vazquez et al., 20 Aug 2025). This is not a large-scale usability study, but it does indicate the specific aspect of the language that practitioners found most helpful.

7. Nomenclature, Scope Boundaries, and Related Confusion

The acronym MERLAN in (Gomez-Vazquez et al., 20 Aug 2025) denotes Multimodal Environment Requirements LANguage, a DSL for multimodal requirements engineering. It should be distinguished from MERLIN, a separate framework for electromagnetic-signal multimodal LLMs introduced in "MERLIN: Building Low-SNR Robust Multimodal LLMs for Electromagnetic Signals" (Shen et al., 9 Mar 2026).

The relationship between the two is limited to a naming confusion explicitly noted in the MERLIN description, which states that MERLIN is “sometimes mis-referred to as ‘MERLAN’” (Shen et al., 9 Mar 2026). The two systems address different technical problems. MERLAN formalizes multimodal requirements through a metamodel, ANTLR syntax, formal semantics, and code generation (Gomez-Vazquez et al., 20 Aug 2025). MERLIN, by contrast, adopts a native MLLM paradigm with a signal encoder, projector, and LLM for robust EM signal understanding under low-SNR conditions, and introduces EM-100k and EM-Bench as data and benchmark resources (Shen et al., 9 Mar 2026).

This distinction matters because MERLAN operates at the requirements-specification and agent-trigger layer, whereas MERLIN operates at the multimodal representation-learning and reasoning layer. A plausible implication is that confusing the two obscures the boundary between declarative system requirements and learned multimodal inference. In the literature represented here, MERLAN is the requirements language, and MERLIN is the EM-domain MLLM framework.