---
title: 'MERLAN: Multimodal Requirements DSL'
url: https://www.emergentmind.com/topics/merlan
type: topic
---

# MERLAN: Multimodal Requirements DSL

Searching arXiv for the cited MERLAN and related MERLIN entries to ground the article in the current record.
MERLAN, short for **Multimodal Environment Requirements LANguage**, is a Domain-Specific Language (DSL) for specifying requirements for multimodal interfaces in AI-enhanced systems that process multiple input types such as text, audio, images, gestures, and temperature [2508.14631]. It is presented as a response to an open challenge in requirements engineering: existing languages and methods do not offer a unified way to declare entities recognized across heterogeneous inputs, a precise notation for logical conditions over combined modalities, or an independent layer separating *what* must be detected from *how* detection is implemented [2508.14631]. MERLAN addresses these issues through a metamodel for entities and attributes, a compositional structure for simple and complex requirements, a textual syntax implemented as an ANTLR grammar, and a transformation chain that generates an executable agent from MERLAN scripts, for example on top of the BESSER framework [2508.14631].

## 1. Problem Setting and Design Objectives

Multimodal User Interfaces (MUIs) are described as increasingly common in AI-enhanced systems, where software must interpret two or more input types and react in real time [2508.14631]. The motivating problem is not merely multimodal perception, but the formalization of requirements attached to multimodal user interactions, potentially involving more than one modality at the same time [2508.14631].

The language is defined around three explicit deficiencies in prior requirements-engineering approaches. First, they do not provide a unified way to declare entities—objects, concepts, and abstractions—that may be recognized across heterogeneous inputs. Second, they do not provide a precise notation to express the logical conditions under which an agent should trigger actions when multiple modalities are combined or constrained. Third, they do not provide an independent layer that cleanly separates “what” to detect from “how” that detection is implemented through ML models, computer-vision pipelines, or dialog engines [2508.14631].

MERLAN is therefore positioned as a requirements formalization layer rather than as a perception model. This distinction is central to its scope. A plausible implication is that MERLAN is intended to sit above modality-specific inference stacks and below executable agent behavior, acting as a bridge between declarative requirements and runtime logic.

## 2. Metamodel and Core Abstractions

The abstract syntax of MERLAN is given through an Ecore/UML-style metamodel centered on **Entity** and **MultimodalRequirement** [2508.14631]. `Entity` is abstract, has a `name: String`, and contains `attributes: Attribute[*]`. Two subclasses are defined: `ConcreteEntity` and `AbstractEntity`. The language also introduces `ArbitraryEntity`, extending `(ConcreteEntity | AbstractEntity)`, and this is stated to be used to define domain-specific entities on the fly [2508.14631].

Attributes are modeled by a class `Attribute` with `key: String` and `value: Value?`, where the value may be literal, `"?"`, or omitted [2508.14631]. This allows entity descriptions to represent both known and unspecified properties.

Requirements are organized through an abstract `MultimodalRequirement` with `name: String`, specialized into `SimpleRequirement` and `ComplexRequirement` [2508.14631]. `SimpleRequirement` contains `modality: ModalityType`, `confidence: Real`, `cardinality: Cardinality?`—only for `ConcreteRequirement`—and `entityRef: Entity`. It is further refined into `ConcreteRequirement` and `AbstractRequirement`. `ComplexRequirement` contains `operator: {AND, OR, NOT}` and `operands: MultimodalRequirement[+]`, with `NOT` constrained to exactly one operand [2508.14631].

The following table summarizes the principal metamodel elements.

| Construct | Declared role | Key fields or relations |
|---|---|---|
| `Entity` | Abstract entity definition | `name`, `attributes` |
| `ConcreteEntity` / `AbstractEntity` | Entity specializations | Extend `Entity` |
| `Attribute` | Entity property | `key`, `value` |
| `SimpleRequirement` | Atomic multimodal requirement | `modality`, `confidence`, `cardinality`, `entityRef` |
| `ConcreteRequirement` / `AbstractRequirement` | Requirement specializations | Extend `SimpleRequirement` |
| `ComplexRequirement` | Boolean composition of requirements | `operator`, `operands` |
| `Cardinality` | Multiplicity constraint | `min`, `max` |
| `ModalityType` | Modality enumeration | `{IMAGE, AUDIO, TEXT, TEMPERATURE, …}` |

This metamodel gives MERLAN a typed representation of both recognition targets and logical trigger conditions. The distinction between concrete and abstract entities, and between simple and complex requirements, is especially important because the later semantics treat them differently [2508.14631].

## 3. Textual Syntax and Grammar

MERLAN is implemented through an ANTLR4 grammar [2508.14631]. The textual syntax is organized around two optional top-level sections, `ENTITIES:` and `REQUIREMENTS:`, within a `script` production [2508.14631]. Entity declarations can be grouped into `CONCRETE:` and `ABSTRACT:` blocks, with each entity followed by zero or more attributes written as `- ID: (STRING | NUMBER | '?')` [2508.14631].

Requirements are named definitions. Each `requirement` is introduced by an identifier and then specified either as a `simple_requirement` or a `complex_requirement` [2508.14631]. Simple requirements support either `CONCRETE` with optional cardinality or `ABSTRACT` followed by attributes. Complex requirements support boolean structure through `AND`, `OR`, and `NOT` [2508.14631].

The grammar excerpt included in the specification is:

$$
\begin{aligned}
\text{script} &::= \text{entities}?\, \text{requirements}? \,;\\
\text{entities} &::= \text{'ENTITIES:'}\,\text{NEWLINE}\,\bigl(\text{concrete\_entities}\bigr)?\,\bigl(\text{abstract\_entities}\bigr)?;\\
\text{concrete\_entities}
&::= \text{'CONCRETE:'}\,\text{NEWLINE}\,\text{concrete\_entity}^*;\\
\text{concrete\_entity}&::= \text{ID}\,\text{NEWLINE}\,\text{attribute}^*;\\
\text{attribute} &::= '\-'~\text{ID}':'~(\text{STRING}\,|\,\text{NUMBER}\,|\,'?')\,\text{NEWLINE};\\[6pt]
\text{requirements} &::= \text{'REQUIREMENTS:'}\,\text{NEWLINE}\,\text{requirement}^+;\\
\text{requirement} &::= \text{ID ':'}\,\text{NEWLINE}\,
(\text{simple\_requirement}\,|\,\text{complex\_requirement});\\
\text{simple\_requirement}
&::= (\text{'CONCRETE'}\,\text{cardinality}?)
|\,(\text{'ABSTRACT'})\,\text{NEWLINE}\,\text{attribute}^*;\\
\text{complex\_requirement}
&::= (\text{'AND'}\,|\,\text{'OR'})\,\text{NEWLINE}\,\text{requirement}^+
\;|\;\text{'NOT'}\,\text{NEWLINE}\,\text{requirement};
\end{aligned}
$$

An excerpt of the actual ANTLR grammar is stated to be available in the MERLAN GitHub repository [2508.14631]. The syntax is therefore not merely illustrative; it is tied to an implementation artifact intended for executable parsing.

## 4. Formal Semantics

MERLAN defines a small semantic domain with `Env = current perceptual state (multimodal observations)` and `Bool = {true, false}` [2508.14631]. Semantic functions are introduced for entities, simple requirements, complex requirements, and multimodal requirements as boolean test functions [2508.14631]. The top-level interpretation of a script is a map from names to predicate functions:

$$
\text{Name} \to (\text{Env} \to \text{Bool})
$$

The semantics of entity recognition are given as follows. For an `Entity` definition $E$ with attribute bindings $a_1,\dots,a_n$,

$$
\llbracket E(a_1,\ldots,a_n)\rrbracket(env) = MLModel(E.modality).detect(E.name, \{a_1\ldots a_n\}, env) \ge confidence
$$

For a `ConcreteRequirement`, the semantics depend on cardinality:

$$
\llbracket R\rrbracket(env) =
\text{let } c = R.cardinality \text{ in }
count(\llbracket R.entityRef\rrbracket(env)) \in [c.min .. c.max]
$$

For an `AbstractRequirement`, the requirement is satisfied when entity recognition reaches the required confidence:

$$
\llbracket R\rrbracket(env) = \llbracket R.entityRef\rrbracket(env) \ge R.confidence
$$

For complex requirements, conjunction and disjunction are interpreted over the operand set, and negation applies to the single operand:

$$
\llbracket R\rrbracket(env) =
\text{if operator=AND then } \forall r \in R.operands.\ \llbracket r\rrbracket(env)
\text{ else } \exists r \in R.operands.\ \llbracket r\rrbracket(env)
$$

$$
\llbracket R\rrbracket(env) = \neg \llbracket R.operands[1]\rrbracket(env)
$$

These semantics formalize MERLAN as a boolean trigger language over perceptual environments. The formalization is significant because it makes explicit that the runtime consumes named predicates rather than raw modality-specific signals. This suggests a clean separation between declarative requirement meaning and backend-specific realization.

## 5. Transformation Chain and Executable Realization

MERLAN includes a toolchain intended to close the gap between requirements and running code [2508.14631]. The transformation flow is described in four stages. First, the ANTLR grammar generates a parser that produces a parse tree. Second, a tree-visitor builds instances of the metamodel, either as EMF/Ecore objects or plain data classes. Third, a model-to-text phase uses a StringTemplate or Acceleo/XTEND template to emit Python code for each `RequirementDefinition` and `Entity`. Fourth, the generated agent skeleton wires these conditions into state-machine transitions or callback handlers [2508.14631].

The code-generation stage emits, among other elements, concrete and abstract entity declarations, `RequirementDefinition("name").set(…R expression…)`, and boolean combinators mapped to `BESSER.AgenticFramework`’s `OR([])`, `AND([])`, and `NOT(r)` calls [2508.14631]. An example transformation rule is given in pseudocode for each `ConcreteRequirement`:

```text
for each ConcreteRequirement R:
  emit ”ConcreteRequirement(name=‘%s’, concrete_entity=%s, attributes=%s)”
    % (R.name, R.entityRef.name, R.allAttributes)
```

The generated agent skeleton is also described as wiring these conditions into state-machine transitions or callback handlers [2508.14631]. This is an important implementation detail because it shows that MERLAN specifications are not merely checked or analyzed statically; they are compiled into executable agents.

The integration claim is that MERLAN is technology-agnostic and that the underlying ML or computer-vision modules can be swapped by reconfiguring the agentic framework while keeping the requirements unchanged [2508.14631]. That statement is central to the language’s architectural intent, because it operationalizes the separation between requirement specification and modality-specific implementation.

## 6. Worked Example and Case-Study Evaluation

A full example in the specification combines speech and gesture recognition in a single trigger condition [2508.14631]. The entity declarations are:

```text
ENTITIES:
CONCRETE:
  person
  - gender: ?
ABSTRACT:
  waving
  - description: "hand raised gesture detected"
```

The requirement definition is:

```text
REQUIREMENTS:
greetVisitor:
AND
  CONCRETE [1]
  - entity: person
  - modality: "audio"
  - confidence: 0.8
AND
  ABSTRACT
  - entity: waving
  - modality: "image"
  - confidence: 0.6
```

The explanation given for this example is exact: `greetVisitor` fires only if exactly one person is recognized in audio with at least `0.8` confidence, and the image pipeline detects a `waving` gesture with at least `0.6` confidence [2508.14631]. The generated Python snippet instantiates `ConcreteEntity("person", {"gender": None})`, `AbstractEntity("waving", {"description":"hand raised gesture detected"})`, constructs the `RequirementDefinition("greetVisitor")`, and hooks the condition into an agent transition through `initial_state.when_requirement_matched_go_to(greetVisitor, welcome_state)` [2508.14631].

Evaluation is reported through a **Case Study (House Agent)** in which four sample rules—smoke alarm, fire+empty house, night+sound lights, and unrecognized car plate—were encoded in MERLAN, used to generate a BESSER-agent, and run on simulated multimodal streams [2508.14631]. The generated agent is reported to have correctly triggered alarm notifications, police calls, and light controls [2508.14631]. Toolchain performance is summarized as parsing plus transformation end-to-end taking **< 200 ms for a 50-rule MERLAN script** [2508.14631].

Early feedback from **two requirements engineers** is also reported. The feedback highlighted that MERLAN’s separation of entity definitions and requirement logic greatly eases iteration when adding new modalities or refining confidence thresholds [2508.14631]. This is not a large-scale usability study, but it does indicate the specific aspect of the language that practitioners found most helpful.

## 7. Nomenclature, Scope Boundaries, and Related Confusion

The acronym **MERLAN** in [2508.14631] denotes **Multimodal Environment Requirements LANguage**, a DSL for multimodal requirements engineering. It should be distinguished from **MERLIN**, a separate framework for electromagnetic-signal multimodal large language models introduced in "MERLIN: Building Low-SNR Robust Multimodal LLMs for Electromagnetic Signals" [2603.08174].

The relationship between the two is limited to a naming confusion explicitly noted in the MERLIN description, which states that MERLIN is “sometimes mis-referred to as ‘MERLAN’” [2603.08174]. The two systems address different technical problems. MERLAN formalizes multimodal requirements through a metamodel, ANTLR syntax, formal semantics, and code generation [2508.14631]. MERLIN, by contrast, adopts a native MLLM paradigm with a signal encoder, projector, and LLM for robust EM signal understanding under low-SNR conditions, and introduces EM-100k and EM-Bench as data and benchmark resources [2603.08174].

This distinction matters because MERLAN operates at the requirements-specification and agent-trigger layer, whereas MERLIN operates at the multimodal representation-learning and reasoning layer. A plausible implication is that confusing the two obscures the boundary between declarative system requirements and learned multimodal inference. In the literature represented here, MERLAN is the requirements language, and MERLIN is the EM-domain MLLM framework.

Source: https://www.emergentmind.com/topics/merlan