---
title: 'MoniTor: Formal Runtime Monitoring'
url: https://www.emergentmind.com/topics/monitor
type: topic
---

# MoniTor: Formal Runtime Monitoring

Searching arXiv for the cited MoniTor-related papers and closely related monitoring work.
arXiv search query: `1601.04263 OR "Sound and Complete Runtime Security Monitor for Application Software" OR "Ain't No Stopping Us Monitoring Now" OR "Monitoring Monitorability" OR "Monitorology the art of observing the world" OR "Amazon SageMaker Model Monitor" OR "Monitoring and Intervention: Concepts and Formal Models" OR "AntMonitor" OR "Intelligent Monitoring Framework for Cloud Services"`
MoniTor is not introduced as a single universally fixed proper noun across the arXiv literature. The closest direct referent is the runtime security monitor at the core of ARMET in "Sound and Complete Runtime Security Monitor for Application Software" [1601.04263]. Later papers use “MoniTor” as a convenient name for generic monitoring frameworks when discussing monitorability, monitor synthesis, or AI oversight [2211.11544], [2512.18311]. This suggests that MoniTor is best understood as a family resemblance term for formally specified, execution-facing monitoring systems: systems that observe behavior, compare it to a model, issue verdicts, and sometimes trigger intervention.

## 1. Terminological scope and conceptual identity

In the most literal sense available in the literature, MoniTor corresponds to a **runtime security monitor (RSM)** that executes an application specification in parallel with an implementation and checks whether observed run-time behavior is consistent with specification-derived predictions [1601.04263]. The paper explicitly notes that the monitor itself is *not* explicitly named “MoniTor,” but that a reference to a “sound and complete runtime security MoniTor for application software” is naturally read as this RSM.

A broader usage appears in work on monitorability. "Ain't No Stopping Us Monitoring Now" frames MoniTor as a monitoring framework that must answer three questions: what a runtime monitor can conclude for a specification $\varphi$, how it can avoid running forever once nothing more can be concluded, and how those conclusions should be presented to users [2211.11544]. "Monitoring Monitorability" uses MoniTor analogously for systems that monitor AI agents, especially reasoning models that expose chain-of-thought, for undesirable properties such as deception, reward hacking, or sycophancy [2512.18311].

At a still more abstract level, "Monitoring and Intervention: Concepts and Formal Models" defines monitoring as a data process that observes the behavior of entities in a context, evaluates that behavior against specified attributes, and records the resulting judgements; intervention is modeled as a subsequent process that inspects those records and may change entity characteristics [1701.07484]. "Monitorology the art of observing the world" describes monitoring as “observation and collection of relevant data about the current state of a system under study” and organizes the design problem around five questions: what to monitor, where to place monitors, how frequently to monitor, how to communicate/store/process data, and how to ensure data quality [1902.09459].

Taken together, these usages place MoniTor at the intersection of formal specification, observable behavior, verdict generation, and operational response. The term is therefore less a single artifact than a recurrent research pattern.

## 2. Runtime security monitor for application software

The RSM of ARMET is a model-based monitor that runs an executable specification in parallel with the application implementation, observes run-time events produced by wrappers, and compares those observations with specification-derived predictions [1601.04263]. Its architecture comprises **AppSpec**, **AppImpl**, a **Wrapper Synthesizer**, and the **RSM**. The Wrapper Synthesizer reads the specification and implementation, generates probes or wrappers, and emits **RTEvents** such as method entry, method exit, resource use, allowable events, and unexpected events. When a deviation is detected, the monitor raises an alarm; ARMET then suspends the application and performs diagnosis.

Operationally, monitoring is event-driven. On startup, an entry event for the top-level component causes the monitor to create the corresponding component instance, set its state flag to `running`, mode to `normal`, and instantiate its subnetwork. For each runtime event, the monitor determines whether it is `ENTRY`, `EXIT`, `ALLOWABLE`, or `NONE`. The central predicate is

$$
monitors(i,\ rte,\ c,\ e,\ e',\ s,\ s')
$$

and the `run` relation specifies the behavior for each event class. In the `ENTRY` case, the monitor checks preconditions, invariants, and absence of attack plans for normal behavior; if the check succeeds, the component remains `running` and its subcomponents are monitored, and if it fails the state is set to `compromised`. In the `EXIT` case, the monitor checks postconditions and invariants; successful checks yield `completed` and `normal`, while failure yields `compromised`. An unexpected `NONE` event also sets the state to `compromised`.

The monitor is configurable at different granularities and levels of abstraction. High-level, coarse-grain monitoring yields low overhead but less precise localization; fine-grain specifications increase overhead but improve containment and localization. The design goal is to exploit modularity and abstraction so that the monitor is efficient enough for real-time systems while still detecting both known and unknown cyber attacks.

The central correctness claim is that the monitor is **sound and complete**. Soundness means it never signals an alarm unless there is a real deviation, eliminating false positives. Completeness means that whenever there is a deviation from the specification, the monitor will eventually raise an alarm, eliminating false negatives, under explicit assumptions. Those properties are proved denotationally rather than asserted operationally.

## 3. Logical language, abstraction, and denotational semantics

The specification language of the RSM is formalized on **monadic second-order logic** and **event calculus** interpreted over **algebraic data structures** [1601.04263]. Its top-level structure has three parts: hierarchical decomposition, behavioral description, and attack plans. Decomposition specifies component interfaces, subcomponents, control-flow and data-flow links, resources, model mappings, and vulnerabilities. Behavioral description gives component-local contracts in different modes such as `normal` and `compromised`, through prerequisites, postconditions, invariants, and allowable events. Attack plans model attack types and their mappings to vulnerabilities and resource/component modes.

This language is explicitly designed to support **hierarchical, modular modeling** and **multiple abstraction levels**. The paper introduces a combined state

$$
State := Store \times Data
$$

together with a merge operator $s;t$ and a semantic abstraction predicate `abstract(v, v')` so that specification state and run-time state, which may be expressed in different languages and at different levels of abstraction, can still be related. Consistency between implementation and specification environments is defined by `consistent(e_r, e_s)`, requiring that corresponding identifiers are related by the abstraction predicate.

The monitor semantics is then stated as a relation between application implementation, application specification, environments, and states. Informally, if specification and implementation begin in equal, normal states and environments, then after a finite number of monitoring steps the monitor reaches one of two cases: either the mode is `normal`, the flag is `completed`, and implementation and specification post-states coincide; or the mode is `compromised` and the post-states differ. This is the formal core behind the monitor’s absence of false positives and false negatives.

The language also gives attack plans first-class semantic status. An attack can correspond to violation of preconditions, postconditions, or invariants; occurrence of an unexpected event; or satisfaction of an attack-plan condition that makes `noatk` false. This is why the monitor can detect **known attacks** through explicit attack models and **unknown attacks** through deviations from the good-behavior model. The paper’s industrial control system example illustrates both cases: parameter tampering in a PID controller can violate postconditions, while false data injection can be detected through inconsistency between the physical plant model and observed sensor readings.

A practical implication is that MoniTor, in this sense, is specification-centric rather than signature-centric. Its guarantees depend on specification correctness, environment consistency, instrumentation coverage, and a trusted monitor/platform.

## 4. Abstract monitoring, monitorability, and intervention

"Ain't No Stopping Us Monitoring Now" places MoniTor in a semantic theory of **monitorability** [2211.11544]. A monitor is a function from observations to verdicts, initially with the three-valued set $V=\{yes,no,?\}$. Given a property $P \subseteq \mathbb{B}$ over behaviors and an observation $\obsa$, the paper defines the **abstract monitor**

$$
\amnt_P(\obsa)=
\begin{cases}
yes & \text{if } \mathbb{B}(\obsa)\subseteq P \\
no & \text{if } \mathbb{B}(\obsa)\cap P=\emptyset \\
? & \text{otherwise.}
\end{cases}
$$

A property is **monitorable** if, for every observation, some refinement eventually yields a non-`?` verdict. This excludes properties for which there are prefixes from which every continuation remains inconclusive forever.

The paper then develops safety and cosafety completions. The **safety completion** $\clop[S](P)$ is the best safety over-approximation of $P$, preserving negative verdicts exactly, and the **cosafety completion** $\inop[](P)$ is the best cosafety under-approximation, preserving positive verdicts exactly. These completions allow partial runtime verification of non-monitorable properties. The key semantic phenomenon is the existence of “ugly” observations: prefixes from which no future refinement can ever yield `yes` or `no`.

To represent that structure, the paper introduces a generalized monitor with six verdicts. In addition to `yes`, `no`, and `?`, it adds verdicts for “no negative verdict will ever be reached,” “no positive verdict will ever be reached,” and “giveup,” the permanently undecidable case. The general result is that for every observation there exists some refinement after which the generalized monitor reaches one of the terminal verdicts `yes`, `no`, or `giveup`. This gives MoniTor a principled stopping rule for properties that are not fully monitorable.

The companion abstraction in "Monitoring and Intervention: Concepts and Formal Models" gives a generic algebraic account of monitoring and intervention [1701.07484]. A context is defined by entities, characteristics, and behaviors, with a behavior mapping

$$
[{-,-}] : Entity \times Characteristics \to Behaviour.
$$

Attributes are properties of behaviors, judgements are the outputs of observation, and a record has the form $(e,\chi,P,j)$. Monitoring is the production of such records; intervention is modeled by a trigger condition on judgements together with an action on characteristics. This framework makes explicit that verdict production and behavioral modification are distinct but composable layers.

A plausible implication is that MoniTor has two separable semantics in the literature: a **verdict semantics**, concerned with what can be concluded from finite observation, and an **intervention semantics**, concerned with what should change once a judgement is produced.

## 5. System architectures and application domains

The monitoring pattern associated with MoniTor appears in several system domains.

| Domain | System or framework | Characteristic contribution |
|---|---|---|
| Application security | ARMET RSM | Sound and complete run-time checking against executable specifications |
| ML operations | SageMaker Model Monitor | Data, concept, bias, and feature attribution drift monitoring |
| Mobile networking | AntMonitor | On-device passive monitoring via Android `VpnService` |
| Cloud services | Intelligent monitoring framework | Ontology-driven monitor recommendation |

"Amazon SageMaker Model Monitor: A System for Real-Time Insights into Deployed Machine Learning Models" presents a fully managed service that continuously monitors the quality of models hosted on Amazon SageMaker and automatically detects **data, concept, bias, and feature attribution drift** in real time [2111.13657]. Its architecture separates **Data Collection**, **Data Analysis**, and **Scheduling / Orchestration**. Inputs and outputs are captured at endpoints, written locally as JSON Lines, uploaded asynchronously to Amazon S3, analyzed in SageMaker Processing, and surfaced through S3, CloudWatch, and SageMaker Studio. The system uses data sketches, approximate histograms, distance-threshold variants of Student’s $t$-test and the two-sample Kolmogorov–Smirnov test, bootstrap confidence intervals for fairness metrics, and **NDCG** for feature attribution drift. The monitored dimensions thereby extend MoniTor from specification compliance into post-deployment statistical stability.

"AntMonitor: A System for On-Device Mobile Network Monitoring and its Applications" instantiates MoniTor as a **pure userspace** Android application built on `VpnService` [1611.04268]. It creates a virtual TUN interface, routes all device traffic through a user-space forwarder, performs on-device inspection and logging, and forwards packets without a remote VPN server. The prototype achieves **throughput of over 90 Mbps downlink and 65 Mbps uplink**, **94% of the throughput without VPN**, and **2-12x less energy** than prior baselines. The same vantage point supports real-time detection and prevention of private information leakage, passive network performance monitoring, and application classification and user profiling. Here MoniTor functions as an execution-path monitor over packet streams rather than over program semantics.

"Intelligent Monitoring Framework for Cloud Services: A Data-Driven Approach" addresses monitor creation itself as a learning problem [2403.07927]. It mines **30,000+ monitors from 791 production services**, derives an ontology with **13 major resource classes** and **9 SLO classes**, and trains a recommendation model that maps service properties to monitor classes. Resource classes include Service-level, API, Dependency, CPU, Compute cluster, Storage, Ram-memory, Cache-memory, Container, Certificates, IO, Paging memory, and None-of-the-above. SLO classes include Success rate, Capacity, Latency, Availability, Throughput, Success rate – QoS, Interruption rate, Freshness, and Others. In a user study with Microsoft engineers, the framework’s usefulness was rated **4.27 out of 5**. In this formulation, MoniTor becomes a monitor-design assistant rather than only a monitor-execution engine.

"Monitorology the art of observing the world" provides a domain-independent counterpart by organizing monitoring around the five challenges of selecting variables, placing monitors, choosing sampling frequency, managing communication/storage/processing, and ensuring data quality [1902.09459]. It explicitly warns against “a flood of meaningless, dumb data,” and thereby frames the systems above as instances of a broader discipline of monitor design.

## 6. AI monitorability, limitations, and research directions

"Monitoring Monitorability" extends MoniTor to AI-agent oversight and defines **monitorability** as “a monitor’s ability to accurately predict a specified property of an agent’s actions, such as whether it will act harmfully” [2512.18311]. The paper introduces three evaluation archetypes: **intervention evaluations**, which ask whether the monitor can detect causal influence from an intervention; **process evaluations**, which ask whether the monitor can detect use of an admissible solution path; and **outcome-property evaluations**, which ask whether the monitor can detect harmful or otherwise labelable properties of final behavior. To compare monitors under class imbalance, it proposes the metric

$$
\text{g-mean}^2 = \mathrm{TPR} \cdot \mathrm{TNR}.
$$

For intervention evaluations it adds a minimal-criterion TPR derived from the total effect of the intervention. The empirical findings are that **CoT monitoring is more effective than action-only monitoring**, that **most models are fairly, but not perfectly, monitorable**, that **longer CoTs are generally more monitorable**, and that **RL optimization does not materially decrease monitorability even at the current frontier scale**. The paper also finds that a smaller model at a higher reasoning effort can match capabilities while yielding higher monitorability at a higher overall inference compute cost, that scaling a weak monitor’s test-time compute improves monitorability, and that asking follow-up questions and exposing the follow-up CoT to the monitor can improve monitorability.

These AI results sharpen a recurring limitation across MoniTor-like systems: the monitor’s guarantees are always conditioned on what is observable and on what assumptions are trusted. In the RSM setting, the assumptions include specification correctness, environment consistency, faithful instrumentation coverage, and a trusted monitor and platform [1601.04263]. In the general monitoring literature, data quality must satisfy conditions such as completeness, consistency, stability over time, accuracy, and timestamping, or downstream decisions become unreliable [1902.09459]. In monitorability theory, some properties are simply not fully decidable from finite observation and require generalized verdicts rather than definitive acceptance or rejection [2211.11544].

The current research trajectory therefore moves along three axes. One is **formalization**, exemplified by denotational proofs of soundness and completeness and by semantic accounts of monitorability. Another is **systems integration**, exemplified by managed ML monitoring, on-device VPN monitoring, and cloud-service recommendation frameworks. The third is **observability of reasoning**, where the monitored object is no longer only an external execution trace but also a model’s exposed deliberative process. This suggests that MoniTor, across its usages, names a continuing effort to reconcile expressiveness, efficiency, and trustworthy verdicts under real operational constraints.

Source: https://www.emergentmind.com/topics/monitor