Terminus-MM: Multimedia Terminal Agent Harness

• Terminus-MM is a multimedia terminal harness that integrates native audio and video tools into terminal agents.
• It employs a workspace-aware routing mechanism that conditionally exposes perception tools based on file presence.
• Empirical results show that full modality access increases binary success to 37.1% and reduces API costs compared to proxy pipelines.

Searching arXiv for the cited paper to ground the article. Terminus-MM is a multimedia harness for terminal agents introduced alongside MultiMedia-TerminalBench (MMTB) in “MMTB: Evaluating Terminal Agents on Multimedia-File Tasks” (Heo et al., 8 May 2026). It is defined as a drop-in extension of Terminus-KIRA, itself an extension of Terminus-2, that adds native audio and video “tools” on top of the existing terminal interface. In the reported formulation, Terminus-MM is designed to let a terminal agent directly operate over workspaces containing audio and video files, rather than relying exclusively on text, code, structured files, or proxy command-line extraction pipelines. Its role is both architectural and evaluative: it supplies multimedia perception primitives to the agent loop and serves as the harness used to study how modality access affects success on multimedia-file tasks.

1. Lineage and system definition

Terminus-MM is situated in a sequence of terminal-agent harnesses. Terminus-2 provides a CLI loop that issues Bash commands through execute_commands, observes stdout/stderr, and uses task_complete to signal the end of a task. Terminus-KIRA extends this setup with view_image(path), which returns a caption or OCR/text dump of an image. Terminus-MM further adds listen_audio(path[, start, end]), which returns a transcript, speaker labels, or audio features, and watch_video(path[, start, end]), which returns a summary of a clip, motion cues, on-screen text, and related information (Heo et al., 8 May 2026).

The paper characterizes these additions as native audio and video “tools” layered on top of the existing terminal interface. This makes Terminus-MM a multimodal terminal-agent harness rather than a general multimedia application framework. Its operational emphasis is on enabling an LLM-driven agent to transform auditory and visual evidence across related files into executable terminal actions.

A central design feature is workspace awareness. At startup, Terminus-MM scans the workspace and dynamically routes only those tools whose modality is present. This means that image, audio, and video tools are exposed conditionally, rather than unconditionally. The paper presents this as a mechanism for controlling the action space seen by the model during inference.

2. Tool routing and agent loop

The paper gives two explicit algorithms for Terminus-MM: workspace-aware perception-tool routing and the main agent loop (Heo et al., 8 May 2026). In the routing procedure, all files under workspace_dir are enumerated, file extensions are mapped to modality indicators, and a retained tool set is constructed from execute_commands, task_complete, and any relevant perception tools. If multimedia files are absent, the corresponding tools are not exposed.

This routing logic has several direct consequences. First, it ties the available API surface to the task workspace rather than to a static global configuration. Second, it removes irrelevant perception calls from the model’s tool schema. The reported ablation shows that this matters empirically: “Terminus-MM w/o masking” attains 32.4 % binary and 42.6 % partial success, whereas routed Terminus-MM attains 37.1 % binary and 46.9 % partial success. The paper attributes this to the prevention of wasted verification steps, such as audio calls on video-only tasks, and describes the gain as recovering 4–5 pp of success.

The agent loop follows the standard pattern used by terminal-agent systems, but with multimedia tool calls included in the dispatch. A prompt is built from a system preamble, tool descriptions induced by the routed schema, prior interaction history, and the task instruction. The LLM then emits either a shell command, an image call, an audio call, a video call, or task_complete. Tool outputs are appended to history until termination. In the ASCII dataflow diagram, this process connects a Harbor task driver, a Terminus-MM agent, and the final workspace containing produced files.

A plausible implication is that Terminus-MM’s main novelty lies less in altering the high-level control loop than in modifying the observational interface available within that loop. The task-solving regime remains tool-mediated and iterative; what changes is the granularity and fidelity of perceptual access to multimedia files.

3. Audio and video perception stack

Terminus-MM relies on a single multimodal LLM backbone, exemplified in the paper by Google Gemini 3.1 Pro, for all three perception channels: image, audio, and video. The perception tools are implemented as API calls to this backbone rather than as purely local standalone decoders. The design therefore centralizes multimodal interpretation within one model family (Heo et al., 8 May 2026).

For audio, the paper specifies the internal sequence triggered by listen_audio(file.wav,start,end). The harness reads the waveform $x[n]$ from file.wav, computes a short-time Fourier transform spectrogram,

$$X(\tau, \omega) = \Sigma_n x[n]\cdot w[n-\tau]\cdot e^{-j\omega n},$$

where $w$ is the analysis window, and feeds log-magnitude $|X(\tau,\omega)|$ frames, or a learned feature sequence, to the LLM’s audio encoder. The LLM then returns a transcript, speaker tags, or feature summaries.

For video, watch_video(file.mp4,start,end) samples frames $\{I_t\}$ at a fixed frame rate, given in the paper as examples such as 1 fps or 2 fps. Each frame is processed by a vision encoder, exemplified as a CNN or ViT patch embedding $$\phi(I_t)`. An OCR layer may optionally extract on-screen text over$$\phi(I_t)`. The resulting frame embeddings, together with an optional audio embedding for the interval, are concatenated and passed through a Transformer to produce a descriptive summary.

The paper states that both audio and video inputs are linear sequences, allowing the LLM’s cross-modal transformer to handle integration and alignment. This suggests that Terminus-MM is architecturally aligned with unified multimodal sequence modeling rather than with a collection of narrowly specialized perception modules. At the same time, the harness abstraction presented in the paper exposes only high-level tool outputs—transcripts, summaries, OCR, motion cues—rather than raw embeddings to the agent.

4. Role within MMTB

Terminus-MM is introduced together with MMTB, a benchmark of 105 self-contained Harbor tasks across 5 meta-categories in which terminal agents directly operate with audio and video files (Heo et al., 8 May 2026). Each task comprises an instruction, a workspace of multimedia files, and an evaluator $V_i$ that checks the final workspace state $y_i$ and returns $s_i \in [0,1]$.

The benchmark defines three reported metrics. Partial success is

$(1/N)\Sigma_i s_i.$

Binary success is

$$X(\tau, \omega) = \Sigma_n x[n]\cdot w[n-\tau]\cdot e^{-j\omega n},$$0

where $$X(\tau, \omega) = \Sigma_n x[n]\cdot w[n-\tau]\cdot e^{-j\omega n},$$1 is a task-specific threshold. The paper also reports API cost per task in USD and mean wall time per task in seconds.

In this evaluation setting, Terminus-MM is not merely an implementation detail; it is the mechanism by which full multimedia access is operationalized. The paper’s comparative setup distinguishes access regimes such as T, T+I, T+I+A, T+I+V, and T+I+A+V, where T denotes terminal access, I image, A audio, and V video. Terminus-MM corresponds to the full T+I+A+V regime.

A common misconception in terminal-agent evaluation is that image access substantially closes the gap between text-only operation and real-world multimedia workflows. The MMTB results argue against that interpretation: in the reported setup, text-only and text-plus-image agents remain below 16 % success, which the paper presents as evidence that many tasks require audio or video grounding.

5. Empirical performance and cost profile

Using a Gemini-3.1-Pro backbone, the paper reports the following comparison across harnesses (Heo et al., 8 May 2026):

Harness	Access	Binary↑ / Partial↑
Terminus-2	T	0.124 / 0.162
Terminus-KIRA	T+I	0.105 / 0.159
Terminus-IA	T+I+A	0.333 / 0.406
Terminus-IV	T+I+V	0.333 / 0.432
Terminus-MM	T+I+A+V	0.371 / 0.469

The corresponding cost and time figures are also reported. Terminus-2 has 0.772 USD cost and 538s mean wall time; Terminus-KIRA 2.061 USD and 544s; Terminus-IA 1.742 USD and 460s; Terminus-IV 1.283 USD and 434s; Terminus-MM 1.228 USD and 442s. The paper summarizes these results with three observations: adding native audio or video yields large jumps over T+I; full T+I+A+V yields the best overall binary success (37.1 %) and partial success (46.9 %); and text-only or text-plus-image agents stall below 16 % success.

The paper also quantifies proxy-conversion overhead when agents lack a modality and must invoke noisy CLI tools such as ffmpeg frames, ASR pipelines, or OCR. On tasks where both a partial agent and Terminus-MM succeed, the average API-cost ratio (partial / MM) ranges from 1.6× to 7.7×, with worst cases up to 42×. The stated interpretation is that repeated transcript, frame, or feature extraction is substantially more expensive than a single native LLM perception tool call.

This result is significant because the improvement is not presented solely as a matter of absolute accuracy. It also concerns the computational and economic consequences of substituting native multimodal access for lossy proxy pipelines. A plausible implication is that multimodal terminal-agent evaluation should treat modality access as an efficiency variable as well as a capability variable.

6. Ablations, solver regimes, and failure structure

The paper includes several analyses intended to isolate what Terminus-MM contributes beyond generic terminal competence (Heo et al., 8 May 2026). In the “modality ladder” ablation with Gemini-3.1, the reported binary success rates are 12.4 % for T alone, 10.5 % for T+I, 33.3 % for T+I+A, 33.3 % for T+I+V, and 37.1 % for T+I+A+V. The lack of gain from T+I relative to T is one of the clearest findings in the paper, while the near parity of audio-only and video-only augmentations indicates that both modalities independently unlock substantial subsets of tasks.

The solver-regime Venn analysis compares Terminus-MM with Codex CLI. It reports 11 tasks solved by both systems, 28 tasks solved only by Terminus-MM, 6 tasks solved only by Codex CLI, and 60 tasks solved by neither. The paper labels the 28 Terminus-MM-only tasks as “grounding-limited” and the 6 Codex-only tasks as “pipeline-limited: simple OCR/ASR suffices.” This framing distinguishes failures caused by absent or lossy multimedia grounding from failures caused by deficiencies in reasoning, tool use, or artifact construction.

Failure signatures sharpen that distinction. Terminus-MM failures concentrate in model reasoning (47 %) and wrong or missed evidence (23 %). Codex CLI failures concentrate in tool timeouts, tool failures, and lossy intermediate extraction (47 %). The paper concludes from these analyses that a multimedia terminal agent needs both native audio and video grounding to avoid lossy CLI proxies and robust tool-use and artifact-construction abilities to avoid execution errors.

These findings also delimit the scope of Terminus-MM’s contribution. The harness addresses one bottleneck—multimedia grounding—more directly than another—general-purpose reasoning and reliable execution. The residual 60 unsolved tasks indicate substantial remaining headroom even when full modality access is available.

7. Interpretation and research significance

Within the framework of MMTB, Terminus-MM functions as an experimental instrument for studying how different forms of multimedia access shape task outcomes and determine which evidence agents rely on to construct executable terminal workflows (Heo et al., 8 May 2026). Its importance therefore lies in both system design and measurement: it concretizes a multimodal terminal-agent interface and exposes the performance consequences of granting or withholding specific modalities.

Several broader implications follow from the reported results. First, the benchmark suggests that multimedia-file tasks are not well approximated by text-only or text-plus-image terminal benchmarks. Second, the routing mechanism shows that modality exposure itself is a controllable variable affecting success, not merely a passive feature of the environment. Third, the cost analysis indicates that native multimedia perception changes the economics of agent execution, especially when compared with repeated proxy extraction through external CLI tools.

At the same time, the empirical profile argues against an overly narrow reading of Terminus-MM as simply “an ASR and video-summary wrapper.” The failure analysis shows that even with native multimedia access, reasoning errors and missed evidence remain major failure modes. This suggests that the main challenge in multimedia terminal agents is not only perception fidelity, but also the coupling of perception with action selection, verification, and artifact construction in long-horizon terminal workflows.

In this sense, Terminus-MM marks a shift in terminal-agent evaluation from predominantly textual interaction toward multimedia-grounded execution. The paper presents it as the harness that makes such evaluation operational, while MMTB provides the task distribution on which its consequences can be measured.