MATE: Multi-Agent Accessibility Framework

• MATE is a multi-agent accessibility framework designed to convert digital content across modalities to accommodate diverse user needs.
• It employs a dedicated ModCon-Task-Identifier with the ModConTT dataset to accurately classify and route modality conversion tasks.
• Its open-source, locally deployable architecture supports integration into varied contexts such as healthcare and institutional systems.

MATE is an open-source, lightweight multi-agent system for accessibility whose core function is to convert information from one modality into another so that content becomes understandable for users with different disabilities or situational needs. The name denotes “LLM-Powered Multi-Agent Translation Environment for Accessibility Applications,” and the system is presented as a multimodal accessibility MAS that infers a requested modality-conversion task from natural-language input and then routes the request to an appropriate agent or sequence of agents. Its canonical example is the conversion of an image received by a user with low vision into an audio description, but the framework is defined more broadly as a general environment for modality adaptation rather than a single assistive interface (Algazinov et al., 24 Jun 2025).

1. Definition and conceptual scope

MATE is defined as a multi-agent accessibility environment centered on modality conversion. In the paper’s formulation, the system does not merely attach one fixed assistive function to a digital interface; it is intended to identify the needed conversion and perform it in a form suitable for the receiving user. This design places modality conversion at the center of the system’s ontology: the relevant unit of action is not a generic command, but a precise transformation from one representational form to another.

The framework is presented as a response to narrow assistive tooling. Existing accessibility tools are described as frequently solving only one problem at a time, such as screen reading or voice control, whereas MATE is proposed as a more general framework for modality adaptation. A common misconception is therefore to treat it as a specialized reader or transcriber. The paper instead positions it as a configurable accessibility MAS whose operational goal is to determine what conversion is being requested and then execute it through an agentic pipeline (Algazinov et al., 24 Jun 2025).

2. Accessibility problem and motivation

The motivating problem is that digital content is often delivered in forms that are inaccessible to specific users, and many technologies are not developed to support the full range of user needs. The paper argues that existing multi-agent systems often cannot provide comprehensive assistance because closed-source designs limit customization. As a result, individuals with disabilities encounter significant barriers when interacting with digital environments.

Within this framing, MATE is aligned with a broader accessibility objective: ensuring that incoming data are converted into an understandable format. The paper’s motivating example is deliberately concrete. If a user cannot see well and receives an image, MATE should convert that image to an audio description. This emphasis on the delivery format of content, rather than on a single interface modality, makes the system applicable to heterogeneous accessibility scenarios in which the same source object may need to be transformed differently for different users (Algazinov et al., 24 Jun 2025).

3. Supported modality conversions

The paper describes MATE as supporting a range of modality adaptation tasks. These are presented as representative conversions that the framework can route and execute.

Source-to-target conversion	Role in the framework
text-to-speech	converts textual content into audio
speech-to-text	converts spoken input into text
image-to-text	converts images into textual descriptions
image-to-audio	converts images into audio descriptions
audio-to-image	converts audio into image form
video-to-text	converts video into text

These tasks are not described as isolated utilities. They are examples of the broader class of modality-conversion operations that MATE is designed to infer from user prompts. The paper’s emphasis on image-to-audio is especially significant because it illustrates the system’s accessibility-first interpretation of multimodality: source and target modalities are selected according to the user’s needs rather than according to a fixed application workflow (Algazinov et al., 24 Jun 2025).

4. System architecture and operational model

At the architectural level, MATE is proposed as a framework that infers the requested modality-conversion task from natural language and routes the request to the right agent or sequence of agents. This routing function is central. The system is not presented as a monolithic model that directly performs every possible conversion internally; instead, it is a multi-agent environment in which task identification and task execution are separated conceptually.

A further architectural feature is model heterogeneity. The framework supports multiple types of models, ranging from LLM API calling to custom machine-learning classifiers. The paper presents this flexibility as a practical property: the system can be adapted to various needs and can remain compatible with a wide variety of hardware. Since the system is expected to run locally, the authors also emphasize privacy and security of sensitive information. This combination of local deployment and model heterogeneity suggests a design intended for configurable, institutionally deployable accessibility support rather than for a closed hosted service (Algazinov et al., 24 Jun 2025).

5. ModCon-Task-Identifier and the ModConTT dataset

A major technical contribution is the introduction of ModCon-Task-Identifier, a dedicated task-recognition component that extracts the precise modality conversion task from user input. In the paper’s framing, this component solves the classification problem that sits upstream of execution: before any agent can be selected, the system must determine exactly which modality transformation the prompt requests.

To support this task, the authors introduce a new dataset, ModConTT, standing for “Modality Conversion Task Type.” The paper reports that a fine-tuned BERT-based ModCon-Task-Identifier outperforms several LLM and classical machine-learning baselines on this task, and the abstract states that numerous experiments show it consistently outperforms other LLMs and statistical models on the custom data. This component is therefore not ancillary. It is the mechanism by which the generality of the MATE framework becomes operational: natural-language requests are mapped to explicit modality-conversion categories before downstream agents are engaged (Algazinov et al., 24 Jun 2025).

6. Deployment contexts, applications, and significance

The paper presents MATE as applicable across a wide range of domains, industries, and areas, explicitly including healthcare. It is also described as being integrable with institutional technologies, such as digital healthcare services, for real-time user assistance. This institutional orientation is technically important because it links the accessibility objective to deployment constraints such as privacy, hardware heterogeneity, and interoperability with existing systems.

The framework is also presented as open source, and the authors state that code and data are publicly available through a GitHub repository. This matters in the specific context identified by the paper: closed-source accessibility systems and non-customizable multi-agent systems are treated as part of the problem that MATE seeks to address. A plausible implication is that the open-source and locally deployable design supports adaptation to organization-specific accessibility requirements and governance constraints, although the paper’s concrete claim is narrower: it emphasizes customization, compatibility with varied hardware, and protection of sensitive information through local execution (Algazinov et al., 24 Jun 2025).