Cued-Agent: A Cue-Centric Design Paradigm

• Cued-Agent is a cue-centric design pattern spanning multimodal speech generation, recognition, delegation, memory recall, and secure computing, emphasizing explicit cue integration.
• In Cued Speech applications, systems convert text into synchronized hand and lip trajectories using architectures like AVTacotron2 with attention-based fusion for high decoding accuracy.
• Cue-guided collaboration in both human–agent tasks and computer-use security enhances system adaptability and robust decision-making across diverse technical domains.

Cued-Agent is a term that appears in recent literature in several technically distinct senses. In its most specific and established usage, it denotes agentic systems for Cued Speech: either an interactive generator that converts text into synchronized hand and lip trajectories, or a collaborative multi-agent recognizer that converts Cued Speech video into phoneme sequences and natural-language sentences. In a broader sense, the label is also used for agents whose behavior is explicitly organized around cues, including task-aware delegation signals, strategy-guided memory recall prompts, queue-length information signals, and software-environment cues in computer-use agents. Taken together, these works suggest a cue-centric design pattern rather than a single canonical architecture (Sankar et al., 8 Jan 2025, Huang et al., 1 Aug 2025).

1. Terminology and scope

Across the cited literature, “Cued-Agent” does not refer to one unified benchmark object or software package. Instead, it spans multiple research lineages that share an emphasis on explicit cues as coordination primitives, control signals, or perceptual anchors. In the Cued Speech literature, the term is tied directly to accessibility technologies for deaf and hard-of-hearing users. In human–agent collaboration, it refers to delegation or recall signals surfaced to users. In computer-use security, it is used as a deployment target whose behavior must be constrained against adversarial environmental cues. In queueing theory, it refers to agents whose entry decisions are shaped by public information cues (Huang et al., 1 Aug 2025, Gu, 11 Mar 2026, Zhao et al., 31 Jul 2025, Liao et al., 28 May 2025, Hirsch et al., 26 Feb 2025).

Usage	Domain	Representative paper
Cued Speech generation	Text-to-gesture synthesis	(Sankar et al., 8 Jan 2025)
Cued Speech recognition	Collaborative multi-agent ACSR	(Huang et al., 1 Aug 2025)
Delegation cues	Human–agent collaboration	(Gu, 11 Mar 2026)
Memory-recall cueing	HCI and memory assistance	(Zhao et al., 31 Jul 2025)
Queue information cues	Mechanism design	(Hirsch et al., 26 Feb 2025)
Computer-use deployment target	Security and reliability	(Liao et al., 28 May 2025, Chen et al., 8 May 2026, Luo et al., 8 Oct 2025)

The terminological overlap matters because the underlying technical objects differ sharply. Some works define Cued-Agent as a multimodal sequence model, some as a collaborative multi-agent pipeline, and some as a conceptual agent class whose actions are shaped by cues. A plausible implication is that the term should always be read with domain qualification.

2. Cued-Agent as automatic Cued Speech generation

In "Cued Speech Generation Leveraging a Pre-trained Audiovisual Text-to-Speech Model" (Sankar et al., 8 Jan 2025), a Cued-Agent is an interactive agent for Automatic Cued Speech Generation (ACSG). Cued Speech is a visual communication system designed to make spoken language fully visually accessible to people with hearing impairment by disambiguating lipreading with manual cues: handshape and hand placement around the face. In French CS (LfPC), consonant groups are encoded by handshape, vowels by hand placement, and coarticulation and sequence dynamics are expressed via timing and motion of lips and hands. ACSG is formulated as a sequence-to-sequence task mapping text or phoneme sequences to temporally aligned continuous visual articulatory trajectories of the lips and the cueing hand, represented as 2D landmark streams.

The model core is a reprogrammed AVTacotron2. The encoder uses 3 convolutional layers with 512 filters and kernel size 5, followed by a bidirectional LSTM with 512 cells. Alignment is handled by location-sensitive attention whose attention weights are processed by a 1D CNN with 32 filters and kernel size 31. The decoder comprises a prenet with a fully connected layer of dimension 256, two unidirectional LSTM layers with 1024 cells each, an 80-dimensional mel head with a 5-layer CNN postnet plus final linear layer, a visual head predicting visual features, and a gate output for end-of-sequence prediction. The multimodal decoder jointly predicts mel-spectrogram frames and visual features, yielding implicit synchronization between audio and articulatory outputs.

The visual representation is continuous rather than label-based. Lip landmarks comprise 42 2D points per frame and hand landmarks 21 2D points per frame, both extracted with Mediapipe. PCA reduces each stream to 10 principal components, retaining up to 99% variance, and the two streams are concatenated into a 20-dimensional visual vector per frame. Audio is resampled to 22,050 Hz and represented as an 80-band mel-spectrogram with window size 1024 and hop size 256; visual features are processed at the same rate as mel frames to enforce synchronous audiovisual sequences. The training objective is autoregressive,

$p(y \mid x)=\prod_{t=1}^{T} p(y_t \mid y_{<t},x),$

with MSE losses for mel, lips, and hand streams and binary cross-entropy for the gate.

Three transfer strategies were studied. S1 trains from scratch on CS. S2 warm-starts from a pre-trained audiovisual TTS model and fine-tunes encoder, decoder, and output heads; this caused catastrophic forgetting for lips, producing degenerate average lip-shape frames. S3 freezes the encoder and fine-tunes the decoder and heads; this preserved learned text-to-phoneme duration and alignment structure and yielded the best CS trajectories. Evaluation uses an automatic CS recognition system, with S1 reaching 17.06% phonetic decoding accuracy, S3 reaching 77.3% on CSF23, and approximately 71% on CSF22. The CSF23 dataset recorded for the study contains 2,654 videos, 66,664 phonetic samples, and 3.5 hours of material, using FullHD 1920×1080 video at stable 30 fps and a single professional French cuer. Stage-1 generation stops at text-to-landmark synthesis; Stage-2 rendering into photorealistic video or avatar animation is identified as future work.

From an agent-systems perspective, the resulting Cued-Agent accepts text, optionally synthesizes mel-spectrograms for audio, generates synchronized hand and lip trajectories, and can drive 2D or 3D rendering through inverse PCA and landmark retargeting. The paper also describes optional ACSR-based monitoring for quality control, GPU-feasible real-time or near-real-time operation, and personalization via lightweight adapter layers to capture user-specific timing, including hand anticipation of lips by up to approximately 120 ms.

3. Cued-Agent as collaborative multi-agent Cued Speech recognition

In "Cued-Agent: A Collaborative Multi-Agent System for Automatic Cued Speech Recognition" (Huang et al., 1 Aug 2025), the term denotes a recognition system rather than a generator. The task is Automatic Cued Speech Recognition (ACSR): mapping Cued Speech video into phoneme sequences and, for the first time in ACSR, end-to-end natural-language sentences. The central technical motivation is the temporal asynchrony between hand and lip movements, which makes frame-synchronous fusion brittle under limited-data conditions.

The system decomposes ACSR into four specialized sub-agents. The Multimodal LLM-based Hand Recognition agent performs keyframe screening and CS expert prompted decoding of hand positions and handshapes. The pretrained Transformer-based Lip Recognition agent extracts lip features from the input video and is finetuned with a hybrid CTC/attention objective. The Hand Prompt Decoding agent performs training-free fusion by combining lip posteriors with sparse hand prompts during beam search. The Self-Correction Phoneme-to-Word agent uses DeepSeek-R1 and CS rule prompts for iterative refinement from phoneme strings to Mandarin sentences.

The hand branch operates on hand ROI $X^h \in \mathbb{R}^{T \times W^h \times H^h}$ and centroid trajectory $X^r \in \mathbb{R}^{T \times 2}$. Slow-motion frames are selected using centroid speed $D_j = \mathrm{distance}(X_{j-1}^r, X_j^r)$ with threshold $\sigma = 6$, grouped with index threshold $\theta = 2$, and represented by group-center keyframes. MLLM predictions are embedded into a hand-prompt matrix $H \in \mathbb{R}^{T \times q}$. The lip branch outputs $L \in \mathbb{R}^{T \times d}$, linearly projected to $L' \in \mathbb{R}^{T \times q}$. Fusion is parameter-free at inference time:

$L^H = L' + \lambda H,$

with $X^h \in \mathbb{R}^{T \times W^h \times H^h}$0 and $X^h \in \mathbb{R}^{T \times W^h \times H^h}$1 in the joint CTC/attention beam search. This design uses hand cues to re-score CTC paths while keeping the attention branch lip-driven.

The dataset contribution is equally central. The work extends the Mandarin Chinese CS Dataset by collecting Multi-Hearing-Impaired MCCSD, adding eight hearing-impaired cuers and producing a mixed dataset of fourteen subjects. MCCSD contains three subsets: 6-cuer normal-hearing with 6,000 samples, 1-cuer normal-hearing with 1,000 samples, and 1-cuer hearing-impaired with 818 samples. MHI-MCCSD adds 5,272 annotated samples covering 1,010 unique sentences. Recordings use 1280×720 video at 30 fps in soundproof rooms, and data splits follow a 4:1 train/test split without sentence overlap.

Performance is reported with phoneme-level CER and WER, and sentence-level S-WER and Semantic Score.

Setting	CER / WER	S-WER / Semantic Score
MCCSD (1-H)	2.61% / 6.56%	12.1% / 89.23%
MCCSD (1-HI)	6.72% / 16.23%	24.38% / 76.48%
MCCSD (6-H)	9.05% / 20.54%	31.28% / 70.75%
MHI-MCCSD (8-HI)	12.67% / 29.86%	40.84% / 60.25%

Ablations attribute the gains to both hand prompting and semantic self-correction. On MCCSD (1-H), CER improves from 3.64 to 2.97 to 2.61 and WER from 10.36 to 8.55 to 6.56 as the system moves from pure lip decoding to hand information and then self-correction. On MHI-MCCSD (8-HI), CER improves from 18.41 to 13.86 to 12.67 and WER from 43.79 to 33.97 to 29.86. The paper states that these results outperform or match state-of-the-art methods and are especially effective in hearing-impaired settings, where lip-only recognition is more error-prone.

4. Cue-guided collaboration and memory-recall agents

Outside Cued Speech, cueing becomes a more general agent design principle. "Task-Aware Delegation Cues for LLM Agents" operationalizes a Cued-Agent as an LLM agent equipped with a task-aware collaboration signaling layer that converts offline human preference data into online, user-facing cues for delegation (Gu, 11 Mar 2026). Prompts from Chatbot Arena are embedded with Sentence-BERT, reduced with UMAP, and clustered with K-means into $X^h \in \mathbb{R}^{T \times W^h \times H^h}$2 task types. For each model $X^h \in \mathbb{R}^{T \times W^h \times H^h}$3 and cluster $X^h \in \mathbb{R}^{T \times W^h \times H^h}$4, the system computes a Capability Profile,

$X^h \in \mathbb{R}^{T \times W^h \times H^h}$5

and a Coordination-Risk Cue,

$X^h \in \mathbb{R}^{T \times W^h \times H^h}$6

At runtime, the agent types the task, surfaces a human-readable cluster label for common-ground verification, routes to a primary model, and escalates to a primary-plus-auditor regime when tie-rate is high or capability margins are small. Predictive probes show that task typing carries actionable structure: ridge logistic regression reaches approximately 0.548 accuracy for winner prediction, and removing cluster features reduces accuracy by approximately 0.007; ridge regression for difficulty prediction reaches approximately 2.463 MSE, and removing cluster features worsens MSE by approximately 0.104.

"MemoCue: Empowering LLM-Based Agents for Human Memory Recall via Strategy-Guided Querying" instantiates a different cueing paradigm: the agent transforms a vague query into a cue-rich one to activate the user’s own memory rather than relying on a finite external memory store (Zhao et al., 31 Jul 2025). The Recall Router uses a 5W Recall Map with five scenarios—Event, Person, Location, Temporal, and Decision—and fifteen strategy patterns. A fine-tuned RoBERTa classifier maps the original query to one scenario, after which a hierarchical recall tree with Monte Carlo Tree Search selects strategies and generates cue-rich responses. Selection uses the UCT score

$X^h \in \mathbb{R}^{T \times W^h \times H^h}$7

with $X^h \in \mathbb{R}^{T \times W^h \times H^h}$8 search iterations and top-$X^h \in \mathbb{R}^{T \times W^h \times H^h}$9 retained paths per query. Evaluation uses the Balance of Recall Score,

$X^r \in \mathbb{R}^{T \times 2}$0

with $X^r \in \mathbb{R}^{T \times 2}$1.

MemoCue is trained on the MemoStrategy instruction-tuning dataset, reported in the main text as 5,805 samples with 5,200 train and 605 test items. Across three representative datasets, the system surpasses LLM-based baselines by 17.74% in recall inspiration under LLM judging, achieves 90.36% average strategy prediction accuracy across five backbones, and shows ablation drops of approximately 3.28 without 5W mapping, approximately 4.96 without MCTS, and approximately 7.64 without both. These two papers together suggest that cues can function as explicit online coordination artifacts: in one case they expose delegation-relevant uncertainty and capability, and in the other they actively shape the user’s internal retrieval process.

5. Deployment, robustness, and adversarial evaluation in computer-use agents

A further branch of the literature uses Cued-Agent as a deployment target in computer-use environments. "Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability" treats a Cued-Agent as a deployment-grade computer-use agent whose design explicitly leverages cues from screen observations, DOM or accessibility structures, tool outputs, memory traces, and permission surfaces to construct trustworthy executable state (Chen et al., 8 May 2026). The paper formalizes the agent as a policy $X^r \in \mathbb{R}^{T \times 2}$2 over a software-facing environment $X^r \in \mathbb{R}^{T \times 2}$3 with actions $X^r \in \mathbb{R}^{T \times 2}$4, and organizes reliability around a Perception–Decision–Execution architecture and a Creation–Deployment–Operation–Maintenance lifecycle. The central claim is that reliability is not captured by task success alone; perception errors, planning drift, memory use, tool mediation, permission scope, and runtime oversight jointly determine whether agent actions remain aligned with user intent.

Empirical security work shows why such controls are necessary. "RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments" presents a hybrid sandbox combining a VM-based OS environment with Docker-based web platforms and introduces RTC-Bench with 864 examples spanning 9 benign goals, 24 adversarial goals, and 4 instantiations (Liao et al., 28 May 2025). Under decoupled evaluation, Claude 3.7 Sonnet | CUA reaches an ASR of 42.9%, Operator reaches 7.6%, and GPT-4o reaches an average ASR up to 66.2%; Attempt Rates peak at 92.5%. In end-to-end evaluation, Claude 3.7 Sonnet | CUA reaches ASR up to 50% and Claude 4 Opus | CUA approximately 48%. The paper states explicitly that it did not evaluate a system named Cued-Agent, but frames these results as directly relevant to any such agent deployed in hybrid web–OS workflows.

"Code Agent can be an End-to-end System Hacker: Benchmarking Real-world Threats of Computer-use Agent" extends this threat picture to terminal-based OS control with AdvCUA, a benchmark of 140 tasks: 40 direct malicious tasks, 74 TTP-based malicious tasks, and 26 end-to-end kill chains (Luo et al., 8 Oct 2025). The benchmark aligns tasks with 77 techniques across 10 ATT&CK tactics on a multi-host Ubuntu 22.04 sandbox and replaces LLM-as-a-Judge with hard-coded Match, Trigger, Probe, and Verify protocols. It reports Bypass Success Rate and Attack Success Rate:

$X^r \in \mathbb{R}^{T \times 2}$5

$X^r \in \mathbb{R}^{T \times 2}$6

On TTP tasks, Cursor CLI achieves the highest average ASR at 69.59%, surpassing ReAct at 52.29% and Cursor IDE at 51.66%; on end-to-end kill chains, Cursor IDE attains the highest average ASR at 34.62%, followed by Cursor CLI at 26.93% and ReAct at 23.37%. These papers collectively suggest that cue-centric design does not remove the need for safe authority binding, least privilege, provenance controls, hard-coded verification, and continual red-teaming.

6. Other technical usages and disambiguation

The term also appears in more distant literatures. "Cued to Queue: Information in Waiting-Line Auctions" studies agents whose queue-entry decisions are shaped by public information cues about queue length and makes the connection explicit by describing the setting as one involving “Cued-Agents” (Hirsch et al., 26 Feb 2025). Here the cue is not multimodal perception but belief update. A core notion is “sudden bad news,” defined by a shift in beliefs over items remaining such that $X^r \in \mathbb{R}^{T \times 2}$7 is strictly FOSD-dominated by $X^r \in \mathbb{R}^{T \times 2}$8 and $X^r \in \mathbb{R}^{T \times 2}$9 is weakly FOSD-dominated by future beliefs. The main welfare result is distribution-sensitive: if the hazard rate is weakly increasing, information policies that release sudden bad news increase total surplus relative to no information; if the hazard rate is weakly decreasing, such policies reduce total surplus. With entry costs and a decreasing hazard rate, announcing only when the queue is full maximizes total surplus.

By contrast, "Cued@wmt19:ewc&lms" is not a cue-centric agent paper in the same sense (Stahlberg et al., 2019). There, CUED denotes the Cambridge University Engineering Department, and the system is a WMT19 machine translation submission built from Transformer baselines, Elastic Weight Consolidation, checkpoint averaging, sentence-level and document-level LLMs, and a source-conditioned $D_j = \mathrm{distance}(X_{j-1}^r, X_j^r)$0-gram LM extracted from SMT lattices. Its primary submissions achieve 43.0 BLEU for English→German and 39.7 BLEU for German→English. The coexistence of this naming lineage with cue-centered agent work makes explicit disambiguation essential.

Viewed across all of these uses, Cued-Agent names either a concrete multimodal Cued Speech system or, more broadly, an agent whose operation is materially shaped by cues: articulatory cues, hand prompts, delegation signals, memory-recall prompts, queue-length announcements, or software-environment observations. The shared motif is the elevation of cues from passive context to an explicit computational object that modulates inference, routing, synchronization, or control.