Multi-Source Inner Guidance

Updated 4 July 2026

Multi-source inner-guidance is an architectural principle that injects diverse, complementary signals directly into internal computational loops for robust adaptation and control.
It employs parallel pathways—such as multimodal inputs, expert subnetworks, and differentiable energy functions—to merge distinct guidance signals through varied injection loci.
The approach is applied in wearable support, domain adaptation, robotics, TTS, and introspection, balancing complementary inductive biases and fusion strategies.

to=arxiv_search 天天中彩票能json {"query":"\"multi-source inner-guidance\" OR \"Mirai: A Wearable Proactive AI \\"Inner-Voice\\" for Contextual Nudging\" OR OmniGuide universal guidance fields", "max_results": 10} The cited literature suggests that multi-source inner-guidance denotes a family of mechanisms in which multiple guidance signals are injected into an internal computational loop—such as adaptation, denoising, control, or dialogue orchestration—rather than being used only for offline supervision or post hoc output selection. The “sources” may be multimodal observations, multiple source domains, expert subnetworks, differentiable energy fields, distinct LLM personas, or disentangled conditioning channels. Across wearable behavioral support, multi-source domain adaptation, frequency-domain fault diagnosis, robot policy steering, hand-focused diffusion guidance, introspective multi-agent systems, and controllable text-to-speech, the unifying theme is internal steering by heterogeneous signals with complementary roles (Fang et al., 4 Feb 2025, Li et al., 2020, Tu et al., 1 Feb 2025, Song et al., 9 Mar 2026, Eum et al., 11 Mar 2025, Jeon et al., 29 Mar 2026, Yin et al., 10 Dec 2025, Lin et al., 2021).

1. Scope and typology

The literature does not present a single canonical definition of multi-source inner-guidance. Instead, the term is realized through several recurring forms. In some systems, multiple sensory or linguistic streams are merged symbolically inside a prompting loop. In others, parallel subnetworks or branch-specific experts are coordinated by a guidance network or by staged residual transfer. In generative models, guidance is often injected directly into the inference trajectory through gradients, energy terms, or chained conditional updates. In dialogical systems, guidance may be instantiated as multiple internal voices represented by separate agents (Fang et al., 4 Feb 2025, Li et al., 2020, Tu et al., 1 Feb 2025, Song et al., 9 Mar 2026, Eum et al., 11 Mar 2025, Jeon et al., 29 Mar 2026, Yin et al., 10 Dec 2025, Lin et al., 2021).

System	Guidance sources	Injection locus
Mirai (Fang et al., 4 Feb 2025)	Egocentric video, live speech, cloned voice persona	Final GPT-4o prompt and proactive speech loop
ML-MSDA (Li et al., 2020)	$N$ source-target branches plus one guidance network	Target prediction alignment via $L_M$
FARNet (Tu et al., 1 Feb 2025)	Amplitude spectrum, phase spectrum, FSIM interactions	Amplitude-to-phase residual guidance
OmniGuide (Song et al., 9 Mar 2026)	Semantic, collision, and human-demo energies	Flow-matching denoising updates
MGHanD (Eum et al., 11 Mar 2025)	Discriminator guidance, LoRA textual guidance, cumulative hand mask	DDIM reverse process
InnerPond (Jeon et al., 29 Mar 2026)	Multiple “I-positions” as LLM agents	Prompt-based orchestration in shared space
DMP-TTS (Yin et al., 10 Dec 2025)	Content, timbre, style conditions	Chained classifier-free guidance
Triple M (Lin et al., 2021)	Basic, forward, and GMM attention	Training-time alignment guidance

This taxonomy indicates that “multi-source” refers less to a fixed sensor configuration than to the coexistence of multiple internal signals with distinct inductive biases. A plausible implication is that the concept is best understood as an architectural principle rather than a domain-specific algorithm.

2. Internal loci of guidance

A central distinction across the literature is where guidance enters the system. Mirai uses two separate pipelines—vision and speech—that are merged symbolically via GPT prompts. Camera and audio run in parallel, and their outputs join only at the final GPT-4o prompt. The system is organized around three major components linked by a WebSocket bus: a User Modeler, a Context-Aware Agent, and a Proactive Speech Agent (Fang et al., 4 Feb 2025).

ML-MSDA places guidance at the level of domain adaptation. It builds $N+1$ subnetworks, each with a feature extractor $G_j(\cdot)$ , domain discriminator $D_j(\cdot)$ , and category classifier $F_j(\cdot)$ . The $N$ branch networks pair each single source with the target domain, while the guidance network pairs the combined multi-source domain with the target domain. Weight-sharing is limited to the first few layers of the feature extractors, while discriminators and classifiers remain separate. Guidance is therefore neither purely early fusion nor purely late fusion; it is mediated by coordinated target-side prediction alignment (Li et al., 2020).

FARNet uses a cascaded inner-guidance arrangement. The amplitude spectrum sub-network first reconstructs the exemplar domain’s amplitude style, and its output is then used to rebuild the input to the phase spectrum sub-network. Simultaneously, the residual $x_{out1}-x_{in}$ is injected by $1\times1$ convolutions into the early FSIM blocks of the phase network. Here, guidance is serial and asymmetric: the amplitude side normalizes “style,” and the phase side is then pushed toward structural alignment under that normalization (Tu et al., 1 Feb 2025).

In OmniGuide, MGHanD, and DMP-TTS, guidance is injected directly into iterative inference. OmniGuide modifies the denoising vector field of a flow-matching VLA policy by subtracting a clipped gradient of the trajectory energy. MGHanD perturbs the DDIM reverse trajectory with a discriminator-derived latent correction and a LoRA-induced textual shift, both masked to the evolving hand region. DMP-TTS evaluates multiple conditional branches at each diffusion step and recombines them through chained classifier-free guidance, so that content, timbre, and style can be scaled independently (Song et al., 9 Mar 2026, Eum et al., 11 Mar 2025, Yin et al., 10 Dec 2025).

Triple M occupies a different point in the design space. Its multi-guidance attention does not fuse multiple attention distributions at runtime. Instead, forward attention and GMM-based attention act as training-time teachers that shape a single deployable location-sensitive attention via a guidance loss. This is a distilled form of inner-guidance: multiple expert signals are present during learning, but only one attention mechanism remains at inference (Lin et al., 2021).

InnerPond transfers the idea to introspection. It externalizes multiple “I-positions” as LLM-based lotus-leaf agents, then coordinates them through prompt templates and lightweight state tracking. The backend provides I-Position Extraction, Single-Agent Interactions, and Multi-Agent Orchestration, but no formal mathematical scoring function. Coordination relies entirely on prompt-based role enforcement (Jeon et al., 29 Mar 2026).

3. Formal mechanisms

The formalization of inner-guidance ranges from simple rule-based gating to full differentiable energy shaping. Mirai does not provide an explicit learned fusion function $f(x_{\text{camera}},x_{\text{audio}},x_{\text{motion}})$ , and it uses no motion sensors or numeric vector fusion. Vision features become discrete scene descriptions; audio features become transcripts; fusion is the combined prompt of $L_M$ 0. The only explicit mathematical mechanism is the debouncer:

$L_M$ 1

where $L_M$ 2, $L_M$ 3 is the frame-batch counter, and $L_M$ 4 triggers injection of $L_M$ 5 into the speech agent. Mirai therefore anticipates intent through a rule-based wrapper around GPT classifications rather than through a probabilistic time-series model (Fang et al., 4 Feb 2025).

ML-MSDA formalizes inner-guidance through mutual learning between each branch network and the guidance network. After conditional adversarial alignment, it minimizes a symmetric KL term on target predictions:

$L_M$ 6

The full objective is

$L_M$ 7

This design lets branches adapt locally to source-specific shifts while the guidance network aggregates globally across combined sources (Li et al., 2020).

FARNet formalizes inner-guidance in frequency space. Signals are decomposed into amplitude and phase:

$L_M$ 8

with

$L_M$ 9

The amplitude and phase reconstruction losses are

$N+1$ 0

combined as

$N+1$ 1

Its manifold triplet loss further warps Euclidean distances through a piecewise scaling $N+1$ 2, then optimizes

$N+1$ 3

Guidance is thus both reconstructive and metric-shaping (Tu et al., 1 Feb 2025).

OmniGuide expresses each guidance source $N+1$ 4 as an energy over the predicted Cartesian trajectory $N+1$ 5:

$N+1$ 6

and aggregates them as

$N+1$ 7

The denoising update becomes

$N+1$ 8

The framework can instantiate semantic attractors, collision repellers derived from a signed distance field, and human-demo attractors, all as differentiable guidance terms in $N+1$ 9 (Song et al., 9 Mar 2026).

MGHanD also performs inner guidance during denoising, but in latent diffusion. Its update can be written as

$G_j(\cdot)$ 0

The visual term is a discriminator gradient computed by “forward guidance,” the textual term comes from a LoRA adapter, and both are restricted by the cumulative mask

$G_j(\cdot)$ 1

Guidance is therefore localized in space and scheduled in time (Eum et al., 11 Mar 2025).

DMP-TTS formalizes multi-source inner-guidance as chained classifier-free guidance. With unconditional, text-only, text-plus-speaker, and text-plus-speaker-plus-style branches,

$G_j(\cdot)$ 2

where $G_j(\cdot)$ 3, $G_j(\cdot)$ 4, and $G_j(\cdot)$ 5 independently scale content, timbre, and style. Hierarchical condition dropout during training ensures that the model learns all intermediate subsets of conditions (Yin et al., 10 Dec 2025).

Triple M uses a guidance loss rather than runtime fusion:

$G_j(\cdot)$ 6

The basic location-sensitive attention is pulled toward forward-attention and GMM-attention alignments during training, but inference uses only the basic attention. This sharply distinguishes training-time guidance transfer from runtime multi-branch composition (Lin et al., 2021).

InnerPond is an explicit counterexample to the assumption that inner-guidance must be formalized numerically. No formal mathematical equations or scoring functions appear in the paper; turn-taking, relation classification, and depth interventions are handled by prompt templates and lightweight state tracking (Jeon et al., 29 Mar 2026).

4. Domain-specific realizations

In wearable behavior support, Mirai operationalizes inner-guidance as an “inner-voice” delivered in the user’s own cloned voice. Its prototype uses an egocentric camera, always-on audio capture, GPT-4o scene description and goal-relevance classification, Deepgram Speech-to-Text, and ElevenLabs “Instant Voice Cloning.” The system demonstrates three application scenarios: healthy eating, productivity/focus, and confident communication. Example nudges include “My body deserves better than this,” “I need to put the phone down and focus on my code,” and “For instance, I manage the project deadlines” (Fang et al., 4 Feb 2025).

In domain adaptation and domain generalization, inner-guidance addresses heterogeneity across source domains rather than moment-to-moment behavioral state. ML-MSDA pairs each source domain with the target domain and adds one guidance network trained on the combined multi-source domain and target domain. FARNet, by contrast, uses Fourier-based multi-source augmentation and cascaded amplitude-to-phase guidance to build a robust generalized model for bearing fault diagnosis under unseen working conditions. The two works share the objective of preventing negative transfer or spurious correlations, but they do so with different internal control structures: prediction-level mutual learning in ML-MSDA and staged spectral reconstruction plus metric shaping in FARNet (Li et al., 2020, Tu et al., 1 Feb 2025).

In robotics, OmniGuide treats arbitrary sources of guidance—such as 3D foundation models, semantic-reasoning VLMs, and human pose models—as differentiable energy functions with task-specific attractors and repellers in 3D space. This lets a base VLA policy remain unchanged while action sampling is steered by external task knowledge backpropagated through differentiable kinematics or dynamics. The paper emphasizes challenging tasks involving complex spatial or semantic understanding, cluttered manipulation, and precise manipulation (Song et al., 9 Mar 2026).

In image generation, MGHanD targets a narrow but persistent failure mode of text-to-image models: realistic hand synthesis. Its multi-source inner-guidance combines a StyleGAN-T–style discriminator over DINOv2-ViT and CLIP-ViT features, a rank-4 LoRA adapter trained on hand-specific prompts, and a cumulative hand mask that is gradually enlarged during reverse diffusion. The hand region is therefore refined without fully overriding the pretrained model’s broader generative prior (Eum et al., 11 Mar 2025).

In speech synthesis, the term spans two distinct mechanisms. Triple M uses multi-guidance attention to improve long-sentence robustness in sequence-to-sequence TTS, while DMP-TTS uses chained classifier-free guidance to disentangle content, timbre, and style in a latent Diffusion Transformer. The latter additionally combines Style-CLAP, trained with contrastive learning and multi-task style supervision, with REPA alignment to Whisper representations. This suggests that inner-guidance in TTS may act either on alignment trajectories or on latent generative conditioning channels (Lin et al., 2021, Yin et al., 10 Dec 2025).

In introspection, InnerPond brings the concept closest to its psychological reading. Building on Dialogical Self Theory, it represents internal perspectives as distinct LLM-based lotus-leaf agents in a pond-like shared environment. Users can co-create approximately 10 “I-positions,” chat with them one-to-one, spatially arrange them through $G_j(\cdot)$ 7, and orchestrate dialogue between selected pairs. A concrete example juxtaposes “Myself, Yearning for Creative Freedom” and “Myself, Desiring Financial Abundance,” with the user mediating toward a hybrid strategy (Jeon et al., 29 Mar 2026).

5. Reported empirical evidence

The strength of evidence varies substantially across systems. Some papers report detailed quantitative gains with explicit baselines and ablations; others emphasize feasibility or design implications rather than statistical efficacy.

System	Reported results	Stated limitation or note
Mirai (Fang et al., 4 Feb 2025)	STT 100 ms; GPT-4o 450 ms; TTS 370 ms; total 920 ms	No precision/recall or user-study metrics
ML-MSDA (Li et al., 2020)	Outperforms comparison methods; achieves state-of-the-art performance	No specific numbers in the provided detail
FARNet (Tu et al., 1 Feb 2025)	57.9% / 55.3% baseline; 84.0% / 82.2% full FARNet	Best $G_j(\cdot)$ 8, $G_j(\cdot)$ 9
OmniGuide (Song et al., 9 Mar 2026)	Init only: +8% success, -18% collisions; denoising only: +20%, -34%; both: +26%, -46%	Denoising guidance found more effective
MGHanD (Eum et al., 11 Mar 2025)	FID 0.9601; KID 0.1368; Hand-conf 0.9009; Hand-prob 0.7250; CLIP-sim 30.1349	Guidance applied from $D_j(\cdot)$ 0 to $D_j(\cdot)$ 1
DMP-TTS (Yin et al., 10 Dec 2025)	Text prompt: 0.64 / 0.85 / 0.73 style accuracies; NMOS/QMOS 3.73 / 3.77; WER 0.038	Over-conditioning degrades naturalness
Triple M (Lin et al., 2021)	WER 3.0%; long-sentence failure 2%; MOS 4.57 ± 0.05	Only basic attention kept at inference
InnerPond (Jeon et al., 29 Mar 2026)	$D_j(\cdot)$ 2; additions $D_j(\cdot)$ 3; deletions $D_j(\cdot)$ 4; 1:1 dialogues $D_j(\cdot)$ 5; turns $D_j(\cdot)$ 6	No inferential statistics or $D_j(\cdot)$ 7-values

Mirai’s evidence is primarily systems-oriented. In 100 trials, with network effects excluded, the average latency is 920 ms, and the paper emphasizes that sub-second latency keeps interventions “non-disruptive.” The evaluation remains qualitative: three scenario demos and a video figure illustrate feasibility rather than statistical efficacy (Fang et al., 4 Feb 2025).

FARNet reports benchmark gains on CWRU and SJTU. Relative to a ResNet-18 baseline at average 57.9% / 55.3%, FARNet augmentation alone reaches 76.4% / 80.3%, adding standard triplet loss reaches 80.2% / 81.3%, and the full model with manifold triplet loss reaches 84.0% / 82.2%. The paper also reports that FSIM + augmentation + manifold-triplet outperforms simpler variants by 3–5 points (Tu et al., 1 Feb 2025).

OmniGuide presents ablations that isolate two guidance stages: initial-noise selection and denoising-time guidance. Initialization only yields +8% success and -18% collisions, denoising guidance only yields +20% success and -34% collisions, and combining both yields +26% success and -46% collisions over baseline. In a cluttered “Multi-Choice” scene, collision guidance raises safety from approximately 20% to 90%, semantic guidance raises success from approximately 30% to 75%, and combining both yields approximately 90% success and approximately 95% safety (Song et al., 9 Mar 2026).

MGHanD reports both automatic metrics and user preferences. Relative to Stable Diffusion, it improves FID from 1.0438 to 0.9601, KID from 0.1476 to 0.1368, Hand-conf from 0.8965 to 0.9009, and Hand-prob from 0.6708 to 0.7250, while maintaining CLIP-sim at 30.1349. In a user study with 40 users and 20 prompts, visual-quality preference is 44.1% for MGHanD versus 23.9% for HandRefiner, 19.6% for Stable Diffusion, and 10.0% for ConceptSlider; prompt-alignment preference is 36.4% for MGHanD (Eum et al., 11 Mar 2025).

DMP-TTS emphasizes disentangled controllability. Under text prompting, style accuracies are 0.64 for emotion, 0.85 for energy, and 0.73 for rate; NMOS/QMOS are 3.73 / 3.77; speaker similarity is 0.71 and WER is 0.038. Under audio prompting, NMOS/QMOS reach 3.82 / 3.83 with speaker similarity 0.72 and WER 0.043. Ablations show that removing multi-task supervision reduces emotion from 0.64 to 0.54 and energy from 0.85 to 0.80, while removing REPA increases WER from 0.038 to 0.046 (Yin et al., 10 Dec 2025).

Triple M reports a reduction in long-sentence failure rate from more than 60% for the baseline to 2% for multi-guidance, with WER improving to 3.0% and a stated 26.8% relative WER reduction versus GMM-only attention. On in-domain MOS, multi-guidance scores 4.57 ± 0.05 versus 4.52 ± 0.08 for the baseline and 4.42 ± 0.08 for GMM-only, compared with 4.65 ± 0.04 for ground truth (Lin et al., 2021).

InnerPond’s evidence is qualitative and interaction-log based rather than inferential. The study involves 17 young adults choosing between two career paths, a three-session protocol, system logs, think-aloud data, field notes, and interviews. The paper explicitly reports thematic coding and states that no inferential statistics or $D_j(\cdot)$ 8-values are reported (Jeon et al., 29 Mar 2026).

6. Limitations, misconceptions, and open directions

A common misconception is that multi-source inner-guidance necessarily implies a learned multimodal fusion network. Several cited systems contradict this. Mirai has no explicit learned fusion function and no probabilistic $D_j(\cdot)$ 9; its core decision to speak is a logical wrapper around GPT classifications. InnerPond has no formal coherence metric or numeric turn-taking formula. Triple M does not fuse multiple attention mechanisms at runtime; it transfers their properties into a single inference-time attention through training losses (Fang et al., 4 Feb 2025, Jeon et al., 29 Mar 2026, Lin et al., 2021).

A second misconception is that more guidance always implies more stable or better behavior. The literature instead describes calibration problems. DMP-TTS reports that varying $F_j(\cdot)$ 0 or $F_j(\cdot)$ 1 from 6 to 21 produces monotonic increases in speaker similarity or emotion accuracy until over-conditioning degrades naturalness, with a mid-range of approximately 12–15 recommended for balanced quality versus control. MGHanD activates visual and textual guidance only from step $F_j(\cdot)$ 2 to $F_j(\cdot)$ 3, and restricts them with a growing hand mask, which suggests that temporally and spatially localized guidance is beneficial when global priors should be preserved (Yin et al., 10 Dec 2025, Eum et al., 11 Mar 2025).

A third misconception is that “inner-guidance” always refers to psychological self-dialogue. InnerPond indeed treats the self as composed of multiple internal perspectives under Dialogical Self Theory, but the same phrase also appears in robot control, TTS, and domain adaptation to describe entirely different internal coordination mechanisms. This suggests that the term is polysemous across subfields: its common denominator is internalized steering by multiple sources, not any single ontology of the “inner” (Jeon et al., 29 Mar 2026, Song et al., 9 Mar 2026, Li et al., 2020).

The current evidence base is uneven. Mirai demonstrates sub-second latency and feasibility but explicitly calls for future work on improving the proactive agent via human feedback and for a longitudinal study in naturalistic settings. InnerPond reports rich qualitative findings but no inferential statistics. ML-MSDA states state-of-the-art performance in the abstract, but the supplied detail emphasizes formulation rather than benchmark numbers. Even in more quantitatively mature systems, such as OmniGuide, MGHanD, DMP-TTS, and Triple M, the reported gains are specific to their own deployment context and guidance schedule (Fang et al., 4 Feb 2025, Jeon et al., 29 Mar 2026, Li et al., 2020, Song et al., 9 Mar 2026, Eum et al., 11 Mar 2025, Yin et al., 10 Dec 2025, Lin et al., 2021).

Taken together, the literature presents multi-source inner-guidance as a broad systems pattern with several stable properties: heterogeneous sources remain at least partially distinct; guidance enters an internal loop rather than acting only externally; complementary signals are coordinated by rules, losses, prompts, or differentiable energies; and empirical success depends strongly on where, when, and how guidance is injected. A plausible implication is that future work will continue to separate source plurality from fusion strategy, treating the former as a source of complementary inductive bias and the latter as the central design problem.