StreamME: Dual Streaming Technologies

• StreamME is a term representing two distinct systems that convert live streaming media into structured artifacts for dialogue generation and avatar reconstruction.
• The dialogue-mining branch employs adaptive persona synthesis and blueprint-guided interactive generation to create task-oriented service dialogues from diverse public streams.
• The 3D avatar reconstruction branch uses on-the-fly geometry-centric 3D Gaussian splatting to synchronously capture and render live head avatars without pre-cached data.

Searching arXiv for the relevant papers on “StreamME” and closely related entries. StreamME is used for two technically distinct systems in recent arXiv literature. In one usage, it is a practical instantiation of the STREAM framework that turns publicly available streaming media into high-value, task-oriented service dialogues by mining authentic interaction signals and synthesizing structured multi-turn sessions for vertical domains such as Automotive, Restaurant, and Hotel (Xue et al., 24 May 2026). In another usage, it is a geometry-centric, on-the-fly 3D Gaussian Splatting system tailored for live-stream head avatars, designed to synchronously record and reconstruct a head avatar from live video streams without any pre-cached data (Song et al., 22 Jul 2025). The shared label reflects a common orientation toward live, streaming inputs, but the two systems address different problems: dialogue data construction and real-time avatar reconstruction.

1. Two established usages

In the cited literature, StreamME names both a data-centric dialogue-mining system and a live-stream avatar-reconstruction method. The former is framed around task-oriented dialogue scarcity, privacy constraints, annotation cost, and temporal staleness; the latter is framed around fast monocular head-avatar reconstruction, privacy-preserving on-device training, and bandwidth reduction for VR systems or online conference (Xue et al., 24 May 2026, Song et al., 22 Jul 2025).

Usage	Core function	Reference
STREAM-derived StreamME	turns publicly available streaming media into high-value, task-oriented service dialogues	(Xue et al., 24 May 2026)
3DGS StreamME	fast 3D avatar reconstruction from live video streams	(Song et al., 22 Jul 2025)

A recurrent misconception is to treat the name as if it referred to a single architecture. The literature instead presents separate systems with different inputs, objectives, and evaluation protocols. One system centers on streaming signal ingestion, persona synthesis, blueprint-guided dialogue generation, and downstream Dialogue State Tracking; the other centers on 3D Gaussian Splatting, geometry-only adaptation, motion-aware anchors, and real-time rendering.

2. StreamME as a dialogue-mining system

Within the STREAM line of work, StreamME is a practical instantiation of a framework intended to resolve the task-oriented dialogue trilemma: complex domain-specific dialogues are scarce; expert annotation is costly; real service conversations are gated by privacy and commercial restrictions; and static corpora rapidly become stale (Xue et al., 24 May 2026). The motivating claim is that traditional TOD corpora emphasize slot-filling and canonical flows but lack realistic behaviors such as deep requirement mining, constraint conflicts, negotiation, and recovery, while many vertical domains require strategic guidance, professional knowledge, and decision-making that crowdsourcing rarely captures.

The pipeline comprises four phases: Streaming Signal Ingestion (SSI), Adaptive Persona Synthesis (APS), Conversational Blueprinting (CB), and Interactive Dialogue Generation (IDG). SSI aggregates three heterogeneous sources—web pages, live streams, and short videos—and extracts five atomic interaction signals: User Questions $(Q_u)$, Agent Responses $(R_a)$, QA Pairs $(P_{qa})$, Dialogue Strategies $(S_{tag})$, and Account Metadata $(I_{acc})$. Platforms monitored include Douyin, Kuaishou, Xiaohongshu, and Ctrip, and more than 320K candidate public accounts, rooms, videos, and pages were screened. ASR outputs were normalized using large domain lexicons, including a 4.1M-entry automotive vocabulary and 43K address entries for location-sensitive hotel and restaurant content. Evidence checking and correction used retrieval-based evidence verification plus LLM-assisted correction for entities such as names, prices, dates, locations, and configurations.

APS constructs paired personas grounded in real signals rather than templates. User Persona Modeling builds $P_u$ as a structured representation of live user questions plus seed dialogues, covering mindset and goals, basic information, core requirements, and plausible utterance variants. Agent Persona Modeling builds $P_a$ from account metadata and professional interaction patterns, encoding role and service scope, style and strategy, and a domain knowledge base $\mathcal{R}$. This pairing is intended to align the agent’s expertise and constraints with the user’s objectives.

Conversational Blueprint construction produces $B$, described as a strategic, executable plan rather than a mere turn template. It encodes an Overall Rhythm Overview, Key Node Definitions, Typical Scenarios and Coping Strategies, and a Dialogue Flow Atlas. IDG then synthesizes multi-turn dialogue histories $H$ by combining $(R_a)$0, $(R_a)$1, and $(R_a)$2 through multi-agent simulation and retrieval-augmented generation. Dialogue Opening Synthesis extracts and rewrites opening patterns from seed dialogues; User Utterance Simulation and Agent Response Generation exchange evidence in a bidirectional loop, grounding turns in retrieved behavioral evidence $(R_a)$3 and in the knowledge base $(R_a)$4.

Each session is represented as a quadruplet $(R_a)$5, and the dataset is formalized as

$(R_a)$6

Dialogue history is updated turn by turn as

$(R_a)$7

For diversity control, the framework constructs a similarity graph over complete dialogues and defines an edge only when both user-side and agent-side aggregated representations exceed thresholds:

$(R_a)$8

Community detection is then used to identify redundancy clusters and preserve nuanced diversity while reducing duplication.

3. Data representation, evaluation, and deployment of the STREAM-derived system

The resulting dataset, StreamDial, covers Automotive, Restaurant, and Hotel, and contains 87,498 dialogue sessions and 1,497,320 turns in total, with an average of 17.11 turns per session and a comparable scale across domains (Xue et al., 24 May 2026). Automotive contains 29,486 dialogues and 566,095 turns with 19.20 average turns; Restaurant contains 27,389 dialogues and 450,703 turns with 16.46 average turns; Hotel contains 30,623 dialogues and 480,522 turns with 15.69 average turns. Intermediate artifacts are retained: Automotive has $(R_a)$9, $(P_{qa})$0, and $(P_{qa})$1; Restaurant has $(P_{qa})$2, $(P_{qa})$3, and $(P_{qa})$4; Hotel has $(P_{qa})$5, $(P_{qa})$6, and $(P_{qa})$7.

Quality control emphasizes high-interaction source prioritization, removal of low-signal content, ASR normalization with large lexicons, retrieval-based evidence checking, LLM-assisted correction for entities and values, and joint temporal, semantic, and entity-level consistency checks. Example behaviors include inventory checks, production-date inquiries, reservation or holding requests, freebie negotiation, follow-up arrangements, and non-linear progress such as mid-dialogue requirement shifts and solution transitions. This suggests that the framework is designed to preserve business-relevant strategic variation rather than only canonical booking or inquiry flows.

Intrinsic evaluation uses LLM-as-a-Judge on six dimensions—Coherence, Informativeness, Naturalness, Diversity, Flexibility, and Overall—with Qwen3-Max, GPT-5.2, and Gemini3-Pro as judges. Representative overall gains from Baseline to Ours are substantial: in Automotive, Qwen3-Max 6.32 to 8.98, GPT-5.2 5.89 to 7.81, and Gemini3-Pro 6.91 to 8.82; in Restaurant, 7.15 to 8.81, 7.01 to 7.93, and 7.99 to 8.85; in Hotel, 7.31 to 8.72, 7.03 to 7.89, and 7.96 to 8.92. A completed human-evaluation set includes 1,431 sessions and 53 annotators, with anonymized, randomized presentation and hidden source labels; the outcome is reported as consistent with automatic judges, with StreamDial rated more informative and flexible while maintaining comparable coherence and naturalness.

Extrinsic evaluation targets Dialogue State Tracking under a controlled training budget of 2,000 dialogues per run. With public test results reported as Joint Goal Accuracy and Slot-value F1, Qwen3-1.7B improves from 93.43/98.81 to 96.97/99.37 in the +Hybrid setting; Qwen3-8B improves from 93.94/98.56 to 96.72/99.41; Gemma3-4B improves from 95.71/98.78 to 97.98/99.54. On X-RiSAWOZ Automotive with Qwen3-8B, multilingual transfer is reported from 84.85/95.43 to 87.63/96.86 in English, 80.30/94.23 to 81.06/94.45 in French, and 84.85/95.98 to 85.61/96.03 in Korean. An Automotive slot-distribution analysis further reports average distinct slot values rising from baseline 6.8 to seed 18.4, stream 21.1, and hybrid 24.2.

The deployment recipe given for StreamME follows six stages: source onboarding and compliance; SSI ingestion and denoising; APS persona construction; CB blueprinting; IDG synthesis with RAG; and packaging and evaluation. Sessions are stored as $(P_{qa})$8, and periodic human validation, slot-coverage audits, and evidence-linked audit trails are recommended. The data release is hosted at the repository specified in the paper.

4. StreamME as a live-stream 3D Gaussian avatar system

In a separate line of work, StreamME is a method for fast 3D avatar reconstruction from live video streams, built upon 3D Gaussian Splatting and centered on what the authors call on-the-fly training (Song et al., 22 Jul 2025). The method synchronously records and reconstructs a head avatar from live video streams without any pre-cached data, and it eliminates the reliance on MLPs in deformable 3DGS, relying solely on geometry. The intended setting is monocular live video at $(P_{qa})$9 resolution, with a real-time face and pose tracker providing per-frame head pose and a deformed head mesh; background removal uses RobustVideoMatting.

The system initializes 3D Gaussian points by UV mesh sampling on the tracked mesh surface, with the first frame’s mesh defining the canonical point cloud $(S_{tag})$0. An optional warm-up of approximately 20–30 seconds learns auxiliary Tex and SH parameters to refine coarse geometry and lighting; after warm-up, Tex and SH are frozen. During streaming, each iteration uses the latest captured frame plus a small set of buffered previous frames to reduce catastrophic forgetting, and clone/prune updates occur every 1500 iterations.

The central modeling choice is geometry-only adaptation. Gaussians are attached to the head mesh surface in canonical space, and per-frame mesh deformations and head pose move the Gaussians directly, without any neural deformation field. For a Gaussian with mean $(S_{tag})$1, covariance $(S_{tag})$2, opacity $(S_{tag})$3, and appearance parameters $(S_{tag})$4, the density is

$(S_{tag})$5

Covariance is parameterized as

$(S_{tag})$6

and the projected 2D covariance is

$(S_{tag})$7

Rendering uses standard 3DGS front-to-back compositing:

$(S_{tag})$8

A distinguishing feature is the primary points simplification strategy. Primary points are motion-aware anchors selected from UV-sampled mesh vertices and their bound Gaussians. Selection is driven by

$(S_{tag})$9

with $(I_{acc})$0. The binary index gates scale and opacity:

$(I_{acc})$1

implemented with a straight-through estimator and a hard threshold on $(I_{acc})$2 with $(I_{acc})$3. Points with $(I_{acc})$4 are cloned for local densification; points with $(I_{acc})$5 are pruned. A point-to-surface regularizer keeps points near the facial surface at anchors:

$(I_{acc})$6

This mechanism is intended to remove motion-irrelevant and over-small Gaussians while preserving motion-critical ones in regions such as the mouth and eyes.

5. Optimization, efficiency, and downstream applications of the avatar system

The training loss combines photometric, structural, and surface terms:

$(I_{acc})$7

where $(I_{acc})$8 is photometric L1, $(I_{acc})$9 is the SSIM-based term, $P_u$0 is the surface regularizer, and $P_u$1 is a warm-up dark channel term used to separate Tex and SH and stabilize lighting and texture (Song et al., 22 Jul 2025). Optimization uses Adam on 3DGS attributes $P_u$2 with learning rates $P_u$3, while warm-up uses learning rates $P_u$4 for $P_u$5.

The reported system metrics emphasize both convergence speed and rendering throughput. The teaser reports basic appearance within 10 seconds at PSNR approximately 21 dB and high quality in about 5 minutes at PSNR approximately 26 dB. On self-reenactment at $P_u$6 on RTX4090, StreamME reports PSNR 29.7 dB, LPIPS 0.095, FPS 139, memory 2.52 MB, and MOS scores 3.9 for Motion Synchronization and 4.1 for Video Quality. The same table reports FlashAvatar at 27.8 dB, 0.109, 94.5 FPS, and 12.6 MB; GaussianBlendshape at 26.4 dB, 0.112, 22.9 FPS, and 872 MB; and AvatarMAV at 24.1 dB, 0.137, 2.58 FPS, and 14.1 MB. Training dynamics on the test set are reported as PSNR/iterations of 10.8 / $P_u$7 at 1 second, 23.1 / $P_u$8 at 10 seconds, 27.2 / $P_u$9 at 2 minutes, and 29.8 / $P_a$0 at 30 minutes.

Simplification is presented as a major contributor to efficiency. After approximately 10 minutes, simplification reduces points by 8–10×, from 200k to 24k, without quality loss; the average converged point count is 9,807, compared with 13,453 for FlashAvatar and 62,530 for GaussianBlendshape. Rendering runs at 139 FPS on RTX4090 for $P_a$1 inference, and streaming only 3DGS parameters instead of full frames yields approximately 70% compression. A plausible implication is that the method’s efficiency claims depend jointly on geometry-only adaptation, aggressive pruning, and the absence of MLP-based deformation fields.

The system is also positioned as directly applicable to downstream tasks such as animation, toonify, and relighting. Cross-identity driving is supported for animation; diffusion-based editing systems such as TextToon and PortraitGen are used in the toonify demonstration; and relighting uses the learned SH lighting basis and specular decomposition acquired during warm-up. The paper links these applications to immediate visualization and on-device training, allowing users to re-record under-trained regions and reducing the need to transmit raw facial video.

6. Limitations, governance, and broader significance

The two StreamME usages share a reliance on public or live streaming inputs, but their limitations differ. In the STREAM-derived system, noise and uneven source quality remain; residual ASR errors and colloquial ambiguity can propagate despite denoising, lexicon normalization, and evidence checking. Domain coverage is limited to Automotive, Restaurant, and Hotel, and expansion to healthcare, legal, or education is said to raise additional governance requirements. Ethical use is stated to require continued transparency and governance, including documentation of filtering criteria, respect for platform terms, and stronger privacy safeguards (Xue et al., 24 May 2026).

In the avatar-reconstruction system, the main constraints arise from monocular capture. The paper states that single-view input leads to incomplete 3D structure and artifacts under extreme poses or occlusions, while monocular constraints limit high-fidelity global illumination and complex hair geometry. Quality also depends on the distribution of captured expressions and poses. These limitations temper the privacy and bandwidth claims: on-device training protects facial privacy and parameters-only transmission reduces bandwidth, but reconstruction fidelity remains conditioned by the available viewpoint and motion coverage (Song et al., 22 Jul 2025).

Taken together, the literature presents StreamME as a label for systems that operationalize streaming media rather than treat it as a passive corpus. In one case, streaming media becomes a continuously refreshed source of authentic interaction signals for synthesizing strategy-rich task-oriented dialogues; in the other, live video becomes the substrate for immediate, geometry-driven avatar formation. The common theme is not a shared model family, but the conversion of live or public streams into structured artifacts—dialogue sessions in one case, animatable 3D Gaussian avatars in the other—under explicit constraints of latency, privacy, timeliness, and system efficiency.