Immersive Voice & Audio Services (IVAS)

• IVAS is a 3GPP codec family that enables mobile, low-latency immersive communication using spatial audio coding and metadata-assisted scene descriptions.
• It harmonizes analysis filterbanks, transport protocols, and rendering techniques like stereo downmix and HOA coding to optimize bitrate, delay, and computational cost.
• The framework supports interactive XR experiences—such as telepresence, remote collaboration, and immersive narrative media—by co-designing codec and rendering semantics.

Searching arXiv for recent IVAS papers to ground the article. Searching for the specific ParamISM IVAS paper and related IVAS work. Immersive Voice and Audio Services (IVAS) is the 3GPP codec family designed for mobile, low-latency, low-bit-rate immersive communication, and it aims to standardize spatial, interactive, interoperable voice and audio services for real-time XR communication and shared experiences. In the cited literature, IVAS extends the EVS mono codec to support stereo, multichannel, scene-based audio, object-based audio, and metadata-assisted spatial audio under an immersive-mode end-to-end delay constraint not exceeding 40 ms; for HOA Scene-Based Audio (SBA), a total end-to-end delay of approximately 38 ms is reported. IVAS is therefore described not only as a coding family, but also as a framework in which low-footprint metadata, low-delay transport, interoperable scene description, and portable rendering semantics are co-designed for conversational and interactive use cases (Eichenseer et al., 7 Jul 2025, Jot et al., 2021, Llave et al., 23 Jun 2026).

1. Scope, standardization context, and design objectives

IVAS targets conversational and interactive XR scenarios in which each participant’s voice and environmental sounds are rendered as spatial audio objects, with synchronized scene state across devices. Its intended use cases include low-latency telepresence and communications in AR/VR, immersive games, interactive calls with head-tracked HOA scenes, co-presence, remote collaboration, and entertainment applications. Within this scope, IVAS can carry object waveforms, object metadata, and optionally HOA or Ambisonic scene layers, which makes it format-agnostic at the rendering layer and interoperable across platforms (Jot et al., 2021).

A central architectural feature is that IVAS harmonizes analysis filterbanks, transport conventions, and rendering components across coding modes. In Parametric ISM (ParamISM), for example, harmonized filterbanks and rendering are explicitly shared across IVAS modes to meet code-size, ROM, RAM, and WMOPS constraints, while preserving seamless decoding and spatial rendering to loudspeaker arrays or binaural. The same general design logic reappears in HOA coding, where spatial parameters are extracted at reduced temporal and frequency resolution and combined with a small number of decorrelated transport channels encoded by EVS. This suggests that IVAS is designed around a strict systems perspective: bitrate, latency, computational cost, and renderer consistency are treated as joint constraints rather than independent optimizations (Eichenseer et al., 7 Jul 2025, Llave et al., 23 Jun 2026).

The relation to EVS is fundamental but nontrivial. EVS codes each object independently in a multi-mono configuration and requires separate post-rendering, so bitrate and complexity scale linearly with object count. IVAS instead introduces metadata-assisted spatial coding modes in which waveform transport is reduced and spatial structure is partly represented parametrically. For object coding, this takes the form of a stereo downmix plus side information; for HOA, it takes the form of decorrelated transport channels plus spatial parameters. In both cases, IVAS departs from independent-channel coding by using joint scene structure to satisfy mobile constraints on bit rate, delay, and complexity (Eichenseer et al., 7 Jul 2025, Llave et al., 23 Jun 2026).

2. Scene representations, metadata, and interoperable rendering semantics

The IVAS-aligned object model described in the literature treats each audio object as a natural sound source having controllable position $\vec r$, orientation, size, radiation or directivity, and environment interactions such as distance, occlusion, obstruction, reflectors, and multiple rooms. Listener state is also part of the model, notably head pose and HRTF profile, because egocentric rendering depends on time-varying listener orientation and position. In metadata terms, IVAS can carry time-stamped object transforms, optional velocity vectors for Doppler, directivity descriptors, dry gain, reflections descriptors, reverb sends, occlusion or obstruction state, and references to waveform or HOA assets (Jot et al., 2021).

The same line of work proposes an open, portable scene description that separates a high-level scene graph from a low-level egocentric rendering API. To achieve interoperability, the model can be distilled into a minimal field set compatible with MPEG-I Audio scene metadata, ITU/EBU ADM, HOA/Ambisonics, and runtime control channels such as OSC and XR scene graphs based on glTF or OpenUSD. The emphasis is on transporting acoustically relevant fields rather than full geometric descriptions, thereby keeping payload small while preserving perceptually relevant control over distance, room behavior, and source directivity (Jot et al., 2021).

Research on immersive audiobook production adopts essentially the same object/scene dichotomy. In that framework, a Narrative Analysis Agent outputs an object/scene graph with timing; metadata includes ADM-like descriptors for object positions, scene bounds, reverberation parameters, and source types; and the system produces both object-based content and HOA-compatible scene content. This convergence is significant because it shows that IVAS-aligned metadata is not limited to conversational speech transport. It also supports narrative rendering, scene analysis, and endpoint adaptation while retaining interoperability with existing scene-description ecosystems (Selvamani et al., 8 May 2025).

At the rendering stage, IVAS-aligned systems support both object-based and HOA decoding. For HOA, binaural rendering can be written as

$y_L = D_L B,\qquad y_R = D_R B,$

with head tracking applied via HOA rotation, while object-based binaural rendering can be expressed as

$$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$

The gain term includes geometric spreading, air absorption, Doppler, and near-field corrections. This representation makes explicit that IVAS is not only concerned with coding efficiency; it is equally concerned with preserving the rendering variables needed for audio/visual and virtual/real congruence (Jot et al., 2021).

3. Object-based low-bitrate coding: ParamISM

Within IVAS, object-based audio includes a parametric coding option, Parametric ISM, tailored for transmitting three or four arbitrarily placed audio objects efficiently at 24.4 or 32 kbit/s while preserving a convincing spatial image. ParamISM sends a stereo downmix plus compact side information derived jointly from object metadata and input object signals. The IVAS bitstream carries two independently coded downmix channels, each handled as a Single Channel Element by the IVAS core codec, together with parametric side information transmitted per 20 ms frame (Eichenseer et al., 7 Jul 2025).

The encoder operates in the time-frequency domain. Using an MDFT analysis filterbank, IVAS obtains $X_i(k,n)$ for each mono input object $x_i$, computes per-tile powers

$P_i(k,n)=|X_i(k,n)|^2,$

and aggregates them into $L=11$ psychoacoustically designed parameter bands:

$P_i(l)=\sum_n \sum_{k=B(l)}^{B(l+1)} P_i(k,n).$

For each band, the encoder selects the two objects with the highest powers and transmits their indices. The dominant-object power ratio is

$$r_1(l)=\frac{P_1(l)}{P_1(l)+P_2(l)},\qquad r_2(l)=1-r_1(l),$$

and $r_1(l)$ is quantized with 3 bits. Per frame, each object’s azimuth and elevation are also quantized once, with 7 bits for azimuth and 6 bits for elevation. For $y_L = D_L B,\qquad y_R = D_R B,$0 bands, dominant-object indices contribute $y_L = D_L B,\qquad y_R = D_R B,$1 bits per frame and power-ratio indices contribute $y_L = D_L B,\qquad y_R = D_R B,$2 bits per frame. The side information is therefore fixed-rate and low-resolution, which addresses mobile bit-rate constraints without scaling with the number of objects (Eichenseer et al., 7 Jul 2025).

The stereo downmix is direction-aware. For object $y_L = D_L B,\qquad y_R = D_R B,$3 with azimuth $y_L = D_L B,\qquad y_R = D_R B,$4, IVAS derives per-object gains from two fixed cardioids oriented toward the left and right hemispheres:

$y_L = D_L B,\qquad y_R = D_R B,$5

The downmix channels are

$y_L = D_L B,\qquad y_R = D_R B,$6

Smoothing between frames prevents discontinuities, and an energy compensation accounts for differences between the total energy of all objects and that of the two dominant ones per band. In matrix form, the short-time mixing relation is $y_L = D_L B,\qquad y_R = D_R B,$7, but ParamISM does not attempt to invert $y_L = D_L B,\qquad y_R = D_R B,$8 to recover the original objects explicitly. Instead, the decoder uses transmitted directional metadata and power-ratio parameters with the two-channel downmix to synthesize a spatial image over the target layout (Eichenseer et al., 7 Jul 2025).

Decoder-side rendering is performed in the covariance domain. The downmix is analyzed in a CLDFB that converts 20 ms frames into 16 time slots and 60 frequency bins. IVAS supports layouts including 5.1, 5.1.4, 7.1, and 7.1.4. For each band, the decoder reconstructs the dominant-object powers as

$y_L = D_L B,\qquad y_R = D_R B,$9

forms a target covariance

$$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$0

and computes a per-time-frequency mixing matrix $$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$1 such that $$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$2. Direct responses are computed from decoded object directions using Edge Fading Amplitude Panning, described as an improved amplitude-panning method versus classic VBAP. The paper’s core claim is that, although a two-channel downmix cannot in general uniquely separate more than two signals, IVAS leverages time-frequency sparsity: the dominant pair varies across bands and time, enabling a convincing spatial image for three or four objects (Eichenseer et al., 7 Jul 2025).

The reported operating region is explicitly low-rate. ParamISM supports three or four arbitrary objects at 24.4 or 32 kbit/s total, adheres to the immersive-mode delay target by using 20 ms frames and low-delay filterbanks, and is generally lower or comparable in WMOPS to multi-mono EVS. For three objects, total WMOPS are 241.14 at 24.4 kbit/s and 197.62 at 32 kbit/s, compared with 237.57 for $$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$3EVS at 8 kbit/s and 313.71 for $$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$4EVS at 9.6 kbit/s. For four objects, ParamISM totals are 248.18 and 205.30, versus 316.70 and 417.86 for the corresponding $$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$5EVS baselines. Two MUSHRA tests on a 7.1.4 loudspeaker layout with 13 expert listeners found that ParamISM achieved good quality for speech and fair quality for music or other content at the evaluated bitrates, and that on average it outperformed EVS at the same total bit rate by 15–20 MUSHRA points. Challenging cases included heavy temporal and spectral overlap and content with many simultaneous transients; for pure music, IVAS offers higher-rate non-parametric options up to 512 kbit/s (Eichenseer et al., 7 Jul 2025).

4. Scene-based audio and HOA coding

For scene-based audio, IVAS includes an SBA mode targeting Ambisonics up to 3rd order. The canonical channel count is

$$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$6

so 3rd-order HOA corresponds to $$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$7 channels, and the encoded sound field is expressed through a spherical-harmonic expansion

$$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$8

Rather than coding all HOA channels independently, IVAS SBA uses a hybrid parametric and residual framework combining SPAR and DirAC, with spatial parameters extracted at low temporal resolutions of 5–20 ms and a limited number of transport channels chosen and processed to be as mutually decorrelated as possible (Llave et al., 23 Jun 2026).

The coding structure separates the HOA vector into transmitted and non-transmitted subsets. The first transport channel is a beamformed combination of the first four Ambisonic components, while subsequent transport channels carry residual directional components. Non-transmitted components are reconstructed from quantized prediction matrices, gains or energies, and decorrelation functions. The bitrate allocations reported for IVAS SBA are: 32 kbps with one transport channel and 8 kbps of spatial parameters; 64 kbps with two transport channels and 10 kbps of spatial parameters; 128 kbps with three transport channels and 10 kbps of spatial parameters; and 256 kbps with four transport channels and 52.4 kbps of spatial parameters. The multi-mono baseline, by contrast, applies EVS mono independently to each of the 16 HOA channels at either 16.4 or 32 kbps per channel, yielding totals of approximately 262 and 512 kbps, respectively (Llave et al., 23 Jun 2026).

The perceptual rationale is interchannel correlation. IVAS exploits covariance and correlation among HOA components to avoid redundant transmission of highly correlated signals, which is especially beneficial for scenes composed of a limited number of plane waves. Multi-mono EVSx16 cannot exploit this structure and therefore often weakens timbre and spatial coherence by spreading the same total bitrate over many redundant channels. The practical consequence is a clear content dependence: IVAS is especially robust for highly correlated HOA signals, whereas diffuse reverberation remains challenging (Llave et al., 23 Jun 2026).

This dependence was evaluated in a MUSHRA study with 19 expert listeners over a loudspeaker dome using AllRADecoder with maxrE weighting. Thirteen items were tested, including native HOA recordings, ideal plane-wave encodings, and reverberant speech scenes. The reported results show that SPK1_ANE, a single talker encoded as a plane wave, maintained “excellent quality (>80)” even at 32 kbps IVAS. More generally, ideal plane-wave items such as SPK1_ANE, POP, SPK3_ANE, and FLK_ANE clustered above approximately 60 for IVAS at 64–256 kbps. At matched bitrates, IVAS 256 kbps scored higher than EVSx16 262 kbps for most items; exceptions included AMB, where EVSx16 outperformed IVAS by approximately 11 MUSHRA points, and BND, where the mean difference was not statistically significant (Llave et al., 23 Jun 2026).

A second experiment isolated the effect of reverberation through paired anechoic and reverberant items. EVSx16 showed a strong bias favoring reverberant signals by more than 20 MUSHRA points, with no bitrate dependence over 262–512 kbps ($$y_L(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_L(\theta_k(t),\phi_k(t),f)\big),\qquad y_R(t)=\sum_k g_k(r_k,t)\,\big(s_k(t)*h_R(\theta_k(t),\phi_k(t),f)\big).$$9). IVAS showed the opposite bias, favoring anechoic plane-wave signals: approximately +20 points at 256 kbps and up to approximately +35 points at 64 kbps, with the bias decreasing significantly as bitrate increased ($X_i(k,n)$0 for 256 vs 128 kbps, $X_i(k,n)$1 for 128 vs 64 kbps). This directly addresses a common misconception that IVAS is uniformly superior across all HOA material. The published evidence instead indicates a structured trade-off: IVAS is strongest when interchannel correlation is high and the scene is object-like, while very diffuse and heavily reverberant content exposes limitations in decorrelation synthesis (Llave et al., 23 Jun 2026).

5. Interactive XR rendering, acoustics, and networked congruence

A broader research line aligned with IVAS formulates a parametric 6-DoF object-based interactive audio engine for shared immersive virtual experiences. The engine is designed to deliver perceptually relevant binaural cues while supporting room reverberation, acoustic reflectors, obstacles, and the juxtaposition of real and virtual environments. It combines pre-computed and real-time acoustic propagation solvers, and it defines a compact object model and open scene description intended to facilitate interoperable applications distributed across mobile devices, wearables, and cloud or edge deployments (Jot et al., 2021).

The rendering pipeline explicitly models direct sound, clustered early reflections, and late reverberation. Obstacles and occluders are represented by transmission and reflectivity properties, early reflections by parameters such as delay, gain, pan, and focus, and multi-room coupling by room fingerprints and adjacency attributes. Source radiation may be represented either through compact parametric directivity or through spherical-harmonic coefficients,

$X_i(k,n)$2

with orientation applied through quaternion rotation. Late reverberation is implemented with room reverberators such as FDNs, while clustered reflections encode the Gerzon energy vector of a reflection cluster. The overall aim is audio/visual and virtual/real congruence rather than only static spatialization (Jot et al., 2021).

Latency and synchronization are treated as first-class system requirements. Motion-to-sound latency budgets are stated as less than 20 ms, ideally 5–10 ms. Head tracking is updated at 60–120 Hz or higher with IMU fusion, audio is processed at 48 kHz with 64–256 sample blocks, and scene updates are time-stamped and synchronized through PTP or NTP with predictive smoothing and hysteresis for acoustically discontinuous state changes such as occlusion. Embedded deployments minimize state payload through 3-band EQ and low-order SH representations, and heavier propagation can be moved to edge servers when needed. This indicates that IVAS-aligned rendering is defined as a networked real-time control problem as much as an acoustic rendering problem (Jot et al., 2021).

6. Application patterns, evaluation regimes, and unresolved issues

IVAS-aligned research extends beyond telepresence into narrative media. A multi-agent framework for immersive audiobook production instantiates IVAS concepts end-to-end through a Narrative Analysis Agent, a Neural Narration Agent based on FastSpeech 2 and VALL-E, a Spatial Sound Design Agent, a Room/Scene Acoustics Agent using HOA and scattering delay networks, a Mixing/Mastering Agent, and a Rendering/Delivery Agent. Synchronization is handled with DTW and LSTM-based offset prediction; diffusion-based generative audio is spatially conditioned for effects and ambiences; and delivery supports HOA decoding to binaural through SOFA HRTFs or to loudspeaker arrays. The workflow is explicitly low-latency, with chunked processing, prefetching, bounded DTW, and streaming buffers of 100–200 ms (Selvamani et al., 8 May 2025).

The associated evaluation culture is correspondingly broad. Reported metrics include MOS for conversational quality and narration quality, localization error, externalization, distance perception accuracy, reverberation fidelity through RT60 and related measures, synchronization offset distributions, throughput, and low-latency profiling. The studies on ParamISM and HOA coding use MUSHRA methodologies with expert listeners, but the broader IVAS-aligned rendering literature adds psychophysical and systems metrics that connect codec behavior to endpoint perception, head-tracking stability, and environment congruence. This suggests that IVAS should be understood as an ecosystem spanning codec design, metadata design, rendering semantics, and perceptual validation rather than as a narrowly defined waveform compressor (Selvamani et al., 8 May 2025).

Several limitations are explicit in the literature. ParamISM assumes two-object dominance per band and therefore relies on time-frequency sparsity and limited overlap; extreme overlaps and dense musical mixtures can reduce spatial fidelity, blur localization, or introduce timbral changes. HOA SBA remains sensitive to pronounced diffuse reverberation, particularly at low bitrates with few transport channels, where decorrelator mismatch and core-codec stress are offered as non-exclusive explanations. In both modes, bitrate increases improve robustness, but maximum bitrate does not guarantee transparency for all material. Security and privacy are also nontrivial because object-level voice and audio, listener head pose, and position can reveal sensitive user state; the proposed mitigations include encrypting audio and control planes, metadata minimization, and access control for room fingerprints and environmental captures (Eichenseer et al., 7 Jul 2025, Llave et al., 23 Jun 2026, Jot et al., 2021).

Open technical questions are framed in terms of interoperability and conformance rather than only compression ratio. The literature identifies the need to standardize reverberation fingerprint representation and normalization across devices, define common directivity models and occlusion or obstruction EQ conventions, specify multi-room panning semantics, and provide reference implementations and test scenes for localization, externalization, RT60 fidelity, and latency. A plausible implication is that the long-term significance of IVAS lies in how successfully these rendering semantics can be stabilized across heterogeneous devices and content classes while preserving the low-footprint, low-delay design envelope that motivated the codec family in the first place (Jot et al., 2021, Selvamani et al., 8 May 2025).