Loss-tolerant neural video codec aware congestion control for real time video communication

Abstract: Because of reinforcement learning's (RL) ability to automatically create more adaptive controlling logics beyond the hand-crafted heuristics, numerous effort has been made to apply RL to congestion control (CC) design for real time video communication (RTC) applications and has successfully shown promising benefits over the rule-based RTC CCs. Online reinforcement learning is often adopted to train the RL models so the models can directly adapt to real network environments. However, its trail-and-error manner can also cause catastrophic degradation of the quality of experience (QoE) of RTC application at run time. Thus, safeguard strategies such as falling back to hand-crafted heuristics can be used to run along with RL models to guarantee the actions explored in the training sensible, despite that these safeguard strategies interrupt the learning process and make it more challenging to discover optimal RL policies. The recent emergence of loss-tolerant neural video codecs (NVC) naturally provides a layer of protection for the online learning of RL-based congestion control because of its resilience to packet losses, but such packet loss resilience have not been fully exploited in prior works yet. In this paper, we present a reinforcement learning (RL) based congestion control which can be aware of and takes advantage of packet loss tolerance characteristic of NVCs via reward in online RL learning. Through extensive evaluation on various videos and network traces in a simulated environment, we demonstrate that our NVC-aware CC running with the loss-tolerant NVC reduces the training time by 41\% compared to other prior RL-based CCs. It also boosts the mean video quality by 0.3 to 1.6dB, lower the tail frame delay by 3 to 200ms, and reduces the video stalls by 20\% to 77\% in comparison with other baseline RTC CCs.

• The paper presents NVC-CC, integrating loss-tolerant neural codecs into reinforcement learning to enhance congestion control efficiency.
• The approach reduces RL training time by 41% while improving video quality metrics by up to 1.6 dB and shortening tail frame delays.
• The research demonstrates practical benefits with video stalls reduced by up to 77%, marking a significant advance in adaptive real-time communication.

Loss-tolerant Neural Video Codec Aware Congestion Control for Real-Time Video Communication

The research paper by Zhengxu Xia, Hanchen Li, and Junchen Jiang provides an insightful contribution to the field of real-time video communication (RTC) through the development of a novel reinforcement learning-based congestion control algorithm, termed NVC-CC. This research builds upon the critical premise that the recent advancements in neural video codecs (NVCs), particularly their loss-tolerance capabilities, provide an untapped opportunity to optimize RTC congestion control strategies beyond the capabilities of traditional codecs.

Background and Motivation

In the landscape of RTC applications such as video conferencing, VR broadcasting, and cloud gaming, there is an essential need for systems that can adapt to dynamic network conditions to ensure high-quality user experiences. Historically, congestion control mechanisms designed for these applications relied on meticulously hand-crafted heuristics that focused on reliability, often at the expense of real-time performance. While RL approaches have shown promise in automating adaptive control logic, they bring along challenges such as potential QoE degradation due to their trial-and-error nature during learning phases, which necessitates provisions like safeguard mechanisms to revert to heuristic-based policies in unsafe conditions.

The prevailing safeguard strategies, although beneficial in maintaining system stability, inadvertently limit the RL models' learning efficiency. This paper addresses these challenges by leveraging the inherent packet loss resilience offered by state-of-the-art NVCs, thereby enabling RL-based congestion control mechanisms to explore and learn more efficiently without the need for extensive safeguard fallbacks.

Key Contributions

1. Novel Approach Utilizing Loss-tolerant NVCs: The crux of the research lies in integrating the loss-tolerant capabilities of NVCs directly into the RL framework to improve congestion control. Prior RL solutions did not fully utilize this codec property, resulting in learning inefficiencies. By redefining the reward function to incorporate frame quality and latency—metrics directly influenced by the NVC's loss resilience—this approach allows smarter and safer exploration of action spaces in RL training.
2. Efficiency Gains: Empirical evaluations demonstrate that the proposed NVC-aware RL-based congestion control reduces training time by 41% compared to existing RL counterparts. This indicates significant improvements in learning efficiency, achieved by eliminating the frequent interventions of safeguard policies and fully utilizing the NVC's properties to mitigate the impact of risky actions during training.
3. Enhanced Quality of Experience: The paper reports substantial gains in video quality metrics, such as a mean video quality increase of 0.3 to 1.6 dB% and a reduction in tail frame delay by 3 to 200 ms compared to existing baseline RTC congestion controls. Furthermore, video stalls are reduced by 20% to 77%, confirming the practical advantages of their approach in operational settings.

Theoretical and Practical Implications

The research acknowledges the transformative potential of combining RL with NVCs in improving RTC applications' congestion control strategies. The capability of training RL models that can inherently handle packet losses through NVCs without safeguarding interventions opens new avenues for developing more robust and adaptable communication systems. This paves the way for future AI developments in networking, potentially leading to more sophisticated integration of machine learning models with domain-specific features like codec loss tolerance.

Speculation on Future Directions

Future research could deepen investigations into the scalability of NVC-CC, particularly for multi-party conferencing scenarios or resource-constrained environments where traditional video codecs are still preferred. Moreover, addressing the sim-to-real generalization gap remains a critical area, enhancing the real-world applicability of these models trained in simulated environments. Additionally, extending this framework to operate harmoniously among diverse congestion control protocols on the internet could yield advancements in fairness and interoperability.

In summary, the paper presents a compelling case for evolving RTC congestion control mechanisms through innovative integration of cutting-edge neural codecs and reinforcement learning paradigms, marking a noteworthy step forward in optimizing the performance and efficiency of real-time communication networks.