Networking for Immersive Communications

Communication systems tried to improve the quality of communication to become close to real in-person communication. The vision of immersive communication is to enable interactions with remote people and environments such that users get the impression of being really inside a remote or virtual environment. Immersive communications may add additional services and functions and even improve real in-person communication. Augmented reality, virtual reality, and haptics are key technologies to enable immersive communications, with extreme requirements in terms of bandwidth, reliability, and latency. The targets are a few milliseconds, combined with high bandwidth (e.g., for 360◦ video), and/or high reliability (e.g., 99.9999 %, for interactive sensor/actor-based applications such as haptics). NICO addresses these extreme challenges and requirements with a set of various approaches on the protocol, network architecture, and implementation architecture levels. In NICO, we intend to come closer to these targets by addressing the following interrelated research challenges, which have to be considered in an integrated way. 

1. Viewport prediction algorithms in 360◦ video based on sensor-based localization, tracking, and rotation/movement predictions using advanced machine learning (ML) concepts in order to improve users’ Quality-of-Experience (QoE), 

2. Disruptive methods to identify QoE parameters in immersive communications combining system measurements and psychological behaviors [1], 

3. Implementation of an advanced integrated hardware and software platform, including operating system scheduling for user-level virtualization and the support of hardware accelerators for immersive communication processing in order to support low latencies and high throughput, 

4. Novel mechanisms to minimize latencies by (extended) caching mechanisms exploiting edge and in-network  processing for immersive communications [2] [3], 

5. Novel network protocols based on information-centric networking and network coding to support low-latency communication, 

6. Original algorithms and mechanisms for service migration to minimize latency between mobile users and service entities, including the novel concept of Floating Services.

Floating AI Services for Mobile Virtual Reality Networks

The part of the project investigates the design and implementation of Floating AI Services for Mobile Virtual Reality Networks (MVRNs), an emerging class of decentralized systems where users interact in shared virtual and real environments via mobile Head-Mounted Devices (HMDs). These networks support immersive applications such as gaming, remote collaboration, and interactive simulations, which increasingly rely on Artificial Intelligence (AI) for tasks like viewport prediction, object recognition, scene understanding, and context-aware interactions.

A central challenge in MVRNs is the limited computational and communication resources of mobile HMDs, which restrict the ability to run AI models locally or exchange large amounts of data through infrastructure-based networks. This thesis explores a fully decentralized framework that leverages device-to-device (D2D) communication to enable floating AI services, AI models that dynamically migrate and adapt across nearby devices. Key contributions include resource-aware model pruning to reduce computational load, proximity-based model sharing to exploit local collaboration, and online learning algorithms that guarantee robust performance under limited feedback. The work aims to provide scalable, efficient, and adaptive AI service provisioning for next-generation mobile virtual reality applications.

References

[1]  H. Xing and T. Braun, "A Hybrid Network For Extended Reality Environments," in IEEE Transactions on Multimedia, doi: 10.1109/TMM.2025.3590913.

[2] C. Gao and T. Braun, "Dual-Engine Intelligent Caching: A Joint Optimization Framework for 360° Mobile VR Video Edge Caching," in IEEE Journal on Selected Areas in Communications, doi: 10.1109/JSAC.2025.3574596.

[3] C. Gao and T. Braun, "Two-Stage Hybrid Edge Caching Framework for 360° VR Video," 2025 IEEE 26th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM), Fort Worth, TX, USA, 2025, pp. 1-10, doi: 10.1109/WoWMoM65615.2025.00015.