Loading…

Vaser: Optimizing 360-Degree Live Video Ingest via Viewport-Aware Neural Enhancement

As a revolutionary technique, 360-degree live video streaming provides users with an immersive and realistic experience for a live event. However, due to the limited upload bandwidth of broadcasters, the ingest of 360-degree live video streams has become a bottleneck that seriously lowers the overal...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on broadcasting 2023-12, Vol.69 (4), p.1-14
Main Authors: Wang, Zelong, Luo, Zhenxiao, Hu, Miao, Chen, Min, Wu, Di
Format: Article
Language:English
Subjects:
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:As a revolutionary technique, 360-degree live video streaming provides users with an immersive and realistic experience for a live event. However, due to the limited upload bandwidth of broadcasters, the ingest of 360-degree live video streams has become a bottleneck that seriously lowers the overall utility of viewers. In this paper, we first conduct a series of measurement studies to unveil the characteristics of 360-degree video viewers and the effectiveness of neural enhancement under various settings. Based on the obtained insights, we further propose a novel neural-enhanced 360-degree live video ingest framework called Vaser . A key innovation of Vaser is to take viewport information into account when conducting neural enhancement and tile uploading. Vaser can significantly reduce the upload bandwidth demand for 360-degree live video ingest without sacrificing user experiences. We also propose an enhanced complexity-based patch selection algorithm to improve the efficiency of SR model training. To better utilize the upload bandwidth, we exploit deep reinforcement learning (DRL) to determine the bitrate of upload streams and the frequency of SR model updates. Last, we implement the whole framework and validate its effectiveness using real network traces. The experimental results show that our proposed framework can reduce the upload bandwidth consumption by 40%-55.6% on average with an upscale ratio of 2 and improve the overall utility of viewers by 1.15x-3.61x compared to other baseline approaches.
ISSN:0018-9316
1557-9611
DOI:10.1109/TBC.2023.3301715