Loading…
Vaser: Optimizing 360-Degree Live Video Ingest via Viewport-Aware Neural Enhancement
As a revolutionary technique, 360-degree live video streaming provides users with an immersive and realistic experience for a live event. However, due to the limited upload bandwidth of broadcasters, the ingest of 360-degree live video streams has become a bottleneck that seriously lowers the overal...
Saved in:
Published in: | IEEE transactions on broadcasting 2023-12, Vol.69 (4), p.1-14 |
---|---|
Main Authors: | , , , , |
Format: | Article |
Language: | English |
Subjects: | |
Citations: | Items that this one cites Items that cite this one |
Online Access: | Get full text |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Summary: | As a revolutionary technique, 360-degree live video streaming provides users with an immersive and realistic experience for a live event. However, due to the limited upload bandwidth of broadcasters, the ingest of 360-degree live video streams has become a bottleneck that seriously lowers the overall utility of viewers. In this paper, we first conduct a series of measurement studies to unveil the characteristics of 360-degree video viewers and the effectiveness of neural enhancement under various settings. Based on the obtained insights, we further propose a novel neural-enhanced 360-degree live video ingest framework called Vaser . A key innovation of Vaser is to take viewport information into account when conducting neural enhancement and tile uploading. Vaser can significantly reduce the upload bandwidth demand for 360-degree live video ingest without sacrificing user experiences. We also propose an enhanced complexity-based patch selection algorithm to improve the efficiency of SR model training. To better utilize the upload bandwidth, we exploit deep reinforcement learning (DRL) to determine the bitrate of upload streams and the frequency of SR model updates. Last, we implement the whole framework and validate its effectiveness using real network traces. The experimental results show that our proposed framework can reduce the upload bandwidth consumption by 40%-55.6% on average with an upscale ratio of 2 and improve the overall utility of viewers by 1.15x-3.61x compared to other baseline approaches. |
---|---|
ISSN: | 0018-9316 1557-9611 |
DOI: | 10.1109/TBC.2023.3301715 |