Micro-cells Enabling Spatio-Angular Joint Attention for High-resolution Light Field Editing

Light Field (LF) processing is a critical step in advancing immersive experiences beyond two-dimensional (2D) images. While LF-based technology facilitates a smooth transition from 2D to three-dimensional (3D) media, processing LFs composed of numerous images remains challenging. Most existing resea...

Full description

Saved in:
Bibliographic Details
Published in:IEEE access 2024-01, Vol.12, p.1-1
Main Authors: Kim, Gwon-Jung, Rhee, Chae Eun
Format: Article
Language:English
Subjects:
Attention mechanisms
Deep learning
Editing
Geometry
High resolution
Image color analysis
Image reconstruction
Image resolution
Image synthesis
Light fields
Media
Scalability
Spatial resolution
Three-dimensional displays
Transformers
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Light Field (LF) processing is a critical step in advancing immersive experiences beyond two-dimensional (2D) images. While LF-based technology facilitates a smooth transition from 2D to three-dimensional (3D) media, processing LFs composed of numerous images remains challenging. Most existing research has provided only limited processing capabilities for low-resolution LFs. For instance, LF editing typically offers basic functions like color changes and object removal. This paper proposes micro-cell-based processing for spatially and angularly large-scale LFs. By partitioning and rearranging the LF into micro-cells, which are then used as input for a vision transformer (ViT), we enable spatio-angular joint attention that significantly enhances performance. The local attention applied to small-sized micro-cells allows for high-resolution LF processing without excessive computational costs. This paper utilizes this combination of micro-cells and the ViT for LF editing applications. Experimental results demonstrate that the proposed LF editing system delivers exceptional performance both quantitatively and qualitatively, enabling high-resolution LF processing with a variety of editing functions.
ISSN:2169-3536
2169-3536
DOI:10.1109/ACCESS.2024.3504730