Adaptive Image-Space Sampling for Gaze-Contingent Real-time Rendering

• Published in: Computer graphics forum 2016-07, Vol.35 (4), p.129-139
• Main Authors: Stengel, Michael, Grogorick, Steve, Eisemann, Martin, Magnor, Marcus
• Format: Article
• Language: English
• Subjects: Acuity; Algorithms; Analysis; Categories and Subject Descriptors (according to ACM CCS); Computer graphics; Defense contracts; Fragments; Helmet mounted displays; I.3.x [Computer Graphics]: Three-Dimensional Graphics and Realism-Virtual Reality; I.3.y [Computer Graphics]: Three-Dimensional Graphics and Realism-Head-mounted Displays; Rendering; Sampling; Shading; Studies
• ISSN: 0167-7055; 1467-8659
• DOI: 10.1111/cgf.12956

With ever‐increasing display resolution for wide field‐of‐view displays—such as head‐mounted displays or 8k projectors—shading has become the major computational cost in rasterization. To reduce computational effort, we propose an algorithm that only shades visible features of the image while cost‐effectively interpolating the remaining features without affecting perceived quality. In contrast to previous approaches we do not only simulate acuity falloff but also introduce a sampling scheme that incorporates multiple aspects of the human visual system: acuity, eye motion, contrast (stemming from geometry, material or lighting properties), and brightness adaptation. Our sampling scheme is incorporated into a deferred shading pipeline to shade the image's perceptually relevant fragments while a pull‐push algorithm interpolates the radiance for the rest of the image. Our approach does not impose any restrictions on the performed shading. We conduct a number of psycho‐visual experiments to validate scene‐ and task‐independence of our approach. The number of fragments that need to be shaded is reduced by 50 % to 80 %. Our algorithm scales favorably with increasing resolution and field‐of‐view, rendering it well‐suited for head‐mounted displays and wide‐field‐of‐view projection.