UniAR: A Unified model for predicting human Attention and Responses on visual content

Progress in human behavior modeling involves understanding both implicit, early-stage perceptual behavior, such as human attention, and explicit, later-stage behavior, such as subjective preferences or likes. Yet most prior research has focused on modeling implicit and explicit human behavior in iso...

Full description

Saved in:
Bibliographic Details
Published in:arXiv.org 2024-10
Main Authors: Li, Peizhao, He, Junfeng, Li, Gang, Bhargava, Rachit, Shen, Shaolei, Valliappan, Nachiappan, Liang, Youwei, Gu, Hongxiang, Ramachandran, Venky, Farhadi, Golnaz, Yang, Li, Kohlhoff, Kai J, Navalpakkam, Vidhya
Format: Article
Language:English
Subjects:
Design
Design optimization
Digital imaging
Feedback
Human behavior
Modelling
Preferences
Ratings
Role models
Viewing
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Progress in human behavior modeling involves understanding both implicit, early-stage perceptual behavior, such as human attention, and explicit, later-stage behavior, such as subjective preferences or likes. Yet most prior research has focused on modeling implicit and explicit human behavior in isolation; and often limited to a specific type of visual content. We propose UniAR -- a unified model of human attention and preference behavior across diverse visual content. UniAR leverages a multimodal transformer to predict subjective feedback, such as satisfaction or aesthetic quality, along with the underlying human attention or interaction heatmaps and viewing order. We train UniAR on diverse public datasets spanning natural images, webpages, and graphic designs, and achieve SOTA performance on multiple benchmarks across various image domains and behavior modeling tasks. Potential applications include providing instant feedback on the effectiveness of UIs/visual content, and enabling designers and content-creation models to optimize their creation for human-centric improvements.
ISSN:2331-8422