Banner layout retargeting with hierarchical reinforcement learning and variational autoencoder

• Published in: Multimedia tools and applications 2022-10, Vol.81 (24), p.34417-34438
• Main Authors: Hu, Hao, Zhang, Chao, Liang, Yanxue
• Format: Article
• Language: English
• Subjects: 1168: Deep Pattern Discovery for Big Multimedia Data; Automation; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Design; Designers; Interpolation; Layouts; Learning; Multimedia Information Systems; Optimization; Quality assessment; Special Purpose and Application-Based Systems
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-022-13325-w

In many advertising areas, banners are often generated with different display sizes, so designers have to make huge efforts to retarget their designs to each size. Automating such retargeting process can greatly save time for designers and let them put creativity on new ads. This paper proposes a hierarchical reinforcement learning-based (HRL-based) method and a variational autoencoder-based (VAE-based) method by treating the automated banner retargeting problem as a layout retargeting task. The HRL and VAE models are trained separately to learn the scaling and positioning policy of the design elements from an original (base) layout. Hence, the proposed method can generate appropriate layouts for different target banner sizes. Meanwhile, evaluation metrics are proposed to assess the quality of generated layouts and are also reward conditions during the training process. To evaluate performances of the two models, SOTA methods such as Non-linear Inverse Optimization (NIO), Triangle Interpolation (TI), and Layout GAN (LGAN) are implemented and compared. Experimental results show that both HRL- and VAE-based methods retarget design layouts effectively, and the VAE model achieves better performance than the HRL model.