Creativity and Perception: Unveiling the Role of Cross‐Modal Audiovisual Integration

• Published in: The Journal of creative behavior 2024-09, Vol.58 (3), p.460-477
• Main Authors: Xiao, Xinyao, Wang, Jian, Shu, Yanyan, Tan, Junying
• Format: Article
• Language: English
• Subjects: audio‐visual modality; creativity; cross‐modal; emotional coherence; emotional valence
• ISSN: 0022-0175; 2162-6057
• DOI: 10.1002/jocb.668

ABSTRACT Multisensory environments rich in modal integration provide cues from various sensory modalities including visually, auditorily, and tactilely. Such modal integration plays a crucial role in cognitive processing, specifically in fostering creativity. Numerous studies highlight that emotional coherence through cross‐modal affective integration enhances cognitive competencies such as memory, attention, and the capacity to generate original, fluent ideas. Nonetheless, current research lacks comparative studies pinpointing how different sensory modalities impact individual creativity. We addressed this research void by employing a cross‐model matching paradigm, anchored on the concept of emotional coherence. Our investigation evaluated the impact of varying emotional stimuli (both positive and negative) on creativity performance, considering single sensory modalities (visual and auditory), as well as their cross. Our study participants were 119 Chinese university students who completed a creative task under varying emotional stimulation modality conditions. We observed that the bimodal audiovisual integration of positive emotional stimuli most effectively enhanced creativity as compared to solely auditory modality. The visual modality seemed the least effective, underscoring the impact of multichannel integration. Interestingly, the bimodal audiovisual integration of negative emotional stimuli significantly boosted originality, albeit with little difference in fluency relative to the auditory modality. Based on the theoretical significance of multimodal emotional integration, our interpretation of these findings suggests that audiovisual cross‐modal emotional integration, with its rich emotional information, serves as a catalyst for enhancing originality. However, we observed that the facilitative effect on overall creativity and fluency is primarily observed in the context of positive emotions. It is important to note that we carefully controlled for participants' creative personality factors and conducted three types of emotional evaluations across modalities to ensure the validity of our results. Furthermore, the impact of cross‐modal audiovisual emotional integration on creativity, both in terms of originality and fluency, is influenced by the distinct emotional perceptual characteristics inherent to each modality.