Stylized Image Generation based on Music-image Synesthesia Emotional Style Transfer using CNN Network

• Baixi Xing,
• Jian Dou,
• Qing Huang,
• Huahao Si,

Vol. 15, No. 4, April 30, 2021
10.3837/tiis.2021.04.015, Download Paper (Free):

• Affective Computing
• Image Style Transfer
• Deep Convolutional Neural Networks

Abstract

Emotional style of multimedia art works are abstract content information. This study aims to explore emotional style transfer method and find the possible way of matching music with appropriate images in respect to emotional style. DCNNs (Deep Convolutional Neural Networks) can capture style and provide emotional style transfer iterative solution for affective image generation. Here, we learn the image emotion features via DCNNs and map the affective style on the other images. We set image emotion feature as the style target in this style transfer problem, and held experiments to handle affective image generation of eight emotion categories, including dignified, dreaming, sad, vigorous, soothing, exciting, joyous, and graceful. A user study was conducted to test the synesthesia emotional image style transfer result with ground truth user perception triggered by the music-image pairs’ stimuli. The transferred affective image result for music-image emotional synesthesia perception was proved effective according to user study result.

Statistics

Cite this article