• KSII Transactions on Internet and Information Systems
    Monthly Online Journal (eISSN: 1976-7277)

CNN-based Visual/Auditory Feature Fusion Method with Frame Selection for Classifying Video Events

Vol. 13, No.3, March 31, 2019
10.3837/tiis.2019.03.033, Download Paper (Free):

Abstract

In recent years, personal videos have been shared online due to the popular uses of portable devices, such as smartphones and action cameras. A recent report[1] predicted that 80% of the Internet traffic will be video content by the year 2021. Several studies have been conducted on the detection of main video events to manage a large scale of videos. These studies show fairly good performance in certain genres. However, the methods used in previous studies have difficulty in detecting events of personal video. This is because the characteristics and genres of personal videos vary widely. In a research, we found that adding a dataset with the right perspective in the study improved performance. It has also been shown that performance improves depending on how you extract keyframes from the video. we selected frame segments that can represent video considering the characteristics of this personal video. In each frame segment, object, location, food and audio features were extracted, and representative vectors were generated through a CNN-based recurrent model and a fusion module. The proposed method showed mAP 78.4% performance through experiments using LSVC[2] data.


Statistics

Show / Hide Statistics

Statistics (Cumulative Counts from December 1st, 2015)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article

[IEEE Style]
Giseok Choe, Seungbin Lee and Jongho Nang, "CNN-based Visual/Auditory Feature Fusion Method with Frame Selection for Classifying Video Events," KSII Transactions on Internet and Information Systems, vol. 13, no. 3, pp. 1689-1701, 2019. DOI: 10.3837/tiis.2019.03.033

[ACM Style]
Choe, G., Lee, S., and Nang, J. 2019. CNN-based Visual/Auditory Feature Fusion Method with Frame Selection for Classifying Video Events. KSII Transactions on Internet and Information Systems, 13, 3, (2019), 1689-1701. DOI: 10.3837/tiis.2019.03.033