• KSII Transactions on Internet and Information Systems
    Monthly Online Journal (eISSN: 1976-7277)

Two person Interaction Recognition Based on Effective Hybrid Learning


Action recognition is an essential task in computer vision due to the variety of prospective applications, such as security surveillance, machine learning, and human–computer interaction. The availability of more video data than ever before and the lofty performance of deep convolutional neural networks also make it essential for action recognition in video. Unfortunately, limited crafted video features and the scarcity of benchmark datasets make it challenging to address the multi-person action recognition task in video data. In this work, we propose a deep convolutional neural network–based Effective Hybrid Learning (EHL) framework for two-person interaction classification in video data. Our approach exploits a pre-trained network model (the VGG16 from the University of Oxford Visual Geometry Group) and extends the Faster R-CNN (region–based convolutional neural network a state-of-the-art detector for image classification). We broaden a semi-supervised learning method combined with an active learning method to improve overall performance. Numerous types of two-person interactions exist in the real world, which makes this a challenging task. In our experiment, we consider a limited number of actions, such as hugging, fighting, linking arms, talking, and kidnapping in two environment such simple and complex. We show that our trained model with an active semi-supervised learning architecture gradually improves the performance. In a simple environment using an Intelligent Technology Laboratory (ITLab) dataset from Inha University, performance increased to 95.6% accuracy, and in a complex environment, performance reached 81% accuracy. Our method reduces data-labeling time, compared to supervised learning methods, for the ITLab dataset. We also conduct extensive experiment on Human Action Recognition benchmarks such as UT-Interaction dataset, HMDB51 dataset and obtain better performance than state-of-the-art approaches.


Show / Hide Statistics

Statistics (Cumulative Counts from December 1st, 2015)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.

Cite this article

[IEEE Style]
M. U. Ahmed, Y. H. Kim, J. W. Kim, M. R. Bashar and P. K. Rhee, "Two person Interaction Recognition Based on Effective Hybrid Learning," KSII Transactions on Internet and Information Systems, vol. 13, no. 2, pp. 751-770, 2019. DOI: 10.3837/tiis.2019.02.015.

[ACM Style]
Minhaz Uddin Ahmed, Yeong Hyeon Kim, Jin Woo Kim, Md Rezaul Bashar, and Phill Kyu Rhee. 2019. Two person Interaction Recognition Based on Effective Hybrid Learning. KSII Transactions on Internet and Information Systems, 13, 2, (2019), 751-770. DOI: 10.3837/tiis.2019.02.015.

[BibTeX Style]
@article{tiis:22005, title="Two person Interaction Recognition Based on Effective Hybrid Learning", author="Minhaz Uddin Ahmed and Yeong Hyeon Kim and Jin Woo Kim and Md Rezaul Bashar and Phill Kyu Rhee and ", journal="KSII Transactions on Internet and Information Systems", DOI={10.3837/tiis.2019.02.015}, volume={13}, number={2}, year="2019", month={February}, pages={751-770}}