• KSII Transactions on Internet and Information Systems
    Monthly Online Journal (eISSN: 1976-7277)

An Automatic and Scalable Application Crawler for Large-Scale Mobile Internet Content Retrieval

Vol. 12, No. 10, October 30, 2018
10.3837/tiis.2018.10.013, Download Paper (Free):

Abstract

The mobile internet has grown ubiquitous across the globe with the widespread use of smart devices. However, the designs of modern mobile operating systems and their applications limit content retrieval with mobile applications. The mobile internet is not as accessible as the traditional web, having more man-made restrictions and lacking a unified approach for crawling and content retrieval. In this study, we propose an automatic and scalable mobile application content crawler, which can recognize the interaction paths of mobile applications, representing them as interaction graphs and automatically collecting content according to the graphs in a parallel manner. The crawler was verified by retrieving content from 50 non-game applications from the Google Play Store using the Android platform. The experiment showed the efficiency and scalability potential of our crawler for large-scale mobile internet content retrieval.


Statistics

Show / Hide Statistics

Statistics (Cumulative Counts from December 1st, 2015)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article

[IEEE Style]
M. Huang, Y. Lyu, H. Yin, "An Automatic and Scalable Application Crawler for Large-Scale Mobile Internet Content Retrieval ," KSII Transactions on Internet and Information Systems, vol. 12, no. 10, pp. 4856-4872, 2018. DOI: 10.3837/tiis.2018.10.013.

[ACM Style]
Mingyi Huang, Yongqiang Lyu, and Hao Yin. 2018. An Automatic and Scalable Application Crawler for Large-Scale Mobile Internet Content Retrieval . KSII Transactions on Internet and Information Systems, 12, 10, (2018), 4856-4872. DOI: 10.3837/tiis.2018.10.013.

[BibTeX Style]
@article{tiis:21897, title="An Automatic and Scalable Application Crawler for Large-Scale Mobile Internet Content Retrieval ", author="Mingyi Huang and Yongqiang Lyu and Hao Yin and ", journal="KSII Transactions on Internet and Information Systems", DOI={10.3837/tiis.2018.10.013}, volume={12}, number={10}, year="2018", month={October}, pages={4856-4872}}