• KSII Transactions on Internet and Information Systems
    Monthly Online Journal (eISSN: 1976-7277)

An Efficient Design and Implementation of an MdbULPS in a Cloud-Computing Environment

Vol. 9, No.8, August 31, 2015
10.3837/tiis.2015.08.026, Download Paper (Free):

Abstract

Flexibly expanding the storage capacity required to process a large amount of rapidly increasing unstructured log data is difficult in a conventional computing environment. In addition, implementing a log processing system providing features that categorize and analyze unstructured log data is extremely difficult. To overcome such limitations, we propose and design a MongoDB-based unstructured log processing system (MdbULPS) for collecting, categorizing, and analyzing log data generated from banks. The proposed system includes a Hadoop-based analysis module for reliable parallel-distributed processing of massive log data. Furthermore, because the Hadoop distributed file system (HDFS) stores data by generating replicas of collected log data in block units, the proposed system offers automatic system recovery against system failures and data loss. Finally, by establishing a distributed database using the NoSQL-based MongoDB, the proposed system provides methods of effectively processing unstructured log data. To evaluate the proposed system, we conducted three different performance tests on a local test bed including twelve nodes: comparing our system with a MySQL-based approach, comparing it with an Hbase-based approach, and changing the chunk size option. From the experiments, we found that our system showed better performance in processing unstructured log data.


Statistics

Show / Hide Statistics

Statistics (Cumulative Counts from December 1st, 2015)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article

[IEEE Style]
Myoungjin Kim, Yun Cui and Hanku Lee, "An Efficient Design and Implementation of an MdbULPS in a Cloud-Computing Environment," KSII Transactions on Internet and Information Systems, vol. 9, no. 8, pp. 3182-3202, 2015. DOI: 10.3837/tiis.2015.08.026

[ACM Style]
Kim, M., Cui, Y., and Lee, H. 2015. An Efficient Design and Implementation of an MdbULPS in a Cloud-Computing Environment. KSII Transactions on Internet and Information Systems, 9, 8, (2015), 3182-3202. DOI: 10.3837/tiis.2015.08.026