• KSII Transactions on Internet and Information Systems
    Monthly Online Journal (eISSN: 1976-7277)

A Network Load Sensitive Block Placement Strategy of HDFS

Vol. 9, No.9, September 30, 2015
10.3837/tiis.2015.09.014, Download Paper (Free):

Abstract

This paper investigates and analyzes the default block placement strategy of HDFS. HDFS is a typical representative distributed file system to stream vast amount of data effectively at high bandwidth to user applications. However, the default HDFS block placement policy assumes that all nodes in the cluster are homogeneous, and places blocks with a simple RoundRobin strategy without considering any nodes。ッ resource characteristics, which decreases self-adaptability of the system. The primary contribution of this paper is the proposition of a network load sensitive block placement strategy. We have implemented our algorithm and justify it through extensive simulations and comparison with similar existing studies. The results indicate that our work not only performs much better in the data distribution but also improves write performance more significantly than the others.


Statistics

Show / Hide Statistics

Statistics (Cumulative Counts from December 1st, 2015)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article

[IEEE Style]
Lingjun Meng, Wentao Zhao, Haohao Zhao and Yang Ding, "A Network Load Sensitive Block Placement Strategy of HDFS," KSII Transactions on Internet and Information Systems, vol. 9, no. 9, pp. 3539-3558, 2015. DOI: 10.3837/tiis.2015.09.014

[ACM Style]
Meng, L., Zhao, W., Zhao, H., and Ding, Y. 2015. A Network Load Sensitive Block Placement Strategy of HDFS. KSII Transactions on Internet and Information Systems, 9, 9, (2015), 3539-3558. DOI: 10.3837/tiis.2015.09.014