• KSII Transactions on Internet and Information Systems
    Monthly Online Journal (eISSN: 1976-7277)
Menu

An Adaptively Speculative Execution Strategy Based on Real-Time Resource Awareness in a Multi-Job Heterogeneous Environment

Vol. 11, No. 2, February 27, 2017
10.3837/tiis.2017.02.004, Download Paper (Free):

Abstract

MapReduce (MRV1), a popular programming model, proposed by Google, has been well used to process large datasets in Hadoop, an open source cloud platform. Its new version MapReduce 2.0 (MRV2) developed along with the emerging of Yarn has achieved obvious improvement over MRV1. However, MRV2 suffers from long finishing time on certain types of jobs. Speculative Execution (SE) has been presented as an approach to the problem above by backing up those delayed jobs from low-performance machines to higher ones. In this paper, an adaptive SE strategy (ASE) is presented in Hadoop-2.6.0. Experiment results have depicted that the ASE duplicates tasks according to real-time resources usage among work nodes in a cloud. In addition, the performance of MRV2 is largely improved using the ASE strategy on job execution time and resource consumption, whether in a multi-job environment.


Statistics

Show / Hide Statistics

Statistics (Cumulative Counts from December 1st, 2015)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article

[IEEE Style]
Q. Liu, W. Cai, Q. Liu, J. Shen, Z. Fu, X. Liu, N. Linge, "An Adaptively Speculative Execution Strategy Based on Real-Time Resource Awareness in a Multi-Job Heterogeneous Environment," KSII Transactions on Internet and Information Systems, vol. 11, no. 2, pp. 670-686, 2017. DOI: 10.3837/tiis.2017.02.004.

[ACM Style]
Qi Liu, Weidong Cai, Qiang Liu, Jian Shen, Zhangjie Fu, Xiaodong Liu, and Nigel Linge. 2017. An Adaptively Speculative Execution Strategy Based on Real-Time Resource Awareness in a Multi-Job Heterogeneous Environment. KSII Transactions on Internet and Information Systems, 11, 2, (2017), 670-686. DOI: 10.3837/tiis.2017.02.004.

[BibTeX Style]
@article{tiis:21351, title="An Adaptively Speculative Execution Strategy Based on Real-Time Resource Awareness in a Multi-Job Heterogeneous Environment", author="Qi Liu and Weidong Cai and Qiang Liu and Jian Shen and Zhangjie Fu and Xiaodong Liu and Nigel Linge and ", journal="KSII Transactions on Internet and Information Systems", DOI={10.3837/tiis.2017.02.004}, volume={11}, number={2}, year="2017", month={February}, pages={670-686}}

Unified Search
(in title, author, abstract, and keywords)

Category Search

KSII Transactions on Internet and Information Systems
© 2007~Present, KSII. All Rights Reserved.