๐”– Bobbio Scriptorium
โœฆ   LIBER   โœฆ

[ACM Press the 8th ACM international conference - Karlsruhe, Germany (2011.06.14-2011.06.18)] Proceedings of the 8th ACM international conference on Autonomic computing - ICAC '11 - ARIA

โœ Scribed by Verma, Abhishek; Cherkasova, Ludmila; Campbell, Roy H.


Book ID
121404162
Publisher
ACM Press
Year
2011
Weight
591 KB
Category
Article
ISBN
1450306071

No coin nor oath required. For personal study only.

โœฆ Synopsis


MapReduce and Hadoop represent an economically compelling alternative for efficient large scale data processing and advanced analytics in the enterprise. A key challenge in shared MapReduce clusters is the ability to automatically tailor and control resource allocations to different applications for achieving their performance goals. Currently, there is no job scheduler for MapReduce environments that given a job completion deadline, could allocate the appropriate amount of resources to the job so that it meets the required Service Level Objective (SLO). In this work, we propose a framework, called ARIA, to address this problem. It comprises of three inter-related components. First, for a production job that is routinely executed on a new dataset, we build a job profile that compactly summarizes critical performance characteristics of the underlying application during the map and reduce stages. Second, we design a MapReduce performance model, that for a given job (with a known profile) and its SLO (soft deadline), estimates the amount of resources required for job completion within the deadline. Finally, we implement a novel SLO-based scheduler in Hadoop that determines job ordering and the amount of resources to allocate for meeting the job deadlines.We validate our approach using a set of realistic applications. The new scheduler effectively meets the jobs' SLOs until the job demands exceed the cluster resources. The results of the extensive simulation study are validated through detailed experiments on a 66-node Hadoop cluster.


๐Ÿ“œ SIMILAR VOLUMES


[ACM Press the 8th ACM international con
โœ David, Howard; Fallin, Chris; Gorbatov, Eugene; Hanebutte, Ulf R.; Mutlu, Onur ๐Ÿ“‚ Article ๐Ÿ“… 2011 ๐Ÿ› ACM Press โš– 716 KB

Energy efficiency and energy-proportional computing have become a central focus in enterprise server architecture. As thermal and electrical constraints limit system power, and datacenter operators become more conscious of energy costs, energy efficiency becomes important across the whole system. Th