Loading...
Loading...

Go to the content (press return)

Resource-aware adaptive scheduling for MapReduce clusters

Author
Polo, J.; Castillo, C.; Carrera, D.; Becerra, Y.; Whalley, I.; Steinder, M.; Torres, J.; Ayguade, E.
Type of activity
Presentation of work at congresses
Name of edition
12th ACM/IFIP/USENIX International Middleware Conference
Date of publication
2011
Presentation's date
2011-12-16
Book of congress proceedings
Middleware 2011: ACM/IFIP/USENIX 12th International middleware conference: Lisbon, Portugal, December 12-16, 2011: proceedings
First page
187
Last page
207
DOI
https://doi.org/10.1007/978-3-642-25821-3_10 Open in new window
URL
http://www.springerlink.com/content/q8t3780224714064/ Open in new window
Abstract
We present a resource-aware scheduling technique for MapReduce multi-job workloads that aims at improving resource utilization across machines while observing completion time goals. Existing MapReduce schedulers define a static number of slots to represent the capacity of a cluster, creating a fixed number of execution slots per machine. This abstraction works for homogeneous workloads, but fails to capture the different resource requirements of individual jobs in multi-user environments. Our te...
Keywords
MapReduce, Performance management, Resource-awareness, Scheduling
Group of research
CAP - High Performace Computing Group

Participants