Right-sizing Resource Allocations for Scientific Applications in Clusters, Grids, and Clouds

Doctoral Dissertation

Abstract

Clouds have joined clusters and grids as powerful environments for large scale scientific computing. While these platforms provide virtually unlimited computing resources, using more resources for an application does not always result in superior performance. The extra amount that does not contribute to any performance increase is a waste. This dissertation seeks to answer the question of how many computing resources should be allocated for a given workload. Two categories of workloads ? static and dynamic, are identified where viable solutions are found for this problem. For static workloads, we show that distributed abstractions allow for accurate performance modeling on distributed, multicore, and distributed multicore systems and thus can automatically make better resource allocation decisions. For dynamic workloads, we present dynamic capacity management as a solution to avoid resource waste without compromising on the application performance. We evaluate the effectiveness of this technique on a selection of workload patterns, ranging from highly homogeneous to completely random, and observe that the system is able to significantly reduce wasted resources with minimal impact on performance. Finally, we show that both solutions have been successfully applied in real world scientific applications in areas such as bioinformatics, economics, and molecular modeling.

Attributes

Attribute NameValues
URN
  • etd-07242013-144913

Author Li Yu
Advisor Douglas Thain
Contributor Marina Blanton, Committee Member
Contributor Douglas Thain, Committee Member
Contributor Jesus Izaguirre, Committee Member
Contributor Scott Emrich, Committee Member
Degree Level Doctoral Dissertation
Degree Discipline Computer Science and Engineering
Degree Name PhD
Defense Date
  • 2013-07-23

Submission Date 2013-07-24
Country
  • United States of America

Subject
  • resource management

  • distributed systems

  • scientific computing

  • cloud computing

Publisher
  • University of Notre Dame

Language
  • English

Record Visibility and Access Public
Content License
  • All rights reserved

Departments and Units

Files

Please Note: You may encounter a delay before a download begins. Large or infrequently accessed files can take several minutes to retrieve from our archival storage system.