Towards Understanding Job Heterogeneity in HPC: A NERSC Case Study
2016 (English)In: 2016 16TH IEEE/ACM INTERNATIONAL SYMPOSIUM ON CLUSTER, CLOUD AND GRID COMPUTING (CCGRID), 2016, 521-526 p.Conference paper (Refereed)
The high performance computing (HPC) scheduling landscape is changing. Increasingly, there are large scientific computations that include high-throughput, data-intensive, and stream-processing compute models. These jobs increase the workload heterogeneity, which presents challenges for classical tightly coupled MPI job oriented HPC schedulers. Thus, it is important to define new analyses methods to understand the heterogeneity of the workload, and its possible effect on the performance of current systems. In this paper, we present a methodology to assess the job heterogeneity in workloads and scheduling queues. We apply the method on the workloads of three current National Energy Research Scientific Computing Center (NERSC) systems in 2014. Finally, we present the results of such analysis, with an observation that heterogeneity might reduce predictability in the jobs' wait time.
Place, publisher, year, edition, pages
2016. 521-526 p.
, IEEE-ACM International Symposium on Cluster Cloud and Grid Computing, ISSN 2376-4414
IdentifiersURN: urn:nbn:se:umu:diva-126538DOI: 10.1109/CCGrid.2016.32ISI: 000382529800067ISBN: 978-1-5090-2453-7OAI: oai:DiVA.org:umu-126538DiVA: diva2:1040719
16th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid), MAY 16-19, 2016, Cartagena, COLOMBIA