The workload on parallel supercomputers: modeling the characteristics of rigid jobs

✍ Scribed by Uri Lublin; Dror G. Feitelson

• Publisher: Elsevier Science
• Year: 2003
• Tongue: English
• Weight: 315 KB
• Volume: 63
• Category: Article
• ISSN: 0743-7315
• DOI: 10.1016/s0743-7315(03)00108-4

✦ Synopsis

The analysis of workloads is important for understanding how systems are used. In addition, workload models are needed as input for the evaluation of new system designs, and for the comparison of system designs. This is especially important in costly largescale parallel systems. Luckily, workload data are available in the form of accounting logs. Using such logs from three different sites, we analyze and model the job-level workloads with an emphasis on those aspects that are universal to all sites. As many distributions turn out to span a large range, we typically first apply a logarithmic transformation to the data, and then fit it to a novel hyper-Gamma distribution or one of its special cases. This is a generalization of distributions proposed previously, and leads to good goodness-of-fit scores. The parameters for the distribution are found using the iterative EM algorithm. The results of the analysis have been codified in a modeling program that creates a synthetic workload based on the results of the analysis.