Scalable Communication Tracing for Performance Analysis of Parallel Applications.
详细信息
文摘
Performance analysis and prediction for parallel applications is important for the design and development of scientific applications,and for the construction and procurement of high-performance computing HPC) systems. As one of the most important approaches,application tracing is widely used for this purpose for being able to provide the computation and communication details of an application. Recent progress in communication tracing has tremendously improved the scalability of tracing tools and reduced the size of the trace file,and thereby opened up novel opportunities for trace-based performance analysis for parallel applications. This work focuses on domain-specific trace compression methodology and puts forth fundamentally new approaches to improve the communication tracing techniques. Facilitated by the advances in this area,novel algorithms are further designed to address the hard problem of performance analysis,prediction,and benchmarking at scale. Specifically,this work makes the following contributions: 1. This work contributes ScalaExtrap,a fundamentally novel performance modeling scheme and tool. With ScalaExtrap,we synthetically generate the application trace for large numbers of MPI tasks by extrapolating from a set of smaller traces. We devise an innovative approach for topology extrapolation of SPMD Single Program Multiple Data) codes with stencil or mesh communication. The extrapolated trace can subsequently be used for trace-based simulation,visualization,and detection of communication inefficiencies and scalability limitations at scale. 2. This work contributes novel methods to automatically generate highly portable and customizable communication benchmarks from HPC applications. We utilize ScalaTrace to collect selected aspects of the run-time behavior of HPC applications. We then generate portable and easy-to-read benchmarks with identical run-time behavior from the collected traces with C and the rich-featured coNCePTuaL network benchmarking language. Because our approach supports code obfuscation,it is particularly valuable for proprietary,export-controlled,or classified applications. 3. This work contributes novel algorithms to improve the trace compression and replay for SPMD applications. Built on our past experience with ScalaTrace,a spectrum of compression techniques,including elastic data element representation,approximate loop matching,loop agnostic inter-node compression,and so on,are designed to improve the trace compression for applications with iteration-specific program behavior and diverging parallel control flow. A fully distributed replay tool for probabilistic traces is also developed for the reproduction of the computation performance of the original application. The respective design has been implemented in ScalaTrace 2,the next generation of the ScalaTrace tracing infrastructure. Overall,this work is centered around scalable tracing of parallel applications. Built upon the prior research,it contributes novel approaches on communication trace compression and trace-based performance analysis. To the best of our knowledge,the algorithms and techniques proposed in this work are without precedence.