Recent years have seen significant strides in individual performance modeling techniques for parallel applications and systems. Each class of individual techniques has important strengths and weaknesses. Abstract, analytical models provide important insights into application performance and are usually extremely fast, but lack the ability to capture detailed performance behavior and most such models must be constructed manually which limits their accessibility to users. Program-driven simulation techniques can capture detailed performance behavior at all levels, and can be used automatically (i.e., with little user intervention) to model a given program, but can be extremely expensive for large-scale parallel programs and systems, not only in terms of simulation time but especially in their memory requirements. Finally, program measurement is an important tool for tuning existing programs and for parameterizing and validating performance models, and it can be very effective for some metrics (such as counting cache misses using on-chip counters), but it is inflexible, limited to a few metrics, and limited to available program and system configurations.
To overcome these limitations of individual modeling approaches, researchers are now beginning to focus on developing comprehensive, end-to-end modeling environments that bring together multiple techniques to enable practical performance modeling for large, real-world applications on very large scale systems. Such an environment would support a variety of modeling techniques for each system component, and enable different models to be used for different system components within a single modeling study. Equally important, such an environment would include compiler support to automatically construct workload information that can drive the different modeling techniques, minimizing the need for user intervention.
For example, the POEMS project aims to create such an environment for the end-to-end modeling of large parallel applications on complex parallel and distributed systems [1]. In addition to the two basic goals stated above, another goal of POEMS is to enable compositional development of end-to-end performance models, using a specification language to describe the system components and a choice of model for each component, and using automatic data mediation techniques (specialized for specific model interactions) to interface the different component models. The project brings together a wide range of performance modeling techniques including detailed execution-driven simulation of message-passing programs and scalable I/O [7], program-driven simulation of single-node performance and memory hierarchies, abstract analytical models of parallel programs such as LogGP [21], and detailed analytical models of parallel program performance based on deterministic task graph analysis [5]. An overview of the POEMS framework and methodology is available in [1]. Several components of the framework are described elsewhere, including the specification language for compositional development of models [8], an overview of the parallel simulation capability for detailed performance prediction of large scale scientific applications [6], and the LogGP model for Sweep3D (an ASCI neutron transport code that has been an initial driving application for the POEMS project) [21].
An important challenge in developing such a comprehensive performance modeling environment is designing an application representation that can support the two basic goals mentioned above. In particular, the application representation must provide a description of program behavior that can serve as a common source of workload information for any modeling technique or combination of modeling techniques to be used in a particular experiment. Directly using the application source code as this representation is inadequate because it can directly support only program-driven modeling techniques such as execution-driven simulation. Instead, we require an abstract representation of the program structure that can enable different modeling techniques to be used for different program components (e.g., computational tasks, memory access behavior, and communication behavior). Nevertheless, the representation should precisely capture all relevant information that affects the performance of the program so that the representation itself does not introduce any a priori approximations into the system. The application representation must also be efficient and flexible enough to capture complex, large-scale applications with high degrees of parallelism and sophisticated parallelization strategies.
To meet the second basic goal mentioned above, it must be possible to compute the application representation automatically for a given parallel program using parallelizing compiler technology. This requires a compile-time representation that is independent of program input values. In particular, this requires symbolic information about the program structure (e.g., numbers of parallel tasks, loop iterations, and message sizes). The representation must also capture part of the static control-flow in the program, in addition to the sequential computations and the parallel structure. It also requires that detailed information about a specific execution of the program on a particular input should be derivable from the static representation.
This paper describes the design of the application representation in the POEMS environment. We begin by describing in more detail the key goals that must be met by this design (Section 2). Section 3 then describes the main design features of the application representation in POEMS, and discusses how these design features meet the major challenges that must be faced for advanced, large-scale applications. We conclude with an overview of related work and a description of status and future plans.