Scientific Applications: Center for Simulation of Advanced Rockets (CSAR): Software Framework for Advanced Rocket Simulation
The Center for Simulation of Advanced Rockets (CSAR) is one of five university-based research centers funded by the U.S. Department of Energy as part of its Advanced Simulation and Computing (ASCI) program. The principal goal of CSAR is detailed, integrated, whole-system simulation of solid propellant rockets under both normal and abnormal operating conditions.

Our research at PPL is focused on application of object-based parallel programming paradigms for computational science and engineering. Past research in our group allows building multi-paradigm adaptive parallel programs based on the Converse runtime system and Charm++ load-balancing framework. We have adapted this framework to procedural languages such as Fortran 90, and message passing programming paradigms. Detailed performance analysis of Adaptive MPI, an implementation of MPI standard on top of Charm++, has been done, and has been optimized to have minimal overhead as compared to native MPI on several machines. Various components of the integrated rocket simulation codes, ROCFLO, ROCSOLID, and ROCFACE have been ported to Adaptive MPI with minimal changes and with little effort, providing them with automatic load balancing capabilities. The AMPI implementation also enhances the modularity of rocket codes by using separate sets of virtual processors for ROCFLO and ROCSOLID. Based on a similar programming paradigm, a prototype finite element method (FEM) framework has been developed. Although originally designed for finite element applications, it is general enough for unstructured mesh applications. Using the FEM framework, code developers write only sequential code that implements the node and element-specific computations of the FEM solver, while the details of parallelism such as partitioning, communication, and load balancing are taken care of by the framework. A 2-D pressure-driven crack propagation application (originally developed by P. Geubelle et al) has been ported to use this framework.

The multi-partitioned object-based approach has several other benefits besides automatic load balancing and efficient cache reuse. We have developed prototype software components that allow parallel programs to shrink and expand at runtime in order to effectively use inherently dynamic computational resources, such as clusters of workstations (COWs), which are becoming increasingly popular parallel architectures. Also, an efficient checkpointing framework has been developed that allows a parallel program to checkpoint and restart on possibly different number of processors.

We are also developing a Parallel Mesh Adaptation Framework for use with our FEM framework. These codes perform mesh refinement/derefinement and quality improvements in parallel.
A System Integration Framework for Coupled Multiphysics Simulations [Engineering with Computers 2006]
| Xiangmin Jiao | Gengbin Zheng | Phil Alexander | Mike Campbell | Orion Lawlor | John Norris | Andreas Haselbacher | Michael Heath
An Integration Framework for Simulations of Solid Rocket Motors [AIAAPropulsion 2005]
[MS Thesis]
An Adaptive Mesh Refinement (AMR) Library Using Charm++ [Thesis 2002]
A New Approach to Software Integration Frameworks for Multi-Physics Simulation Codes [IFIP Conference on the Architecture of Scientific Software 2000]
Object-Based Adaptive Load Balancing for MPI Programs [ICCS 2001]
A Parallel Framework for Explicit FEM [HiPC 2000]