PRACE JUGENE

Parallel space-time approach to turbulence: computation of unstable periodic orbits and the dynamical zeta function

Official Site: http://www.prace-project.eu/

Allocation on PRACE JUGENE.

The CCS has been awarded 40.5M CPU hrs on PRACE’s JUGENE (> 1 petaflops machine in Jülich) between 1 May 2011 and 30 April 2012 for work in the nanomaterials domain, based on the use of large scale molecular dynamics.

Project title: Parallel space-time approach to turbulence: computation of unstable periodic orbits and the dynamical zeta function

Abstract:

The challenge of predicting the properties of turbulent fluids is one of the most important fundamental problems still faced by current research, sometimes hailed as the last great unsolved problem from classical mechanics. It is of the utmost practical relevance in areas as diverse as weather forecasting, transport and dispersion of pollutants, gas flows in engines, blood circulation and cosmology. In this work, we intend to apply a novel methodology that efficiently utilizes petascale resources in the computation of turbulent quantities from first principles.

The algorithmic complexity of the Navier-Stokes equations (NSE), which describe fluid turbulence, typically scales as a power of a non-dimensional parameter, known as the Reynolds number. In turbulent flow the value of this quantity is often of the order of many millions, quickly invalidating the possibility of Direct Numerical Simulation (DNS) of these equations. An alternative approach, based on the modern mathematical theory of dynamical systems, which we follow in this work, is to identify the unstable periodic orbits (UPOs), from which the statistical properties of the NSE in the turbulent regime can then be computed, in an exact and systematic fashion.

The importance of these orbits to the description of turbulence has been known for some time. However, only recently, with the emergence of petascale resources, has it become feasible to use a space-time approach to compute these, due to the substantial memory requirements involved. This is a novel approach that has huge transformational potential for the area of turbulence studies and very likely for other forced dissipative dynamical systems as well. Algorithms such as these, which efficiently parallelize time as well as space, are going to be increasingly relevant to the new generation of petascale machines, as well as for the next generation of exascale computational resources.

For this purpose we have developed a fully parallel software package, called HYPO4D, which is the recipient of the TeraGrid’08 5K scalability award for its impressive parallel performance. Previous work by us, on a range of supercomputing resources in both the UK and the USA, with special emphasis on the Blue Gene/P machine at Argonne National Laboratory, has demonstrated the validity and efficiency of our approach, with the computation of the first UPOs in such high-dimensional systems. We now aim to extend this work in order to perform the first reasonable calculations of turbulent quantities from first principles, using this novel methodology. The largest UPO we intend to study will require in excess of 200,000 cores, making it essential for us to have access to JUGENE.

Tier-0 resource: JUGENE (IBM Blue Gene/P) at FZJ, Jülich, Germany

Resource awarded: 40.5M core-hours

HYPO4D timing results on JUGENE, given as site updates per second (SUPS) for the initial value problem versus number of cores used. The total size of the system increases linearly with the number of cores used. Each core is given a $96^3$ sublattice in the spatial decomposition algorithm, and the memory usage of each core is maximized. Timing results are shown for the original code (green) and an improved code (red). In the latter case, the main halo-exchange routine was re-written and a few other collective operations were optimized.