Close
An amphiphile is a chemical species whose molecules consist of a hydrophobic tail attached to a hydrophilic head. This dichotomy causes the molecules to self-assemble into complex morphologies when dispersed in solvents, binary immiscible fluid mixtures or melts. Some mesophases are liquid crystalline, with features intermediate between a liquid and a solid. They play important functions in biological systems, and have applications in membrane protein crystallisation, controlled drug release and biosensors [1].
| High-density isosurface of the late-time configuration in a gyroid-genic ternary amphiphilic fluid as simulated on a 64 3 lattice by LB3D [3]. Gyroid ordering coexists with defect-rich, sponge-like regions. The dynamical behaviour of such defect-rich systems can only be studied with very large scale simulations, in conjunction with high-performance visualisation and computational steering. |  |
Being one of these mesophases, the gyroid exhibits weak crystallinity and the presence of defects, which play an important rôle in determining its mechanical properties. The objective of TeraGyroid is to study defect pathways and dynamics in gyroid self-assembly via the largest set of lattice-Boltzmann (LB) simulations ever performed, involving lattices of over one billion sites. Our LB model (LB3D) has an unrivalled ability to describe the dynamics and hydrodynamics of binary immiscible and ternary amphiphilic fluids [2,3]. The TeraGyroid project couples cutting-edge grid technologies, high-performance computing, visualisation and computational steering capabilities to produce a major leap forward in soft condensed matter simulation.
The lattice-Boltzmann model
Our model provides a hydrodynamically correct mesoscale fluid simulation method which describes the equilibrium, kinetic, and flow properties of complex surfactant-containing fluids with applications to bulk and confined geometries. [4] The mesoscopic, particulate nature of the LB model means that it can be applied to the study of condensed matter on length and timescales intermediate between the microscopic and the macrosopic, while retaining the hydrodynamic interactions that are critical to the evolution of structure. Our LB model is implemented in the LB3D code, which manifests exceptional scaling on all parallel machines [5]; it was recently the recipient of a gold-star rating on HPCx, the UK's fastest supercomputer, for its performance on 1024 processors. Such a scalable code, combined with access to massive computing power,allows much larger amounts of condensed matter to be probed, effectively bridging the gap in length and time scales between micro and macro descriptions that has afflicted simulations for decades.High Performance ComputingThe TeraGyroid project has access to a substantial fraction of the world's largest supercomputing resources, including the whole of the UK's supercomputing facilities at CSAR (Manchester) and HPCx (Daresbury/Edinburgh) and the USA's TeraGrid machines at Pittsburgh Supercomputing Center, NCSA (Illinois) and San Diego Supercomputing Center. The largest simulations to be performed within this project, involving in excess of one billion lattice sites, can only be accommodated on the HPCx, NCSA and PSC platforms.
Visualisation
Amphiphilic fluids produce exotic mesophases with a range of complex morphologies. These can only be fully apprehended using sophisticated visualisation software, semi-immersive or fully immersive virtual reality techniques being essential in some cases.The complexity of these data sets makes visualisation of large images itself a major graphics challenge. Using the VTK library, we are able to view up to 512 3 data sets of the gyroid morphology using a PC (2 GB memory and hardware volume rendering are minimum requirements) but visualisation of billion node models requires 64-bit hardware and multiple rendering units (such as multipipe OpenGL or chromium , with which SGI IR4 pipes and TeraGrid Viz resources are equipped).
Computational Steering
Traditionally, large compute-intensive simulations are run non-interactively, through submission of one or more jobs to a batch queue. The simulation runs according to the pre-prepared input file and results are output to a disk for subsequent analysis.There are many problems with this modus operandi : for example, something may have been wrong with the input file, the simulation fails to evolve in the desired direction, or all the important action occurs early on and the rest of the expensive CPU cycles are wasted. Even when a run proves successful, the data output from a terascale simulation is vast and can takemany months to analyse.A more efficient alternative is provided by grid-enabled computational steering. Large scale lattice-Boltzmann simulations run sufficiently fast on massively parallel computers that one has the possibility to interact with the model as it evolves in real time. With the concurrent use of high performance visualisation engines, one can render the simulation output, compress the resulting graphics frames and send these across a low bandwidth network to a person using a lightweight steering device (e.g. a laptop or even a PDA). Computational steering enables the scientist to interact with the visualised output and on this basis make decisions to alter the evolution of the simulation in any number of ways, reducing the consumption of CPU cycles and storage resources.In this project, we use the RealityGrid steering API and associated libraries to steer the LB3D code. [6] The advanced checkpointing capabilities available in this middleware also permit us to migrate the code across architectures at run time, to exploit dynamically varyingcompute resources so as to reduce execution time, and to spawn new simulations from an initial one when interesting phenomena are discovered. All these capabilities reduce the time to discover new scientific results.
The RealityGrid Project
RealityGrid is a major UK e-Science Pilot Project funded by EPSRC involving partners at several UK, US and European universities, and various non-academic institutions. Its purpose is to develop and deploy grid based middleware to assist in the advancement of condensed matter science. Within the TeraGyroid project, we perform multiple steered and spawned simulations as part of our study of defect dynamics, the visualised output being streamed to a distributed set of collaborators located at Access Grid nodes across the USA and UK.
Combined US/UK Grid Technologies
This ambitious project is the result of an international collaboration linking the USA's TeraGrid and the UK's e- Science Grid, jointly funded by NSF and EPSRC. Trans-Atlantic optical bandwidth is supported by British Telecommunications.
[1] W. M. Gelbart, D. Roux and A. Ben-Shaul: "Modern ideas and problems in amphiphilic science." (Springer: Berlin, 1993); D. Chapman & M. N. Jones: "Micelles, monolayers and biomembranes." (Wiley: Chichester, 1995.)
[2] N. González-Segredo, M. Nekovee & P. V. Coveney, Phys. Rev. E 67 , 046304 (2003); M. Nekovee and P. V. Coveney, J. Am. Chem. Soc. 123 , 12380 (2001).
[3] N. González-Segredo & P. V. Coveney, preprint (URL: http://arxiv.org/cond-mat/0310390) .
[4] H. Chen, B. M. Boghosian, P. V. Coveney & M. Nekovee, Proc. R. Soc. London A 456 , 2043 (2000).
[5] P. J. Love, M. Nekovee, P. V. Coveney, J. Chin, N. González-Segredo & J. M. R. Martin, Comp. Phys. Commun. 153 , Issue 3, 340-358 (2003).
[6] J. Chin, J. Harting, S. Jha, P. V. Coveney, A. R. Porter & S. M. Pickles, Contemporary Physics 44 , 417-434 (2003).