HPC for data driven research project
15 February 2017
The introduction of Grace in 2015, and its extension in 2016, has dramatically improved UCL’s ability to support “traditional” high performance computing by providing a dedicated platform for performing large parallel workloads previously handled by Iridis and Legion.
Although Iridis has been retired, Legion will be ten years old this year. Given the typical lifespan of computer hardware, that’s quite an incredible feat; however, much like the Ship of Theseus (or “Trigger’s broom”), its longevity is due to the continual replacement and updating of old hardware. Continuing this lineage, and with the removal of its MPI workload, there is now a prime opportunity to harness the most recent improvements in Legion and re-focus the service to research areas requiring access to computing resources suited to high throughput and data-intensive workloads.
In order to support this new remit, RITS will be undertaking the most significant refresh to Legion since its introduction in 2007, with a > £1m upgrade that will enhance its ability to support data intensive research at UCL. The refresh project will focus on a replacement of the current Legion interconnect with a new low latency, high bandwidth interconnect (e.g. InfiniBand, OmniPath) to drastically improve internode communications and I/O to storage; replacement of the older islands of compute nodes within the system; upgraded storage and a dedicated fast connection to the Research Data Storage Service. In addition, the project will also explore the development of other capabilities, such as the ability to host databases and large data sets locally to the system.
The refresh project is currently in its early stages. Recognising that the system being developed marks a step away from the current, traditional use cases for HPC, the Research Computing Group are keen to speak to researchers and groups who have current or, future need for a system of this type to help shape requirements for the project, for example:
- Researchers who are currently involved in “Data Intensive” or “Big Data” research projects;
- Researchers whose needs are not sufficiently met by desktops or locally hosted solutions where volume, velocity, or variety of data being analysed has become a bottle-neck;
- Researchers who have a dedicated local facility for data science research which could be used as a use case for other research areas.
Understanding the current, and anticipated future needs of research groups in these areas will be key to ensuring the success of the project and we warmly encourage members of the community to contact us to discuss their requirements for this system.
If you would like to get in touch the please contact firstname.lastname@example.org