GPFS Pilot Service
These draft pages will be updated regularly as required. We appreciate feedback on mistakes and what other information you'd like to see on the Research Data support pages.
- Access to storage is currently only available over ssh.
- You use your central username and password for accessing the service.
- Each project is assigned an RD internal name, you will have been provided this by email.
- Each project will have had a central posix group created for it (starting "rd" and containing 6 characters) which will contain the users associated with the project. You will have been provided this by email.
- All project areas have a initial quota which can be increased if necessary, you will have been told by email what this quota is and when increases can be allocated.
- Please DO NOT use the ssh nodes for compute work. Please contact Research Computing for access to compute resources.
Host Name for ssh: ssh.rd.ucl.ac.uk
Path to project space: /mnt/gpfs/UCL/projectname
This example assumes that your username is ucapd01 and the project name is solarsystem.
To login in
ssh -l username ssh.rd.ucl.ac.uk
ssh -l ucapd01 ssh.rd.ucl.ac.uk
Once you've successfully logged in, you will be placed in your home directory (/mnt/gpfs/UCL/homedirs/username). Note that you will not be able to login to our access node unless you are on the UCL network.
To change to your project directory
To check your project quota
/usr/lpp/mmfs/bin/mmlsquota -j projectname UCLGPFS --block-size auto
/usr/lpp/mmfs/bin/mmlsquota -j solarsystem UCLGPFS --block-size auto
Who do I contact for support?
The Research Data Service support team. We are in the process of setting up a ticketing service but in the mean time, please contact one or all of the following (in no particular order):
- Daniel Hanlon
- Alastair Smith
- Dugan Witherick
If you have been dealing with one individual, then for the meantime, please contact them and if they are unavailable you can contact one of the others.
How do I access my data when external to the UCL network?
The GPFS pilot service is not visible externally to the UCL network because it is behind the UCL institutional firewall; we are currently looking into ways in which we can provide secure access externally.
Until we have external access available, you can do one or other of the following:
Login via the socrates.ucl.ac.uk gateway
socrates.ucl.ac.uk provides a standard ssh login service which is visible externally to UCL. To access our services, first login to socrates i.e.
ssh -l username socrates.ucl.ac.uk
and then follow the above instructions on logging into our service.
Use the UCL VPN service
UCL runs a VPN (Virtual Private Network) service which enables one to be virtually placed inside of instituonal firewall (after which you follow the above instructions on logging into our service). See UCL VPN Service for more information on using the VPN service.
Please note that by its very nature, when you connect to the VPN service you will likely lose connection to any network services you are using before you started the connection process.
SSH File Transfer Clients
Here is a list of a few suggested file transfer tools. Please send us suggestions for more.
- OSX built in command line support for ssh, scp, sftp and rsync over ssh
- Cyberduck - GUI file transfer application
- SSHFS - enables the remote storage to be mounted like a drive
- Linux built in command line support for ssh, scp, sftp and rsync over ssh
- For GUI access, the various windows managers have different levels of built in support for sftp, sshfs etc. For example, nautilus has built in support for sshs.
How much quota do I have for my home directory?
We have purposely kept this quota relatively small (500MB with a hard limit of 1000MB) because the system is not intended to be used as a general purpose Linux facility. To the current status of your home area run
mmlsquota --block-size auto
and your home area usage will be listed where the fileset is called "homedirs".
Can I request more space?
Please contact us as soon as you think you may require additional space.
I can't read/write to some files in our project area, how do I gain access?
The posix file system does not support the concept of files/directories owned by groups (unlike Windows). This unfortunately means that for you to be able to read the files, they should have the same primary group as your project and, have the correct read/write/execute permissions for group access. To be able to read the file/s, your colleague who is the primary owner on the file needs to change the permissions accordingly e.g.
chmod g+rwx file
would provide read/write/execute permissions to the group. It is possible to ensure that newly created files in a directory have write permissions for the group, please ask your designated project administrator (or PI) to contact us if you require this option to be set.
How do I find out what groups I am in?
at the command line after logging in and this will display the list of groups you are in and the corresponding unix ids.
How do I find out who else in in my group?
getent group groupname
at the command line and this will display a list of users in the group.
How do I add users to a group/project?
These requests should come through the project PI or designated project administrator. Send the request to the Research Data Services technical team and we will forward the request to user services.
Transfer rates to our storage are affected by the many different layers of networking and other equipment between our servers and where you are transferring from (as well as the protocol used for the transfer). We are monitoring these transfer rates and will be looking the resolve bottlenecks where possible. However there are some ways in which transfer rates MAY be improved:
- Change the cipher used for ssh encryption to a lightweight one such as arcfour (this is set as an option to ssh, scp, sftp).
- Run tar over ssh (see http://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/ for an example of how to do this.
Can I build code/run compute jobs on your ssh nodes?
No. The task of the ssh nodes is to handle data transfer operations and are not compute nodes. Running compute jobs will have a serious impact on your own and other user's ability to transfer data on and off the system. Please contact Research Computing for access compute nodes.
Page last modified on 08 nov 13 11:54