Instructions for Downloading

Once your project data is ready to download you will receive a download token via e-mail. In the examples below the token for project ID 20178_Bloggs_Joe is "sKbcd69HNOIdoje9KKooo6969klnNIdA". Anyone who knows the token can access your data so be careful who you share it with. You may view the file listing using a web browser at https://transfer.genomics.ed.ac.uk/<token>, but there is no good way to download multiple large files with a web browser, so you need to use encrypted FTP.

Eddie

The Eddie compute cluster is directly connected to the Edinburgh University datastore, so downloading via Eddie is a fast way to get your files to your datastore area without going via your local PC (if you are at home then transferring files via your home internet and back will be very slow).

Eddie requires that large transfers are done via the 'staging' nodes so after logging in, do this:

$ qlogin -q staging

All being well, you should get a command prompt on a staging node.

Eddie has the lftp software which is the best way to quickly and securely fetch data:

$ cd /where/you/want/to/save

$ lftp transfer.genomics.ed.ac.uk <<<'login anonymous lftp@ ; mirror -vvc sKbcd69HNOIdoje9KKooo6969klnNIdA .'

Obviously, substitute your own token for the one above. The <<< shell syntax avoids making the secret token visible to other users of the Eddie transfer node.

After downloading, you should verify the file checksums to be sure that all were fully and correctly transferred. For each md5sums.txt file included with the data, change to the directory containing that file and run: 

$ md5sum -c md5sums.txt

 

Other Linux, Mac Command Line

Most Linux systems will have either wget or lftp commands available. You may need to install these using the system package manager. They are also available for Mac users through the Homebrew package manager. Once installed, use the same lftp command as for Eddie above.

It's also possible to use wget. This doesn't work on Eddie because their version of wget (as at October 2020) does not support encrypted FTP and will say "Unsupported scheme ‘ftps’", but on more up-to-date systems it should be fine.

$ wget -crnH --cut-dirs=1 -i - <<<'ftps://transfer.genomics.ed.ac.uk/sKbcd69HNOIdoje9KKooo6969klnNIdA/.'

Again, the <<< syntax avoids making the secret token visible to other users on your system (if you have any!). The other arguments to wget enable some sensible recursive downloading options.

Both the wget and lfpt methods will resume partial downloads, but to be sure you have fetched all files intact you should use the md5sum check as noted above.

 

Windows and Mac FTP Clients

On both platforms, the free Cyberduck client may be used to download the files via encrypted FTP but have slightly different instructions. You will need the delivery token and the project ID.

Cyberduck on Windows

  1. Launch Cyberduck and click the "Open Connection" button.
  2. The top dropdown will default to "FTP (File Transfer Protocol)". Select the dropdown and choose "FTP-SSL".
  3. In the "Server:" field, enter "transfer.genomics.ed.ac.uk"
  4. Check the "Anonymous Login" checkbox.
  5. Click the "More Options" button and a few more options will appear. In the "Path:" field, enter the token then a forward slash, then the project ID (for example:  sKbcd69HNOIdoje9KKooo6969klnNIdA/20178_Bloggs_Joe)
  6. Click connect and the available files will appear in the browser pane. Files can be dragged and dropped from this pane. 

Cyberduck on Mac

  1. Launch Cyberduck and click the "Bookmark Menu" and select "New Bookmark"
  2. The top dropdown will default to "FTP (File Transfer Protocol)". Select the dropdown and choose "FTP-SSL".
  3. In the "Server:" field, enter "transfer.genomics.ed.ac.uk"
  4. Check the "Anonymous Login" checkbox.
  5. Click the "More Options" button and a few more options will appear. In the "Path:" field, enter the token, then a forward slash, then the project ID (for example:  sKbcd69HNOIdoje9KKooo6969klnNIdA/20178_Bloggs_Joe)
  6. Close this dialogue box, and click on the bookmark you have created.
  7. The available files will appear in the browser pane. Files can be dragged and dropped from this pane. 

On Windows, WinSCP also works well, and is also free. To connect to our server you must do the following:

  1. Start a new session with the file protocol as FTP (not SFTP!)
  2. Set the encryption to "TLS/SSL Explicit Encryption"
  3. Set the host name to transfer.genomics.ed.ac.uk
  4. Enable anonymous login
  5. Click advanced, and under Environment > Directories > Remote directory put the download token (eg. sKbcd69HNOIdoje9KKooo6969klnNIdA)

If your connection to the server fails, double-check each step 1-4. If you are connected to the server but do not see any files available, then check that you added the correct token at step 5.

Unlike MacOS and Linux, Windows does not have a built-in md5sum checker tool, or at least not one that can check multiple files at once. We suggest using the free one from http://getmd5checker.com/.

 

Still stuck?

Please contact the Edinburgh Genomics team (eg-bioinformatics@mlist.is.ed.ac.uk) if you are having any problems with downloading or checksumming your files.