Page tree
Skip to end of metadata
Go to start of metadata

Please note that massdata is not designed for small files. Attempting to store or retrieve files less than a few megabytes will result in extremely poor performance for all users. If you wish to store lots of small files to massdata, please use a utility such as tar to combine them into a single, larger file.

 

To transfer files from your local machine to massdata, we recommend to transfer files to raijin /short using rsync first and then from /short to massdata using mdss put/get or netcp/netmvmdss command can be run either interactively or in a job script while as netcp/netmv commands are used to generate a job script and submit it to copyq.

With appropriate options, rsync is resumable should the connection drop mid-transfer. The recommended command line to use on your local machine is:

rsync -avPS src dst

where  will be the path to the files on your local machine and  will be the path to your destination on the appropriate host.

For example:

//1. Transfer files from local machine to raijin: (use r-dm.nci.org.au, the dedicated data mover)


rsync -avPS /home/me/files user@r-dm.nci.org.au:/short/project/user/my_files
//where user is your NCI username, and project is the three letter/digit code identifying your project.


//2. Then transfer files from raijin to mdss:
ssh user@raijin.nci.org.au 
mdss put /short/project/user/my_files user/destination_folder

//or 
netcp/netmv -t archivename.tar /short/project/user/my_file user/destination_folder