MATLAB

MATLAB is a general purpose analysis and graphics plotter from MathWorks. With more than 600 mathematical, statistical, and engineering functions, MATLAB provides immediate access to high-performance numerical computing. This functionality is extended with interactive graphical capabilities for creating plots, images, surfaces, and volumetric representations.

A toolbox algorithms enhance MATLAB's functionality in domains such as signal and image processing, data analysis and statistics, mathematical modeling, and control design. Toolboxes are collections of algorithms, written by experts in their fields, that provide application-specific numerical, analysis, and graphical capabilities.

Usage

Typically, running a Matlab job on Gadi that benefits from the speedup of multi-cores would require, 

1. joining the matlab software group for your institution, matlab_<institution>, to allow your access to the Matlab license
2. load modulefile for the corresponding license
3. setting up both your .m file and the job submission script to allow the computation- and data-intensive problems using multi-cores, GPUs and multi-nodes. 
4. Make use of the smpd deamon available in Matlab for parallelisation.

   When using smpd, please specify -smpdfile to $PBS_JOBFS/<file> to avoid creating smpd files in /tmp.

Regarding the license access, please check the section License below for more details.   

Once get the access to the matlab license on Gadi, please confirm with your institution that whether the license comes with either of the features 'Parallel Computing Toolbox' and 'Distributed Computing Server'. Although some linear algebra and numerical functions, such as matrix products, fft, eig, sort, are implicitly multi-threaded and make use of all the cores available to the job without any extra programming efforts, see an incomplete list provided by Mathworks Support Team here, the majority of the functions provided by matlab runs only on a single core and users need to explicitly use the functionalities provided by these two features to enable the utilisation of multi-cores and multi-compute nodes, respectively. As of 30 Oct 2020, the majority of NCI-hosted matlab licenses provides only the feature Parallel Computing Toolbox, therefore, supports jobs running only within a single compute node. 

In order to utilise multi-cores within a single compute node, typically, the minimum programming effort would be the revision of the for loops into parfor loops in the .m file. For example, to parallelise the matlab jobs with the majority computation in a for loop read as

A = 512;
N = 1024;
y = NaN(1,N);
for i = 1:N
  y(i) = max(abs(eig(rand(A))))
end

simply do 

numberOfWorkers=8;
A = 512;
N = 1024;
y = NaN(1,N);

localCluster = parcluster('local');
localCluster.NumWorkers = numberOfWorkers;
parpool(localCluster);

parfor (i=1:N,numberOfWorkers)
  y(i) = max(abs(eig(rand(A))))
end

For further explorations of the functionality provided by the Parallel Computing Toolbox, please refer to Mathworks website [https://au.mathworks.com/help/distcomp/index.html].  If you want to run Matlab jobs across more than one compute nodes and confirm that you get access to the feature Distributed Computing Server, please contact help@nci.org.au for further instructions.

Once the .m file is ready, you will need to load the MATLAB application and license module.

To load the specific version of R2019b, please run

module load matlab/R2019b

To load the license module for your own matlab software group, please run

module load matlab_licence/<institution>

An example job submission script (matlabjob.sh) is provided below. It requests 48 processors, 190GB of memory, 400GB of local disk space for 10 hours in the normal queue and tells the job to mount /g/data/ab01 and /scratch/ab01 project folder into it. Within the job it passes the number of CPUs as the number of matlabWorkers and the PBS_JOBFS folder as the output directory outputDir, both of which are variables used in the matlab script mfile.m.

Note that the submission script uses the PBS directive`-l software`, to ask the job scheduler for the reservation of matlab token seats from license matlab_<institution> where the placeholder <institution> needs to be replaced by your own institution, see details in the section License requirements below. To change the number of processors, memory, or jobfs required, and/or project folders to mount, simply modify the appropriate PBS resource requests at the top of this file.

#!/bin/bash
#PBS -q normal
#PBS -l walltime=02:00:00
#PBS -l ncpus=48
#PBS -l mem=190GB
#PBS -l jobfs=400GB
#PBS -l storage=gdata/ab01+scratch/ab01
#PBS -l software=matlab_<institution>
#PBS -l wd

module load matlab/R2019b
module load matlab_licence/<institution>
matlab -nodisplay -nosplash -r "outputDir='$PBS_JOBFS',numberOfWorkers=$PBS_NCPUS, mfile, exit" > /g/data/ab01/$USER/job_logs/$PBS_JOBID.log

Please note, the flag -nodisplay passed to matlab enables running without the GUI, while the flag -nosplash disables the display of the Matlab logo.

Once the job submission script is saved to the same directory in which the matlab script mfile.m is located, run 

qsub matlabjob.sh 

inside that directory to submit your job to the PBS scheduler.

Running Single-core Jobs

Since Matlab is automatically multithreaded and, by default, it generates as many matlabWorkers as it is allowed to, it is always safe to explicitly set the maximum number of computational threads to one by adding the following line 

maxNumCompThreads(1)

at the beginning of the .m file or starting matlab with the option -singleCompThread as shown in the following example submission script.

#!/bin/bash
#PBS -q normal
#PBS -l ncpus=1
#PBS -l walltime=00:02:00
#PBS -l mem=250MB
#PBS -l storage=gdata/ab01+scratch/ab01
#PBS -l software=matlab_<institution>
#PBS -l wd

module load matlab/R2019b
module load matlab_licence/<iname>
matlab -nosplash -nojvm -singleCompThread < mfile.m > /g/data/ab01/$USER/job_logs/$PBS_JOBID.log

License requirements

NCI has a Hosting Provider Agreement with MathWorks that allows us to install MATLAB on our systems and have users provide their own licence to access it. In particular, if your institution has a licence for MATLAB that you can use on the computer in your office then it is most likely possible for you to use it on NCI systems. As of 30 Oct. 2020, there are 12 institutions have set up their configurations with NCI. If it’s not already set up, contact your licence administrator to ask them  to get in touch with us.

To get access to your own institution provided Matlab licence, join the corresponding software group in the table below on my.nci.org.au first. Once your group membership request is approved, the corresponding licence module becomes visible to use. Run `module avail matlab` to see the license modulefile name. Jobs need to load the license module together with the application module to run Matlab.

* CSIRO has a special arrangement with NCI that allows only a specific group of users to use their license. Please contact CSIRO licensing service to discuss the access. We can only add users to the group on their approval.