MATLAB

Overview

MATLAB is a programming and numeric computing platform used by millions of engineers and scientists to analyze data, develop algorithms, and create models. Simulink is a dynamic simulation environment to design and simulate systems for model development. Together these flagship platforms contain thousands of pre-written and validated functions to accelerate your workflows across multiple CPU and GPU cores.

Beyond MATLAB and Simulink, there are over 100 specialist toolboxes that extend and enhance the core functionality through additional, domain specific, computational capabilities. These range from biomedical and computational financial analysis to geologic and mapping investigations via multibody simulations and scenario exploration. Toolboxes are written by domain experts to simplify your work in these specific areas.

Usage

Typically, running a Matlab job on Gadi that benefits from the speedup of multi-cores would require, 

1. joining the matlab software group for your institution, matlab_<institution>, to allow your access to the Matlab license
2. load modulefile for the corresponding license
3. setting up both your .m file and the job submission script to allow the computation- and data-intensive problems using multi-cores, GPUs and multi-nodes. 
4. Make use of the smpd deamon available in Matlab for parallelisation.

   When using smpd, please specify -smpdfile to $PBS_JOBFS/<file> to avoid creating smpd files in /tmp.

Regarding the license access, please check the section License below for more details.   

Once get the access to the matlab license on Gadi, please confirm with your institution that whether the license comes with either of the features 'Parallel Computing Toolbox' and 'Distributed Computing Server'. Although some linear algebra and numerical functions, such as matrix products, fft, eig, sort, are implicitly multi-threaded and make use of all the cores available to the job without any extra programming efforts, see an incomplete list provided by Mathworks Support Team here, the majority of the functions provided by matlab runs only on a single core and users need to explicitly use the functionalities provided by these two features to enable the utilisation of multi-cores and multi-compute nodes, respectively. As of 30 Oct 2020, the majority of NCI-hosted matlab licenses provides only the feature Parallel Computing Toolbox, therefore, supports jobs running only within a single compute node. 

In order to utilise multi-cores within a single compute node, typically, the minimum programming effort would be the revision of the for loops into parfor loops in the .m file. For example, to parallelise the matlab jobs with the majority computation in a for loop read as

% Initialisation
A = 512;
N = 1024;
y = NaN(1,N);

% Executing serial `for` loop
for i = 1:N
  y(i) = max(abs(eig(rand(A))))
end

% Print results
disp(y);
disp("Done");

simply do 

% Initialisation
numberOfWorkers = 48;
A = 512;
N = 1024;
y = NaN(1,N);

% Parallel pool creation
parpool('local', numberOfWorkers);

% Executing parallel `for` loop
parfor(i = 1:N, numberOfWorkers)
  y(i) = max(abs(eig(rand(A))))
end

% Print results
disp(y);
disp("Done");

For further explorations of the functionality provided by the Parallel Computing Toolbox, please refer to Mathworks website [https://au.mathworks.com/help/distcomp/index.html].  If you want to run Matlab jobs across more than one compute nodes and confirm that you get access to the feature Distributed Computing Server, please contact help@nci.org.au for further instructions.

Once the .m file is ready, you will need to load the MATLAB application and license module.

To load the specific version of R2019b, please run

$ module load matlab/R2019b

To load the license module for your own matlab software group, please run

$ module load matlab_licence/<institution>

An example job submission script (matlabjob.sh) is provided below. It requests 48 processors, 190GB of memory, 400GB of local disk space for 2 hours in the normal queue. Within the job it passes the number of CPUs as the number of matlabWorkers and the PBS_JOBFS folder as the output directory outputDir, both of which are variables used in the matlab script mfile.m.

Note that the submission script uses the PBS directive`-l software`, to ask the job scheduler for the reservation of matlab token seats from license matlab_<institution> where the placeholder <institution> needs to be replaced by your own institution, see details in the section License requirements below. To change the number of processors, memory, or jobfs required, and/or project folders to mount, simply modify the appropriate PBS resource requests at the top of this file.

#!/bin/bash

#PBS -q normal
#PBS -l walltime=02:00:00
#PBS -l ncpus=48
#PBS -l mem=190GB
#PBS -l jobfs=400GB
#PBS -l software=matlab_<institution>
#PBS -l wd

# Load modules, always specify version number.
module load matlab/R2019b
module load matlab_licence/<institution>

# Must include `#PBS -l storage=scratch/ab12+gdata/yz98` if the job
# needs access to `/scratch/ab12/` and `/g/data/yz98/`. Details on:
# https://opus.nci.org.au/display/Help/PBS+Directives+Explained

# Run matlab application
matlab -nodisplay -nosplash -r "outputDir='$PBS_JOBFS',numberOfWorkers=$PBS_NCPUS, mfile, quit" > $PBS_JOBID.log

Please note, the flag -nodisplay passed to matlab enables running without the GUI, while the flag -nosplash disables the display of the Matlab logo.

Once the job submission script is saved to the same directory in which the matlab script mfile.m is located, run 

$ qsub matlabjob.sh 

inside that directory to submit your job to the PBS scheduler.

Running Single-core Jobs

Since Matlab is automatically multithreaded and, by default, it generates as many matlabWorkers as it is allowed to, it is always safe to explicitly set the maximum number of computational threads to one by adding the following line 

maxNumCompThreads(1)

at the beginning of the .m file or starting matlab with the option -singleCompThread as shown in the following example submission script.

#!/bin/bash

#PBS -q normal
#PBS -l ncpus=1
#PBS -l walltime=00:02:00
#PBS -l mem=250MB
#PBS -l software=matlab_<institution>
#PBS -l wd

# Load modules, always specify version number.
module load matlab/R2019b
module load matlab_licence/<institution>

# Must include `#PBS -l storage=scratch/ab12+gdata/yz98` if the job
# needs access to `/scratch/ab12/` and `/g/data/yz98/`. Details on:
# https://opus.nci.org.au/display/Help/PBS+Directives+Explained

# Run matlab application
matlab -nosplash -nojvm -singleCompThread < mfile.m > $PBS_JOBID.log

License requirements

NCI has a Hosting Provider Agreement with MathWorks that allows us to install MATLAB on our systems and have users provide their own licence to access it. In particular, if your institution has a licence for MATLAB that you can use on the computer in your office then it is most likely possible for you to use it on NCI systems. As of 22 Oct. 2021, there are 14 institutions have set up their configurations with NCI. If it’s not already set up, contact your licence administrator to ask them  to get in touch with us.

To get access to your own institution provided Matlab licence, join the corresponding software group in the table below on my.nci.org.au first. Once your group membership request is approved, the corresponding licence module becomes visible to use. Run `module avail matlab` to see the license modulefile name. Jobs need to load the license module together with the application module to run Matlab.

* CSIRO has a special arrangement with NCI that allows only a specific group of users to use their license. Please contact CSIRO licensing service to discuss the access. We can only add users to the group on their approval.