JupyterLab Sessions for Deep Learning Jobs

ARE runs on the Gadi supercomputer and jobs can be run on any Gadi queues. Deep Learning jobs in general run on GPUs. Gadi currently has three GPU queues, namely gpuvolta , analysis , and dgxa100 . We recommend to run the deep learning JupyterLab session on the gpuvolta queue, ask for the 1gpu size and use the code of the NCI project which supports you with its compute grant (SU).

Request a JupyterLab Session

On the ARE dashboard page, select JupyterLab and request resources following the example request form below. To fit the form in the page, it has been cut into three parts and put into three columns, from left to right, with repeated field to indicate the continuity.

In the "Project" field, fill in the NCI project code you want to run your JupyterLab session. The cost will be charged from the named project.

In the "Storage" field, declare all the project directories that you need access to on /scratch and /g/data. For example, add gdata/rt52 if you want to read from the ERA5 dataset. If you are not a member of rt52 yet, please join the project here.

The minimum PBS storage directive required for this workshop is "gdata/dk92+gdata/wb00+gdata/rt52"+scratch/<PROJECT>, where <PROJECT> should be identical to the project code used in the Project field of the ARE JupyterLab request form. In the example we used the project om02. Replace it with your own project code in both fields.

In the "Jobfs size" field, request the amount of local disk required by the Cylc suite. If missing, the default value is 100 MiB, which, in the majority of cases, is not enough to support local tasks that actively writing data to `$TMPDIR`. In the following example, 100GiB jobfs is requested.

Click the "Advanced options" button to unfold entire form.

In the field of "Module directories", type in the path of the module files.

In the field of "Modules" type in the module names separated by space.

In case the queue is very busy, you can submit the job 24 hours in advance and tell PBS not to consider starting the job until 11:30am, 23 Nov 2023 in the "PBS flags" field.