Suite Run Directory

At the module(site) level, we set suite run directories inside ~/cylc-run as symlinks to /scratch. 

To override this default configuration,  you can add the modification into the user level configuration ~/.metomi/rose.conf.  An example to symlink the suite run directory to /g/data  with the exception of ~/cylc-run/<suiteid>/share  to /scratch  is provided as below.

To be specific, the created suite run directory looks like:

$ readlink -f ~/cylc-run/u-cs809/
/g/data/xy99/abc111/cylc-run/u-cs809
$ readlink -f ~/cylc-run/u-cs809/share
/scratch/xy99/abc111/cylc-run/u-cs809/share

Remote Host

There are two modes available for running a Cylc7 suite on the NCI Gadi supercomputing platform.

Localhost Mode

In the localhost mode, Cylc can execute background jobs and submit PBS jobs directly from the persistent session itself. This mode is recommended for running Cylc suites at NCI as it avoids SSH logins and eliminates the need for initialization steps and changes to your system environment, see details here. If you're developing new Cylc suites to run at NCI, it is advisable to use this mode from the outset. For existing suites, it is highly encouraged to modify them so that they can be executed in the localhost  mode.

The primary requirement for a Cylc suite to run in the localhost mode is to specify "localhost" as the value of "host" for all jobs. For instance, you should set "host = localhost" for both "background" and "PBS" jobs as shown below.

[runtime]
    [[root]]
        [[[environment]]]
            STREAM_ROOT=$CYLC_SUITE_SHARE_DIR/stream
    [[local]]
        inherit = root
        [[[remote]]]
            host = localhost
        [[[job]]]
            batch system = background
    [[HPC]]
        inherit = root
        [[[remote]]]
            host = localhost
        [[[job]]]
            batch system = pbs

By following these, you can take full advantage of the localhost mode and simplify the execution of your Cylc suites on the NCI Gadi supercomputing platform.

Accessdev Compatible Mode

For some existing Cylc7 suites that have been migrated from accessdev, a simple change of all "host" values to "localhost" may not be sufficient to make them work. While additional efforts may be required to transition these suites to the localhost mode, you still have the chance to execute the original Cylc7 suite in a mode compatible with accessdev, with the possible inclusion of an extra steps which depends on where to run cylc/rose commands. 

Make sure the PBS storage option contains "gdata/hr22+gdata/ki32" in your Cylc suite. For example, the current UM trunk does contain the above two file systems and you need to add them into the file "trunk/rose-stem/site/nci/family.rc",

i.e. "  -l storage = scratch/access+gdata/access+gdata/hh5+gdata/hr22+gdata/ki32".