To run your Cylc suites, whether launched from Gadi login or from ARE, you will typically need to start a meta-scheduler session using the NCI's persistent session service. This is because suites tend to run for long periods of time, and you will want to close your ARE sessions, or logout of Gadi, while your complex suites scheduled onto Gadi continue to run on your behalf. The Cylc metascheduler session allows continuous monitoring of all your PBS jobs and submissions of the scheduled tasks in your Cylc suite to PBS. You can check on the status of your suites either inside a new ARE VDI session or a new Gadi login shell later on.
Power users of Met Office based suites frequently have complex workflows that need to run continuously for months, please see below our recommendation for running such long suites on Gadi.
Re-creation of the meta-scheduler session and then restarting suites is required on occasions such as quarterly maintenances and/or unscheduled system maintenances.
Start a Meta-scheduler Session
To start running long Cylc suites using the NCI persistent session service, you will first need to start the metascheduler session. Creation of the session can be done either in a ARE VDI session, or a Gadi login shell. After the session started, it will stay running until you kill it, or the node is rebooted. Typically nodes are rebooted at each system maintenance, so you will need to start new meta-scheduler session and restart your Cylc workflows after the quarterly maintenance.
To start the session, follow our suggestions on this page.
Note, you only need one meta-scheduler session to serve all your Cylc suites. It is capable of accommodating both Cylc7 and Cylc8 suites running at the same time as each login to the session has its own shell environment that can be configured differently for different suites.
Run Suites
To use the meta-scheduler persistent session in your Cylc suite runs, you don't have to have any special SSH/Rose/Cylc configurations of your personal account. Loading the right Cylc7/Cylc8 module is enough. Refer to this page for recommended ways of running Cylc7 suites and this one for Cylc8 suites.
Monitor Suites
We have our suggested way to monitor suites on the corresponding Cylc7 and Cylc8 page. One can monitor Cylc7 suites' progress either on the login node or inside the ARE VDI session, but for Cylc8, because `cylc gui` uses a browser it is recommended to be inside an ARE VDI session.
For monitoring suites progress across your research team, we have our recommendations of how to do it here.
Support for Short Suites and Debugging
For jobs that run for a short period of time, you can also use the workflow described above. If the short suite or debugging process has to run without a meta-scheduler session, by setting the environment variable `CYLC_SESSION` to localhost , you can run it all inside an ARE VDI session.