There are several possible ways to run your Cylc7 suites on Gadi. Depending on which mode you are using and where you start the suite run, it requires different steps. We list the step sequence for running the Cylc7 suites in different ways as below and you can follow the details instruction for each step in their corresponding section. The major difference between `localhost` and `accessdev compatible` modes is whether to support `ssh gadi` inside the suite run.
ROSE_ORIG_HOST† | Localhost Mode | Accessdev Compatible Mode |
|---|---|---|
ARE VDI; or Gadi login node |
| 0. Initialization Step (once)
|
Persistent Session |
|
|
†ROSE_ORIG_HOST is the host where you run "cylc/rose" commands to start the workflow.
Here are the quick links to individual steps. If your own suite fails to run after following the instructions on this page, try an example suite first. We have examples suite run available for testing your environment.
No matter which way in the above you are using, the workflow will keep running inside the persistent session and you can look up the progress later.
Note
Charging rate using persistent session local resources is high. Try not to run anything else other than the Cylc metascheduler itself, unless it has to be run inside your persistent session. For example run `cylc gui` on the login node or inside an ARE VDI session rather than within the persistent session.
Initialisation Step (once - only for accessdev compatible-mode)
If you are running in "Localhost" mode or inside your persistent session, this is the wrong section to look at.
In this initialization step, you need to create an SSH key file named "~/.ssh/id_rsa-rose-cylc-gadi" by executing the script "/g/data/hr22/bin/gadi-cylc-setup-ps -y".
In the case of a non-existing ~/.ssh directory, the script will create the following files under ~/.ssh directory to make Cylc suites work in the passwordless login to gadi or gadi.nci.org.au via localhost.
~/.ssh/authorized_keys ~/.ssh/id_rsa-rose-cylc-gadi ~/.ssh/id_rsa-rose-cylc-gadi.pub ~/.ssh/id_rsa-rose-cylc-gadi-restricted.pub
Importantly, this initialisation step does NOT update your ~/.ssh/config file and thus it does NOT change any existing ssh configurations.
You only need to run this initialisation step once.
Specify Target Session
A user may have several active sessions concurrently. It's essential to specify which specific session you intend to to run the Cylc7 suite before loading the environment module.
There are two ways to specify the target persistent session for Cylc7 workflow. You could either
- insert the session name into a file named ~/.persistent-sessions/cylc-session ( Cylc7 module will keep fetching the session name from it).
or
- specify your session name with an environment variable CYLC_SESSION ( It will also rewrite the content of ~/.persistent-sessions/cylc-session with the new session name).
before loading the Cylc module.
For instance, the file could contain the following session name.
[abc111@gadi-login-02 ~]$ more ~/.persistent-sessions/cylc-session cylc-test.abc111.xy99.ps.gadi.nci.org.au
In general, the Cylc7 module will initially attempt to retrieve the session name from the CYLC_SESSION environment variable. In cases where the CYLC_SESSION environment variable is not available, the module will then seek to obtain the session name from the file located at ~/.persistent-sessions/cylc-session. If neither of them exists, loading the module fails.
After loading the module, the file ~/.persistent-sessions/cylc-session will be updated and made consistent with the last used target session name.
Load the Cylc Module
From the $ROSE_ORIG_HOST, you need to load the Cylc7 module first before starting your Cylc7 suite.
$ module use /g/data/hr22/modulefiles $ module load cylc7/23.09
Please note, DO NOT use any previous versions of cylc7 module earlier than 23.09 to run your suite that needs to launch Cylc meta-scheduler remotely as they are not configured to work with the persistent sessions.
If this is the first time you run Cylc jobs on Gadi, it is very likely that you need to checkout suites from MOSRS before actually starting the suite run. In order do this, you need to do authentication first. Run
$ mosrs-auth
will lead you through the MOSRS authentication.
If you have one of the following directories missing,
- ~/.gnupg
- ~/.metomi
- ~/.subversion
it will go through the MOSRS configuration setup, asking for your MOSRS account once and the corresponding password twice. Otherwise, it refreshes your MOSRS credentials.
Start Your Cylc Suite
Before starting your suite run, if the suite is going to run for the first time using a persistent session inside Gadi, please double check our suggestions to prepare the suite.
From the $ROSE_ORIG_HOST, you could start your Cylc7 suite under the ~/roses directory. For example, you could start a suite with the following command or other ones:
$ rose suite-run
Check the last several line of the output to make sure it is running in your persistent session.
... [INFO] [INFO] *** listening on https://cylc-test.abc111.xy99.ps.gadi.nci.org.au:1861/ *** [INFO] [INFO] To view suite server program contact information: [INFO] $ cylc get-suite-contact u-cz168 [INFO] [INFO] Other ways to see if the suite is still running: [INFO] $ cylc scan -n 'u-cz168' cylc-test.abc111.xy99.ps.gadi.nci.org.au [INFO] $ cylc ping -v --host= cylc-test.abc111.xy99.ps.gadi.nci.org.au u-cz168 [INFO] $ ps -opid,args 206 # on cylc-test.abc111.xy99.ps.gadi.nci.org.au [INFO]
Now your suite starts running in the persistent session and you can quit Gadi login node or ARE VDI.
Monitor Your Cylc Suite Run
Your Cylc7 workflow continues to operate within the persistent session. You have the flexibility to monitor its progress conveniently by executing the following commands after logging into either the Gadi login node or an ARE VDI session.
[abc111@gadi-login-07 ~]$ module use /g/data/hr22/modulefiles [abc111@gadi-login-07 ~]$ module load cylc7/23.09 [abc111@gadi-login-07 ~]$ cylc gscan