In the following example, we run the suite `u-cq161` using the persistent session. If you don't have any persistent sessions alive, follow the instructions here to create one. Again, if you already have a persistent session alive, you don't have to create a new one for running the suite u-cq161. It is a very simple suite. there are only two tasks, `get_stream` executes locally inside the persistent session, and "build_stream" submitted as a pbs job which takes a few seconds to execute. And it needs only the standard project membership: hr22 , ki32 , ki32_mosrs and access.
0. Confirm Project Memberships
To ensure you have the right project memberships in the meta-scheduler session, run the command `groups` like the following.
[abc111@gadi-login-01 ~]$ ssh -Y demo.abc111.xy99.ps.gadi.nci.org.au [abc111@demo ~]$ groups | grep "hr22\|ki32\|access" access hh5 cws_help rt52 dk92 up99 hr22 ki32 ki32_mosrs ki32_nemo
If any missing ones, have a look at the `Check Group Memberships` section on this page for our suggestions about how to proceed.
1. Load Modules
One can either launch the Cylc workflow directly inside the persistent session or from the login node. In this example of the suite u-cq161, we launch the run from inside the persistent session. After confirm the project memberships, you should have been inside the persistent session already. If not, run
[abc111@gadi-login-01 ~] $ ssh -Y demo.abc111.xy99.ps.gadi.nci.org.au
Then load the modules
$ module use /g/data/hr22/modulefiles/ $ module load cylc7/23.09 Loading cylc7/23.09 Loading requirement: mosrs-setup/1.0.1
2. Check out Suites
To checkout the suite, provide your MOSRS credentials. If your cache is still valid, you may skip this step and checkout the suite directly. If you never used your MOSRS account on Gadi, the command `mosrs-auth` leads you through the configuration steps in which you need to enter your MOSRS username once and password twice.
$ mosrs-auth INFO: You need to enter your MOSRS credentials here so that GPG can cache your password. Please enter the MOSRS password for testuser: INFO: Checking your credentials using Subversion. Please wait. INFO: Successfully accessed Subversion with your credentials. INFO: Checking your credentials using rosie. Please wait. INFO: Successfully accessed rosie with your credentials.
Once successfully verified, you can checkout the suite on Gadi.
$ rosie co u-cq161 [INFO] u-cq161: local copy created at /home/111/abc111/roses/u-cq161
3. Run Suites
$ cd ~/roses/u-cq161 $ rose suite-run [INFO] export CYLC_VERSION=7.9.7 [INFO] export ROSE_ORIG_HOST=demo.abc111.xy99.ps.gadi.nci.org.au [INFO] export ROSE_SITE=nci [INFO] export ROSE_VERSION=2019.01.7 [INFO] create: /scratch/xy99/abc111/cylc-run/u-cq161 [INFO] symlink: /scratch/xy99/abc111/cylc-run/u-cq161 <= /home/111/abc111/cylc-run/u-cq161 [INFO] create: log.20231002T022419Z [INFO] symlink: log.20231002T022419Z <= log [INFO] create: log/suite [INFO] create: log/rose-conf [INFO] symlink: rose-conf/20231002T132419-run.conf <= log/rose-suite-run.conf [INFO] symlink: rose-conf/20231002T132419-run.version <= log/rose-suite-run.version [INFO] create: bin [INFO] install: bin [INFO] source: /home/111/abc111/roses/u-cq161/bin [INFO] install: rose-suite.info [INFO] source: /home/111/abc111/roses/u-cq161/rose-suite.info [INFO] install: suite.rc [INFO] REGISTERED u-cq161 -> /home/111/abc111/cylc-run/u-cq161 [INFO] create: share [INFO] create: share/cycle [INFO] create: work [INFO] chdir: log/ [INFO] ._. [INFO] | | The Cylc Suite Engine [7.9.7] [INFO] ._____._. ._| |_____. Copyright (C) 2008-2019 NIWA [INFO] | .___| | | | | .___| & British Crown (Met Office) & Contributors. [INFO] | !___| !_! | | !___. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ [INFO] !_____!___. |_!_____! This program comes with ABSOLUTELY NO WARRANTY; [INFO] .___! | see `cylc warranty`. It is free software, you [INFO] !_____! are welcome to redistribute it under certain [INFO] [INFO] *** listening on https://demo.abc111.xy99.ps.gadi.nci.org.au:43094/ *** [INFO] [INFO] To view suite server program contact information: [INFO] $ cylc get-suite-contact u-cq161 [INFO] [INFO] Other ways to see if the suite is still running: [INFO] $ cylc scan -n 'u-cq161' demo.abc111.xy99.ps.gadi.nci.org.au [INFO] $ cylc ping -v --host=demo.abc111.xy99.ps.gadi.nci.org.au u-cq161 [INFO] $ ps -opid,args 21711 # on demo.abc111.xy99.ps.gadi.nci.org.au [INFO]
4. Monitor the Workflow
Given X11 forwarding enabled, one can monitor any Cylc 7 workflows on the login node. For example,
Sue@local $ ssh -Y abc111@gadi.nci.org.au [abc111@gadi-login-08 ~]$ module use /g/data/hr22/modulefiles/ [abc111@gadi-login-08 ~]$ module load cylc7/23.09 Loading cylc7/23.09 Loading requirement: mosrs-setup/1.0.1 [abc111@gadi-login-08 ~]$ cylc gui
A GCylc window will pop up. Select "File→Open Another Suite", then suite u-cq161 to look up the progress.
After all jobs finish, simple checks, such as listing the exit status of all the PBS jobs, can be useful.
$ grep "Exit Status" -R ~/cylc-run/u-cq161/log/job/ /home/111/abc111/cylc-run/u-cq161/log/job/1/run_stream/NN/job.out: Exit Status: 0 /home/111/abc111/cylc-run/u-cq161/log/job/1/run_stream/01/job.out: Exit Status: 0 /home/111/abc111/cylc-run/u-cq161/log/job/1/build_stream/NN/job.out: Exit Status: 0 /home/111/abc111/cylc-run/u-cq161/log/job/1/build_stream/01/job.out: Exit Status: 0
Note, the PBS jobs can return with the zero "Exit Status" but still have errors depending on how the tasks are composed. Closer inspections are recommended to look into individual tasks.