Specialised Environments

Page tree

In the following example, we run the suite `u-cz535` using the persistent session. If you don't have any persistent sessions alive or you suspect your current meta-scheduler session is broken, follow the instructions here to create a new one. Again, if you already have a working persistent session alive, you don't have to create a new one for running the suite u-cz535. It is a very short suite. There are only three tasks, `get_stream` executes locally inside the persistent session,  the other two, "build_stream" and "run_stream", are pbs jobs, each takes a few seconds to execute. And it needs only the standard project membership: hr22 , ki32 , ki32_mosrs  and access.

0. Confirm Project Memberships

To ensure you have the right project memberships in the meta-scheduler session, run the command `groups` like the following. 

[abc111@gadi-login-01 ~]$ ssh demo.abc111.xy99.ps.gadi.nci.org.au 
[abc111@demo ~]$ groups | grep "hr22\|ki32\|access"
access hh5 cws_help rt52 dk92 up99 hr22 ki32 ki32_mosrs ki32_nemo

If any missing ones, have a look at the `Check Group Memberships` section on this page for our suggestions about how to proceed. 

1. Load modules

One can either launch the Cylc workflow directly inside the persistent session or from the login node. 

From the login node, set the environment variable `CYLC_SESSION` before loading the modules.

[abc111@gadi-login-01 ~] $ export CYLC_SESSION=demo.abc111.xy99.ps.gadi.nci.org.au

If lauching the workflow directly inside the persistent session, login first

[abc111@gadi-login-01 ~] $ ssh demo.abc111.xy99.ps.gadi.nci.org.au

Then load the modules

$ module use /g/data/hr22/modulefiles/
$ module load cylc/8.2.1
Using the cylc session demo.abc111.xy99.ps.gadi.nci.org.au

Loading cylc/8.2.1
  Loading requirement: mosrs-setup/1.0.1

2. Checkout Suites

To checkout the suite, provide your MOSRS credentials. If your cache is still valid, you may skip this step and checkout the suite directly. If you never used your MOSRS account on Gadi, the command `mosrs-auth` leads you through the configuration steps in which you need to enter your MOSRS username once and password twice.  

$ mosrs-auth
INFO: You need to enter your MOSRS credentials here so that GPG can cache your password.
Please enter the MOSRS password for testuser: 
INFO: Checking your credentials using Subversion. Please wait.
INFO: Successfully accessed Subversion with your credentials.
INFO: Checking your credentials using rosie. Please wait.
INFO: Successfully accessed rosie with your credentials.

Once successfully verified, you can checkout the suite on Gadi.

$ rosie co u-cz535
[INFO] u-cz535: local copy created at /home/111/abc111/roses/u-cz535

3. Run Suites

$ cylc install
INSTALLED u-cz535/run1 from /home/111/abc111/roses/u-cz535
$ cylc play u-cz535
Loading cylc/8.2.1
  Loading requirement: mosrs-setup/1.0.1

 ▪ ■  Cylc Workflow Engine 8.2.1
 ██   Copyright (C) 2008-2023 NIWA
▝▘    & British Crown (Met Office) & Contributors

2023-10-01T11:19:35+11:00 INFO - Extracting job.sh to /home/111/abc111/cylc-run/u-cz535/run1/.service/etc/job.sh
u-cz535/run1: demo.abc111.xy99.ps.gadi.nci.org.au PID=19919

4. Monitor the Workflow

It is necessary to start an ARE VDI session to monitor the Cylc 8 workflow. Once inside the VDI session, open the Mate terminal and run

$ module use /g/data/hr22/modulefiles
$ module load cylc/8.2.1
$ cylc gui --new u-cz535

Mozilla Firefox will display the page. Click on a task to see more details of the associated job.

After all jobs finish, simple checks, such as listing the exit status of all the PBS jobs, can be useful.

$ grep "Exit Status" -R ~/cylc-run/u-cz535/runN/log/job
/home/111/abc111/cylc-run/u-cz535/runN/log/job/1/run_stream/NN/job.out:   Exit Status:        0
/home/111/abc111/cylc-run/u-cz535/runN/log/job/1/run_stream/01/job.out:   Exit Status:        0
/home/111/abc111/cylc-run/u-cz535/runN/log/job/1/build_stream/NN/job.out:   Exit Status:        0
/home/111/abc111/cylc-run/u-cz535/runN/log/job/1/build_stream/01/job.out:   Exit Status:        0

Note, the PBS jobs can return with the zero "Exit Status" but still have errors depending on how the tasks are composed. Closer inspections are recommended to look into individual tasks.



  • No labels