Persistent Sessions

To support the use of some long-running, low CPU and memory demand processes, NCI provides a persistent sessions service on Gadi. This service is primarily designed for the use of workflow management tools that automatically submit and monitor PBS jobs.

Starting Sessions

All management of persistent sessions can be undertaken by logging into the Gadi login nodes using SSH, or the ARE environment.

To create a session use the following commands:

Replace the <name> parameter with the name you'd like to use for your session. Names must start with a letter (a-z), and only contain letters, numbers, and the hyphen character.

You can optionally specify the project to run the session under with the -p  parameter. Replace <project> with the project code of the project you'd like to run the session under.

Your session will automatically be allocated a hostname of the form <name>.<user>.<project>.ps.gadi.nci.org.au. These names are only visible within Gadi, they are not visible outside of Gadi.

Connecting to Sessions

Once a session has been created, you can connect to it from anywhere inside Gadi with SSH. 

[abc777@gadi-login-01 ~]$ ssh <name>.<user>.<project>.ps.gadi.nci.org.au

Warning: Permanently added '[<name>.<user>.<project>.ps.gadi.nci.org.au]:2222,[10.9.1.1]:2222' (ED25519) to the list of known hosts.

The first time you connect to a session you will see SSH configuring the host key for the session. This will not happen on future connections.

The SSH configuration on Gadi will automatically select the appropriate SSH keys and ports to connect to your persistent session. For more advanced users, if you have edited your SSH configuration file ensure that you do not have configuration in your own ~/.ssh/config file on Gadi that may prevent this working correctly.

Inside the Session

Once you have connected to the session using ssh  above, you can load whatever modules you require for your persistent session, and start the tools you need to use in the persistent session. If the software you are running inside the persistent session listens on the network, you should connect to it using the hostname. Do not hard code the IP address as it will likely be different each time the session is started.

Remember that persistent sessions are only designed for lightweight processes like a workflow manager. Attempting to run heavy tasks such as compute may result in your session being terminated.

If your tool makes regular requests to the PBS service to manipulate jobs, ensure that you are not placing an undue load on the PBS service by ensuring that tasks like qstat  are run at most once a minute. Large numbers of connections to the PBS service may reduce the performance of the PBS service for all users, and potentially result in the termination of your sessions.

Listing Running Sessions

You can list your running sessions with the persistent-sessions  tool:

[abc777@gadi-login-01 ~]$ persistent-sessions list [ -p <project> ] [ <uuid> ]


                                UUID  PROJECT         ADDRESS      CPUTIME  MEMORY

12345678-90ab-cdef-1234-567890abcdef     ab12        10.9.X.X 00:00:05.230    2.9M

You can optionally specify the project to list sessions for with the -p  parameter. Replace <project> with the project code of the project you'd like to list sessions for.

If you've already known the unique UUID for the session you'd like to check, simply provide the <uuid> parameter.

Terminating Sessions

Users can terminate any sessions they are running:

[abc777@gadi-login-01 ~]$ persistent-sessions kill <uuid>

To determine the UUID of the session you want to kill, use the list command as documented above.

Session Charging

Persistent sessions are currently not charged to the project's SU allocation. We will be adding charging for persistent sessions, with the charging details to be documented once the rates have been determined.

Note that the usage of persistent sessions will be closely monitored and usage that is not appropriate may result in actions including the termination of sessions, or suspension of NCI access for repeated violations.