Reasons for performing housekeeping tasks
Resource usage is an immediate and ongoing consideration that justifies regular housekeeping.
- The compute grant provided by your project is limited.
- The storage provided by your project is limited.
Regular housekeeping
The following regular housekeeping tasks will help reduce the resource usage of your project.
- Delete unwanted jobs
- Archive data
- Remove runtime directories
- Terminate idling VDI sessions
Delete unwanted jobs
To list all your current jobs in queues, run
$ qstat -u $USER -sw
gadi-pbs:
Req'd Req'd Elap
Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time
------------------------------ --------------- --------------- --------------- -------- ---- ----- ------ ----- - -----
12345678.gadi-pbs abc123 normal-exec example1 2108380 1 2 9216m 02:00 R 00:20
Job run at Mon Jan 30 at 16:00 on (gadi-cpu-clx-1489:ncpus=2:mem=9437184kb:jobfs=4194304kb)
12345679.gadi-pbs abc123 normal-exec example2 -- 1 48 190gb 01:00 W --
--
12345680.gadi-pbs abc123 normal-exec example3 -- 148 7104 28tb 05:00 Q --
--
12345685.gadi-pbs abc123 normal-exec example4 -- 1 12 48gb 40:00 Q --
--
For jobs that are no longer needed, delete them from the queue. For example, to delete jobs 12345680 and 12345685, run
$ qdel 12345680 12345685
Archive data
The rose suite-run command creates files on the /scratch file system. Files on /scratch expire if they have not been accessed in the last 100 days. To archive any files you want to keep for a longer time, move them to a directory on /g/data.
Remove runtime directories
Every suite creates its own runtime directory on /scratch. After a few suite runs the files created on /scratch can easily accumulate to a few hundreds of GiB of storage. Make sure to clean up the runtime directory of each suite once you have finished with it. For example, to clean up the runtime directory for the suite u-cs809, proceed as below.
[abc123@gadi-cpu-bdw-0005 ~]$ cd ~/roses/u-cs809 [abc123@gadi-cpu-bdw-0005 u-cs809]$ rose suite-clean Clean u-cs809? y/n (default n) y [INFO] delete: localhost:/scratch/om02/abc123/cylc-run/u-cs809 [INFO] delete: localhost:cylc-run/u-cs809/work [INFO] delete: localhost:cylc-run/u-cs809/share/cycle [INFO] delete: localhost:cylc-run/u-cs809/share [INFO] delete: localhost:cylc-run/u-cs809
Terminate idling VDI sessions
This page shows all active interactive sessions you own on ARE. At any time you should have no more than one active VDI desktop session running. Running sessions have the button Launch VDI Desktop shown at the bottom. An example of the active session can be found here. If you accidentally have multiple VDI desktops listed, launch them to find idling ones and use the red Delete button to terminate them.