Gadi /scratch maintenance was completed and normal operations resumed at 12:15 am AEST 17 June.
Downtime begins: 6:00 pm AEST Wed 16 June 2021
Downtime ends: 10:00 am AEST Thu 17 June 2021 (estimated)
Services affected: Gadi login, compute and data mover nodes
NCI has been advised by the vendor that urgent maintenance of the /scratch file system is necessary as soon as possible. The /scratch file system will undergo essential maintenance between 6:00 pm AEST Wednesday 16 June and 10:00 am Thursday 17 June. Gadi login, compute and data mover nodes will be offline and unavailable to users during this maintenance period.
To support this urgent maintenance two operational constraints will be in effect from 12:00 pm AEST Thursday 10 June until Friday 18 June:
- Changes to /scratch allocations will not be permitted.
- Job wall time extensions will not be permitted.
A standard job wall time horizon is in place for this urgent maintenance. Queues will drain as normal in preparation for the maintenance downtime. Gadi jobs which would be scheduled to run into or start during the maintenance period will be held, and then released when normal operations resume.
The /g/data file systems and associated data and virtual services not dependent on Gadi /scratch are expected to remain operational during the scheduled maintenance period.
/scratch Hard and Soft Quotas Reset - IMPORTANT
Due to capacity limitations on /scratch and to ensure overall stability, the vendor has advised NCI that hard quotas on /scratch must be reset to match the project (soft) quotas. This change will result in strict enforcement of project quotas. There will no longer be a grace period on /scratch if a project is over quota.
This quota policy change will take effect on Thursday 17 June when operations resume following /scratch maintenance. From 17 June, projects that are over quota on /scratch will be blocked from running jobs.
At present the /scratch file system is heavily subscribed, with a number of projects over quota. System monitoring has identified a large number of user files on /scratch which have not been accessed recently and should be removed or relocated to persistent storage. All users are encouraged to clean up unused or unneeded files on scratch as soon as possible. /scratch is intended for temporary working files only.
Queen's Birthday Public Holiday – Monday 14 June
Monday 14 June is the Queen's Birthday public holiday in all states and territories except Queensland and Western Australia. NCI systems are expected to operate as normal on that day (subject to operational constraints to support /scratch maintenance described above). Support requests received during the public holiday long weekend 12-14 June will be addressed as soon as possible starting Tuesday 15 June.
NCI User Services