The NCI-DLWP-CS repository contains notebooks to train the DLWP-CS model with the NCI WeatherBench, which is a Deep learning benchmark developed at NCI.

Details about the DWLP-CS model can be found in the paper: 
https://doi.org/10.1029/2020MS002109

Details about the NCI WeatherBench can be found here: https://dx.doi.org/10.25914/q5na-a058

How to use 

0) Before you start.
    You must be a member of the following NCI projects

  wb00
  dk92
  vp91


Additionally, you need a project that has enough resources to run V100 GPUs for several hours. 
One can apply to join NCI projects here: https://my.nci.org.au/


1) Download the notebooks in any suitable location on Gadi 

cd /to/a/Gadi/location
git clone https://github.com/nci/NCI-DLWP-CS


2) Launch an NCI ARE (https://are.nci.org.au) Jupyter instance with the following parameters. 

Walltime (hours): 2 hours
Queue: gpuvolta
Compute Size: 1gpu (minimum or custom for more memory)
Project: <Choose one that has enough resources>
Storage: gdata/dk92+gdata/wb00+scratch/vp91+<All other storage that you need>

Module directories: /g/data/dk92/apps/Modules/modulefiles
Modules: nci-dlwp-cs/2024.04.30 
Jobfs size: 380GB


3) From the ARE instance, navigate to the notebook's location and run them individually.

Notebooks

The structure of direcoty 

.
├── 1 - Downloading and processing ERA5 - NCI.ipynb
├── 2 - Remapping to the cubed sphere - NCI.ipynb
├── 3 - Training a DLWP-CS model - NCI.ipynb
├── 4 - Predicting with a DLWP-CS model - NCI.ipynb
├── LICENSE
└── README.md


 - The notebooks are self-contained and one should be able to run without any extra input. 
- Each notebook contains information about the purpose and produced output.

Data location 

- All input data is read from the NCI WeatherBench under the `wb00` project (`/g/data/wb00/NCI-Weatherbench/`)
- Project `dk92` contains the modules to run the DWLP-CS model. 

Contact us

- All the code, data, and Notebooks in this repository are maintained by the NCI data and software modernisation team (https://opus.nci.org.au/display/DAE/Specialised+Environments+Home). 
- Please, contact us for any suggestion or bug report. 

  • No labels