Introduction

This page describes four state-of-the-art weather prediction models that are deployed on the Gadi. Those models are Graphcast, Panguweather, and the two versions of FourCastNet. Originally the code was developed by ECMWF (https://github.com/ecmwf-lab/ai-models) and this is a customized version for NCI

The following table shows the model names along with the reference papers and ECMWF repository links.  

Model nameDeveloperReferenceECMWF repository
FourCastNet
(Adaptive Fourier Neural Operator)		Nvidiahttps://arxiv.org/pdf/2202.11214.pdfhttps://github.com/ecmwf-lab/ai-models-fourcastnet
FourCastNet-v2
(Spherical Fourier Neural Operator)		Nvidiahttps://arxiv.org/pdf/2306.03838.pdfhttps://github.com/ecmwf-lab/ai-models-fourcastnetv2
GraphCast Google Deepmindhttps://arxiv.org/pdf/2212.12794.pdfhttps://github.com/ecmwf-lab/ai-models-graphcast
Panguweather
https://arxiv.org/pdf/2211.02556.pdfhttps://github.com/ecmwf-lab/ai-models-panguweather


ARE 

The models can be accessed by loading the 'ai-models/2023.11.14' module from the '/g/data/dk92/apps/modulefiles/ai-models' directory. To open a Jupyter Notebook session fill in the following: 

  1. Login to the ARE website: https://are.nci.org.au/
  2. And fill up the following fields: 

    Walltime (hours): 2
    Queue: gpuvolta
    Compute Size: 1gpu
    Project: <Your project code>
    Storage: gdata/dk92+gdata/wb00+gdata/rt52+gdata/<your project code>+scratch/<your project code>

    In the advanced section: 

    Module directories: /g/data/dk92/apps/Modules/modulefiles
    Modules: ai-models/2023.11.14
    Jobfs size: 380GB
  3.  Click Launch to start a JupyterLab session.

Following video replicates the steps.


Note

Your NCI account needs access to the rt52 (join) and dk92 (join) projects to run the notebooks. Also, your project must have enough service units (SU) to run the tasks. 

Please visit the following page to join NCI projects: https://my.nci.org.au/


Notebooks download 

There are four notebooks available, one for each model. They use our NCI replica of the ERA5 dataset from the rt52 project to run the models. To run the models: 

a) Log in to the Gadi ( gadi.nci.org.au ) using ssh.

b) Go to the directory, where you want to store the notebooks, this could be any directory of your choosing. Then, clone the repository

cd /to/the/user/directory
git clone https://github.com/nci/NCI-AI-Models.git

c) The folder will have the following files:

-rw-r--r-- 1  40873169 Nov 20 18:39 AI-Models-fourcastnet.ipynb
-rw-r--r-- 1   8115820 Nov 20 18:39 AI-Models-fourcastnet_v2.ipynb
-rw-r--r-- 1  10367345 Nov 20 18:39 AI-Models-graphcast.ipynb
-rw-r--r-- 1  19961265 Nov 20 18:39 AI-Models-panguweather.ipynb
-rw-r--r-- 1      1066 Nov 20 18:39 LICENSE
-rw-r--r-- 1        15 Nov 20 18:39 README.md

d) Now, in the previously started ARE session, browse to the location and open the notebooks.

Next, visit the following page to get details about running the notebooks: https://opus.nci.org.au/x/iINnDw


  • No labels