...
The Dask notebooks listed in the table below are designed to be run on both NCI's VDI and using Pangeo on Gadi. Some of these examples are compute and memory intensive and will greatly benefit in performance by scaling up using up using a Dask dask cluster on Gadi.
| filename | description | dataset | data project to join |
|---|---|---|---|
| dask_data_chunks_CMIP6.ipynb | Dask array basics; NetCDF chunks vs Dask chunks; chunking practices | oi10 | |
| dask_diagnositc_tools.ipynb | Introduce a few diagnostic tools such as visualising task graphs, local and distributed diagnostics tools | fs38 | |
| dask_intensive_calculation_cmip6.ipynb | Explore some of the Coupled Model Intercomparison Project (CMIP6) replication data to demonstrate how Dask handles expensive calculations | oi10 | |
| dask_interactive_visualisation_CMIP6.ipynb | Calculate time and zonal mean of the temperature of CMIP6 GFDL models and interactively visualise data | oi10 | |
| dask_memory_compute_management.ipynb | Strategies of managing larger-than-memory data using partition; saving data onto disk; cleaning ram; executing in the background | oi10 | |
| dask_xarray_CMIP6.ipynb | Use standard xarray operations on Dask Array; persist data into memory to speed up I/O; customise workflows and automatic parallelisation | fs38 | |
| dask_xarray_precipitation.ipynb | Calculate the intra-ensemble range for all the mean daily temperature and average seasonal precipitation in Australia using historical precipitation data of the CESM2 model within CMIP6 | fs38 |
...