Introduction
PyAOS is a conda collection of packages deployed under project dk92 for our data analysis software and modules.
An overview of the PyAOS stack is shown below and you can find its details here and the package index of PyAOS is given here.
Accessing the module
Resources can be accessed by joining the project dk92 through mancini. Note that no storage or compute resources are provided by project dk92 as it is purely for accessing the software. You will need to use your existing compute NCI project code for computational resources.
The PyAOS module can be loaded as below
$ module use /g/data/dk92/apps/Modules/modulefiles/ $ module load pyaos/22.08
You can load PyAOS module on Gadi, OOD and ARE in a similar way with loading the NCI-data-analysis module. For more details, please see here.
Program environments
The PyAOS environment consists of plenty of python modules in general purposed data science, atmosphere and ocean science, together with the distributed computing frameworks, such as MPI, Dask, Ray etc. To view the full package list please run the following command
$ conda list
A list of feature packages in the latest PyAOS module are listed below
Name | Package | Version | Description |
|---|---|---|---|
all-purpose | |||
xarray | 2022.6.0 | A python library provides data models for working with labeled arrays and datasets. | |
iris | 3.2.1 | A python package for analysing and visualising Earth science data. | |
xcdat | 0.3.1 | Xarray Extended With Climate Data Analysis Tools. | |
cf-python | 3.13.0 | An Earth Science data analysis library that is built on a complete implementation of the CF data model. | |
pygeode | 1.4.1 | A python package intended to simplify the manipulation, analysis and visualization of geophysical datasets. | |
dask | 2022.8.1 | Distributed computing framework. | |
scipy | 1.9.0 | Fundamental algorithms for scientific computing in Python. | |
numpy | 1.22.4 | Fundamental package for scientific computing in Python. | |
matplotlib | 3.4.3 | A comprehensive library for creating static, animated, and interactive visualizations in Python. | |
General Utilises | |||
GeoCAT-comp (GeoCAT project) | geocat-comp | 2022.07.0 | Computational routines from the NCAR Command Language (NCL). |
PyFerret (PMEL) | pyferret | 7.63 | Quick exploration of oceanographic data. |
eofs (Andrew Dawson) | eofs | 1.4.0 | EOF analysis of spatial-temporal data. |
windspharm (Andrew Dawson) | windspharm | 1.7.0 | Computations on global wind fields in spherical geometry. |
regionmask (Mathias Hauser) | regionmask | 0.9.0 | Masks for commonly used geographic regions (Giorgi, SREX, etc) |
ocetrac (Hillary Scannell) | ocetrac | 0.1.4 | Label and track the evolution of unique geospatial features in gridded datasets. |
Data Access | |||
siphon | 0.9 | Utilities for downloading data from remote data services. | |
climetlab | 0.11.31 | Simplified access to climate and meteorological datasets. | |
Visualisation | |||
cartopy | 0.20.2 | Geographic map projections for plotting. | |
geoviews | 1.9.5 | Interactive exploration and visualisation of geographical, meteorological, and oceanographic datasets. | |
cmocean (Kristen Thyng) | cmocean | 2.0 | Beautiful colormaps for oceanography. See xcmocean for xarray integration. |
GeoCAT-viz (GeoCAT project) | geocat-viz | 2022.07.0 | Visualization routines from the NCAR Command Language (NCL). Examples at GeoCAT-examples. |
cf-plot | 3.1.23 | cf-python related functions for common contour, vector and line plots used in climate research. | |
Meteorology | |||
metpy | 1.3.1 | Tools for reading, visualising and performing calculations with weather data. | |
satpy | 0.37.1 | Reading, manipulating, and writing data from remote-sensing earth-observing meteorological satellite instruments. | |
arm_pyart | 1.12.5 | Weather radar algorithms and utilities. | |
pydda | 1.1.0 | Direct data assimilation framework for wind retrievals. | |
act-atoms | 1.1.9 | Toolkit for working with atmospheric time-series datasets of varying dimensions. | |
PyDSD (Joseph Hardin and Nick Guy) | pydsd | 1.0.6.2 | Utilities for working with disdrometer data. |
xcape | 0.1.4 | Fast convective parameters for numpy, dask, and xarray. | |
Oceanography | |||
gsw | 3.4.0 | Python implementation of the Thermodynamic Equation of Seawater 2010 (TEOS-10). | |
gsw-xarray | 0.2.1 | A wrapper that adds CF attributes to xarray outputs. | |
argopy (Guillaume Maze) | argopy | 0.1.12 | Argo data access, visualisation and manipulation. |
mixsea (Jesse Cusack & Gunnar Voet) | mixsea | 0.1.1 | Turbulence parameter estimation from fine scale oceanographic data. |
PyCO2SYS (Matthew Humphreys et al.) | pyco2sys | 1.8.1 | Marine carbonate system solver. |
Climate | |||
esmvaltool | 2.4.0 | Diagnostics and performance metrics for the evaluation of CMIP models. | |
pcmdi_metrics | 2.4.0 | Diagnostics and performance metrics for the evaluation of CMIP models. | |
cmip6_preprocessing (Julius Busecke) | cmip6_preprocessing | 0.6.0 | Tools for cleaning/standardization of the metadata associated with CMIP6 data files. |
xclim | 0.37.0 | Functions to compute climate indices from observations or model simulations. | |
icclim | 5.4.0 | Index Calculation for CLIMate. | |
climpred (Riley Brady and Aaron Spring) | climpred | 2.2.0 | Verification of weather and climate forecasts. |
climtas (Scott Wales) | climtas | 0.3.2 | Climtas is a package for working with large climate analyses. It focuses on the time domain with custom functions for Xarray and Dask data. |
climate-indices (James Adams) | climate-indices | 1.0.10 | Various climate index algorithms relating to precipitation and temperature. |
Spatial grids | |||
xESMF (Jiawei Zhuang) | xesmf | 0.6.3 | Universal regridder for geospatial data. |
gridded (NOAA-ORR-ERD) | gridded | 0.3.6 | A single way to work with results from any hydrodynamic/oceanographic model regardless of what type of grid it was computed on. |
pyresample | 1.25.1 | Resampling geospatial image data. | |
esmpy | 8.2.0 | Interface to the Earth System Modeling Framework (ESMF) regridding utility. | |
pyproj | 3.3.0 | Interface to PROJ (cartographic projections and coordinate transformations library). | |
GCM-Filters (Ocean Transport and Eddy Energy Climate Process Team) | gcm_filters | 0.3.0 | Diffusion-based spatial filtering of gridded data. |
uxarray | 2022.08.0 | Reading and recognizing unstructured grid models. | |
Modeling | |||
wrf-python (GeoCAT project) | wrf-python | 1.3.4.1 | A collection of diagnostic and interpolation routines for use with output from the Weather Research and Forecasting (WRF-ARW) Model. |
climlab (Brian Rose) | climlab | 0.8.1 | Process-oriented climate modeling. |
climt (Joy Monteiro et al) | climt | 0.16.25 | Climate modelling and diagnostics toolkit. |
xgcm | 0.8.0 | General circulation model post-processing with xarray. | |
Climate and Forecast (CF) metadata conventions | |||
| cf-checker (CEDA) | cfchecker | 4.1.0 | Check compliance of netCDF files against the CF Convention. |
| compliance-checker (IOOS) | compliance-checker | 5.0.2 | Check compliance of netCDF files against CF, ACDD, and IOOS Metadata Profile file standards. |
| cfdm (NCAS-CMS) | cfdm | 1.9.0.3 | Implements the data model of the CF metadata conventions. |
| cf-xarray (Deepak Cherian) | cf_xarray | 0.7.4 | Lightweight accessor for xarray objects that interprets CF attributes. |
Workflow management | |||
| aospy (Spencer Hill) | aospy | 0.3.1 | Automated climate data analysis and management. |
| cmdline-provenance (Damien Irving) | cmdline_provenance | 1.0.0 | For keeping track of your data processing steps. |
Version history
NCI-data-analysis module is regularly updated for more recent versions of the software components. We encourage you to access the latest module where-ever possible. The latest version 22.08 is singularity image based Conda environment.
PyAOS module | Platform version |
|---|---|
| 22.08 | python 3.9 |