Catalog File

An intake catalog file is a YAML or JSON file that contains a description of a collection of data sources, such as CSV, HDF5, NetCDF, and other formats.  They can include information about the location, format, and any relevant metadata associated with the data sources. Each dataset in the NCI dataset indexing scheme has its own catalog file in YAML or Json formats.

For NCI, the catalog file of intake-ESM scheme is in JSON format.

Data Source File

In the NCI Intake-ESM scheme, the data source file encapsulates essential metadata within several columns, commonly employed in data analysis and visualization. The intake-esm catalog file in CSV format is a structured document that serves as a lightweight and human-readable inventory of Earth System Model (ESM) datasets. Offering key details about accessible datasets, it streamlines their discovery and seamless integration into data analysis workflows. The adoption of the CSV format enhances simplicity and compatibility with a diverse range of data analysis tools.

Data layout

Intake-ESM catalog files can be loaded into a Pandas DataFrame, providing a tabular representation of the catalog's metadata. 

NCI intake-ESM data catalogues 

NCI currently provides intake-esm data catalogs for a number of data collections. See our documentation for more details: https://opus.nci.org.au/x/HQG9D.