Date: Fri, 29 Mar 2024 18:27:43 +1100 (AEDT) Message-ID: <1436706213.287.1711697263950@opus-prod1.nci.org.au> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_286_291606823.1711697263944" ------=_Part_286_291606823.1711697263944 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
HDF5 is a data model, library, and= file format for storing and managing data. It supports an unlimited variet= y of datatypes, and is designed for flexible and efficient I/O and for high= volume and complex data. HDF5 is portable and is extensible, allowing appl= ications to evolve in their use of HDF5. The HDF5 Technology suite includes= tools and applications for managing, manipulating, viewing, and analyzing = data in the HDF5 format. For more information on using the HDF5 library, or= any of the provided utilities see the HDF5 support page at the HDFgroup website.
To use HDF5 on Gadi, first load one of the HDF5 modules.
$ modul= e load hdf5/1.10.7
Please note that it is important to specify a version when loading a HDF= 5 module, as some files created by later HDF5 versions cannot be opened by = earlier versions. In order to build an application on Gadi against HDF5, on= ly the required HDF5 libraries need to be linked
$ gcc -= o read_h5_file read_h5_file.c -lhdf5 -lhdf5_hl
The libraries that HDF5 depends on are linked automatically by default u= sing data stored in the shared HDF5 libraries. You do not need to load any = underlying modules in order to compile your application. To use the fortran= interface, the fortran libraries also need to be linked
$ gfort= ran -o read_h5_file read_h5_file.f90 -lhdf5_fortran -lhdf5hl_fortran -lhdf5= -lhdf5_hl
Every HDF5 installation on Gadi has been built to be compatible with all=
GNU and Intel compilers available on Gadi. In the above commands, ic=
c
can be substituted for gcc
and ifor=
t
can be substituted for gfortran
without a=
ny other changes. The HDF5 libraries corresponding to your choice of compil=
er are selected at compile time by NCI's compiler wrappers. Every parallel =
HDF5 installation on Gadi is compatible with every MPI distribution install=
ed on Gadi by a similar mechanism, with one minor exception:
--mca io ompio
&nb=
sp; to mpirun
when you run your application. This is=
due to a break in compatibility between the default MPI-IO implementation =
between OpenMPI 3 and 4, which are otherwise binary compatible.We do not usually recommend the use of static linking on Gadi, as it pre=
vents applications from receiving important security or bug fixes that woul=
d otherwise be applied automatically when shared libraries are updated. If =
your application uses HDF5, and is compiled statically (e.g. by passing the=
-static
flag to the compiler) then you will need to=
include the libraries that HDF5 depends on. The easiest way to do this is =
to use the compiler wrappers provided with HDF5
$ h5cc = -static -o read_h5_file read_h5_file.c $ h5fc -static -o read_h5_file read_h5_file.f90
These wrappers can be used with both the Intel and GNU compilers. The un= derlying compiler used by each wrapper is selected based on the modules loa= ded in your environment. Note that MPI applications cannot be built statica= lly on Gadi.
As of HDF5 1.12, releases are now divided into 'Experimental' and 'Maint=
enance' releases, as outlined in the Introducing Experimental Releases page. For =
this reason, NCI will not install odd-numbered minor (i.e. 'Experimental') =
releases unless a specific feature new to that release is requested by user=
s. NCI will continue to proactively support even-numbered minor (i.e. 'Main=
tenance') releases. These will be installed on /apps
as =
soon as practical after they become available. As is standard policy for&nb=
sp;/apps
, NCI will not install any versions of HDF5 that were =
released earlier than when Gadi became generally available to users.
There are two HDF5 modules available for each software version:
$ modul= e avail hdf5 -------------------------- /apps/Modules/modulefiles ----------------------= ----- hdf5/1.8.21 hdf5/1.10.5 hdf5/1.10.7(default) hdf5/1.12.1 hdf5/1.8.21p hdf5/1.10.5p hdf5/1.10.7p hdf5/1.12.1p
The hdf5/<version>p
(henceforth referred t=
o as 'parallel') module variants denote that these installations have been =
built with parallel IO enabled. Unless your application specifically uses p=
arallel HDF5 features, e.g., by invoking the MPI-IO VFD using
herr_t = H5Pset_fapl_mpio( hid_t fapl_id, MPI_Comm comm, MPI_Info info )
we recommend that you use the serial HDF5 modules (i.e. the modules with=
out p
in the version number), even if you are compil=
ing an MPI-enabled application.
The following features are enabled for all HDF5 installations on Gadi:= p>
The C++ interface is not compatible with parallel HDF5, but has been ena= bled on the serial HDF5 installations.
With the release of HDF5 1.10, new features were introduced that, when e= nabled for a particular file, cause that file to be unreadable to HDF5 1.8 = and earlier. Specifically, creating a file with any of the following featur= es enabled:
Or creating a file with H5P_SET_LIBVER_BOUNDS ( fapl_id, low, high )
where hdf5/1.8.21
. =
Applications that build against HDF5 1.8 should be able to be built against=
HDF5 1.10 with no modification to the source.
HDF5 1.12 introduces further new features, and as a result, there is an = API break between HDF5 1.10 and HDF5 1.12. Applications that build against = HDF5 1.10 cannot be built against HDF5 1.12 without modification. Though th= ere is a compatibility option provided to build HDF5 1.12 with a 1.10 compa= tible API, we have not enabled this feature on Gadi as we have several HDF5= 1.10 builds, and using the HDF5 1.10 API disables use of new HDF5 1.12 fea= tures. For more information see the Migrating From HDF5 1.10 To HDF5 1.12 on the HDF= group website.