Introduction
A variety of pythonlib modules ( with python2) that reside in /g/data/access (called "/access") currently conflict with the hr22 cylc7 module. A job may fail if calling those "/access" python lib packages from native rose/cylc7 python (e.g., "rose-ana*" tasks).
To address such issue, we have deployed py27env module under hr22 to provide compatible python2 packages with the hr22 cylc7 module.
Usage
module use /g/data/hr22/modulefiles module load py27env/22.12
Package list
Selected packages in the py27env module are listed below
| py27env | 22.12 |
|---|---|
| bzip2 | 1.0.8 |
| cdat-lite | 6.0.1 |
| cdat_info | 2.10 |
| cffi | 1.14.0 |
| distarray | 2.12.2 |
| hdf4 | 4.2.15 |
| hdf5 | 1.10.1 |
| jasper | 1.900.1 |
| jpeg | 9d |
| netcdf4 | 1.3.1 |
| numpy | 1.11.3 |
| python | 2.7.18 |
| six | 1.16.0 |
| ecmwf-api-client | 1.6.1 |
| h5netcdf | 0.7.4 |
| matplotlib | 2.2.3 |
| f90nml | 1.4.3 |
| h5py | 2.8.0 |
| mock | 3.0.5 |
Compatibility with /access pythonlib modules
Currently the pythonlib projects under /access contains the following modules:
$ module use ~access/modules $ module avail pythonlib --------------------------------- /g/data/access/projects/access/modules ---------------------------------- pythonlib/cartopy/0.10.0 pythonlib/matplotlib/1.4.3 pythonlib/python-bufr/0.2-5 pythonlib/cartopy/0.11.1 pythonlib/mock/1.0.1 pythonlib/python-bufr/0.2-6 pythonlib/cartopy/0.12.0 pythonlib/mule/2017.8.1 pythonlib/six/1.9.0 pythonlib/cartopy/0.13.1 pythonlib/mule/2019.01.1 pythonlib/um2netcdf4/2.0 pythonlib/cdat-lite/6.0rc2-fixed pythonlib/mule/2020.01.1 pythonlib/umfile_utils/access_cm2 pythonlib/ecmwf-api-client/1.6.1 pythonlib/mule/2022.05.1 pythonlib/umfile_utils/cmip6 pythonlib/f90nml/1.0.2 pythonlib/netCDF4/1.5.3 pythonlib/umfile_utils/latest(default) pythonlib/h5netcdf/0.2.1 pythonlib/netCDF4/1.5.5 pythonlib/umfile_utils/raijin pythonlib/h5py/2.5.0 pythonlib/python-bufr/0.2-2
Some of them are incompatible with the hr22 cylc7 module with a common issue as below
$ module use /g/data/hr22/modulefiles/ $ module load cylc7/23.09 Using the cylc session mytest.****.***.ps.gadi.nci.org.au Loading cylc7/23.09 Loading requirement: mosrs-setup/1.0.1 $ module use ~access/modules $ module load pythonlib/netCDF4/1.5.5 $ /g/data/hr22/apps/cylc7/bin/python Python 2.7.18 |Anaconda, Inc.| (default, Apr 23 2020, 22:42:48) [GCC 7.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import netCDF4 Traceback (most recent call last): File "<stdin>", line 1, in <module> File "build/bdist.linux-x86_64/egg/netCDF4/__init__.py", line 3, in <module> File "build/bdist.linux-x86_64/egg/netCDF4/_netCDF4.py", line 7, in <module> File "build/bdist.linux-x86_64/egg/netCDF4/_netCDF4.py", line 6, in __bootstrap__ ImportError: /lib64/libssh.so.4: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b >>>
In this case, if you see this error, you should then load hr22 "py27env" module instead of access "pythonlib" modules to utilise the compatible packages with hr22 cylc7 module, i.e.
module use /g/data/hr22/modulefiles module load py27env/22.12