User can use their own Julia packages if they need more than the default ones provided by the `NCI-data-analysis` module under the project dk92 are not enough, no matter if the julia session is running from an OOD VDI Desktop, within the ARE jupyter notebook or through command line on a gadi node.
Please see examples below for package installation into your project folder under /g/data. We recommend to test the installation using the built-in test files in the new package whenever possible and include the test in our examples.
...
.
OOD VDI
On the start of a Julia session, it uses packages provided by the module. The default location `/opt/julia` exists only in the singularity image and no modification is allowed. New packages have to be installed to another location with write permission.
To use another environment, add the path as the first element in `DEPOT_PATH`. In our example below, we choose to build the new environment in `/g/data/z00/.julia`.
When it is a new environment, the initialisation is required. An empty `Project.toml` file in the directory `/g/data/z00/.julia/environments/v<version>/` is the minimum requirement for the Julia package manager to find the new environment. Once found, it is ready to add the missing package to the new location.
To shift back to the default environment, simply pop the first element out of `DEPOT_PATH`.
| Code Block |
|---|
[abc123@ood-vn1 ~]$ module use /g/data/dk92/apps/Modules/modulefiles
[abc123@ood-vn1 ~]$ module load NCI-data-analysis/22.06
[abc123@ood-vn1 ~]$ julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.2 (2022-02-06)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> DEPOT_PATH
1-element Vector{String}:
"/opt/julia"
(@v1.7) pkg> st GraphIO
No Matches in `/opt/julia/environments/v1.7/Project.toml`
julia> ENV["PROJECT"]
"z00"
julia> pushfirst!(DEPOT_PATH,"/g/data/"*ENV["PROJECT"]*"/.julia")
2-element Vector{String}:
"/g/data/z00/.julia"
"/opt/julia"
shell> mkdir -p /g/data/z00/.julia/environments/v1.7
shell> touch /g/data/z00/.julia/environments/v1.7/Project.toml
(v1.7) pkg> st
Status `/g/data4/z00/.julia/environments/v1.7/Project.toml` (empty project)
Resolving package versions...
Installed Compat \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500------------ v4.1.0
Installed ArnoldiMethod \u2500----- v0.2.0
Installed GraphIO \u2500\u2500\u2500\u2500\u2500\u2500\u2500----------- v0.6.0
Installed Graphs \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500------------ v1.5.0
Updating `/g/data4/z00/.julia/environments/v1.7/Project.toml`
[aa1b3936] + GraphIO v0.6.0
Updating `/g/data4/z00/.julia/environments/v1.7/Manifest.toml`
...
Precompiling project...
5 dependencies successfully precompiled in 16 seconds (9 already precompiled)
(v1.7) pkg> test GraphIO
Testing GraphIO
Status `/local/z00/abc123/tmp/jl_jswoue/Project.toml`
[944b1d66] CodecZlib v0.7.0
[8f5d6c58] EzXML v1.1.0
[aa1b3936] GraphIO v0.6.0
[86223c79] Graphs v1.5.0
[fae87a5f] ParserCombinator v2.1.1
[ae029012] Requires v1.3.0
[699a6c99] SimpleTraits v0.9.4
[8bb1440f] DelimitedFiles `@stdlib/DelimitedFiles`
[8dfed614] Test `@stdlib/Test`
Status `/local/z00/abc123/tmp/jl_jswoue/Manifest.toml`
[ec485272] ArnoldiMethod v0.2.0
...
Test Summary: | Pass Total
GraphIO | 152 152
Testing GraphIO tests pass
julia> popfirst!(DEPOT_PATH)
"/g/data/z00/.julia"
(@v1.7) pkg> st Pluto
Status `/opt/julia/environments/v1.7/Project.toml`
[c3e4b0f8] Pluto v0.14.7
|
ARE Jupyter Notebooks
To install a package maintained in the Julia Package Listing requires internet access. Preferably, request the resource from `copyq` to Unlike Julia running in an OOD session which by default has the access to the external network, Julia notebook has to be running inside an ARE session that requests resources from `copyq` queue to get the internet access. [we might need to include analysis queue in the future]
The above example is also available in \[this notebook\] [writing another notebook to github].
Gadi Nodes
Not all gadi nodes have the access to the external network. Ideally, it is feasible to install and test Julia packages on the login node. However, If the installation or the test is too big to run on a login node and doesn't comply with the login node resource usage limit, simply submit an interactive copyq job and do the installation/test from there.
...