You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Next »

User can use their own Julia packages if the default ones provided by the `NCI-data-analysis` module under the project dk92 are not enough, no matter if the julia session is running from an OOD VDI Desktop, within the ARE jupyter notebook or through command line on a gadi node.

Please see examples below for package installation into your project folder under /g/data. We recommend to test the installation using the built-in test files in the new package whenever possible and include the test in our examples. 

OOD VDI 


[abc123@ood-vn1 ~]$ module use /g/data/dk92/apps/Modules/modulefiles
[abc123@ood-vn1 ~]$ module load NCI-data-analysis/22.06
[abc123@ood-vn1 ~]$ julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.7.2 (2022-02-06)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |

julia> DEPOT_PATH
1-element Vector{String}:
"/opt/julia"

(@v1.7) pkg> st GraphIO
  No Matches in `/opt/julia/environments/v1.7/Project.toml`

julia> ENV["PROJECT"]
"z00"

julia> pushfirst!(DEPOT_PATH,"/g/data/"*ENV["PROJECT"]*"/.julia")
2-element Vector{String}:
 "/g/data/z00/.julia"
 "/opt/julia"

shell> mkdir -p /g/data/z00/.julia/environments/v1.7

shell> touch /g/data/z00/.julia/environments/v1.7/Project.toml

(v1.7) pkg> st
      Status `/g/data4/z00/.julia/environments/v1.7/Project.toml` (empty project)
Resolving package versions...
   Installed Compat \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 v4.1.0
   Installed ArnoldiMethod \u2500 v0.2.0
   Installed GraphIO \u2500\u2500\u2500\u2500\u2500\u2500\u2500 v0.6.0
   Installed Graphs \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500 v1.5.0
    Updating `/g/data4/z00/.julia/environments/v1.7/Project.toml`
  [aa1b3936] + GraphIO v0.6.0
    Updating `/g/data4/z00/.julia/environments/v1.7/Manifest.toml`
...
Precompiling project...
  5 dependencies successfully precompiled in 16 seconds (9 already precompiled)

(v1.7) pkg> test GraphIO
     Testing GraphIO
      Status `/local/z00/abc123/tmp/jl_jswoue/Project.toml`
  [944b1d66] CodecZlib v0.7.0
  [8f5d6c58] EzXML v1.1.0
  [aa1b3936] GraphIO v0.6.0
  [86223c79] Graphs v1.5.0
  [fae87a5f] ParserCombinator v2.1.1
  [ae029012] Requires v1.3.0
  [699a6c99] SimpleTraits v0.9.4
  [8bb1440f] DelimitedFiles `@stdlib/DelimitedFiles`
  [8dfed614] Test `@stdlib/Test`
      Status `/local/z00/abc123/tmp/jl_jswoue/Manifest.toml`
  [ec485272] ArnoldiMethod v0.2.0
...
Test Summary: | Pass  Total
GraphIO       |  152    152
     Testing GraphIO tests pass



ARE Jupyter Notebooks

To install a package maintained in the Julia Package Listing requires internet access. Preferably, request the resource from `copyq` to 


Gadi Nodes

Not all gadi nodes have the access to the external network. Ideally, it is feasible to install and test Julia packages on the login node. However, If the installation or the test is too big to run on a login node and doesn't comply with the login node resource usage limit, simply submit an interactive copyq job and do the installation/test from there.

The environment variable `SINGULARITYENV_JULIA_DEPOT_PATH` passes in its value to the environment variable `JULIA_DEPOT_PATH` within the singularity image in which julia and all its packages packages are installed. It overwrites the default value `/opt/julia`. If at any point a package installed in the module is required, simply point back to that Project.toml file by inserting the path back to  the `DEPOT_PATH` vector as the first element.

login node $ JULIA_PKGDIR=/g/data/$PROJECT/$USER/.julia.$(date +%Y).$(date +%m).$(date +%d).$(date +%H)
login node $ mkdir -p $JULIA_PKGDIR
login node $ export SINGULARITYENV_JULIA_DEPOT_PATH=$JULIA_PKGDIR
login node $ module use /g/data/dk92/apps/Modules/modulefiles
login node $ module load NCI-data-analysis/22.06
login node $ julia
julia > DEPOT_PATH
1-element Vector{String}:
 "/g/data/z00/abc123/.julia.2022.06.21.10"
(@v1.7) pkg> st
  Installing known registries into `/g/data/z00/abc123/.julia.2022.06.21.10`
      Status `/g/data/z00/abc123/.julia.2022.06.21.10/environments/v1.7/Project.toml` (empty project)
(@v1.7) pkg> add GraphIO
    Updating registry at `/g/data/z00/abc123/.julia.2022.06.21.10/registries/General.toml`
   Resolving package versions...
...
    Updating `/g/data/z00/abc123/.julia.2022.06.21.10/environments/v1.7/Project.toml`
  [aa1b3936] + GraphIO v0.6.0
    Updating `/g/data/z00/abc123/.julia.2022.06.21.10/environments/v1.7/Manifest.toml`
... 
Precompiling project...
  14 dependencies successfully precompiled in 11 seconds

(@v1.7) pkg> test GraphIO
     Testing GraphIO
      Status `/scratch/z00/abc123/tmp/jl_MbtVIQ/Project.toml`
...
Precompiling project...
  12 dependencies successfully precompiled in 3 seconds (14 already precompiled)
     Testing Running tests...
...
Test Summary: | Pass  Total
GraphIO       |  152    152
     Testing GraphIO tests passed 
julia> using GraphIO

julia> names(GraphIO)
9-element Vector{Symbol}:
 :CDFFormat
 :DOTFormat
 :EdgeListFormat
 :GEXFFormat
 :GMLFormat
 :Graph6Format
 :GraphIO
 :GraphMLFormat
 :NETFormat

julia> pushfirst!(DEPOT_PATH,"/opt/julia")
2-element Vector{String}:
 "/opt/julia"
 "/g/data/z00/yxs900/.julia.2022.06.21.10"

(@v1.7) pkg> st Graphs
      Status `/opt/julia/environments/v1.7/Project.toml`
  [86223c79] Graphs v1.7.0

julia> using Graphs
  • No labels