Ansys CFX is the industry-leading computational fluid dynamics software for turbomachinery applications. It offers highly scalable, high-performance computing (HPC) to help solve simulations quickly and cost effectively.

Usage

NCI installs CFX as part of the ANSYS package, load ansys module to add the executables into your search path.

module load ansys/2021r1

There are also license modulefiles for users to configure where to check out ANSYS licence(s). Users need to join the corresponding ansys software group before they can see the license modulefile, for example, only members of ansys_nci group can see the license module `ansys_licence/nci`. The software group membership gives not only the access to the license module,  and also allows checkouts from the corresponding licensing server.

To load the licence module for your own ansys software group, run

module load ansys_licence/<institution>

A example job submission script (cfxjob.sh) is provided below. It requests 96 processors, 380 GB of memory, 400GB of local disk space for 10 minutes in the normal queue and tells the job to mount /g/data/ab01 and /scratch/ab01 project folder into the job. Within the job it tells CFX to use the start method "Gadi_openmpi" and the loaded openmpi by the ansys module to run the simulation on $PBS_NCPUS processors.

#!/bin/bash
#PBS -q normal
#PBS -l walltime=00:10:00
#PBS -l ncpus=96
#PBS -l mem=380Gb
#PBS -l jobfs=400GB
#PBS -l storage=gdata/ab01+scratch/ab01
#PBS -l software=<license_string>
#PBS -l wd

module load ansys/2021r1
module load ansys_licence/<institution>

export CFX5_OPENMPI_DIR=$OPENMPI_ROOT
cfx5solve -def "StaticMixer.def" -part $PBS_NCPUS -start-method Gadi_openmpi

Note that the submission script uses the PBS directive`-l software`, to ask the job scheduler for the reservation of ansys licence seats from license  <license_string> , see details in the section `License requirements` below. To change the number of processors, memory, or jobfs required, and/or project folders to mount, simply modify the appropriate PBS resource requests at the top of this file.

The maximum number of CPU cores a CFX job can use is limited by the license availability. For example, NCI owns an academic ANSYS license file which has 128 seats for the hpc feature and it supports jobs request up to 128 CPU cores. If you submit many 128-core fluent jobs, they are expected to run one at a time.

Once the job submission script is saved to the same directory in which the input file StaticMixer.def is located, run 

qsub cfxjob.sh 

inside that directory to submit your job to the PBS scheduler.

Restart Jobs

If you need to run CFX for a very long period of time which exceeds the default walltime limit set for the job, see the walltime limit for different queues and number of cores on the Queue Limits page, we suggest to add checkpoints to the long job, save restart files and use the last one as input to the next job. 

License requirements

Users from the following institutions have access to their own institution licenses and should join the corresponding software group. To join a software group, login to my.nci.org.au first, search for the group by the software group name, and use the tab `Join` on its project page to submit the membership request. It may take up to 1 business day for the lead CI to respond. Once your group membership request is approved, the corresponding licence module becomes visible and ready for use. Run `module avail ansys` to list all ansys related modulefiles. Jobs need to load the license module together with the application module to run CFX.

* NCI owns an academic ANSYS license file which provides 25 CFD feature seats and 128 HPC feature seats. Users who have no access to the above licenses are welcome to join the software group ansys_nci to enable the access to modulefile ansys_licence/nci and the checkouts from licence ansys_nci.