NCI Help

Page tree

Overview

OpenFOAM is the leading free, open source software for computational fluid dynamics (CFD).

On Gadi we have two types of OpenFOAM.  One comes from The OpenFOAM Foundation https://openfoam.org/ and has single or double digit version numbers, like 7, 8 or 5,x

Another flavor is developed by OpenCFD Lmd. since 2004 https://www.openfoam.com/ and has version number resembling year, such as 2006.

How to use

First you need to decide on the version of the software you want to use. Use

$ module avail OpenFOAM

to check what versions are available. We normally recommend using the latest version available. For example, to load the version 8 of OpenFOAM use

$ module load OpenFOAM/8

For more details on using modules see our software applications guide.

An example job submission script (openFoam_pbs.sh) is provided below. It requests 96 CPUs, 140 GiB of memory, 100 GiB of local disk space, and 10 hours of walltime for the job. This script should be saved in the working directory from which the analysis will be done.

Note that the submission script uses the PBS software resource flag -l software=OpenFOAM. To change the number of processors, memory, or jobfs required, simply modify the appropriate PBS resource requests at the top of this file.

#!/bin/bash
 
#PBS -l ncpus=96
#PBS -l mem=140GB
#PBS -l jobfs=100GB
#PBS -l walltime=10:00:00
#PBS -l software=OpenFOAM
#PBS -l wd
 
# Load module, always specify version number.
module load OpenFOAM/8
 
# Must include `#PBS -l storage=scratch/ab12+gdata/yz98` if the job
# needs access to `/scratch/ab12/` and `/g/data/yz98/`
 
mpirun <foamExec> <otherArgs> -parallel > output.log

Replace the <foamExec> with the executable, such as icoFoam, and <otherArgs> with the actually arguments passing to the executable that you are using.

To run the job you would use the command

$ qsub openFoam_pbs.sh

We suggest that you carefully go through the OpenFOAM's user guide before any job submissions to Gadi. The user guide PDF files are available on Gadi under the corresponding installation directories. 

OpenFOAM is compiled with GNU compilers.

By default OpenFOAM simulations produces many small files. Please use collated file format to reduce the number of files.

Compiling your own solver

Loading OpenFOAM modules sets a number of environment variables. This is what needs to be used instead  of running OpenFOAM/etc/bashrc.

Variable FOAM_USER_LIBBIN specifies where the user libraries will be located and FOAM_USER_APPBIN points to the directory for executables. Users may want to change these directories if required.

Authors: Andrey Bliznyuk, Mohsin Ali
  • No labels