The optimal set up for OpenFOAM on Qarnot This document explains the requirements for the optimal set up of your existing OpenFOAM case on Qarnot.
It complements the existing example-oriented documentation and focuses on the practical requirements for running real user cases, especially in parallel and for cases with specific layouts such as multi-region simulations.
Scope This guide is intended for users who already have an OpenFOAM case working in their own environment and want to port it to Qarnot.
The objective is not to explain OpenFOAM basics, but to describe the conventions and settings required so that:
the case can run correctly on Qarnot, the parallel execution behaves as expected, the output volume remains manageable, and the platform can provide support under expected performance conditions. Existing references For a first introduction to OpenFOAM on Qarnot, refer to:
This page focuses on the additional technical requirements needed when adapting an existing case.
General requirements for a case submitted to Qarnot An OpenFOAM case submitted to Qarnot must be self-contained and batch-oriented.
In practice, this means:
the case must be launchable through an Allrun `Allrun all paths used in scripts must be relative to the case directory, the case must not depend on user-specific local environment settings, all commands invoked by Allrun If your case requires custom libraries, additional binaries, or a specific runtime setup not available by default, please contact Qarnot support at:
support-compute@qarnot-computing.com
How Qarnot runs an OpenFOAM case On Qarnot, the case is expected to be launched through its `Allrun` script.
Parallel execution relies on a Qarnot-compatible integration of runParallel
The number of MPI processes is determined from the value of `numberOfSubdomains` found in:
system/decomposeParDictThis value is used by Qarnot to determine how many cores must be used for the run.
Mandatory root-level system/decomposeParDict A root-level system/decomposeParDict
Qarnot uses this file to read numberOfSubdomains
At minimum, this root-level file must contain the correct value for:
numberOfSubdomains N;where N
Multi-region cases Some OpenFOAM cases, especially multi-region ones, may store decomposition settings in region-specific locations instead of using a single root-level system/decomposeParDict
A common example is a case with separate `fluid` and `solid` regions, each having its own decomposition settings.
In such situations, Qarnot still expects a root-level: system/decomposeParDict
Even if region-specific decomposition dictionaries already exist, you must create a root-level system/decomposeParDictnumberOfSubdomains
This root-level file can be a minimal or "dummy" file whose main purpose is to expose the expected numberOfSubdomains
MPI options and OMPI_OPTS Qarnot provides a default set of MPI options adapted to its Open MPI integration.
For the vast majority of users, the default configuration is the correct one and should not need any change.
Advanced users may add or override MPI flags through the OMPI_OPTS variable.
This mechanism is reserved for experienced users who understand the impact of MPI runtime options on performance and behavior. In normal usage, the default value should be kept unchanged.
Mandatory I/O settings for Qarnot OpenFOAM can generate very large numbers of files and significant metadata activity when run in parallel. This can create a very high I/O load, especially on demanding HPC workloads.
On Qarnot, the following settings are mandatory in order to keep the I/O pattern compatible with the platform constraints and to maintain acceptable performance and support conditions.
Without these settings, the storage and I/O infrastructure may not keep up efficiently with the workload.
1. Use collated file handling The following setting is mandatory:
fileHandler collated;This significantly reduces I/O pressure by avoiding the standard pattern where each processor writes its own set of files independently.
In practice, this greatly reduces file fragmentation and the number of small files created during the run, which is essential for large parallel jobs.
2. Write results in binary format The following setting is mandatory:
writeFormat binary;Compared with ASCII output, binary output typically reduces file size by around a factor of 3 on average.
This directly reduces:
output storage volume, snapshot size, transfer time, and pressure on the filesystem. 3. Enable output compression The following setting is mandatory:
writeCompression on; This further reduces the amount of data written to disk and helps keep output volume manageable.
Recommended output retention policy When possible, the following setting is strongly recommended:
purgeWrite 1;This keeps only the most recent written time directory.
It is recommended whenever the user only needs the final state, or only the most recent written state, because it drastically limits disk usage and output growth.
However, this setting is not appropriate for all cases.
For example, it may not be suitable for transient simulations or workflows that require access to previous time steps for restart, time-history analysis, or post-processing.
Example settings in controlDict Below is an example of a controlDict` configuration aligned with Qarnot requirements:
fileHandler collated;In summary:
`fileHandler collated;` is mandatory, `writeFormat binary;` is mandatory, `writeCompression on;` is mandatory, `purgeWrite 1;` is strongly recommended when previous written states are not needed. Example of a minimal root decomposeParDict For cases where the real decomposition logic is handled elsewhere, the root-level file can still be minimal as long as it exposes the expected number of subdomains:
numberOfSubdomains 64;This allows Qarnot to determine that the case must run on 64 cores.
Example expectations for Allrun Your Allrun
run without user interaction, rely only on relative paths, call tools available in the default OpenFOAM environment, and be compatible with parallel execution through runParallel If the case requires additional setup beyond that default environment, please contact support before running in production.
Final checklist before submitting a case to Qarnot Before launching your case on Qarnot, verify the following points:
the case is launched through Allrun Allrun all paths in scripts are relative, the case does not rely on a user-specific local environment, all commands called by Allrun a root-level system/decomposeParDict system/decomposeParDictnumberOfSubdomainsmulti-region cases still provide a root-level system/decomposeParDict. ffileHandler collated; writeFormat binary;writeCompression on;purgeWrite 1;Support If your case requires custom libraries, additional executables, or a non-standard setup, contact: support-compute@qarnot-computing.com
