Developers

CMake is our primary build system. If you are new to CMake, this short tutorial from the HEP Software foundation is the perfect place to get started. If you just want to use CMake to build the project, jump into sections 1. Introduction, 2. Building with CMake and 9. Finding Packages.

Dependencies

Before you start, you will need a copy of the pyAMReX source code:

git clone https://github.com/AMReX-Codes/pyamrex.git $HOME/src/pyamrex
cd $HOME/src/pyamrex

pyAMReX depends on popular third party software. On your development machine, follow the instructions here.

Note

Preparation: make sure you work with up-to-date Python tooling.

python3 -m pip install -U pip
python3 -m pip install -U build packaging setuptools wheel pytest
python3 -m pip install -U -r requirements.txt

Compile

From the base of the pyAMReX source directory, execute:

# find dependencies & configure
cmake -S . -B build -DAMReX_SPACEDIM="1;2;3"

# compile & install, here we use four threads
cmake --build build -j 4 --target pip_install

That’s all!

You can inspect and modify build options after running cmake -S . -B build with either

ccmake build

or by adding arguments with -D<OPTION>=<VALUE> to the first CMake call, e.g.:

cmake -S . -B build -DAMREX_GPU_BACKEND=CUDA -DAMREX_MPI=OFF -DAMReX_SPACEDIM="1;2;3"

That’s it!

Developers could now change the pyAMReX source code and then call the install lines again to refresh the installation.

Tip

If you do not develop with a user-level package manager, e.g., because you rely on a HPC system’s environment modules, then consider to set up a virtual environment via Python venv.

Build Options

CMake Option

Default & Values

Description

BUILD_TESTING

ON/OFF

Build tests

CMAKE_BUILD_TYPE

RelWithDebInfo/Release/Debug

Type of build, symbols & optimizations

CMAKE_INSTALL_PREFIX

system-dependent path

Install path prefix

CMAKE_VERBOSE_MAKEFILE

ON/OFF

Print all compiler commands to the terminal during build

AMReX_OMP

ON/OFF

Enable OpenMP

AMReX_GPU_BACKEND

NONE/SYCL/CUDA/HIP

On-node, accelerated GPU backend

AMReX_MPI

ON/OFF

Enable MPI

AMReX_PRECISION

SINGLE/DOUBLE

Precision of AMReX Real type

AMReX_SPACEDIM

3

Dimension(s) of AMReX as a ;-separated list

AMReX_BUILD_SHARED_LIBS

ON/OFF

Build AMReX library as shared (required for app extensions)

pyAMReX_IPO

ON/OFF

Compile with interprocedural/link optimization (IPO/LTO)

pyAMReX_INSTALL

ON/OFF

Enable install targets for pyAMReX

PYINSTALLOPTIONS

None

Additional options for pip install, e.g., -v --user

Python_EXECUTABLE

(newest found)

Path to Python executable

pyAMReX can be configured in further detail with options from AMReX, which are documented in the AMReX manual:

Developers might be interested in additional options that control dependencies of pyAMReX. By default, the most important dependencies of pyAMReX are automatically downloaded for convenience:

CMake Option

Default & Values

Description

BUILD_SHARED_LIBS

ON/OFF

Build shared libraries for dependencies

CCACHE_PROGRAM

First found ccache executable.

Set to -DCCACHE_PROGRAM=NO to disable CCache.

pyAMReX_amrex_src

None

Path to AMReX source directory (preferred if set)

pyAMReX_amrex_repo

https://github.com/AMReX-Codes/amrex.git

Repository URI to pull and build AMReX from

pyAMReX_amrex_branch

we set and maintain a compatible commit

Repository branch for pyAMReX_amrex_repo

pyAMReX_amrex_internal

ON/OFF

Needs a pre-installed AMReX library if set to OFF

pyAMReX_pybind11_src

None

Path to pybind11 source directory (preferred if set)

pyAMReX_pybind11_repo

https://github.com/pybind/pybind11.git

Repository URI to pull and build pybind11 from

pyAMReX_pybind11_branch

we set and maintain a compatible commit

Repository branch for pyAMReX_pybind11_repo

pyAMReX_pybind11_internal

ON/OFF

Needs a pre-installed pybind11 module if set to OFF

For example, one can also build against a local AMReX copy. Assuming AMReX’ source is located in $HOME/src/amrex, add the cmake argument -DpyAMReX_amrex_src=$HOME/src/amrex. Relative paths are also supported, e.g. -DpyAMReX_amrex_src=../amrex.

Or build against an AMReX feature branch of a colleague. Assuming your colleague pushed AMReX to https://github.com/WeiqunZhang/amrex/ in a branch new-feature then pass to cmake the arguments: -DpyAMReX_amrex_repo=https://github.com/WeiqunZhang/amrex.git -DpyAMReX_amrex_branch=new-feature.

You can speed up the install further if you pre-install these dependencies, e.g. with a package manager. Set -DpyAMReX_<dependency-name>_internal=OFF and add installation prefix of the dependency to the environment variable CMAKE_PREFIX_PATH. Please see the introduction to CMake if this sounds new to you.

If you re-compile often, consider installing the Ninja build system. Pass -G Ninja to the CMake configuration call to speed up parallel compiles.

Configure Your Compiler

If you don’t want to use your default compiler, you can set the following environment variables. For example, using a Clang/LLVM:

export CC=$(which clang)
export CXX=$(which clang++)

If you also want to select a CUDA compiler:

export CUDACXX=$(which nvcc)
export CUDAHOSTCXX=$(which clang++)

Note

Please clean your build directory with rm -rf build/ after changing the compiler. Now call cmake -S . -B build (+ further options) again to re-initialize the build configuration.

Run

We provide the public imports amrex.space1d, amrex.space2d and amrex.space3d, mirroring the compile-time option AMReX_SPACEDIM.

Due to limitations in AMReX, currently, only one of the imports can be used at a time in the same Python process. For example:

import amrex.space3d as amr

A 1D or 2D AMReX run needs its own Python process. Another dimensionality cannot be imported into the same Python process after choosing a specific dimensionality for import.