Ctrl+K

Installation

Installation#

How to install#

The following instructions will install Utopia into a development environment on your machine. If you simply want to run Utopia, you can do so via a ready-to-use docker image; see below for more information.

Step-by-step Instructions#

These instructions are intended for ‘clean’ macOS (both Intel and Apple Silicon) or recent Ubuntu setups. Since Windows supports the installation of Ubuntu via Windows Subsystem for Linux, Utopia can also be used on Windows. Follow the WSL Installation Guide to install Ubuntu, then follow the instructions for Ubuntu in this README.

Note: Utopia is always tested against a recent Ubuntu release, currently Ubuntu 22.04. However, you can use Utopia with any earlier release, as long as the dependencies can be fulfilled.

⚠️ If you encounter difficulties, have a look at the troubleshooting section. If this does not resolve your installation problems, please file an issue in the GitLab project.

1 — Clone Utopia#

First, create a Utopia directory at a place of your choice. This is where the Utopia repository will be cloned to. When working with or developing for Utopia, auxiliary data will have a place there as well.

In your terminal, enter the Utopia directory you just created and invoke the clone command:

git clone https://gitlab.com/utopia-project/utopia.git

Alternatively, you can clone via SSH, using the address from the “Clone” button on the project page.

After cloning, there will be a new utopia directory (mirroring this repository) inside your top-level Utopia directory.

2 — Install dependencies#

Install the third-party dependencies using a package manager.

Note: If you have Anaconda installed, you already have a working Python installation on your system, and you can omit installing the python packages below. However, notice that there might be issues during the configuration step. Have a look at the troubleshooting section to see how to address them.

On Ubuntu (22.04)#
apt update
apt install cmake gcc g++ gfortran git libarmadillo-dev libboost-all-dev \
            libhdf5-dev libspdlog-dev libyaml-cpp-dev pkg-config \
            python3-dev python3-pip python3-venv

Further, we recommend installing the following optional packages:

apt update
apt install ffmpeg graphviz doxygen

You will probably need administrator rights.

Note: For simplicity, we suggest installing the meta-package libboost-all-dev which includes the whole Boost library. If you want a minimal installation (only the strictly required components), use the following packages instead: libboost-dev libboost-test-dev libboost-graph-dev libboost-regex-dev.

On macOS#

First, install the Apple Command Line Tools:

xcode-select --install

There are two popular package managers on macOS, Homebrew and MacPorts; we recommend to use Homebrew. Here are the installation instructions for both:

  • Homebrew:

    Install the required packages:

    brew update
brew install armadillo boost cmake hdf5 pkg-config python3 spdlog yaml-cpp

    Further, we recommend installing the following optional packages:

    brew update
brew install ffmpeg graphviz doxygen

  • MacPorts:

    Please be aware that port commands typically require administrator rights (sudo).

    Install the required packages:

    port install armadillo boost cmake hdf5 python37 py37-pip spdlog yaml-cpp

    Select the installed versions of Python and Pip:

    port select --set python python37
port select --set python3 python37
port select --set pip pip37
port select --set pip3 pip37

    Further, we recommend installing the following optional packages:

    port install ffmpeg graphviz doxygen

3 — Configure and build#

Enter the repository and create your desired build directory:

cd utopia
mkdir build

Now, enter the build directory and invoke CMake (and mind the caveats below):

cd build
cmake ..

Note: If you are using MacPorts, append the location of your Python installation to the CMake command (this is only required when calling CMake on a clean build directory):

cmake -DPython_ROOT_DIR=/opt/local ..

The terminal output will show the configuration steps, which includes the installation of further Python dependencies and the creation of a virtual environment.

After this, you can build a specific or all Utopia models using:

make dummy     # builds only the dummy model
make -j4 all   # builds all models, using 4 CPUs

4 — Run a model 🎉#

You should now be able to run a Utopia model. Being in the build directory, call:

source ./activate

to enter the virtual environment, where the Utopia Command Line Interface (CLI) is available. (If you later want to exit the virtual environment, call the deactivate command.)

Note: If you are using csh or fish shells, use the respective activate scripts located in build/utopia-env/bin/ (see below).

The utopia command is now available and gives you control over running and evaluating model simulations:

utopia run dummy

The model output will be written into ~/utopia_output/dummy/<timestamp>. For more information on how to use the command line interface, see the information for users below and the documentation.

5 — Make sure everything works#

This step is optional, but recommended.

To make sure that Utopia works as expected on your machine, build and carry out all tests.

Optional Installation Steps#

The following instructions will enable additional, optional features of Utopia.

Enable Multithreading in Utopia Models#

  1. Install the optional dependencies for multithreading.

    • On Ubuntu, we recommend using the GCC compiler with Intel TBB:

      apt update && apt install libtbb-dev

      Alternatively, one may install the Intel oneAPI base toolkit following these installation instructions. The only required package is intel-basekit. It includes the oneDPL library we use for parallelization.

    • On macOS, with Homebrew (please mind the note below!):

      brew update && brew install onedpl

    • On macOS with MacPorts, we are currently unsure whether multithreading is workable.

  2. Enter the Utopia build directory, and call CMake again. This time, enable the use of multithreading with the MULTITHREADING option:

    cd build
cmake -DMULTITHREADING=On ..

    At the end of its output, CMake should now report that the “Multithreading” feature has been enabled. If the requirements for multithreading are not met, however, the build will fail.

  3. Re-compile the models. Inside the build directory, call

    make all

  4. Depending on the algorithms used inside the respective model, it will automatically exploit the multithreading capabilities of your system when executed! 🎉

Utopia via Docker#

Docker is a free OS-level virtualization software. It allows running any application in a closed environment container.

The Utopia docker image is a way to run Utopia models and evaluate them without having to go through the installation procedure. It is also suitable for framework and model development.

The image and instructions on how to create a container from it can be found on the ccees/utopia docker hub page.

Quickstart#

This section gives a glimpse into working with Utopia. It’s not more than a glimpse; after playing around with this, consult the documentation to gain access to more information material, and especially: the Utopia Tutorial.

How to run a model?#

The Utopia command line interface (CLI) is, by default, only available in a Python virtual environment, in which utopya (the Utopia frontend) and its dependencies are installed. To conveniently work with the frontend, you should thus enter the virtual environment. Execute one of the commands below, depending on which type of shell you’re using:

source ./build/activate                  # For bash, zsh, or similar
source ./build/utopia-env/activate.csh   # For csh
source ./build/utopia-env/activate.fish  # For fish

Now, your shell should be prefixed with (utopia-env). (To exit the virtual environment, simply call deactivate.) All the following should take place inside this virtual environment.

As you have already done with the dummy model, the basic command to run a model named SomeModel is:

utopia run SomeModel

where SomeModel needs to be replaced with a valid model name. To get a list of available models, run the utopia models ls command. Alternatively, have a look at the src/utopia/models directory, where they are implemented.

The utopia run command carries out a pre-configured simulation for that model, loads the data, and performs automated plots.

Note:

  • The script will always run through, even if there were errors in the individual parts. Thus, you should check the terminal output for errors and warnings (red or yellow, respectively). If you want the CLI to fail on errors, you can also set the --debug flag.

  • By default, you can find the simulation output in ~/utopia_output/SomeModel/<timestamp>. It contains the written data, all the used configuration files, and the default plots.

For further information, e.g. on how you can pass a custom configuration, check the CLI help:

utopia --help
utopia run --help

The best place to continue from here is the tutorial.

Utopia Documentation#

This Readme only covers the very basics. For all further documentation, tutorials, guides, model descriptions, frequently asked questions and more, you should consult the actual Utopia documentation.

The latest build (corresponding to the latest commit to master) is available online.

See below on how to build the documentation locally.

Information for Developers#

SSH Repository Access#

To work comfortably with Utopia, access to the GitLab needs to be easy. The best way to achieve that is by registering a so-called SSH key with your GitLab account.

To do that, follow the linked instructions to generate a key pair and to add a key to your GitLab account.

Building the documentation locally#

To build the documentation locally, first make sure that all submodules are downloaded, because they are needed for building the documentation:

git submodule update --init --recursive

Then navigate to the build directory and run

make doc

In case you also want to generate (some of) the figures that are embedded into the documentation, set the UTOPIA_DOC_GENERATE_FIGURES environment variable:

UTOPIA_DOC_GENERATE_FIGURES=True make doc

After building, carrying out a link check and running some documentation tests is advisable:

make check_docs

The Sphinx-built user documentation will then be located at build/doc/html/. The C++ doxygen-documentation can be found at build/doc/doxygen/html/. Open the respective index.html files to browse the documentation.

Choosing a different compiler#

CMake will inspect your system paths and use the default compiler. You can use the CC and CXX environment variables to select a specific C and C++ compiler, respectively. As compiler paths are cached by CMake, changing the compiler requires you to delete the build directory and re-run CMake.

Ubuntu 22.04 LTS (Jammy Jellyfish), for example, provides GCC versions 10 to 12. To use GCC 12, for example, first install it via APT:

apt update && apt install gcc-12 g++-12

Then create the build directory, enter it, and set the CC and CXX environment variables when calling CMake:

mkdir build && cd build
CC=/usr/bin/gcc-12 CXX=/usr/bin/g++-12 cmake ..

Alternatively, you can export these variables in separate bash commands before executing CMake.

Build Types#

If you followed the instructions above, you have a Release build which is optimized for maximum performance. If you need to debug, you should reconfigure the entire project with CMake by navigating to the build folder and calling

cmake -DCMAKE_BUILD_TYPE=Debug ..

Afterwards, call

make <something>

to rebuild executables. CMake handles the required compiler flags automatically.

The build type (as most other CMake flags) persists until it is explicitly changed by the user. To build optimized executables again, reconfigure with

cmake -DCMAKE_BUILD_TYPE=Release ..

Testing#

Utopia contains unit tests to ensure consistency by checking whether class members and functions are working correctly. This is done both for the C++ and the Python code. The tests are integrated into the GitLab Continuous Integration pipeline, meaning that tests are run upon every push to the project and failing tests can be easily detected. Tests can also be executed locally, to test a (possibly altered) version of Utopia before committing and pushing changes to the GitLab.

See the user manual for more details on how to run tests.

Setting up a separate repository for models#

Working inside a clone or a fork of this repository is generally not a good idea: it makes updating harder, prohibits efficient version control on the models, and makes it more difficult to include additional dependencies or code.

To make setting up such a project repository as easy as possible, we provide a template repository, which can be used to start a new Utopia project! Follow the instructions in the models_template project for more information.

Make sure to call the following command to export package information to the CMake project registry, such that a separate repository can find Utopia:

cmake -DCMAKE_EXPORT_PACKAGE_REGISTRY=On ..

Dependencies#

Software Required Version Tested Version Comments
GCC >= 9 11.2 Full C++17 support required
or: clang >= 9 14.0 Full C++17 support required
or: Apple LLVM >= 9 Full C++17 support required
CMake >= 3.16 3.22
pkg-config >= 0.29 0.29
HDF5 >= 1.10.4 1.10.7
Boost >= 1.67 1.74 required components: graph, regex and unit_test_framework
Armadillo >= 9.600 10.8.2
yaml-cpp >= 0.6.2 0.7.0
spdlog >= 1.3 1.9.2
Python3 >= 3.6 3.10.4

Utopia aims to allow rapid development, and is thus being tested only against the more recent releases of its dependencies. Currently, Utopia is tested against the packages provided by Ubuntu 22.04. However the above version requirements (i.e., those enforced by the build system) can be fulfilled also with Ubuntu 19.10.

To get Utopia running on a system with an earlier Ubuntu version, the above dependencies still need to be fulfilled. You can use the Ubuntu Package Search to find the versions available on your system. If a required version is not available, private package repositories may help to install a more recent version of a dependency.

If you encounter difficulties with the installation procedure for any of these dependencies, please file an issue in the GitLab project.

Python#

Utopia also has some Python dependencies, which are automatically installed during the configuration step of the installation.

The following table includes the most important Python dependencies:

Software Version Comments
utopya >= 1.2 The (outsourced) Utopia frontend package
dantro >= 0.18 Handle, transform, and visualize hierarchically organized data
paramspace >= 2.5 Makes parameter sweeps easy

In addition, the following packages are optionally used for development of the framework or its models.

Software Version Comments
pytest For model tests
pre-commit >= 2.18 For pre-commit hooks
black >= 22.6 For formatting python code
Sphinx == 5.* Builds the Utopia documentation

These requirements are defined in the .utopia-env-requirements.txt file; in case installation fails, a warning will be emitted during configuration.

Troubleshooting#

  • If you have a previous installation and the build fails inexplicably, removing the build directory completely and starting anew from the configuration step should help.
    In cases where the installation used to work but at some point stopped working, this should be a general remedy. If the problem does not seem to be related to the Python environment, deleting only build/CMakeCache.txt may already suffice and save some configuration time.

  • If you have trouble with more recent HDF5 versions on macOS, one workaround is to use an older version:

    brew install hdf5@1.10
brew link hdf5@1.10 --overwrite

  • Anaconda ships its own version of the HDF5 library which is not compatible with Utopia. To tell CMake where to find the correct version of the library, add the following argument (without the comments!) to the cmake .. command during configuration:

    -DHDF5_ROOT=/usr/           # on Ubuntu
-DHDF5_ROOT=/usr/local/     # on macOS (Homebrew)

    For Ubuntu, the full command would then be:

    cd build
cmake -DHDF5_ROOT=/usr/ ..
Contents