Utopia::Models::HdfBench::HdfBenchModel Class Reference

The HdfBench Model. More...

#include <HdfBench.hh>

Inheritance diagram for Utopia::Models::HdfBench::HdfBenchModel:

Collaboration diagram for Utopia::Models::HdfBench::HdfBenchModel:

Public Types
using	Base = Model< HdfBenchModel, HdfBenchModelTypes >
	The base model type.
using	DataSet = typename Base::DataSet
	Data type for a dataset.
using	Config = typename Base::Config
	Data type that holds the configuration.
using	Clock = std::chrono::high_resolution_clock
	Type of clock.
using	Time = std::chrono::high_resolution_clock::time_point
	Type of a time point, retrieved from the clock.
using	DurationType = std::chrono::duration< double >
	Type of the duration measure, should be a floating-point type.
using	BenchFunc = std::function< double(const std::string, Config)>
	Type of a benchmark function pointer.
Public Types inherited from Utopia::Model< HdfBenchModel, HdfBenchModelTypes >
using	Config = typename ModelTypes::Config
	Data type that holds the configuration.
using	DataManager = DataIO::Default::DefaultDataManager< HdfBenchModel >
	The data manager to use, specialized with the derived model.
using	DataGroup = typename ModelTypes::DataGroup
	Data type that is used for storing datasets.
using	DataSet = typename ModelTypes::DataSet
	Data type that is used for storing data.
using	RNG = typename ModelTypes::RNG
	Data type of the shared RNG.
using	Space = typename ModelTypes::Space
	Data type of the space this model resides in.
using	Time = typename ModelTypes::Time
	Data type for the model time.
using	Monitor = typename ModelTypes::Monitor
	Data type for the monitor.
using	MonitorManager = typename ModelTypes::MonitorManager
	Data type for the monitor manager.
using	Level = typename ModelTypes::Level
	Data type for the hierarchical level.

Public Member Functions
template<class ParentModel >
	HdfBenchModel (const std::string &name, ParentModel &parent_model, const DataIO::Config &custom_cfg={})
	Construct the HdfBench model.
void	perform_step ()
	Iterate a single step.
void	monitor ()
	Monitor model information.
void	write_data ()
	Write the result times of each benchmark.
Public Member Functions inherited from Utopia::Model< HdfBenchModel, HdfBenchModelTypes >
	Model (const std::string &name, const ParentModel &parent_model, const Config &custom_cfg={}, std::tuple< WriterArgs... > w_args={}, const DataIO::Default::DefaultDecidermap< HdfBenchModel > &w_deciders=DataIO::Default::default_deciders< HdfBenchModel >, const DataIO::Default::DefaultTriggermap< HdfBenchModel > &w_triggers=DataIO::Default::default_triggers< HdfBenchModel >)
	Constructs a Model instance.
const std::shared_ptr< Space > &	get_space () const
	Return the space this model resides in.
Time	get_time () const
	Return the current time of this model.
Time	get_time_max () const
	Return the maximum time possible for this model.
Config	get_cfg () const
	Return the config node of this model.
std::string	get_name () const
	Return the name of this model instance.
std::string	get_full_name () const
	Return the full name of this model within the model hierarchy.
std::shared_ptr< DataGroup >	get_hdfgrp () const
	Return a pointer to the HDF group this model stores data in.
Time	get_write_start () const
	Return the parameter that controls when write_data is called first.
Time	get_write_every () const
	Return the parameter that controls how often write_data is called.
DataManager	get_datamanager () const
	return the datamanager
hsize_t	get_remaining_num_writes () const
	Return the number of remaining write_data calls this model will make.
std::shared_ptr< RNG >	get_rng () const
	Return a pointer to the shared RNG.
std::shared_ptr< spdlog::logger >	get_logger () const
	Return a pointer to the logger of this model.
Monitor	get_monitor () const
	Return the monitor of this model.
std::shared_ptr< MonitorManager >	get_monitor_manager () const
	Get the monitor manager of the root model.
Level	get_level () const
	Return the hierarchical level within the model hierarchy.
virtual void	prolog ()
	A function that is called before starting model iteration.
virtual void	epilog ()
	A function that is called after the last iteration of a model.
void	iterate ()
	Iterate one (time) step of this model.
void	run ()
	Run the model from the current time to the maximum time.
std::shared_ptr< DataSet >	create_dset (const std::string name, const std::shared_ptr< DataGroup > &hdfgrp, std::vector< hsize_t > add_write_shape, const std::size_t compression_level=1, const std::vector< hsize_t > chunksize={})
	Create a new dataset within the given group.
std::shared_ptr< DataSet >	create_dset (const std::string name, const std::vector< hsize_t > add_write_shape, const std::size_t compression_level=1, const std::vector< hsize_t > chunksize={})
	Create a new dataset within the model's base data group.
std::shared_ptr< DataSet >	create_cm_dset (const std::string name, const CellManager &cm, const std::size_t compression_level=1, const std::vector< hsize_t > chunksize={})
	Create a dataset storing data from a CellManager.
std::shared_ptr< DataSet >	create_am_dset (const std::string name, const AgentManager &am, const std::size_t compression_level=1, const std::vector< hsize_t > chunksize={})
	Create a dataset storing data from a AgentManager.

Protected Member Functions
template<bool setup = false>
double	benchmark (const std::string &bname)
	Carries out the benchmark associated with the given name.
double	time_since (const Time start)
	Returns the time (in seconds) since the given time point.
double	time_between (const Time start, const Time end)
	Returns the absolute time (in seconds) between the given time points.
Protected Member Functions inherited from Utopia::Model< HdfBenchModel, HdfBenchModelTypes >
void	__perform_step ()
	Perform the computation of a step.
void	__monitor ()
	Monitor information in the terminal.
void	__write_data ()
	Write data; calls the implementation's write_data method.
void	__write_initial_state ()
	Write the initial state.
void	increment_time (const Time dt=1)
	Increment time.
void	__prolog ()
	The default prolog of a model.
void	__epilog ()
	The default epilog of a model.
HdfBenchModel &	impl ()
	cast to the derived class
const HdfBenchModel &	impl () const
	const cast to the derived interface

Protected Attributes
BenchFunc	setup_nd
BenchFunc	setup_nd_with_chunks
BenchFunc	write_const
	Writes a constant value into the dataset.
Protected Attributes inherited from Utopia::Model< HdfBenchModel, HdfBenchModelTypes >
const std::string	_name
	Name of the model instance.
const std::string	_full_name
	The full name within the model hierarchy.
const Level	_level
	The level within the model hierarchy.
const Config	_cfg
	Config node belonging to this model instance.
const std::shared_ptr< RNG >	_rng
	The RNG shared between models.
const std::shared_ptr< spdlog::logger >	_log
	The (model) logger.
std::shared_ptr< Space >	_space
	The space this model resides in.
Time	_time
	Model-internal current time stamp.
const Time	_time_max
	Model-internal maximum time stamp.
const std::shared_ptr< DataGroup >	_hdfgrp
	The HDF group this model instance should write its data to.
const Time	_write_start
	First time at which write_data is called.
const Time	_write_every
	How often to call write_data from iterate.
Monitor	_monitor
	The monitor.
DataManager	_datamanager
	Manager object for handling data output; see DataManager.

Private Member Functions
std::map< std::string, Config >	load_benchmarks ()
	Load the benchmark configurations into a map.

Private Attributes
std::map< std::string, BenchFunc >	_setup_funcs
	A map of implemented setup functions for datasets.
std::map< std::string, BenchFunc >	_write_funcs
	A map of implemented write functions.
const std::vector< std::string >	_benchmarks
	Names of benchmarks.
const std::map< std::string, Config >	_bench_cfgs
	Configuration for the benchmarks.
std::map< std::string, double >	_times
	The results of the measurements, stored under the benchmark name.
std::shared_ptr< DataSet >	_dset_times
	Dataset to store the write times in.
std::map< std::string, std::shared_ptr< DataSet > >	_dsets
	Dataset to write test data to are stored in a map of dataset pointers.
const bool	_delete_afterwards
	Whether to delete datasets after the last step.
const std::chrono::duration< double >	_sleep_step
	Sleep time in seconds at the beginning of each step.
const std::chrono::duration< double >	_sleep_bench
	Sleep time in seconds before each benchmark.

Additional Inherited Members
Static Protected Attributes inherited from Utopia::Model< HdfBenchModel, HdfBenchModelTypes >
static constexpr WriteMode	_write_mode
	Which data-writing mode the base model should use.

Detailed Description

The HdfBench Model.

This model implements a benchmark of Utopia's Hdf5 writing capabilities.

It does not implement a manager or a grid but focusses on benchmarking the write times, given iterable data.

Member Typedef Documentation

Base

using Utopia::Models::HdfBench::HdfBenchModel::Base = Model<HdfBenchModel, HdfBenchModelTypes>

The base model type.

BenchFunc

using Utopia::Models::HdfBench::HdfBenchModel::BenchFunc = std::function<double(const std::string, Config)>

Type of a benchmark function pointer.

Clock

using Utopia::Models::HdfBench::HdfBenchModel::Clock = std::chrono::high_resolution_clock

Type of clock.

Config

using Utopia::Models::HdfBench::HdfBenchModel::Config = typename Base::Config

Data type that holds the configuration.

DataSet

using Utopia::Models::HdfBench::HdfBenchModel::DataSet = typename Base::DataSet

Data type for a dataset.

DurationType

using Utopia::Models::HdfBench::HdfBenchModel::DurationType = std::chrono::duration<double>

Type of the duration measure, should be a floating-point type.

Time

using Utopia::Models::HdfBench::HdfBenchModel::Time = std::chrono::high_resolution_clock::time_point

Type of a time point, retrieved from the clock.

Constructor & Destructor Documentation

HdfBenchModel()

template<class ParentModel >

Utopia::Models::HdfBench::HdfBenchModel::HdfBenchModel ( const std::string &  name, ParentModel &  parent_model, const DataIO::Config &  custom_cfg = {}  )

inline

Construct the HdfBench model.

This model aims to allow benchmarking of the Utopia Hdf5 library in a setting that is close to the actual use case, i.e.: as means for storing model output.

Parameters

name	Name of this model instance
parent_model	The parent model this model instance resides in
custom_cfg	A custom configuration to use instead of the one extracted from the parent model using the instance name

                                         {}
    )
    :
        // Initialize first via base model
        Base(name, parent_model, custom_cfg),
 
        // Set maps for setup and write functions
        _setup_funcs(),
        _write_funcs(),
 
        // Get the set of enabled benchmarks from the config
        _benchmarks(get_as<std::vector<std::string>>("benchmarks", this->_cfg)),
        _bench_cfgs(load_benchmarks()),
 
        // Create the temporary map for measured times and the times dataset
        _times(),
        _dset_times(this->create_dset("times", {_benchmarks.size()})),
        _dsets(),
 
        // Extract config parameters applicable to all benchmarks
        _delete_afterwards(get_as<bool>("delete_afterwards", this->_cfg)),
        _sleep_step(get_as<double>("sleep_step", this->_cfg)),
        _sleep_bench(get_as<double>("sleep_bench", this->_cfg))
    {
        // Check arguments
        if (_delete_afterwards) {
            throw std::invalid_argument("delete_afterwards feature is not yet "
                                        "implemented!");
        }
 
        // Set up the function mappings . . . . . . . . . . . . . . . . . . . .
        // FIXME Creating func maps should be possible in initializer list, but
        //       although it compiles, it leads to segfaults ...
 
        this->_log->debug("Associating setup functions ...");
        _setup_funcs["setup_nd"] = setup_nd;
        _setup_funcs["setup_nd_with_chunks"] = setup_nd_with_chunks;
 
 
        this->_log->debug("Associating write functions ...");
        _write_funcs["write_const"] = write_const;
 
 
        this->_log->debug("Associated {} setup and {} write function(s).",
                          _setup_funcs.size(), _write_funcs.size());
 
 
        // Carry out the setup benchmark  . . . . . . . . . . . . . . . . . . .
        const bool initial_write = get_as<bool>("initial_write", this->_cfg);
        this->_log->debug("initial_write: {},  sleep_step: {}s,  "
                          "sleep_bench: {}s", initial_write ? "yes" : "no",
                          _sleep_step.count(), _sleep_bench.count());
 
        this->_log->info("Performing setup and initial benchmarks ...");
 
        for (const auto &bname : _benchmarks) {
            // Setup the dataset and store the time needed
            _times[bname] = this->benchmark<true>(bname);
 
            // Perform one write operation, if configured to do so, and add
            // the time on top
            if (initial_write) {
                _times[bname] += this->benchmark(bname);
            }
        }
 
 
        // Add information to the dataset attributes
        _dset_times->add_attribute("dim_name__1", "benchmark");
        _dset_times->add_attribute("coords__benchmark", _benchmarks);
        _dset_times->add_attribute("initial_write", initial_write);
 
        // Done now.
        this->_log->debug("Finished constructing HdfBench '{}'.", this->_name);
    }

Member Function Documentation

benchmark()

template<bool setup = false>

double Utopia::Models::HdfBench::HdfBenchModel::benchmark ( const std::string &  bname )

inlineprotected

Carries out the benchmark associated with the given name.

                                             {
        // Get the benchmark configuration entry
        const auto bcfg = _bench_cfgs.at(bname);
 
        // Get the name of the setup/benchmark function, then resolve it
        BenchFunc bfunc;
        if constexpr (setup) {
            bfunc = _setup_funcs.at(get_as<std::string>("setup_func", bcfg));
        }
        else {
            bfunc = _write_funcs.at(get_as<std::string>("write_func", bcfg));
        }
 
        // Call the function; its return value is the time it took to execute
        const auto btime = bfunc(bname, bcfg);
 
        // Log the time, then return it
        this->_log->debug("Benchmark result {:>20s} {} : {:>10.3f} ms",
                          bname, setup ? "setup" : "write", btime * 1E3);
        return btime;
    }

load_benchmarks()

std::map< std::string, Config > Utopia::Models::HdfBench::HdfBenchModel::load_benchmarks ( )

inlineprivate

Load the benchmark configurations into a map.

                                                {
        this->_log->debug("Loading benchmark configurations ...");
        std::map<std::string, Config> cfg;
 
        for (const auto &bname : _benchmarks) {
            this->_log->trace("Loading benchmark configuration '{}' ...",
                              bname);
 
            try {
                cfg[bname] = get_as<Config>(bname, this->_cfg);
            }
            catch (std::exception &e) {
                std::cerr << "Could not find a benchmark configuration with "
                             "name '" << bname << "'! Make sure the given "
                             "configuration contains such an entry."
                          << std::endl << "Original error message:  ";
                throw;
            }
        }
 
        this->_log->debug("Got {} benchmark configurations.", cfg.size());
        return cfg;
    }

monitor()

void Utopia::Models::HdfBench::HdfBenchModel::monitor ( )

inline

Monitor model information.

Here, functions and values can be supplied to the monitor that are then available to the frontend. The monitor() function is only called if a certain emit interval has passed; thus, the performance hit is small.

    {
        // Can supply information to the monitor here in two ways:
        // this->_monitor.set_entry("key", value);
        // this->_monitor.set_entry("key", [this](){return 42.;});
    }

perform_step()

void Utopia::Models::HdfBench::HdfBenchModel::perform_step ( )

inline

Iterate a single step.

The "iteration" in this model is the step that creates the data that is written in the write_data method, i.e.: it carries out the benchmarks and stores the corresponding times in the _times member, to be written out in write_data

                         {
        // Sleep before the actual step is carried out
        std::this_thread::sleep_for(_sleep_step);
        // NOTE Duration might be zero, not triggering a sleep. Same below.
 
        // Carry out the benchmarks, optionally sleeping some time before that
        for (const auto &bname : _benchmarks) {
            std::this_thread::sleep_for(_sleep_bench);
 
            _times[bname] = this->benchmark(bname);
        }
    }

time_between()

double Utopia::Models::HdfBench::HdfBenchModel::time_between ( const Time  start, const Time  end  )

inlineprotected

Returns the absolute time (in seconds) between the given time points.

                                                          {
        const DurationType seconds = abs(end - start);
        return seconds.count();
    }

time_since()

double Utopia::Models::HdfBench::HdfBenchModel::time_since ( const Time  start )

inlineprotected

Returns the time (in seconds) since the given time point.

                                        {
        return time_between(start, Clock::now());
    }

write_data()

void Utopia::Models::HdfBench::HdfBenchModel::write_data ( )

inline

Write the result times of each benchmark.

                       {
        _dset_times->write(_benchmarks.begin(), _benchmarks.end(),
                           [this](const auto& bname) {
                                return this->_times.at(bname);
                            });
    }

Member Data Documentation

_bench_cfgs

const std::map<std::string, Config> Utopia::Models::HdfBench::HdfBenchModel::_bench_cfgs

private

Configuration for the benchmarks.

_benchmarks

const std::vector<std::string> Utopia::Models::HdfBench::HdfBenchModel::_benchmarks

private

Names of benchmarks.

_delete_afterwards

const bool Utopia::Models::HdfBench::HdfBenchModel::_delete_afterwards

private

Whether to delete datasets after the last step.

_dset_times

std::shared_ptr<DataSet> Utopia::Models::HdfBench::HdfBenchModel::_dset_times

private

Dataset to store the write times in.

_dsets

std::map<std::string, std::shared_ptr<DataSet> > Utopia::Models::HdfBench::HdfBenchModel::_dsets

private

Dataset to write test data to are stored in a map of dataset pointers.

_setup_funcs

std::map<std::string, BenchFunc> Utopia::Models::HdfBench::HdfBenchModel::_setup_funcs

private

A map of implemented setup functions for datasets.

_sleep_bench

const std::chrono::duration<double> Utopia::Models::HdfBench::HdfBenchModel::_sleep_bench

private

Sleep time in seconds before each benchmark.

_sleep_step

const std::chrono::duration<double> Utopia::Models::HdfBench::HdfBenchModel::_sleep_step

private

Sleep time in seconds at the beginning of each step.

_times

std::map<std::string, double> Utopia::Models::HdfBench::HdfBenchModel::_times

private

The results of the measurements, stored under the benchmark name.

_write_funcs

std::map<std::string, BenchFunc> Utopia::Models::HdfBench::HdfBenchModel::_write_funcs

private

A map of implemented write functions.

setup_nd

BenchFunc Utopia::Models::HdfBench::HdfBenchModel::setup_nd

protected

Initial value:

= [this](const auto& bname, auto cfg){
        
        auto shape = get_as<std::vector<hsize_t>>("write_shape", cfg);
        shape.insert(shape.begin(), this->get_time_max() + 1);
 
        const auto start = Clock::now();
        
 
        
        _dsets[bname] = this->_hdfgrp->open_dataset(bname);
        _dsets[bname]->set_capacity(shape);
 
        
        return time_since(start);
    }

                                                            {
        // Determine the shape of the final dataset
        auto shape = get_as<std::vector<hsize_t>>("write_shape", cfg);
        shape.insert(shape.begin(), this->get_time_max() + 1);
 
        const auto start = Clock::now();
        // -- benchmark start -- //
 
        // Create the dataset and set its capacity
        _dsets[bname] = this->_hdfgrp->open_dataset(bname);
        _dsets[bname]->set_capacity(shape);
 
        // --- benchmark end --- //
        return time_since(start);
    };

setup_nd_with_chunks

BenchFunc Utopia::Models::HdfBench::HdfBenchModel::setup_nd_with_chunks

protected

Initial value:

= [this](const auto& bname, auto cfg){
        
        const auto time_setup = this->setup_nd(bname, cfg);
 
        
        const auto chunks = get_as<std::vector<hsize_t>>("chunks", cfg);
 
        const auto start = Clock::now();
        
 
        
        _dsets[bname]->set_chunksize(chunks);
 
        
        return time_setup + time_since(start);
    }

                                                                        {
        // Call the regular setup_nd to set up the dataset
        const auto time_setup = this->setup_nd(bname, cfg);
 
        // Extract the chunks argument
        const auto chunks = get_as<std::vector<hsize_t>>("chunks", cfg);
 
        const auto start = Clock::now();
        // -- benchmark start -- //
 
        // Set the chunks value
        _dsets[bname]->set_chunksize(chunks);
 
        // --- benchmark end --- //
        return time_setup + time_since(start);
    };

write_const

BenchFunc Utopia::Models::HdfBench::HdfBenchModel::write_const

protected

Initial value:

= [this](const auto& bname, auto cfg){
        
        const auto val = get_as<double>("const_val", cfg);
 
        
        const auto shape = get_as<std::vector<std::size_t>>("write_shape", cfg);
        const auto it_len = std::accumulate(shape.begin(), shape.end(),
                                            1, std::multiplies<std::size_t>());
 
        const auto start = Clock::now();
        
 
        
        _dsets[bname]->write(boost::counting_iterator<std::size_t>(0),
                             boost::counting_iterator<std::size_t>(it_len),
                             [&val]([[maybe_unused]] auto &count){
                                return val;
                             });
 
        
        return time_since(start);
    }

Writes a constant value into the dataset.

                                                               {
        // Determine the value to write
        const auto val = get_as<double>("const_val", cfg);
 
        // Determine iterator length by factorizing the shape
        const auto shape = get_as<std::vector<std::size_t>>("write_shape", cfg);
        const auto it_len = std::accumulate(shape.begin(), shape.end(),
                                            1, std::multiplies<std::size_t>());
 
        const auto start = Clock::now();
        // -- benchmark start -- //
 
        // Can use the counting iterator as surrogate
        _dsets[bname]->write(boost::counting_iterator<std::size_t>(0),
                             boost::counting_iterator<std::size_t>(it_len),
                             [&val]([[maybe_unused]] auto &count){
                                return val;
                             });
 
        // --- benchmark end --- //
        return time_since(start);
    };

---

The documentation for this class was generated from the following file:

• src/utopia/models/HdfBench/HdfBench.hh