Low-level interface for using HDF5 with modern C++ and STL. More...

Collaboration diagram for HDF5:

Classes
class	Utopia::DataIO::HDFAttribute
	Class for hdf5 attribute, which can be attached to groups and datasets. More...

class	Utopia::DataIO::HDFBufferFactory
	Class which turns non-vector or plain-array containers into vectors. If the value_types are containers themselves, these are turned into vectors as well, because HDF5 cannot write something else. More...

class	Utopia::DataIO::HDFDataset
	Class representing a HDFDataset, wich reads and writes data and attributes. More...

class	Utopia::DataIO::HDFDataspace
	Class that wraps an HDF5 dataspace and takes care of managing its resources. More...

class	Utopia::DataIO::HDFFile
	Class representing a HDF5 file. More...

class	Utopia::DataIO::HDFGroup
	Class represting a HDFGroup, an object analogous to a folder for HDFFiles. More...

class	Utopia::DataIO::HDFIdentifier
	Wrapper class around an hdf5 identifier, used to manage reference counts of the object this identifier refers to. More...

class	Utopia::DataIO::HDFObject< objectcategory >
	Common base class for all HDF5 classes in the DATAIO Module i.e., for all classes that wrap HDF5-C-Library functionality like HDFDataset, HDFGroup, HDFFile. This class is not intendet as something to be used independently, but only as a base class to be inherited from. More...

class	Utopia::DataIO::HDFType
	Class which handles the conversion of C-types into hdf5types. More...

Typedefs
using	Utopia::DataIO::HDFDataspace::Base = HDFObject< HDFCategory::dataspace >

Enumerations
enum class	Utopia::DataIO::HDFCategory { Utopia::DataIO::file = H5I_FILE , Utopia::DataIO::group = H5I_GROUP , Utopia::DataIO::datatype = H5I_DATATYPE , Utopia::DataIO::dataspace = H5I_DATASPACE , Utopia::DataIO::dataset = H5I_DATASET , Utopia::DataIO::attribute = H5I_ATTR }
	Enumerate the different HDF5 object types for use in HDFObject class. More...

Functions
void	Utopia::DataIO::swap (HDFAttribute &lhs, HDFAttribute &rhs)
	Swaps the states of Attributes rhs and lhs. More...

void	Utopia::DataIO::swap (HDFDataset &lhs, HDFDataset &rhs)
	Exchange state between lhs and rhs. More...

hsize_t	Utopia::DataIO::HDFDataspace::rank ()
	Get thet dataspace's rank, i.e., number of dimensions. More...

std::pair< arma::Row< hsize_t >, arma::Row< hsize_t > >	Utopia::DataIO::HDFDataspace::get_properties ()
	Get the properties object: size and capacity. @notice The dimensions can be infered from the size of the returned vectors. More...

arma::Row< hsize_t >	Utopia::DataIO::HDFDataspace::size ()
	Get the current size of the dataspace in each dimension. More...

arma::Row< hsize_t >	Utopia::DataIO::HDFDataspace::capacity ()
	Get the capacity of the dataspace in each dimension. More...

void	Utopia::DataIO::HDFDataspace::open ()
	Open the dataspace - set it to be equivalent to any data that later will be used to write or read. More...

template<typename Object >
void	Utopia::DataIO::HDFDataspace::open (Object &&object)
	Open the dataspace with an HDF5 object, i.e., dataset or attribute. More...

void	Utopia::DataIO::HDFDataspace::open (std::string name, hsize_t rank, arma::Row< hsize_t > extent, arma::Row< hsize_t > capacity)
	Open a new dataset of type 'simple', which is equivalent to a N-dimensional array of dimension N = 'rank', a given extent, and a given maximum capacity that in each dimension must be greater or equal to the extent. More...

std::pair< arma::Row< hsize_t >, arma::Row< hsize_t > >	Utopia::DataIO::HDFDataspace::get_selection_bounds ()
	Get the selection bounding box, i.e., the start and end vector of the currently selected subset of the dataspace. More...

void	Utopia::DataIO::HDFDataspace::select_slice (arma::Row< hsize_t > start, arma::Row< hsize_t > end, arma::Row< hsize_t > stride)
	Select a slice in the dataspace defined by [start, end, stride] in the manner of numpy. Overwrites old selections. More...

void	Utopia::DataIO::HDFDataspace::select_all ()
	Select the entire dataspace as hyperslap to be read/written to. More...

void	Utopia::DataIO::HDFDataspace::resize (arma::Row< hsize_t > new_size)
	Resize the dataspace. The new size needs to fit into the dataspaces capacity. More...

void	Utopia::DataIO::HDFDataspace::release_selection ()
	Release a previously defined selection. More...

	Utopia::DataIO::HDFDataspace::HDFDataspace ()
	Construct HDFDataspace per default. Equivalent to using H5S_ALL when employing the pure C interface. More...

	Utopia::DataIO::HDFDataspace::HDFDataspace (const HDFDataspace &)=default
	Copy constructor. More...

	Utopia::DataIO::HDFDataspace::HDFDataspace (HDFDataspace &&)=default
	Move constructor. More...

HDFDataspace &	Utopia::DataIO::HDFDataspace::operator= (const HDFDataspace &)=default
	Copy assign dataspace. More...

HDFDataspace &	Utopia::DataIO::HDFDataspace::operator= (HDFDataspace &&)=default
	Move assign dataspace. More...

	Utopia::DataIO::HDFDataspace::HDFDataspace (std::string name, hsize_t rank, std::vector< hsize_t > extent, std::vector< hsize_t > capacity)
	Construct HDFDataspace from the given arguments. More...

template<typename Object >
	Utopia::DataIO::HDFDataspace::HDFDataspace (Object &&object, std::enable_if_t< not std::is_same_v< std::decay_t< Object >, HDFDataspace >, int >=0)
	Construct a new HDFDataspace object from an HDFDataset or HDFAttribute. This loads the file-dataspace that belongs to the dataset or attribute such that it gets managed by this class. More...

virtual	Utopia::DataIO::HDFDataspace::~HDFDataspace ()=default
	Destroy the HDFDataspace object. More...

void	Utopia::DataIO::HDFDataspace::swap (HDFDataspace &other)
	Swap state with argument. More...

void	Utopia::DataIO::swap (HDFDataspace &lhs, HDFDataspace &rhs)
	Swap states of lhs and rhs. More...

void	Utopia::DataIO::swap (HDFGroup &lhs, HDFGroup &rhs)
	Swap lhs and rhs. More...

bool	Utopia::DataIO::operator== (const HDFIdentifier &lhs, const HDFIdentifier &rhs)
	Comparsion operator for equality. More...

bool	Utopia::DataIO::operator!= (const HDFIdentifier &lhs, const HDFIdentifier &rhs)
	Comparsion operator for inequality. More...

void	Utopia::DataIO::swap (HDFIdentifier &lhs, HDFIdentifier &rhs)
	Exchange the states of lhs and rhs. More...

template<HDFCategory cat>
void	Utopia::DataIO::swap (HDFObject< cat > &lhs, HDFObject< cat > &rhs)
	Exchange state of lhs and rhs. More...

bool	Utopia::DataIO::operator== (const HDFType &lhs, const HDFType &rhs)
	Check equality of argument typefactories. Two typefactories are considered equal when they refer to the same HDF5 type, e.g., H5T_INTEGER, or H5T_VLEN. More...

bool	Utopia::DataIO::operator!= (const HDFType &lhs, const HDFType &rhs)
	TODO. More...

bool	Utopia::DataIO::check_validity (htri_t valid, const std::string_view object_name)
	Check for validity of a hdf5 htri_t type or similar. More...

htri_t	Utopia::DataIO::path_exists (hid_t loc_id, std::string path, hid_t link_property_list=H5P_DEFAULT)
	Checks iteratively if each segment of a path exists. More...

auto	Utopia::DataIO::path_is_valid (hid_t id, std::string path)
	Check if the path given relative to the object identified by 'id' exists and points to a valid hdf5 object. More...

template<HDFCategory category>
std::string	Utopia::DataIO::category_to_string ()
	Turn category enum into a string that names it. More...

template<class Object >
std::string	Utopia::DataIO::generate_object_name (const Object &object)
	Use category and path variable of object to make a string that identifies the object it is applied to. More...

template<typename Object >
hid_t	Utopia::DataIO::open_dataspace (Object &&object)
	Depending on object category, opens a dataset or attribute dataspace. More...

template<typename Object >
hid_t	Utopia::DataIO::open_type (Object &&object)
	Depending on object category, invokes H5Dget_type or H5Aget_type. More...

Detailed Description

Low-level interface for using HDF5 with modern C++ and STL.

HDF5 Backend Module

Overview

This backend is a replacement of the HDF5 C++ wrappers. It does not implement the full HDF5 standard, but only the features we deemed useful and needed for the Utopia project. It was created because the C++ wrappers supplied by the HDF5 group do not support STL containers and in general no modern C++ features. Furthermore, development of the pure C implementation is much faster and it is generally more complete.

Implementation

In this module, C++ classes are created which represent HDF5 files, groups, datasets and attributes, with the associated object creation- and data I/O- capabilities, limited to one and two-dimensional datasets of arrays, containers or scalars. Additionally, helper classes for organizing type mapping and type conversion are supplied, which normally are irelevant for users. Finally a number of helper functions are supplied which are used to assert correctness.

Typedef Documentation

◆ Base

using Utopia::DataIO::HDFDataspace::Base = HDFObject< HDFCategory::dataspace >

Enumeration Type Documentation

◆ HDFCategory

enum Utopia::DataIO::HDFCategory

strong

Enumerate the different HDF5 object types for use in HDFObject class.

Enumerator
file
group
datatype
dataspace
dataset
attribute

Function Documentation

◆ capacity()

arma::Row< hsize_t > Utopia::DataIO::HDFDataspace::capacity ( )

inline

Get the capacity of the dataspace in each dimension.

Returns: arma::Row< hsize_t > Vector containing the dataspace's current capacity in each dimension

◆ category_to_string()

template<HDFCategory category>

std::string Utopia::DataIO::category_to_string ( )

Turn category enum into a string that names it.

Template Parameters

◆ check_validity()

bool Utopia::DataIO::check_validity	(	htri_t	valid,
		const std::string_view	object_name
	)

Check for validity of a hdf5 htri_t type or similar.

Parameters

[in]	valid	parameter to check
[in]	object	name Name of object to be referenced in thrown exceptions if valid <= 0

Returns: The valid argument for further use

This function is necessary because for instance H5Iis_valid does not return a boolean (non existant in C), but a value which is > 0 if everything is fine, < 0 if some error occurred during checking and 0 if the object to check is invalid. This has to be taken into account in order to be able to track bugs or wrong usage properly. See here:

https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.6/hdf5-1.6.7/src/unpacked/src/H5public.h which yields the following snippet:

Boolean type. Successful return values are zero (false) or positive (true). The typical true value is 1 but don't bet on it. Boolean functions cannot fail. Functions that return ‘htri_t’ however return zero (false), positive (true), or negative (failure). The proper way to test for truth from a htri_t function is:

if ((retval = H5Tcommitted(type))>0) { printf("data type is committed\n"); } else if (!retval) { printf("data type is not committed\n"); } else { printf("error determining whether data type is committed\n"); }

◆ generate_object_name()

template<class Object >

std::string Utopia::DataIO::generate_object_name ( const Object & object )

Use category and path variable of object to make a string that identifies the object it is applied to.

Template Parameters

Object automatically determined

Parameters

object object to determine info string for

Returns: std::string category and path in the form "category `category` at `path`".

◆ get_properties()

std::pair< arma::Row< hsize_t >, arma::Row< hsize_t > > Utopia::DataIO::HDFDataspace::get_properties ( )

inline

Get the properties object: size and capacity. @notice The dimensions can be infered from the size of the returned vectors.

Returns: auto pair containing (size, capacity) armadillo rowvectors

◆ get_selection_bounds()

std::pair< arma::Row< hsize_t >, arma::Row< hsize_t > > Utopia::DataIO::HDFDataspace::get_selection_bounds ( )

inline

Get the selection bounding box, i.e., the start and end vector of the currently selected subset of the dataspace.

Returns: std::pair<arma::Row<hsize_t>, arma::Row<hsize_t>>

◆ HDFDataspace() [1/5]

Utopia::DataIO::HDFDataspace::HDFDataspace ( )

inline

Construct HDFDataspace per default. Equivalent to using H5S_ALL when employing the pure C interface.

◆ HDFDataspace() [2/5]

Utopia::DataIO::HDFDataspace::HDFDataspace ( const HDFDataspace & )

default

Copy constructor.

◆ HDFDataspace() [3/5]

Utopia::DataIO::HDFDataspace::HDFDataspace ( HDFDataspace && )

default

Move constructor.

◆ HDFDataspace() [4/5]

template<typename Object >

Utopia::DataIO::HDFDataspace::HDFDataspace	(	Object &&	object,
		std::enable_if_t< not std::is_same_v< std::decay_t< Object >, HDFDataspace >, int >	= `0`
	)

inline

Construct a new HDFDataspace object from an HDFDataset or HDFAttribute. This loads the file-dataspace that belongs to the dataset or attribute such that it gets managed by this class.

Template Parameters

Object automatically dete rmined

Parameters

object object to open the dataset space to

◆ HDFDataspace() [5/5]

Utopia::DataIO::HDFDataspace::HDFDataspace	(	std::string	name,
		hsize_t	rank,
		std::vector< hsize_t >	extent,
		std::vector< hsize_t >	capacity
	)

inline

Construct HDFDataspace from the given arguments.

Parameters

rank
extent
capacity

◆ open() [1/3]

void Utopia::DataIO::HDFDataspace::open ( )

inline

Open the dataspace - set it to be equivalent to any data that later will be used to write or read.

◆ open() [2/3]

template<typename Object >

void Utopia::DataIO::HDFDataspace::open ( Object && object )

inline

Open the dataspace with an HDF5 object, i.e., dataset or attribute.

Template Parameters

Object

Parameters

o	HDF5 object to retrieve the dataspace for

◆ open() [3/3]

void Utopia::DataIO::HDFDataspace::open	(	std::string	name,
		hsize_t	rank,
		arma::Row< hsize_t >	extent,
		arma::Row< hsize_t >	capacity
	)

inline

Open a new dataset of type 'simple', which is equivalent to a N-dimensional array of dimension N = 'rank', a given extent, and a given maximum capacity that in each dimension must be greater or equal to the extent.

Parameters

rank	Dimension of the dataspace
extent	Current extent of the dataspace
capacity	Total capacity of the dataspace

◆ open_dataspace()

template<typename Object >

hid_t Utopia::DataIO::open_dataspace ( Object && object )

Depending on object category, opens a dataset or attribute dataspace.

Template Parameters

Object

Parameters

object

Returns: hid_t

◆ open_type()

template<typename Object >

hid_t Utopia::DataIO::open_type ( Object && object )

Depending on object category, invokes H5Dget_type or H5Aget_type.

Template Parameters

Object

Parameters

object

Returns: hid_t

◆ operator!=() [1/2]

bool Utopia::DataIO::operator!=	(	const HDFIdentifier &	lhs,
		const HDFIdentifier &	rhs
	)

Comparsion operator for inequality.

Parameters

other Object to compare with

Returns: true other != this; false otherwise

◆ operator!=() [2/2]

bool Utopia::DataIO::operator!=	(	const HDFType &	lhs,
		const HDFType &	rhs
	)

TODO.

Parameters

lhs
rhs

Returns: true; false

◆ operator=() [1/2]

HDFDataspace& Utopia::DataIO::HDFDataspace::operator= ( const HDFDataspace & )

default

Copy assign dataspace.

Returns: HDFDataspace&

◆ operator=() [2/2]

HDFDataspace& Utopia::DataIO::HDFDataspace::operator= ( HDFDataspace && )

default

Move assign dataspace.

Returns: HDFDataspace&

◆ operator==() [1/2]

bool Utopia::DataIO::operator==	(	const HDFIdentifier &	lhs,
		const HDFIdentifier &	rhs
	)

Comparsion operator for equality.

Parameters

other Object to compare with

Returns: true other == this; false otherwise

◆ operator==() [2/2]

bool Utopia::DataIO::operator==	(	const HDFType &	lhs,
		const HDFType &	rhs
	)

Check equality of argument typefactories. Two typefactories are considered equal when they refer to the same HDF5 type, e.g., H5T_INTEGER, or H5T_VLEN.

Template Parameters

T
U

Parameters

lhs
rhs

Returns: true; false

◆ path_exists()

htri_t Utopia::DataIO::path_exists	(	hid_t	loc_id,
		std::string	path,
		hid_t	link_property_list = `H5P_DEFAULT`
	)

Checks iteratively if each segment of a path exists.

Checks if a given path exists in a hdf5 object identitifed by its id.

Parameters

loc_id	Identifier of the file or group to query.
path	The path of the link to check. This can be a relative or an absolute path, but (as with H5Lexists) it can NOT use the `../` syntax to go to the parent object. For such cases, an absolute path needs to be given.
link_property_list	Link access property list identifier.

Returns: htri_t Variable which tells if the given path exists (true > 0, false = 0, error < 0)

◆ path_is_valid()

auto Utopia::DataIO::path_is_valid	(	hid_t	id,
		std::string	path
	)

Check if the path given relative to the object identified by 'id' exists and points to a valid hdf5 object.

Parameters

id	id of the starting point
path	path from the object identified by 'id' to the object to check

Returns: auto > 0 if valid and existing, 0 if not

◆ rank()

hsize_t Utopia::DataIO::HDFDataspace::rank ( )

inline

Get thet dataspace's rank, i.e., number of dimensions.

Returns: auto rank of the dataspace

◆ release_selection()

void Utopia::DataIO::HDFDataspace::release_selection ( )

inline

Release a previously defined selection.

◆ resize()

void Utopia::DataIO::HDFDataspace::resize ( arma::Row< hsize_t > new_size )

inline

Resize the dataspace. The new size needs to fit into the dataspaces capacity.

Note: Dataset needs to be chunked when new_size != capacity of the dataspace

Parameters

new_size Desired new size of the dataspace

◆ select_all()

void Utopia::DataIO::HDFDataspace::select_all ( )

inline

Select the entire dataspace as hyperslap to be read/written to.

◆ select_slice()

void Utopia::DataIO::HDFDataspace::select_slice	(	arma::Row< hsize_t >	start,
		arma::Row< hsize_t >	end,
		arma::Row< hsize_t >	stride
	)

inline

Select a slice in the dataspace defined by [start, end, stride] in the manner of numpy. Overwrites old selections.

Parameters

start
end
stride

◆ size()

arma::Row< hsize_t > Utopia::DataIO::HDFDataspace::size ( )

inline

Get the current size of the dataspace in each dimension.

Returns: arma::Row< hsize_t > Vector containing the dataspace's current size in each dimension

◆ swap() [1/7]

void Utopia::DataIO::swap	(	HDFAttribute &	lhs,
		HDFAttribute &	rhs
	)

Swaps the states of Attributes rhs and lhs.

Template Parameters

ParentObject1
ParentObject2

Parameters

lhs	First Attribute to swap
rhs	Second Attribute to swap

◆ swap() [2/7]

void Utopia::DataIO::swap	(	HDFDataset &	lhs,
		HDFDataset &	rhs
	)

Exchange state between lhs and rhs.

Parameters

lhs	The left hand side
rhs	The right hand side

Template Parameters

HDFObject The type of the parent object

◆ swap() [3/7]

void Utopia::DataIO::swap	(	HDFDataspace &	lhs,
		HDFDataspace &	rhs
	)

Swap states of lhs and rhs.

Parameters

lhs	Dataspace to swap states
rhs	Dataspace to swap states

◆ swap() [4/7]

void Utopia::DataIO::HDFDataspace::swap ( HDFDataspace & other )

inline

Swap state with argument.

Parameters

other Other HDFDataspace instance

◆ swap() [5/7]

void Utopia::DataIO::swap	(	HDFGroup &	lhs,
		HDFGroup &	rhs
	)

Swap lhs and rhs.

Parameters

lhs	The left hand side
rhs	The right hand side

◆ swap() [6/7]

void Utopia::DataIO::swap	(	HDFIdentifier &	lhs,
		HDFIdentifier &	rhs
	)

Exchange the states of lhs and rhs.

Parameters

lhs
rhs

◆ swap() [7/7]

template<HDFCategory cat>

void Utopia::DataIO::swap	(	HDFObject< cat > &	lhs,
		HDFObject< cat > &	rhs
	)

Exchange state of lhs and rhs.

Template Parameters

cat	automatically determined

Parameters

lhs
rhs

◆ ~HDFDataspace()

virtual Utopia::DataIO::HDFDataspace::~HDFDataspace ( )

virtualdefault

Destroy the HDFDataspace object.

Classes

Typedefs

Enumerations

Functions

Detailed Description

HDF5 Backend Module

Overview

Implementation

Typedef Documentation

◆ Base

Enumeration Type Documentation

◆ HDFCategory

Function Documentation

◆ capacity()

◆ category_to_string()

◆ check_validity()

◆ generate_object_name()

◆ get_properties()

◆ get_selection_bounds()

◆ HDFDataspace() [1/5]

◆ HDFDataspace() [2/5]

◆ HDFDataspace() [3/5]

◆ HDFDataspace() [4/5]

◆ HDFDataspace() [5/5]

◆ open() [1/3]

◆ open() [2/3]

◆ open() [3/3]

◆ open_dataspace()

◆ open_type()

◆ operator!=() [1/2]

◆ operator!=() [2/2]

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ operator==() [1/2]

◆ operator==() [2/2]

◆ path_exists()

◆ path_is_valid()

◆ rank()

◆ release_selection()

◆ resize()

◆ select_all()

◆ select_slice()

◆ size()

◆ swap() [1/7]

◆ swap() [2/7]

◆ swap() [3/7]

◆ swap() [4/7]

◆ swap() [5/7]

◆ swap() [6/7]

◆ swap() [7/7]

◆ ~HDFDataspace()