utopya.model_registry package¶

This submodule implements a registry of Utopia models, which is used to provide the required model information to the frontend and use it throughout.

Submodules¶

utopya.model_registry._exceptions module¶

All model registry related error types

exception utopya.model_registry._exceptions.ModelRegistryError[source]¶

Bases: ValueError

Raised on errors with model registry

__module__ = 'utopya.model_registry._exceptions'¶

__weakref__¶: list of weak references to the object (if defined)

exception utopya.model_registry._exceptions.BundleExistsError[source]¶

Bases: utopya.model_registry._exceptions.ModelRegistryError

Raised when a bundle that compared equal already exists.

__module__ = 'utopya.model_registry._exceptions'¶

utopya.model_registry.entry module¶

This module implements the ModelRegistryEntry class, which contains a set of ModelInfoBundle objects. These can be combined in a labelled and unlabelled fashion.

class utopya.model_registry.entry.ModelRegistryEntry(model_name: str, *, registry_dir: str)[source]¶

Bases: object

Holds model config bundles for a single model.

Instances of this class are associated with a file in the model registry directory. Upon change, they update that file.

Initialize a model registry entry from a registry directory.

Expects a file named <model_name>.yml in the registry directory. If it does not exist, creates an empty one.

__init__(model_name: str, *, registry_dir: str)[source]¶

Initialize a model registry entry from a registry directory.

Expects a file named <model_name>.yml in the registry directory. If it does not exist, creates an empty one.

property model_name: str¶

property registry_dir: str¶

property registry_file_path: str¶

__len__() → int[source]¶: Returns total number of bundles, labelled and unlabelled.

__contains__(obj) → bool[source]¶

Checks if the given _object_ is part of this registry entry.

Note that this does not check for keys!

__str__() → str[source]¶: Return str(self).

__eq__(other) → bool[source]¶: Check for equality by inspecting stored bundles and model name

__getitem__(key: Union[str, int]) → utopya.model_registry.info_bundle.ModelInfoBundle[source]¶: Return a bundle for the given label or index. If None, returns the tries to return the item.

item() → utopya.model_registry.info_bundle.ModelInfoBundle[source]¶: Retrieve the single bundle for this model, if not ambiguous.

keys() → Iterator[Union[str, int]][source]¶: Returns legitimate keys for item access, i.e.: all registered keys. Starts with labelled bundle names, continues with indices of unlabelled bundles.

values() → Generator[utopya.model_registry.info_bundle.ModelInfoBundle, None, None][source]¶: Returns stored model config bundles, starting with labelled ones

items() → Generator[Tuple[Union[str, int], utopya.model_registry.info_bundle.ModelInfoBundle], None, None][source]¶: Returns keys and registered bundles, starting with labelled ones

add_bundle(*, label: Optional[str] = None, overwrite_label: bool = False, update_registry_file: bool = True, **bundle_kwargs) → utopya.model_registry.info_bundle.ModelInfoBundle[source]¶

Add a new configuration bundle to this registry entry.

This makes sure that the added bundle does not compare equal to an already existing bundle.

Parameters

label (str, optional) – The label under which to add it. If None, will be added as unlabelled.
overwrite_label (bool, optional) – If True, overwrites an existing bundle under the same label. This option does not affect unlabelled bundles or registry file updates.
update_registry_file (bool, optional) – Whether to write changes directly to the registry file.
**bundle_kwargs – Passed on to construct the ModelInfoBundle that is to be stored.

Raises

ModelRegistryError – If exist_ok is False and such a bundle already exists in this registry entry.

pop(key: Union[str, int], *, update_registry_file: bool = True) → utopya.model_registry.info_bundle.ModelInfoBundle[source]¶: Pop a configuration bundle from this entry.

clear(*, update_registry_file: bool = True)[source]¶: Removes all configuration bundles from this entry.

classmethod to_yaml(representer, node)[source]¶

Creates a YAML representation of the data stored in this entry. As the data is a combination of a dict and a sequence, instances of this class are also represented as such.

Parameters

representer (ruamel.yaml.representer) – The representer module
node (type(self)) – The node to represent, i.e. an instance of this class.

Returns

A YAML representation of the given instance of this class.

__dict__ = mappingproxy({'__module__': 'utopya.model_registry.entry', '__doc__': 'Holds model config bundles for a single model.\n\n Instances of this class are associated with a file in the model registry\n directory. Upon change, they update that file.\n ', '__init__': <function ModelRegistryEntry.__init__>, 'model_name': <property object>, 'registry_dir': <property object>, 'registry_file_path': <property object>, '__len__': <function ModelRegistryEntry.__len__>, '__contains__': <function ModelRegistryEntry.__contains__>, '__str__': <function ModelRegistryEntry.__str__>, '__eq__': <function ModelRegistryEntry.__eq__>, '__getitem__': <function ModelRegistryEntry.__getitem__>, 'item': <function ModelRegistryEntry.item>, 'keys': <function ModelRegistryEntry.keys>, 'values': <function ModelRegistryEntry.values>, 'items': <function ModelRegistryEntry.items>, 'add_bundle': <function ModelRegistryEntry.add_bundle>, 'pop': <function ModelRegistryEntry.pop>, 'clear': <function ModelRegistryEntry.clear>, '_load_from_registry': <function ModelRegistryEntry._load_from_registry>, '_update_registry_file': <function ModelRegistryEntry._update_registry_file>, '_as_dict': <classmethod object>, 'to_yaml': <classmethod object>, '__dict__': <attribute '__dict__' of 'ModelRegistryEntry' objects>, '__weakref__': <attribute '__weakref__' of 'ModelRegistryEntry' objects>, '__hash__': None, '__annotations__': {}})¶

__hash__ = None¶

__module__ = 'utopya.model_registry.entry'¶

__weakref__¶: list of weak references to the object (if defined)

utopya.model_registry.info_bundle module¶

Implements the ModelInfoBundle class, which holds a single bundle of information about a model.

class utopya.model_registry.info_bundle.ModelInfoBundle(*, model_name: str, paths: dict, metadata: Optional[dict] = None, project_name: Optional[str] = None, registration_time: Optional[str] = None, missing_path_action: str = 'log', **additional_kwargs)[source]¶

Bases: object

A bundle of model information; behaves like a read-only dict

Initialize a ModelInfoBundle

PATH_KEYS = (('binary', <class 'str'>, True), ('source_dir', <class 'str'>), ('default_cfg', <class 'str'>, True), ('default_plots', <class 'str'>), ('base_plots', <class 'str'>), ('python_model_tests_dir', <class 'str'>), ('python_model_plots_dir', <class 'str'>))¶

PATH_KEYS_REL_TO_SRC = ('default_cfg', 'default_plots', 'base_plots')¶

SRC_DIR_SEARCH_PATHS = (('default_cfg', '{}_cfg.yml'), ('default_plots', '{}_plots.yml'), ('base_plots', '{}_base_plots.yml'))¶

METADATA_KEYS = (('version', <class 'str'>), ('long_name', <class 'str'>), ('description', <class 'str'>), ('author', <class 'str'>), ('email', <class 'str'>), ('website', <class 'str'>), ('dependencies', <class 'list'>), ('utopya_compatibility', <class 'str'>), ('misc', <class 'dict'>))¶

__init__(*, model_name: str, paths: dict, metadata: Optional[dict] = None, project_name: Optional[str] = None, registration_time: Optional[str] = None, missing_path_action: str = 'log', **additional_kwargs)[source]¶: Initialize a ModelInfoBundle

__eq__(other) → bool[source]¶: Compares equality by comparing the stored configuration. Only if another ModelInfoBundle is compared against does the model name also take part in the comparison.

__str__() → str[source]¶: Return str(self).

property model_name: str¶

property registration_time: str¶: Registration time string of this bundle

property as_dict: dict¶: Returns a deep copy of all bundle data. This does NOT include the model name and the registration time.

__getitem__(key: str)[source]¶: Direct access to the full bundle data

property paths: dict¶: Access to the paths information of the bundle

property metadata: dict¶: Access to the metadata information of the bundle

property project_name: str¶: Access to the Utopia project name information of the bundle

property missing_paths: dict¶: Returns those paths where os.path.exists did not evaluate to True

classmethod to_yaml(representer, node)[source]¶

Creates a YAML representation of the data stored in this bundle.

Parameters

representer (ruamel.yaml.representer) – The representer module
node (type(self)) – The node to represent, i.e. an instance of this class.

Returns

A YAML representation of the given instance of this class.

__dict__ = mappingproxy({'__module__': 'utopya.model_registry.info_bundle', '__doc__': 'A bundle of model information; behaves like a read-only dict', 'PATH_KEYS': (('binary', <class 'str'>, True), ('source_dir', <class 'str'>), ('default_cfg', <class 'str'>, True), ('default_plots', <class 'str'>), ('base_plots', <class 'str'>), ('python_model_tests_dir', <class 'str'>), ('python_model_plots_dir', <class 'str'>)), 'PATH_KEYS_REL_TO_SRC': ('default_cfg', 'default_plots', 'base_plots'), 'SRC_DIR_SEARCH_PATHS': (('default_cfg', '{}_cfg.yml'), ('default_plots', '{}_plots.yml'), ('base_plots', '{}_base_plots.yml')), 'METADATA_KEYS': (('version', <class 'str'>), ('long_name', <class 'str'>), ('description', <class 'str'>), ('author', <class 'str'>), ('email', <class 'str'>), ('website', <class 'str'>), ('dependencies', <class 'list'>), ('utopya_compatibility', <class 'str'>), ('misc', <class 'dict'>)), '__init__': <function ModelInfoBundle.__init__>, '__eq__': <function ModelInfoBundle.__eq__>, '__str__': <function ModelInfoBundle.__str__>, 'model_name': <property object>, 'registration_time': <property object>, 'as_dict': <property object>, '__getitem__': <function ModelInfoBundle.__getitem__>, 'paths': <property object>, 'metadata': <property object>, 'project_name': <property object>, 'missing_paths': <property object>, '_parse_paths': <function ModelInfoBundle._parse_paths>, 'to_yaml': <classmethod object>, '__dict__': <attribute '__dict__' of 'ModelInfoBundle' objects>, '__weakref__': <attribute '__weakref__' of 'ModelInfoBundle' objects>, '__hash__': None, '__annotations__': {}})¶

__hash__ = None¶

__module__ = 'utopya.model_registry.info_bundle'¶

__weakref__¶: list of weak references to the object (if defined)

utopya.model_registry.registry module¶

This module implements the ModelRegistry, which combines ModelRegistryEntry objects and makes it possible to register new models.

class utopya.model_registry.registry.KeyOrderedDict(*args, key: Optional[Callable] = None, **kwds)[source]¶

Bases: dantro.utils.ordereddict.KeyOrderedDict

A key-ordered dict that expects string keys and sorts by the lower-case representation of keys.

Initialize a KeyOrderedDict, which maintains key ordering. If no custom ordering function is given, orders by simple “smaller than” comparison.

Apart from that, the interface is the same as for regular dictionaries.

Parameters

*args – a single sequence of (key, value) pairs to insert
key (Callable, optional) – The callable used to
**kwds – Passed on to update method

Raises

TypeError – on len(args) > 1

DEFAULT_KEY_COMPARATOR(k)¶

__module__ = 'utopya.model_registry.registry'¶

class utopya.model_registry.registry.ModelRegistry(utopia_cfg_dir_path: str = '/root/.config/utopia')[source]¶

Bases: object

The ModelRegistry class takes care of providing model information to the rest of the utopya package and, at the same time, maintains the model registry it is associated with.

It provides a dict-like interface to access the stored registry entries under their model name. Via reigster_model_info, a model can be registered and information can be added to it.

Additionally, there are some functions that provide an overview over the registered models and with which information they

# TODO Consider making this class a singleton?

Loads the utopia model registry from the configuration at the given path.

Parameters: utopia_cfg_dir_path (str, optional) – The path to store the model registry folder in.

__init__(utopia_cfg_dir_path: str = '/root/.config/utopia')[source]¶

Loads the utopia model registry from the configuration at the given path.

Parameters: utopia_cfg_dir_path (str, optional) – The path to store the model registry folder in.

property registry_dir: str¶: The model registry directory path

__len__() → int[source]¶

__str__() → str[source]¶: Return str(self).

property info_str: str¶

property info_str_detailed: str¶

keys()[source]¶

values()[source]¶

items()[source]¶

__contains__(model_name: str) → bool[source]¶: Whether an entry for the given model name exists in the registry

__getitem__(model_name: str) → utopya.model_registry.entry.ModelRegistryEntry[source]¶: Retrieve a deep copy of a model registration entry for the given model name.

register_model_info(model_name: str, *, exists_action: Optional[str] = None, **bundle_kwargs) → utopya.model_registry.entry.ModelRegistryEntry[source]¶

Register information for a single model. This method also allows to create a new entry if a model does not exist.

However, it will raise an error if the model was already registered and neither the skip nor the remove options were explicitly specified.

Parameters

model_name (str) – The name of the model to register
exists_action (str, optional) –
The action to take when a model of the given name already exists. Possible values:
- None: Continue with registering the info bundle; note
  that this might still lead to errors if the exact same bundle already exists.
- skip: Skip bundle registration
- raise: Raise an error
- clear: Clear all existing bundles
  from the corresponding model registry entry.
- validate: Makes sure the bundle that will be created
  from the given kwargs is part of the registry entry.
**bundle_kwargs – Passed on to ModelRegistryEntry.add_bundle

Returns

The registry entry for this model.

Return type

ModelRegistryEntry

Raises

ValueError – On exists_action == 'raise' and model already existing.

remove_entry(model_name: str)[source]¶: Removes a registry entry and deletes the associated registry file

__dict__ = mappingproxy({'__module__': 'utopya.model_registry.registry', '__doc__': 'The ModelRegistry class takes care of providing model information to the\n rest of the utopya package and, at the same time, maintains the model\n registry it is associated with.\n\n It provides a dict-like interface to access the stored registry entries\n under their model name. Via ``reigster_model_info``, a model can be\n registered and information can be added to it.\n\n Additionally, there are some functions that provide an overview over the\n registered models and with which information they\n\n # TODO Consider making this class a singleton?\n ', '__init__': <function ModelRegistry.__init__>, 'registry_dir': <property object>, '__len__': <function ModelRegistry.__len__>, '__str__': <function ModelRegistry.__str__>, 'info_str': <property object>, 'info_str_detailed': <property object>, 'keys': <function ModelRegistry.keys>, 'values': <function ModelRegistry.values>, 'items': <function ModelRegistry.items>, '__contains__': <function ModelRegistry.__contains__>, '__getitem__': <function ModelRegistry.__getitem__>, 'register_model_info': <function ModelRegistry.register_model_info>, 'remove_entry': <function ModelRegistry.remove_entry>, '_add_entry': <function ModelRegistry._add_entry>, '_load_from_registry_dir': <function ModelRegistry._load_from_registry_dir>, '__dict__': <attribute '__dict__' of 'ModelRegistry' objects>, '__weakref__': <attribute '__weakref__' of 'ModelRegistry' objects>, '__annotations__': {}})¶

__module__ = 'utopya.model_registry.registry'¶

__weakref__¶: list of weak references to the object (if defined)

utopya.model_registry.utils module¶

Utility functions that work on the already initialized model registry.

utopya.model_registry.utils.get_info_bundle(*, model_name: Optional[str] = None, info_bundle: Optional[utopya.model_registry.info_bundle.ModelInfoBundle] = None, bundle_key: Optional[Union[str, int]] = None) → utopya.model_registry.info_bundle.ModelInfoBundle[source]¶

Determine the model info bundle in cases where both a model name and an info bundle are allowed as arguments.

Parameters

model_name (str, optional) – The model name.
info_bundle (ModelInfoBundle, optional) – The info bundle object. If given, will directly return this object again.
bundle_key (Union[str, int], optional) – In cases where only the model name is given, the bundle_key can be used for item access, e.g. in cases where more than one bundle is available and access would be ambiguous.

Returns

The selected info bundle item

Return type

ModelInfoBundle

Raises

ValueError – If neither or both model_name and info_bundle were None

utopya.model_registry.utils.load_model_cfg(**get_info_bundle_kwargs) → Tuple[dict, str, dict][source]¶

Loads the default model configuration file for the given model name, using the path specified in the info bundle.

Furthermore, extract_validation_objects() is called to extract any Parameter objects that require validation, replace them with their default values, and gather the Parameter class objects into a separate dict.

Parameters

**get_info_bundle_kwargs – Passed on to get_info_bundle

Returns

The corresponding model configuration, the path: to the model configuration file, and the Parameter class objects requiring validation.

Return type

Tuple[dict, str, dict]

Raises

FileNotFoundError – On missing file
ValueError – On missing model