utopya.model_registry package
Contents
utopya.model_registry package#
This submodule implements a registry of Utopia models, which is used to provide the required model information to the frontend and use it throughout.
Submodules#
utopya.model_registry._exceptions module#
All model registry related error types
- exception utopya.model_registry._exceptions.ModelRegistryError[source]#
Bases:
ValueError
Raised on errors with model registry
- __module__ = 'utopya.model_registry._exceptions'#
- __weakref__#
list of weak references to the object (if defined)
- exception utopya.model_registry._exceptions.BundleExistsError[source]#
Bases:
utopya.model_registry._exceptions.ModelRegistryError
Raised when a bundle that compared equal already exists.
- __module__ = 'utopya.model_registry._exceptions'#
utopya.model_registry.entry module#
This module implements the ModelRegistryEntry class, which contains a set of ModelInfoBundle objects. These can be combined in a labelled and unlabelled fashion.
- class utopya.model_registry.entry.ModelRegistryEntry(model_name: str, *, registry_dir: str)[source]#
Bases:
object
Holds model config bundles for a single model.
Instances of this class are associated with a file in the model registry directory. Upon change, they update that file.
Initialize a model registry entry from a registry directory.
Expects a file named <model_name>.yml in the registry directory. If it does not exist, creates an empty one.
- __init__(model_name: str, *, registry_dir: str)[source]#
Initialize a model registry entry from a registry directory.
Expects a file named <model_name>.yml in the registry directory. If it does not exist, creates an empty one.
- property model_name: str#
- property registry_dir: str#
- property registry_file_path: str#
- __contains__(obj) bool [source]#
Checks if the given _object_ is part of this registry entry.
Note that this does not check for keys!
- __getitem__(key: Union[str, int]) utopya.model_registry.info_bundle.ModelInfoBundle [source]#
Return a bundle for the given label or index. If None, returns the tries to return the item.
- item() utopya.model_registry.info_bundle.ModelInfoBundle [source]#
Retrieve the single bundle for this model, if not ambiguous.
- keys() Iterator[Union[str, int]] [source]#
Returns legitimate keys for item access, i.e.: all registered keys. Starts with labelled bundle names, continues with indices of unlabelled bundles.
- values() Generator[utopya.model_registry.info_bundle.ModelInfoBundle, None, None] [source]#
Returns stored model config bundles, starting with labelled ones
- items() Generator[Tuple[Union[str, int], utopya.model_registry.info_bundle.ModelInfoBundle], None, None] [source]#
Returns keys and registered bundles, starting with labelled ones
- add_bundle(*, label: Optional[str] = None, overwrite_label: bool = False, update_registry_file: bool = True, **bundle_kwargs) utopya.model_registry.info_bundle.ModelInfoBundle [source]#
Add a new configuration bundle to this registry entry.
This makes sure that the added bundle does not compare equal to an already existing bundle.
- Parameters
label (str, optional) – The label under which to add it. If None, will be added as unlabelled.
overwrite_label (bool, optional) – If True, overwrites an existing bundle under the same label. This option does not affect unlabelled bundles or registry file updates.
update_registry_file (bool, optional) – Whether to write changes directly to the registry file.
**bundle_kwargs – Passed on to construct the ModelInfoBundle that is to be stored.
- Raises
ModelRegistryError – If exist_ok is False and such a bundle already exists in this registry entry.
- pop(key: Union[str, int], *, update_registry_file: bool = True) utopya.model_registry.info_bundle.ModelInfoBundle [source]#
Pop a configuration bundle from this entry.
- clear(*, update_registry_file: bool = True)[source]#
Removes all configuration bundles from this entry.
- classmethod to_yaml(representer, node)[source]#
Creates a YAML representation of the data stored in this entry. As the data is a combination of a dict and a sequence, instances of this class are also represented as such.
- Parameters
representer (ruamel.yaml.representer) – The representer module
node (type(self)) – The node to represent, i.e. an instance of this class.
- Returns
A YAML representation of the given instance of this class.
- __dict__ = mappingproxy({'__module__': 'utopya.model_registry.entry', '__doc__': 'Holds model config bundles for a single model.\n\n Instances of this class are associated with a file in the model registry\n directory. Upon change, they update that file.\n ', '__init__': <function ModelRegistryEntry.__init__>, 'model_name': <property object>, 'registry_dir': <property object>, 'registry_file_path': <property object>, '__len__': <function ModelRegistryEntry.__len__>, '__contains__': <function ModelRegistryEntry.__contains__>, '__str__': <function ModelRegistryEntry.__str__>, '__eq__': <function ModelRegistryEntry.__eq__>, '__getitem__': <function ModelRegistryEntry.__getitem__>, 'item': <function ModelRegistryEntry.item>, 'keys': <function ModelRegistryEntry.keys>, 'values': <function ModelRegistryEntry.values>, 'items': <function ModelRegistryEntry.items>, 'add_bundle': <function ModelRegistryEntry.add_bundle>, 'pop': <function ModelRegistryEntry.pop>, 'clear': <function ModelRegistryEntry.clear>, '_load_from_registry': <function ModelRegistryEntry._load_from_registry>, '_update_registry_file': <function ModelRegistryEntry._update_registry_file>, '_as_dict': <classmethod object>, 'to_yaml': <classmethod object>, '__dict__': <attribute '__dict__' of 'ModelRegistryEntry' objects>, '__weakref__': <attribute '__weakref__' of 'ModelRegistryEntry' objects>, '__hash__': None, '__annotations__': {}})#
- __hash__ = None#
- __module__ = 'utopya.model_registry.entry'#
- __weakref__#
list of weak references to the object (if defined)
utopya.model_registry.info_bundle module#
Implements the ModelInfoBundle class, which holds a single bundle of information about a model.
- class utopya.model_registry.info_bundle.ModelInfoBundle(*, model_name: str, paths: dict, metadata: Optional[dict] = None, project_name: Optional[str] = None, registration_time: Optional[str] = None, missing_path_action: str = 'log', **additional_kwargs)[source]#
Bases:
object
A bundle of model information; behaves like a read-only dict
Initialize a ModelInfoBundle
- PATH_KEYS = (('binary', <class 'str'>, True), ('source_dir', <class 'str'>), ('default_cfg', <class 'str'>, True), ('default_plots', <class 'str'>), ('base_plots', <class 'str'>), ('python_model_tests_dir', <class 'str'>), ('python_model_plots_dir', <class 'str'>))#
- PATH_KEYS_REL_TO_SRC = ('default_cfg', 'default_plots', 'base_plots')#
- SRC_DIR_SEARCH_PATHS = (('default_cfg', '{}_cfg.yml'), ('default_plots', '{}_plots.yml'), ('base_plots', '{}_base_plots.yml'))#
- METADATA_KEYS = (('version', <class 'str'>), ('long_name', <class 'str'>), ('description', <class 'str'>), ('author', <class 'str'>), ('email', <class 'str'>), ('website', <class 'str'>), ('dependencies', <class 'list'>), ('utopya_compatibility', <class 'str'>), ('misc', <class 'dict'>))#
- __init__(*, model_name: str, paths: dict, metadata: Optional[dict] = None, project_name: Optional[str] = None, registration_time: Optional[str] = None, missing_path_action: str = 'log', **additional_kwargs)[source]#
Initialize a ModelInfoBundle
- __eq__(other) bool [source]#
Compares equality by comparing the stored configuration. Only if another ModelInfoBundle is compared against does the model name also take part in the comparison.
- property model_name: str#
- property registration_time: str#
Registration time string of this bundle
- property as_dict: dict#
Returns a deep copy of all bundle data. This does NOT include the model name and the registration time.
- property paths: dict#
Access to the paths information of the bundle
- property metadata: dict#
Access to the metadata information of the bundle
- property project_name: str#
Access to the Utopia project name information of the bundle
- property missing_paths: dict#
Returns those paths where os.path.exists did not evaluate to True
- classmethod to_yaml(representer, node)[source]#
Creates a YAML representation of the data stored in this bundle.
- Parameters
representer (ruamel.yaml.representer) – The representer module
node (type(self)) – The node to represent, i.e. an instance of this class.
- Returns
A YAML representation of the given instance of this class.
- __dict__ = mappingproxy({'__module__': 'utopya.model_registry.info_bundle', '__doc__': 'A bundle of model information; behaves like a read-only dict', 'PATH_KEYS': (('binary', <class 'str'>, True), ('source_dir', <class 'str'>), ('default_cfg', <class 'str'>, True), ('default_plots', <class 'str'>), ('base_plots', <class 'str'>), ('python_model_tests_dir', <class 'str'>), ('python_model_plots_dir', <class 'str'>)), 'PATH_KEYS_REL_TO_SRC': ('default_cfg', 'default_plots', 'base_plots'), 'SRC_DIR_SEARCH_PATHS': (('default_cfg', '{}_cfg.yml'), ('default_plots', '{}_plots.yml'), ('base_plots', '{}_base_plots.yml')), 'METADATA_KEYS': (('version', <class 'str'>), ('long_name', <class 'str'>), ('description', <class 'str'>), ('author', <class 'str'>), ('email', <class 'str'>), ('website', <class 'str'>), ('dependencies', <class 'list'>), ('utopya_compatibility', <class 'str'>), ('misc', <class 'dict'>)), '__init__': <function ModelInfoBundle.__init__>, '__eq__': <function ModelInfoBundle.__eq__>, '__str__': <function ModelInfoBundle.__str__>, 'model_name': <property object>, 'registration_time': <property object>, 'as_dict': <property object>, '__getitem__': <function ModelInfoBundle.__getitem__>, 'paths': <property object>, 'metadata': <property object>, 'project_name': <property object>, 'missing_paths': <property object>, '_parse_paths': <function ModelInfoBundle._parse_paths>, 'to_yaml': <classmethod object>, '__dict__': <attribute '__dict__' of 'ModelInfoBundle' objects>, '__weakref__': <attribute '__weakref__' of 'ModelInfoBundle' objects>, '__hash__': None, '__annotations__': {}})#
- __hash__ = None#
- __module__ = 'utopya.model_registry.info_bundle'#
- __weakref__#
list of weak references to the object (if defined)
utopya.model_registry.registry module#
This module implements the ModelRegistry, which combines ModelRegistryEntry objects and makes it possible to register new models.
- class utopya.model_registry.registry.KeyOrderedDict(*args, key: Optional[Callable] = None, **kwds)[source]#
Bases:
dantro.utils.ordereddict.KeyOrderedDict
A key-ordered dict that expects string keys and sorts by the lower-case representation of keys.
Initialize a KeyOrderedDict, which maintains key ordering. If no custom ordering function is given, orders by simple “smaller than” comparison.
Apart from that, the interface is the same as for regular dictionaries.
- Parameters
*args – a single sequence of (key, value) pairs to insert
key (Callable, optional) – The callable used to
**kwds – Passed on to
update
method
- Raises
TypeError – on len(args) > 1
- DEFAULT_KEY_COMPARATOR(k)#
- __module__ = 'utopya.model_registry.registry'#
- class utopya.model_registry.registry.ModelRegistry(utopia_cfg_dir_path: str = '/root/.config/utopia')[source]#
Bases:
object
The ModelRegistry class takes care of providing model information to the rest of the utopya package and, at the same time, maintains the model registry it is associated with.
It provides a dict-like interface to access the stored registry entries under their model name. Via
reigster_model_info
, a model can be registered and information can be added to it.Additionally, there are some functions that provide an overview over the registered models and with which information they
# TODO Consider making this class a singleton?
Loads the utopia model registry from the configuration at the given path.
- Parameters
utopia_cfg_dir_path (str, optional) – The path to store the model registry folder in.
- __init__(utopia_cfg_dir_path: str = '/root/.config/utopia')[source]#
Loads the utopia model registry from the configuration at the given path.
- Parameters
utopia_cfg_dir_path (str, optional) – The path to store the model registry folder in.
- property registry_dir: str#
The model registry directory path
- property info_str: str#
- property info_str_detailed: str#
- __contains__(model_name: str) bool [source]#
Whether an entry for the given model name exists in the registry
- __getitem__(model_name: str) utopya.model_registry.entry.ModelRegistryEntry [source]#
Retrieve a deep copy of a model registration entry for the given model name.
- register_model_info(model_name: str, *, exists_action: Optional[str] = None, **bundle_kwargs) utopya.model_registry.entry.ModelRegistryEntry [source]#
Register information for a single model. This method also allows to create a new entry if a model does not exist.
However, it will raise an error if the model was already registered and neither the skip nor the remove options were explicitly specified.
- Parameters
model_name (str) – The name of the model to register
exists_action (str, optional) –
The action to take when a model of the given name already exists. Possible values:
None
: Continue with registering the info bundle; notethat this might still lead to errors if the exact same bundle already exists.
skip
: Skip bundle registrationraise
: Raise an errorclear
: Clear all existing bundlesfrom the corresponding model registry entry.
validate
: Makes sure the bundle that will be createdfrom the given kwargs is part of the registry entry.
**bundle_kwargs – Passed on to ModelRegistryEntry.add_bundle
- Returns
The registry entry for this model.
- Return type
- Raises
ValueError – On
exists_action == 'raise'
and model already existing.
- remove_entry(model_name: str)[source]#
Removes a registry entry and deletes the associated registry file
- __dict__ = mappingproxy({'__module__': 'utopya.model_registry.registry', '__doc__': 'The ModelRegistry class takes care of providing model information to the\n rest of the utopya package and, at the same time, maintains the model\n registry it is associated with.\n\n It provides a dict-like interface to access the stored registry entries\n under their model name. Via ``reigster_model_info``, a model can be\n registered and information can be added to it.\n\n Additionally, there are some functions that provide an overview over the\n registered models and with which information they\n\n # TODO Consider making this class a singleton?\n ', '__init__': <function ModelRegistry.__init__>, 'registry_dir': <property object>, '__len__': <function ModelRegistry.__len__>, '__str__': <function ModelRegistry.__str__>, 'info_str': <property object>, 'info_str_detailed': <property object>, 'keys': <function ModelRegistry.keys>, 'values': <function ModelRegistry.values>, 'items': <function ModelRegistry.items>, '__contains__': <function ModelRegistry.__contains__>, '__getitem__': <function ModelRegistry.__getitem__>, 'register_model_info': <function ModelRegistry.register_model_info>, 'remove_entry': <function ModelRegistry.remove_entry>, '_add_entry': <function ModelRegistry._add_entry>, '_load_from_registry_dir': <function ModelRegistry._load_from_registry_dir>, '__dict__': <attribute '__dict__' of 'ModelRegistry' objects>, '__weakref__': <attribute '__weakref__' of 'ModelRegistry' objects>, '__annotations__': {}})#
- __module__ = 'utopya.model_registry.registry'#
- __weakref__#
list of weak references to the object (if defined)
utopya.model_registry.utils module#
Utility functions that work on the already initialized model registry.
- utopya.model_registry.utils.get_info_bundle(*, model_name: Optional[str] = None, info_bundle: Optional[utopya.model_registry.info_bundle.ModelInfoBundle] = None, bundle_key: Optional[Union[str, int]] = None) utopya.model_registry.info_bundle.ModelInfoBundle [source]#
Determine the model info bundle in cases where both a model name and an info bundle are allowed as arguments.
- Parameters
model_name (str, optional) – The model name.
info_bundle (ModelInfoBundle, optional) – The info bundle object. If given, will directly return this object again.
bundle_key (Union[str, int], optional) – In cases where only the model name is given, the bundle_key can be used for item access, e.g. in cases where more than one bundle is available and access would be ambiguous.
- Returns
The selected info bundle item
- Return type
- Raises
ValueError – If neither or both model_name and info_bundle were None
- utopya.model_registry.utils.load_model_cfg(**get_info_bundle_kwargs) Tuple[dict, str, dict] [source]#
Loads the default model configuration file for the given model name, using the path specified in the info bundle.
Furthermore,
extract_validation_objects()
is called to extract any Parameter objects that require validation, replace them with their default values, and gather the Parameter class objects into a separate dict.- Parameters
**get_info_bundle_kwargs – Passed on to get_info_bundle
- Returns
- The corresponding model configuration, the path
to the model configuration file, and the Parameter class objects requiring validation.
- Return type
Tuple[dict, str, dict]
- Raises
FileNotFoundError – On missing file
ValueError – On missing model