Implementing your own DAG operations#
On this page, you will see how to
make your own DAG operations available using the :py:func:` ~utopya.eval.transform.is_operation` decorator or
See also the full article in the dantro docs on registering custom operations.
As documented in the overview of DAG operations, the data transformation framework has access to a plethora of standard operations you can use for your data analysis.
However, you may need a special operation for your own purposes.
In such a situation, you can include a custom operation and register it using the
Add something like the following to your model-specific plot module:
"""model_plots/MyModel/__init__.py""" # Your regular imports here # Import the is_operation decorator ... from utopya.eval import is_operation # ... and register your operation from some custom callable @is_operation("MyModel.my_operation") def my_operation(data, *, some_parameter): """Some operation on the given data""" # ... do something with data and the parameter ... return new_data
Of course, custom operations can also be defined somewhere else within your plot modules, e.g. in an
operations.py file, and imported into
from .operations import my_operation.
Note that you should ideally not override existing operations. To avoid naming conflicts, it is advisable to use a unique name for custom operations, e.g. by prefixing the model name for some model-specific operation as done in the example above.
Important: Your model-specific custom operations should be defined in the model-specific plot module, i.e.: accessible after importing
Prior to plotting, the
PlotManager pre-loads that module, such that the
register_operation calls and the decorator definitions are actually invoked.
You can also use the
register_operation() function to register pre-existing operations, e.g. from numpy:
import numpy as np from utopya.eval import register_operation # Register custom operations from some imported module ... register_operation(name="np.mean", func=np.mean) # ... or from a lambda register_operation(name="my_custom_square", func=lambda d: d**2)
If the operation does not become available despite you importing it, this may have several reasons:
model_plots/<model_name>/__init__.pyis directly imported; that module should explicitly import the module that you define your operations in, otherwise the operation will not become available.
You may have a
ModuleNotFoundErrorsomewhere in your model-specific plot definitions. Due to restrictions of the importing function, these actual errors cannot be distinguished from other (acceptable) import errors, and will thus fail silently. This may be another point to look at if your operation does not become available.
There are some example implementions in the Utopia SEIRD model and the Utopia Opinionet model.