Implementing your own DAG operations

Contents

Implementing your own DAG operations#

Summary

On this page, you will see how to

  • make your own DAG operations available using the :py:func:` ~utopya.eval.transform.is_operation` decorator or register_operation() function.

See also the full article in the dantro docs on registering custom operations.

As documented in the overview of DAG operations, the data transformation framework has access to a plethora of standard operations you can use for your data analysis. However, you may need a special operation for your own purposes. In such a situation, you can include a custom operation and register it using the is_operation() decorator.

Add something like the following to your model-specific plot module:

"""model_plots/MyModel/__init__.py"""

# Your regular imports here

# Import the is_operation decorator ...
from utopya.eval import is_operation

# ... and register your operation from some custom callable
@is_operation("MyModel.my_operation")
def my_operation(data, *, some_parameter):
    """Some operation on the given data"""
    # ... do something with data and the parameter ...
    return new_data

Of course, custom operations can also be defined somewhere else within your plot modules, e.g. in an operations.py file, and imported into __init__.py using from .operations import my_operation.

Note that you should ideally not override existing operations. To avoid naming conflicts, it is advisable to use a unique name for custom operations, e.g. by prefixing the model name for some model-specific operation as done in the example above.

Important: Your model-specific custom operations should be defined in the model-specific plot module, i.e.: accessible after importing model_plots/<your_model_name>/__init__.py. Prior to plotting, the PlotManager pre-loads that module, such that the register_operation calls and the decorator definitions are actually invoked.

You can also use the register_operation() function to register pre-existing operations, e.g. from numpy:

import numpy as np
from utopya.eval import register_operation

# Register custom operations from some imported module ...
register_operation(name="np.mean", func=np.mean)

# ... or from a lambda
register_operation(name="my_custom_square", func=lambda d: d**2)

Hint

If the operation does not become available despite you importing it, this may have several reasons:

  • Only the model_plots/<model_name>/__init__.py is directly imported; that module should explicitly import the module that you define your operations in, otherwise the operation will not become available.

  • You may have a ModuleNotFoundError somewhere in your model-specific plot definitions. Due to restrictions of the importing function, these actual errors cannot be distinguished from other (acceptable) import errors, and will thus fail silently. This may be another point to look at if your operation does not become available.

Examples#

There are some example implementions in the Utopia SEIRD model and the Utopia Opinionet model.

Contents