Working with custom functions

What are custom functions?

A custom function is a multi-argument calculation that produces one or more outputs. Custom functions are typically run as part of a multi-function pipeline where the output of the first function is used as input to the next function and so on.

Typically, the custom functions you write involve multiple lines of code and are too long to paste into Maximo Monitor as a simple function. Custom functions that you develop must be stored in a Python package in an external repository, such as GitHub Enterprise, and registered with Maximo Monitor. The external repository must have a URL that is accessible from a pip installation.

Starting with a base class

The custom functions that you create extend the base classes in the base module of the in GIT with some base and sample classes IoT Functions package.

For instructions on how to use each class, see the help text of each class in base.py.

Before you write new classes, explore the built-in functions in the bif.py module or in the catalog. Create a sample device type and explore the data items and their definitions. You might decide to reuse some of the built-in functions or extend them based on the sample data.

Setting up your environment

You must install the in GIT with some base and sample classes IoT Functions package in your environment and its prerequisite packages. For more information, see Tutorial: Adding a custom function.

• In a production environment, use the production branch to install or upgrade. In a beta environment, use the beta branch.
• Make sure you are always using the latest version of IoT Functions.
• Install Python 3.9.x. Check the version of each Python module that is supported by Maximo Monitor. If you do not use the exact version that is supported, your pipeline might break.

Writing the function code

The custom functions that you create extend the base classes in the base module of the in GIT with some base and sample classes IoT Functions package.

Write your custom functions in a way that they can be reused across multiple organizations or multiple device types with no modification to the function code. You can externalize input and output configuration items through Maximo Monitor.

The name of your custom function must be unique for your tenant. In this topic, where you see a reference to MultiplyByFactor, use a unique name for the function in your own code, such as MultiplyByFactor<YourInitials>. For example, you might name your function MutlipleByFactorLD.

A company develops robotic arms. The company is introducing a new robotic arm and is testing that is performs as well as the existing robotic arm. In early testing, the team discovered that for some of the robotic arms, the distance traveled during testing was too high. After investigation, the team discovered that the tools used to do the testing were adding a delay to the speed and travel time of each arm. The operations manager wants to adjust both values by a factor of 0.9.

The analyst identifies that a similar calculation might be useful in other calculations and generalizes the function by writing a MultiplyByFactor class. For example:

import inspect
import logging
import datetime as dt
import math
from sqlalchemy.sql.sqltypes import TIMESTAMP,VARCHAR
import numpy as np
import pandas as pd

from iotfunctions.base import BaseTransformer
from iotfunctions import ui

logger = logging.getLogger(__name__)

# Specify the URL to your package here.
# This URL must be accessible via pip install.
# Example assumes the repository is private.
# Replace XXXXXX with your personal access token.
# After @ you must specify a branch.

PACKAGE_URL = 'git+https://XXXXXX@github.com/<user_id><path_to_repository>@prod'
# If your code is hosted in GitLab, use the format 'git+https://<deploy_token_username>:<deploy_token_password>@gitlab.com/<user_id><path_to_repository>@prod'

class MultiplyByFactor<YourInitials>(BaseTransformer):

     
    def __init__(self, input_items, factor, output_items):
    #Initalization method.  Define input and output items here.  

        self.input_items = input_items
        self.output_items = output_items
        self.factor = float(factor)
        super().__init__()
        

    def execute(self, df):
    #Execute method. 
        
        df = df.copy()
        for i,input_item in enumerate(self.input_items):
        df[self.output_items[i]] = df[input_item] * self.factor
        return df
				

By configuring the input item later through Maximo Monitor this function can be used in several calculations.

The calculation transforms the input item by multiplying it by a factor to produce a new metric. In this example, the function extends the BaseTransformer class in GIT with some base and sample classes IoT Functions package.

Defining configuration settings

Input and output parameters to a function are set from Maximo Monitor.

Your function class must implement a build_ui() class method of the base classes in the in GIT with some base and sample classes IoT Functions package. Use this method to describe how the inputs and outputs to the function are represented in Maximo Monitor.

In the build_ui() class method, you must specify what UI control to use for input and output items.

The following UI controls are used for input fields:

• UISingleItem: A field with single selection.
• UIMultiItem: A field with multiple selection.
• UISingle: A constant value field with single selection.
• UIMulti: A constant value field with multiple selection.

The following UI controls are used for output fields:

• UIFunctionsOutSingle: A field with single selection.
• UIFunctionOutMulti: A field with multiple selection.

Add the build_ui() class method to your custom function.

The MultiplyByFactor example function is updated to include the build_ui().

@classmethod
    def build_ui(cls):
        #define arguments that behave as function inputs
        inputs = []
        inputs.append(ui.UIMultiItem(
                name = 'input_items',
                datatype=float,
                description = "Input items to be adjusted",
                output_item = 'output_items',
                is_output_datatype_derived = True)
                      )        
        inputs.append(ui.UISingle(
                name = 'factor',
                datatype=float)
                      )
        outputs = []
        return (inputs,outputs)    

Laying out the Python package

For any function code that you host externally, you must lay out files in Python project using the following directory structure:

functions
|
|__ setup.py
|
|__ scripts
  |
  |_ local_test_of_function.py
|
|__ custom
  |
  |_ functions.py
  |_  __init__.py
			

Change the module name, custom, to a unique name, such as customyourinitials. For example, customld. Your package can include one or more function modules.

Hosting the function code externally

You must create an external repository, such as in GitHub or GitLab, to host the function code. The best approach is to use a private repository and use a personal access token to access the function.

For instructions for creating tokens, for GitHub, see Creating a personal access token and for GitLab, see Creating a deploy token.

A setup.py file includes metadata that describes the project.
For example:

  from setuptools import setup, find_packages

  setup(name='customld', version='0.0.1', packages=find_packages(),
        install_requires=['iotfunctions@git+https://github.com/ibm-watson-iot/functions.git@production'])

An __init__.py file initializes the package. For example:

__version__ = '1.0'

The package includes one or more custom modules with Python code to run a function.

Testing custom functions

To test the function, you can write a script that tests the function in your local environment. In the script, you create an instance of the custom function and use the execute_local_test() method to run it offline. For example:

fn = MultiplyByFactor(
        input_items = 'temperature',
        output_item = 'Real_feel')

df = fn.execute_local_test(generate_days = 1,to_csv=True)
print(df)

The execute_local_test() class method builds a local device type, adds the function to the device type, runs the function, and saves the results to a file (to_csv=True).

See Tutorial: Adding a custom function for a step-by-step example of how to test a custom function.

Establishing a connection

You must connect your external function code to Maximo Monitor before you can register it. Set your credentials in a credentials file to connect to the data lake.

1. Download the credentials_as.json file.
2. Replace the variables with your data and then save the file to your local machine.

The credentials file is used to run or test the function locally. Do not push this file to your external repository.

In your test script, add code to load your credentials:

with open('credentials_as.json', encoding='utf-8') as F:
  credentials = json.loads(F.read())

Add code to the script to select the default database and to build a connection to Maximo Monitor using the credentials file:

 db_schema = None
 db = Database(credentials=credentials)
				

Registering custom functions

To register a specific function, use the db.register_functions() method and pass the name of the function to the method.

from custom<yourinitials>.multiplybyfactor<yourinitials> import MultiplyByFactor<YourInitials>

db.register_functions([MultiplyByFactor<YourInitials>])

To register your custom function, your function class must implement a build_ui() class method.

To register all functions in a module, for example, iotfunctions.bif , use the db.register_module() method and pass the name of the module to the method. For example:

import iotfunctions.bif as bif
db = Database(credentials = credentials, tenant_id=credentials['tenantId'])
db.register_module(bif)

To unregister one or more functions, use db.unregister_functions method and pass the names of the functions to the method. For example:

db.unregister_functions(['MyFunction1','MyFunction2'])

Using custom functions

To apply a complex function to a device type, you create a metric, select the function from the catalog, specify the input and output configuration items, and apply it to a device type.