Working with Python models and custom classes

When you're working with Python models, all classes on which the models depend must be available in the Python search path for any session that loads or writes the models. The session can be remote, such as livy sessions via Hadoop Integration or local (Watson Studio Local).

For custom Python classes, you must ensure that the class definition exists within a module (Python file) that is in your Python search path. There are two ways to accomplish this:
  • Put your Python files into a .zip file in your local project workspace. For any code that loads, writes, or executes the model, add logic to add the zip file into to your session's active search path. For example:
    • Local (Watson Studio Local) session: sys.path.insert(0, "/path/to/your/zip").
    • Remote (Hadoop livy session): sc.addPyFile("hdfs:///path/to/your/zip/uploaded/to/hdfs")
    Note: When you're using code-generation utilities, such as those for generating batch scoring or model evaluation scripts, the logic to load your custom classes from your zip file will not be automatically included; you will need to manually add it before accessing your model.
  • Ask your Watson Studio Local admin to place the Python files in which your custom classes are declared directly into one of the predefined global directories. The directories are automatically loaded into the Python search path for the default Watson Studio Local Python images. Example global directories include:
    • /user-home/_global/python-2.7
    • /user-home/_global/python-3.5
    • /user-home/_global/python-3.6
    This will allow your jobs and environments to automatically pick up your custom classes, making them usable by all model-related functionality on the local Watson Studio Local side. If the admin then proceeds to re-push the relevant image to a registered Hadoop edge node, your custom packages will automatically become available for all model-related operations on the remote side, as well, assuming your livy session is configured to run with that re-pushed image.
    Note: With this approach, other Watson Studio Local users can use your custom classes, as well. If your custom class includes anything private, do not use this approach.