Repository library APIs

You can use the ML for IBM z/OS® repository library APIs to train and save a spark-based model.

Class com.ibm.analytics.ngp. repository_v3.MLRepositoryClient

An MLRepositoryClient instance is an utility used to save a spark-based model with MLz repository library.

Method
def MLRepositoryClient(defBasePath: Option[String] = None, 
                       defApiInvoker: Option[MLApiInvoker] = Some(new MLApiInvoker), 
                       defAuthorizationBasePath: Option[String] = Some(null))

Initializes self.

Where:
  • defBasePath is the url of the repository service.
  • defAuthorizationBasePath is the url for authorization.
How to use:
import com.ibm.analytics.ngp.repository_v3._
val repositoryServicePath = "https://<service-url>"
val client = MLRepositoryClient(repositoryServicePath)
Method
def authorize(token: String): Unit

Authorizes with the MLz JWT token.

Where:
  • token is the credential for the MLz core service.
How to use:
val token = <Bearer-token>
client.authorize(token)
Method
def models(): ArtifactCollection[ModelArtifact, ArchiveArtifact] = modelCollection

Collection that allows to operate on repository pipeline models.

Class com.ibm.analytics.ngp. repository_v3.MLRepositoryClient.ModelCollection

Method
def all(): Try[Seq[ModelArtifact]]

Gets all models.

How to use:

import com.ibm.analytics.ngp.repository_v3._
val repositoryServicePath = "https://<service-url>"
val client = MLRepositoryClient(repositoryServicePath)
val token = <Bearer-token>
client.authorize(token) 
client.models.all()
Method
def get(artifactId: String): Try[ModelArtifact]

Get model metadata for this artifactId.

Where:
  • artifactId is the id of the model in the MLz core service.

How to use:

import com.ibm.analytics.ngp.repository_v3._
val repositoryServicePath = "https://<service-url>"
val client = MLRepositoryClient(repositoryServicePath)
val token = <Bearer-token>
client.authorize(token) 
val model = client.models.get("6aa8fb29-ce9c-4410-b4d5-e553bbb4eb28")
Method
def getModelId(modelName: String): Try[String]

Get model id by the model artifact’s name.

Where:
  • modelName is the name of the model.

How to use:

import com.ibm.analytics.ngp.repository_v3._
val repositoryServicePath = "https://<service-url>"
val client = MLRepositoryClient(repositoryServicePath)
val token = <Bearer-token>
client.authorize(token) 
val modelId = client.models.getModelId("model_test")
Method
def remove(artifactId: String): Try[Unit]

Remove a model by artifact’s Id from the MLz core service.

Where:
  • artifactId is the name of the model.

How to use:

import com.ibm.analytics.ngp.repository_v3._
val repositoryServicePath = "https://<service-url>"
val client = MLRepositoryClient(repositoryServicePath)
val token = <Bearer-token>
client.authorize(token) 
val modelId = client.models.getModelId("model_test").get
client.models.remove(modelId)
Method
def save(artifact: ModelArtifact): Try[ModelArtifact]

Save the model to the repository service.

Where:
  • artifactId stores all the information of the model which includes model id, name, training data, model metadata and so on.
How to use:
import com.ibm.analytics.ngp.repository_v3._
val repositoryServicePath = "https://<service-url>"
val client = MLRepositoryClient(repositoryServicePath)
val token = <Bearer-token>
client.authorize(token) 
val modelArtifact = MLRepositoryArtifact(
                        tentModel,
                        trainDF,
                        "tentModel",
    			       MetaNames.DESCRIPTION -> "Tent Model", 
    			       MetaNames.LABEL_FIELD -> "TENT_LABEL")
client.models.save(modelArtifact)

MLz core service also supports that you associate a custom runtime with the model.

You can create the custom runtime then supply the name of the custom runtime in the MetaName.CUSTOM_RUNTIME_NAME.

Example:

The custom runtime has been created and named "myCustomRuntime".

import com.ibm.analytics.ngp.repository_v3._
val mlRepositoryArtifact = MLRepositoryArtifact(
    tentModel,
    trainDF,
    "tentModel_custom",
    MetaNames.DESCRIPTION -> "Tent Model with custom runtime", 
    MetaNames.LABEL_FIELD -> "TENT_LABEL",
    MetaNames.CUSTOM_RUNTIME_NAME ->  "myCustomRuntime")
val modelArtifact = client.models.save(mlRepositoryArtifact)
Method
def save(mlPipelineModel: PipelineModelLike, trainingData: DataFrame, 
modelId: String, props: MetaProps): Try[ModelArtifact]

Creates a new version of existing model artifact.

Where:
  • mlPipelineModel specifies the type of a trained model and defines the pipeline for creating and training a model.
  • training_data specifies the subset of data used for training a Spark model.
  • modelId is the id of the existing model.
  • props specifies the metadata of this model version.

How to use:

Create a new version based on the exiting model "tentModel".

import com.ibm.analytics.ngp.repository_v3._
val repositoryServicePath = "https://<service-url>"
val client = MLRepositoryClient(repositoryServicePath)
val token = <Bearer-token>
client.authorize(token) 
val modelId = client.models.getModelId("tentModel").get
val props = MetaProps(Map(
                  "MetaNames.AUTHOR_EMAIL" -> "tester01@ibm.com"))
val modelVersionArtifact = client.models.save(tentModel, trainDF, modelId, props)

MLz core service also supports that you associate a custom runtime when saving a new model version.

You can create the custom runtime and supply the name of the custom runtime in the MetaName.CUSTOM_RUNTIME_NAME.
val props = MetaProps(Map(
              "MetaNames.CUSTOM_RUNTIME_NAME" -> "myCustomRuntime"))
client.models.save(tentModel, trainDF, modelId,props)
Method
def version(artifactVersionHref: String): Try[ModelArtifact]

Get a specific version of the artifact identified by 'href'.

Where:
  • artifactVersionHref is the url of the model version which includes the model id and version id.

How to use:

import com.ibm.analytics.ngp.repository_v3._
val repositoryServicePath = "https://<service-url>:<port>"
val client = MLRepositoryClient(repositoryServicePath)
val token = <Bearer-token>
client.authorize(token) 
val modelId= client.models.getModelId("tentModel").get
val modelArtifact = client.models.get(moddelId)
val versionId = modelVersionArtifact.get.meta.prop(MetaNames.VERSION)
val artifactVerHref1 = repositoryServicePath + "/v3/ml_assets/models/"+  modelId + "/versions/" + versionId
val model1_version = client.models.version(artifactVerHref1)
Method
def versions(artifactId: String): Try[Seq[ModelArtifact]]

Get all versions info for this modelId.

Where:
  • artifactId is the id of the model in the MLz core service.
How to use:
import com.ibm.analytics.ngp.repository_v3._
val repositoryServicePath = "https://<service-url>"
val client = MLRepositoryClient(repositoryServicePath)
val token = <Bearer-token>
client.authorize(token) 
val versions = client.models.versions(modelId)