Scheduling a model for evaluation

As your data changes over time, the quality of the predictions from a particular model can degrade. With IBM Watson® Machine Learning for z/OS, you can schedule periodic evaluations of the accuracy of your models.

1. Sign into the WML for z/OS web user interface with your user name and password.
2. From the sidebar, navigate to the Model Management page and click the Models tab.
3. Select the TentModelVB model and from the ACTIONS menu, select Setup evaluation.
4. Select an Evaluator, such as binary. For Use performance metrics to monitor this model, select areaUnderROC and click Notify when less than. Specify a value or keep the default.
5. For Data Source, select an existing data source and data set and specify the minimum trigger records. If needed, create a new data source.
6. If you want to use Time range, name the time range column and choose a time.
7. Click Setup to complete the evaluation setup.
8. From the ACTIONS menu, click Deploy to deploy or redeploy the model.
9. Go to the Deployments tab and from the ACTIONS menu for the TentModelVB model, select Schedule evaluation.

   The Schedule page displays the evaluator and data set that you already selected.

   If a model has multiple deployments, you can schedule all of them for evaluation. However, you cannot schedule more than one deployment of the same base model for evaluation at a time. You must remove the evaluation setup from one deployment before you can schedule another one for evaluation.

10. For Starts at, specify the earliest time that the model can be reevaluated.
    The model is not reevaluated before this time.
11. For Repeat, select the frequency for evaluation.
    The first reevaluation occurs at the next scheduled repeat time. Optionally, specify the last time when a scheduled evaluation will occur.
12. Select the automatic retain option if the result of the scheduled evaluation is poor. You can click the deployment name to open the deployment details and review the evaluation results.
13. Click Schedule to save the scheduled evaluation.