Understanding how debiasing works
Watson OpenScale corrects for both direct and indirect bias by using a scoring endpoint.
The debiased scoring endpoint can be used exactly as the normal scoring endpoint of your deployed model. In addition to returning the response of your deployed model, it also returns the
debiased_predictioncolumn contains the debiased prediction value. In IBM Watson Machine Learning, this column is an encoded representation of the prediction. For example, if the model prediction is either “Loan Granted” or “Loan Denied”, IBM Watson Machine Learning can encode these two values to “0.0” and “1.0”. The
debiased_predictioncolumn contains such an encoded representation of the debiased prediction. If you specify a string column as prediction column,
debiased_predictioncolumn can also contain a string value.
debiased_probabilitycolumn represents the probability of the debiased prediction. This array of double values represents the probability of the debiased prediction that belongs to one of the prediction classes.
debiased_decoded_targetcolumn still exists in the response, but it contains the same value as the one in the
debiased_predictioncolumn. Because there is no mapping set in the Model details wizard, you can now directly specify the string column as a prediction column.
debiased_decoded_targetcolumn contains the string representation of the debiased prediction. In the previous example, where the prediction value was either “0.0” or “1.0”, the
debiased_decoded_targetcontains either “Loan Granted” or “Loan Denied”.
Steps to check the debias endpoint
To check the debias endpoint, from the Endpoint box, select Debiased transactions. You can then view and copy the endpoint in different formats, such as cURL, Java, or Python.
Ideally, you would directly call this endpoint from your production application, instead of directly calling the scoring endpoint of your model that is deployed in your machine learning provider. Watson OpenScale stores the
debiased values in the payload logging table of your model deployment. Then, all scoring done through this endpoint is automatically debiased.
Because this endpoint deals with runtime bias, it continues to run background checks for the scoring data from the payload logging table. It keeps updating the bias mitigation model, which is used to debias the scoring requests sent. In this way, Watson OpenScale is always up to date with the incoming data, and with its behavior to detect and mitigate bias.
Finally, Watson OpenScale uses a threshold to decide that data is now acceptable and is deemed to be unbiased. That threshold is taken as the least value from the thresholds set in the Fairness monitor for all the fairness attributes configured.
- For more information, see Debiasing options.