Six steps to fairness

1. Identify risks. Who might be impacted by the proposed application?
   • Is this an area where people with disabilities have historically experienced discrimination? If so, can the project improve on the past? Identify what specific outcomes there should be, so these can be checked as the project progresses.
   • Which groups of people might be unable to provide the expected input data (e.g. clear speech), or have data that looks different (e.g. use sign language, use a wheelchair)? How would they be accommodated?
   • Consider whether some input data might be proxies for disability.
2. Involve stakeholders. Having identified potentially impacted groups, involve them in the design process. Approaches to developing ethical AI applications for persons with disabilities include actively seeking the ongoing involvement of a diverse set of stakeholders (Cutler et al., 2019 – Everyday Ethics for AI) and a diversity of data to work with. It may be useful to define a set of ’outlier’ individuals and include them in the team, following an inclusive design method. These ‘outliers’ are people whose data may look very different from the average person’s data. What defines an outlier depends on the application. For example, in speech recognition, it could be a person with a stutter or a person with slow, slurred speech. Outliers also are people who belong in a group, but whose data look different. For example, Alan may use different software from his peers because it works better with his screen reader technology. By defining outlier individuals up front, the design process can consider, at each stage, what their needs are, whether there are potential harms that need to be avoided, and how to achieve this.
3. Define what it means for this application to be ‘fair’. In many jurisdictions, fair treatment means that the process using the application allows individuals with disabilities to compete on their merits, with reasonable accommodations.Decide how fairness will be measured for the application itself, and also for any AI models used in the application.  If different ability groups are identified in the data, group fairness tests can be applied to the model.  These tests measure fairness by comparing outcomes between groups. If the difference between the groups is below a threshold, the application is considered to be fair. If group membership is not known, individual fairness metrics can be used to test whether ‘similar’ individuals receive similar outcomes. With the key stakeholders, define the metric for the project as a whole, including accommodations, and use diverse individuals for testing.
4. Plan for outliers. People are wonderfully diverse, and there will always be individuals who are outliers, not represented in the AI model’s training or test data. Design solutions that also can address fairness for small groups and individuals, and support reasonable accommodations. One important step is providing explanations and ways to report errors or appeal decisions. IBM’s AI Explainability 360 toolkit includes ‘local explanation’ algorithms that describe factors influencing an individual decision. With an explanation, users like Alan can gain trust that the system is fair, or steps can be taken to address problems.
5. Test for model bias and mitigate.
   • Develop a plan for tackling bias in source data in order to avoid perpet­uating previous discriminatory treatment. This could include boosting representation of peo­ple with disabilities, adjusting for bias against specific disability groups, or flagging gaps in data coverage so the limits of the resulting model are explicit.
   • Bias can come in at any stage of the machine learning pipeline. Where possible, use tools for detecting and mitigating bias during development. IBM’s AI Fairness 360 Toolkit offers many different statistical methods for assessing fairness. These methods require protected attributes, such as disability status, to be well defined in the data. This could be applied within large organizations when scrutinizing promotion practices for fairness, for example.
   • Test with outliers, using input from key stakeholders. In recruitment and other contexts where candidate disability information is not available, statistical approaches to fairness are less applicable. Testing is essential to understand how robust the solution is for outlier individuals. Measure against the fairness metrics defined previously to ensure the overall solution is acceptable.
6. Build accessible solutions. Design, build and test the solution to be usable by people with diverse abilities, and to support accommodations for individuals. IBM’s Equal Access Toolkit provides detailed guidance and opensource tools to help teams understand and meet accessibility standards, such as the W3C Web Content Accessibility Guidelines (WCAG) 2.1.

Considerations for policymakers