IBM Responds to U.S. Department of Commerce RFI on Artificial Intelligence

Today, IBM submitted comments to the United States Department of Commerce’s National Institute of Standards and Technology (NIST) regarding the development of standards and tools to build and advance trusted Artificial Intelligence (AI) systems and applications.

As a global leader in the advancement of trusted AI, IBM welcomes the opportunity to respond to the Request for Information (RFI) on Artificial Intelligence Standards issued by NIST on May 1, 2019. We believe NIST has an important role to play in driving U.S. technical leadership in AI and promoting broader awareness of and trust in the technology. As such, IBM is proposing 10 actionable recommendations for NIST in its RFI response that are built around the following key priorities in the areas of:

  1. Trustworthy AI (includes Fairness, Explainability, Robustness and Transparency): Establish an overall AI accountability framework that provides a shared conceptual foundation with a consistent set of definitions for trustworthy AI and fosters development of trust-related tools including evaluations, data sets, and metrics.
  2. Ethics and Responsibility: Maintain awareness of global efforts related to AI ethics, including best practices, guidelines and standards, and follow the related activities from Institute of Electrical and Electronics Engineers (IEEE), the European Commission, and the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC).
  3. Privacy: Ensure any additional standards or tools touching on privacy and useful for the development of AI should build on the NIST Privacy Framework.
  4. Evaluations: Create evaluations for trustworthy AI and look for ways to expand these to incorporate essential aspects of fairness, explainability, robustness, and transparency, in addition to accuracy, for a growing set of cross-cutting and industry-specific AI tasks and data modalities. One suggestion is for NIST to create a series of challenges within the AI community that will help determine how best to judge meaningful explanations.
  5. Data sets: Make larger and more diverse public data sets available to the AI community for training, developing, and testing trusted AI systems.
  6. Metrics: Develop metrics for trustworthy AI that provide technical measures for fairness, explainability, robustness, transparency, and effectiveness of AI systems.

AI is emerging as an important foundational technology, and the fast pace at which the AI field is developing is a strong indication of its enormous potential to transform business and society in a tremendously positive way. NIST has an important opportunity to accelerate this innovation and enable development of trustworthy AI and ensure U.S. competitiveness.

IBM and Trustworthy AI

Fairness, explainability, robustness, and transparency are the foundation of IBM’s Principles for Trust & Transparency. We believe these principles are essential throughout the entire AI lifecycle to strengthen trust in AI as a force for good in the world. Today, IBM offers multiple services including Watson OpenScale, AI Fairness 360 Toolkit, the Adversarial Robustness Toolkit, and the Everyday Ethics Guide for Artificial Intelligence that provide organizations, developers, and designers with tools and insights to help align their use of AI systems with human values and expectations.

IBM looks forward to continuing its collaborations with NIST, as well as industry , academia, and other organizations to ensure the development and deployment of trusted AI technologies that make the world a smarter, healthier and more prosperous place.

-John R. Smith, IBM Fellow, AI Tech, IBM Research AI; and Mark O’Riley, IBM Government and Regulatory Affairs Executive