DeepSeek-R1 is a reminder on the importance of AI governance

Open-source foundation models, such as the much-discussed DeepSeek-R1, can give enterprises a head start when creating their own custom models and can therefore increase ROI. But what has been less discussed is the role of AI governance. Governance is required for all forms of AI and ML, but there are unique considerations for open-source models from 3^rd party providers. If you want your organization’s use of AI to be safe, transparent and responsible, AI governance is crucial.

Given these developments, let’s discuss how you can take advantage of open-source models such as DeepSeek’s, while identifying how AI governance can mitigate some of the major challenges.

An IBM study found that 51% of businesses using open-source tools saw positive ROI, compared to just 41% of those that weren’t. These new and emerging, smaller open-source models reinforce IBM’s stance on the future of AI being open, which will help businesses realize the benefits of AI in a way that balances efficiency with trust and safety. While many tech leaders were surprised and concerned with the implications of DeepSeek, IBM has long been a proponent of a more open approach. “Smaller, efficient models can deliver real results without massive, proprietary systems,” IBM CEO Arvind Krishna recently said in an article published in Fortune.

When using open-source foundation models, AI governance is critical across the model lifecycle. Starting with a risk assessment, organizations must first decide whether the model is safe for use. What is considered safe for general purpose or individual use might not be suitable for enterprise use cases.

In the case of DeepSeek-R1, organizations might decide that self-hosting the model or running it through an AI studio product such as IBM® watsonx.ai™ is relatively secure. However, using the DeepSeek app might present data and security challenges. Many countries have banned the use of the DeepSeek app for this reason.

After models are approved, model validators can add them to a model inventory, which tracks model utilization and performance while also designating the use cases to which models could be applied. This collection of models allows AI developers to identify available models and for assurance, data and security teams to track where the models are being used.

AI factsheets capture model facts throughout the lifecycle and increase transparency about what is within the model and how it is being used. Factsheets demonstrate which foundation model was used, including prompt parameters, templates and evaluation.

There are concerns with the safety of some open-source models and the ease of prompt injection and jailbreaking to generate harmful output results. Observability and guardrails can monitor model health, accuracy and drift, and limit hate speech, profanity and other harmful content. 

The downsides of AI without AI governance are immense. Organizations that use AI to interact with individuals in the EU must comply with the EU AI Act. Failure to comply could result in a EUR 35 million fine or 7% annual turnover, whichever is greater.

The risks to companies’ brands, reputations and customer and partner trust are also significant. Imagine if your organization compromises the personal data of customers, or insults or defrauds a customer in an interaction. They won’t care whether the original fault was with your team or the foundation model, your brand is damaged either way. Without safeguards in place, using generative AI for any production use case becomes too risky.

AI governance allows organizations to manage and mitigate a range of risks and empowers them to use AI safely and responsibly. To support these efforts, IBM Research® developed the AI Risk Atlas, a tool designed to help organizations identify and map various AI risks.

A robust AI governance framework and solution is crucial for your organization, but building one from scratch is a lengthy, costly process that can be detrimental to your AI ROI. For organizations that want to scale their AI efficiently, securely and responsibly, we built watsonx.governance™.

An AI governance solution, such as watsonx.governance, enables leaders to decide whether and how to allow models such as DeepSeek’s in their organization’s AI. By integrating with IBM Guardium® AI Security, watsonx.governance can also identify unknown instances of the model and manage specific risks such as jailbreak, DDOS and PII data prompts. It can govern any model and be applied to any cloud or run on premises. IBM’s AI Risk Atlas

DeepSeek is just one in a series of open-source innovations and serves as a reminder that, compared to previous technologies, every AI advancement must place governance and security at the core of your organization’s strategy. They are not optional but foundational requirements.