Generative AI is a transformative technology that many organizations are experimenting with or already using in production to unlock rapid innovation and drive massive productivity gains. However, we have seen that this breakneck pace of adoption has left business leaders wanting more visibility and control around the enterprise usage of GenAI.

When I talk with clients about their organization’s use of GenAI, I ask them these questions:

• Do you have visibility into which third-party AI services are being used across your company and for what purposes?
• How much is your company cumulatively paying for LLM subscriptions, including signups by teams and individuals, and are those costs predictable and controllable?
• Are you able to address common vulnerabilities when invoking LLMs, such as the leakage of sensitive data, unauthorized user access and policy violations?

These questions can all be answered if you have an AI Gateway.

What is an AI Gateway? 

An AI gateway provides a single point of control for organizations to access AI services via APIs in the public domain and brokers secured connectivity between your different applications and third-party AI APIs both within and outside an organization’s infrastructure. It acts as the gatekeeper for data and instructions that flow between those components. An AI Gateway provides policies to centrally manage and control the use of AI APIs with your applications, as well as key analytics and insights to help you make decisions faster on LLM choices. 

Announcing AI Gateway for IBM API Connect 

Today IBM is announcing the launch of AI Gateway for IBM API Connect, a feature of our market-leading and award-winning API management platform. This new AI gateway feature, generally available by the end of June, will empower customers to accelerate their AI journey. When this feature launches in June, you will be able to get started with centrally managing watsonx.ai APIs, with the ability to manage additional LLM APIs planned for later this year. 

Key benefits of AI Gateway for IBM API Connect 

1. Faster and more responsible adoption of GenAI: Centralized, controllable self-service access to enterprise AI APIs for developers. 
2. Insights and cost management: Address unexpected or excessive costs for AI services through limiting the rate of requests within a certain duration and by caching AI responses, use built-in analytics and dashboards to get visibility into enterprise-wide use of AI APIs. 
3. Governance and compliance: By funneling LLM API traffic through the AI Gateway, you can centrally manage the use of AI services through policy enforcement, data encryption, masking of sensitive data, access control, audit trails and more, in support of your compliance obligations. 

Take the next step 

Learn how to complement watsonx.ai and watsonx.gov with AI Gateway for IBM API Connect by visiting our webpage or requesting a live demo to see it in action.