LLM APIs: Tips for bridging the gap

When visiting a country whose language you don’t know, you might rely on a friend to translate conversations or a translation app when asking for directions. That way, you wouldn't need to learn the whole language, especially for short trips.

In the realm of large language models (LLMs), application programming interfaces (APIs) act as translators, allowing seamless exchange between LLMs and artificial intelligence (AI) applications. These interfaces facilitate the integration of natural language processing (NLP) and natural language understanding capabilities into software systems.

Through LLM APIs, businesses can harness AI models in their workflows. Online retailers, for instance, can connect their customer service chatbot to a language model for more tailored responses that foster natural and engaging interactions. Similarly, companies can link their AI coding assistant to an LLM for more robust code analysis and generation.

LLM APIs are typically based on a request-response architecture that follows a series of steps:

1. An application sends a request—generally in the form of a hypertext transfer protocol (HTTP) request—to the API. Before transmission, the app first converts the request into the API’s required data format (usually in JavaScript Object Notation or JSON), which contains information such as the model variant, the actual prompt and other parameters.

2. After the API receives the request, it forwards it to the LLM for processing.

3. The machine learning model draws upon its NLP skills—be it content generation, question answering, sentiment analysis, text generation or text summarization—to produce a response that it relays to the API.

4. The API delivers this response back to the application.

To access an LLM API, users will need to sign up with their chosen provider and generate API keys for authentication.

Pricing is an important component of LLM APIs. Providers offer varied price points based on their models.

To understand how LLM API pricing works, you’ll need to first grasp the concept of tokens. For language models, tokens are machine-readable representations of words. A token can be a letter, a punctuation mark, part of a word or the entire word itself.

Tokens are the smallest units of text that a model can take in and process as input and generate as output. They serve as the basis for pricing. Most providers use a pay-as-you-go pricing model, charging for LLM API access per thousand or million tokens, with separate pricing for input and output tokens.

This token-based pricing reflects the computational and processing costs associated with running LLMs. It also allows for transparency and flexibility, accommodating different usage patterns among businesses.

Combining enterprise data or services with the AI layer that LLM APIs bring makes for more powerful real-world applications. Here are a few benefits LLM APIs can offer:

• Accessibility: Businesses can take advantage of AI language capabilities without requiring comprehensive knowledge of and expertise in AI. They also won’t need to invest in developing their own models and the associated infrastructure costs.
• Customization: Through LLM APIs, organizations can fine-tune large language models to fit their specific tasks or domains.
• Periodic updates: Providers regularly update their algorithms to improve performance and keep up with the rapid pace of change in AI.
• Scalability: LLM APIs can usually handle large volumes of requests simultaneously, scaling as a business grows.

Despite these gains, LLM APIs also come with challenges:

• Cost: These interfaces can be expensive, particularly for high-volume or large-scale use. Enterprises must manage their costs effectively to maximize the value of LLM APIs.
• Security vulnerabilities: Bad actors can use API endpoints for malicious purposes, such as extracting sensitive data, installing malware or conducting distributed denial of service (DDoS) attacks by sending a flood of requests.

LLM APIs open up possibilities for enterprises to realize the full potential of their applications through AI. Here are five techniques to help businesses use LLM APIs more efficiently:

Select the language model that best suits your use case. Start with basic features and gradually work your way up to more advanced ones.

For instance, if you’re only after sentiment analysis, a smaller, older, more cost-efficient model will do. However, if rapid and real-time responses are what you’re after, such as with customer service chatbots and translation apps, you might opt for a larger, newer model. More complex tasks might require the newest, most powerful model variant.

Some providers even supply APIs and models tailored for specific use cases. OpenAI’s Assistants API is targeted for building AI assistants, while Mistral has APIs for coding and computer vision tasks. You can also consider fine-tuning APIs to fine-tune a model with your organization’s training data.

The cost of using LLM APIs can stack up quickly, so keep an eye on your usage. Most providers have dashboards or tools to monitor token usage and set monthly spending limits to manage your costs. Stay updated on pricing and algorithm changes that might better suit your budget and deliver more value.

Some providers offer lower prices or discounts on certain services. Google’s Gemini API, like OpenAI, has a cheaper price point for context caching, wherein a set of input tokens are stored in a cache for retrieval by succeeding requests. This practice is helpful when repetitive content is passed to a model—whether it’s a recurring instruction from a chatbot, repeated queries for a dataset or similar bug fixes for a codebase.

Meanwhile, OpenAI offers a discount for batch processing through its Batch API (Anthropic and Mistral have similar APIs). This asynchronous processing can be a cost-effective option for sending groups of requests on large datasets that don’t require immediate responses, such as summarizing lengthy documents or classifying content.

Take advantage of free LLM API tiers. These tiers are free of charge but have limits on tokens or usage. For enterprises on a tight budget, free LLM API tiers might be suitable for testing apps or building prototypes.

API security is a must for any organization. Here are some ways to secure API interactions with LLMs:

• Implement secure protocols to encrypt the information that passes through the LLM API, thus protecting data in transit.
• Establish access control policies so that only authorized users can access API keys and to limit access to the API itself.
• Remove any sensitive information from datasets before sending them through LLM APIs.
• Evaluate the security measures and policies of your chosen LLM API provider.

Tokens drive cost, so minimizing the input token count can help lower cost and improve performance. One way to minimize the input token is through token optimization, which borrows heavily from prompt engineering tactics.

Here are a few strategies for token optimization:

• Craft clear and concise prompts. Use direct language and focused instructions.
• Break down long prompts into smaller, meaningful parts, if a long prompt can't be avoided.
• Remove redundant data and unnecessary details.
• Provide short, highly representative examples in a structured and consistent format, in terms of context. Only include information that’s critical for a model to understand the task.

After you’ve applied the relevant optimization techniques, continuously refine your prompts based on the model’s outputs. Verify those outputs to make sure they’re correct and accurate.

Observe your usage patterns to see whether they’re in line with your budget and whether you’re implementing the most cost-effective model. Employ API monitoring solutions to track LLM API performance according to key metrics such as response time, latency and error rates to maximize the effectiveness of your chosen model.

LLM APIs are a growing market. Many LLM developers have their own APIs, while other external API providers supply access to various large language models.

Independent benchmarking firm Artificial Analysis maintains a popular LLM API leaderboard (link resides outside ibm.com) that compares and ranks different API endpoints across metrics such as latency, output speed, quality and price.

Here are some popular LLM APIs:

AI research firm Anthropic has APIs (link resides outside ibm.com) for its Claude family of large language models. These models include Claude 3.5 Sonnet, the company’s latest premium offering; Claude 3.5 Haiku, its fastest and most cost-effective model; and Claude 3 Opus, a powerful model for complex tasks. APIs are also available for older model versions such as Claude 3 Haiku and Claude 3 Sonnet.

There are three ways to access the API (link resides outside ibm.com): Anthropic’s web console, developer libraries in Python and TypeScript on GitHub, and on partner platforms such as Amazon Bedrock and Google Cloud Vertex AI.

AI company Cohere provides its own API (link resides outside ibm.com) for Command R+, its LLM purpose-built for enterprise use cases, and Command R, a generative AI model optimized for retrieval-augmented generation (RAG) and agentic AI functionality. Developers can access the API (link resides outside ibm.com) by using Cohere’s command-line interface tool or through Go, Java, Python and TypeScript libraries on GitHub.

Google offers APIs (link resides outside ibm.com) for its Gemini suite of large language models. These models include Gemini 1.5 Flash, its fastest multimodal AI model; Gemini 1.5 Flash-8B, its smallest model; Gemini 1.5 Pro, its next-generation model; and Gemini 1.0 Pro, its first-generation model.

Developers can access the Gemini API (link resides outside ibm.com) on Google AI Studio and Google Cloud Vertex AI. Software development libraries are also available in different programming languages.

IBM® Granite™ is the IBM flagship series of LLM foundation models. Developers can use APIs on the IBM watsonx™ platform to access the Granite 3.0 models, specifically Granite 3.0 2B Instruct and Granite 3.0 8B Instruct, instruction-tuned models with 2 and 8 billion parameters. The Granite 3.0 open source models are also available through platform partners such as Google Vertex AI and Hugging Face.

Llama is Meta’s collection of open source AI models. The Llama 3 models, particularly the 3.1 versions, can be accessed through the APIs of Meta’s various ecosystem partners (link resides outside ibm.com).

Meta also released Llama Stack (link resides outside ibm.com) to streamline the development and deployment of AI apps built on top of Llama models. Llama Stack consists of a set of interoperable APIs for agents, inference, memory and safety, among others.

Mistral AI has different API endpoints (link resides outside ibm.com) for its premier models—such as Mistral Large, Mistral Small and Ministral—and free models, including Mistral NeMo and Mistral 7B. The company also offers a fine-tuning API. The Mistral API can be accessed through its own La Plateforme development platform and partner platforms such as IBM watsonx and Microsoft Azure AI.

OpenAI, the company behind ChatGPT, provides APIs for its multiple models (link resides outside ibm.com). These APIs include its latest generative pretrained transformer (GPT) models GPT-4o and GPT-4o mini, and older OpenAI GPT models such as GPT-4 Turbo and GPT-3.5 Turbo.

OpenAI’s text generation models employ a chat completion API endpoint, but other APIs include an Images API for OpenAI’s image model, an Audio API for its text-to-speech model and a Realtime API for low-latency applications. Developers can access the OpenAI API through the OpenAI platform and software development libraries in various programming languages.

LLM APIs play a vital role in the AI pipeline. By combining the reasoning power of LLMs with the usability of programmed interfaces, LLM APIs bridge the gap between large language models and enterprise applications. Understanding the inner workings of LLM APIs and how to use them efficiently can help businesses better blend AI into their systems.