The power of small

Large language models (LLMs) have dominated the AI conversation, fueled by the popularity of ChatGPT and similar gen AI apps. However, small language models (SLMs) are on the rise. More compact and efficient, SLMs use less memory and processing power. This makes them well-suited for resource-constrained environments.

Their ability to produce unique content and insights from vast datasets has captivated the world and spurred new tools and apps, further cementing their cultural impact, especially in the enterprise. With Dr. Juan Bernabe Moreno, Director of IBM Research® Europe for UK and Ireland, we’ll explore how SLMs can benefit enterprise AI adoption.

Generative artificial intelligence (gen AI) holds significant productivity potential for enterprises. However, when implementing AI models, bigger isn’t always better. There are 2 major LLM issues preventing enterprise adoption. 

First, many LLMs are general-purpose, limiting their value. “Average” company models are ineffective because no “average” company exists. Effective AI models must be built, tuned and deployed to specific organizational needs. 

Second, many proprietary LLMs are ‘black boxes’—a closed model in which only the company that owns it can see the components of—lacking data transparency and hindering tuning with enterprise data, where AI’s true value lies. This leaves enterprises responsible for model performance without insight or control. 

These issues prevent trust and understanding of model safety. Without effectively choosing gen AI solutions to meet industry, legal and regulatory requirements, companies can’t fully use gen AI’s power.  

For successful gen AI, enterprises need 3 things:

1. Efficiency in performance and cost: Models need to be cost-effective for scaling. Larger models with more parameters and tokens are proving too expensive to run. Efficient SLMs offer companies AI models that only contain data that is relevant to the specific tasks the model supports, saving cost and energy.
   
   
2. Transparent data: Businesses need control over their AI. Accessible training data can help ensure accurate, trustworthy results. 
   
   
3. Flexibility and choice: Organizations need options for gen AI solutions that address their unique use cases. They want to build AI based on models that can easily be adapted to new scenarios and use cases. SLMs offer this flexibility.

To meet these needs and maximize AI value, enterprises are turning to SLMs and discovering the power of small. SLMs offer a compelling alternative to general-purpose LLMs. 

SLMs have received far less attention and fanfare, but they offer a compelling option for organizations of all sizes. Their energy efficiency, data transparency and strong performance—often matching or exceeding larger models—unlock responsible gen AI adoption without hindering innovation. 

Generally, anything under 30 billion parameters is considered an SLM. Key advantages include lower costs, reduced energy consumption and improved data transparency and integrity. A new generation of smaller models, such as IBM® Granite™, built on cleaned, filtered datasets for specific tasks, reduce risks such as bias and inappropriate output while increasing data visibility. 

This trustworthy base model enables confident integration of proprietary data, unlocking AI’s true value. IBM provides an intellectual property (IP) indemnity for all Granite models further boosting confidence in merging their data with the models.

IBM believes open-source base models empower organizations to create specialized, data-infused models. To support this, IBM open sourced its Granite family of customizable SLMs, trained on transparent, filtered datasets.

Combining a small Granite model with enterprise data can achieve task-specific performance rivaling larger models at a fraction of the cost. Early proofs-of-concept show IBM Granite models costing significantly less (between 3 and 23 times less) than large frontier models, while outperforming or matching similarly sized competitors on key benchmarks.

Also, new techniques such as InstructLab—introduced by IBM® and RedHat® in May 2024—simplify enterprise data infusion into LLMs. InstructLab enables enterprises to customize AI models using far less human-generated information and computing resources than traditional retraining.

SLMs, including Granite models, are already making an impact. Global sports institutions use Granite models, tuned with their own domain data, to enhance fan experiences with AI-generated commentary. Internally, IBM uses Granite models to power its human resources (HR) service platform, AskHR. By using natural language prompts, IBMers can access HR services in one place, saving time for both employees and HR professionals. 

Vast LLMs are not the only way to benefit from gen AI. Smaller, more accessible, specialized models offer needed efficiency, trust, flexibility and performance at a lower cost, financially and environmentally.