About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Home
Case Studies
IBM Big AI Models - Turbonomic
Optimizing GPUs for gen AI
Saving critical resources with IBM Turbonomic
Maximizing high-demand GPUs for efficiency and performance
Artificial intelligence has reached a crucial milestone, where training large language models (LLMs) is one of the most computationally demanding tasks. High-performance computing is essential for generative AI (gen AI) and LLM workload optimization, and Graphics Processing Units (GPUs) can be expensive and scarce. GPUs are specialized computer chips designed to handle complex mathematical calculations and parallel processing, making them ideal for complex computations required in training and inference for deep learning models. As a result, GPUs are in high demand, and optimizing their utilization is critical for AI success.
The IBM® Big AI Models (BAM) team, that supports the primary research and development environment for engineering teams to test and refine their gen AI projects, saw an opportunity for improvement. As more projects went through the testing stage, the team recognized the importance of optimally using each instance to avoid resource wasting.
Transforming GPU management: from chaos to control
To optimize their GPU resources and manage their LLM Kubernetes instances, the IBM BAM team deployed IBM Turbonomic®, an advanced application resource management software tool. Using real-time data, Turbonomic generated AI-driven recommendations for automated actions to optimize resource utilization and efficiency. By identifying optimal resource allocation strategies, the solution generated tailored suggestions that the team could configure to execute automatically, enabling AI-driven resource optimization.
As an IBM internal software explicitly tailored for optimizing hybrid cloud management, including containerized applications, virtual machines and public clouds, IBM Turbonomic provided seamless integration within the existing infrastructure.
Tom Morris, AI Platform Researcher, summarizes: “Enabling Turbonomic to scale up and down our LLM inference servers has allowed me to spend less time monitoring performance.”
BEFORE
AFTER
Better performance, reduced costs: the outcomes of efficient GPU resource allocation
With Turbonomic, the IBM BAM team was able to create a scalable and agile infrastructure that could adapt to the evolving demands of their business, supporting their LLM services and running over 100 NVIDIA A100 GPUs.
By scaling down overprovisioned instances, the team show the ability to increase the idle GPU resources from 3 to 16 (5.3 times), to allow those resources to handle additional workloads.
The results included:
- Resource allocation
With the automated solution, dynamic scaling became second nature, generating optimal utilization of available GPUs according to varying needs.
- Cost efficiency
Scaling LLM services on demand allowed time-sharing of GPUs, optimizing the total number of GPUs required. Now, with scaling and sharing, IBM BAM team showed 13 fewer GPUs will be necessary in a full automation environment.
- Labor efficiency
Automatic scaling of LLM inference servers allowed the IBM BAM team to spend less time monitoring performance.
- Scalability and performance
After fully automating the scaling of LLM services, the originally over-provisioned GPU resources were freed up to be shared by other workloads based on demand. The throughput increase gives the opportunity to enhance performance by addressing latency issues.
Applying Turbonomic automation capabilities, the IBM BAM team successfully scaled and optimized LLM services. This improvement positioned the team to reallocate their time for strategic projects.
The IBM Big AI Models (BAM) team is a group of researchers and engineers within IBM Research® that focuses on developing and applying large-scale AI models. These models are designed to process and analyze vast amounts of data, enabling applications such as natural language processing, computer vision and predictive analytics.
Legal
© Copyright IBM Corporation 2024. IBM, the IBM logo, Turbonomic, and IBM Research are trademarks or registered trademarks of IBM Corp., in the U.S. and/or other countries. This document is current as of the initial date of publication and may be changed by IBM at any time. Not all offerings are available in every country in which IBM operates.
Client examples are presented as illustrations of how those clients have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.