Db2 Genius Hub: Trusted on prem AI inferencing with Intel Gaudi 3
IBM Db2 Genius Hub is an AI-Powered console experience designed to bring intelligence and autonomy directly into database operations.
As enterprises move from experimenting with AI to operationalizing it at scale, one reality becomes clear: AI must work reliably where enterprise data lives. For many organizations that means running AI models locally more closer to production systems, tightly governed and built for consistency rather than compromise.
In this context, AI inferencing plays a critical role. Inferencing is where AI delivers real value interpreting signals, generating insights and guiding action in real-time. For mission critical systems like enterprise databases, inferencing performance directly impacts operational confidence and user experience.
This is the foundation on which IBM Db2 Genius Hub delivers AI-driven autonomy and where Intel Gaudi inferencing has helped elevate the on-prem AI experience.
IBM Db2 Genius Hub is an AI-Powered console experience designed to bring intelligence and autonomy directly into database operations. It continuously correlates performance signals and operational context across the Db2 estate, helping teams quickly understand what changed, why it happened and what to do next through optimal, expert-backed recommendations.
At the core of this experience is AI inferencing. Every recommendation, explanation and insight generated by Db2 Genius Hub relies on the ability to efficiently run AI models against live operational data often across complex, production scale environments.
Delivering this capability on-prem places demanding requirements on performance, responsiveness and scalability. AI must operate fast enough to be actionable, consistent enough to be trusted and efficient enough to integrate seamlessly into existing enterprise infrastructure.
This is where Intel Gaudi inferencing plays a key role as inferencing servers. By leveraging Gaudi to accelerate AI inferencing workloads, Db2 Genius Hub strengthens how AI operates across on-prem air-gapped Db2 environments. Gaudi’s multicard architecture and RedHat vLLM engine, combined with support for running concurrent instances of LLM, enables efficient distribution of inferencing workloads and seamless handling of high request concurrency. This design helps maintain optimal latency for each request, delivering more responsive, consistent, and elevated user experience for Db2 Genius Hub in on-prem deployments.
These improvements directly supports Db2 Genius Hub’s mission to reduce manual overhead and shift teams from reactive firefighting to proactive database management while maintaining transparency, explainability and control in production environments.
Delivering a trusted, high-quality AI experience in on-prem environments requires a close collaboration across technology layers and teams. It requires continuous validation against real workloads, real concurrency and real user experience.
As part of the collaboration between IBM and intel, Db2 Genius Hub served as a practical, real-world validation ground for Intel Gaudi inferencing. The focus was not just on raw performance, but on how inferencing behaves under realistic enterprise scenarios where multiple requests arrive concurrently, contextual understanding matters and response quality must remain consistent.
A key part of this effort involved evaluating Db2 Genius Hub’s AI agents using Gaudi as the inferencing server. This included running exhaustive stress tests and domain specific evaluations across scenarios such as contextual search and tool-calling workflows mirroring how users interact with the system in production. These tests were executed across varying levels of concurrency to understand how inferencing performance and response quality held up under load.
For each evaluation, detailed response data and scoring was captured and shared with intel team. This feedback loop made it possible to identify subtle issues ranging from response consistency to quality degradation under higher concurrency and address them iteratively. Improvements were then re-tested, measured again and refined further.
This collaboration underscores the value of an ecosystem-led approach to AI innovation. By aligning hardware capabilities with real enterprise use cases, IBM and Intel were able to advance Gaudi’s readiness for production AI ultimately delivering a more responsive, dependable and enterprise-grade on-prem AI experience.