Monitoring AI host platforms

The AI host platforms monitoring, such as vLLMs and GPUs, involve tracking key performance indicators (KPIs) to make sure the usage of optimal resources and application health. This tracking includes monitoring GPU memory and compute usage, and model inference latency. Real-time insights into these metrics enable proactive issue detection, performance tuning, and efficient scaling of AI workloads. Effective monitoring ensures stable and efficient deployment of AI models.

Instana monitors AI host platforms and provides real-time visibility into GPU usage, vLLM performance, and containerized environments. AI host platforms offer granular insights into inference latency and resource consumption that enable proactive performance optimization and rapid issue resolution. Instana's AI-powered analytics correlate metrics, traces, and logs for comprehensive troubleshooting.

Following are the supported AI host platforms:

• vLLM
• GPU