New Relic integrates with NVIDIA NIM inference microservices to assist ship high-performing fashions optimized for NVIDIA GPUs
New Relic, the all-in-one observability platform for each engineer, introduced it’s integrating its platform with NVIDIA NIM inference microservices to scale back the complexity and prices of creating, deploying, and monitoring generative AI (GenAI) apps. Now, clients can use New Relic AI monitoring to realize complete visibility throughout the AI stack for functions constructed with NVIDIA NIM, all with a simplified setup and whereas making certain knowledge safety. This enhances the strong safety features and ease of use of NVIDIA NIM’s self-hosted fashions, which accelerates generative AI utility supply. Collectively, New Relic built-in with NVIDIA NIM may help clients undertake AI quicker and obtain faster ROI.
Observability is Mission Crucial to Deploying Price-Efficient, Excessive-Efficiency Fashions
Organizations are quickly adopting generative AI to reinforce digital experiences, enhance productiveness, and drive income. Gartner predicts that over 80% of enterprises will use GenAI or deploy GenAI apps by 2026. Fast deployment and quicker ROI are essential for organizations to realize market benefit and Observability is the important thing. It affords a holistic, real-time view of the AI utility stack – throughout providers, infrastructure, and the AI layer – to make sure environment friendly, dependable, and cost-effective operation.
“In as we speak’s hyper-competitive market, organizations can not afford to attend years for AI ROI,” mentioned New Relic CEO Ashan Willy. “Observability solves this by offering visibility throughout the AI stack. We’re pioneering AI observability by extending our platform to incorporate AI apps constructed with NVIDIA NIM. Combining NVIDIA’s AI expertise with our experience in observability and APM provides enterprises a aggressive edge within the AI race.”
“As enterprises race to undertake generative AI, NVIDIA NIM may help companies shortly deploy functions in manufacturing,” mentioned NVIDIA Director of AI Software program Amanda Saunders. “New Relic’s integration with NVIDIA NIM allows IT and improvement groups to optimize their AI functions by quickly observing and responding to operational insights.”
New Relic Speeds Sooner ROI for AI Functions Constructed with NVIDIA NIM
AI functions can complicate tech stacks, improve safety considerations, and be value prohibitive. New Relic AI monitoring offers a complete view of the AI stack, together with key metrics on throughput, latency, and prices whereas making certain knowledge privateness. It additionally traces the request flows throughout providers and fashions to know the internal workings of AI apps. New Relic extends its in-depth monitoring to NVIDIA NIM, supporting a variety of AI fashions together with–Databricks DBRX, Google’s Gemma, Meta’s Llama 3, Microsoft’s Phi-3, Mistral Giant and Mixtral 8x22B, and Snowflake’s Arctic. This helps organizations deploy AI functions constructed with NVIDIA NIM confidently, speed up time-to-market, and enhance ROI.
Key options and use circumstances for AI monitoring embrace:
- Full AI stack visibility: Spot points quicker with a holistic view throughout apps, NVIDIA GPU-based infrastructure, AI layer, response high quality, token depend, and APM golden alerts.
- Deep hint insights for each response: Repair efficiency and high quality points like bias, toxicity, and hallucinations by tracing all the lifecycle of AI responses
- Mannequin stock: Simply isolate model-related efficiency, error, and value points by monitoring key metrics throughout all NVIDIA NIM inference microservices in a single place
- Mannequin comparability: Examine the efficiency of NVIDIA NIM inference microservices operating in manufacturing in a single view to optimize mannequin alternative based mostly on infrastructure and person wants.
- Deep GPU insights: Analyze vital accelerated computing metrics akin to GPU utilization, temperature, and efficiency states; perceive context and resolve issues quicker.
- Enhanced knowledge safety: Along with NVIDIA’s self-hosted mannequin’s safety benefit, New Relic lets you exclude monitoring of delicate knowledge (PII) in your AI requests and responses.
New Relic deepens its 60+ AI integration ecosystem with NVIDIA
This integration follows New Relic’s latest addition to NVIDIA’s AIOps partner ecosystem. Leveraging NVIDIA AI’s accelerated computing, New Relic combines observability and AI to streamline IT operations and speed up innovation via its machine studying, and generative AI assistant, New Relic AI. New Relic affords probably the most complete observability answer with 60+ AI integrations together with NVIDIA GPUs and NVIDIA Triton Inference Server software program.
Join the free insideBIGDATA newsletter.
Be part of us on Twitter: https://twitter.com/InsideBigData1
Be part of us on LinkedIn: https://www.linkedin.com/company/insidebigdata/
Be part of us on Fb: https://www.facebook.com/insideBIGDATANOW
