Nutanix Exec On The Challenges Of Integrating GenAI Workloads

IT leaders have made AI integration a priority but deploying AI in existing environments comes with a slew of challenges.

(Lee Caswell, Senior Vice President of Product Solutions Marketing , Nutanix)

Seventy-eight percent of business leaders surveyed in a 2024 Q4 report from Deloitte said they expected to increase their AI spending this year.

Implementing generative AI will account for a large position of that budget, according to those surveyed for Deloitte’s The State of Generative AI in the Enterprise report.

However, there are pain points organizations face implementing GenAI solutions: worries about regulations, concerns over having tech staff skilled enough in AI, and lack of a solid adoption strategy are some of the challenges leaders cited in the report.

MES Computing spoke with Lee Caswell, senior vice president of product solutions marketing at Nutanix, about the No. 1 issue organizations, including those in the midmarket, have with adopting GenAI—its implementation.

What are the biggest hurdles for the midmarket with implementing GenAI into their existing infrastructures?

Midmarket customers are looking at GenAI. They’ve got a couple of big questions.

First, when you see the level of investment required to build what’s called the large language model that you would use, it becomes obvious really quickly that they’re not doing that. They’re relying on LLMs that are built by either the hyperscalers or companies like Anthropic or Meta.

And so, it becomes: I’m going to either take a large language model, which I could run myself, or there’s certainly SaaS-level services, right? You’ve got things like Microsoft Copilot and ServiceNow.

But ... if you think about, how can you use generative AI, either to go and improve your internal operations or to build more sticky external customer relations? How would I do that as a midmarket company when I’m concerned about the privacy of my data, and so this becomes really interesting to think for a lot of these customers now of how can I go and take my infrastructure that I’m familiar with running and how would I make it generative AI-ready?

How is Nutanix helping those customers with integrating GenAI?

There are certain things that you expect in a generative AI use case. One is they’re generally running not just on CPUs, but they need GPUs. And so, the idea that all of our partners that we’re working with today have certified GPU-enabled servers. So from Dell, Cisco, HPE, Lenovo, others, you can get GPU-enabled servers, and that’s really important, because you’d have all the GPUs which are changing relatively quickly from Nvidia today and possibly others in the future. You can get access to those, and we can help you size those for the particular application that you’d want.

Number two is for a lot of these folks the large language models are also changing relatively quickly. Nvidia has a set themselves called the Nvidia libraries. Hugging Face has a million models that you can choose from. You can choose public models like Llama 2, Llama 3, Mistral ... What we’ve done at Nutanix is we’ve built what’s effectively like an app store for LLMs. You can go pick the LLM you want to deploy on your GPU-enabled clusters, and then if you want to go change it over time, you can go change it and you have you’ve never bought the wrong thing. If you want to do a video application ... a fraud detection system, or you want to build a support copilot ... for a lot of [the] midmarket, probably the biggest question to ask first and/or to answer first is, what’s the application that I want? And how could AI help?

Then there’s the last piece of this, which is for a lot of these LLMs because they’re new. They’re based on containers, and this can be new for midmarket companies [that may not] have separate container teams or Kubernetes. With Nutanix, you can take our Nutanix Kubernetes platform, run Kubernetes and containers just the same way you run [virtual machines] today.

What is the difference that Nutanix is offering from let’s say a customer would get from an Azure or an AWS? Is it that hybrid scenario where they can have some infrastructure on-prem and some in the cloud?

Although the LLMs come from the public cloud, a lot of these companies are going to want to run them in their private data center, so [they aren’t] helping train a public model. So, there’s a policy aspect that’s a core element.

Another element is, even in the midmarket you may have different branches, for example, that you want to have running and so the edge starts playing into [that]. Maybe you have four different manufacturing facilities. You’re like, “Oh, I want to go and run AI in each one of those.” And relative to the cloud, we have a model that can extend all the way from the edge to a data center into the public cloud. And here’s a really important distinction, is you can scale our system down really effectively.

Also ... you can run when you’re disconnected from the cloud, and Nutanix is very good for that, so you don’t have to have an active cloud connection in order to run like a local manufacturing facility or local retail outlet.