AI Redefines The CIO As A Chief Enterprise Orchestrator

The push to integrate AI in almost every facet of an organization is changing the CIO role into that of a companywide orchestrator—harmonizing AI models, tools and people.

AI is not just changing how we work; in some cases, it’s shifting traditional IT roles.

As AI proliferates across enterprises, the CIO role is rapidly evolving from one of a technology enabler to an enterprise orchestrator.

AI orchestration is such a big, nebulous topic, but it can be broken generally into several bite-sized pieces. It comprises governing AI autonomy, preparing data readiness, and executing AI projects at scale.

“That’s really like the origin of this word ‘orchestration’ because it’s really all about a number of things,” said Mary Jesse, an engineer and the founder and CEO of ACME Brains—a Seattle-based company that develops and commercializes a proprietary AI personal context engine.

“Who’s doing what role? Which instruments are you bringing to bear, and what exactly are they doing, and when are they doing it? What is the sequence of what they’re doing? And when you start this, when you stop it, you know, how is it made complete? So that’s exactly the components of orchestration,” she said.

Just as the conductor of the Philharmonic guides the various sections of the orchestra into a harmonious symphony, today’s CIO must harmonize AI models, tools and people to work cohesively—setting a tempo of sorts.

It’s a daunting task as different departments have various AI tools, demands and goals. While orchestration today is mostly a focus for larger enterprises, that is rapidly changing as midmarket organizations, especially ones in heavily regulated industries, are going through their own orchestration pain points and as CIOs struggle to control the very AI they are enabling for their end users.

The 80 Percent Automation Goal

The first leg of many small to midsize enterprises’ orchestration journeys begins with automation. Some opt to begin with automating customer service.

“[For] smaller to midmarket organizations, a good place to start is customer support. We worked with a fairly small company in the U.K., and they’re a utility. They provide water to commercial clients. And they have, obviously, customer support; there’s no water, or there’s an issue with the water, or billing, or all kinds of different problems,” Daniel Meyer, Camunda’s CTO, said in an interview with MES Computing.

“We want to get to a place where AI does 80 percent ... of the work, things like data mapping, validation, transformation, the stuff that is the low-level, tactical work that humans are doing today, but AI that humans can now coach AI on, so a human does the work, passes it to AI and then AI automates it,” Michael Bevilacqua, vice president of AI product management at Adeptia, told MES Computing.

Governance Versus Orchestration

Where things get tricky for IT leaders is the leap from deploying AI to perform fairly manageable, isolated tasks—transcribing an online meeting, for instance—to governing autonomous AI agents. Such agents could perform tasks ranging from identifying security anomalies to scheduling sales follow-ups.

Yet, introducing autonomous AI demands a skilled AI enterprise orchestrator because of the risks possible. Opening the door to threats, disclosing protected company IP, or passing hallucinated information to customers are just a few of those risks.

One of the more frustrating aspects of managing AI is sifting through the buzzwords and marketing jargon that accompanies so many vendor pitches and promises of AI. There also seems to be a marketing overlap as to what constitutes AI governance versus orchestration.

AI governance and orchestration are “really different things,” Jesse said. Governance, she explained, defines AI rules and policies.

Governance “is really the guidelines. It states the rules of the game,” she said. Orchestration, on the other hand, is “tactical.”

“It’s actually implementing the rules of the game,” she added.

The midmarket needs “governance that scales with them as they grow their AI strategy,” Emre Kazim, co-founder and co-CEO of Holistic AI, told MES Computing in an email statement.

“I don’t think AI governance is any different than security governance, than performance governance ... governance done right is an enabler. It’s not a control function. The point of the goal of governance is to move faster safely, not to slow things down,” Bevilacqua said.

“Orchestration is really the underlying principle of what is different about an agent is that you are not just orchestrating an interaction with an LLM, but now you’re orchestrating a whole system,” Jesse said.

How Midmarket Leaders Can Become AI Enterprise Orchestrators

Experts say one of the first ways to become a successful AI orchestrator is getting your organization’s data AI-ready.

“Ninety percent of the world’s data is unstructured. It’s messy. It’s dirty. It doesn’t follow a schema. It’s stored all over the organization. It’s stored in people’s emails. How do we get that data into a place that can be utilized in a central knowledge base of the company?” Bevilacqua said.

Adeptia, he said, offers an intelligent data mapper that uses AI to access the source of data.

“Maybe it’s a PDF, here’s an email, maybe it’s a Word doc, maybe it’s code, and then we have a target schema that we say we want to standardize and normalize that data into a common data model that is across the organization.”

That doesn’t mean necessarily asking the CFO or board for a budget to go out right away and buy an AI orchestration platform.

Instead, setting up policies from the start of AI implementation is a good way to begin.

“You have to figure out what LLM [to use] and how you’re going to handle it. And then you have to make sure you have boundary conditions ... possibly going in, possibly going out. So, there’s a number of steps that have to happen to have AI sort of just function at the base level,” Jesse said.

Implementing an “enterprise data strategy” is key, she said.

“That’s what underlines everything. Because that’s what you might be feeding into AI. AI might be feeding it back out to you. You have your corporate data, you have your customer data, you have all this data, right? So, it always comes back to the data, and it’s leveraging that data. That’s what AI really helps you do—leverage your data better and automate to make it more efficient. It’s sort of a hyped-up version in a lot of ways of what we’ve lived through in machine learning ... all of that data—cleansing and data lakes and all the stuff that led up to, just before LLMs were launched, that stuff’s all still applicable. It’s just now you have this front-end tool that allows you to just kind of turbocharge making use of all of that. But all of that still is important,” she added.

Midmarket tech leaders overall seem to be dancing cautiously with AI.

“Orchestration in general is a great idea given the amount of tools that are available within a given workspace, and it ensures that there’s a consistent experience for all staff members,” Marques Stewart, vice president at Achievement First, said in a comment to MES Computing.

“As for AI orchestration, I would be wary of that because sometimes AI can make logic leaps that seem helpful but are just areas that you have not yet considered—so definitely consider having some sort of ‘stopgap’ in place if there’s a chance the orchestration could wreak havoc,” Stewart added.

For midmarket IT leaders, focusing on making the organization’s data AI-ready, clearly defining goals and purposes for the use of AI across departments, and setting strong AI policies even before entering into any contract with a vendor are strong initial steps to leading as an enterprise AI orchestrator.