AI Insights

By Joe Wolski, CTO, Godel.

It’s very easy to make a series of seemingly sensible AI decisions in isolation, only to look up 18 months later and realise you’ve accidentally created a complex ecosystem – one that’s become difficult to govern and scale. But that’s exactly what organisations are doing.

It’s happening because different people in an organisation are experimenting with AI, so while one team introduces an AI coding assistant, another might have begun experimenting with agentic workflows. Somewhere else, an internal AI gateway gets built, and a few open-source models are deployed, all of which leads to different orchestration frameworks appearing across the business. Each one solves a slightly different problem and in isolation, they look like sensible decisions. However, collectively, a sprawling AI estate is beginning to form, not unlike the kind of operational sprawl the industry creating during the DevOps and cloud tooling era, and which then took years to simplify.

In regulated industries, the challenge is making the technology governable and easy to support across the enterprise environment without accidentally creating another layer of operational sprawl. The difficult part of agentic AI isn’t necessarily the model itself, it’s everything surrounding it. Governance, permissions, auditability, orchestration, security controls, and accountability suddenly become just as important as the intelligence layer underneath. Complexities grow quickly and in heavily regulated markets like banking, insurance and healthcare, that growth must be compliant.

Every independently adopted framework creates another integration point, governance gap and operational dependency that the wider organisation eventually has to absorb. The industry has already lived through a version of this once before with DevOps – nobody intentionally designed fragmented toolchains, but over time businesses accumulated different deployment frameworks, testing tools, orchestration layers and security processes because individual teams made rational local decisions. Years later, organisations found themselves spending more time managing the ecosystem than delivering value through it.

Governance hasn’t caught up

As it stands, agentic AI risks creating the same problem – only much, much faster. AI adoption also feels different from previous technology shifts because most organisations don’t feel they have time to move cautiously. Boards expect AI strategies now, investors want productivity gains and the competition is already redesigning products and services around AI-enabled operating models. This means CTOs are being pushed towards tactical adoption before the organisation has properly thought through the longer-term operating model implications.

A lot of organisations have already spent heavily on AI tooling over the last 12 months, yet many CTOs still aren’t seeing the operational gains they expected. The assumption was that buying more AI tools would automatically make software delivery dramatically faster, but the tooling itself was never really the bottleneck.

Raw development speed is becoming commoditised because with AI, almost anyone can now generate code quickly, so in many ways, speed is already solved. The real differentiator is how efficiently you use the models, how you govern them, how you assure the quality of what gets produced and whether you can genuinely support, secure and underwrite those outputs at enterprise scale.

That’s also why many CTOs are becoming increasingly frustrated with the current tooling market because spending significant money on AI platforms still doesn’t guarantee faster delivery or higher quality outcomes. The market is full of noise around agentic engineering and AI-enabled delivery, but underneath that noise, organisations are still trying to work out how to operationalise AI safely.

Build or buy is no longer a straightforward decision

Historically, enterprise software leaned heavily towards buying platforms because it was faster, cheaper and lower risk than building internally. AI changes that dynamic because the closer intelligence gets to customer experience, operational workflows and decision making, the more valuable proprietary context becomes. Your data, processes and knowledge matter, which creates a stronger argument for bespoke capability.

However, bespoke capability and building an internal AI platform are not the same thing.

If organisations choose to build internally, they’re not simply deploying AI capability, they’re effectively building themselves into a permanent platform engineering responsibility. They need teams building orchestration layers, governance frameworks, internal IP and operational controls, and then they need to maintain it indefinitely because the ecosystem itself keeps evolving underneath them.

Buying heavily packaged AI platforms creates its own frustrations too – businesses often end up paying significant premiums for tooling that limits flexibility around model choice, workflow design and operational control, while still having no guarantees that delivery outcomes will materially improve. Many CTOs are now discovering they’ve spent heavily on AI tooling over the last 12 months and yet nothing actually feels faster.

The choice is build internally and inherit long-term complexity, or buy externally and risk paying significant premiums for tooling that still doesn’t fully solve the operational challenge underneath. For most businesses, there’s very little strategic value in owning foundational AI infrastructure themselves. Governance tooling, orchestration layers and operational frameworks are becoming commodity capabilities in the same way cloud infrastructure eventually did. The advantage comes from what you build on top, and that’s probably the more sensible way to think about AI too.

The organisations moving fastest at the moment are generally separating foundational capability from differentiating capability. They’re buying the layers that standardise, govern and operationalise AI safely across the business, while focusing internal effort on the workflows, products and intelligence that genuinely create competitive advantage. The ones struggling are often trying to do both at the same time.

Every engineer assigned to building internal orchestration capability is an engineer not modernising legacy systems, reducing technical debt or accelerating customer-facing innovation elsewhere in the business. There’s always an opportunity cost attached to platform engineering, particularly in large enterprises where technical debt already consumes significant delivery capacity.

That doesn’t mean building internally is always wrong, some organisations absolutely have the scale, technical maturity or regulatory requirements to justify it. But I do think many CTOs are underestimating the operational burden they’re signing up for because once you become the owner of an internal AI platform, you inherit the long-term governance, compliance and lifecycle management obligations that come with it. And unlike traditional enterprise technology stacks, the AI ecosystem itself is evolving monthly.

That’s why the bigger risk for many organisations probably isn’t moving too slowly on AI anymore – it’s creating an AI operating environment so complex they spend the next decade trying to untangle it.