Innovation Wants to Experiment. Regulation Wants To Be Certain.
There is no shortage of excitement around enterprise AI. New capabilities, new possibilities, another opportunity to innovate. But when it comes down to brass tacks, organizations in regulated industries are wrestling with a more practical question: how do you experiment with a technology that's still evolving, in an environment that leaves little room for error?
THE SETUP
I was recently part of a pilot to develop a customer-facing AI assistant in a highly regulated space. The vision was compelling: trained on proprietary company knowledge and data, supporting customers with accurate, trusted information in an interactive way. We had an engaged cross-functional team, enthusiastic stakeholders, a well-defined customer journey, and clear legal and regulatory guardrails to guide the development.
On paper, every ingredient for success was there.
What we underestimated was the tension between the way AI learns and the way regulated organizations operate.
WHERE IT GOT HARD
The first outputs were rough - noticeably off in places. The model needed to learn. But with more training, the model improved quickly, and the team could see real progress within just a few cycles.
The harder truth showed up later. For the experts responsible for the underlying content, "quite good" wasn't the bar - it was 100% accuracy, full stop. And that last stretch, from good to fully reliable, proved elusive. It took far longer than any of us expected, and none of us could say with confidence whether we'd get there at all.
As confidence in the pace of progress wavered, so did enthusiasm for investing the time needed to keep training the model. The project reached an uncomfortable point: we believed the system could improve, but couldn't confidently say how much effort it would take, how long, or whether the required performance was even achievable.
THE REAL LESSON
Looking back, it wasn't a technology problem. It was an expectations problem.
Traditional software projects, even the most agile, move toward a known destination. AI projects don't work that way. Part of the work is discovering what the technology can do, how your data behaves, and what performance is realistic. Learning isn't a by-product of the project. It's one of its primary outcomes.
That uncertainty asks for a different mindset from everyone involved. It also changes the role of subject matter experts: their contribution can't be limited to validating a finished product. They have to become active participants in shaping it - which takes time, patience, and trust, especially when the payoff isn't yet guaranteed.
WHAT I’D DO DIFFERENTLY
If I approached a similar pilot again, I wouldn't necessarily choose different technology. But I would change the timeline.
We scoped this as a fast pilot, because we all believed the development would be fast. It was. What we didn't account for was that the refinement would be slow — and that refinement, not development, was where the real work lived. Next time, I'd double the time we budgeted, not for building the model, but for the iterative cycle of testing, feedback, and trust-building with subject matter experts that came after.
I'd also spend more time preparing the organization for the nature of AI experimentation itself: involving domain experts earlier, building shared expectations about what early outputs will look like, and being upfront that progress gets measured in learning before it's measured in performance.
In the end, the organization decided not to move forward with further refinement. But the work wasn't wasted. The model we'd built will become a foundation for other applications, and the team walked away with a much clearer picture of what AI experimentation actually requires.
Perhaps that's the paradox of enterprise AI. Innovation depends on experimentation. Regulation depends on certainty. The organizations that succeed won't simply have the best models - they'll be the ones that learn how to create space for experimentation while holding onto the trust and rigor their industries demand.