Why AI Agents Need Ontologies to Avoid Going Off the Rails

AI has made software delivery faster, but speed alone does not guarantee better outcomes. As teams adopt AI agents to drive software development, the real challenge is keeping requirements, context, specs, and compliance aligned, so the output doesn’t drift from original intent. Ontologies give structure to what an agent knows, making them the AI agent guardrails that hold against hallucination, drift, and infinite loops. They provide a comprehensive view of the existing system, making the blast radius and full impact of any change immediately clear.

Why AI-assisted development still breaks down

Only 39% of technology leaders are confident in the financial impact of their current AI investments, according to a Gartner survey of 353 data and analytics and AI leaders. The investment curve and the results curve have stopped moving together.

I watched Dr. Frank Coyle, a longtime computer science educator now at UC Berkeley, present at the AI Engineer World’s Fair this year. His question was simple. As AI agents start perceiving, deciding, and acting on their own, what actually stops them from drifting into nonsense, issuing a duplicate refund, looping forever, or attempting something that’s semantically impossible in the real domain? Coyle’s answer traced back to a single gap. Agents lack a shared, structured understanding of the domain they operate in, so they default to guessing at meaning instead of knowing it.

He tied that risk directly to how agentic systems behave once they start looping. “Loops can drift as agents start talking to each other,” he said, warning that unchecked iteration burns tokens and budget without producing anything usable.

Most teams have tried to patch this gap with vector databases and larger context windows. Embedding everything and retrieving by similarity looks impressive in a demo. It rarely holds up once a system needs to run reliably for months. Agents need more than raw data to draw from. They need definitions, relationships, and a shared understanding of what customers, products, policies, and processes actually mean. Without that foundation, an agentic system can sound intelligent while still being fundamentally unreliable. That gap is what ontologies close.

What an ontology actually is

In 1993, Tom Gruber defined an ontology as “an explicit specification of a conceptualization.” In practice, that means a structured, explicit definition of what the concepts in a domain actually mean and how they relate to each other. A large language model has no innate concept of any of it, because all it’s really doing is predicting the next most probable word.

Take something as ordinary as a customer record. A sales team calls that person a prospect. Billing calls them an account. Support calls them a user. Each system is technically correct within its own definition, and each one is also blind to the other two. An ontology settles that ambiguity once, so every agent touching that record works from the same definition instead of three competing ones.

What breaks without AI agent guardrails

An agent, stripped down to its mechanics, is a loop. It generates a response, checks whether a tool is needed, runs it, feeds the result back in, and repeats. That loop is what makes agents powerful, as well as risky. Coyle put it plainly: “Loops give us the last piece in the equation,” the piece that turns a language model into something that can, in principle, compute anything a machine can compute. Power and risk arrive in the same mechanism.

I’ve watched that mechanism go wrong in the same handful of ways, across very different enterprises.

1. The first is the loop that never resolves. An agent retries a step, doesn’t get the result it expects, retries again, and keeps going. So, token counts crank up as the loops continue, and the business rarely notices until the final bill comes in.
2. The second is quieter and more dangerous. An agent completes an action that’s technically valid and operationally wrong. It approves a purchase order against a vendor contract that expired the week before, because nothing in its context told it the contract had lapsed. The mistake only surfaces later in a procurement review.
3. The third shows up the moment you ask why. An agent recommends one vendor over another, and when a human asks for the reasoning, it can only point back to the data it retrieved, not the relationships that should have shaped the decision. It isn’t hiding anything. It never had the structure to reason with in the first place.
4. The fourth is the one that should worry enterprises most. Agents that lack an ontology can’t check whether an action is allowed before they take it. They can only attempt it and see what happens. In a chat interface, that’s an awkward answer. In a system that touches real transactions, that’s a live risk running in production.

None of these are model failures. As Coyle put it, things “go off the rails” not because the model got dumber, but because nothing was holding the rails in place to begin with.

Why this matters more as agents get more autonomous

Large language models are probabilistic by design. Ask the same question twice and a good model gives you two answers that are close in meaning but not identical in form.

This is the real argument for ontologies, and it has nothing to do with making agents smarter. It’s about giving a probabilistic system something deterministic to reason against. The model keeps its ability to generate, infer, and adapt. It just does that against a structure that doesn’t shift depending on how the question was phrased. Symbolic structure paired with probabilistic generation, each one compensating for what the other can’t do alone.

That pairing matters more as autonomy increases. A single agent operating on a bad assumption can be contained easily. But a fleet of agents, each reasoning from a slightly different, ungrounded picture of the same domain, can drift out of alignment quickly. The organizations moving fastest into agentic systems right now need the guardrail in place before they scale, not after.

Closing thoughts

Ontologies aren’t a new idea. Gruber’s definition is three decades old, and enterprises have used them in narrower forms for years, mostly buried inside data governance projects nobody outside the team ever saw. What’s changed is the stakes. When ontologies are missing from a system of autonomous agents making decisions and taking actions on their own, the cost is a decision nobody can explain, an action nobody approved, or a loop that quietly drains a budget before anyone notices.

The organizations getting real value from AI right now are the ones that gave their agents something stable to reason against before asking them to operate independently. That’s the gap Coyle named, and it’s the gap enterprises will keep running into until they close it structurally instead of patching around it.

We built Forge to address that exact problem. It gives AI agents a persistent, structured understanding of an enterprise’s own systems, so they can build new software or modernize legacy applications without losing track of what the business actually means by its own data. The goal isn’t faster output alone. It’s software an enterprise can trust because every step traces back to intent it actually approved.

To see exactly how Forge does this, read the companion piece, How Forge Gives AI Agents a Living Ontology.