Siddhartha Rao is a VP & Distinguished Engineer at Oracle. He has been working on AI and machine learning for the last decade.
Nearly every serious conversation about enterprise AI now includes the same four words: âContext is the moat.â Then everyone nods.
But ask what context actually is, and the room goes quiet. That silence is expensive. Companies are funding agents to âchat with their dataâ without being able to say what those agents need to know.
I lead generative AI services for Oracleâs AI Data Platform, so Iâm not a neutral observer. Still, Iâve watched enough pilots stall to have opinions about why. Here are six of those opinions:.
Recent findings suggest that the fashionable answer to the question of context is data gravity. Bring the models to where the data already lives.
Fine. Do it. But a model sitting next to a petabyte it doesnât understand is like a tourist outside of a library in a language they canât read.
The advantage belongs to whomever understands the data (like which of three revenue tables finance actually trusts) and knows which instrument fits which question, whether that means forecasting for demand, Bayesian estimates for risk or deterministic rules for eligibility.
The language modelâs job, working out what the data means, is narrower than the hype suggests.
Context includes metadata, the workflows a business actually runs, an ontology of the concepts it reasons in, and an understanding of how its structured and unstructured data relate.
Recent findings suggest that context is not raw rows stuffed into a prompt. Size is not meaning, and we should stop pretending otherwise.
The piece most teams miss is the tool description. An agent never reads your data model. It reads the descriptions of the tools itâs allowed to call. A generic ârun SQLâ tool tells it nothing. A tool described in the businessâs own language, scoped to what that agent should see, tells it how to think.
According to recent reports, prompt engineering was about how you ask. Context engineering is about what the model is told the world contains.
In a recent development, should context live in one platform or stay spread across lakehouses, databases and applications? Itâs a false choice. No CIO wants yet another copy of everything, with permissions that drift from the source. Leave the data where it lives and query it in place.
Sources indicate that what has to consolidate is meaning; one semantic layer that bounds the model, orchestrates its tools and decides what each user may touch. That layer, not storage, is the control point.
According to recent reports, gartner predicted in June 2025 that over 40% of agentic AI projects will be canceled by the end of 2027, citing costs, unclear value and inadequate risk controls. But read that list again. Nowhere does it say the model wasnât smart enough.
Pilots run on curated data for a few users who can see everything. Production runs on thousands of users with row-level, column-level and role-based permissions. Agents make it harder by creating data the access model never anticipated, such as summaries, memories or knowledge graphs.
If an agent tells one user what it learned from anotherâs query, then all youâve done is build a very polite data leak.
Recent findings suggest that will better reasoning models make proprietary context obsolete? No. The frontier has moved from capability to cost. A model that rediscovers your schema on every question burns tokens, takes a different path each time and returns an answer nobody can trace. Thatâs a brilliant latest hire with amnesia, billing by the hour.
Sources indicate that context is amortized reasoning. You pay once to understand the business, and every later question inherits it. That math changes only when inference is nearly free, reproducible and able to audit itself back to source rows. I wouldnât bet the roadmap on all three.
In a recent development, when an agent acts on stale or over-permissioned context, who owns the failure? Juvenal asked the governance question nearly 2,000 years ago: Who watches the watchmen? If you canât reconstruct what the agent saw, how old it was and whose permissions applied, the answer is the CIO.
Sources indicate that so, build governance in. Citations should trace to the business concept the agent thought it was answering and to the exact rows it used. Memory should point back to source data with a freshness timestamp. A stale finding presented as current is the most dangerous kind of wrong, because itâs plausibly right.
If you fund one thing on Monday morning, make it the expertise, in-house or through a partner, to build an ontology, semantic models and progressive disclosure. Give the model the map first, then reveal detail only as each question demands. Itâs cheaper, more accurate and easier to govern because every disclosure becomes a checkpoint for permissions, freshness and lineage.
Industry observers note that before you sign with any vendor, put these six problems on the table. Their answers will tell you more than their demos.
According to recent reports, forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
