Retrieval over the ontology beats retrieval over documents

A company connects a language model to its document store: contracts, procedures, policies, meeting notes, years of them. This is retrieval-augmented generation, usually shortened to RAG: when someone asks a question, the system finds the passages most similar to it and gives them to the model to answer from.

A customer service employee asks what delivery terms a particular customer has. The system finds three versions of that customer’s contract, a draft annex that was never signed, and an email in which someone proposed different terms. The model blends them into a fluent, confident answer that matches none of them. On another day, it finds the company’s general terms instead of the customer-specific annex that overrides them.

The cause is how document retrieval works. It finds text whose meaning resembles the question. It does not know which version is in force, which document is a draft, which customer a clause applies to, or that an annex overrides the general terms. Those are not properties of text. They are states and relations, which is exactly what a model of the company, an ontology, is there to hold.

Where document retrieval breaks

Versions. A superseded contract is as similar to the question as the current one, so it is retrieved just as readily. Similarity has no notion of “in force”.

Relations lost in the cutting. Documents are split into chunks before they are indexed. A clause about delivery terms ends up in one chunk, the page naming the customer in another, and the link between them is gone.

Questions that are really counts. “How many open complaints does this customer have?” is not written in any document. It is a count over records, and no amount of text similarity will produce it reliably.

Permissions applied too late. If the index contains documents the person asking may not see, filtering them out after retrieval is fragile, and a mistake shows confidential text to the wrong person.

Freshness. The index is as current as its last rebuild. The contract signed this morning is not in it yet.

Citations that do not settle anything. The answer quotes a passage, but nothing tells the reader whether that passage comes from the document currently in force.

Retrieve from the model first, then from documents

Start from the object. The question names a customer. Resolve that to the customer object in the ontology, then follow its relations: the contracts in force, their annexes, open orders, open complaints. The model of the company does the narrowing that similarity cannot.

Search documents attached to records. The current contract is the document attached to the contract record whose status is “in force”. Text search then runs within that document, not across everything the company has ever written.

Answer structured questions with queries. Counts, sums, dates and statuses come from the records, computed, not from passages that happen to mention them.

Apply permissions before retrieval. The person’s rights decide which objects are visible at all. The model never receives the rest, so it cannot leak it.

Cite the record and the version. The answer links to the contract record, its version and the clause, so the person can check it in one click.

Document retrieval remains useful for knowledge that genuinely lives in prose: procedures, manuals, accumulated know-how. Even there it works better when each document is attached to the objects it describes, so that retrieval can start from the right place.

That is how Hurukan is designed. Every document is attached to its record, with revisions and reviews, as described under Projects and Documents. The Ontology layer holds the objects, relations and states, and the Super Intelligence layer is designed to ground its models in that structure first, with permissions already applied. It is in build. Where such a model comes from in the first place is the subject of your company already has an ontology.

A test you can run today

Collect ten questions your staff actually asked this month about customers, orders, suppliers or contracts. For each, write down where the correct answer really lives:

• in one document,
• in several documents that have to be combined,
• in a record,
• or in a count or sum over many records.

Then put the same ten questions to your current document-retrieval system, if you have one, and check every answer against the real source.

Note where the wrong version, the wrong customer or a made-up count appears. If the failures cluster in every category except “one document”, you have found the limit of document retrieval, and it is a limit of design, not of the model. The broader question of where language models do and do not help is covered in where language models earn their keep.