Content first, AI later: Why 90% of your company's data isn't ready for AI
Introduction
In the summer of 2025, MIT published a number that landed in boardrooms like a bomb: of all corporate AI pilot projects, around five percent produce a measurable gain. The other 95 percent fizzle out after money, time and hope have gone into them. The reflex is to conclude that AI isn't up to the job. That is explicitly not what the researchers found. The problem, they write, is not model quality but the way companies integrate them. Gartner names poor data quality as the first reason projects get shelved — ahead of cost and ahead of unclear benefit. IDC puts the consequence most bluntly: an AI strategy without a data strategy is not a strategy.
The promise these projects start with is seductive, and not even wrong: you throw in contracts, case files, manuals, ask a question in plain language and get a reliable answer. No more hunting through folders. You can build applications like that — but between "in principle" and "in production" sits the work that no glossy slide talks about. The demo shows the nice question and the nice answer. It doesn't show the bit in between, and it runs on three hand-picked documents that are cleanly structured and hold no surprises.
The real document base is version chaos. Around 90 percent of corporate data is unstructured, and that share has barely moved in years: PDFs, Word files, scans and faxes at worst. Knowledge sitting in email that exists in exactly one inbox. Slide decks that are in fact process documentation, because back then it was done in PowerPoint. On top of that, duplicates, near-duplicates, superseded contract versions, chains of schedules, addenda agreed by email. A human works through that because they bring context. A large language model sees fragments, loses the thread and produces convincing answers that are simply wrong. The problem is worst exactly where the pressure is highest: in banking, insurance, law firms and pharma, where the document base is most complex and a wrong answer costs the most.
Key quotes
In plain terms: it isn't the AI. It's what we feed it.
— 00:01:00 (translated from German)
The whole episode hangs on this line. It moves the question from choosing technology to taking stock — to a place where you can work without signing a licence agreement first.
Without ownership of your content, every AI strategy is built on sand.
— 00:06:01
Setting up a centre of excellence before anyone can say which version of a contract is in force gives you a committee, not a foundation.
The footnote with the source is real, but the sentence in front of it is invented.
— 00:11:35
That is mis-grounding, and it is nastier than a plain hallucination: it survives any surface-level check and even satisfies the rule that the AI should cite its sources. On paper the answer looks evidenced. Auditability and the duty to document are still not met.
The pilot dies of that shock to trust, not because it wasn't doing anything useful.
— 00:22:57
A single confidently wrong answer in front of a subject-matter team — or worse, in front of a client — costs more trust than twenty correct ones can rebuild.
The awkward thing about your document base is that it isn't a photograph. It's a film.
— 00:23:57
Which is why preparation is not a project with an end date but a process. Clean up once and carry on as before, and twelve months later you are back where you started — only with an invoice.
What you'll learn in this episode
Why RAG is not a magic wand — Retrieval augmented generation can cut hallucination substantially, but only if retrieval, chunking, metadata, access layers and source handling are right. Otherwise it produces new failure modes rather than fewer. Even legal AI tools wired to curated statute and case-law databases return wrong answers between 17 and 34 percent of the time — more than one in three. And these are expensive specialist tools, not an afternoon's work.
How chains of cross-references send a model astray — Section 2 points to Schedule 7, Schedule 7 departs from the standard terms as of March, and those exist in four versions. The clause actually in force appears in an email addendum written two years later. So the decisive question isn't which model to buy. It's whether you can reliably find the right document in the right version.
What mis-grounding is, and why it's harder to catch than hallucination — The source exists, the model does reach for it, but the statement isn't supported by it. Usually because the correct document was never findable in the first place.
Why handing users a raw prompt is risky — Models tend to tell you what you want to hear. Build a false premise into your question and legal tools will very often construct a legally tidy argument on top of it, instead of pushing back or asking. The sources never get consulted at all.
A four-question self-diagnosis — Are there important documents that exist only as an image? Multiple versions where nobody knows which one applies? Heavily cross-referencing contracts and files? Connections that live only in a few people's heads? Two yeses are enough to locate the problem — and it costs nothing in model budget.
What preparation actually involves — Turn scans into real text by OCR and store that version properly. Resolve cross-references; denormalise complex contract sets in pre-processing so schedules and references sit together in one readable version. Preserve document structure so headings, clauses and footnotes stay visible. Extract metadata such as names, deadlines and amounts once, rather than making the model guess at every query.
Why it doesn't work without governance — Flag final versions, separate drafts from approved results, take duplicates and superseded versions out of the AI's reach. That is an organisational question, not a technical one — and it decides whether your clean base drifts back into chaos within months.
Why the work pays twice — A prepared document base isn't a sunk cost, it's an asset. It carries not only this application but the next one: search, deadline monitoring, risk analysis, quarterly reporting. And it helps the humans first, who finally know which version of a contract is in force.
You will swap models several times and keep adding applications. The document base stays. It is the structural engineering beneath the elegant room the demo shows you — invisible, and without it the whole thing comes down.
