A VP of engineering at a mid-market manufacturer told me last year that his team had spent four months on an AI project that “just stalled.” They had a good use case: flag at-risk purchase orders before they slipped, using the patterns buried in years of procurement history. They had a capable data scientist. They had budget and executive air cover. And the thing never shipped.
When we pulled the thread, the model was never the problem. The problem was that “years of procurement history” lived in three systems that disagreed with each other. The ERP had one version of a supplier’s lead time. A homegrown Access database that one purchasing manager maintained had another. The shared drive full of spreadsheets had a third, and it was the one people actually trusted. Nobody could say which was correct, so the data scientist spent four months arbitrating data disputes instead of building anything. By the time the source-of-truth question got escalated, the project had lost its momentum and its sponsor’s patience.
This is the most common failure mode I see, and it almost never shows up in the kickoff deck. Teams scope AI work as a modeling problem when it is overwhelmingly a data problem. The model is maybe twenty percent of the effort. The other eighty percent is getting clean, governed, accessible data into a shape the model can use without lying to you.
So before we greenlight a build at Ryshe, we run a readiness pass. It is not glamorous and it does not take long, but it catches the failures that would otherwise surface in month three. Here is the checklist, with the reasoning behind each item, so you can run it yourself.
Start With Access and Ownership, Not Algorithms
The first question is embarrassingly basic and almost always under-answered: can you actually get the data, and who has to say yes?
Not “does the data exist.” It exists. The question is whether your project team can read it, at the cadence the project needs, without filing a ticket every week and waiting on a database administrator who reports to a different VP. I have watched promising projects die because the one person who could grant access to the production replica was protective of it, and nobody above them was willing to force the issue.
Work through these before anything else:
- Where does the data physically live, and do you have a credentialed, repeatable way to read it? A one-time CSV export someone emailed you is not access. It is a hostage situation.
- Who owns the system, and have they agreed in writing that this project can use it? Ownership here means the person accountable for it, not the vendor who installed it.
- What are the read constraints? Can you hit it during business hours, or will analytics queries degrade the application everyone depends on? You may need a replica, and that has lead time.
- Is there a contractual or regulatory limit on how this data can be used? In regulated environments this is not optional, and discovering it after you have trained a model is genuinely expensive.
If you cannot check these boxes, stop. Everything downstream assumes you can read the data on demand. If you can’t, the rest is theater.
Interrogate Quality Honestly
Every team believes their data is messier than average. They are usually right, because “average” is a fiction and most operational data is a mess. The job is not to achieve perfect data. It is to know specifically how it is broken so you can decide whether the breakage matters for your use case.
The trap is the confident dashboard. A clean-looking BI report can sit on top of data that is forty percent null in the fields your model actually needs, because the dashboard quietly filters out the rows that would embarrass it.
Pull the raw data and look at it directly:
- Completeness. What percentage of records have the fields you need populated? Not “should have” — actually have.
- Consistency. Does “lead time” mean calendar days in one table and business days in another? Are units, currencies, and time zones consistent, or merely assumed to be?
- Timeliness. How stale is the data when you read it? A field updated quarterly cannot power a daily prediction, no matter how much you want it to.
- Duplication and identity drift. Is one real-world supplier represented as four entities because someone typed the name differently each time?
You do not need to fix all of this before deciding to proceed. You need an honest inventory of what is broken and a judgment about which problems are fatal to the specific thing you are trying to do.
Trace Lineage So You Can Trust the Inputs
Lineage is the question of where each field came from and what happened to it on the way to you. It sounds like governance bureaucracy. It is actually the thing that lets you trust a number at two in the morning when a prediction looks wrong and you need to know why.
If a value is computed — a margin, a risk score, an “expected” date — you need to know the formula and the inputs. We have seen teams train models on a field that was itself the output of an older, undocumented heuristic. The model dutifully learned to imitate a rule of thumb someone wrote in 2014, and nobody noticed until the predictions started reproducing that rule’s blind spots.
For the fields that matter most to your use case, you should be able to answer:
- What system originated this value, and was it entered by a human, calculated, or imported?
- What transformations happened between origin and the table you are reading?
- If it changed, can you see the history, or only the current state? Models that predict the future usually need to learn from the past, and a system that overwrites in place has thrown that away.
You don’t need full lineage on every column in the warehouse. You need it on the handful of fields your project actually depends on.
Nail Down Identity and Permissions
This is where engineering and manufacturing companies get bitten more than most, because their data is full of entities that should be the same but aren’t: the same part with three numbers across three plants, the same customer under a legacy account and a current one, the same employee in the HR system and the badge system with no shared key.
If you cannot reliably join records about the same real-world thing, your model will treat one entity as several and learn nothing coherent. Identity resolution is unglamorous plumbing, and it is frequently the single largest chunk of the real work.
Permissions belong in the same conversation. Ask early:
- Whose data is in here, and who is allowed to see model outputs derived from it? A prediction about an employee or a customer can be more sensitive than the raw record.
- Does the use case cross a boundary — personnel data feeding an operational tool, or one business unit’s data informing another’s — that someone needs to approve?
- Will the model’s output need an audit trail showing what data drove a given decision? In regulated settings, assume yes.
Getting this wrong does not produce a bad model. It produces a model you legally cannot deploy, which is worse, because you find out at the end.
Establish the System of Record (and Stop the Arguments)
Back to the manufacturer from the opening. Their real failure was the absence of a declared system of record. Three sources disagreed and no one had the authority to say which one wins.
For every important entity in your project — supplier, part, order, customer, asset — one system must be the designated truth. Not because it is perfect, but because decisions require a tiebreaker. Once you declare it, the four-month arbitration disappears: when sources conflict, the system of record wins, and the others become candidates for cleanup or retirement.
This is an organizational decision more than a technical one, and it usually needs someone with authority to make it stick. If your project depends on data where the system of record is genuinely contested, surface that to leadership now. It will not resolve itself, and it will absolutely resurface as a model defect later wearing a different costume.
A useful test: pick a single record and ask three people in different departments for its current value. If you get three answers and a debate, you have found work that has to happen before, not during, the build.
Map the Integration Points Before You Need Them
A model that produces a brilliant prediction nobody can act on is a science project. Readiness includes knowing how data gets in and how results get out.
On the input side, a one-time data pull is fine for a prototype but useless in production. You need a sustainable path: an API, a scheduled export, a change-data-capture stream — something that delivers fresh data on the cadence the use case demands, without a human babysitting it.
On the output side, decide where the prediction lands. A score in a database that no one looks at changes nothing. It has to surface where the decision actually gets made — in the ERP screen the planner already uses, in an alert that reaches the right person, in a workflow that was going to happen anyway.
Walk the round trip explicitly:
- How does data get from the source system to the model, on what schedule, and what happens when that pipe breaks at 3 a.m.?
- Where does the output go, and is that destination something a real person already touches in their normal day?
- Who owns each integration once you are gone, and do they know it?
Integration is where prototypes go to die. The demo on a laptop works; the production path through three systems and two teams is the actual project.
Make the Go / Fix-First Call
The point of the checklist is a decision, not a report. When you have worked through it, every item lands in one of three buckets:
- Green: ready. Access works, quality is adequate for the use case, lineage and identity are sound, the system of record is declared, integration is feasible.
- Yellow: fixable in flight. Real but boundable — a known quality gap you can clean, an integration you can build alongside the model.
- Red: fix first. No reliable access, a contested system of record, an unresolved permissions boundary, or identity so broken you cannot trust a join. These are not model problems and a model will not paper over them.
The discipline is simple and most teams skip it: if anything sits in red, you fix it before you build, full stop. It is far cheaper to spend six weeks declaring a system of record and standing up clean access than to spend four months watching a data scientist arbitrate disputes that were never theirs to settle.
This is also the honest answer to “are we ready for AI?” Readiness is not a feeling or a budget line. It is this checklist coming back mostly green, with a credible plan for the yellow and nothing left in red.
If you want a second set of eyes on where your data actually stands, that is exactly the kind of thing we talk through when you book a 30-minute call — no deck, just walking your real systems and giving you a straight read on green, yellow, and red. Whether you do that with us or run the checklist yourself, run it before the build, not during it. The four months you save are almost always your own.
