A VP of operations at a 250-person manufacturing firm told me something last quarter that stuck with me. “I have eleven different AI pilots running across the business,” she said. “I can’t tell you which ones are working, who owns them, or whether any two of them could share the same code. And my CFO just asked me for a number.”
That is the actual state of AI at most mid-market companies right now. Not a lack of activity — there is plenty of activity. A sales team running a chatbot. A finance analyst who built something clever in a spreadsheet with an API key. An engineer who automated a report and now everyone depends on a script no one else understands. What’s missing is anything connecting these efforts: a way to decide what’s worth doing, a standard for doing it safely, and a mechanism for one team’s win to become another team’s starting point.
The big companies solved this with a Center of Excellence — a central team that sets standards, prioritizes investment, and spreads reusable patterns across the organization. It works. But it also assumes you can hire eight to fifteen people with “data scientist” or “ML engineer” in their titles, and that you have enough AI work to keep them busy. If you’re a 200-person firm, you can’t, and you don’t. The math doesn’t close, and you know it doesn’t, which is why you’ve been quietly putting off the question.
Here’s the part nobody tells you: the CoE was never really about the headcount. It was about the outcomes. And you can get most of those outcomes without building the org chart.
What a Center of Excellence Actually Produces
Strip away the staffing and a CoE does five concrete things. Worth naming them plainly, because each one is achievable on its own.
- Prioritization. It says no to most ideas so the few good ones get real resources. Without this, you get eleven pilots and zero production systems.
- Governance. It defines what “safe to ship” means — data handling, model behavior, human review, audit trails — so individual teams aren’t each inventing their own risk posture.
- Reusable patterns. It captures what worked once and makes it cheap to repeat. The second use case should cost a fraction of the first.
- Internal champions. It builds people inside the business who can carry the work forward without external help on every step.
- Vendor partnership. It manages the relationship with outside expertise so you’re buying capability, not just hours.
Notice that only one of those — reusable patterns — strictly requires deep technical staff to originate. The other four are leadership functions. They need judgment, authority, and consistency, not a PhD. This is the unlock for mid-market leaders: you can stand up four of the five functions with people you already have, and partner for the fifth.
Start With Prioritization, Not Technology
The most expensive mistake I see is starting with a tool. A company buys a platform, or commits to a model provider, and then goes looking for problems to point it at. That’s backward, and it’s how you end up with eleven pilots.
Begin instead with a short, honest inventory. Pull your leaders into a room and list every place AI is currently being used or seriously discussed. For each one, ask three questions:
- What measurable thing gets better if this works — hours recovered, errors avoided, revenue unblocked?
- Who actually owns the outcome, by name?
- What happens if it produces a wrong answer and nobody catches it?
That third question quietly does a lot of work. It separates the use cases where a mistake is an annoyance from the ones where a mistake is a liability. A typical engineering firm will find that its highest-enthusiasm pilot is also its highest-risk one, and that a boring document-retrieval use case nobody was excited about is the one that’s safe, valuable, and ready.
Rank ruthlessly. Pick two or three. Kill or pause the rest — not forever, just not now. The discipline of saying “not now” to a champion’s pet project is the single most important behavior of a CoE, and it costs you nothing but the willingness to be unpopular for a meeting.
Governance That Fits on Two Pages
Governance in a mid-market firm should not look like a Fortune 500 compliance binder. If your AI policy is forty pages, no one will read it and it will quietly become fiction. Aim for two pages that a busy manager can actually internalize.
The questions worth standardizing
- Where does the data go? Which systems can touch customer data, drawings, financials, or anything regulated. For firms in regulated spaces — and if you’re in aerospace, healthcare, or anything HIPAA-adjacent, you know who you are — this is non-negotiable and should be answered before the first line of code.
- When does a human decide? Define which outputs ship directly and which require a person to approve. The dividing line is consequence, not sophistication.
- How do we know it’s still working? A model that was accurate in March can drift by September. Decide who looks at performance, and how often.
- What’s the paper trail? If a regulator, a customer, or your own auditor asks how a decision was made, you need to be able to answer.
You don’t need to answer these perfectly. You need to answer them consistently, so that every new use case inherits the same baseline instead of relitigating it. Consistency is the product here, not perfection.
Reusable Patterns: Build the Second One Cheaply
This is where most mid-market AI efforts leak the most money. The first use case costs what it costs — discovery, plumbing, integration, learning. The waste happens when the second use case costs just as much because nothing from the first was captured.
A pattern is just a solved problem written down well enough to reuse. The retrieval setup that answers questions over your engineering specs is, structurally, the same machinery that answers questions over your contracts or your safety manuals. The pipeline that pulls data from one system, cleans it, and hands it to a model is most of the pipeline you’ll need for the next three projects.
You don’t need a platform team to capture patterns. You need three habits:
- Write down how each system is built, in plain language, while it’s fresh.
- Keep the reusable pieces — the data connectors, the prompts, the review steps — somewhere a future project can find them.
- Before starting anything new, ask what already exists that’s eighty percent of the way there.
We’ve seen teams that get this right bring the cost of their third and fourth projects down to a fraction of their first. That’s not magic. It’s just refusing to start from zero every time.
Champions Beat Hires
You will not out-hire your problem, and you shouldn’t try. What you can do is grow two or three people inside the business who become fluent enough to own the work day to day.
These are not data scientists. They’re the operations analyst who’s naturally curious, the engineer who already automates things on the side, the project manager everyone trusts. Their job isn’t to build models from scratch. It’s to understand the systems well enough to run them, spot when something’s off, ask vendors the right questions, and translate between “what the business needs” and “what the technology can do.”
What a champion actually needs
- Enough technical literacy to call nonsense when they hear it.
- Real ownership — a name attached to outcomes, not a committee.
- Air cover from leadership to spend time on this, not squeeze it around their day job.
- Access to someone more expert when they hit the edge of what they know.
That last point is where the external partner earns its keep. A good delivery partner doesn’t just hand you a system and leave. They build alongside your champions so that capability stays in the building when the engagement ends. If your vendor’s incentive is to keep you dependent, you have the wrong vendor.
Where the Partner Fits — and Where It Shouldn’t
Here’s the honest version, since I’d rather you hear it from me than learn it the hard way. An external delivery partner is the most efficient way to get the one CoE function you genuinely can’t fake: the deep technical capability to build the first few systems right and to establish the patterns your team will reuse.
What you should expect a partner to do:
- Build the first two or three use cases to production standard, not pilot standard.
- Leave behind documented, reusable patterns your team owns.
- Train your champions in the process, not after it.
- Tell you when an idea isn’t worth building.
What you should not let a partner do:
- Own your prioritization. That’s a business judgment only you can make.
- Become a permanent dependency for routine changes.
- Sell you a platform before they understand your problem.
The arrangement that works is a partner who supplies the technical depth and a transferable way of working, while you keep the prioritization, the governance, and the ownership in-house. You’re renting the expertise you can’t justify hiring full-time, and buying down the cost of every project that follows. That’s the whole model. It’s how a firm with no data science org gets center-of-excellence outcomes — Ryshe Forge exists precisely to fill that build-and-transfer gap.
The Takeaway
A Center of Excellence is a set of outcomes, not an org chart. Prioritization, governance, reusable patterns, internal champions, and a vendor partnership you control — four of those five you can stand up this quarter with people already on your payroll, and the fifth you can rent without apology.
The companies that get stuck are the ones waiting until they’re big enough to “do it properly,” running eleven disconnected pilots in the meantime and wondering why none of them turned into anything. The companies that pull ahead are the ones that act small, decide hard, and refuse to start from zero twice.
If you want to pressure-test where your own efforts stand — what to keep, what to kill, and what a sane first build looks like — that’s a good thirty-minute conversation. You can book a 30-minute call. No deck, no pitch. Just a straight read on whether what you’re doing adds up, and what the next concrete step would be.
