How ready is your revenue data for AI?

How AI accuracy on business data is measured

Accuracy on company data is not a property of the model. The same assistant that answers a question correctly at one company answers it wrongly at another, because what differs is the data it reasons over — whether the customer records reconcile, whether a metric has one definition, whether the numbers are current, and whether the conversations behind them can be read at all.

So the useful question is not how good the model is. It is which business questions your systems can support. This assessment sorts them into three verdicts: answerable, where the data exists and joins up; unverifiable, where an answer can be produced but nobody can confirm it against a source; and guesswork, where the underlying signal is missing entirely. The second verdict is usually the uncomfortable one, because those answers look exactly like the good ones.

The five factors, and what each is worth

Weights reflect how much each factor determines whether an answer can be trusted, not how hard it is to fix. Entity resolution carries the most because every cross-system question depends on it: if the join is wrong, every downstream number is wrong and still looks plausible.

1. Fragmentation20% of the score

   How many disconnected systems hold business data?

   Connecting an assistant over MCP

   Real gain — connectors reach systems that were unreachable.

   Building it in-house

   Warehouse consolidation + connector maintenance — 2–4 quarters, 1–2 data engineers, ongoing.

   With a context layer

   Managed sync across all your business systems.

2. Entity resolution25% of the score

   Is there one account ID that works everywhere?

   Connecting an assistant over MCP

   No gain — MCP has no concept of a resolved account key, so the model guesses the join, differently each time.

   Building it in-house

   Master data management (MDM) initiative — 3–6 quarters, cross-team, high failure rate.

   With a context layer

   Verified joins on a resolved account key.

3. Semantic consistency20% of the score

   Is “active customer” defined the same way in every tool?

   Connecting an assistant over MCP

   No gain — with no shared glossary the model invents a definition per query, so two people get two answers.

   Building it in-house

   Semantic layer build + definition governance — 2–3 quarters to build, permanent to maintain.

   With a context layer

   Shared glossary with enforced column roles.

4. Freshness15% of the score

   Real-time, nightly, or a manual CSV export?

   Connecting an assistant over MCP

   Real gain — a live query beats last night's batch.

   Building it in-house

   Orchestration + pipeline observability — 1–2 quarters, then on-call forever.

   With a context layer

   Monitored pipelines with freshness SLAs.

5. Unstructured coverage20% of the score

   Are calls, tickets, and emails queryable at all?

   Connecting an assistant over MCP

   Partial — it can read the Gong or Zendesk API, but transcripts still aren't joined to accounts.

   Building it in-house

   Transcript/ticket ingestion + retrieval pipeline — 2–4 quarters, ML-adjacent hiring.

   With a context layer

   Gong / Zoom / Slack / ticket ingest, joined to accounts.

What companies can do to improve it

Start with entity resolution, whatever else you do. One verified account key that works across the CRM, the billing system, and the support tool is the single change that makes cross-system answers checkable, and without it the other four factors cannot compound.

Then agree definitions before building anything. A glossary with one enforced meaning per metric costs very little compared with the engineering around it, and it eliminates the most common failure — two people asking the same question and getting two different numbers, both citing real data.

Treat freshness as a visible property rather than an assumption. A nightly batch is fine if everyone knows the answer is from last night; the damage comes from staleness that nothing in the answer discloses.

Bring conversations into scope last, and only joined to accounts. Calls, tickets, and emails hold the reasons behind the numbers, but a searchable transcript archive that does not connect to a customer record will not improve a single business answer.

Each of these is buildable in-house, and the efforts above are honest estimates of what that takes. The reason teams buy instead is that none of them ever finish: every new system, metric, and acquisition reopens all five. Riley maintains them as a managed context layer and exposes it over MCP, so the assistants your teams already use keep working while the agreements live in one place.