Your AI Strategy Has a Data Problem—And It Started Before You Picked Up a Tool

Most AI strategies begin with the tool. Which platform, which vendor, which use case to pilot first. It’s a natural instinct, and the urgency is real — but it tends to skip over the question that ends up mattering most. The organizations we work with that see the most ROI from their AI investments aren’t necessarily the ones who moved first. They’re the ones who had their data in order before they started—with a strong data foundation and clear data governance inside their system. And for most midmarket companies, that’s a story that runs straight through their ERP.

When the Data Isn’t Swimming in the Same Lake: Why Fragmented ERP Data Breaks AI

Here’s what fragmented data looks like in action when there’s no shared data foundation or consistent data governance across systems. A company grows through acquisitions — seven deals in seven years, each target with its own ERP, chart of accounts, and expense categories. At first, everyone manages. The finance team manually reconciles across systems. The IT team creates integrations to fill the gaps. When a data migration occurs, historical records are often moved in bulk rather than properly cleaned, because the go-live date can’t be changed and something has to give.

Fast forward a few years, and that company wants to layer AI on top of its operations. Maybe it’s an agent to handle customer service calls. Maybe it’s demand forecasting or spend anomaly detection. The AI tools exist, and the use case is clear. But the data underneath is inconsistent, incomplete in places, and spread across systems that have never shared a common structure. The AI doesn’t know what to do with that. And worse, it doesn’t tell you it doesn’t know. It just produces answers that look confident and turn out to be wrong.

One of the executives we work with described their ERP migration in straightforward terms: garbage in, garbage out. They had been using the same system for over twenty years across multiple countries, and even three years after switching to a new platform, data quality issues still appeared. You can’t shortcut your way out of that. AI doesn’t fix messy data. It only amplifies what already exists.

This is why the “swimming in the same lake” framing matters. AI works effectively across your business when all the relevant data resides in a shared, structured environment. The moment you ask it to reason across systems that don’t communicate with each other — or pull from historical records that were never properly migrated — you’ve already limited what’s possible before writing a single prompt.

The Historical Data Trap: How ERP Migration Shortcuts Limit AI Models

The main issue often occurs during ERP migrations, especially in how organizations handle historical data. The pressure during an implementation is always focused on the go-live date. Migrating historical data is expensive, time-consuming, and frankly unglamorous — it doesn’t get much attention in demos. As a result, teams often take shortcuts. They transfer summary records instead of transaction-level data. They leave certain entities on legacy systems longer than planned. They migrate the structure without ensuring data quality, so old errors get carried over into the new system.

Those decisions seem reasonable at the time. Three years later, they look very different when you’re trying to build an AI model using records that don’t go back far enough, or that mean different things depending on which business unit entered them.

And the ERP is often just the starting point. The companies furthest along in AI adoption have typically gone through the same consolidation exercise with their CRM, their warehouse management system, and other core platforms — not because those systems matter less, but because AI doesn’t respect the boundaries between them. A voice agent handling customer calls needs to see order history, account status, and billing data in the same breath. If that information is spread across systems that don’t share a common structure, the agent is working with one hand tied behind its back, regardless of how sophisticated the underlying model is.

The executives who have experienced this are clear about what they would do differently: treat the data migration as the main project, not just an annoying side task of the technology project. The value of the ERP depends entirely on the data inside it. And the AI you build on top of it is only as valuable as the actual content of the ERP.

What “Ready” Looks Like

A great illustration of what it takes to do this right is a distribution company that’s been working with Velosio on its digital transformation for several years. Their story isn’t dramatic like AI demos tend to be. It’s actually pretty straightforward. For roughly four years, they did the work. Four ERP conversions. Multiple new distribution centers with warehouse management technology. A full consolidation onto a single CRM platform. A shift from 17% cloud to 68% cloud. Every acquisition that came in got migrated onto the same platform. Not eventually, but deliberately, as a matter of policy.

When they were ready to deploy a voice agent in their customer care operation, they didn’t need to solve a data problem first. The data was already available, structured, and consistent across business units. About 25% of their customer calls now go through that agent. When a call must be routed to a live person, the system automatically hands off the customer record so the agent doesn’t have to start from scratch.

And when they want to roll the capability out to another business unit, the foundation is already in place — there’s no translation layer to build, no reconciliation problems to solve, because everything runs on the same platform.

That scalability didn’t come from the AI. It came from the years of ERP work before it.

Where Most Organizations Actually Are

If you’re reading this and feeling like your organization is behind, you’re probably not as far back as you think — but the gap is real, and it’s worth understanding and addressing.

When we speak with senior leaders across industries about their AI progress, most rate themselves as a two or three on a five-point scale. For their ERP maturity, they typically score slightly higher — a three or four. The gap between these two numbers highlights the real issue. Organizations that are more advanced in ERP consolidation are almost always more advanced in AI adoption, and it’s no coincidence.

A few questions worth honestly considering: How many systems does your financial data currently reside in? When you last migrated to a new ERP, what happened to your historical records — were they cleaned and moved or left behind? If you wanted to deploy an AI tool across two different business units today, would they be working from the same data or would you need to build a bridge first?

These aren’t trick questions. Most organizations have at least one uncomfortable answer in there. The point is to understand where you truly stand before you begin evaluating AI platforms and building use case roadmaps.

The reframe

The way we’d encourage senior leaders to think about this: the ERP work you may be putting off, or deprioritizing, or doing halfway — that is the AI strategy. The organizations moving fastest right now aren’t doing AI and ERP as separate initiatives. They’re recognizing that you can’t have one without the other, and they’re planning their steps accordingly.

That doesn’t mean waiting to consider AI until the ERP is perfect. It means recognizing which AI use cases your current data foundation can genuinely support, and being realistic about what needs to happen before moving forward. Sometimes that’s a complete consolidation effort. Sometimes it’s a focused data cleanup in existing systems. Sometimes it’s deciding to migrate that last legacy entity you’ve been delaying for two years. It’s not as exciting as the AI demo, but it’s genuinely more important.

Take our AI Readiness Assessment to understand where your organization stands on ERP maturity and AI readiness.

Don’t worry, Velosio works with companies at every stage of this journey. Reach out to our team to get started today.