Lead Scoring Recalibration After a CRM Data Migration
I manage marketing operations for a company that migrated CRM platforms last year, and while most of the migration project focus went into making sure records and pipeline data transferred correctly, our lead scoring model quietly broke in the process in ways that took nearly two months to fully surface and fix. I want to walk through why this happened and what I'd do differently, because I don't think our migration was unusual, I think most CRM migrations create this exact risk and most teams underestimate it.
Field Mapping Errors Don't Always Throw Visible Failures
Our old CRM had a lead scoring model built on roughly fifteen behavioral and firmographic fields, things like company size, industry, email engagement history, and website activity. When we migrated, most fields mapped cleanly to equivalent fields in the new system, but a handful didn't map one to one, either because the new platform structured the data differently or because a field that used to be a simple dropdown became a multi-select in the new system with a different underlying data format.
The scoring model itself migrated as configuration, referencing field names and value logic that assumed the old data structure. Several scoring rules referenced fields that technically still existed in the new system under the same name but now held differently formatted or differently populated data, which meant the scoring logic kept running without any error, it just started producing systematically wrong scores because the underlying data it was evaluating no longer meant what the rules assumed it meant. Nothing broke loudly. Everything just quietly got worse.
The First Sign of Trouble Was a Sales Complaint, Not a Dashboard
We found out something was wrong not through any monitoring we had in place, but because a sales manager flagged that leads coming through as high priority felt noticeably less qualified than they used to, a soft, subjective signal that could easily have been dismissed as normal variance if he hadn't pushed on it. When I actually pulled the data and compared score distributions before and after the migration, the shift was dramatic, a field that used to reliably differentiate engaged leads from cold ones was now returning null or malformed values for a meaningful share of records, silently zeroing out what should have been a significant positive scoring factor.
This experience changed how I think about migration validation generally. Verifying that data transferred is necessary but insufficient. You also have to verify that every piece of downstream logic that depends on that data, scoring models, routing rules, automated workflows, still behaves correctly against the new data's actual structure and values, not just that the values technically exist somewhere in the new system.
Rebuilding Required Auditing Every Rule Against Real Current Data
Fixing this properly meant going through every single scoring rule individually, checking the field it referenced against how that field actually looked in the new system, and confirming the logic still produced the intended behavior. This was tedious, genuinely unglamorous work, but skipping it in favor of a faster patch would have just meant fixing the obvious broken rules while leaving other, less obviously broken ones still quietly producing wrong scores.
In the process, I found that a few rules had actually been somewhat broken or poorly calibrated even before the migration, edge cases and stale logic that had accumulated over years of incremental changes to the scoring model, that nobody had caught because the model's overall output still looked roughly reasonable in aggregate. The migration forced a level of scrutiny on the whole model that hadn't happened in years, which turned an unwelcome incident into an opportunity to genuinely improve the model, not just restore it to its previous, already somewhat imperfect state.
Recalibrating Thresholds, Not Just Fixing Broken Rules
Beyond fixing rules that referenced broken or malformed fields, I also had to recalibrate the actual score thresholds that determined lead routing and sales handoff, because the distribution of scores had shifted enough during the broken period that our existing threshold, tuned years earlier against the old distribution, no longer represented a meaningful cutoff between genuinely qualified and unqualified leads. I pulled a fresh sample of recent leads, had sales manually assess qualification quality independent of the scoring model, and used that fresh ground truth to recalibrate thresholds against current data and current buyer behavior rather than assuming the old thresholds were still correct once the underlying rules were fixed.
Building Ongoing Validation Instead of Relying on Someone Noticing
The most important structural change coming out of this incident was building an ongoing score distribution monitor, tracking the shape of our lead score distribution over time and alerting on statistically significant shifts, rather than relying on a sales manager happening to notice something felt off and being willing to push on that feeling until it got investigated. This kind of anomaly detection wouldn't have prevented the original migration issue entirely, since some validation gap was always going to exist during a major platform change, but it would have caught the problem within days instead of the nearly two months it actually took, based purely on informal, subjective feedback eventually reaching the right person.
Documentation Debt Made This Worse Than It Needed to Be
Part of what made the audit process slower than it should have been was that our scoring model's original rule logic and the business reasoning behind each rule weren't well documented anywhere, so reconstructing why a given rule existed and what it was actually supposed to measure took real detective work in some cases, going back through old team communications and asking people who'd been involved years earlier. I now maintain living documentation for every scoring rule, what it measures, why it exists, and what data it depends on, specifically so that the next platform migration, or any other major change to our data infrastructure, doesn't require this same archaeological effort to understand what the model was actually supposed to be doing.
What I'd Tell Someone Managing a CRM Migration
Don't treat data transfer validation as sufficient on its own, explicitly audit every piece of downstream logic, scoring, routing, automation, against how the new system's data actually looks, not just whether the values technically migrated. Expect field mapping issues to fail silently rather than loudly, and build monitoring that can catch a quiet distribution shift rather than waiting for someone to notice something feels off. Use a migration as an opportunity to genuinely recalibrate thresholds and clean up accumulated model debt, not just to restore the prior state. And document your scoring logic and its underlying reasoning as you go, because reconstructing that reasoning after the fact, under pressure, during an active incident, is a genuinely bad time to be doing it for the first time.
