I'm working on a data migration of several hundred nodes from a Drupal 6 to a Drupal 7 site. I've got the data exported to the new site and I want to check it. Harkening back to my statistics classes, I recall that there is some way to figure out a random number of nodes to check to give me some percentage of confidence that the whole process was correct. Can anyone enlighten me as to this practical application of statistics? For any given number of units, how big must the sample be to have a given confidence interval?

  • 5
    I probably wouldn't want to use statistics as a measure of success. This isn't dart throwing. Its a migration. You know what you started with so you should see that at the end
    – Rig
    Sep 12, 2012 at 13:16
  • This migration isn't a simple 1:1 mapping. I'm splitting fields, combining them, and creating more complex nested structures out of several fields. To verify all migrated data to validate my existing code, I would have to write new, other code that does the same thing. And then, if both codebases have different bugs, how would I know which was correct?
    – user1936
    Apr 23, 2013 at 17:55

2 Answers 2


I found this sample size calculator. For my population of 215 items, if I want a 95% confidence with +/- 5% confidence interval, I'll need to randomly sample 138 items.

Edit: Here's the actual formula that I was looking for.


Computers are really good at doing repetitive tasks and comparisons, it wouldn't take long to write a small app that verifies all the data, and unless the amount of data you transferred was huge it could be done overnight or over the weekend and get 100% certainty. Stats rely on getting a true random sample of the entire population, and failing to get a valid set makes that confidence worthless and both humans and computers are really bad at true random. If something does come up its a lot better to be 100% certain than 95%.

  • 2
    It would take significant effort to write a script comparing Drupal 6 to 7 data with any custom fields. A small enough sample size would make it more practical to compare by hand as long as 100% confidence isn't necessary. It's what interns were made for.
    – Matt S
    Jan 10, 2013 at 19:29
  • See my comment on the section. This isn't a 1:1 migration; I have complex transfomations of the data. That "small app that verifies data" would have to have all the functionality that the real app does -- otherwise it wouldn't know if the transformations happened correctly. It would seem quite a nightmare to troubleshoot bugs between two codebases of identical functionality.
    – user1936
    Apr 23, 2013 at 17:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.