Study: 47 Percent of Newly-Created Data Records Have Critical Errors

Chris Gaetano
Published Date:
Sep 11, 2017

While much is said on how data analytics can provide new insights and solutions for business, a recent study has found that the quality of the data being analyzed leaves much to be desired, as 47 percent of studied samples have critical errors and only 3 percent would pass even the loosest of quality standards, according to the Harvard Business Review

Managers were asked to construct 100 data records by assembling 10-15 critical data attributes for the last 100 units of work completed by their departments. Managers and their teams were them instructed to work through each record and mark down obvious errors. They then counted up the total number of error-free records. This number, ranging from 0 to 100, is then marked as their Data Quality Score, the percent of data created correctly. 

The researchers conducted this exercise with 75 different executives at Irish businesses. What they found was, on average, 47 percent of newly-created data records had at least one critical error, defined as something that would actually impact work. Looking at the data more granularly, a quarter of newly-created data sets had scores below 30 percent and half had scores below 57 percent. In fact, only 3 percent of the Data Quality scores overall could be rated "acceptable" by the same managers who took part in the exercise (none ever thought less than the high-nineties as acceptable). They did note, though, that there is a great deal of variability between scores. Individual tallies ranged from 0 percent to 99 percent. 

The results found that poor data quality is everywhere: there is no sector, government agency or department that is any better or worse than another. 

Click here to see more of the latest news from the NYSSCPA.