Never Pristine

Data is never pristine. Why? Because man produces it, something your's truly often forgets when writing about tech and science but Nate Silver never forgets as seen in his terrific article titled Be Skeptical of Both Piketty and His Skeptics.

Data never has a virgin birth. It can be tempting to assume that the information contained in a spreadsheet or a database is pure or clean or beyond reproach. But this is almost never the case. All data is collected and compiled by someone — either an individual researcher or a government agency or a scientific laboratory or a news organization or someone or something else. Sometimes, the data collection process is automated or programmatic. But that automation process is initiated by human beings who write code or programs or algorithms; those programs can have bugs, which will be faithfully replicated by the computers.

This is another way of saying that almost all data is subject to human error. It’s important both to reduce the error rate and to develop methods that are more robust to the presence of error. And it’s important to keep expectations in check when a controversy like the one surrounding the French economist Thomas Piketty arises.

Buyer beware, anyone? :)