Survey Says… “Oops”

Max Fisher at The Washington Post ran a blog post last week featuring a world map of “racial tolerance” based on data from the World Values Survey (WVS), and it didn’t take long before the collective peer review power of Tufts University and Reddit found at least two examples of “fat fingers” where a “no, I don’t mind living next to other races” was mistakenly swapped with a “yes, I’m totally racist when it comes to choosing neighbors” for Bangladesh and Hong Kong, thanks to mistranslation and poor survey design. Others have pointed out the inherent flaw in assuming that the construct of “race” is universal and that news organizations’ need to feed the content beast creates a game of telephone where complex data is oversimplified and misinterpreted without real scrutiny.

I first encountered the WVS last year in my coursework on International Librarianship, where Geert Hofstede’s book Cultures and Organizations: Software of the Mind was used as a primary text and it served as solid jumping-off point for discussion. Hofstede is the granddaddy of cross-cultural studies quants, building a cross-cultural theory and corporate consulting brand out of data he developed as head of HR for IBM Europe in the 1960s, the same time Robert McNamara was using IBM mainframes to plot effective firebombing raids of Japan. Those were the salad days for punchcards. Like the WVS, Hofstede has survey datasets from dozens of countries over decades from which he and others glean tantalizing correlations (which so easily slip towards causation) between conceptual frameworks that “emerge” from the data (individualism vs. collectivism, for example) and GDP, economic growth, war, etc. At first, the book was fascinating as an American living in China – “wow, this validates so many things I anecdotally observe with hard data!” By the end of the book, however, I was completely disenthralled. The assumptions, generalizations, and seeming contradictions piled up in a doomed effort to render “national culture,” if such a thing is quantifiable, legible (yes, I’m finally reading Scott’s Seeing Like a State), looking like nothing more than the psychologist and sociologist equivalent to the old hack joke “White people drive like this, but black people drive like that!” Unless you’re a committed professional like Michael Harris Bond (who developed the original Chinese Values Survey) who will spend years wrestling with the data and appreciating its limitations, you’re better off watching Russell Peters.