Mark D. White

Mark D. White

Kevin Drum at Mother Jones recently highlighted a new paper by Kimmo Eriksson (Mälardalen University and Stockholm University) published in Judgment and Decision Making titled "The Nonsense Math Effect" (7/6, November 2012). Here's the abstract:

Mathematics is a fundamental tool of research. Although potentially applicable in every discipline, the amount of training in mathematics that students typically receive varies greatly between different disciplines. In those disciplines where most researchers do not master mathematics, the use of mathematics may be held in too much awe. To demonstrate this I conducted an online experiment with 200 participants, all of which had experience of reading research reports and a postgraduate degree (in any subject). Participants were presented with the abstracts from two published papers (one in evolutionary anthropology and one in sociology). Based on these abstracts, participants were asked to judge the quality of the research. Either one or the other of the two abstracts was manipulated through the inclusion of an extra sentence taken from a completely unrelated paper and presenting an equation that made no sense in the context. The abstract that included the meaningless mathematics tended to be judged of higher quality. However, this "nonsense math effect" was not found among participants with degrees in mathematics, science, technology or medicine.

It's a short paper and well worth the quick read (or read Drum's post, which summarizes it well). Eriksson reports that humanities/social science readers tended to be enchanted by the irrelevant equations, with 60-65% rating the adulterated abstract higher, but economists are not broken out of that very broadly defined group (which only includes 84 people as it is). Given some (most?) economists' predilection for mathyness, though, I would not be surprised at some degree of unconscious bias for research that promises greater mathematical sophistication (though I assume any such bias would melt away once the paper was read).

But I think many other economists, especially heterodox economists who are more skeptical about the benefits of mathematical modeling, might go the other way. I know that when I read an interesting abstract and then skim the paper, my eyes glaze over when I hit math–not because it doesn't add anything to support the author's thesis but because I'm afraid it will leave out many things in the interest of abstraction and simplicity, such the very nonquantitative aspects of the model that I found fascinating in the first place! Some things must be left out of a model, of course, but these factors should be omitted because they are relatively unimportant, not because they're don't fit into the modeling framework.

As Eriksson writes in his introduction to the paper,

In areas like sociology or evolutionary anthropology I found mathematics often to be used in ways that from my viewpoint were illegitimate, such as to make a point that would better be made with only simple logic, or to uncritically take properties of a mathematical model to be properties of the real world, or to include mathematics to make a paper look more impressive.

He very well could have included economics in there as well–I'm curious if his exclusion of it was intentional or random. Gee, I'll bet we could model that…