<link rel="stylesheet" href="//fonts.googleapis.com/css?family=Roboto%3A300%2C400%2C500%2C700%7CRoboto+Slab%3A400%2C700">Evolutionary psychology has problems; and it isn't improving - Filthy Monkey Men

Evolutionary psychology (EP) attempts to study how evolution has shaped our brains, thoughts and behaviour. However, it’s ability to obtain reliable results is hampered by the fact it’s trying to study all humans yet 75% of EP research is conducted only on Westerners. Of course, I’m hardly breaking new ground with this criticism; researchers have been pointing it out for years. Which raises the question: is all this criticism having an impact? Is evolutionary psychology improving?  

Evolutionary psychology

The percentage of evolutionary psychology research conducted on various populations around the world. Westerners are often referred to as “WEIRD” (or Western, Educated, Industrialised, Rich, Democracies).

The criticism

The issue of sampling practices in psychology is an old one. If your research doesn’t examine a representative cross-section of the population then your results may not apply to the whole population. In fact, they might not apply to anyone outside the narrow group you examined; effectively meaning you wasted your time. As a result of this, late last century people were debating over whether psychology examined enough non-whites1, or whether it focused too much on university students2.

The percentage of psychology studies conducted on populations around the world. Europe, North America and other Western (WEIRD) countries make up >90% of it.

Arnett’s results, revealing the percentage of psychology studies conducted on populations around the world. Europe, North America and other Western (WEIRD) countries make up >90% of it.

In recent years the focus has shifted somewhat. People began asking “even if we studied a perfect cross-section of America would those results be representative of humanity as a whole?” The answer seemed to be no3. In 2008, Arnett hammered this problem home by publishing one of the largest reviews of psychology research ever. He found that 95% of research examined WEIRD populations4. That’s 95% of research conducted on an unrepresentative sample.

Most of this discussion may have focused on psychology in general, but EP has been paying attention. Researchers have written about how Arnett’s results do not bode well for the field3,5 and the eponymous textbook Evolutionary Psychology devotes a fair bit of time to how bad sampling practices are bad6. And that’s not counting all the blogs, books and other articles on the issue.

In short, evolutionary psychology knows using WEIRD samples too much is bad. So surely their use is decreasing over time, in favour of more representative samples that would produce more reliable results?

The test

Last week I wrote about a review of EP research I conducted. At the end of that little experiment I was left with a lot of data on how much EP research used WEIRD samples from 2003 – 2007 and in 2013. So having finished with my previous review, I went over to this data set and asked it “has the use of such samples been decreasing over time?”

Hopefully you all read the headline of this post before clicking on it so by now you know the answer: No.

How the use of WEIRD samples changed over time

How the use of WEIRD samples changed over time

As you can see – aside from a brief dip in 2005 – there was effectively no change over this period. Statistics confirms this; indicating there was not a statistically significant change in the use of WEIRD samples from 2003 – 2007. As if that wasn’t bad enough, when you break it down by journal it turns out that some have actually increased their use of WEIRD samples.

How the use of WEIRD samples changed over time, broken down by journal examined

How the use of WEIRD samples changed over time, broken down by journal examined. Note: whilst some journals increased their use of WEIRD samples and others decreased, none of these results were statistically significant

But Adam” I hear my dear readers ask, “these results are from 2007. Surely it’s unfair to dismiss evolutionary psychology based on old data“. And right you are, reader who lives in my head. Which is why I also examined the latest complete issue of these journals (which was at the time the 2013 issue).

And the results, sadly, were the same. There was no statistically significant decrease in the use of WEIRD samples between the 2003 – 2007 data and the 2013 data. Evolutionary psychology has had 10 years to get its act together, and it has completely failed. Which is a real shame because this is a field with so much potential. It could tell us so much. Yet at the moment it’s often being done badly; and it doesn’t show any signs of changing.

References

  1. Graham, S. (1992). ” Most of the subjects were White and middle class”: Trends in published research on African Americans in selected APA journals, 1970–1989. American Psychologist, 47(5), p.629.

  2. Sears, D. (1986). College sophomores in the laboratory: Influences of a narrow data base on social psychology’s view of human nature. Journal of personality and social psychology, 51(3), p.515.
  3. Henrich, J., Heine, S. and Norenzayan, A. (2010). Beyond WEIRD: Towards a broad-based behavioral science. Behavioral and Brain Sciences, 33(2-3), pp.111–135.
  4. Arnett, J. (2008). The neglected 95%: why American psychology needs to become less American. American Psychologist, 63(7), p.602.
  5. Henrich, J., Heine, S. and Norenzayan, A. (2010). The weirdest people in the world?. Behavioral and brain sciences, 33(2-3), pp.61–83.
  6. Buss, D. (2008). Evolutionary psychology. 3rd ed. Boston: Allyn and Bacon.

Related posts




5 Comments

Cosmic Life · 6th January 2015 at 5:32 pm

Well, this is depressing! I would like to study some EP since it would be absolutely fascinating but I’m waiting for the quality of research to increase first. What’s the use of buy a textbook about it if the majority of the studies it references are biased and flawed in both sampling and methodology?

To be fair, research in EP sounds arduous consider how broad your samples must be to accurately represent the human population. Sounds like doing a proper study would take a lot of money and time.

P.S. Thanks for properly using “raises the question” instead of “begs the question”. It’s a pet peeve of mine ^_^

    Adam Benton · 6th January 2015 at 5:39 pm

    I would never discourage anyone from studying the subject. As I say in the closing, it’s fascinating. We just need more people going into the field aware of these problems and willing to fix them.

    And it’s not like the studies on WEIRD groups are completely worthless. They can be a quick and cheap way to test a lot of ideas. Like testing drugs on mice before embarking on human trials. The issue is more that nobody seems to perform those human trials. “Oh we looked at college students and found everyone loves beards. Let’s end there. No more research needed.”

    P.S. Don’t give me too much credit, I made that mistake before and had it corrected by another commenter. Now I know better.

Andre Salzmann · 7th January 2015 at 3:40 am

It is really a pity such research could not be done in South Africa. It is most probably
the perfect sample set. We must have 4 to 6 distinguishable groups.

Would it not be possible for a USA or UK institution to pilot such research and then
to co-op a university here and take properly controlled samples here?

Sound work in this respect could benefit the world tremendously. I think. We all
need objective insight.

I live close to a university and could perhaps assist with linking such academics? Masters
and Phd students? Our academic year is about to begin within a week or so.

As it is I am the at present the chairman of a local civil rights organisation and have
the necessary contacts. there could even be a possibility of this organisation contributing
some funding for such work. No promises now however. I would have to take that up
with the top structure of course.

    Adam Benton · 7th January 2015 at 2:07 pm

    I wouldn’t go so far as to say that South Africa is a perfect sample set; but I do think it could be critically important. It offers a different (and perhaps more diverse) sample than you would see in most WEIRD countries. It’s not a perfect representative sample of all humanity, but definitely a step in the right direction

    But crucially research there could arguably be conducted relatively cheaply. One of the common complaints against using non-WEIRD samples is that it costs too much. If people could work with South African institutions to lower those costs it could be hugely beneficial.

    It might go something like you conduct the initial research really cheaply in America. See if there is a real effect worth studying. Then spend a bit more money to vindicate it in South Africa (or similar locale). See if it still holds up. Then maybe conduct a more widespread and more expensive study.

Andre Salzmann · 11th January 2015 at 11:00 am

If I can be of assistance, please let me know.

Leave your filthy monkey comments here.

This site uses Akismet to reduce spam. Learn how your comment data is processed.