In the past year or so, the world of psychology has been rocked by scandals of scientific fraud involving fabricated data and dubious analysis techniques, among other things. The ensuing debates within the discipline have uncovered a host of other problems such as positive result publication bias, which means studies that have positive results supporting a hypothesis are favoured by academic journals.
Additionally, studies with surprising or counterintuitive findings tend to have a better chance of being published, which is good news for both the journal (as more people are likely to download the paper and earn the journal publisher some money) and the academic, who might get publicity in mainstream media as a result – not bad news at all for one’s CV!
Another recent and heated debate revolves around the proposed solution to the issue of scientific fraud: replication. The credibility of science as a whole rests on the replicability of its findings, i.e. whether another researcher is able to repeat a particular study and reproduce the same findings. If that isn’t the case, well, we should have less faith that the original research revealed anything meaningful about the world. The only problem is that academic journals are more interested in novel findings with a significant contribution to the discipline rather than publishing results that confirm (or disprove) the results of existing research while those with negative results which might disprove a theory are less likely to be published – leading to a bias towards studies which contain false positives.
Why is all this relevant to us as market researchers?
The rising interest in using behavioural economics and other findings from the ‘brain industry’ in market research has meant that more and more people are reading popular science books like Predictably Irrational, or perhaps the currently slightly less fashionable stuff by writers like Malcolm Gladwell or Jonah Lehrer. Lists of best social psychology books provide plenty of food for thought and are a great resource for getting to grips with these topics if you don’t have a background in psychology.
Popular books about social science tend to fall into two camps. Those written by journalists, who summarise research findings from a range of other people, and those written by the scientists who are actually carrying out the research. However, while books by “real” scientists like Ariely or Kahneman are just as likely to have been weaved into an engaging narrative, it’s less likely that the findings they report have been selectively cherry-picked and moulded into a story-study-lesson model to support their ‘big idea’ – a common issue with many titles in the section Waterstone’s now calls ‘Smart Thinking’. (For a detailed critique on the rise of “brain pseudoscience” in general, see here.)
Why does it matter?
We are naturally more drawn to simple narratives that make it easier to understand and remember things. This also makes us more susceptible to accept more engaging science stories as “truth”, simply because it’s easy to understand. However, most theories and concepts in psychology are not straight-forward or unambiguous, and usually involve numerous limitations on the generalisability of the findings.
While simplifying theories to make a topic more accessible to a wider readership is acceptable, there is a danger that, similarly to academic journals, only the most sensational and counterintuitive findings make the final cut, which can then distort or bias the conclusions we as readers take from the book. If we then let these ideas guide and inform our work as market researchers, we are introducing an additional bias into the work we do for clients.
So what should we do?
The most important thing is to stay critical of new research findings – especially the more sensational ones. Real science needs validation and replication before we can truly believe what it’s telling us about the world.
In the context of behavioural economics, there are even concerns within the behavioural economics academic sphere that some of the well-known effects and biases may not, in fact, operate in the way we have thought, or at least their existence may have been exaggerated by publication bias. But, if our only knowledge of the field comes from findings weaved into a narrative form, we remain unaware of the critique around these theories as well as what the limitations of the research might be.
Given all this, it’s a good idea to treat new theories and ideas with caution and question whether, given our own experiences of the world, they sound plausible or not. And if we really want to take something further and integrate it into the work we do for clients, it’s good practice to check how widely accepted a certain theory or idea really is and maybe even understand its limitations.
Yes, that takes time. But do we not owe it to our clients? Behavioural economics and other academic research has the potential to make our market research practice better, more accurate and more insightful – we just need to make sure we use it correctly.
We need to be storytellers, but not at the expense of science.