IA partner interviewed on leading psychology blog

Standard
This week our partner Leigh Caldwell was interviewed by InDecision – a blog aimed mainly at academic researchers in the field of psychology of decision making. Every so often they feature  a practitioner applying the science in the commercial world and previous interviewees have included e.g. Rory Sutherland from Ogilvy Change, Matthew Willcox from advertising agency Draftfcb and behavioural finance expert Daniel Egan from Betterment. (Original interview here.)

Chicago headshot 1

This week in our practitioner series we’re featuring Leigh Caldwell who is a behavioural economist and founding partner of pricing research consultancy Irrational Agency. He’s been applying decision making research commercially since the mid-2000s, making him quite an early adopter of this discipline, and is also active in academic economic research, working in the emerging field of cognitive economics. He has founded and run several businesses in technology and professional services, and recently condensed his experience in pricing and marketing these businesses into a new book The Psychology of Price. He is also the sub-editor for our upcoming interview series on applications of decision-making psychology in economics and public policy.

Tell me about your work: how does decision making psychology fit in it? I see my work as scaling up. I start from decision-making research that applies to individuals, and expand it into an understanding of how groups of people, companies or markets or whole economies, operate.

As a consultant, I do this for companies who want to know how to design a pricing or marketing strategy while taking into account consumer psychology. As a researcher, I do it with economic theory, building models of how markets work and how economies experience growth or recessions.

To do either of these jobs calls for mathematical models – models of how people behave which strike the balance between being psychologically realistic, but simple enough to work with. Old style economics went too far down the simplification route; but modern empirical psychology doesn’t produce simple models. So my work involves figuring out just how much simplification is enough, then doing the mathematics to expand it to an economic scale.

This field as a whole is called cognitive economics. Its goal is to build models of the economy that are based on a realistic foundation of how people really make decisions, and to bring an understanding of positive psychology into economic modelling – how people get utility or happiness from non-material goods, by modifying their cognitive state. It tackles questions like: what determines whether a company invents a new product or competes in an existing category? Why do companies make profits when economics says all profit should be competed away? Why are people unemployed? Why do people invest and save and borrow in the way they do? When people can get psychological benefit from intangible things, why do they still rely so much on material possessions? These are all really important questions which traditional economics can’t answer. Cognitive economics uses the discoveries of decision-making psychology to figure out why these things happen.

How did you first become interested in decision making psychology? I was running a business, a software company, and had been trying to work out for years how much money I should charge for our products. I could tell that my customers weren’t making decisions through rational cost-benefit analysis, so I wanted to know what else was going on. The same pattern showed up when we built software – whenever people started using it, they insisted on ignoring the “correct” processes and used it in whatever way they felt like. The Sheldon Cooper in me was frustrated by all this irrationality. I had to figure out what was going on!

I had read lots of marketing books with some foundations in folk psychology – anything from Dale Carnegie to Ries & Trout – but none of them seemed very scientific. As a mathematician and programmer with an economics background, my natural approach was to try to build a predictive model of people’s behaviour and figure out what was going on. When I started looking into the psychology research, I found out that there were plenty of researchers examining the same kinds of problems…but no coherent structure for how to apply the discoveries either to economics or business. That was where I discovered my niche. I decided to start applying this science, first in my software business and eventually set up a new business selling pricing advice. Having got involved along the way in academic research in order to find these answers, it seemed natural to keep working both on new research and on business applications.

What type of research do you find most interesting, useful or exciting? Like everyone, I’m entertained by the range of provocative topics people study in this discipline: the psychology of online dating, whether people called Michel are more likely to buy Michelin stock, or how easily people can be manipulated into saying the opposite of what they apparently believe – there’s always something fun.

But I’m always more interested in foundational work. This field is full of ad hoc papers, with lots of experiments focusing on individual standalone phenomena. Those are all fine in their own right, but they are hard to apply to real world problems. You need to do a new experiment every time you want to investigate anything. Theoretical work that unifies a spectrum of different results into a smaller set of principles makes it easier to solve new problems. That kind of work is what really fascinates me.

Do you see any challenges to the wider adoption of decision making psychology in your field? There’s resistance from the economics side of the discipline – many economists insist that people are fundamentally rational, even if they make occasional mistakes in their decisions. Their idea is that all the mistakes basically cancel out, or disappear once consumers learn to overcome them. That is part of why I want to turn all the disparate effects in this field into a unified theory: to find out whether our general cognitive limitations have an impact on the efficiency of markets or on whether societies end up rich or poor.

From the business side, the issue isn’t any direct resistance, just a lack of rigour and knowledge. Businesses are often run on superstition more than on evidence. The barrier here is inertia: a concerted effort will be needed to persuade companies and governments to take up these ideas. Fortunately, capitalism provides an incentive to make that effort – there are big rewards awaiting the agencies or consultancies who can win that role as a bridge from science to business.

How do you see the relationship between academic researchers and practitioners? Tenuous.

Two other interviewees in this series responded to this question with the word “symbiotic”. That’s true, but it’s also idealised. In reality, the culture – or to be technical, the habitus – of these two worlds are so different that it’s hard for them to work together. So far.

Academics mostly agree that it’s a good thing to make their work relevant for business or public policy applications, but many of them don’t have a clear idea of how to do that. (Business schools are a major exception – I’ve been impressed by the decision-making research conducted in the top business schools.) However, academics who are hired as consultants often struggle to make their work have an impact. Consultancy needs to be followed up by strong and simplified implementation steps in order to work, and academics rarely enjoy distilling their work in that way. Then again, that’s true of most commercial consultancy too.

Businesspeople are more skeptical of the potential for collaboration. No pharmaceutical company would deny the importance of rigorous biochemical science in creating their products, but it wouldn’t occur to most of them that decision-making science is relevant to their marketing and pricing too. I don’t think this means they’re anti-academic or anti-science, just that they don’t understand it and so it is easier for them to rely on gut feeling and intuitive judgment in this area. Quite a lot of my commercial work ends up being about translating scientific concepts into business language, and then demonstrating why business should be more open to using scientific methods and knowledge.

Mostly, the interface between the two worlds is limited to popular science books, a few intrepid people from marketing agencies who visit academic conferences, and the occasional consulting contract for a professor somewhere. I would love to be part of changing this. Right now we have two separate worlds and a few people who occasionally cross the border between them.

Imagine we could create a continuous spectrum instead: at one end theoretical academic research on mathematics or abstract models, then empirical research testing hypotheses, then an “engineering” discipline who knows the science and also how to implement it in business, through to marketing departments who use what the engineers have developed, all the way to operations or finance departments who could become aware of how to incorporate consumer psychology in the service they deliver and the way they bill for it. That would transform the practice of both business and academia.

How do we get there? Maybe we all need to apply some decision making psychology to understanding our own barriers and how to change our own behaviour.

What advice would you give to young researchers who might be interested in a career in your field? The areas I work in get their richness and value from the interplay of three disciplines: marketing, empirical psychology and economics. Researchers who are interested in pricing and other business applications will want to understand how people work in business as well as the scientific process of psychology and the modelling and mathematics that comes into economics.

For example, if you’re an economist and haven’t done empirical psychology work before, try getting involved in some pricing experiments at a business school so you can see how that works. If you’re a psychology researcher who hasn’t worked in a company, try working with a small business to try to redesign the pricing of their products or services. To get a feel for practical applications in pricing, you might also want to take a look at Priceless by William Poundstone (or my book). And if you’re a practitioner who wants to bring more science into your work, go along to some academic conferences or seminars just to get a feel for how people work.

Sometimes people are worried that they won’t understand the science (or the economics) or the maths will be too difficult. Try it anyway and just understand as much as you can. People in other fields aren’t any cleverer, they just speak a different language – the people who work in that field learned it, and you could learn it too if you wanted to.

Practical applications aside, if you’re interested in cognitive economics research, you will have to have an independent spirit. There are not many people working in the field yet, so you probably won’t find a supervisor who specialises in it. You can get in touch with me and I can help you identify a list of papers to start with, and then see what kind of research question you’re interested in. I think cognitive economics will be an increasingly important field and this is a good time to get into it; but it is always more challenging to work in an emerging field because the directions of research and the conventions aren’t clear yet.

 

 

Advertisements

Science vs. storytelling, and why it matters to us as researchers

Standard

In the past year or so, the world of psychology has been rocked by scandals of scientific fraud involving fabricated data and dubious analysis techniques, among other things. The ensuing debates within the discipline have uncovered a host of other problems such as positive result publication bias, which means studies that have positive results supporting a hypothesis are favoured by academic journals.

Additionally, studies with surprising or counterintuitive findings tend to have a better chance of being published, which is good news for both the journal (as more people are likely to download the paper and earn the journal publisher some money) and the academic, who might get publicity in mainstream media as a result – not bad news at all for one’s CV!

Another recent and heated debate revolves around the proposed solution to the issue of scientific fraud: replication. The credibility of science as a whole rests on the replicability of its findings, i.e. whether another researcher is able to repeat a particular study and reproduce the same findings. If that isn’t the case, well, we should have less faith that the original research revealed anything meaningful about the world. The only problem is that academic journals are more interested in novel findings with a significant contribution to the discipline rather than publishing results that confirm (or disprove) the results of existing research while those with negative results which might disprove a theory are less likely to be published – leading to a bias towards studies which contain false positives.

Why is all this relevant to us as market researchers?

The rising interest in using behavioural economics and other findings from the ‘brain industry’ in market research has meant that more and more people are reading popular science books like Predictably Irrational, or perhaps the currently slightly less fashionable stuff by writers like Malcolm Gladwell or Jonah Lehrer. Lists of best social psychology books provide plenty of food for thought and are a great resource for getting to grips with these topics if you don’t have a background in psychology.

Popular books about social science tend to fall into two camps. Those written by journalists, who summarise research findings from a range of other people, and those written by the scientists who are actually carrying out the research. However, while books by “real” scientists like Ariely or Kahneman are just as likely to have been weaved into an engaging narrative, it’s less  likely that the findings they report have been selectively cherry-picked and moulded into a story-study-lesson model to support their ‘big idea’ – a common issue with many titles in the section Waterstone’s now calls ‘Smart Thinking’. (For a detailed critique on the rise of “brain pseudoscience” in general, see here.)

Why does it matter? 

We are naturally more drawn to simple narratives that make it easier to understand and remember things. This also makes us more susceptible to accept more engaging science stories as “truth”, simply because it’s easy to understand. However, most theories and concepts in psychology are not straight-forward or unambiguous, and usually involve numerous limitations on the generalisability of the findings.

While simplifying theories to make a topic more accessible to a wider readership is acceptable, there is a danger that, similarly to academic journals, only the most sensational and counterintuitive findings make the final cut, which can then distort or bias the conclusions we as readers take from the book. If we then let these ideas guide and inform our work as market researchers, we are introducing an additional bias into the work we do for clients.

So what should we do?

The most important thing is to stay critical of new research findings – especially the more sensational ones. Real science needs validation and replication before we can truly believe what it’s telling us about the world.

In the context of behavioural economics, there are even concerns within the behavioural economics academic sphere that some of the well-known effects and biases may not, in fact, operate in the way we have thought, or at least their existence may have been exaggerated by publication bias. But, if our only knowledge of the field comes from findings weaved into a narrative form, we remain unaware of the critique around these theories as well as what the limitations of the research might be.

Given all this, it’s a good idea to treat new theories and ideas with caution and question whether, given our own experiences of the world, they sound plausible or not. And if we really want to take something further and integrate it into the work we do for clients, it’s good practice to check how widely accepted a certain theory or idea really is and maybe even understand its limitations.

Yes, that takes time.  But do we not owe it to our clients?  Behavioural economics and other academic research has the potential to  make our market research practice better, more accurate and more insightful – we just need to make sure we use it correctly.

We need to be storytellers, but not at the expense of science.