Through the looking glass: an agency researcher goes to the ESOMAR client conference

DZNXl4XWAAEFJQqLast week I had the pleasure of seeing what life is on the other side of the research fence when I attended the ESOMAR World Inspiration Network conference in Amsterdam where the only presenters were clients.

After attending 40+ industry conferences where agencies mostly speak to other agencies about their work, I was really looking forward to hearing exclusively client-side perspectives into our industry. As a cherry on top, the conference was held at the top of Heineken Experience – a quick 10min bike ride from my home in Amsterdam!

Overall it was a highly insightful experience and I felt like I learned more than I do in an average conference so hats off to the programme committee and ESOMAR for putting together a great programme! The two days were packed with great talks and I’ll share some highlights from the conference through my tweets at the time.

The day was kicked off by Programme Committee Chair Silke Muenster from Philip Morris International who reminded us, the handful of clients in the room, of our differing perspectives – of course, as agencies, we focus on research, but it was a good wakeup call first thing in the morning that we must spend more time stepping into the shoes of the clients we work with.

Sanita Pinchback’s keynote explored the different challenges Unilever has from a HR perspective. One particular slide on cultural differences in expressing emotion caught my attention – I completely agree with that, but one thing that market research is currently missing (somewhat) is valid and up to date theoretical understanding of how cultural context influences emotional expression. I realise that that discussion will be quite technical and academic, but without a solid theoretical understanding of the phenomenon you want to measure, increasingly sophisticated technology will be just shiny new things – not necessarily better.

Sanita also talked about the amount of internal training and upskilling they do at Unilever – the number of sessions and learning topics was incredible! Although we already invest heavily in learning internally, this really hammered home just how important it is for us to stay on top of the latest developments in our industry – after all, research is our business so if clients are upskilling, we need to do it too.

Another talk that really made me think about our work on agency side was by Tony Costella on how the insights function is changing from “farmers to fishers” – in other words, in addition to “farming” commissioned custom research to also “fishing” around in readily available behavioural data.

As a part of that change, he talked about the changing needs for the skills profile required of an insights professional: while it was perfectly acceptable to have a T-shaped profile with deep knowledge in a specific area and an ability to work across a broad area outside that, these days that’s no longer enough. Instead, we need people with an M-shaped skills/knowledge profile: multiple deep skills combined with an ability to apply across situations and domains.

That resonated particularly strongly because I feel like we need even more than that on the agency side: not only is it enough for a small specialist agency to have deep knowledge on our specialism (behavioural science), we also need to understand clients’ industries so that we can more easily apply our expertise across different areas. Adding to that, we also need to stay top of technical innovations such as AI, chatbots, machine learning etc. Finally, even the deep specialist knowledge in behavioural science isn’t just one “prong”: solving client problems requires a broad knowledge of a wide range of theories and concepts across different fields of behavioural science. As the saying goes, if you only have a hammer, every problem looks like a nail.

One great practical idea I took from the conference was from the presentation of Olga Kornilova from Ferring Pharmaceuticals. Insights Vernissage as an alternative to expressing themes from research was really engaging, and Olga did a great job of bringing this method to life.

The best quote of the day was definitely the above: the best way to have a good idea is to have a lot of ideas – and I certainly got a lot of ideas from ESOMAR WIN and am already looking forward to next year!


If you found this post interesting, we’d be grateful if you’d nominate us as an Innovative Supplier in the GRIT industry survey. Just click here, answer the survey (it should take 10-15 minutes) and put Irrational Agency as an innovative supplier. In return for completing the survey, GreenBook will give you early access to the report when it’s published.

Going beyond the gilded cage – addendum

This post is an addendum to an article originally published in the January 2016 issue of Impact magazine. (Registration required)

This month’s Impact magazine features an article I wrote about gender stereotypes in the market research industry. The article was inspired by a study we did for Women In Research (in collaboration with the Market Research Society) to understand nonconscious gender perceptions in the market research industry through a wide-scale experiment.

As there is limited space in a magazine, I couldn’t share as much of the results as I would have liked, so I wanted to write a post to complement the print article (with a couple of paragraphs repeated so that this post makes sense to readers who have not yet seen the print article or are from other industries). I also received a surprising amount of feedback about the methodology from researchers so I felt it would be good to provide more detail about that we well. I’ll start by explaining what we actually did, followed by the reasons why. I will also explain some of the specific biases we are naturally susceptible to, and how they influence our impressions of other people. I’ll then include snapshots of some of the results and conclude with some further reading.

What we did

A survey invitation was sent to all MRS members, yielding 440 responses (37% male/63% female). Over half of the sample had worked in the industry over 15 years, and were therefore at least director level with both quantitative and qualitative experience. Two-thirds came from agency side, with a relatively balanced split between different sized agencies.

Some of the feedback I received was concerned that we biased the results by mentioning Women in Research on the landing page, and therefore giving away the purpose of the research. That point is very valid, but in this case it would not have been fair to not disclose the organisation that was mainly driving the initiative (WIRe, MRS kindly agreed to partner with them) and also because the purpose would have revealed itself sooner or later anyway. We were fully aware of this and actually it was not a concern to us if it made gender perceptions and/or discrimination more salient in people’s minds – quite the opposite. The reason for that is that this study was more an experiment than a traditional survey, and as is typical of psychological experiments, we did not want to reveal this upfront which has subsequently caused some confusion with many people.


Stage 1

CVs tested

In the first stage we showed people 2 CVs that were meticulously anonymised from real CVs provided by recruitment agency Hasson Associates. Each respondent saw one male and one female CV (Simon/Susan Taylor/Adams) with the order of genders and CVs randomised, and was asked to rate them on a number of dimensions.


Some of those who sent me feedback felt that the CV did not give them enough information to rate them on e.g. ease of working with, and  felt it was difficult to give an opinion. This exercise was actually modelled on a relatively well known case study from Columbia ​Business School​ called the Heidi/Howard case where they asked students to read a CV for an entrepreneur called Heidi or for an entrepreneur called Howard, and subsequently rate them on several dimensions. What they found was that “students ​​rated Heidi ​​as highly competent and effective as Howard, but they evaluated her as ​​unlikeable and ​​selfish, and ​​​would not have want​ed​ to hire her or work with her.​”​ (More on this and similar work on the likability-competence dilemma women have here.) I agree that perhaps both the original and our version could have been designed better, but I felt it was nonetheless interesting to replicate such a well known case study in our industry. ​At least in the original case, people had very strong feelings a​bout the person based on their CV, but I appreciate that they did so in a different situation than in the context of a survey like this. Nevertheless, this question worked as an explicit benchmark, demonstrating a rational response, as the female candidate received more favourable ratings across the board.

Stage 2

statements from recruitment profiles
Statements tested

In the second stage, we showed respondents phrases that might appear in a recruitment agency’s cover letter of a candidate (based on real examples provided by Hasson Associates) and asked them to first person, in absence of other information, they would want to interview for a job, followed a task where they had to choose whether a phrase was more likely to describe a male or a female candidate. These were formulated to test various hypotheses of whether certain behaviours, skills, attributes and achievements were more credible for either gender, and also to understand how they might impact one’s likelihood to get invited to an interview. While the second part of this stage is more directly linked to gender perceptions, the data also gives us interesting and useful insights into what is valued by employers.

More on the likeability-competence dilemma and other issues with gendered behaviour norms here:


Stage 3

attributes used
Attributes tested

In the third stage, we drilled down to specific words and asked respondents in which word would be more positive to appear in a performance review, followed by a task to choose whether a word would be more likely to appear in Simon or Susan’s review.

There is a fair amount of academic research that suggests most people have at least some level of association for different words with specific genders which affects both men and women. There’s also plenty of research supporting the idea that internalised stereotypes can influence both genders’ behaviour and particularly performance: for example, countries with high gender inequality girls tend to do worse in math than boys, whereas the reverse is true in countries with low inequality. In fact, there’s also evidence that simply priming people by making their gender (or race) salient can influence their performance in intellectual tasks​ s​​​o it’s quite a complex issue with widespread societal consequences​.​

More on gendered language use and social norms here:



Why we did it

Despite the ​extensive academic research supporting these ideas, I felt it was important to ground ​them ​in our own context – even if this research is not flawless, it’s at least something we can start a conversation with​ and​ I was personally very excited to see the results as I feel passionately about appreciating everyone as who they are without the need to factor in their gender. I’m still sometimes just as biased myself though – it’s only human, and we can’t really help it however much we’d like.


At the beginning of the article I mention this riddle which is designed to reveal gender stereotypes:

A father and son are in a horrible car crash that kills the dad. The son is rushed to the hospital and just as he’s about to go under the knife, the surgeon says, “I can’t operate—that boy is my son!”

An old riddle like this is a good example of the impact gender schemas can have on our thinking. Most people are puzzled by it, and suggest wildly creative options instead of the most obvious one: that the surgeon is a female and the boy’s mother. One particular place where these stereotypes come to play is career choices, progression and recruitment. Everyone likes to think of themselves as enlightened but in reality we all store a range of belief patterns in our minds. We learn these schemas in childhood because they help us generalise and explain life, even though they do not necessarily reflect our personal values. Despite what many think, gender stereotypes also do not just affect women: when the riddle was changed, most people struggled to imagine that a nurse could indeed be a man. Curiously, after writing the article, I came across a real life example that hopefully demonstrates that we are all equally prone to these gender schemas – including women.
HONY screenshots
These are screenshots of a Facebook discussion that ensued after one particular Humans Of New York post about Syrian refugees who have been approved to enter the US. It was sparked by a woman realising her own prejudices (assuming the doctor in the story was the man), followed by comments from several female doctors (some of whom additionally also either identify as Muslim or are of the Middle Eastern origin) who had – much to their shock – also made the same assumption.

Some people were also concerned that was it was easy for people to answer tactically when they (assumed they) had figured out the research. That point is very valid that it was indeed aimed at understanding stereotypes, but not necessarily in the way that some people have thought. Stereotypes affect people’s decision making in many subtle ways – including the stereotypes women hold of themselves, which were of particular interest to us as this research was conducted on the back of conversations at the previous WIRe event about the impostor syndrome and especially how women sometimes hold themselves back – which is in fact part of what we found.

It’s also not simply a question of discrimination against women like some people have assumed: these stereotypes can also limit men in what behaviour is perceived to be acceptable for them. If, for example, being described creative is more associated with women than men, then that’s also an issue for a man who is, in fact, more creative. However, the first step to taking action in any direction or trying to make a difference is to understand what exactly is it that we are dealing with.

The short-cuts we use to judge people

The truth is that we are all cognitive misers who like to preserve mental energy and processing capacity who only spend it when we really have to – not because we are lazy, but because there is simply too much going on for our brains to handle it.  The limitations of our minds mean that all human thought is a trade-off between speed and accuracy, and most often we choose speed because that kept our ancestors out of lions’ mouths on the savannah. While we have now mostly accepted that we have fast and slow ways of thinking about brands, we can also apply it to how we perceive people.

In the first stage, our automatic, effortless System 1 uses heuristics and other mental shortcuts to grasp the essence of whatever we are presented with. Just like we are unaware of these shortcuts when thinking about brands, we are also not aware of how they alter the way we perceive and interpret information about people before ever realising it. Unfortunately, research suggests that we don’t even need to believe in a stereotype for it to affect our thinking – they happen automatically if we are simply aware of them. The second stage of perceiving people where we correct and suppress our prejudices is much more effortful and therefore ultimately the kingdom of System 2 which the cognitive miser would ultimately prefer to avoid altogether.

The cognitive miser in us loves certain short cuts in particular when it is evaluating people. First impressions count because the information we get about someone early on when observing them affects our perception of them more than any information we get later on (the primacy effect). These interpretations can be hard to change because we have a tendency to reconcile the psychological pain conflicting beliefs cause us by changing them (cognitive dissonance).  Unfortunately, first impressions are usually out of own hands because confirmation bias filters our perception to see in others what we already expect to see. Stereotypes are one type of filter: by categorising things they ease the load on our minds and help us make sense of the world. They are beliefs about categories of people – some positive, some negative – that are often more like probabilities based on our experiences: on average, we’ve seen more women with long hair then men, so a stereotypical woman has longer hair than a man.

Just like we can’t help but make snap judgments about brands and products, we automatically make snap judgments about people whether we like it or not. And just like it’s not humanly possible to stop System 1 thinking altogether, we also can’t stop being influenced by stereotypes even if we wanted to – they are our way of making sense of the world. Thinking that we are immune from it only makes us more likely to fall prey to it, so our best chance is to accept it. But before that, we need to be aware of the prejudices we have.

It was clear from the research that market researchers are a very enlightened bunch when it comes to gender perceptions, and unlike many industries qualities associated with women are also ones that we value. While this is great news for diversity, we shouldn’t lull ourselves into a false sense of security that gender equality is not a challenge for us: both men and women all put themselves and each other in clear boxes. These perceptions and labels can make us miss both existing talent inside our companies as well as when we are recruiting for new people. Stereotypes can indeed be valuable guides in navigating a complex world, but we should not ourselves become blinkered.

Diving into the results

These charts are copied from the presentation at this fall’s Women in Research event – some quality issues emerged when moving screenshots to blog. There is, of course, even more data than this -if you are interested in anything specific, please drop me a note and I’ll dig it out!

sample details.png
Who we spoke to

sample details 2.png
Where respondents were from

cv results.png
How the CVs were rated


what gets you in an interview 1
What to say if you want to get into an interview…

what gets you in an interview 2
…and what not to say.

how we put each other in boxes
How we put each other in boxes – overall perceptions of each gender, and descriptions that could apply to both

how women are limiting their ambitions
How each gender sees itself

Valued words in a performance review.png
Which words are most positive in a performance review?

words seen as positive by gender
Words seen as most positive by gender

top 5 words to come up in a review for each gender.png
Top 5 words that are seen as more likely to come up in either a man’s or a woman’s performance review


matrix as an image for blog

How the attributes map out by gender association and valence

Some further reading

Banaji, M. R., & Greenwald, A. G. (2013). Blindspot: Hidden biases of good people. Delacorte Press.

Devine, P. G. (1989). Stereotypes and prejudice: their automatic and controlled components. Journal of personality and social psychology56(1), 5.

Halvorson, H. G. (2015) No One Understands You And What To Do About It. Cambridge: Harvard Business Review Press.

Mueller, J. S., Goncalo, J. A., & Kamdar, D. (2011). Recognizing creative leadership: Can creative idea expression negatively relate to perceptions of leadership potential?. Journal of Experimental Social Psychology47(2), 494-498.

Pratkanis, A. R. (1988). The attitude heuristic and selective fact identification.British journal of social psychology27(3), 257-263.

Steele, C. (2011). Whistling Vivaldi: And other clues to how stereotypes affect us (issues of our time). WW Norton & Company.


Behavioural Polling: The UK Election

This post originally appeared on RWConnect on 21st May 2015

UK readers will already be fed up of hearing about it, but the rest of the world might not yet be aware of how spectacularly the British opinion polls failed to predict the results of our general election this month. Google will happily provide you with a long series of post-mortems on what went wrong, but if you start with Jane Bainbridge, Wikipedia and Peter Kellner via Ben Farmeryou’ll get a decent introduction to what happened.

Although something clearly went wrong, the polling companies deserve to be defended against some of the wilder accusations made in blog posts and tweets over the last two weeks (mostly by people outside the market research industry, but occasionally by insiders too). Many of the issues that have been blamed for the failure to predict the election are well-known to pollsters, who make strenuous attempts to correct for them:

  • the difficulty of sampling voters who are hard to reach by telephone
  • skew in the demographics of voters who are online and respond to polls
  • different turnout rates of people who support different parties
  • respondents’ tendency to overestimate their likelihood of voting
  • “Shy Tories” and “shy kippers” – people who feel the interviewer might disapprove of their choice, so avoid reporting their intention to vote Tory or UKIP

It does seem possible that some of these factors were estimated wrongly by the polling models this time round, for reasons that remain to be seen. But I am no more expert in how to correct for these factors than you are (and possibly less), so I’ll leave those to the experts at ICM, Ipsos MORI, YouGov and the rest.

I can, however, offer a contribution from psychology and behavioural economics – including an understanding of the tendency for people to say one thing and do another. Understanding and adjusting for this tendency, with an approach we would call behavioural polling, is a key opportunity for the polling industry.

The (fairly accurate, though not exact) exit polls from 7th May had the advantage of asking respondents about concrete historical behaviour (very recent behaviour, moreover) instead of predictions of future actions. It is unsurprising therefore that they were more accurate than the polls before the election. However, the better we can understand the differences between predicted and actual behaviour, the more accurate our behavioural polls will be.

Why might people have said they would vote Labour and then not do so? Or vote Conservative despite claiming they wouldn’t?

  • Because they changed their mind in the last few days
  • Because they were lying
  • Because they wrongly predicted what they would do
  • Because action is different from belief
  • Because they ended up not voting at all.

Each of these reasons has a corresponding explanation in behavioural science: the biases and heuristics at play include false certainty, overconfidence, status quo bias, social norm effects, ambiguity aversion, and others. We can explore them all by designing our research methods accordingly – as always in research, getting the right answer depends on asking the right question in the right way.

One potential change is around priming, to put the respondent in the right mindset and get them thinking in the same way they might think in the ballot box. Another is to replicate the ballot box context – the physical and mental process of choosing a candidate, putting a cross in a box and placing the ballot anonymously into a sealed box is quite specific, and activates idiosyncratic connotations and habits that may not occur in an online or telephone poll.

The effort required to get to the polling station and vote is another difference between survey and reality, and methods which replicate that effort barrier will better understand the likelihood of voting. The impact of last-minute newspaper headlines or TV coverage can also be estimated through simulation of hypothetical messages.

A further gap between prediction and reality is the tendency of respondents to underestimate the impact of emotional and heuristic processes on their own behaviour. Just as many people insist that they are not influenced by brand advertising, despite the evidence of the sales figures of Unilever and Coca-Cola, voters will tell us that Daily Mail headlines or bacon sandwich photographs would never influence their vote. In reality, they can have a subconscious effect on behaviour of which the respondent is not even aware. These subconscious influences can be measured with implicit tools and the effect on voting behaviour estimated with some precision.

It would be expensive to incorporate all of these methodological adjustments in a single poll, let alone in the whole series of polls that take place in the months leading up to a general election. So we can borrow another technique from scientific research and split samples into two treatments: a control group using existing techniques and an experimental group using the new method. By measuring the difference in results between the groups, we can estimate the size of impact of each specific behavioural effect. These effects can then be applied as adjustments to conventional polls.

The adjustments won’t always be simple, as the strength of each effect may vary according to the candidates on offer and the mood of the voter, but with a series of experiments it will be possible to develop a model of how to adjust the prediction to better match reality.

Although we may not be able to determine the success of a full behavioural polling cycle for five more years, there are plenty of elections in the meantime in which to test new approaches. There will be elections in London, Wales, Scotland, Northern Ireland or for local councils every year between now and 2020. We plan to partner with a polling company to test these new methods in those elections, so that 2020 won’t be a repeat of 2015 – in terms of polling accuracy at least, if not political outcome!

Leigh Caldwell is a consultant and writer on pricing and cognitive economics and partner at The Irrational Agency

What to do with decision theories?

This post originally appeared on RWConnect on 4th March 2014

In my last post, I explained three broad theories of human decision-making which different psychologists put forward as alternative explanations for the discoveries of behavioural economics.

decision ahead - proceed slowlyHeuristics and biases assumes that we’re basically meant to be rational but some design flaws get in the way; adaptive toolbox says that there’s no such thing as rationality, we just use a collection of ad hoc mental tools to solve problems as best we can; and information processing viewpoints propose an underlying mechanism in our minds, which act as a sort of imperfect computer to make decisions based on the information the world throws at us.

In this article I won’t try to answer which model might be better, but let’s say you’ve decided which one best fits the consumers you’re studying. What should you do next?

Each of these models gives you a way to understand how customers are making the decision to buy, or not to buy, your (or your client’s) product. Each of them also challenges the traditional, unstated, powerful assumption that most market researchers make about consumer decisions.

You may be one of the rare marketers who does not make this assumption. While others do not know they are making it. But the assumption is revealed every time a questionnaire asks a respondent how much they like something, and every time an interviewer says “Tell me why you bought that.”

What is this hidden assumption? It is this: that the decision process is based on consumers buying products that they like, because of the attributes of the product, and knowing why they do so. In other words, that it’s an economically rational process. All three parts of this assumption are disproven by behavioural economics. The three models each give an alternative description of the decision and buying process, and this process is where researchers can focus their efforts in order to make behavioural economics work for their clients.

The heuristics and biases model says that the basic economically rational process is still the right framework, but consumers make errors of judgement in deciding how much they like things. If you follow this model, you should still look at product attributes but you should also look at how they are communicated and what context they are in, because this affects the value that consumers place on them. By understanding these context effects, you can learn how to emphasise the attributes on which your client’s product is strongest, which contexts it will do best in, and how to segment consumers by their biases and contexts. A methodology for analysing this process can be based on listing product attributes, enumerating the contexts in which the product (and its competitors) will be seen, and using a cognitive biases list as a checklist for how consumers might be influenced to put a higher or lower value on each attribute.

If you use the adaptive toolbox model, you will need to instead think of how consumers might make a good-enough decision about this product category – how they will satisfy themselves that the product they are choosing is OK and meets most of their needs. There are a number of standard heuristics or rules of thumb that consumers typically take to do this. One is to pick the one most important product attribute and focus on that (this might be price, or something category-specific like miles per gallon). Another is to copy what their friends or peers are doing. A third is to do what they did last time, if there were no significant negative consequences. You can find out which rule consumers are using by observing them, asking them (in the right way!), or testing their behaviour in a controlled experiment which is designed to distinguish between these rules. When you know the rule(s) they use, you’ll know the basic parameters about how to design product communications, how to position and price the product and how to change consumer behaviour for the better.

Finally, the information processing model says that consumers make their decisions by gathering information in pursuit of a goal. To analyse this process, you would start by understanding what information the consumer already has – this both forms a baseline to ask what new information they will seek out, and influences the goals they choose to satisfy. In this model, other drives such as emotions and preferences are seen as specific types of information. Then consider the capabilities of the consumer to gather information, the sources they use, the way they interpret and combine new facts with existing knowledge, and some basic parameters like how quickly they can read and interpret information, and their rate of consumption of social or other media. You can then quantitatively and qualitatively determine which (and how many) products each consumer might consider. You can also use the same framework to estimate which contextual, product or communication factors they are most likely to focus on when making their final choice.

Whichever model you use, the tools of traditional market research can fit into it. These new frameworks give a new way to think about the consumer and how to understand them, but they do not in themselves provide new methodologies. Qualitative approaches, survey research, concept tests, panels, eye tracking, ethnography and all the rest can fit as important tools into any of the three models. However, by starting from an understanding of how the consumer thinks – informed by behavioural economics research – you’ll have a more powerful and effective way to use those tools and achieve business results for your clients.

Leigh Caldwell is a consultant and writer on pricing and cognitive economics and partner at The Irrational Agency

How Behavioural Economics Points to Something Deeper

This post originally appeared on RWConnect on 28th November 2013

lightbulb-headReaders of this blog are likely to be already familiar with many of the experimental results of behavioural economics (BE). Discoveries such as anchoring, hyperbolic discounting, loss aversion and other cognitive biases are now quite well-known in the market research world. Each of them comes with its own tricks for how to influence consumers, or pitfalls to look out for when designing questions. (those who are less familiar can find out lots more about them from some of the leading BE books:Predictably Irrational by Dan Ariely, Thinking Fast and Slow by Daniel Kahneman or Basic Instinctsby Pete Lunn).

These experiments and their associated influence tricks are the most visible aspect of behavioural economics as a field. And they’re useful too – but only in specific circumstances. Most research projects don’t have a specific need for an understanding of hyperbolic discounting or loss aversion. To put this into practice in market research, we’d like to have a clearer set of rules about what BE says about consumer insight.

There is a more powerful way to look at the empirical discoveries of BE. As well as standalone discoveries, they are also a set of clues to deeper and more important underlying insights about how people think and decide. These general lessons are applicable in many different situations – and can lead us towards finding the specific biases, limitations, heuristics or methods of influence that apply to our own consumers.

The drawback is: there is no single theory of how people make decisions. Scientific psychologists, working backwards from the results of experiments, have come up with a number of alternative frameworks. They aren’t mutually exclusive – think of them as different, valid, ways to look at the world. As a researcher or marketer, you may want to understand more than one of these models in order to decide which one to use in a particular project.

In this article I’ll briefly look at three of the leading theories of decision making. Each of these can be useful in understanding how consumers think about, and hopefully how they decide to buy, your clients’ products.

The first is the information processing model of Payne, Bettman and Johnson. This theory says that when we make a decision, we have to process the information available to us by using a series of smaller individual steps. The steps include small tasks such as estimating how good a product is, comparing two different products, or choosing to look for more information before making the decision. When confronted with a choice such as which car to buy, we decide on a strategy, gather and process more information until we’re ready to make the decision, and then choose one of the options.

Payne and Bettman also propose that while doing this, we are governed by “meta-motives” or goals that we want to satisfy during the decision-making process itself. There are four possible meta-motives:

  •  maximising decision accuracy
  •  minimising cognitive effort
  •  minimising negative emotions such as regret or anxiety
  •  being able to justify our decision to others

Different people focus on different meta-goals, so in order to appeal to the widest set of consumers, your clients might want to communicate in several different ways to match these four decision-making styles.

A second theory is the fast and frugal heuristics approach of Gerd Gigerenzer, Peter Todd, Ralph Hertwig and other researchers in the “ABC” school. This theory says that we have a toolbox of standard mental shortcuts which we use in different situations. For instance, in evaluating products we might use the “Take The Best” rule, which say that we first compare the available products on their most important feature; if one is clearly the best product on this dimension, that’s the one we buy; otherwise we move onto the second most important feature, and so on. Collectively, these shortcuts are known as the adaptive toolbox and they are thought to have been developed by evolutionary pressures as near-optimal solutions for tricky or dangerous environments.

The third model is the modified expected utility (or subjective expected utility) approach, which says that we take a generally “rational” view of our decisions – roughly estimating our expected outcome from each option, and picking the one that seems best – but subject to some modifications or approximations such as avoidance of risk. Under this theory, we mostly avoid risky options, those which might lead to a negative outcome or those whose outcomes are ambiguous, and therefore act in a relatively conservative way. The prospect theory model of Kahneman and Tversky is an example of this approach.

Other models include decision field theory (Busemeyer and Townsend), which says that we gradually “drift” towards a decision as we randomly consider various aspects of the different options available to us; decision by sampling (Neil Stewart), which suggests we compare options with randomly selected experiences from memory and see whether they appear to be better or worse than those memories; and ACT-R (John Anderson), which is less a theory of decision processes than a model of the structure of the mind, and is often used to simulate various different decision approaches and find which best matches the behaviour of real individuals.

Sometimes “theories” that we hear about, such as “nudge theory” are not general theories as such, but collections of techniques for influencing decisions. Nudge theory, as well as most of the experimental observations of behavioural economics, is compatible with several of the above models.

Any of the above frameworks can be used to understand more about how your respondents make decisions either in a real purchase context or during your research process. However, instead of a list of dozens of cognitive biases, you now have several competing decision making frameworks to choose between – a partial improvement but still no clear answer. So in future posts, I’ll suggest ways to unify these into a practical approach you may be able to use in your daily work.

Leigh Caldwell is a consultant and writer on pricing and cognitive economics and partner at The Irrational Agency

Globally irrational or locally rational?

This post appeared originally on RWConnect on 28th November 2013

Why we need to understand cultural context when applying behavioural economics

The increase in research in markets such as Asia and Latin America makes understanding the impact of cultural context on consumer decision making more important than ever before. While quantitative researchers have long accepted that survey research is affected by culture through phenomena such as acquiescence bias or extreme response styles, cultural differences have far more diverse and wide-ranging implications for marketing and market research.

Behavioural economics is being enthusiastically adopted across the market research industry all over the world. Researchers everywhere are applying insights from decision making science and embracing the concept that we’re all a little bit irrational. But are we irrational in the same way?

We’re so WEIRD
Much of the research on decision making that is in the public realm has been conducted almost exclusively in Western countries and especially in the US. This means that we are implicitly assuming that these cognitive biases are universal and function largely in similar ways in different cultures. However, 96% of samples in psychological studies come from countries with only 12% of the world’s population, which means that a randomly selected American is 300 times more likely to be a research participant than is a randomly selected person from outside of the West. These countries are commonly referred to as WEIRD (Western, Industrialised, Educated, Rich and Democratic), which makes them vastly unrepresentative as a sample in psychological research.

Even though cross-cultural research into decision making is still in its infancy, a growing body of evidence suggests that behavioural economics as a field will hugely benefit from it as differences between cultures help unpack the deeper foundations of behaviour. Given the emphasis of many cognitive decision making theories on the impact of immediate context such as framing or priming, it’s surprising how little culture is taken into account. While social psychology has a wealth of knowledge on how cultural context affects us, theories in cognitive psychology rarely consider culture as a factor due to implicit assumptions about the universality of cognitive processes: according to these researchers, what we think about may vary, but how we think is always the same.

The Muller-Lyer illusion

However, research has shown even fundamental cognitive functions such as how we perceive colour can significantly differ based on the cultural context you’ve grown up in. Similarly, perception of seemingly simple optical illusions such as Müller-Lyer arrows have been shown to vary across cultures and even age groups.

Through the looking glass of culture
When talking about culture, we often refer to aspects such as values, attitudes, social norms, beliefs and traditions. However, despite long-standing debates within academia, no commonly accepted definition of culture actually exists. Instead, researchers tend to focus on certain aspects of culture depending on their area of interest and the phenomena they are investigating.

On a general level there is a wealth of evidence that economic, social and linguistic environments strongly shape people’s behaviour, motivations and preferences: for example, a study investigating time discounting (i.e. whether we value immediate rewards more than those in the future) in 45 countries found that differences at country level related to wealth and education as well as cultural factors such as individualism, the importance of tradition and whether time was conceptualised as linear or cyclical. However, without a unifying framework of conceptualising culture research such remains too scattered and almost makes it harder to grasp the bigger themes underlying cultural differences. A more effective way of understanding culture’s impact on how BE biases work differently in different countries is to look at some measurable differences between cultures which doaffect how a person’s cognition works while they make decisions. While other frameworks exist, one of the most powerful ones is a person’s self-concept.

“Me, myself and I” vs. “All together now”
The most widely analysed dimensions of culture are individualism and collectivism. Often discussed in the context of Geert Hofstede’s Taxonomy of Cultural Differences, these dimensions have received a lot of attention both among academics and practitioners. In a nutshell, individualism is characterised by detachment from relationships and community with the individual seeing himself as relatively independent from others, whereas collectivism is characterised by the importance placed on relationships, roles and status within the social system, with the individual seeing himself inseparable from his network of social relations.

However, at the level of the individual, these cultural mindsets affect how we see the world through organising the information we have about ourselves, directing our attention to information that is perceived to be relevant, shaping motivations and influencing how people appraise situations that influence their emotional experiences. These self-concepts can be placed on a continuum between two poles: independent and interdependent selves. Independent self-concepts are typically more prevalent in individualistic countries, whereas interdependent ones tend to be more common in collectivistic ones, although variation exists within countries.

Those termed independent define themselves through internal attributes such traits, abilities, personal values and preferences, and see behaviour as being under the control of the individual, arising from internal attributes such as preferences (e.g. what you buy reflects your identity). Conversely, those termed interdependent define themselves through relationships with others and don’t necessarily see behaviour as a reflection of internal traits but situated in a specific context – your preferences might radically change depending on what social circumstances you are in.

This has profound consequences for some of the fundamental concepts in consumer psychology such as cognitive dissonance: if you see your behaviour reflecting your true self, which is ideally consistent across time and circumstances, holding two or more conflicting ideas will make you feel uncomfortable. However, if you instead assume that your preferences merely reflect the current social circumstances and can therefore change from one moment to the next, conflicting ideas will not pose a threat to your identity, which means the concept of cognitive dissonance exerts much less power on consumers in e.g. East Asian cultures. As cognitive dissonance, often seen as irrational, is commonly used in advertising, understanding the extent to which it is prevalent in the cultural context is crucial to efficient marketing communications.

I am what I buy… or am I?
Whether or not we see ourselves as separate individuals or intertwined with others is also important in understanding consumer choice. Is choice an individual endeavour, reflecting our internal attributes or one that takes other people into account and says little about the chooser? In Western cultures, choice is seen as an act of self-expression: uniqueness is desirable and choices are a way to paint a portrait of yourself for the outside world, so we vary our choices in an attempt to gain a sense of “specialness”. In behavioural economics, this is termed as diversification bias where we seek variety in both what and how we choose which may sometimes lead to seemingly irrational behaviour. However, the majority of the research on this effect has been conducted in Western countries and specifically in the US where personal choice is almost one of the key cultural values.

When choice is an act of self-expression, it becomes hugely important for the individual, and the psychological impact of either lack of choice or failed choice is larger, which leads to strategies such as variety-seeking. However, in in collectivistic cultures choice is often an interpersonal task which means the success or failure of making a choice that portrays oneself in the most positive light is not as big a concern. Subsequently, recent research has shown that the diversification bias is weaker in these cultural contexts.

Self-concepts also affect the strength of another well-known behavioural economics concept: theendowment effect, where simply owning an object enhances its perceived worth, and owners value objects substantially (and irrationally) more than potential buyers do. Because owning an object activates an association between it and the self, the Western focus on self-enhancement means this association automatically boosts the object’s value. Therefore, the strength of the endowment effect is influenced by the degree to which self-enhancement is culturally valued, with recent research suggesting that the effect is indeed stronger in a Western context. In practice this means that sales tactics such as free trial or “bait and switch” may be less effective in these non-Western contexts with weaker endowment effect combined with weaker cognitive dissonance.

Fifty shades of irrationality
Understanding the potential cultural influences on thought is crucial for everyone attempting to accurately describe and predict consumers’ decision making.  Insights from behavioural economics might well be applicable in different cultures, but we need to have highly nuanced sense of the specific characteristics of each cultural context and its impact on consumer decision making to ensure effective applications. As behavioural economics professor Dan Ariely notes, the biggest challenge for the field in the next 10 years is understanding the generality of the findings so far and to what extent the effects discovered carry over in different contexts. As market research gradually abandons the error of rationality and adopts more principles from behavioural economics, let’s make sure we don’t entrench a new mistake: universality.

Elina Halonen is a founding partner of the Irrational Agency and editor at InDecision blog.

What’s life like on the other side of the behavioural fence?

Last week we co-hosted (along with Hunting Dynasty) an event to celebrate the second anniversary of the London Behavioural Economics Network. To mark the special occasion, we invited two illustrious speakers to discuss life on both sides of the behavioural fence: commercial and academic. Representing the academic side, we had George Loewenstein from Carnegie Mellon University and for the commercial side, there was only one option – Rory Sutherland.

2014-04-23 20.17.23

Thanks to Brian Tarran from the Market Research Society, we can share with you a transcript of the event (link to our video at the end of this post). Brian also wrote an excellent review of the event which you can read on Research Live.


George Loewenstein: Why is it that British electrical outlets have a switch on them? If you don’t have anything plugged in, why bother to turn it off, and if you do have something plugged in, why bother to turn it off?
Rory Sutherland: I’ve always seen it as a failing of other countries that you don’t have a switch. But there may have been also, bizarrely… if you go back to the early days of electricity, there was a considerable fear that if you left the switch on with nothing plugged in the electricity would leak into the room. So it might have been a device by the electricity boards to prevent people from being paranoid.
GL: I’ve read in various places that you studied classics at Cambridge and taught classics. Now, I don’t want to rake the coals on any deep regrets that you feel about leaving teaching, but I’m wondering if you’ve gained any insights from your career in advertising that would apply to education?
RS: A classical education is one of those things that works obliquely. It’s also worth asking: is a large component of education signalling, rather than value? Your level of educational attainment correlates very well with your career outcome if you get a paid job. It doesn’t correlate at all with people who are self-employed. This raises the question: is there a large amount of education that is effectively a spiralling, competitive credentialism? We don’t have any evidence that people with OxBridge firsts are better employees than people with firsts from Leeds. There’s no real empirical evidence that your degree class predicts your level of value to your company. There’s an aspect, which I know is a very cynical one, who argue that this is in a sense a peacock’s tail thing; basically it’s a three-year long IQ test combined with proof of commitment.
GL: Take something like drop-outs, or students not studying as hard as you’d like them to. Are there any lessons from behavioural economics or the marketing world that would discourage and encourage those people?
RS: There are some interesting questions about education. Does it disproportionately privilege certain personality types, those that are particularly well-suited to the system they find themselves in. One of the interesting things about an ad agency is that, as a place to work, it is a complimentary mix of skills. You need the beard-stroking Oxbridge intellectuals dotted around the place, but what you find in the most successful ad agencies is that the people who make up the management of that organisation tend to come from an extraordinary mix of educational and social background. That’s not just a politically correct, nice thing to have, it is a source of competitive advantage.
GL: That’s a nice segue into a question I was going to ask, about the new master’s programmes that are beginning in the UK, focusing on behavioural economics. Is this the kind of training that you would value at Ogilvy, or are you looking for the pot-smoking… what was it?
RS: They are not mutually exclusive, of course. You can have an MSc in behavioural science and a severe pot habit.
RS: One of the strange things is, you would think that advertising agencies and clients are full of behavioural scientists and psychologists, devising the next evil way in which to mislead people into buying stuff. I must admit, I was rather hoping to find that kind of thing when I went into the business myself. I always assumed I’d be able to turn some strange corner and find a room full of people attaching electrodes to rats. That room does not exist. But the reason private enterprise looks like it’s good at behavioural science is that it stumbles on things by accident. It happens to produce things that work. Capitalism is sort of semi-Darwinist: if you stumble on something that is disproportionately successful, it makes money, so you tend to expand that particular area. Now, some time after the early Mad Men era, the links between academic psychology and advertising and marketing were actually quite strong. This was in the late-50s, early-60s, when there were various people dotted around ad agencies with possibly fake, possibly genuine Viennese accents, who claimed to have met Sigmund Freud. These people came up with ideas, like how ‘plink, plink, fizz’ would create a social norm around using two Alka-Seltzer rather than one. They came up with lines like, ‘How else can a month’s salary last a lifetime’ for DeBeers, which was an extraordinarily good piece of framing in terms of anchoring what a man should spend on a commitment device for his future wife. They stumbled on these things by accident, but they made no effort whatsoever to codify it. What the business world is very good at is stumbling on successes by accident, but it’s terrible at making sense of them. We had all the information to create a science of behavioural economics back in the 50s and 60s, but we squandered it because nobody tried to make sense of it.
GL: How do you use behavioural economics in your practice? Can you give an example of a campaign that wouldn’t have happened without behavioural economics.
RS: The most important thing about behavioural economics is the widespread awareness among clients and agency people that these things are important. Before, in modelling and attempting to predict consumer behaviour, we tended to rely on the two pairs of broken binoculars: neo-classical economic assumptions, which are quite seriously wrong in some areas, and market research. Enormous business and government decisions are based on these two flawed lenses. Neo-classical economics tends to assume that psychology isn’t matter – which, as a result, creates an imaginary fantasy world of perfect information, perfect trust and perfect efficiency where marketing and advertising needn’t exist. But there are loads of things for which there is no rational reason. Stripy toothpaste, say – let’s get really trivial. There is no rational reason for stripy toothpaste because once you put it into your mouth it all gets mixed up. Nobody in market research would ever say that toothpaste should be stripy. No neo-classical economist would argue that there was any benefit to it. But someone stumbled on this idea that it’s much easier for people to believe a toothpaste does three things if there are three different visible components. That kind of thing, just understanding something that would never have emerged from research or neo-classical economics – those kind of things can make the difference between triumph and disaster. If all I do is get people to understand that, and get people to accept that it is worth testing things that don’t appear to make any sense to test, that’s all you have to do.
GL: The stereotype of advertisers and marketers is that they are playing on motives like power, sex, fear and so on. Is that true, and to what extent is behavioural economics capturing the motives that really drive people to purchase products?
RS: I’m very uncomfortable with the use of the word irrational, because an awful lot of behaviour that is currently seen as irrational is meta-rational or evolutionary rational. It’s the product of evolved psychological instincts that may or may not be useful in the modern world. For example, people’s willingness to pay a premium for a famous brand is rational if you understand a bit of reputational game theory, which is that someone with a valuable reputation has more to lose from selling a bad product than someone you’ve never heard of. There are lots of mechanisms that we employ instinctively, actually make very good sense. I also think loss aversion, in evolutionary terms, needs to be better understood. Rationality is always about maximising something, but I would argue that any sensible evolved system is not going to care about whether something is good or really good, but it would desperately care about the chances of something being fatally terrible. If you look at it from the absence of bad, rather than presence of good, an awful lot of human decision making makes an awful lot of sense. The idea that rationality means getting the best of something… I mean, you can go to a Michelin-starred restaurant and spend the afternoon on the bog, but say what you like about Maccy D’s but you don’t get ill there. Satisficing is what most people do all the time to the extent where, when people don’t satisfice there’s probably something going wrong. When people maximise, it tends to be in competitive situations where people are thinking as game theorists.
GL: But sometimes what appears to be maximising, isn’t maximising at all. I had a colleague who always tried to maximise the gain from any kind of negotiation and it was a disastrous strategy…
RS:… Because no-one would negotiate with him?
GL: That’s right. Another case is, if you always want to get the parking spot closest to the theatre, you’re going to waste a lot of time looking for a parking place. I’m actually a big believer in satisficing, but really a believer that satisficing can be a form of maximising. On the idea of loss aversion being connected with having evolved, it would have to be a very crude kind of evolution that produced that effect, because we’re loss averse over very small amounts: like gain $10/lose $10, or gain $15/lose $10 – most people are going to turn that down. But $10 has no prospect of killing you, so you would have to say it’s something we learned through evolution and then generalised inappropriately.
RS: Do you accept the claim that some psychologists make that the term behavioural economics steals for economics credit that is owed to psychology?
GL: There are a lot of psychologists who are annoyed at economics, and are annoyed at behavioural economics specifically. A lot of the work on changing health behaviours, for instance, was first done by psychologists, didn’t get a huge amount of attention, and then behavioural economists started doing it, and have gotten a tremendous amount of attention since then. Psychologists are, rightfully in my opinion, bothered about that. In fact, a lot of psychologists have started rebranding themselves as behavioural economists and getting more attention. I think they are right to have a grudge, but I guess you could tell us that it’s all a matter or marketing, and behavioural economists are much better at the marketing game.
RS: I got into this weird Twitter row, and Nicholas Christakis, I think, had the last word on the question when he said, “Look, there is a branding problem with psychology, which is very simple: the president of the United States cannot have a council of psychological advisors – but he can have a council of economic advisors”. On the other hand, for reasons I don’t fully understand, economics has completely disproportionate influence in business, and especially in policy making, out of all the other social sciences. Part of my theory is that economics is the one academic discipline where it is ok to be greedy. My brother is an astrophysicist, and if he went off to work in a high-frequency trading firm for two years and make a fortune and then try to get back into astrophysics, his career would be dead because he’d sold out. Economics, because it preaches self interest as a virtue, getting two-year gig at Citibank for £300,000 a year is positively career enhancing. Is that a valid theory?
GL: I think there is a good reason why economists are so tied into policy, because a lot of economics is oriented towards policy. But I actually want to pick up on a little piece of what you talked about, which is greed. When economists talk about greed they misuse the term. For an economist, greed is self interest. But when lay people use the term, they are using it in a very different sense. For a lay person, greed is self interest taken too far; to the point where it is destructive. So, it’s fine in my opinion to be self interested, but the danger is where you take it too far.
RS: We do have a social epistemology, and our idea of the good life is probably massively informed by what we think other people’s idea of the good life is. So, I think the extent to which we decide individually is a really interesting question. Although we’re consciously determined to see ourselves as individuals, I think our vision of what’s good is massively driven by our assumptions of what other people think.
GL: Exactly, and I recently co-authored a paper on mattering maps. When you’re in a particular situation, let’s say a social group, there’s typically something that matters more than anything else. Among academics it might be publications. If you are a musician it might be how well you play a piece. But one of the interesting features of mattering maps is they can change so abruptly as a result of whatever social milieu you’re in. So, I couldn’t agree more with you, that whatever we’re seeking at any given time is completely socially determined. The think we have some kind of free will about is how we’re going to seek it out. But we don’t really have a lot of control over what we’re seeking.
RS: We won’t get into free will and determinism here, because that’s probably a bit heavy. But a final thing: there is this attack on behavioural economics that it is just a collection of findings, or anomalies – that it is stamp collecting for psychologists. Critics say, ‘Where is the unifying theory?’ But if you’re dealing with psychology, human behaviour, human health – any of these complex emergent phenomena – is there ever going to be a unifying theory?
GL: Let me just say that something I’ve noticed about Brits is that you find pretension toxic. And what you don’t realise is that Rory is actually making fun of me with this question…
RS: No, I’m not!
GL: …because I was having breakfast with him and I told him that I was working with Nick Chater on a theory of everything. I actually meant it a little bit tongue in cheek, but not totally. So Rory’s making fun of me. But, ok, the first thing is that it’s kind of surprising as a behavioural economist to be questioned about the desire for a unified theory, because we’re always being attacked by traditional economists who say, ‘We have a unified theory and you’re just a bunch of disparate findings’. My view is that traditional economics is not nearly as unified as it claims to be because whatever phenomenon the economist is looking at, the utility function metamorphoses to deal with the phenomena that’s important to them. But I do think that a unifying theory, even a unifying mathematical theory, can be a beautiful thing. And, that’s what social science is all about: trying to take disparate social phenomena and come up with a unified account. That’s when I get a chill up my spine, when disparate things come together. So my aspiration is for behavioural economics to be grounded on a unified theory, or maybe a few unified theories.


For a video of the event, click here.

To sign up to the next London Behavioural Economics Event:

Join the LBEN Facebook group:

IA partner interviewed on leading psychology blog

This week our partner Leigh Caldwell was interviewed by InDecision – a blog aimed mainly at academic researchers in the field of psychology of decision making. Every so often they feature  a practitioner applying the science in the commercial world and previous interviewees have included e.g. Rory Sutherland from Ogilvy Change, Matthew Willcox from advertising agency Draftfcb and behavioural finance expert Daniel Egan from Betterment. (Original interview here.)

Chicago headshot 1

This week in our practitioner series we’re featuring Leigh Caldwell who is a behavioural economist and founding partner of pricing research consultancy Irrational Agency. He’s been applying decision making research commercially since the mid-2000s, making him quite an early adopter of this discipline, and is also active in academic economic research, working in the emerging field of cognitive economics. He has founded and run several businesses in technology and professional services, and recently condensed his experience in pricing and marketing these businesses into a new book The Psychology of Price. He is also the sub-editor for our upcoming interview series on applications of decision-making psychology in economics and public policy.

Tell me about your work: how does decision making psychology fit in it? I see my work as scaling up. I start from decision-making research that applies to individuals, and expand it into an understanding of how groups of people, companies or markets or whole economies, operate.

As a consultant, I do this for companies who want to know how to design a pricing or marketing strategy while taking into account consumer psychology. As a researcher, I do it with economic theory, building models of how markets work and how economies experience growth or recessions.

To do either of these jobs calls for mathematical models – models of how people behave which strike the balance between being psychologically realistic, but simple enough to work with. Old style economics went too far down the simplification route; but modern empirical psychology doesn’t produce simple models. So my work involves figuring out just how much simplification is enough, then doing the mathematics to expand it to an economic scale.

This field as a whole is called cognitive economics. Its goal is to build models of the economy that are based on a realistic foundation of how people really make decisions, and to bring an understanding of positive psychology into economic modelling – how people get utility or happiness from non-material goods, by modifying their cognitive state. It tackles questions like: what determines whether a company invents a new product or competes in an existing category? Why do companies make profits when economics says all profit should be competed away? Why are people unemployed? Why do people invest and save and borrow in the way they do? When people can get psychological benefit from intangible things, why do they still rely so much on material possessions? These are all really important questions which traditional economics can’t answer. Cognitive economics uses the discoveries of decision-making psychology to figure out why these things happen.

How did you first become interested in decision making psychology? I was running a business, a software company, and had been trying to work out for years how much money I should charge for our products. I could tell that my customers weren’t making decisions through rational cost-benefit analysis, so I wanted to know what else was going on. The same pattern showed up when we built software – whenever people started using it, they insisted on ignoring the “correct” processes and used it in whatever way they felt like. The Sheldon Cooper in me was frustrated by all this irrationality. I had to figure out what was going on!

I had read lots of marketing books with some foundations in folk psychology – anything from Dale Carnegie to Ries & Trout – but none of them seemed very scientific. As a mathematician and programmer with an economics background, my natural approach was to try to build a predictive model of people’s behaviour and figure out what was going on. When I started looking into the psychology research, I found out that there were plenty of researchers examining the same kinds of problems…but no coherent structure for how to apply the discoveries either to economics or business. That was where I discovered my niche. I decided to start applying this science, first in my software business and eventually set up a new business selling pricing advice. Having got involved along the way in academic research in order to find these answers, it seemed natural to keep working both on new research and on business applications.

What type of research do you find most interesting, useful or exciting? Like everyone, I’m entertained by the range of provocative topics people study in this discipline: the psychology of online dating, whether people called Michel are more likely to buy Michelin stock, or how easily people can be manipulated into saying the opposite of what they apparently believe – there’s always something fun.

But I’m always more interested in foundational work. This field is full of ad hoc papers, with lots of experiments focusing on individual standalone phenomena. Those are all fine in their own right, but they are hard to apply to real world problems. You need to do a new experiment every time you want to investigate anything. Theoretical work that unifies a spectrum of different results into a smaller set of principles makes it easier to solve new problems. That kind of work is what really fascinates me.

Do you see any challenges to the wider adoption of decision making psychology in your field? There’s resistance from the economics side of the discipline – many economists insist that people are fundamentally rational, even if they make occasional mistakes in their decisions. Their idea is that all the mistakes basically cancel out, or disappear once consumers learn to overcome them. That is part of why I want to turn all the disparate effects in this field into a unified theory: to find out whether our general cognitive limitations have an impact on the efficiency of markets or on whether societies end up rich or poor.

From the business side, the issue isn’t any direct resistance, just a lack of rigour and knowledge. Businesses are often run on superstition more than on evidence. The barrier here is inertia: a concerted effort will be needed to persuade companies and governments to take up these ideas. Fortunately, capitalism provides an incentive to make that effort – there are big rewards awaiting the agencies or consultancies who can win that role as a bridge from science to business.

How do you see the relationship between academic researchers and practitioners? Tenuous.

Two other interviewees in this series responded to this question with the word “symbiotic”. That’s true, but it’s also idealised. In reality, the culture – or to be technical, the habitus – of these two worlds are so different that it’s hard for them to work together. So far.

Academics mostly agree that it’s a good thing to make their work relevant for business or public policy applications, but many of them don’t have a clear idea of how to do that. (Business schools are a major exception – I’ve been impressed by the decision-making research conducted in the top business schools.) However, academics who are hired as consultants often struggle to make their work have an impact. Consultancy needs to be followed up by strong and simplified implementation steps in order to work, and academics rarely enjoy distilling their work in that way. Then again, that’s true of most commercial consultancy too.

Businesspeople are more skeptical of the potential for collaboration. No pharmaceutical company would deny the importance of rigorous biochemical science in creating their products, but it wouldn’t occur to most of them that decision-making science is relevant to their marketing and pricing too. I don’t think this means they’re anti-academic or anti-science, just that they don’t understand it and so it is easier for them to rely on gut feeling and intuitive judgment in this area. Quite a lot of my commercial work ends up being about translating scientific concepts into business language, and then demonstrating why business should be more open to using scientific methods and knowledge.

Mostly, the interface between the two worlds is limited to popular science books, a few intrepid people from marketing agencies who visit academic conferences, and the occasional consulting contract for a professor somewhere. I would love to be part of changing this. Right now we have two separate worlds and a few people who occasionally cross the border between them.

Imagine we could create a continuous spectrum instead: at one end theoretical academic research on mathematics or abstract models, then empirical research testing hypotheses, then an “engineering” discipline who knows the science and also how to implement it in business, through to marketing departments who use what the engineers have developed, all the way to operations or finance departments who could become aware of how to incorporate consumer psychology in the service they deliver and the way they bill for it. That would transform the practice of both business and academia.

How do we get there? Maybe we all need to apply some decision making psychology to understanding our own barriers and how to change our own behaviour.

What advice would you give to young researchers who might be interested in a career in your field? The areas I work in get their richness and value from the interplay of three disciplines: marketing, empirical psychology and economics. Researchers who are interested in pricing and other business applications will want to understand how people work in business as well as the scientific process of psychology and the modelling and mathematics that comes into economics.

For example, if you’re an economist and haven’t done empirical psychology work before, try getting involved in some pricing experiments at a business school so you can see how that works. If you’re a psychology researcher who hasn’t worked in a company, try working with a small business to try to redesign the pricing of their products or services. To get a feel for practical applications in pricing, you might also want to take a look at Priceless by William Poundstone (or my book). And if you’re a practitioner who wants to bring more science into your work, go along to some academic conferences or seminars just to get a feel for how people work.

Sometimes people are worried that they won’t understand the science (or the economics) or the maths will be too difficult. Try it anyway and just understand as much as you can. People in other fields aren’t any cleverer, they just speak a different language – the people who work in that field learned it, and you could learn it too if you wanted to.

Practical applications aside, if you’re interested in cognitive economics research, you will have to have an independent spirit. There are not many people working in the field yet, so you probably won’t find a supervisor who specialises in it. You can get in touch with me and I can help you identify a list of papers to start with, and then see what kind of research question you’re interested in. I think cognitive economics will be an increasingly important field and this is a good time to get into it; but it is always more challenging to work in an emerging field because the directions of research and the conventions aren’t clear yet.



Science vs. storytelling, and why it matters to us as researchers

In the past year or so, the world of psychology has been rocked by scandals of scientific fraud involving fabricated data and dubious analysis techniques, among other things. The ensuing debates within the discipline have uncovered a host of other problems such as positive result publication bias, which means studies that have positive results supporting a hypothesis are favoured by academic journals.

Additionally, studies with surprising or counterintuitive findings tend to have a better chance of being published, which is good news for both the journal (as more people are likely to download the paper and earn the journal publisher some money) and the academic, who might get publicity in mainstream media as a result – not bad news at all for one’s CV!

Another recent and heated debate revolves around the proposed solution to the issue of scientific fraud: replication. The credibility of science as a whole rests on the replicability of its findings, i.e. whether another researcher is able to repeat a particular study and reproduce the same findings. If that isn’t the case, well, we should have less faith that the original research revealed anything meaningful about the world. The only problem is that academic journals are more interested in novel findings with a significant contribution to the discipline rather than publishing results that confirm (or disprove) the results of existing research while those with negative results which might disprove a theory are less likely to be published – leading to a bias towards studies which contain false positives.

Why is all this relevant to us as market researchers?

The rising interest in using behavioural economics and other findings from the ‘brain industry’ in market research has meant that more and more people are reading popular science books like Predictably Irrational, or perhaps the currently slightly less fashionable stuff by writers like Malcolm Gladwell or Jonah Lehrer. Lists of best social psychology books provide plenty of food for thought and are a great resource for getting to grips with these topics if you don’t have a background in psychology.

Popular books about social science tend to fall into two camps. Those written by journalists, who summarise research findings from a range of other people, and those written by the scientists who are actually carrying out the research. However, while books by “real” scientists like Ariely or Kahneman are just as likely to have been weaved into an engaging narrative, it’s less  likely that the findings they report have been selectively cherry-picked and moulded into a story-study-lesson model to support their ‘big idea’ – a common issue with many titles in the section Waterstone’s now calls ‘Smart Thinking’. (For a detailed critique on the rise of “brain pseudoscience” in general, see here.)

Why does it matter? 

We are naturally more drawn to simple narratives that make it easier to understand and remember things. This also makes us more susceptible to accept more engaging science stories as “truth”, simply because it’s easy to understand. However, most theories and concepts in psychology are not straight-forward or unambiguous, and usually involve numerous limitations on the generalisability of the findings.

While simplifying theories to make a topic more accessible to a wider readership is acceptable, there is a danger that, similarly to academic journals, only the most sensational and counterintuitive findings make the final cut, which can then distort or bias the conclusions we as readers take from the book. If we then let these ideas guide and inform our work as market researchers, we are introducing an additional bias into the work we do for clients.

So what should we do?

The most important thing is to stay critical of new research findings – especially the more sensational ones. Real science needs validation and replication before we can truly believe what it’s telling us about the world.

In the context of behavioural economics, there are even concerns within the behavioural economics academic sphere that some of the well-known effects and biases may not, in fact, operate in the way we have thought, or at least their existence may have been exaggerated by publication bias. But, if our only knowledge of the field comes from findings weaved into a narrative form, we remain unaware of the critique around these theories as well as what the limitations of the research might be.

Given all this, it’s a good idea to treat new theories and ideas with caution and question whether, given our own experiences of the world, they sound plausible or not. And if we really want to take something further and integrate it into the work we do for clients, it’s good practice to check how widely accepted a certain theory or idea really is and maybe even understand its limitations.

Yes, that takes time.  But do we not owe it to our clients?  Behavioural economics and other academic research has the potential to  make our market research practice better, more accurate and more insightful – we just need to make sure we use it correctly.

We need to be storytellers, but not at the expense of science.