Behavioural Polling: The UK Election

This post originally appeared on RWConnect on 21st May 2015

UK readers will already be fed up of hearing about it, but the rest of the world might not yet be aware of how spectacularly the British opinion polls failed to predict the results of our general election this month. Google will happily provide you with a long series of post-mortems on what went wrong, but if you start with Jane Bainbridge, Wikipedia and Peter Kellner via Ben Farmeryou’ll get a decent introduction to what happened.

Although something clearly went wrong, the polling companies deserve to be defended against some of the wilder accusations made in blog posts and tweets over the last two weeks (mostly by people outside the market research industry, but occasionally by insiders too). Many of the issues that have been blamed for the failure to predict the election are well-known to pollsters, who make strenuous attempts to correct for them:

  • the difficulty of sampling voters who are hard to reach by telephone
  • skew in the demographics of voters who are online and respond to polls
  • different turnout rates of people who support different parties
  • respondents’ tendency to overestimate their likelihood of voting
  • “Shy Tories” and “shy kippers” – people who feel the interviewer might disapprove of their choice, so avoid reporting their intention to vote Tory or UKIP

It does seem possible that some of these factors were estimated wrongly by the polling models this time round, for reasons that remain to be seen. But I am no more expert in how to correct for these factors than you are (and possibly less), so I’ll leave those to the experts at ICM, Ipsos MORI, YouGov and the rest.

I can, however, offer a contribution from psychology and behavioural economics – including an understanding of the tendency for people to say one thing and do another. Understanding and adjusting for this tendency, with an approach we would call behavioural polling, is a key opportunity for the polling industry.

The (fairly accurate, though not exact) exit polls from 7th May had the advantage of asking respondents about concrete historical behaviour (very recent behaviour, moreover) instead of predictions of future actions. It is unsurprising therefore that they were more accurate than the polls before the election. However, the better we can understand the differences between predicted and actual behaviour, the more accurate our behavioural polls will be.

Why might people have said they would vote Labour and then not do so? Or vote Conservative despite claiming they wouldn’t?

  • Because they changed their mind in the last few days
  • Because they were lying
  • Because they wrongly predicted what they would do
  • Because action is different from belief
  • Because they ended up not voting at all.

Each of these reasons has a corresponding explanation in behavioural science: the biases and heuristics at play include false certainty, overconfidence, status quo bias, social norm effects, ambiguity aversion, and others. We can explore them all by designing our research methods accordingly – as always in research, getting the right answer depends on asking the right question in the right way.

One potential change is around priming, to put the respondent in the right mindset and get them thinking in the same way they might think in the ballot box. Another is to replicate the ballot box context – the physical and mental process of choosing a candidate, putting a cross in a box and placing the ballot anonymously into a sealed box is quite specific, and activates idiosyncratic connotations and habits that may not occur in an online or telephone poll.

The effort required to get to the polling station and vote is another difference between survey and reality, and methods which replicate that effort barrier will better understand the likelihood of voting. The impact of last-minute newspaper headlines or TV coverage can also be estimated through simulation of hypothetical messages.

A further gap between prediction and reality is the tendency of respondents to underestimate the impact of emotional and heuristic processes on their own behaviour. Just as many people insist that they are not influenced by brand advertising, despite the evidence of the sales figures of Unilever and Coca-Cola, voters will tell us that Daily Mail headlines or bacon sandwich photographs would never influence their vote. In reality, they can have a subconscious effect on behaviour of which the respondent is not even aware. These subconscious influences can be measured with implicit tools and the effect on voting behaviour estimated with some precision.

It would be expensive to incorporate all of these methodological adjustments in a single poll, let alone in the whole series of polls that take place in the months leading up to a general election. So we can borrow another technique from scientific research and split samples into two treatments: a control group using existing techniques and an experimental group using the new method. By measuring the difference in results between the groups, we can estimate the size of impact of each specific behavioural effect. These effects can then be applied as adjustments to conventional polls.

The adjustments won’t always be simple, as the strength of each effect may vary according to the candidates on offer and the mood of the voter, but with a series of experiments it will be possible to develop a model of how to adjust the prediction to better match reality.

Although we may not be able to determine the success of a full behavioural polling cycle for five more years, there are plenty of elections in the meantime in which to test new approaches. There will be elections in London, Wales, Scotland, Northern Ireland or for local councils every year between now and 2020. We plan to partner with a polling company to test these new methods in those elections, so that 2020 won’t be a repeat of 2015 – in terms of polling accuracy at least, if not political outcome!

Leigh Caldwell is a consultant and writer on pricing and cognitive economics and partner at The Irrational Agency


What to do with decision theories?

This post originally appeared on RWConnect on 4th March 2014

In my last post, I explained three broad theories of human decision-making which different psychologists put forward as alternative explanations for the discoveries of behavioural economics.

decision ahead - proceed slowlyHeuristics and biases assumes that we’re basically meant to be rational but some design flaws get in the way; adaptive toolbox says that there’s no such thing as rationality, we just use a collection of ad hoc mental tools to solve problems as best we can; and information processing viewpoints propose an underlying mechanism in our minds, which act as a sort of imperfect computer to make decisions based on the information the world throws at us.

In this article I won’t try to answer which model might be better, but let’s say you’ve decided which one best fits the consumers you’re studying. What should you do next?

Each of these models gives you a way to understand how customers are making the decision to buy, or not to buy, your (or your client’s) product. Each of them also challenges the traditional, unstated, powerful assumption that most market researchers make about consumer decisions.

You may be one of the rare marketers who does not make this assumption. While others do not know they are making it. But the assumption is revealed every time a questionnaire asks a respondent how much they like something, and every time an interviewer says “Tell me why you bought that.”

What is this hidden assumption? It is this: that the decision process is based on consumers buying products that they like, because of the attributes of the product, and knowing why they do so. In other words, that it’s an economically rational process. All three parts of this assumption are disproven by behavioural economics. The three models each give an alternative description of the decision and buying process, and this process is where researchers can focus their efforts in order to make behavioural economics work for their clients.

The heuristics and biases model says that the basic economically rational process is still the right framework, but consumers make errors of judgement in deciding how much they like things. If you follow this model, you should still look at product attributes but you should also look at how they are communicated and what context they are in, because this affects the value that consumers place on them. By understanding these context effects, you can learn how to emphasise the attributes on which your client’s product is strongest, which contexts it will do best in, and how to segment consumers by their biases and contexts. A methodology for analysing this process can be based on listing product attributes, enumerating the contexts in which the product (and its competitors) will be seen, and using a cognitive biases list as a checklist for how consumers might be influenced to put a higher or lower value on each attribute.

If you use the adaptive toolbox model, you will need to instead think of how consumers might make a good-enough decision about this product category – how they will satisfy themselves that the product they are choosing is OK and meets most of their needs. There are a number of standard heuristics or rules of thumb that consumers typically take to do this. One is to pick the one most important product attribute and focus on that (this might be price, or something category-specific like miles per gallon). Another is to copy what their friends or peers are doing. A third is to do what they did last time, if there were no significant negative consequences. You can find out which rule consumers are using by observing them, asking them (in the right way!), or testing their behaviour in a controlled experiment which is designed to distinguish between these rules. When you know the rule(s) they use, you’ll know the basic parameters about how to design product communications, how to position and price the product and how to change consumer behaviour for the better.

Finally, the information processing model says that consumers make their decisions by gathering information in pursuit of a goal. To analyse this process, you would start by understanding what information the consumer already has – this both forms a baseline to ask what new information they will seek out, and influences the goals they choose to satisfy. In this model, other drives such as emotions and preferences are seen as specific types of information. Then consider the capabilities of the consumer to gather information, the sources they use, the way they interpret and combine new facts with existing knowledge, and some basic parameters like how quickly they can read and interpret information, and their rate of consumption of social or other media. You can then quantitatively and qualitatively determine which (and how many) products each consumer might consider. You can also use the same framework to estimate which contextual, product or communication factors they are most likely to focus on when making their final choice.

Whichever model you use, the tools of traditional market research can fit into it. These new frameworks give a new way to think about the consumer and how to understand them, but they do not in themselves provide new methodologies. Qualitative approaches, survey research, concept tests, panels, eye tracking, ethnography and all the rest can fit as important tools into any of the three models. However, by starting from an understanding of how the consumer thinks – informed by behavioural economics research – you’ll have a more powerful and effective way to use those tools and achieve business results for your clients.

Leigh Caldwell is a consultant and writer on pricing and cognitive economics and partner at The Irrational Agency

How Behavioural Economics Points to Something Deeper

This post originally appeared on RWConnect on 28th November 2013

lightbulb-headReaders of this blog are likely to be already familiar with many of the experimental results of behavioural economics (BE). Discoveries such as anchoring, hyperbolic discounting, loss aversion and other cognitive biases are now quite well-known in the market research world. Each of them comes with its own tricks for how to influence consumers, or pitfalls to look out for when designing questions. (those who are less familiar can find out lots more about them from some of the leading BE books:Predictably Irrational by Dan Ariely, Thinking Fast and Slow by Daniel Kahneman or Basic Instinctsby Pete Lunn).

These experiments and their associated influence tricks are the most visible aspect of behavioural economics as a field. And they’re useful too – but only in specific circumstances. Most research projects don’t have a specific need for an understanding of hyperbolic discounting or loss aversion. To put this into practice in market research, we’d like to have a clearer set of rules about what BE says about consumer insight.

There is a more powerful way to look at the empirical discoveries of BE. As well as standalone discoveries, they are also a set of clues to deeper and more important underlying insights about how people think and decide. These general lessons are applicable in many different situations – and can lead us towards finding the specific biases, limitations, heuristics or methods of influence that apply to our own consumers.

The drawback is: there is no single theory of how people make decisions. Scientific psychologists, working backwards from the results of experiments, have come up with a number of alternative frameworks. They aren’t mutually exclusive – think of them as different, valid, ways to look at the world. As a researcher or marketer, you may want to understand more than one of these models in order to decide which one to use in a particular project.

In this article I’ll briefly look at three of the leading theories of decision making. Each of these can be useful in understanding how consumers think about, and hopefully how they decide to buy, your clients’ products.

The first is the information processing model of Payne, Bettman and Johnson. This theory says that when we make a decision, we have to process the information available to us by using a series of smaller individual steps. The steps include small tasks such as estimating how good a product is, comparing two different products, or choosing to look for more information before making the decision. When confronted with a choice such as which car to buy, we decide on a strategy, gather and process more information until we’re ready to make the decision, and then choose one of the options.

Payne and Bettman also propose that while doing this, we are governed by “meta-motives” or goals that we want to satisfy during the decision-making process itself. There are four possible meta-motives:

  •  maximising decision accuracy
  •  minimising cognitive effort
  •  minimising negative emotions such as regret or anxiety
  •  being able to justify our decision to others

Different people focus on different meta-goals, so in order to appeal to the widest set of consumers, your clients might want to communicate in several different ways to match these four decision-making styles.

A second theory is the fast and frugal heuristics approach of Gerd Gigerenzer, Peter Todd, Ralph Hertwig and other researchers in the “ABC” school. This theory says that we have a toolbox of standard mental shortcuts which we use in different situations. For instance, in evaluating products we might use the “Take The Best” rule, which say that we first compare the available products on their most important feature; if one is clearly the best product on this dimension, that’s the one we buy; otherwise we move onto the second most important feature, and so on. Collectively, these shortcuts are known as the adaptive toolbox and they are thought to have been developed by evolutionary pressures as near-optimal solutions for tricky or dangerous environments.

The third model is the modified expected utility (or subjective expected utility) approach, which says that we take a generally “rational” view of our decisions – roughly estimating our expected outcome from each option, and picking the one that seems best – but subject to some modifications or approximations such as avoidance of risk. Under this theory, we mostly avoid risky options, those which might lead to a negative outcome or those whose outcomes are ambiguous, and therefore act in a relatively conservative way. The prospect theory model of Kahneman and Tversky is an example of this approach.

Other models include decision field theory (Busemeyer and Townsend), which says that we gradually “drift” towards a decision as we randomly consider various aspects of the different options available to us; decision by sampling (Neil Stewart), which suggests we compare options with randomly selected experiences from memory and see whether they appear to be better or worse than those memories; and ACT-R (John Anderson), which is less a theory of decision processes than a model of the structure of the mind, and is often used to simulate various different decision approaches and find which best matches the behaviour of real individuals.

Sometimes “theories” that we hear about, such as “nudge theory” are not general theories as such, but collections of techniques for influencing decisions. Nudge theory, as well as most of the experimental observations of behavioural economics, is compatible with several of the above models.

Any of the above frameworks can be used to understand more about how your respondents make decisions either in a real purchase context or during your research process. However, instead of a list of dozens of cognitive biases, you now have several competing decision making frameworks to choose between – a partial improvement but still no clear answer. So in future posts, I’ll suggest ways to unify these into a practical approach you may be able to use in your daily work.

Leigh Caldwell is a consultant and writer on pricing and cognitive economics and partner at The Irrational Agency

Globally irrational or locally rational?

This post appeared originally on RWConnect on 28th November 2013

Why we need to understand cultural context when applying behavioural economics

The increase in research in markets such as Asia and Latin America makes understanding the impact of cultural context on consumer decision making more important than ever before. While quantitative researchers have long accepted that survey research is affected by culture through phenomena such as acquiescence bias or extreme response styles, cultural differences have far more diverse and wide-ranging implications for marketing and market research.

Behavioural economics is being enthusiastically adopted across the market research industry all over the world. Researchers everywhere are applying insights from decision making science and embracing the concept that we’re all a little bit irrational. But are we irrational in the same way?

We’re so WEIRD
Much of the research on decision making that is in the public realm has been conducted almost exclusively in Western countries and especially in the US. This means that we are implicitly assuming that these cognitive biases are universal and function largely in similar ways in different cultures. However, 96% of samples in psychological studies come from countries with only 12% of the world’s population, which means that a randomly selected American is 300 times more likely to be a research participant than is a randomly selected person from outside of the West. These countries are commonly referred to as WEIRD (Western, Industrialised, Educated, Rich and Democratic), which makes them vastly unrepresentative as a sample in psychological research.

Even though cross-cultural research into decision making is still in its infancy, a growing body of evidence suggests that behavioural economics as a field will hugely benefit from it as differences between cultures help unpack the deeper foundations of behaviour. Given the emphasis of many cognitive decision making theories on the impact of immediate context such as framing or priming, it’s surprising how little culture is taken into account. While social psychology has a wealth of knowledge on how cultural context affects us, theories in cognitive psychology rarely consider culture as a factor due to implicit assumptions about the universality of cognitive processes: according to these researchers, what we think about may vary, but how we think is always the same.

The Muller-Lyer illusion

However, research has shown even fundamental cognitive functions such as how we perceive colour can significantly differ based on the cultural context you’ve grown up in. Similarly, perception of seemingly simple optical illusions such as Müller-Lyer arrows have been shown to vary across cultures and even age groups.

Through the looking glass of culture
When talking about culture, we often refer to aspects such as values, attitudes, social norms, beliefs and traditions. However, despite long-standing debates within academia, no commonly accepted definition of culture actually exists. Instead, researchers tend to focus on certain aspects of culture depending on their area of interest and the phenomena they are investigating.

On a general level there is a wealth of evidence that economic, social and linguistic environments strongly shape people’s behaviour, motivations and preferences: for example, a study investigating time discounting (i.e. whether we value immediate rewards more than those in the future) in 45 countries found that differences at country level related to wealth and education as well as cultural factors such as individualism, the importance of tradition and whether time was conceptualised as linear or cyclical. However, without a unifying framework of conceptualising culture research such remains too scattered and almost makes it harder to grasp the bigger themes underlying cultural differences. A more effective way of understanding culture’s impact on how BE biases work differently in different countries is to look at some measurable differences between cultures which doaffect how a person’s cognition works while they make decisions. While other frameworks exist, one of the most powerful ones is a person’s self-concept.

“Me, myself and I” vs. “All together now”
The most widely analysed dimensions of culture are individualism and collectivism. Often discussed in the context of Geert Hofstede’s Taxonomy of Cultural Differences, these dimensions have received a lot of attention both among academics and practitioners. In a nutshell, individualism is characterised by detachment from relationships and community with the individual seeing himself as relatively independent from others, whereas collectivism is characterised by the importance placed on relationships, roles and status within the social system, with the individual seeing himself inseparable from his network of social relations.

However, at the level of the individual, these cultural mindsets affect how we see the world through organising the information we have about ourselves, directing our attention to information that is perceived to be relevant, shaping motivations and influencing how people appraise situations that influence their emotional experiences. These self-concepts can be placed on a continuum between two poles: independent and interdependent selves. Independent self-concepts are typically more prevalent in individualistic countries, whereas interdependent ones tend to be more common in collectivistic ones, although variation exists within countries.

Those termed independent define themselves through internal attributes such traits, abilities, personal values and preferences, and see behaviour as being under the control of the individual, arising from internal attributes such as preferences (e.g. what you buy reflects your identity). Conversely, those termed interdependent define themselves through relationships with others and don’t necessarily see behaviour as a reflection of internal traits but situated in a specific context – your preferences might radically change depending on what social circumstances you are in.

This has profound consequences for some of the fundamental concepts in consumer psychology such as cognitive dissonance: if you see your behaviour reflecting your true self, which is ideally consistent across time and circumstances, holding two or more conflicting ideas will make you feel uncomfortable. However, if you instead assume that your preferences merely reflect the current social circumstances and can therefore change from one moment to the next, conflicting ideas will not pose a threat to your identity, which means the concept of cognitive dissonance exerts much less power on consumers in e.g. East Asian cultures. As cognitive dissonance, often seen as irrational, is commonly used in advertising, understanding the extent to which it is prevalent in the cultural context is crucial to efficient marketing communications.

I am what I buy… or am I?
Whether or not we see ourselves as separate individuals or intertwined with others is also important in understanding consumer choice. Is choice an individual endeavour, reflecting our internal attributes or one that takes other people into account and says little about the chooser? In Western cultures, choice is seen as an act of self-expression: uniqueness is desirable and choices are a way to paint a portrait of yourself for the outside world, so we vary our choices in an attempt to gain a sense of “specialness”. In behavioural economics, this is termed as diversification bias where we seek variety in both what and how we choose which may sometimes lead to seemingly irrational behaviour. However, the majority of the research on this effect has been conducted in Western countries and specifically in the US where personal choice is almost one of the key cultural values.

When choice is an act of self-expression, it becomes hugely important for the individual, and the psychological impact of either lack of choice or failed choice is larger, which leads to strategies such as variety-seeking. However, in in collectivistic cultures choice is often an interpersonal task which means the success or failure of making a choice that portrays oneself in the most positive light is not as big a concern. Subsequently, recent research has shown that the diversification bias is weaker in these cultural contexts.

Self-concepts also affect the strength of another well-known behavioural economics concept: theendowment effect, where simply owning an object enhances its perceived worth, and owners value objects substantially (and irrationally) more than potential buyers do. Because owning an object activates an association between it and the self, the Western focus on self-enhancement means this association automatically boosts the object’s value. Therefore, the strength of the endowment effect is influenced by the degree to which self-enhancement is culturally valued, with recent research suggesting that the effect is indeed stronger in a Western context. In practice this means that sales tactics such as free trial or “bait and switch” may be less effective in these non-Western contexts with weaker endowment effect combined with weaker cognitive dissonance.

Fifty shades of irrationality
Understanding the potential cultural influences on thought is crucial for everyone attempting to accurately describe and predict consumers’ decision making.  Insights from behavioural economics might well be applicable in different cultures, but we need to have highly nuanced sense of the specific characteristics of each cultural context and its impact on consumer decision making to ensure effective applications. As behavioural economics professor Dan Ariely notes, the biggest challenge for the field in the next 10 years is understanding the generality of the findings so far and to what extent the effects discovered carry over in different contexts. As market research gradually abandons the error of rationality and adopts more principles from behavioural economics, let’s make sure we don’t entrench a new mistake: universality.

Elina Halonen is a founding partner of the Irrational Agency and editor at InDecision blog.

What’s life like on the other side of the behavioural fence?

Last week we co-hosted (along with Hunting Dynasty) an event to celebrate the second anniversary of the London Behavioural Economics Network. To mark the special occasion, we invited two illustrious speakers to discuss life on both sides of the behavioural fence: commercial and academic. Representing the academic side, we had George Loewenstein from Carnegie Mellon University and for the commercial side, there was only one option – Rory Sutherland.

2014-04-23 20.17.23

Thanks to Brian Tarran from the Market Research Society, we can share with you a transcript of the event (link to our video at the end of this post). Brian also wrote an excellent review of the event which you can read on Research Live.


George Loewenstein: Why is it that British electrical outlets have a switch on them? If you don’t have anything plugged in, why bother to turn it off, and if you do have something plugged in, why bother to turn it off?
Rory Sutherland: I’ve always seen it as a failing of other countries that you don’t have a switch. But there may have been also, bizarrely… if you go back to the early days of electricity, there was a considerable fear that if you left the switch on with nothing plugged in the electricity would leak into the room. So it might have been a device by the electricity boards to prevent people from being paranoid.
GL: I’ve read in various places that you studied classics at Cambridge and taught classics. Now, I don’t want to rake the coals on any deep regrets that you feel about leaving teaching, but I’m wondering if you’ve gained any insights from your career in advertising that would apply to education?
RS: A classical education is one of those things that works obliquely. It’s also worth asking: is a large component of education signalling, rather than value? Your level of educational attainment correlates very well with your career outcome if you get a paid job. It doesn’t correlate at all with people who are self-employed. This raises the question: is there a large amount of education that is effectively a spiralling, competitive credentialism? We don’t have any evidence that people with OxBridge firsts are better employees than people with firsts from Leeds. There’s no real empirical evidence that your degree class predicts your level of value to your company. There’s an aspect, which I know is a very cynical one, who argue that this is in a sense a peacock’s tail thing; basically it’s a three-year long IQ test combined with proof of commitment.
GL: Take something like drop-outs, or students not studying as hard as you’d like them to. Are there any lessons from behavioural economics or the marketing world that would discourage and encourage those people?
RS: There are some interesting questions about education. Does it disproportionately privilege certain personality types, those that are particularly well-suited to the system they find themselves in. One of the interesting things about an ad agency is that, as a place to work, it is a complimentary mix of skills. You need the beard-stroking Oxbridge intellectuals dotted around the place, but what you find in the most successful ad agencies is that the people who make up the management of that organisation tend to come from an extraordinary mix of educational and social background. That’s not just a politically correct, nice thing to have, it is a source of competitive advantage.
GL: That’s a nice segue into a question I was going to ask, about the new master’s programmes that are beginning in the UK, focusing on behavioural economics. Is this the kind of training that you would value at Ogilvy, or are you looking for the pot-smoking… what was it?
RS: They are not mutually exclusive, of course. You can have an MSc in behavioural science and a severe pot habit.
RS: One of the strange things is, you would think that advertising agencies and clients are full of behavioural scientists and psychologists, devising the next evil way in which to mislead people into buying stuff. I must admit, I was rather hoping to find that kind of thing when I went into the business myself. I always assumed I’d be able to turn some strange corner and find a room full of people attaching electrodes to rats. That room does not exist. But the reason private enterprise looks like it’s good at behavioural science is that it stumbles on things by accident. It happens to produce things that work. Capitalism is sort of semi-Darwinist: if you stumble on something that is disproportionately successful, it makes money, so you tend to expand that particular area. Now, some time after the early Mad Men era, the links between academic psychology and advertising and marketing were actually quite strong. This was in the late-50s, early-60s, when there were various people dotted around ad agencies with possibly fake, possibly genuine Viennese accents, who claimed to have met Sigmund Freud. These people came up with ideas, like how ‘plink, plink, fizz’ would create a social norm around using two Alka-Seltzer rather than one. They came up with lines like, ‘How else can a month’s salary last a lifetime’ for DeBeers, which was an extraordinarily good piece of framing in terms of anchoring what a man should spend on a commitment device for his future wife. They stumbled on these things by accident, but they made no effort whatsoever to codify it. What the business world is very good at is stumbling on successes by accident, but it’s terrible at making sense of them. We had all the information to create a science of behavioural economics back in the 50s and 60s, but we squandered it because nobody tried to make sense of it.
GL: How do you use behavioural economics in your practice? Can you give an example of a campaign that wouldn’t have happened without behavioural economics.
RS: The most important thing about behavioural economics is the widespread awareness among clients and agency people that these things are important. Before, in modelling and attempting to predict consumer behaviour, we tended to rely on the two pairs of broken binoculars: neo-classical economic assumptions, which are quite seriously wrong in some areas, and market research. Enormous business and government decisions are based on these two flawed lenses. Neo-classical economics tends to assume that psychology isn’t matter – which, as a result, creates an imaginary fantasy world of perfect information, perfect trust and perfect efficiency where marketing and advertising needn’t exist. But there are loads of things for which there is no rational reason. Stripy toothpaste, say – let’s get really trivial. There is no rational reason for stripy toothpaste because once you put it into your mouth it all gets mixed up. Nobody in market research would ever say that toothpaste should be stripy. No neo-classical economist would argue that there was any benefit to it. But someone stumbled on this idea that it’s much easier for people to believe a toothpaste does three things if there are three different visible components. That kind of thing, just understanding something that would never have emerged from research or neo-classical economics – those kind of things can make the difference between triumph and disaster. If all I do is get people to understand that, and get people to accept that it is worth testing things that don’t appear to make any sense to test, that’s all you have to do.
GL: The stereotype of advertisers and marketers is that they are playing on motives like power, sex, fear and so on. Is that true, and to what extent is behavioural economics capturing the motives that really drive people to purchase products?
RS: I’m very uncomfortable with the use of the word irrational, because an awful lot of behaviour that is currently seen as irrational is meta-rational or evolutionary rational. It’s the product of evolved psychological instincts that may or may not be useful in the modern world. For example, people’s willingness to pay a premium for a famous brand is rational if you understand a bit of reputational game theory, which is that someone with a valuable reputation has more to lose from selling a bad product than someone you’ve never heard of. There are lots of mechanisms that we employ instinctively, actually make very good sense. I also think loss aversion, in evolutionary terms, needs to be better understood. Rationality is always about maximising something, but I would argue that any sensible evolved system is not going to care about whether something is good or really good, but it would desperately care about the chances of something being fatally terrible. If you look at it from the absence of bad, rather than presence of good, an awful lot of human decision making makes an awful lot of sense. The idea that rationality means getting the best of something… I mean, you can go to a Michelin-starred restaurant and spend the afternoon on the bog, but say what you like about Maccy D’s but you don’t get ill there. Satisficing is what most people do all the time to the extent where, when people don’t satisfice there’s probably something going wrong. When people maximise, it tends to be in competitive situations where people are thinking as game theorists.
GL: But sometimes what appears to be maximising, isn’t maximising at all. I had a colleague who always tried to maximise the gain from any kind of negotiation and it was a disastrous strategy…
RS:… Because no-one would negotiate with him?
GL: That’s right. Another case is, if you always want to get the parking spot closest to the theatre, you’re going to waste a lot of time looking for a parking place. I’m actually a big believer in satisficing, but really a believer that satisficing can be a form of maximising. On the idea of loss aversion being connected with having evolved, it would have to be a very crude kind of evolution that produced that effect, because we’re loss averse over very small amounts: like gain $10/lose $10, or gain $15/lose $10 – most people are going to turn that down. But $10 has no prospect of killing you, so you would have to say it’s something we learned through evolution and then generalised inappropriately.
RS: Do you accept the claim that some psychologists make that the term behavioural economics steals for economics credit that is owed to psychology?
GL: There are a lot of psychologists who are annoyed at economics, and are annoyed at behavioural economics specifically. A lot of the work on changing health behaviours, for instance, was first done by psychologists, didn’t get a huge amount of attention, and then behavioural economists started doing it, and have gotten a tremendous amount of attention since then. Psychologists are, rightfully in my opinion, bothered about that. In fact, a lot of psychologists have started rebranding themselves as behavioural economists and getting more attention. I think they are right to have a grudge, but I guess you could tell us that it’s all a matter or marketing, and behavioural economists are much better at the marketing game.
RS: I got into this weird Twitter row, and Nicholas Christakis, I think, had the last word on the question when he said, “Look, there is a branding problem with psychology, which is very simple: the president of the United States cannot have a council of psychological advisors – but he can have a council of economic advisors”. On the other hand, for reasons I don’t fully understand, economics has completely disproportionate influence in business, and especially in policy making, out of all the other social sciences. Part of my theory is that economics is the one academic discipline where it is ok to be greedy. My brother is an astrophysicist, and if he went off to work in a high-frequency trading firm for two years and make a fortune and then try to get back into astrophysics, his career would be dead because he’d sold out. Economics, because it preaches self interest as a virtue, getting two-year gig at Citibank for £300,000 a year is positively career enhancing. Is that a valid theory?
GL: I think there is a good reason why economists are so tied into policy, because a lot of economics is oriented towards policy. But I actually want to pick up on a little piece of what you talked about, which is greed. When economists talk about greed they misuse the term. For an economist, greed is self interest. But when lay people use the term, they are using it in a very different sense. For a lay person, greed is self interest taken too far; to the point where it is destructive. So, it’s fine in my opinion to be self interested, but the danger is where you take it too far.
RS: We do have a social epistemology, and our idea of the good life is probably massively informed by what we think other people’s idea of the good life is. So, I think the extent to which we decide individually is a really interesting question. Although we’re consciously determined to see ourselves as individuals, I think our vision of what’s good is massively driven by our assumptions of what other people think.
GL: Exactly, and I recently co-authored a paper on mattering maps. When you’re in a particular situation, let’s say a social group, there’s typically something that matters more than anything else. Among academics it might be publications. If you are a musician it might be how well you play a piece. But one of the interesting features of mattering maps is they can change so abruptly as a result of whatever social milieu you’re in. So, I couldn’t agree more with you, that whatever we’re seeking at any given time is completely socially determined. The think we have some kind of free will about is how we’re going to seek it out. But we don’t really have a lot of control over what we’re seeking.
RS: We won’t get into free will and determinism here, because that’s probably a bit heavy. But a final thing: there is this attack on behavioural economics that it is just a collection of findings, or anomalies – that it is stamp collecting for psychologists. Critics say, ‘Where is the unifying theory?’ But if you’re dealing with psychology, human behaviour, human health – any of these complex emergent phenomena – is there ever going to be a unifying theory?
GL: Let me just say that something I’ve noticed about Brits is that you find pretension toxic. And what you don’t realise is that Rory is actually making fun of me with this question…
RS: No, I’m not!
GL: …because I was having breakfast with him and I told him that I was working with Nick Chater on a theory of everything. I actually meant it a little bit tongue in cheek, but not totally. So Rory’s making fun of me. But, ok, the first thing is that it’s kind of surprising as a behavioural economist to be questioned about the desire for a unified theory, because we’re always being attacked by traditional economists who say, ‘We have a unified theory and you’re just a bunch of disparate findings’. My view is that traditional economics is not nearly as unified as it claims to be because whatever phenomenon the economist is looking at, the utility function metamorphoses to deal with the phenomena that’s important to them. But I do think that a unifying theory, even a unifying mathematical theory, can be a beautiful thing. And, that’s what social science is all about: trying to take disparate social phenomena and come up with a unified account. That’s when I get a chill up my spine, when disparate things come together. So my aspiration is for behavioural economics to be grounded on a unified theory, or maybe a few unified theories.


For a video of the event, click here.

To sign up to the next London Behavioural Economics Event:

Join the LBEN Facebook group:

On the identity and methods of behavioural economics

The FT has a very good article from Tim Harford today, surveying behavioural economics and asking some important questions about it. People within a field can be so immersed in their unconscious assumptions and practices that it takes an outsider to point out some of the questions they are not asking.

Tim says:

The past decade has been a triumph for behavioural economics…[which] is one of the hottest ideas in public policy….Yet, as with any success story, the backlash has begun. Critics argue that the field is overhyped, trivial, unreliable, a smokescreen for bad policy, an intellectual dead-end – or possibly all of the above. Is behavioural economics doomed to reflect the limitations of its intellectual parents, psychology and economics? Or can it build on their strengths and offer a powerful set of tools for policy makers and academics alike?

Quite. That, of course, is a journalistic question – not one intended to be answered within the article, but designed to provoke the prospect of a good ding-song. But the substantive points come soon. Note that Tim, writing for a generalist FT-reading audience, chooses to address his article to public policy so it doesn’t look like an abstruse argument between academics. But actually it’s about the effectiveness of BE, and economics in general, as a tool at all. Public policy, private decisions, how businesses operate – all can be informed by whatever economic theory we believe in.

…there is something unnerving about a discipline in which our discoveries about the past do not easily generalise to the future…This patchwork of sometimes-fragile psychological results hardly invalidates the whole field but complicates the business of making practical policy.

Indeed – and it divides the field, into those who believe a (more) unified theory is available, and those who believe rational choice is still the main theory available and that behavioural results are only meaningful in relation to that.

The line between behavioural economics and psychology can get a little blurred. Behavioural economics is based on the traditional “neoclassical” model of human behaviour used by economists. This essentially mathematical model says human decisions can usefully be modelled as though our choices were the outcome of solving differential equations. Add psychology into the mix – for example, Kahneman’s insight (with the late Amos Tversky) that we treat the possibility of a loss differently from the way we treat the possibility of a gain – and the task of the behavioural economist is to incorporate such ideas without losing the mathematically-solvable nature of the model.

Consider the example of, say, improving energy efficiency. A psychologist might point out that consumers are impatient, poorly-informed and easily swayed by what their neighbours are doing. It’s the job of the behavioural economist to work out how energy markets might work under such conditions, and what effects we might expect if we introduced policies such as a tax on domestic heating or a subsidy for insulation.

And the problem today is that, without a clear theory, behavioural economists can’t work that out. All they can do is suggest various effects that might happen, and design an experiment to test them. Nothing wrong with that, but it’s a bit ad hoc.

The most well-known critique of behavioural economics comes from a psychologist, Gerd Gigerenzer of the Max Planck Institute for Human Development. Gigerenzer argues that it is pointless to keep adding frills to a mathematical account of human behaviour that, in the end, has nothing to do with real cognitive processes.

David Laibson, a behavioural economist at Harvard…concedes that Gigerenzer has a point but adds: “Gerd’s models of heuristic decision-making are great in the specific domains for which they are designed but they are not general models of behaviour.” In other words, you’re not going to be able to use them to figure out how people should, or do, budget for Christmas or nurse their credit card limit through a spell of joblessness.

We come back again to the need for a general theory, and one of behavioural economics’ regular combatants agrees:

For some economists, though, behavioural economics has already conceded too much to the patchwork of psychology. David K Levine, an economist at Washington University in St Louis, and author of Is Behavioral Economics Doomed? (2012), says: “There is a tendency to propose some new theory to explain each new fact. The world doesn’t need a thousand different theories to explain a thousand different facts. At some point there needs to be a discipline of trying to explain many facts with one theory.”

The challenge for behavioural economics is to elaborate on the neoclassical model to deliver psychological realism without collapsing into a mess of special cases…The question is, how many special cases can behavioural economics sustain before it becomes arbitrary and unwieldy? Not more than one or two at a time, says Kahneman.

Thaler says: “…if you want one unifying theory of economic behaviour, you won’t do better than the neoclassical model, which is not particularly good”

It seems that Kahneman and Thaler actually agree with Levine in a way; all three doubt that behavioural economics can crystallise into a single theory, though only Levine thinks this is a serious problem.

George Loewenstein and Peter Ubel wrote in The New York Times that “behavioural economics is being used as a political expedient, allowing policy makers to avoid painful but more effective solutions rooted in traditional economics.”

This point is different but important: if policymakers expect behavioural economics to be a substitute for regular economics they’ll be disappointed. The two are complementary, and the most important policy contribution of BE may be to tell us which economic incentives will have the biggest impact, and which will have unwanted side-effects, rather than to obviate the need for traditional incentives altogether.

Should we be trying for something more ambitious than behavioural economics? “I don’t know if we know enough yet to be more ambitious,” says Kahneman.

That’s a provocative point. Yet it acknowledges that whatever field eventually manages to incorporate both traditional and behavioural economics may have to be called something different.

Laibson says behavioural economics has only just begun to extend its influence over public policy. “The glass is only five per cent full but there’s no reason to believe the glass isn’t going to completely fill up.

I and many readers of this blog will probably be with Laibson on this point. But perhaps without a new approach, behavioural policy is going to run more and more often into the wall of adhockery – the lack of general theories making us redo things from the ground up in each new situation.

Tim isn’t the only person to write about this recently. For a contrary word, try Chris Dillow’s comment, which makes some good challenges from his usual half-libertarian, half-Marxist point of view.

Then, here are some links and thoughts from Diane Coyle, including “Is behavioural economics the past or the future” by Chris House. Diane hones down one of Tim’s questions into Kao and Velupillai’s distinction between classical and modern behavioural economics: modern assumes people are (biased) optimisers, while classical assumes they are satisficers. This is the same distinction drawn by Gerd Gigerenzer, though his research looks at a broader range of decision-making heuristics, of which satisficing is just one. Diane asks, effectively: is the best mathematical approach to tweak the models of maximisation, or to try to build a new behavioural economics based on heuristics?

Chris House’s post says:

…in 2007-2008 we were again told that behavioral economics would finally come into full bloom. It didn’t happen though. The wave of behavioralists never came.

While this isn’t true in psychology or behavioural policy and marketing – all thriving and fast-growing fields – it is true of economics. My experience is that many new economics undergraduates or entrants to economics PhD programs are intrigued by behavioural ideas, they are often guided by supervisors into more traditional areas where it is easier to define a research question that is going to produce safe, publishable papers. Barkley Rosser, commenting on House’s post, mentions the new journal Review Of Behavioural Economics, which along with other emerging initiatives may help to change this.

Otherwise, Chris raises that same point:

Behavioral economics won’t get very far if it ends up being just a pile of “quirks.” Are these anomalies merely imperfections in a system which is largely characterized by rational self-interest or is there something deeper at play? …if behavioral is to somehow fulfill its earlier promise then there has to be some transcendent principle or insight which comes from behavioral economics that we can use to understand the world.

Then there is the David Levine paper that Tim mentions, “Is Behavioural Economics Doomed?“. In this, Levine says (among many other interesting things!):

For most decisions of interest to economists these external helpers [computers, paper and pencil etc] play a critical role – and no doubt lead to a higher level of rationality in decision making than if we had to make all decisions on the fly in our heads.

What a brave claim! Do we really rule out from the realm of economically interesting decisions all consumer purchases, the consumer’s intuitive feelings about how safe they feel with a certain amount of savings in the bank, and all the decisions about cars, houses and jobs that – although someone might sit and think about them for a while – still involve a big chunk of emotion?

Actually, there is no need to throw out these kinds of decisions in order to meet Levine’s key challenge of “trying to explain many facts with one theory.” He asserts that mainstream economics is already successful at explaining many facts. But perhaps, when he discards all those “uninteresting” decisions it isn’t so hard to explain what’s left. Indeed, it’s those “uninteresting” decisions which classical economics does struggle with, and only behavioural economics can illuminate. Contrary to Levine, I am convinced that these decisions actually make up the majority of important economic events. But I do recognise his critique – echoed by Tim and implicitly by Velupillai and Gigerenzer: that behavioural economics does not offer a full theory to replace that of mainstream economics. However, it has given us good empirical evidence which we could build a theory on.

As well as defining away a large portion of the economy as “not interesting”, Levine also co-opts some of the parts that he does consider interesting, saying they are already handled by mainstream economics: notably the subject of learning. Non-behavioural economists have considered consumers’ imperfect ability to learn the preferences of other consumers, or the rules of the “game” they are playing, as a factor in non-optimal decisions. But psychologists know much more about exactly how people learn than economists do – so a successful model of learning as part of economics can only be built with an openness to psychological research. Where Levine may be right is that behavioural economics will not replace mainstream economics, but instead the two fields will merge – with the behaviour of consumers predicted by a combination of objective economic, and subjective psychological, factors.

Anyway, arguments over the boundaries of disciplines are rarely productive: I don’t really mind if Levine considers a model to be behavioural or not, as long as the model advances the cause of making successful predictions.

The real questions are: does standard economics fail to address some important problems? How good is behavioural economics at addressing them instead? And does behavioural economics need a unified approach in order to address them?

Most of the people mentioned above have different answers to those questions:

  • Levine wants a unified theory – but think we have to exclude many types of “uninteresting” decision in order to get one.
  • Kahneman and Thaler want different theories for several different areas – but those incompatible theories will not be able to deal with the many boundaries where different aspects of economics interact with each other.
  • The classical economists already have a unified theory – but there are many things it can’t explain.
  • Gigerenzer has a philosophy – but no overall theory. And I’m not sure if he expects or really wants a unifying theory any more than Kahneman does (this may be one of the few things they agree on).

[Update: much of this debate was anticipated in this Werner Guth paper of 2007]

My view, which I think concurs with Laibson’s: a single broader theory is possible. I think we’ve hit a theoretical dead end with the traditional maximising agent, so it will have to be based on more psychologically realistic foundations, such as those of Velupillai, Gigerenzer or Bettman, Payne & Johnson. To achieve this, we need to carefully choose the right elements to build into our model of decision-making in a way, so that it can make useful predictions of how those elements might operate. I have a paper coming out later this year which suggests one direction towards this.