Goodhart’s Law, Campbell’s Law, and the Cobra Effect.

We’ve mentioned Goodhart’s Law a few times in previous newsletters about measuring psychological safety, alongside its siblings, Campbell’s Law and The Cobra Effect. I thought it would be useful to do a piece on what exactly these are, and how they’re relevant to psychological safety.

Goodhart’s Law

Goodhart’s Law is “When a measure becomes a target, it ceases to be a good measure.” It’s named after economist Charles Goodhart, after he published an article on UK monetary policy in 1975 that stated:

“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” 

In this case he was criticising the then Thatcher government for setting particular kinds of targets for monetary policy, and is a variation of the “Lucas critique”. However, it actually represents a broader phenomenon applicable to nearly all social contexts, and it was anthropologist Marilyn Strathern who in fact popularised the now much better known generalisation of Goodhart’s Law in her 1997 paper “‘Improving ratings’: audit in the British University system”

Credit: Sketchplanations

Essentially, if people are aware that they’ll be rewarded for hitting a target or punished for missing it, they will optimise their behaviour to meet that target – even if that behaviour has other negative consequences. For example, if a call centre operative is rewarded with a bonus for answering a certain number of calls in a week, they may try to prematurely end customer calls, even though that might lower overall customer satisfaction. The reward doesn’t even need to be significant: the sheer existence of the measure can be enough to invoke Goodhart’s Law.

This has relevance for psychological safety. When working with teams, we should understand what they’re targeted, rewarded and punished for. What metrics and targets matter to them? Targets and rewards drive behaviour. In particular they can in some cases drive unhealthy, competitive behaviour, which adversely affects psychological safety by incentivising undermining others on the team and reducing interpersonal trust. So by changing (or simply removing) these targets, we can create the foundation for more healthy and psychologically safe behaviours in the group.

Credit: XKCD

Goodhart’s Law also has relevance to measuring psychological safety. If we were to measure psychological safety in teams and then rank or even just publicise the scores of those teams or managers of them, we’d be indirectly invoking Goodhart’s Law. And it would mean that the next measurement we took would likely be inaccurate: publicising the results of a survey can make the next survey less reliable. Team members, and the managers of them may well answer survey questions differently to how they truly feel, knowing how they’d be shared afterwards. This can occur even if publishing the results was never intended to create a target – because no one ever wants to be at the bottom of a ranking.

Which brings us to Campbell’s Law

Campbell’s Law

Campbell’s law builds on Goodhart’s Law, and is in a sense an amplified version of Goodhart’s in practice. It states that: 

“The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

Donald T. Campbell, a psychologist and social scientist, was writing on research methodology, and was making the point that the more important a target is, the worse it is as a measure, and the more likely we are to corrupt the thing we’re measuring.

Credit: Sketchplanations

In stating this, Campbell was specifically referring to educational testing, and the distortion of test scores as actual indicators of educational attainment if the test scores become targets for schools and teachers. In 1976, he wrote: “Achievement tests may well be valuable indicators of general school achievement under conditions of normal teaching aimed at general competence. But when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways.”

Of course, this may be exactly what happens if we measure psychological safety and place any sort of target on it, for example by rewarding managers whose teams score highly. What Campbell’s Law additionally tells us is that it’s not just the people subject to the measurement, but those involved in testing too, who can distort it. For example, a consultancy might be given a target to improve psychological safety scores in the organisation, on the understanding (explicit or otherwise) that if they manage this, their contract will be renewed for another year. There’s clearly a pressure for “corruption”. This may even be unconscious, but as a result the consultancy might interpret and present the data in more flattering ways. In extreme cases, they may actually manipulate the data itself – if the contract size (the reward) is high enough.

Which of course leads us to the famous example, The Cobra Effect.

The Cobra Effect

The Cobra Effect is all about “perverse incentives” – incentives that have undesired and unintended results, potentially harming the very thing that they were intending to improve. 

In the case that gives the phenomenon its name, it is said that The British Colonial Government in Delhi implemented a bounty program to decrease the population of cobras. Initially successful in reducing the number of observed cobras, the scheme ultimately resulted in the opposite: an increase in the population of cobras. Plenty of dead cobras were presented to officials, but this wasn’t due to increased captures; instead it was due to entrepreneurial individuals breeding cobras to exploit those financial incentives. Recognising this unintended consequence, government officials terminated the program, leading breeders to release their cobras into the wild, ultimately resulting in a higher cobra population in Delhi compared to pre-program levels (Hollins, 2017).

Credit: Sketchplanations

There are many, many real-life examples of perverse incentives, and here are just a few on the Perverse Incentive Wiki page.

From Qword (2023) Maybe you should store passwords in plaintext., QWORD. (Thanks Balázs Szakmáry for the article share)

The Cobra Effect is a potential danger in measuring psychological safety, if the incentives are high enough. We may observe survey results improving after a few surveys, but in fact that data may be masking the underlying reality – that answers and scores are being influenced by Goodhart’s and Campbell’s Law effects. This very phenomenon could be damaging psychological safety itself, by suppressing the ability to speak up and provide real answers to survey questions.

Does this mean we shouldn’t measure anything? Well, in my opinion, we can have a tendency to over-measure. We also tend to measure proxies for the thing we want to measure, often without really questioning what we’re doing – for example, we might measure the number of students scoring A’s in exams, believing that an A means the student understands the topic well, when it might just be that they’re good at cramming, and the teacher is good at teaching to the test. We even begin to believe that the proxy is the thing that’s important, because our attention becomes focussed on that above all else.

So what should we do instead? Being mindful of these effects is the starting point. Then when we want to measure anything, we can employ these:

Five strategies to address the Goodhart-Campbell-Cobra effect 

1 – Revisit the aim

One way to mitigate the Goodhart-Campbell-Cobra effect (I’m naming this aggregation as the GCC Effect) is to regularly revisit the actual aim. Are we trying to increase the number of A’s in tests, or are we trying to ensure that students leave school with a deep understanding of the subject? Is our measure still actually measuring that?

2 – Diversify the metrics

Another tactic is to diversify the metrics that we measure and use metrics that compensate for each other, and help to triangulate on the real goal. For example, in order to compensate for call centre operatives ending calls early, we may also introduce a metric for actual completed requests. If we might find that operatives are falsely completing requests without the customer being satisfied, we may add another metric for customer satisfaction where we ask them directly. You can see how this goes, and we do need to be cautious with this, because before long we have an awful lot of metrics, and a great deal of effort being expended in measuring and analysing them. 

3 – Narrowly define your metrics

The closer a metric (a proxy) is to the actual goal, the better that metric reflects the progress towards the goal. So define metrics narrowly, with the intention of avoiding the risk of gaming. This is largely impossible in practice, since we cannot predict in advance how they’ll be gamed, so we can only try to reduce the risk and impact of gaming the metric, never get it to zero.

4 – Use qualitative measures as well as metrics

Use interviews, conversations, feedback and focus groups to gain insights about what’s really happening. These measures provide deep insight, and are much less likely (although not impossible) to be gamed, intentionally or otherwise.

5 – Make it easier and more rewarding to improve the system instead of gaming it.

Most of these negative effects are due to people finding it easier to distort the data or game the system in order to hit the target. Instead, we should make it easier for people to change the system in order to hit the targets, rather than gaming it. The customer service representative could give the product team valuable feedback to improve the product in order to improve customer satisfaction, and be rewarded for it. This is hard to do, but worthwhile in order to improve and trust your metrics, and more importantly, improve your organisation and the experience of the people working within it. 

None of this is about avoiding measurement altogether, instead it is about recognising the time, energy and effort that is often wasted through inadequate and sometimes counter-productive measurement. If we can refocus on what matters most, we can save ourselves a lot of trouble, and free ourselves up to actually improve the outcomes we are striving for. 

References:

Campbell, Donald T (1979). “Assessing the impact of planned social change”. Evaluation and Program Planning. 2 (1): 67–90. doi:10.1016/0149-7189(79)90048-X

Coy, Peter (2021-03-26). “Goodhart’s Law Rules the Modern World. Here Are Nine Examples”. Bloomberg.com. Archived from the original on 2021-04-25. Retrieved 2021-06-03.

Dubner, Stephen J. (11 October 2012). “The Cobra Effect: A New Freakonomics Radio Podcast”. Freakonomics, LLC

Hollins, M. 2017. Curbing the Cobra effect. Insight, The Financial Conduct Authority. Available at: https://www.fca.org.uk/insight/curbing-cobra-effect 

Kerr, S., 1975. On the folly of rewarding A, while hoping for B. Academy of Management journal, 18(4), pp.769-783.

Koretz, Daniel M. (2009). Measuring Up. Harvard University Press. ISBN 978-0-674-03972-8.

Nichols, Sharon L.; Berliner, David C. (March 2005). “The Inevitable Corruption of Indicators and Educators Through High-Stakes Testing” (PDF). East Lansing, MI: The Great Lakes Center for Education Research & Practice. Archived from the original (PDF)

Qword (2023) Maybe you should store passwords in plaintext., QWORD. Available at: https://www.qword.net/2023/04/30/maybe-you-should-store-passwords-in-plaintext 

Siebert, Horst (2001). Der Kobra-Effekt. Wie man Irrwege der Wirtschaftspolitik vermeidet (in German). Munich: Deutsche Verlags-Anstalt. ISBN 3421055629.

Sohl-Dickstein, J. (2022) Too much efficiency makes everything worse: overfitting and the strong version of Goodhart’s law, Jascha’s blog. Available at: https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart.html 

Strathern, Marilyn (1997). “‘Improving ratings’: audit in the British University system”. European Review. 5 (3): 305–321. doi:10.1002/(SICI)1234-981X(199707)5:3<305::AID-EURO184>3.0.CO;2-4

Waters, Tony (14 May 2013). “Campbell’s law, planned social change, Vietnam war deaths, and condom distributions in refugee camps”. Ethnography.com

Further reading

Metrics and psychological safety
Prepare to Measure Psychological Safety.
Measuring Psychological Safety
Psychological Safety Case Study

Psychological Safety in Practice

Top Ten Ways to Build Psychological Safety

We recently published our Top Ten Ways to build psychological safety, according to our experience working with teams and organisations for the past few years. Our fab placement student turned research assistant Bea Poyton turned it into the excellent infographic. Read the in-depth article here and download a high res pdf of the top ten ways to build psychological safety here.

When a Team Member Speaks Up — and It Doesn’t Go Well

This great article from Megan Reitz and Amy Edmondson uses a fictional case study of Elena, a new team member who attempts to speak up, and it doesn’t go well… It makes the very good point that some corporate training programmes on psychological safety put the onus on the individual to simply speak up, skilfully, boldly and articulately. In fact, even if when someone speaks up and it doesn’t go well, it should be treated as a learning opportunity – everything is an experiment!

The Reason People Are Not Telling Biden the Truth

This is a very interesting article about Biden’s presidential campaign, which includes discussion on the implications and risk of groupthink, and suggests that Biden’s advisers are afraid to express concerns due to fear and futility – essentially a lack of psychological safety. “Weak leaders stifle dissent and leave themselves weaker. Strong leaders welcome dissent and make themselves stronger.”

Complexity

In the fields of and around psychological safety, we’re constantly dealing with complexity – of people, politics, technology, sociotechnical systems and more. This is a neat map of the complexity sciences by Brian Castellani and Lasse Gerrits.

Skateboarding and Failure

I love this episode of Fail Better with David Duchovny, featuring Tony Hawk. Skateboarding is so, so difficult, that every successful trick is essentially the pinnacle of a few thousand failures. In this episode, Tony Hawk reflects on that, and also describes the first successful 900 at the X Games in 1997. Honestly, watch the video – it’s inspiring. Tony also gives a shout out to Matt Hoffman, one of my childhood heroes, who executed the first 900 on a BMX in 1989.

This week’s poem:

‘Kind Hearts are the Gardens’, by Henry Wadsworth Longfellow 

Kind hearts are the gardens,
Kind thoughts are the roots,
Kind words are the flowers,
Kind deeds are the fruits.

Take care of your garden,
And keep out the weeds,
Fill it with sunshine;
Kind words and kind deeds.