Start With Why: Questions to Ask Yourself before Measuring Psychological Safety.

Welcome to the psychological safety newsletter and thanks for subscribing. You are amazing. This week discusses some things to think about before measuring psychological safety.

Measuring Psychological Safety

I’ve had a lot of conversations recently about measuring psychological safety, and thought it’d be good to revisit this twitter thread where I laid out some of my thoughts about the questions to ask yourself before measuring or surveying psychological safety.

– Why do I want to measure it? What’s my aim? What’s the outcome that I want to achieve? 

– Do I want to observe a trend over time and response to change (longitudinal), or am I looking for correlations and differences between teams, sections, functions and locations (cross sectional)?

– Am I looking for numbers and metrics (quantitative data) or detail, context, and sentiment (qualitative data)? Or combining both approaches in a mixed-methods study?

(My personal note: I suggest being very careful when conducting quantitative research in sociological contexts, particularly when applications and algorithms are involved in the analysis. If you’re not sure how the algorithm is interpreting data and generating results, don’t trust it. This is a great resource from Rachel Thomas on the subject.)

– Is this a purely exploratory study? I.e. a Grounded Theory research project intended to generate a testable theory? If we’re doing this, we should be careful to avoid the Texas Sharp Shooter Fallacy.

– How much time and effort do I want to invest in measurement? Do I actually have the time to do it? Do I have other people to help out, especially if I’m not around? What will happen to the project if I go on holiday or sick leave?

– How many questions or statements will I include in the survey? Fewer questions will increase the response rate, but provide fewer insights. More questions provide greater and more granular insights, but decrease response rate. 

– How will I visualise the data that I collect? Numbers by themselves are dry, but the best visualisations of data help to foster engagement as well as understanding.

– Have I considered how I will act upon the findings? If people respond to the survey and their feedback isn’t acted upon quickly, it could harm our ability to carry out further surveys, and damage trust as well as psychological safety in the organisation.

– Is this an academic study that should be replicable in different organisations, or do I actually care about replicability at all? If we’re only ever going to measure this organisation, it almost certainly doesn’t matter.

– Have I considered what we’ll do if the data doesn’t show me what I hope it shows? 

– How many languages, cultures, contexts and domains do I need to address to ensure that everyone completing the survey is having the same experience and able to provide comparable answers? How will I address usability and inclusivity? For example, visually impaired people using screen readers.

– Will respondents be anonymous? How can I ensure anonymity? Can I ensure that the data cannot be de-anonymised?

– How will I segment the results? Will I aggregate results for the entire survey, or break results down into office, department, function and team? What then about very small teams – how will they maintain anonymity?

– If I’m segmenting the results, how will I ensure that teams and managers don’t begin (even unconsciously) to compete against other teams, damaging the validity of further surveys?

– If I’m averaging results, what about ranges too? If one member of a team “scores” very low, whilst the rest of the team “score” highly, how will I identify that as a team with a problem? And how do I address it without highlighting the individual?

– How will I know that my measurements are correct? How will I validate my methods and results? Validity (whether the results really do represent what they are supposed to measure) is important. Or is it? Does it matter, if in the end, I helped to foster greater psychological safety in the organisation?

– How often will I repeat the survey? Once a year is probably too infrequent to detect issues and address them, but every week would be far too arduous and cause survey fatigue amongst respondents.

– How will I check for reliability? That is, if I repeat the same survey a second, third, or fourth time, would the results be the same? Reliability is the extent to which the survey would give the same results if the measurement were to be taken again under the same conditions: its consistency each time it is run. If it’s not reliable, I may end up trying to fix problems that don’t exist, and miss the real issues.

– How will I avoid making individual level assumptions from population level data? This is known as the “ecological fallacy“.  An example of this is the assumption made from population data that red wine prevents heart attacks, when in fact it’s that in general, people who can afford to regularly drink red wine benefit from improved health due to their socioeconomic conditions, not that they drink red wine regularly (sorry, wine drinkers!). 

– Am I surveying everyone in the organisation, or a sample population? If sampling, how will I ensure that the sample population is representative of the whole?

– How will I check that I’m not subject to sampling bias? What if the only people responding to the survey are those who feel safe to do so (or those who just love filling out surveys!)?

– How will I try to make sure that the act of measurement doesn’t negatively impact the thing (psychological safety) that I’m trying to measure? Remember, the observer effect doesn’t just apply to particle physics 🙂

– Am I measuring psychological safety, or proxies for psychological safety? How do I know that they are causally correlated?

– How will I address effects such as Campbell’s Law or Goodhart’s Law? How will I know that people aren’t just telling me what they think I want to hear, or what they’re incentivised to say?

– What’s the opportunity cost of measurement? Could I spend that time doing something even more impactful? 

– Is there another way that I could achieve the same goal? Could I look for analogues or signals of safety instead? 

– What about conversations? Could I experiment with focus groups, interviews, or set up communities?

This might read as though I’m suggesting that you shouldn’t measure psychological safety, but that’s not my intention 🙂 Measuring anything related to human behaviour and interactions can be really powerful, but it’s also dangerous – be careful, check your assumptions, plan carefully, get expert assistance, and be ready to change your approach if necessary. And if you are embarking on a programme of measurement and surveying, we can help!

Note on last week’s events

Last weekend, coinciding with this year’s Trans Day of Remembrance, the LGBTQIA community suffered another horrific attack, in Colorado Springs, USA. My heart goes out to all those affected, and the entire LGBTQ community, who continue daily to suffer prejudiced rhetoric as well as physical violence. Here are some resources that I hope might help anyone in need of them:

• TransCanWork (USA)
• Resources for Supporting Transgender Employees 
• LGBT Foundation (UK)
• Switchboard: LGBT+ helpline (UK)

And a reminder to us all, myself included, that it’s the frequent small acts, the constant attention to being inclusive and kind, as well as calling out and addressing exclusive behaviours and micro-aggressions, that add up to the big wonderful changes. Just as hateful rhetoric leads to violent acts of hate, aggregated small acts of kindness turn into worldwide movements for the better.

This newsletter is sponsored by Conflux

Conflux is the leading business consultancy worldwide helping organizations to navigate fast flow in software. We help organizations to adopt and sustain proven, modern practices for delivering software rapidly and safely.

‘The Fearless Organization’ by Amy Edmondson is considered by many to be essential reading on the topic of psychological safety. In this article, Sophie Weston, Principal at Conflux, has put together some key takeaways from the book.

Psychological Safety in the Workplace

I was wondering when we were going to get an Amy Edmondson and Kim Scott collab, and here it is! 
I absolutely love this piece, and there are some key points I want to draw out:

• Ask for feedback in every 1-1. 
• Reward people’s candour. 
• You don’t have to agree with someone’s feedback, but you should examine where there is value in it.
• Give helpful criticism. “There’s nothing safe about feeling unsure of where one stands.”
• Give praise. At least as much as, ideally more than, helpful criticism.
• Gauge your feedback: Intentions don’t matter. Impact does.

And we have four podcasts to share this week!

Here’s a two-part podcast episode from the Golf Performance Newsletter: Coach Noah Talks, with Dr Jamie Taylor, discussing psychological safety, thriving under pressure, clear communication, and “tough love”.

And here’s The Teamwork Advantage with Gregg Gregory, talking to Stephan Wiedner about psychological safety, discussing aspects such as the Andon Cord and measuring psychological safety.

Amy Edmondson would say that psychological safety isn’t about being “nice”, but here’s a podcast all about being nice, with Dave Delaney: “Candid conversations about communication, collaboration, community, kindness, and *nice* leadership.“, talking about health, wellness, and psychological safety at work with Kim Farmer.

This is part 2 of a 3-part podcast series by The L&D Leaders’ Lounge, in this, Laura Walker, Eleanor Hudgell & John Shephard talk about modelling psychological safety and highlight the power of experimentation that is enabled through an environment of psychological safety.

Things to do and try:

*Sorry for the swearies below* – skip this part if you’re not a fan 🙂

This is a game inspired by an article I shared in a previous newsletter about fuck-ups and feedback, from Creative Hackers, about “The Subtle Art of Fucking Up“. The folks at Dandy People turned this into a game, – the Biggest Fuck Ups Game – which is essentially a facilitation tool to enable a discussion about past mistakes and what we can learn, or have learned, from them. There are a few obvious mistakes on the game board too – which is either ironic, serendipitous, very clever, or all three 🙂 

This week’s poem:

Song For The Kicked Out, by Kay Ulanday Barrett

the streets are not paved with gold, they lied
I got a rough throat, i got a rough life
the streets are not paved with gold, they lied
I got too much queer in me to live their way tonight.

she found me waist up in you
she had found me mouthful, drinkin’ you

mama said that I was the devil, made this journey here a waste,
made too American and too unruly
couldn’t I just wear dresses, make money, and behave?

mama said leave this house, her spirit broken by ache
all my belongings freckled the streets
I slept outside, spirit stayin’ up, my journey isn’t self-made

I say all you laughin’ & jukin’ in the alleys
I say all you sleepin’ to roaches cries
I say all you couch surfin’ until the next night
I say all you kissin’ despite the fists and the fights

together we are a prayer, a chant, a song, no matter what
together we are an anthem despite their cis hetero fuss
together we are stronger than the world’s unsaids
together we are as mighty as our ancestors up from the dead

we are bigger than the skylines that hold us
we are bigger than the sirens that stab our hearts
we are bigger than the talk of boystown progress & bars
we are bigger than bleeding our blood to the stars

together we are a prayer, no matter what
together we are an anthem despite their cis hetero fuss
together we are stronger than the world’s unsaids
together we are as mighty as our ancestors up from the dead

Mission statement

The mission of this newsletter is to make the world of work a better, safer, more inclusive and equitable place. We work towards this by sharing, as widely as possible, new research and opinion, case studies, news, practices and methods related to psychological safety. 

This includes topics such as inclusion, human factors, facilitation, safety critical environments, systems thinking, leadership and management, diversity and equity, mental health, organisational dynamics, culture, ethics, and more. 

A core goal is to amplify the voices of minoritised groups and those who are less represented in this field. This includes (but is not limited to) voices of people of colour, neurodiverse people, LGBTQ+ people, women in leadership and tech, and more.

Another goal is to share insights and best practices across different domains: for example, so that those of us in tech may learn from healthcare, and those of us in education can learn from aviation.

I believe knowledge is worthless unless it’s accessible. A key principle of this newsletter is to only share content that everyone can access. This excludes content behind paywalls, research papers that are not open-access, and proprietary tools that don’t open source their algorithms (preventing us from observing how systems make decisions about us). There may be some exceptions to this principle in practice, for example, where authors are able to share original texts of papers.

Whilst this statement will evolve this over time, the core principles of equity, amplification, and progressive & humanistic leadership shouldn’t change.