MEASURING SAFETY

Suzette Woodward

I often get asked how can we measure whether we are safer today than we were yesterday. I suspect that this is often with the desire for me to provide a small number of clever measures that would clearly show whether safety has improved or not. This is so much harder than people realise. Especially those that govern, scrutinise, and lead our NHS organisations.

Some people even ask if I could give them just one measure. When asked that I often think of Timpsons, the company that does key cutting, dry cleaning and a few other handy things. There is book about Timpsons called the Happiness Index, so that gives you a clue as to what kind of organisation it is. Anyway, they ask just one question in their staff survey. They ask:

“How happy are you with the support you get from your manager (or team)?”

Employees score it on a scale from 1 to 10 and can then add as many comments as they like. The emphasis is less on measuring engagement through dozens of questions and more on understanding whether people feel supported by their leaders. Imagine that, if we in the NHS had just one question in our staff survey.

Even if we don’t have just the one question, we certainly could be more nuanced and intelligent with our measurement of safety. In particular, we cannot just have the number of incidents as a measure. It is hugely flawed. It only shows you how many incidents are reported, not how many actually happen, and is biased in that the incidents reported are the easy to see, easy to report or what people are expected to report. They are also not always related to safety and often about management or operational issues and sometimes even worse, they are used as a threat towards another individual.

What people should be doing is to ask:

“How can we break that up into bite size chunks and make the overwhelming amount of data more meaningful (or even reduce the overwhelming amount of data)“

We could do this by looking at our priorities and how do we want me measure them. For example, if our priority is to study deteriorating patients in a unit or organisation, then we could ask for that to be the focus for a particular period. In that period, we could look at how many patients deteriorated, when and how they were picked up, and all the factors associated; such as what the conditions were, staffing levels, activity levels, time of day, date etc. However, crucially we should also count how many patients did not deteriorate in that time period and what were the factors associated with those patients. This way we arrive at a balanced view of what safety looks like from the perspective of one focus, deterioration. We can do this for many more topics but should be careful not to overwhelm ourselves in a different way.

Another key question when something happens which was unexpected is to ask:

“Has this happened before?“

Undoubtedly the answer will be yes. At this point, instead of deciding to study in depth the incident itself, we should then ask:

“What recommendations were made the last time?“

This would lead us to study the recommendations and look at whether they were the right recommendations or whether we should revisit them. Often when we change things there are unintended or unexpected consequences that we do not foresee when we make those changes. Also, if we change simply based on the times something went wrong, we may impact on the times it goes right. Change is not always the right thing to do. Sometimes it is ok to keep going on the same path, to continue the direction rather than change it.

We would only know this if we ask another question which is:

“Why did it go wrong this time, when most of the time it goes ok?“

This question helps us then start to study safety and not the absence of safety or failure. Safety is when a system is functioning so that it is as safe as it can be given the fact we work in a complex adaptive system that can be very difficult to get right first time or all of the time. It is also a system that is not isolated from the people who work within it who are human beings. As human beings, they do not become somehow more superhuman than others as they step into a healthcare institution. They are just as fallible and flawed as the next person. All of this needs to be taken into account when we want to know how to improve.

That leads on to the next aspect of measuring which is culture. If you have a fearful culture, where people are worried about their future, or the personal consequences of something going wrong, then the likelihood of reporting or speaking up or asking questions or sharing how they are feeling will be significantly impacted. Putting it bluntly people will not share information if they are scared. If they don’t share information then we cannot possibly learn. Again you cannot measure what is hidden.

If we go back to that one question that Timpsons ask, the underlying principle championed by Sir John Timpson is that if people feel well supported by their manager, most other aspects of workplace culture tend to follow. Thus safety, is dependent upon three key things here:

1. Do staff feel supported by their manager when things go wrong?
2. Do they trust what will happen to them when it does?
3. Is that consistent across the unit, ward, department or organisation?

These are the characteristics of a just culture. Where there is a supportive approach that helps people cope and learn from incidents, but also ensures accountability and responsibility for learning for the future. It is also supporting people when they are struggling with their roles or abilities. It is about doing the kind thing, and asking what can be done to help. It is not about ignoring the individual factors and only looking at the system, but seeing both of them in context.

In summary:

Measurement is hard to do

It is never one piece of data

The only way we can do it is to break it down and consider what should be collected and why

Dont change things for the sake of change – often things are predominantly working well, it would be good to try and measure that instead of simply failure

Learning is totally reliant on a culture that is kind and supportive where people feel able to speak up and share

We know from the history of safety, that simply relying on harm or incident data has not got us as far as we would have hoped. The same things are happening time and time again. So maybe we should think about trying something new.