I have been working with loads of amazing people both here and abroad over the last year. One of the many things that comes up time and time again is measurement: ‘how do we measure success when it comes to patient safety?. The questions we have are challenging to answer with confidence; how safe are we today? are we continuously improving? what are the early warning signals I should be looking out for? are our systems and processes functioning effectively and safely? how will we know when we are improving’?. This seemingly straightforward task of measurement comes with its own set of intricacies and challenges.
Patient safety is a multifaceted concept that encompasses a wide range of factors. From understanding the hazards we deal with every day, the risks associated with those hazards and the clinical processes, the probability and likelihood of those risks leading to any form of harm. We have individual areas of concern; infection control, usability of equipment, medication management for example. There are multiple policies and protocols that can impact, both good and bad, on the safety of care. There are behavioural and relational aspects that are so crucial for safety such as the way in which we communcate with each other, the impact of good and bad relationships, the impact of rudeness, incivility and bullying, the need for a just and learning culture and psychological safety. And of course we have ‘culture’; safety culture, reporting culture, learning culture, organisational culture. All these dimensions of safety are intricate and interconnected.
Healthcare as we know is a complex adaptive system, by its very nature it is dynamic, with constant advancements in knowledge, technologies, and treatments. Keeping safety metrics aligned with these changes requires continuous updates and adjustments. Static measurement approaches may fail to capture the evolving landscape of healthcare, leading to outdated safety assessments. This complexity makes the task of measurement inherently challenging.
Patient safety is not a one-size-fits-all concept. We are often searching for a score, whether that be a single score or a composite score that tells us how safe the whole organisation is. This is nuts when you think about it. One significant hurdle in measuring safety lies in the variation in systems, levels of risk, and level of adaption across departments and systems within healthcare organisations. An ultra safe system requires a much lower tolerance of risk versus an ultra adaptive system that understands risks are inherent in almost everything that is done. All specialties, departments, and care settings within healthcare have unique safety considerations. Determining universally applicable metrics that effectively capture safety across this diversity is a constant challenge if not impossible. A standardised approach often falls short in accommodating these different healthcare contexts.
Our approach to incident reporting is another obstacle in the measurement and monitoring of safety. While I completely understand that incident reports should provide us with data that should help us find particular problems and then fix them, the reality is far from this ideal. Incident reporting systems are not designed in a way that helps us collect meaningful data, they are predominantly reported by nurses, are usually only one person’s side of the story and can only provide a very short description of what has happened. The catch all approach in terms of what we ask to be reported or that gets reported means we are drowning in data and unable to truly see what needs to be fixed as well as we would like.
Measuring safety goes beyond tangible metrics; It isn’t just a case of counting incidents or the numbers of falls or doing a culture survey. In safety numbers can be very misleading. It involves capturing the subjective experiences and perceptions of healthcare professionals and patients. The subjective nature of safety perceptions also introduces an additional layer of complexity, as individual perspectives can vary widely, making it challenging to derive objective and standardised measures.
In January 2023 a paper was published The Safety of Inpatient Health Care authored by David Bates and colleagues. This followed up, three decades later, the Harvard Medical Practice Study (HMPS) conducted in a sample of patients hospitalised in New York State in 1984, published in 1991. The first study was a catalyst for the work on patient safety in the US and led to Sir Liam Donaldson, the then Chief Medical Officer in the UK, to form a group of experts that would lead to the work we have done in patient safety since the year 2000. It also formed the basis for numerous studies worldwide trying to quantify safety and harm. In the UK the pilot study was conducted in acute care by Charles Vincent and colleagues which led to the statistic that we all use to day of 10% of admissions are associated with incidents in hospital care with 5% of these incidents being avoidable.
The authors of the follow up study state ‘Patient safety has changed substantially since the HMPS was performed‘ … ‘However, documenting the extent to which patient safety has improved has been challenging.‘ They evaluated the frequency and types of health care–associated adverse events (we describe as patient safety incidents in the UK) approximately three decades after the original HMPS and found that ‘adverse events remain common and are preventable nearly one fourth of the time. Preventable adverse events were identified in approximately 7% of all admissions, and preventable adverse events categorised as serious, life-threatening, or fatal were identified in approximately 1%‘. Of these, medication incidents were the most common type, followed by incidents related to a surgical or other procedure, falls and pressure ulcers, and health care–associated infections. Incidents such as falls and pressure ulcers as well as medication incidents were the most likely events to be preventable.
The authors naturally state that comparison should be with caution as some data is not more accessible, there are new adverse events as a result of new treatments. Over the last three decades healthcare has become more complex, and more advanced. The way we delivery healthcare has also changed with the most severely ill patients being treated in acute care hospitals.
The authors conclude that ‘our ability to measure many important types of adverse events in an efficient, reliable, and continuous manner remains limited, and our results underscore the need to develop practical measurement tools‘ and ‘despite stunning advances in medical science, we still have important gaps in patient safety‘.
Ref: Bates et al. N Engl J Med 2023; 388:142-153
DOI: 10.1056/NEJMsa2206117
Next time I am interested in exploring another idea that has recently come to my attention, that of Success Cause Analysis. Watch this space!
PATIENT SAFETY NOW 