The evidence base for OSH leading indicators

What we looked at 

Lloyd’s Register Foundation’s strategy promotes the use of the best evidence and insight to understand the complex factors that affect safety. Leading indicators are used widely as performance metrics across many industries in the belief that using more proactive measures of safety will have beneficial impact on accident and incident rates.   

Leading indicators are defined as proactive, preventive, and predictive metrics that inform how effective health and safety practices are. 

The Foundation wanted to explore whether there is robust evidence to support the use of leading indicators in occupational safety and health (OSH), and whether the evidence shows that they make a difference to safety outcomes in terms of preventing or reducing incidents. This work aimed to review what is known about the effectiveness of leading indicators in OSH and establish the current evidence base for them. 

What did we do? 

Researchers from York Health Economics Consortium (YHEC) were commissioned to conduct a literature review and assessment of the evidence found. They are specialists in the research methods necessary to review bodies of literature and were supported by an expert panel of safety practitioners. They conducted a rapid assessment of the literature to establish what evidence exists examining the impact that leading indicators have on lagging indicators (measures of past occurrences such as work-related injuries or fatalities) and specifically to answer the following questions: 

• Which leading indicators have been evaluated? 
• From which industries and in which countries does this evidence come from? 
• Are different leading indicators evaluated in lower to middle income countries, compared to high income countries? 
• How has effectiveness been evaluated? 
• Is there any evidence for the effectiveness of leading indicators, and what is the nature of that evidence? 
• How robust is this evidence base, and how can it be improved? 

They detail their findings in the full report available from the YHEC website. 

What have we learnt? 

The review found a substantial evidence base evaluating the impact of leading indicators on safety lagging indicators. However, that evidence base was disparate, and the quality of the studies included suggests that it is difficult to demonstrate a causal link between leading indicators and an improvement in safety measures from it, for three reasons: 

• The literature found was entirely observational studies which are not designed to demonstrate causality. 
• The robustness of the research methods used in the included studies was evaluated as moderate to low. Over 80% of the included studies were assessed as being of moderate or high risk of bias due to the methods used to conduct them. They are therefore more likely to report distorted estimates of effects of leading indicators. 
• The evidence base is poorly generalisable across settings, due to limited and inconsistent reporting of key study information. 

What did the authors recommend? 

The relevance of reviews of research evidence can be enhanced by considering the implications for practice arising from the findings. However, in this instance the findings make this step difficult. The review found that there is a huge amount of variation in the evidence base, making it difficult to tease out recommendations and to compare findings across studies. Increased standardisation of data collection and wider data sharing would encourage and enhance this. 

The report concludes that there is a clear need for research in this area and suggests perspectives and methods that would need to be considered to produce robust research that could improve the relevance to practice. The diversity of evidence in this review is likely to reflect the complexity of how leading indicators are used in practice. A key challenge for future research will be to establish how to evaluate groups of indicators and the levels of variation that are acceptable to compare results across different studies, and in different contexts. 

What are we going to do next? 

The findings of this rapid evidence assessment build on other research commissions by the Evidence & Insight team at the Foundation. Informed by these, the Foundation has conducted extensive groundwork to explore the potential impact that using a ‘what works’ approach might bring to safety challenges, which will include working with industry to further understand decision making and assess the impact on safety outcomes of translating evidence and research into practice. To take this approach forward, we will establish a Global Safety Evidence Centre (GSEC) at the Foundation, and work will commence on the centre during the second half of 2024. 

‘What works’ is a method that can be used to improve the impact that research findings have on people’s lives. It is based on the principle that good decision making is underpinned by good evidence, and if that evidence isn’t available, robust ways of generating that evidence should be established. ‘What works’ recognises that research evidence on its own is not enough; you need to know how and why something works, for who, and finally, how to implement what is known.   

An evidence standard is a framework to support the assessment of a body of evidence in a field; they vary depending on the type of research evidence being considered and that is most appropriate for a field or sector. This report suggests that evidence standards specific to the OSH context would help to ensure that future research is able to contribute toward improving leading indicator practice and implementation. Developing those standards through active engagement with stakeholders would ensure that research produces the most appropriate evidence, maximising the potential for it to impact on safety practices.   

Developing evidence standards for research in safety is part of the research commissioned by the Foundation to support the establishment of the Lloyd’s Register Foundation GSEC. It is regarded as one of the core underpinning activities to support the activities of the future centre. More information on this research and the GSEC will be available on the Foundation website later in 2024. 

Steve Naylor, Senior Scientist at the UK Health & Safety Executive Science and Research Centre and a member of the advisory panel for this work, discusses the report and how organisations should consider which leading indicators to use in the absence of a robust evidence base.  

Please get in touch via the form below for further information on GSEC and how you might get involved.