Pneumonia is one of the biggest killers in the world. Every year, it causes the death of more than 800,000 children under five – taking a life every 39 seconds. Yet simply by identifying a fast breathing rate, pneumonia can be diagnosed and a child given life-saving antibiotics.

One of the main reasons why it still kills so many children is that in many low and middle-income countries we rely on humans – often community health workers – to count manually the number of breaths per minute a sick child takes. This leaves a huge margin for error. What if they get it wrong?

It is an incredibly challenging task to complete: how do you define what is and is not a breath, especially if the child is moving, crying or breathing rapidly? And how do you not lose count? 

To improve this situation, we need to develop reliable, yet cheap automated respiratory rate counters that can be made available and easily used by anyone. These counters also need to be able to differentiate between breaths and ‘non-breath’ movements – for example a wriggle or a twitch.

At Malaria Consortium we have been doing extensive operational research into respiratory rate counters, in particular whether the aids currently on the market are adopted easily by community health workers, and whether they are reliable. (You can read about our research into the former in our research brief here.)

Testing the reliability of these devices poses a whole new set of problems. How do you do it?

In our first study, we put together a video panel of three medical professionals who watched films of children breathing. They were asked to count the number of breaths in a minute to identify their respiratory rates.

The idea was that we would then compare that to the result from the automated respiratory rate counter – and therefore validate the counter’s reliability.

Interestingly, what we found was that there was disagreement between the people on the video panel. They weren’t coming up with the same answer. This created a new challenge: to come up with an improved reference standard.

The BREATHE study

Our most recent study – entitled Breath Recognition Aid To Heath Experts (BREATHE) – aimed to solve this problem and create a new reference standard.

For the BREATHE study, we used annotation software developed by a senior scientist at Philips Research. This time, the panel watched the video as they did before, but they marked the breaths on the video as it played.

Not only did they mark the breaths, but they also marked the non-breath movements. It is very difficult when you are in the field or watching a video at full speed to know whether it’s a breath or a movement. But with this annotation tool they could slow the video down and get to grips with what they were looking at.

We trained a panel of 10 experts and then randomly selected five to review each video. That panel annotated a total of 51 videos using the tool. We also conducted a focus group discussion with the reviewers to understand the usability and acceptability of the annotation tool.

The hope was that we would get a stronger consensus between the panellists this time, and therefore provide a reference standard that we can then use to test the accuracy of the automated respiratory rate counters.

The results

The full analysis is still being completed, but at this stage we can say that the annotation tool offers ‘good to excellent’ reliability at gaining consensus on breathing rates.

This is a positive result: it shows that the video annotation software is a promising tool to improve reference standards for introducing new automated respiratory rate counters.

Looking at the bigger picture, these results represent a small but crucial step towards being able to test automated respiratory rate counters, find the most reliable and cost-effective ones, and scale them up.

By getting these counters into the hands of the health workers who need them most, we can help improve that accuracy of pneumonia diagnoses – and save lives.

The BREATHE research was funded by the Philips Foundation.

Charlotte Ward will be presenting these findings at ASTMH at 4pm on Saturday 23 November in Scientific Session 157, National Harbor 4/5 (National Harbor Level)

Charlotte Ward is the Senior Research Specialist at Malaria Consortium