An Artificial Intelligence program developed by Sydney doctors predicts patient deterioration up to 48 hours in advance – better than current manual early warning systems
Dr Levi Bassin and Dr David Bellhave have combined their background in cardiothoracic surgery with their respective degrees in computer science and mathematics to develop an algorithm that improves on the existing Early Warning Score system used in NSW public hospitals to detect patient deterioration, known as ‘Between The Flags’.
“When we looked back over patient observation data, we could see that there were trends in patient deterioration that were not raising alerts,” explained Dr Bassin, who is trained in advanced minimally invasive heart procedures.
“This was particularly true at night when fewer staff are on duty. David and I, who both come from a computer modelling background, set out to build a model using machine learning that could pick up a deteriorating trend earlier.”
Dr Bassin and Dr Bell developed the the Ainsoff Index in collaboration with Information Services and clinical teams at the Sydney Adventist Hospital.
The AI model was developed using anonymised patient demographics, ward-based observations, laboratory values and their trends. Using machine-learning techniques, the data was processed to develop a logistic model and deterioration index to predict patient deterioration prior to an adverse event.
Initial results published in the Journal of Critical Care Medicine show the statistically derived index is superior to other early warning scores at predicting adverse events while there is still time to intervene.
The deterioration index was tested on historical data acquired from 258,732 admissions at two Sydney hospitals for which there were 8,002 adverse events.
Addition of vital sign and laboratory trend values to the logistic model increased the area under the curve (AUC) from 0.84 to 0.89 and the sensitivity to predict an adverse event 1–48 hours prior from 0.35 to 0.41.
A 48-hour simulation showed that the model had a higher AUC than the Modified Early Warning Score and National Early Warning Score (0.87 vs 0.74 vs 0.71).
During the silently run prospective trial, the sensitivity of the deterioration index to detect adverse event any time prior to the adverse event was 0.369 at one hour prior, and 0.327 at four hours prior, with a specificity of 0.972.
A 12-month clinical trial of the system at the Sydney Adventist Hospital has just closed, and the results will be reported in a forthcoming publication.
“Importantly, what we have developed is a highly sensitive and specific tool that improves on just looking at a single set of observations that only alert when a deterioration has occurred,” said Dr Bassin.
“By taking into account the trend in observations and pathology results, as well as an individual patient’s age, sex and their personal statistics, it is a shift away from a ‘one size fits all’ approach to patient deterioration.
“Alert fatigue is a major problem in hospital clinical decision support and the Ainsoff Index produced just 10 per cent of the false alarms when compared to other alert mechanisms. Moreover, it correctly identified more unwell patients than existing systems.
“So we are intervening before people get very sick, to keep them out of the ICU, and using staff time more efficiently. This should lead to improved health outcomes for patients, a more efficient use of the health workforce, and savings for hospitals.”
Sydney Adventist Hospital is the first hospital in Australia to implement this system running in real time for every patient, where the Ainsoff Index has been integrated into the electronic medical record.
It allows any staff member coming onto a ward to see an overview of all patients and so any member of the clinical staff can determine in seconds whether a patient is at risk of deterioration. Staff can quickly be allocated accordingly, with a senior staff member going immediately to care for sicker patients.
There is no additional work required for staff with all data coming from standard patient observations, providing better insight for the existing staff as to where to focus their attention first.
“We see the Ainsoff Index as part of the new era in medical informatics, where AI is being used to assist staff,” said Dr Bassin. “Given our results showing it is superior to currently available early warning systems, it could be considered as an eventual replacement of these less sensitive systems”
Senior nurses at the Sydney Adventist Hospital have said they like the ability to view the health of the entire ward in one snapshot. “The Ainsoff Index for every patient is displayed on one page, which allows us to allocate senior staff to sicker patients, and also to ensure that unwell patients don’t get missed,” said one
Their health analytics company was recently acquired by Beamtree Holdings, which will facilitate widespread adoption of the Ainsoff Index.