How comprehensive should an augmented intelligence (AI) triage system be?

At Healthily, we often get asked: “Does your AI self-assessment platform cover every medical condition?”

The answer is simple: no. And it will never cover all conditions, because there’s no real benefit in doing so.

One key reason for this is that no one really knows what ‘all’ conditions means. Clinical knowledge grows annually and the definitions and sub-divisions of conditions have a near exponential quality.

Take the World Health Organization’s (WHO) International Statistical Classification of Diseases and Related Health Problems (ICD). Begun in 1893, it’s now on its 11th edition (2022). In recent years, the number of conditions deemed suitable for a classification has risen from 12,420 to 68,000.

Given it takes our medical team 20 hours to research, discuss and test every condition we place in our dynamic online triage (DOT) self-assessment tool, it would take us 1,360,000 hours or 170,000 working days or 809 man years to cover all 68,000 conditions in the ICD.

Yes, we use humans. We tried using medical records to train our AI at the start of our journey, but realised 2 things:

• you get some funny results because of unexplained ‘correlations’
• we didn’t need to learn the symptoms of a condition because we had hundreds of years of medical knowledge to draw on.

That’s why we call it ‘augmented intelligence’ rather than ‘artificial intelligence’. The data the system learns from is inputted by our doctors and not randomly scraped from medical records or research papers.

Healthily is currently ‘aware’ of 620 conditions and 1,500 symptoms in its system. We have plans to grow this to between 1,000 and 1,500 in the coming years, but we estimate that our current set represents 80% to 90% of the conditions seen by a primary care doctor in an average year.

This estimate is based on a study that found 75% of problems managed by doctors required knowledge of 100 problems.

This is also supported by a paper published in the BMJ in 2020 showing that Healthily covered 64.5% of conditions with an accuracy of 92.6%. Since then, we’ve added 200 more conditions – easily moving us into the 80% to 90% range. It is worth noting Healthily doesn’t cover childhood conditions and is only suitable for adults over the age of 16.

Our coverage of conditions is representative rather than comprehensive.

The WHO notes that more than half of the 68,000 conditions in the ICD list are defined as rare and affecting fewer than 5 in 10,000 people globally.

But what if you happen to have one of those conditions?

That’s the beauty of DOT. We don’t need to know every condition to give you next-steps advice. You see, we’re not trying to diagnose you but provide you with medically safe information to help you understand the reported symptom or symptoms and most likely possible causes and make a safe decision. In fact, no digital self-assessment tool can really diagnose you. Diagnosis requires a blood test, X-ray, mucus sample or physical observation.

What we’re focusing on is assisting everyone in the world in finding the answer to the question, “What should I do?” rather than “What is wrong with me?”. This is the fundamental difference between our approach and pretty much everyone else’s in this space.

Prof Maureen Baker CBE, Chief Medical Officer at Healthily and a primary care doctor of 35 years, explains: “When you go to the doctor for your first appointment you generally are not diagnosed. Rather, the doctor assesses you and then, based on your symptoms, examination and medical history, decides whether you need investigations or whether it's likely you have a condition that will either resolve spontaneously or can be treated at that point.”

DOT is built with a suite of ‘red flag’ symptoms and combination of symptoms based on clinical guidelines, which forces augmented intelligence to understand when something isn’t right.

The information provided supports you when deciding what to do next by helping you identify less serious conditions that you can manage by yourself and when deciding the need to see a healthcare professional. In healthcare this is called triage and should not be confused with diagnosis.

DOT doesn’t need to match your symptoms to a condition to realise you need to see a healthcare professional. For instance, if you say you are coughing up blood, the possible conditions are irrelevant – you need to have the symptom properly investigated by a medical professional immediately.

And if the system is in doubt, DOT errs on the side of caution.
Test DOT out for yourself by downloading the Healthily app. Find DOT at the bottom of the home screen and start chatting.