Babylon AI Achieves Equivalent Accuracy With Human Doctors in Global Healthcare First

Food and Healthcare Press Releases Thursday June 28, 2018 09:15
LONDON--28 Jun--PRNewswire/InfoQuest
  • Transatlantic collaboration between the London-based medical Artificial Intelligence (AI) company and the Royal College of Physicians, Stanford Primary Care and Yale New Haven Health yields significant breakthrough

Babylon Health tonight announced a world-first during a presentation streamed live from London's Royal College of Physicians: The company's AI, in a series of robust tests (including the relevant sections of the MRCGP exam), has demonstrated its ability to provide heath advice which is on-par with practicing clinicians[1].

The MRCGP exam is the final test for trainee General Practitioners (GPs), set by the Royal College of General Practitioners (RCGP). Trainee GPs who pass this assessment have demonstrated their competence and clinical skills to a level which is sufficiently high enough for them to undertake independent practice.

A key part of this exam tests a doctor's ability to diagnose.

1. Babylon's technology provides health information, rather than a medical diagnosis, for regulatory reasons. The tests carried out relate to the diagnostic exams taken by doctors as a benchmark for accuracy, however, Babylon's AI service remains an information service, rather than a medical diagnosis.

Babylon took a representative sample-set of questions testing diagnostic skills from publicly available RCGP sources[2], as well as independently published examination preparation materials, and mapped these to the current RCGP curriculum in order to ensure the questions resembled actual MRCGP questions as closely as possible.

The average pass mark over the past five years for real-life doctors was 72%[3]. In sitting the exam for the first time, Babylon's AI scored 81%. As the AI is continues to learn and accumulate knowledge, Babylon expects that subsequent testing will produce significant improvements in terms of results.

Important though exams are, doctors are presented with a much wider range of illnesses and conditions in their daily practice. Therefore, to further test the AI's capabilities, Babylon's team of scientists, clinicians and engineers next collaborated with the Royal College of Physicians, Dr Megan Mahoney (Chief of General Primary Care, Division of Primary Care and Population Health, Stanford University), and Dr Arnold DoRosario (Chief Population Health Officer, Yale New Haven Health) to test Babylon's AI alongside seven highly-experienced primary care doctors using 100 independently-devised symptom sets (or 'vignettes').

Babylon's AI scored 80% for accuracy, while the seven doctors achieved an accuracy range of 64-94%.

The accuracy of the AI was 98% when assessed against conditions seen most frequently in primary care medicine. In comparison, when Babylon's research team assessed experienced clinicians using the same measure, their accuracy ranged from 52-99%.

2. As the RCGP does not publish past papers, Babylon used example questions - some published directly by the College, some which were sourced from publicly available resources (which all are referenced) - during its AI exam preparation and testing.

3. Average CSA pass mark by trainee GPs was calculated using publicly available RCGP test result data from the period 2012 - 2017.
Crucially, the safety of the AI was 97%. This compares favourably to the doctors, whose average was 93.1%.

Dr Ali Parsa, Babylon's Founder and CEO said of tonight's news: "The World Health Organisation estimates that there is a shortage of over 5 million doctors globally, leaving more than half the world's population without access to even the most basic healthcare services. Even in the richest nations, primary care is becoming increasingly unaffordable and inconvenient, often with waiting times that make it not readily accessible. Babylon's latest artificial intelligence capabilities show that it is possible for anyone, irrespective of their geography, wealth or circumstances, to have free access to health advice that is on-par with top-rated practicing clinicians.

"Tonight's results clearly illustrate how AI-augmented health services can reduce the burden on healthcare systems around the world. Our mission is to put accessible and affordable health services into the hands of every person on Earth. These landmark results take humanity a significant step closer to achieving a world where no-one is denied safe and accurate health advice."

Babylon's research paper, entitled Evaluation of AI Powered Symptom Checker, can be downloaded from the company's website and will be available over coming days via ArXiv.com.
About Babylon

Babylon's mission is to put an accessible and affordable health service in the hands of every person on earth. Babylon uses a combination of cutting-edge technology and the best available medical expertise to deliver 24-hours-a-day, 7-days-a-week access to digital health tools (including health assessment, triage and health tools), to people across Europe, North America, Asia, the Middle East and Africa, as well as video doctor consultations.

For more information, please visit: https://www.babylonhealth.com/
To access photographs and b-roll assets from tonight's event, visit: https://www.babylonhealth.com/press/assets
To access the research paper as well as additional copies of this press release, visit: https://www.babylonhealth.com/press/press-room
Source: Babylon Health

Latest Press Release

Future Health Index, Commissioned by Philips, Identifies Key Recommendations for Better Health Information Exchange to Drive Value in Healthcare

Royal Philips (NYSE: PHG) (AEX: PHIA), a global leader in health technology, today released a new Future Health Index (FHI) report " Moving data to the heart of health systems: Increasing vital technology adoption to make value-based healthcare a reality...

PULSUS Group collaborates with Anbu Kochi to Contribute towards Kerala Flood Victims

In a humanitarian effort, PULSUS Group - in collaboration with Anbu Kochi - is responding to the calamity of the recent, unprecedented floods in the Indian state of Kerala by providing contributions to aid relief and rehabilitation measures. PULSUS Group...

Menarini Ricerche Announces New Positive Results in Acute Myeloid Leukemia Cell Lines for MEN1112/OBT357 at the 60th Annual SIC Meeting

Menarini Ricerche will present tomorrow, September 20th, at the 60th Annual Meeting of the Italian Cancer Society (SIC), new preclinical data showing that pre-treatment with 5-Azacytidine and Decitabine enhances the Antibody-dependent cellular...

Experts Call for Improved Identification of #High Burden# Rosacea Patients in Everyday Practice

- Additional analysis of global survey data reveals quality of life is impacted by several factors extending beyond severity of rosacea alone Additional analysis of a global survey conducted and presented earlier this year, as an expert-authored report...

BookDoc collaborates with Chow Tai Fook: Get more rewards by staying active together with Chow Tai Fook

Chow Tai Fook is collaborating with BookDoc, the largest integrated healthcare app in Southeast Asia by healthcare network in giving more rewards to BookDoc users for maintaining an active healthy lifestyle. Users can now enjoy discounts and redeem...

Related Topics