Evaluation of a digital system leveraging mobile technology and artificial intelligence for digitalization, remote analysis and supported differential cell count of bone marrow aspirates

ABSTRACT

Background
Microscopic examination of bone marrow aspirates (BMA) is still, in almost every organization world-wide, a task that is performed with conventional microscopes relying on an in-person diagnostic process. Being these samples left outside the digital world, it is not possible to leverage tools such as Artificial Intelligence (AI) algorithms to support diagnosis in hematological medical images. Concerning the BMA analysis, the differential cell count (DCC) remains non-automated, being a tedious and time consuming task that is frequently affected by considerable interobserver variability.

Aims
To support the work of hematologists in the analysis of BMA samples by implementing a simple digitization system to develop and validate an AI algorithm for the automatic recognition of different bone marrow cell lineages. This system aims at decreasing the interobserver variability associated with bone marrow DCC and improving the efficiency due to the increased speed of cell quantification.

Methods

100 BMA samples from 95 patients were digitized by photographing 20 fields for each sample (1000x magnification) by hematology researchers. The digitization equipment consists of a 3D printed device to align the smartphone to the ocular lens of a conventional microscope; an acquisition app, and a web-based telemedicine platform to remotely visualize and analyze the images. Resulting images were used to train and validate an AI algorithm for the recognition of different bone marrow cell lineages.

Subsequently, a total of 32,229 cells were identified and each of them was remotely classified and tagged by a panel of 3 different hematologists (randomly selected from the 6 who participated in the study) according to 6 different lineages: Myeloid, Erythroid, Monocytic, Lymphoid, Blasts and Plasma cells.

The AI algorithm was trained only with cells in which all 3 experts agreed on the class (n=28,058). Validation of the algorithm was performed on an independent test set composed by 7,063 cells not included in the training set. Interobserver agreement was calculated for all possible pairs of experts and mean agreement was calculated for each lineage class. 95% confidence intervals (CI) were properly calculated.

A two-stage cell detection and lineage classification deep learning-based algorithm was developed to automatically identify and differentiate the aforementioned 6 lineage classes. Ethics approval for the study was obtained from the Clinical Research Ethics Committee.

Results

The detection algorithm achieved high overall accuracy for detecting nucleated cells in microscopy images (sensitivity and precision of 91.6% and 91.3% respectively). Identified cells were further classified by the algorithm into 6 different lineage classes with an overall accuracy of 92.6%.

Additionally, and for comparative purposes, interobserver agreement was also calculated. Average interobserver agreement was 88.1%. The Figure shows the 95% CI for AI performance and agreement between observers for each lineage class independently. As it can be shown, the AI algorithm demonstrated similar performance to that of a hematologist.

Conclusion
It is possible to digitize BMA samples in a standardized way just relying on smartphones and a simple 3D printed device, transforming BMAs’ DCC in a digital, asynchronous and ubiquitous process. A reliable AI algorithm has been generated to perform an automatic bone marrow DCC with comparable performance to that of a human expert. This AI algorithm can be embedded in any smartphone and has the potential to run in any lab in the world, increasing efficiency and objectivity.