Residual Tumor Cellularity Assessment of Breast Cancer after Neoadjuvant Therapy using Image Transformer

Residual tumor cellularity (RTC) is assessed routinely after neoadjuvant therapy (NAT) for Breast Cancer (BC) patients to determine the effectiveness of therapy and plan the treatment. RTC is also considered a prognostic biomarker associated with metastatic recurrence and survival of patients. Traditionally, experts perform the assessment by manually counting the tumor cells in representative tumor regions on hematoxylin and eosin (H&E) specimens under the microscope. This manual assessment is tedious, time-consuming, vulnerable to intra- and inter-observer variability and dependent on the availability of an expert. An automated assessment would be more practical and efficient. Several automated methods were proposed; however, they failed to achieve sufficient accuracy and practical usability for the automated evaluation in digital pathology. This paper presents a fully automated RTC assessment method using H&E whole slide images (WSI) and artificial intelligence (AI) featuring digital pathology. This method utilized a vision transformer (ViT) to select representative tumor regions and a data-efficient image transformer (DeiT) to assess the tumor cellularity based on the selected representative tumor regions. The proposed method was demonstrated on heterogeneous data in which it achieved 97.8% accuracy in evaluating RTC with an Intra-class Correlation Coefficient (ICC) of 0.99, outperforming the state-of-art (0.88). The Mathew Correlation Coefficient (MCC) was 0.971, indicating a perfect agreement between the pathologists and the proposed method. The accuracy of automated tumor selection was 99.7% for test data. High accuracy, strong agreement in RTC assessment and support for automatic tumor selection ensured the practical use of the proposed system.