Simulia Living Heart Pathology

Computational Pathology: A Survey Review and The Way Forward Mahdi S. Hosseinia,∗, Babak Ehteshami Bejnordib,1, Vincent Quoc-Huy Trinhc, Lyndon Chand, Danial Hasand, Xingwen Lid, Stephen Yangd, Taehyo Kimd, Haochen Zhangd, Theodore Wud, Kajanan Chinniahd, Sina Maghsoudloua, Ryan Zhangd, Jiadai Zhud, Samir Khakid, Andrei Buine, Fatemeh Chajia, Ala Salehif, Bich Ngoc Nguyeng,

Dimitris Samarash, Konstantinos N. Plataniotisd

bQualcomm AI Research, Qualcomm Technologies Netherlands B.V., Amsterdam, The Netherlands eHuron Digitial Pathology, St. Jacobs, ON N0B 2N0, Canada This work is dedicated to the beloved memories of Kuanhou Fang, Shahnaz Habibpanah, Zakiyeh Khaliji-Oskoui, James Liang, Mahsa MohammadiMoghaddam, Vily Panoutsakopoulou, Athanasia Samara, Huoyuan Yu, Dexi Zhang and all people around the world who have lost their lives because of cancer Abstract.

Computational Pathology (CPath) is an interdisciplinary science that augments developments of computational approaches to analyze and model medical histopathology images. The main objective for CPath is to develop infrastructure and workflows of digital diagnostics as an assistive CAD system for clinical pathology, facilitating transformational changes in the diagnosis and treatment of cancer that are mainly address by CPath tools. With evergrowing developments in deep learning and computer vision algorithms, and the ease of the data flow from digital pathology, currently CPath is witnessing a paradigm shift. Despite the sheer volume of engineering and scientific works being introduced for cancer image analysis, there is still a considerable gap of adopting and integrating these algorithms in clinical practice. This raises a significant question regarding the direction and trends that are undertaken in CPath. In this article we provide a comprehensive review of more than 700 papers to address the challenges faced in problem design all-the-way to the application and implementation viewpoints. We have catalogued each paper into a model-card by examining the key works and challenges faced to layout the current landscape in CPath. We hope this helps the community to locate relevant works and facilitate understanding of the field’s future directions. In a nutshell, we oversee the CPath developments in cycle of stages which are required to be cohesively linked together to address the challenges associated with such multidisciplinary science. We overview this cycle from different perspectives of data-centric, model-centric, and application-centric problems. We finally sketch remaining challenges and provide directions for future technical developments and clinical integration of CPath. For updated information on this survey review paper and accessing to the original model cards repository, please refer to GitHub.

Keywords: digital pathology, whole slide image (WSI), deep learning, computer aided diagnosis (CAD), clinical pathology, survey

2.1

Clinical Pathology Workflow . . . . . . . . . . . . . .

Diagnostic Tasks

. . . . . . . . . . . . . . . . . . . .

2.3

Prognosis . . . . . . . . . . . . . . . . . . . . . . . .

2.4

Prediction of treatment response . . . . . . . . . . . .

Organs And Diseases

. . . . . . . . . . . . . . . . . .

Tissue Slide Preparation

. . . . . . . . . . . . . . . .

10

∗Corresponding author. 1Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc.

Whole Slide Imaging (Wsi)

. . . . . . . . . . . . . .

3.3

Cohort Selection, Scale, and Challenges . . . . . . . .

4.1

Supervised Annotation . . . . . . . . . . . . . . . . .

4.2

Optimum Labeling Workflow Design . . . . . . . . . .

5.1

Classification Architectures . . . . . . . . . . . . . . .

5.2

Segmentation Architectures . . . . . . . . . . . . . . .

5.3

Object Detection Architectures . . . . . . . . . . . . .

Multi-Task Learning

. . . . . . . . . . . . . . . . . .

Multi-Modal Learning

. . . . . . . . . . . . . . . . .

5.6

Vision-Language Models . . . . . . . . . . . . . . . .

Sequential Models

. . . . . . . . . . . . . . . . . . .

5.8

Synthetic Data and Generative Models . . . . . . . . .

Multi-Instance Learning (Mil) Models

. . . . . . . .

2

5.10 Contrastive Self-Supervised Learning for Few-Shot Generalization . . . . . . . . . . . . . . . . . . . . . .

20

5.11 Novel CPath Architectures . . . . . . . . . . . . . . .

20

5.12 Model Comparison . . . . . . . . . . . . . . . . . . .

Clinical Validation

. . . . . . . . . . . . . . . . . . .

6.2

FDA Regulations . . . . . . . . . . . . . . . . . . . .

7.1

Contrastive Self-Supervised Learning becomes Main- stream . . . . . . . . . . . . . . . . . . . . . . . . . .

7.2

Prediction becoming increasingly High-Level . . . . .

7.3

Spatial and Hierarchical Relationships receiving At- tention . . . . . . . . . . . . . . . . . . . . . . . . . .

7.4

Vision-Language Models for Explainable Predictions .

Synthetic Data Now Realistic Enough

. . . . . . . . .

8.1

CPath as Anomaly Detection . . . . . . . . . . . . . .

8.2

Leveraging Existing Datasets . . . . . . . . . . . . . .

8.3

Creating New Datasets . . . . . . . . . . . . . . . . .

8.4

Pre- and Post-Analytical CAD Tools . . . . . . . . . .

8.5

Multi Domain Learning . . . . . . . . . . . . . . . . .

8.6

Federated Learning for Multi-Central CPath . . . . . .

8.7

CPath-specific Architecture Designs . . . . . . . . . .

8.8

Digital and Computational Pathology Adoption . . . .

8.9

Institutional Challenges . . . . . . . . . . . . . . . . .

26

8.10 Clinical Alignment of CPath Tasks . . . . . . . . . . .

26

8.11 Concluding Remarks . . . . . . . . . . . . . . . . . .

9.1

Clinical Pathology Workflow . . . . . . . . . . . . . .

Diagnostic Tasks

. . . . . . . . . . . . . . . . . . . .

9.3

Prognosis . . . . . . . . . . . . . . . . . . . . . . . .

Prediction Of Treatment Response

. . . . . . . . . . .

9.5

Cancer Statistics . . . . . . . . . . . . . . . . . . . . .

9.6

Whole Slide Imaging . . . . . . . . . . . . . . . . . .

Organs And Diseases

. . . . . . . . . . . . . . . . . .

9.8

Ground Truth Labelling and Annotation . . . . . . . .

9.9

Surveyed Datasets . . . . . . . . . . . . . . . . . . . .

9.9.1

Table Creation Details . . . . . . . . . . . . .

59

9.10 Organ Overview . . . . . . . . . . . . . . . . . . . . .

59

9.11 Technicalities by Task . . . . . . . . . . . . . . . . . .

59

9.12 Model Card Categorization . . . . . . . . . . . . . . .

81

9.12.1 Template . . . . . . . . . . . . . . . . . . . .

81

9.12.2 Samples . . . . . . . . . . . . . . . . . . . . .

1. Introduction

April 2017 marked a turning point for digital pathology when the Philips IntelliSite digital scanner received the US Food & Drugs Administration (FDA) approval (with limited use case) for diagnostic applications in clinical pathology [1, 2]. A sub- sequent validation guideline was created to help ensure the pro- duced Whole Slide Image (WSI) scans could be used in clin- ical settings without compromising patient care, while main- taining similar results to the current gold standard of optical microscopy [3, 4, 5, 6]. The use of WSIs offers significant ad- vantages to the pathologist’s workflow: digitally captured im- ages, compared to tissue slides, are immune from accidental physical damage and maintain their quality over time [7, 8].

Clinics and practices can share and store these high-resolution images digitally enabling asynchronous viewing/collaboration worldwide [9, 10]. The development of digital pathology shows great promise as a framework to improve work efficiency in the practice of pathology [11, 10]. Adopting a digital workflow also opens immense opportunities for using computational methods to augment and expedite their workflow–the field of Computa- tional Pathology (CPath) is dedicated to researching and devel- oping these methods [12, 13, 14, 15, 16, 17].

However, despite the aforementioned advantages, the adop- tion of digital pathology, and hence computational pathology, has been slow. Some pathologists consider the analysis of WSIs as opposed to glass slides as an unnecessary change in their workflow [18, 19, 9, 20] and recent surveys indicate that the switch to digital pathology does not provide enough financial incentive [21, 22, 23, 24, 25, 8]. This is where advances from CPath can address or overpower many of the concerns in adopt- ing a digital workflow. For example, CPath models to identify morphological features that correlate with breast cancer provide substantial benefits to clinical accuracy. Further, CPath models that identify lymph node metastases with better sensi- tivity while reducing diagnostic time can streamline work- flows to increase pathologist throughput and generate more rev- enue [28, 29].

Similar to digital pathology, the adoption of CPath methods has also lagged despite the many benefits it offers to improve efficiency and accuracy in pathology [30, 31, 2, 32]. This lack of adoption and integration into clinical practice raises a sig- nificant question regarding the direction and trends of current work in CPath. This survey looks to review the field of CPath in a systematic fashion by breaking down the various steps in- volved in a CPath workflow and categorizing CPath works to both determine trends in the field and provide a resource for the community to reference when creating new works.

Existing survey papers in the field of CPath can be clus- tered into a few groups. The first focuses on the design and applications of smart diagnosis tools [33, 34, 35, 36, 37, 38, 17, 16, 15, 39, 40, 41, 42, 43].

These Works Focus On De-

signing novel architectures for artificial intelligence (AI) mod- els with regards to specific clinical tasks, although they may briefly discuss clinical challenges and limitations.

A Sec-

ond group of works focus on clinical barriers for AI integra- tion discussing specific certifications and regulations required for the development of medical devices under clinical settings [44, 45, 46, 47, 48, 49]. Lastly, the final group focuses on both the design and the integration of AI tools with clinical applica- tions [50, 51, 12, 13, 52, 29, 53, 54, 14, 55, 56]. These works speak to both the computer vision and pathology communities in developing machine learning (ML) models that can satisfy clinical use cases.

Our work is situated in this final group as we breakdown the end-to-end CPath workflow into stages and systematically re- view works related to and addressing those stages. We oversee this as a workflow for CPath research that breaks down the pro- cess of problem definition, data collection, model creation, and clinical validation into a cycle of stages. A visual representation of this cycle is provided in Figure 1. We review over 700 papers

3

Fig. 1: We divide the data science workflow for pathology into multiple stages, wherein each brings a different level of experience. For example, the anno- tation/ground truth labelling stage (c) is where domain expert knowledge is consulted as to augment images with associated metadata. Meanwhile, in the evaluation phase (e), we have computer vision scientists, software developers, and pathologists working in concert to extract meaningful results and implica- tions from the representation learning.

from all areas of the CPath field to examine key works and chal- lenges faced. By reviewing the field so comprehensively, our goal is to layout the current landscape of key developments to allow computer scientists and pathologists alike to situate their work in the overall CPath workflow, locate relevant works, and facilitate an understanding of the field’s future directions. We also adopt the idea of generating model cards from and de- signed a card format specifically tailored for CPath. Each paper we reviewed was catalogued as a model card that concisely de- scribes (1) the organ of application, (2) the compiled dataset, (3) the machine learning model, and (4) the target task. The com- plete model card categorization of the reviewed publications is provided in Appendix 9.12 for the reader’s use.

In our review of the CPath field, we find that two main ap- proaches emerge in works: 1) a data-centric approach and 2) a model-centric approach. Considering a given application area, such as specific cancers, e.g. breast ductal carcinoma in-situ (DCIS), or a specific task, e.g. segmentation of benign and malignant regions of tissue, researchers in the CPath field fo- cus generally on either improving the data or innovating on the model used.

Works with data-centric approaches focus on collecting pathology data and compiling datasets to train models on cer- tain tasks based on the premise that the transfer of domain ex- pert knowledge to models is captured by the process of collect- ing and labeling high-quality data [58, 51, 59]. The motivation behind this approach in CPath is driven by the need to 1) ad- dress the lack of labeled WSI data representing both histology and histopathology cases due to the laborious annotation pro- cess and 2) capture a predefined pathology ontology pro- vided by domain expert pathologists for the class definitions and relations in tissue samples. Regarding the lack of labeled WSI data our analysis reveals that there are a larger number of datasets with granular labels, but there is a larger total amount of data available for a given organ and disease application that have weakly supervised labels at the Slide or Patient-level. Al- though some tasks, such as segmentation and detection, require WSI data to have more granular labels at the region-of-interest (ROI) or image mosaic/tiles (known as patch) levels, to capture more precise information for training models, there is a poten- tial gap to leverage the large amount of weakly-supervised data to train models that can be later used downstream on smaller strongly-supervised datasets for those tasks. When considering the ontology of pathology as compared to the field of computer vision, we note that pathology datasets have far fewer classes than computer vision (e.g. ImageNet-20K contains 20,000 class categories for natural images whereas CAMELYON17 has four annotated classes for breast cancer metastases ), but has much more variation within each of these classes in terms of representations and fuzzy boundaries around the grade of cancers which subdivides each class into many more in reality.

There are also very rare classes in the form of rare diseases and cancers, as presented in Figure 12 and discussed in Section 2, which present a class imbalance challenge when compiling data or training models. If one considers the complexities involved in representational learning of related tissues and diseases, it raises the question of whether there is a clear understanding and consensus in the field of how an efficient dataset should be compiled for model development. Our survey analyzes the availability of CPath datasets along with what area of applica- tion they address and their annotation level in detail in Section 3.3, and the complete table of datasets we have covered is avail- able in Appendix 9.9. Section 4 goes into more depth about the various levels of annotation, the annotation process, and select- ing the appropriate annotation level for a task.

The model-centric approach, by contrast, is favoured by computer scientists and engineers, who design algorithmic ap- proaches based on the available pathology data.

Selection

of a modelling approach, such as self-supervised, weakly- supervised, or strongly-supervised learning, is dictated directly by the amount of data available for a given annotation level and task. Currently, many models are developed on datasets with strongly-supervised labels at the ROI, Patch, or Pixel-levels to address tasks such as tissue type classification or disease de- tection. However, a recent trend is developing to apply self- supervised and weakly-supervised learning methods to leverage the large amount of data with Slide and Patient-level annota- tions . Models are trained in a self or weakly supervised manner to learn representations on a wider range of pathol- ogy data across organs and diseases, which can be leveraged for other tasks requiring more supervision but without the need for massive labeled datasets [63, 64, 65]. This trend points to the future direction of CPath models following a similar trend to that in computer vision, where large-scale models are being

4

pre-trained using self-supervised techniques to achieve state-of- the-art performance in downstream tasks [66, 67]. Although data and model centric approaches are both im- portant in advancing the performance of models and tools in CPath, we note a need for much more application centric work in CPath. We define a study to be application centric if the primary focus is on addressing a particularly impactful task or need in the clinical workflow, ideally including clinical valida- tion of the method or tool. To this end, Section 2 details the clinical pathology workflow from specimen collection to report generation, major task categories in CPath, and specific applica- tions per organ. Particularly, we find that very few works focus on the pre or post-analytical phases of the pathology workflow where many errors can occur, instead focusing on the analyt- ical phase where interpretation tasks take place. Additionally, certain types of cancer with deadly survival rates are underrep- resented in CPath datasets and works. Very few CPath models and tools have been validated in a clinical setting by pathol- ogists, suggesting that there may still be massive barriers to actually using CPath tools in practice. All of this points to a severe oversight by the CPath community towards considering the actual application and implementation of tools in a clinical setting. We suspect this to be a major reason as to why there is a slow uptake in adopting CPath tools by pathology labs.

The contributions of this survey include the provision of an end-to-end workflow for developing CPath work which outlines the various stages involved and is reflected within the survey sections.

Further, We Propose And Provide A Comprehensive

conceptual model card framework for CPath that clearly cat- egorizes works by their application of interest, dataset usage, and model, enabling consistent and easy comparison and re- trieval of papers in relevant areas. Based on our analysis of the field, we highlight several challenges and trends, including the availability of datasets, focus on models leveraging exist- ing data, disregard of impactful application areas, and lack of clinical validation. Finally, we give suggestions for address- ing these aforementioned challenges and provide directions for future work in the hopes of aiding the adoption and implemen- tation of CPath tools in clinical settings.

The structure of this survey closely follows the CPath data workflow illustrated in Figure 1. Section 2 begins by outlining the clinical pathology workflow and covers the various task do- mains in CPath, along with organ specific tasks and diseases.

The next step of the workflow involves the processes and meth- ods of histopathology data collection, which is outlined in Sec- tion 3. Following data collection, Section 4 details the corre- sponding annotation and labeling methodology and considera- tions. Section 5 covers deep learning designs and methodolo- gies for CPath applications. Section 6 focuses on regulatory measures and clinical validation of CPath tools. Section 7 ex- plores emerging trends in recent CPath research. Finally, we provide our perceived challenges and future outlook of CPath in Section 8.

2. Clinical Applications For Cpath

The field of CPath is dedicated to the creation of tools that address and aid steps in the clinical pathology workflow. Thus, a grounded understanding of the clinical workflow is of paramount importance before development of any CPath tool. The outcomes of clinical pathology are diagnostics, prog- nostics, and predictions of therapy response. Computational pathology systems that focus on diagnostic tasks aim to as- sist the pathologists in tasks such as tumour detection, tumour grading, quantification of cell numbers, etc. Prognostic systems aim to predict survival for individual patients while therapy re- sponse predictive models aid personalized treatment decisions based on histopathology images. Figure 3 visualizes the goals pertaining to these tasks. In this section, we provide detail on the clinical pathology workflow, the major application areas in diagnostics, prognostics, and therapy response, and finally de- tail the cancers and CPath applications in specific organs. The goal is to outline the tasks and areas of application in pathology where CPath tools and systems can be developed and imple- mented.

2.1. Clinical Pathology Workflow

This subsection provides a general overview of the clini- cal workflow in pathology covering the collection of a tissue sample, its subsequent processing into a slide, inspection by a pathologist, and compilation of the analysis and diagnosis into a pathology. Figure 2 summarizes these steps at a high level and provides suggestions for corresponding CPath appli- cations. The steps are organized under the conventional pathol- ogy phases for samples: pre-analytical, analytical, and post- analytical. These phases were developed to categorize qual- ity control measures, as each phase has its own set of potential sources of errors , and thus potential sources of corrections during which CPath and healthcare artificial intelligence tools could prove useful. For details about each step of the workflow, please refer to the Appendix 9.1.

Pre Analytical Phase The first step of the pre-analytical phase is a biopsy performed to collect a tissue sample, where the biopsy method is dependent on the type of sample required and the tissue characteristics. Sample collection is followed by accessioning of the sample which involves entering of the pa- tient and specimen information into a Laboratory Information System (LIS) and linking to the Electronic Medical Records (EMR) and potentially a Slide Tracking System (STS). Af- ter accessioning, smaller specimens that have not already been preserved by fixation in formalin are fixated. Once the basic specimen preparation has occurred, the tissue is analyzed by the pathology team without the use of a microscope; a step called grossing. Grossing involves cross-referencing the clin- ical findings and the EMR reports, with the operator localiz- ing the disease, locating the pathological landmarks, describing these landmarks, and measuring disease extent. Specific sam- pling of these landmarks is performed, and these samples are then put into cassettes for the final fixation. Subsequently, the samples are then sliced using a microtome, stained using the relevant stains for diagnosis, and covered with a glass slide.

Analytical Phase After a slide is processed and prepared, a pathologist views the slide to analyze and interpret the sample. The approach to interpretation varies depending on the spec- imen type. Interpretation of smaller specimens is focused on

5

Fig. 2: Quality assurance and control phases developed by pathologists to over- see the clinical pathology workflow into three main phases of pre-analytical, analytical, and post-analytica phases. We further show how each of these pro- cesses can be augmented under the potential CPath applications in an end-to- end pipeline.

diagnosis of any disease. Analysis is performed in a decision- tree style approach to add diagnosis-specific parameters, e.g. esophagus biopsy →type of sampled mucosa →presence of folveolar-type mucosa →identify Barrett’s metaplasia →iden- tify degree of dysplasia. Once the main diagnosis has been identified and characterized, the pathologist sweeps the remain- ing tissue for secondary diagnoses which can also be charac- terized depending on their nature. Larger specimens are more complex and usually focus on characterizing the tissue and identifying unexpected diagnoses beyond the prior diagnosis from a small specimen biopsy. Microscopic interpretation of large specimens is highly dependent on the quality of the gross- ing and the appropriate detection and sampling of landmarks.

Each landmark (e.g., tumor surface, tumor at deepest point, surgical margins, lymph node in mesenteric fat) is character- ized either according to guidelines, if available, or according to the pathologist’s judgment.

After The Initial Microscopic

interpretation additional deeper cuts (“levels”), special stains, immunohistochemistry (IHC), and/or molecular testing may be performed to hone the diagnosis by generating new material or slides from the original tissue block.

Post-Analytical Phase The pathologist synthesizes a diagno- sis by aggregating their findings from grossing and microscopic examination in combination with the patient’s clinical informa- tion, all of which are included in a final pathology report. The classic sections of a pathology report are patient information, a list of specimens included, clinical findings, grossing report, microscopic description, final diagnosis, and comment. The length and degree of complexity of the report again depends on the specimen type. Small specimen reports are often suc- cinct, clearly and unambiguously listing relevant findings which guide treatment and follow-up. Large specimen reports depend on the disease, for example, in cancer resection specimens the grossing landmarks are specifically targeted at elements that will guide subsequent treatment.

In the past, pathology reports had no standardized format, usually taking a narrative-free text form. Free text reports can omit necessary data, include irrelevant information, and con- tain inconsistent descriptions . To combat this, synoptic re- porting was introduced to provide a structured and standardized reporting format specific to each organ and cancer of interest [69, 70]. Over the last 15 years, synoptic reporting has enabled pathologists to communicate information to surgeons, oncolo- gists, patients, and researchers in a consistent manner across in- stitutions and even countries. The College of American Pathol- ogists (CAP) and the International Collaboration on Cancer Re- porting (ICCR) are the two major institutions publishing synop- tic reporting protocols. The parameters included in these pro- tocols are determined and updated by CAP and ICCR respec- tively to remain up-to-date and relevant for diagnosis of each cancer type. For the field of computational pathology, synoptic reporting provides a significant advantage in dataset and model creation, as a pre-normalized set of labels exist across a vari- ety of cases and slides in the form of the synoptic parameters filled out in each report. Additionally, suggestion or prediction of synoptic report values are a possible CPath application area.

2.2. Diagnostic Tasks

Computational pathology systems that focus on diagnostic tasks can broadly be categorized as: (1) disease detection, (2) tissue subtype classification, (3) disease diagnosis, and (4) seg- mentation. These tasks are visually depicted in Figure 3. Note how the detection tasks all involve visual analysis of the tissue in WSI format. Thus computer vision approach is primarily adopted towards tackling diagnostic tasks in computer aided di- agnosis (CAD). For additional detail on some previous works on these diagnostic tasks, we refer the reader to Appendix 9.2 Detection We define the detection task as a binary classifica- tion problem where inputs are labeled as positive or negative, indicating the presence or absence of a certain feature. There may be variations in the level of annotation required, e.g. slide- level, patch-level, pixel-level detection depending on the fea- ture in question.

Although Detection Tasks May Not Provide

an immediate disease diagnosis, it is a highly relevant task in many pathology workflows as pathologists incorporate the pres- ence or absence of various histological features into synoptic reports that lead to diagnosis. Broadly, detection tasks fall into two main categories: (1) screening the presence of cancers and (2) detecting histopathological features specific to certain diag- noses.

Cancer detection algorithms can assist the pathologists by fil- tering obviously normal WSIs and directing pathologist’s focus

6

Fig. 3: The categorization of diagnostic tasks in computational pathology along with examples A) Detection: common detection task such as differentiating positive from negative classes like malignant from benign, B) Tissue Subtype Classification: classification task for tumorous tissue, Stroma, and adipose tis- sue, C) Disease Diagnosis: common disease diagnosis task like cancer stag- ing, D) Segmentation: tumor segmentation in WSIs, and E) Prognosis tasks: shows a graph comparing survival rate and months after surgery.

to metastatic regions . Although pathologists have to re- view all the slides to check for multiple conditions regardless of the clinical diagnosis, an accurate cancer detection CAD would expedite the workflow by pinpointing the ROIs and summariz- ing results into synoptic reports, ultimately leading to a reduces time per slide. Due to this potential impact, cancer detection tasks have been explored in a broad set of organs. Additionally, the simple labeling in binary detection tasks allows for deep learning methods to generalize across different organs where similar cancers form [72, 73, 74].

Tissue Subtype Classification Treatment and patient progno- sis can vary widely depending on the stage of cancer, and finely classifying specific tissue structures associated with a specific disease type provides essential diagnostic and prognostic in- formation . Accordingly, accurately classifying tissue sub- types is a crucial component of the disease diagnosis process.

As an example, discriminating between two forms of glioma (a type of brain cancer), glioblastoma multiforme and lower grade glioma, is critical as they differ by over 45% in patient survival rates . Additionally, accurate classification is key in col- orectal cancer (CRC) diagnosis, as high morphological varia- tion in tumor cells makes certain forms of CRC difficult to diagnose by pathologists . We define this classification of histological features as tissue subtype classification.

Disease Diagnosis The most frequently explored design of deep learning in digital pathology involves emulating pathol- ogist diagnosis. We define this multi-class diagnosis problem as a disease diagnosis task. Note the similarity with detection– disease diagnosis can be considered a fine-grained classification problem which subdivides the general positive disease class into finer disease-specific labels based on the organ and patient context.

Segmentation The segmentation task moves one step beyond classification by adding an element of spatial localization to the predicted label(s). In semantic segmentation, objects of inter- est are delineated in an image by assigning class labels to every pixel. These class labels can be discrete or non-discrete, the lat- ter being a more difficult task . Another variant of the seg- mentation task is instance segmentation, which aims to achieve both pixel-level segmentation accuracy as well as clearly de- fined object (instance) boundaries. Segmentation approaches can accurately capture many morphological statistics and textural features , both of which are relevant for cancer di- agnosis and prognosis. Most frequently, segmentation is used to capture characteristics of individual glands, nuclei, and tu- mor regions in WSIs.

For Instance, Glandular Structure Is A

critical indicator of the severity of colorectal carcinoma , thus accurate segmentation could highlight particularly abnor- mal glands to the pathologist as demonstrated in [82, 83, 84].

Overall, segmentation provides localization and classification of cancer-specific tumors and of specific histological features that can be meaningful for the pathologist’s clinical interpreta- tion.

2.3. Prognosis

Prognosis involves predicting the likely development of a disease based on given patient features. For accurate survival prediction, models must learn to both identify and infer the effects of histological features on patient risk. Prognosis rep- resents a merging of the diagnosis classification task and the disease-survivability regression task.

Training a model for prognosis requires a comprehensive set of both histopathology slides and patient survival data (i.e. a variant of multi-modal representation learning).

Despite The

complexity of the input data, ML models are still capable of extracting novel histological patterns for disease-specific sur- vivability [85, 86, 87]. Furthermore, strong models can dis- cover novel prognostically-relevant histological features from WSI analysis [88, 89]. As the quality and comprehensiveness of data improves, additional clinical factors could be incorpo- rated into deep learning analysis to improve prognosis.

2.4. Prediction Of Treatment Response

With the recent advances in targeted therapy for cancer treat- ment, clinicians are able to use treatment options that precisely identify and attack certain types of cancer cells. While the num- ber of options for targeted therapy are constantly increasing, it becomes increasingly important to identify patients who are potential therapy responders to a specific therapy option and avoid treating non-responding patients who may experience se- vere side effects. Deep learning can be used to detect structures and transformations in tumour tissue that could be used as pre- dictive markers of a positive treatment response. Training such deep learning models usually requires large cohorts of patient data for whom the specific type of treatment option and the cor- responding response is known.

2.5. Organs And Diseases

This section presents an overview of the various anatomical application areas for computational pathology grouped by the targeted organ. Each organ section gives a brief overview of the types of cancers typically found and the content of the pathol- ogy report as noted from the corresponding CAP synoptic re- porting outline (discussed at 2.1). Figure 4 highlights the inter- section between the major diagnostic tasks and the anatomical

7

Fig. 4: Distribution of diagnostic tasks in CPath for different organs from Table 9.11. This distribution includes more than 400 cited works from 2018 to 2022 inclusive. The x-axis covers different organs, the y-axis displays different di- agnostic tasks, and the height of the bars along the vertical axis measures the number of works that have examined the specific task and organ. Please refer to Table 9.11 in the supplementary section for more information.

focuses in state-of-the-art research. The majority of papers are dedicated to the four most common cancer sites: breast, colon, prostate, and lung . Additionally, a significant amount of re- search is also done on cancer types with highest mortality, brain and liver. . Note that details of some additional works that may be of interest for each organ type can be found in Appendix

9.7

Breast Breast cancers can start from different parts of the breast and majorly consist of 1) Lobular cancers that start from lobular glands, 2) Ductal cancers, 3) Paget cancer which in- volves the nipple, 4) Phyllodes tumor that stems from fat and connective tissue surrounding the ducts and lobules, and 5) An- giosarcoma which starts in the lining of the blood and lymph vessels. In addition, based on whether the cancer has spread or not, breast cancers can be categorized into in situ or inva- sive/infiltrating forms. DCIS is a precancerous state and is still confined to the ducts. Once the cancerous cells grow out of the ducts, the carcinoma is now considered invasive or infiltrative and can metastasize .

Synoptic reports for breast cancer diagnosis are divided based on the type of cancers mentioned above. For DCIS and invasive breast cancers, synoptic reports focus on the histo- logic type and grade, along with the nuclear grade, evidence of necrosis, margin, involvement of regional lymph nodes, and biomarker status. Notably, architectural patterns are no longer a valuable predictive tool compared to nuclear grade and necro- sis to determine a relative ordering of diagnostic importance for DCIS . In contrast to DCIS and invasive cancers, Phyl- lodes tumours vary due to their differing origin in the fat and connective tissue, focusing on analyzing the stroma character- istics, existence of heterologous elements, mitotic rate, along with the involvement of lymph nodes. Finally, to determine therapy response and treatments, biomarker tests for Estrogen, Progesterone and HER-2 receptors are recommended, along with occasional tests for Ki67 antigens [95, 96].

Most breast cancer-focused works in CPath propose vari- ous solutions for carcinoma detection and metastasis detection, an important step for assessing cancer stage and morbidity.

Metastasis detection using deep learning methods was shown to outperform pathologists’ exhaustive diagnosis by 9% free- response receiver operating characteristic (FROC) in .

Prostate Prostate cancer is the second most prevalent cancer among the total population and the most common cancer among men (both excluding non-melanoma skin cancers). However, most prostate cancers are not lethal. Prostate cancer can oc- cur in any of the three prostate zones: Central (CZ), Periph- eral (PZ), and Transition (TZ), in increasing order of aggres- siveness.

Prostate Cancers Are Almost Always Adenocarcino-

mas, which develop from the gland cells that make prostate fluid. The other types of prostate cancers are small cell car- cinomas, neuroendocrine tumors, transitional cell carcinomas, isolated intraductal carcinoma, and sarcomas (which are very rare). Other than cancers, there are multiple conditions that are important to identify or diagnose as precursors to cancer or not. Prostatic intraepithelial neoplasia (PIN) is diagnosed as either low-grade PIN or high-grade PIN. Men with high-grade PIN need closely monitored follow-up sessions to screen for prostate cancer. Similarly, atypical small acinar proliferation (ASAP) is another precancerous condition requiring follow-up biopsies.

To grade and score tumours, pathologists use a Tumor, Nodes, Metastasis (TNM) framework. In the synoptic report, pathologists identify and report the histologic type and grades, and involvement of regional lymph nodes to help grade and provide prognosis for any tumours. Specifically for prostate analysis, tumour size and volume are both important factors in prognosis according to multiple studies [99, 100, 101, 102].

Similarly, location is important to note for both prognosis and therapy response . Invasion to nearby (except perineural invasion) tissues is noted and can correlate to TMN classifi- cation . Additionally, margin analysis is especially im- portant in prostate cancers as the presence of a positive margin increases the risk of cancer recurrence and metastasis . Fi- nally, intraductal carcinoma (IDC) must be identified and dis- tinguished from PIN and PIA; as it is strongly associated with a high Gleason score, a high-volume tumor, and metastatic dis- ease [106, 107, 108, 109, 110].

After a prostate cancer diagnosis is established, pathologists assign a Gleason Score to determine the cancer’s grade: a grade from 1 to 5 is assigned to the two most common areas and those two grades are summed to make a final Gleason Score . For Gleason scores of 7, where survival and clinical out- comes demonstrate large variance, the identification of Crib- riform glands is key in helping to narrow possible outcomes [112, 113].

Ovary Ovarian cancer is the deadliest gynecologic malig- nancy and accounts for more than 14, 000 deaths each year . Ovarian cancer manifests in three types: 1) epithelial cell tumors that start from the epithelial cells covering the outer sur- face of the ovary, 2) germ cell tumors which start from the cells that produce eggs, and 3) stromal tumors which start from cells that hold the ovary together and produce the hormones estrogen and progesterone. Each of these cancer types can be classified into benign, intermediate and malignant categories. Overall, epithelial cell tumors are the most common ovarian cancer and have the worst prognosis .

When compiling a synoptic report for ovarian cancer diag-

8

nosis, pathologists focus on histologic type and grade, extra- organ involvement, regional lymph nodes, T53 gene mutations, and serous tubal intraeptithelial carconma (STIC). Varying his- tologic tissue types are vital to determine the pathology char- acteristics and determining eventual prognosis. For example, generally endometrioid, mucinous, and clear cell carcinomas have better outcomes than serous carcinomas . Addition- ally, lymph node involvement and metastasis in both regional and distant nodes has a direct correlation to patient survival, grading, and treatment. Determining the presence of STICs correlates directly to the presence of ovarian cancer, as 60% of ovarian cancer patients will also have an associated STIC .

Finally, T53 gene mutations are the most common in epithelial ovarian cancer; which has the worst prognosis among ovarian cancers, so determining their presence is critical to patient can- cer risk and therapy response [117, 118]. There are not a large number of works dedicated to the ovary specifically, but most works on ovary focus on classification of its five most common cancer subtypes: high-grade serous (HGSC), low-grade serous (LGSC), endometriod (ENC), clear cell (CCC), and mucinous

(Muc) [119, 120].

Lung Lung cancer is the third most common cancer type, next to breast and prostate cancer . Lung cancers mostly start in the bronchi, bronchioles, or alveoli and are divided into two major types, non-small cell lung carcinomas (NSCLC) (80−85%) and small cell lung carcinomas (SCLC) (10−15%).

Although NSCLS cancers are different in terms of origin, they are grouped because they have similar outcomes and treatment plans.

Common Nscls Cancers Are 1) Adenocarcinoma, 2)

squamous cell carcinoma 3) large cell carcinoma, and some other uncommon subtypes . For reporting, histologic type helps determine NSCLC vs SCLC and the subtype of NSCLC. Although NSCLC gener- ally has favourable survival rates and prognosis as compared to SCLC, certain subtypes of NSCLC can have lower survival rates due to co-factors . Histologic patterns are applicable in adenocarcinomas, consisting of favourable types: lepidic, in- termediate: acinar and papillary, and unfavourable: micropap- illary and solid . Grading each histologic type aids in cat- egorization but is differentiated based on each type, and thus is out of scope for this paper. Importantly for lung cancers, tu- mour size is an independent prognostic factor for early cancer stages, lymph node positivity, and locally invasive disease. Ad- ditionally, the size of the invasive portion is an important fac- tor for prognosis of nonmucinous adenocarcinoma with lepidic pattern [125, 126, 127, 128, 129, 123]. Other important lung specific features are visceral pleural invasion, which is associ- ated with worse prognosis in early-stage lung cancer even with tumors < 3cm , and lymphatic invasion which indicates an unfavourable prognostic finding [125, 131].

Colon and Rectum Colorectal cancers are two of the five most common cancer types . Cancer cells usually start to develop in the innermost layer of the colon and rectum walls, known as the mucosa, and continue their way up to other layers.

In other layers, there are lymph and blood vessels that can be used by cancer cells to travel to nearby lymph nodes or other organs . Colorectal cancers usually start with the creation of different types of polyps, each possessing a unique risk of developing into cancer. Most colorectal cancers are adenocarci- nomas, which are split into three well-studied subtypes: classic adenocarcinoma (AC), signet ring cell carcinoma (SRCC), and mucinous adenocarcinoma (MAC). In most cases, AC has a bet- ter prognosis than MAC or SRCC. Other types, albeit uncom- mon, of colorectal cancers are: carcinoid tumors, gastrointesti- nal stromal tumors (GISTs), lymphomas, and sarcomas .

As in other cancers, histologic grade is the most important factor in cancer prognosis along with regional lymph node sta- tus and metastasis. The tumor site is also important in deter- mining survival rates and prognosis . Vascular invasion of both small and large vessels are important factors in adverse outcomes and metastasis [135, 136, 137], and perineural inva- sion has been shown in multiple studies to be an indicator of poor prognosis [138, 139, 137]. Additionally, microsatellite in- stability (MSI) is shown to be a good indicator of prognosis and is divided into three stages in decreasing adversity of Stable (MSI-S), Low (MSI-L), and High (MSI-H) . Finally, some studies have indicated the usefulness of biomarkers in colorec- tal cancer treatment, with biomarkers such as BRAF mutations, KRAS mutations, MSI, APC, Micro-RNA, and PIK3CA .

Works are relatively well-distributed among various tasks in- cluding disease diagnosis, segmentation, and detection. Ex- panding on colorectal cancer detection, work from used feature analysis for colorectal and mucinous adenocarcinomas using heatmap visualizations. They discovered that adenocar- cinoma is often detected by ill-shaped epithelial cells and that misclassification can occur due to lumen regions that resemble the malformed epithelial cells. Similarly for mucinous carci- noma, the model again recognizes the dense epithelium, but this time ignores the primary characteristic of the carcinoma (abundance of extracellular mucin). These findings suggest that a thorough analysis of class activation maps can be helpful for improving the classifier’s accuracy and intuitiveness.

Bladder There are several layers within the bladder wall with most cancers starting in the internal layer, called the urothelium or transitional epithelium. Cancers remaining in the inner layer are non-invasive or carcinoma in situ (CIS) or stage 0. If they grow into other layers such as the muscle or fatty layer, the cancer is now invasive. Nearly all bladder cancers are urothe- lial carcinomas or transitional cell carcinomas (TCC). However, there are other types of cancer such as squamous cell carci- nomas, adenocarcinomas, small cell carcinomas, and sarcomas which all are very rare. In the early stages, all types of bladder cancers are treated similarly but as their stage progresses, and chemotherapy is needed, different drugs might be used based on the type of the cancer . As with other organs, histo- logic type and grade also play a role in prognosis and treatment , and lymphovascular invasion is independently associated with poor prognosis and recurrence .

Works focusing on the bladder display promising results that could lead to rapid clinical application. For example, a predic- tion method for four molecular subtypes (basal, luminal, lumi- nal p53, and double negative) of muscle-invasive bladder cancer was proposed in , outperforming pathologists by 30% in classification accuracy when restricted to a tissue morphology-

9

based assessment.

Further Improvements In Accuracy Could

help expedite diagnosis by complementing traditional molec- ular testing methods. Kidney Each kidney is made up of thousands of glomeruli which feed into the renal tubules. Kidney cancer can occur in the cells that line the tubules (renal cell carcinoma (RCC)), blood vessels and connective tissue (sarcomas), or urothelial cells (Urothelial carcinoma). RCC accounts for about 90% of kidney cancers and comes in two types: 1) clear cell renal car- cinoma, which are most common and 2) non-clear cell renal carcinoma consisting of papillary, chromophobe and some very rare subtypes . The CAP’s cancer protocol template for the kidney is solely focused on RCCs , likely due to their high probability. Tumour size is directly associated with malig- nancy rates, with 1cm size increases resulting in 16% increases in malignancy . Additionally, the RCC histologic type is correlated with metastasis, with clear cell, capillary, collecting ducts (Bellini), and medullary being the most aggressive .

Many works are focused on glomeruli segmentation, as the number of glomeruli and glomerulosclerosis constitute standard components of a renal pathology report . In addition to glomeruli detection, some works have also detected other rele- vant features such as tubules, Bowman’s capsules, and arteries . The results display strong performance on PAS-stained nephrectomy samples and tissue transplant biopsies, and there seems to be a strong correlation between the visual elements identified by the network and those identified by renal patholo- gists.

Brain There are two main types of brain tumors: malig- nant and non-malignant. Malignant tumors can be classified as primary tumors (originating from the brain) or secondary (metastatic) [153, 154]. The most common type of brain can- cers are gliomas, occurring 50.4% of the time, and are classified into four grades . In the synoptic reporting, tumour loca- tion is noted as it has some impact on the prognosis, with pari- etal tumours showing better prognosis compared to other loca- tions . Additionally, focality of glioblastomas (a subtype of gliomas) is important to determine as multifocal glioblas- toma is far more aggressive and resistant to chemotherapy as compared to unifocal . A recent summary of the World Health Organization’s (WHO) classification of tumors of the central nervous system has indicated that biomarkers as both ancillary and diagnostic predictive tools . Additionally, in a recent WHO edition of classification of tumours of the central nervous system, molecular information is now integrated with histologic information into tumor diagnosis for cases such as diffuse gliomas and embryonal tumors .

Accordingly, most works focus on gliomas and more specif- ically glioblastoma, the most aggressive and invasive form of glioma. Due to glioblastoma’s extremely low survival rate of 5% after 5 years, compared to a low grade glioma’s survival rate of over 50% after 5 years [158, 76], it is critical to distin- guish the two forms for improved patient care and prognosis.

Liver Liver cancer is one of the most common causes of can- cer death . In particular, hepatocellular carcinoma (HCC) is the most common type of primary liver cancer and has var- ious subtypes, but they generally have little impact on treat- ment . Histogolical grade is divided into nuclear features and differentiation, which directly correlate to tumour size, pre- sentation, and metastatic rate [161, 162]. Notably, high-grade dysplastic nodules are included in synoptic reports for HCC but are difficult to assess and have high inter-observer disagreement , and thus is an area where CAD systems could be lever- aged to normalize assessments. Current grading of this cancer suffers from an unsatisfactory level of standardization , likely due to the diversity and complexity of the tissue. This could explain why relatively low number of works are dedi- cated to liver disease diagnosis and prognosis. Instead, most works focus on the segmentation of cancerous tissues.

Lymph Nodes There are hundreds of lymph nodes in the hu- man body that contain immune cells capable of fighting infec- tions. Cancer manifests in lymph nodes in two ways: 1) cancer that originates in the lymph node itself known as lymphoma and 2) cancer cells from different origins that invade lymph nodes . As mentioned in the prior organ sections, lymphocytic infiltration is correlated with cancer recurrence on multiple or- gans and lymph nodes are the most common site for metastasis.

The generalizable impact to multiple organs and importance of detecting lymphocytic infiltration is why many works focused on lymph nodes address metastasis detection .

Organ Agnostic The remaining papers focus on segmenta- tion, diagnosis, and prognosis tasks that attempt to generalize to multiple organs, or target organ agnostic applications. An interesting approach to increase the generalization capability of deep learning in histopathology is proposed in . Cur- rently, publicly available datasets with thorough histological tis- sue type annotations are organ specific or disease specific and thus constrain the generalizability of CPath research. To fill this gap, a novel dataset called Atlas of Digital Pathology (ADP) is proposed . This dataset contains multi-label patch-level annotations of Histological Tissue Types (HTTs) arranged in a hierarchical taxonomy. Through supervised training on ADP, high performance on multiple tasks is achieved even on unseen tissue types.

3. Data Collection For Cpath

One of the first steps in the workflow for any CPath research is the collection of a representative dataset. This procedure of- ten requires large volumes of data that should be annotated with ground-truth labels for further analysis [62, 168, 64]. How- ever, creating a meaningful dataset with corresponding anno- tations is a significant challenge faced in the CPath community [62, 168, 169, 64, 170].

This section outlines the entire process of the data-centric de- sign approach in CPath, including tissue slide preparation and WSI scanning–the first two stages in the proposed workflow shown in Figure 1. Additionally, the trend in dataset compi- lation across the 700 papers surveyed is discussed regarding dataset sizes, public availability, and annotation types; see Table 9.11 in the Supplementary Material for information regarding the derivations and investigation of said trends.

3.1. Tissue Slide Preparation

For the application development stages in CPath, the creation of a new WSI dataset must begin with selection of relevant glass slides. High quality WSIs are required for effective analysis, however, considerations must be made for potential slide arti- facts and variations inherently present. As described in Section 2.1, pathology samples are categorized as either biopsy or re- section samples, with most samples being prepared as perma- nent samples and some intra-operative resection samples being prepared as frozen samples.

Variations and Irregularities Throughout the slide section- ing process, artifacts and irregularities can occur which reduce the slide quality, including: uncovered portions, air bubbles in between the glass seal, tissue chatter artifacts, tissue folding and tears, ink markings present on the slide, and dirt, debris, microorganisms, or cross-contamination of slides by unrelated tissue from other organs[171, 172, 173]. Frozen sections can present unique irregularities and variations, such as freezing ar- tifacts, cracks in the tissue specimen block, or delay of fixa- tion causing drying artifacts[174, 175]. Beyond these irregular- ities, glass slides may vary in stain colouring, occurring due to differences in slide thickness, tissue thickness, fixation, tissue processing schedule, patient variation, stain variation, and lab variation[176, 174, 177, 178, 179, 180].

All such defects and variations are important to keep in mind when selecting glass slides for the development and applica- tion process in CPath, as they can both reduce the quality of the WSI as well as impact the performance of developed CAD tools trained with these WSIs [171, 172, 177]. A more detailed dis- cussion on the surveyed works in CPath which seek to identify and correct issues in slide artifacts and colour variation in WSIs is found in Section 3.2. However, prior to digitization, artifacts, and irregularities can be kept at a minimum by following good pathology practices. While an in-depth discussion of this topic is outside the scope of this paper, some research provides an extensive list of recommendations for reducing such errors in slide sectioning .

3.2. Whole Slide Imaging (Wsi)

WSI Scan Once a glass slide is prepared, it must be digitized into a WSI. The digitization and processing workflow for WSIs can be summarized as a four-step process : (1) Image ac- quisition via scanning; (2) Image storage; (3) Image editing and annotation; (4) Image display . As the first two steps of the digitization workflow are the most relevant for WSI collection and with regards to the CPath workflow, they are discussed to a greater extent below.

Slide scanning is carried out through a dedicated slide scan- ner device. A plethora of such devices currently exist or are in development; see Appendix Table 1 for a collection of commer- cially available WSI scanners. Additionally, some research has investigated and compared the capabilities and performances of various WSI scanners [183, 184, 185, 186].

In order to produce a WSI that is in focus, which is especially important for CPath works, appropriate focal points must be chosen across the slide either using a depth map or by selecting arbitrarily spaced tiles in a subset . Once focal points are chosen, the image is scanned by capturing tiles or linear scans of the image, which are stitched together to form the full im- age [180, 187]. Slides can be scanned at various magnification levels depending on the downstream task and analysis required, with the vast majority being scanned at 20× (∼0.5µm/pixel) or 40× (∼0.25µm/pixel) magnification .

WSI Storage and Standards WSIs are in giga-pixel dimen- sion format [30, 188]. For instance a tissue in 1cm × 1cm size (uncompressed) with 50, 000 × 50, 000 pixels. Due to this large size, hardware constraints may not support viewing entire WSIs at full resolution, thus WSIs are most often stored in a tiled for- mat so only the viewed portion of the image (tile) is loaded into memory and rendered . When building CAD tools for CPath, this large WSI dimensionality must be taken into ac- count in determining how much compute is required to analyze a WSI. Alongside the WSI, metadata regarding patient, tissue specimen, scanner, and WSI information is stored for reference [30, 188, 190]. Due to their clinical use, it is important to de- velop effective storage solutions for WSI images and metadata, allowing for robust data management, querying of WSIs, and efficient data retrieval [191, 192]. Further details on WSI image formats and storage methods are discussed in Appendix 9.6 To develop CPath CAD tools in a widespread and general manner, a standardized format for WSIs and their correspond- ing metadata is essential . However, there is a general lack of standardization for WSI formats outputted by various scan- ners, as shown in Table 1, especially regarding metadata stor- age. The Digital Imaging and Communications in Medicine (DICOM) standard provides a format for CPath image format- ting and data management through Supplement 145[190, 193], and has been shown in research to allow for efficient access and interoperability of data between varying medical centers and devices . However, few scanners are DICOM-compliant and thus there are challenges to using different models of scan- ners, thus different image formats and metadata structures, in the context of dataset aggregation and processing.

Apart from storage format, a general framework for storing and distributing WSIs is also an important pillar for CPath. In other medical imaging fields such as radiology, images are of- ten stored in a picture archiving and communications systems (PACS) in a standardized DICOM format, with DICOM storage and retrieval protocols to interface with other systems .

The need for standardization persists in pathology for WSI stor- age solutions; few works have proposed solutions to incorpo- rate DICOM-based WSIs in a PACS, although some research has successfully implemented a WSI PACS consistent using the DICOM standard using a web-based service for viewing and image querying .

WSI Defects and Variations Certain aspects of the slide scanning process can introduce unfavorable irregularities and variations . A major source of defects is out-of-focus re- gions in a generated WSI; often caused by glass slide artifacts, such as air bubbles and tissue folds, which interfere with selec- tion of focus points for a slide [171, 195]. Out-of-focus regions degrade WSI quality and are detrimental to the performance of CAD tools developed with these WSIs, presenting concerns

11

for performance with studies showing high false-positive er- rors [196, 172]. Additionally, as WSIs are scanned in strips or tiles, any misalignment between sections can introduce strip- ing/stitching errors in the final image . Another source of error may appear during tissue-background segmentation where the scanner may misidentify some tissue regions as background, potentially missing crucial tissue areas on the glass slide from being digitized .

Variations in staining refers to differences in colour and con- trast of the tissue structures in the final WSI occurring due to differences in the staining process, staining chemicals, and tis- sue state. Variations in colour can lead to difficulty in general- izing CAD tools to WSIs from different labs, institutions, and settings [199, 200]. Even identical staining techniques can yield different WSIs due to scanner differences in sensor design, light source and calibration [180, 201], creating challenges for cross- laboratory dataset generation. These additional sources of vari- ation add layers of complexity to the WSI processing workflow, and must be kept in mind during slide selection and dataset cu- ration for CAD tool development and deployment.

Addressing Irregularities and Variations Much work has gone into identifying areas of irregularities within WSIs, most notably blur and tissue fold detection [195, 196]. Some re- search has explored automated deep learning tools to iden- tify these irregularities at a more efficient pace than manual inspection [195, 196].

Developing Techniques For Address-

ing staining variation has also been a significant research area [202, 203, 204, 177, 205, 206, 207] as the use of techniques addressing stain variation is important for all future works. We list some computational approaches proposed to address these issues: An example method proposed in uses a stain nor- malization technique, attempting to map the original WSI onto a target color profile. In this technique, a color deconvolution matrix is estimated to convert each image to a target hema- toxylin and eosin (H&E) color space and each image is normal- ized to a target image colour profile through spline interpolation . A second approach applies color normalization using the H channel with a threshold on the H channel on a Lym- phocyte Detection dataset.

Recent Studies Have Shown

promise in having deep neural networks accomplish the stain normalization in contrast to the previous classical approaches [208, 209, 203, 210], commonly applying generative models such as generative adversarial networks (GANs) to stain nor- malization. Furthermore, Histogram Equalization (HE) tech- nique for contrast enhancement is used in , where novel preprocessing technique is proposed to select and enhance a portion of the images instead of the whole dataset, resulting in improved performance and computational efficiency.

An alternative approach to address the impact of stain varia- tion on training CAD tools is data augmentation. Such methods augment the data with duplicates of the original data, contain- ing adjustments to the color channels of the image, creating im- ages of varying stain coloration, and training train models that are accustomed to stain variations. This method has been frequently used as a pre-processing step in the development of training datasets for deep learning [212, 213, 214]. A form of medically-irrelevant data augmentation based on random style

Verhoeff

Fig. 5: WSI tissue images with different types of histological stains. Each stain highlights different areas and structures of the tissue in order to aid in visual- izing underlying characteristics. Amongst this diversity, there is Hematoxylin and Eosin or H&E which is mainly used in studies as most histopathological processes can be understood from this stain. All images provided are under a Creative Commons license, specifics on the license can be found in the refer- ences.

transfer, called STRAP, was proposed by researchers and out- performed stain normalization . Similar to style transfer, proposes stain transfer which allows one to virtually gen- erate multiple types of staining from a single stained WSI.

3.3. Cohort Selection, Scale, And Challenges

The data used to create/train CPath CAD tools can greatly impact the performance and success of the tool. Curating the ideal dataset, and thus selecting the ideal set of WSIs for the development of a CAD tool is a nontrivial task. Several works suggest that datasets for deep learning in CPath should include a large quantity of data with a degree of variation and artifacts in the WSIs [62, 172]. Some works also recommend the inclu- sion of difficult or rarely diagnosed cases; other works indicate that inclusion of extremely difficult cases may decrease the per- formance of advanced models [172, 223].

A study highlighting the results of the 2016 Dataset Ses- sion at the first annual Conference on Machine Intelligence in Medical Imaging outlines several key attributes to create an ideal medical imaging dataset , including: having a large amount of data to achieve high performance on the desired task, quality ground truth annotations, and being reusable for fur- ther efforts in the field. While the scope of this conference did not include CPath, many of the points made regarding medical imaging datasets are also relevant to the development of CPath datasets. The session also outlined the impact that class imbal- ances can have on ML models, an issue also prevalent in CPath as healthy or benign regions often outnumber diseased regions by a significant margin .

Our survey of past works in the literature reveals some trends in CPath datasets.

Currently, The Majority Of Datasets Pre-

sented in the literature for CAD tool development are small- scale datasets , using a small number of images, and/or im- ages from a small number of pathology laboratories. Exam- ples of these smaller datasets include a dataset with 596 WSIs

12

(401 training, 195 testing) from four centres for breast cancer detection and the Breast Cancer Histology (BACH2018) dataset, which has 500 ROI images (400 training, 100 testing) and 40 WSIs (30 training, 10 testing) . Although curat- ing a dataset from fewer pathology laboratories may be sim- pler, these smaller scale datasets may not be able to effectively generalize to data from other pathology centres [199, 120]. An example of this can be seen in which data from different pathol- ogy centres are clustered disjointly in a t-distributed stochas- tic neighbor embedding (t-SNE) representation demonstrated in . Another alternative was proposed in : using a swarm learning technique multiple AI models were trained on different small data sets separately and then unified into one central model.

Additionally, stain variations, slide artifacts, and variation of disease prevalence may sufficiently shift the feature space such that a deep learning model may not sustain high perfor- mance on unseen data in new settings [120, 229]. As artifacts in WSIs are inevitable, with some artifacts, such as ink mark-up on glass slides, being an important part of the pathology work- flow , the ability of CAD tools to become robust to these artifacts through exposure to a diverse set of images is an im- portant consideration.

Compared to the number of studies conducted on small-scale datasets, relatively fewer studies have been performed using large-scale, multi-centre datasets [62, 231, 172].

One Study

uses over 44,715 WSIs from three organ types, with very lit- tle curation of the WSIs for multi-instance learning detailed in . Stomach and colon epithelial tumors were classified using 8,164 WSIs in . A similar study uses 13,537 WSIs from three laboratories to test a machine learning model trained on 5,070 WSIs and achieves high performance .

Despite some advancements, there exist major barriers to us- ing such large, multi-centre datasets in CAD development. No- tably, for strongly supervised methods of learning, an immense amount of time is needed to acquire granular ground truth anno- tations on a large amount of data . To combat this, some researchers have implemented weakly-supervised learning by harvesting existing slide level annotations to forego the need for further annotation . Additionally, it may be difficult to ag- gregate data from multiple pathology centres due to regulatory, privacy, and attribution concerns, despite the improvements that diverse datasets offer. section 5 discusses model architectures and training techniques that harness curated datasets of various annotation levels.

Dataset Availability In general computer vision, progress can be tracked by the increasing size and availability of datasets used to train models, e.g. ImageNet grew from 3.2 million im- ages and 5000 classes in 2009 to 14 million images and 21,000 classes in 2021 . We infer a similar trend in dataset growth and availability indicates progress in CPath. In our survey of over 700 CPath papers, we determine the current landscape by noting the dataset(s) used in work, along with dataset details such as the organ(s) of interest, annotation level, and stain type, tabulating the results into Table 9.11 of the supplementary ma- terials, with summarized findings from Table 9.11 are shown in Figure 6.

From Figure 6 we can clearly see that the majority of datasets used for research developments in computational pathology are privately sourced or require additional registration/request.

With organs represented in a small number of datasets, such as the liver, thyroid, brain, etc, having a smaller proportion of freely accessible datasets as compared to the Breast, Colon, or Porstate. This can be problematic when trying to create CAD tools for cancers in these organs due to a lack of accessible data.

We additionally note that although data sets requiring registra- tion/request for access can be easily accessible, as in the case of Breast Cancer Histopathological Database (BreakHis) being used in multiple works [234, 235, 236], the need for reg- istration presents a barrier to access as requests may go unan- swered or take much time to review.

In our categorization of CPath datasets, we find that a few prominent datasets have been released publicly for use by the research community. Many such datasets are made available through grand challenges in computational pathology ,

Such As The Camelyon16 And Camelyon17 Challenges

for breast lymph node metastases detection [238, 61, 239], and the Gland Segmentation in Colon Histology Images Challenge (GLaS) competition for colon gland segmentation in conjunc- tion with Medical Image Computing and Computer Assisted Intervention (MICCAI) 2015 [240, 241].

Notable Amongst

publicly available data repositories is the cancer genome atlas (TCGA) , a very large-scale repository of WSI-data con- taining many organs and diseases, along with covering a variety of stain types, magnification levels, and scanners. Data col- lected from TCGA has been used in a large number of works in the literature for the development of CAD tools [200, 243, 244].

As such, TCGA represents an essential repository for the devel- opment of computational pathology. While patient confidential- ity is a general concern when compiling and releasing a CPath dataset, large-scale databases such as TCGA prove that it is possible to provide relatively unrestricted data access without compromising patient confidentiality. Further evaluating pub- lic source datasets, it seems that the majority of them use data extracted from large data repositories, such as TCGA, without specifying the IDs of the images used, which provides a chal- lenge in comparing datasets or CAD tool performance across works. However, there are a few datasets that are exceptions to that phenomenon [245, 246, 247, 64].

Figure 6 also provides some insights on the dataset break- down by organ, stain type, and annotation level. Per organ, it can be seen that the breast, colon, prostate/ovary, and lung tissue datasets are amongst the most common, understandably since cancer occurrence in these regions is the most frequent –complying with cancer statistics findings in 9.5. Multi- organ datasets are the other most common type, where we have designated a dataset to be multi-organ if it compiles WSIs from several different organs. To note, multi-organ datasets are espe- cially useful for the development of generalized image analy- sis tools in computational pathology. The annotation level pro- vided in the datasets did not indicate any pattern across most organs.

Dataset Bias It is also important to note the potential for bias in datasets that may influence the ability of any deep learning

13

Fig. 6: (left) shows the distribution of datasets per organ as a capture of the current trend in datasets, although the number of datasets can change over time an understanding of what organs have more available data is important for developing CAD tools. Along the vertical axis, we list different organs, while the horizontal axis shows the number of datasets; wherein the darker color denotes public availability while the light color includes unavailable or by request statuses. (right) Distribution of staining types, annotation levels, and magnification details per organ color coded consistently with the bar graph. Organs have been sorted based on the abundance of datasets. For more details, please refer to Table 9.11 in the supplementary section.

algorithm to generalize on unseen data [248, 249]. This prob- lem is a prevalent issue in general machine learning applications [250, 251, 252, 253], and CPath is not immune to it. The sur- vey review in reviews a large number of other examples in machine learning that exhibit such bias, both from a dataset- standpoint and an algorithm-standpoint.

Such a lack of generalizability in CPath can impact the ability of machine learning models trained on biased data to meet the needs of patients. As noted in , minority groups may be disproportionately negatively impacted if care is not taken in curating a diverse dataset that adequately reflects the relevant demographics for the problem to be solved.

Several works have delved into the issue of dataset bias in CPath specifically [255, 256]. A notable example is in , where the study was able to demonstrate that deep learning models trained on WSIs from TCGA were able to infer the organization that contributed the slide sample. Notably, some features, such as genetic ancestry, patient prognosis, and sev- eral key genomic markers were significantly correlated with the site the WSI was provided from. As the vast majority of data in TCGA is acquired from 24 origin centers , such site- specific factors may impact the ability of a DL model to per- form well on patient data from different sites.

As discussed previously, having a large set of diverse data may help to mitigate generalization issues [199, 120, 248]. Ad- ditionally, the study makes the suggestion that training data should be from separate sites than validation data, and that per-site performance of a model should be reported when val- idating a model. In doing this, the robustness of the model to site-specific variation, including both stain and demographic re- lated variation, can be evaluated.

4. Domain Expert Knowledge Annotation

A primary goal of CPath is to capture and distill domain ex- pert knowledge, in this case the expertise of pathologists, into increasingly efficient and accurate CAD tools to aid pathol- ogists everywhere.

Much Of The Domain Knowledge Trans-

fer is encompassed within the process of human experts, in this case pathologists, generating diagnostically-relevant anno- tations and labels for WSIs. It must be emphasized, that without some level of label, a WSI dataset is not directly usable to train a model for most CAD tasks that involve the generation of di- agnoses, prognoses, or suggestions for pathologists. Thus, the process of obtaining and/or using annotations at the appropriate granularity and quality is paramount in the field. This section focuses on describing various types of ground-truth annotation to cover the spectrum of weak to strong supervision of labels, discussing the practicality of labeling across this supervision spectrum, and how a labeling workflow can be potentially de- signed to optimize related annotation tasks.

4.1. Supervised Annotation

In contrast to general computer vision, computer scientists do not have expert-level knowledge of histopathology and thus they are not as efficient at generating annotations or labels of pathology images. Further, labels cannot be easily obtained by outsourcing the task to the general public. As a result, pathol- ogists must be leveraged to obtain labels at some stage of the data collection and curation process, and in many annotation pipelines the first step involves recruiting the help of patholo- gists for their expertise in labelling.

Obtaining Expert Domain Knowledge The knowledge of pathologists is essential in the development of accurate ground

14

truth annotations–a process most commonly completed by en- circling ROI . However, there are studied instances of inter-observer variance between pathologists when determin- ing a diagnosis [257, 258, 259]. As obtaining the most cor- rect label is essential when training a model for CAD, this issue must be addressed and a review of the data by several pathol- ogists can result in higher quality ground truth data as com- pared to that of a single pathologist. As a result, most datasets are curated by involving a group of pathologists in the anno- tation process. If there exists a disagreement between the ex- pert pathologists on the annotation of a ground truth, one of several methods is usually employed to rectify the discrepancy.

A consensus can be reached on the annotation label through discussion amongst pathologists, as is done in the triple nega- tive breast cancer (TNBC-CI) dataset , the Breast Cancer Surveillance Consortium (BCSC) dataset and the mini- malist histopathology image analysis dataset (MHIST) dataset . Alternatively, images, where disagreements occur, can be discarded, as is done in some works [263, 264]. Further, the disagreement between annotators can be recorded to deter- mine the difficulty level of the images, as is done in the MHIST dataset . This extra metadata aids in the development of CAD tools for analysis.

Pathologists can also be involved indirectly in dataset an- notation. Both the Multi-organ nuclear segmentation dataset

(Monuseg) And Adp Have Non-Expert Labelers An-

notate their respective datasets. A board-certified pathologist is then tasked with reviewing the annotations for correctness. Alternatively, some researchers have employed a pathologist in performing quality control on WSIs for curating a high-quality dataset with minimal artifacts [87, 266]. To enable the large scale collection of accurate annotated data, Lizard was developed using a multi-stage pipeline with several significant “pathologist-in-the-loop” refinement steps.

Existing pathological reports, along with the metadata that comes from public large-scale databases like TCGA, can also be leveraged as additional sources of task-dependent annota- tions without the use of further annotation. For example, TCGA metadata was used to identify desirable slides in , while pathological diagnostic reports were used for breast ductal car- cinoma in-situ grading in .

To note, there are some tasks where manual annotation by pathologists can be bypassed altogether.

For Instance, Ihc

was applied to generate mitosis figure labels using a Phospho- Histone H3 (PHH3) slide-restaining approach in , while immunofluorescence staining was used as annotations to iden- tify nuclei associated with pancreatic adenocarcinoma .

These works parallel the techniques that pathologists often use in clinical practice, such as the use of IHC staining as a sup- plement to H&E stained slides for difficult to diagnose cases . They demonstrate high performance on their respective tasks wherein the top-performing models on the Tumor Prolif- eration Assessment Challenge 2016 (TUPAC16) dataset were achieved . Importantly, these techniques still utilize supervision, albeit weakly, by leveraging lab techniques that have been developed and refined to identify the desired regions visually.

Ground-Truth Diagnostic Information Understanding dif- ferent annotation levels and their impact on the procedural de- velopment of ML pipelines is an important step in solving tasks within CPath. There are five possible levels of annotation, in or- der of increasing granularity (from weakly-supervised to fully- supervised): patient, slide, ROI, patch, and pixel.

Figure 7

overviews the benefits and limitations of each level. For addi- tional information regarding each annotation level please refer to Appendix 9.8.

Picking the Annotation Level Selecting an annotation level depends largely on the specific CPath task being addressed, as shown in Figure 8. For example, segmentation tasks tend to fa- vor pixel-level annotations as they require precise delineation of a nucleus or tissue ROI. Conversely, disease diagnosis tends to favor datasets with ROI-level annotations, as diagnosis tasks are predominantly associated with the classification of diseased tis- sue, the higher-level annotations may provide a sufficient level of detail and context for this task .

Figure 8 shows that tasks that use stronger supervision are more likely to be used in CAD tool model development. How- ever, due to the high cost of pixel-level annotation, fully super- vised annotations are challenging to develop. Even patch-based annotations often require the division and analysis of a WSI into many small individual sub-images resulting in a similar problem to pixel-based annotations [63, 212]. In contrast, WSI data is most often available with an accompanying slide-level pathology report regarding diagnosis thus making such weakly labeled information at the WSI level significantly more abun- dant than ROI, patches, or pixel-level data [274, 275]. Different levels of annotation can be leveraged together, as demonstrated by a framework to use both pixel and slide level annotations to generate pseudo labels in . Additionally, it is common in CPath to further annotate the slide-level WSIs on an ROI or patch level structure [85, 277, 278, 32].

Active Learning Tools Active learning annotation tools bridge the gap between the need for highly supervised labels and the current abundance of less informative annotations. Such works seek to ease the annotation process by using compu- tational approaches to assist the human annotator. For exam- ple, in , a platform was developed for creating nuclei and gland segmentation ground truth labels quickly and efficiently.

A convolutional neural network (CNN), trained on similar co- hort data, was used to segment nuclei and glands with different mouse actions . Alternatively, Awan et al. presented the HistoMapr™platform to assist in diagnosis and ground truth data collection. Through this tool, a pathologist selects one of several proposed classes for each given ROI, thus mitigating the need for hand-drawn annotations or manual textual input . Similarly, an active learning model called the Human- Augmenting Labeling System (HALS) was developed to increase data efficiency by guiding annotators to more informa- tive slide sections. Quick Annotator (QA) is another tool which provides an easy-to-use online interface for annotating ROIs and was designed to create a significant improvement in the annotation efficiency of histological structures by several orders of magnitude.

There are other active learning annotation tools proposed for

15

Fig. 7: Details of the five different types of annotations in computational pathology. From left to right: a) Patient-level annotations: can include high level information about the patient like status of cancer, test results, etc. b) Slide-level: are annotations associated with the whole slide, like a slide of normal tissue or a diseased one c) ROI-level annotations: are more focused on diagnosis and structure details d) Patch-level: are separated into Large FOV (field of view) and small FOV, each having different computational requirements for processing, and finally e) Pixel-level: includes information about color, texture and brightness Fig. 8: A snapshot of the distribution of different annotation levels based on the CPath task being addressed in the surveyed literature for the purposes of highlighting the trend of datasets. The x-axis displays the different annotation levels studied in the papers (from left to right): Patient, Slide, ROI, Patch, and Pixel. The y-axis shows the different tasks (top to bottom): Detection, Diagno- sis, Classification, Segmentation, and Prognosis. The height of the bars along the vertical axis measures the number of works that have examined the specific task and annotation level.

different applications in computer vision that can be investi- gated for use in the pathology datasets.

Such Examples In-

clude methods to produce object segmentation masks for still images [281, 282] as well as video . One notable exam- ple is DatasetGAN ; the model is proposed as a training data creator, and it is shown that the model can produce seg- mentation masks with a small number of labelled images in the training data. While these systems are for general computer vision, they may be adoptable in computational pathology, and would facilitate the necessary relationship between pathologists and computer scientists in the development of CAD tools. As such, they may prove to be a valuable contributor to the CAD system development workflow.

Tissue-Class and Disease Complexity Much of the current CPath research operates under the umbrella of supervised learn- ing tasks, and correspondingly uses labeled data to develop automated CAD tools. We refer to supervised learning to in- clude a diverse spectrum of annotation i.e. weak-supervision (e.g. patient-level) all the way to strong-supervision (e.g. pixel- level).

Classes within a dataset can be task-dependent, for example as shown in Table 9.11 of the supplementary mate- rial, datasets primarily used for segmentation such as MoNuSeg

And Cpm-17 Have Classes For Each Annotated Pixel

indicating the presence or absence of nuclei. However, classes need not be task-dependent; datasets such as CAMELYON16 outline metastases present in WSIs that can be used for a variety of applications, including disease detection and segmentation tasks .

The current paradigm for dataset compilation in computa- tional pathology, particularly for disease detection and diag- nosis, treats different disease tissue types as separate inde- pendent classes. For example, BreakHis divides all data into benign/malignant breast tumours .

At The Roi Level,

GLaS divides colon tissue into five classes: healthy, adenoma- tous, moderately differentiated, moderately-to-poorly differen- tiated, and poorly differentiated . So far, this approach to class categorization has resulted in high-performing CAD tools [80, 235, 236, 286, 83, 84]. However, the treatment of different disease tissue types as an independent class is perceived differ- ently in computer vision domain where the representation learn- ing of normal objects is done differently compared to anoma- lies. A similar synergy can be found by differentiating healthy tissue classes from diseased ones and one should be mindful about defining meaningful tissue ontology for annotation and labeling.

4.2. Optimum Labeling Workflow Design

This section focuses on the steps required for compilation of a CPath dataset which is broken into three main sub-tasks: Data Acquisition, Data Annotation, and Data Management, as per Figure 9. Each sub-task is discussed below with reference to its individual components in the hierarchical structure in Figure 9.

16

Data Acquisition Database compilation starts from data ac- quisition. When collecting data, it is vital that there are large amounts of data , along with having sufficient diversity [62, 172]. Specifically, diversity in CPath data occurs in multi- ple ways, such as staining methods, tissue types and regions, laboratory processes, and digital scanners.

We Advise That

CPath researchers consult expert pathologists on the diversity of data required for various tasks. Ideally, all the data acquired in pathology should be perfect without any irregularity and ar- tifacts. However, some level of artifacts and irregularity are unavoidable and introducing realistic artifacts that are represen- tative of real-world scenarios into the data increases the robust- ness and generalizability of CAD tools.

Data Annotation After collecting sufficient data, the next task is annotation of the data. Data annotation is a costly pro- cess in both time and money, thus a budget and schedule should always be established when generating labels. There are often various approaches for annotating different structures , so a specific labelling taxonomy should be defined a priori. As mentioned previously, annotation should involve expert pathol- ogists due to the domain knowledge requirement and impor- tance of label correctness. A table of commonly used com- mercially available annotation software for annotating different slide formats are show in Table 2, along with their compatible image formats which is important to note when trying to build compatible and accessible datasets.

Once the ontology of class-definitions is defined (in collabo- ration with expert pathologists), there will be two ways to gen- erate labels or annotations in general: domain expert labelling or non-expert labelling. The domain expert labelling refers to having pathologists annotate data that they are specialized at, which is labor-extensive. On the other hand, non-expert la- belling can use crowdsourcing techniques to generate weak la- bels or have non-experts, such as junior pathologists or students, label the data. This process is cheaper and quicker, but it may be harder to maintain the same level of quality as domain ex- pert labeling . Regardless of the labelling methodology used, labels generated from both should be validated. Finally, to determine the sufficiency of label quantity, one should con- sider the balance between the number of classes, representation size of each class, and complexity of class representation. Tech- niques from active-learning could be also leveraged to compen- sate for lack of resource management as well as maintain the quality of labeling as discussed above.

Data Management Data management is an important aspect of any dataset creation process, and is the one that is most likely to be overlooked. Proper data management should have con- siderations for reusability, medical regulations/procedures, and continuous integration and development.

Reusability can be broken down into detailed documentation of the data, accessible and robust hosting of data, and consid- eration for image standards. Poor cross-organizational docu- mentation can lead to missing metadata, ultimately resulting in discarding entire datasets .

Adherence To An Estab-

lished image standard, such as DICOM, can help resolve some of these issues in reusability. Medical regulations/procedures can be broken down into the construction of a Research Ethics Fig. 9: Tree diagram for the optimum labeling workflow, where a CPath dataset is divided into tasks and sub-tasks based on its initial characteristics.

Board (REB) and proper consideration for whom is curating the data. Through incentives for data excellence for medical practitioners, the issue of misaligned priorities between data scientists, domain experts, and field partners can be resolved . To ensure that models used on actual patients remain rel- evant and hidden errors do not propagate, continuous integra- tion/development (CI/CD) must be implemented. These sys- tems must include at least two components, a method to audit predictions from the model, as well as a way to refine the train- ing data accounting for discrepancies found through auditing.

Several algorithms deployed in high-risk environments, includ- ing medical diagnosis, proved to only work well when data was updated after initial deployment [290, 289]. Throughout the data management process, consultation with domain experts is a vital step in ensuring the success of data compilations .

5. Model Learning For Cpath

Once an application domain and corresponding dataset have been chosen, the next step of developing a CPath tool involves designing of an appropriate model and representation learning paradigm. Representation learning refers to a set of algorith- mic techniques dedicated to learning feature representations of a certain data domain that can be used in downstream tasks . In CPath, the amount of data available for a given an- notation level and task are the key determinants to designing a model and learning technique. The last decade has shown neu- ral network architectures to become the dominant method in many machine-learning domains because they are rich enough to avoid handcrafted features and offer superior performance.

. The annotation level of the data pertaining to the task corresponds to the level of supervision for the learning tech- nique applied. This relationship between data annotation level and learning supervision level is surveyed in Figure 11.

This section details the various types of models and learn- ing techniques, along with the tasks they have been applied to in CPath. Figure 10 highlights the most common backbone ar- chitectures used for feature encoding in SOTA research, based on the corresponding tasks. More details are provided in Ta- ble 9.11 from the supplementary materials. The selection of

17

architectures is then compared to draw useful insights into ac- curacy, computational complexity, and limitations. Lastly, ex- isting challenges in model design are investigated.

5.1. Classification Architectures

In CPath, general classification architectures are the most prevalent due to their straightforward applicability to a wide range of tasks including tissue subtype classification, disease diagnosis and detection (more details in Section 2 and Figure 4). Architectures commonly used for natural images, in partic- ular CNNs, are widely adopted for CPath. To maximize model performance, it is a common approach to pre-train the model on large datasets like ImageNet before subsequent fine-tuning them to perform well for the specific CPath task, a task known as transfer learning [294, 295, 296, 297, 274, 298, 299, 87, 300, 301, 302, 303, 304, 305, 86, 306, 307, 257, 308, 268, 309, 310, 64, 311, 312, 313, 314, 206, 315, 316, 317, 318, 319]. Trans- fer learning for CPath allows for: 1) improved generalizability, particularly for tasks with limited amount of data; and 2) im- proved ease in finetuning a model compared to training from scratch .

Graph Convolutional Neural Networks (GCN) [321, 322] is an alternative architecture that can be used to improve the learn- ing of context-aware features across the WSI. GCNs typically consist of nodes representing elements and edges defining re- lationships between nodes. In , a GCN was defined on a WSI, where the nodes represent patches and edges represent the connections among patches; this work obtained remarkable re- sults on cancer prognosis task outperforming the SOTA in four out of five cancer types .

Vision Transformers (ViT) , have recently emerged as a direct application of Transformer models to the image domain. In ViT, images are sub-patched and flattened into a 1D embedding along with a positional encoding which is then classified by an MLP head. By using the positional encoding, the model’s attention mechanism can focus computation on the most relevant areas of the image. ViT models have been ap- plied with great success to CPath tasks, especially in conjunc- tion with pre-trained CNN models . We refer the reader to a comprehensive survey of transformer methods in medical image analysis for more details .

General classification architectures are also commonly used as a foundation for novel architectural designs. For example, Squeeze-and-Excitation (SE) modules were introduced to re- duce the number of parameters in ResNet and DenseNet blocks while maintaining high accuracy [328, 236]. A fully-connected conditional random field (CRF) was incorporated on top of a CNN encoder to improve performance while maintaining the same level of computational complexity . Lastly, patch sampling and pooling were used with AlexNet to perform slide- level disease diagnosis and segmentation .

Finally, in order to achieve superior performance, many re- searchers often rely on ensemble or multi-stage techniques which combine the predictive power or feature extraction abili- ties of multiple models to form a final output. These approaches have shown performance improvements compared to traditional single model classifiers [330, 331, 332, 333, 334, 335]. How- ever, this often comes at the expense of higher computational requirements.

5.2. Segmentation Architectures

Segmentation is widely used in CPath, as shown in Figure 4, and enable localizing the area of interest at the pixel level . U-Net was initially developed for neuronal structure seg- mentation in electron microscopy image stacks , but has become one of the most common architectures for segmenta- tion in CPath [337, 338, 339, 340, 244, 341, 342, 285, 330, 343, 344, 345, 346, 347, 152, 280, 348, 349, 350, 311, 351, 352, 353, 354, 355, 205, 356, 207]. U-Net has an encoder-decoder struc- ture: an encoder to contract features spatially and a decoder to expand them again to capture semantically related context and generate pixel-level predictions . The U-Net model has been used to segment nuclei for creating a novel dataset with unsupervised learning , but it should be noted that this process also relies on the Mask R-CNN framework and pathol- ogists for quality-checking purposes.

Another common approach for segmentation is to use fully convolutional networks (FCNs) [357, 265, 358, 359, 80, 360, 198, 277, 84, 361, 362, 356], customized architectures con- structed by combining multiple components of various archi- tectures, or introducing new components to pre-existing archi- tectures [241, 363, 168, 364, 365, 366, 367, 368, 369, 370, 266, 371, 307, 84, 372, 373, 374, 350, 375, 351, 276, 273, 376, 316].

For example, one work used a custom CNN to predict whether each pixel was benign or malignant, while a second CNN was used to refine the initial prediction through probability fusion .

5.3. Object Detection Architectures

In this section, we specifically focus on architectures that are used for object detection in CPath, where bounding boxes are predicted around regions of interest. A major CPath application for object detection is mitosis detection with the primary goal for counting the number of mitosis instances. To this end, a large number of studies has been dedicated to this application [377, 199, 225, 269, 378, 344, 379, 200, 380, 381, 382, 383, 384, 385, 350, 386]. Object detection has been additionally ap- plied for nuclei [387, 388, 77, 389, 390, 391, 392], colorectal gland [80, 393, 348] and glomeruli detection [394, 395, 396]; however, it can also be applied to the detection of a variety of histopathological objects including tumor-infiltrating lympho- cytes or keratin pearls .

In CPath, object detection employs a combination of pre- existing off-the-shelf architectures and customized neural net- works to perform object detection tasks, as shown in Figure 10.

A model called CircleNet, which uses a deep layer aggrega- tion network as a backbone, was proposed to detect round ob- jects . Their approach involves using an anchor-free “cen- ter point localization” framework in order to output a heatmap with center points followed by a conversion into a bounding circle for the detection of kidney glomeruli. A multi-stage deep learning detection model based on Fast R-CNN was proposed in . First, a modified Fast R-CNN generated region pro- posals, then a ResNet-50 model eliminated false positives, and

18

Fig. 10: Distribution of the most common Neural Network architectures used in the surveyed literature, based on the CPath task. The x-axis displays the Neural Network architectures used in the papers (from left to right): Custom CNN, Inception, ResNet, VGG, U-Net, and AlexNet. The y-axis shows the different tasks (top to bottom): Detection, Classification, Disease Diagnosis, Segmenta- tion, WSI Processing, Patient Prognosis, and Others. For more details, please refer to Table 9.11 in the supplementary section.

a Feature Pyramid Network detected mitosis in sparsely anno- tated images using a ResNet backbone .

5.4. Multi-Task Learning

Multi-task models are individual models predicting for mul- tiple tasks at once (e.g. classification and segmentation), as de- fined in Section 2. Multi-task learning (MTL) can be beneficial over independent task learning because sharing representations between related tasks can create more generalizable represen- tations and encourage the task heads to make logically consis- tent predictions. This type of model, however, is uncommon in CPath, as it requires annotating multiple tasks for each image [120, 398, 82, 384, 399, 32, 400]. We discuss some of these papers in further detail below.

In one work, a ResNet-50 backbone followed by indepen- dent decoders (a pyramid scene parsing network for segmen- tation and a fully-connected layer for classification) was used to solve 11 different tasks (4 segmentation based and 7 classi- fication based) . With significantly less computation, the MTL model achieved comparable or better results to single task learning in classification, but comparatively worse results in segmentation. Similarly, in , a ResNet-50 with two parallel branches to perform segmentation and classification, was able to achieve comparable results on both tasks through an MTL approach.

While the results are impressive, there is still work to be done in this field. One work found that model performance may be sensitive to the number and type of tasks used during training . If the tasks are unrelated, this could deteriorate the per- formance compared to a single-task setting. How to weigh dif- ferent task objectives and select optimal tasks to be trained to- gether is yet an active area of research [401, 402]. MTL repre- sents an interesting field of research in CPath as it may reduce the necessity to train multiple deep neural networks to perform different tasks.

5.5. Multi-Modal Learning

As opposed to multi-task networks where multiple tasks are learned simultaneously, the multi-modal approach involves us- Fig. 11: Details of types of learning using varying levels of supervision. Note that the types of tasks each type of learning can address vary based on the data that is available, as noted in the Example Task portion of the figure. However, from left to right, models trained with less supervision can still learn salient rep- resentations of the data that can be used to fine-tune models for tasks requiring more supervision. In that sense, for CPath there is a spectrum of supervision from self to strongly supervised learning that aligns well with the annotation levels shown in Figure 7.

ing network input features from multiple domains/modalities at once . In the case of CPath, modalities can be repre- sented as pathologists’ reports, gene expression data, or even WSI images. Most commonly, immunohistochemistry (IHC) stains alongside the H&E stain to better visualize specific pro- teins [404, 405, 406].

As A Result, Models Can Learn Better

unified/shared latent representations which capture correlations from multiple indicators, since some information may not be captured by individual indicators . This approach can be viewed as adding hand-crafted features to boost performance.

While the use of deep learning normally implies using learned features to replace hand-crafted ones, using hand-crafted fea- tures can nonetheless improve performance compared to strictly deep learning approaches when data is limited . Indeed, many works have obtained best performance by combining manual and learned features [378, 390, 409]. This was demon- strated in the case of mitotic cell classification when an en- sembled classifier model using hand-crafted features set a new record for the MITOS-ATYPIA 2014 challenge with an F-score of 86% . However, where data is plentiful, CNNs alone can outperform all other hand-crafted features.

In The Same

MITOS-ATYPIA 2014 challenge, the previous record was bro- ken this way with a new F-score of 96% . Although one cannot compare these two works directly as they use different classifier heads and dataset balancing methods, one can argue that the optimal choice of approaches from deep learning, clas- sical ML, and different modalities should depend on the situ- ation. Multi-modal approaches are gaining traction in CPath for specific problems, especially where useful additional data is available [412, 413]. For example, gene expression data and WSI images are often combined to improve cancer prognosis prediction [414, 415] .

5.6. Vision-Language Models

Following its successful use in the natural image domain, vision-language data (consisting of histopathology images paired with relevant natural language text) is becoming increas- ingly prominent in CPath. Whether it be the development of foundational models extending to CPath, or fine tun- ing state-of-art large large models for use in downstream tasks

19

[417, 418, 419], leveraging the semantic information embedded in the natural language data is becoming more evidently bene- ficial. It was only recently that foundational language mod- els advanced enough to become useful in CPath, and this has triggered an explosion of interest into building models at the intersection of visual and language information. At the mo- ment, language data is primarily used to address Multi-Instance Learning, although this is still an extremely new field and we anticipate that future works will surely address more advanced tasks (see Section 7.4 for further discussion).

5.7. Sequential Models

Recurrent Neural Networks (RNNs) are typically used in tasks with temporally-correlated sequential data, such as speech or time series . Since RNNs consider the past through the hidden state, they are suited for handling contextual infor- mation.

While Images Are The Default Data Format In Cpath

(and hence poorly-suited for RNNs), some works opt to com- bine RNNs with CNNs as a feature extractor [62, 88, 421, 231, 422, 266, 423, 264, 424, 425, 213, 426, 427], most com- monly by aggregating patches or processing feature sequences [62, 231, 266, 425, 428]. Another application of RNNs is to consider spatial relations between patches, which can be lost after extracting from the slide [264, 213].

A particularly exciting use of RNNs is in deciding which region within an image should be examined next [424, 429]. In the “Look, Investigate, and Classify” 3-stage model, an long short-term memory (LSTM) was used to classify the ROI cropped from the current patch and predict the next region to be analyzed, and achieved good performance while only us- ing 15% of pixels from the original image . Similarly, an LSTM network was used to better predict ROIs by treat- ing state features similar to time-series data, thus identifying only relevant examples to use for training . And an LSTM transformer with “Feature Aware Normalization” (FAN) units for stain normalization was used in parallel with a VGG-19 net- work . More recently, transformers using attention mech- anisms have been used to allow parallelization and better se- quence translation compared to older RNNs or LSTM networks .

5.8. Synthetic Data And Generative Models

With annotated data difficult to obtain in CPath, especially with granular labels (see Section 3.3 for more discussion), this is problematic for training generalizable models. Hence, gener- ating synthetic data from a controlled environment (either via simulation or a trained model) for augmenting the available training set of annotated data shows much promise. Originally developed for visual style transfer in general computer vision, generative models learn to create novel instances of samples from a given data distribution - they form the dominant ap- proach in CPath.

Initial works primarily utilized Generative Adversarial Net- works (GANs) for patch synthesis [337, 431, 432, 433], stain normalization, [434, 208, 209, 210, 435, 206], style transfer [436, 437, 438, 206], and various other tasks . One unsu- pervised pipeline relied on a non-GAN model to create an initial patch that was refined by a GAN . In another work, one CycleGAN generated tumor images and another non-tumor, in order to train a classification network . One work used neural image compression to learn the optimal encoder to map image patches to spatially consistent feature vectors . An- other work first classified bone marrow cell representations and then used an unsupervised GAN to generate more instances from each cluster . A self-supervised CycleGAN was also used for stain normalization, and shown to improve model per- formance in subsequent detection and segmentation tasks .

Similarly, a CycleGAN pipeline was applied to perform artifi- cial IHC re-staining . Recent works in GANs attempt to model spatial awareness of tissues and improve the realism of the generated samples .

Lately, diffusion models have become the SOTA in general computer vision and now produce far more semantically plau- sible and noise-free images than GANs. These improvements promise to make synthetic data finally accepted by pathologists and the broader CPath community as reliable training data and significantly improve the generalizability of models trained on them .

5.9. Multi-Instance Learning (Mil) Models

Multi-instance learning (MIL) involves training from data that is labelled as high-level bags consisting of numerous un- labelled instances. In the context of CPath, these labelled bags often represent annotated slides of far more unlabelled patch instances . As labels at the WSI level are much easier to obtain (and hence more prevalent) than patch-level annota- tions, MIL has been applied to CPath by a significant number of papers [62, 425, 445, 446, 63, 447, 448, 449, 444, 274, 450, 451, 452, 453, 454, 455, 456, 315, 275, 323, 457, 64, 456, 458, 459, 460, 461, 462, 463]. Since both utilize coarser annota- tions for training on massive images, MIL is similar to weakly- supervised learning. However, weak supervision predicts at a finer level (e.g. pixel segmentation from labelled patches) than the provided annotation while MIL prediction is typically at the same level.

One notable work used a two stage approach to first encode patches with a CNN from a slide into feature vectors and then pass the most cancer-likely ones to an slide-level classification RNN. A similar work first detected abnormal regions in the WSI before adaptively fusing the instance-level features with an importance coefficient . Adding additional instance- specific attributes tends to improve MIL performance.

One

work applied a nuclei grading network to provide a cell-level prediction for each patch, and demonstrate this out-performs hand-crafted cell features for overall slide classification .

Recent works explore the morphological and spatial relation- ships between instances, which conforms with pathologist diag- nostic intuitions and have demonstrably improved performance, especially with unbalanced data .

As not all instances are equally relevant to the bag label, many works focus on building attention mechanisms to adap- tively focus on more relevant instances. One work used such a mechanism to highlight regions of interest and improve local- ization relative to other SOTA CNNs . MIL models can

20

be improved by considering multi-scale information: one work notably used embeddings from different magnification levels and self-supervised contrastive learning to learn WSI classi- fiers .

Some Works Explicitly Encode The Patient-Slide-

patch hierarchy in the attention mechanism [464, 465], with one work using a cellular graph for top-down attention . Graph Neural Networks (GNNs) have been explored to lever- age intra- and inter-cell relationships, enabling cancer grading , classification , and survival prediction [469, 470].

These hierarchy- and morphology-aware models are the current SOTA and pave the way for future improvements. One persistent challenge with using MIL in CPath, compared to natural image computer vision, is the lack of large-scale WSI datasets . One recent work addressed issues related to small sample cohorts by splitting up large bags (and their la- bels) into smaller ones through pseudo-bags .

5.10. Contrastive Self-Supervised Learning for Few-Shot Gen-

Eralization

The idea of using contrastive learning (CL) for self- supervised learning (SSL) dates back to 2005, yet only re- cently gained momentum in CPath [473, 474, 275, 475, 476, 477, 313, 478]. By using a contrastive loss, a feature embed- ding is learned to ensure similar (positive) examples are close in vector space, while dissimilar (negative) examples are dis- tant [473, 474]. Contrastive learning is an attractive approach for CPath because when used as self-supervision for few-shot learning [275, 475, 313], it does not require labelling the mas- sive self-supervision image set but only labelling the small sub- set used for training on the downstream task, an approach that has recently achieved SOTA performance in a wide array of tasks in CPath . SimCLR was originally proposed to learn representations invariant to different augmentation transforms (such as crop, noise) for natural images , and when applied to CPath, was found to match or outperform SOTA supervised techniques . Self-supervised pre-training has been shown to perform best against fully-supervised pre-training when ap- plied to small but visually-diverse datasets . Recent works have focused on transferring the self-supervised representations to the downstream task more intelligently: through latent space transfers , with an awareness of the patient-slide-patch hi- erarchy [480, 481], or with semi-supervised pseudo-label guid- ance .

5.11. Novel Cpath Architectures

In this section, we discuss papers that made significant changes to the model design or completely designed an archi- tecture from scratch for CPath tasks [284, 483, 389, 484, 64, 364, 485, 368, 486]. Typically, model architectures are adapted from the natural image domain and minor changes applied for CPath tasks, rather than being designed from scratch for CPath directly. Unfortunately, general computer vision architectures typically require large computational resources not necessar- ily available in clinical settings and are prone to overfitting on smaller CPath training sets .

More importantly, CPath tasks often comprise of multiple specialized sub-tasks not addressed by common architectures - in such cases, CPath-specific architectures perform better.

“PlexusNet” achieved SOTA performance with significantly fewer parameters and “Hover-Net” used a three-branched architecture for nuclei classification and instance segmentation . Path R-CNN similarly used one branch to generate ep- ithelial region proposals and another to segment tumours .

In other cases, custom architectures are designed to obtain better performance with respect to certain metrics [85, 302, 384, 395, 345] or improve computational efficiency and speed , where since model inference can be a bottleneck for WSI pro- cessing. To automate architecture design, neural architecture search (NAS) is often used. This is an umbrella term covering evolutionary algorithms (EA), deep learning (specifically Rein- forcement Learning), and gradient-based NAS searches. There are two approaches to EA: (1) Neuroevolution, which more generally optimizes at the neuron-level to find optimal weights, and (2) Evolutionary-Algorithms based NAS (EANAS), which searches for optimal combinations of mid-sized neural net- work blocks and conducts training after this architectural search [487, 488]. In CPath, reinforcement learning-based NAS has designed models for cancer prediction which were found to train faster, have fewer parameters, and perform comparably with manually designed models . Another work demon- strated that a significantly smaller model can outperform exist- ing SOTA models on a variety of CPath tasks using an adaptive optimization strategy .

We hypothesize that NAS has yet to be explored significantly in CPath due to the lack of annotated data (see Section 3.3) and its relative recency as a research area. According to the “no free lunch theorems” [490, 491], no single model can perform best on all tasks. However, computationally efficient but per- formant models are crucial for CPath applications, and NAS is the most promising approach to computationally design such architectures without manual engineering.

5.12. Model Comparison

The various model architectures and types discussed above can and should be compared on common benchmarks to de- termine the best models for a given task . Numerous pa- pers have conducted such benchmarking work on CNNs. One work comparing GoogLeNet, AlexNet, VGG16, and FaceNet on breast cancer metastasis classification found that deeper net- works (i.e. GoogLeNet) predictably performed better .

Another work found that using ResNet-34 with a custom gra- dient descent performed best .

Finally, Vgg-19 Per-

formed best in colorectal tissue subtype clasification, showing that deeper SOTA networks do not necessarily perform better universally. Which CNN performs best depends on the task, the nature of the data, the metrics used, training time, hyperpa- rameters, and/or hardware constraints.

Likewise, third parties have organized “grand-challenges” to facilitate the fair comparison of different techniques on a common CPath task and dataset.

In Some Cases, Sota

CNNs achieve the best results, such as the adapted GoogLeNet that obtained the highest AUC and the AlexNet that achieved highest accuracy for breast cancer detection in the CAMELYON16 challenge. Likewise, SqueezeNet, which

21

is an existing SOTA network performs best in colorectal tis- sue subtype classification . On the contrary, the best per- forming models for mitosis detection in the TUPAC16 and MITOS12 challenges both relied on custom CNN ar- chitectures. For breast cancer diagnosis, a novel Hybrid CNN achieved the best results in the BACH18 (ICIAR18) dataset while the two teams achieving the best classification ac- curacy in the BreakHis dataset used differing approaches: one directly used ResNet-50 and the other used an ensem- ble of VGG networks . For nuclei segmentation on the Kumar-TCGA dataset, a novel framework using ResNet and another existing model achieved the highest F1-score .

Lastly, a custom CNN achieved the best results for gladn seg- mentation on the GLaS dataset . However, as mentioned in Section 3.3, many grand chal- lenges use private datasets or even extract data from larger public repositories without referencing the original WSIs used.

Furthermore, benchmark datasets address different tasks and lack standardization. As models that are hyper-optimized to for specific sets of data continue to be released, the lack of more standardized benchmark datasets and model comparison stud- ies make it impossible to systematically compare new models against existing ones or assess their robustness in clinical set- tings, thus impeding model development in CPath.

6.1. Clinical Validation

Within the domain of CPath, clinical validation is essen- tial for substantiating the decisions produced by deep learning models so that they are more readily accepted by the medical community. Generally, acceptable clinical criteria are deter- mined by authoritative professional guidelines, consensus, or evidence-based sources. However, in CPath, prediction results are generated by the computer scientists and engineers who build the model, and may not be completely aware of where their work fits into the clinical pathology workflow–the clini- cal implications of this arrangement are often unknown .

By incorporating pathologist expertise, clinical validation can better align the technical work with clinical objectives. Despite the importance of this step for real-world deploy- ment, very few works have performed clinical validation with expert pathologists. We identify three prominent types of clin- ical validation in the CPath literature: (1) direct performance comparison of CAD tools with pathologists on a similar task, (2) impact of CAD tool assistance on pathologist performance, and (3) pathologist validation of CAD tool outputs. Each topic is further discussed in the sections alongside notable results.

Direct Performance Comparison with Pathologists To val- idate the benefits of deep learning methods, it is desirable that they equal or surpass the performance humans to gain the trust of pathologists in their decisions and their willingness to use them as a second opinion . With this in mind, many pa- pers directly compared their models with pathologists in tasks such as prognosis and diagnosis.

One study on cancer detection found that the top com- putational models from the CAMELYON16 challenge out- performed the 11 pathologists with a two-hour time constraint and performed similarly to the expert pathologist without a time constraint . This suggests that deep learning mod- els could be particularly useful in clinical scenarios with ex- cessive numbers of time-critical cases to diagnose. Similarly, for tissue subtype classification, another study performed sim- ilarly to, or slightly better than individual pathologists. The proposed model agreed with all pathologists 66.6% of the time and agreed with two-thirds of pathologists 76.7% of the time . An additional study claimed their deep learning model outperformed pathologists without gynecology-specific train- ing in ovarian carcinoma classification . This pushes the idea that CAD predictions can be used as a second opinion due to the potential for human error by individual pathologists.

One paper on diagnosis demonstrated that deep learn- ing models can correctly classify images that even individ- ual pathologists failed to correctly identify. However, another paper found that 50% of the examples misclassified by their model were also misclassified by at least one pathologist .

This suggests that deep learning models can aid pathologists in decision-making, but as they tend to achieve a specificity and sensitivity similar to pathologists, they must be applied cau- tiously to avoid reinforcing the biases or errors of individual pathologists.

Deep learning models for prognosis have been shown to achieve performance similar to or better than experts as well [245, 388, 296]. In one study, the best model for renal clear cell carcinoma classification achieved 83% accuracy, outperforming the inter-pathologist accuracy of 80% . This shows that deep learning models and pathologists may perform similarly on patient prognosis.

Overall, AI approaches are not perfect but have approached expert-level ability in a variety of tasks. Deep learning could play an important role as a second opinion and in democra- tizing the knowledge distilled from many pathologists to other pathology centres. Specifically, deep learning models appear to be best used as a tool to enhance the pathologist workflow, and could provide aid in making quick decisions with high accuracy .

Impact of CAD Tool Assistance Much of CPath research is conducted under the assumption that the resulting AI tools will be intuitive, usable, and beneficial to pathologists and patients.

However, CAD tools that are developed without feedback from pathologists could fail to integrate into a realistic pathologist workflow or impact the most significant diagnostic tasks. Thus, a valuable validation experiment is to compare and comprehend the performance of expert pathologists in clinical tasks before and after being given the assistance of a CAD tool.

In one study, a CAD system called Paige Prostate Alpha leveraged a weakly-supervised algorithm to highlight patches in a WSI with the highest probability of cancer . When used by pathologists, the model significantly improved sensitiv- ity, average review time, and accuracy over unaided diagnosis.

Likewise, another study using the LYNA algorithm examined the performance of six pathologists on breast cancer tumor clas- sification before and after being able to see the LYNA-predicted patch heatmaps. The results indicate using LYNA substantially improved sensitivity, average review time, and the subjective

22

“obviousness” score for all breast cancer types [72, 27]. These studies suggest that integrating CAD tools into the clinical workflow will greatly improve pathologist efficiency.

However, there is a general lack of research on the impact of CAD tools on pathology efficiency. Such studies would shed more light on the impact of CAD tools and identify approaches for implementation in clinical settings.

6.2. Fda Regulations

Despite the ongoing development of CAD tools in CPath and its potential for triaging cases and providing second opinions, the regulations regarding this technology pose an obstacle to the testing and deployment of these devices. The FDA cur- rently provides three levels of clearance on AI/ML-based med- ical technologies: 510(k) clearance, premarket approval, and the De Novo pathway. While one source lists 64 AI/ML-based medical solutions that are currently FDA-approved or cleared, none of these are in the field of CPath . A few compa- nies, such as Paige AI, hold the 510(k) clearance for their digi- tal pathology image viewer; however, an automated diagnostic system has yet to be approved. This may indicate a reluctance to change, and the lack of clarity in the process of FDA approval has prevented numerous impactful technologies from being de- ployed. There is a need for collaboration between researchers, doctors, and governmental bodies to establish a clear pathway for these novel technologies to be validated and implemented in clinical settings.

7. Emerging Trends In Cpath Research

Computational pathology research has seen a sudden shift of focus in the past year of 2023. Driven by recent technological advances in the field of computer vision for natural images and the release of capable foundational models in natural language processing, formerly difficult research problems in CPath have been solved, opening up exciting new avenues of research, es- pecially the difficulty of training models on adequate annotated data. We will discuss the main research trends below in further detail and make simple predictions of where the field is headed.

7.1. Contrastive Self-Supervised Learning becomes Main-

Stream

Data annotation for CPath is a persistent problem - it is easy to collect large amounts of visual data but much harder to an- notate them. Transfer learning can help but it is difficult to transfer a model trained on one dataset to generalize to another.

Whereas past efforts focused on carefully engineered methods, the recent development of contrastive self-supervised learning means that it has become the mainstream approach in CPath [313, 275, 475]. Not only does it utilize the massive amounts of unlabelled images typically available in CPath, but it also as a result only requires finetuning on a small set for the downstream task. We anticipate that this will lead to the development of general-use foundational models in the future to perform the most common CPath tasks, as more pathology images are collected and models become more advanced.

7.2. Prediction Becoming Increasingly High-Level

We noticed that recent research works are increasingly ad- dressing higher level prediction tasks than before.

Whereas

patch classification [294, 295, 296, 297, 274] or pixel segmenta- tion [336, 337, 339, 340, 244] was formerly mainstream, these problems appear to have been largely solved, and now there is far more research into higher-level problems dominate, such as multiple-instance learning [444, 62, 464, 465, 466]. As com- putational methods continue to improve, it is natural that they are applied not merely as attention aids for pathologists (i.e. at the pixel or patch level), but furthermore are used to make in- telligent slide- and patient-level decisions on their own. Indeed, they promise to vastly improve pathologist efficiency when used with human pathologists in the loop to validate the automated decisions, especially when paired with modern natural language capabilities.

7.3. Spatial and Hierarchical Relationships receiving Attention Inspired by the approach taken for natural image computer vision, the mainstream approach in CPath currently requires breaking up large WSIs into smaller patches and perceiving them independently (see Figure 7). However, this ignores the spatial relationships between cells and tissues or between the patches and their parent slides in histopathology images, which are often relevant or even crucial when making decisions. Many works have recently found success in explicitly encoding an awareness of these inter-cell relationships [467, 469, 470] and the patch-slide-patient hierarchy [466, 464, 465], especially us- ing Graph neural networks, but these suffer from higher latency than conventional CNNs. We anticipate future works will seek to speed up GNNs for tasks where spatial and hierarchical re- lationships are important and continue developing hierarchy- aware attention for MIL techniques.

7.4. Vision-Language Models for Explainable Predictions One persistent problem in CPath has been developing mod- els that can explain their decisions for human validation. One obvious route is to develop models that produce natural lan- guage output (and even converse with the human user to explain their decisions), but until recently, this would have required col- lecting massive amounts of pathology text paired with images.

With foundational vision-language models widely available and able to generalize to great effect in the natural image domain [500, 501, 502], recent works have shown that they perform excellently when applied with minimal re-training to CPath im- ages [417, 418, 419]. Further advances require collecting more pathology-specific data, but we anticipate that crowd sourc- ing of public pathology annotations will become mainstream and this will lead to the development of foundational vision- language models. As natural language capabilities continue im- proving, we also anticipate that synoptic report automation will become feasible and reinforcement learning from human feed- back (RLHF) will become common for improving CPath language models.

7.5. Synthetic Data Now Realistic Enough

Whereas one way to combat the difficulty of annotating CPath data is to develop models that require fewer annota- tions, another trend is to generate more annotated data for train- ing. Whereas concerns were previously raised about their real- ism, new advances in generative image models have now been leveraged to produce realistic histopathology images and pixel- accurate annotations simultaneously. However, current works are limited by specific tissues, organs, diseases [337, 439, 212, 440], or stains [210, 441] and are limited by their inability to easily expand to other histopathology content. We note that generating synthetic data via game engines and 3D model assets is a recent trend in the natural image domain [504, 505, 506], but visual modelling of histopathology entities is little explored.

We anticipate that future works will attempt to improve syn- thetic histopathology image generation by: (1) attempting to create generative models that can generalize to a broad variety of histopathology images and (2) create simulation software to generate realistic histopathology images without learned mod- els.

8.1. Cpath As Anomaly Detection

Typically in computer vision, the various classes represent distinct normative entities, such as airplanes or bears [507, 508]. There exist abundant “normal” samples and potentially few “anomalous” samples, which are considered data points sig- nificantly dissimilar to the majority within a given class .

These anomalies are not only out of distribution from the sam- ples in a dataset, there is also a lack of consensus on understand- ing anomalous representations as effectively identifying anoma- lies requires ML models to learn a feature space encompassing all “normal” samples within each class .

In other words, in general computer vision, each class can- not simply be considered as an anomalous version of any other class. However, in CPath, since each class is often considered a different disease state on a single tissue type, diseased classes are essentially extensions of the “normal” healthy class into the “anomalous” zone. From a pathologist’s perspective, similar to the general computer vision approach, the curriculum learning process of a resident pathologist first involves training on histol- ogy and gaining a mastery of normal tissue identification, and then train on diseased tissues, so they are able to flag the sample as anomalous and follow up with possible diagnosis.

In light of this, it may be illuminating to approach the prob- lem from an anomaly detection viewpoint: provided a model has sufficient variety of healthy tissue, any anomalies must then be diseased. The output of such an anomaly detection algo- rithm is dependent on the task at hand. One source describes several meaningful output types that may be produced : an anomaly score which describes how anomalous a sample is and a binary label indicating whether a sample is normal or anomalous. If only identifying anomalous samples is enough, a binary classification procedure may be sufficient. However, if it is necessary to identify the particular stage of progression of a disease type, then a more granular approach in assigning some anomaly score may be more appropriate as explored in a previous work . This work found that the confidence score in tissue classification was inversely correlated with dis- ease progression, thus the confidence score may act as a proxy for an anomaly score. Theoretically, such approaches may bet- ter replicate the behaviour of pathologists. While several works have used an anomaly detection approach on medical image data outside of CPath [511, 512, 513], few works tackle the problem for WSI data in CPath.

8.2. Leveraging Existing Datasets

As mentioned in Section 3.3 of this paper, a minority of datasets in CPath are available to be freely used by the public. Additionally, the level of annotations varies for each dataset.

However as can be noted in Table 9.11 of the supplemen- tary material, for prominent public datasets such as CAME- LYON16, CAMELYON17, GlaS, BreakHis, and TCGA, there is far more available data annotated at the slide level as op- posed to more granular predictions. For example, considering breast datasets, there are 399 WSIs annotated at the Slide and

Roi Levels In Camelyon16 And 1399 Wsis Annotated

at the Patient, Slide, and ROI levels in CAMELYON17 , in contrast, the TCGA-BRCA dataset contains 1163 diagnostic slides and 1978 tissue slides that are accompanied with labels at the Patient and Slide levels and diagnostic reports with labels for tissue features and tumor grades .

The lack of publicly available datasets with granular annota- tions is a major challenge in CPath. To address this lack some training data, techniques have been proposed to efficiently ob- tain labels, such as an active deep learning frameworks that use a small amount of labelled data to suggest the most pertinent unlabelled samples to be annotated . Alternatively, other works propose models to synthetically create WSI patches, usu- ally with the use of GANs. For example, Hou et al. in- troduced an unsupervised pipeline that was capable of synthe- sizing annotated data at a large scale, noting that even patholo- gists had difficulty distinguishing between real and synthesized patches. However, despite these promising results, the issue of acquiring accurate and large datasets remains a prevalent issue within CPath.

Generally, tasks such as tissue classification or gland seg- mentation require labels at the ROI, Patch, or Pixel levels. How- ever, existing data annotated at the patient and slide levels can be used for these tasks by leveraging weakly supervised tech- niques such as MIL [64, 444], or by learning rich representa- tions using self-supervised techniques such as DINO [516, 65] and contrastive learning that can be used in downstream tasks.

Specifically, work is being done to develop training methodologies and architectures that are more data efficient for patient- and slide-level annotations, such as CLAM, which is a MIL technique that is used to train a performant CPath model with as little as 25% of the training data to get over 0.9 AUC .

Another recent work used self-supervised learning on WSIs without labels to train a hierarchical vision transformer and used the learned representations to fine-tune for cancer sub- typing tasks. This finetuned model outperformed other SOTA methods that used supervised learning methods on both the full

24

training set and when all models used only 25% of the train- ing set. These examples demonstrate a recent trend in the ap- plication of weakly and self-supervised learning techniques to leverage pre-existing and available data with weak labels, show- casing that a large amount of granular labels are not necessarily required for achieving SOTA performance. We urge researchers in the CPath field to follow this trend and focus on how to lever- age existing weakly labelled datasets, especially to learn rich representations as a pre-training step for learning on smaller strongly labelled datasets.

8.3. Creating New Datasets

Although we mention the availability of many datasets and comment on how to leverage this existing data, there is still a need for new CPath datasets that address overlooked clinical and diagnostic areas. Therefore, creation of new CPath datasets should focus on addressing two main goals: (1) tasks that are not addressed adequately by existing datasets and (2) accumu- lating as large a dataset as possible with maximal variety.

Regarding the first goal, there are still organs, diseases, and pathology tasks without freely available data or sufficient an- notations to develop CAD tools. For example, in Figure 6, we see that whereas breast tissue datasets are abundant, there are few public datasets for the brain and none for the liver. Col- lecting and releasing datasets for these organs would have sig- nificant impact in enabling further works focusing on these ap- plications. Further, analysis of specific organ synoptic reports can guide CPath researchers to build CAD tools to identify or discriminate the most impactful diagnostic parameters. In the case of the prostate, which is discussed in Section 2.5, the syn- optic report requires distinguishing IDC from PIN and PIA as it correlates to high Gleason scores. This is important, as high- grade PIN is a cancer precursor requiring follow-up sessions for screenings. These parameters are identified and noted in the re- port by the pathologist and factor into the final diagnosis and grading. Thus, collecting annotated datasets for such parame- ters can be crucial to developing CAD tools that are relevant to clinical workflows and can enrich learned representations.

The second goal is concerned with the scaling laws of deep learning models with respect to the amount of data available and their application to diverse clinical settings. As seen in the gen- eral computer vision domain, larger datasets tend to improve model performance, especially when used to learn rich model representations through pre-training that can be used for down- stream tasks such as classification and semantic segmentation . Additionally, ensuring that datasets capture the under- lying data distribution and thus sufficiently encompass the test distribution has been shown to be especially important in the medical domain . For CPath, this means ensuring a dataset captures the expected variations in tissue structure, disease pro- gression, staining, preparation artifacts, scanner types, and im- age processing. Collecting a sufficiently large dataset continues to be problematic, however, so recent works have focused on using crowd sourcing to annotate histopathology data posted publicly on Twitter and YouTube [416, 418], a practice that is similar to that commonly used for natural images.

8.4. Pre- And Post-Analytical Cad Tools

In recent years, advances in image analysis, object detection, and segmentation have motivated new approaches to support the analytical phase of the clinical workflow, especially in the two steps where CAD tools could significantly increase efficiency and accuracy: (1) specimen registration and (2) pathology re- ports. This need is highlighted by a study determining that the pre-analytical and post-analytical phases (as shown in Figure 2 account for up to 77% of medical errors in pathology .

Likewise, Meier et al. classify 14% of medical errors as diag- nostic errors, with an even smaller proportion being misinter- preted diagnoses in their study . Other authors attribute approximately 15 −25% of diagnostic errors to slide interpre- tation [521, 522, 523, 524, 525]. These results reinforce the need for CPath applications that address more than just the an- alytical phase . Considering post-analytical step of com- piling a pathology report, a few natural language processing efforts have been used to analyze completed pathology reports [527, 528, 529], extract primary site codes from reports, and generate of captions or descriptive texts for WSI patches . However, to the best of our knowledge, there are no works that reliably extract clinical data from service requests and electronic medical records to automatically generate syn- optic or text reports. Developing such a tool that could ex- plicitly identify the most significant parameters for its decisions would directly improve clinical workflow and increase the in- terpretability of the results at the same time. We encourage the field of CPath to expand its efforts in creating tools for the pre- and post-analytical steps in order to reduce the large proportion of clinical errors attributed to those phases, and suggest some potential applications in Figure 2.

8.5. Multi Domain Learning

Despite being particularly well-suited for CPath, multi- domain learning (MDL) is still a relatively unexplored topic. MDL aims to train a unified architecture that can solve many tasks (e.g. lesion classification, tumour grading) for data com- ing from different domains (e.g. breast, prostate, liver). Dur- ing inference, the model receives an input image and the corre- sponding domain indicator and is able to solve the correspond- ing task for the given domain. There are two reasons that make MDL attractive for CPath. The first is that the additional infor- mation from a source domain (coming from a related organ such as the stomach) can be informative for improving performance in the target domain (e.g. colon). By sharing representations between related domains, the model is enabled to generalize to other domains. The second motivation is to alleviate the data sparsity problem where one domain has a limited number of la- beled data. Through MDL, the domain with limited data can benefit from the features that are jointly learned with other re- lated tasks/domains [531, 532].

8.6. Federated Learning For Multi-Central Cpath

Data-driven models require a large amount of data to yield strong performance. In CPath, this requires incorporating di- verse datasets with varying tissue slide preparations, staining quality, and scanners. An obvious solution to train such models

25

is to accumulate the data from multiple medical centers into a centralized repository. However, in practice, data privacy reg- ulations may not permit such data sharing between medical in- stitutions, especially between countries. A possible solution lies in privacy-preserving training algorithms, such as federated learning [533, 534], which can make use of decentralized data from multiple institutions while maintaining data privacy. In federated learning, training starts with a generic machine learn- ing model in a centrally located server. But instead of trans- ferring data to a centralized server for training, copies of the model are sent to individual institutions for training on their lo- cal data. The learning updates are encrypted and sent to the central server and then aggregated across the institutions. Ming Lu et al. demonstrated the feasibility and effectiveness of applying federated, attention-based weakly supervised learn- ing for general-purpose classification and survival prediction on WSIs using data from different sites. Using such algorithms for CPath can facilitate cross-institutional collaborations and can be a viable solution for future commercial solutions that need to continuously augment and improve their ML models using decentralized data.

8.7. Cpath-Specific Architecture Designs

Many deep learning architectures are not designed for CPath specifically, which raises a serious question about the optimal- ity of using “borrowed” architectures from general computer vision. For instance, notes that traditional CV architec- tures may not be well suited for CPath due to a large number of parameters that risk overfitting. Additionally, the field of pathology has much domain-specific knowledge that should be taken into account before choosing an ML model. For example, under varying magnifications different morphological patterns are captured, from cellular-level details to tissue architecture features . Naively applying an architecture without con- sidering such details could discard key visual information and lead to deteriorated performance.

Unlike natural images, WSIs exhibit translational, rotational, and reflective symmetry and CNNs for general vision ap- plications do not exploit this symmetry. The conventional ap- proach to overcome this issue is to train the model with aug- mented rotations and reflections, but this increases training time and does not explicitly restrict CNN kernels to exploit those symmetries. Rotation-equivariant CNNs, which are inherently equivariant to rotations and reflections were introduced for dig- ital pathology , significantly improving over a comparable CNN on slide level classification and tumor localization. Sim- ilarly, Lafarge et al. designed a group convolution layer leveraging the rotational symmetry of pathology data to yield superior performance in mitosis detection, nuclei segmentation, and tumor classification tasks. These results motivate the appli- cation and further research of rotation-equivariant models for CPath.

In general, we note that the SOTA computer vision architec- tures used in computational pathology have tended to lag be- hind those used for natural images by a couple of years. This delay in knowledge propagation from the mainline computer vision research in natural images may be due to the data-centric nature of the CPath field. As data labelling is specialized and expensive to conduct, annotating more data or clever training tweaks to finetune established architectures is more attractive than developing advanced, specialized architectures. However, we recommend that CPath researchers should still use the most powerful relevant models available for the simple reason that they tend to perform best given the computational resources available. While computational efficiency is generally not as important during training, it is imperative at inference time if models are to be run in real-time on medical devices with lim- ited computational resources.

8.8. Digital And Computational Pathology Adoption

Despite the numerous advantages to the clinical workflow and applications offered by using digital pathology and CPath, the adoption of digital pathology remains the first barrier to clinical use. A major reason for adoption hesitancy is the com- mon opinion that digital slide analysis is an unnecessary step in a pathologist’s workflow which has been refined over decades to produce reproducible and robust diagnoses without digitiza- tion [18, 19, 9, 20]. In terms of clinical efficiency, studies have shown mixed results, with two finding that digitization actually decreased efficiency (by increasing turnaround time by 19%) [536, 537]. However, another study demonstrated a clear in- crease in productivity and reduction of turnaround-time .

One of the co-authors (B.N.) has implemented digital pathology at a public tertiary institution, which began as a pilot study over three years including three experienced academic pathologists and showed that digitization reduced turnaround time by 18% for biopsies and 25% for resections, and increased case output by 17%. These trial results led to all pathologists not retiring within two years to transition to a digital pathology workflow in 2019. Due to the varied nature of results and outcomes in studies analyzing the effectiveness of digital pathology there is more work to be done to have a multi-institution and lab analy- sis for more general and concrete results.

A major factor in the adoption of digital or computational pathology practices is the source of funding and the pay struc- ture of pathologists.

A Few Cost-Analysis Studies Show That

the transition to digital pathology becomes financially advan- tageous in 2 years, with savings projected to be up to $5M after 5 years in a sizeable tertiary center [538, 9, 539, 540]. The financial impact will also be viewed differently in public vs pri- vate healthcare settings. Public healthcare is primarily limited by funding and universal access to healthcare whereas for pri- vate lab networks improvements in processes and services are directly linked to the prospects of obtaining additional contracts and increased profitability. However, studies considering mul- tiple institutions and funding settings are still required to fully characterize the financial impact compared to clinical benefit.

Additionally, on an individual pathologist level, compensation structures can affect buy in for implementation. For example, at our co-authors’ B.N. and V.Q.T institution, a fee-for-service structure is used to compensate pathologists thus an increase in throughput and productivity has a direct correlation to increased pay. We propose that this fee-for-service model contributes to the widespread embracement of DP at this institution. In con- trast, pathologists in a salary-based environment are paid based

26

based on a combined package of services which includes di- agnostics, research, teaching, administration, quality control, etc. An increase in clinical productivity would technically not benefit them directly, as it would translate to a high number of rendered diagnostics over the same amount of time.

Integration CPath into the clinical workflow is relatively un- derstudied as few papers have actually deployed, or performed clinical validation of their results. Works in this area have ei- ther proposed methods to deploy their models in the clinic or developed tools to enable the use of their research in the clinic [294, 497, 541]. However, as a primary goal of CPath is the use of CAD tools in clinical settings, more works should consider how to integrate models and tools into the clinical workflow, especially in conjunction with expert clinicians.

8.9. Institutional Challenges

Several institutional challenges may affect the implementa- tion of CPath tools, and similar challenges in implementing dig- ital pathology workflows at medical institutions have been well- described by many studies [542, 543, 544, 545, 546]. As noted by multiple studies considering the digital transition of pathol- ogy laboratories [542, 543, 544, 545, 546], the importance of a common shared goal and frequent communication between the involved parties is necessary to successfully deploy a digital system. These lessons are likely extendable in the context of CPath and CAD development as well. Specifically, Cheng et al. reported on their experiences and lessons learned as a 7-point-based system to efficiently deploy a digital pathology system in a large academic center. We believe similar system- atic approaches will need to be developed to implement CPath applications in a clinical setting.

Another institutional challenge concerns the regulatory over- pathology association, state/provincial, and federal levels. Reg- ulatory measures underlying WSI scanners are well established, as well as the technical and clinical validation of their use [547, 548, 549].

On The Other Hand, Patient Confidentiality,

ethics, medical data storage regulations, and data encryption laws are equally, if not more, time-consuming and intensive to comply with. These issues can be mitigated by deploying a standardized digital pathology system throughout multiple in- stitutions at the state/provincial level. For example, our co- author (B.N.) has obtained governmental approval and funding to distribute a set of digital pathology systems throughout the province’s public anatomical pathology laboratories. Similarly, a unified set of standards for processing and digitizing slides, along with unifying storage and access to WSIs for research use in collaborative efforts is paramount in moving forward in both the development and implementation of CAD systems.

8.10. Clinical Alignment Of Cpath Tasks

Researchers in the CPath field must ensure that the CAD tools they create are clinically relevant and applicable to pathol- ogy so that effort and resources are not allocated towards ex- traneous or clinically irrelevant tasks.

For Example, Certain

CADs have been proposed to facilitate case triaging and reduce turnaround time for critical diagnoses [550, 72, 73, 74, 551]]. However, several regulatory agencies in pathology aim for 90% of cases to be completed within a timeframe of 72 hours for signing-out resection specimens and up to 48 hours for biop- sies [552, 553]. In this context, triaging becomes extraneous, as signing out cases faster than 48-72 hours has no clinical impact.

However, in the context of an institution operating at longer turnaround times or struggling to keep up with the caseload, this method could be lifesaving. Alternatively, identifying mi- totic figures and counting positive Ki-67 nuclei are appreciated tools already in use in multiple digital pathology settings, de- spite these tools being seldom applied to the large caseload pro- portion of most practicing pathologists.

As noted previously, the overall number of pathologists in the USA has decreased 17% from 2007 to 2017 and caseloads have increased by 41.7% . This trend places further emphasis of developing CAD tools towards specific challenges encountered by pathologists and where sub-specialists may not be readily available. For example, a large consortium generated a prostate cancer CAD that achieved a 86.8% concordance with expert genitourinary pathologists , a significant breakthrough for healthcare settings where prostate biopsies are not signed out by sub-specialists. Additionally, targeting specific diagnoses with high rates of medical errors and inter-observer variance, notably in dermatological, gynecological, and gastrointestinal pathol- ogy, should be prioritized and integrated into practice quickly to support patient care . Finally, advanced CAD tools capa- ble of diagnosing features out of reach by conventional pathol- ogy could have a great impact. For example, identifying the ori- gin of metastases from morphological cues on the WSIs without added IHC or CADs capable of calculating the exact in- volvement of cancer on a biopsy core for prognostic purposes .

8.11. Concluding Remarks

Bringing pathologists and computer scientists together and initiating meaningful collaborations with shared gains between all parties is likely the most efficient path forward for CPath and CAD integration. Opportunities to facilitate collaborations should be promoted by parties such as the Pathology Innova- tion Collaborative Community and the Digital Pathology Asso- ciation. Furthermore, we encourage involved pathologists and computer scientists to communicate and collaborate on studies towards the common goal of providing patients with fast, repro- ducible, and high-quality care.

Acknowledgment

Authors would like to thank Huron Digital Pathology for providing support and insightful discussions related to digital pathology hardware infrastructures. Authors would like to also thank Xin Zhao, Koosha E. Khorasani, Mona Sharifi, Kiana Abtahi, Alejandra Zambrano Luna, Cassandre Notton, Tomas Pereira, and Amirhossein Mohammadi for their help in initial preparation of our model-cards.

This Work Was Partly Sup-

ported from NSERC-CRD (CRDPJ515553-17) funding grant and MITACS-Elevate award (Ref. IT14369).

Related Journal Articles & DOI Links

Selected peer-reviewed publications relevant to 12 Lead ECG Acquisition. Click the DOI to access the full paper (may require institutional access).

Why Choose Us?

Bangalore guidance for robotics, Spectre and autonomous systems projects.