Transfer of manual annotation in digital pathological images of colorectal cancer with different scanners

doi:10.3877/cma.j.issn.2095-3224.2020.05.008

Abstract

Abstract:

Objective

To create an accurate transfer method of the digital slide of cancer tissue annotated by pathologist in the whole slide image (WSI) constructed by different scanners in the process of constructing the artificial intelligence pathological diagnosis model of colorectal cancer.

Methods

In this study, we proposed a annotation transfer method based on image registration to establish affine mapping between WSIs from different scanners. The best affine mapping parameters are estimated by minimizing the mutual information between the two WSI thumbnails to avoid and change the specific characteristics of the scanner and reduce the calculation time. We used 181 colorectal cancer pathological sections and two brands of scanners to obtain the corresponding WSI to test the above annotation transfer method.

Results

The scanning results of 181 H&E slides showed that WSI constructed by different scanners in the same slide had different performance in color, position, size and other attributes. Using our proposed annotation transfer method, a total of 179 images were successfully transferred between WSIs constructed by different scanners, and 125 pairs of them took less than 1 minute to compute using a single CPU core.

Conclusion

We propose a fast and accurate automatic annotation transfer method, which is used to transfer manual annotation between WSIs constructed by different scanners. In the process of preparing deep learning training data, we can not only avoid the new image reannotation by pathologists, but also avoid the difference in annotation between pathologists.

Key words: Colorectal neoplasms, Annotation transfer, Whole slide image, Scanner, Pathology, Deep learning

Jiangtao Li, Bo Zheng, Yi Pan, Shuhao Wang, Cancheng Liu, Ning Lyu, Zhuo Sun, Shuangmei Zou. Transfer of manual annotation in digital pathological images of colorectal cancer with different scanners[J]. Chinese Journal of Colorectal Diseases(Electronic Edition), 2020, 09(05): 475-481.

/ / Recommend

Add to citation manager EndNote|Ris|BibTeX

URL: https://zhjzcjbdzzz.cma-cmc.com.cn/EN/10.3877/cma.j.issn.2095-3224.2020.05.008

https://zhjzcjbdzzz.cma-cmc.com.cn/EN/Y2020/V09/I05/475

Figures/Tables 6

References 32

[1]	Metter DM, Colgan TJ, Leung ST, et al. Trends in the US and Canadian Pathologist Workforces From 2007 to 2017 [J]. JAMA Netw Open, 2019, 2(5): e194337.
[2]	Williams B J, Lee J, Oien K A, et al. Digital pathology access and usage in the UK: results from a national survey on behalf of the National Cancer Research Institute's CM-Path initiative [J]. Journal of Clinical Pathology, 2018, 71(5): 463-466.
[3]	LeCun Y, Bengio Y, Hinton G. Deep learning [J]. Nature, 2015, 521(7553): 436-444.
[4]	Kamilaris A, Prenafeta-Boldú FX. Deep learning in agriculture: A survey [J]. Computers and Electronics in Agriculture, 2018, 147: 70-90.
[5]	Helber P, Bischke B, Dengel A, et al. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(7): 2217-2226.
[6]	Shen D, Wu G, Suk HI. Deep Learning in Medical Image Analysis [J]. Annu Rev Biomed Eng, 2017, 19: 221-248.
[7]	Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis [J]. Med Image Anal, 2017, 42: 60-88.
[8]	Ortiz A, Munilla J, Górriz JM, et al. Ensembles of Deep Learning Architectures for the Early Diagnosis of the Alzheimer's Disease [J]. Int J Neural Syst, 2016, 26(7): 1650025.
[9]	Milletari F, Ahmadi S A, Kroll C, et al. Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound [J]. Computer Vision and Image Understanding, 2017, 164: 92-102.
[10]	Işın A, Direkoğlu C, Şah M. Review of MRI-based brain tumor image segmentation using deep learning methods [J]. Procedia Computer Science, 2016, 102: 317-324.
[11]	Cha KH, Hadjiiski L, Samala RK, et al. Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets [J]. Med Phys, 2016, 43(4): 1882.
[12]	Roth H R, Farag A, Lu L, et al. Deep convolutional networks for pancreas segmentation in CT imaging. Medical Imaging 2015: Image Processing [J]. International Society for Optics and Photonics, 2015, 9413: 94131G.
[13]	Li X, Chen H, Qi X, et al. H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes [J]. IEEE Trans Med Imaging, 2018, 37(12): 2663-2674.
[14]	Li W. Automatic segmentation of liver tumor in CT images with deep convolutional neural networks [J]. Journal of Computer and Communications, 2015, 3(11): 146.
[15]	Tomczak K, Czerwińska P, Wiznerowicz M. The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge [J]. Contemp Oncol (Pozn), 2015, 19(1A): A68-A77.
[16]	Zhu Y, Qiu P, Ji Y. TCGA-assembler: open-source software for retrieving and processing TCGA data [J]. Nat Methods, 2014, 11(6): 599-600.
[17]	Litjens G, Bandi P, Ehteshami Bejnordi B, et al. 1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset [J]. Gigascience, 2018, 7(6): giy065.
[18]	Cruz-Roa A, Gilmore H, Basavanhally A, et al. Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent [J]. Sci Rep, 2017, 7: 46450.
[19]	Tellez D, Balkenhol M, Otte-Höller I, et al. Whole-slide mitosis detection in H&E breast histology using PHH3 as a reference to train distilled stain-invariant convolutional networks [J]. IEEE transactions on medical imaging, 2018, 37(9): 2126-2136.
[20]	Ehteshami Bejnordi B, Mullooly M, Pfeiffer RM, et al. Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies [J]. Mod Pathol, 2018, 31(10): 1502-1512.
[21]	Steiner DF, MacDonald R, Liu Y, et al. Impact of Deep Learning Assistance on the Histopathologic Review of Lymph Nodes for Metastatic Breast Cancer [J]. Am J Surg Pathol, 2018, 42(12): 1636-1646.
[22]	Mårtensson G, Ferreira D, Granberg T, et al. The reliability of a deep learning model in clinical out-of-distribution MRI data: a multicohort study [J]. Medical Image Analysis, 2020, 101714.
[23]	Van Leemput K, Maes F, Vandermeulen D, et al. Automated model-based bias field correction of MR images of the brain [J]. IEEE Trans Med Imaging, 1999, 18(10): 885-896.
[24]	Tustison NJ, Avants BB, Cook PA, et al. N4ITK: improved N3 bias correction [J]. IEEE Trans Med Imaging, 2010, 29(6): 1310-1320.
[25]	Dong P, Brankov JG, Galatsanos NP, et al. Digital watermarking robust to geometric distortions [J]. IEEE Trans Image Process, 2005, 14(12): 2140-2150.
[26]	Pluim JP, Maintz JB, Viergever MA. Mutual-information-based registration of medical images: a survey [J]. IEEE Trans Med Imaging, 2003, 22(8): 986-1004.
[27]	Schroeder W, Ng L, Cates J. The ITK software guide second edition updated for ITK version 2.4 [J]. FEBS Lett, 2005, 525: 53-58.
[28]	Klein S, Staring M, Murphy K, et al. Elastix: a toolbox for intensity-based medical image registration [J]. IEEE Trans Med Imaging, 2010, 29(1): 196-205.
[29]	Fischl B. Free Surfer [J]. Neuroimage, 2012, 62(2): 774-781.
[30]	Jenkinson M, Beckmann CF, Behrens TE, et al. FSL [J]. Neuroimage, 2012, 62(2): 782-790.
[31]	Kazemi K, Noorizadeh N. Quantitative Comparison of SPM, FSL, and Brain suite for Brain MR Image Segmentation [J]. J Biomed Phys Eng, 2014, 4(1): 13-26.
[32]	Aljabar P, Heckemann RA, Hammers A, et al. Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy [J]. Neuroimage, 2009, 46(3): 726-738.

Please choose a citation manager

Content to export