切换至 "中华医学电子期刊资源库"

中华结直肠疾病电子杂志 ›› 2020, Vol. 09 ›› Issue (05) : 475 -481. doi: 10.3877/cma.j.issn.2095-3224.2020.05.008

所属专题: 文献

论著

不同扫描仪构建的结直肠癌全切片数字病理图像中人工标注迁移的研究
李江涛1, 郑波1, 潘怡1, 王书浩2, 刘灿城2, 吕宁1, 孙卓2,(), 邹霜梅1,()   
  1. 1. 100021 北京,国家癌症中心/国家肿瘤临床医学研究中心/中国医学科学院北京协和医学院肿瘤医院病理科
    2. 100102 透彻影像(北京)科技股份有限公司研发部
  • 收稿日期:2020-09-15 出版日期:2020-10-25
  • 通信作者: 孙卓, 邹霜梅
  • 基金资助:
    中国医学科学院医学科学创新基金(No.2018-I2M-AI-008)

Transfer of manual annotation in digital pathological images of colorectal cancer with different scanners

Jiangtao Li1, Bo Zheng1, Yi Pan1, Shuhao Wang2, Cancheng Liu2, Ning Lyu1, Zhuo Sun2,(), Shuangmei Zou1,()   

  1. 1. Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
    2. Department of Research and Development, Thorough Images, Beijing 100102, China
  • Received:2020-09-15 Published:2020-10-25
  • Corresponding author: Zhuo Sun, Shuangmei Zou
  • About author:
    Corresponding authors:Zou Shuangmei, Email:
    Sun Zhuo, Email:
引用本文:

李江涛, 郑波, 潘怡, 王书浩, 刘灿城, 吕宁, 孙卓, 邹霜梅. 不同扫描仪构建的结直肠癌全切片数字病理图像中人工标注迁移的研究[J]. 中华结直肠疾病电子杂志, 2020, 09(05): 475-481.

Jiangtao Li, Bo Zheng, Yi Pan, Shuhao Wang, Cancheng Liu, Ning Lyu, Zhuo Sun, Shuangmei Zou. Transfer of manual annotation in digital pathological images of colorectal cancer with different scanners[J]. Chinese Journal of Colorectal Diseases(Electronic Edition), 2020, 09(05): 475-481.

目的

研究结直肠癌人工智能病理诊断模型构建过程中,病理医师对数字切片癌组织的人工标注在不同扫描仪构建的全切片图像(WSI)中准确迁移的方法。

方法

在本研究中,我们提出了一种基于图像配准的标注迁移方法,在来自不同扫描仪的WSI之间建立仿射映射。通过多分辨率最小化两个WSI缩略图之间的互信息来估计最佳仿射映射参数,以避免和改变扫描仪特定特性的影响,减少计算时间。我们使用了181张结直肠癌病理切片,使用两个品牌的扫描仪获得相应的WSI,对上述标注迁移方法进行测试。

结果

181张HE切片的扫描结果表明,同一张切片由不同扫描仪构建的WSI在颜色、位置、大小等属性上都有不同的表现。使用我们提出的标注迁移方法,其中179张图像的人工标注成功地在不同扫描仪构建的WSI中迁移,其中125对使用单个CPU核心的计算时间不到1分钟。

结论

我们提出了一种快速、准确的全自动的标注迁移方法,用于在不同扫描仪构建的WSI之间传递人工标注。在准备深度学习训练数据过程中,既可以避免病理医师对新图像的重新标注,也可以避免病理医师之间在标注上的差异。

Objective

To create an accurate transfer method of the digital slide of cancer tissue annotated by pathologist in the whole slide image (WSI) constructed by different scanners in the process of constructing the artificial intelligence pathological diagnosis model of colorectal cancer.

Methods

In this study, we proposed a annotation transfer method based on image registration to establish affine mapping between WSIs from different scanners. The best affine mapping parameters are estimated by minimizing the mutual information between the two WSI thumbnails to avoid and change the specific characteristics of the scanner and reduce the calculation time. We used 181 colorectal cancer pathological sections and two brands of scanners to obtain the corresponding WSI to test the above annotation transfer method.

Results

The scanning results of 181 H&E slides showed that WSI constructed by different scanners in the same slide had different performance in color, position, size and other attributes. Using our proposed annotation transfer method, a total of 179 images were successfully transferred between WSIs constructed by different scanners, and 125 pairs of them took less than 1 minute to compute using a single CPU core.

Conclusion

We propose a fast and accurate automatic annotation transfer method, which is used to transfer manual annotation between WSIs constructed by different scanners. In the process of preparing deep learning training data, we can not only avoid the new image reannotation by pathologists, but also avoid the difference in annotation between pathologists.

图1 不同扫描仪扫描的HE切片的差异说明。1A~1D:病例1~4。每个图左侧显示来自KF-PRO-005-EX扫描仪的WSI,右侧显示来自EasyScan6扫描仪的WSI
图2 显示直接将病理医师在WSI Im(左)空间上的标注应用于WSI In(右)空间的结果。结果表明,直接应用标注会导致在新的WSI空间中的偏差(2A~2D:病例1~4)
图3 最佳仿射变换迁移图像的图示。带标注的目标图像Im(左);带标注的仿射迁移图像(右)(3A~3D:病例1~4)
图4 标注迁移结果的图示。带标注Cm的目标图像Im(左);仿射迁移标注Cn的图像In(右)(4A~4D:病例1~4
图5 实验中的两个失败案例,每一行对应一对。左栏显示目标图像Im,右栏显示图像In。(5A、5B:失败病例1,5C、5D:失败病例2 )
图6 每个图像对的计算时间成本分布
[1]
Metter DM, Colgan TJ, Leung ST, et al. Trends in the US and Canadian Pathologist Workforces From 2007 to 2017 [J]. JAMA Netw Open, 2019, 2(5): e194337.
[2]
Williams B J, Lee J, Oien K A, et al. Digital pathology access and usage in the UK: results from a national survey on behalf of the National Cancer Research Institute's CM-Path initiative [J]. Journal of Clinical Pathology, 2018, 71(5): 463-466.
[3]
LeCun Y, Bengio Y, Hinton G. Deep learning [J]. Nature, 2015, 521(7553): 436-444.
[4]
Kamilaris A, Prenafeta-Boldú FX. Deep learning in agriculture: A survey [J]. Computers and Electronics in Agriculture, 2018, 147: 70-90.
[5]
Helber P, Bischke B, Dengel A, et al. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(7): 2217-2226.
[6]
Shen D, Wu G, Suk HI. Deep Learning in Medical Image Analysis [J]. Annu Rev Biomed Eng, 2017, 19: 221-248.
[7]
Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis [J]. Med Image Anal, 2017, 42: 60-88.
[8]
Ortiz A, Munilla J, Górriz JM, et al. Ensembles of Deep Learning Architectures for the Early Diagnosis of the Alzheimer's Disease [J]. Int J Neural Syst, 2016, 26(7): 1650025.
[9]
Milletari F, Ahmadi S A, Kroll C, et al. Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound [J]. Computer Vision and Image Understanding, 2017, 164: 92-102.
[10]
Işın A, Direkoğlu C, Şah M. Review of MRI-based brain tumor image segmentation using deep learning methods [J]. Procedia Computer Science, 2016, 102: 317-324.
[11]
Cha KH, Hadjiiski L, Samala RK, et al. Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets [J]. Med Phys, 2016, 43(4): 1882.
[12]
Roth H R, Farag A, Lu L, et al. Deep convolutional networks for pancreas segmentation in CT imaging. Medical Imaging 2015: Image Processing [J]. International Society for Optics and Photonics, 2015, 9413: 94131G.
[13]
Li X, Chen H, Qi X, et al. H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes [J]. IEEE Trans Med Imaging, 2018, 37(12): 2663-2674.
[14]
Li W. Automatic segmentation of liver tumor in CT images with deep convolutional neural networks [J]. Journal of Computer and Communications, 2015, 3(11): 146.
[15]
Tomczak K, Czerwińska P, Wiznerowicz M. The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge [J]. Contemp Oncol (Pozn), 2015, 19(1A): A68-A77.
[16]
Zhu Y, Qiu P, Ji Y. TCGA-assembler: open-source software for retrieving and processing TCGA data [J]. Nat Methods, 2014, 11(6): 599-600.
[17]
Litjens G, Bandi P, Ehteshami Bejnordi B, et al. 1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset [J]. Gigascience, 2018, 7(6): giy065.
[18]
Cruz-Roa A, Gilmore H, Basavanhally A, et al. Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent [J]. Sci Rep, 2017, 7: 46450.
[19]
Tellez D, Balkenhol M, Otte-Höller I, et al. Whole-slide mitosis detection in H&E breast histology using PHH3 as a reference to train distilled stain-invariant convolutional networks [J]. IEEE transactions on medical imaging, 2018, 37(9): 2126-2136.
[20]
Ehteshami Bejnordi B, Mullooly M, Pfeiffer RM, et al. Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies [J]. Mod Pathol, 2018, 31(10): 1502-1512.
[21]
Steiner DF, MacDonald R, Liu Y, et al. Impact of Deep Learning Assistance on the Histopathologic Review of Lymph Nodes for Metastatic Breast Cancer [J]. Am J Surg Pathol, 2018, 42(12): 1636-1646.
[22]
Mårtensson G, Ferreira D, Granberg T, et al. The reliability of a deep learning model in clinical out-of-distribution MRI data: a multicohort study [J]. Medical Image Analysis, 2020, 101714.
[23]
Van Leemput K, Maes F, Vandermeulen D, et al. Automated model-based bias field correction of MR images of the brain [J]. IEEE Trans Med Imaging, 1999, 18(10): 885-896.
[24]
Tustison NJ, Avants BB, Cook PA, et al. N4ITK: improved N3 bias correction [J]. IEEE Trans Med Imaging, 2010, 29(6): 1310-1320.
[25]
Dong P, Brankov JG, Galatsanos NP, et al. Digital watermarking robust to geometric distortions [J]. IEEE Trans Image Process, 2005, 14(12): 2140-2150.
[26]
Pluim JP, Maintz JB, Viergever MA. Mutual-information-based registration of medical images: a survey [J]. IEEE Trans Med Imaging, 2003, 22(8): 986-1004.
[27]
Schroeder W, Ng L, Cates J. The ITK software guide second edition updated for ITK version 2.4 [J]. FEBS Lett, 2005, 525: 53-58.
[28]
Klein S, Staring M, Murphy K, et al. Elastix: a toolbox for intensity-based medical image registration [J]. IEEE Trans Med Imaging, 2010, 29(1): 196-205.
[29]
Fischl B. Free Surfer [J]. Neuroimage, 2012, 62(2): 774-781.
[30]
Jenkinson M, Beckmann CF, Behrens TE, et al. FSL [J]. Neuroimage, 2012, 62(2): 782-790.
[31]
Kazemi K, Noorizadeh N. Quantitative Comparison of SPM, FSL, and Brain suite for Brain MR Image Segmentation [J]. J Biomed Phys Eng, 2014, 4(1): 13-26.
[32]
Aljabar P, Heckemann RA, Hammers A, et al. Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy [J]. Neuroimage, 2009, 46(3): 726-738.
[1] 郏亚平, 曾书娥. 含鳞状细胞癌成分的乳腺化生性癌的超声与病理特征分析[J]. 中华医学超声杂志(电子版), 2023, 20(08): 844-848.
[2] 唐玮, 何融泉, 黄素宁. 深度学习在乳腺癌影像诊疗和预后预测中的应用[J]. 中华乳腺病杂志(电子版), 2023, 17(06): 323-328.
[3] 康夏, 田浩, 钱进, 高源, 缪洪明, 齐晓伟. 骨织素抑制破骨细胞分化改善肿瘤骨转移中骨溶解的机制研究[J]. 中华乳腺病杂志(电子版), 2023, 17(06): 329-339.
[4] 代莉, 邓恢伟, 郭华静, 黄芙蓉. 术中持续输注艾司氯胺酮对腹腔镜结直肠癌手术患者术后睡眠质量的影响[J]. 中华普通外科学文献(电子版), 2023, 17(06): 408-412.
[5] 王得晨, 杨康, 杨自杰, 归明彬, 屈莲平, 张小凤, 高峰. 结直肠癌微卫星稳定状态和程序性死亡、吲哚胺2,3-双加氧酶关系的研究进展[J]. 中华普通外科学文献(电子版), 2023, 17(06): 462-465.
[6] 唐旭, 韩冰, 刘威, 陈茹星. 结直肠癌根治术后隐匿性肝转移危险因素分析及预测模型构建[J]. 中华普外科手术学杂志(电子版), 2024, 18(01): 16-20.
[7] 张生军, 赵阿静, 李守博, 郝祥宏, 刘敏丽. 高糖通过HGF/c-met通路促进结直肠癌侵袭和迁移的实验研究[J]. 中华普外科手术学杂志(电子版), 2024, 18(01): 21-24.
[8] 李婷, 张琳. 血清脂肪酸代谢物及维生素D水平与结直肠癌发生的关系研究[J]. 中华普外科手术学杂志(电子版), 2023, 17(06): 661-665.
[9] 李晓阳, 刘柏隆, 周祥福. 大数据及人工智能对女性盆底功能障碍性疾病的诊断及风险预测[J]. 中华腔镜泌尿外科杂志(电子版), 2023, 17(06): 549-552.
[10] 倪文凯, 齐翀, 许小丹, 周燮程, 殷庆章, 蔡元坤. 结直肠癌患者术后发生延迟性肠麻痹的影响因素分析[J]. 中华结直肠疾病电子杂志, 2023, 12(06): 484-489.
[11] 范小彧, 孙司正, 鄂一民, 喻春钊. 梗阻性左半结肠癌不同手术治疗方案的选择应用[J]. 中华结直肠疾病电子杂志, 2023, 12(06): 500-504.
[12] 关旭, 王锡山. 基于外科与免疫视角思考结直肠癌区域淋巴结处理的功与过[J]. 中华结直肠疾病电子杂志, 2023, 12(06): 448-452.
[13] 顾睿祈, 方洪生, 蔡国响. 循环肿瘤DNA检测在结直肠癌诊治中的应用与进展[J]. 中华结直肠疾病电子杂志, 2023, 12(06): 453-459.
[14] 萨仁高娃, 张英霞, 邓伟, 闫诺, 樊宁. 超声引导下鼠肝消融术后组织病理特征的变化规律及影响[J]. 中华消化病与影像杂志(电子版), 2023, 13(06): 394-398.
[15] 徐军, 姬园园, 陈君平, 王健. 伴菊形团结构的脑膜瘤合并颅骨侵犯一例并文献复习[J]. 中华临床医师杂志(电子版), 2023, 17(08): 916-919.
阅读次数
全文


摘要