切换至 "中华医学电子期刊资源库"

中华结直肠疾病电子杂志 ›› 2020, Vol. 09 ›› Issue (05) : 475 -481. doi: 10.3877/cma.j.issn.2095-3224.2020.05.008

所属专题: 文献

论著

不同扫描仪构建的结直肠癌全切片数字病理图像中人工标注迁移的研究
李江涛1, 郑波1, 潘怡1, 王书浩2, 刘灿城2, 吕宁1, 孙卓2,(), 邹霜梅1,()   
  1. 1. 100021 北京,国家癌症中心/国家肿瘤临床医学研究中心/中国医学科学院北京协和医学院肿瘤医院病理科
    2. 100102 透彻影像(北京)科技股份有限公司研发部
  • 收稿日期:2020-09-15 出版日期:2020-10-25
  • 通信作者: 孙卓, 邹霜梅
  • 基金资助:
    中国医学科学院医学科学创新基金(No.2018-I2M-AI-008)

Transfer of manual annotation in digital pathological images of colorectal cancer with different scanners

Jiangtao Li1, Bo Zheng1, Yi Pan1, Shuhao Wang2, Cancheng Liu2, Ning Lyu1, Zhuo Sun2,(), Shuangmei Zou1,()   

  1. 1. Department of Pathology, National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing 100021, China
    2. Department of Research and Development, Thorough Images, Beijing 100102, China
  • Received:2020-09-15 Published:2020-10-25
  • Corresponding author: Zhuo Sun, Shuangmei Zou
  • About author:
    Corresponding authors:Zou Shuangmei, Email:
    Sun Zhuo, Email:
引用本文:

李江涛, 郑波, 潘怡, 王书浩, 刘灿城, 吕宁, 孙卓, 邹霜梅. 不同扫描仪构建的结直肠癌全切片数字病理图像中人工标注迁移的研究[J]. 中华结直肠疾病电子杂志, 2020, 09(05): 475-481.

Jiangtao Li, Bo Zheng, Yi Pan, Shuhao Wang, Cancheng Liu, Ning Lyu, Zhuo Sun, Shuangmei Zou. Transfer of manual annotation in digital pathological images of colorectal cancer with different scanners[J]. Chinese Journal of Colorectal Diseases(Electronic Edition), 2020, 09(05): 475-481.

目的

研究结直肠癌人工智能病理诊断模型构建过程中,病理医师对数字切片癌组织的人工标注在不同扫描仪构建的全切片图像(WSI)中准确迁移的方法。

方法

在本研究中,我们提出了一种基于图像配准的标注迁移方法,在来自不同扫描仪的WSI之间建立仿射映射。通过多分辨率最小化两个WSI缩略图之间的互信息来估计最佳仿射映射参数,以避免和改变扫描仪特定特性的影响,减少计算时间。我们使用了181张结直肠癌病理切片,使用两个品牌的扫描仪获得相应的WSI,对上述标注迁移方法进行测试。

结果

181张HE切片的扫描结果表明,同一张切片由不同扫描仪构建的WSI在颜色、位置、大小等属性上都有不同的表现。使用我们提出的标注迁移方法,其中179张图像的人工标注成功地在不同扫描仪构建的WSI中迁移,其中125对使用单个CPU核心的计算时间不到1分钟。

结论

我们提出了一种快速、准确的全自动的标注迁移方法,用于在不同扫描仪构建的WSI之间传递人工标注。在准备深度学习训练数据过程中,既可以避免病理医师对新图像的重新标注,也可以避免病理医师之间在标注上的差异。

Objective

To create an accurate transfer method of the digital slide of cancer tissue annotated by pathologist in the whole slide image (WSI) constructed by different scanners in the process of constructing the artificial intelligence pathological diagnosis model of colorectal cancer.

Methods

In this study, we proposed a annotation transfer method based on image registration to establish affine mapping between WSIs from different scanners. The best affine mapping parameters are estimated by minimizing the mutual information between the two WSI thumbnails to avoid and change the specific characteristics of the scanner and reduce the calculation time. We used 181 colorectal cancer pathological sections and two brands of scanners to obtain the corresponding WSI to test the above annotation transfer method.

Results

The scanning results of 181 H&E slides showed that WSI constructed by different scanners in the same slide had different performance in color, position, size and other attributes. Using our proposed annotation transfer method, a total of 179 images were successfully transferred between WSIs constructed by different scanners, and 125 pairs of them took less than 1 minute to compute using a single CPU core.

Conclusion

We propose a fast and accurate automatic annotation transfer method, which is used to transfer manual annotation between WSIs constructed by different scanners. In the process of preparing deep learning training data, we can not only avoid the new image reannotation by pathologists, but also avoid the difference in annotation between pathologists.

图1 不同扫描仪扫描的HE切片的差异说明。1A~1D:病例1~4。每个图左侧显示来自KF-PRO-005-EX扫描仪的WSI,右侧显示来自EasyScan6扫描仪的WSI
图2 显示直接将病理医师在WSI Im(左)空间上的标注应用于WSI In(右)空间的结果。结果表明,直接应用标注会导致在新的WSI空间中的偏差(2A~2D:病例1~4)
图3 最佳仿射变换迁移图像的图示。带标注的目标图像Im(左);带标注的仿射迁移图像(右)(3A~3D:病例1~4)
图4 标注迁移结果的图示。带标注Cm的目标图像Im(左);仿射迁移标注Cn的图像In(右)(4A~4D:病例1~4
图5 实验中的两个失败案例,每一行对应一对。左栏显示目标图像Im,右栏显示图像In。(5A、5B:失败病例1,5C、5D:失败病例2 )
图6 每个图像对的计算时间成本分布
[1]
Metter DM, Colgan TJ, Leung ST, et al. Trends in the US and Canadian Pathologist Workforces From 2007 to 2017 [J]. JAMA Netw Open, 2019, 2(5): e194337.
[2]
Williams B J, Lee J, Oien K A, et al. Digital pathology access and usage in the UK: results from a national survey on behalf of the National Cancer Research Institute's CM-Path initiative [J]. Journal of Clinical Pathology, 2018, 71(5): 463-466.
[3]
LeCun Y, Bengio Y, Hinton G. Deep learning [J]. Nature, 2015, 521(7553): 436-444.
[4]
Kamilaris A, Prenafeta-Boldú FX. Deep learning in agriculture: A survey [J]. Computers and Electronics in Agriculture, 2018, 147: 70-90.
[5]
Helber P, Bischke B, Dengel A, et al. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification [J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(7): 2217-2226.
[6]
Shen D, Wu G, Suk HI. Deep Learning in Medical Image Analysis [J]. Annu Rev Biomed Eng, 2017, 19: 221-248.
[7]
Litjens G, Kooi T, Bejnordi BE, et al. A survey on deep learning in medical image analysis [J]. Med Image Anal, 2017, 42: 60-88.
[8]
Ortiz A, Munilla J, Górriz JM, et al. Ensembles of Deep Learning Architectures for the Early Diagnosis of the Alzheimer's Disease [J]. Int J Neural Syst, 2016, 26(7): 1650025.
[9]
Milletari F, Ahmadi S A, Kroll C, et al. Hough-CNN: deep learning for segmentation of deep brain regions in MRI and ultrasound [J]. Computer Vision and Image Understanding, 2017, 164: 92-102.
[10]
Işın A, Direkoğlu C, Şah M. Review of MRI-based brain tumor image segmentation using deep learning methods [J]. Procedia Computer Science, 2016, 102: 317-324.
[11]
Cha KH, Hadjiiski L, Samala RK, et al. Urinary bladder segmentation in CT urography using deep-learning convolutional neural network and level sets [J]. Med Phys, 2016, 43(4): 1882.
[12]
Roth H R, Farag A, Lu L, et al. Deep convolutional networks for pancreas segmentation in CT imaging. Medical Imaging 2015: Image Processing [J]. International Society for Optics and Photonics, 2015, 9413: 94131G.
[13]
Li X, Chen H, Qi X, et al. H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT Volumes [J]. IEEE Trans Med Imaging, 2018, 37(12): 2663-2674.
[14]
Li W. Automatic segmentation of liver tumor in CT images with deep convolutional neural networks [J]. Journal of Computer and Communications, 2015, 3(11): 146.
[15]
Tomczak K, Czerwińska P, Wiznerowicz M. The Cancer Genome Atlas (TCGA): an immeasurable source of knowledge [J]. Contemp Oncol (Pozn), 2015, 19(1A): A68-A77.
[16]
Zhu Y, Qiu P, Ji Y. TCGA-assembler: open-source software for retrieving and processing TCGA data [J]. Nat Methods, 2014, 11(6): 599-600.
[17]
Litjens G, Bandi P, Ehteshami Bejnordi B, et al. 1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset [J]. Gigascience, 2018, 7(6): giy065.
[18]
Cruz-Roa A, Gilmore H, Basavanhally A, et al. Accurate and reproducible invasive breast cancer detection in whole-slide images: A Deep Learning approach for quantifying tumor extent [J]. Sci Rep, 2017, 7: 46450.
[19]
Tellez D, Balkenhol M, Otte-Höller I, et al. Whole-slide mitosis detection in H&E breast histology using PHH3 as a reference to train distilled stain-invariant convolutional networks [J]. IEEE transactions on medical imaging, 2018, 37(9): 2126-2136.
[20]
Ehteshami Bejnordi B, Mullooly M, Pfeiffer RM, et al. Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies [J]. Mod Pathol, 2018, 31(10): 1502-1512.
[21]
Steiner DF, MacDonald R, Liu Y, et al. Impact of Deep Learning Assistance on the Histopathologic Review of Lymph Nodes for Metastatic Breast Cancer [J]. Am J Surg Pathol, 2018, 42(12): 1636-1646.
[22]
Mårtensson G, Ferreira D, Granberg T, et al. The reliability of a deep learning model in clinical out-of-distribution MRI data: a multicohort study [J]. Medical Image Analysis, 2020, 101714.
[23]
Van Leemput K, Maes F, Vandermeulen D, et al. Automated model-based bias field correction of MR images of the brain [J]. IEEE Trans Med Imaging, 1999, 18(10): 885-896.
[24]
Tustison NJ, Avants BB, Cook PA, et al. N4ITK: improved N3 bias correction [J]. IEEE Trans Med Imaging, 2010, 29(6): 1310-1320.
[25]
Dong P, Brankov JG, Galatsanos NP, et al. Digital watermarking robust to geometric distortions [J]. IEEE Trans Image Process, 2005, 14(12): 2140-2150.
[26]
Pluim JP, Maintz JB, Viergever MA. Mutual-information-based registration of medical images: a survey [J]. IEEE Trans Med Imaging, 2003, 22(8): 986-1004.
[27]
Schroeder W, Ng L, Cates J. The ITK software guide second edition updated for ITK version 2.4 [J]. FEBS Lett, 2005, 525: 53-58.
[28]
Klein S, Staring M, Murphy K, et al. Elastix: a toolbox for intensity-based medical image registration [J]. IEEE Trans Med Imaging, 2010, 29(1): 196-205.
[29]
Fischl B. Free Surfer [J]. Neuroimage, 2012, 62(2): 774-781.
[30]
Jenkinson M, Beckmann CF, Behrens TE, et al. FSL [J]. Neuroimage, 2012, 62(2): 782-790.
[31]
Kazemi K, Noorizadeh N. Quantitative Comparison of SPM, FSL, and Brain suite for Brain MR Image Segmentation [J]. J Biomed Phys Eng, 2014, 4(1): 13-26.
[32]
Aljabar P, Heckemann RA, Hammers A, et al. Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy [J]. Neuroimage, 2009, 46(3): 726-738.
[1] 李洋, 蔡金玉, 党晓智, 常婉英, 巨艳, 高毅, 宋宏萍. 基于深度学习的乳腺超声应变弹性图像生成模型的应用研究[J]. 中华医学超声杂志(电子版), 2024, 21(06): 563-570.
[2] 张嘉炜, 王瑞, 张克诚, 易磊, 周增丁. 烧烫伤创面深度智能检测模型P-YOLO的建立及测试效果[J]. 中华损伤与修复杂志(电子版), 2024, 19(05): 379-385.
[3] 谢馨, 李一鸣, 胡晓均, 邓飞龙. 高压蒸汽灭菌次数对钛基底-聚醚醚酮种植扫描杆扫描准确度的影响[J]. 中华口腔医学研究杂志(电子版), 2024, 18(04): 230-236.
[4] 罗青杉, 梅海涛, 郝家领, 蔡锦锋, 周润楷, 温玉刚. 连接蛋白43通过调控细胞周期抑制结直肠癌的增殖机制研究[J]. 中华普通外科学文献(电子版), 2024, 18(05): 344-349.
[5] 伍先权, 张立果, 周璇, 梁建深. 乳腺包裹性乳头状癌的临床病理与手术策略联系[J]. 中华普通外科学文献(电子版), 2024, 18(04): 294-297.
[6] 谢丽春, 欧庆芬, 张秋萍, 叶升. 简化和标准肝脏MRI方案在结直肠癌肝转移患者随访中的临床应用[J]. 中华普外科手术学杂志(电子版), 2024, 18(04): 434-437.
[7] 施烨鑫, 马翔, 鲁明, 夏青城, 王鹏超, 宋青雨, 赵庆洪. 腹腔镜下结直肠肿瘤定位研究进展[J]. 中华普外科手术学杂志(电子版), 2024, 18(04): 463-466.
[8] 季鹏程, 鄂一民, 陆晨, 喻春钊. 循环外泌体相关生物标志物在结直肠癌诊断中的研究进展[J]. 中华结直肠疾病电子杂志, 2024, 13(04): 265-273.
[9] 李佳莹, 王旭丹, 梁雪, 张雷, 李佳英. 1990~2021年中国结直肠癌死亡趋势分析[J]. 中华结直肠疾病电子杂志, 2024, 13(04): 274-279.
[10] 戈伟, 陈刚. 纳米炭导航行淋巴示踪在结直肠癌TNM分期中淋巴分期价值的临床研究[J]. 中华结直肠疾病电子杂志, 2024, 13(04): 288-293.
[11] 崔精, 鲍一帆, 沈晓明, 杨增辉, 高森, 鲍传庆. 结直肠癌中circMFSD12对肿瘤细胞功能及5-FU敏感性的调控[J]. 中华结直肠疾病电子杂志, 2024, 13(04): 294-302.
[12] 李月刚, 关旭, 赵志勋, 权继传, 庄孟, 汤坚强. 结直肠癌NOSES手术的国际研究现状[J]. 中华结直肠疾病电子杂志, 2024, 13(04): 321-328.
[13] 国超凡, 彭琪博, 郑章强, 于向阳. 低位前切除综合征风险预测及治疗的研究进展[J]. 中华结直肠疾病电子杂志, 2024, 13(04): 335-340.
[14] 王曦娅, 尹弘青, 丁伟, 徐滨, 于海源, 马东升, 邵军. 桥本背景下甲状腺乳头状癌多参数分析预测大容量淋巴结转移[J]. 中华临床医师杂志(电子版), 2024, 18(06): 548-554.
[15] 曹磊, 邵轶普, 张志中, 王晨潮, 孙开文, 董阳, 闫东明, 李红伟, 杨波. 基于遗传基因的烟雾病与烟雾综合征生物信息学分析机制研究[J]. 中华脑血管病杂志(电子版), 2024, 18(04): 350-356.
阅读次数
全文


摘要