切换至 "中华医学电子期刊资源库"

中华结直肠疾病电子杂志 ›› 2024, Vol. 13 ›› Issue (03) : 217 -228. doi: 10.3877/cma.j.issn.2095-3224.2024.03.007

论著

基于卷积神经网络实现结直肠息肉的实时检测与自动NICE分型(附视频)
陈健1, 周静洁1, 夏开建2, 王甘红3, 刘罗杰1, 徐晓丹1,()   
  1. 1. 215500 苏州,常熟市第一人民医院(苏州大学附属常熟医院)消化内科
    2. 215500 苏州,常熟市医学人工智能与大数据重点实验室
    3. 215500 苏州,常熟市中医院消化内科
  • 收稿日期:2023-11-22 出版日期:2024-06-25
  • 通信作者: 徐晓丹
  • 基金资助:
    苏州市科技发展计划项目(SLT2023006); 常熟市医学人工智能与大数据重点实验室能力提升项目(CYZ202301); 常熟市医药卫生科技计划项目(CSWS202316)

Implementing real-time detection and automatic NICE classification of colorectal polyps using convolutional neural networks

Jian Chen1, Jingjie Zhou1, Kaijian Xia2, Ganhong Wang3, Luojie Liu1, Xiaodan Xu1,()   

  1. 1. Department of Gastroenterology, Changshu First People's Hospital (Changshu Hospital Affiliated to Soochow University), Suzhou 215500, China
    2. Changshu Key Laboratory of Medical Artificial Intelligence and Big Data, Suzhou 215500, China
    3. Department of Gastroenterology, Changshu Traditional Chinese Medicine Hospital, Suzhou 215500, China
  • Received:2023-11-22 Published:2024-06-25
  • Corresponding author: Xiaodan Xu
引用本文:

陈健, 周静洁, 夏开建, 王甘红, 刘罗杰, 徐晓丹. 基于卷积神经网络实现结直肠息肉的实时检测与自动NICE分型(附视频)[J]. 中华结直肠疾病电子杂志, 2024, 13(03): 217-228.

Jian Chen, Jingjie Zhou, Kaijian Xia, Ganhong Wang, Luojie Liu, Xiaodan Xu. Implementing real-time detection and automatic NICE classification of colorectal polyps using convolutional neural networks[J]. Chinese Journal of Colorectal Diseases(Electronic Edition), 2024, 13(03): 217-228.

目的

本研究旨在开发一个能够自动定位息肉并按NICE分型的深度学习目标检测模型,以促进更有效的诊断和治疗规划。

方法

收集2018年1月至2023年6月来自苏州大学附属常熟医院、常熟市中医院和常熟市辛庄人民医院的4个结肠息肉数据集,包括静态图像和视频。所有样本经病理学证实并按NICE分型分类。图像使用LabelMe工具进行标注,随后转换成MSCOCO格式以适配深度学习模型训练。采用预训练的Faster R-CNN模型,结合实时数据增强和多种图像处理技术,通过迁移学习策略进行模型训练。模型的性能评估遵循COCO标准,关注交并比(IoU)、平均精度(AP)和召回率。此外,对模型在NICE分型中的识别能力进行了详细评估。

结果

分析1 835例患者的2 248个结直肠息肉,所有息肉经病理学证实,并按NICE分型分类,具体为NICE 1型575例,NICE 2型1 143例,和NICE 3型530例。通过迁移学习技术,开发了基于ResNet-50和ResNet-101骨干网络的Faster R-CNN模型,其中以ResNet-101为基础的版本被命名为Faster R-CNN-NICE。性能分析显示,虽然Faster R-CNN-NICE模型在处理速度上略有下降(减少1.72帧/秒),但在边界框平均精度(bbox_mAP达0.542)和息肉分类综合平均精度(mAP为0.830)上显著提升。与内镜医生相比,此模型在大部分情况下展示了更高的置信度和准确性,特别是在NICE 2型息肉的预测中,其性能与医生的预测结果的差异存在统计学意义(χ2=4.30,P<0.05)。模型转换为ONNX格式后,在不同硬件环境下显示了优化的执行效率,并在视频实时检测中有效地实现了息肉的快速定位和精确分类。

结论

本研究根据NICE分型系统构建了一个结肠息肉图像数据集,并开发出Faster R-CNN-NICE深度学习模型,有效地实现了结肠息肉的即时定位与精确鉴别。这项技术预期将为内镜医生的临床工作提供支持,显著提升诊断的效率和准确度。

Objective

This study aims to develop a deep learning object detection model capable of autonomously locating polyps and classifying them according to the NICE classification, thereby facilitating more effective diagnostic and treatment planning.

Methods

This study utilized four colorectal polyp datasets collected from Changshu Hospital Affiliated to Soochow University, Changshu Traditional Chinese Medicine Hospital and Changshu Xinzhuang People's Hospital from January 2018 to June 2023, comprising both static images and videos. All samples were pathologically confirmed and classified according to the NICE typology. The images were annotated using the LabelMe tool and subsequently converted to MSCOCO format to suit deep learning model training. We employed a pre-trained Faster R-CNN model, combined with real-time data augmentation and various image processing techniques, and trained the model using a transfer learning strategy. Model performance was evaluated following the COCO standards, focusing on Intersection over Union (IoU), Average Precision (AP), and recall rate. Additionally, the model's capability in NICE classification recognition was thoroughly assessed.

Results

We analyzed 2 248 colorectal polyps from 1 835 patients, all pathologically confirmed and classified as 575 NICE type 1, 1 143 NICE type 2, and 530 NICE type 3. Using transfer learning techniques, Faster R-CNN models based on ResNet-50 and ResNet-101 backbone networks were developed, with the ResNet-101-based version named Faster R-CNN-NICE. Performance analysis showed that although the Faster R-CNN-NICE model experienced a slight decrease in processing speed (reduction by 1.72 frames per second), it significantly improved in bounding box average precision (bbox_mAP reaching 0.542) and comprehensive average precision for polyp classification (mAP at 0.830). Compared to endoscopists, this model demonstrated higher confidence and accuracy in most cases, particularly in predicting NICE type 2 polyps, where its performance significantly differed from that of the doctors (χ2=4.30, P<0.05). After conversion to the ONNX format, the model exhibited optimized execution efficiency in various hardware environments and effectively achieved rapid localization and accurate classification of polyps in real-time video detection.

Conclusion

This research constructed a colon polyp image dataset based on the NICE classification system and developed a Faster R-CNN-NICE deep learning model, which effectively achieves immediate localization and precise identification of colon polyps. This technology is expected to support the clinical work of endoscopists, significantly enhancing the efficiency and accuracy of diagnoses.

图1 数据集中息肉图像示例。1A~5A:NICE 1型息肉;1B~5B:NICE 2型息肉;1C~5C:NICE 3型息肉
图2 数据集图像特征分析。2A:数据集图像尺寸分布;2B:各类别图像的分布情况
表1 结肠直肠病变的特征[例(%)]
图3 基于ResNet-50(3A、3B)和ResNet-101(3C、3D)骨干网络的Faster R-CNN模型在训练过程中的性能指标。3A和3C部分展示了损失函数的变化;3B和3D部分展示了准确率的变化
图4 基于ResNet-50和ResNet-101骨干网络的Faster R-CNN在不同指标上的整体性能
表2 不同骨干网络的Faster R-CNN模型在测试集上的分类评估性能
图5 人工智能模型与内镜医师诊断性能对比。左图:Faster R-CNN-NICE模型的预测。右图:内镜医师的预测。5A:NICE 1型息肉;5B:NICE 2型息肉;5C:NICE 3型息肉
图6 人工智能模型与不同资历内镜医师在外部测试集的诊断性能对比。注:准确率(Accuracy),精确度(Precision),召回率(Recall),F1分数(F1-score)
图7 使用Faster R-CNN-NICE模型预测新图像的结果。7A、7B:标注为NICE 1型的息肉;7C、7D:标注为NICE 2型的息肉;7E、7F:标注为NICE 3型的息肉
[1]
Ferlitsch M, Moss A, Hassan C, et al. Colorectal polypectomy and endoscopic mucosal resection (EMR): European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline[J]. Endoscopy, 2017, 49(3): 270-297.
[2]
Ono H, Yao K, Fujishiro M, et al. Guidelines for endoscopic submucosal dissection and endoscopic mucosal resection for early gastric cancer (second edition)[J]. Dig Endosc, 2021, 33(1): 4-20.
[3]
Tontini GE, Neumann H. Endoscopic classification for colorectal tumors using narrow-band imaging[J]. Dig Endosc, 2016, 28(5): 537-538.
[4]
Wang Y, Li W, Wang Y, et al. Diagnostic performance of narrow-band imaging international colorectal endoscopic and Japanese narrow-band imaging expert team classification systems for colorectal cancer and precancerous lesions[J]. World J Gastrointest Oncol, 2021, 13(1): 58-68.
[5]
Zhuo X, Lu Y, Sun J. IDDF2021-ABS-0082 Application and learning curve of nice classification for colorectal polyps under non-magnifying endoscopy[J]. Gut, 2021, 70(Suppl. 2): A111.
[6]
Gong EJ, Bang CS, Lee JJ, et al. No-code platform-based deep-learning models for prediction of colorectal polyp histology from white-light endoscopy images: development and performance verification[J]. J Pers Med, 2022, 12(6): 963.
[7]
Garcia-Rodriguez A, Tudela Y, Cordova H, et al. In vivo computer-aided diagnosis of colorectal polyps using white light endoscopy[J]. Endosc Int Open, 2022, 10(9): E1201-E1207.
[8]
Wang P, Liu P, Glissen Brown JR, et al. Lower adenoma miss rate of computer-aided detection-assisted colonoscopy vs routine white-light colonoscopy in a prospective tandem study[J]. Gastroenterology, 2020,159(4): 1252-1261.
[9]
Hassan C, East J, Radaelli F, et al. Bowel preparation for colonoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2019[J]. Endoscopy, 2019, 51(8): 775-794.
[10]
Tanaka S, Sano Y. Aim to unify the narrow band imaging (NBI) magnifying classification for colorectal tumors: current status in Japan from a summary of the consensus symposium in the 79th Annual Meeting of the Japan Gastroenterological Endoscopy Society[J]. Dig Endosc, 2011, 23(Suppl. 1): 131-139.
[11]
Russell BC, Torralba A, Murphy KP, et al. LabelMe: a database and web-based tool for image annotation[J]. IJCV, 2008, 77(1): 157-173.
[12]
Athalye C, Arnaout R. Domain-guided data augmentation for deep learning on medical imaging[J]. PloS One, 2023, 18(3): e282532.
[13]
Kang LW, Wang IS, Chou KL, et al. Image-based real-time fire detection using deep learning with data augmentation for vision-based surveillance applications[C]. Taipei: 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2019.
[14]
Qiu Z, Rong S, Ye L. YOLF-ShipPnet: improved retinanet with pyramid vision transformer[J]. Int J Comput Int Sys, 2023, 16(1): 58.
[15]
Shin H, Roth HR, Gao M, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning[J]. IEEE transactions on medical imaging, 2016, 35(5): 1285-1298.
[16]
Ren S, He K, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6): 1137-1149.
[17]
Berbís MA, Aneiros-Fernández J, Mendoza Olivares FJ, et al. Role of artificial intelligence in multidisciplinary imaging diagnosis of gastrointestinal diseases[J]. WJG, 2021, 27(27): 4395-4412.
[18]
Yang R, Yu Y. Artificial convolutional neural network in object detection and semantic segmentation for medical imaging analysis[J]. Front Oncol, 2021, 11: 638182.
[19]
Rezatofighi H, Tsoi N, Gwak J, et al. Generalized intersection over union: a metric and a loss for bounding box regression[C]. Long Beach, CA: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[20]
Hewett DG, Kaltenbach T, Sano Y, et al. Validation of a simple classification system for endoscopic diagnosis of small colorectal polyps using narrow-band imaging[J]. Gastroenterology, 2012, 143(3): 599-607.
[21]
Chang JY. Artificial intelligence-based colorectal polyp histology prediction using narrow-band image-magnifying colonoscopy: a stepping stone for clinical practice[J]. Clin Endosc, 2022, 55(5): 699-700.
[22]
Urban G, Tripathi P, Alkayali T, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy[J]. Gastroenterology, 2018, 155(4): 1069-1078.
[23]
Gkionis IG, Flamourakis ME, Tsagkataki ES, et al. Multidimensional analysis of the learning curve for laparoscopic colorectal surgery in a regional hospital: the implementation of a standardized surgical procedure counterbalances the lack of experience[J]. BMC Surgery, 2020, 20(1): 308.
[24]
Chen H, Wang R, Du J, et al. Feature refinement method based on the two-stage detection framework for similar pest detection in the field[J]. Insects, 2023, 14(10): 819.
[1] 罗刚, 泮思林, 孙玲玉, 李志新, 陈涛涛, 乔思波, 庞善臣. 一种新型语义网络分析模型对室间隔完整型肺动脉闭锁和危重肺动脉瓣狭窄胎儿右心发育不良程度的评价作用[J]. 中华医学超声杂志(电子版), 2024, 21(04): 377-383.
[2] 赫兰, 杨泽堃, 张颖, 王玉东, 陈伟导, 王一同, 申锷. 双输入BCNN-ResNet模型对超声颈动脉斑块稳定性的分类诊断价值[J]. 中华医学超声杂志(电子版), 2024, 21(02): 137-142.
[3] 成汉林, 史中青, 戚占如, 王小贤, 曾子炀, 单淳劼, 钱隼南, 罗守华, 姚静. 基于深度学习的超声心动图动态图像切面识别研究[J]. 中华医学超声杂志(电子版), 2024, 21(02): 128-136.
[4] 刘韩, 王胰, 舒庆兰, 彭博, 尹立雪, 谢盛华. 基于深度学习的超声心动图三尖瓣反流严重程度智能评估方法研究[J]. 中华医学超声杂志(电子版), 2024, 21(02): 121-127.
[5] 王瑞, 张嘉炜, 张克诚, 周增丁. 基于深度学习的皮肤烧烫伤创面图像分割与分类及检测的研究进展[J]. 中华损伤与修复杂志(电子版), 2024, 19(02): 172-175.
[6] 杜彦斌, 黄涛, 寇天阔, 石英. 双镜联合根治术与腹腔镜根治术在早期结肠癌患者中的应用效果[J]. 中华普外科手术学杂志(电子版), 2024, 18(03): 275-278.
[7] 袁成雪, 张宗霞, 许婷, 斯郎拉姆. 三种内镜手术治疗结肠息肉的效果及安全性观察[J]. 中华普外科手术学杂志(电子版), 2024, 18(01): 78-81.
[8] 杨龙雨禾, 王跃强, 招云亮, 金溪, 卫娜, 杨智明, 张贵福. 人工智能辅助临床决策在泌尿系肿瘤的应用进展[J]. 中华腔镜泌尿外科杂志(电子版), 2024, 18(02): 178-182.
[9] 刘宝帅, 高显华, 潘受禹, 曹强坚, 刘连杰, 张卫. 家族性腺瘤性息肉病患者结直肠息肉的内镜下诊断治疗进展[J]. 中华结直肠疾病电子杂志, 2024, 13(01): 26-31.
[10] 董力, 李赫妍, 魏文斌. 人工智能在糖尿病视网膜病变中的应用进展[J]. 中华眼科医学杂志(电子版), 2024, 14(01): 57-61.
[11] 谢家兴, 李学民, 敖明昕. 人工智能在白内障诊断领域的应用进展[J]. 中华眼科医学杂志(电子版), 2023, 13(06): 361-365.
[12] 李帅, 李开南. 人工智能在骨科诊断技术中的研究进展[J]. 中华老年骨科与康复电子杂志, 2024, 10(01): 46-50.
[13] 陈健, 张子豪, 卢勇达, 夏开建, 王甘红, 刘罗杰, 徐晓丹. 基于深度学习构建结直肠息肉诊断自动分类模型[J]. 中华诊断学电子杂志, 2024, 12(01): 9-17.
[14] 王靖玺, 赵丽, 吕滨. 人工智能在肺栓塞CT检查中的临床研究进展[J]. 中华心脏与心律电子杂志, 2024, 12(01): 26-31.
[15] 胡兵. 结直肠息肉冷切[J]. 中华胃肠内镜电子杂志, 2024, 11(01): 72-72.
阅读次数
全文


摘要