切换至 "中华医学电子期刊资源库"

中华结直肠疾病电子杂志 ›› 2024, Vol. 13 ›› Issue (03) : 217 -228. doi: 10.3877/cma.j.issn.2095-3224.2024.03.007

论著

基于卷积神经网络实现结直肠息肉的实时检测与自动NICE分型(附视频)
陈健1, 周静洁1, 夏开建2, 王甘红3, 刘罗杰1, 徐晓丹1,()   
  1. 1. 215500 苏州,常熟市第一人民医院(苏州大学附属常熟医院)消化内科
    2. 215500 苏州,常熟市医学人工智能与大数据重点实验室
    3. 215500 苏州,常熟市中医院消化内科
  • 收稿日期:2023-11-22 出版日期:2024-06-25
  • 通信作者: 徐晓丹
  • 基金资助:
    苏州市科技发展计划项目(SLT2023006); 常熟市医学人工智能与大数据重点实验室能力提升项目(CYZ202301); 常熟市医药卫生科技计划项目(CSWS202316)

Implementing real-time detection and automatic NICE classification of colorectal polyps using convolutional neural networks

Jian Chen1, Jingjie Zhou1, Kaijian Xia2, Ganhong Wang3, Luojie Liu1, Xiaodan Xu1,()   

  1. 1. Department of Gastroenterology, Changshu First People's Hospital (Changshu Hospital Affiliated to Soochow University), Suzhou 215500, China
    2. Changshu Key Laboratory of Medical Artificial Intelligence and Big Data, Suzhou 215500, China
    3. Department of Gastroenterology, Changshu Traditional Chinese Medicine Hospital, Suzhou 215500, China
  • Received:2023-11-22 Published:2024-06-25
  • Corresponding author: Xiaodan Xu
引用本文:

陈健, 周静洁, 夏开建, 王甘红, 刘罗杰, 徐晓丹. 基于卷积神经网络实现结直肠息肉的实时检测与自动NICE分型(附视频)[J/OL]. 中华结直肠疾病电子杂志, 2024, 13(03): 217-228.

Jian Chen, Jingjie Zhou, Kaijian Xia, Ganhong Wang, Luojie Liu, Xiaodan Xu. Implementing real-time detection and automatic NICE classification of colorectal polyps using convolutional neural networks[J/OL]. Chinese Journal of Colorectal Diseases(Electronic Edition), 2024, 13(03): 217-228.

目的

本研究旨在开发一个能够自动定位息肉并按NICE分型的深度学习目标检测模型,以促进更有效的诊断和治疗规划。

方法

收集2018年1月至2023年6月来自苏州大学附属常熟医院、常熟市中医院和常熟市辛庄人民医院的4个结肠息肉数据集,包括静态图像和视频。所有样本经病理学证实并按NICE分型分类。图像使用LabelMe工具进行标注,随后转换成MSCOCO格式以适配深度学习模型训练。采用预训练的Faster R-CNN模型,结合实时数据增强和多种图像处理技术,通过迁移学习策略进行模型训练。模型的性能评估遵循COCO标准,关注交并比(IoU)、平均精度(AP)和召回率。此外,对模型在NICE分型中的识别能力进行了详细评估。

结果

分析1 835例患者的2 248个结直肠息肉,所有息肉经病理学证实,并按NICE分型分类,具体为NICE 1型575例,NICE 2型1 143例,和NICE 3型530例。通过迁移学习技术,开发了基于ResNet-50和ResNet-101骨干网络的Faster R-CNN模型,其中以ResNet-101为基础的版本被命名为Faster R-CNN-NICE。性能分析显示,虽然Faster R-CNN-NICE模型在处理速度上略有下降(减少1.72帧/秒),但在边界框平均精度(bbox_mAP达0.542)和息肉分类综合平均精度(mAP为0.830)上显著提升。与内镜医生相比,此模型在大部分情况下展示了更高的置信度和准确性,特别是在NICE 2型息肉的预测中,其性能与医生的预测结果的差异存在统计学意义(χ2=4.30,P<0.05)。模型转换为ONNX格式后,在不同硬件环境下显示了优化的执行效率,并在视频实时检测中有效地实现了息肉的快速定位和精确分类。

结论

本研究根据NICE分型系统构建了一个结肠息肉图像数据集,并开发出Faster R-CNN-NICE深度学习模型,有效地实现了结肠息肉的即时定位与精确鉴别。这项技术预期将为内镜医生的临床工作提供支持,显著提升诊断的效率和准确度。

Objective

This study aims to develop a deep learning object detection model capable of autonomously locating polyps and classifying them according to the NICE classification, thereby facilitating more effective diagnostic and treatment planning.

Methods

This study utilized four colorectal polyp datasets collected from Changshu Hospital Affiliated to Soochow University, Changshu Traditional Chinese Medicine Hospital and Changshu Xinzhuang People's Hospital from January 2018 to June 2023, comprising both static images and videos. All samples were pathologically confirmed and classified according to the NICE typology. The images were annotated using the LabelMe tool and subsequently converted to MSCOCO format to suit deep learning model training. We employed a pre-trained Faster R-CNN model, combined with real-time data augmentation and various image processing techniques, and trained the model using a transfer learning strategy. Model performance was evaluated following the COCO standards, focusing on Intersection over Union (IoU), Average Precision (AP), and recall rate. Additionally, the model's capability in NICE classification recognition was thoroughly assessed.

Results

We analyzed 2 248 colorectal polyps from 1 835 patients, all pathologically confirmed and classified as 575 NICE type 1, 1 143 NICE type 2, and 530 NICE type 3. Using transfer learning techniques, Faster R-CNN models based on ResNet-50 and ResNet-101 backbone networks were developed, with the ResNet-101-based version named Faster R-CNN-NICE. Performance analysis showed that although the Faster R-CNN-NICE model experienced a slight decrease in processing speed (reduction by 1.72 frames per second), it significantly improved in bounding box average precision (bbox_mAP reaching 0.542) and comprehensive average precision for polyp classification (mAP at 0.830). Compared to endoscopists, this model demonstrated higher confidence and accuracy in most cases, particularly in predicting NICE type 2 polyps, where its performance significantly differed from that of the doctors (χ2=4.30, P<0.05). After conversion to the ONNX format, the model exhibited optimized execution efficiency in various hardware environments and effectively achieved rapid localization and accurate classification of polyps in real-time video detection.

Conclusion

This research constructed a colon polyp image dataset based on the NICE classification system and developed a Faster R-CNN-NICE deep learning model, which effectively achieves immediate localization and precise identification of colon polyps. This technology is expected to support the clinical work of endoscopists, significantly enhancing the efficiency and accuracy of diagnoses.

图1 数据集中息肉图像示例。1A~5A:NICE 1型息肉;1B~5B:NICE 2型息肉;1C~5C:NICE 3型息肉
图2 数据集图像特征分析。2A:数据集图像尺寸分布;2B:各类别图像的分布情况
表1 结肠直肠病变的特征[例(%)]
图3 基于ResNet-50(3A、3B)和ResNet-101(3C、3D)骨干网络的Faster R-CNN模型在训练过程中的性能指标。3A和3C部分展示了损失函数的变化;3B和3D部分展示了准确率的变化
图4 基于ResNet-50和ResNet-101骨干网络的Faster R-CNN在不同指标上的整体性能
表2 不同骨干网络的Faster R-CNN模型在测试集上的分类评估性能
图5 人工智能模型与内镜医师诊断性能对比。左图:Faster R-CNN-NICE模型的预测。右图:内镜医师的预测。5A:NICE 1型息肉;5B:NICE 2型息肉;5C:NICE 3型息肉
图6 人工智能模型与不同资历内镜医师在外部测试集的诊断性能对比。注:准确率(Accuracy),精确度(Precision),召回率(Recall),F1分数(F1-score)
图7 使用Faster R-CNN-NICE模型预测新图像的结果。7A、7B:标注为NICE 1型的息肉;7C、7D:标注为NICE 2型的息肉;7E、7F:标注为NICE 3型的息肉
[1]
Ferlitsch M, Moss A, Hassan C, et al. Colorectal polypectomy and endoscopic mucosal resection (EMR): European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline[J]. Endoscopy, 2017, 49(3): 270-297.
[2]
Ono H, Yao K, Fujishiro M, et al. Guidelines for endoscopic submucosal dissection and endoscopic mucosal resection for early gastric cancer (second edition)[J]. Dig Endosc, 2021, 33(1): 4-20.
[3]
Tontini GE, Neumann H. Endoscopic classification for colorectal tumors using narrow-band imaging[J]. Dig Endosc, 2016, 28(5): 537-538.
[4]
Wang Y, Li W, Wang Y, et al. Diagnostic performance of narrow-band imaging international colorectal endoscopic and Japanese narrow-band imaging expert team classification systems for colorectal cancer and precancerous lesions[J]. World J Gastrointest Oncol, 2021, 13(1): 58-68.
[5]
Zhuo X, Lu Y, Sun J. IDDF2021-ABS-0082 Application and learning curve of nice classification for colorectal polyps under non-magnifying endoscopy[J]. Gut, 2021, 70(Suppl. 2): A111.
[6]
Gong EJ, Bang CS, Lee JJ, et al. No-code platform-based deep-learning models for prediction of colorectal polyp histology from white-light endoscopy images: development and performance verification[J]. J Pers Med, 2022, 12(6): 963.
[7]
Garcia-Rodriguez A, Tudela Y, Cordova H, et al. In vivo computer-aided diagnosis of colorectal polyps using white light endoscopy[J]. Endosc Int Open, 2022, 10(9): E1201-E1207.
[8]
Wang P, Liu P, Glissen Brown JR, et al. Lower adenoma miss rate of computer-aided detection-assisted colonoscopy vs routine white-light colonoscopy in a prospective tandem study[J]. Gastroenterology, 2020,159(4): 1252-1261.
[9]
Hassan C, East J, Radaelli F, et al. Bowel preparation for colonoscopy: European Society of Gastrointestinal Endoscopy (ESGE) Guideline - Update 2019[J]. Endoscopy, 2019, 51(8): 775-794.
[10]
Tanaka S, Sano Y. Aim to unify the narrow band imaging (NBI) magnifying classification for colorectal tumors: current status in Japan from a summary of the consensus symposium in the 79th Annual Meeting of the Japan Gastroenterological Endoscopy Society[J]. Dig Endosc, 2011, 23(Suppl. 1): 131-139.
[11]
Russell BC, Torralba A, Murphy KP, et al. LabelMe: a database and web-based tool for image annotation[J]. IJCV, 2008, 77(1): 157-173.
[12]
Athalye C, Arnaout R. Domain-guided data augmentation for deep learning on medical imaging[J]. PloS One, 2023, 18(3): e282532.
[13]
Kang LW, Wang IS, Chou KL, et al. Image-based real-time fire detection using deep learning with data augmentation for vision-based surveillance applications[C]. Taipei: 16th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2019.
[14]
Qiu Z, Rong S, Ye L. YOLF-ShipPnet: improved retinanet with pyramid vision transformer[J]. Int J Comput Int Sys, 2023, 16(1): 58.
[15]
Shin H, Roth HR, Gao M, et al. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning[J]. IEEE transactions on medical imaging, 2016, 35(5): 1285-1298.
[16]
Ren S, He K, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6): 1137-1149.
[17]
Berbís MA, Aneiros-Fernández J, Mendoza Olivares FJ, et al. Role of artificial intelligence in multidisciplinary imaging diagnosis of gastrointestinal diseases[J]. WJG, 2021, 27(27): 4395-4412.
[18]
Yang R, Yu Y. Artificial convolutional neural network in object detection and semantic segmentation for medical imaging analysis[J]. Front Oncol, 2021, 11: 638182.
[19]
Rezatofighi H, Tsoi N, Gwak J, et al. Generalized intersection over union: a metric and a loss for bounding box regression[C]. Long Beach, CA: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[20]
Hewett DG, Kaltenbach T, Sano Y, et al. Validation of a simple classification system for endoscopic diagnosis of small colorectal polyps using narrow-band imaging[J]. Gastroenterology, 2012, 143(3): 599-607.
[21]
Chang JY. Artificial intelligence-based colorectal polyp histology prediction using narrow-band image-magnifying colonoscopy: a stepping stone for clinical practice[J]. Clin Endosc, 2022, 55(5): 699-700.
[22]
Urban G, Tripathi P, Alkayali T, et al. Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy[J]. Gastroenterology, 2018, 155(4): 1069-1078.
[23]
Gkionis IG, Flamourakis ME, Tsagkataki ES, et al. Multidimensional analysis of the learning curve for laparoscopic colorectal surgery in a regional hospital: the implementation of a standardized surgical procedure counterbalances the lack of experience[J]. BMC Surgery, 2020, 20(1): 308.
[24]
Chen H, Wang R, Du J, et al. Feature refinement method based on the two-stage detection framework for similar pest detection in the field[J]. Insects, 2023, 14(10): 819.
[1] 李洋, 蔡金玉, 党晓智, 常婉英, 巨艳, 高毅, 宋宏萍. 基于深度学习的乳腺超声应变弹性图像生成模型的应用研究[J/OL]. 中华医学超声杂志(电子版), 2024, 21(06): 563-570.
[2] 罗刚, 泮思林, 孙玲玉, 李志新, 陈涛涛, 乔思波, 庞善臣. 一种新型语义网络分析模型对室间隔完整型肺动脉闭锁和危重肺动脉瓣狭窄胎儿右心发育不良程度的评价作用[J/OL]. 中华医学超声杂志(电子版), 2024, 21(04): 377-383.
[3] 赫兰, 杨泽堃, 张颖, 王玉东, 陈伟导, 王一同, 申锷. 双输入BCNN-ResNet模型对超声颈动脉斑块稳定性的分类诊断价值[J/OL]. 中华医学超声杂志(电子版), 2024, 21(02): 137-142.
[4] 孔德铭, 刘铮, 李睿, 钱文伟, 王飞, 蔡道章, 柴伟. 人工智能辅助全髋关节置换三维术前规划准确性评价[J/OL]. 中华关节外科杂志(电子版), 2024, 18(04): 431-438.
[5] 张嘉炜, 王瑞, 张克诚, 易磊, 周增丁. 烧烫伤创面深度智能检测模型P-YOLO的建立及测试效果[J/OL]. 中华损伤与修复杂志(电子版), 2024, 19(05): 379-385.
[6] 叶莉, 杜宇. 深度学习在牙髓根尖周病临床诊疗中的应用[J/OL]. 中华口腔医学研究杂志(电子版), 2024, 18(06): 351-356.
[7] 杨建波, 马欢, 黄小梅, 刘华柱. 结肠镜辅助下EMR、CSP和RFA术治疗直径<1cm结直肠息肉的疗效和安全性比较[J/OL]. 中华普外科手术学杂志(电子版), 2025, 19(01): 76-79.
[8] 杜彦斌, 黄涛, 寇天阔, 石英. 双镜联合根治术与腹腔镜根治术在早期结肠癌患者中的应用效果[J/OL]. 中华普外科手术学杂志(电子版), 2024, 18(03): 275-278.
[9] 黄俊龙, 李文双, 李晓阳, 刘柏隆, 陈逸龙, 丘惠平, 周祥福. 基于盆底彩超的人工智能模型在女性压力性尿失禁分度诊断中的应用[J/OL]. 中华腔镜泌尿外科杂志(电子版), 2024, 18(06): 597-605.
[10] 犹成亿, 尤恒, 叶东樊, 张雯, 刘禹, 王仁宇, 苏琳茜, 甘慧, 徐智. 基于3D Res U-Net-Faster RCNN 技术和CT 影像学特征的肺结节性质预测模型的建立[J/OL]. 中华肺部疾病杂志(电子版), 2024, 17(05): 673-679.
[11] 赵毅, 李昶田, 唐文博, 白雪婷, 刘荣. 腹腔镜术中超声主胰管自动识别模型的临床应用[J/OL]. 中华腔镜外科杂志(电子版), 2024, 17(05): 290-294.
[12] 潘清, 葛慧青. 基于机械通气波形大数据的人机不同步自动监测方法[J/OL]. 中华重症医学电子杂志, 2024, 10(04): 399-403.
[13] 丁亚琴, 方旭, 戴彦成. 基于紧密型医联体的“一免三优先政策”在基层消化道肿瘤筛查中的应用探讨[J/OL]. 中华消化病与影像杂志(电子版), 2024, 14(06): 481-484.
[14] 孙铭远, 褚恒, 徐海滨, 张哲. 人工智能应用于多发性肺结节诊断的研究进展[J/OL]. 中华临床医师杂志(电子版), 2024, 18(08): 785-790.
[15] 林一鑫, 董晶, 贾建文, 黄菊梅, 武军元, 王双坤, 柳云鹏, 汪阳. 基于人工智能分析颈内动脉颅外段迂曲特征及对称性的应用性评价[J/OL]. 中华脑血管病杂志(电子版), 2024, 18(03): 202-209.
阅读次数
全文


摘要


AI


AI小编
你好!我是《中华医学电子期刊资源库》AI小编,有什么可以帮您的吗?