南方医科大学学报

南方医科大学学报 ›› 2023, Vol. 43 ›› Issue (12): 2118-2125.doi: 10.12122/j.issn.1673-4254.2023.12.17

• • 上一篇    下一篇

基于注意力机制和多模态特征融合的猕猴脑磁共振图像全脑分割

吴雪扬，张 煜，张 华，钟 涛   

  1. 南方医科大学生物医学工程学院//广东省医学图像处理重点实验室，广东 广州 510515
  • 出版日期:2023-12-20 发布日期:2023-12-29

Whole-brain parcellation for macaque brain magnetic resonance images based on attention mechanism and multi-modality feature fusion

WU Xueyang, ZHANG Yu, ZHANG Hua, ZHONG Tao   

  1. School of Biomedical Engineering//Guangdong Provincial Key Laboratory of Medical Image Processing, Southern Medical University, Guangzhou 510515, China
  • Online:2023-12-20 Published:2023-12-29

RichHTML

11

PDF

92

摘要： 目的 提出并探讨一种新的基于注意力机制和多模态特征融合的深度学习算法（DDAM），实现对猕猴脑MRI图像的全脑分割。方法 共收集68例年龄分布在13~36月的多模态猕猴脑MRI图像数据，且均包含对应的真实标签。针对多模态数据信息复杂且互补的特点，采用多编码器结构分别适应不同模态并进行特征提取。在解码器部分引入注意力机制构建多模态特征融合模块（AMFF），利用模态间信息丰富且互补的特点，充分融合不同尺度和复杂度的多模态特征，进而提升分割性能。另外，进行消融实验分析并对结果进行统计学检验。结果 多编码器结构以及注意力机制的引入能够有效地提升模型对多模态特征的融合能力，使得猕猴数据的全脑分割平均DSC达到0.904，ASD低至0.131（P<0.05）。消融实验结果验证了DDAM方法各组成部分的有效性。结论 本文针对多模态数据特点构建深度学习算法模型，提出的DDAM方法，能够更有效地提取并融合多模态特征，从而实现全脑分割精度的显著提高。

关键词: 猕猴大脑；磁共振全脑分割；深度学习；注意力机制；多模态特征融合

Abstract: Objective We propose a novel deep learning algorithm based on attention mechanism and multimodality feature fusion (DDAM) to achieve whole-brain parcellation of macaque brain magnetic resonance (MR) images. Methods We collected multimodality brain MR images of 68 macaques (aged 13 to 36 months) with corresponding ground truth labels. To address the complexity and complementary nature of the multimodality data, we employed a multi- encoder structure to adapt to different modalities and performed feature extraction. In the decoder, an attention mechanism was introduced to construct the Attention-based Multimodality Feature Fusion module (AMFF). Leveraging the rich and complementary information between modalities, AMFF effectively integrated multimodality features of varying scales and complexities to enhance the performance of parcellation. We conducted ablation experiments and statistically analyzed the results. Results The incorporation of the multi-encoder structure and attention mechanism significantly improved the performance of the network for integrating the multimodality features, and achieved an average DSC of 0.904 and an ASD as low as 0.131 for macaque whole-brain parcellation (P<0.05). The ablation experiments validated the effectiveness of each component of the DDAM method. Conclusion The proposed DDAM method, with its enhanced ability to extract and integrate multimodality features, can significantly improve the accuracy of whole-brain MR image segmentation.

Key words: macaque brain; magnetic resonance imaging; whole-brain parcellation; deep learning; attention mechanism; multi-modality feature fusion