A multi-feature fusion-based model for fetal orientation classification from intrapartum ultrasound videos

doi:10.12122/j.issn.1673-4254.2025.07.24

Abstract

Abstract:

Objective To construct an intelligent analysis model for classifying fetal orientation during intrapartum ultrasound videos based on multi-feature fusion. Methods The proposed model consists of the Input, Backbone Network and Classification Head modules. The Input module carries out data augmentation to improve the sample quality and generalization ability of the model. The Backbone Network was responsible for feature extraction based on Yolov8 combined with CBAM, ECA, PSA attention mechanism and AIFI feature interaction module. The Classification Head consists of a convolutional layer and a softmax function to output the final probability value of each class. The images of the key structures (the eyes, face, head, thalamus, and spine) were annotated with frames by physicians for model training to improve the classification accuracy of the anterior occipital, posterior occipital, and transverse occipital orientations. Results The experimental results showed that the proposed model had excellent performance in the tire orientation classification task with the classification accuracy reaching 0.984, an area under the PR curve (average accuracy) of 0.993, and area under the ROC curve of 0.984, and a kappa consistency test score of 0.974. The prediction results by the deep learning model were highly consistent with the actual classification results. Conclusion The multi-feature fusion model proposed in this study can efficiently and accurately classify fetal orientation in intrapartum ultrasound videos.

Key words: intrapartum ultrasound, fetal orientation, deep learning, attention mechanism

Ziyu ZHENG, Xiaying YANG, Shengjie WU, Shijie ZHANG, Guorong LYU, Peizhong LIU, Jun WANG, Shaozheng HE. A multi-feature fusion-based model for fetal orientation classification from intrapartum ultrasound videos[J]. Journal of Southern Medical University, 2025, 45(7): 1563-1570.

Figures/Tables 9

Tab.1 Data distribution

Classes	Train (Number of images)	Val (Number of images)	Video Test (number of video/number of frames)
OA	1006	125	3/482
OP	1708	213	3/491
OT	1270	158	2/346
Total	3984	496	8/1319

Fig.1 Model structure diagram.

Tab.2 Definition of the model evaluation metrics

Model evaluation metrics	Definition
ACC	$T P + T N T P + T N + F P + F N$
Precision (P)	$T P T P + F P$
Recall (R)	$T P T P + F N$
PR-AUC	The average precision, which is the average of the precision over different recall points, is shown as the area under the P-R curve on the P-R curve plot
kappa	$A c c u r a c y - p e 1 - p e p e = ∑ i (S u m o f e l e m e n t s i n r o w i * S u m o f e l e m e n t s i n c o l u m n i) T o t a l s u m o f a l l e l e m e n t s i n t h e m a t r i x 2$
ROC-AUC	The area under the ROC curve

Tab.2 Definition of the model evaluation metrics

Model evaluation metrics	Definition
ACC	$T P + T N T P + T N + F P + F N$
Precision (P)	$T P T P + F P$
Recall (R)	$T P T P + F N$
PR-AUC	The average precision, which is the average of the precision over different recall points, is shown as the area under the P-R curve on the P-R curve plot
kappa	$A c c u r a c y - p e 1 - p e p e = ∑ i (S u m o f e l e m e n t s i n r o w i * S u m o f e l e m e n t s i n c o l u m n i) T o t a l s u m o f a l l e l e m e n t s i n t h e m a t r i x 2$
ROC-AUC	The area under the ROC curve

Fig.2 Test results of the model on the validation dataset.

Fig.3 Methods for predictive statistics.

Tab.3 Classification results of videos by different models

Label	Ours	EfficientNet_b2	DenseNet169	ResNet50	Yolo8s_cls	ConvNeXt	MPViT	Swim transformer
OA1	OA	OA	OP	OA	OA	OP	OA	OT
OP1	OP	OP	OP	OP	OP	OP	OA	OP
OT1	OT	OT	OT	OT	OT	OT	OA	OT
OA2	OA	OA	OA	OA	OA	OA	OA	OT
OP2	OP	OP	OP	OP	OP	OP	OA	OP
OA3	OA	OA	OA	OA	OA	OA	OA	OT
OP3	OP	OP	OP	OP	OP	OP	OA	OT
OT3	OT	OT	OT	OT	OT	OT	OA	OT

Tab.4 Comparison of results from different models

Model	PR-AUC	ROC-AUC	ACC	kappa
EfficientNet_b2	0.92545	0.96126	0.91431	0.86892
DenseNet169	0.92836	0.96201	0.91519	0.86879
ResNet50	0.92621	0.95507	0.92307	0.88284
Yolo8s_cls	0.87686	0.94205	0.87456	0.80322
ConvNeXt	0.91083	0.94177	0.91145	0.86351
MPViT	0.47209	0.57401	0.37571	0.00162
Swim transformer	0.49753	0.62991	0.48685	0.22014
Yolov8+CBAM	0.88311	0.94350	0.90500	0.85025
Yolov8+CSAM	0.55933	0.75915	0.64312	0.42443
Ours	0.91545	0.95944	0.92499	0.88553

Fig.4 PR curves and ROC curves of different models on the test set.

Tab.5 Ablation experiment

Modules				PR-AUC	ROC-AUC	ACC	kappa
CBAM	PSA	ECA	AIFI	PR-AUC	ROC-AUC	ACC	kappa
0	0	0	0	0.87686	0.94205	0.87456	0.80322
1	0	0	0	0.88311	0.94350	0.90500	0.85025
0	1	0	0	0.91193	0.94745	0.89150	0.83678
0	0	1	0	0.89821	0.94370	0.89865	0.84478
0	0	0	1	0.86844	0.93363	0.86554	0.79148
1	1	1	1	0.91545	0.95944	0.92499	0.88553

References 33

[1]	Leung KY. Applications of advanced ultrasound technology in obstetrics[J]. Diagnostics (Basel), 2021, 11(7): 1217. doi：10.3390/diagnostics11071217
[2]	Malvasi A. Intrapartum Ultrasonography for Labor Management[M]. Cham: Springer International Publishing, 2021. doi：10.1007/978-3-030-57595-3
[3]	Yang TZ, Luo H. Practical Ultrasonography in Obstetrics and Gynecology[M]. Cham: Springer Nature Singapore, 2022. doi：10.1007/978-981-16-4477-1
[4]	Wiafe YA, Whitehead B, Venables H, et al. Comparing intrapartum ultrasound and clinical examination in the assessment of fetal head position in African women[J]. J Ultrason, 2019, 19(79): 249-54. doi：10.15557/JoU.2019.0037
[5]	张勤建, 颜建英. 产时超声辅助产程管理[J]. 中国实用妇科与产科杂志, 2024, 40(2): 142-7.
[6]	孙璐鑫, 韩英杰. 产时超声在产程监测中的应用[J]. 国际生殖健康/计划生育杂志, 2025, 44(1): 84-8.
[7]	Ghi T, Eggebø T, Lees C, et al. ISUOG Practice Guidelines: intrapartum ultrasound[J]. Ultrasound Obstet Gynecol, 2018, 52(1): 128-39. doi：10.1002/uog.19072
[8]	Lieberman E, Davidson K, Lee-Parritz A, et al. Changes in fetal position during labor and their association with epidural analgesia[J]. Obstet Gynecol, 2005, 105(5 Pt 1): 974-82. doi：10.1097/01.AOG.0000158861.43593.49
[9]	Ghi T, Dall’Asta A. Sonographic evaluation of the fetal head position and attitude during labor[J]. Am J Obstet Gynecol, 2024, 230(3S): S890-900. doi：10.1016/j.ajog.2022.06.003
[10]	Guittier MJ, Othenin-Girard V, Irion O, et al. Maternal positioning to correct occipito-posterior fetal position in labour: a randomised controlled trial[J]. BMC Pregnancy Childbirth, 2014, 14: 83. doi：10.1186/1471-2393-14-83
[11]	杨建成, 马琰, 徐颖, 等. 胎方位异常阴道分娩第二产程会阴超声测量产程进展参数的临床研究[J]. 现代妇产科进展, 2024, 33(9): 656-61.
[12]	Bertholdt C, Piffer A, Pol H, et al. Management of persistent occiput posterior position: The added value of manual rotation[J]. Int J Gynaecol Obstet, 2022, 157(3): 613-7. doi：10.1002/ijgo.13874
[13]	Vitner D, Paltieli Y, Haberman S, et al. Prospective multicenter study of ultrasound-based measurements of fetal head station and position throughout labor[J]. Ultrasound Obstet Gynecol, 2015, 46(5): 611-5. doi：10.1002/uog.14821
[14]	Senécal J, Xiong X, Fraser WD, et al. Effect of fetal position on second-stage duration and labor outcome[J]. Obstet Gynecol, 2005, 105(4): 763-72. doi：10.1097/01.aog.0000154889.47063.84
[15]	de Vries B, Phipps H, Kuah S, et al. Transverse occiput position: using manual rotation to aid normal birth and improve delivery OUTcomes (TURN-OUT): a study protocol for a randomised controlled trial[J]. Trials, 2015, 16: 362. doi：10.1186/s13063-015-0854-3
[16]	Simkin P. The fetal occiput posterior position: state of the science and a new perspective[J]. Birth, 2010, 37(1): 61-71. doi：10.1111/j.1523-536X.2009.00380.x
[17]	Fiorentino MC, Villani FP, Di Cosmo M, et al. A review on deep-learning algorithms for fetal ultrasound-image analysis[J]. Med Image Anal, 2023, 83: 102629. doi：10.1016/j.media.2022.102629
[18]	Burgos-Artizzu XP, Coronado-Gutiérrez D, Valenzuela-Alcaraz B, et al. Evaluation of deep convolutional neural networks for automatic classification of common maternal fetal ultrasound planes[J]. Sci Rep, 2020, 10(1): 10200. doi：10.1038/s41598-020-67076-5
[19]	Ghi T, Conversano F, Zegarra RR, et al. Novel artificial intelligence approach for automatic differentiation of fetal occiput anterior and non-occiput anterior positions during labor[J]. Ultrasound Obstet Gynecol, 2022, 59(1): 93-9. doi：10.1002/uog.23739
[20]	Dall’Asta A, Conversano F, Minopoli M, et al. OC02.02: Artificial intelligence for automatic classification of anterior/posterior/transverse occiput positions using transperineal sonography[J]. Ultrasound Obstet Gynecol, 2022, 60(S1): 4-5. doi：10.1002/uog.24995
[21]	Eisenstat J, Wagner MW, Vidarsson L, et al. Fet-net algorithm for automatic detection of fetal orientation in fetal MRI[J]. Bioengineering (Basel), 2023, 10(2): 140. doi：10.3390/bioengineering10020140
[22]	Zegarra RR, Conversano F, Dall’Asta A, et al. A deep learning approach to identify the fetal head position using transperineal ultrasound during labor[J]. Eur J Obstet Gynecol Reprod Biol, 2024, 301: 147-53. doi：10.1016/j.ejogrb.2024.08.012
[23]	Woo S, Park J, Lee JY, et al. CBAM: convolutional block attention module[M]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 3-19. doi：10.1007/978-3-030-01234-2_1
[24]	Wang QL, Wu BG, Zhu PF, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 13-19, 2020, Seattle, WA, USA. IEEE, 2020: 11531-9. doi：10.1109/cvpr42600.2020.01155
[25]	Wang A, Chen H, Liu LH, et al. YOLOv10: real-time end-to-end object detection[EB/OL]. 2024: 2405.14458. .
[26]	Zhao YA, Lv WY, Xu SL, et al. DETRs beat YOLOs on real-time object detection[EB/OL]. 2023: 2304.08069. . doi：10.1109/cvpr52733.2024.01605
[27]	Tan MX, Le QV. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL]. 2019: 1905.11946. . doi：10.1007/978-1-4842-6168-2_10
[28]	Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). July 21-26, 2017, Honolulu, HI, USA. IEEE, 2017: 2261-9. doi：10.1109/cvpr.2017.243
[29]	He KM, Zhang XY, Ren SQ, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). June 27-30, 2016, Las Vegas, NV, USA. IEEE, 2016: 770-8. doi：10.1109/cvpr.2016.90
[30]	Liu Z, Mao HZ, Wu CY, et al. A ConvNet for the 2020s[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-24, 2022, New Orleans, LA, USA. IEEE, 2022: 11966-76. doi：10.1109/cvpr52688.2022.01167
[31]	Lee Y, Kim J, Willette J, et al. MPViT: multi-path vision transformer for dense prediction[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 18-24, 2022, New Orleans, LA, USA. IEEE, 2022: 7277-86. doi：10.1109/cvpr52688.2022.00714
[32]	Liu Z, Lin YT, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). October 10-17, 2021, Montreal, QC, Canada. IEEE, 2021: 9992-10002. doi：10.1109/iccv48922.2021.00986
[33]	Yu Hung AL, Zheng HX, Zhao K, et al. CSAM: a 2.5D cross-slice attention module for anisotropic volumetric medical image segmentation[J]. IEEE Winter Conf Appl Comput Vis, 2024, 2024: 5911-20. doi：10.1109/wacv57701.2024.00582