Abstract: Objective To propose a deep learning target detection model AM- YOLO that integrates coordinate attention and efficient attention mechanism. Methods Mosaic image enhancement and MixUp mixed-class enhancement were used for image preprocessing. In the target detection model YOLOv5s with One-Stage structure and modified backbone network and neck network, the maximum pooling layer of the spatial pyramid of the backbone network was replaced with a two-dimensional maximum pooling layer, and the coordinate attention mechanism and the efficient channel attention mechanism were integrated into the C3 module and the backbone network of the model, respectively. The improved model was compared with the unmodified YOLOv5s model, YOLOv3 model, YOLOv3-SPP model, and YOLOv3-tiny model for relevant algorithmic indicators in comparative experiments. Results The AM-YOLO model incorporating coordinate attention and efficient channel attention mechanism effectively improved the accuracy of melanoma recognition with also a reduced size of the model weight. This model showed significantly better performance than other models in terms of precision, recall rate and mean average precision, and its mean average precision for benign and malignant melanoma reached 92.8% and 87.1%, respectively. Conclusion The deep learning-based target object detection algorithm model can be applied in recognition of melanoma targets.

Key words: deep learning; object detection; attention mechanism; YOLOv5s; melanoma