Defect Detection of Steel Wire Rope in Coal Mine Based on Improved YOLOv5 Deep Learning

1. Introduction

As the most crucial lifting system component in the coal mine, wire ropes are the guarantee line of safety production in coal mine. Due to the harsh working environment in the mine and because steel wire ropes maintain a high strength stress state for a long duration, in the working process, the steel wire ropes from the components such as the pulley and derrick produce friction [1]; therefore, it is exceedingly crucial to test the health of the steel wire ropes before utilizing them. In 2010, Hu and Xu [2] found that artificial visual inspection methods were backward and could not discover the potential safety hazards in a timely manner. Therefore, they proposed a detection probe based on weak magnetic excitation technology, thereby realizing the qualitative and semi-quantitative detection pertaining to the defects of steel wire ropes utilized in coal mine. Wang et al. [3] proposed a novel probe for wire rope detection in 2017, which utilized a Hall element array combined with magnetic focusing technology to realize qualitative detection of wire rope defects. In 2021, Zhang [4] realized the defect detection of steel wire ropes by utilizing the axial action of external magnetic field to influence its permeability.

By analyzing the defect detection methods of steel wire ropes utilized in coal mine, it is found that the magnetization characteristics of steel wire ropes are utilized to achieve detection, and that the external magnetic field or other interference factors are generally applied to them, which may affect their normal working state. Herein, a defect detection method based on computer vision and deep learning image detection is proposed to realize the defect detection of steel wire ropes utilized in coal mine. This method does not need to impose external factors on steel wire ropes; moreover, it does not interfere with the production process and can effectively detect the defect state of steel wire ropes, which provides a novel solution for the safety production inspection in coal mine.

2. Defect Type of Steel Wire Rope Utilized in Coal Mine

Due to the harsh production environment in coal mines, steel wire rope defects are occasioned by various factors, which not only affect the service life of steel wire rope, but also endanger the life safety of workers [5]. Mechanical stretching is a crucial causative factor for defects that affect the process of use, which may lead to many problems such as wire rope breakage and wear. Fig. 1 shows the common defects of steel wire ropes utilized in coal mines.

2.1 Abrasion

In the working process of wire ropes, the friction between the outer rope circumference and the brackets or channels produces some wear. As the contact surface between the rope and the support is generally fixed, one-sided wear occurs in long-term use, and the one-sided wear of wire ropes leads to the unbalanced force of a certain section and leads to fracture in the middle. In the use process, regular reversing is often utilized to extend the service life. Although it can balance the force, long-term wear makes wire ropes thinner, thereby making it unable to attain the safe use standard. The other scenario is explained as follows: when wire ropes are bent or rolled, the bending component is squeezed, and the load is concentrated there, occasioning friction between the wires and wear inside wire ropes, which leads to broken wires inside. Long-term wear of steel wire ropes can critically affect their service life and occasion immense safety risks.

2.2 Broken Wires

There are two main reasons for wire rope fractures: (1) when wire ropes leave the factory, fractures are occasioned by the manufacturing process or production error, and (2) in the use process, fractures are occasioned by severe wear, overload, and bending fatigue because the mechanical strength of wire ropes becomes overloaded. In regard to the production of wire fractures, the performance of wire ropes must be tested before use to avoid wire fractures. Wire fractures during use is an urgent research topic.

In addition, corroded and worn wires are also two common defects. In the process of use, factors such as moisture may lead to the chemical corrosion of the wire rope. Long-term deterioration of such corrosion can lead to the fracture of the wire rope. Through observation, there are apparent differences in wire rope fracture occasioned by different causes of wire breakage. For example, the section formed by wire breakage due to overload exhibits an oblique stubble shape and flat fracture. In summary, to ensure the safety of coal mine production, it is necessary to detect the fault before the fault occurs in the use of wire rope. Therefore, this study presents a reliable scheme for wire rope detection before use.

3. Design of the Wire Rope Detection System

3.1 Test Platform Design

Fig. 2 indicates the design scheme of the steel wire rope defect detection platform in a coal mine. It mainly includes a wire rope transmission device, camera, a lighting device, an embedded system, and the whole bracket and other components. The wire rope conveying device is assembled outside the mine head and moves the wire rope gently in front of the wellhead through a parallel guide rail. The lighting system can ensure that the picture collected by the camera is evenly illuminated and provide enough brightness for detection at night, thereby avoiding the interference occasioned by shadows or dark environments on detection. The embedded system supports image processing, the execution of deep learning programs, and the output of recognition results. The system exhibits strong stability and antiinterference, and can adapt to the harsh working environment in the coal mine. The support component utilizes I-beam steel to provide stable support for each component, thereby avoiding unnecessary shaking and error in the detection process.

3.2 Detection System Design

Fig. 3 illustrates the overall block diagram of the steel wire rope defect detection system utilized in coal mines. The figure indicates that the images collected by the camera are transmitted to the deep learning network in the embedded system through the acquisition card transcoding, and that the real-time detection is conducted, and the results are transmitted to the host computer in the monitoring room to provide feedback on the state of the wire rope. The camera’s sampling frequency can be adjusted according to the motion speed of the wire rope or the detection requirements. Although when the sampling frequency is higher, the detection precision that can be achieved is higher, the system also requires strong computing power support.

4. Coal Mine Steel Wire Rope Defect Detection based on YOLOv5

4.1 YOLOv5 Deep Learning Network Model

4.1.1 YOLO introduction of algorithm

You Only Look Once (YOLO) is an image recognition algorithm based on deep learning one-stage series of algorithms with exceedingly advantageous recognition speed and excellent target recognition ability [6]. YOLOv5 adds features such as Mosaic enhancements and adaptive anchor frames. The onestage YOLOv5 network model is selected as the detection algorithm of the proposed system, which can immensely relieve the pressure attributed to utilizing computing equipment with finite edge power.

4.1.2 Enhanced YOLOv5 model

According to the International Society for Optics and Photonics (SPIE), in a broad perspective, an area ratio of less than 0.12% of the whole image is considered as a small target. However, small targets generally exhibit peculiar characteristics, namely low clarity, small area, and insufficient information, and they are easily confused with other features and noise interference. Therefore, small targets are difficult to analyze using traditional feature extraction algorithms, which creates a scenario in which it is exceedingly difficult for neural networks to learn their features. According to the problems existing in small target detection, this study adopts the YOLOv5 model as the basis and optimizes it for the problem of wire rope defect detection. An enhanced multi-scale model based on YOLOv5 is proposed, which adds a feature extraction scale on the basis of the original network; therefore, it can adapt to smaller detection targets and overcome the shortcomings of small target detection scale. The overall structure of the enhanced network is depicted in Fig. 4.

1) Network backbone

The optimized YOLOv5 is composed of the Backbone, the Neck, and the Head. The Backbone consists of the Focus, CSPNet, CBL, and SPP structure (Fig. 5).

The Focus structure slices the input image and subsequently performs a convolution to obtain the feature map of 320×320×32. This structure effectively reduces the feature loss and enhances the operation speed.

CSPNet is composed of residual components, which integrates gradient variation into the feature graph to solve the gradient redundancy problem and reduce the number of parameters.

The CBL usually refers to the convolution-BN-activation function, which is a basic down-sampling module in the model.

The SPP utilizes pooling layers of different kernel sizes to extract features of different scales, and subsequently utilizes the obtained features in the form of spatial pyramids to achieve multi-scale feature extraction.

2) Network neck and head

The combination of the Neck and the Head of the network realizes multi-scale prediction. With the enhanced Pass Aggregation Network (PANet) structure in the neck, the features are utilized in a new superposition manner, which endows the model with a multi-scale target detection ability. The network’s head is the key detection module. It chooses the feature map at the end of the model as the base, and utilizes anchor boxes to integrate category information and confidence into output results, thus realizing target detection.

3) Expansion of scale

The enhanced YOLO network utilized herein adds a scale to the original one, namely 160×160. The main purpose of adding this scale is to solve the detection needs of small target defects. Herein, an enhanced multi-scale model based on YOLOv5 is proposed, which adds a feature extraction scale to the original network. The aforementioned model can adapt to smaller detection targets and overcome shortcomings related to the small target detection scale. For a sample image, the minimum detectable target is 1/(160×160).

4) Activation function

Herein, the YOLOv5 network is enhanced and optimized for the wire rope samples. The ACON-C activation function is utilized in the Backbone component, and its graph is illustrated in Fig. 6. The sparse processing forced by ReLU reduces the model’s effective capacity, and the gradient of ReLU on the negative axis is 0, thereby occasioning the death of some neurons. The ACON Family can adaptively choose whether to activate neurons, which effectively prevents neuronal death occasioned by ReLU.

Herein, the Backbone component adopts the most widely utilized ACON-C activation function in the ACON Family [6,7], namely

[TeX:] $$p_1 \text{ and } p_2$$ are two parameters that can learn autonomously for adaptive adjustment, and their graphs are illustrated in Fig. 6.

Considering Eq. (3), we can observe that as x approaches positive infinity, its gradient approaches 1, and as x approaches negative infinity, its gradient approaches -1. If we utilize the second derivative of ACON-C, we obtain

Let the upper expression be 0, and the upper and lower bounds of the first derivative are respectively,

According to Formula (3) and (5), the gradient of ACON-C is completely determined.

5) Loss function

The loss function of the YOLOv5 series image detection network model comprises three components, including bounding box regression loss, confidence prediction loss, and category prediction loss. In the model designed herein, GIoU is utilized as the bounding box regression loss, and its formula is as follows:

where GIoU is the smallest rectangular area C that can contain both A and B for the predicted bounding box A and the actual bounding box B. Considering the overlap and complete separation of the actual box and the predicted box, we can observe that the GIoU loss ranges from -2 to 0. Confidence and category prediction losses are binary cross entropy losses, whose formula is expressed as follows:

where [TeX:] $$\sigma(x)$$ denotes the sigmoid function.

4.1.3 YOLOv5 algorithm implementation

The algorithm is implemented in Python 3.7 based on the PyTorch framework. Herein, an AMD R7- 4800H CPU and a Windows 10 PC with a GTX 1650 GPU are utilized to pre-train the network model.

Before training, we should configure a virtual environment and install torch 1.7.1, torchvision 0.8.2 and other supporting libraries and related dependencies in the Python 3.7 environment. Subsequently, we should process the sample dataset, mark the acquired steel wire rope defect images with LabelImg software, place the sample dataset in the JPEG Images folder, and place the .xml files generated by the marking software in the Annotations folder. Subsequently, we randomly divide the dataset into training, validation, and test sets according to 8:1:1, and feed the annotation and classification information into the YOLO Labels folder. After the dataset is ready, we can start the network training. The training adopts the transfer learning mode, utilizes the native weight parameter as the network hyper parameter, and sets the other initialization parameters as illustrated in Table 1. After 1,000 epochs of training, we save the best weight results as the best.pt file, which indicates that the training is completed.

4.1.4 Model test

The video sample of the wire rope with known defects was input into the trained network, the test program was debugged and executed, and the identification result was finally saved in the runs/detect folder.

4.2 Model Evaluation

The test results of model experiment are illustrated in Fig. 7.

The accuracy curve in the network training process is illustrated in Fig. 8, which depicts the correspondence between the results and labels. As illustrated in Fig. 8, the accuracy rate attained 85.46% after the 200th iteration. Fig. 9 illustrates the recall curve, which is the ratio of the identified objects to the total number of objects, indicating how well the samples are detected without omissions.

The network model fully converges after the 892nd training. The accuracy rate attains 96.70%, and the recall rate attains 91.87% at this point. The network model converges and produces rational results during the training process.

In this experiment, the receiver operating characteristic (ROC) curve and the precision-recall (PR) curve were utilized to analyze and evaluate the accuracy of the target detection model. The ROC curve is a curve reflecting the relationship between sensitivity and specificity, which can describe how the sensitivity and specificity of the detection model changes continuously with respect to the images of wire rope defects. The position of the ROC curve divides the graph into two components, and the area under the curve is the AUC value. The larger the AUC value is, or equivalently, the closer the ROC curve is to the upper left, the higher the target detection accuracy. In case of an imbalance in the proportions of positive and negative samples, the PR curve can further determine the model classification performance. Similar to the ROC curve, the larger the AUC value is, the closer the PR curve is to the upper right, and the higher the target detection accuracy.

As illustrated in Fig. 10, which matches the ROC curves of each model with the data of wire rope defect, the defect detection accuracy is higher, and the fault type identification accuracy is 93%, which leads to more optimal defect state classification. Combining the PR curve in Fig. 11, we can observe that even though the ROC-AUC of the proposed model is only 0.02 higher than that of the CSPD-YOLO model, the defect state accuracy rate is much higher for the proposed model. The proposed model outperforms other network models in regard to PR curves because it is closer to the upper right corner of the coordinate axis and exhibits larger AUC values for its PR curves. It can be concluded that the accuracy of the proposed model is higher than that of other diagnostic models in the detection of wire rope defects.

5. Conclusion

This paper focuses on the qualitative detection pertaining to the defective components of the steel wire rope utilized for mining before entering the shaft. Therefore, this study proposes a steel wire rope defect detection method based on an enhanced YOLO V5 model. This method can affect accurate and real-time detection for rope body defects before the rope enters the shaft, thus avoiding accidents occasioned by the rope body defects. Using the YOLOv5 deep learning model, a defect detection device for steel wire ropes applied for coal mining was designed to achieve the real-time detection of steel wire rope defects, and attained a comprehensive identification accuracy rate (96.7%). The device can check the results onsite and transmit the identification information to the host computer in the main control room via HTTP protocol. The device can detect defects of single or multiple wire ropes based on the requirements of actual working conditions.