Using Machine Learning To Detect Railway Defects

Epigraph

When it is obvious that the goals cannot be reached, don't adjust the goals, adjust the action steps. 

Confucius

Summary

The article describes the successful experience of using machine learning (artificial intelligence) to automatically detect defects during ultrasonic control of railway rails. An effective method of creating a training dataset for a neural network using mathematical modeling is presented, which made it possible to achieve high accuracy indicators for recognizing real defectograms. There given examples of prototypes of trained neural network operation with achieved prediction accuracy of real defectograms equal to 92%.

Introduction

Railway rails are heavily stressed during operation. This results in defects that can cause train accidents.

Figure 1 Photographs of rail defects and train accidents caused by their presence [1-6]

Predictive and preventative testing of rails capable of detecting defects is an extremely important area in the field of railway safety. The article will consider ultrasound flaw detection, which is widely used to diagnose rails in tracks.

One of the main problems of analyzing the results of flaw detection is the lack of automatic detection of defects onset of data captured over hundreds of kilometers of the track. When manually searching for defects, a high probability of missing them remains, as well as a strong dependence of the result on the experience of a flaw detector and a human factor.

Problem Statement

Within the framework of this work, the task is to create a working prototype of a neural network for automatic detection of flaws on defectograms of ultrasound inspection of rails with an accuracy of more than 85%.

Classification of Rail Defects During Ultrasound Inspection

For the training of the neural network, initial data in the digital form of ultrasound rails examination are required, these data can be obtained using flaw detectors in the form of the corresponding defectogram, which are displayed in the form of B-sweeps (BSCAN).

The principle of BSCAN formation is based on the input of a pulsed ultrasound signal into the rail at certain angles with a certain pitch in coordinate and recording of the reflected signal (Fig. 2). Intensity graphs are generated as points of different brightness (BSCAN) for different input angles. The use of sounding with different angles of input of the ultrasound signal is due to the different properties of defects to reflect the signal depending on the depth of their occurrence and shape.

Figure 2 Example of rail sounding scheme using six transducers [7]

To form classes of defects, the principle of classification of rail defects has been considered. All rail defects in the classification are encoded with a three-digit number according to the document: "Statement on the decryption of the results of non-destructive rail testing," which was introduced by PJSC Russian Railways in 2017 [8].

To create a dataset, the 8 most common defects were selected. The view of the selected rail defects, their codes, and visual representation of BSCAN are shown in tables 1-2.

Table 1 List of defect codes and their defectograms. Part 1

Table 2 List of defect codes and their defectograms. Part 2

The First Version of Neural Network Trained on Frames of Real Defectograms

Analysis of the existing database of defectograms showed the impossibility of their joint use due to the production of the equipment by various manufacturers with various signal processing algorithms, different numbers of emitters and receivers, etc. For the selected Avikon-11 flaw detector (OJSC Radioavionica, Russia) for each group of defects, according to the estimated data of flaw detectors, there are no more than 20 defects obtained from the sections of a railway between two stations, which is critically small to create an effective neural network. Therefore, the set of defectograms was also carried out on the existing site of the railway track with specially located reference defects. Such a section of the railway is called the "Control Dead End." The view of the defectogram of one rail from the control dead-end is shown in Fig. 3.

Figure 3 Example of defectogram section of one rail, Avikon-11

The continuous defectogram was "cut" into separate fragments, which were then classified and saved as separate files - frames. Each cut frame Fig. 3 (point 5) includes at once all the areas of rail inspection along its section. This joint consideration of the measurement channels allows all available information features to be used to classify the defectogram frame. The size of such a frame of [500, 800] conditional points is quite large and requires significant time and computational costs when teaching the neural network and in addition, it requires a larger generalized data set. To increase the data set with defects, an offset frame method was used, as shown in Fig. 4. Thus, it is possible to get more than 50 frames on one defect. This method allowed to increase the dataset from 1,000 to 60,000 with 9 classes, where class 0 is a defect-free rail.

Figure 4 Frame motion for the creation of an enlarged dataset with defects

In the process, the synthesis of a convolutional neural network for unambiguous multiclass classification with a fully connected classifier at the output (CNN) was carried out. The trained neural network showed 35% accuracy on validation data. Which is 3 times better than a purely random classifier for each class of 9 available.  It was not possible to achieve an improvement in recognition accuracy due to the selection of the structure and hyperparameters of the neural network.  At this stage, the lack of a sufficiently large and generalized data set is a crucial factor in low network efficiency. 

Data preparation and synthesis of the above-stated neural network was implemented as part of a startup by Neuroscaner LLC [10].

The Second Version of Neural Network Trained on Model Data

The low efficiency of the first version of the neural network shows that the reason is not in the algorithms used, but in the absence of a representative balanced data set for its training. This is the key point. The increase in the dataset due to the obtaining of new data when testing railways is a long process that can last several years.

The task is to consider the possibility of using model data to train the neural network.

One of the ways to accelerate the process of increasing the data set is their modeling. It is necessary to obtain samples of frames of defectograms with defects of different classes based on mathematical models. 

Modeling Type Selection

The main principle of the neural network operation and its training is to abstract the visible picture that it receives at the input and transform it into high-level visual concepts while filtering unimportant visual details.  The neural network should remember only the image, and the details only "hinder" it from recognizing [11]. That is why the creation of accurate physical models of ultrasound propagation in the rail is not necessary. Accurate physical models of ultrasound propagation in the rail do not solve the problem of generating various forms of defects and their location. Such defects need to be created manually in the amount of tens of thousands. The problem of the model of introducing ultrasound into the rail and measuring the reflected signal, their stochasticity, dependence on many factors (generator location, acoustic contact, etc.) is also not solved. And this is on top of significant computing resources.

This work proposes a method that consists of creating parametric simulation models based on which you can generate a dataset for training a neural network in volumes above several thousand for each class of defects. 

To implement a working prototype of such a neural network, a channel with an ultrasound input angle of alpha = 0 (strictly down) is selected. The length of the frame (along the length of the rail) was increased to 1000 conditional points, which is important for the accuracy of recognition.

Mathematical models were developed in the LabVIEW environment for prototyping and modeling.  The obtained mathematical models take into account not only the waveform obtained from the reflectors but also the distribution of amplitudes.  Each frame obtained at the output of the common parametric model is unique due to random generators for each parameter of the model. Table 3-4 shows examples of BSCANs of measured and model data of individual heterogeneous structures in rails. (Table 3-4 shows examples of B-spreads of measured and model data of individual rail failures.)

Table 3 BSCAN of the measured and model data on the bolt hole

Table 4 BSCAN of measured and model defects from group 30 (surface defects and defects of railhead)

Measured reflection of railhead defect	Model reflection of railhead defect 1	Model reflection of railhead defect 2

Comparison of the measured data of ultrasound examination of individual rail failures with the obtained model data allows us to talk about their qualitative similarity. The adequacy of the obtained models is proposed to be checked on the basis of an indicator of the accuracy of recognition of real defects by a neural network that will learn from model data.

From separate models of rail failures, frames of ultrasound inspection of rails were obtained. At the same time, the parameters of the models varied to obtain a variety of combinations that can occur in real conditions, namely:

• various depths of defects;
• various coordinates of defects;
• various relative positions of defects and process reflectors in the form of bolt holes;
• different combinations of bolt joint of rails by number of holes;
• different forms of defects and process heterogeneous structures in rails;
• stochasticity of response by amplitude for all heterogeneous structures in rails.

Examples of Model Data Samples

Fig. 5 shows the samples of model frames with a defect of group 30 - surface defect and head defect.

Figure 5 Samples of model frames with defect of group 30 - surface defect and head defect

For comparison, Fig. 6 shows the real data from the Avikon-11 flaw detector.

Figure 6 Samples of real frames with defect of group 30 - surface defect (head defect)

Fig. 7 shows model frame samples with group 33 defect - rail neck defect.

Figure 7 Samples of model frames with group 33 defect - rail neck defect

For comparison, Fig. 8 shows real data from the Avikon-11 flaw detector.

Figure 8 Samples of real frames with group 33 defect - rail neck defect

Fig. 9 shows samples of model frames with a defect of group 55 - a defect in the rail neck. Fig. 10 shows real frames with a defect of group 55.

Figure 9 Samples of model frames with defect of group 55 - rail neck defect

Figure 10 Samples of real frames with group 55 defect - rail neck defect

Fig. 11 shows samples of model frames with non-defective conditions (with and without bolt holes).

Figure 11 Samples of model frames with non-defective condition (with and without bolt holes)

Each group of model frames was assigned a class number (coded mark) according to Table 5.

Table 5 - Assigning Class

Class number for neural network	Name/Group	Model Frame View
0	Non-defective condition
1	Group 30
2	Group 33
3	Group 55

The created mathematical models made it possible to generate a balanced set of 10,000 training data, 1,000 validation data, and 1,000 test data for 4 given classes (0, 1, 2, 3 as in the table above). The generation time of such a set was 10 minutes.

The Effectiveness of the Idea

Further, the work carried out the synthesis of a model of a convolutional neural network (CNN), its training, and its verification on the synthesized (model) data of the created dataset. 

The accuracy of recognition on training data was 98%, on test data 97%.

To check the accuracy of recognition of real frames of flaws (measured by a flaw detector), marked frames from a control dead-end (Avikon-11 flaw detector, a check dataset - 3000 samples) were supplied to the input of the created neural network.  The accuracy of recognition of real data from the Avikon 11 flaw detector was 92%.

The accuracy obtained is an excellent result. However, there are some ways to increase it.  The mismatch matrix (Fig. 12) helps to understand on which frames the neural network made mistakes.

Figure 12 Mismatch Matrix

After analyzing the matrix, it becomes clear that the system makes the main prediction error in frames of class 3 (defect group - 55), predicting class 0 (defect-free state) - the extreme positions of one bolt hole the system recognizes as a defect of group 55, due to its similarity (Figure 13 below). In this case, it is necessary to add informational signs of other channels, thanks to which the accuracy of the forecast will increase (in this case, this is the neck control channel - the presence of bolt hole). This explains Fig. 13.

Figure 13 Example of the wrong assignment of a frame to class 3

At the moment, an algorithm of highlighting information signs on the frame for which the neural network has "decided" to classify (Fig. 14 below (yellow background) has been added to the recognition system.

Figure 14 Example of neural network highlighting of detected defects (highlighted in yellow)

The rest of the classified frames can be viewed at this link.

Conclusions and Further Development Strategies

The obtained accuracy of 92% shows the possibility of applying model data to train the network and its application to automatic recognition of real defectograms of ultrasound inspection of rails, which solves the problem of a small amount of training data and will greatly accelerate progress in the field of creating software with an automatic expert defect recognition system. Further development of this effective idea is the following steps:

1. Synthesis of models for railhead control channels, neck control channels, and rail sole control channels,
2. Tuning models out of noise,
3. "Planting" (placement, introduction) of generated defects on real defectograms (real noise patterns with introduced defect),
4. Improving models to enhance neural network accuracy.

The presented stages of development of the working prototype will allow increasing:

• accuracy of defect recognition,
• number of recognized defect classes,
• create a complete system for automatic detection of defects through all measuring channels of the selected flaw detector.

The obtained models can be adapted to the required flaw detector.

References

1. https://www.steeldata.info/macro/demo/data/2586.html
2. https://zen.yandex.ru/media/id/5da46baeba281e00b3fa94b4/defekty-relsov-klassifikaciia-kody-vidy-gruppy-i-instrukciia-po-defektnosti-jeleznodorojnyh-putei-5ddce8dccfa4bb2f43c81941
3. https://zen.yandex.ru/media/zaalan/ultrazvuk-na-straje-bezopasnosti-kak-vyiavliaiutsia-defekty-v-relsah-5d4849785ba2b500adf92ac0
4. http://scbist.com/photoplog/index.php?n=8125
5. https://ucrazy.ru/zhest/print:page,1,1196284967-kak_poezda_s_rels_sletaut.html
6. http://scbist.com/photoplog/index.php?n=8123
7. Instructions for decryption of results of ultrasonic and magnetic inspection of rails using "CRUZ-M" recorder and "ECHO-COMPLEX" flaw detector.
8. "Statement on the decryption of the results of non-destructive rail testing," introduced by PJSC Russian Railways in 2017 [8].
9. Cygan, H. , Girardi, L. ,  Aknin, Patrice ,  Simard, Patrice. (2003). «B-scan ultrasonic image analysis for internal rail defect detection»,  in Conference: WCRR 2003 Edinburgh Scotland, Oct 2003.
10. Neuroscaner - Development of a system for determining hidden defects of rails: report on research and development/LLC "Neuroscaner"; hands. R.N. Yuriev. Samara, 2020. 98 s. Bibliography: p. 138. IKRBS NIOKTR No. AAAA-A20-120092490082-3
11. Francois Chollet.  «Deep Learning with Python».  Manning Publications Co, 2018, p.362.
12. https://drive.google.com/drive/folders/1EclZjz56fOLt6xGKu12kAqmvq4ZRFLRh?usp=sharing