Robust automatic train pass-by detection combining deep learning and sound level analysis Open Access

With the continued expansion of highly dense urban areas, noise pollution has become an increasing concern. For several decades, reports have highlighted noise-related health issues.^1,2 Long-term noise overexposure is a risk factor for hearing loss, hypertension, and other cardiovascular conditions.^3,4

Among existing sound sources, industrial and transport noise have been identified as the most harmful and unpleasant due to their omnipresence in our everyday lives.^5,6 Recommendations have been outlined by health institutes,⁶ along with several independent studies.⁷ Transport noise is now regulated by national or international standards.⁸ Most of these standards require detection of the targeted source activity. Noise levels during active periods of the targeted source are used to compute its noise contribution. Thus, an effective acoustic-based train pass-by detector enables railway traffic monitoring as well as railway noise contribution estimation.

Event detection on acoustic measurements is known in the literature as Sound Event Detection (sed). With the rise of machine learning, sed has become an efficient way to identify specific sound sources within a signal.^9,10 Depending on the soundscape and the studied source, specific features can be extracted from raw audio data.¹¹ Time-frequency representations, mainly mel-spectrograms,¹² are favored in most sed systems. Spectrograms efficiently highlight the most energetic spectral patterns over sliding window applied on a signal. Such representations also benefit from the Convolutional Neural Network (CNN) popularity and accuracy.¹³ Thanks to layers of optimized filters, CNNs are efficient to extract meaningful information on images, facilitating their classification. CNNs and their variations are a baseline for diverse sed tasks, especially in environmental contexts.^14,15 In recent years, other complex networks (e.g., Transformers^16,17) have emerged, but there is little application on long-term acoustic recordings. The smaller size and relatively low computational cost of CNNs¹⁸ remain a considerable asset, especially for rare sed.¹⁹ 

Contrary to most environmental sed systems that target various sources or source sets,^10,14 our proposed work focuses on train pass-by events. In this regard, Molina-Moreno et al.²⁰ have recently presented a two-step procedure. First, they design an event detector consisting of a 66 dB(A) threshold to be sustained for 4 s on the 1 s A-weighted sound pressure level (L_Aeq,1s). Then, a classifier separates train pass-bys from other events. Two classifiers are proposed by:²⁰ either a logistic regression classifier or a recurrent neural network-based autoencoder. Both only use features extracted from L_Aeq,1s as inputs. Compared to mel-spectrograms, L_Aeq,1s provides signal energy information with a larger temporal resolution and without frequency bands. This choice lowers computational costs, what matters in long-term acoustic monitoring.^20–22 However, additional steps or complementary features are often necessary to ensure generalization on new data.

Our proposed acoustic-based train pass-by detection method relies on both mel-spectrograms and L_Aeq-based features. Two acoustic classifiers use mel-spectrograms to detect vehicle noise and then to specify the vehicle type. Meanwhile, an L_Aeq analysis unit enhances detected events demarcation thanks to advanced signal processing techniques. Applied on raw acoustic recordings, our proposed method automatically detects train pass-by and computes railway noise contribution.

In order to introduce our method, a quick overview is presented in Fig. 1. It displays the major stages, their inputs, and the global outcome: the detected train pass-bys.

Generic acoustic classifier YAMNet is used for preclassification (Sec. 2.1): it allows the identification of several frames associated with transport noise, called frames of interest (FoI) in Fig. 1. Then, an L_Aeq-based algorithm (Sec. 2.2) extends these frames into complete continuous events and discards those with inconsistent properties. This second stage is called event boundary determination and filtering. Finally, over each event, an internal complementary classifier (Sec. 2.3) is applied to distinguish train pass-bys from other vehicle sound events.

Working on detection of a specific sound source often requires different processing layers to first detect events of interest and then narrow down to the desired source. The main objective of the first step is to detect periods of interest, without missing any train pass-bys. Instead of thresholds on sound pressure level, the use of a broad classifier working on raw signals or on transformations of these signals permits the detection of low-noise events. However, detection accuracy may suffer from overlapping sounds in polyphonic environments (multiple sources at the same time).

To select the first classifier, we compared three different “state-of-the-art” (SOA) acoustic classifiers: YAMNet,²³ E-PANNS,²⁴ and Audio Spectrogram Transformer (AST).¹⁶ They were all trained on the AudioSet corpus,²⁵ audio clips drawn from YouTube videos, whose ontology contains 521 labels, covering environmental acoustic sources and events. Among these labels, we identified three major ones that can relate to the sounds of a train pass-by: C_rail = {Vehicle, Train, Rail transport}. Each time the used classifier assigns one of these labels to a frame, we consider that the model predicted railway noise on this particular frame. Preliminary assessments, later confirmed by a complete evaluation (Sec. 3.3), show that YAMNet provides a great recall on railway noise frames classification. A high recall is crucial for a preclassifier because it proves that few real frames are missed by the model. Such performance justifies our selection of YAMNet as the preclassifier in our context.

YAMNet is a CNN developed by Google, based on the initial MobileNet version 1 architecture.²³ The acoustic signal is processed within a sliding window (960 ms with 50% overlap) to form a series of frames. A mel-spectrogram is computed for each frame and used as input to the CNN for frame-level classification. The 64 mel filters are applied from 125 Hz to 7500 Hz. Such filters cover most of the relevant environmental acoustics frequency range.

FoI become the input for the following stage (Fig. 1): Event boundary determination and filtering. By definition, these are timestamps, multiple of the sliding window hop size δt = 480 ms (window length × overlap). To maintain temporal consistency, the 480-ms duration, denoted as δt, becomes the reference time step throughout the method.

Figure 2 provides a detailed layout of the proposed method. The three stages appearing in Fig. 1 are shown in the form of units within the dashed lines. The colored areas correspond to the intermediate outputs after each stage. Thus, in the YAMNet preclassification unit, FoI of a 20-minute audio signal are represented by colored areas.

On the example from Fig. 2, FoI do not directly provide well-demarcated events. YAMNet frame classification lacks continuity, as each frame is processed individually, without using information from its neighborhood. Thus, we need to improve event demarcation in order to get complete and continuous pass-by events. To address this issue, we draw inspiration from the train pass-by detection carried out by human experts.

For detecting train pass-bys on long-term acoustic signals, human experts usually rely on the evolution of A-weighted sound pressure level (L_Aeq,δt). On the L_Aeq,δt curve, a linear mobile sound source emerging from background noise during several δt periods, follows a specific pattern. Ascending and descending phases of similar duration should be observed, sometimes separated by a sound level plateau for low source speeds or long linear sources. This pattern, common to road and rail vehicles, can be observed multiple times on the L_Aeq,δt evolution of Fig. 2. Acoustics experts use this property, in addition to other signal processing criteria, to recognize train pass-bys on L_Aeq,1s evolution. To adapt and automate this process, two successive algorithms are implemented. First, event demarcation is addressed by extending FoI to the real event boundaries. Then, L_Aeq,δt profiles are analyzed to reject events whose profile differs from the expected pattern.

Typical train pass-by patterns on sound level evolution curve have been established above. They consist of ascending and descending phases of similar duration with a potential sound level plateau. Thus, a proper train pass-by demarcation can be obtained by extending preselected FoI to the closest significant sound level local minima.

Our selection of significant sound level local minima includes those whose IP exceeds a IP_min threshold, empirically fixed at 1 dB(A). In Fig. 3, two local minima (in t₀ and t₂) could be considered significant thanks to a large IP. On the other hand, local minima in t₃, t₄, and t₅ show little IP and would likely be considered non-significant.

Initial FoI are extended to the closer significant sound level local minimum in each direction. This operation improves event demarcation, as disjointed frames turn into continuous events. A typical outcome is displayed on Fig. 2, where colored areas of the event boundary determination and filtering unit represent well-demarcated events.

Thresholds are applied on both metrics to discard events whose properties largely differs from typical train pass-by. On our evaluation dataset, optimal thresholds are 5 s for event duration and 10 dB(A) for relative amplitude. Such thresholds do not wrongfully reject any real train pass-by on evaluation dataset. However, false positives also pass these thresholds because truck or motorcycle L_Aeq,1s profiles do resemble train pass-by L_Aeq,1s profiles. Therefore, a complementary classifier is built to handle remaining misclassifications.

To complement previous stages and to obtain accurate results, we aim at improving distinction between different types of vehicles. For this matter, a complementary classifier has been internally developed. In various soundscapes, we collected data and labeled it across five distinct classes: railway vehicles, roadway vehicles, construction machines, aircraft noise, and other surrounding noise. Around 14 hours of verified data thus form the training dataset of our complementary classifier. In addition to the last class, these sources emit similar noise when engine sounds prevail. Therefore, precise features are required to optimize classification of these sound sources. To such extent, mel-spectrogram representation is often preferred to L_Aeq,δt.

Thanks to this process, a single probability vector represents the L_Aeq,δt-weighted class likelihood for each event. A final threshold on railway vehicle probability provides the decision to keep the event as a train pass-by or discard it. In the complementary classifier unit of Fig. 2, the railway vehicle probability is displayed for each event, and the color represents the final decision. Unless otherwise specified, threshold value is constant for all recording sites. However, depending on measurement conditions, a site-by-site threshold adjustment increases overall performances.

This section describes the evaluation process, from the used labeled dataset to the evaluation metrics computation. Two main aspects should be carefully assessed:

• Method robustness to new and complex environments.

• Validity of the railway noise contribution resulting from the automatic detection.

Both these subjects matter in the definitions of our evaluation dataset, method, and metrics.

The evaluation dataset is crucial to estimate the value of our proposed automatic train pass-by detection method. For this purpose, two elements are prioritized when building our dataset: a large quantity and a wide variety of acoustic recordings. As a reminder, railway noise is evaluated over long representative periods, and most of our acoustic measurements last more than 24 h. The recording device (class 1 sound level meter) stands between 20 and 50 m away from the railway line. With a daily average around 80 train pass-bys (including high-speed “TGV” trains, freight and passenger trains), 16 measurements are selected to bring the total of expected events well over 1000. Above all, specific attention is paid to the soundscape around each recording sites, split into three categories:

• Urban: High level of background noise due to various sound sources. High probability of overlapping sources.

• Rural (with road traffic): High level of background noise due to high-speed roads near the recording area.

• Rural (without road traffic): Low level of background noise. Low probability of overlapping sound sources.

Eight urban and eight rural soundscapes (four with and four without road traffic) form the evaluation dataset. This context identification enables a deeper analysis of the process strengths and weaknesses, via the method described below.

In order to assess sed systems, frame-based and event-based approaches can be considered.¹² Event-based evaluation is easier to understand but does not provide any information about event demarcation. Thus, frame-based approach is favored for assessing intermediate results. It consists of a frame-to-frame comparison between the system output and the ground truth. Ultimately, the final detection prediction is also evaluated using the event-based approach.

First, precision and recall, respectively, evaluate the method tendency to include false positives and false negatives. F1-score, harmonic mean of precision and recall, is computed, too.

Finally, we also compare railway noise contribution resulting from our method to what is obtained with the reference annotation. This is a way to verify that the most annoying pass-bys are properly classified and demarcated. The L_den,railway indicator (see definition in the supplementary material) is chosen to make this comparison. In substance, it provides an equivalent noise level over train pass-bys with penalties during the evening and night. The absolute difference between L_den,railway obtained through the reference annotation and the proposed system is noted ǀL_den,railwayǀ.

First, we are willing to evaluate the progress through the three method stages (Figs. 1 and 2), and compare it to existing models. As established earlier (Sec. 3.2), it is more convenient to assess intermediate results using frame-based evaluation. Table 1 shows the frame-based precision, recall, and F1-score for major SOA classifiers trained on Audioset.²⁵ Table 2 shows the same information after each stage of the proposed method.

Table 1 clarifies the purpose and relevance of each step. First, it shows that, despite a significantly lower number of parameters, YAMNet provides an equivalent or higher recall than E-PANNS or AST. Recall is the most crucial metric while assessing a preclassifier: we want to avoid missing events and will mostly rely on the following steps to discard false positives. Using YAMNet preclassified frames as input, our second stage helps improving frame-based recall from 84% to 92% and frame-based precision from 6% to 54%. Finally, after complementary classification, a 77% precision and 90% recall are reached. Such results represent a substantial improvement compared to direct application of pretrained classifiers.

To complement these first observations, Table 3 provides an event-based evaluation, differentiating the three formerly defined measurement contexts. Only the final prediction is assessed in Table 3. F1-score with optimal threshold refers to the average F1-score obtained when railway vehicle probability threshold is adjusted site-by-site in order to maximize accuracy. Any other metric is computed with a constant railway vehicle probability threshold for all recording sites.

Such results underscore the challenge of detecting train pass-bys in the presence of road traffic noise. This results in a significant precision gap between soundscapes with road traffic (48% on average) and those without (92%). However, recall remains high in any context, meaning that expected train pass-bys are detected. Also, adjusting the threshold site-by-site largely improves F1-score in complex soundscapes (+19% in urban context). In terms of railway noise contribution, the average ǀL_den,railwayǀ is low, meaning that our automatic detection closely aligns with the reference. The maximal observed gap, at 1.2 dB(A), is relatively low for noise contribution estimations.

In this paper, a new acoustic-based automatic train pass-by detection method is proposed. The open-source audio classifier YAMNet is used to identify vehicle noise within long-term audio signals. Despite a low precision (6%) on our evaluation dataset, it provides a higher recall (84%) than E-PANNS (81%) or AST (69%) with lower computational cost. After YAMNet preclassification, two successive stages are introduced to improve detection performances, especially precision.

First, an event boundary determination and filtering algorithm is applied. It relies on A-weighted sound level evolution to extend preclassified frames into complete and continuous pass-by events. Two criteria on sound level properties are also introduced to discard events that largely differ from train pass-bys. Overall, this stage permits a large precision improvement (+48%) compared to the preclassification. Second, a complementary classifier is introduced to identify train pass-bys among the remaining events. Trained on internally collected and labeled data, this classifier distinguishes railway vehicles from other sources, including road vehicles, construction machines, and aircraft noise. Application of this classifier also results in a large precision improvement (+23%) while maintaining a high recall.

On our evaluation dataset, the complete method provided a frame-based 77.2% precision and 90% recall. Above all, the estimated railway noise contribution is very close to what is obtained manually. Average difference regarding the L_den,railway indicator is around 0.43 dB(A). This demonstrates the reliability of the method for railway noise contribution estimation. Nonetheless, further improvements could focus on refining event boundaries and reducing computational cost.

See the supplementary material for a brief introduction to L_den,railway indicator and its use.

This work was supported by ACOUSTB and GIPSA-Lab, as part of a collaborative PhD thesis in an ANRT frame.

The authors have no conflicts to disclose.

The data that support the findings of this study are available from the corresponding author upon reasonable request.