Effects of spatialized water-sound sequences for traffic noise masking on brain activities

Although conventional noise mitigation methods focus on constraining noise sources and their transmission by sound insulation and absorption techniques in noisy environments,^1,2 the soundscape approach develops alternative solutions, optimizing the sonic environment's relationship and human perception. Numerous studies have demonstrated the viability of introducing natural sounds (e.g., water sound and bird songs) into noisy urban environments for masking traffic noise.^3–6 

The masking effect of water sound on noise perception can be achieved on the sensation level, namely, “energetic masking” and perception level, namely, “informational masking.”⁷ Many studies have tested various water sounds at different signal-to-noise ratios (SNRs) to optimize the soundscape quality and desired sound levels to set the water sounds playback.^5,8 However, research about the effects of the spatial distribution of water sound on noise masking are limited. Several studies have shown the influence of spatial variation of urban space on perceived sound quality and emotional feelings.^9,10 Hong et al. explored the effects of spatial separations between target noise and water sound on perceived loudness of target noise (PLN) and overall soundscape quality (OSQ) through laboratory experiments. The results indicated that the effects of spatial separations between traffic noise and water sound were significant in PLN and OSQ. Specifically, the PLN increase at 135° separation was equivalent to an estimated target noise level increment of ∼1–2 dB. Moreover, the OSQ decrease, at 135° and 180° separations, was equivalent to an estimated target noise level increase of ∼2–5 dB.¹¹ 

For real-life applications, some researchers and designers have recently devoted themselves to introducing sound installations, such as sound screens and loudspeakers, into urban parks as practical measures for noise control.^12,13 Masullo et al. used immersive virtual reality technology to investigate the effects of combining audio and visual elements of installations with water features on traffic noise mitigation in urban green parks. They confirmed that the informational masking with water sounds at levels 3 dB lower than the road traffic background noise (BGN) improved the subjective perception of the environmental quality of urban parks. Moreover, installations with water features improve their restorativeness on escaping and fascination components.¹⁴ However, the influences of the spatial arrangement of those installations are barely investigated.

Compared to the post hoc oral reports of sonic environments, many of the advantages of electrophysiological measurement of human responses are objective and reliable to the external environments. Various studies have investigated the neural effects of different urban spaces, including green space,^15,16 indoor environments,^17–19 and contemplative landscape,²⁰ with their soundscape qualities on the human mind and mental health through electroencephalogram (EEG) measurements. Those studies have tried to connect the positive effect of natural elements with the patterns of the alpha band, which was considered a neural indicator of relaxation and comfort state. However, rather than changes of the alpha band, changes from the theta,¹⁸ beta,^16,17,19,20 and gamma bands¹⁹ were observed. Li et al. compared the soundscape components and EEG reactions in typical mountainous urban parks. The results showed that the relative power of the alpha band was more evident at the birdsong-dominant site than at the traffic-noise-dominant site under the audio-only and audio-visual conditions. Besides, more restorative EEG reactions were found within the audio stimuli than within the audio-visual stimuli.²¹ 

Electrophysiological measures are not only used as neural indicators of sonic environments. They can also provide spatial-temporal information about the procedure of auditory attention and noise masking. Most of those studied focused on speech processing in a multi-speaker environment. Tóth et al. compared the whole-brain functional networks underlying the process of focusing attention on a single speech stream with dividing attention between two streams. The results showed that focusing attention on a single speaker compared to dividing attention between two concurrent speakers was predominantly associated with connections involving the frontal cortices in the delta (0.5–4 Hz), alpha (8–10 Hz), and beta bands (13–30 Hz), whereas dividing attention between two parallel speech streams was linked with stronger connectivity involving the parietal cortices in the delta and beta frequency bands.²² Szalárdy et al. used functional connectivity of EEG signals between different brain regions to investigate the neuronal correlates of informational and energetic masking in a multi-talker situation. They found energetic masking was predominantly associated with a stronger connectivity between the frontal and temporal regions at the lower alpha and gamma bands, and informational masking was associated with a distributed network between parietal, frontal, and temporal regions at the theta and beta bands.²³ This methodology provides the potential to interpret the fundamental mental processes induced by the mask sounds in noisy environments regarding acoustic comfort, health, and well-being to enable policymakers and designers to extrapolate solid results.

This paper used water-sound sequences with different spatialization settings to investigate their effects on masking road traffic noise (RTN). This was performed through EEG measurements, which provided more insight into the overall mental state assessment and brain network changes of cognitive processes referring to attention control when using the spatialization sequences of water sound for the informational masking of noise. The results will also provide us with more thoughts about workable measures of urban design for noise mitigation.

A within-subjects experimental design was used. The independent variable was the spatialization of the water sounds. Four levels of spatialization were defined: frontal position-fixed water sound (FPW), a two-position switching water sound (TSW), a four-position-randomized moving water sound (FMW), and an empty water sound, all of which combined the RTN on the frontal position as background. The two-position pair of TSW included four different settings: frontal-left pair, frontal-right pair, back-left pair, and back-right pair (only adjacent positions were considered for avoiding the distance differences of two-position pairs). The dependent variables were the neural signals obtained by a wearable device during each condition. The study hypothesized that a structural, spatial representation of water sounds in a noisy environment would produce more positive subjective feelings and better efficiency on informational masking than the fixed location water sound, leading to decreased mental stress and increased restorative qualities. The results obtained from the neural signals were also compared with those obtained by self-reported questionnaires about the perceived characteristics of the sound environment and with those based on the emotional saliency.

The sound sources included a 3-min traffic noise, recorded with a Zoom H6 Hand-Recorder device (Zoom North America, New York) and a Soundfield SPS200 microphone [SoundField Ltd., Sydney, Australia; A-weighted equivalent continuous sound level, LAeq, 65 dB(A)] as BGN, and a 5 s water stream sound, recorded by the same device with a RØDE NTG-2 microphone (RØDE, Sydney, Australia). To optimize the effect of the water sound-based informational masking, the sound level of the water stream sound was set at −3 dB^3,5,24,25 with respect to the background traffic noise (SNR = −3 dB). Water stream sound was used to create 3-min-long soundtracks (A/B) for spatial sound reproduction. The soundtracks combined repeated 5 s of water stream sound with 2 s of fade-in and fade-out, alternating between positions with 2 s of overlap [see Figs. 1(a) and 1(b)]. They were played back within the Sens i-Lab of the Department of Architecture and Industrial Design of the Università degli Studi della Campania “Luigi Vanvitelli” through the Astro Spatial Audio (Astro Spatial Technologies BV., Odiliapeel, Netherland), an object-based audio system which drives 25 Adorn A55 Martin Audio and 2 Sx110 Martin Audio (Martin Audio Ltd., London), and rendered by SARA II Premium Rendering Engine (Astro Spatial Technologies BV., Odiliapeel, Netherland). This audio system uses the wave field synthesis principles to reproduce the sound emitted by audio objects, point source, or plane front (called plan wave) in the listening area of the room.^26–28 Both of the previous audio objects were used for the playback: the plane wave object, reproducing the RTN, and the point source objects, reproducing the water sounds [see Figs. 1(c) and 1(d)].

FPW was set as a fixed position of water sound in the frontal position with RTN as background. As for TSW settings, four two-position pairs were defined: the frontal-right, right-back, back-left, and left-front. The distance between each position soundtrack up to the subject was the same as that for FPW. For FMW, the pseudo-random routine of the water sound selected from the four-position (frontal/back/left/right) was defined at the ASA application (see Table I). The experimenter controlled the order of these sound sequences via browser-based GUI during the listening test.

The listener was sitting at the center of the test room of the Sens i-Lab [see Fig. 1(c)] at about 3.5 m from the position of the virtual sound sources. The audio stimuli at listeners were recorded using a dual channel system Sympnonie (01 dB, Limonest, France) and an Mk1 Cortex manikin (01 dB, Limonest, France). They reproduced realistic auditory scenarios of about 57 dB(A), similar to those measured inside an existing urban park.²⁴ The audio stimuli were then imported and analyzed with the software Artemis Head Acoustics (HEAD acoustics GmbH, Herzogenrath, Germany). In Fig. 2, the spectrograms of the left and right channels at the dummy head were reported for all of the sounds spatialization conditions of the experiment.

The self-reported questionnaire was used to collect participants' subjective responses to objective and emotional aspects of the sound environment. In particular, the first part of the questionnaire was focused on assessing general characteristics of the sound environment, including naturalness,^13,29,30 mechanicalness,^13,30 smoothness,²⁹ rhythmicalness,³¹ spaciousness,³² and familiarity,³³ whereas the second investigated the emotional saliency of sounds.³⁴ The latter part of the questionnaire combines items derived from the circumplex model of soundscape perception^33,35 with others focused on the emotional feeling of the sound environment.

Twenty subjects gave informed consent and were instructed to sit in the center of the test room to be immersed in virtual sound environments. Before the formal experiment, the subject filled out two pages of the initial questionnaire, which contained basic information such as age [average, 30 years old; standard deviation (SD), 5.90], gender (male, 12; female, 8), working environment, Weinstein noise sensitivity scale^36,37 (average score, 3.73; SD, 0.50), and personal well-being scale^38,39 (average score, 53.78; SD, 12.59). After wearing the portable EEG device and passing the impedance check of EEG electrodes, the subject was asked to listen to five sequences with a comfortable sitting position and eyes open in the predefined balanced order (two TSW conditions were randomly selected from the four TSW conditions, considering that the recommended duration of the entire EEG test should be less than 30 min in the case of the signal-noise-rate losing, caused by the effect of discomfort and fatigue⁴⁰) Each sequence lasted 3 min. Next, the subject must fill out the questionnaire, including the perceived contents and his/her feelings about each sound based on previous works.^13,29–34 After finishing the questionnaire, the subject informed the experimenter to play the following sound sequence. Finally, the subject took a 1-min rest with his/her eyes closed. The neural activities during this period were used for baseline correction for EEG analysis.

During the whole process, the brain data of each subject were continuously recorded by a DSI-24 wireless EEG headset (Wearable Sensing Ltd., San Diego, CA) with 20 dry electrodes signals referenced to the Pz electrode at locations corresponding to the 10–20 international system (see Fig. 3). The light and temperature in the laboratory were kept constant during the test. The EEG data were sampled at 300 Hz and streamed from the measurement device to the recording laptop using the Lab Recorder application based on the Lab Streaming Layer protocol (Swartz Center for Computational Neuroscience, San Diego, CA) to synchronize the neural data with sound sequences. The Ethical Committee for Scientific Research of the department approved the protocol.

The continuous EEG data were imported into matlab and the EEGLAB toolbox and preprocessed using the automated PREP pipeline for data cleaning.^41,42 The data of two subjects were excluded because of less clean data (both of the percentages of invalid data were higher than 50%, and the average of the valid data was 87.06%). Then, a 1–45 Hz bandpass filter was applied. After re-referencing the EEG signal to the average (except for A1 and A2 mastoid electrodes), their independent components were calculated using the Infomax algorithm. Eye-blink and ocular movement artifacts were deleted based on the standard topographic profiles of the individual components and distinctive temporal pattern. After the removal of eye movement artifacts, the EEG data during each sound's perception were extracted.

The cleaned EEG data were analyzed using MATLAB and the FieldTrip toolbox.⁴³ Time-frequency-resolved activity was obtained using the multitaper method (4 cycles width) based on Hanning sequences between 1 and 45 Hz (stepsize, 1 Hz), from which the average powers of each frequency band (delta band was defined as the range of 1–4 Hz; theta band, 4–8 Hz; alpha band, 8–13 Hz; beta band, 13–30 Hz; low-gamma band, 30–45 Hz) were derived (see Table II). The interested electrodes were divided into five regions: the frontal (Fp1, Fp2, F3, and F4), left temporal (F7, T3, and T5), central (Cz, C3, and C4), right temporal (F8, T4, and T6), and posterior regions (P3, P4, O1, and O2), respectively.

The relative power of each given band/sum of power from 1 to 45 Hz was calculated by

where P(·) indicates the power, RP(·) indicates the relative power, and f₁ and f₂ indicate the low and high frequency, respectively. The relative power for each band and power ratios for different frequency bands were averaged in each region. The ratios of power for different frequency bands in each electrode were also computed for possible pairs of frequency bands, such as P(theta)/P(alpha) and P(alpha)/P(beta).

EEG connectivity analysis could be conducted at the sensor-level or source-level for network analysis. Sensor-level connectivity helps us understand the temporal changes of functional networks of the brain in a regional scale (referring to frontal, occipital, parietal, and temporal lobes), but the information of precise neuroanatomy locations of those connectivity changes requires source-level analysis. Sensor-level connectivity analysis was used in our study as for the connectivity changes of the brain regional network were the main points in our study, and it also ensured the analysis reliability as the recommended number of electrodes for source-level connectivity analysis should not less than 32.⁴⁶ The 3-min EEG data during each sound's condition were epoched by 7 s fixed length and analyzed by the MNE toolbox using the spectral connectivity algorithm.⁴⁷ The spectral connectivity was computed for the debiased weighted phase lag index (dwPLI). The dwPLI is a debiased estimator of the squared weighted phase lag index (wPLI) developed by Vinck et al., correcting for sample-size bias in phase-synchronization indices.⁴⁸ 

The results related to the objective descriptors of each sound environment (naturalness, mechanicalness, smoothness, rhythmicalness, spaciousness, and familiarity) were analyzed. Two main differences were found between the four conditions. The subjects felt more familiar with traffic noise rather than water sound conditions, and more rhythmic features were detected from FMW and TSW conditions than traffic noise [see Table III and Fig. 4(a)]. On the other hand, the scores of adjectives items' responses, including pleasant, happy, stimulating, attractive, energetic, and calm were averaged to compute the positive component of the emotional saliency (ES+). The results indicated the better masking effect of TSW compared to that of the FPW condition from the analysis of variance (ANOVA) post hoc test (t_TSW-FPW = 3.02, p = 0.019). However, all of the water sound conditions did not show significant improvement in terms of ES+ cores compared to the RTN condition. To be more specific, the main differences existed in three positive emotional items, which included stimulating, happy, and energetic (see Table III). FMW and TSW were more stimulating and energetic than FPW rather than RTN, and TSW was also felt to be happier than FPW rather than RTN. No distinctions between the four conditions emerged in the negative component (ES-; averaged by the scores of boring, unpleasant, nervous, weak, sad, and unattractive items).³⁴ 

The relative power of the alpha band showed significant differences between four conditions in the whole brain [F(3,51) = 9.43, p < 0.001; see Table IV and Fig. 5 for the results of each region]. FPW and FMW had higher relative powers of the alpha band than RTN in most of the brain regions. The higher relative power of the alpha band of TSW sound only occurred in the frontal and left regions compared to the RTN condition [see Fig. 5(a) for post hoc comparison]. The relative power of the theta band showed differences between four conditions in the frontal region from ANOVA results but no post hoc analysis differences. The relative power of delta, beta, and gamma bands showed no significant differences between the four conditions [see Table IV and Fig. 5(a)].

The index of the theta-alpha ratio showed major differences between different conditions within each brain region except for the frontal position (see Table V). From the post hoc multiple pairwise statistical comparison, RTN ratios were much higher than FPW, FMW, and TSW in the central region and left region, and the same ratios of RTN were significantly greater than FPW and FMW in the posterior region and right region [see Fig. 6(a)].

The index of the alpha-beta ratio showed significant changes between different conditions within each brain region (see Table V). From the one-way ANOVA analysis, the ratios of RTN were clearly lower than those of FPW, FMW, and TSW in most of the regions except for the posterior region. For the central region, FMW was significantly greater than RTN, FPW and TSW [see Fig. 6(b)].

The dwPLI connectivity results showed significant differences between different conditions within local regions across frequency bands. In the delta band, the main changes were in the posterior position, whereas in the alpha band, they differentiated in the central position. The differences of the beta band and gamma band also existed in the frontal position. The inter-regions connectivity metrics also illustrated crucial changes across frequency bands. In the delta band, the connections in the frontal-central, frontal-posterior, central-left, and central-posterior regions were significantly distinct between the four conditions. In the alpha band, the connections in the frontal-posterior and central-left regions were significantly different. In the beta band, six inter-regions had large differences, including frontal-right, frontal-posterior, central-left, left-right, left-posterior, and right-posterior inter-regions. In the gamma band, the main differences of inter-network connectivity were only found in the left-right and right-posterior regions (see Table VI for detailed information).

From the post hoc analysis of delta band connectivity data, the coherence of the local posterior region and most inter-regions in the RTN condition were significantly higher than those of the FPW, FMW, and TSW conditions (see Table VII and Fig. 7). As for the alpha band connectivity results, only the frontal-posterior connections of TSW were much greater than those of FPW, FMW, and RTN, the central-left connections of FPW and RTN were clearly greater than those of FMW, and the local central coherence of TSW was significantly higher than those of FPW, FMW, and RTN (see Table VII and Fig. 7). Most of inter-regions coherences of TSW and FMW conditions in the beta band connectivity were higher than those of the FPW and RTN conditions. The frontal-right coherence of FPW was essentially lower than RTN. The local frontal coherence of FPW in the beta band was significantly lower than those of FMW, TSW, and RTN (see Table VII and Fig. 7). Furthermore, gamma band connectivity data showed the differences of RTN in the inter-regions network. The left-right coherence of RTN was lower than those of FPW, FMW and TSW, and similar results happened in the right-posterior region. The local frontal coherence of FPW was much lower than that of RTN (see Table VII and Fig. 7).

The subjective results showed that the spatial water sound (including TSW and FMW conditions) was better than fixed location water sound, but there was no water sound effect of FPW compared to the RTN condition, which was inconsistent with previous work.^1–4,6 There were controversial results referring to the overall effect of the temporal variability of water sound for traffic noise masking.^4,5 The work of Coensel et al. indicated that low temporal variability of water sound reduced the loudness of RTN.⁴ Meanwhile, these studies also found that water sounds with high temporal variability produced more pleasant feelings than sounds with a steady-state character.^5,25 Other that the differences between FPW and RTN conditions in rhythmicalness and familiarity dimension, the spatial effects of each sound could also induce the results. Because the spatial configurations were the key factors in our study, the dynamic aspects of subjective assessments (including energetic and stimulating items) could be biased toward the spatial effects (characterized by spacious) and undermine the temporal factors (characterized by rhythmic). It indicated a more complicated relationship between temporal variability of water sounds and the subjective feelings of the masking effect, and the spatial variability should be also considered within this relationship, which could give more ideas about future studies.

From the EEG spectrum results, the increase in the relative power of the alpha band clearly indicated the enhancement of neural relaxation caused by the introduction of water sound in a traffic noise environment. Compared to the absence of the alpha band changes of previous work,^16,19,20 those changes across the whole brain and between three water sound conditions and RTN condition, were evident. This could be related to more involvement of the default mode network rather than the task-related network of the brain caused by the less task engagements and more rest states in our experiment.⁴⁹ The alpha-beta ratio, which was considered an index of mental relaxation opposite to mental fatigue⁵⁰ and mental stress,⁵¹ was derived from the phenomenon that when mental stress level increases, the beta activity in the brain also increases with the decrement of alpha band power. The results of the alpha-beta ratios confirmed the positive mental effect of the introduction of water sound. The findings supported and extended previous studies related to perceived restorativeness in urban parks⁵² and the natural environment.⁵³ 

While the FMW condition induced similar reactions with FPW in the spectral results of the alpha band, the TSW condition seemed to produce the lowest relaxation effect on traffic noise, which was contrary to the more positive subjective results. Similiarly, the relative lower alpha-beta ratio of TSW in sound conditions, especially the posterior region related to visual perception, could indicate a more active state of cognitive process involved in the TSW condition. Some in situ and experimental studies had confirmed that sound source visibility was an influencing factor on auditory impression.^54,55 As mentioned previously, in the study of Li et al., the visual stimuli of audio-visual conditions were associated with decreases in restorative EEG rhythm at traffic-noise-dominant sites and birdsong-dominant sites compared to audio-only conditions.²¹ Xu et al. found that the noise annoyance ratings of road traffic were higher when the sound source was visible.⁵⁶ Similar to the spatial attention implicitly involved in thoses study through visual cues, our study explicitly strengthed the spatial representation of water sound in the TSW compared to FPW condition, which could bring more attention to the water sound and produce similar EEG rhythm transfer. Therefore, the discussion about the attentional network was necessary, and the investigation of multisensory inputs were asked for future works.

The investigation of the theta-alpha ratio, often used as the task load index, considered that an increase in mental load was associated with a decrease in alpha power and an increase in theta power.^57–59 Our results showed that water sound sequences were able to decrease the mental load of traffic noise regardless of their spatial settings.

Although several studies have shown the effect of spatial variation on soundscape perception,^9–11 the research about the cognitive processes of spatial sound on noise masking was limited. The analysis of the functional connectivity of EEG across different frequency bands was used for revealing the interactions between different brain networks while there was some evidence suggesting that these connections supported attention control and auditory processing functions in the speech field.^22,60,61

EEG oscillation studies suggested low-frequency phase synchronization, including delta and theta frequency bands, increases between the frontal and parietal regions in tasks requiring attentional orientation.^62–64 Functional imaging studies found that the posterior medial frontal cortex (pMFC) and lateral prefrontal cortex (LPFC) transmit excitatory or inhibitory signals to regions involved in information selection through theta-band phase synchronization.^65,66 Meanwhile, delta-band activity reflected fronto-parietal sensorimotor processes elicited by the detection of the sensory target.⁶⁷ The evidence explains that the activation of information selection networks was more evident in the RTN condition than masking conditions, which was illustrated by the stronger connectivity within the delta band of RTN than those of TSW, FPW, and FMW across most intra-regions and inter-regions. The research of Tóth et al. suggested strong connectivity in fronto-parietal networks in the alpha band associated with selective attention when listeners were instructed to listen to one as opposed to two audio streams,²² and gamma oscillation over the sensory cortices is supported for enhanced attention to the sensory events in the visual⁶⁸ and auditory domains.^69,70 In our results, the connectivity networks of the alpha and beta bands showed that TSW was the highest activated condition in the focused attention network and FMW was the second, which indicated that water sound with a more solid spatial setting could enhance sustained attention, and then induce better masking effects.

In a previous work, Szalárdy et al. found that noise masking was related to the EEG connectivity of the alpha band to energetic masking, especially in frontal-temporal regions, and the connectivity of the beta band was related to informational masking.²³ The differences of brain networks for energetic and informational masking were also explained for speech perception through a functional magnetic resonance imaging study.⁷¹ The results showed a network of activation in the bilateral temporal lobes, prefrontal cortex, and parietal lobes, which were commonly activated across all of the masking conditions. Meanwhile, informational masking additionally activated clusters of activity in the bilateral superior temporal gyrus and right primary auditory cortex. The obvious higher activation of the connectivity networks across intra- and inter-regions in the beta band in the FMW compared to FPW condition were likely to support the results of Szalárdy's work²³ as the moving information was the only difference between the FMW and FPW conditions. However, further study still needed to clarify the relationship between brain oscillation connectivity and energetic/informational masking.

Traffic noise is treated as a health threat for citizens in urban cities. Environmental designers and managers are striving for noise absorption and abatement. The introduction of wanted sounds, such as water sound, has been proven effective in mitigating traffic noise. Our study used three different water-sound sequences and one control condition with only traffic noise to investigate the role of spatialization of water-sound sequences on traffic noise masking. The sequences included a frontal fixed-position water sound, a TSW sound, and a FMW, which was played back within a spatial audio system in our Sens i-Lab. The portable EEG device recorded the neural responses of 20 participants in the experiment laboratory during the spatial sound playback time. The mental effects and attention process related to informational masking were assessed by the analysis of the EEG spectral power distribution and sensor-level functional connectivity along with subjective assessments. The changes in the relative power of the alpha band and the ratio of the alpha-beta band among four conditions showed an increased relaxation state triggered by the introduction of water sounds. Different spatial settings of water-sound sequences, especially the two-position switching setting, induced more attentional network activations related to the information masking process for noise mitigation along with more positive subjective effects.

This work was supported by Programma V:ALERE 2019 “VALERE: VAnviteLli pEr la RicErca” and Project MIELE, Multisensory Investigation for ELderly-centred design of common living urban Environments.