Robust defect detection under uncertainties using spatially adaptive capacitive imaging

Structural integrity of safety-critical infrastructures such as aircrafts, bridges, nuclear power plants, oil and gas transmission pipelines, and load-bearing metal structures in highways decrease with time. Nondestructive evaluation (NDE) plays an important role in maintenance of these infrastructures by detecting significant erosions and other defects that crop up with usage. For providing effective diagnosis, NDE methods use several inline inspection techniques to access these structures without causing any further damage in a non-invasive manner.¹ The advancement in sensing technology has led to the development of different NDE sensing modalities.^2–14 However, often in practice, it is extremely difficult to set up new non-destructive inspection procedures abiding the ideal laboratory conditions under which these methods were developed and tested. As a result, in these situations, NDE data are collected under uncertainties such as lift-off, probe tilt, fluctuations in sampling rates, and step sizes.^15,16 Contemporary NDE methods based on robotic systems can conduct very fast scans but often suffer from the aforementioned uncertainties due to anomalies in probe positioning. Lift-off is the variation of distance between the probe and material under test (MUT) whereas probe tilt is the angular deviation of probe from perpendicularity. NDE data are often contaminated by the presence of unknown lift-offs due to coarseness of MUT surface, non-conducting coatings, or irregular paintings on the MUT.^17,18 While lift-offs and probe tilts are the most common uncertainties, fluctuations in the sampling rate and step sizes can also contaminate NDE data due to operator’s movement related discrepancies such as changes in frictional force between probe and material surfaces as well as misalignment of probe and MUT coordinates. These uncertainties in the data collection process give rise to different kinds of corruption in the recorded signals. Defect identification algorithms that were developed with sensor data collected under ideal laboratory conditions fail to provide accurate defect analysis under these uncertainties which corrupt data with various kinds of noise contaminations. Subsequently, there is an urgent need to develop robust, automated, reliable defect identification methods that can operate with high precision under uncertainties that practitioners encounter in real-world applications.¹⁹ 

Here, we develop a high-Q capacitive sensing based shortwave NDE method which is low cost, highly flexible, and easily implementable as it does not require extensive hardware setup at MHz frequencies. Unlike other popular NDE methods, it can be employed under a wide range of robust conditions for it enjoys the following advantageous properties over the competing NDE methods: (a) unlike capacitive sensing, ultrasonic testing (UT) normally requires a coupling medium and physical contact with MUT.⁸ Air coupled UT, however, does not need coupling medium. But, it uses long sound pulse for excitation for which accurate measurement of timing is difficult. Moreover, it is not applicable for metal inspections due to high acoustic impedance of metal structures.²⁰ (b) Thermographic methods have low sensitivity unless we use very expensive thermal camera for imaging and are non-robust to temperature variations.^21,22 X-ray imaging is not only expensive but also needs proper screening because of the detrimental ionizing radiations.⁹ (c) Microwave imaging based NDE detection do not penetrate deep in conducting materials and use high frequency for operation in GHz ranges which require cumbersome data acquisition circuitry.²³ (d) Conventional electro-magnetic (EM) based NDE techniques, such as magnetic flux leakage (MFL) and eddy-current (EC) methods, are mostly effective in inspecting highly conductive or ferromagnetic materials.^{2–4,10–14}

We concentrate on the highly flexible, easily applicable capacitive sensing method and undertake a disciplined approach to understand how signals collected from defective samples get corrupted as uncertainties such as lift-off, speed, and fluctuations in sampling rates are introduced in the data collection process. In Sec. II, we describe the data collection methodology as well as the different robust setups considered in this study. Existing NDE literature mainly focused on the effect of lift-offs based uncertainties in EC and MFL methods. Lift-off corrections in these traditional EM techniques have been developed based on novel hardware-based probes,^24,25 new sensor array designs,²⁶ and by usage of signal processing techniques.^17,27 Novel probing methods introduced in Ref. 17 use reference signals to cancel lift-off induced contaminations in pulse eddy current (PEC) signals whereas dual frequency mode of EC inspection is used in Ref. 28 for measuring lift-off and accurate defect sizing. Signal processing techniques based on dynamic trajectories of fast Fourier transform of PEC scans are used to reduce the lift-off effects in Ref. 29, and features extracted based on rise time, zero crossing, differential time to peak are used in Ref. 30. Unlike traditional EM methods, there has been very limited research on the scope of capacitive imaging (CI) under uncertainties. Specialized capacitive sensors have been designed to tackle lift-off effects^31–34 but they have limited applications. Here, we develop a very easy-to-implement capacitive sensing probe and provide a systematic study on the impact of uncertainties on signals captured by it. Based on the study, we develop a novel spatial adaptation based signal processing algorithm that is exhibited to provide very high defect identification rates. The resulting NDE method is cost effective, less time consuming, and can be employed across a wide range of applications for it does not require any domain specific modifications or usage of complex probes or guidance of NDE experts.

In Sec. III, using a disciplined statistical framework, we compare the voltage readings recorded by the capacitive sensors under different kinds of uncertainties. We show that the corruption in the voltages due to uncertainties which decreases signal strength for defect defection is not merely due to homogeneous addition of noise over all the inspection readings. We analyze the differences in the capacitive sensing voltage readings collected under various kinds of lift-offs by Gaussian mixtures models. In each case, we document the target-to-background voltage comparisons where the target is the defective area amidst the background of non-defective scan points. Based on this mixture model framework, in Sec. IV, we report the theoretically possible lowest limit on the defect misclassification error rates across varied uncertainties by considering the Bayes estimator. In most NDE applications, it is much more harmful to misclassify defects as non-defects [which is termed as false negatives (FN) or “missing”] than to misclassify non-defects as defects [which is termed as false positives (FP) or “false alarm”]. In this context, we evaluated the Bayes error under the weighted classification loss and reported the minimum achievable FP rates when the FN rates are constrained below 10%, i.e., when we, at worst, underestimate 10% of the defect sizes. Based on experimental data collected under different types of uncertainties, we found that the FP rates were quite low even when FN rates are controlled below 10%. This suggests that it is theoretically possible to recover defect sizes under the uncertainties considered here.

Computation of the Bayes rules and errors reported in Sec. IV needed pertinent information on defects that is not available for real applications. Thus, those error rates mark the best theoretically possible errors that any non-spatial decision rule can obtain. In practice, it is very difficult to get a solely data driven procedure that achieves such error rates. In Sec. V, we propose a data driven classifier of defective scan points. The classifier borrows information across geographically close scan points and is spatially adaptive. Aggregating the data from nearby locations can increase the background-to-signal ratio and increase the defect detection rate under certainty. The proposed approach non-parametrically recovers the structure of the defects without prior knowledge on the uncertainties in the data collection process. We explain the benefits of spatial adaptation and show that the proposed method yields accurate defect identification in highly robust conditions.

Figure 1 shows the framework of the proposed study. In order to provide a disciplined analysis of robust defect identification using capacitive sensing, we further divide the study into the following components:

1. The first stage involves collection of experimental data using common place capacitive sensing methods. In this stage, as described in Sec. II A, we set up a cheap capacitive-sensing probe with its parameters tuned to operate optimally in usual non-robust laboratory settings.

2. Thereafter, varied types of uncertainties are systematically introduced in the data collection process through an experimental design (see Table I) that involves 36 experiments on defective samples containing a single defect of varying sizes.

3. We compute the Bayes error in defect misclassification to understand the impact of corruption in the voltage signals recorded by capacitive sensing under uncertainties. The nature of the corruption under different types of uncertainties were characterized.

4. Based on the above characterization, a data driven method was developed to adaptively denoise capacitive sensing based voltage signals. The proposed non-parametric method uses spatial filtering and is exhibited to detect defective scan points with very high accuracy.

We use a capacitive sensing-based probe that consists of a parallel plate capacitor along with a coupled inductor coil. The capacitive probe detects the change in the dielectric constant due to distortion in electrostatic fields in the presence of defects in the sample. The voltage recorded by the probe across a grid of scan points produces image that can lead to defect identification.^7,35,36

To understand the working principle of the probe, note that the resonant frequency $fR$ of the sensor is inversely proportional to its net capacitance (C) as $fR=1/(2πLC)$⁠, where $L$ is the inductance. We know that $C=κAd−1$⁠, where $A$ is the area of the probe, $d$ is the distance between the excitation and ground, and $κ$ is the dielectric constant of the medium. In the presence of defects, $κ$ changes as the medium is altered compared to non-defective scan points. This changes the capacitive loading $C$ of the fringing field which impacts $fR$⁠.

The sensor is designed as in Ref. 36. It is made up of Roger’s $4350$ board with dielectric constant of $3.66$ and thickness of 1.5 mm. A commercial inductor coil of 100 $μ$H is used to form the LC tank. The experimental scanning setup consists of the following ingredients: (a) LC tank based capacitive probe with pickup coil; (b) scanning robot arm to move the probe along the sample; (c) power splitter and directional coupler; (d) Data Acquisition System (DAS) and code to produce image; and (e) other associated units.

We used Aerotech AGS$1000$ programmable XYZ scanner for scanning. The probe is mounted on the scanner. A radio frequency (RF) source is used as input. It is operated at resonant frequency of the probe which was estimated at 5 MHz when the medium was just air with no sample. The probe is read in the reflection mode. As shown in Fig. 1, it is connected to a power splitter which generates the reference and the measurement signal. The directional coupler is fed with the measurement signal and is connected with the pickup coil of the probe. As in Refs. 37 and 38, the reflected signal captured by the probe is passed from the directional coupler to the lock-in amplifier which generates voltage signal proportional to the difference in the reference and the measured reflected signals. This voltage signal thereafter is passed into the DAS consisting of a National Instrument Data Acquisition Card PCIe-$6341$⁠. The voltage signal is sampled and digitized by a routine in the DAS, and the output is recorded. The complete experimental setup used in the laboratory is shown in Fig. 2.

To understanding how data collected by capacitive sensors is altered as uncertainties increases in the system, we collect data across different regimes corresponding to varied defect types and capacitive sensing sampling methods. Table I describes 12 different regimes/experiments. For each experiment, we collect data under three scenarios: (1) capacitive sensing with no lift-off; (2) capacitive sensing with moderate lift-off of 3 mm; and (3) capacitive sensing with increased lift-off of 5 mm. Aside from the lift-off uncertainties, other variations were also incorporated in the capacitive sensing methods. These variations were introduced across the 12 different experiments and they were kept invariant across the three scenarios of the experiments. Table I shows the details for each experimental setup. Also, to reflect data collection uncertainties due to non-smoothness of the inspected sample surfaces, we used cheap steel samples bought from the shelves of a popular retailer instead of perfectly lab-calibrated expensive samples for collecting our data. Steel samples containing through wall hole (TWH) circular defects of radii 2, 4, and 5 mm were used. Figure 3 shows one such sample.

Figure 4 shows the capacitive sensing images across the three scenarios of experiment 1. Note that, due to coarseness of the samples in use, voltage readings collected under no lift-off has some fluctuations in the readings at the non-defective scan points. The leftmost plot of Fig. 4 depicts this variation in the voltage readings as non-defective scan points at the bottom right corner show higher voltage than the non-defective scan points in the top left. The defective points are, however, clearly detectable under no lift-off with much lower voltage than the voltage at almost all non-defective scan points. With increase in lift-off, the voltage across all the scan points decreases as shown in the legends of middle and right most plots of Fig. 4. While defective scan points still have voltages on the lower end, the difference in voltage between defective and non-defective scan points is much more decreased particularly under increased lift-off. This make classifying defective scan points based on the voltage readings much difficult as lift-off increases.

The aforementioned patterns in the three scenarios of Fig. 4 is broadly seen also across the other experiments. Figures 13–15 in  Appendix B shows the images of the voltages for all the other cases. Comparing the leftmost (no lift-off scenario) images in Figs. 13–15, we observe that the voltage magnitudes vary across experiments as uncertainties change across regimes, but the general pattern of diminishing voltages due to increased lift-off was consistent across the experiments. These figures demonstrate that across all the considered regimes which encompass three different defect sizes, as lift off increases we see significant decrease in the gap between the averages of the voltage readings from the defective and the non-defective scan points. The reduction in this gap decreases the signal strength in classifying defective scan points. In Sec. III, we validate this phenomenon by considering a rigorous statistical framework and quantitatively study the impact of lift off on this gap.

We first define a few notations. For the $j$th scenario of the $i$th experiment, consider observing readings $Yij={yl(i,j):l∈Λi}$ over grid $Λi$⁠.The grids vary over experiments $i$ when the step sizes are different. We next check if the change in the readings due to lift-offs can be explained by independent corruption in signal intensity across the grid. For this purpose, we consider the additive noise model: $yl(i,j)=yl(i,1)+ϵl(i,j)$ for $j=2,3$⁠. For any fixed $1≤i≤12$ and $j=1,2$⁠, we test the null hypothesis that $ϵl(i,j)$ are independent across $l∈Λi$⁠. All the 24 p-values are less than $10−5$⁠, and so, across all the scenarios, the null hypothesis that changes in the signal intensity is independent across locations was conspicuously rejected. This suggests that the corruption in the signals due to uncertainties in the data generation process is not homogeneous across all the scan-points inspected in the sample. We characterize the heterogeneity in the corruptions to the capacitive sensor signals in the following paragraph.

In Figs. 4 and 13–15, we witness that the lift-off based corruptions in the voltages recorded at the defective scan points is different from the corruptions in the background non-defective scan points. To explain this difference in corruption, we consider a bivariate normal mixture model that uses different voltage distributions for the non-defective and the defective scan points. Noting that for any experiment i, the defective scan points are invariant across scenarios j, let $Θi={θl(i):l∈Λi}$ denote the defective scan points in experiment i, i.e., $θl(i)=1$ if l is a defective scan point and $θl(i)=0$ otherwise. For experiment i and scenario j, consider the following conditional model on the capacitive sensor readings: for the non-defective background points, consider the readings to be independent and identically distributed (i.i.d.) from a normal distribution,

and the readings from the defective scan points also are i.i.d. from a normal distribution with possibly different location and scale than the background,

We further assume that conditioned on $Θi$⁠, $yl(i,j)$ are independent. In practice, we do not observe $Θi$⁠. But, for the experiments in Table I, we know the ground truth regarding the defect size and location. Using the knowledge that defects in all the experiments in Table I were circular and placed at the center of the grid, we used contour plot based thresholding on the capacitive sensor readings under no lift to estimate $Θi$ by $Θior$⁠. Note that, we use the suffix or in $Θior$ to remind that it is not a solely data driven estimator of the support of the defective points but uses oracle information on defects. Figure 5 shows how $Θior$ is estimated for experiment 1 based on the readings from case (1). As the readings in case (1) do not have corruptions due to lift-off $Θior$ is well estimated. All the scan points in the grid which has voltage lower than $−2.8$ were classified as defects and those with voltage higher than $−2.8$ were classified as non-defects. As we had symmetric regular shaped singular defects in all the experiments in Table I, the simple thresholding scheme used here to calculate $Θior$ was adequate in correctly pin pointing $Θi$⁠. Figures 16–18 in  Appendix B shows the contour plots for the rest of the experiments.

In Table II, based on $Θior$⁠, we report the means $Δ0$⁠, $Δ1$ and standard deviations $σ0$⁠, $σ1$ for the defective and the background readings. Using these means and standard deviations, Fig. 6 shows the 95% confidence interval for the voltage readings at the defective and non-defective scan points. From Table II and Fig. 6, we see the following general pattern across the 12 experiments: (a) the difference in means of the voltage between the defective and non-defective area decreases with the increase in uncertainties in the data collection process. The defective scan points always have a lower average voltage; (b) the standard deviation of the signals from the defective areas, however, decreases with lift-off suggesting that though the mean difference in voltage readings between defects and non-defects decreases with lift-off, the readings at the defective point have lower variability and, thus, are more concentrated with lift-off. Figure 7 shows the distributions of the voltage readings at the defective and non-defective scan points for three cases of experiment 1. Figure 19 in  Appendix B shows the voltage reading of the other 11 experiments. Figures 7 and 19 also illustrate the above discussed patterns for changes in voltage readings with lift-off.

In practice, we only observe $Yij$ and not $Θij$⁠. For scenario j of experiment i, the marginal distribution of $Yij$ is

where $πi=P(θl(i)=1)$ is the proportional size of the defect in the grid of inspected points. For any i and j, if we had known $πi$⁠, $Δ0(i,j)$⁠, $Δ1(i,j)$⁠, $σ0(i,j)$⁠, $σ1(i,j)$⁠, then based on observing $yl(i,j)$⁠, we can estimate the support of defective scan points as a point-wise classification problem. In Table III (last three columns), using the oracle values of the above parameters from Table II, we report the Bayes misclassification error rate (BME) for point-wise classification. We also report the defect misdetection or false negative (FN) rate and the false positive rate (FP) of misclassifying non-defects as defects. BME is the sum of FP and FN.

As the proportions of defective scan points were quite small in all the experiments in Table I, the FN rate (column 8 of Table III) were quite high as compared to FP rate (column 7 of Table III). However, in these NDE applications, FN are more harmful than FP, and so, we next consider minimizing the FP rate when the FN rate is controlled below 10%.

With the FN rate of defective scan points controlled below 10%, we can detect at least 90% of the defective scan points which would be adequate to identify defect shapes provided the FP rate is also low. In order to minimize FP rate keeping FN rate controlled below 10%, we consider minimizing the weighted classification loss as in the compound decision theory setup of Ref. 39, where $wij≥0$ is the relative weight of a false positive. For the $j$th case of the $i$th experiment, the weighted classification loss for a data driven estimator $Θ^(i,j)$ of the support of the defective points $Θi$ is

When $wij=1$⁠, we get back the usual classification loss. The Bayes estimator $Θ^w(i,j)$ that minimizes the weighted classification loss is, for location $l∈Λi$ set $Θ^w(i,j)[l]=1$⁠, if

and 0 otherwise. Note that for presentational ease, we have dropped the suffix $(i,j)$ from the mean and standard deviation parameters in the above expression. The detailed calculations are provided in  Appendix A. We calculate the FP and FN rates of this weighted Bayes estimator $Θ^w(i,j)$ as the weights $wij$ are varied over a wide range of values in $(0,1]$⁠.

We consider weights in $S={1,1/5,1/10,…,1/50}∪{1/75,1/100,1/200}$⁠. In Table III, we report the optimal weights $wij$ in $S$ that minimizes the FP rate keeping the corresponding FN rate below 10%. Note that in all the cases the sum of the FP and FN rates for the optimal weighted Bayes classifier is more than that of the BME of the unweighted case which minimizes the unweighted misclassification error. However, the optimal weighted Bayes classifier controls the defect misdetection rate below 10% of the defective scan points. From Table III, we witness that FP rates of the optimal weighted Bayes classifier increases with lift-off but are usually well controlled below 15%. In experiments X and XII, the FP rates are a little higher. However, they also have high FP rates under no lift-off which suggest that better FP vs FN trade-offs can be obtained by slightly increasing the FN rates in those two experiments. Note that the classified defective scan points are clustered together at the defect location. If the wrongly classified non-defective scan points are randomly distributed over the grid they can be easily weeded off and reclassified as long as their proportion is controlled based on local density of defective scans. In Sec. IV, we undertake such an approach that leverages the spatial connectedness of the defective scan points. The low FN and FP error rates for the weighted Bayes estimator in Table III (columns 3 and 4) shows that capacitive sensing-based voltage readings contains significant information to detect defect under lift-offs considered in cases 2 and 3 of Table I. In Fig. 8, the False Positive (FP) vs False Negative (FN) curves for the three scenarios of experiment 1 are shown as the weights are varied in the weighted classification loss. For each curve, the right extreme point denotes the FP and FN proportions for the unweighted loss whereas the left extreme point corresponds to weight $1/200$⁠. The points that corresponds to optimal weights that minimize FP controlling FN below 10% are marked in red. Across all defect sizes considered in Table I, we validate that there is a consistent increase in Bayes misclassification error rates with increase of lift-offs. Note that, as the data used in this paper was solely collected by laboratory experiments, the voltage signals captured by our capacitive sensors had very high signal strength (as shown in Figs. 7 and 19) when there were no uncertainties. All our validation results are based on ground truth that is recovered from signals without uncertainties.

The errors reported in Table III mark the theoretically possible lower limit on the error rates that any decision rule that conduct point-wise classification without borrowing spatial information can produce. To obtain the results in Table III, we have used knowledge of $Θior$ which is not available for real world NDE applications. In this section, we develop a solely data based estimator which does not use any prior knowledge on defect size, shapes, or locations. Achieving the error rates in Table III with solely data driven estimators is not always possible as it would require highly accurate estimates of $πi$⁠, $Δ0(i,j)$⁠, $Δ1(i,j)$⁠, $σ0(i,j)$⁠, and $σ1(i,j)$⁠. Estimating $πi$⁠, $Δ1(i,j)$⁠, and $σ0(i,j)$ in the presence of noise is extremely difficult and lead to significant estimation error.^40–43 To mitigate these estimation problems, we develop a spatially adaptive procedure to estimate $Θi$⁠.

First, a naïve thresholding estimator $Θ^T[l]=1{yl<cα}$ is considered, which classifies the $l$th scan point as defect if its voltage is below the $α$th quantile of voltage readings over the grid. In Table IV in  Appendix B, we report the performance of $Θ^T$ for a range of $α$ values. As $α$ increases, coverage rates of the defect (defined as $1$-FN) increase at the cost of increased FP. We observe that while $Θ^T$ works very well for case 1 when there is no lift-off, its performance drastically deteriorates as signals are corrupted due to increase in lift-off uncertainty. We next consider denoising the signals collected under lift-off uncertainty by using the adaptive shrinkage estimator proposed in Ref. 44. For denoising, we use the denoiseR package of Ref. 45 that estimates a low-rank signal assuming Gaussian noise by minimizing the risk estimation criterion in Refs. 44 and 46. On the denoised data, we again tried the elementwise thresholding described above. We observed that under lift-off its performance is not much better than naïve thresholding of voltage signals (see Table IV in  Appendix B). Next, we consider a spatial adaptive estimator that is based on local variances in the capacitive imaging data. For voltage readings collected over any grid $Λ$⁠, consider a symmetric neighborhood $Nl$ around the $l$th scan point in the grid. $Nl$ is a set of grid points that includes the scan point and its neighbors. Consider the variance $vl$ associated with $Nl$⁠,

We compute variance over each point in the grid. For $Nl$⁠, we consider a square window with 25 scan points centered around the $l$th scan point. We use this local variance filter on the voltage readings and estimate the support of defective grid points as

where $cβ1=quantile(yl:l∈Λ;β1)$ and $cβ2=quantile(νl:l∈Λ;1−β2)$ are the quantiles of the voltage readings and local variances. We universally set the values of $β1$ and $β2$ at 40%. Note that, as we use large values of the tuning parameters $β1$ and $β2$⁠, the estimator is not very sensitive to minor changes in the values. Figure 9 shows $Θ^S$ for case 3 of experiment I. Consider the decomposition,

where $Θ^S1[l]=1{yl<cβ1}$ and $Θ^S2[l]=1{vl>cβ2}$⁠.

Figure 9 (top left plot) shows $Θ^S1$ contains almost all defective locations along with a large proportion of non-defective scan points. These non-defective scan points are not uniformly distributed over the grid which would have made our task of filtering them easier but are concentrated in the top left quadrant. This is a result of lift-off uncertainty compounded with probe tilting. Applying the filter $Θ^S2$ based on the local variances (Fig. 9, top right plot) helps in cleaning these non-defective scan points in $Θ^S1$⁠, keeping most of the defective scan points. The resultant estimator $Θ^S$ (see Fig. 9, bottom left image) still contains sporadic non-defective scan points. As these non-defective scan points are not concentrated unlike in $Θ^S1$⁠, we can easily weed them off by using a uniform local filter. For this purpose, we consider the averaging filter FA which divides the grid $Λ$ into 100 rectangles and finds the average of signals over each of those rectangles. We apply FA on $Θ^S$⁠. This smooth (Fig. 9, bottom right plot) out the sporadic non-defective misclassified scan points keeping the concentrated mass of correctly detected scan points intact. Finally, using these as the local weights on the original voltage readings, we consider the spatially adaptive estimator: $Θ^SA=FA(Θ^S)∗Y~$⁠, where $Y~=max(Y)−Y$⁠.

We conduct a binary classification of grid points as defects by thresholding $Θ^SA$ and considering $Θ^TSA[l]=1{Θ^SA[l]>t1−α}$⁠, where $t1−α=quantile(Θ^SA[l]:l∈Λ;α)$⁠. In Table IV, we report the FP and coverage rates of the proposed estimator. As shown in recent works,⁴³ using spatially adaptive local weights can tremendously help in recognizing underlying patterns from very noisy imaging data. Here, thresholding the spatially adaptive estimator $FA(Θ^S1∗Θ^S2)$ instead of the original signal or low rank model based denoised signal, we observe that we can reasonably filter out the effects of probe tilt and lift-off based uncertainties.

Table IV in  Appendix B and Fig. 10 compares the efficacy of the proposed method with naïve thresholding and low rank signal denoising. They show that using the aforementioned spatially adaptive estimator based on local features of the imaging data, it is possible to attain appreciable coverage and low false positive rate in detecting defective scan points under the varied uncertainties considered in the paper. From Table IV and Fig. 10, we observe that thresholding based on raw voltage signals is effective under no lift-off but provides very low coverage as lift-off increases. Low rank model based denoising also does not help. It reduces false positive rate a bit but suffers from insufficient coverage. The proposed spatial adaptive estimator, however, can provide considerably high coverage with very tight control on the false positive rates. As such with false positive rates controlled at only 2%, the spatially adaptive estimator provides sufficient coverage under lift-off in all experiments barring VII, X, XI, and XII. When the limit on the false positive proportion is raised to $10$%, the spatially adaptive estimator provides sufficient coverage across all the 36 regimes considered. It is to be noted that there is discernable difference in the performance of the estimator (as shown in Fig. 10) between regimes $1$–$30$ and $31$–$36$⁠. This is due to the fact that regimes $31$–$36$ contain defects of considerably different characteristics than those in the previous regimes. Overall, the results in Table IV and Fig. 10 show that the spatially adaptive estimator based on local features of the imaging data can attain appreciable coverage and low false positive rate in detecting defective scan points on the grid under the different lift-off uncertainties considered in the paper.

In (6), we considered fixed sized rectangular neighborhoods. They can be easily replaced by Gaussian kernel filters that weighs each scan points in the lattice inversely proportional to the exponent of its distance from the scan point under study. In this context, the proposed filters are similar to signal processing techniques that uses Gaussian filters. However, we found that the local variances in these data sets contain important information, and so, we filtered the variances. In this attribute, we differ from traditional filtering where filtering is done based on signal intensity or amplitude and not variances.

All the experiments considered in Table IV contained a single defect. Next, we consider two experiments (MI and MII) where we had multiple coexisting defects. Each experiment had three scenarios as before pertaining to no, 3, and 5 mm lift off along with other embedded uncertainties. Figure 11 shows the structure of the defects. There were 4 and 12 defects of varying shapes and sizes in experiments MI and MII, respectively.

Figure 12 shows the image plots of the voltage readings collected by our capacitive sensing probe.

Table V in  Appendix B, we report the false positive proportion and defect detection coverage rate that was achieved by applying our proposed spatial denoising algorithm on the capacitive sensing data. We observe that our proposed method not only outperforms the aforementioned competing algorithms but also obtains high coverage and low false positive rate in detecting the multiple, disconnected defect clusters.

We develop a robust NDE methodology using short waves based capacitive sensing. Our proposed method is cheap as it does not use complex expensive sensors. It does not need any expert supervision and can produce instantaneous defect identification under highly robust conditions by using spatially adaptive denoising. Moreover, the apparatus involved for data collection is light and flexible and can be implemented across a wide range of NDE applications.

We demonstrate high efficacy of the proposed NDE methodology in non-smooth samples under varied lift-off uncertainties. We provide a Gaussian mixture model based disciplined analysis to check when defective scan points in the sample can be detected with high accuracy and with low false positive misclassification rates for non-defective scan points. This would lead to identification of defect characteristics such as location, size, and shape with desired accuracy. The results are validated on experimental data on samples containing single defect of three different sizes as well as on samples containing multiple coexisting defects. We show that the proposed spatially adaptive denoising algorithm leverages the spatial contiguity of defective scan points and can well identify defects based on capacitive sensing data that is convoluted with the uncertainties due to lift-off, probe tilt, and low sampling rates.

The performance of the defect defection algorithm is sensitive to the threshold and neighborhood size parameters. Larger threshold value will increase coverage but also result in higher false discovery rates. The threshold value needs to be set based on applications. Here, we have considered fixed neighborhood shapes and sizes. Data driven choices of neighborhood size can further improve our methodology. This can be incorporated by using Gaussian kernel density based filters instead of the square grids used in Sec. V and then to adaptively choose the kernel bandwidth. We plan to develop theoretical support for selecting these hyper-parameter in future.

Our method would need some manual adaptation under varying weather conditions and humidity levels. We need the RF source to be set near the resonant frequency of the probe which was estimated to be $5$ MHz for our lab experiments. If the medium between the sample and the probe is much different from our experimental setup, then the RF source needs to manually reset to the corresponding resonant frequency. The problem can be solved by using multiple probes each operating at different frequencies. However, it would require multi-parametric estimation. We aim to study it in our future works. Another potentially useful direction of research would be to study the applicability of the spatially adaptive estimator developed in this paper for analyzing NDE data collected with uncertainties by other NDE methods.

The work was supported in part by the DOT/PHMSA, Slow Crack Growth Evaluation of Vintage Polyethylene Pipes (Core Research Program) with Award No. DTPH5615T00007. The work was also partially supported by the DOT/PHMSA CAAP program with Award Nos. 693JK31850007CAAP and 693JK32050002CAAP. We thank the associate editor and two referees for constructive suggestions to improve the paper.

The authors have no conflicts to disclose.

S. Mukherjee: Conceptualization (equal); Formal analysis (lead); Methodology (lead); Software (lead); Writing – original draft (equal). D. Kumar: Data curation (equal); Methodology (equal); Software (equal). L. Udpa: Methodology (equal); Resources (equal); Supervision (equal). Y. Deng: Conceptualization (equal); Funding acquisition (equal); Methodology (equal); Project administration (equal); Supervision (equal); Writing – review & editing (equal).

The data that support the findings of this study are available from the corresponding author upon reasonable request.

By Theorem 2 of Ref. 39, we know that for experiment $i$ and case $j$⁠, the weighted classification loss is minimized by the estimator,

where $ϕ(y|Δ,σ)$ denotes the normal density with mean $Δ$ and standard deviation $σ$ evaluated at $y$⁠. Taking logarithm on both sides of the inequality above, $Θ^w(i,j)[l]$ reduces to

The weighted Bayes classifier in the paper follows directly from above by noting that $log⁡ϕ(y|Δ,σ)=−(y−σ)2/2/σ2−log⁡σ$⁠. The FP and FN error rates of this estimator are given by

We calculate these by Monte Carlo simulations and then optimize the weights for each $(i,j)$ pairs. The results are reported in Table III which also contains the FP and FN rates for unweighted classification loss when $wij=1$⁠.

Tables IV and V described in Sec. V are presented here. We also provide pictorial description of the capacitive sensing data for all the 12 regimes. In the main paper, Figs. 4, 5, and 7 describe the different attributes of the data from the three cases of experiment I. In Figs. 13–19, we provide the corresponding plots for the other experiments. Figures 13–15 supplements Fig. 4, Figs. 16–18 supplements Fig. 5, and Fig. 19 corresponds to Fig. 7 in the main paper. These figures display reproducible signaling patterns that are described in the main paper across the concerned experiments. These tables and figures are provided after the references.