A lifting-wavelet-based iterative thresholding correction for atomic force microscopy images with vertical distortion

By virtue of its atomic-scale resolution, atomic force microscopy (AFM) has provided many outstanding results in fields including biological science,^1–6 medical research,^7–10 physical measurement,^11–13 and microelectronics technology.^14,15 However, the complexity of AFM¹⁶ means its imaging quality is easily degraded by many factors, such as the hardware performance of the scanning system, the scanning algorithm, environmental noise, and data processing.¹⁷ Hence, distortions such as drift, artifacts, and deformation are common in AFM imaging,¹⁸ which inevitably limits its applications.

When AFM is applied to morphology characterization, the sample must be placed horizontally on the stage and scanned line-by-line with the probe driven by a scanner. The morphology image of the sample is thus constructed using the acquired data. Hence, vertical distortion may be caused by vertical drift and illusory slopes in practical AFM applications.¹⁹ More specifically, environmental noise (including acoustic and electronic noise), temperature changes, probe contamination, and mechanical vibration during the scanning process may cause vertical drift between the probe and the sample,²⁰ leading to the induction of artificial ripples in the morphology image. Furthermore, the creeping of the piezoelectric scanner and the inclination of the installation in the vertical direction can also induce an illusory slope overlayed on the image of the sample surface,^21,22 causing the obtained image to show an uneven brightness distribution. In summary, the sample morphology profiles of the acquired images are highly anamorphic due to vertical distortion, which reduces the quality of AFM images.

Upgrading hardware to correct the distortion caused by vertical drift is a feasible solution that reduces vibration during the scanning process. Possible upgrades include installing the AFM hardware in a low-noise and clean environment and employing shock absorbers such as mass-spring systems and sound insulation covers. Nevertheless, hardware improvements alone are not enough since the vertical drift is induced by many random factors.²³ Moreover, upgrading hardware also increases the economic cost and amount of space required for experiments, limiting the applications of AFM in life science and other fields.²⁴ Researchers have thus proposed many correction algorithms to eliminate vertical drift. An adaptive contact-mode imaging method has been proposed to actively inhibit probe vibration by reducing the interaction force between the probe and the specimen.²⁵ A robust feedforward controller has also been designed to effectively suppress the probe vibration caused by acoustic noise,²⁶ and a robust controller combined with an iterative learning controller has been designed to effectively reduce the mechanical vibration of the XY-scanner and improve the imaging quality with a high scanning speed.²⁷ A modified repetitive controller based on a cross-coupling compensation approach has been proposed to compensate for the periodic disturbances of the AFM piezoelectric scanner.²⁸ For the imaging lifting algorithms, a self-intersecting scanning path-based method has been designed to distinguish drift from topographic features for height drift correction.²⁹ An approach based on image processing and morphological prediction has been proposed to correct vertical drift by extracting distorted bases.³⁰ A spiral scanning path has been designed to perform real-time online drift correction on scanning blocks.³¹ A generative adversarial network has been proposed to eliminate the superimposed envelope on top of the true profile caused by vertical drift.³² In addition, some correction algorithms for other distortions are also suitable for the correction of vertical drift. A three-axis correction method based on orthogonal scanning image comparison has been proposed for position drift and works by solving the minimum error constraint.³³ An algorithm based on cross-correlation and the least squares technique has been proposed to determine the drift details for image correction.³⁴ A method using two orthogonal fast scanning AFM images has been adopted to calculate the drifts and to correct the distorted image by reasonably segmenting and comparing with the corresponding image blocks.³⁵ A deep learning method based on synthetic data training has been designed for AFM image correction and can eliminate artifacts and noise induced by the scanning process.³⁶ A related study involving the effects of weights has also provided guidance for various AFM image processing and correction applications.³⁷ 

Moreover, to eliminate illusory slopes, a line-by-line fitting method based on the least squares method is commonly employed in commercial AFM systems. However, the effectiveness of this approach is limited due to the lack of specific analysis of different sample characteristics. For this reason, many correction algorithms capable of removing illusory slopes have been reported. A recursive least squares method has been proposed to provide an accurate three-dimensional representation of an illusory slope.³⁸ An adaptive feature recognition algorithm based on image edge detection has been proposed to automatically identify sample features to achieve final line-fitting-based correction.³⁹ An image flattening method based on image segmentation has been designed to eliminate the slope by performing surface fitting after removing specimen profiles.⁴⁰ A robust line fitting method based on sparse sampling consensus has been proposed to correct the distortion caused by the illusory slope and improve the quality of AFM images.⁴¹ 

Inspired by the above research, a lifting-wavelet-based iterative thresholding correction method is proposed here to adaptively correct the vertical distortion caused by vertical drift and illusory slopes. As a robust correction algorithm with strong universality, it avoids complex hardware structure redesign and multiple hyperparameter adjustments. A data preprocessing strategy is first applied to filter out the obvious specimens that disturb the distortion modeling by internal double screening based on lifting wavelet analysis (LWA). As partial vague specimens persist after rough screening, a refined screening strategy based on an iterative thresholding algorithm (ITA) is proposed to separate the base from the specimens precisely. Afterward, the profile distortion is modeled by line fitting the fined screening base data. On this basis, the corrected image is obtained by subtracting the modeled distortion from the raw profile line by line. Finally, the results of experiments and applications verify the effectiveness of the proposed method.

The remainder of the paper is organized as follows: Section II presents an analysis of vertical distortion and the basic idea of the proposed correction method. Section III implements the data preprocessing and presents the technical detail of distortion modeling and correction. Section IV performs the experiments and applications to validate the effectiveness and universality of the proposed method. The paper is concluded in Section V.

Figure 1(a) presents the distorted morphology image of a standard two-dimensional (2-D) grid sample generated by AFM. Normally, for a standard 2-D grating sample, square grids of consistent height are evenly distributed on the flat substrate. Therefore, the brightness distribution of the undistorted image should be highly consistent, and the grid morphology should be clear. However, it can be seen from Fig. 1(a) that, due to vertical drift, the image is obviously anamorphic, as reflected in the production of non-existent ripples, which seriously damages the uniformity of the overall brightness distribution and makes the grids blurry. Therefore, the distorted image cannot reliably reflect the true morphology of the sample. Moreover, the ripples in the image are mainly along the fast scanning direction, and the distortion distribution is uncertain in the slow scanning direction. Furthermore, the distortion on each fast scanning line is relatively consistent since the scanning time of each fast line is generally short; however, the distortion on different fast scanning lines is inconsistent due to the randomness of the vertical drift.

Furthermore, The distorted image induced by the illusory slope of a standard one-dimensional (1-D) grating sample is shown in Fig. 1(b). The undistorted image of a standard 1-D grating sample should also exhibit a uniform brightness distribution and clear grid contours. However, the overall brightness of the scanned image is uneven since the illusory slope added to the real image results in higher morphology for the profile lying on the slope. More specifically, the brightness on the left side of the image is significantly higher than that on the right side, which does not accurately reflect the true morphology of the sample. The illusory slope on each fast scanning line is approximately linear.

Based on the above analysis, the vertical distortion can be regarded as additive noise on each line of the morphology image. It is thus feasible to eliminate distortion through a line-by-line correction approach. This paper proposes a lifting-wavelet-based iterative thresholding correction method to address the problem caused by vertical distortion in AFM imaging. For each distorted profile along the fast scanning direction, the correction is divided into two steps: data preprocessing and distortion modeling. The data preprocessing step roughly filters out most obvious specimen profiles by internal double screening based on lifting wavelet analysis. On this basis, an iterative thresholding algorithm is applied to screen out precise base profiles, which are further used to simulate the distortion. Next, an accurate representation of the distortion is obtained by line fitting the data screened out by the refined screening. Finally, the image is corrected by removing the distortion representation from the raw profile line by line. The overall flowchart of the proposed method is shown in Fig. 2.

Although line fitting could screen out the data suitable for distortion modeling, the result of the rough screening is inaccurate due to the interference of convex or concave specimen morphology. The specific manifestation of this is that the fitted line is elevated by convex specimen areas and is simultaneously pulled down by concave specimen areas, resulting in some base parts suitable for distortion modeling being misclassified into the set P, thus reducing the accuracy of the distortion modeling in subsequent calculations. Therefore, it is inexact to rely solely on line fitting to filter out the interference data for distortion modeling.

To identify the base parts misclassified into P, a secondary screening strategy based on lifting wavelet analysis is applied to generate more reliable modeling data. In particular, although the profile height of the scanned sample is greatly distorted, the height gradient of the distorted profile can still accurately reflect the distribution of sample features. Therefore, it is possible to utilize the height gradient obtained from wavelet analysis to distinguish specimens from the base parts. This paper applies lifting wavelet transformation combined with a filtering operation to analyze and extract height gradient features of distorted profiles and provide more complete base data for further distortion modeling.

After the lifting wavelet analysis of the distorted profile, the large amplitudes in the reconstructed detail component correspond to specimen outlines with obvious height changes. Low amplitudes correspond to outlines with smooth morphological heights, including the distorted base and partial specimen profiles. Hence, it is feasible to use the detail component to correct the misclassified data in the rough screening so as to construct more reliable modeling data.

After data preprocessing, an iterative thresholding algorithm is proposed for accurate distortion modeling using $Q̃0$⁠, based on which the distorted image is effectively corrected to produce a high-quality AFM image. A specific flowchart of the modeling and correction methods is presented in Fig. 3. The key strategy of the ITA is to accurately screen out the optimal data for distortion modeling, and this process is implemented iteratively under a certain error constraint.

The experimental platform applied in this paper is the Flex-FPM AFM system produced by Nanosurf. The scanning process is performed in contact mode in air, and the scanning frequency is set to 1 Hz. The proposed method is verified by correcting the distorted images for vertical distortion. The adopted samples include gratings with periodic regular features, a polymer sample, and an Escherichia coli (E. coli) sample with smooth topographical profiles. Specifically, 1-D and 2-D gratings are scanned in experiments for quantitative evaluation of the proposed method. Moreover, a correction to the distorted morphology images of E. coli cells is performed to demonstrate the performance of the proposed method for a biological sample with practical significance. This shows that the method could provide more accurate sample morphology and assist in practical micro-nano research.

In the 1-D grating experiments, multiple comparative results were obtained to verify the performance of the proposed method over the commercial method. The commercial method involves first performing line fitting on the distorted morphological profile of each row using the least squares method, then implementing the correction by subtracting the fitted line from the distorted profile row by row.

The selected standard flat 1-D grating sample height was about 0.1 μm with a fixed period of 3 μm. The morphological image with vertical distortion is presented in Fig. 4(a). The image exhibits an uneven brightness gradient due to the illusory slope and vertical drift. Meanwhile, the feature height of the sample is also distorted and does not truly reflect the actual shape of the sample. The distorted image was corrected using the commercial method, with the result shown in Fig. 4(b). Although the problem of an uneven brightness distribution was greatly improved by the commercial method, a slope remains on the grid surface due to the lack of sample profile analysis, which is reflected in the fact that the brightness in the upper left of the image is lower than the overall average brightness of the image. Furthermore, the result of the proposed method in Fig. 4(c) shows an effective correction has been achieved through accurate distortion modeling with analysis of sample profiles. It can be seen that the brightness of the grid surface is consistent.

To further visually represent the correction effect, the cross-section profiles of the distorted and corrected images along the scanning direction and the orthogonal direction, shown as the dot-dashed lines in Fig. 4, are sampled in Fig. 5. It can be seen from Fig. 5(a) that the profile in the scanning direction represented by the black solid curve is seriously distorted, with an obvious slope adding to the base. The modeling result of the commercial method is shown as the pink line with circles, based on which the profile is corrected to give the pink dotted curve, which still shows a slope. In contrast, thanks to data preprocessing and fine iterations, the proposed method achieves accurate distortion modeling, as shown by the blue line with stars, and the corrected result, represented as the blue dot-dashed curve, is satisfactory since the base height is consistent. Furthermore, for the proposed method, the correction for the distorted image of the 1-D grating sample was completed within 1.66 s, demonstrating high calculation efficiency.

Furthermore, the profiles in the orthogonal direction are shown in Fig. 5(b) and further verify the effectiveness of the proposed method. Specifically, the expected profile corresponding to the pink dot-dashed line in Fig. 4 is a straight line [the red dashed line in Fig. 5(b)]. Unfortunately, the vertical distortion in the orthogonal direction is more obvious, which results in the curved profile displayed as the black solid curve in Fig. 5(b). The corrected results demonstrate that the profile corrected by the proposed method (the blue dot-dashed curve) is closer to the desired profile than that of the commercial method (pink dotted curve).

Moreover, indices including the root mean square error (RMSE), the peak signal-to-noise ratio (PSNR), and the structural similarity index measurement (SSIM), were chosen to quantitatively evaluate the quality of the corrected images. Quantitative comparisons of the distorted image and corrected images of the 1-D grating are shown in Table I. The SSIM of the image corrected by the proposed method reaches 0.9606, exceeding that of the commercial method. Furthermore, the RMSE of the proposed method (32.5220) is lower than that of the commercial method (37.0026), and the PSNR of the proposed method (17.8873 dB) is higher than that of the commercial method (16.7662 dB), further validating the efficacy proposed method.

A 2-D grating sample with a profile height of 20 nm and a fixed period of 3 μm was scanned for another experiment. The morphological image of the 2-D grating includes square grids with a uniform distribution. Furthermore, to fully demonstrate the superiority of the proposed method, apart from the commercial correction method, three more reference methods were selected to comprehensively compare with the proposed method, including two advanced segmentation-based correction methods and an improved line-fitting-based correction method. The segmentation-based correction methods first adopt object detection to extract the flat base. Line fitting is then performed for the base part of each scanning line to simulate distortion and achieve further correction. More concretely, an Otsu-thresholding-based baseline correction method, combined with least-squares-based line fitting technology, was selected for comparative experiments due to its strong correction performance.³⁹ Another selected correction method uses an adaptive thresholding (AT) algorithm combined with sliding-window line fitting technology and has been validated for correcting distorted AFM images.⁴⁰ An improved line-fitting-based correction method was also referenced for performance comparison.⁴¹ This method is a sparse sample consensus (SPASAC)-based line fitting method that achieves precise distortion modeling and correction by setting a global constraint condition for all distorted profiles. On this basis, richer correction results were obtained to fully verify the superiority of the proposed method.

Figure 6 presents the distorted and corrected images of the 2-D grating. To compare the correction details more intuitively, the same area in the red dotted box in each image is enlarged. Figure 6(a) shows the scanned image with vertical distortion. The grid shape is blurred with an uneven brightness distribution, and the image has obvious ripples. The image corrected by the commercial method is displayed in Fig. 6(b). Although the morphological image is largely restored, there are still obvious artifacts between grids, which cannot convincingly represent the actual profile of the sample. Figures 6(c) and 6(d) show the corrected results of the Otsu-based method and the AT-based method, respectively. Benefitting from fine distortion modeling with the analysis of sample profiles, these two methods achieve better correction performance with more uniform brightness and realistic morphological features. However, some artifacts persist as the brightness is still uneven in the corrected images, especially in the localized amplification regions in the red boxes. Moreover, Fig. 6(e) shows the correction result of the SPASAC-based method, which appears as a high-quality topography image with a flatter grid surface and base part. Nevertheless, some visible artifacts at the edges of the grids remain since the global constraint condition for distortion modeling is difficult to adapt to all the different profiles, as reflected by the ripples in the enlarged area. Finally, the corrected result of the proposed method is shown in Fig. 6(f). The brightness of the image is more even than those of the other reference methods and without artifacts. In addition, the correction time for the distorted image of the 2-D grating sample is 1.60 s for the proposed method.

Similarly, two orthogonal cross-section profiles in Fig. 6 are sampled to further contrast the correction effects. Figure 7(a) shows the distorted and corrected profiles along the scanning direction, where the black solid curve represents the distorted profile with an obvious illusory slope. The modeling result of the commercial method is shown as the pink line with circles, and the corrected profile from the commercial method is presented as the pink curve with hexagons, which preserves an obvious slope induced by inaccurate distortion modeling and deviates from the expected baseline (the black line with rhombuses). Furthermore, the correction results of the Otsu-based method (the green curve with crosses) and the AT-based method (the brown dotted curve) also show visible deviations from the expected baseline, which are caused by the inexact line fitting of the Otsu-based method (the green line with squares) and the AT-based method (the brown line with pentagons). Thanks to the precise analysis of sample features, the correction results of the SPASAC-based method (the red dashed curve) and the proposed method (the blue dot-dashed curve) exhibit good performance with appropriate base heights. The profiles along the orthogonal direction are shown in Fig. 7(b). The expected profile corresponding to the pink dot-dashed line in Fig. 6 is a straight line [the black line with rhombuses in Fig. 7(b)]. However, the actual profile is severely distorted, as shown by the black solid curve in Fig. 7(b). It can be seen from the correction results that, compared with the four reference methods, the profile corrected by the proposed method is closer to the expected one and also conforms to the standard height of the grid, thus verifying the superiority of the proposed method.

Moreover, the quantitative results shown in Table II also indicate that the correction effect of the proposed method is better than those of the reference methods. The RMSE of the image corrected by the proposed method is the lowest at 31.7267, the PSNR is the highest at 18.1023 dB, and the SSIM is the highest at 0.9347. In summary, the effectiveness of the proposed method has been verified by the convincing results of the comparative experiments.

Furthermore, a PS-LDPE-12M sample, which is a blend of polystyrene and polyolefin elastomer coated onto a silicon substrate with approximate circular shapes, was used to verify the universality and robustness of the proposed method. For the PS-LDPE-12M sample, the correction time was 2.29 s with the proposed method.

The correction results of different methods for the PS-LDPE-12M sample are shown in Fig. 8, where the red dotted elliptic curves indicate some obvious artifacts in the images. Figure 8(a) shows the raw image with serious vertical distortion. It has varying brightness and ripples that degrade the image quality. The image corrected by the commercial method in Fig. 8(b) still has some artifacts around some feature areas, causing an uneven brightness distribution. In addition, the corrected results of the other reference methods in Figs. 8(c)–8(e) induce partial obvious artifacts, although the image brightness distribution is more even than that of the raw image and the image corrected with the commercial method. The image corrected with the proposed method is presented in Fig. 8(f). It provides satisfactory performance without obvious artifacts, thereby further verifying the effectiveness and generality of the proposed method.

Considering practical applications, a biological sample of E. coli with irregular and smooth profiles was scanned to verify the universality and robustness of the proposed method. Normally, E. coli is a kind of rod-shaped cell with micron-scale length and diameter, cultured on a flat surface and randomly distributed with varying morphological heights. Therefore, in actual observations, the E. coli cells should be easy to identify from a flat substrate.

The corrected results from the proposed method and four reference methods are presented in Fig. 9. To show the correction performance more clearly, some obvious artifacts are marked as yellow dotted elliptic curves in the images. The distorted image in Fig. 9(a) shows obvious ripples with different brightnesses caused by vertical drift. Furthermore, an illusory slope is induced in the image along the scanning axis, which results in the approximate gradient of the brightness in the overall image. After commercial correction, although the major distortion is eliminated in Fig. 9(b), partial prominent artifacts around the specimens are actually magnified. Moreover, in Figs. 9(c)–9(e), the corrected images of the other reference methods provide effective performance with even brightness distribution in the base areas. However, the correction results with the details in the yellow dotted elliptic curves still retain partial artifacts in Figs. 9(c)–9(e), which cannot reflect the true morphology of the E. coli sample. More concretely, the Otsu-based method has poor robustness due to inaccurate identification of partial cells and border profiles, resulting in multiple artifacts around the specimens and impairing the uniformity of the overall brightness. For the image corrected by the AT-based method in Fig. 9(d), the issue of residual artifacts is significantly mitigated thanks to the block-based adaptive threshold recognition algorithm. Nevertheless, partially obvious artifacts persist in the yellow dotted elliptic curves. Furthermore, the image corrected by the SPASAC-based method is shown in Fig. 9(e). Since a single constraint is difficult to adapt to all the distorted profiles, the image quality is impaired by the remaining ripples, as reflected in particular in the yellow dotted elliptic curves. In contrast, the image corrected by the proposed method in Fig. 9(f) is of high quality with an even brightness distribution in the non-specimen area, which indicates that the correction result is consistent with expectations, thereby verifying the effectiveness and generality of the proposed method.

In addition, to further highlight the advantages of the proposed method, more corrections were implemented on two different E. coli samples under more challenging conditions to verify the robustness of the proposed method. Specifically, the first E. coli sample was placed on a slope along the slow scanning direction, and the second sample was placed on a higher slope in the fast scanning direction, accompanied by vertical drift in the slow axis. The distorted images with different cell distributions are shown in Figs. 10(a1) and 10(a2), respectively. Obvious artifact areas are marked by green elliptic curves. The corrected results of the proposed method are shown in Figs. 10(b1) and 10(b2). They show high-quality correction with a uniform brightness distribution and clear cellular features, thus demonstrating the efficiency and reliability of the proposed method.

In this paper, a lifting-wavelet-based iterative thresholding correction method has been proposed to eliminate the additive noise caused by vertical distortion in AFM imaging. The proposed method adopts a line-by-line precise correction strategy to achieve high-quality AFM imaging. The core of the correction is the precise modeling of the distortion for each scanning row. First, a data preprocessing approach based on double screening is applied to roughly filter out the sample topography height. Next, an iteration thresholding algorithm is employed to finely pick out the topography height data for accurate distortion modeling. Afterward, the image is corrected by subtracting the distortion model from the distorted profile line by line.

Experiments and comparisons were carried out to verify the effectiveness of the proposed method. The results showed that the proposed method achieves effective correction for distorted images containing sufficient substrate areas. However, for distorted images with features that almost cover the imaging area, the proposed method may suffer from poorer performance due to a lack of sufficient distortion modeling data. Hence, future work will focus on developing distortion correction algorithms suitable for a broader range of samples.

This work was supported by the National Natural Science Foundation of China under Grant No. 21933006.

The authors have no conflicts to disclose.

Data sharing is not applicable to this article as no new data were created or analyzed in this study.

Yifan Bai received a B.S. degree in intelligence science and technology from Hebei University of Technology, Tianjin, China, in 2022. He is currently working toward an M.S. in artificial intelligence at the Institute of Robotics and Automatic Information System, College of Artificial Intelligence, Nankai University, Tianjin, China. His research interests include atomic force microscopy imaging.

Yinan Wu received a B.S. degree in intelligence science and technology from Nankai University, Tianjin, China, in 2013. He received his Ph.D. degree from the Institute of Robotics and Automatic Information System, Nankai University, Tianjin, China, in 2018. From 2019 to 2021, he was a postdoctoral fellow at the same institute, where he is now an associate professor. His research interests include nanopositioning and atomic force microscopy imaging.

Yongchun Fang received a B.S. degree in electrical engineering and an M.S. degree in control theory and applications from Zhejiang University, Hangzhou, China, in 1996 and 1999, respectively, and a Ph.D. degree in electrical engineering from Clemson University, Clemson, SC, in 2002. From 2002 to 2003, he was a postdoctoral fellow with the Sibley School of Mechanical and Aerospace Engineering, Cornell University, Ithaca, NY. He is currently a professor at the Institute of Robotics and Automatic Information System, College of Artificial Intelligence, Nankai University, Tianjin, China. His research interests include AFM-based nanosystems, visual servoing, and the control of underactuated systems, including overhead cranes.