Vascular recognition system based on photoacoustic detection

The physical and virtual exposure of humans is continually expanding with the advancement of global information and networking. Existing biometric methods such as fingerprint recognition, face recognition, iris recognition, and vein recognition are mainly based on physiological or behavioral features for identity authentication. Moreover, one's biometric features are unique, immutable, and innate, thereby providing convenient access and antitheft solutions. Therefore, biometrics has gradually become the mainstream identification method worldwide.^1–4 However, the rapid development and widespread application of biometrics have led to methods that involve spoofing, copying, or forging one's biometrics, thus posing a significant threat to such systems.^5,6 Willis and Matsumoto et al. proposed a method of forging fingerprints with molds, where the rate of deceiving a fingerprint identification system exceeded 60%.^7,8 Masuko et al. attacked a speech authentication system with speech synthesis programs and reported a false acceptance rate of 20%.^9,10 Similarly, Matsumoto and Thalheim proposed to deceive an iris recognition system with high-resolution images. The results revealed that only one fake iris failed to deceive the system.^11,12 Thus, the effective prevention of illegal users from spoofing identification systems has garnered scholarly attention.^13–15 

Recently, several studies have been conducted on enhancing the security of biometrics.^16–18 The recognition system first distinguishes whether the target information is a biological feature from a legitimate user, thus resisting spoofing attacks and ensuring the security of the system. In addition, certain useful biometric detection technologies have been developed by researchers and these have been mentioned in Refs. 19–22. Existing biometric detection technologies include methods for detecting blink behavior and turn, dynamic changes in faces through optical flow, facial textures, and body temperature of living persons through thermal infrared technology; in addition, present biometric detection technologies include the multispectral detection of light reflected from the human face and time-of-flight (TOF) camera ranging.^23–30 These methods effectively improve the biopsy performance of systems. However, they also pose new challenges to the preprocessing stage in systems. For instance, human–computer interaction leads to poor concealment of systems. Moreover, attackers can derive new breaching methods, and the purchase of additional equipment increases the system cost. Furthermore, factors such as lighting conditions, facial expressions, and occlusion tend to reduce the robustness of the system.

Photoacoustic imaging (PAI) is a laser-excited, ultrasonic detection imaging technology that has high optical contrast, high resolution, and a large penetration depth. In recent years, PAI has been widely used in multiple interdisciplinary areas such as tumor detection, drug evaluation, atherosclerotic plaque identification, and subcutaneous vascular imaging.^31–33 When biological tissue is irradiated with a pulsed laser, the tissue instantly absorbs the laser energy and expands because of the heat, consequently generating ultrasonic waves, i.e., photoacoustic (PA) signals that are detected using an ultrasonic transducer.^31–36 The amplitude of the detected signal reflects the optical absorption performance of the sample; this information can be employed to reconstruct the contrast image of the vasculature against a background. The delay of the maximum-value arrival time of the PA signal represents the depth of the vascular system. Moreover, PAI has attained a micron-level spatial resolution³⁵ and an adequate vascular image quality; furthermore, the image preprocessing is facile. Therefore, PA signals exhibit both optical and acoustic characteristics as well as imaging capabilities that are appropriate for vasculatures.^31–34 

As such, existing PA modules can enable covert interactions between people and systems via a cover glass or other planar materials that can transmit ultrasonic waves in water or via an ultrasound coupling gel that can enable coupling and act as a target platform. Furthermore, they can realize multimodal detection without any additional equipment, overcome shadowing and pollution, and easily capture vascular characteristics in the field of biometric recognition. PA modules have low cost, great robustness, and a simple data acquisition process. Although certain studies have reported the use of PA in the field of biometrics,^35–38 the implementation of PA in the antispoofing of biometrics has not been considered to date. Herein, we used the peak-to-peak value and the maximum arrival delay of the PA signal as the biological characteristics in a simple PA system to resist spoofing attacks. The peak-to-peak value reflects the optical absorption characteristics of vasculatures, and the difference in maximum arrival time reflects the depth of vasculatures.^31–45 In the stated PA detection process, the user is not required to actively cooperate with the system. Therefore, the detection method is concealed and does not attract the attention of the attacker. Thereafter, the system detects fake vascular biometrics based on the optical absorption and depth of vasculatures. Moreover, the PA system could eliminate adverse external factors such as masking and pollution owing to the poor optical absorption properties of these pollutants. Thus, a gallery with only real vasculatures was established, and the quality of the vascular images was found to be adequate. Finally, the preprocessing, feature extraction, and recognition of real vascular images in the gallery using the proposed PA detection system allowed the recognition rate to improve from 95% to 97.5%. The recognition rates of currently reported algorithms are generally under 100%. In addition, the PA system could detect fake vascular images in advance because it detects false signals of fake vascular biometrics. Thus, the recognition rate of the proposed recognition system was improved by removing the signals from fake vasculatures. Owing to the inherent information components of PA signals, the vascular recognition proposed herein has the benefits of low cost, no interaction, excellent detection ability, great robustness, and high recognition rate. The advancements proposed in this paper are expected to vitally influence the field of biometrics in the future.

A schematic of the system and the volumetric layout of the experimental interface are presented in Figs. 1(a) and 1(b), respectively. An optical parametric oscillator (OPO) (OPOletteTM 532, Opotek LLC) was used for PA excitation at a wavelength of 532 nm with a maximal repetition rate of 20 Hz and a duration of ∼7 ns. The output power of the laser was controlled using a computer. In addition, the 532-nm solid-state laser is cost-effective, and the optical absorption performance of blood is adequate at 532 nm. The output beam was spatially attenuated, reshaped, and connected to the single-mode fiber using a fiber optic coupler (customized, Liansheng) with a wavelength range of 190–1200 nm and a numerical aperture (NA) of 0.22. Finally, the output beam was spread onto an optical lens barrel (customized, Yuanming) using the single-mode fiber. The optical lens barrel comprised a collimator lens with a focal length (customized, Yuanming) of 30 mm, and an objective lens (customized, Yuanming) with a numerical aperture of 0.60 was employed to achieve microscope optical illumination at the wavelength of 400–700 nm. The spot diameter for the vasculature was approximately 20 μm, which is suitable for imaging the main vasculature with larger diameters, whereas the peripheral vasculature with a smaller diameter is filtered out during the image preprocessing. Subsequently, the optical lens was mounted on a three-dimensional mechanical displacement platform (3D stage) using a holding device (NCCC50, Zolix) to enable scanning. The PA signal was received by a focused ultrasonic transducer (I5P10N, Doppler) with a central frequency of 4.53 MHz and a –6 dB bandwidth of 152.8%. Furthermore, the signals were amplified by ∼60 dB using an amplifier (ATA-5620, ATA), and the device was digitized by using a personal computer with a multifunctional data acquisition card (PCI8520, ART). The sample platform comprised a displacement platform (MJ60, Zolix) and an ultrasonic transducer.

Excess chloral hydrate was introduced to the rat (Sprague-Dawley, Jiangxi University of Traditional Chinese Medicine) by intraperitoneal injection, and the rat was euthanized by anesthesia. Hair removal cream was applied to the rat, and its body hair was gently depilated to avoid the effect of the hair on the photoacoustic signal. A thin layer of ultrasound gel was applied to the mouse skin with vasculature ex vivo being used for imaging; 160 vascular images from 10 rats were reconstructed.

Forty fake vasculatures were developed using white cotton threads with red ink, which has a much lower absorption coefficient than blood at 532 nm. The fake vasculatures were placed over the real vasculature on the rat skin, and the distance between the fake and the real vasculatures corresponded to the depth of the real vasculature. Moreover, the red ink exhibits a weak optical absorption capacity at 532 nm. The amplitude and the delay of the maximum-value arrival time of the PA signal for the fake and real vasculatures were detected.

The protocol of the experiments in this study was approved by the ethics committee of Jiangxi Science and Technology Normal University. We followed the Chinese guidelines for animal handling and care.

An attack on the subsequent processing part of the system is highly unlikely because it is expensive, requires professional knowledge, and is difficult to perform. A deceptive attack on the preprocessing part only requires camouflaging the detection target, which is simple. The peak-to-peak value and the difference in the PA signal are inherent properties of the detection target, which are difficult to counterfeit. Moreover, the amplitude and the depth of the PA signal of the vasculatures were detected before imaging. As the system detected fake vascular PA signals, only the real vasculatures were imaged.

An equal amount of laser energy was radiated on the fake and real vasculatures at the same depth via collimation and focusing. As the two liquids generated PA signals via the thermoelectric effect, the peak-to-peak values of their PA signals were individually compared as detection parameters to enhance the antispoofing ability of the system.

The fake vasculature was placed over the real vasculature on the rat skin. The distance between both vasculatures was approximately 0.1 mm, which corresponded to the depth of the real vasculature of the rat. The output power of the laser was adjusted by using a computer to produce the same peak-to-peak values for the fake and real vasculatures. At this stage, the difference in the maximum values of the arrival time of the PA signal reflected the depth information, which can be employed as a detection parameter to resist spoofing attacks. The depth of the two detected targets can be expressed as “depth = difference between the maximum arrival duration × speed of ultrasound.” During the experiment, the relative positions of the laser source, sample, and the ultrasonic transducer remained constant.

The images needed to be normalized in size owing to the selection of the various imaging areas during PA imaging. Therefore, the PA images of the real vasculature were normalized to grayscale before establishing the image base, which consisted of only the real vasculature images. Subsequently, the PA images were converted into standardized grayscale images. Thereafter, the vascular images were segmented using the Niblack algorithm,⁴⁶ which evaluated the mean and variance of the pixels in the neighborhood of each pixel. The binarization of the obtained data was performed using Eq. (1),

where T is the threshold, m and s are the mean and standard deviation of the pixel neighborhood window, respectively, and k is the preset correction value. In this study, k was set to –0.2, and the size of the field was 20 × 20.

The vascular images were smoothed to remove noise points. The vascular images were refined and trimmed the burrs, and only the structure of the main vasculature was reserved as the recognition object. Thereafter, the algorithm analyzed the images and obtained seven invariant moments as the recognition features of the target area. These invariant moments exhibited rotational and scale invariance and can be used in the field of vascular recognition. Moreover, the feature vectors of these invariant moments formed global features f(n) = [f(1), f(2), f(3), f(4), f(5), f(6), f(7)] that were inputted to a support vector machine (SVM)⁴⁷ classifier for classification and recognition. In this study, the “libsvm3.22” SVM was used with its default parameters. Note that the SVM is an established machine learning algorithm based on statistical learning theory; notably, it produces superior pattern recognition results even with a small number of training samples.

The fake vasculature was placed over the real one on the rat skin, as shown in Fig. 2(a). The fake vasculature appears redder in color than the real one. In addition, the general recognition method converted the vein image into grayscale, as presented in Fig. 2(b). There was no observable difference between the color and grayscale images of the fake and real vasculatures.

Figure 3 presents the PA signals of the fake and real vasculatures shown in Fig. 2(a) under the same laser energy, where the fake vascular amplitude was much less than the real one. As the real vasculature displays superior optical absorption ability, its PA signal was clearer than that of the fake vasculature. The ability of the optical absorption of the fake vasculature is poor, and its signal-to-noise ratio (SNR) is low; thus, its PA signal is difficult to capture. The peak-to-peak values of the real and fake vascular images were 942.2 and 188 mv, respectively, which differed by almost 80.04% owing to the varied optical absorption properties of red ink and blood. The PA amplitude $p(r,t)$ satisfied the equation $p(r,t)=ΓH(r,t)$⁠. The PA amplitude was proportional to the absorbed optical energy $H(r,t)$⁠, and the optical absorption performance was dominant.³⁶ Moreover, the optical absorption ability is an inherent property of the sample itself, indicating that a greater ability results in more energy absorption, stronger thermoelectric effect, and a higher amplitude of the excited photoacoustic signal that is eventually employed to detect the nonvascular substances.

The real vascular signal after altering the energy output of the laser using the gradient transmittance plate and the unaltered fake vascular signal are plotted in Fig. 4. The peak-to-peak values of the real and fake vascular signals were 182.9 and 188 mV, respectively. The deviation between the peak-to-peak values of the fake and real vascular images was only (188–182.9)/182.9 ≈ 2.79%, which is negligible from a physical perspective. The maximum arrival time of the fake vascular signal was 7.042 μs. A delay of 0.07 μs was seen in the maximum arrival time of the real and fake vascular signals owing to the placement of the fake vascular over the real one, as it was farther from the ultrasonic transducer surface. The speed of ultrasound propagation in biological tissue is known to be 1500 m/s. The fake vasculature on the surface of the skin is situated at a depth of ∼0.105 mm from the real vasculature. The vasculature of the rats was located approximately 0.1 mm below the epidermis, as the body volume varied with each rat. Moreover, the discretization during the calculations performed in MATLAB may have induced errors in the results. The exact TOF of the PA signal was difficult to obtain because the data processed by MATLAB were discrete. Thus, the difference in the maximum arrival time of the photoacoustic signal directly reflects the depth comparison of the samples. As the depth of the forged vasculature was different from that of the real one, the photoacoustic signal was either delayed or advanced, thus facilitating resistance toward spoofing attacks.

The vascular amplitude and the time delay represent the optical absorption characteristic and the depth, respectively. Thus, the peak-to-peak value and the maximum-value arrival time delay information were set as the antispoofing parameters for data preprocessing. Therefore, the PA signal could be employed to accurately detect the fake vasculatures and eliminate them from the imaging process. The results of the antispoofing detection based on PA signals are listed in Table I, where only four vasculatures were detected incorrectly: two fake vasculatures were detected as real and two real vasculatures were detected as fake. The photoacoustic signals of 160 vasculatures passed the PA detection. The signals were regarded as real vascular images by the recognition system. Subsequently, these real vasculatures were imaged.

The choice of threshold mainly relies on experience in this paper, because the relative experimental conditions are unchanged, such as the distance between the laser beam and the sample, and the energy output power. In Sec. III A 1, the amplitude of the real vessel is about five times that of the fake one. But the blood oxygen content varies in each sample. Therefore, the amplitude threshold is set to 500 mv. In Sec. III A 2, the vasculars of the rats were located approximately 0.1 mm below the epidermis as the volume of the rats was different. From a physical point of view, the error of 5% is negligible. Combined with the actual situation of the experiment (the volume of the rats is different), the threshold is set to 1 mm (1 ± 15%). In fewer experiments, the laser exceeds the standard working time and the output power is unstable. This may be the cause of the wrong detection result in the PA detection. This method has limitations. Neural network is a more scientific method. But, it is not applicable here because of the small number of samples. The larger the number of samples, the massive the experiments will be. Threshold selection based on artificial intelligence is a meaningful area of research and will be studied in depth.

Figure 5 depicts a portion of the image base of real vasculatures detected using the PA system; all images were normalized by size and grayscaled.

Figures 6(a)–6(f) present the real vascular image, normalized PA image (by size and grayscale), binarized vascular image, smoothed vascular image, thinned vascular image, and burr-trimmed vascular image, respectively. The features of each vascular image were extracted as a data set.

The distribution of the data between the training and the test sets influences the recognition result. In this study, the training set size was set to 10, 20, 30, 40, 50, 60, 70, 80, 100, 120, and 140. The corresponding recognition rates, shown in Fig. 7, are monotonically increasing and nonlinear in nature. In addition, the recognition rate drops faster for a narrow training set, and it rises more slowly as the training set becomes broader. In particular, the recognition rate reached 100% when the training set included more than 120 images (three-quarters of the total sample). Conversely, the recognition rate was less than 75% for a training set with less than 40 images (one-quarter of the total sample). Furthermore, the recognition rate dropped below 50% when the training set included less than 20 images (one-eighth of the total sample). Thus, the intermediate value of 80 images was appropriately selected for the training set in this research.

The dataset comprised 160 vascular images of 10 rats, which were equally distributed (80 images each) between the training and the test set. The results of vascular recognition are listed in Table II. As the recognition rate reached 97.5% with the method using the SVM, the actual recognition rate of the method was 97.5%. Thus, the recognition rate of the real and fake vasculatures was improved by 2.63% [(97.5–95)/95], when compared with the results directly obtained from the SVM. The recognition rate of vasculatures is greatly improved because it is difficult to improve when the recognition rate of the system is inherently high. The proposed recognition of the system outperforms other methods because traditional machine learning methods are suitable only for a small number of samples in this experiment. The PA data preprocessing is not limited to the use of SVM, and it can be combined with other algorithms. Therefore, the recognition rate can be improved.

The remaining part of this section discusses the current limitations and possible improvements to the system.

The imaging speed is mainly limited by the scanning speed of the mechanical displacement platform, the pulse frequency of the laser, and the acquisition rate of the data acquisition card. In this study, all blood vessel images in the photoacoustic library took several hours to obtain because of the mechanical scanning platform, which significantly impacts the practical application potential of the proposed system. In practice, users certainly cannot accept such a long-duration authentication method. The latest photoacoustic systems can shorten the imaging time to several minutes,⁴⁸ which sets the foundation for the use of photoacoustic imaging to resist spoofing attacks.

Based on prior research on photoacoustic imaging, an objective lens with a large numerical aperture is generally used to focus the laser beam for obtaining high-resolution images, and the short working distance of such a lens, at the millimeter level, is unavoidable. Thus, we tried to build a photoacoustic system with a dark field using a reflective objective. A miniature ultrasonic transducer was placed at the dark field in the middle of the reflective objective. The ultrasonic transducer requires medium coupling and cannot be used directly in air. Thus, we used a light-permeable medium as the coupling material in the optical path but it inevitably increased the scattering of light, thus resulting in a decrease in resolution to the millimeter level. This cannot be employed relative to the diameter of the isolated mouse blood vessels used in this study. There are ultrasonic transducers that can be directly coupled to air but they are highly expensive. Thus, we changed the light-transmitting medium to a scattering medium and coupled the construction of wavefronts. The photoacoustic signal was used as a feedback signal,⁴⁹ and the phase of the incident laser beam was changed by a spatial light modulator or digital micromirror device to improve the focus. However, an ideal photoacoustic system could be built based on the dark-field backward mode.

Photoacoustic imaging mainly relies on the optical selective-absorption characteristics of the sample to develop a high-contrast image. In this study, the effective target was oxygenated or hypoxic hemoglobin, and the background target was skin (lipid). The samples were irradiated with a 532-nm Nd:YAG laser beam. The light absorption efficiencies of lipids and oxygenated or hypoxic hemoglobin at 532 nm are stated in Ref. 39, and they differ by almost 5 orders of magnitude. Theoretically, the effects of skin can be considered as negligible. Moreover, there existed a noise caused by the system circuit, which was filtered out using a filter. Subsequently, the photoacoustic signal was amplified using a signal amplifier to increase the signal-to-noise ratio. Furthermore, we used an image preprocessing method to remove the noise points (artifacts and useless signal points generated by small blood vessels) in the image to improve vein recognition.

In this study, we demonstrated a PA detection technique with low cost, no interactive interface, great robustness, excellent anticounterfeit detection capabilities, and a high recognition rate that can be applied for vascular recognition. In addition, the PA data preprocessing could eliminate fake vascular biometrics using the amplitude and depth of the PA signal. Moreover, the anticounterfeiting ability of the PA system was enhanced, and the recognition rate was effectively improved. However, the 3D stage entailed movement errors during PA vascular image acquisition in the experiment, and the acquisition speed for the PA vascular images was slow. Moreover, large numerical aperture objectives have shorter working distances of only a few millimeters. Besides other characteristics of PA, tackling this problem may be a future direction for this work as listed: (1) galvanometer or linear array PA head, (2) more convenient backward PA mode, and (3) content of blood oxygen and 3D structure of vasculatures. This study will contribute to the research of vascular recognition in the field of antispoofing.

This study was supported by the National Natural Science Foundation of China (NSFC) (Nos. 51763011, 61650402, and 61861020), the 2019 Innovation of Outstanding Young Personnel Training Program (No. 20192BCBL23015), and the project of Key Laboratory of Optic-electronic Detection and Information Processing of Nanchang City (No. 2019-NCZDSY-008).