We have proposed a novel image-based feedback control method to enhance the path-following capability for position and orientation of a magnetic catheter at its distal part. The proposed control method uses only a navigation status of the magnetic catheter for manipulation without utilizing complex analytical models. By comparing the positions of two markers of the magnetic catheter relative to a predefined path, we classify the navigation status of the magnetic catheter into four cases and determine the external magnetic field and feeding velocity according to a proposed control algorithm. Finally, we conduct a path-following experiment in an environment with multiple branches to verify the improved position and minimization of the orientation error regarding the path-following capability of the proposed control method.
I. INTRODUCTION
A catheter is a long flexible tube inserted into the body to deliver drugs or treatment devices to a target lesion. Conventional catheters are manipulated by the hands of a medical doctor from outside the patient’s body, exhibiting low steering capability and maneuverability. To overcome these limitations, magnetic catheters (MCs), which have permanent magnets at the distal part, are widely investigated nowadays.1 Magnetic torque is generated at the embedded magnets when an external magnetic field (EMF) is applied, and the MC can be actively manipulated using the magnetic torque for better steering capability and maneuverability.
Various analytical models have been utilized to estimate the motions of the MC such as Euler-Bernoulli beam theory2,3 or Cosserat rod theory.4 However, control methods based on complex analytical models have several limitations to their application in real in-vivo operations. First, they cannot consider the full set of dynamic real-time boundary conditions such as friction and reaction forces. Moreover, since complex analytical models require computation time for numerical calculations, the control method makes MC manipulation challenging in real-time due to time-delay. Additionally, most of the conventional control methods control the position of only one distal point of the MC. However, in the case of the MC equipped with treatment modules at its distal part, the orientation of the MC should be controlled as well to prevent damage which may be caused by contact between the treatment modules and the inner wall of the environment.
We propose a novel image-based feedback control (IBFC) method of the MC utilizing the positions of two markers located at both ends of the distal part without analytical models. In the IBFC method, the normal pixel distances between each marker and a predefined path are calculated, and the status of the MC is classified into one of four cases depending on the normal pixel distances. For each case, we set different values for the feeding velocity of the MC and angular velocity of the EMF to minimize both the position and orientation error. The proposed IBFC method was verified by controlling a MC in an environment with multiple branches. Further, to validate the path-following performance of the proposed IBFC method, trajectories and average position error of each marker were measured and compared with the results obtained by controlling the MC using the conventional one-point control method.
II. MANIPULATION OF THE MC USING THE IBFC METHOD
Figure 1 shows the structure of the MC and an overall procedure of the proposed IBFC method. The MC has three permanent magnets to generate magnetic torque under an EMF and two markers at both ends of the distal part for tracking. The magnetic torque generated at the magnets under the EMF can be expressed as follows:5
where m and B are the magnetic moment of the magnets and the magnetic flux density of the EMF, respectively. Since the magnetic torque aligns the magnets with the direction of the EMF, a magnetic navigation system (MNS) that generates and control the EMF can manipulate the direction of the MC. A feeding device pushes and pulls the MC axially. A path composed of several line segments is predefined by the operator, and navigation status is determined by comparing the normal distances between each marker and the nearest line segment. Based on the current status of the MC, the IBFC method controls angular velocity of the EMF (ωEMF) and feeding velocity of the MC (vF) by utilizing the MNS and the feeding device, respectively, to maximize the path-following capability.
Structure of the MC and an overall procedure of the proposed IBFC method.
First, a camera captures real-time images of the workspace, and the RGB color tracking module detects the markers from the images. The positions of the markers are transferred to the status classification module, which calculates the normal distance between the markers and nth line segment to identify the navigation status of the MC. The normal distance (dq) between marker q and nth line segment shown in Figure 2(a) can be calculated as follows:
where (xn, yn), (xn+1, yn+1), and (xq, yq) are the pixel positions of nth and (n+1)th points composing the nth line segment and the pixel position of marker q, respectively. From the image acquisition of the workspace to the calculation of the normal distance, all procedures progress based on the pixel coordinates without any calibration process for mapping to real coordinates. This minimizes the errors of the control system, which may be caused by the distortion of the lens and imprecise calibration.
(a) Normal distances between two markers and the predefined path composed of several line segments. (b) Four cases of navigation status of the MC. (c) Flow chart to classify the status of the MC into one of the four cases and determine suitable values for vF and ωEMF.
(a) Normal distances between two markers and the predefined path composed of several line segments. (b) Four cases of navigation status of the MC. (c) Flow chart to classify the status of the MC into one of the four cases and determine suitable values for vF and ωEMF.
Then, the navigation status of the MC is classified into one of four cases, as shown in Figure 2(b), by the status classification module. When both markers are close enough to the predefined path that both d1 and d2 are smaller than dallow (which is set by the operator), this status is categorized to case A. In case A, vF and ωEMF are maximized and minimized, respectively, for fast and effective navigation. If any marker has normal distance greater than dallow, the status is categorized depending on the signs of d1 and d2. When d1 and d2 have same sign, the status is categorized as case B, and vF and ωEMF are minimized and maximized, respectively, to approach the path quickly. If d1 and d2 have different sign, the status is categorized as case C or D depending on which is smaller. For cases C and D, vF and ωEMF are selected differently to minimize d1 and d2 simultaneously. The determined vF and ωEMF are transmitted to the feeding device and the MNS to manipulate the MC. This categorization process is represented in Figure 2(c) as a flow chart. Assuming the current iteration is the kth, the direction of the EMF for the next iteration (θk+1) is calculated as follows:
where θk and tk are the direction of the EMF and a measured processing time for the kth iteration of the IBFC method, respectively.
III. RESULTS AND DISCUSSION
To manipulate the MC, the feeding device and closed-loop magnetic navigation system (CMNS) were utilized, as shown in Figures 3(a) and (b). The feeding device consists of three step motors (17HD2041-04N-A, MOONS motor, China) and generates translational and rotational motion of the MC. The CMNS composed of eight electromagnets and backyoke structure can generate EMF in all three-dimensional directions in the workspace.6 A commercial MC (Thermocool®, Biosense Webster, USA) with two visual markers shown in Figure 3(c) was used to verify the IBFC.
(a) Feeding device, (b) closed-loop magnetic navigation system (CMNS) and (c) commercial MC used for the experiment.
(a) Feeding device, (b) closed-loop magnetic navigation system (CMNS) and (c) commercial MC used for the experiment.
The proposed IBFC method was verified by controlling the MC in an environment with multiple branches. We first predefined the path to a target branch, then manipulated the MC using the IBFC method. To verify the effectiveness of the IBFC method based on the two markers, we conducted an experiment using a control method based on a single marker at the distal tip of the MC. The control method based on a single marker is similar to the IBFC method but manipulates the MC just to minimize the position error of front marker without status classification. The rear marker is not utilized to control the MC in the control method based on a single marker, but it is used to evaluate the path following capability of the whole distal part of MC. vF and ωEMF were set to the average values of the minimum and maximum values for the IBFC method. During manipulation, the positions of the markers were tracked in real-time by an RGB camera (acA2000-165uc, BASLER, Germany) and by utilizing color tracking algorithms. We set dallow, the maximum and minimum vF, and the maximum and minimum ωEMF to be 12 pixels (≈2 mm), 1.2 mm/s, 0.3 mm/s, rad/s, and rad/s, respectively. The magnitude of the EMF at the center of the workspace was fixed to 40 mT.
According to results represented in Figures 4(a) and (b), the MC manipulated by the IBFC method tends to maintain its distal part near the center of the predefined path without biasing towards the inner wall closer than the MC manipulated by the control method based on a single marker. To evaluate how good the whole distal part of the MC follows the path for each control method, we consider the normal distances of the front marker, the rear marker and the middle point of the front and rear markers comprehensively. Figure 4(c) shows the average normal distance errors of two markers and a middle point between the front and rear markers. The results show that the proposed IBFC method has better position accuracy at the middle point and the rear marker than the control method based on a single marker. Also, the total position error and the orientation error of each iteration during the navigation represented in Figures 4(a) and (b) were calculated as follows:
where d1,n, d2,n, eposition,n and eorientation,n are the normal distances of the front and rear markers, and the total position and orientation errors of the nth iteration, respectively. The average position error and the average orientation error of the IBFC method were calculated from the total position error and orientation error from 1st to nth iterations and reduced by 18.64% and 7.5% compared to them of the control method based on a single marker, respectively. The average processing time for one iteration of the IBFC method (tk) was 0.1 second, which is sufficient for real-time control of the MC. Therefore, the proposed IBFC method has not only good path-following capability without the use of complex analytical models, but also short processing time, which is necessary for real-time control of medical operations.
Trajectories in which the distal part of the MC sweep during navigation with (a) the proposed IBFC and (b) the control method based on a single marker. (c) Average normal distance errors of two markers and a middle point between the front and rear markers.
Trajectories in which the distal part of the MC sweep during navigation with (a) the proposed IBFC and (b) the control method based on a single marker. (c) Average normal distance errors of two markers and a middle point between the front and rear markers.
IV. CONCLUSION
We developed IBFC method for the MC to enhance the path-following capability of the position and orientation at its distal part. The proposed IBFC method calculates the normal distances of each marker, classifies the status of MC into one of four cases, and determines vF and ωEMF to minimize d1 and d2 simultaneously for each case. We verified the proposed IBFC method by controlling the MC. According to the results, the IBFC method showed good accuracy regarding the position and orientation related to the path-following capability and fast processing for real-time control of the MC. This research contributes to improving the accuracy and safety of MC manipulation and to expanding the biomedical applications of MC and magnetic manipulation methods.
ACKNOWLEDGMENTS
This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI19C0642)