Investigation on stereo-ToF data fusion for the inspection of used industrial parts

Optical systems for automated or partially automated inspection have been making important contributions to ensure the quality and functionality of technical products for many years. Often used to monitor the quality of newly produced goods, vision systems also aim to play an important role in identification and condition description of used industrial parts such as aged vehicle components. In this work, a passive stereo system and a Time-of-Flight (ToF) sensor of the latest generation were used to create the desired sensor system. In the first step, the pixel-based information of both sensors was exploited to spatially calibrate the transformation between the left stereo camera and the ToF sensor by forming a 2D-3D correspondence set of detected feature points. To compensate for the resolution difference of the sensors, numerous interpolation points were randomly sampled on the reconstructed sparse surface mesh of the ToF sensor to create the missing sub-pixel information. It could be shown that the fused sensor information led to an increase in incompleteness by 7.81% on average for all components examined. The higher noise in the ToF measurement data in the fill-ins could be mitigated by using an adapted median kernel filtering. The average deviation of the measurement from a reference dataset was 1.30mm for the stereo system, 2.51mm for the ToF system, and 1.42mm for the fused result. The result of this work is promising as the quality of the surface mesh could be raised especially for critical surface areas and the underlying RGB data itself can be used for pixel-wise classification and segmentation.