Evaluation of Flaw Detection Algorithm Using Simulated X-Ray Computed Tomography of Ground Truth Data

A framework to generate simulated X-ray computed tomography (XCT) data of ground truth (denoted here as “GT”) flaws was developed for the evaluation of flaw detection algorithms using image comparison metrics. The flaws mimic some of those found in additively manufactured parts. The simulated flaw structure gave a GT data set with which to quantitatively evaluate, by calculating exact errors, the results of flaw detection algorithms applied to simulated XCT images. The simulated data avoided time-consuming manual voxel labeling steps needed for many physical data sets to generate GT images. The voxelated pore meshes that exactly match GT images were used in this study as opposed to using continuum pore meshes. The voxelated pore mesh approach avoids approximation error that occurs when converting continuum pore meshes to voxelated GT images. Spherical pores of varying sizes were randomly distributed near the surface and interior of a cylindrical part. XCT simulation was carried out on the structure at three different signal-to-noise levels by changing the number of frames integrated for each projection. Two different local thresholding algorithms (a commercial code and the Bernsen method) and a global thresholding algorithm (Otsu) were used to segment images using varying sets of algorithm parameters. The segmentation results were evaluated with various image evaluation metrics, which showed different behaviors for the three algorithms regarding “closeness” to the GT data. An approach to optimize the thresholding parameters was demonstrated for the commercial flaw detection algorithm based on semantic evaluation metrics. A framework to evaluate pore sizing error and binary probability of detection was further demonstrated to compare the optimization results.