The British Journal of Radiology vol:85 issue:1020 pages:e1233-E1241
Objective: To compare two methods for assessment of image processing algorithms in digital mammography: free-response receiver operating characteristic (FROC) for the specific task of microcalcification detection and visual grading analysis (VGA).Methods: The FROC study was conducted prior to the VGA study reported here. 200 raw data files of low breast density (BI-RADS I-II) mammograms (Siemens Novation DR, Germany)-100 of which abnormal-were processed by four image-processing algorithms: Raffaello (IMS, Bologna, Italy), Sigmoid (Sectra, Linköping, Sweden), and OpView v. 2 and v. 1 (Siemens, Erlangen, Germany). Four radiologists assessed the mammograms for the detection of microcalcifications. 8 months after the FROC study, a subset (200) of the 800 images was reinterpreted by the same radiologists, using the VGA methodology in a side-by-side approach. The VGA grading was based on noise, saturation, contrast, sharpness and confidence with the image in terms normal structures. Ordinal logistic regression was applied; OpView v. 1 was the reference processing.Results: In the FROC study all algorithms performed better than OpView v. 1. From the current VGA study and for confidence with the image, Sigmoid and Raffaello were significantly worse (p<0.001) than OpView v. 1; OpView v. 2 was significantly better (p = 0.01). For the image quality criteria, results were mixed; Raffaello and Sigmoid for example were better than OpView v. 1 for sharpness and contrast (although not always significantly).Conclusion: VGA and FROC discordant results should be attributed to the different clinical task addressed.Advances to knowledge: The method to use for image processing assessment depends on the clinical task tested.