Author:

Keywords:

C3/19/036#55510260

Abstract:

Pear fruit are prone to developing internal disorders that leave consumers dissatisfied and unwilling to repeat their purchase. It must thus be prevented that defect fruit reach the consumer. Internal disorders, however, are invisible from the surface of the fruit. In practice, batches of fruit are, therefore, often discarded based on the result of a manual destructive inspection of a small number of randomly sampled fruit. This leads to unacceptable financial losses and waste due to the disposal of the healthy fruit still present in the batch. Moreover, the sampled fruit may not be representative of the whole batch. Prior research has shown that X-ray imaging is a promising instrumental technique for detecting internal disorders. The aim of this PhD was, therefore, to provide novel, performant, and nondestructive methods to analyze X-ray images automatically in an objective way. This was done by implementing deep learning, which is a new paradigm in machine learning, that allows algorithms to learn directly from data. First, a method was developed to detect pears with internal disorders with X-ray Computed Tomography (X-ray CT) data using a conventional machine learning strategy. Herein, an image processing algorithm was developed to extract features from the 3D data. Thereafter, a classifier was trained to distinguish healthy and defect pears based on the extracted features. The classifier achieved classification accuracies ranging between 90.2 and 95.1 % depending on the cultivar and number of features that were used. However, the proposed method had several disadvantages. It required a handcrafted feature extraction algorithm. Potentially better features, which were not thought of during development, remained unexplored. In addition, the feature extraction algorithm is possibly application specific. Furthermore, while the method allowed for classifying defect and sound fruit, it could not quantify the severity of the disorders which may be of high importance for consumers and thus for making decisions on discarding fruit or not. To circumvent these disadvantages, a deep neural network was used to segment different internal structures in CT images, including healthy tissue, the core, cavities and tissue affected by internal browning. The model was trained on manually annotated CT scans of healthy and defect fruit. On an independent test set, a very good agreement was found between the predicted and ground truth "healthy tissue", "core" and "cavity" labels (average intersection over union (IoU) > 0.95). Interestingly, low IoU scores were found for the "internal browning" label, even though visually most predictions seemed sufficiently accurate. It turned out this was mainly caused by negligible errors on small volumes and volume edges. From the predicted labels of the model, the severity of the internal disorders could be quantified by calculating the affected volumes. The resulting quantitative data was used to classify "consumable" vs "non-consumable" fruit at high accuracy (99.4 %) on the one hand and "healthy" vs "defect but consumable" vs "non-consumable" classification on the other hand (92.2 %). Herein, the identification of "defect but consumable" fruit showed to be the most difficult. A concern with X-ray CT, however, is that it is currently not applicable inline at the speed of commercial sorting lines (10 fruit/s). X-ray radiography, on the other hand, can easily be implemented inline using an X-ray source and detector on either side of a conveyor belt. A downside of X-ray radiography, however, is that it only produces a 2D projection of the absorption by a 3D object. Compared to X-ray CT, it is thus more challenging to distinguish contrast in the image caused by internal disorders and contrast caused by the shape and internal structure of the fruit. An anomaly detection approach using deep learning was proposed to detect internal disorders in X-ray radiographs of pears, recognizing recent advantages in deep learning, while overcoming the need for annotated data normally required for supervising the model during training. The anomaly detection model was trained exclusively on X-ray radiographs of healthy pears, after which they were evaluated on a test set with images of healthy and defect pears. Defect pears could be identified based on the anomaly score produced by the model. It was shown that performance could be significantly improved by using synthetic anomalies. Herein fake defects were added to X-ray images of healthy pears. ROC analysis showed that the proposed method was on par (mean area under the curve (AUC) up to 0.962) with a state-of-the-art benchmark method which was given several advantages (mean AUC = 0.963). The best anomaly detection model achieved an overall accuracy of 95 %, with true positive and false positive rates equal to 91.8 and 0.8 %, respectively. By investigating the performance as a function of internal disorder severity, it was shown that using the proposed method, defect fruit with a cavity percentage > 1.0 % could be detected 100 % accurate, while for lower cavity percentages the accuracy depended on the internal browning severity. The black-box nature of neural networks was addressed by producing heatmaps of the anomalous regions found by the models. In this research, a large step forward was made towards internal disorder detection in pears using X-ray imaging. It is expected that deep learning and X-ray imaging will increasingly be adopted by various industries for quality inspection. Hereto, the presented methods could easily be used for the inspection of other fruits or vegetables, or be implemented in other applications, such as foreign object detection in foods. Future work should focus on further investigating deep learning-based approaches, such as unsupervised learning, and to further discover the possibilities, but also limitations, of X-ray based inspection of foods. To bring quality inspection to an even higher level, further research is required in fast inline X-ray CT systems, and other X-ray based methods, such as X-ray phase contrast imaging.