Comprehensive Evaluation of Paprika Instance Segmentation Models Based on Segmentation Quality and Confidence Score Reliability
2025
Nozomu Ohta | Kota Shimomoto | Hiroki Naito | Masakazu Kashino | Sota Yoshida | Tokihiro Fukatsu
Fruit instance segmentation models are widely researched for inference tasks such as yield prediction and automated harvesting. Previous studies evaluated these models only on the basis of mask average precision: they overlooked segmentation quality and confidence score reliability&mdash:both crucial for inference tasks. This study proposes an evaluation method that incorporates, in addition to mask average precision, the Aggregated Jaccard Index to assess segmentation quality and the coefficient of determination between confidence scores and Intersection over Union for reliability evaluation. We compared YOLO11, Mask R-CNN, and their improved variants using a dataset that included stable and comprehensive imaging obtained by monitoring equipment in a large-scale commercial paprika greenhouse. Results show that mask scoring R-CNN excels in segmentation quality, while YOLO11 performs better in mask average precision and confidence score reliability. These findings suggest that when evaluating instance segmentation models for real-world application scenarios, we should not rely solely on mask average precision, but a combination of multiple metrics that assess different aspects of the model must be utilized.
显示更多 [+] 显示较少 [-]