Improving obstacle awareness for robotic harvesting of sweet-pepper

Abstract Obstacles are densely spaced in a sweet-pepper crop and they limit the free workspace for a robot that can detach the fruit from the plant. Previous harvesting robots mostly attempted to detach a fruit without using any information of obstacles, thereby reducing the harvest success and damaging the fruit and plant. The hypothesis evaluated in this research is that a robot capable of distinguishing between hard and soft obstacles, and capable of employing this knowledge, improves harvest success and decreases plant damages during harvesting. In line with this hypothesis, the main objective was to develop a sweet-pepper harvesting robot capable of distinguishing between hard and soft obstacles, and of employing this knowledge. As a start, the thesis describes the crop environment of a harvesting robot, reviews all harvesting robots developed for high-value crops, and defines challenges for future development. Based on insights from this review, we explored the ability to distinguish five plant parts. A multi-spectral imaging set-up and artificial lighting were developed and pixels were classified using a decision tree classifier and a feature selection algorithm. Classification performance was found insufficient and therefore post-processing methods were employed to enhance performance and detect plant parts on a blob basis. Still, performance was found insufficient and a focussed study was conducted on stem localization. The imaging set-up and algorithm developed for stem localization were used to provide real stem locations for motion planning simulations. To address the motion planning problem, we developed a new method of selecting the grasp pose of the end-effector. The new method and the stem localization algorithm were both integrated in the harvesting robot, and we tested their contribution to performance. This research is the first to report a performance evaluation of a sweet-pepper harvesting robot tested under greenhouse conditions. The robot was able to harvest sweet-peppers in a commercial greenhouse, but at limited success rates: harvest success was 6% when the Fin Ray end-effector was mounted, and 2% when the Lip-type end-effector was mounted. After simplifying the crop, by removal of fruit clusters and occluding leaves, harvest success was 26% (Fin Ray) and 33% (Lip-Type). Hence, these properties of the crop partly caused the low performance. The cycle time per fruit was commonly 94 s, i.e. a factor of 16 too long compared with an economically feasible time of 6 s. Several recommendations were made to bridge the gap in performance. Additionally, the robot’s novel functionality of stem-dependant determination of the grasp pose was evaluated to respond to the hypothesis. Testing the effect of enabling stem-dependent determination of the grasp pose revealed that, in a simplified crop, grasp success increased from 41% to 61% for the Lip-type end-effector, and stem damage decreased from 19% to 13% for the Fin Ray end-effector. Although these effects seem large, they were not statistically significant and therefore resulted in rejection of the hypothesis. To re-evaluate significance of the effects, more samples should be tested in future work. In conclusion, this PhD research improves the obstacle awareness for robotic harvesting of sweet-pepper by the robot’s capability of perceiving and employing hard obstacles (plant stems), whereas previous harvesting robots either lumped all obstacles in one obstacle class, or did not perceive obstacles. This capability may serve as useful generic functionality for future robots.