Authors:
Gabriel Lins Tenorio
1
;
2
;
Weria Khaksar
2
and
Wouter Caarls
1
Affiliations:
1
Electrical Engineering Department, Pontifical Catholic University of Rio de Janeiro - PUC-Rio, Rio de Janeiro, Brazil
;
2
Faculty of Science and Technology, Norwegian University of Life Sciences (NMBU), Ås, Norway
Keyword(s):
3D Instance Segmentation, Strawberry Detection, Precision Agriculture.
Abstract:
This paper presents an investigation into the use of 3D Deep Learning models for enhanced strawberry detection in polytunnels. We focus on two main tasks: firstly, fruit detection, comparing the standard MaskRCNN
and an adapted version that integrates depth information (MaskRCNN-D), both capable of classifying strawberries based on their maturity (ripe, unripe) and health status (affected by disease or fungus); secondly, for the
identification of the widest region of strawberries, we compare a contour-based algorithm with an enhanced
version of the VGG-16 model. Our findings demonstrate that integrating depth data into the MaskRCNN-D
results in up to a 13.7% improvement in mean Average Precision (mAP) from 0.81 to 0.92 across various
strawberry test sets, including simulated ones, emphasizing the model’s effectiveness in both real-world and
simulated agricultural scenarios. Furthermore, our end-to-end pipeline approach, which combines the fruit
detection (MaskRCNN-D) and wides
t region identification models (enhanced VGG-16), shows a remarkably
low localization error, achieving down to 11.3 pixels of Root Mean Square Error (RMSE) in a 224 × 224
strawberry cropped image. This pipeline integration, combining the strengths of both models, provides the
most effective result, enabling their application in autonomous fruit monitoring systems.
(More)