S06 - Session O1 - Smartphone-based strawberry plant growth monitoring by using R-CNN

Authors: Seitaro Toda *, Tetsutaka Sakamoto, Yuya Imai, Ryoga Maruko, Takeru Kanoh, Naomichi Fujiuchi, Kotaro Takayama

In recent years, strong demand for numerical indice based qualification and quantification of the plants' growth condition has become an urgent topic for the efficient agricultural production. For example, the weekly evaluation of plants' growth by measuring stem elongation, stem thickness, leaf expansion with a tape measure is popular in tomato production. On the other hand, the weekly evaluation of strawberry plant growth requires counting the number of flowers and fruits by human eyes. Counting the flowers and fruits in a commercial greenhouse takes much time and many farmers do not want to do such cumbersome activity even if the counting is quite effective to monitor the growth balance and yield forecast. In this study, to provide a low-cost and easy strawberry plant growth monitoring tool to farmers, we developed a deep learning model using the R-CNN algorithms for quick counting of the number of flowers and fruits of a strawberry canopy in a commercial greenhouse. A smartphone, an iPhone, is used as the imaging device and takes color images of the strawberry canopy. YOLOv3, a representative object detection algorithm, was used to develop the flower and fruit detection model. Taking into account the variations of shape and color of fruits, the fruits were classified into 3 classes "small green (immatured)", "green (immatured)", and "red (matured)". And the 400 images were annotated with 4 labels of "flower", "small green fruit", "green fruit", and "red fruit", and the annotated images were used for training (including validation) of the model. The developed model was tested with 20 images, which were not used for the training. The developed model detected the 4 labels with enough accuracy at the F-measures from 0.66 to 0.83. By using the developed model, we conducted strawberry plant growth monitoring for 3 months. The coefficient of the correlation between the detected number of "red fruit" and the number of harvested fruits was 0.92. These results indicate that the developed deep learning model has enough accuracy to evaluate the strawberry plant growth in commercial greenhouses.