What can herbaria teach us about tropical plant growth dynamics? Evaluation of the potential contributions of automated visual analysis?

Recent advances in information technology have enabled the mass production of botanical visual datasets. This new material, directly produced in the field or resulting from the digitization of natural history collections, offers scientists new opportunities to study plant diversity at an unprecedented taxonomic, temporal and geographic scale. The study presented here is part of the CEBA-PhenObs project. It is dedicated to the experimentation and the development of new deep learning approaches to study tropical plant growth dynamics, based on digitized specimens of the Cayenne herbarium (CAY).
The exploitation of huge volumes of digitized herbarium specimens is mainly limited by the difficulty of searching, filtering and extracting the most relevant material for specific studies. Indeed, the metadata associated with these images are generally limited to the location, date and taxonomic identity of the sample. On the other hand, information on the phenological or morphological characters expressed by the specimens is rarely provided. Recovering such missing information necessitates resorting to the visual analysis of the scans. In this study, we propose to develop new deep learning methods for the automated extraction of phenological characters to foster the study of plant growth at an unprecedented taxonomical scale. This study focuses on plant species from the Amazonian region whose growth dynamics show a greater diversity of patterns than those of temperate species.
The characteristics of the scientific challenge addressed, the data produced to approach it, and the preliminary results obtained will be presented during this seminar. This work, far from having resolved all the difficulties raised, (i) allows a better understanding of the potential of herbaria to study the vegetative growth of tropical plants, (ii) highlights the specificities of the underlying visual recognition tasks, and (iii) provides an overview of the performance of the most advanced visual recognition techniques. In particular, this work shows that the approaches usually implemented on this type of data for the study of the phenology of the reproductive system are not directly applicable for the analysis of the vegetative system. It proposes a framework to make progress in this challenge, and thus pursue the valorisation of herbarium collections on this research topic.

Recent bibliographical reference:
- Pearson et al., 2020. Machine Learning Using Digitized Herbarium Specimens to Advance Phenological Research. BioScience, Volume 70, Issue 7, July 2020, Pages 610–620. https://doi.org/10.1093/biosci/biaa044
- Davis C. et al., 2020. A new method for counting reproductive structures in digitized herbarium specimens using Mask R-CNN. Frontiers in Plant Science 11 (2020): 1129. https://doi.org/10.3389/fpls.2020.01129
- Goëau et al., 2020. A new fine‐grained method for automated visual analysis of herbarium specimens: A case study for phenological data extraction. Applications in plant sciences 8.6 (2020): e11368. https://doi.org/10.1002/aps3.11368
- Lorieul T. et al., 2019. Toward a large‐scale and deep phenological stage annotation of herbarium specimens: Case studies from temperate, tropical, and equatorial floras. Applications in Plant Sciences, 7(3), e01233. https://doi.org/10.1002/aps3.1233