Influence of data and methods on high-resolution imagery-based tree species recognition considering phenology: The case of temperate forests

Seasonal phenological transformations alter tree appearances, notably by influencing the size and color of the foliage. It has long been anticipated that such phenology induced characteristics can help address the tree-species recognition problem, a fundamental challenge in forest science. Yet, studies on tree-species recognition using remote sensing and phenological characteristics have been rare, due to the very limited availability of high spatiotemporal resolution observations. Moreover, the interactions between the effectiveness of phenological characteristics, remote sensing data, and the analytical methodologies have not yet been sufficiently explored. The understanding of how to integrate multi-temporal observations and phenological characteristics in tree-species recognition has been lacking. This study aims to identify principles for optimizing species recognition by combining data, methods, and phenological dynamics. This involves understanding the impact factors of various methodologies, and how they interact with phenological characteristics and datasets at different times and/or frequencies. The study was carried out using multi-temporal high-resolution optical images of a temperate forest, which were collected in 2021 during leaf growth and senescence periods between May and October, i.e., three leaf growth (May–August) and three leaf senescence (September–October) periods. The test site comprised 14 different tree classes, including 11 species, 2 genera, and 1 dead tree class. The experimental results showed that, for deep learning approaches, the current main limitations in the tree species recognition lie in sample imbalance as the targeted species number increases. With the state-of-the-art data and methods, distinguishing between species within a same genus is much more challenging than differentiating between species from different genera or families. It is also revealed that the best timing for tree species classification is early autumn (September) or late spring (May) when a single-temporal (one-timepoint) data is applied; all-temporal (six-timepoint) data improves the recognition results in comparison with single-temporal observations; however, the improvements from adding additional timepoints became marginal after two timepoint are used with one from late spring and other from early autumn. Furthermore, prior knowledge of individual crown boundaries, typically obtained through individual tree crown delineation, is essential for efficiently incorporating phenological variations into species recognition.