Deep Learning

SenNet: A dual-branch image semantic segmentation network for wheat senescence evaluation and high-yielding variety screening

Abstract

Wheat is one of the three primary staple crops globally, with the senescence of its leaves having a direct effect on yield. However, conventional senescence evaluation methods are mainly based on visual scoring, which are subjective, time-consuming, and hamper the investigation of mechanisms between senescence process and yield formation. High-throughput image-based plant phenotyping techniques offer a promising approach. However, extracting senescence-related semantic information from images presents challenges, including blurred edge segmentation, inadequate characterization of senescence features, and interference from complex field environments. Therefore, this study proposes a dual-branch image senescence segmentation model (SenNet), which integrates edge priors and local–global attention mechanisms, including local–global hierarchical attention mechanisms, gated convolution, and positional encoding modules. First, a wheat senescence dynamics image dataset (19530 images) was constructed, comprising 509 wheat varieties from a two-year and two-replicate field experiments. Then, the SenNet model achieved senescence image segmentation for various wheat varieties, enabling senescence dynamics analysis and high-yielding variety screening. The results showed that: 1) The mean Intersection over Union (mIoU) of the SenNet model was 95.41 %, which represented a 4.01 % improvement over the average mIoU of seven state-of-the-art models. 2) The contributions of the local–global hierarchical attention mechanism, gated convolution, and positional encoding module to the accuracy improvement of SenNet were 3.15 %, 1.62 %, and 1.03 %, respectively. 3) SenNet can be transferred across years and locations. The mIoU accuracy of the SenNet across locations is 96.01 %. Furthermore, the model trained in 2023 can be transferred to 2022 and 2024, achieving mIoU accuracies of 93.75 % and 93.27 %. 4) High-yielding varieties typically experience a later onset of senescence and faster senescence in later stages. Based on the senescence law, this study further constructed new dynamic traits of senescence (e.g., AreaUnderCurve). Leveraging the random forest-based yield prediction (R^2^ = 0.68) from the dynamic traits, high-yielding varieties were screened with an average precision, recall, F1 score, and accuracy of 81 %, 79 %, 80 %, and 87 %, respectively. This study provides an efficient method for monitoring senescence dynamics and predicting yield, offering new insights into the screening of high-yielding varieties.

PhenoNet: A two-stage lightweight deep learning framework for real-time wheat phenophase classification

Abstract

The real-time monitoring of wheat phenology variations among different varieties and their adaptive responses to environmental conditions is essential for advancing breeding efforts and improving cultivation management. Many remote sensing efforts have been made to relieve the challenges of key phenophase detection. However, existing solutions are not accurate enough to discriminate adjacent phenophases with subtle organ changes, and they are not real-time, such as the vegetation index curve-based methods relying on entire growth stage data after the experiment was finished. Furthermore, it is key to improving the efficiency, scalability, and availability of phenological studies. This study proposes a two-stage deep learning framework called PhenoNet for the accurate, efficient, and real-time classification of key wheat phenophases. PhenoNet comprises a lightweight encoder module (PhenoViT) and a long short-term memory (LSTM) module. The performance of PhenoNet was assessed using a well-labeled, multi-variety, and large-volume dataset (WheatPheno). The results show that PhenoNet achieved an overall accuracy (OA) of 0.945, kappa coefficients (Kappa) of 0.928, and F1-score (F1) of 0.941. Additionally, the network parameters (Params), number of operations measured by multiply-adds (MAdds), and graphics processing unit memory required for classification (Memory) were 0.889 million (M), 0.093 Giga times (G), and 8.0 Megabytes (MB), respectively. PhenoNet outperformed eleven state-of-the-art deep learning networks, achieving an average improvement of 3.7% in OA, 5.1% in Kappa, and 4.1% in F1, while reducing average Params, MAdds, and Memory by 78.4%, 85.0%, and 75.1%, respectively. The feature visualization and ablation analysis explained that PhenoNet mainly benefited from using time-series information and lightweight modules. Furthermore, PhenoNet can be effectively transferred across years, achieving a high OA of 0.981 using a two-stage transfer learning strategy. Furthermore, an extensible web platform that integrates WheatPheno and PhenoNet and ensures that the work done in this study is accessible, interoperable, and reusable has been developed ( https://phenonet.org/).

Simultaneous prediction of wheat yield and grain protein content using multitask deep learning from time-series proximal sensing

Abstract

Wheat yield and grain protein content (GPC) are two main optimization targets for breeding and cultivation. Remote sensing provides nondestructive and early predictions of yield and GPC, respectively. However, whether it is possible to simultaneously predict yield and GPC in one model and the accuracy and influencing factors are still unclear. In this study, we made a systematic comparison of different deep learning models in terms of data fusion, time-series feature extraction, and multitask learning. The results showed that time-series data fusion significantly improved yield and GPC prediction accuracy with R2 values of 0.817 and 0.809. Multitask learning achieved simultaneous prediction of yield and GPC with comparable accuracy to the single-task model. We further proposed a two-to-two model that combines data fusion (two kinds of data sources for input) and multitask learning (two outputs) and compared different feature extraction layers, including RNN (recurrent neural network), LSTM (long short-term memory), CNN (convolutional neural network), and attention module. The two-to-two model with the attention module achieved the best prediction accuracy for yield (R2 = 0:833) and GPC (R2 = 0:846). The temporal distribution of feature importance was visualized based on the attention feature values. Although the temporal patterns of structural traits and spectral traits were inconsistent, the overall importance of both structural traits and spectral traits at the postanthesis stage was more important than that at the preanthesis stage. This study provides new insights into the simultaneous prediction of yield and GPC using deep learning from time-series proximal sensing, which may contribute to the accurate and efficient predictions of agricultural production.