OSNet: an oriented instance segmentation network of breeding plot extraction from UAV RGB imagery

Abstract

Drones have enabled large-scale breeding and cultivation experiments. However, extracting individual breeding plots from aerial images is a key prerequisite and urgent demand for extracting variety-level traits. The main difficulties in plot extraction include irregular rotation angles of the plots, ambiguous gaps both within and between plots, and variable color contrasts between the vegetation and the background. To solve these challenges, a novel oriented instance segmentation network (OSNet) is proposed by leveraging a global context transformer (GCT) and an oriented region proposal network (RPN). The performance was assessed using a welllabeled dataset with 960 plots of 160 wheat varieties across two years. Results show that OSNet achieved the AP@0.5 of 0.917, F1-score of 0.959, Accuracy of 0.966, IoU of 0.912, Recall of 0.934, and Plot-a of 0.999. OSNet outperformed five state-of-the-art (SOTA) networks with an average improvement of 3.08 %, 1.42 %, 1.19 %, 1.70 %, 1.79 %, and 0.04 % in AP@0.5, F1-score, Accuracy, IoU, Recall, and Plot-a, respectively. The sensitivity analysis proved that OSNet consistently achieved stable segmentation accuracy across different rotation angles and growth stages. The interpretability through ablation analysis showed that OSNet benefits from the oriented proposal and global information. Furthermore, OSNet can be transferred to new datasets with various years, crops, and data dimensions, supporting typical phenotyping tasks such as 2D wheat spike detection (r = 0.91) and 3D canopy height measurement (r = 0.89). The innovative methodology will be a fundamental tool for processing drone imagery, accelerating phenotypic trait extraction across various varieties and thereby expediting the breeding process.

Contributions

• To construct a series of real datasets for rotated breeding plot extraction based on two-year field experiments involving multiple crops, rotation angles, and growth stages, as well as to generate a virtual dataset with various plot patterns.
• To design a novel DL network (OSNet) for rotated breeding plot extraction and evaluate its generalizability across diverse datasets spanning multiple years, crops, and data dimensions.
• To accelerate the extraction of phenotypic traits by constructing two variety-level phenotyping pipelines based on OSNet-derived individual plots, including wheat spike detection from 2D images and canopy height measurement from 3D point clouds.

Citation

@article{zhang2025osnet,
  title={OSNet: an oriented instance segmentation network of breeding plot extraction from UAV RGB imagery},
  author={Zhang, Ruinan and Zhang, Yuanhao and Jin, Shichao and Zang, Jingrong and Zhao, Ruofan and Yao, Jiaqi and Li, Shaochen and Li, Qing and Su, Yanjun and Wu, Jin and others},
  journal={Computers and Electronics in Agriculture},
  volume={236},
  pages={110436},
  year={2025},
  publisher={Elsevier}
}