Plant size estimation based on the construction of high-density corresponding points using image registration

• Published in: Computers and electronics in agriculture 2019-02, Vol.157, p.288-304
• Main Authors: Rakun, Jurij, Stajnko, Denis, Zazula, Damjan
• Format: Article
• Language: English
• Subjects: Accuracy; Affine transformation; Algorithms; Cameras; Color matching; Dense corresponding points; Density; Digital imaging; Elastic deformation; Euclidean geometry; Image detection; Image registration; Mathematical analysis; Matrix algebra; Matrix methods; Multiview geometry; Objects in natural scenes; Pixels; Plant size estimation; Registration; Shades; Template matching; Triangles
• ISSN: 0168-1699; 1872-7107
• DOI: 10.1016/j.compag.2018.12.053

•We successfully estimated the sizes of natural objects by using outdoor images.•We estimated the distances to the objects by using images of the natural scene.•We reveal a novel approach to detect dense corresponding points on outdoor images.•Dense correspondence can be detected even on surfaces, where other algorithms fail.•We introduce a measure of goodness to obtained the best corresponding pairs. This paper presents an approach to estimating the plant or fruit size from objects on the digital images of natural scenes. Two images are taken from camera positions with a known distance between them. For a selected object and a pair of its boundary pixels in the first image, a corresponding pixel pair is found in the second image. The Euclidean distances within the two pairs compare in similar triangles and combine with the known distance between the camera positions to estimate the size of the object in metric units. A dense correspondence among images is first required. It can be determined by joint information of multiple 2D views of the same scene utilizing image registration techniques. These are used to align corresponding image points, without having to rely on distinct features, such as geometric properties. This is most advantageous for weak textured areas, e.g. uniform fruit shades, where many methods for corresponding point detection fail. The pixel neighbourhoods of candidate corresponding pairs are compared by template matching to verify their similarity and to select the most reliable correspondences. A stratified rigid-to-elastic registration approach generates a deformation matrix whose elements define translational vectors on a pixel basis. The accuracy of correspondence, checked by colour template matching, is assessed by mean absolute error, D, between corresponding pixel-pair intensities within the templates. By matching 26 random fruit image pairs, the proposed approach detected 1390.5 ± 1129.8 reliable corresponding points on the surface of the fruit with the accuracy of D≤5, on average. This means approximately 18% of all objects’ pixels and considerably exceeds the results of comparable methods for high-density correspondence detection, such as correlation-based correspondence, non-rigid dense correspondence (NRDC), and scale invariant feature transform (SIFT). The number and quality of corresponding points obtained by the proposed algorithm turned out to be reliable and sufficiently robust for an accurate estimation of distances be