Multi-granular Angle Description for Plant Leaf Classification and Retrieval Based on Quotient Space

1. Introduction

Plant identification is an important issue in agricultural information and botany research. Plants’ leaves are the most commonly used morphological character to supply valuable discriminatory information for plant classification [1,2]. With the wide use of digital cameras and computers, automatic leaf identi¬fication using image understanding and computer vision techniques becomes a hot topic recently [3].

General practices for plant leaf classification involve several steps, including leaf image preprocessing, feature extraction/matching, and classifier design. Among these steps feature extraction and classifier design are crucial components. There are some studies using machine learning approaches to classify plant leaves, such as deep convolutional neural network, multi-layer perceptron and support vector machines [4]. Some approaches need feature extraction work, while some approaches needn’t. Many studies focus on designing novel leaf feature extraction methods [5-7]. Leaf shape, texture and venation features are developed to distinguish leaves of different species. As one of contents in leaf images, shape can provide many valuable evidences for identifying the plant species due to the rich morphological variations. Hence shape features play an important role in leaf retrieval and classification. How to extract leaf shape feature efficiently becomes a crucial issue to complete classification and retrieval.

Leaf shape features extracted using existing shape description methods are adopted in some studies. Elliptic Fourier harmonic functions and a complexity index of the leaf shape are generated from leaf contours in reference [8], and good classification performance was achieved for weed species identification. In [1], fast curvelet transform and invariant Hu moments were used to capture leaf shape features. A set of leaf features including morphological features, shape-defining feature and the well-known Fourier descriptors, were extracted from leaf images for plant classification [9]. Fourier descriptor was also used together with Hu moment invariants in reference [2], combined with texture features for leaf identification. The normalized multi-scale bending energy was used as shape description method, and optimized for finding parameters that best fit the descriptor for leaf classification [10]. There are also some studies developing new shape description methods for plant leaf classification. In [6], a variation of inner-distance shape context (IDSC), a well-known contour-based descriptor, has also been used to calculate global and local information of leaf shape independently. And a pattern counting strategy was advocated for leaf matching. Recently, Wang et al. [11] proposed the multi-scale arch height (MARCH) to describe geometric features of the leaf contours, and leaf image retrieval application was developed in the study. This method extracted hierarchical arch height features at K-scale for each leaf contour point, and achieved better classification rate and retrieval accuracy performance on four leaf datasets compared with several well-known leaf shape representation methods. However, it’s a worthy topic to determine the value of critical parameter K reasonably. In [5], the multi-scale R-angle was proposed to describe leaf contour curvature using angle features derived from contour points, and outperformed many state-of-the-art leaf classification methods such as IDSC, triangle-area representation (TAR), MARCH and multi-scale distance matrix. However, the multi-scale generation strategy was produced by varying the radius parameter R at each contour point, and cannot describe the local detail information very well. An extensive overview of leaf shape description methods is given in [12].

Existing research results indicate that angle feature derived from contour points has good properties and has been used for shape retrieval and leaf classification tasks [5,13-15]. Angle feature is intrinsically invariant to rotation, translation and scaling. Furthermore, it is easy to extend to multi-scale description, with which hierarchical features from local contour variations to global information can be captured well. Hence angle feature is a promising feature for leaf description. However, it is a key issue to determine the scale parameters of angle feature. Because there is lack of theoretical support and no guarantee for the classification performances. In addition, different variations of angle feature can be studied to take full advantage of this feature.

The granular computing of the quotient space theory is a new tool for solving complex problems, and attracts more and more attentions from researchers [16,17]. The essential characteristics of objects in images can be excavated and analyzed at different levels with the theory, and it accords with human cognition principle. The granular computing of the quotient space has demonstrated very good performances in several research fields, such as image segmentation, image retrieval and remote sensing image analysis. Yin et al. [18] applied granular theory in image segmentation. Image features were extracted with different granu¬larities from reduced images and respectively, and original images are partitioned by the fusion of features according to quotient space theory.

In this paper, granular computing of the quotient space theory is extended to describe leaf contours for plant leaf classification and retrieval. The main contribution of our work can be summarized as follows: (1) A multi-granular leaf feature description method is proposed. The method uses granular computing to extract angle information from leaf contours, and can finely describe leaf information from coarse to fine. (2) Three kinds of angle features are derived from the angle information for each leaf to make a comprehensive leaf descriptor. And in leaf matching stage, similarities of each pair of leaves are measured and synthesized using the three features under multi-granularities according to quotient space theory. Finally, plant leaf classification and retrieval experiments are conducted on two challenging leaf image databases. Compared with machine learning approaches which need learning special parameters to classify plant leaves, the proposed method is much simpler and very timesaving because of no training phase. Besides, there may be not sufficient training data available for machine learning approaches, and the classification performance of machine learning approaches would be affected. Hence the proposed method has great advantages over machine learning approaches.

The remainder of this paper is organized as follows: Section 2 introduces the proposed multi-granular leaf contour description method and the three kinds of angle features in detail. Section 3 presents the experi-mental results and the comparison with other methods. Conclusions derived from the study are presented in Section 4.

2. The Multi-Granular Angle Description Based on Quotient Space

The granular computing studies the difference between descriptions under different granularities of the same problem, and obtains a solution from the comprehensive study of these descriptions. Quotient space theory is a new mathematic model for granular computing. The model represents problem to be solved as triple elements function [TeX:] $$(\mathrm{X}, \mathrm{F}, \mathrm{T})$$, where the symbols X, F and T denote the domain, property function and the structure of the domain respectively. For a given equivalence relation R, the quotient space corresponding to [TeX:] $$(\mathrm{X}, \mathrm{F}, \mathrm{T})$$ is [TeX:] $$([\mathrm{X}],[\mathrm{F}],[\mathrm{T}])$$. And [TeX:] $$[\mathrm{X}]$$ is defined as quotations set of X corresponding to R. [TeX:] $$[\mathrm{T}]$$ is a quotient topology, denoted by [TeX:] $$\left\{\mathrm{u} \mid \mathrm{p}^{-1}(\mathrm{u}) \in \mathrm{T}, \mathrm{u} \in[\mathrm{X}]\right\}$$ where p is a natural projection from X to[TeX:] $$[\mathrm{X}]$$. Parameter F represents the property function. If R is defined under a certain granularity, then [TeX:] $$([\mathrm{X}],[\mathrm{F}],[\mathrm{T}])$$ can be viewed as a description under this granularity. Problems can be analyzed under multiple granularities, and different levels of image features can be obtained corresponding to different granu-larities. In order to obtain comprehensive feature information of an image, the image should be described under multiple granularities and these features can be fused effectively. The optimal criterion of fusion is with regard to image description granularities, and needs to be determined based on the specific application. As for the leaf shape description problem, most existing methods tend to describe and match leaves under a certain granularity. Hence many shape descriptors can finely capture either global or local features of the contour. And these descriptors with only global or local information may fail to be robust enough for leaf classification and retrieval tasks.

2.1 Quotient Space for Leaf Classification and Retrieval

From the perspective of quotient space, leaf classification and retrieval tasks can be achieved by dividing leaf image database into two disjoint subsets. The leaves in one subset belong to the same species to query leaf, and leaves in another set are different from the query. According to the principle of quotient space, leaf image database is represented as a triple elements function [TeX:] $$(\mathrm{X}, \mathrm{F}, \mathrm{T})$$, where X denotes the domain composed of leaves in database. If there are N leaf images in database, then [TeX:] $$\mathrm{X}=\left\{\mathrm{S}_{1}, \mathrm{S}_{2}, \cdots, \mathrm{S}_{N}\right\}$$, and [TeX:] $$\mathrm{S}_{\mathrm{i}}(\mathrm{i}=1, \ldots, \mathrm{N})$$ represents the [TeX:] $$i^{\mathrm{th}}$$ leaf image. Property F represents feature functions of leaves in image database under a certain granularity. The structure of the domain T represents similarity function between leaf images.

Leaf classification and retrieval results can be obtained using a triple elements function [TeX:] $$(\mathrm{X}, \mathrm{F}, \mathrm{T})$$ under a certain granularity, and denoted by [TeX:] $$([\mathrm{X}],[\mathrm{F}],[\mathrm{T}])$$. The representation [TeX:] $$([\mathrm{X}],[\mathrm{F}],[\mathrm{T}])$$ means different image database division corresponding to a certain granularity. Specifically, [X] is the leaf image subsets under the granularity, and [F] is feature attributes of leaves in [X], and [T] represents structural relationship of leaf subsets determined by the similarity function between leaves. Implement of the transformation from [TeX:] $$(\mathrm{X}, \mathrm{F}, \mathrm{T})$$ to [TeX:] $$([\mathrm{X}],[\mathrm{F}],[\mathrm{T}])$$ under a certain granularity means completion of leaf classification and retrieval under the same granularity. The corresponding equivalence relationship is the similarity matrix of leaf features.

2.2 Multi-granular Leaf Feature Extraction

Leaf description under different granularities based on quotient space can capture leaf shape features from coarse to fine. In this paper, three kinds of angle features are derived for each leaf contour under multi-granularities to make a comprehensive leaf descriptor. Multi-granular leaf feature extraction includes four steps: leaf preprocessing, contour sampling, multi-granular description using angle features, and feature fusion.

For leaf preprocessing, original color leaf images are transformed into binary images, and leaf contours are extracted from these binary images using boundaries detection algorithm. Then leaf contours are sampled with equal-length interval, so each leaf contour can be expressed uniformly by an ordered (clockwise or counterclockwise) coordinates set, [TeX:] $$C=\left\{p_{i}=\left(x_{i}, y_{i}\right)\right\}$$, for [TeX:] $$i=1,2, \ldots, M$$, where [TeX:] $$x_{i}$$ and [TeX:] $$y_{i}$$ represent the coordinates of point [TeX:] $$p_{i}$$, and M is the number of leaf contour points.

The multi-granular description using angle features is detailed as follows. For each point on leaf contour C, angle features can be calculated with pairs of neighbor contour points. The selection strategy for neighbor contour points under different granularities has an important effect on the describing abilities of angle features. The multi-scale R-angle method introduces the intersections of the contour with a circle of radius R as neighbor points. And the multi-scale R-angle is produced by measuring the sine of the angle between the intersections using the intersections and sampled points. However, location of intersections with a circle of variable radius for each point is time-consuming, and R-angle features will misclassify points with similar contour curvature but different center distance. In this paper, pairs of neighbor points are selected under a certain granularity with different arc-length interval. For a leaf contour point pi, we trace the contour in the forward and reverse directions with arc length interval equal to d, and two neighbor points pr, pl are selected. The angle of triangle [TeX:] $$\left(p_{l}, p_{i}, p_{r}\right)$$ at [TeX:] $$p_{i}$$ can be considered as description of contour curvature under granularity with interval equal to d. When the interval is small, the angle describes contour curvature under fine-granularity and local contour variations can be captured well. And when the interval is large, the angle can be viewed a representation under coarse-granularity, and global information of contour is preserved. By changing the arc length interval from small to large, the angle features can describe the leaf contour from coarse to fine. Assume that there are S granularities used. Examples of angle features at [TeX:] $$p_{i}$$ under granularity s (denoted by [TeX:] $$\theta_{i s}$$ for [TeX:] $$\mathrm{s}=1,2, \ldots \mathrm{S}$$) are given in Fig. 1.

To describe leaf comprehensively, three kinds of angle features are derived from triangle [TeX:] $$\left(p_{l}, p_{i}, p_{r}\right)$$ for each leaf contour point and each granularity. The calculation processes are presented as follows. First, the angular value [TeX:] $$\theta_{i s}$$ at point [TeX:] $$p_{i}$$ under granularity s ([TeX:] $$(s=1 \ldots S)$$) is calculated using formula (1).

where [TeX:] $$\theta_{i s} \in[0, \pi]$$. And angle feature vector under granularity s are [TeX:] $$\theta_{s}=\left[\theta_{1 s}, \theta_{2 s}, \ldots, \theta_{M s}\right]$$ with [TeX:] $$\mathrm{s}=1 \ldots \mathrm{S}$$.

Second, angular histogram is generated using the angle feature vector under the corresponding granularity. Suppose that there are K bins in angular histogram [TeX:] $$H_{s}$$ under granularity s. Since angular value ranges from 0 to [TeX:] $$\pi$$, they are equally divided into K bins. The angular value [TeX:] $$\theta_{i s}$$ belongs to [TeX:] $$k^{\mathrm{th}}$$ bin of [TeX:] $$H_{s}$$ if it satisfies the following condition.

Count the number of angular for each bin to construct the histogram for each granularity. And angle histogram feature of leaf contour C under granularity s are [TeX:] $$H_{s}=\left[\operatorname{bin}_{1}, \operatorname{bin}_{2}, \ldots, \operatorname{bin}_{K}\right]$$ with [TeX:] $$\mathrm{s}=1 \ldots \mathrm{S}$$.

Third, the ternary angular pattern (TAP) is proposed to describe relations between the adjacent angular features under the same granularity. Inspired by binary angular pattern, TAP employs more reasonable coding rules to form a feature vector. For point [TeX:] $$p_{i}$$ under granularity s ([TeX:] $$s=1 \ldots \mathrm{S}$$), the angular value [TeX:] $$\theta_{i s}$$ are compared with angular values [TeX:] $$\theta_{l s}$$ and [TeX:] $$\theta_{r s}$$, respectively for pattern coding. Since there cases of contour segments for leaves, the coding rules use 1, 0, -1 to represent convex points, concave points and points on linear segment respectively. Take the comparison between [TeX:] $$\theta_{i s}$$ and [TeX:] $$\theta_{r s}$$ for example, the coding rules of TAP at [TeX:] $$p_{i}$$ for the neighbor point [TeX:] $$p_{r}$$ under the granularity s are

where is the [TeX:] $$T A P_{i r s}$$ at [TeX:] $$p_{i}$$ for [TeX:] $$p_{r}$$, and [TeX:] $$\theta_{t h}$$ is the comparison threshold with a positive value. The TAP at [TeX:] $$p_{i}$$ for [TeX:] $$p_{l}$$ is calculated in a same way and is denoted by [TeX:] $$T A P_{i l s}$$ . And the TAP at [TeX:] $$p_{i}$$ is coded with [TeX:] $$\left[T A P_{i l s} T A P_{i r s}\right]$$ under granularity s. TAP feature of leaf contour C are [TeX:] $$T A P_{s}=\left[T A P_{1 l s}, T A P_{1 r s},, \ldots, T A P_{M l s}, T A P_{M r s}\right]$$ under granularity s with [TeX:] $$\mathrm{s}=1 \ldots \mathrm{S}$$. For each leaf, there are three features [TeX:] $$\theta_{s}, H_{s}$$ and [TeX:] $$T A P_{s}$$ corresponding to the granularity s.

The quotient space for the leaf image database is [TeX:] $$(\mathrm{X}, \mathrm{F}, \mathrm{T})$$, where X are the leaf images, and F are the three features, and T is the similarity structure between features of leaf images under multi-granularities. So, the next step is to measure leaf similarity using features under multi-granularities.

2.3 Leaf Similarity Measurement and Synthesis

Several measurements can be using as metrics of leaf similarity, such as commonly used city block and dynamic programming. Since dynamic programming algorithm is much time-consuming, it is unsuitable for large leaf image database. And city block metric cannot apply directly on these leaf features, because the angle feature vector or TAP changes with shift of starting point of leaf contour C and affect leaf matching results. Hence fast Fourier transform (FFT) is applied after angle feature vector and TAP are extracted under each granularity. And only the magnitudes of the coefficients are used to make them independent from the starting point.

Then the similarity between each pair of leaves for each feature is measured using city block under each granularity. Take angle feature for example. Let [TeX:] $$\mathrm{F}_{\mathrm{As}}=\left[\mathrm{f}_{\mathrm{A} 1 \mathrm{s}}, \mathrm{f}_{\mathrm{A} 2 \mathrm{s}}, \ldots \mathrm{f}_{\mathrm{AXs}}\right]$$ and [TeX:] $$\mathrm{F}_{\mathrm{Bs}}=\left[\mathrm{f}_{\mathrm{B} 1 \mathrm{s}}, \mathrm{f}_{\mathrm{B} 2 \mathrm{s}}, \ldots \mathrm{f}_{\mathrm{B} \mathrm{X} \mathrm{s}}\right]$$ denote the final angle feature vector which are extracted from leaf A and leaf B under granularity s, respectively. And parameter X is the dimension of final angle feature vector. The leaf dissimilarity between A and B under granularity s can be calculated by

The same operations can be applied for both angle histogram feature and TAP. The smaller the dissimilarity is, the greater the similarity is.

In the multi-granularity synthesis stage, the optimal criterion is constructed according to leaf classification and retrieval performance. From the multi-granular leaf feature extraction process, we know that local contour information will be captured well under fine-granularity. As the arc-length interval increases, the granularity becomes coarser. A good leaf descriptor should capture the hierarchy information ranging from global to local variations of leaf contours. And leaf local information changes rapidly, so more fine-granularities should be included in synthesis stage. While global information of leaf contours changes slowly, and less coarse-granularities in synthesis stage can avoid redundant description.

In addition, the three kinds of angle feature make unequal contributions to distinguish leaves for completing retrieval and classification tasks. Hence, unequal weights should be given to the three features for feature fusion. Assume that there are S granularities used in this paper. The distance matrix corresponding to [TeX:] $$\theta_{s}, H_{s}$$ and [TeX:] $$T A P_{s}$$ are denoted by [TeX:] $$D_{1 s}, D_{2 s}$$ and [TeX:] $$D_{3 s}$$, respectively under granularity s. Thus, the final leaf similarity is defined as follows:

where [TeX:] $$w_{1}, w_{2}$$ and [TeX:] $$w_{3}$$ serve as weights of the distance matrix and satisfy formula (9). S is the total number of granularities.

3. Plant Leaf Classification and Retrieval Experimental Results

To evaluate the effectiveness of the proposed method, two challenging leaf datasets, the Swedish leaf dataset and Flavia dataset are used to perform plant leaf classification and retrieval experiments. And the leaf classification and retrieval performances of the proposed method are compared with that of the state-of art leaf description methods. All the experiments are carried out on a notebook computer with MATLAB software.

3.1 Leaf Database and Performance Evaluation Measure

The Swedish leaf database contains 1,125 leaf images, which are from 15 different Swedish tree species with 75 leaves each species [19]. Fig. 2 shows representative leaves from the 15 species. The Swedish leaf database is suitable for classification and retrieval testing because of large interclass distance and certain similarities between different classes.

The second Flavia leaf database is challenging leaf database, and used in many plant leaf identification and retrieval experiments. There are 1,907 leaf images in total, and 32 different species with 50–77 images for each species [20]. Fig. 3 shows representative leaves from the 32 species.

The mean average precision (MAP), precision and recall curves are used to evaluate leaf retrieval and classification performances, which are the most popular measure in CBIR. Precision measures the accuracy of the retrieval and the speed of recall, and recall measures the robustness of the retrieval. MAP is the mean of average precision of all queries, and calculated using the same approach as [5].

3.2 Parameter Setting and Result Comparisons

In our experiments, the number of leaf contour points is 256. In the construction process of multi-granular angle description, the granularity settings, bin numbers and weights of feature vectors will affect retrieval results directly. First, we tested the impact of value of granularity S using angle feature vector. As mentioned in Section 2.3, more fine-granularities with small interval and less coarse-granularity with large interval should be included in synthesis stage. In the multi-granularity synthesis stage, we use a simple way to satisfy this requirement. The adjacent granularities are determined by double the arc-length interval of fine-granularity, and the fine-granularity with the smallest interval is with arc-length interval equal to round ([TeX:] $$M / 100$$). Hence, the arc length interval is set to 3, 5, 10, 20, 41, and 82, corresponding to granularity 1–6, respectively. There are six angle feature vectors in total for each leaf image. And the retrieval performance of the angle feature under each granularity on the Swedish leaf database is shown in Fig. 4.

From Fig. 4, it can be seen that angle feature under single granularity with only global or local information fail to be robust enough for leaf retrieval task. The retrieval performance of multi-granular angle feature after synthesis is also tested. The optimal criterion is decided by the contribution of features under each granularity. Considering that global and local information play an equally important role for leaf contour description, each granularity is synthesized equally. To make the matching faster, only top 7 elements of angle feature vector under each granularity are used, and the retrieval performance is also shown in Fig. 4 (please see the blue line with [TeX:] $$\Delta$$). It can be seen that multi-granular angle feature description achieves better performance than each single-granular angle feature description.

Second, the effect of bin numbers in angular histogram is tested. Since angular value ranges from 0 to [TeX:] $$\pi, 5^{\circ}$$ is used as the step interval to construct angular histogram [TeX:] $$H_{s}$$. That is to say the bin number K is set to 36 ([TeX:] $$5^{\circ}$$), 18 ([TeX:] $$10^{\circ}$$), 12 ([TeX:] $$15^{\circ}$$), 9 ([TeX:] $$20^{\circ}$$) and 6 ([TeX:] $$30^{\circ}$$), respectively ([TeX:] $$25^{\circ}$$ is not considered here because 180/25 is not an integer.). And the precision and recall curves of TAP under multi-granularities with different K are shown in Fig. 5.

It can be seen from Fig. 5 that the appropriate value for bin number is 18 or 12, because the precisions at each level of recall in most cases are higher than that for bin number with 36, 9 or 6. And the bin number is set to 12 finally in consideration of compactness of the angular histogram.

Third, the weights of the distance matrix in the feature fusion to calculate the final leaf similarity are also determined. As indicated in [5], angle feature has strong descriptive ability, and can be assigned with bigger weight. The angular histogram is suitable for describing the global information of leaf, and derived from the angle feature with some information missed. So, the angular histogram should be assigned with small weight. TAP just preserve relationship of size between angles, and smaller weight is appropriate. In quantification, the weight parameters are set to [TeX:] $$w_{1}=0.67, w_{2}=0.3$$ and [TeX:] $$w_{3}=0.03$$. For TAP feature the weight parameters is smaller than the other two. It makes sense for two reasons to keep the TAP value in the method. Theoretically, TAP can describe relations between the adjacent angular features, and this relative information can represent the convex, concave and/or linear variation. Experimentally, no TAP in the proposed method will reduce MAP score by 0.84 percentage points for Swedish leaf image database, and 1.48% points for Flavia leaf database. And the MAP scores for each species in Swedish leaf database are shown in Fig. 6. The MAP score over the database is 75.47%.

The proposed method is compared with the state-of-art multi-scale leaf description methods in terms of retrieval performance, including multi-scale R-angle, MARCH [5,11]. In [5,11], both MARCH and multi-scale R-angle are fused with existing global contour features such as eccentricity, solidity and rectangularity. In order to facilitate comparison, the compared methods are re-implemented without fusion with global contour features. And, there MAP scores are listed in Table 1.

From Table 1, it can be seen that the retrieval performance of the multi-granular angle feature is better than that of multi-scale R-angle and MARCH. The multi-granular angle feature gets the highest MAP value with 75.47%, which is 6.62% higher than the second, multi-scale R-angle descriptor.

To make a comprehensive comparison, leaf classification experiments are also conducted on the Swedish leaf database. For each leaf species, 25 samples are used as training images and the remaining 50 samples are used as testing images. And 1-NN classification rule is taken to calculate the classification rate, which is the same as in [11]. The classification rates of the multi-granular angle feature and the state-of-the-art approaches, including the well-known inner distance (IDSC), MARCH, multi-scale R-angle and multi-scale convexity concavity (MCC) representation [21], are listed in Table 2.

These classification performances in Table 2 demonstrate that the multi-granular angle feature achieves comparable classification rate with MCC on Swedish leaf database, and higher than multi-scale R-angle, MARCH, and IDSC. The major reason for the promotion compared with multi-scale R-angle is that the multi-granular angle feature uses reasonable granularities and more angle information to capture the hierarchy information ranging from global to local information. On the contrary, multi-scale R-angle is affected by the centroid in essence. The classification rate of the proposed method is slightly lower than that of MCC, and the main reason lies in the matching stage. In MCC the optimal matching of two leaves is achieved using dynamic programming (DP) approach, and shape complexity is estimated to improve the similarity measure. However, the MCC suffers from being computationally expensive. Regarding the computational complexity, MCC takes a matching complexity of [TeX:] $$O\left(N^{3}\right)$$, where N is the number of contour points. While the matching complexity of the proposed method is [TeX:] $$O(1)$$, which is far less than that of MCC. In addition, it is difficult and has no theoretical foundation to determine the optimal parameter of each scale for MCC.

Leaf retrieval performance is also tested on Flavia leaf database for the multi-granular angle feature. The same evaluation method in [5,11] is adopted here. The MAP scores of each species in Flavia leaf database are shown in Fig. 7, and the MAP score over the database is 67.29%.

Compared with the results on the Swedish leaf database, the MAP scores on Flavia leaf database are much lower. This is because that Flavia leaf database is more challenging than the Swedish leaf database. Firstly, Flavia leaf database has many more leaf images than Swedish leaf database. Secondly, similarities between different classes are important causes of the misclassification, which appear in Flavia leaf database more than in Swedish leaf database. It can be seen from Figs. 6 and 7 that for the best classified species, species 7 in Swedish leaf database has minimal similarities with other species, which is with 100% MAP score. And species 17 in Flavia leaf database has 99.2% MAP score, which has similarities with species 13, 18, and 30. And the minimum value of MAP scores is 0.52 for species 9 in Swedish leaf database, while there are eight leaf species with MAP scores under the 50% in Flavia leaf database (species 9, 10, 13, 15, 16, 22, 25, 30).

We also compared the proposed method with the state-of-art leaf description methods on Flavia leaf database. Besides MARCH and multi-scale R-angle, IDSC, Shape Contexts with DP, TAR [22], and MDM-RM [23] are also used for comprehensive comparison. The MAP scores of these methods are listed in Table 3.

Two conclusions can be drawn from Table 3. Firstly, the compared methods also achieved much lower MAP scores on Flavia leaf database compared with Swedish leaf database. Secondly, the multi-granular angle feature descriptor achieves the best classification rate with MAP score 67.29% on Flavia leaf database. The MAP score is 0.99% higher than the second-best Shape Contexts, and 3.79% higher than the third best multi-scale R-angle. Hence the multi-granular angle feature achieves an excellent performance in comparison with these notable approaches.

4. Conclusions

In this paper, we have proposed the multi-granular angle feature descriptor based on quotient space for plant leaf classification and retrieval tasks. The descriptor extracts angle features from leaf contour points under different granularities, and is a comprehensive leaf descriptor composed of three kinds of angle features. Similarities of each pair of leaves are measured and synthesized using the multi-granular angle feature under multi-granularities. Plant leaf classification and retrieval experiments are conducted on two challenging databases, Swedish leaf and Flavia leaf database. The performances of the proposed method are compared with state-of-the-art methods. The results of classification and retrieval experiments demonstrate that the proposed method has very promising retrieval performance.

Acknowledgement

This paper is supported partially by Universities Key Scientific Research Project of He’nan (Project No. 19A520029, 19B520017).