LANDMARK EXTRACTION FROM LEAVES WITH PALMATE

VENATION

Application to Grape

Rafﬁ Enﬁciaud and Soﬁ

ene Mouine

INRIA Paris-Rocquencourt - IMEDIA, Domaine de Voluceau - Rocquencourt - B.P. 105, 78153 Le Chesnay, France

Keywords:

Plant identiﬁcation, Venation extraction, Mathematical morphology, Computational botany.

Abstract:

The growing interest of Content Base Image Retrieval techniques in the context of plant identiﬁcation requires

the development of appropriate features. A considerable amount of information about the taxonomic identity

of a plant is contained in its leaves, and most of the botanical expertise uses jointly the contour and the

venation network. The current work focuses principally on the extraction of the venation network, the base

and secondary landmarks of leaves with uncluttered background, assuming only their structure as palmate.

Morphological operators are used to extract a ﬁrst approximation of the venation network, which is then

ﬁltered by a voting scheme and reconstructed using a connected component like algorithm. The base point

and the primary veins are then extracted with an accuracy of 100%, which allows identiﬁcation of the lobes

and the measurement their relative length.

1 INTRODUCTION

The wine industry is ruled by laws on exploitation and

trade. Identifying the exact type of vine is a critical is-

sue for people involved in its exploitation. A wide va-

riety of actors is concerned by vine identiﬁcation, but

only a few of them has knowledge in botany. This is in

contrast with the difﬁculty of the task, since the iden-

tiﬁcation is almost solely achievable by experts in this

speciﬁc ﬁeld. The difﬁculty of the identiﬁcation ex-

plains the growing interest in automated procedures,

such as Content Based Image Retrieval (CBIR) tech-

niques, which application in this ﬁeld is relatively new

and has shown to provide promising performances.

Aside from any technical feasibility issue, CBIR en-

dows non-expert users with fast retrieval, similarity

and browsing methods, which potentially could solve

the task of identiﬁcation without the need for exter-

nal expertise. A considerable amount of information

about the taxonomic identity of a plant is contained in

its leaves. In addition to their discriminative power,

leaves are parts of plants that are observable at al-

most any stage of maturity. CBIR was successful

in classifying Orchid species (Coutaud et al., 2009)

from scan of their leaves and using common colour

and texture descriptors. In this paper, we present a

method for extracting meaningful landmarks from the

leaves, only under the assumption of a palmate struc-

ture, a family to which the grapes belong. These

landmarks are chosen to be as close as possible to

the existing botanical expertise, in order to develop

feature extraction techniques that remain humanly in-

terpretable. To the authors knowledge, the speciﬁc

task of grapes identiﬁcation from their leaves is ad-

dressed exclusively in the botany community. Hence,

no particular reference nor ground truth are available

for evaluations. In the botany community, a lot of

work has been achieved so far and the current state of

knowledge is constantly renewed. The reader is re-

ferred to (OIV expert group “Vine selection”, 2009),

which describes the identiﬁcation cues for wide va-

riety of grapes. Many of these rely on the general

appearance of the leaf (eg. the number of lobes, the

overall shape of the blade.) Others are deﬁned directly

on the venations (eg. length and relative angles), or on

a combination of the boundary of the blade and the

venation (eg. length and width of the tooth.) The au-

thors of (Park et al., 2006) extract the venation pattern

of the leaves. Combined with the shape (Nam et al.,

2008), the venation improves the accuracy of the iden-

tiﬁcation. The paper is organized as follows. Section

2 presents the general settings. Section 3 describes a

method for extracting the venation network. These re-

sults are used for reconstructing the venation network

and detecting the base point in Section 4. Section 5

concludes and provides directions for future work.

520

Enﬁciaud R. and Mouine S. (2012).

LANDMARK EXTRACTION FROM LEAVES WITH PALMATE VENATION - Application to Grape.

In Proceedings of the 1st International Conference on Pattern Recognition Applications and Methods, pages 520-524

DOI: 10.5220/0003715705200524

 SciTePress

2 GENERAL SETTINGS

Our database contains 20 varieties of grapes, each of

which containing 19 images. Each image contains

one leaf on an uncluttered background. The grape

leaves are all palmate, which means that several pri-

mary veins split from the petiole (leaf stalk). From

primary veins then start secondary veins at bifurca-

tion points. In the image plane, the location where the

petiole joins the blade is considered as the base point.

The leaves have very different aspects even within a

class, as they are non-rigid objects and their planarity

is not guaranteed. They also may or not have lobes

that create local occlusions. The venation seems to

be the proper element on which we can normalize the

measurements. The settings make the segmentation

of the blade straightforward: a chromatic threshold

in the GHLS (Levkowitz and Herman, 1993) colour

space provides very accurate results. Some meaning-

ful information cannot be retrieved from the contour

of the leaf alone. This is the case for instance for the

lobes, their number and their apexes. Indeed the tooth

of the blade is of the same amplitude for the apex and

its surroundings, which makes the apex of the lobe

hardly distinguishable.

3 VENATION EXTRACTION

The veins are locally linear structures that have a con-

trast with the blade. They may be either brighter or

darker than the blade, depending on the variety, and

the contrast level is not known a priori. These prop-

erties make mathematical morphology operators, es-

pecially top-hat and opening with line structuring el-

ements (SE) following (Zana and Klein, 2001) ideal

candidates for extracting these structures.

3.1 Morphological Processing

We denote S

the segment SE of length ` and orienta-

tion α. We also denote γ

the morphological opening

using the SE S

. Given a particular orientation α, we

want to extract the meaningful segments with direc-

tion α from the image. We ﬁrst extract the elements

that cannot ﬁt into a line of length `

and of direction

⊥

= α + π/2 with a morphological top-hat, in order

to extract the structures having a contrast along α

⊥

We then apply a morphological opening using S

extract the segments of direction α and of length at

least `

. Efﬁcient implementation of directional ﬁlter-

ing exists (eg.(Soille and Talbot, 2001).)

The processing is performed on the luminance

channel. In our experiments, we considered 12 direc-

tions uniformly distributed on [0,π[. The sizes of the

SE play a major role in the detection of the venation.

Ideally `

should be of the magnitude of the thick-

ness of the leaves, and `

should be wide enough to

ﬁlter out small structures that cannot be associated to

segments of interest. However even with ﬁnely tuned

parameters, the retrieved veins will, in all cases, be

disconnected due to ﬁrst, the curvature and bifurca-

tion points of the venation network and second, to

the angular resolution. Rather, we consider this step

as an initial ﬁltering of the venation with a certain

amount of false detection, that we further use to re-

cover the base and the primary veins. Although the

SE are straight lines, the thickness of the veins al-

lows to consider slightly curved approximations, and

we approximate the connected components (CC) of

f ilt

with a 2-polynomial using a linear regression.

The 2-polynomial representation allows several com-

putations in closed form (intersection, length...) that

are easily expressed, and accommodates the curvature

of the detections.

3.2 Meaningful Veins

The number of false alarms produced by the previous

processing is often high, and we propose a method

for keeping the meaningful polynomials. As the scale

of the leaf is not a priori known, no parameter on the

size of the polynomials should be used. We denoted

by s

the segment indexed by k ∈ {(i,α),( j,θ)} and

∩ s

the intersection points of polynomials s

and

. Alg.1 favours the polynomials that are on the path

of other polynomials. Also, the longer the polynomi-

als, the higher the associated vote. The purpose is to

reduce the number of entities of interest in each direc-

tion, for instance by keeping the n highest votes in

Algorithm 1: Meaningful veins - Pseudo-code.

Data: l

input CC in each direction indexed by i

and α, n

: complexity parameters, n:

number of meaningful detection per α

Result: o

: detected meaningful veins

1 forall the α do

2 s

← polynomials l

sorted by decreasing

length;

3 v

← 0;

4 forall the α, θ, θ 6= α, i ∈ N

∗

, j ∈ N

∗

5 forall the t ∈ s

∩ s

, p ∈ t,

(k, k

) ∈ {(i, α),( j,θ)}

, k 6= k

6 if p ∈ s

then v

← v

+ 1;

7 forall the α do

8 o

←ﬁrst n element of s

sorted by v

;

LANDMARK EXTRACTION FROM LEAVES WITH PALMATE VENATION

- Application to Grape

521

each direction

. Due to the angular resolution, some

directions do not have any relevant polynomials but

still are in the ﬁnal set of detected segments (see Fig.1

& 3, ﬁrst row.) This issue will be addressed in the next

section.

4 BASE DETECTION

Since the venation network is palmate, we can argue

that the base point is at the intersection of the primary

veins. We hence compute the intersection points for

each pair of candidate polynomials of Alg.1, and keep

the points that either lie within the range used for the

approximation of both polynomials, or are close to it.

In fact, keeping only intersection points lying within

the range of approximation ﬁlters out relevant inter-

section point, explained by the fact that the base point

is often a large area of venation, which makes the di-

rectional top-hat miss the zone. In our experiments,

we considered intersection points that are not further

away than 5% of the length of the candidate vein.

We associate to each intersection x

a weight w

that

favours angles close to π/2 at the intersection point

between the two involved polynomials.

Given the intersections, we then use the mean-

shift (MS) (Comaniciu and Meer, 2002) algorithm to

ﬁnd the local maximum density locations, which we

consider as being candidates for the base of the leaf.

Using the normal density estimates, the update rule

yields

i+1

∑

k∈N

· w

· exp





−x





∑

k∈N

· exp





−x





(1)

where N

stands for an appropriate neighbourhood of

the intersection point x weighted by w. In our exper-

iments, we took the neighbourhood as being a ball of

radius 1% of the image size. The result of the MS

algorithm is a set of local modes, that we cluster to-

gether as explained in (Comaniciu and Meer, 2002),

resulting in a set of candidates C. Selecting the base

as the mode with highest density (denominator of

Equ.1) provides 99.74% of correct results. However,

in this setting, some polynomials may appear with

high similarity at different orientations, each of which

creating intersection points. Moreover, the number

of votes/intersections grows quadratically in the num-

ber of polynomials. This is why we believe that rely-

ing on the density alone will not behave correctly for

other databases and will be sensitive to noise.

in our experiments, we took n

= 10, n

= 5 and n = 3.

4.1 Network Reconstruction

As mentioned, the candidate venation network com-

puted in §3.2 is disconnected. Moreover, some poly-

nomials are redundant as they were detected at close

angular directions. We consider a path as a sequence

of 2-polynomials along with their boundary. The

complete algorithm proceeds in two similar steps:

ﬁrst it prunes, merges and splits intersecting polyno-

mials into an initial set of paths P . It then merges

this set of paths with close and compatible orienta-

tions boundaries. The algorithm starts by sorting the

initial set of polynomials S by decreasing length. It

then proceeds in a way similar to a region growing

by hill climbing, with the difference that it may create

new polynomials during the growing, which should

then be correctly handled. A FIFO queue c is initial-

ized with the ﬁrst element of S. Two other queues

p,g, used respectively for the current path, and the

pruned polynomials, and a priority queue q are also

initialized. For each popped-out element e of c, all

polynomials t in S , but not in p, c or g, and inter-

secting e within their respective boundary and with

an angle lower than π/12, are selected. Let’s call i the

intersection of t and e meeting these constraints. If the

boundaries of t are completely included in those of e,

then t is targeted for pruning and added into g. Other-

wise, e and t are split at their intersection i. From the

four parts obtained, the longest conﬁguration is kept,

yielding left and right polynomials l

t,c

. The new

polynomial l

t,c

∪ r

t,c

is inserted into q. After all can-

didates t have been tested, either q is empty, in which

case e is added to the current reconstructed path p, or

q contains new polynomials not in S. In that case, for

the t generating the longest new polynomial, e and t

are both removed from S, and l

t,c

and r

t,c

are added

both into S and c (region growing). Finally, when no

growing is possible, P = P ∪ p and S = S − p ∪ g.

For the second part of the reconstruction, instead

of considering intersecting polynomials, paths not

distant from more than 2 × R , with R the radius of

the kernel used in Equ.1 are considered. Indeed, poly-

nomials separated by at most 2 × R will get selected

by a base candidate (see next section).

4.2 Base Selection

For each of the MS candidates c ∈ C , we select

the subset Q

of paths from P such that Q

= {p ∈

P ,∃ p

∈ p st. d(p

,c) < R }, which amounts to solve

the roots of a 3-polynomial. We then consider the

base point B as being

B = arg max

c∈C

∑

p∈Q

length(p)

ICPRAM 2012 - International Conference on Pattern Recognition Applications and Methods

522

With the proposed method, the achieved accuracy in

the detection of the base is 100% for the speciﬁed

database, even for leaves with defects (see Fig.4.) We

should note that the detection of the base point is of

principal interest, since the structure of the venation

network is hierarchical and rooted on the base point.

Hence its detection should also be considered for join-

ing disconnected veins. Indeed, prematurely linking

some venation paths together irremediably degrades

the accuracy of the venation structure on the whole

database.

Figure 1: “Servant” variety. Top: ﬁltered venation network

(red) and MS candidates C (white). Bottom: detection of

the base (white) and associated primary veins Q

(red).

Figure 2: “Servant” variety: zoom over the base area. The

damage prevents from linking the base to the horizontal

path.

4.3 Other Landmarks

From the reconstructed primary venation network P ,

the extremities of the lobes are easily extracted by

Figure 3: “Merlot” variety. See Fig.1 for legend.

Figure 4: More results from “Servant” variety. See Fig.1

(bottom) for legend.

extrapolating the path. We used a linear extrapola-

tion with direction given by the mean of the tangent

of the 2-polynomials close to their boundary. This

is explained by the fact that the 2-polynomial repre-

sentation is accurate only within the CC used for its

regression, and may strongly diverge outside the CC.

We also recall that the leaves are not assumed to be

planar. The other base candidates C \ B are lying on

LANDMARK EXTRACTION FROM LEAVES WITH PALMATE VENATION

- Application to Grape

523

bifurcation points, which enables the measurements

of relative lengths and angles of bifurcations.

5 CONCLUSIONS AND FUTURE

WORK

We have presented a set of methods for detecting par-

ticular landmarks from grape leaves, with a focus on

the reconstruction of the venation network and the

base point. Our method starts from an initial detec-

tion of segments in the image, which are then ﬁltered

based on the number of their intersections. The ar-

eas of intersections are clustered, each cluster being

a candidate base point. The initial venation network

being disconnected, we proposed a hill-climbing like

algorithm for its reconstruction, resulting in a set of

venation paths. Finally, the base point is selected as

being the candidate from which the total length of the

closest paths is maximal.

The proposed scheme detects the base point with

an accuracy of 100% for the grape leaves database,

and is robust to scale, orientation and contrast or illu-

mination change. The main assumption being made

is related to the palmate structure of the leaves, which

induces a hierarchical venation network rooted on the

base point. The landmark extraction is a preliminary

yet necessary step toward a proper classiﬁcation and

identiﬁcation of the leaves. We applied successfully

the venation and base extraction on the palmate subset

of a more general database. As future work, we can

mention the classiﬁcation of the leaves based on the

morphometric features, by the joint use of the blade,

the reconstructed venation network and the extracted

landmarks. Also, the discriminative power and sen-

sitivity of the descriptors against the number of vari-

eties and the amount of data are of principal interest.

ACKNOWLEDGEMENTS

This research has been conducted with the support

of the Agropolis Foundation through the Pl@ntNet

project. The authors would also like to thank

the members of the Pl@ntGrape case study, and

in particular Jean-Michel Boursiquot & Thierry La-

combe (INRA, UMR AGAP, DAVEM team), Thierry

Dessup (INRA, Vassal Domain) and Christophe

Sereno (INRA, UMR Amap).

REFERENCES

Comaniciu, D. and Meer, P. (2002). Mean shift: a robust

approach toward feature space analysis. IEEE Trans.

on PAMI, 24(5):603–619.

Coutaud, M. et al. (2009). Advances in taxonomic identiﬁ-

cation by image recognition with the generic content-

based image retrieval Ikona. In e-Biosphere.

Levkowitz, H. and Herman, T. (1993). GLHS: A gen-

eralized Lightness, Hue, and Saturation color mod-

els. CVGIP: Graphical Models and Image Process-

ing, 55(4):271–285.

Nam, Y. et al. (2008). A similarity-based leaf image

retrieval scheme: Joining shape and venation fea-

tures. Computer Vision and Image Understanding,

110(2):245–259.

OIV expert group “Vine selection” (2009). The OIV de-

scriptor list for grape varieties and Vitis species. Tech-

nical report, International Organisation of Vine and

Wine (OIV).

Park, J.-K. et al. (2006). A venation-based leaf image classi-

ﬁcation scheme. In Information Retrieval Technology,

volume 4182 of LNCS, pages 416–428. Springer.

Soille, P. and Talbot, H. (2001). Directional morphological

ﬁltering. IEEE Trans. on PAMI, 23(11):1313–1329.

Zana, F. and Klein, J. (2001). Segmentation of vessel-like

patterns using mathematical morphology and curva-

ture evaluation. IEEE Transactions on Image Process-

ing, 10(7):1010–1019.

ICPRAM 2012 - International Conference on Pattern Recognition Applications and Methods

524