An Android Application for Leaf-based Plant Identification

Sofiene Mouine† , Itheri Yahiaoui†‡ , Anne Verroust-Blondet† , Laurent Joyeux† ,

Souheil Selmi† , Hervé Goëau†
†
Inria Paris-Rocquencourt 78153 Le Chesnay, France
‡
CReSTIC Reims University, 51687 Reims, France

sofiene.mouine@inria.fr, itheri.yahiaoui@univ-reims.fr, anne.verroust@inria.fr,

laurent.joyeux@inria.fr, souheil.selmi@inria.fr, herve.goeau@inria.fr

ABSTRACT
3G/3G+/4G/WIFI
This paper presents an Android application for plant iden-
tification. The system relies on the observation of leaf im-
ages. Unlike other mobile plant identification applications,
the user may choose the leaf characters that will guide the
identification process. For this purpose, two kinds of de- Android device
scriptors are proposed to the user: a shape descriptor based
on a multiscale triangular representation of the leaf margin
and a descriptor of the salient points of the leaf. The ap- List of species using
plication achieves good identification accuracy and provides a KNN classifier
Android users a useful system for plant identification.

Categories and Subject Descriptors Most similar Server 2

images
H.5.1 [Information Interfaces and Presentations]: Mul- Leaf image
timedia Information Systems retrieval

General Terms
Algorithms, Experimentations Figure 1: Plant identification process

Keywords set of leaf descriptors that have shown a noteworthy perfor-

Android application, plant identification, leaf descriptor, shape mance on several public leaf databases. In addition to leaf
representation, local descriptor species identification, the user will also be able to compare
the effectiveness of the descriptors for a given leaf image.
1. INTRODUCTION AND MOTIVATION
Identifying plants is a challenging task considering the 2. PLANT IDENTIFICATION PROCESS
large number of existing species in the world. The inter- The plant identification process is summarized in Figure
species similarity and the intra-species variability make the 1. The user captures a leaf image with an Android device.
identification task particularly difficult and time consuming. Hypotheses made on the taken images are the same as those
We present here a practical plant identification tool based in [4]: the image contains a centred single leaf on a uniform
on the visual information provided by leaves. Our tool is an background. Before launching the identification, the user
Android application which has been developed within the has to select a leaf character that will be the basis of the
Pl@ntNet1 project. The application is intended for mobile identification (margin, venation points). Then, the leaf im-
devices to allow a user to identify plants on the spot. It can age is sent to a primary intermediate server with a degraded
also be used as an observation collector tool to enrich the quality to save bandwidth. However, this step requires an
knowledge database. Unlike similar applications where the adequate bandwidth. For this purpose, a 3G network or
identification is a black-box processing, our mobile system later networks are required. The role of the primary server
enables the user to choose a specific leaf descriptor (margin, is to store the image in order to broaden the knowledge
venation points, both of them) that will be used as the ba- about plant species. The identification step is performed on
sis of identification. Within this application, we focus on a the second server using the descriptor previously selected
by the user. The descriptors have been embedded in the
Copyright is held by the author/owner(s). IKONA content-based image retrieval system [1]. Finally, a
ICMR’13, April 16–20, 2013, Dallas, Texas, USA. 1
ACM 978-1-4503-2033-7/13/04. http://www.plantnet-project.org/

309
ranked list of leaf species is returned and displayed on the 2.3 Plant identification methods
Android device (cf. Figure 2). We use a knowledge database, Two kinds of methods are used within the Android appli-
which is off-line indexed using each of the descriptors sug- cation:
gested to the user. On the other hand, the signature of the - A shape-based approach that describes the leaf margin us-
image sent by the user is computed on-line and a large scale ing a multiscale triangular representation [6].
matching algorithm returns the most similar images [3]. A - A shape context based descriptor SC2 that represents the
KNN Classifier is then used to build a list of species. salient points of the leaf (essentially venation points) in the
context defined by the leaf boundary [5].
- The combination of the methods mentioned above by a
late fusion algorithm.
These techniques have shown their effectiveness for leaf im-
age retrieval. The SC2 descriptor was tested on the Image-
CLEF2011 leaf dataset. The shape-based approach using
multiscale triangles has been evaluated on four public leaf
datasets: Swedish, Flavia, ImageCLEF 2011 and 2012. It
has also shown good robustness to partial occlusion. The
identification results given by these descriptors can be found
in [5, 6].

3. CONCLUSION
In this paper, an Android application for leaf species iden-
tification has been presented. It is based on a set of leaf de-
scriptors that have given promising results on leaf datasets.
The accuracy of the identification makes this application
useful to amateur stakeholders as well as experts. Future
work aims to expand the knowledge database by including
(a) (b) leaf images from other species.

Figure 2: Screenshots of the Android application (a) Acknowledgements

Leaf image taken on a uniform background (b) Iden- This research has been conducted with the support of the
tification results. A ranked list of species is returned Agropolis Foundation through the Pl@ntNet project.

4. REFERENCES
2.1 Knowledge database
[1] N. Boujemaa, J. Fauqueur, M. Ferecatu, F. Fleuret,
To identify the species of a leaf image, a retrieval technique V. Gouet, B. Lesaux, and H. Sahbi. Ikona: Interactive
is used. We need to have a knowledge database, i.e. a specific and generic image retrieval. In International
collection of annotated leaf images where a plant species is workshop on Multimedia Content-Based Indexing and
associated with each leaf image. This database will be used Retrieval (MMCBIR), 2001.
to find the most similar images to a query image. To do so,
[2] H. Goëau, P. Bonnet, A. Joly, I. Yahiaoui,
we use a subset of the training set of ImageCLEF 2012 plant
D. Barthélémy, N. Boujemaa, and J.-F. Molino. The
identification task [2] freely available2 : the training sets of
IMAGECLEF 2012 Plant identification Task. In CLEF
scans and scan-like images are merged into one to form our
2012, Rome, Italy, Sept. 2012.
knowledge database. It contains 6698 images belonging to
[3] A. Joly and O. Buisson. Random maximum margin
122 species. Thus, our Android application make it possible
hashing. In CVPR, pages 873–880, 2011.
to identify leaves from 122 different species.
[4] N. Kumar, P. N. Belhumeur, A. Biswas, D. W. Jacobs,
2.2 Architecture W. J. Kress, I. C. Lopez, and J. V. B. Soares. Leafsnap:
A computer vision system for automatic plant species
The linkage between the GUI on the mobile device and
identification. In 12th European Conference on
the identification system on the second server is done by a
Computer Vision (ECCV 2012), pages 502–516,
RESTful java web service which offers an attractive scalable
Florence, Italy, Oct. 2012.
computing architecture. In our case, our RESTful identifi-
[5] S. Mouine, I. Yahiaoui, and A. Verroust-Blondet.
cation web service can be either a server or a client. It plays
Advanced shape context for plant species identification
the role of a server when it receives the leaf image taken by
using leaf image retrieval. In Proceedings of the 2nd
the user. It is a client when it interfaces the identification
ACM International Conference on Multimedia
server using the selected descriptor. The data exchange be-
Retrieval, pages 49:1–49:8, 2012.
tween the web service and the identification server is ensured
via sockets. Besides the abstraction of the architecture, the [6] S. Mouine, I. Yahiaoui, and A. Verroust-Blondet. A
web service allows simple communication between the An- shape-based approach for leaf classification using a
droid device and our identification system. multiscale triangular representation. In Proceedings of
the 3rd ACM International Conference on Multimedia
2 Retrieval, 2013.
http://imedia-ftp.inria.fr:8080/imageclef2012/ImageCLE
F2012PlantIdentificationTaskFinalPackage.zip

310