2018 Joint 10th International Conference on Soft Computing and Intelligent Systems and 19th International Symposium

on Advanced Intelligent Systems

Object Recognition Based Robust Mapping for the

Autonomous Mobile Robot under Dynamic
Environment
1st Bin Zhang 2nd Masahide Kaneko 3rd Hunok Lim
dept. of Mechanical Engineering dept. of Mechanical Engineering and Intelligent Systems dept. of Mechanical Engineering
Kanagawa University The University of Electro-Communications Kanagawa University
Yokohama, Japan Tokyo, Japan Yokohama, Japan
zhangbin@kanagawa-u.ac.jp kaneko@radish.ee.uec.ac.jp holim@kanagawa-u.ac.jp

Abstract—Simultaneous Localization and Mapping (SLAM) is map. However, there may be many matched places in the
an essential technique for autonomous mobile robot. 2D mapping map. Especially under familiar environment like the corridor,
has been greatly developed recently and widely used in navigation the point clouds may be matched everywhere. Secondly, the
tasks since the it is easier to be generated and has fewer
computation amount than 3D mapping. However, 2D information
generated map based on laser scanner sensor only contains
is not enough to reflect real 3D environment. This paper proposes objects information from the plane where the sensor is located.
a 2D mapping method by using immobile area grid map method. It is hard to provide measurement in 3D environment where
3D objects are detected by 3D sensor and the object recognition objects are in different heights. For example, desks and chairs
results are reflected to the 2D map by a probability model, in are normal in indoor environment, but a 2D scanner sensor
which the object recognition results are used to calculate the
degree of using observed information to generate the map. In
may only detect their legs as objects, which turn to four
this way, 3D objects are reflected to the map and potential points on the generated map. Collision may happen during
mobile objects are deleted from the map. The effectiveness of our autonomous navigation for the robot. This problems can be
proposed method is proven by conducting experiments under an solved by using multiple 2D sensors or 3D sensors to detect the
indoor environment, where 3D objects (e.g. desks and chairs) and environment [7]–[9]. Unfortunately, the accuracy of multiple
human beings are all contained. With the proposed method, the
generated 2D map shows the areas of 3D objects well and deletes
2D sensor is still limited by the number of sensors, and the
the potential moving objects (static human beings) correctly. cost of multiple 2D sensors or 3D sensors with wide view
Index Terms—object recognition, SLAM, dynamic environ- angle are usually high. Recently, 3D sensors with low cost
ment, 2D mapping, 3D information integration like Microsoft Kinect sensor have been released but the view
angle is still narrow and the information is insufficient for
I. I NTRODUCTION robot localization. Research in [10] tried to rotate a 2D scanner
In recent years, SLAM has become one of the most fasci- sensor by a motor to detect 3D information, and the real
nating research fields [1], [2]. Localization and mapping are time performance is limited. Thirdly, most of conventional
two essential information for any autonomous mobile robot researches [12], [15], [20], [23] tried to remove the moving
working in human living environment. As the basic technique objects from the generated map, but it takes a lot time to
for path planning and activity planning, the generated map remove potential moving objects that are static at the moment
can also be used for robot navigation later in the same but actually able to move in the future by these methods.
environment [3]. Laser scanners have been widely used for Ito et al. [16] tried to deal with this problem by using the
SLAM [4], [5] as they usually have high accuracy to detect immobile area grid map method, in which a probability model
distances in front and have a wide view angle. For example, of describing the degree of the observed object to generate the
Wang et al. [4] successfully realized SLAM under crowed map is added. The static human beings are removed from the
environment and generated a map without moving objects. map quickly when they move. However, the object recognition
However, conventional methods still have a few problems to results are limited in their method by only using laser sensors.
be solved. Firstly, the accuracy of localization needs to be In this paper, an object recognition based robust mapping
improved. The odometry information gotten from the mobile method for the autonomous mobile robot under dynamic
robot can be used for localization. Whereas, its accumulative environment is proposed. SLAM is realized by integrating
error caused by the differences between the two wheels of the the information of a laser scanner sensor called Laser Range
robot, rode condition or sensor noise will be fatal with time Finder (LRF) and a Kinect 3D sensor. The LRF sensor is
going on. Iterative closest point (ICP) [6] method is usually used to generate the basic 2D map to take advantage of its
used for localization in SLAM process by matching the point wide sensing range, and the Kinect sensor is used to recognize
cloud gotten from the laser sensor with that in the global different kinds of 3D objects and reflect them to the basic

978-1-5386-2633-7/18/$31.00 ©2018 IEEE 1336

DOI 10.1109/SCIS-ISIS.2018.00209
2D map. In this way, the generated 2D map contains 3D
information so that the environment can be better reflected
to the 2D map. The idea of sensor placement is familiar with
Lin et al. [11], but different from their work, the generated
map is more robust in removing potential moving objects.
The immobile area grid map based SLAM method is used
to represent the map by an immobile probability for each
grid. The map update parameter is set adaptively from the
certainty of identification result. Potential moving objects are
not mapped if the identification results are corresponding
to moving objects. The experimental results show that the
proposed method is more robust to generate a 2D method
containing 3D information while deleting potential mobile
objects from the map. Fig. 1. Working process of the iterative closest point (ICP) method.

This paper is organized as follows. Section I shows the

introduction of the paper. Section II explains the method of
improving the accuracy of robot self-localization. In section
III, the idea of integrating 2D and 3D information is proposed.
Section IV explains the probabilistic model of mapping under
dynamic environment, by using immobile Area Gird Map
Based SLAM The experimental results and analyses are shown
in section V. Finally, section VI presents the conclusions and
future work.

II. ROBOT S ELF -L OCALIZATION

To guide the users correctly, the robot needs to locate itself

accurately. The ICP method can be used for localization [6].
Its working process is shown in Fig. 1. The newly detected
distance data by the laser sensor is matched with the model Fig. 2. Matching in multiple places when using the iterative closest point
(ICP) method.
data, as shown in Fig. 1 (a). These two types of data should
be similar throughout continuous frames, and the location
with the closest match can be determined by minimizing
the distance between corresponding points, as shown in Fig. III. I NTEGRATION OF 2D AND 3D I NFORMATION
1 (b). The calculated positional relation shows the motion To understand the environment around the robot, distance
of the robot, and the localization of the robot can also be sensors are usually used for SLAM. In the proposed system,
calculated as shown in Fig. 1 (c). However, this method has the LRF sensor is used for getting the 2D distance information,
the problem that there may be multiple matched places with and Kinect sensor is used for getting the 3D information
the map. For example, in the corridor, the model data are around the robot. LRF sensor is chosen as the UTM-30LX
similar with the new detected sensor and matched with the made by Hokuyo Co.. It can sense the range of 270◦ in front
detected distance data everywhere, as shown in Fig. 2. The of the robot with the maximum distance of 10m. The degree
odometry information received from the mobile robot can step is 0.25◦ . The LRF sensor is shown in Fig. 3 (a) and
also be used for localization. Notably, the accumulative error its sensing range is shown in Fig. 3 (b). Kinect sensor can
caused by the differences between the two wheels of the get the color and depth information at the same time in front
robot, road condition, or sensor noise will be fatal as time of the sensor. It is made by Microsoft Co. and the Kinect
goes on. Thus, these two types of information are integrated V2 is chosen for the experiment. The sensing range of the
by using the Monte Carlo Markov Chain (MCMC) [17] Kinect sensor is 70◦ in horizontal and 60◦ in vertical with
particle filter method. During the prediction step, the motion the maximum distance of 8m. The color image resolution of
information obtained from odometry is used to predict the the Kinect sensor is 1920*1080 pixels. The Kinect sensor is
position of the robot, mixed with Gaussian noise in order to shown in Fig. 4 (a) and its sensing range is shown in Fig. 4
use it as the motion model. During the evaluation step, the (b).
prediction results were evaluated and re-weighted according to LRF has a large sensing range but it can only detect the
the matching rates between the detected distance data and the distance information in a horizontal plane. The objects in front
model data, same with the ICP method. The robot position then of the robot can be detected well on the sensing plane. For the
can be calculated as the re-weighted mean of all prediction scene shown in Fig. 5 (a), the object detection result is shown
results. in Fig. 5 (b). Kinect can detect 3D distance information and

1337
 270

(a) A scene for object detection (b) Detection result by Kinect

(a) LRF sensor (b) Sensing range of LRF Fig. 6. Environment sensing result by Kinect.

Fig. 3. Introduction of LRF sensor.

0.5m 8m
(a) Kinect sensor (b) Sensing range of Kinect

Fig. 7. Environment sensing result after integrating two kinds of information.

Fig. 4. Introduction of Kinect sensor.

IV. I MMOBILE A REA G IRD M AP BASED SLAM

color information at the same time, but its sensing range is The occupancy grid map used in SLAM is a conventional
limited. Thus, these two kinds of information are integrated method which makes a map by an occupancy probability in
for generating a robust map, which can reflect the objects well each grid. When the occupancy probability is high, it means
in the space. The distance information of Kinect sensor (3D that the probability of existing an object for the grid is high.
information) is projected to the ground plane to integrate with When the occupancy probability is low, it means that the
that gotten from LRF sensor (2D information). Meanwhile, probability of existing an object for the grid is low, and the
the human detection can be easily processed by Kinect sensor probability of noting for the grid is high. The robot keeps
[18]. From the distance information of Kinect sensor, the sensing the environment around and updating the occupancy
planes can be detected by using Point Cloud Library (PCL) probability in each grid. In this way, the areas where objects
[19]. After the planes are detected, the ceiling plane can exist or not can be expressed by probability. However, this
be separated as the highest horizontal plane over 2.5m, and method has the problem that even the moving objects would
the wall planes can be recognized as the vertical planes that be registered on the map if they are observed in a high
connected to the ceiling plane. For the scene shown in Fig. 6 probability. As the robust environment map should not contain
(a), the human detection result is shown in Fig. 6 (b). After moving objects and potential moving objects (like a human
combining these two kinds of information on the 2D map, the being keeps still), other methods which can delete the moving
result is shown in Fig. 7. The human beings, walls and 3D objects are used for the researches [20]- [23]. To solve the
obstacles like desks are well reflected on the map. problem, the immobile grid map [16] is applied to generate
the map. The environment is divided into girds first, same
with the occupancy grid map method. However, the probability
for each grid is calculated by the probability of suitable
immobile area instead of the occupancy probability. When the
occupancy grid map is used, the updating weigh will increase
in the exponential way if the object existing probability is
high. Moreover, the observed objects might be moving objects
under the dynamic environment. The map will be generated
correctly only when the objects are still objects as the moving
(a) A scene for object detection (b) Detection result by LRF objects should not be shown in the environment map. For the
immobile area grid map method, the map updating weight can
Fig. 5. Environment sensing result by LRF. be adjusted adaptively even if the object existing probability
is high, by using a changeable parameter that controlled by

1338
the object recognition results. are used for updating the map by a big updating coefficient
The event that one grid is immobile area is set as I, and the and the objects can be quickly reflected on the map.
event that some objects are observed on the same grid is set The value of λ is set based on the likelihood of the object
as O. The immobile area probability Pt (I) can be expressed recognition results. The likelihood of the object recognition
as Equation (1) at time t. results expresses how much the recognition result can be
trusted. When the likelihood is low, the objects are better to be
Pt (I) ∝ Pt (O) · Pt−1 (I) (1)
thought as unknown objects, and the value of λ is around 2.
Here, Pt (O) is the object existing probability. From Equation With the increasing of the likelihood, the value of λ increases
(1), the probability Pt (I) that one gird is immobile area at from 2 if the objects are recognized as still objects, and the
time t can be calculated from the observed object existing value of λ decreases from 2 if the objects are recognized as
probability Pt (O) at time t and the immobile area probability moving objects.
Pt−1 for the same grid at time t − 1. The event I can exist
depending on the event O can be observed. V. G LOBAL M APPING OF THE W ORK E NVIRONMENT
Meanwhile, the event that one grid is not immobile area is The map of the work environment is generated in advance.
set as I, and the event that any objects cannot be observed on The mobile platform is chosen as the PIONEER 3-DX made
the same grid is set as O. The non-immobile-area probability by MobileRobots Co.. The height of the LRF sensor is set as
Pt (I) can be expressed as Equation (2) at time t. 32cm and the Kinect sensor is set as 100cm above the ground
on the robot. The Laser Range Finder (LRF) sensor is used
Pt (I) ∝ Pt (O) · Pt−1 (I) + Pt (O) · Pt−1 (I) (2)
for getting the 2D distance information and the Kinect sensor
From Equation (2), the event I can exist depending on the is used for getting the 3D distance information.
event O or the event O can be observed. It means that the sum Firstly, the experiment is conducted in a room where kinds
of the cases that the event O that noting is observed happens of objects and a person are inside. The person moved from still
and the event O that some objects are observed as moving status during the experiment. The scene is shown in Fig. 8.
objects happens. From Equation (2), the following equation The generated map by the proposed method and conventional
can be gotten. method [11] are shown in Fig. 9 and Fig. 10 separately. The
Pt (I) ∝ Pt−1 (I) (3) conventional SLAM method is based on occupancy grid map,
which is most widely used by researchers since the open
From Equation (3), the event I exists only depending on a source is released already and possible to apply for kinds of
constant value. To divide the Equation (1) by Equation (3), indoor environments. In Fig. 9 and Fig. 10, the black points
the Equation (4) can be gotten. show the immobile areas, the white points show the mobile
Pt (I) Pt−1 (I) areas, and the gray points show the unknown areas in the map.
∝ λ · Pt (O) · (4)
Pt (I) Pt−1 (I) By comparing the two results, the chairs and desks are detected
as some areas in the proposed method , but only detected as
Here, λ is the constant coefficient. The updating coefficient
some points in the conventional method. These results make
for immobile area probability is defined as β, Equation (5)
sure that the robot cannot go through in these areas so that
can be gotten.
collisions can be avoided. By our proposed method, the person
β = λ · Pt (O) (5)
can be detected well from the beginning even if he was static,
Here, λ is defined as the variable parameter that is controlled and these areas in the map are not updated as objects. On the
by the object recognition results. The observed objects are other hand, the conventional method can only recognize the
recognized as the immobile ares or moving objects first, and person after he started to move. Moreover, the chair that the
the results are used for setting the value of λ. Then we can person sat before moving can be also reflected to the map well
adjust the value of β by the value of λ. The potential moving by the proposed method.
objects can be deleted for updating the map by adjusting the The proposed method is used for generating the map of
value of λ. the lab environment where the guide robot will work. The
The robot senses the environment by its sensors, and the mapping result is shown in Fig. 11. We observe that the
observed objects are recognized first. The value of λ is humans are all deleted from the map, no matter they are static
adjusted depending on the recognition results. If the results are or moving. The areas of desks and chairs are shown as areas
unknown objects, the object is ambiguous and Pt (O) = 0.5. with objects, rather than some points (e.g. only the legs of
The immobile area probability should not be changed as the desks or chairs). Moreover, the mobile objects are deleted from
observed information is meaningless for updating the map. the map in a fast speed. This map is good enough for path
Thus, the value of λ is set as 2 to make sure the updating planning of mobile robots.
coefficient for immobile area probability β = 1. If the
recognition results are moving objects, the value of λ is set VI. C ONCLUSION
smaller than 2 so that the observed objects are not used for This paper proposed a robust 2D mapping method contain-
updating the map. If the recognition results are still objects, ing 3D object recognition results. 3D information are reflected
the value of λ is set bigger than 2 so that the observed objects to the basic 2D map to make the 2D map more accurate and

1339
 (a) (b) (c) (d)

(e) (f) (g) (h)

Fig. 8. The scene of the environment.

(a) (b) (c) (d)

(e) (f) (g) (h)

Fig. 9. Mapping results by the proposed method.

robust. For example, desks and chairs are reflected as four Future work will focus on recognizing more objects and
points on the map by using laser sensor, but the planes of improving the accuracy of object recognition result. The
the desks and chairs are recognized by the 3D sensor and attributes of the objects will also be reflected to the map to
reflected as areas in the 2D map. This helps the robot to make it more easy to use for path planning of autonomous
improve safely when moving around. The map is generated mobile robots. For example, the robot should not go through
and updated by using immobile area grid map. The 3D object the space under a desk, and should not stand by behind a door.
recognition results are used to calculate the degree of using In this way, the robots can move more naturally like human
the observed objects to update the map by probability. In this beings and the social acceptability of robot will be improved.
way, the potential mobile objects are quickly deleted from the
R EFERENCES
map. The effectiveness of the proposed method is proven by
experiments. This method has great performance under indoor [1] H. Durrant-Whyte and T. Bailey, “Simultaneous localisation andmapping
(SLAM): Part I,” IEEE Robotics and Automation Magazine, vol.13, no.
dynamic environment containing multiple objects (e.g. desk 2, pp. 99-110, 2006.
and chair) and potential mobile objects (e.g. humans who keep [2] T. Bailey and H. Durrant-Whyte, “Simultaneous localisation andmapping
static in the beginning). (SLAM): Part II,” IEEE Robotics and Automation Magazine, vol.13, no.
3, pp. 108-117, 2006.

1340
 (a) (b) (c) (d)

(e) (f) (g) (h)

Fig. 10. Mapping results by the conventional method.

of Information Science and Engineering 28, pp. 131-144, 2012.

[12] D. Wolf and G. Sukhatme, “Mobile robot simultaneous localization and
mapping in dynamic environments,” IEEE International Conference in
Robotics and Automation, vol.19, no.1, pp.53-65, 2005.
[13] D. Hahnel, W. Burgard, D. Fox and S. Thrun, “An efficient fastSLAM
algorithm for generating maps of large-scale cyclic environments from
raw laser range measurements,” IEEE/RSJ International Conference on
Robotics and Systems, vol. 1, pp. 206-221, 2003.
[14] S. Yang and C..Wang, “Feasibility grids for localization and mapping
in crowded urban scenes,” IEEE International Conference on Robotics
and Automation, pp. 2322-2328, 2011.
Fig. 11. Mapping results of the lab environment. [15] D. Holz, C. Lorken and H. Surmann, “Continuous 3D sensing for
navigation and SLAM in cluttered and dynamic environments,” 11th
International Conference on Information Fusion, pp. 1-7, 2008.
[16] A. Ito, K. Takahashi, and M. Kaneko, “Robust mapping for mobile robot
based on immobile area grid map considering potential moving objects,”
[3] Y. Liu, J. Dong and F. Sun, “An efficient navigation strategy for mobile
IEEJ Transactions on Electronics, Information and Systems, vol. 134,
robots with uncertainty estimation,” 7th World Congress on Intelligent
no. 2, pp. 192-204, 2014.
Control and Automation, pp. 5174-5179, 2008.
[17] R. Kurazume, H. Yamada, K. Murakami, Y. Iwashita and T. Hasegawa,
[4] C.-C. Wang, C.E. Thorpe, S. Thrun, M. Hebert, and H.F. Durrant-Whyte, “Target tracking using sir and mcmc particle filters by multiple cameras
“Simultaneous localization, mapping and moving object tracking,” I. J. and laser range finders,” Proc. 2008 IEEE/RSJ International Conference
Robotic Res., Vol.26, No.9, pp. 889-916, 2007. on Intelligent Robots and Systems (IROS08), pp. 3838-3844, 2008
[5] M. Liu, S. Huan and G. Dissanayake, “Feature based SLAM using [18] Munaro, Matteo, Filippo Basso, and Emanuele Menegatti, “Tracking
laser sensor data with maximized information usage,” IEEE International people within groups with RGB-D data,” 2012 IEEE/RSJ International
Conference on Robotics and Automation, pp. 1181-1186, 2011. Conference on Intelligent Robots and Systems. IEEE, 2012.
[6] P. Besl and N. McKay, “A method for registration of 3-D shapes,” IEEE [19] R. Rusu and S. Cousins, “3D is here: Point Cloud Library (PCL),” IEEE
trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. International Conference on Robotics and Automation, pp. 1-4, 2011.
239-256, 1992. [20] D. Hahnel, D. Schulz, andW. Burgard, “Mobile robot mapping in
[7] D. M. Cole and P. M. Newman, “Using laser range data for 3D SLAM populated environments,” Advanced Robotics, Vol.17, No.7, pp.579-598,
in outdoor environments,” IEEE International Conference on Robotics 2003
and Automation, pp. 1556-1563, 2006. [21] T.-D. Vu, J. Burlet, and O. Aycard, “Grid-based localization and lo-
[8] R. Halterman and M. Bruch, “Velodyne HDL-64E LIDAR for unmanned cal mapping with moving object detection and tracking,” Information
surface vehicle obstacle detection,” Proc. International Society for Op- Fusion, Vol.12, No.1, pp.58-69, 2011.
tical Engineering, vol. 7692, pp. 224-231, 2010. [22] N.C. Mitsou and C.S. Tzafestas, “Temporal occupancy grid for mo-
[9] J. Welle, D. Schulz, T. Bachran and A. Cremers, “Optimization tech- bile robot dynamic environment mapping,” Control Automation, 2007.
niques for laser-based 3D particle filter SLAM,” IEEE International MED’07. Mediterranean Conference on, pp.1-8, 2007.
Conference on Robotics and Automation, pp. 3525-3530, 2010. [23] S.-W. Yang and C.-C.Wang, “Feasibility grids for localization and
[10] J. Weingarten and R. Siegwart, “EKF-based 3D SLAM for structured mapping in crowded urban scenes,” 2011 IEEE International Conference
environment reconstruction,” IEEE/RSJ International Conference on on Robotics and Automation (ICRA), pp.2322-2328, 2011.
Intelligent Robots and Systems, pp. 3834-3839, 2005.
[11] K. Lin, C. Chang, A. Dopfer and C. Wang, “Mapping and localization in
3D environments using a 2D laser scanner and a stereo camera,” Journal

1341