2020 21st IEEE International Conference on Mobile Data Management (MDM)

Counting People by Using Convolutional Neural

Network and A PIR Array
Peng-Rong Tsou Cheng-En Wu Yen-Ru Chen Yun-Ting Ho
Dept. EE of NCHU Dept. EE of NCHU Dept. EE of NCHU Dept. EE of NCHU
Taichung, Taiwan R.O.C. Taichung, Taiwan R.O.C. Taichung, Taiwan R.O.C. Taichung, Taiwan R.O.C.
apollosun1997@gmail.com a886911464747@gmail.com joe860519@gmail.com timothy.ho584@gamil.com

Jun-Kai Chang Hsiao-Ping Tsai

Dept. EE of NCHU Dept. EE of NCHU
Taichung, Taiwan R.O.C. Taichung, Taiwan R.O.C.
a22197518@gmail.com hptsai@nchu.edu.tw

Abstract—Counting the number of people is a common and the information about which rooms are occupied and also
basic computing operation in many applications. Most of the the number of people inside is available as a skyscraper fire
people counting techniques need a sensing device like camera and happens, the information can support the decision making
apply image processing methods to track pedestrians. However,
counting people with cameras in private places raises a lot of about which room to cut in next to improve rescue perfor-
security and privacy issues. The passive infra-red sensor (PIR) mance. Other examples are using people counting techniques
can detect the body temperature of the infrared and thus provides for safety assistance for elderly care, flow control for activities,
another promising solution. Although a single PIR can easily exhibition, customer understanding at museum or shopping
identify the passing situations (i.e., in or out) of a single person, malls, etc.
the signals of a single PIR is not sufficient to identify the
complex situations of multiple people. In the paper, we design a Nowadays, many solutions with difference sensors and tech-
people counting device with a PIR array to detect the passing niques have be proposed for people counting, such as thermal
situations and generate data records with higher discriminability. image counter, stereo camera counting, WiFi counting, and IoT
In addition, we apply the machine learning classification methods sense people counting with radar or infrared beams [9] [10] [8]
including the CNN, the RBM+LR, Decision Tree, and Naive- [11] [12] [13] [14]. Most of the people counting techniques use
Bayes on the collected data records to identify the passing
situations. To validate our design, we conduct experiments to image processing methods to estimate the number of people.
study the feasibility and classification performance and explore They need at least one camera as the sensor to track individual
the impact factors. The experimental results show that the CNN people and may also require some light source such as IR in
outperforms the other and achieves the best accuracy, i.e., about stringent environments. However, using cameras to monitor
92%. Also, the results show that the captured data records of people in public places could violate people’s privacy. Even
the PIR array contain sufficient characteristics for identifying
complex passing situations and the configuration of the PIR array worse, counting people with cameras in private places raises
including the sensor direction and the field of view (FOV) of a lot of security and privacy issues. As for the WiFi counting,
a PIR modified by the metal tape can significantly impact the although it is not as aggressive as the image-based approaches,
discriminability of the collected data. the RF based approaches rely on specialized sensors and
Index Terms—- require users to carry RF devices. In addition, it is more
people counting, passive infrared, PIR array, machine learn-
ing, CNN, RBM
suitable for counting the crowd in a zone but not passing an
entrance. On the other hand, the passive infrared sensor (PIR)is
a kind of infrared and is relatively cheap. It can detect the body
I. I NTRODUCTION
temperature of the infrared with low power consumption. It is
Nowadays, the IoT applications of smart home, intelligent widely utilized to sense the temperature changes to detect the
city, and intelligent transportation have being going deep into movement of a test person and is suitable for the applications
in our daily life, where smart sensors are widely deployed of people counting. Motivated by the fire safety of a smart
to collect data that are further analysed for customer under- building, we study the problem of using PIRs to monitor
standing, business decision making, elderly care, life quality the people passing the entrance of a room and estimate the
improvement and public safety. number of people inside. Note that there are already papers
Among them, people counting is a common and basic that use PIRs to detect the passing of a single person or
computing operation of IoT applications where sensing devices his passing direction [3] [5] [7]. However, the situations that
are used to count the number of people traversing or passing multiple users passing simultaneously or consecutively is less
a certain passage or entrance. People counting has many addressed. Since a single PIR can only identify simple single-
important applications. Taking public safety as an example, person passing in/out behaviours, we propose the use of a

2375-0324/20/$31.00 ©2020 IEEE 342

DOI 10.1109/MDM48529.2020.00075

Authorized licensed use limited to: Cornell University Library. Downloaded on August 21,2020 at 17:33:42 UTC from IEEE Xplore. Restrictions apply.
PIR array to periodically generate a set of more differentiable B. Related Work
PIR analogue output waveforms for facilitate the classification For tracking users in an office space, F. Wahl et al. [3]
of multi-people passing behaviours. In addition, we adopt proposed to set a pair of PIRs at individual entrances. The
the machine learning techniques including as Decision Tree, PIR pairs are used to detect the entry and exit of a pedestrian
RBM+Logistic Regression, and the CNN to train the classifica- as he enter a room and leave the room, and then Markov
tion models. To study the performance of the proposed system, probability model is used to learn the transition probability in
we implement the system with the Python Django and conduct user’s trajectories. Finally, the signals of the PIR pairs together
several experiments. The experimental results show that the with the Markov model are used to estimate the number of
CNN outperforms the other classifiers with a good accuracy people in each room. However, this paper doesn’t address the
above 90%. In addition, the results also show that using the situations where multiple people walk abreast or follow each
PIR array can help improve the classification accuracy and other closely to pass an entrance.
the proposed system can achieve good performance for people In [4], S. Narayana et al. found that a PIR sensor’s analogue
counting. signal are of significant difference under different movement
modes. Fig. 2 and Fig. 3 show that the curves of a PIR can
II. P RELIMINARY indicate the speed, distance, relative direction of movement of
passing users. However, the complex situations that multiple
A. Passive Infra-red Sensor (PIR) people walk abreast or follow closely while passing a PIR
is less addressed. Since the amplitude of the PIR sensor is
Fig.1 shows the exterior of a commonly seen passive triggered to it maximal value easily as a passing user is close
infrared sensor (PIR), which is a high-sensitivity, low-cost, to the PIR, so that it is difficult to distinguish the case of a
low-power sensor. It is used to receive infrared rays emitted by single user or multiple ones. This is the work that inspires our
the human body and to detect human movement. The infrared idea. Based on the characteristics of PIRs, we may infer the
sensor in the PIR has a coating (band pass filter), which can movement behaviour of a passing user easily. If we can set
block most of the infrared rays, but the infrared rays with tem- up multiple PIRs with each targeting a specified monitoring
perature close to 36.5 degrees (the wavelength is about 10 µm) area, the differences of their measure curve could help classify
can pass the coating. On top of that, a piece of filter (Fresnel more complex situations.
lens) will filter out 30% of infrared rays, finally, infrared rays
with wavelength about 7-10 µm will enter the infrared sensor
in the PIR. Note that since the infrared radiation emitted by the
human body is very weak, the tolerance of signals received by
the infrared sensor cannot be ignore. Fresnel is characterized
by a short focal length, and uses less material and smaller
volume than traditional lenses. In addition, the lens were
divided into multiple concentric circles to achieve the optical
focusing effect. Therefore, through the Fresnel lens, PIR can
sense the infrared more accurately. Because the wavelength Fig. 2. PIR’s analogue signals under the situation of different speeds [4]
of infrared rays emitted by the human body is about 9-10
µm, it is suitable for human motion detection. Because of
its low power consumption and sensing angles of up to 100
degrees and sensing distances in the range of 3 meters to 7
meters, PIR is often used in energy-saving and alarm systems.
Since the analogue signal of the infrared sensor are converted
into digital signal and controlled the MCU e.g., BISS0001.
Upon the analogue signal is higher than a threshold, the digital
output is triggered high for a specified delay time. The digital
output signals are frequently used in the autonomous control Fig. 3. PIR’s analogue signals under the situation of different movement
of smart appliances for energy saving [1] [2]. directions [4].

In [5], Piero Zappi et al. proposed to stick metal tape on

the Fresnel lens so as to change the field of view (FOV) of a
PIR. And J. Yun and J. Woo [7] proposed to arrange a PIR
array of four on the ceiling to detect the precise movement
direction of a pedestrian. They compare the classification
performance of classical machine learning methods and a
Fig. 1. A Passive Infrared Sensor (PIR). deep learning one. The results show that all of the leaning
algorithms can detect the movement direction precisely but

343

Authorized licensed use limited to: Cornell University Library. Downloaded on August 21,2020 at 17:33:42 UTC from IEEE Xplore. Restrictions apply.
the deep learning algorithm requires fewer training data and
provides better stability with unknown subject. However, this
paper only addresses the condition with a single pass-by. The
performance of the case that multiple pedestrians passing still
needs further study.

III. PROPOSED METHOD

A. Data Collection
To count the number of people in a room, our idea is
to monitor each entrance and identify the passing situation
of pass-bys. For a room with multiple entrances, it is thus
necessary to monitor each to keep the statistics correct. For the
sake of description, we focus on the case of a single entrance
because it can be easily extended for the case of multiple
ones. To monitor an entrance, we designed the people counting
device that mainly comprises a PIR array to sense pass-bys
and based on the detected the signals to identify the number
Fig. 5. Deployment of the people counting device.
of people who entering or exiting the room. As shown in Fig
5, the people counting device comprises two shelves that are
hanged on the wall, above the entrance door, measured up from horizontal concatenation format is composed of 1800×16 one-
the floor about 2.37m, inside and outside the room. Each of dimensional integer values, made by concatenating the integer-
the two shelve carries an Arduino Mega board and a PIR array lists of individual PIRs into one string. A record in vertical list
of eight PIRs to sense passbys periodically. The signal data format is a two-dimensional integer array, where each column
of the PIR array are collected for a period and then classified indicates the signals of a PIR in 6 sec. while a record in color
to identify the passing situation, based on which the number image format is a color image with 3 × 224 × 224 resolution,
of people is updated. Besides, the device also includes two made by overlapping of the curves of the sixteen PIRs. In
ultra sonic sensor (HC-SR04), each is on one side of door, addition, the data records are normalized by using the min-
measured up from the floor about 1 m and distanced from the max scaling before the training step as well as the classification
door by about 1.7m. They are utilized to trigger the PIR array step.
to start sensing immediately once it detects passbys. To make
the sensing data more discriminative, we arrange them as a
rectangle grid, as shown in Fig 5(b) and stick metallic tape on
the PIRs to limit the overlapping range of their receptive fields
as Fig 4. Once the PIR array is triggered, the device starts to
detect signals at a sampling rate of 30 Hz for a period of 5
second periodically until the signals of all PIRs turn down to
under a minimal threshold. Fig.7 shows an example of the data
collected by the PIR array.

Fig. 6. The passing situation of a single user entering.

Fig. 4. PIR sensing range after sticking metallic tape

B. Classification Methods
The sensing data are collected and recorded, based on whom Instead of tracking individuals to know whether he/she is
we identify the passing situation. In this work, we compile the leaving or entering the room, we target to figure out the
data records in three different formats, i.e., the vertical list for- number of people entering or leaving in a period based on
mat, color image format and horizontal concatenation format, the data collected. Note that the identification of the pass-
as shown in Fig. 8, Fig 10and Fig 9 respectively. A record in ing situations is actually a classification problem. Since the

344

Authorized licensed use limited to: Cornell University Library. Downloaded on August 21,2020 at 17:33:42 UTC from IEEE Xplore. Restrictions apply.
 passing situations are various and depend on the number of
people and the individuals’ moving speeds as well as moving
directions, we adopt several classification methods including
the CNN, the pipleline of the RBM and logistic regression
(RBMLR), decision tree (DT), Naı̈ve-Bayes (NB) to study the
discrimination and classification performance of using a PIR
array.
1) BernoulliRBM+Logistic Regression : The Bernoulli Re-
stricted Boltzmann Machines (RBM) are artificial neural net-
works that use a layer of hidden variables to model the
distribution over its input variables and are frequently used
in non-linear feature extraction [15] [16]. As the curves of a
Fig. 7. An example of a data record that contains the signals of the sixteen data record shown in Fig 8 are non-linear, we build a RBM
sensors in the passing situation of a single user.
classifier by pipelining a RBM and a Logistic Regression
following behind.
2) CNN: A convolutional neural network (CNN) is a neural
network that has one or more convolutional layers as well
as pooling layers. It has great capability in capturing the
correlation among pixels and exploring the features of various
granularities. Its success in many image processing, classi-
fication, segmentation applications has attracted tremendous
attention recently. Here, we use a CNN to extract the features
among the curves of the PIR array for classifying the passing
situations. Two CNN are trained. The first CNN takes a image
record as input and output the category of the passing situation.
Fig. 8. A record of integers in the horizontal concatenation format (Passing Fig.11 illustrate the architecture of the convolution neural
Situation: A single User Entering). network, where the input is a image with the resolution of
3 × 244 × 244. Fig.12 illustrates the 2nd one that takes the
matrix as shown in Fig.9 as input.

Fig. 9. A record of integers in the vertical list format (Passing Situation: A

single User Entering).

Fig. 11. Illustration of the CNN architecture taking the data records in color
image format as its input.

Fig. 12. Illustration of the 2nd CNN architecture taking the data records in
vertical list format as its input.
Fig. 10. A record in the color image format

345

Authorized licensed use limited to: Cornell University Library. Downloaded on August 21,2020 at 17:33:42 UTC from IEEE Xplore. Restrictions apply.
 IV. E XPERIMENTAL R ESULTS
To test the discrimination ability of using the PIR array
and to study the classification performance of the proposed
system, we build the dataset by using the people counting
device. Specifically, we classify the passing situations into
22 categories (as shown in Table I). To generate the datasets
for training the classifiers, seven students participate in build-
ing the training dataset. A specified number of students are
randomly chosen to pass the entrance and about twenty data
records are collected for each category. A data record contain-
ing the signals of all of the sixteen PIRs is generated by the
people counting device with the sampling rate of 30Hz for a
period of roughly around 6 seconds. The collected data is then
compiled into the records in both raw-data format and color
image format. To classify the data records, we adopt the well- Fig. 13. Performance of the classification methods with the PIR array
known classification methods including the CNN, the pipleline
of the RBM and logistic regression (RBMLR), decision tree
(DT), Naı̈ve-Bayes (NB). Several experiments are conducted
to study the classification accuracy of the classifiers on all
categories and individual categories, and also the impact of
the configuration of the PIR array. To study the Configuration
of the PIR array on the accuracy, we setup 4 configurations.
The first configuration (denoted by 16 in the figures) contains
the signals of all of the sixteen PIRs. The 2nd and the third
contain those of two set of eight PIRs, i.e., 8 1 and 8 2 contain
Pirs 1, 2, 7, 8, 9, 10, 15, and 16 and PIRs 1, 2, 3, 4, 13, 14,
15 and 16 respectively. The fourth configuration (denoted by
4) contain 4 PIRS, i.e., 1, 2, 15 and 16.
As shown in Fig. 13, we can see that the CNN outperforms
the others for the four configurations. The CNN with color Fig. 14. Performance of each situations with different method
image as its input achieves the best accuracy about 92.75%
as all PIRs are used. As the number of PIRs is reduced, the
accuracy of all methods tends to decline, which indicates that
more PIRs indeed captures more information that is helpful to TABLE I
identify the passing situations. Also, the accuracy of the CNN C ATEGORIES OF PASSING S ITUATIONS
with configurations 8 2 is better than that with 8 2 that is the
Label Description
signals of individual PIRs are of importance regarding to the 1 One In
discrimination and classification accuracy. The results lead to 2 One Out
an interesting problem of finding the best configuration for 3 Two In (One after the other)
the people counting problem. The problem is left as a future 4 Two Out (One after the other)
5 Two In (Side by side)
work. 6 Two Out (Side by side)
Moreover, Fig. 14 shows the accuracy of the classification 7 Three In(one after another)
methods on individual categories. The results show that some 8 Three Out(one after another)
9 Three In (Two: Side by side/ One: one after another)
of the categories are more difficult of to classify. According
10 Three Out (Two: Side by side/ One: one after another)
to our analysis, we found part of the data records of of 11 Four In
these categories are of bad quality, i.e, the curves are vague 12 Four out
and damaged. The reasons of the bad records need further 13 Four In (Two: Side by side/ Two: side by side)
14 Four Out (Two: Side by side/ Two: side by side)
investigation. Also, we compare the CNN with two image 15 Five (Two In, Three Out)
formats. The results show that the CNN with color image 16 Five (Three In, Two Out)
slightly outperforms that that with vertical-list matrix. 17 Five In(Two: Side by side/Three: one after another)
18 Five Out(Two: Side by side/Three: one after another)
V. C ONCLUSION 19 Two (One In, One Out)
20 Three (One In, Two Out)
In this paper, we devised a people counting system based 21 Three (Two In, One Out)
on passive infra-red sensors to generate the training and 22 Four (Two In, Two Out)
testing datasets. The preliminary study results show that the
CNN achieves better classification accuracy and robustness. In

346

Authorized licensed use limited to: Cornell University Library. Downloaded on August 21,2020 at 17:33:42 UTC from IEEE Xplore. Restrictions apply.
addition, the CNN classifier taking image records as the inputs
gets a slightly higher accuracy than the CNN taking input
records in the vertical list format. The experimental results
also show that the impact of the number of categories on the
classification accuracy is not obvious. Last, we discover that
the configuration of the PIR array affect the discrimination
ability of the collected data significantly. The future work
includes more study to figure out the best configuration of
the PIR array, building an on-line people counting system to
study the practicality and comparing it with the image-based
people counting methods.
VI. ACKNOWLEDGEMENT
The work was supported in part by the MOST of Taiwan,
R.O.C., under Contracts 108-2218-E-005-012-.
R EFERENCES
[1] Z. Han, R.X. Gao and Z.Fan, ”Occupancy and indoor environment
quality sensing for smart buildings,” In Proc. of the 5th European DSP
Education and Research Conference (EDERC’12), pp. 882–887, 2012.
[2] C.-H. Tsai, Y.-W. Bai, C.-A. Chu, C.-Y. Chung, M.-B. Lin, ”PIR-sensor-
based lighting device with ultra-low standby power consumption,” IEEE
Trans. Consum. Electron, No. 57, pp. 1157–1164, 2011.
[3] F. Wahl, M. Milenkovic, and O. Amft, ”A distributed PIR-based ap-
proach for estimating people count in office environments,” in Proc.
IEEE 15th Int. Conf. CSE, pp. 640-647, 2012.
[4] S. Narayana, R. V. Prasad, V. S. Rao, T. V. Prabhakar, S. S. Kowshik,
and M. S. Iyer. ”PIR sensors: Characterization and novel localization
technique,” In IPSN, pp.142-153, 2015.
[5] P. Zappi, E. Farella, and L. Benini, ”Tracking Motion Direction and
Distance With Pyroelectric IR Sensors,” IEEE SENSORS J., vol. 10,
no. 9, pp.1486-1494, 2010.
[6] Is-215t Datasheet, Honeywell Security and
Custom Electronic., 2008.[Online]. Available:
http://library.ademconet.com/MWT/fs1/9/4434.pdf
[7] J. Yun and J. Woo, ”A Comparative Analysis of Deep Learning
and Machine Learning on Detecting Movement Directions using PIR
Sensors,” in IEEE Internet of Things Journal.
[8] J. Yun and S.-S. Lee, ”Human Movement Detection and Identification
Using Pyroelectric Infrared Sensors,” Sensors, no. 14, pp. 8057-8081,
2014.
[9] V-Count 3D Alpha+, https://v-count.com/solutions/people-counting/
[10] VIVOTEK, https://www.vivotek.com/solutions/people-counting
[11] I. Amin, A.J. Taylor, F. Junejo,A. Al-Habaibeh, and R. Parkin, ”Au-
tomated people-counting by using low-resolution infrared and visual
cameras,” Measurement, No. 41, pp.589-599, 2008.
[12] Traf-Sys Gazelle People Counter, https://www.trafsys.com/traf-sys-
gazelle-people-counter/
[13] H. Zou, Y. Zhou, J. Yang, W. Gu, L. Xie and C. Spanos, ”FreeCount:
Device-Free Crowd Counting with Commodity WiFi,” IEEE Global
Communications Conference (GLOBECOM 2017), pp. 1-6, 2017.
[14] IoT Sense People Counting, https://www.singtel.com/business/products-
services/iot/products/people-counting
[15] H. Larochelle and Y. Bengio, ”Classification using discriminative re-
stricted Boltzmann machines,” Proc. of the 25th int’l conf. on Machine
learning(ICML), p. 536, 2008.
[16] H. Larochelle, M. Mandel, R. Pascanu and Y. Bengio, “Learning
Algorithms for the Classification Restricted Boltzmann Machine,” J. of
Machine Learning Research, vol. 13, no. 22, pp. 643-669, 2012.

347

Authorized licensed use limited to: Cornell University Library. Downloaded on August 21,2020 at 17:33:42 UTC from IEEE Xplore. Restrictions apply.