[Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

Design and Analysis of a Novel Sidewalk

Following Visual Controller for an Autonomous
Wheelchair
Efgan UĞUR1, Tolgay KARA1, Abdulhafez ABDULHAFEZ2, Ismail Haj OSMAN1
1
Department of Electrical and Electronics Engineering, Gaziantep University, 27310, Turkey
2
Department of Software Engineering, Hasan Kalyoncu University, 27010, Turkey
efganugur@gantep.edu.tr
θ Deflection angle of the wheelchair
Abstract—This paper presents a study that focuses on  Derivative of wheelchair deflection angle
sidewalk following problem of an autonomous wheelchair. The φ Specified angle of Hough line parameter
main goal is to propose a solution to the urban mobility ζ Damping ratio
problem of people with walking disabilities. The study offers an ωn Natural frequency
efficient control system design for this task. A linearized TCP Transmission control protocol
wheelchair model is constructed and image-based visual Va Motor voltage as input of system
servoing is introduced to evaluate the performance of tracking x, y Cartesian coordinate axes
yellow tactile pavement on sidewalk with optimal control.
Reference trajectory sets are created using robust vanishing
I. INTRODUCTION
point for sidewalk following by employing the Hough Lines
method. These reference paths are tested with two control Walking does not assume a problem for the majority of
methods of Linear Quadratic Regulator (LQR) control and people, but people with walking disability have difficulty in
Pole Placement (PP) control. Both control methods are applied walking and this may affect their physical and emotional
through simulation on the autonomous wheelchair model, and
well-being significantly. Human beings are social creatures
efficacy of sidewalk following under these control methods is
discussed comparatively. Disturbance attenuation results of the and they need to interact with their family, friends, and other
given optimal control methods and simulation outputs prove people in their community. Also, people have to be mobile
the efficacy of the model and the designed control systems. for daily activities such as going to school or work, which
LQR control method proves to have better performance in may be partly or fully compromised by walking disabilities.
system response in comparison to PP control method. Researches show that difficulty in walking can lead to
increasing isolation, anxiety, and depression [1]. Power
Index Terms—assistive technology, control design, modeling,
optimal control, visual servoing.
wheelchairs are one of the most common assistance
solutions for people with walking disabilities, but power
NOMENCLATURE wheelchairs may comprise problems of ergonomics for
people with reflex disabilities of upper limb mobility
b Intercept of the lines
COG Center of gravity problems, as in using the joy-stick.
DC Direct current A recent statistical analysis by the Turkish Statistical
EMF Electromotive force Institute reveals that a considerable number of people are in
Fh Horizontal forces this situation, which justifies the necessity of and motivates
g Gravitation for this research on autonomous wheelchair control [2].
HDR High dynamic range Smart or autonomous wheelchair technologies are
IAE Integral absolute error
developed to help these people.
ITAE Integral time-weighted absolute error
Ip Body inertia There are many research and development studies on
Iw Wheel inertia power wheelchairs and smart wheelchairs in particular, and
ke back EMF assistive technologies for the disabled people [3]. SYSIASS
km Motor torque project [4], iChair [5], ATEKS [6], RADHAR [7], smart
l Distance from body’s COG wheelchair [8] are among the examples of autonomous
LQR Linear quadratic controller
wheelchairs that are currently available, but it should be
MSE Mean squared error
m Slope of the lines noted that they are multi-sensor systems with distributed
mp Body mass with load architecture of autonomous wheelchair design. In
mw Wheel mass comparison to aforementioned designs, we propose in this
PP Pole placement research that a camera is employed for visual sensing as the
PID Proportional-Integral-Derivative only sensor, for the fact that vision proves to be the most
Q and R Weighting matrices of LQR
versatile sensing technique in robotic application [9].
r Distance from the origin
rw Wheel radius Vision based motion control is a difficult task, and it may
x Displacement of the wheelchair take tremendous amount of effort to solve an autonomous
x Velocity of the wheelchair wheelchair’s motion control problem. In an important study
x0 Initial state vector on vision based control process, the autonomous corridor
xd Desired state vector following problem is solved using features like vanishing

Digital Object Identifier 10.4316/AECE.2024.01001

3
1582-7445 © 2024 AECE
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

point and straight lines for the indoor environment [10]. II. WHEELCHAIR MODELING
Design of a line following wheelchair for visually impaired The general structure of a power wheelchair consists of
and paralyzed patients is also one of the indoor environment the chassis, two front free wheels (casters), and two driven
studies [11-12]. As a particular task in this study, we tackle rear wheels, as presented in Fig. 1. Two rear wheels are
sidewalk following in outdoor environment. driven separately by two DC motors. To obtain the
Some previous studies have already touched the problem mathematical model of the power wheelchair, some
of sidewalk following. A pedestrian based approach makes assumptions and limitations are introduced below.
use of natural human motion to allow a robot to adapt to
sidewalk navigation [13]. Vision based navigation studies
are modelled on mobile robot model [14-15]. Another study
is related with curb detection for road and sidewalk
detection proposed for mobile robots [16]. In the present
study, we aim to develop a solution for wheelchair motion
model to follow yellow tactile pavement on the sidewalk
based on visual data.
This study does not need the presence of a pedestrian and
the wheelchair model is compatible for application of a
variety of control design methods. In addition to that, Hough
Lines method is employed which has been extensively used
as a powerful algorithm for extracting straight lines from Figure 1. Power wheelchair model
images. For controller design, mathematical model of the
wheelchair is developed with the purpose of vision based Assumptions:
navigation. A study that relates optimal tuning of dynamic 1. Friction forces on motor armature are negligible;
controller via LQR in a power wheelchair is available in 2. No slipping occurs on the ground and the wheels;
literature [17], but it is focused only on dynamic and 3. Cornering forces are negligible.
kinematic model of the wheelchair. Another study related In building the mathematical model of the wheelchair
with balancing control of wheelchair system includes LQR dynamics shown in Fig. 1, it is taken into consideration that
controller design [18], which differs from our work in the torque is applied to the wheels by the DC motors, and the
aspect of using principle of wheeled inverted pendulum wheels are moved by the horizontal forces Fh exerted at the
model. center of the wheel along the horizontal coordinate axis. The
Available in related literature is a study on automatic line wheelchair movement in the vertical direction is taken as
following navigation system for an intelligent robotic zero, so the deflection angle θ and displacement of the
wheelchair using fuzzy control [19-20], which needs a fuzzy wheelchair x are calculated from x and y coordinate
logic table instead of system modeling. However, this movements only. The equation of motion for the right and
method depends on the designer’s control experience on the left rear wheels is derived using Newton’s second law of
system and the fuzzy control table contains many motion.
parameters as well as tedious and complicated experiments Equation (1) represents the dynamic model for the
[21]. wheels, where equations of motion are derived using Table I
Additionally, many recent studies show that LQR and PP for wheelchair parameters and Assumptions 1 and 3 are
gives very good results on path tracking [22-25], so we applied [18].
decided to apply LQR and PP controllers to our model. In I k k k
x  2 m Va  2 m e   Fh
2( w2  mw )  (1)
the light of and as an addition to the extensive research that rw Rrw Rrw
has been carried out on comparison of LQR and PP control
Also, the equation of motion for the chassis is derived by
[26-31], this paper presents new results on the two methods
using Newton’s second law of motion using Assumptions 1
with applications to the vision based wheelchair control
to 3 as in (2) and (3).
problem.
I k k k
The aim of this study is to assess the performances and (2 w2  2mw  m p ) 
x  2 m Va  2 m 2e x
investigate the differences between LQR and PP controllers rw Rrw Rrw (2)
so that an efficient control methodology can be decided  m p l cos( )  m p l 2 sin( )
based on visual data.
The rest of the paper is structured as follows. In the k k k
( I p  m p l 2 )  2 m Va  2 m e 
x   m p lx cos( ) (3)
second section, we present the mathematical model of the R Rrw
wheelchair and its linearized state space model. The third
The model is linearized around the operating point of:
section is devoted to image modelling and controller design.
The computational results obtained through the application   0 and ( d / dt ) 2  0
of the proposed methodology are presented in fourth section. Then, the linearized equations of motion become:
Experimental study, hardware of the system and results are 2k a 2k k a m 2p gl 2
shared in the fifth section. Some brief conclusions and x  m Va  m 2 e x 
  (4)
Rrw b Rrw b b
future perspectives are given in the last section.

4
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

2k m m p la III. SIDEWALK FOLLOWING CONTROLLER DESIGN

   (1  )Va 
RI T rw b Sidewalk following control problem is tackled using
(5) image based visual servoing technique in this study. Main
2km ke m p la m3p gl 3 goal of the study is designing a controller that minimizes the
(1  ) x  
Rrw I T rw b IT b error between the set of observed visual features and the set
where, of their desired values. For an efficient system design, we
consider the features of vanishing point and lines from the
2Iw
M T  (2mw   mp) (6) sidewalk. Additionally, efficacy of the optimal control
rw2 approach has a vital effect on the success of the study.
b  ( M T I T  m 2p l 2 ) (7) A. Visual Features Extraction and Estimation of
IT  ( I p  m p l ) 2
(8) Parameters
Extraction of features from visual data, which are
a  ( IT  m p l ) (9) vanishing point, median line and middle line is described in
Equations (4) and (5) can be arranged in state space form this section for sidewalk following control purposes.
as follows: Vanishing point can be defined as the point at which
 x  Ax  Bu receding parallel lines viewed in perspective appear to
 (10) converge. The middle line is defined as a line that goes right
 y  Cx  Du through the middle of the image. Eventually, the median line
where, is drawn based on the average of the selected points on the
x yellow tactile pavement of the sidewalk. More detail on
  these three features and the extraction process can be found
x
x  (11) in the subsequent paragraphs of this section in the context of
  Hough lines method. After selected features are stated,
 

  visual feature extraction process is started.
Proposed method for visual feature extraction is the
u  Va  (12) intersection method, which is constructed on calculations of
1 0 0 0 image pixels. Firstly, the values of x coordinate and y
C   (13) coordinate for any image are calculated. Also, the lines of
0 0 1 0 the image are detected by using Hough lines method [32-
0 0 0 0 33]. Hough line transform is based on slope and intercept
D  (14) value of any line. The straight line is normally
0 0 0 0
parameterized as:
0 1 0 0 y  mx  b (17)
 2 2 
0 2km ke a m gl where m is the slope and b is the intercept. Note that m goes
0
p
 
 Rrw2 b b  to infinity for vertical lines.
A  (15) The line equation can be represented in polar coordinates
0 0 1 0 as:
  r  x cos( )  y sin( ) (18)
0 2k m ke m p la m3p gl 3
 (1  ) 0  where r is the distance from the origin to the closest point on
 Rrw I T rw b IT b 
the straight line and (r, φ) corresponds to the Hough space
0  representation of a line as shown in Fig. 2. In this represe
 2k a  ntation, (r, φ) is accepted as 0 set value. Hough line
 m 
 Rrwb  transform calculates the (r, φ) pair values using x and y
B  (16) coordinates of image pixels. Lines are found at the peak
  2k m (1  m p la )  values of the (r, φ) pair.
 RI T rw b 
 
0 
TABLE I. PARAMETERS OF THE WHEELCHAIR
Parameter Description Value Units
g Gravitation 9.81 m/s2
rw Wheel radius 0.1778 m
mw Wheel mass 2.8 kg
mp Body mass with load 75.4 kg
Iw Wheel inertia 0.03 Kg m2 Figure 2. Hough line parameters on the image coordinates to specify
straight line by the angle φ
Ip Body inertia 0.44 Kg m2
l Distance from body’s COG 0.52 m
km Motor torque 0.75 Nm/A Another representation for Hough line transformation
ke Back EMF 0.75 V/(rad/s) theory is in Cartesian coordinate form, which is exemplified
R Terminal resistance 2.38 Ω in Fig. 3. The red and blue points are mapped to blue line in
Hough domain. Red line and blue line is intersected in a
point then this point yields the values m and b of the

5
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

equation (17). More points that lie on the same line tend to
more lines in Hough domain and that will increase voting to
the intersection point indicating that there are many points
that belong to the line in image domain with that slope and
y-intercept [34].

Figure 3. Image space and parameter space of any image to detect lines
for sidewalk following
Figure 6. Theta angle and vanishing point on the yellow line-sidewalk

Also, the needed median line is found by calculating

average of selected points on the yellow tactile pavement.
Using simple tangent calculation, theta θ is calculated as the
angle between the middle line and the median line as in Fig.
6. In Fig. 6, Red line shows median line, green lines show
most intersected Hough lines for the yellow tactile
pavement, blue line shows vanishing line and black line
shows middle line in Fig. 6. Fig. 7 shows more examples for
image extracted features.
B. Controller Design
The image-based sidewalk following controller is
designed according to the theta values that are acquired from
the image processing part. Theta values are used as
reference values for path following, and controller is
Figure 4. Hough lines on the yellow line-sidewalk
designed for efficient control.
Hough lines method can be applied to the sidewalk
following problem as shown in Fig. 4, where Hough lines
are highlighted in green. After Hough lines are generated,
the most intersected two lines for the image are chosen and
this intersection point is accepted as the vanishing point as
seen Fig. 5. The vanishing point is marked in Fig. 6 with a
red cycle. After the vanishing point is determined, some
calculations are carried out to find the corresponding
velocity, which depends on the theta angle θ. By dividing ( (
the x coordinate of image pixels by two, the middle line of ) b)
image is found.

( (
) d)
Figure 7. More examples for image extracted features on the yellow line-
sidewalk: a) example-1; b) example-2; c) example-3; d) example-4

In modern control theory, LQR has justified its

significance as an optimal control approach [35] and has
been successfully applied on control problems for dynamic
systems with uncertainties [36]. It should be that LQR is a
closed-loop control strategy which is among the
Figure 5. Most intersected two lines on the yellow line-sidewalk

6
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

fundamental control methods applicable to many advanced The elements of Q and R matrices are selected from the
control problems. Thus, LQR controller has good potential Bryson’s rule combined with trial-and-error [40].
for combining the orthogonal collocation optimization to State feedback gain KLQR is provided by the following
upgrade the optimization and control performances of equations (22-24):
dynamic systems with disturbances [37]. K LQR  R 1 B T P (22)
On the other hand, PP control method is quite popular in
control design for letting the control designers choose their U   K LQR x (23)
desired performance through preferred closed-loop pole AT P  PA  Q  PBR 1 B T P  0 (24)
locations [38]. where the Algebraic Riccati Equation in (24) is used here to
The stability and various control performance indicators find P that represents constant matrix and calculated to find
of a closed-loop linear time-invariant system mostly depend optimal input of the system. So, KLQR gain matrix is:
on pole locations. For this reason, the poles of the closed-
K LQR   3.1623 20.4120 142.3382 0.7952  (25)
loop system should be located at those positions that have
rational and expected performance during control system 2) PP controller
design. There are several approaches for selection of pole
In modern feedback control, it is possible to collect more locations for good design. We have used Dominant Second
control information through state feedback in linear and Order Poles approach for PP controller design here. The
time invariant systems. Accordingly, state feedback has approach focuses on pole selection without explicit regard
been widely applied in deriving optimal control law and for their influence on control effort; but, the designer is able
eliminating the effect of disturbances [39]. In this research, to temper the choices to take control effort into account. The
it is aimed to apply two control methods of LQR and PP in pole which is nearer to the origin or imaginary axis is
state feedback configuration because of above mentioned referred to as a dominant pole. When we look at the poles of
reasons. the system, it can be seen that the nearest pole to the origin
is the real pole located at -0.0744. Poles with multiplicity
should not be greater than rank of B matrice, so the other
pole can be selected as -0.075. The rise time, overshoot and
settling time can be deduced directly from the pole
locations. Damping ratio ζ=1 will meet the overshoot
requirement and for this damping ratio, a rise time of 6 sec.
suggests a natural frequency ωn of about 1. There are four
poles in total, so the other two need to be placed far from the
dominant pair; for our purposes, “far ” means the transients
Figure 8. Visual data based control block diagram due to the fast poles should be over (significantly faster)
well before the transients due to the dominant poles and we
Performances of the two control methods are tested on the assume a factor of 1 in the respective undamped natural
wheelchair model, which is known to be completely state frequencies to be adequate [41]. From these considerations
controllable, and results are shared in section IV. Visual data desired poles are given in the following matrix:
based control block diagram of the proposed feedback
pole   1  i 1  i 0.075 0.0744  (26)
system for sidewalk following control is shown in Fig. 8.
1) LQR controller To find the control gain for PP, Kpp with desired poles,
As a state-feedback controller, LQR algorithm is applied well known Ackermann’s formula is used [42].
on the wheelchair model. State feedback law is designed K   0 0  0 1 S 1 ac ( A) (27)
with the purpose of minimizing the cost function given in where matrix A is defined in Equation (10) and matrix S is
(19), given by:
 
¥

S B A2 B  An 1 B

J  ( x Qx  u Ru ) dx AB (28)
T R
(19)
0 and the notation ac(A) is given by:
where x and u are determined earlier in Equation (10) that ac ( A)  An  an 1 An 1    a1 A  a0 I (29)
represent the state variables of the model and the input
So, Kpp is calculated from equations (27-29) as:
voltage to provide motion of the wheelchair respectively.
One of the weighting matrices Q that is used to penalize K pp   0.0057 4.0682 71.3825 0.9229  (30)
bad performance is selected as: 3) Overall system
 10 0 0 0  Overall system’s operation procedure for both controllers
0  is shown in the Figs. 9 and 10 in the form of a flowchart.
10 0 0
Q  The architecture of the proposed system comprises two main
(20)
0 0 10 0  sections. First section is visual feature extraction and this
  common part is identical for the both control methods. As
0 0 0 5 obviously seen in the flowcharts, θ value is the output of the
and the other weighting matrix R that is used to penalize image processing part. Then, the angle θ is used for
actuator effort is selected as: generating the state reference signal of both controllers.
R  (1) (21)

7
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

Figure 9. Flowchart of LQR controlled system

Figure 10. Flowchart of PP controlled system

8
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

Aforementioned in the previous sections, Hough lines,

LQR and PP control methods can be seen in the Figs. 9 and
10. K gain value for LQR comes from the Riccati Equation,
whereas K gain value for PP comes from the Ackermann’s
formula. The presented flowcharts are aimed to be useful for
illustrating the differences between the two control methods.

IV. SIMULATION TESTS AND RESULTS

Four simulations test results are presented in this section.
Firstly, responses to set points variations of the LQR and PP
controlled system are shared. Trajectory tracking tests are
shown in the second part. Thirdly, disturbance attenuation
tests and results are presented for the LQR and PP
controller. Robustness to body weight variations are Figure 11. Responses of states for LQR controlled system
presented as last test in this section.
A. Responses to Set Point Variations
The designed feedback control system is tested via
computer simulations under LQR and PP control strategies
for various scenarios. Fig. 11 shows the responses of the
states for LQR control with KLQR gain matrix, where the
desired state xd and inital state x0 values are taken as
following:
xd = [5;2;10;3] (31)
x0 = [3;4;5;8] (32)
The state variables in x0 and xd are displacement x,
velocity x , wheelchair directional angle θ, and its derivative
 in respective order.
Step responses of the feedback control system under LQR Figure 12. Step responses of the LQR and PP controller
control and PP control for θ state variable are shown in Fig.
12. When the system is started, θ angle can reach the
expected value of 1 radian and settles stably in 58 s with
small overshoot for LQR control. Designed PP controlled
system settles stably in 140 s, which means PP controlled
system needs more time to reach the settling point.
Fig. 13 shows the step response test result of the proposed
LQR control method compared with the traditional control
strategy of Proportional-Integral-Derivative (PID) control.
In this test, displacement x state vector is neglected and θ
state vector is taken into consideration. The best gains are
selected as proportional gain 0.5081, integral gain 0.001 and
derivative gain 0.001 for PID controller after auto-tune is
employed and fine tuning is applied in computer simulation
environment. Fig. 13 clearly shows that LQR controller has Figure 13. Step responses of the LQR and PID controller
good performance with rise time 6.42 s, settling time 14.85 s
and overshoot 2.23 percent. When the PID controller
performance is analyzed, the rise time is 12.56 s, settling
time 44.78 s, and overshoot 10.62 percent. Presented step
response test results reveal that LQR controller has
advantages over conventional PID method.
B. Trajectory Tracking Tests
In the second test, desired angle sets are assumed to be
sine wave and square wave as reference trajectory to
evaluate the system performance of both controllers. It is
observed in Figs. 14 and 15 that the reference trajectory is
quickly followed by LQR and PP and moreover LQR
controller achieves to track the reference trajectory with less
deviation compared to PP controller.
Figure 14. Theta tracking path for LQR and PP controller (square wave
reference)

9
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

C. Disturbance Attenuation Tests

Disturbance attenuation tests are applied on the
simulation model block diagram given in Fig. 16. This
diagram is used also for PP control by changing the value of
K gain matrix.
Moreover, this block diagram is used for adequate
disturbance types, which are added to the theta state of the
PP and LQR controlled model. Random noise is applied to
the LQR controlled model as disturbance at sample time 70,
with mean 0, variance 0.05 and seed 0. Effect of the random
noise can be seen in Fig. 17. Step disturbance is added to the
LQR controlled model on sample time 0, with step time 50
and final value 0.3. Effect of step disturbance can be seen in
Fig. 18.
Figure 15. Theta tracking path for LQR and PP controller (sine wave
Square wave disturbance is added to the LQR controlled
reference model frequency 0.005 Hertz and amplitude 0.5. Effect of
square wave disturbance can be seen in Fig. 19. The
TABLE II. ERROR INDEX VALUES OF THE SQUARE WAVE AND SINE WAVE disturbance attenuation performance with LQR is analyzed
TESTS
on the basis of MSE, IAE and ITAE error index values and
Reference Wave Controller MSE IAE ITAE
LQR 3.13 145.3 1.42x104 presented in Table III.
Square
PP 14.15 6151 6.34x105 Random noise is added to the PP controlled model as
LQR 0.03781 144.4 1.45x104 disturbance on sample time 70, mean 0, variance 0.05 and
Sine
PP 0.0384 149.2 1.49x104
seed 0. Effect of random noise can be seen on Fig. 20. Step
disturbance is added to the PP controlled model on sample
The efficacy of LQR in tracking the reference is also time 0, step time 50 with final value 0.3. Effect of step
verified by the error index values of the two tests. As the disturbance can be seen on Fig. 21. Square wave disturbance
error indices representing the control efficacy, the Mean is added to the PP controlled model on frequency 0.005
Squared Error (MSE), the Integral Absolute Error (IAE), Hertz and amplitude 0.5.
and the Integral Time-weighted Absolute Error (ITAE) are
selected. MSE, IAE, and ITAE values of Figs. 14 and 15 are
given in Table II.

Figure 16. Simulation model for observing disturbance attenuation performance of LQR controlled model

10
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

Figure 17. Random noise added to theta state of the LQR controlled model
Figure 21. Step disturbance added to theta state of the PP controlled model

Figure 18. Step disturbance added to theta state of the LQR controlled
model Figure 22. Square wave disturbance added to the PP controlled model

Effect of square wave disturbance can be seen on Fig. 22.

The disturbance attenuation performance of the PP control
method is revealed by the MSE, IAE and ITAE values in
Table IV.
The given error index values in Tables II-IV are used to
assess the performance of each controller, where smaller
values of the performance indices indicate that the controller
has a good performance. It is clearly observed that LQR
controlled system has lower error indices than PP controlled
system except IAE and ITAE values in Table II. This is
interpreted as the result of the overshoots exhibited by the
output responses of LQR controlled model. In the
Figure 19. Square wave disturbance added to theta state of the LQR disturbance attenuation tests, it is considering that as
controlled model example holes on the way or broken roads cause distortion
motion of the autonomous wheelchair and effect of this
distortions and controller responses can be seen on Figs. 17-
22. Lastly, when Table III and Table IV are compared in
terms of error values of LQR and PP, LQR controller
provides much better disturbance attenuation performance
compared to PP.
TABLE III. ERROR INDEX VALUES OF FIGURES 16-18 FOR LQR
Input Disturbance MSE IAE ITAE
Random Noise 0.029 349.1 6.13x104
Step Disturbance 0.028 348.7 6.12x104
Square Wave 0.029 351.7 6.13x104

TABLE IV. ERROR INDEX VALUES OF FIGURES 19-21 FOR PP

Figure 20. Random noise added to theta state of the PP controlled model Input Disturbance MSE IAE ITAE
Random Noise 0.069 803.6 1.35x105
Step Disturbance 0.028 1059 1.09x105
Square Wave 1.046 3023 5.70x105

11
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

V. ROBUSTNESS TO BODY WEIGHT VARIATIONS A 24V DC battery is employed to supply power to both
Simulation tests up to this point are realized for an the wheelchair's motors and the Sabertooth motor driver.
average body mass of 75.4 kg with the moment of inertia Additionally, a separate 5V DC power source is utilized to
0.44 kg.m2. In order to verify the robustness of proposed energize the Raspberry Pi module and the camera. The
controller to variations of driver mass, the model is also Sabertooth motor driver is used to control the movement of
tested for various body masses and corresponding moments the motors of the wheelchair. Communication between the
of inertia. Sabertooth module and Raspberry Pi is established via serial
With this purpose, eight different body mass values communication protocols. For a more comprehensive
between 40-150 kg and moments of inertia between 0.234- understanding of the interconnections and the physical
0.875 kg.m2 are simulated in computer environment. The positioning of these components on the actual wheelchair,
test results are shown in Fig. 23, and a zoomed-in view is refer to Fig. 26.
presented in Fig. 24 to better observe the results. The The Sabertooth motor driver facilitated wheelchair
graphical results show that the proposed LQR controlled movement based on velocity signals from the Raspberry Pi.
model is stable and robust to driver mass variations, with This entire process runs at about 7 Hz, taking around 143 ms
minimal deviation from the set value. The steady-state error per motion cycle. This frequency ensures prompt interaction
variation between the assumed minimum and maximum among system components, ensuring efficient wheelchair
mass values is only 0.21 percent. movement.

VI. EXPERIMENTAL STUDY

For experimental verification of the suggested control
strategy, a test setup with a standard power wheelchair and a
laptop computer is used. The wheelchair is equipped with
the camera and required electronics to realize the suggested
control system. The wheelchair test setup image is shown in
Fig. 25, and the hardware used for the tests are explained on
the wheelchair image in Fig. 26. Real experiments are
conducted on the Gaziantep University adjacent sidewalk
with yellow tiles for disabled individuals. The test setup
comprises two distinct power sources. Figure 25. A photo from our experimental study

Image Acquisition and Visual Signals Obtaining Process:

The experimental setup is configured such that it is
possible to access the Raspberry pi camera images from a
remote server (that is the laptop computer in our case). This
configuration facilitates the image processing and control
command calculation procedures to be realized in the laptop
computer environment, while the motor movement drive
and image acquisition procedures are realized by the
Raspberry Pi.

Figure 23. Step responses LQR controlled model for various values of the
body weight and inertia

Figure 26. Wheelchair hardware system and camera’s position

The experimental test procedure can be summarized in

steps as follows:
 Open a socket connection between the host computer
Figure 24. Step responses LQR controlled model for various values of the and Raspberry pi over TCP for motor commands;
body weight and inertia (zoomed in)
 Read frames from Raspberry Pi camera;
 Transmit the frames to the image processing and

12
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

control code; A computer simulation model is built to see the results of

 Return the calculated speeds depending on the theta the study and simulation test outcomes provide the basis for
values to Raspberry Pi. the assessment of control performances for successful
Experimental results representing the yellow line tracking sidewalk following. Compared with PP control, LQR
performance of the wheelchair are shown in Fig. 27. Real control method proves to have better performance in system
wheelchair’s deflection angle on the sidewalk and response. Disturbance attenuation tests on the system also
wheelchair’s position on the coordinate system are shown in show satisfactory performance in agreement with the
Fig. 29. When θ is taken as 0 as the reference value, LQR specifications. Consequently, the wheelchair model has the
and PP controlled models follow the yellow line ability to follow sidewalk sucessfully with visual feedback
successfully. As in the simulation results, LQR gives better under suggested control methods, and the autonomous
results in the experimental study. MSE (Mean Squared wheelchair dynamics and simulation results prove the
Error) values of both control methods are calculated. The efficacy of the model and designed control system. As a
results show that MSE for PP is 3.4838x10-4 and MSE for future perspective of this study, this control strategy and
LQR is 1.4871x10-4, which clearly reveals that LQR applied control methods will be tested on the real wheelchair
provides much better performance in the real experimental in real life conditions.
study.
APPENDIX
VII. CONCLUSION Raspberry Pi V2.1 Camera Characteristics:
In this paper it is aimed to present the results of a study on The camera’s capability to output 4K video at 60 Hz and
two different control strategies for yellow tactile pavement support for dual monitors enable high-quality visual
tracking control of an autonomous wheelchair model for feedback, which is a crucial feature for advanced robotic
sidewalk following. The task of sidewalk following is applications. Additionally, the camera module delivers good
achieved by exploiting visual features as vanishing point image quality and supports high dynamic range (HDR)
and angle between middle line and median line using Hough imaging, enabling robots to capture detailed and well-
Lines method. Sets of this angle are used as reference balanced visuals in diverse environments. Some other
trajectory values for designed control systems. The features are as follows:
characteristics of wheelchair system are studied to establish  Still resolution: 8 megapixels;
the mathematical model, and linearized model is constructed  Video modes: 1080p47, 1640×1232p41 and
for LQR and PP controllers. 640×480p206;
 Sensor resolution: 3280 × 2464 pixels;
 Sensor image area: 3.68 x 2.76 mm (4.6 mm diagonal);
 Pixel size: 1.12 µm x 1.12 µm;
 Depth of field: Approximately 10 cm to ∞;
 Focal length: 3.04 mm;
 Horizontal field of view: 62.2 degrees;
 Vertical field of view: 48.8 degrees.

REFERENCES
[1] L. I. Iezzoni, E. P. Mccarthy, R. B. Davis and H. Siebens, “Mobility
difficulties are not only a problem of old age,” Journal of General
Internal Medicine, vol. 16, pp. 235–243, 2001. doi:10.1046/j.1525-
1497.2001.016004235
[2] Turkey Health Interview Survey, “The percentage of individuals not
able to walk, walk up and down stairs without any aid or assistance by
sex and age group, 2008, 2010, 2012, 2014, 2016,” 2018.
https://data.tuik.gov.tr/Kategori/GetKategori?p=Saglik-ve-Sosyal-
Koruma-101 (accessed Nov. 23, 2023)
Figure 27. Experimental results for LQR and PP controlled model [3] J. Leaman and H. M. La, “A comprehensive review of smart
wheelchairs: past, present and future,” IEEE Transactions on Human-
Machine Systems, 47, pp. 486-499, 2017.
doi:10.1109/THMS.2017.2706727
[4] A. Kokosy, T. Floquet, G. Howells, H. Hu, M. Pepper, M. Sakel and
C. Donze, “SYSIASS - an intelligent powered wheelchair,” in
International Conference on Systems and Computer Science, Lille,
2012
[5] J. Leaman and H. M. La, “iChair: Intelligent powerchair for severely
disabled people,” in The ISSAT International Conference on
Modeling of Complex Systems and Environments (MCSE), Da Nang,
2015, pp. 1-6. doi:10.13140/RG.2.1.3404.0808
[6] U. Yayan, B. Akar, F. İnan and A. Yazici, “Development of indoor
navigation software for intelligent wheelchair,” in 22nd Signal
Processing and Communications Applications Conference (SIU),
Trabzon, 2014, pp. 405-408. doi:10.1109/SIU.2014.6830251
[7] E. Demeester, E. Vander Poorten, A. Hüntemann and J. De Schutter,
“Wheelchair navigation assistance in the fp7 project radhar:
Objectives and current state,” IROS 2012 Workshop on Progress,
Figure 28. Real wheelchair’s deflection angle representation on the Challenges and Future Perspectives in Navigation and Manipulation
sidewalk Assistance for Robotic Wheelchairs, 2012

13
 [Downloaded from www.aece.ro on Saturday, July 06, 2024 at 04:13:07 (UTC) by 68.36.208.214. Redistribution subject to AECE license or copyright.]

Advances in Electrical and Computer Engineering Volume 24, Number 1, 2024

[8] S. A. V. Kuppa, M. S. H. Reddy, A. Sanjana, J. Sridevi and V. U. [24] M. N. Yazar, Z. A. Erbudak, A. Bay, İ. Dost and A. Semiz, “Path
Rani, “Design and development of smart wheelchair,” in IEEE 2nd tracking methods employed in lane centering systems,” in 7th
International Conference on Sustainable Energy and Future Electric International Conference on Mechanical Engineering and Robotics
Transportation (SeFeT), Hyderabad, 2022, pp. 1-5. Research (ICMERR), Krakow, 2022, pp. 13-17.
doi:10.1109/SeFeT55524.2022.9908983 doi:10.1109/ICMERR56497.2022.10097807
[9] F. Pasteau, V. K. Narayanan, M. Babel and F. Chaumette, “A visual [25] J. Ni, Y. Wang, H. Li and H. Du, “Path tracking motion control
servoing approach for autonomous corridor following and doorway method of tracked robot based on improved LQR control,” in 41st
passing in a wheelchair,” Robotics and Autonomous Systems, 75, pp. Chinese Control Conference (CCC), Hefei, 2022, pp. 2888-2893.
28-40, 2016. doi:10.1016/j.robot.2014.10.017 doi:10.23919/CCC55666.2022.9902113
[10] F. Pasteau, M. Babel and R. Sekkal, “Corridor following wheelchair [26] E. V. Kumar and J. Jerome, “Robust LQR controller design for
by visual servoing,” in International Conference on Intelligent Robots stabilizing and trajectory tracking of inverted pendulum,” Procedia
and Systems (IEEE/RSJ), Tokyo, 2013, pp. 590-595. Engineering, 64, pp. 169-178, 2013.
doi:10.1109/IROS.2013.6696411 doi:10.1016/j.proeng.2013.09.088
[11] G. R. Shoovo, B. Dey, M. K. Akash, T. Motahara and M. H. Imam, [27] N. Razmjooy, A. Madadi, H. R. Alikhani and M. Mohseni,
“Design of a line following wheelchair for visually impaired “Comparison of LQR and pole placement design controllers for
paralyzed patient,” in 2nd International Conference on Robotics, controlling the inverted pendulum,” Journal of World’s Electrical
Electrical and Signal Processing Techniques (ICREST), Dhaka, 2021, Engineering and Technology, vol. 3(2), pp. 83-88, 2014
pp. 398-402. doi:10.1109/ICREST51555.2021.9331218 [28] S. C. Saini, Y. Sharma, M. Bhandari and U. Satija, “Comparison of
[12] H. İ. Şahin and A. R. Kavsaoğlu, “Autonomously controlled pole placement and LQR applied to single link flexible manipulator,”
intelligent wheelchair system for indoor areas,” in 3rd International in International Conference on Communication Systems and Network
Congress on Human-Computer Interaction, Optimization and Robotic Technologies (CSNT), Gujrat, 2012, pp.843-847.
Applications (HORA), Ankara, 2021, pp. 1-6. doi:10.1109/CSNT.2012.183
doi:10.1109/HORA52670.2021.9461335 [29] P. Chalupa, J. Trefil and J. Novák, “Modelling and hardware design
[13] Y. Du, N. J. Hetherington, C. L. Oon, W. P. Chan, C. P. Quintero, E. of double inverted pendulum,” in 24th International Conference on
Croft and H. M. Van der Loos, “Group surfing: A pedestrian-based Process Control (PC), Štrbské Pleso, 2023, pp. 18-23.
approach to sidewalk robot navigation,” in International Conference doi:10.1109/PC58330.2023.10217698
on Robotics and Automation (ICRA), Montreal, 2019, pp. 6518-6524. [30] S. Zarghoon and C. Belavý, “LQR and pole placement controllers
doi:10.1109/ICRA.2019.8793608 with integral action design technique and performance analysis for
[14] M. Wen, J. Zhang, T. Chen, G. Peng, T. Chia and Y. Ma, “Vision control of temperature field in continuous casting process,” in 24th
based sidewalk navigation for last-mile delivery robot,” in 17th International Carpathian Control Conference (ICCC), Miskolc-
International Conference on Control, Automation, Robotics and Szilvásvárad, 2023, pp. 489-494.
Vision (ICARCV), 2022, Singapore, pp. 249-254. doi:10.1109/ICCC57093.2023.10178886
doi:10.1109/ICARCV57592.2022.10004362 [31] B. N. Soufiani and M. A. Adli, “Pole placement and LQR control of
[15] T. Senlet and A. Elgammal, “Design of a vision-based autonomous slosh-free liquid transportation with dual-arm cooperative
robot for street navigation,” in 22nd Signal Processing and robot,” Journal of the Faculty of Engineering and Architecture of Gazi
Communications Applications Conference (SIU), Trabzon, 2014, pp. University, vol. 35(4), pp. 2255-2267, 2020.
862-865. doi:10.1109/SIU.2014.6830366 doi:10.17341/gazimmfd.703520
[16] M. Cheng, Y. Zhang, Y. Su, J. M. Alvarez and H. Kong, “Curb [32] P. V. C. Hough, “A method and means for recognizing complex
detection for road and sidewalk detection,” IEEE Transactions on patterns,” U.S. Patent No. 3069654, 1962
Vehicular Technology, 67(11), 10330-10342, Aug. 2018. [33] R. O. Duda and P. E. Hart, “Use of the Hough transformation to
doi:10.1109/TVT.2018.2865836 detect lines and curves in pictures,” Communications of the
[17] E. M. Araújo Filho and J. V. Da Fonseca Neto, “Optimal tuning of ACM, 15(1), pp. 11-15, 1972. doi:10.1145/361237.361242
dynamic controller via LQR in a powered wheelchair,” American [34] A. Alaa, E. Adel and A. Anwar, “Hough transform,”
Journal of Engineering Research, vol. 6(11), pp. 44-53, Nov. 2017 https://sbme-tutorials.github.io/2021/cv/notes/4_week4.html#original-
[18] S. Doung and U. Wasiwitono, “Multibody dynamics modeling and hough-transform-cartesian-coordinates (accessed Nov. 23, 2023)
control of wheelchair balancing system,” International Seminar on [35] H. Sun, Y. Liu, F. Li and X. Niu, “Distributed LQR optimal protocol
Intelligent Technology and Its Applications (ISITIA), Surabaya, 2021. for leader-following consensus,” IEEE Transactions on Cybernetics,
pp. 123-128. doi:10.1109/ISITIA52817.2021.9502215 vol. 49, pp. 3532-3546, July 2018. doi:10.1109/TCYB.2018.2850760
[19] W. C. Cheng and C. C. Chiang, “The development of the automatic [36] Y. Li, M. Xu, J. Chen and X. Wang, “Nonprobabilistic reliable LQR
lane following navigation system for the intelligent robotic design method for active vibration control of structures with
wheelchair,” in IEEE International Conference on Fuzzy Systems uncertainties,” AIAA Journal, Apr. 2018. doi:10.2514/1.J056510
(FUZZ-IEEE), Taipei, 2011, pp. 1946-1952. [37] L. Fan, P. Liu, H. Teng, G. Qiu, P. Jiang, “Design of LQR tracking
doi:10.1109/FUZZY.2011.6007742 controller combined with orthogonal collocation state planning for
[20] S. A. Wani, I. Nasiruddin, M. Shahid and S. Khatoon, “Intelligent process optimal control,” IEEE Access, vol. 8, Dec. 2020.
controller design for motion control of smart wheelchair,” in doi:10.1109/ACCESS.2020.3043498
International Conference on Power, Instrumentation, Energy and [38] S. Das, K. Halder and A. Gupta, “Delay handling method in
Control (PIECON), Aligarh, 2023, pp. 1-5. dominant pole placement based PID controller design,” IEEE
doi:10.1109/PIECON56912.2023.10085867 Transactions on Industrial Informatics, vol. 16, pp. 980-991, May,
[21] L. Wei and W. Yao, “Design and implement of LQR controller for a 2019. doi:10.1109/TII.2019.2918252
self-balancing unicycle robot,” in IEEE International Conference on [39] Y. Lan and M. Fei, “Design of state-feedback
Information and Automation (ICIA), Lijiang, 2015, pp. 169-173. controller by pole placement for a coupled set of inverted
doi:10.1109/ICInfA.2015.7279279 pendulums,” in 10th International Conference on
[22] I. Siradjuddin, Y. W. Syaifudin, T. Winarno, E. Rohadi, F. A. Electronic Measurement & Instruments (ICEMI), Chengdu, 2011, pp.
Salsabilla and S. Adhisuwignjo, “Linear controller design using pole 69-73. doi:10.1109/ICEMI.2011.6037857
placement method for nonholonomic mobile robot trajectory [40] G. F. Franklin, J. D. Powell, M. L. Workman, “Digital control of
tracking,” in Sixth International Conference on Vocational Education dynamic systems,” pp. 313-314, Addison-Wesley, 1998
and Electrical Engineering (ICVEE), Surabaya, 2023, pp. 122-126. [41] G. F. Franklin, J. D. Powell, A. Emami-Naeini, “Feedback control of
doi:10.1109/ICVEE59738.2023.10348262 dynamic systems,” pp. 477-480, Prentice Hall, 2002
[23] Y. Sun, Q. Dai, J. Liu, X. Zhao and H. Guo, “Intelligent vehicle path [42] H. Saadat, “Power System Analysis”, pp. 569-572, McGraw-Hill,
tracking based on feedback linearization and LOR under extreme 1993
conditions,” in 41st Chinese Control Conference (CCC), Hefei, 2022,
pp. 5383-5389. doi:10.23919/CCC55666.2022.9901797

14