1

Gabriela. Egüez, Cristian. Fernández, Omar. Ortiz y Abigail. Rubio

Control and planning of flight

Universidad de las Fuerzas Armadas-ESPE
gceguez@espe.edu.ec, cpfernandezp@espe.edu.ec, xoortiz1@espe.edu.ec,
karubio@espe.edu.ec

trajectories of multirotor UAVs (or

fixed wing) based on reinforcing
learning

METHODOLOGY
Abstract— The generation of trajectories and their planning in
the interpreters for unmanned aerial vehicles is a growing discipline,
in this document the information of the trajectories of the unmanned C.Control of Quadrotor
aerial vehicles is used to control the automation of unmanned aerial The most basic and necessary sensor in any quadrant is the IMU
vehicles. These planning schemes, this means that a route has been sensor, this consists of an accelerometer and a gyroscope.
planned and the flight has been planned once an account has been
planned. 1) PID Controller
The model has four input forces that are basically the thrust
Index Terms— UAVs, Reinforcing Learning, control systems, provided by each propeller connected to each rotor with a fixed
autonomous navigation, planning. angle. The forward (rearward) movement continues to increase
(decrease) the speed of the front (rear) rotor speed while decreasing
I. INTRODUCTION (increasing) the rear (front) rotor speed simultaneously, which means
changing the tilt angle. Movement to the left and to the right is

R
oute tracking and reinforcement learning are one of the central
elements of a UAVs unmanned aerial vehicle navigation achieved by changing the angle of rotation in the same way. The
system, UAVs are autonomously controlled vehicles using front and rear engines rotate counterclockwise, while other engines
programmed flight plans, they are currently used for tasks involving rotate clockwise, so that the steering command is derived by
some difficulty or risk for conventional vehicles manned by people increasing (decreasing) the speed of the engines in the opposite
[1], the programming that is made to the UAVs are by way of route direction clockwise while decreasing (increasing) the motor speeds
tracking which is a process that deals with how to determine the in a clockwise direction. [5]
speed and direction settings at each moment of time for the UAV to
follow a certain route, a route consists of a set of points that 2) Fuzzy Controller
represent the positional coordinates of a particular course, when a The fuzzy control strategy implements the logic used to control this
route tracking algorithm is implemented, a responsible route vehicle. The controller decisions are taken based on the desired
registration unit must also be implemented to save all the combination of the four motions: height, pitch, roll and yaw.
coordinates that constitute the course, in addition the route tracking These actions increase or reduce the power of each of the motors to
algorithm also n has to handle deviations oriented position or get the specifications and to follow a trajectory. The inputs of the
orientation of the route [2]; These deviations can be caused by new controller are the error of the state variables, three for each vector x,
obstacles that occur in the path that should be avoided, for this it is v, α, and ω.
necessary to reinforce learning which is a type of automatic learning
technique that allows UAVs to learn in an interactive environment D.Path Tracking
through the test and error, using the feedback of the actions or
experiences that it has, that is to say it uses a mapping between the Optimal time, is one of the most important aspects when talking
entrance and exit [3]. about achieving maximum performance, is a problem of high
difficulty [6]. One way to approach this problem is to see it in two
stages, the first is path planning, which is the planning of the path
through a geometric path [7]. The other stage path tracking, the
II.THEORY optimal time through the geometric path is determined [8]. It is also
known that the optimal trajectory depends on the maximum
A. Path Tracking acceleration and deceleration that engines can have [9]
Route tracking and reinforcement learning are one of the central
elements of a UAVs unmanned aerial vehicle navigation system, Tracking the route is the process of determining the speed and
UAVs are autonomously controlled vehicles using programmed direction settings at each instant of time for a robot to follow a
flight plans, they are currently used for tasks involving some certain route.
difficulty or risk for conventional vehicles manned by people [4].
1) Follow the carrot
B. Reinforcement Learning
This method also is known as adaptative dynamic programming, The carrot point, or goal point, is then defined to be the point on
is a useful tool for resolve complex sequential problems, like the path a look-ahead distance away from the intersection point of
Markov´s decision problem and its variants. The great utility of this line. The most important parameter is the orientation error,
learning reinforcement (RL) is due to its bases in classical dynamic defined to be the angle between current vehicle heading and the line
programming (DP) and stochastic processes. drawn from the center of the vehicle coordinate system to the carrot
point.

 2

2) Pure pursuit  Define the environment and the characteristics.

The concept of the pure pursuit approach is to calculate the  Establish the “sub-paths” that are constitute by two
curvature that will take the vehicle from its current position to a goal
nodes.
position [10]. A circle is then defined in such a way that it passes
through both the goal point and the current vehicle position. Finally,  Establish a height H, in which the terrain is obstacle-
a control algorithm chooses a steering angle in relation to this circle. free, it means that the A* algorithm find a track for
In fact, the robot vehicle changes its curvature by repeatedly fitting each “sub-path”, in the opposite case increase H
circular arcs of this kind, always pushing the goal point forward. until a path can be found.
 Establish a P probability, which gives the accessible
locations for the UAV, in values below P.
 The risk of the UAV in all the track is calculated.
 If the risk is above the umbral R, the P probability
decreases.
Re-planification: alternative election
In reaction for threats, a re-planification is made, for that is
Fig. 1. Pure pursuit approach [17] necessary a new flight plan. The UAV take decisions to adjust
the speed, inside the limitations imposed [12].
3) Vector Pursuit
Vector pursuit uses both the location and the orientation of the Dynamic Environment via Partially Observable Markov
look-ahead, giving it an advantage over the other algorithms as it Decision Process
ensures that the vehicle arrives at the current goal point with the This algorithm collects information that is taken by the
proper steering angle. A screw consists of a centerline and a pitch
sensors, with this information paths are constructed and make
defined in a given coordinate system. The centerline can be defined
by using Plücker line coordinates. A line can be represented using the necessaries calculations.
only two points represented by the vectors r1 and r2. This line can
also be defined as a unit vector S, in the direction of the line and a
moment vector S0, of the line about the origin. [11]

E. Reinforcement Learning
1) Rapidly Exploring Random trees (RRT) Tree of
decisions
This method consists in the construction of a tree, which grows
from an origin point.

Fig. 4. Process of the UAV

III. DESARROLLO
Fig. 2. Tree growth
A. BUILDING
This algorithm is capable of generate a tree, which allows
the explanation of the free space. 1) Materials:

 Arduino: It is a microcontroller based on the ATmega328P

microchip. The board is equipped with several digital or
analogic input/output pins that can be interconnected to other
expansion boards and other circuits. The costs for this product
is $10.

Fig. 3. Evolution of the tree in an environment.

2) A* algorithm (A star)

Initial flight plan: Fig. 5. Arduino

 3

 Base of carts: Acrylic base, used by the facility to incorporate

the materials. The cost is $8.

Fig. 9. Engines 120 [rpm] 12 [V]

 Batteries: 9 V battery to energize the circuit. The cost

for this product is $1.

Fig. 6. Base

 Wheels: The cost for this product is $4.

Fig. 10. Battery 9 [V]

 Bluetooth module Hc-06: Enables the transmission of

data between different devices. The cost of this product
is $9.

Fig. 7. Crazy Wheel

Fig. 11. Bluetooth module

 H bridge module: This module is compatible with

Arduino and it allow us to reverse the rotation og an
engine. The cost for this product is $4.

Fig. 8. Wheels

 Engines: 120 rpm gearmotors. It provides 120

revolutions per minute in vacuum whith a consumption Fig. 12. H bridge module
of 60 mA. The motor shaft is 6n mm and fits perfectly
with the diferent caps and adapters of robot wheels. 2) Software:
Great reliability of this engine in any application. The
cost is $8.  IDE Arduino Software: This software is part of the
Arduino Create platform, which enables dynamic use
functions, such as the ability to store the code written in
the cloud with the intention that it can be accessed from
anywhere.
 4

{
digitalWrite(IN1,LOW);
digitalWrite(IN2,HIGH);
digitalWrite(IN3,LOW);
digitalWrite(IN4,HIGH);
Serial.println("adelante");
}
if(val=='1')
{
digitalWrite(IN1,HIGH);
digitalWrite(IN2,LOW);
digitalWrite(IN3,HIGH);
digitalWrite(IN4,LOW);
Fig. 13. IDE Arduino Serial.println("atras");
}
ARDUINO CODE if(val=='2')
{
Bluetooth connection digitalWrite(IN1,HIGH);
digitalWrite(IN2,LOW);
digitalWrite(IN3,LOW);
#include <SoftwareSerial.h> // libreria que permite establecer
digitalWrite(IN4,HIGH);
pines digitales
Serial.println("derecha");
// para comunicacion serie
}
if(val=='3')
SoftwareSerial miBT(10, 11); // pin 10 como RX, pin 11 como
{
TX
digitalWrite(IN1,LOW);
digitalWrite(IN2,HIGH);
void setup(){
digitalWrite(IN3,HIGH);
Serial.begin(9600); // comunicacion de monitor serial a 9600
digitalWrite(IN4,LOW);
bps
Serial.println("izquierda");
Serial.println("Listo"); // escribe Listo en el monitor
}
miBT.begin(38400); // comunicacion serie entre Arduino y el
if(val=='4')
modulo a 38400 bps
{
}
digitalWrite(IN1,LOW);
digitalWrite(IN2,LOW);
void loop(){
digitalWrite(IN3,LOW);
if (miBT.available()) // si hay informacion disponible desde
digitalWrite(IN4,LOW);
modulo
Serial.println("para");
Serial.write(miBT.read()); // lee Bluetooth y envia a monitor
}
serial de Arduino

if (Serial.available()) // si hay informacion disponible desde el CONNECTION

monitor serial
miBT.write(Serial.read()); // lee monitor serial y envia a
Bluetooth

Motor Control

int IN1=8;
int IN2=7;
int IN3=6;
int IN4=5;
char val;
void setup() {
pinMode(IN1,OUTPUT);
pinMode(IN2,OUTPUT);
pinMode(IN3,OUTPUT);
pinMode(IN4,OUTPUT);
Serial.begin(9600);
Fig. 14. Connection
}
 Phyton
void loop() {
if(Serial.available()>0)
{ IV. IMAGE PROCESSING AND CONTROLLER
val=Serial.read();
}
if(val=='0')
 5

A. IMAGINE PROCESSING VI. CONCLUSION

 The control of the cart by means of the microcontroller
facilitates the communication between Phyton and the
B. CONTROLLER bridge H with respect to other microcontrollers.
The control of the circuit implemented and explained above is  The data transmitted by means of bluetooth to the cart
controlled by an ARDUINO UNO microcontroller, which receives helps to have more agility in the control of the car as it
the data of the movements of the computer. Once the data is prevents jacking and helps the cart to move better.
obtained, the Arduino connects to a bridge H, receives a one or a 2 conclusiones mas
zero depending on the movement to be made.
VII. REFERENCES
Once the data is obtained, the H bridge activates its pins and
sends its voltage to supply power to the motors. To verify the [1] S. Gupte, P. Teenu & J. Conrad, “A Survey of Quadrotor
operation of the implemented circuit, we proceeded to download a Unmanned Aerial Vehicles”, Proceedings of IEEE
PUTTY program through which we sent the data for the control of Southeastcon, 2012
the car. [2] J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd
ed., vol. 2. Oxford: Clarendon, 1892, pp.68-73.
Checked the operation of the cart, will connect to Phyton through [3] «Csiro,» [En linea] .https://www.csiro.au/en/ [Ultimo acceso :
a serial library, this will allow the transmission of data to the 05 05 2019]
arduino. [4] Z. Yang, F. Ling & B. Chen, “Survey of Autopilot for Multi-
rotor Unmanned Aerial Vehicles”, IECON 2016 - 42nd Annual
The data received in the arduino is the turn that the cart will Conference of the IEEE Industrial Electronics Society
make, and a distance of 1m will be placed so that at that moment it [5] D. Garijo, J. López y I. Pérez, “Control de un vehículo aéreo no
stops and the camera captures again the place where it is required to tripulado”, Madrid, 2009.
arrive. This control uses a reinforcement learning type Decision tree. [6] M. Lundgren, “Path Tracking for a Miniature Robot”, 2003.M.
[7] P. Bajaj, «GeeksforGeeks,» [En línea]. Available:
V. TEST AND RESULTS https://www.geeksforgeeks.org/what-is-reinforcement-learning/.
[Último acceso: 05 05 2019].
[8] Gosavi A, “Reinforcement learning: A tutorial survey and
The vehicle coordinate system is defined where the y-axis is in recent advances”. Informs J Comput 21(2):178–192
the forward direction of the vehicle, the z-axis is down, and the x- [9] S. Bouabdallah, P. Murrieri and R. Siegwart, “Design and
axis forms a right-handed coordinate system. Therefore, all Control of an Indoor Micro Quadrotor”, pp 4393-4398, New
coordinates used must first be transformed to vehicle coordinates for Orleans, Proceedings of IEEE International Conference on
the algorithm to work properly. Robotics and Automation, April 2004
[10] D. Verscheure, B. Demeulenaere, J. Swevers, J. De Schutter,
This can also be compared to the way humans drive their cars. We and M. Diehl “Time-Optimal Path Tracking for Robots: A
usually look at some distance in front of the car and pursue that Convex Optimization Approach”, IEEE TRANSACTIONS ON
spot. Assuming that the points that constitutes the path and the AUTOMATIC CONTROL, vol. 54, no. 10, October 2009
successive positions of the robot vehicle belongs to the same [11] F. Pfeiffer and R. Johanni, “A concept for manipulator
coordinate system, the algorithm can be described by these simple trajectory planning”, IEEE J. Robot. Autom., vol. RA-3, no. 2,
steps: pp. 115–123, Apr. 1987
[12] H. Choset, W. Burgard, S. Hutchinson, G. Kantor, L. E.
1. Obtain current position of the vehicle Kavraki, K. Lynch, and S. Thrun, “Principles of Robot Motion:
2. Find the goal point: Theory, Algorithms, and Implementation”. Cambridge, MA:
2.1. Calculate the point on the path closest to the vehicle MIT Press, Jun. 2005
(xc, yc)
2.2. Compute a certain look-ahead distance D
2.3. Obtain goal point by moving distance D up the path
from point (xc,yc)
3. Transform goal point to vehicle coordinates
4. Compute desired curvature of the vehicle γ = 2Δx/D2
5. Move vehicle towards goal point with the desired
curvature
6. Obtain new position and go to point 2.

Time in a straight line:

Time between current position and objective:

Medium squaered error to the reached position: