A UAV-Cloud System for Disaster Sensing

Applications

Chunbo Luo, James Nightingale Ekhorutomwen Asemota Christos Grecos

AVCN Research Centre, Technology Services Independent image consultant,
School of Eng. & Computing, UWS Capgemini Glasgow, UK
Paisley, UK UK christos.grecos@gmail.com
{chunbo.luo, ekhor.asemota@capgemini.co.uk
james.nightingale}@uws.ac.uk

Abstract—The application of small civilian unmanned aerial Although UAV-based projects represent the cutting edge in
vehicles (UAVs) has attracted great interest for disaster sensing. disaster management technology, there are several existing
However, the limited computational capability and low energy more traditional disaster observation and monitoring
resource of UAVs present a significant challenge to real-time approaches, such as the use of ground observation vehicles,
data processing, networking and policy making, which are of satellites and man-piloted airplanes. Ground observation
vital importance to many disaster related applications such as oil- vehicles can respond quickly but their coverage and
spill detection and flooding. In order to address the challenges geographical accessibility are limited. Additionally their view
imposed by the sheer volume of captured data, particularly video of an area may also be restricted by the presence of buildings,
data, the intermittent and limited network resources, and the
debris or natural features [4]. Satellites provide large coverage
limited resources on UAVs, a new cloud-supported UAV
and fair resolution, but accuracy and the time taken to position
application framework has been proposed and a prototype
system of such framework has been implemented in this paper. them before data can be acquired may restrict their usefulness
The framework integrates video acquisition, data scheduling, in time-critical disaster situations [5]. Man-piloted airplanes
data offloading and processing, and network state measurement can react quickly and cover large areas but are not suitable for
to deliver an efficient and scalable system. The prototype of the applications where there is a high risk to human beings (e.g.
framework comprises of a client-side set of components hosted on radioactive disasters) or where repeated/prolonged coverage is
the UAV which selectively offloads the captured data to a cloud- needed, they are also costly to maintain. Other specialized
based server. The server provides real-time data processing and monitoring systems designed for specific disasters do exist but
information feedback services to the incident control centre and are not discussed due the space limitation of this paper.
client device/operator. Results of the prototype system are
presented to demonstrate the feasibility of such framework. UAVs appear to be a potentially promising choice that can
enhance traditional disaster surveillance systems and overcome
Keywords—UAV; cloud; scheduler; wireless network; video the limitations of traditional methods. In particular, small
communication civilian UAVs have the advantages of good coverage,
downward facing high definition cameras, low costs and fast
deployment. However, the limited onboard battery power and
I. INTRODUCTION
computation processing capability of small UAV’s can limit
Disasters, whether caused by nature or man-made actions, their effectiveness. In order to address these challenges, a
have had a devastating impact on the quality of human life recent study proposed the idea of integrating computing cloud
across the world. For example, global climate change and infrastructure with a UAV system [6]. Another enhancement
extreme weather have resulted in more frequent and severe proposed in [3] is UAV cooperation so that the work load can
flooding events in both the UK and worldwide. Statistics from be shared by multiple UAVs which improves efficiency and
the United Nations Office for Disaster Risk Reduction show increases capability. These innovative techniques have inspired
that there were 3455 major flood disasters between 1980 and two questions. Firstly how to effectively utilize the large
2011. The data contained in their report also highlights a volume of data generated by UAVs during disaster
steeply rising trend in the frequency of major flood events [1]. management operations and secondly the overall design and
Similarly, large scale man-made disasters have also occurred implementation of a UAV-cloud framework. This paper aims
frequently in recent years, e.g. oil spill accidents [2]. In order to to fill this gap in knowledge by proposing a novel service-
deal with the challenges posed by such events, researchers in oriented computing framework which forms a UAV-cloud
both academia and industry have increasingly turned to the use system.
of UAVs which have been shown to provide large area
coverage and real-time high-quality data acquisition During the recovery period from natural disasters,
capabilities, as well as have low capital costs and fast telecommunications infrastructures may be destroyed or
deployment time [3]. severely impaired, requiring UAV communications to be
routed over ah-hoc networks provided by the tactical radio

This research is sponsored by the RCUK Digital Economy Theme

Sustainable Society Network+ and Royal Society-NSFC Grant No. IE131036.

978-1-4799-8088-8/15/$31.00 ©2015 IEEE

 Fig. 1. The UAV cloud framework.

networks of civilian or military responders. In such challenging computationally-intense signal processing algorithms for object
deployments, these tactical networks often suffer from identification, and communicates the outputs from advanced
impairments such as lost, intermittent or limited connectivity processing services to the control centre.
and are typically described as disconnected, intermittent and
limited (DIL) environments. The rest of this paper is organized as follows: Section II
describes disaster sensing scenarios and challenges; Section III
Generally, the introduction of cloud computing can introduces design goals and assumptions; Section IV provides
overcome the resource limitations of UAVs via data an overview of the whole framework; Section V presents the
offloading. However, the offloading process is not without cost prototype system; Section VI discusses the current design and
as, for example, offloading large volumes of data can seriously future developments, and Section VII concludes this paper.
drain the energy resource of the UAV and consume a large
proportion of the available network bandwidth. Existing work II. PROBLEM STATEMENT AND CHELLENGES
on mobile cloud for smartphone applications has proposed the
use of customized algorithms to determine the strength and The deployment of small UAVs for collecting critical
availability of network infrastructure before offloading [7]. information in disaster areas has been an innovative but
These algorithms have inspired the design of the UAV-cloud challenging task. Specific challenges include
framework in this paper which describes the key UAV, • Ground areas may have been severely changed or
computing cloud and network components of the proposal disrupted by the disaster. The infrastructure may have
scheme. The need for network state measurement is even more been damaged or destroyed. Disaster scenarios may also
acute in DIL tactical network environments than in mobile be highly unpredictable and complex.
networks supported by robust and redundant commercial
infrastructures and is an important component of the proposed • UAVs have battery constraints which substantially limit
scheme. their flying time, communication capability and
computational capacity.
A client-server configuration delivers two broad sets of
techniques. The client is hosted on the UAV, which collects • The UAV may be required to communicate over DIL
video data and context information from its hardware. It also tactical network environments where network state
has a context-aware video scheduler that selectively offloads awareness must be factored into any algorithm which
captured data to the cloud based on contextual information. decides when data can be transmitted to the cloud-based
The server, hosted within the cloud infrastructure, listens for server.
incoming traffic from the client. When meaningful data is
received, the server provides valuable services such as • The overall disaster sensing task has stringent
requirement on timeliness. For example, the first task
 following a major disaster is usually search and rescue, second is used as a representative sampling rate, thereby
which would require image/video information collected reducing the amount of data offloaded to the cloud.
and processed in a timely fashion, preferably in real
time. • Sufficient UAV onboard storage. The storage should be
large enough to store all video data captured during the
• The mobility pattern of UAVs directly affects the whole deployment time. This should be easily attainable
collected data and the performance of image processing given current solid state drive technologies.
algorithms. Specifically, the quality of acquired data
may vary during the flying time. If all data were to be • Adequate wireless communications between UAV,
transmitted to the base station, unnecessary power and wireless base station, cloud and the control centre. The
bandwidth would be consumed. installation of pre-processing algorithms on the UAV
can filter out a large number of unnecessary frames,
• Object detection and other disaster applications require with the remaining critical frames reliably transmitted
high quality image and video information. UAVs to the cloud. Commands from the control centre should
should be able to adjust their flight patterns to optimise also be transmitted to the UAVs reliably.
data quality.
• Performance enhancement through cloud computing.
• Intelligent video capturing and processing tools can The performance and scalability of the cloud and thus
significantly improve the quality of decision making in the whole system can be enhanced with reasonable
disaster scenarios. effort.
Similar challenges can be found in mobile communication
networks where mobile devices are restricted by their power IV. UAV SENSING FRAMEWORK ARCHITECTURE
and computational capability. Researchers have developed The block diagram displayed in Fig. 1 provides a higher
joint smartphone and cloud systems to deal with such problems level view of the framework. From the design perspective, it
by applying mobile cloud computing and virtualization [7]. can be split into two categories: client and server. The modules
Initial results have indicated significant reductions in energy within the client mainly include a context collector, a context-
consumption of mobile phones and reduced processing times aware scheduler, and a video capturing and pre-processing unit.
using mobile cloud. One study [8] suggests that it is not always The server hosted in the cloud mainly manages video
efficient to connect to a cloud. The deployment of UAVs in retrieving, object detection, other data mining algorithms and
disaster management has much higher complexity than using data storage. It provides an interface through which the control
smartphones on mobile networks. Additional factors such as centre can access processed information, aiding decision
time, flight path, task management and data processing, all making by human operators. The client and server are
require specific innovative solutions. connected through wired and wireless networks.
Inspired by the aforementioned studies, this paper proposes
a UAV-cloud framework to address the challenges within A. Client components
disaster sensing scenarios. To further enhance the framework, a The three major modules of the client work as follows. The
context-aware scheduler which systematically adjusts video video capture and pre-processing unit collects video data,
and data transportation based on a pre-processing algorithm stores them in the on-board hard disk, and carries out simple
forms part of the design. Fig.1 shows the block diagram of the pre-processing. The video data is then fed to a context-aware
whole system. To the best of our knowledge, no similar scheduler that also receives data from a context collector.
innovative framework has been previously proposed or Critical UAV system information including battery level,
designed. mobility, and location information, as well as key information
received from the control centre including video spatial &
III. DESIGN GOAL AND ASSUMPTIONS temporal data and control commands etc. are collected and
passed to the scheduler. Armed with contextual awareness and
The design of the proposed UAV-cloud framework is based network state awareness, the scheduler can make intelligent
on the following assumptions which are already true or can be decisions about video data transportation by considering all of
implemented with reasonable effort: the available contextual information. A more detailed
• Efficient on-board video pre-processing algorithms. In explanation of each component’s functionality is given in
the prototype system, the image histogram distribution Section V where the prototype implementation is outlined.
metric is used to distinguish between frames for a
specific mission. As a relatively cost-efficient image B. Server components
processing technique, histogram distribution is The server components are hosted in a computing cloud
calculated for every captured video frame in the which supports virtualization and scalable resource allocation
implementation. More advanced algorithms can be in order to satisfy power hungry, computationally-intense data
researched and implemented in the framework for processing requirements. The main modules of the server
improved performance. include a video retrieval unit and data storage algorithms which
underpin a range of data mining and signal processing services.
• Frame-based video data segmentation and
The server listens for client information and retrieves video
communication. In the prototype system, one frame per
data for deep processing with results (e.g. detected potential
targets), sent to the control centre for evaluation by a human
operator. The cloud also provides scalability and easy-
configuration features such as the update of post-processing
algorithms.

C. Communications
The client, server and control centre are supported by two
types of networks: wireless and wired connections. UAVs and
the control centre are connected by a wireless base station. The
control centre has wired/wireless connection to the cloud which
exchanges data with the wireless base station through a wired
network. In each case data may, in disaster recovery situations,
be transmitted over tactical radio networks exhibiting DIL
environment characteristics. Whilst not explicitly considered in
this pilot study, the choice of communications model may suit
a service oriented environment where the reliability of
communication is achieved through a web service middleware.
Fig. 3. A screenshot of the processing results at the client side.

V. PROTOTYPE SYSTEM IMPLEMENTATION Firstly, the scheduling algorithm checks the UAV status
In order to test the proposed framework and demonstrate its information and ensures it has enough residual power to
design features, a prototype of the proposed architecture has transmit data. If this is true, it then checks the mobility pattern
been fully implemented. A picture of the UAV is shown in Fig. of the device to ensure the acquired data is of sufficient quality
2 and a screenshot of the client processing result is shown in or contains objects of interest. The third step of the algorithm is
Fig. 3. to compare the histogram distribution of the current frame with
those computed on previously acquired frames of the same
mission. If a significant difference is detected, the network
state information is evaluated to determine if it will be possible
to transmit the frame within a timeframe which would make it
useful for real-time evaluation. If there is sufficient network
availability and capacity, the frame will be sent to the cloud for
further processing.
The server components are hosted in a virtual machine
supported by a computing cloud. The video retrieval unit
constantly listens for incoming data and reconstructs the video
data prior to passing it to the post-processing algorithm. In the
pilot implementation, the post-processing component consists
of a single service designed to detect oil spillage in marine
environments [9]. The server also stores received video data for
future processing by other services that may be requested by
operators or incident controllers when additional information is
Fig. 2. 3DR Hex-Rotor UAV Platform. needed to support a command decision. The server also
provides an access interface to the control centre. The pilot
The video capture unit of the client records video data implementation described in this paper provided limited server
which is stored on an on-board hard disk. It then selects one functionality. Future research will investigate more accurate
frame from each frame set (a pre-defined sampling period e.g. detection algorithms, communication protocols (between
every nth frame) captured and sends it to the pre-processing control centre and UAV), automatic annotation techniques of
unit. The pre-processing algorithm calculates the histogram historical UAV data (to facilitate further data mining) and
distribution of the image. This current histogram image is then implement the most suitable ones for deployment. The final
sent to the context-aware scheduler. implementation may also create a service oriented middleware
to manage services available to operators and incident
Simultaneously, the context collector polls the on-board commanders. A screenshot of the server side processing results
sensors and systems to collect UAV real-time mobility is shown in Fig. 4. In order to show the design features of the
patterns, battery power level, spatial coordinates and network framework and the efficiency of this architecture, an
state metrics. experiment using practical UAV video data has been conducted
within the current prototype system. We compared the number
These data are fed into the context aware scheduler together
of video frames transmitted from the UAV to the cloud before
with the output data from the video pre-processing stage. The
and after the adoption of the framework. Fig.5 shows the result.
scheduler then determines if the frame should be sent to the
From the figure, we can see that approximately only 10% of
cloud for more rigorous analysis or not.
raw video data was sent and processed by the scheduler, which
demonstrates the efficiency of the proposed pre-processing and be improved in terms of efficiency and detection performance.
context aware approach. Future work will enhance these algorithms, test the scalability
of the cloud and improve the overall framework.
Disaster scenarios are usually unpredicted and cover a large
geographical area. To support emergency services would
require coordinated actions from multiple aerial vehicles and/or
ground robotic systems. The proposed UAV-cloud framework
will be extended for such application scenarios and include the
essential multiple-agent cooperation and data sharing
mechanisms.

VII. CONCLUSION
This paper proposes a UAV-cloud framework for disaster
sensing applications under the condition of DIL networks. Its
major components include client units hosted by the UAV
onboard system and server units hosted by the remote
computing cloud infrastructure which provides service-oriented
resource support. In order to save energy and improve disaster
Fig. 4. A screenshot of the video results at the server side. sensing performance, the onboard client filters acquired video
data and only offloads those frames that are essential to the
cloud for advanced processing. The cloud runs sophisticated
4
10 power-hunger algorithms, e.g. object detection, and reports to
the control centre. The proposed framework is suitable for the
scenarios which have a large amount of video data requiring
3
10
real-time or close to real-time processing which is essential in
disaster related applications.

REFERENCES
Frames

2
10

[1] UNISDR Disaster Statistics. [Online]. Available: http://www.unisdr.org/

we/inform/disaster-statistics
1
10
[2] Fingas, Merv, and Carl Brown. "Review of oil spill remote sensing."
Marine pollution bulletin 83.1 (2014): 9-23.
UAV Raw Video [3] S. Cameron, S. Hailes, S. McClean, and et al., “Suaave: Combining
Server Processed Video aerial robots and wireless networking,” in SUAAVE. University of
0
10 Oxford, University College London, University of Ulster, Feb. 2010, pp.
0 1000 2000 3000 4000 5000
Iteration
6000 7000 8000 9000
1 – 14.
[4] Grocholsky, Ben, et al. "Cooperative air and ground surveillance."
Robotics & Automation Magazine, IEEE 13.3 (2006): 16-25.
Fig. 5. Comparing of the change of the number of frames through the [5] Son, Byungrak, Yong-sork Her, and Jung-Gyu Kim. "A design and
adoption of the proposed framework. implementation of forest-fires surveillance system based on wireless
sensor networks for South Korea mountains." International Journal of
Computer Science and Network Security (IJCSNS) 6.9 (2006): 124-130.
VI. DISCUSSION AND FURTHER WORK [6] Michael Cochez, Jacques Periaux, Vagan Terziyan, Kyryl Kamlyk, Tero
Tuovinen, “Evolutionary Cloud for Cooperative UAV Coordination”,
The current prototype system does not address the impact Reports of the Department of Mathematical Information Technology,
of network impairments, which is important to the performance Series C. Software and Computational Engineering, No. C 1/2014.
of the whole system. For example, the scheduler should also [7] Kosta, Sokol, et al. "Thinkair: Dynamic resource allocation and parallel
include network state metrics in its decision on transmission. If execution in the cloud for mobile code offloading." INFOCOM, 2012
the wireless channels between the UAVs and base station can Proceedings IEEE. IEEE, 2012.
support high data rates, higher definition video data and more [8] Namboodiri, Vinod, and Toolika Ghose. "To cloud or not to cloud: A
mobile device perspective on energy consumption of applications."
frames can be sent to the cloud. If the channel quality is World of Wireless, Mobile and Multimedia Networks (WoWMoM),
stringent, only critical frames and messages should be 2012 IEEE International Symposium on a. IEEE, 2012.
exchanged in the network. [9] Mengmeng Di, Huajun Song, Chunbo Luo and Peng Ren. "Elongated
Strip Oil Spill Segmentation Based on A Cooperative Model.", 2014
Both the algorithms for pre- and post-processing, hosted in IEEE International Conference on Multisensor Fusion and Information
the UAV and cloud respectively, are fairly simple and should Integration, 2014.