Lu Cse 19 003
Lu Cse 19 003
Abstract—The paper is a literature survey of RTOS (Real-Time Operating System) development. GPOS (General Purpose Operating
System) existed before RTOS but did not meet performance requirements for time sensitive systems. Many GPOS have put forward
adaptations to meet the requirements of real-time performance, and this paper compares RTOS and GPOS and shows their pros and
cons. Furthermore, comparisons among select RTOS such as VxWorks, RTLinux, and FreeRTOS have been conducted in terms of
scheduling, kernel, and priority inversion. Various tools for WCET (Worst-Case Execution Time) estimation are discussed. This paper
also presents a use case of RTOS, i.e. JetOS, and future advancements such as multi-core RTOS, new RTOS architecture and RTOS
security.
Index Terms—Real-time operating system, General purpose operating system, RTLinux, FreeRTOS, VxWorks, multi-core RTOS.
1 I NTRODUCTION
RTOS (Real-Time Operating System) is used in a wide This survey of RTOS covers related technologies and
range of industrial systems, such as process control sys- gives an introduction of them. It also includes select topics
tems, avionics, and nuclear power plants. Most real-time including new adaptations on solving priority inversion, a
operating systems run on embedded systems consisting of newly purposed platform for WCET (worst-case execution
pieces of hardware that work as controllers with dedicated time) tool development, and a RTOS that supports multi-
functions within mechanical or electronic systems. RTOS is core processing.
critical for those mechanical or electronic systems with real- This paper starts with a comparison between GPOS and
time requirements because systems could not operate safely RTOS, and their advantages and disadvantages for general
without it. In many real-time systems a missed deadline can computing and for real-time systems. Section 2 discusses
lead to disastrous consequences. different techniques used to handle task scheduling and
solve priority inversion. Section 3 provides a comparison
1.1 Hard Real Time vs. Soft Real Time between three RTOS implementations, namely VxWorks,
RTLinux, and FreeRTOS. This comparison covers kernels
Hard real time systems and soft real time systems are of the operating systems, schedulers used, and how they
both used in industry for different tasks [15]. The primary solve priority inversion. The next section discusses WCET
difference between hard real time systems and soft real time analysis tools used today in industry along with the devel-
systems is that their consequences of missing a deadline dif- opment on a new platform to build and compare these tools.
fer from each other. For instance, performance (e.g. stability) Section 5 provides a use case of a recently developed RTOS,
of a hard real time system such as an avionic control system i.e. JetOS in avionics. The final section of this paper dis-
or a nuclear power plant, is dependent on the timeliness cusses future developments of RTOS systems ranging from
of the operation results and the correctness of the results. a multi-core RTOS (HIPPEROS) to a new RTOS platform
However, for soft real time systems such as a multimedia (HERCULES).
on-demand system, their performance is solely dependent
on the results. A hard real time system is used in systems
that are time sensitive and must meet their deadlines in 2 GPOS VS RTOS
order to avoid disaster. However, deadlines in soft real time
A general comparison between RTOS and GPOS is provided
system are less strict so that if they miss their deadlines it
below to show the advantages and weaknesses of these
does not result in disaster. Part of this is because hard real
operating systems when used for real time system appli-
time systems do not have an easy way to recover from a
cations.
failure; where as a soft real time system can be time-elastic.
Some of the advantages of a hard real time system is that
it has a faster reaction time than soft real time systems. The 2.1 Task Scheduling
hard real time system also has predefined deadlines and The first part of differences is the way where GPOS and
predictable performance, where the soft real time systems RTOS perform their task scheduling. Generally speaking,
may experience degraded performance. a GPOS utilizes a fairness policy that allows all processes
to share the processor. This hinders the GPOS ability to
• A. Serino was with a REU program hosted by the Department of handle time sensitive tasks because it cannot guarantee
Computer science and engineering at Lehigh University.
E-mail: serinot@yahoo.com task dispatch latencies. The RTOS uses a priority based
• L. Cheng is with the Department of Computer science and engineering at preemptive scheduling system. This enables a high priority
Lehigh University. E-mail: cheng@lehigh.edu task to take the processor from a lower priority task, and
allows the high priority task to run without interruption.
2
2.1.2 Kernel
Generally the GPOS does not support preemption. Preemp-
tion is when processes and threads with higher priorities can
take the processor from lower priority tasks. By not allowing
preemption the GPOS suffers when trying to complete a
task that is time sensitive as it can make the task wait
and thus miss its deadline. The GPOS is unable to cancel
system calls even if they are from a lower priority task and
create unpredictable delays. The advantages of its kernel
are in its support for widely used application programming
interfaces (APIs) and customizable operating system com-
Figure 2. FIFO Scheduling ponents for application-specific demands.
The RTOS fully supports preemption where it imposes
an upper bound on how long the preemption has its inter-
Rate-monotonic scheduling. This scheduling method is rupts disabled. This allows a high priority task to run to
a static-priority algorithm that sets the priority level for completion without interruption and thus let time sensitive
each task in the order of their period information. Short tasks meet their deadlines. The kernel of a RTOS try to use
period tasks execute frequently, and a long period refers the least amount of resources possible. To keep it simple
to infrequent execution. Short period tasks are given higher only services with short execution paths are allowed in the
priorities while long period ones are given low priorities. kernel, and its process loading happens outside along with
This lets high priority tasks run first, and it is best used its file systems. This architecture makes it so that if one of
when there are well defined periodic tasks with the same these systems fails it does not corrupt other services or the
CPU run time length [4] [5]. kernel. The benefit of the RTOS kernel is that there is only a
3
small core of fundamental operating system services in the 2.3 Modified GPOS
kernel. These fundamental services are signal, timers, and A modified GPOS is the result produced by people who
the scheduler [14]. have tried to adapt and change GPOS to have the same
capabilities of both GPOS and RTOS. It has a major ad-
2.2 Priority Inversion vantage in that if it works like a GPOS it can be used
Priority inversion occurs when two tasks of different more widely for a greater amount of tasks, and if it has
priorities share a resource and the higher priority task the capabilities of an RTOS then it supports tasks for time
cannot get the resource from the lower priority task. This critical systems. For example, Linux 2.6 added preemption
is an issue for time sensitive tasks as it can prevent it from [11]. Generally when a GPOS is directly modified to support
meeting its deadline by forcing it to wait for a resource. RTOS functionality high-resolution timers will be used. This
This needs to be fixed as it can result at best in blocking, but modification enables the process scheduler to make the
can also result in chain blocking or worse a full deadlock. system more reactive and event-driven. However, Linux 2.6
There are a couple of protocols to solve priority inversion. is not as fast as other RTOS and the low latency patches for
the GPOS timers do not solve the priority inversion issue
[14].
2.2.1 Priority inheritance Another way people have tried to improve GPOS is by
The priority inheritance protocol is where if a higher introducing a new architecture called dueling kernels where
priority process is waiting for a lower priority process the GPOS is ran on top of an RTOS [14]. This architecture
for a resource the process scheduling algorithm gives a sends real time tasks to run on the RTOS, with a higher
higher/highest priority to the lower priority task on the priority than other tasks running on the GPOS. The RTOS
processor so that it cannot be preempted by a different task gives these tasks the ability to preempt the tasks on GPOS,
until it completes the execution of its critical section. For and then gives the CPU back to the GPOS when it finishes
example, there is a process A on the processor of priority running the high priority tasks. This system has an issue
2 and there is a higher priority process B that could not where the tasks running on the RTOS have limited use of
preempt it at priority 7. The process scheduling algorithm the GPOS services due to preemption issues. This causes
would assign process A priority 9 temporarily so that it RTOS to recreate services that exist in the GPOS. RTOS tasks
finishes its critical section on the processor without being also cannot use the memory management unit which is used
interrupted by other processes which would delay process by the GPOS for non-realtime processes. GPOS services that
B further. This allows process B to get access to the resource are ported often have different vendor extensions that do
as fast as possible, and once the resource is freed by process not work with other vendor’s extensions.
A the process scheduling algorithm reverts process A back
to its original priority of 2 [2].
3 V X W ORKS , F REE RTOS, RTL INUX
The RTOS chosen for comparisons in this paper are some of
2.2.2 Priority ceiling the industries’ top competing RTOS. VxWorks and RTLinux
The priority ceiling is where each resource is assigned a have been extensively compared to each other through
priority ceiling where the priority is equal to the highest research due to the continuing development on both the
priority of any task which may lock the resource. This commercially available VxWorks and the free RTLinux.
works by temporarily raising the priority of tasks in certain FreeRTOS being a more recent RTOS compared to VxWorks
situations. Basically if process A tries to preempt the critical and RTLinux has seen little comparisons with either Vx-
section of another process to execute in its own critical Works or RTLinux directly. Important aspects to compare
section Z; then the priority of the new section should be them include their kernels, schedulers, and how they handle
higher than the priority of inherited properties of all the priority inversion.
preempted sections. If this fails then process A is denied
entry into the critical section and is suspended [2]. 3.1 Kernel
3.1.1 RTLinux
2.2.3 Other priority inversion solutions RTLinux has a special design for its kernel because it has
Priority remapping. As an improvement to the priority in- two kernels. RTLinux uses a specialized real time kernel
heritance protocol, the priority remapping method expands called the RTCore [4]. The second kernel is the standard
the highest priority from 64 to 128 without change its inter- Linux kernel which is used for regular applications that
face. This means that users still only see 64 priorities, but its do not have time constraints. Both interrupt handling and
internal task creation is multiplied by 2. Its internal priority thread handling are controlled by RTCore. The RTCore will
is extended to (0,2,4,...,126,128) even priority, where the send these interrupts to the appropriate interrupt handler.
odd priorities are left for changing when priority inversion This RTCore also restricts the Linux kernel by making
happens. it unable to disable interrupts to make sure it does not
Priority exchange. This method is also an improvement interfere with process scheduling. Thus the Linux kernel
on priority inheritance protocol. It swaps the tasks priorities can only run when there is a task that is not real time.
when a higher priority task is blocked by a lower priority Real time applications can communicate with Linux kernels
task. The priorities will be swapped back after the critical through first-in-first-out pipes. This enables the RTCore
section finishes running the lower priority task [3]. API threading that helps programmers learn to program
4
on this system. The duel kernels gives RTLinux the full 3.2 Scheduler
functionality of Linux, while adding real time capacity to it
The scheduler of the RTOS is an important part of how
[6].
an RTOS decides the next task to be put on the processor,
and to make sure that all tasks meet their deadlines. This
section of the paper will discuss similarities and differences
between the schedulers used by RTLinux, VWorks, and
FreeRTOS.
3.2.1 RTLinux
RTLinux has a flexible scheduler by allowing different
scheduling techniques to be used based off the programmers
needs. The RTLinux scheduler supports FIFO scheduling,
EDF scheduling, and rate-monotonic scheduling. This
flexibility allows for better suited schedulers for different
systems [5].
4.1 AbsInt aiT The aiT tool is a popular tool for specific processors. The
AbsInT aiT is a worst-case execution time analysis tool. OTAWA framework should allow for newly developed
This tool solves the cache and pipeline issue by statically WCET analysis tools to be compared with existing WCET
analyzing the task’s cache and pipeline behavior. This analysis tools, and help the further development of WCET
enables it to get the correct upper bounds of WCET of the tools by providing a standard framework and C++ library
task. This tool uses the technique of abstract interpretation, [17].
and offers a graphical user interface to visualize the WCET
path, and allows for an interactive way to inspect pipelines 5 A U SE C ASE OF RTOS
and caches [18]. The abstract interpretation is a method Avionics is a good example where real time systems are
uses a semantic based method for safe and static program used. There has been recent development of a specific
analysis. The overall method applied by AbsInT aiT breaks type of RTOS, i.e. JetOS, to fully meet the ARINC653
it into multiple steps to find the WCET. international standard for aircraft usage. JetOS originates
from POK RTOS, an open source project, which partially
1. Reconstruction of the control flow meets the ARINC653 standards. The critical parts that have
2. Value analysis: computation of address ranges for been reworked or added to meet the standard include
instructions access memory pok’s scheduler, network stack, memory manager, added
3. Cache analysis: classifies memory references as cache hits separate memory, and a reduction of the kernel size for less
or misses errors. The system uses ordinary partitions which separates
4. Pipeline analysis: predicts the behavior of the program memory, and system partitions are used to utilize services
on the processor outside of the ARINC653 standard. Both of these types
5. Path analysis: determine the worst case execution path of partitions from the kernel point of view are the same.
6. Analysis of loops and recursive procedures Currently there is a working prototype of JetOS and it is in
its testing stages [13].
Steps 2, 3, and 4 are done with abstract interpretation;
path analysis is done using integer linear programming [18]. The kernel of JetOS drops POK’s AADL (Architecture
Analysis and Design Language) configuration tools for
4.2 Bound-T Tool XML based configuration files. Furthermore it dropped
the SPARC platform in favor of building JetOS on top of
The Tidorum Ltds Bound-T tool is software used for static
a platform that other avionic systems use. The platform
analysis of code to estimate its WCET and stack usage for
chosen to replace it was the x86 and powerPC. The kernel
embedded systems. This tool in its current state is not going
is built to support multiple schedulers because different
through further development, and is currently open source.
partitions can utilize different schedulers, and is configured
The Bound-T tool uses the same static analysis approach as
statically where the number of partitions, partition memory
that used by AbsInT aiT. The Bound-T tool was developed
size, port, names, etc. cannot be changed [13].
for local and safe analysis of simple control flows. This
system used a PA (Presburger Arithmetic)-based analysis
Each partition may have one or more processes. Parti-
where an approximate model of computer arithmetic is
tions are scheduled based on a round-robin algorithm. Intra-
used, which assumes that integer variables never overflow
partition schedulers implement lock-wait-unlock and pri-
or wrap around. However this approximation leads to an
ority scheduling. The resources are pre-allocated to ensure
issue of not being able to find a feasible execution path. In
reliability. The memory is pre-allocated to every partition.
order for Bound-T to start getting back on track to finding
These partitions are scheduled differently than kernel mod-
WCET for embedded systems it either has to drop the PA-
ules where partitions are run in user mode with time and
based analysis, or find a new form of preliminary analysis
space constraints [13].
to ensure that the PA analysis can be applied [16].
6 F UTURE D IRECTIONS
4.3 OTAWA Toolbox
With the rising need for both faster computing speed and
The OTAWA toolbox is a proposed WCET analysis tool
better resource usage, multi-core computing has existed in
framework that allows it to host researchers WCET algo-
general-use computers for a number of years. However,
rithms and includes an abstraction layer that separates the
RTOS have been using a mono-core system that is becoming
hardware analysis from the instruction set architecture. This
obsolete in terms of speed and resource usage. A multi-core
framework should also allow for the comparison between
RTOS as new architecture for embedded systems aims at
tools to be easier to do, and support the development of
providing a major increase in both speed and resource usage
new tools. The OTAWA toolbox was used in the MERASA
including energy efficiency [10].
project to help find the most optimal WCET analysis tool
for a multicore processor running mixed-critical workloads
[17]. 6.1 Multicore Real-Time Operating System
In summary, the aiT tool and the Bound-T tool both A challenge in developing multicore RTOS is how to
use static analysis to find the WCET. The difference is evaluate it according to the industry standards. Uni-core
that Bound-T has run into trouble in that it uses PA-based systems meet the requirements of standards such as
analysis that can no longer reliably find upper bounds. ARINC653 and AUTOSAR. Researchers have put forward
6
the use of general purpose operating systems, such as 6.2 HERCULES Framework
Linux, in order to provide an environment where they HERCULES is a project for the development of a high-
could evaluate a multicore real-time system. Although this performance real-time architecture for low-power embed-
approach has the advantage of being able to reuse code ded systems. Estimated effects of the HERCULES project
that already exists, it runs into issues where Linux was include a reduction in energy consumption, productivity
not built to support hard real-time systems or to meet the improvement in programming and maintaining advanced
constraints for safety-critical applications. HIPPEROS, a computing systems, increased concurrency and parallelism
multicore RTOS project launched in 2010, is built from in applications, and enhanced trust of embedded systems.
scratch so that it can implement hard real-time techniques The objective of the project is to introduce predictability into
with multicore design principles. Its kernel is built to scale embedded high-performance computing. There are two use
with an increasing amount of cores [10]. cases for the HERCULES project: avionic and automobiles.
The avionic use case considers future airplanes that
will have more complex needs in regards to image pro-
6.1.1 Kernel cessing or computer vision, which may be used during
landing, surveillance activities, and navigation. Moreover,
HIPPEROS’ kernel is able to run on multiple different the number of cameras on board an airplane is expected
architectures and platforms with an arbitrary number to increase, possible direct video streams for the pilot and
of cores. It uses distributed asymmetric micro-kernel crew, and possible automation to extract meaningful data
architecture that allows each core to execute a local part from the video streams. Machine learning techniques can be
of the kernel. This enables a dedicated core to execute the applied and have been proven to do image processing at the
kernel’s system calls, scheduler, and resource handling and cost of higher algorithmic complexity and computational
frees up parts of the kernel to be able to be executed in requirements. A visual object tracking application based on
parallel. The kernel is configurable in that the developer Airbus high-speed machine learning techniques was used to
or system designer can select the scheduling policy or test the HERCULES framework with various programming
resource allocation protocol at its build time. To manage models and GPU-based platforms. [7].
hard real-time tasks it uses a popular process model that The automobile use case is involved with autonomous
gives executable and timing information like deadline, driving for valet parking. The HERCULES framework was
period, and worst-case execution time. chosen to be tested for valet parking for three reasons.
First it has subset functionalities required for self-driving
a. Asymmetric Kernel Architecture cars, and testing is affordable within the time frame of the
HERCULES project. Second, the project team has simulators
The problem with symmetric kernel architecture design and test environments to create the scenarios. Third, the
is that the cores are executed with the same kernel code agents drive at low speeds, which offers clear safety ad-
and protected data structures with fine-grained lock vantages. This use case study shows four major areas where
mechanisms. The asymmetric design allows for one core algorithms will be applied are the perception area (sensor
to fully dedicate itself to running the scheduler and data processing), the data fusion area (creates environmental
dispatching the processes to other cores. For the HIPPEROS model), the decision area (action and path planning), and
project the dedicated core is called the master core and localization area (GPS and MAP data management) [7].
is responsible for managing global resources, scheduler, These use case studies show the flexibility of the HER-
system calls, and message passing to allow for the kernel CULES framework, which can be adapted to different in-
to be executed in parallel. This works as follows whenever dustries.
there is a scheduling decision the master core must be
woken up to notify the slave core (any core other then
the master core) to perform a context switch. The slave 6.3 Security Research related to RTOS
cores can only perform context switch after receiving an As RTOS has been used in devices and SCADA (Supervisory
inter-processor interrupt (IPI) for the master core. This Control And Data Acquisition) systems for time-sensitive
system of master and slave cores does not require locking and mission-critical tasks, its security is an important area
mechanisms, and it is expected to be able to handle up to of research. Several security vulnerabilities of RTOS have
8 cores before overloading the master core without using been discussed in a report on RTOS security [?], such as lack
clustering for enhancement [1] [10]. of authentication enforcement, inefficiency of encryption,
code injection, exploiting shared memory, priority inversion,
denial of service attacks, and inter-process communication
attacks.
6.1.2 Scheduler
Adversaries may attack the devices or embedded sys-
The scheduler’s API is preemptive and priority based. When tems through accessing the remote debugging tools of
a task switches state (blocked to ready), a scheduler module RTOS. For example, US-Cert Vulnerability Note VU362332
is called and it decides if context switching must occur [19] stated that VxWorks debug service was enabled by
due to tasks priorities. If a task misses its deadline, then default so that an attacker might be able to fully com-
a configurability policy enacts a range of responses that prise the embedded systems and create severe damages to
can terminate the process, ignore the event, or change the the physical processes controlled by the SCADA systems
priority of the process [10]. through remote memory dump and remote function calls.
7
Thus security should be a top priority when designing, [3] Silambarasan, Ramanatha Venkatesan. Handling of priority
implementing, and configuring RTOS for remote debugging inversion problem in RTLinux using priority ceiling protocol. In
Proceedings of the International Journal of Advanced Engineering
tools. Research and Science (IJAERS). Vol. 3, Issue 6. June 2016.
RTOS security may also be enhanced by detecting attacks
based on systems call sequences, network activities, and [4] Daniel Forsberg and Magnus Nilsson. Comparison between
power consumption patterns. For example, a class of at- scheduling algorithms in RTLinux and VxWorks. Technical Report,
Computer Science and Engineering at the University of Linkping,
tacks, known as payload attacks and successfully launched November 2006.
by Stuxnet, modifies PLC control programs (i.e., the pay-
load for PLC firmware which may be a RTOS) and causes [5] Stefan Holmer, Osker Hermansson. A comparison between the
damages to the physical system. In [21] the authors studied scheduling algorithms used in RTLinux and in VxWorks - both
from a theoretical and a contextual view, Technical Report,
firmware-level detection of PLC payload attacks that alter Computer Science and Engineering at the University of Linkping,
the timing behavior of the payload. WCET analysis and 2006.
monitoring have also been used to enhance RTOS security
[22]. [6] Federico Reghenzani, Glueppe Massari, William Fornaclari. The
real-time Linux kernel: A survey on Preempt RT. ACM Computing
RTOS also needs to address the memory fragmentation Surveys. Volume 52, Issue 1, February 2019.
issue to improve its reliability and avoid program stalls
as field devices such as PLCs (Programmable Logic Con- [7] Marko Bertogna. High-Performance Real-time Architectures for
trollers) using RTOS may continuously run for many years Low-Power Embedded Systems. H2020-EU.2.1.1. Project Website:
https://cordis.europa.eu/project/rcn/199161/factsheet/en. 2016-
without rebooting [20]. 2018.
[2] Lui Sha, Ragunathan Rakumar, John Lehoczky. Priority inheritance [18] AbsInt. The Industry Standard for Static Timing Analysis.
protocols: an approach to real-time synchronization. IEEE https://www.absint.com/ait/index.htm
Transactions on Computers, Vol. 39, No. 9, pp. 1175-1185.
September 1990. [19] US-Cert, Vulnerability Note VU362332, Wind River
Systems VxWorks debug service enabled by default,
8
https://www.kb.cert.org/vuls/id/362332/
[21] Huan Yang, Liang Cheng, and Mooi Choo Chuah, Detecting
payload attacks on programmable logic controllers (PLCs), IEEE
Conference on Communications and Network Security (CNS),
Beijing, China, May 2018.
Liang Cheng Dr. Liang Cheng received his PhD from Rutgers, The
State University of New Jersey in 2002. He focuses his research on
enabling intelligent infrastructure based on real-time sensing, model-
driven data analytics, and machine learning. He is an expert in cyber-
physical systems and ad hoc networks. His research has been funded
by U.S. federal funding agencies such as NSF, DOE, DOT, and DARPA,
Pennsylvania government, and companies including ABB, Agere Sys-
tems, East Penn Manufacturing, and PPL. He has advised 7 Ph.D.
students to their graduation, supervised 2 postdocs, advised 23 Master’s
degree theses, and co-authored more than 100 peer-reviewed papers.
More information is available at http://liangcheng.info.