ICETA 2023

21st Year
of International Conference
on Emerging eLearning Technologies
and Applications

PROCEEDINGS

October 26 – 27, 2023

Grand Hotel Starý Smokovec, High Tatras, Slovakia

 Website: http://www.iceta.sk

ISBN: 979-8-3503-7068-3
IEEE Catalog number: CFP2338M-USB
Proceedings editor: Štefan Fejedelem

Copyright and Reprint Permission: Abstracting is permitted with credit to the source. Libraries
are permitted to photocopy beyond the limit of U.S. copyright law for private use of patrons
those articles in this volume that carry a code at the bottom of the first page, provided the per-
copy fee indicated in the code is paid through Copyright Clearance Center, 222 Rosewood
Drive, Danvers, MA 01923. For reprint or republication permission, email to IEEE Copyrights
Manager at pubs-permissions@ieee.org. All rights reserved. Copyright ©2023 by IEEE.

3
 COMMITTEES
Honorary Committee Chairman in Memoriam

Jakab František, Technical University of Košice

Honorary Committee

Molnár Ľudovít, Slovak University of Technology, Bratislava

Newman B. Harvey, California Institute of Technology, Pasadena
Rudas Imre, Óbuda University, Budapest

Program Committee Chair

Feciľak Peter, Technical University of Košice

Program Committee

Babič František, Technical University of Košice

Bauer Pavol, Delft University of Technology
Bederka Andrej, Slovak National Coalition for Digital Skills and Jobs, Bratislava
Behun Marcel, Technical University of Kosice
Berke József, John von Neumann Computer Society
Bieliková Mária, Slovak University of Technology, Bratislava
Bours Patrick, Norvwegian University of Science and Technology
Brestenská Beáta, Comenius University Bratislava
Cehlar Michal, Technical University of Košice
Chovanec Martin, Technical University of Kosice
Čičák Pavol, Slovak University of Technology, Bratislava
Čižmár Anton, Technical University of Kosice
Dado Milan, University of Žilina
Doboš Ľubomír, Technical University of Košice
Dolnák Ivan, University of Žilina
Doroshenko Anatoliy, Institute of Software Systems of NASU, Kiev
Drozdová Matilda, University of Žilina
Dzupka Peter, Technical University of Košice
Feciľak Peter, Technical University of Kosice
Galgoci Gabriel, AT&T Bratislava
Gavurova Beata, Technical University of Košice
Genči Ján, Technical University of Košice
Haľama Marek, Technical University of Košice
Hämäläinen Timo, University of Jyväskylä
Hautamaki Jari, University of Appl. Science
Hluchý Ladislav, Slovak Academy of Sciences, Institute of Informatics
Horváth Pavol, SANET - Slovak Academic Network, Bratislava
Huba Mikuláš, Slovak University of Technology, Bratislava
Hudak Radoslav, Technical University of Košice
Hvorecký Jozef, City University of Seattle, Bratislava

4
Jelemenská Katarína, Slovak University of Technology, Bratislava
Juhár Jozef, Technical University of Košice
Juhas Gabriel, Slovak University of Technology in Bratislava
Kainz Ondrej, Technical University of Kosice
Kess Pekka, University of Oulu
Kireš Marian, University of Pavol Jozef Šafárik
Klačková Ivana, University of Žilina
Klimo Martin, University of Žilina
Korshunov Alexander, Kalashnikov Izhevsk State Technical University of the name M.T.
Kalashnikov
Kotuliak Ivan, Slovak University of Technology
Kováčiková Tatiana, COST Office, Brusel
Kovács Levente, Óbuda University, Budapest
Kyselovič Ján, Slovak Centre of Scientific and Technical Information, Bratislava
Lavrin Anton, Technical University of Kosice
Lelovsky Mario, Slovak IT Association Bratislava
Levický Dušan, Technical University of Košice
Lumnitzer Ervin, Technical University of Kosice
Mesaroš Peter, Technical University of Košice
Meško Dušan, Jessenius Faculty of Medicine, Martin
Michalko Miroslav, Technical University of Kosice
Modrak Vladimir, Technical University of Kosice
Mulesa Oksana, Uzhorod National University
Mytnyk Mykola, Ternopil National Ivan Pul`uj Technical University
Nakano Hiroshi, Kumamoto University
Pavlik Tomaš, Technical University of Košice
Peciar Peter, Faculty of Mechanical Engineering STU Bratislava
Pietrikova Alena, Technical University of Kosice
Pisutova Katarina, Comenius University
Piteľ Jan, Technical University of Košice
Poruban Jaroslav, Technical University of Košice
Radács László, University of Miskolc
Restivo Maria Teresa, University of Porto
Ristvej Jozef, University of Zilina
Salem Abdel-Badeeh M., Ain Shams University of Cairo
Semanišin Gabriel, University of Pavol Jozef Šafárik
Simonics István, Obuda University, Budapest
Šimšík Dušan, Technical University of Košice
Sinčák Peter, Technical University of Košice
Sobota Branislav, Technical University of Košice
Sovak Pavol, University of Pavol Jozef Šafárik
Stopjakova Viera, Slovak University of Technology in Bratislava
Strémy Maximilián, Slovak University of Technology in Bratislava - Advanced Technologies
Research Institute – University Science Park CAMBO, Trnava
Stuchlikova Lubica, Slovak University of Technology
Šveda Dušan, UPJŠ Košice
Szabo Stanislav, Technical University of Košice
Szentirmai László, University of Miskolc

5
Turna Jan, Slovak Centre of Scientific and Technical Information, Bratislava
Usawaga Tsuyoshi, Kumamoto University
Vagan Terziyan, University of Jyvaskyla
Vasková Iveta, Technical University of Košice
Vokorokos Liberios, Technical University of Košice
White Bebo, SLAC National Accelerator Laboratory, Stanford University
Zivcak Jozef, Technical University of Košice
Zolotová Iveta, Technical University of Košice

Organizing Committee Chair

Fejedelem Štefan, elfa, s.r.o.

Organizing Committee

Čuntalová Darina, elfa, s.r.o., Košice, Secretary

Rakoci František, elfa, s.r.o., Košice, Webmaster
Zámečníková Iveta, elfa, s.r.o., Košice, Finance Chair

6
 TABLE OF CONTENTS
Committees ........................................................................................................................ 4

Table of Contents .............................................................................................................. 7

Interaction of police officers with automated vehicles .................................................. 13

J. Andraško, M. Kordík

AI – Quo Vadis Education ................................................................................................. 19

V. Bakonyi, Z. Illés, D. Szabó, S. Korom

The Impact of Sentiment in S&P 500 volatility prediction

with the use of Deep Learning ......................................................................................... 25
V. Balara, M. Mach, K. Machova

The Role of Diversity in Process of Education and their influence

on the productivity of work .............................................................................................. 31
A. Balcova, P. Balco, F. Delaneuville

Asynchronous Motor Speed Servo Drive ....................................................................... 38

I. Bélai, I. Bélai

Possibilities for interactive animation elements on web browsers .............................. 45

M. Beňo, K. Pribilová, M. Ölvecký

The Need for the Creation of a Course in Cyber Security Awareness ......................... 51
E. Beňová, A. Juhásová, K. Kánová, K. Pišútová, K. Révayová

Simulating Vehicle-to-Vehicle (V2V) Communication in Urban Traffic Scenarios ...... 56

D. Bilik, P. Lehoczký, I. Kotuliak

Development of a Module for Collecting Students’ Feedback

in an Interactive Manner ................................................................................................... 62
L. Blahova, J. Kostolny

Development of Online Anatomical Atlas for Biomedical Informatics ......................... 68

D. Bobak, A. Gabrisova, M. Juricek, P. Kulas, A. Listvan, A. Jelinek, K. Kardosova, M.
Kvassay

Proposed modifications of the computer science study programs

to address space safety industry and academia requirements .................................... 76
P. Butka, M. Sarnovský

ASM Modeling for Training System and Handwritten Recognition .............................. 83

P. Campanella

Exploring the Capabilities and Possible Applications of Large Language Models

for Education ..................................................................................................................... 91
M. Čavojský, G. Bugár, T. Kormaník, M. Hasin

7
Teaching Synchronization in Parallel Computing .......................................................... 99
M. Čerňanský

Application of PID controller and CNN to control Duckiebot robot.............................. 105

M. Długosz, P. Skruch, M. Szelest, A. M. Magiera

Usage of workshop programming for turning with live tools ....................................... 111
T. Dodok, N. Čuboňová, M. Čierňava

Crimes in Slovak Republic - Statistical Analysis of Trends .......................................... 117

D. Dubovec, M. Kvet

Preparation and implementation of educational games in the teaching

of Controlling in the construction industry .................................................................... 123
A. Ďuriš, J. Smetanková, R. Ručinský, P. Mésároš, K. Krajníková

Early Detection of Cyber Grooming in Online Conversations:

A Dynamic Trust Model and Sliding Window Approach ............................................... 129
T. N. Eilifsen, B. Shrestha, P. Bours

Embodiment and body awareness and their impact on student interactions

in higher education ........................................................................................................... 135
M. Filipová, P. Balco

Artificial intelligence in education, issues and potential of use

in the teaching process .................................................................................................... 141
D. Gabriska, K. Pribilova

Chain Collision Avoidance Using Vehicle-to-Everything (V2X) Communication ........ 147

M. Galinski, J. Juraško, P. Trúchly, L. Šoltés

The Good, the Bad and the Ugly ethics of automated vehicles .................................... 153
Z. Gyurász, P. Dražová

Example of Fire Detection Using UAV and Cellular Networks ...................................... 159
R. Haluška, S. Marchevský, M. Pleva, S. Pillár

Mobile Educational Application for Keystroke Dynamics Identification Systems ...... 165
R. Haluška, D. Molčanyi, M. Pleva, D. Hládek, J. Staš, M. H. Su, Y. F. Liao

User Recognition in Mobile Applications Using Fingerprint Sensors ......................... 171

R. Haluška, E. Švenk, M. Pleva, S. Ondáš, M. H. Su, Y. F. Liao

English language teaching through social media and digital tools ............................. 176
T. Havlaskova, T. Javorcik, K. Kostolanyova

Puzzle-Driven Learning: Developing and Assessing IT Challenges for Varied

Experience Levels ............................................................................................................. 183
M. Horváth, E. Pietriková

8
Developing a User-Centric Queueing Simulation Engine
for Educational Purposes ................................................................................................. 189
R. Horváth

Serious games: The Possibilities of Data Mining and Data Analysis

through Neuro and Biofeedback ...................................................................................... 195
M. Hosťovecký

On the border between automatic control and artificial intelligence ........................... 200
M. Huba, P. Bisták

Selected Issues Causing Meta-stability in Digital Circuits and Approaches

to Avoiding Them .............................................................................................................. 206
A. Hudec, R. Ravasz, R. Ondica, V. Stopjakova

Learning enhancement with AI: From idea to implementation ..................................... 212

L. Huraj, J. Pospíchal, I. D. Luptáková

Developing Prerequisite Knowledge for 3D Modelling and 3D Printing ...................... 220

J. Hvorecký, A. Schmid

Small and affordable platform for research and education in Connected,

Cooperative and Automated Mobility .............................................................................. 225
M. Janeba, S. Bohumel, J. Hrnčár, M. Galinski, I. Kotuliak

Analysis of methods and technologies used to predict and detect fires in forestry .. 231
M. Janovec, J. Papán

Analysis of available technologies for the design of a tracking device

for monitoring endangered animal .................................................................................. 237
M. Janovec, J. Papán, M. Hraška

Diagnostics of affective components of digital competences

in elementary school pupils – a pilot study .................................................................... 243
T. Javorcik, T. Havlaskova

Low-Code Languages in IT Education: Integrating Theory and Practice .................... 249

G. Juhás, A. Juhásová, L. Petrovič

Open-book versus closed-book exams as a part of hybrid education

in computer science and humanities in the post-COVID era ........................................ 258
J. Jurinová, J. Miština

Network System Healthcheck .......................................................................................... 264

D. Kafka, J. Saxa, P. Segeč, A. Straka, T. Jurík

Educational Web Solution for Cyber Security ................................................................ 270

O. Kainz, S. Nečeda, M. Michalko, M. Murin, I. Nováková

Web-Based Visualization and Control of Traffic Simulation ......................................... 277

J. Kapusta, P. Trúchly

9
Automated Monitoring of Network Infrastructures Based
on the Zabbix Solution...................................................................................................... 283
E. A. Katonová, J. Džubák, P. Feciľak

Implementation of IDS Functionality into IoT Environment using Raspberry PI ........ 289
E. A. Katonová, P. Nehila, P. Fecilak, O. Kainz, M. Michalko, F. Jakab, R. Petija

Auditory Testing in Noise with Language Barrier — A Case Study ............................. 295
E. Kiktová, P. Getlík

The Influence of Energy Efficiency on the Production of Emissions

in Safety Small Production Systems ............................................................................... 301
I. Klačková, D. Wiecek, V. Benko, T. Dodok

Digital Twin and Modelling a 3D Human Body in Healthcare ........................................ 307

M. Klimo, M. Kvassay, N. Kvassayova

Game Engine Based Application for Neurorehabilitation

in Collaborative Virtual Reality ........................................................................................ 313
Š. Korečko, P. Nehila, B. Sobota

Exploring GitOps: An Approach to Cloud Cluster System Deployment ...................... 318

T. Kormaník, J. Porubän

Enhancing Learning Outcomes with Interactive Courses ............................................. 324

J. Kostolny, V. Karcolova, M. Vaclavkova, L. Blahova

Fitness Tracker Data Extraction and Visualization Methods ........................................ 330

A. Kotvan, P. Helebrandt

Automated environment to support competition in computer networking ................. 337

S. Kubinský, P. Feciľak, M. Michalko, F. Jakab

Early Recognition of the Speaker’s Age ......................................................................... 341

E. Kupcová, R. Haluška, M. Popovič, M. Pleva, M. S. Heng, P. Bours

Using a selected machine learning method in the R language

in statistics learning.......................................................................................................... 348
I. D. Luptakova

Comparison of Deep Learning and Ensemble Learning

in Classification of Toxic Comments .............................................................................. 353
K. Machova, T. Tomcik

Eye-Tracking System as a Part of the Phishing Training .............................................. 359

M. Madleňák, K. Kampová

How to Develop CT Skills Using Robotics Projects Combined

with Arts - a Pilot Study .................................................................................................... 365
K. Miková, Z. Kubincová

10
Online Educational Game - Tool for Education in Raw Material ................................... 371
M. Molokáč, G. Alexandrová, Z. Babicová, D. Tometzová, L. Molitoris

Automated deployment of the OpenStack platform ...................................................... 375

M. Moravcik, P. Segec

Mobile application to support the teaching of computer networks.............................. 381

M. Murin, K. Nalevanko, E. A. Katonová, R. Petija, P. Feciľak

Efficiency study of MPPT algorithms in HDL for full integration

of solar-powered voltage converter ................................................................................ 389
R. Ondica, A. Hudec, D. Maljar, V. Stopjaková

Exploring Oracle APEX for the University Data Analysis .............................................. 395
I. Pastierik, M. Kvet

Emerging Technologies in Education: Security and Data Protection .......................... 403

P. Pekarcik, E. Chovancova, T. Kuchcakova

Mapping the Environment Using an Ultrasonic and Infrared Sensor........................... 409

K. Pribilová, D. Gabriska, Z. Mäsiarova

Use of Modern Software Environments for Online Teaching

of Microprocessor Programming ..................................................................................... 415
V. Režo, F. Gerhát, M. Weis

Experience with creating toolkits and using innovative methods in the educational
process in the field of acquiring and processing land resources in the conditions
of higher education ........................................................................................................... 421
R. Rybár, L. Bednárová, Z. Šimková

Smart home lighting control environment ...................................................................... 429

R. Sabol, P. Feciľak, M. Michalko, F. Jakab

Mapping Network Entity Relationships Based on SNMP Data Collection ................... 433
P. Segec, M. Sterbak, M. Moravcik, M. Plostica

Using a honeypot to improve student cybersecurity awareness ................................. 440

M. Šimon, L. Huraj, D. Hrinkino

Duckietown project pros and cons .................................................................................. 446

P. Skruch, M. Długosz, M. Szelest, A. M. Magiera

Investigating the Impact of Confined Space Factors on Signal Propagation.............. 451

E. Skýpalová, T. Loveček

Construction proceedings in the Slovak Republic: An overview of tools

for efficient exchange and management of information in the BIM environment ....... 456
J. Smetanková, A. Ďuriš, R. Ručinský, P. Mésároš, L. Zemanová

Virtual Educational Collaborative Environments for Low-Cost Mobile VR Headsets. 463

B. Sobota, T. Hulina, Š. Korečko, M. Mattová

11
User Interfaces in Virtual Reality Environments. ........................................................... 469
B. Sobota, Š. Korečko, M. Mattová, W. Steingartner, C. Szabó, G. Stromp

Possible use of virtual reality in the therapistpatient interaction ................................. 474

B. Sobota, J. Miňová, Š. Korečko, M. Mattová, W. Steingartner

Importance and implementation of virtual reality

for organic photovoltaic educational purposes ............................................................. 479
M. Sobota, F. Kolencik, L. Stuchlíková, M. Weis

Comparison of Sentiment Classifiers on Slovak Datasets:

Original versus Machine Translated ................................................................................ 485
Z. Sokolová, M. Harahus, J. Juhár, M. Pleva, D. Hládek, J. Staš

Analysis and Detection of Speech under Emotional Stress ......................................... 493

J. Staš, D. Hládek, Z. Sokolová, M. Čech, K. Škotková, P. Poremba

The Next Big Thing in University Education: a Threat or an Opportunity? ................. 499
L. Stuchlikova, J. Stuchlik, M. Weis

National center for digital transformation of education ................................................ 507

D. Šveda, V. Hubenáková, K. Kozelková, K. Lukáčová, A. Mišianiková, V. Ondová,
M. Filčáková, J. Kozáková, M. Kireš

On the Distribution of Electrodermal Activity Properties as a Tool

for Teaching Soft Skills .................................................................................................... 519
L. Tomaszek, M. Miklošíková, M. Malčík, P. Šaloun

Increasing cyclists safety using intelligent Bicycle Light

Based On Artificial Intelligence ....................................................................................... 525
A. Tomčala, R. Bencel

Authorship Attribution in Astroturfing Detection

and the Impact of Google Translate on Cross-lingual Text Analysis ........................... 531
I. K. Torgersen, S. Leibold, P. Bours

High school and university students' use of social networks

to support their self-education ........................................................................................ 536
D. Tran, K. Kostolányová

LoRa™ Lab: Laboratory Network for Educational Purposes ........................................ 542

A. Valach, L. Zemko, P. Čičák, K. Jelemenská

Practical teaching of production and characterization of next-gen transistors

based on sol-gel methods. ............................................................................................... 550
T. Vincze, M. Weis

Ultrasonic water leak detection system with real-time transmission

of measured values ........................................................................................................... 556
D. Zadžora, E. A. Katonová, M. Murín, P. Feciľak, M. Michalko, F. Jakab

Author Index ...................................................................................................................... 561

12
 Interaction of police officers with automated
vehicles
Jozef Andraško* and Marek Kordík*
* Comenius University in Bratislava, Faculty of Law/Institute of Information Technology Law and Intellectual Property
Law, Department of Criminal Law, Criminology and Criminalistics, Bratislava, Slovakia
jozef.andrasko@flaw.uniba.sk
marek.kordik@flaw.uniba.sk

Abstract—The article deals with the topic of interaction of II. ACT ON AUTOMATED VEHICLES
police officers with automated vehicles. First of all, authors
will clarify terms such as automated vehicle, fully By adopting Act no. 429/2022 Coll. amending some
automated vehicle, automated driving system and laws in connection with the development of automated
automated delivery vehicle in terms of the new legislation vehicles (hereinafter referred to as the "Act on automated
that was adopted in the Slovak Republic. Furthermore, vehicles") [3], the Slovak Republic was included among
authors will point out the possible application problems in the states that allow the operation of automated vehicles
the exercise of the police officer's authorizations in subject to certain conditions. Act on automated vehicles
interaction with automated vehicles.
amends several legal acts, in particular:
I. INTRODUCTION
x Act No. 8/2009 Coll. on road traffic and on
The operation of automated vehicles that perform amendments to certain laws (hereinafter referred
driving tasks (e.g. steering, acceleration and braking) to as the "Road traffic act") [4],
instead of the driver on public roads also entails the
interaction of such vehicles with police officers and other x Act no. 106/2018 Coll. on the operation of
first responders such as firefighters. vehicles in road traffic and on the amendment of
certain laws (hereinafter referred to as "the Act
In practice, there have already been situations where on the operation of vehicles in road traffic") [5],
an automated vehicle was stopped on the instructions of a
police officer and subsequently moved to another x Act No. 575/2001 Coll. on the organization of
location. Officers from the San Francisco Police government activities and the organization of the
Department stopped the automated vehicle because it did central state administration, as amended [6],
not have its lights on. The automated vehicle stopped, the x Act no. 145/1995 Coll. on administrative fees
officer approached its window and tried unsuccessfully to and on amendments to certain laws, as amended
open the door. There were no passengers in the vehicle. [7].
Subsequently, the police officer went to the police vehicle
and the automated vehicle moved to another location The Act on Automated Vehicles adds to the Act on
where it stopped. The reason the automated vehicle left the operation of vehicles in road traffic the definitions of
where the original stopped was because the automated the terms automated vehicle, fully automated vehicle,
vehicle was trying to find a safer place to stop [1]. automated driving system and automated delivery
vehicle.
Another example is from February 2023, when a
Waymo vehicle stopped at the scene of a fire and did not The legal definitions of the terms automated vehicle
want to leave the road. The police officer ordered the and fully automated vehicle follow the terms from the
vehicle to stop and placed a flare in front of the vehicle to regulation (EU) 2019/2144 of the European Parliament
prevent the vehicle from passing a fire hose that was and of the Council of 27 November 2019 on type-
placed on the road [2]. approval requirements for motor vehicles and their
trailers, and systems, components and separate technical
However, can similar cases be expected in the units intended for such vehicles, as regards their general
conditions of the Slovak Republic? After the adoption of safety and the protection of vehicle occupants and
new legislation in the field of automated vehicles, vulnerable road users (hereinafter referred to as the
certainly yes. The authors of the article will first of all "General safety regulation") [8], however definitions used
clarify terms such as automated vehicle, fully automated in Act on automated vehicles are expanded.
vehicle, automated driving system and automated
delivery vehicle in terms of the new legislation. In the Pursuant to Section 2 (2) (ac) of the Act on the
next part of the article, the tasks and authorizations of operation of vehicles in road traffic is automated vehicle:
members of the Police Force in interaction with "an automated vehicle according to a special
automated vehicles will be clarified. In the end, the regulation6a) or another motor vehicle designed and
authors will point out possible legal problems in the constructed in such a way that it can move autonomously
exercise of police officers' powers in interaction with for a certain periods of time without the continuous
automated vehicles. supervision of the driver, but in respect of which driver
intervention is still expected or required." The special

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 13

regulation in question is the General safety regulation, operation automated delivery vehicles that can deliver
which directly refers to the definition of the term cargo directly to the customer's door.
automated vehicle in the provision of Art. 3 (21) of this
regulation. In comparison to the definition used in the Until the adoption of the Act on automated vehicles, a
General safety regulation, the definition of the term driver was defined in the Road traffic act as a person
automated vehicle in the Act on the operation of vehicles driving a vehicle. The understanding of the driver as a
in road traffic is significantly expanded to include any person who drives the vehicle and is physically in the
other motor vehicle designed and constructed in such a vehicle represented a limit in the deployment of
way that it can move autonomously for a certain period of automated vehicles that use automated driving systems
time without the continuous supervision of the driver, but that perform driving tasks instead of the driver. Because
in respect of which driver intervention is still expected or of this, the Act on automated vehicles creates a new kind
required. An explanation of the reason for expanding the of driver. In addition to the person who drives the
definition to include another motor vehicle can be found vehicle, the driver is also considered to be: "a person who
in the explanatory report to the draft law on automated supervises a vehicle that uses an automated driving
vehicles. Due to the fact that the General safety system to drive." (Road traffic act, Section 2 [2] [v])
regulation is a regulatory act for the granting of type In relation to the person who supervises a vehicle that
approval of vehicles of categories M and N, the national uses an automated driving system for driving (hereinafter
regulator considers that an automated vehicle: "can also referred to as "the person who supervises the vehicle"),
be of a category other than M and N, for example a four specific rights and obligations are regulated in Section 5
wheel vehicle of category L, a wheeled tractor category (6) of the Road traffic act. In particular, the person who
T, rack-laying tractors category C and the like." [9]. supervises the vehicle is: "obliged to take control of the
A fully automated vehicle is pursuant to Section 2 (2) vehicle in a timely and safe manner at the request of the
(ad) of the Act on the operation of vehicles in road traffic: automated driving system or, if the circumstances require
"a fully automated vehicle according to a special it, even without this request, possibly even remotely."
regulation6b) or another motor vehicle designed and For clarity, we present the types of drivers in
constructed so that it can move autonomously." The accordance with the Road traffic act in the following
special regulation in question is a General safety table.
regulation which directly refers to the definition of the
term fully automated vehicle in the provisions of Art. 3 TABLE 1
(22) of this regulation. As in the case of the term TYPES OF DRIVERS ACCORDING TO THE ROAD
automated vehicle, the definition of fully automated TRAFFIC ACT
vehicle has been expanded compared to the definition in
the General safety regulation. The reason is the same as in Types of drivers Explanation
the above case, and thus according to the national
Traditional driver A person who drives a non-
regulator, a fully automated vehicle can also be of a
automated vehicle or an
different category than M and N (e.g. four wheel vehicle
automated vehicle at a time when
of category L, a wheeled tractor category T, rack-laying
the automated driving systems are
tractors category C and the like).
not performing driving tasks.
The automated driving system is pursuant to Section 2
(2) (ae) of the Act on the operation of vehicles in road The person who 1. A traditional driver
traffic: "a vehicle control system that permanently uses supervises the physically present in the
hardware and software to ensure dynamic control over vehicle automated vehicle at the time
the vehicle." The concept of automated driving system is when the driving tasks are
the subject to regulation by SAE standard on Driving performed by the automated
Automation Systems (hereinafter referred to as the "SAE control system. At the
request of the system, the
standard") [10] and the new provision of Convention on
person supervising the
Road Traffic signed on November 8, 1968 in Vienna
(hereinafter referred to as the "Vienna road traffic vehicle must take control of
convention") (Article 1 (ab)) [11]. Automated driving the vehicle.
systems are used in automated vehicles of automation 2. Remote driver - a person
level 3 and above. The Act on automated vehicles does who supervises the vehicle,
not define the concept of dynamic control. This term is, who is not in the automated
for example, defined in a new provision of the Vienna vehicle and can take control
road traffic convention (Article 1 (ac)). of the automated vehicle
Automated delivery vehicles are pursuant to Section 4 remotely.
(8) (h) of the Act on the operation of vehicles in road
traffic: "automated vehicles, fully automated vehicles or
remotely controlled vehicles that move partially or In addition to the specific obligations, the person who
completely autonomously and serve to transport cargo." supervises the vehicle is obliged to fulfill the same
These vehicles belong to other vehicles of category V and obligations and the same requirements apply to him as a
will be used to transport mainly small cargo, e.g. for food traditional driver. The driver, and thus also the person who
delivery as a delivery service. Companies like Nuro [12], supervises the vehicle, must be medically and mentally fit
Starship Technologies [13], Kiwibot [14] are examples of to drive the vehicle (Road traffic act, Section 86 et seq.).
companies that have been able to design and put into The driver must also take a professional competence

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 14

exam, proving sufficient knowledge (theoretical test) and A proposal for a trial operation permit within the trial
skills (motor vehicle driving test) necessary for driving a operation of an automated vehicle or a fully automated
vehicle (Road traffic act, Section 79). The driver must vehicle is submitted by the manufacturer or the
hold a driver's license and must have other valid manufacturer's representative for automated vehicles or
documents with him, such as registration certificate part I fully automated vehicles using an automated driving
or part II (Road traffic act, Section 94 et seq. and Section system that he is developing or manufacturing (Act on the
4 [1] [b]). operation of vehicles in road traffic, Section 49 [2] [b]).
The list of eligible applicants is exhaustive, which means
The driver is also a participant in road traffic, as he that, apart from the above-mentioned entities, no one else
directly participates in road traffic, which represents the can apply for a trial operation permit.
use of highways, roads, local roads and purpose-built
roads by vehicle drivers and pedestrians (Road traffic act, The proposal in question must contain all the data
Section 2 [2] [u] and Section 2 [1]). For this reason, he is required by Section 49 of the Executive Regulation of the
also obliged to fulfill general obligations pursuant to Ministry of Transport and Construction of the Slovak
provisions of the Section 3 of the Road traffic act, where Republic No. 131/2018 Coll., which establishes details in
the general obligations of road traffic participants are the field of vehicle approval (hereinafter referred to as
defined. In particular, driver is obliged to obey an "Executive regulation No. 131/2018 Coll.") [17]. One of
instruction resulting from a traffic sign or traffic device the content requirements of the proposal is the approval
and to obey an instruction, call or command of a police opinion of the Police Force authority regarding the trial
officer related to the exercise of his authority in operation of an automated vehicle or a fully automated
supervising the safety and smoothness of road traffic, to vehicle at a specified time, on a specified territory or on a
tolerate the exercise of his authority, as well as the specified route from the point of view of affecting the
instructions of other persons authorized to do so by Road safety and smoothness of road traffic (Executive
traffic act or a special regulation (Road traffic act, regulation No. 131/2018 Coll., Section 49 [f])
Section 3[2] [c]).
The Act on automated vehicles also adds provisions to
For the sake of completeness it should be added that the Act on the operation of motor vehicles in road traffic
the remote driver concept introduced by the Act on regarding requirements for the authorization of an
automated vehicles is not a new concept in technical automated delivery vehicle operation and the details of
community. Such a concept is considered by the SAE the procedure for the authorization of an automated
standard, which states that the remote driver performs delivery vehicle operation.
some or all dynamic driving tasks and emergency
dynamic driving tasks in real time (including braking, Entitled person to submit a proposal for permission to
steering, acceleration and gear shifting in real time) (SAE operate an automated delivery vehicle to the type
standard, point 3.24). approval authority is the vehicle operator who will
operate the vehicle in road traffic (Act on the operation of
The issue of the remote driver is also the subject of vehicles in road traffic, Section 52 [3]). The operator of
discussions at the meetings of the Global Forum for Road the vehicle is the owner of the vehicle or the holder of the
Traffic Safety (WP.1). The most up-to-date documents in vehicle designated by him (Act on the operation of
the field of remote control are: vehicles in road traffic, Section 2 [27]).
x an informal document entitled "Automated As for the content of the proposal in question, such a
driving. Situations when a driver operates a proposal contains, in addition to the applicant's
vehicle from the outside of the vehicle " [15] identification data, also data and documents to the extent
and established by the executive regulation. The executive
regulation in question is Executive regulation No.
x an informal document entitled "Remote 131/2018 Coll. One of the content requirements of the
management of automated vehicles " [16]. proposal is the approval opinion of the Police Force
III. RESPONSIBILITIES OF POLICE FORCE AUTHORITIES authority, which is required to permit the operation of an
IN THE PROCESS OF TYPE APPROVAL OF AUTOMATED automated delivery vehicle at a specified time, on a
VEHICLES specified territory or on a specified route from the point
of view of affecting the safety and smoothness of road
The Act on automated vehicles introduces the traffic. In addition to this approval opinion, the applicant
possibility to operate automated vehicles and fully must also obtain the approval opinion of the administrator
automated vehicles within the framework of a trial of the road on which the automated delivery vehicle is to
operation, as well as the operation of automated delivery be operated and the approval opinion of the municipality
vehicles. on whose territory the automated delivery vehicle is to be
Pursuant to the new provision, in particular Section 49 operated (Executive regulation No. 131/2018 Coll.,
(1) of the Act on the operation of vehicles in road traffic Section 53a [e] [f] [g])
can be operated in road traffic based on the permission of Automated delivery vehicles may be operated in road
the type approval authority for the purpose of trial drives traffic only if the vehicle and its equipment meet the
(trial operation): "an automated vehicle or a fully technical requirements established by the executive
automated vehicle using an automated driving system in regulation. The executive regulation in question is the
road traffic that has not been approved for operation in Executive Regulation of the Ministry of Transport and
road traffic, for the purpose of trial drives during the Construction of the Slovak Republic No. 134/2018 Coll.
development, production or approval of the vehicle, its which establishes the details of the operation of vehicles
systems, components or separate technical units." in road traffic. [18]

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 15

 From the abovementioned, it follows that the the Slovak Republic, cooperating in ensuring
authorities of the Police Force, probably the traffic police public order; supervising the safety and fluidity
department of the Presidium of the Police Force, have the of road traffic and cooperating in its
possibility to significantly influence the authorization of management, etc.) [20],
the trial operation of automated vehicles, fully automated
vehicles and the operation of automated delivery x combating criminal and other antisocial
vehicles, since without their consent, such operation will activities (detecting criminal offences and
not be permitted. Due to the absence of guidelines and identifying their perpetrators, cooperating in the
procedures on the issue of approval of automated detection of tax evasion, illegal financial
vehicles, it is probably possible to expect the use and operations, the legalisation of proceeds from
application per analogies of the law on the operation of crime and the financing of terrorism, conducting
vehicles in road traffic for the approval and use of an criminal investigations and summary criminal
automated vehicle in road traffic. investigations)
x the exercise of state administration pursuant to
special regulations (e.g. the Act on identity cards
IV. INTERACTION OF POLICE OFFICERS WITH THE [21], Act on residence of foreigners [22], Act on
AUTOMATED VEHICLES AND POSSIBLE LEGAL OBSTACLES firearms and ammunition [23], etc.),
At present, there is no specific legislation regulating x other tasks, if provided for by special regulations
the specific powers that members of the Police Force may (e.g. Criminal Code [24], Civil Dispute Code
use against automated vehicles or the drivers of such [25], Civil Non-dispute Code [26],
vehicles. In the context of the operation of automated Administrative Procedure Code [27],
vehicles, fully automated vehicles and automated delivery Administrative Procedures [28], etc.)
vehicles, it can certainly be expected that such vehicles
will also be operated on public roads where, in various x prevention tasks.
cases, there will also be interaction with members of the As indicated above, the police officer may use some
Police Force for the purpose of checking the person in the of the powers, in the exercise of which this power may
vehicle, checking the vehicle itself for authorisation: also be directed towards the automated vehicle, persons
x requesting proof of identity pursuant to Section in control of the vehicle, persons and things in the
18 of Act No. 171/1993 Coll. on the Police automated vehicle or in relation to the remote driver. For
Force (hereinafter referred to as "Police force the effective and smooth exercise of any authorisation, it
act") [19], is necessary that the information system used by the
police officer in the performance of his duty, after
x to seize a person pursuant to Section 19 of the reading the registration number of the vehicle, should
Police force act, simultaneously provide the information that:
x seize an object pursuant to Section 21 of the x it is an automated vehicle;
Police force act,
x what control mode the vehicle is currently in;
x to stop and search a means of transport pursuant
to Section 23 of the Police force act, x if the vehicle is remotely controlled by the
driver, information about this fact as well as the
x to ensure the safety of designated persons location where the remote driver is at the time of
pursuant to Section 25 of the Police force act, controlling the vehicle;
x to prohibit entry to or remaining in a specified x if necessary, a link to the remote driver [29],
place pursuant to Section 27 of the Police force
act, x verification of the identity of the remote driver
and verification of the ability to drive the
x to close publicly accessible places pursuant to automated remotely [30].
Section 28 of the Police force act,
The rules of the road should de lege ferenda include
x to clarify offences pursuant to Section 32 of the a mandatory rule that, when the automated driving system
Police force act and is performing driving tasks, the system should also follow
the officer's instructions to stop or invite the driver to take
x to ensure the safety and continuity of railway control and stop the automated vehicle himself. It should
transport pursuant to Section 33 of the Police be stressed that it is possible to prove from the data stored
force act. in the vehicle on the driving course who was performing
The aforementioned powers may be used by a police the driving tasks at the time of the infringement, whether
officer for the purpose of carrying out tasks in the field the driver or the automated driving system. For this
of: purpose, e.g. the Data Storage System for Automated
Driving (DSSAD) is used. The DSSAD is a mandatory
x protection of public order and security feature of vehicles that are equipped with an Automated
(cooperating in the protection of fundamental Lane Keeping System (ALKS) [31].
rights and freedoms, in particular in the
protection of life, health, personal liberty and The biggest application problem can be foreseen in the
security of persons and in the protection of setting of traffic rules and the exercise of the police
property, ensuring the control of the borders of officer's authority, especially in the context of driving an

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 16

automated vehicle by a remote driver. In this case, the problematic. In relation to it, the exercise of the police
limit is not the technician, but the complicated legal officer's powers of road traffic control should be adapted
framework, which combines the powers of the police and changed so that they are enforceable in practice.
officer, e.g. to restrict personal liberty or to seize the
automated vehicle in administrative proceedings under
the Police force act, the possible need to enter an ACKNOWLEDGMENT
establishment, or the possibility of entering the vehicle,
and the possibility of the police officer's arresting the This article was drafted with the support from a grant
automated vehicle. Finally, the legal nature of driving a awarded by the Slovak Research and Development
vehicle remotely, which is considered teleoperation and Agency No. APVV – APVV-20-0346 Legal and technical
thus the need to obtain traffic and location data for the aspects of introducing autonomous vehicles.
purposes of administrative proceedings, which are
currently subject to telecommunications secrecy, which
can only be broken for the purposes of criminal
proceedings, but not for the clarification of REFERENCES
misdemeanours or administrative offences. At the same
time, de lege ferenda consideration should be given as to [1] [online], Available at:
whether, in view of the above specificities, it should be <https://www.theverge.com/2022/4/10/23019303/heres-what-happens-
mandatory for the remote driver to carry out the control cops-pull-over-a-driverless-cruise-vehicle-general-motors>
[2] [online], Available at:
over the vehicle only in the state where the vehicle is <https://missionlocal.org/2023/05/waymo-cruise-fire-department-
located, whereby the crossing of the border would be police-san-francisco/>
taken over by the remote driver in the state concerned. [3] Act no. 429/2022 Coll. amending some laws in connection
with the development of automated vehicles
[4] Act No. 8/2009 Coll. on road traffic and on amendments to
V. CONCLUSIONS certain laws
[5] Act no. 106/2018 Coll. on the operation of vehicles in road
The new legislation in the field of automated vehicles traffic and on the amendment of certain laws
in the Slovak Republic on the one hand creates [6] Act No. 575/2001 Coll. on the organization of government
procedures for the authorisation of the trial operation of activities and the organization of the central state administration, as
amended
automated vehicles, fully automated vehicles and the [7] Act no. 145/1995 Coll. on administrative fees and on
operation of automated delivery vehicles and on the other amendments to certain laws, as amended
hand creates room for the emergence of situations where [8] Regulation (EU) 2019/2144 of the European Parliament and
the Police Force authorities will face new situations in the of the Council of 27 November 2019 on type-approval requirements for
performance of their tasks and the exercise of their motor vehicles and their trailers, and systems, components and separate
powers. The performance of the tasks and powers of the technical units intended for such vehicles, as regards their general safety
police authorities in interaction with automated vehicles, and the protection of vehicle occupants and vulnerable road users,
amending Regulation (EU) 2018/858 of the European Parliament and of
fully automated vehicles and automated delivery vehicles, the Council and repealing Regulations (EC) No 78/2009, (EC) No
which can also be controlled remotely by the driver, may, 79/2009 and (EC) No 661/2009 of the European Parliament and of the
in application practice, call into question the hitherto Council and Commission Regulations (EC) No 631/2009, (EU) No
established practices of the police authorities that were 406/2010, (EU) No 672/2010, (EU) No 1003/2010, (EU) No 1005/2010,
used for years in interaction with traditional vehicles and (EU) No 1008/2010, (EU) No 1009/2010, (EU) No 19/2011, (EU) No
the traditional driver, who sat in the driver's seat and 109/2011, (EU) No 458/2011, (EU) No 65/2012, (EU) No 130/2012,
(EU) No 347/2012, (EU) No 351/2012, (EU) No 1230/2012 and (EU)
performed driving tasks. 2015/166. OJ L 325, 16.12.2019, p. 1–40.
As outlined above, a police officer may exercise some [9] Dôvodová správa k návrhu zákona o automatizovaných
vozidlách. [online], Available at:
of the powers, in the exercise of which that power may
<https://www.nrsr.sk/web/Dynamic/DocumentPreview.aspx?DocID=51
also be directed towards the automated vehicle, persons 5513>
in control of the vehicle, persons and things in the [10] SAE standard J3016_202104, “Taxonomy and Definitions
automated vehicle, or in relation to the remote driver. In for Terms Related to Driving Automation Systems for On-Road Motor
relation to the person of the driver driving the automated Vehicles.” 2021
vehicle, it can be stated that the activities of the members [11] Convention on Road Traffic signed on November 8, 1968 in
of the Police Force will be governed by the same Vienna
[12] [online], Available at: <https://www.nuro.ai/>
procedures as before, while no major problems are to be [13] [online], Available at: <https://www.starship.xyz/>
expected in the exercise of the police officer's powers in [14] [online], Available at: <https://www.kiwibot.com/>
the performance of his duties, since the scope of the [15] UNECE: Automated driving. Situations when a driver
exercise of the powers as well as the manner of their operates a vehicle from the outside of the vehicle. Informal document
execution are likely to remain the same. The current legal No. 1/Rev.2. (6th March 2023)
situation also maintains the status quo in relation to the [16] UNECE: Remote management of automated vehicles.
Informal document No.16. (9th March 2023)
necessary documents to be carried by the driver of an
automated vehicle, while a change in the scope of the [17] Executive Regulation of the Ministry of Transport and
Construction of the Slovak Republic No. 131/2018 Coll., which
necessary documents to be carried by the driver while establishes details in the field of vehicle approval
driving cannot be ruled out in the future. [18] Executive Regulation of the Ministry of Transport and
Construction of the Slovak Republic no. 134/2018 Coll. which
From the point of view of the exercise of the police establishes the details of the operation of vehicles in road traffic
officer's powers to enforce road safety and traffic flow [19] Act No. 171/1993 Coll. on the Police Force
and the related enforcement of public order, the new type [20] PEŤOVSKÝ, M., ŠKRINÁR, T., ODLEROVÁ, M.: Zákon
of remote driver of a motor vehicle seems to be o PZ - aplikačná prax. Plzeň: Aleš Čeněk, 2017, p. 10.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 17

[21] Act No. 395/2019 Coll. on Identity Cards [29] PEARL, T.H.: Traffic Stops in the Age of Autonomous
[22] Act No. 404/2011 Coll. on residence of foreigners and Vehicles. In MARREIROS, G., MARTINS, B., PAIVA, A., RIBEIRO,
amendment and supplementation of certain Acts B., SARDINHA, A. (eds) Progress in Artificial Intelligence. EPIA
[23] Act No. 190/2003 Coll. on firearms and ammunition and on 2022. Lecture Notes in Computer Science, vol 13566. Springer, Cham.
amendments to certain acts [online], 2022, Available at: <https://doi.org/10.1007/978-3-031-16474-
[24] Act No. 300/2005 Coll. Criminal Code 3_7>
[25] Act no. 160/2015 Coll. Civil Dispute Code [30] UNECE: Recognizability of ADS from the perspective of
[26] Act no. 161/2015 Coll. Civil Non-dispute Code enforcement. Informal document No.11. (13th September 2022)
[27] Act no. 162/2015 Coll. Administrative Procedure Code [31] UN Regulation No. 157 Uniform provisions concerning the
[28] Act No. 71/1967 Coll. on Administrative Procedures approval of vehicles with regard to Automated Lane Keeping Systems

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 18

 AI – Quo Vadis Education
V. Bakonyi*, Z Illés*, D. Szabó* and Sz. Korom*
* Eötvös Loránd University/Media & Educational Informatics Department, Budapest, Hungary
hbv@inf.elte.hu, illes@inf.elte.hu

Abstract— Over the last decades several new technologies,

global changes have caused serious issues in education. The B. Smart mobile phones
first one might has been the spread of internet and in The next upcoming technology was the spread of smart
parallel with this the smart mobile technology is also phones. The penetration quickly raised especially among
indispensable. During the last couple of years, virtual online our students who are future programmers. We made
classes have been widely used as a result of the emergency surveys from 2015 till 2019. The availability of Hungarian
situation. Most recently AI based applications have shocked surveys are:
us and the education as well, maybe producing the biggest x 2015-2016, https://bit.ly/3evmSlW;
changes not only in education. As almost all inventions, they
can be used for good or bad goals – e.g. think of the idea of x 2019 https://forms.gle/iTrsquf9LETeruJS7),
nuclear power, which can be used for generating power, or and more than 95%-99% of students had smart phones [1].
contrary, making weapons. The above-mentioned new Why is this interesting from the viewpoint of education?
technologies (internet, smart devices) have given people new They do not have to go home or to a laboratory to switch
possibilities to acquire better and quicker information but on a computer for browsing and getting new information,
in the meantime, they can be used for evil aims too, e.g. for they always have their smart phones with them. Anytime
cybercrime. Narrowing this to students, undoubtedly they can browse the internet, even during the lectures. We
students can use them for a better and deeper preparation, have noticed that they check everything and try to find
but all of these technologies have raised new challenges new details and data. It can be a very good learning tool
towards professors about how to stop possible cheating. (but naturally sometimes they watch social media or play
Most recently we are facing maybe the most interesting instead of paying attention) [2][3]. So we decided to use
challenge of all. AI based chatbots, text directed voice and their devices in the teaching process and created an own
image generators take the possibilities of cheating to the Classroom Response System with Bring Your Own
next level. What can we do against it? Can we do anything Device method called E-Lection [4][5][6] .
at all? We all know the saying: If we cannot ban it, then let’s
use it! We are showing one possible way how to use it! As expected, smart mobile phones have become a new
tool in cheating as well. Anywhere and anytime, they are
able to reach the internet using their phones, even during
I. INTRODUCTION tests and exams. institutes organized more supervisors
against cheating, whose task was to check for secretly
Educational challenges have forced out constant
used smart devices and take them away. But it does not
changes not only during the last decades. Think about the
solve everything – there was a student who brought 3
industrial revolution. It caused many changes in all
phones into the exam hall!
aspects of life and therefore in education as well.
C. Online classes, emergency situation
A. Internet
2020 was the year of emergency situation when our
30 years ago, the first technology causing significant
university – as so many others - started to use Ms Teams
impact was the global network.(internet) Since the 90th
for online synchron lessons and Canvas for LMS. We can
internet began to spread, although internet penetration in
send chat messages in it, stream and record lectures, get
less wealthy countries was somewhat slower. It meant that
immediate feedbacks – students enjoyed the new
instead of going to the library, students turned on a
possibilities to learn but cheating had become a major
computer and searched for the needed information, or
issue again. Everything was online, even tests and exams
instead of sending a classical letter the much quicker e-
– they were able to discuss the solutions and help each
mails were available. It was very useful to download
other using Teams itself. Sometimes they said they do not
documents or to get more detailed data to help preparing
have web-cameras or microphones which was not true so
themselves. Using search functionality in browsers
they can “hide” themselves during the lessons or use extra
sometimes gave ready-made solutions or they can send
devices to solve tasks [7]. There are such applications as
each other the results. Therefore, it was sure that the tests
Measure Learning provide proctoring
and exams written in the school were almost surely
(https://www.meazurelearning.com/). During personal
students’ intellectual product– though the ownership of
face-to-face exams, web-cams and microphones were
homeworks became a bit questionable. In the case of
compulsory but there tricks existed with extra persons or
informatics, in the case of laboratory tests and exams,
devices hiding somewhere. Therefore, professors might
some kind of “switch off network” method could solve the
not be sure about fair grading circumstances.
problem.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 19

D. AI based chatbots
In November 2022 the free ChatGPT
(https://openai.com/) appeared which is an Artificial
Intelligence (AI) based chatbot. Today the largest IT
companies like Microsoft, Google
(https://bard.google.com/), Apple (not published yet) are
all working on AI researches. There are free available text
directed AI based image creators (e.g.
https://text2img.org/), music, voice generators (e.g.
https://rb.gy/5nxv7) and naturally chatbots, which are very
popular now. (ChatGPT has produced the quickest
penetration ever seen in the last months, though it stopped
for now [8].) These tools are built into other applications
e.g. into Microsoft Bing. It is a new IT revolution which
will change our life extremely.
It opens a new level in getting information. These
chatbots react as human beings, we can discuss any topic Figure 1. HTML file generated with Bing
with them. (In section II, you can read some discussions.)
Moreover, chatbots can produce new essays, can analyze
B. Computer Systems, simple shell script
program codes, implement programs or make
documentations without the help of humans. It might be a Our question was: “There is a text file in which there
phantastic tool for learning, but naturally it can be used for are Neptun-codes line by line. Count, how many different
cheating too! Instead of students, chatbots are able to Neptun-codes are in the file? Neptun code is a 6-length
create good quality homeworks or can be used at tests or long string, containing both characters and digits. Give,
exams. It was tested by professors and the chatbot got a how many lines do not contain Neptun codes.”
grade C, while in another training grade B [9]. The result solves the given problem see Fig. 2. Though
Naturally these experiences have heightened fears when I said there might be equal Neptun codes and I want
among professors about the fairness of their grading the number of different – chatbot skipped this request in
systems. What is the situation in the case of informatics the result.
trainings?
The suggested questions are:
II. INFORMATICS TRAININGS & AI BASED CHATBOTS
x “What is the meaning of double [?” or
Let us see some practice tasks and big homeworks from
the first semester subjects. (The attached results are x “What re regular expressions?”
created during the first days of August 2023.)
A. Computer Systems, Web-page
The very first homework from Computer Systems
subject is to create a simple homepage about a topic.
ChatGPT created a solution with a non-existing image and
link, but otherwise it is good. Using Bing with the build in
AI based chat we are asked to give an image but after it
provides the required webpage.
Our request was: “Make a html page about Internet of
Things. It should contain an image, a table and at least a
link to another page!”
The AI’s response was: “Sure, I can help you with that.
However, I cannot create images here in the chat box. Can
you provide me with an image that you would like to use
on the page?”
Figure 2. A script file generated with Bing
We gave the link: “The image is at https://rb.gy/hoo35”
and the result is in Fig. 1. Both are very important to understand when you learn
It is a real human like conversation with asking back bash. It is very useful in the learning process to clarify
the needed information. What is really-really useful every detail if it is needed. During the term we have tests
during the learning process is the AI suggested questions and in them there are several similar tasks, which can be
that could be interesting for the learner: solved by chatbots.
x “How do I modify the code to add a border to the C. Programming subject, homework
table?” In this subject, students learn to plan and solve
x “What are some other IoT applications?” problems in C# language.
x “Can you explain how the img tag works in more The question is: “Create a C# code which reads from
detail?” the keyboard an integer called N, which is between 1 and
But what about the coding tasks? 100. After it read N integers from the keyboard. Give me if

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 20

there is a peek in the list of numbers or not. Peek if the
close neighbors are smaller.”
We get a good, working, but not so nice solution. The
main part of the code is as follows:

bool hasPeak = false;

for (int i = 1; i < n - 1; i++)
{
if (a[i] > a[i - 1] && a[i] > a[i + 1])
{
hasPeak = true;
break;
}
}
if (hasPeak)
{
Console.WriteLine("There is a peak");
}
else
{
Console.WriteLine("There is no peak");
}
The suggested question possibilities are good, it really
helps students:
x “What is an array in C#?” Figure 3. A C# homework generated with Bing
x “How do I modify the code to print all peaks in
the list of numbers?”
x “Can you explain how the for loop works in E. Operating Systems, exam task
more detail?” This subject is in the 4th or 5th semester. The exam task
is more complex than the previous ones. It’s description is
D. Programming subject, big homework longer – and we were interested how it is handled by AI?
Our question was: “Implement a C# code which reads The task description was: “Solve the under-detailed
in some integer values from a text file. The number of the task in C programming language, which can be executed
integers are in the first line and after it line by line there on a Linux system. Starts the actual daily corona virus
comes an integer. The task is to find the longest part of the news conference where the Communication Officer
integers where all of them are bigger than 0. Write out the (parent process), the Police Lieutenant-colon (child) and
first and last index of the integer list, if there is no such the National Head Physician (child) take part and answer
part, write out "NO". the press questions. The Communication Officer
It gives an executable program code, of which logic commandeers the event and waits all of the answers.
looks good at first sight, but it is not good at all! Though (After all the parent waits the end of children and all
it is good in calculating the maxLen, but it always along keeps the contact with them.)
overwrites the found values of start and end variables see 1, At the beginning the Communication Officer (parent)
Fig. 3. So now we found that it sometimes makes mistakes waits since both other two members (children) to settle
even in simple tasks. down on the podium and they nod to him (send a signal)
that they are ready to start. (Write on the console) After it
the Communication Officer asks the first question (using
pipe) to the Police Lieutenant-colon - "Is it compulsory to
wear a mask in the shops?" He answers (through pipe) -
"Yes, it is compulsory to wear the mask when you leave
your flat!" (Please write the communication step by step
to the console.) (If you want a better grade, go on to task
2, if not, the parent should wait the end of children and
then terminates).
2, After this the National Head Physician remarked "To
wear a mask is really very important to save other people
and ourselves against the virus in the shops and on the
roads" (this remark is sent to the Communication Officer
through a message queue) The Communication Officer
writes it out to the console.
3, Now the National Head Physician inform the
audience about the number of new infected people. It is a
freely given number which she writes into the shared

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 21

memory. The Communication Officer read it and write it suggestions were helpful. In more complex tasks it was
to the console. //If you want a better grade, go on to task also successful, though we need new experiences time-to-
4, if not, the parent should wait the end of children and time to have an actual opinion.
then it terminates.” All together it might be a very useful tool for practicing
To tell the truth we tried a similar complex question in and learning. Let’s not forget it was published for free trial
December 2022 with ChatGPT and it did not give an only half years ago so it is going to get better and better!
executable code, only tips about the structure of the code
skipping some requests like pipe usage. III. STUDENTS & CHATBOTS
Now we get an executable code from Bing which As we can see from the above-mentioned examples
contains fork calls, pipes, message queue and shared ChatGPT and the like AI based chatbots are able to help
memory too. The code is too long to insert here. In students in learning, but it is quite a new tool. We were
December we were calm that there is no problem with this interested whether our students use it or not, and for what
kind of tasks, AI is not able to produce an executable and how they like it.
solution. the solution could possibly be uploaded by Circumstances of our survey:
students, so we asked a similar question which had never
been an exam task: x Anonym, not compulsory survey in May 2023 in
Google Forms.
F. Operating Systems, similar x It is in Hungarian, but we have an English version
The task was: “Create a C program in which there are too.
a parent and a child process. The parent sends a text to x Available
the child given in the command line using a pipe and the https://forms.gle/GD4zRCpLumADv6i66,
child reads it out and write to the screen. The child https://forms.gle/ys8Eg31epUdyzXPn7
answers back using the shared memory "It is OK". The
x Filled by 233 students (49 secondary school, 13
parents read it and write on the screen.” Again, we got an
first year university students and 171 from higher
executable, well working solution, a part of it see Fig. 4.
semesters)
There are 11 questions (we are not analyzing all of
them now)
Now we are working only with the Hungarian
responses because we have too few English responses.
A. Chatbots are known
Our first question was: “Have you tried the new
ChatGPT or other voice-based AI options?” Possible
answers: Yes, I like it; Yes, I do not like it; No, but I am
interested; No, I am not interested.
57% of secondary school students, 100% of first
semester students and 91% of higher semester students
tried it. Mostly they like it: 44%, 76%, 80% of the total
students (in the above given order).

B. The purpose of use

We were interested in for what purpose they use it see
Fig. 5.
The question was: “What did you use it for, if you used
it?” Possible answers:
x Talking, testing, having fun with machine
intelligence
x To solve everyday problems (e.g. writing a letter,
Figure 4. A C parallel program generated with Bing finding out about a topic (where the cow buzzards
live)
The suggested question also serves the learning process x Solve professional and/or school tasks (e.g.
well: programming, comprehension, composition)
x “What is a pipe in C?” x Voice-based cloning, speech generation
x “How does shared memory work in C?” x Image-based cloning, image, motion picture
x “Can you explain how the fork function works in generation
more detail?” x Other At Fig. 5 we can see, that at about 70%
Summarizing these experiences (and some others are ELTE university students use this tool for professional
not inserted the paper) in many cases first semester tasks purposes too, not only for having fun.
were solved well, but not always! The learning

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 22

 documents[10]. We fear that the result is not 100%
&ŽƌǁŚĂƚƚŚĞLJƵƐĞ/ĐŚĂƚďŽƚƐ in many cases.
ϵϬй x Cancel the availability of chatbots in school. In the
ϴϬй
ϳϬй USA there are districts where it is banned [11].
ϲϬй Whilst there is mobile net possibility, it is not the
ϱϬй
ϰϬй perfect solution either. Moreover, it prevents to use
ϯϬй
ϮϬй the advantages of AI in the learning process in
ϭϬй
Ϭй school. Naturally at home they can also work with
WƌŽĨĞƐƐŝŽ
dƌŝĂů ĂŝůLJůŝĨĞ
Ŷ
sŽŝĐĞ /ŵĂŐĞ KƚŚĞƌƐ it. Without using the new trends, it may cause
^ĞĐŽŶĚĂƌLJƐĐŚŽŽů ϰϯй Ϯϵй Ϯϳй Ϭй Ϯй ϭϬй
problems later in the labour market.
&ŝƌƐƚLJĞĂƌƐƚƵĚĞŶƚƐ ϲϮй ϴϱй ϲϵй ϴй ϯϭй ϴй x Switch off internet and mobile internet during
,ŝŐŚĞƌƐƚƵĚĞŶƚƐ ϲϲй ϱϵй ϳϰй ϱй Ϯϲй ϯй exams. Today Hungarian law does not permit to
^ĞĐŽŶĚĂƌLJƐĐŚŽŽů &ŝƌƐƚLJĞĂƌƐƚƵĚĞŶƚƐ ,ŝŐŚĞƌƐƚƵĚĞŶƚƐ disturb mobile availability.
x Use test detecting services like Measure Learning
Figure 5. Using AI based chatbots. in dedicated laboratories, but small smart phones
may do the trick.
x Use AI detecting tools like ZeroGPT or GTPZero.
C. Using chatbots over the tests, exam works. During lessons students
The next question we focus is: “Have you ever used it may use chatbots and learn how to handle
in any accountancy? (exam, test etc.)” The possible it[12][13]. It is a pity, but AI detection tools are not
answers were 1 (never) - 5 (several times). At TABLE I you working good at the moment, 20% of the results
can see, that more than 30 % of the university students are wrong [14].
already used AI chatbots during test or exams! If we x In the case of a written test or exam, focus on
consider the results only among the students who tried it questions that are not producing “good” results by
already the result is even worse. the chatbots now for the test. It requires an extra
work from the professors.
TABLE I. x Lighten the weight of homeworks in the grading
HAVE YOU EVER USED IT IN ANY ACCOUNTANCY? (EXAM, TEST ETC.) system. Though it may have a result that students
do not want to work on homeworks. Organize
Percentage Percentage
School type
total number who tried already
more teachers to check personal written exams.
Good choice to go back to personal oral exams if it
Secondary school 12% 21% is possible and wait for some protecting idea,
application [10].
First year students 31% 31% We are stating that prohibition is never a good solution.
Higher students 37% 41% It encourages cheating rather than compliance and at the
end of the day it causes more injustice. Moreover, it is
We calculated the result counting the values which are inexplicable to students why they are banned to use a
bigger than 1, divided by the total number of students. The modern tool, which they experience as good and has many
other number was calculated by the students’ number who benefits [16]– especially in informatics trainings [17].
already tried it. the real result is possibly a bit higher. (It In our opinion, the only way forward is to adapt the
would give a clearer picture to look at the frequency of chatbots and reorganize the courses to use chatbots during
use, but for now we think it's enough to look at the the lessons[15]. Instead of banning during tests and
number of people who use such not allowed tools.) exams, we must create an environment where cheating is
Therefore, there is no question we must handle this as low as possible. Today the safest solution is the oral
situation. The genie is out of the bottle! exam, where there is a discussion between the professor
and the student. In the case of informatics trainings where
IV. WHAT CAN WE DO FOR GRADING FAIRNESS? coding must be a part of exams, the solution is not easy. It
It is clear, that we got a new revolutionary tool for may be a good idea to find such tasks which AI result is
learning with human-like interactivity though nowadays it not good or not efficient.
produces false results too. “Even on specific questions that
involve combining knowledge across domains, the OpenAI V. SUMMARY
chat is frankly better than the average MBA at this point.” In the last decades education has faced several
Kevin Bryan [10] technology revolutions, which all gave new possibilities to
It is sure sooner or later professors will be forced to learn better, but meanwhile added new cheating tools. In
modify their courses and build AI based possibilities in education, professors want to grade fair. After each period
the learning process. The question is what to do with fair they modified their courses to avoid problems as much as
gradings. There are several suggestions and practices in they can and use the benefits of the novelty.
this moment. The last challenge appeared in November 2022, the AI
x Create ethical rules and discussions with students based chatbots which can generate texts, images, music
not to use it during tests & exams – inserting AI and what is the best that they are able to communicate as a
connected plagiarism into the university human and discuss problems. We made some examples
how it solves informatic problems and how it can discuss
the problems with the user.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 23

 We created a survey to ask students’ opinion about Research and Practice, Budapest, Magyarország : ELTE
chatbots. As we expect it from people majoring in Informatikai Kar (2021) pp. 132-142., 11 p.
informatics, they have tried it. Though it was a surprise [7] V. Bakonyi and Z. Illés, "Education challenges after Covid-19,"
2022 20th International Conference on Emerging eLearning
that more than 30% also used it for cheating. That means Technologies and Applications (ICETA), Stary Smokovec,
the ghost is out of the bottle, professors all over the world Slovakia, 2022, pp. 34-39, doi:
must solve the problem related to new AI technology in 10.1109/ICETA57911.2022.9974620.
informatics trainings too. [8] Cecily Mauran, “ChatGPT monthly traffic has dropped for the
There are several possibilities to handle the situation, first time” available at https://mashable.com/article/chatgpt-
we have collected some of them. Prohibition does not monthly-traffic-decline
solve anything; we have to change, that is the solution. [9] Drake Bennett, “ChatGPT Is an OK Law Student. Can It Be an
OK Lawyer?” in Blomberg Newsletter, 2023 Junuary, available at
Change in the teaching methods and change in the grading (https://rb.gy/fw2xp)
processes as well. Maybe the most important we have to [10] Amy Blitchok, “How are Colleges and Universities Responding to
change the wording of the exercises; we have to forget the ChatGPT?” available at https://www.universities.com/news/how-
factual wording of elementary exams. There are some are-colleges-and-universities-responding-to-chatgpt
subjects, AI can’t help today. For example, IPC related [11] Arianna Johnson, “ChatGPT In Schools: Here’s Where It’s
tasks solutions are out of scope, but let there be no doubt, Banned—And How It Could Potentially Help Students”, in
tomorrow they will be solved as well. Forbes, available at
https://www.forbes.com/sites/ariannajohnson/2023/01/18/chatgpt-
in-schools-heres-where-its-banned-and-how-it-could-potentially-
REFERENCES help-students/?sh=2a9ee8f86e2c
[12] AI detector GPTZero available at https://gptzero.me/
[1] Judit, Bakonyi Viktória, supervisor Illés Zoltán, “Valós-idejű [13] AI detector, ZeroGPT available at https://www.zerogpt.com/
Oktatási Alkalmazások Okoseszközök Használatával”, dissertation [14] Victor Tangerman, “There's a Problem With That App That
2021. Detects GPT-Written Text: It's Not Very Accurate”, available at
[2] Dr. Illés Zoltán; Ifj Illés Zoltán; H Bakonyi Viktória, “Mobile https://futurism.com/gptzero-accuracy
Driven Changes in Education” in EDUKACJA TECHNIKA [15] Tim Fütterer, et al., “ChatGPT in Education: Global Reactions to
INFORMATYKA (2080-9069 2450-9221): 6 1 pp 3010-3015 AI Innovations”, preprint, available at
(2015) https://www.researchgate.net/publication/370658302_ChatGPT_in
[3] Illés Zoltán; Ifj Illés Zoltán; H Bakonyi Viktória, “Changing the _Education_Global_Reactions_to_AI_Innovations,
learning attitude of students by a BYOD system” In: New methods DOI:10.21203/rs.3.rs-2840105/v1
and technologies in education and practice XXIX DidMattech [16] Khaddage, Ferial; Flintoff, Kim “Goodbye to Structured Learning
2016, Conference place & time: Budapest, Hungary 2016.08.25. - ChatGPT in Education, is it a Threat or an Opportunity?”,
2016.08.26.,Budapest: ELTE Informatikai Kar, pp 122-127 (2016) Conference: Society for Information Technology & Teacher
[4] Viktória H. Bakonyi;Zoltán Illés; Ifj Zoltán Illés,” Real-time Tools Education International ConferenceAt: New Orleans, 2023 March,
in Classroom” in CENTRAL-EUROPEAN JOURNAL OF NEW available at
TECHNOLOGIES IN RESEARCH EDUCATION AND https://www.researchgate.net/publication/370838637_Goodbye_to
PRACTICE ( 2676-9425): 1 1 pp 1-8 (2019) _Structured_Learning_ChatGPT_in_Education_is_it_a_Threat_or
[5] Victoria H. Bakonyi, Zoltan Illes, “Using Real-time Applications _an_Opportunity
in Education” in International Journal of Advances in Electronics [17] Sedat Golgiyaz, “ChatGPT in Computer Software Education”,
and Computer Science, ISSN(p): 2394-2835 Volume-6, Issue-9, Conference: ICHEAS, 4th International Conference On Health,
Sep.-2019, p 33-39 Engineering And Applied SciencesAt: Dubai2023 March,
[6] Viktória, Bakonyi; Zoltán, Illés, “New methods in education, new available at
E-Lection version” In: Abonyi-Tóth, Andor; Stoffa, Veronika; https://www.researchgate.net/publication/371762576_ChatGPT_in
Zsakó, László (szerk.) Proceedings of XXXIV. DidMatTech 2021 _Computer_Software_Education
Conference : New Methods and Technologies in Education, .

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 24

 The Impact of Sentiment in S&P 500 volatility
prediction with the use of Deep Learning
V. Balara*, M. Mach*, K. Machova*,
* Technical University of Košice/Department of Cybernetics and Artificial Intelligence, Košice, Slovakia
marian.mach@tuke.sk, kristina.machova@tuke.sk, viliam.balara@tuke.sk

Abstract— The automatic recognition of the impact of the such as N-Beats or Transformer architectures as an
sentiment in online space can help teachers to teach an effective tool for the purpose of estimating the
information literacy and a critical thinking. The paper forecasting volatility.
focuses on the gauging of the magnitude by which the added
sentimental component affects the predictions of the S&P Resulting from the spread of economic
500 market index. The sentiment of market’s participants globalization, accentuated after the impact of the
bears increased significance in the age of social media, contemporary international economic fluctuations, the
which is reflected in periods of increased social media stock market has experienced numerous significant
activity followed by increased market action regarding fluctuations in the span of the several past years that have
commodity. The paper presents an experiment containing a had an impact on broad range of sectors. This volatility
multitude of deep learning methods in order to ascertain the increases the uncertainty and risk of the stock market,
level of improvement to the volatility prediction provided by
which is detrimental to the normal operation of
the addition of the sentiment to the market data. Primarily,
we will focus on the comparison of LSTM, N-Beats and economies around the globe as well as common
Temporal Fusion Transformer methods and results of each participants in any market, therefore negatively impacting
method with and without the component of sentiment in the common consumer in a wide variety of areas. In order to
training data. decrease the associated uncertainty, it is of highest
importance to conduct accurate volatility measurements
of stock indexes. During the computation of volatility
I. INTRODUCTION forecasts, it is of highest priority to correctly evaluate the
The stock market is often considered to be chaotic, underlying contemporary market behavior in respect to
complex, volatile and dynamic. Undoubtedly, its volatility in order to generate accurate predictions. The
prediction is one of the most challenging tasks in the area importance is amplified in the periods of higher volatility
of time series forecasting. The problem of estimating in comparison to the periods described by increased
volatility has always been the biggest challenge for levels of tranquility.
financial modelers. This is mainly caused by the fact that The forecasting of stock market volatility can be
volatility data are extremely complex, highly nonlinear epitomized as one of the fundamental tasks positioned on
and exhibit a substantial degree of temporal variability. the intersection of financial and computer science. In
Predicting stock market behavior is an area of strong regard to this particular issue, a widely known efficient
appeal for both academic researchers and industry market hypothesis (EMH) provides a notion with the
practitioners alike, as it is both a challenging task and implication that financial market remains permanently
could lead to increased profitability or provide additional efficient [1], which purports that both the technical and
level of security for existing portfolios on either personal fundamental analysis would not yield consistently over-
or corporate level, thus reducing necessary associated average profits to market participants. On the other hand,
expenditures. In the past few years, numerous advances a multitude of researchers are of different opinion in
in the field of machine learning have presented favorable contrast to EMH [2]. Several studies are striving for a
results for speech recognition, image classification and proper measurement technique to gauge the different
language processing. Methods applied in digital signal efficiency levels in the cases of mature and emerging
processing can also be utilized to market data as it falls markets, whereas other studies are trying to devise
into the category of time series. Predicting volatility is prediction models with increased efficacy for stock
challenging task, where modern artificial intelligence can markets, which also serves as the scope of this particular
come at rescue. Conventional techniques, mostly based work.
on GARCH modeling and its derivates have proven to be
The EMH claims that all current relevant
possess certain imperfections in terms of volatility
information is already included in the prices of securities
forecasting. In this paper, it is our goal to assess whether
and additional information may lead to unpredictable
deep learning can be utilized to build models of capable
stock prices. Existing studies in the field of sentiment
of providing viable results in the field of volatility
analysis have found out that there is a notable correlation
forecasting. In particular, the paper focuses on novel
between the sudden movements of stock prices and the
approaches based upon the recurrent neural networks

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 25

publication of related articles [3]. A multitude of studies learning, each level learns to transform its input data into a
focused on stock market related sentiment analysis were slightly more abstract and composite representation.
conducted at various levels using predominantly Several architectures that belong into the category of deep
algorithms such as Support Vector Machines, Naive learning have been utilized in the field of stock market
Bayes regression, and Deep learning. The accuracy of volatility predictions.
Deep learning algorithms is highly dependent on In this work, the supervised learning variant of DL was
provided amount of training data. However, thorough used. Deep learning into the branch of supervised machine
sentiment analysis is beyond the scope of this paper, and
learning of artificial neural networks consisting of a large
therefore pre-processed data in a form of user-reported
number of hidden layers. In the case of text data
sentiment from American Association of Individual
processing, recurrent neural networks are often considered
Investors (AAII) accompanied by the sentiment of Fear
to be the best choice, due to their ability to transfer
and Greed Index as well as CBOE, which are often used information from the processing of one input to the
as a measure of volatility and thus will serve as a processing of the next input and thus model the existing
sentimental component in this work. For the price
relationship between individual words located in a text.
prediction, technical and fundamental analysis are being
used. Technical analysis is based on Dow Theory [4], the A. LSTM
price development history of a given security serves as a Long Short-Term Memory Networks (LSTM) constitute
basis for the prediction. The scope of it spans from the an advanced form of RNN, a sequential network, that
traditional statistical modelling to artificial intelligence- possess the ability that allows the preservation of
based methods such as machine learning [5]. In literature information. First introduced by Schmidhuber and
mentioned models are Artificial Neural Networks (ANN) Hochreiter in 1997 [12], this novel type of network was
which are subsequently evaluated against the explicitly designed to enable the avoidance of persistent
performance of statistical models in the area of the stock long term dependency problems. The main perk of LSTM is
volatility forecasting. The concept of ANN has also been the capability to handle the problem vanishing gradient
compared with different data mining classification which had to be commonly faced by existing RNN
algorithms [6] and the conducted comparison implies the architectures. From a high-level perspective LSTM operates
superiority of ANN models [7]. Several studies, however, to a large extent similarly in comparison with an RNN cell.
The internal functioning of the LSTM is following: the
have shown that the idea of ANN possesses several LSTM cell comprises three parts, as shown in the image
drawbacks and therefore does not represent a suitable below and each part conducts its own individual function.
candidate for stock prediction due to the fact that stock
market data contains substantial amount of noise and
complex dimensionality. ANN expresses inconsistent and
unpredictable behavior on the data. The majority of
neural networks are based on the shallow architectures
and are thus furnished with only single hidden layer. One
of the main reasons they may be unsuccessful is their
training strategy. Another problem with neural networks
is that many of them tend to fall into a local optimum
solution and have a significant proclivity for over-fitting
[8]. However, deeper architectures are able to overcome
aforementioned complications [9] and have already
yielded promising results in various fields including Figure 1 LSTM cell
language, speech and image domains. First proposed by
the Hinton was the idea of deep learning which harnessed The first part, forget gates, determines whether the
the yield of model spreading beyond three-level net provided information sourced from the previous
timestamp should be preserved or whether its relevance is
architecture [10]. Deep neural networks (DNN) have miniscule and can therefore be discarded. In the following
been applied to a wide multitude of time series gate, the so-called input gate, the cell attempts to collect
forecasting problems and have shown a track record of new information from the input to this cell. As a final step,
satisfactory results [11]. Successful applications in the taking place in the third part, the output gate, the updated
field of speech recognition have led to the idea that since information is conveyed further by the cell to the
speech is considered as a time series data and stock data following timestamp from the current one. The previously
falls into the same category of time-series as well, that mentioned three parts of an LSTM cell are colloquially
this method can also be put to use for stock market known as gates. Similarly, to an RNN, an LSTM contains
volatility prediction. a hidden state where ‫ݐ(ܪ‬−1) represents the hidden state of
the previous timestamp and ‫ܪ‬ሺ‫ݐ‬ሻ stands for the current
II. USED DEEP LEARNING METHODS timestamp’s hidden state. In addition to that LSTM also
Deep learning (DL) is a class of machine learning possesses a cell state represented by ‫ݐ(ܥ‬−1) and ‫ )ݐ(ܥ‬for
previous and current timestamp respectively. The input
algorithms that uses multiple layers to progressively gate determines about which information will be
extract higher-level features from the raw input. In deep preserved in memory. This only applies for the
information coming from the current input and short-term

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 26

memory from the previous timestep. At the input gate, the grammatical point of view, but also from proves to be
information is being shifted from variables that are not difficult as a function of the relationship that takes place in
market as useful. The forget gate decides which a real world. Due to the self-attention mechanism applied
information from long term memory is going to be in this architecture, it performs outstandingly in the
preserved or discarded and which is fulfilled by learning how to apply context in a data-driven way. The
multiplying the incoming long-term memory by a forget second one, this architecture itself is considerably more
vector generated by the current input and incoming short apt for computing parallelization during training as well as
memory. The output gate takes the current input, the inference phase. The fact allows for significantly faster
previous short-term memory and newly computed long- throughput when learning from training examples and
term memory in order to conceive new short-term allows the training of larger networks with increased
memory which is going to be passed further down on to amount of training data at a provided timeframe. The
the cell in the following time step. largest one of transformer models in the original paper
was equipped with 213 million parameters and was
B. N-Beats trained for over 3.5 days with 8 GPUs. Self-attention, as
The abbreviation N-Beats, standing for Neural Basis the name suggests, is a mechanism for the neural network
Expansion Analysis for Time Series, is a model devised to contextualize words by paying attention to other words
by ElementAI [13]. This model is comprised from a that make up its context in a body of text. To solve the
sequence of stacks, each stack is built from a multitude of problem of context confusion in the case of possible
blocks. The feedforward networks are bound together by equivocal words or phrases for instance Jaguar-animal and
block via backcast and forecast links. Each block focuses Jaguar-colloquial abbreviation for vehicle brand name,
on the residual error created by preceding blocks, creating transformer models use neural networks to generate a
a partial forecast and focusing on the local characteristics vector called query, and a vector called key for every
of a time-series data. Partial forecasts are aggregated from single present word. In the case when the query of one
the blocks creating the stack’s result, which is afterwards word is equal to the key from another word, this implies
forwarded to the following stack. The purpose of a stack is that the second word possesses a context of relevance in
to recognize non-locally bound patterns following the relation to the first word. In order to provide appropriate
entire time-axis by looking back. At the end, all partial context from the second word to the first word, a third
forecasts are assembled into a resulting global forecast. vector called value is generated which is afterwards
combined with the first word to get a more contextualized
meaning of the first word.
Additionally, to the mechanism of self-attention,
transformers were also introduced with the significant
novel idea of positional encodings, thus rendering the
network-structure agnostic to the relative position of a
word in the sequence. The position information is then
added back as an input in the form of trigonometric
functions.

III. IMPLEMENTATION AND TESTING

A. Methodology
We extracted stock-market data and associated data
representing sentiment from publicly available media
sources, which were after pre-processing combined into
resulting dataset. This dataset was separated into two
Figure 2 N-Beats architecture [13] versions, one containing the data of sentiment and the
second version without sentimental stance. The dataset
C. Transformers was used for the training of detection models using classic
The Transformer model was introduced in 2017 by DL method (LSTM) as well as novel approaches (N-Beats
Vaswani et al., 2017 [14] and achieved remarkable and Transformers). The models were evaluated using the
performance in the field of language translation using only RMSE. The methodology of our approach is depicted on
a fraction of the previous training times. The architecture the Figure 3.
is of an encoder–decoder type. The title of the source All models were implemented in the programming
paper was ‚Attention is all you need ‘because the central language Python (version being 3). The IDE used was
idea of this architecture is to rely on attention and self- Jupiter Notebook, which was then utilized to employ
attention mechanisms instead of the feedback loop which Scikit-learn, NumPy, Matplotlib and Pandas libraries. All
is interpreted in the case of RNNs. There are two methods were trained on both datasets in order to properly
distinctive cases in which transformer architecture has assess the impact of added sentimental element on the
proven to show extraordinary performance. The first of prediction task.
them is that transformers are adept at learning how to The conducted experiment was performed in a two-step
apply context. For instance, in the case of sentence
manner, evaluating the performance of selected methods
processing, the underlying meaning of a word or even a
phrase can be completely altered based upon a context in on sentimentally enriched data in comparison to data
which it is being applied to. In most cases, the way this without sentiment, thus assessing the possible influence of
context is being applied is not only arduous from sentiment on the resulting prediction. The forecasts were

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 27

made for a single day period. The selected methods for (RMSE). For each method, three models were trained
evaluation were LSTM serving as a benchmark separately and for each model the one with the best results
representing traditional approaches while novel methods was selected into final comparison. The reason for three
were represented by N-Beats and Temporal-Fusion- models for each selected architectures are three followed
Transformer (TFT) architectures. Representing an prediction goals: the close price for the selected day, the
example of a regression task, in order to assess the percentual change of the selected day and Fear and Greed
efficacy of each architecture, the overall performance was value for the selected day. For each method, the best
evaluated based on root-mean-square-error metric results for respective setting/architecture are displayed.

Figure 3. Methodology Scheme

B. Data Description
The used dataset was created by combining multiple
available sources for each entity included in the final
dataset. The data are separated into 3 groups, all of which
were acquired separately and combined into resulting
dataset. For the task, S&P 500 index was chosen as a
benchmark for market since it represents 500 largest
companies in the U.S market. Another reason behind this
choice was to offset the considerable daily changes which
can occur in individual companies or sector often caused
by information published after trading hours, therefore not
available to the public and thus not affecting the
sentiment. Since the index includes companies from broad
range of sectors, it alleviates this issue by offsetting the
impact of sudden changes to other sectors of market
economy. The dataset contains 3066 data points,
representing every trading day in the specified period
from 4-01-2011 to 17-03-2023. During this timeframe, the
majority of trading days had negative percentual daily
change (1655 negative days in comparison to 1411
positive days). However, the average magnitude of the
positive daily changes outweighs the one of the negatives,
therefore the overall movement of the S&P 500 prices is
constantly rising in this period. Figure 4. Dataset Structure

The basic data for S&P 500 index (ticker

designation GSPC) and CBOE (ticker designation VIX)
were acquired through the use of YFinance python-
language library which is being continuously developed
and maintained by Yahoo Finance. The data range was
chosen upon the availability of sentimental data. The data

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 28

in this form contain daily values in the selected date The RMSE is defined by the following formula:
range, which are following: Date, Open price, Highest
achieved price for particular day, Lowest achieved price
for particular day, number of traded units for particular
day, closing price for the particular day and CBOE value
for particular day. It is important to note that despite not
belonging to the basic data group, CBOE is included in
the same acquisition step since it is listed as a separate IV. TEST RESULTS AND DISCUSSION
index and therefore can be accessed in similar manner to
S&P 500 data, while the remaining sentimental data had In this chapter, the best performing variants for each
to be obtained either by web-scraping or file-scraping. selected methods are displayed and compared to the rest.
Fear and Greed index data were acquired through web- The overall best performance was achieved by Temporal
scraping the CNN official website, extracting the daily Fusion Transformer representing the Transformer
values in a specified range. The daily value and its methods, which proved to display superior results. The
corresponding sentiments were obtained from the scraped summary of best performing models for each architecture
data; however, values of constituent metrics are not is portrayed by Table 1. The abbreviation SD stands for
included in the used dataset. data with sentiment included, the term standard refers to
AAII data were freely available for download at data without additional sentiment.
official AAII website. From the provided file, AAII TABLE I.
THE RESULTS OF BEST SELECTED DL MODELS FOR EACH
sentiment data which are collected from a weekly survey METHOD
conducted at AAII members were included in the
resulting dataset. The AAII data are represented as a Model LSTM N-Beats TFT
percentual value under ‘Bullish’, ’Bearish’ and ’Neutral’
Type of Dataset SD SD SD
columns. The first two are colloquial terms commonly
used to describe positive and negative sentiment present RMSE(Close) 78,852 84,345 77,863
among market participants. In this section of the data, the
moving-average (MA) was chosen as a metric intended RMSE(FG) 1,279 1,273 1,086
for application. In this work, moving averages were RMSE(% change) 0,985 0,929 0,939
calculated on 5,10,20-60-day basic to represent short-
term as well as medium-term attribute fluctuations. The
followed attributes upon which MA’s were computed As can be seen on Table 1, the TFT achieved lowest
were closing prices and CBOE to also gauge the RMSE values while predicting Close price and Fear and
sentiment fluctuation. By calculating the moving Greed index values. While predicting Percentual change,
average, the impacts of random, short-term fluctuations TFT scored RMSE value of 0,946, which earns it a second
on the price of a stock over a specified time frame were place following N-Beats. N-Beats has best score only in
mitigated. In this work, performed MA calculations were prediction of percentual change, however, provided worst
simple moving averages, therefore exponential method results while predicting Close price. When predicting Fear
which puts an emphasis on closer values on time range and Greed index this method achieved RMSE score of
was not applied. 1,273 earning it a second place in this comparison. The
LSTM was outperformed by its counterparts; however, it
C. Measures of Effectiveness of Models is important to point out that it scored second on close
The root mean square error (RMSE) measures the price prediction.
average difference between a statistical model’s predicted
values and the actual values. Mathematically, it is the V. CONCLUSION
standard deviation of the residuals. Residuals represent Our conclusion is that due to the volatile nature of used
the distance between the regression line and the data data which do not encompass all available information
points. RMSE quantifies how dispersed these residuals regarding the S&P 500 index and sentiment as well, the
are, revealing how tightly the observed data clusters differences may be prone to be caused by external factors
which are not reflected in used dataset. The performance
around the predicted values. It measures the scatter of the
of models in each of the followed attributes is comparable
observed values around the predicted values. and with little differences which may be attributable to
different settings. The models do predict close price
values which are close to the previous day values,
however in cases when abrupt price change occurs the
models do not perform satisfactorily, which is caused by
their behavior of placing the values close to previous
values which is in majority of cases the usual situation. In
terms of Fear and Greed index prediction, the behavior
remains similar to the close price prediction behavior.
Only different case occurs during percentual case
prediction, when LSTM and TFT models make
predictions which are close to the average percentual daily

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 29

change whereas N-Beats creates predictions in broader [5] Omer Berat Sezer, Murat Ozbayoglu, & Erdogan Dogdu (2017). A
range, however, the overall result is not improved based Deep Neural-Network Based Stock Trading System Based on
on RMSE. The experiments were conducted successfully Evolutionary Optimized Technical Analysis Parameters. Procedia
Computer Science, 114, 473-480.
considering the initial goal of this work. The sentimentally [6] Chenn-Jung Huang, Dian-Xiu Yang, & Yi-Ta Chuang (2008).
enriched dataset proved to increase the prediction Application of wrapper approach and composite classifier to the stock
accuracy in majority of test cases, however, the results trend prediction. Expert Systems with Applications, 34(4),2870-2878.
were not significantly improved in comparison to models [7] Erkam Guresen, Gulgun Kayakutlu, & Tugrul U. Daim (2011).
using dataset without sentiment. The automatic Using artificial neural network models in stock market index prediction.
recognition of the impact of the sentiment in online space Expert Systems with Applications, 38(8), 10389- 10397.
[8] Hugo Larochelle, Yoshua Bengio, Jérôme Louradour, & Pascal
provides increased information value to teachers, enabling Lamblin (2009). Exploring Strategies for Training Deep Neural
them to perform their duties with additional efficacy. Networks. J. Mach. Learn. Res., Volume 10, pp. 1–40.
[9] Deng, L. (2016). Deep learning: From speech recognition to
ACKNOWLEDGEMENT language and multimodal processing. APSIPA Transactions on Signal
and Information Processing, 5, E1. doi:10.1017/ATSIP.2015.22.
This work was supported by the Scientific Grant Agency [10] Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning
of the Ministry of Education, Science, Research and algorithm for deep belief nets. Neural computation, Volume 18, Issue 7,
Sport of the Slovak Republic, and the Slovak Academy of 1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527.
[11]Takashi Kuremoto, Shinsuke Kimura, Kunikazu Kobayashi, &
Sciences under grant no. 1/0685/21 and by the Slovak
Masanao Obayashi (2014). Time series forecasting using a deep belief
Research and Development Agency under Contract no. network with restricted Boltzmann machines. Neurocomputing, 137,
APVV–22–0414. pp.47-56.
[12] Hochreiter, Sepp & Schmidhuber, Jürgen. (1997). Long Short-term
REFERENCES Memory. Neural computation. Volume 9. pp.1735-80.
doi:10.1162/neco.1997.9.8.1735.
[1] Fama, E. F. (1965). Random Walks in Stock Market Prices. [13] Boris N. Oreshkin, Dmitri Carpov, Nicolas Chapados, & Yoshua
Financial Analysts Journal, Volume 21Issue 5, pp. 55–59. Bengio (2020). N-BEATS: Neural basis expansion analysis for
[2] Malkiel, B. (2003). The Efficient Market Hypothesis and Its Critics. interpretable time series forecasting. In International Conference on
The Journal of EconomicPerspectives, Volume 17, pp. 59-82. Learning Representations.
[3] Wan, X., Yang, J., Marinov (2021) Sentiment correlation in [14] Vaswani, A. ; Shazeer, N; Parmar,N ; Uszkoreit, J ; Jones,L ;
financial news networks and associated market movements. Sci Rep 11, Gomez, A.N ; Kaiser, Lukasz & Polosukhin, I(2017). Attention Is All
3062. You Need, ’Advances in Neural Information Processing Systems’.
[4] Murphy JJ (1999) Technical analysis of the financial markets: A 5598-6008.
comprehensive guide to trading methods and applications. ISBN 978-0-
7352-0066-1.


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 30

 The Role of Diversity in Process of Education
and their influence on the productivity of work
A. Balcova, P. Balco, F. Delaneuville
Faculty of Management, Comenius University in Bratislava, Slovakia
Odbojárov 10, P.O.BOX 95, 820 05 Bratislava 25
balcova9@uniba.sk , peter.balco@fm.uniba.sk, frederic.delaneuville@fm.uniba.sk,

Abstract — The use of diversity in the educational process and Diversity in its broadest sense refers to the idea that the
in creating an effective work environment is essential to aim is to create conditions throughout society, and
ensure quality education, work productivity, and the especially in the business world, that enable all people,
promotion of science and research. The process of regardless of their individual differences, to develop to their
implementing diversity whether in an educational or work full potential and to show their personal potential. It is an
environment is challenging nowadays, and in some countries, opportunity to ensure a cultured society through an
it is a topic that may be difficult to understand and alien to understanding of each other's diversity and, at the same
certain individuals in society. For a good implementation of time, a solution for eliminating discrimination and
diversity in society, it is necessary to prepare the environment prejudice in the world. Diversity in teams, in society
and the individuals within it who will proactively promote promotes different perspectives that lead to more effective
this policy. decision making and also develops creativity [9].
Key words: DEI, diversity, equity, inclusion, education
C. Dimensions of Diversity
A. Introduction
According to Hubbard [3], diversity consists of two
Diversity is any element by which differences between dimensions. The inner circle is called the core and is
groups or individuals can be identified. By accepting that composed of six parts. The outer layer consists of parts that
people are different in terms of age, gender, ethnicity, are tied to the primary and we refer to them as secondary.
religion, disability, sexual orientation, education, or The components of the nucleus have a significant influence
nationality, we understand that there are some visible as on human socialization at every single stage of life. These
well as invisible differences between us. By taking a variables represent the qualities and characteristics that are
proactive approach to the topic of diversity, it is the part of the core of each individual. Humans have different
invisible aspects of diversity that come to the fore, which dimensions of diversity that are shaped by the environment
can greatly influence the working environment [4]. in which they grow up and live. The secondary dimension
and the so-called mantle of diversity are made up of
elements that play a very important role in shaping people's
B. The Role of Diversity values and personal experiences. These elements can
Diversity is everywhere in the world and is shaped by the include work environment, communication style, socio-
socialisation of individuals in different communities. In our economic and family status, religion, educational style and
neighbourhood there are people with different traditions, so on.
experiences, values and, of course, education. Learning These dimensions can help us to understand, in
about these cultural differences has its advantages for perceiving ourselves and our surroundings. The primary
anyone who is not afraid of them and is willing to learn dimensions represent fundamental and irreversible aspects
about them. Making use of these differences in business of our identity, thus including our genetic characteristics
companies helps to break down stereotypes and also that we tend to consider as part of our natural being.
prejudices in the workplace. In addition, people from other Humans are more sensitive to them because they cannot
cultures contribute to the development of the company with change them and only some can be influenced.
their language skills, way of thinking, experience and
On the other hand, secondary dimensions are less
different perspectives.
obvious and more influential. These include aspects that we
Companies in Slovakia are aware of this situation and are have chosen or that we have created in the course of our
following and copying trends from abroad. According to a lives. These dimensions give us more choice and control
PWC (2018) survey [8], 97% of Slovak-based companies over who we become. Figure 1 figure one illustrates these
surveyed say that diversity is one of their core corporate dimensions.
values. Moreover, according to the results, up to 73% of
companies in Slovakia actively communicate their
diversity goals in public communications. This clearly D. The Role of Diversity in Process of Education
signals their commitment to transparency and commitment
There are a number of reasons for promoting diversity
to this topic. Employees of these companies have regular
access to information on their diversity achievements and in the educational process, including human rights or
up to 83% of companies regularly communicate their individual or societal benefits [6].
progress on diversity and inclusion to their employees The OECD's Strength through Diversity project has
within their teams and management. identified six main themes that are also the pillars for

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 31

creating an equitable and quality education system. In which we perceive and analyse the situations and people
addition to these six dimensions, however, the authors take around us; this lens is influenced by our thoughts,
into account two other factors, namely geographical experiences, and worldview. For many organisations, the
location and socio-economic position in society. Figure 2 value of diversity in culture and mindset has become
shows the intersectionality between diversity dimensions apparent, driving innovation by enabling the discovery of
new ideas that change the way industry, product and service
and overarching factors.
sectors think and act [7].
Armache (2012) states that organizations that actively
promote diversity in the workplace can achieve several
significant benefits. In addition to becoming more
competitive, they have the ability to influence the
perceptions of external stakeholders regarding the topic of
diversity within their organization [1].
Diversity in the workplace brings many potential
benefits. One of them is improved decision-making. When
an organization has a team with diverse perspectives and
experiences, it can lead to more informed and
comprehensive decisions. Different perspectives and
opinions coming from different cultures and backgrounds
can enrich discussions and lead to better solutions to
problems.
Improved problem solving is another significant benefit
of diversity. Diversity allows an organization to approach
problems from different angles and to seek new, innovative
approaches. A team that is able to combine different
perspectives is often more effective in finding creative
solutions.
Diversity also helps to increase idea generation. When
people of different cultures and backgrounds work
Figure 1. Dimensions of Diversity (Source: Hubbard et al. 2004) together, they create a fertile environment for the exchange
of ideas and thoughts, which can lead to new projects and
initiatives. This creative environment can help an
organization innovate and grow. In addition, working in a
diverse environment is also beneficial for company
employees, who can develop their own views and gain
greater confidence in their own unique talents, qualities and
ideas.
Last but not least, diversity in an organisation can
strengthen its ability to expand into international markets.
With teams that understand different cultures and markets,
an organization is better prepared to operate successfully in
a global context. It can more easily expand its product and
service offerings to different regions.
Along with these benefits, organizations that promote
Figure 2. Intersectionality between dimensions of diversity and diversity also actively shape their perceptions among
overarching factors (Source: OECD Education Working Papers No. 260
et al. 2021) external stakeholders. To them, they become known as
innovative and inclusive, which can positively influence
The OECD report highlights the importance of taking a their reputation and relationships with clients, business
holistic approach to diversity in education. Only then can partners and the public. Overall, diversity in the workplace
we ensure effective education where students are supported is not only a growth strategy, but also a way for
regardless of their background, origin and other variables organizations to contribute to positive change in the world.
that distinguish them from their peers. F. Best practice of implementing diversity
E. Diversity in the workplace Today, companies have various options to implement
Business companies face many challenges today and one diversity in their company. There are also a number of
of them is the implementation of diversity in the workplace alternative programs from which employers can select only
and its active use. We are moving in a global world that is certain elements of diversity as a current priority for their
increasingly interconnected and it is necessary for company.
companies to respond adequately to such an environment. Based on various studies, Kreiz (2008) identified the
Therefore, if a business company is interested in building a most important attributes for effective implementation of
globally competitive organisation, it needs to integrate diversity policies in the workplace. These are:
diversity into it. Top leadership commitment - the approach of senior
Diversity plays a key role in stimulating innovation. leaders will be proactive and clear about the division's
Quite simply, each of us has our own unique lens through

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 32

policy, and they will communicate their vision to their highest number of responses, more than one-third, from
subordinates. people aged 21 to 26. The analysis of the respondents is
Diversity as part of an organisation's strategic plan - the shown in the graph shown in Figure 4.
theme of diversity is aligned with the organisation's
mission.
Diversity linked to performance - the organisation
understands that a diverse work environment can increase
productivity and contribute to better performance at both
44;
the individual and organisational level. (47%) women
49;
Measurement - a set of quantitative and qualitative (53%) men
measurements are used to assess the impact of different
aspects of the diversity program.
Accountability - hold leaders accountable for achieving
diversity goals by linking their performance reviews and
rewards to diversity progress.
Succession planning - the organization systematically Figure 3. Sex of respondent
identifies and develops diverse talent to prepare future
leaders and work with the company's succession pipeline.
Recruitment - the process of attracting qualified 37
candidates who contribute to the diversity of the 40
organisation and the use of blind recruitment. 30 26
Diversity training - The organization systematically 17
identifies and develops diverse talent to prepare future 20 12
leaders [5]. 10
1
Internal audits are now in place to verify the
effectiveness of implementation. However, in countries 0
where the diversity environment in companies is also 15-20 y 21-26 y 27-35 y 36-50 y 50 and
defined by national legislation, there are external audits and older
controls directly established for this issue.
Figure 4. Age of respondent
In Slovakia, the topic of diversity in the work
environment is still voluntary. Nevertheless, in 2017, a We also learned from the initial redistribution of the
Diversity Charter was created on the initiative of sample that more than 95% of the respondents were of
companies and organisations, which is part of the European Slovak nationality and the remaining respondents were of
Diversity Charter Platform. Its ambassadors are mostly four other different nationalities.
larger companies, often with an international background.
In further identifying the respondents, we also looked for
The role of the Diversity Charter in Slovakia is to further
a correlation between their age and length of employment.
spread the idea of a healthy working environment in
Based on the data, we found a direct correlation, i.e., the
companies that will be able to thrive. It also offers the
older the employees are the longer they have worked for a
opportunity in Slovakia to measure the diversity of your
given trading company.
workplace. This tool is primarily designed for employers to
be able to check what level of diversity their company is The last part of the identification of employees, was their
achieving after inputting their company data [4]. job classification. The most frequent enquirers were
operations management with a greater proportion of almost
II. SCOPE OF RESEARCH AND RESEARCH 40% (36) and the second most frequent were rank and file
METHODS employees (22). The rest of the respondents were working
at the head office, as senior management or as operations
The subject of our research were employees of an administrators.
international trading company operating in Slovakia. In
total, we managed to collect 93 responses. Men to women III. EVALUATION OF THE QUESTIONNAIRE
participated in the survey in almost equal proportions, 47% DIGITIZATION OF THE UNIVERSITY
to 53% for women. The graph in figure 3 shows the ENVIRONMENT
responses of the respondents.
The purpose of the survey was to find out the employees'
education in the field of diversity and also their perception
We analysed the views of employees from a number of of the situation of this issue in their workplace. The output
perspectives: is recommendations and measures to promote a diverse
- age, workplace environment. Since we have limited space here,
- the nationality of the respondent, we present only the most important findings from our
- the duration of the employment relationship, survey. The first question we asked focused on the term
diversity itself and whether employees had already
- sector of operation, encountered the phrase in the workplace.
- according to the job position. Based on the information obtained, we can determine
We found that the research involved a diverse age how the employer works in the field of education on this
composition of the company's employees. We received the issue. It should be mentioned that the research took place

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 33

in a business company which is one of the ambassadors of 48;
60
the Diversity Charter in Slovakia. As this is a survey from (51,61%)
2021 it is possible that education in this area has been 50
pushed to a higher level. However, the data we have 40 26;
available to date is not entirely satisfactory. More than 30 (27,96%) 16;
50% of the respondents are familiar with the term (17,2%)
diversity, but only almost 35% of the total number of 20
respondents have encountered the term in their 10 3; (3,23%) 0
organisation. The rest of the employees (43) have not 0
encountered the term, which is not insignificant for the Definetly Yes Maybe No Definetly
given sample of respondents [2]. This information is yes no
presented in Figure 5.
Figure 6. Evaluating creativity in a workplace made up of a
43; multicultural team.
50
(46,2%)
32;
40
(34,4%)
30 4; (4%)
18;
(19,4%)
20
10 Yes
21; (23%)
0 No
Yes No Encountered the 68; (73%)
I don´t know
term outside the
organisation

Figure 5. Encountering the concept of intercultural diversity

The next question focused on employees' subjective Figure 7. Disadvantage or discrimination against people of pensionable
opinion regarding the interestingness of working in a age, disabled people, or young parents
multicultural team. It is interesting to observe that only a
very small proportion, namely 3,23% of employees,
answered negatively (no) to this question. This means that In the following graph in figure 8, we can see that there are
very few consider working in a multicultural team to be also employees in the company who show prejudice or are
uninteresting or unfavourable in terms of diversity. At the reluctant to work together in a multicultural team. The
same time, 17,2% of respondents were undecided, reasons for this behaviour can be varied, including
indicating that they are undecided or have mixed feelings language barriers, negative past experiences, or fear of the
about the interestingness of working on a multicultural unknown. However, it is positive that the percentage of
team. This group of people may have mixed experiences these employees, namely 4%, is relatively low.
or feel that it depends on the specific situation and team. Interestingly, those employees who indicated that they
The majority of respondents who answered the question have a problem working in a culturally diverse team, when
expressed a positive view. They either said a definite 'yes' asked "why?" did not answer. This may suggest that these
or were positive about working in a multicultural team. issues may be complex and subjective, and not always easy
The evaluated results are illustrated in the graph in the to explain.
following figure 6.
Overall, the results of this question suggest that most
employees find working in a multicultural team a positive 4; (4%)
and interesting aspect that contributes to their positive
perception of diversity in the work environment.
In the survey, employees were given the opportunity to
give their views on whether they felt that people of Yes
pensionable age, disabled people or young parents in their No
work environment were discriminated against. It is
important to note that only a very small number of 89; (96%)
respondents (4) indicated that they believed these groups
were discriminated against. While this is a small number,
it is necessary to take such a situation seriously as well.
The overall results of employees' responses to this question
are illustrated in Figure 7. Figure 8. Challenging to work in a multicultural team

It is important to stress that, despite society's efforts to

prevent discrimination and problems in the workplace, it is
not always possible to eliminate these problems

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 34

completely. The graph in Figure 9 illustrates employees' organisation in question has measures in place to promote
responses to whether they have experienced discrimination effective communication or provides training for
in the workplace. More than half of respondents marked employees on teamwork issues in the workplace, etc.
their answer as a definite "no," indicating that some
employees may feel that there is a discrimination or
inequality problem in the organization that is not being 12; (13%)
fully addressed.

60 52;
Yes
(55,92%)
50 No
32; 81;
40 (34,4%) (87%)
30
20
9; (9,68%)
10
Figure 11. Misunderstandings or miscommunications about cultural
0 differences between the respondent and his or her supervisor
Yes No I don´t know
The previous chart has indicated to us the fact that
Figure 9. Problem situation related to culture, origin, age or religion. problems related to cultural differences in the employee-
supervisor relationship are not frequent in the company in
Although some organisations strive to create a family- question. This positive finding was further reflected in the
like environment for their employees, this approach is not graph below in figure 12, where up to 91.4% of employees
always applied equally to all. are satisfied with the communication with their supervisors.
One of the problems that can arise in organisations is the This high level of satisfaction with communication with
favouritism shown to some employees by managers. This supervisors indicates that there is a good communication
means that some employees may be favoured and given culture in the organization that enables effective transfer of
more opportunities or better conditions than others. If this information between employees and supervisors.
behaviour is based on personal preference or unfair Employees feel comfortable communicating with their
discrimination, it can lead to dissatisfaction among supervisors, which is a key aspect of creating a productive
employees and a deterioration in the working environment. and harmonious work environment.
Figure 10 shows the results of the survey.
60 51; (54,8%)
7; (8%) 50
40 34; (36,6%)

30
Yes 20
7; (7,5%)
No 10 1; (1,1%)
86, 0
92% Extremely Satisfied Less satisfied Dissatisfied
satisfied

Figure 12. Employee satisfaction with workplace communication in

relation to supervisors
Figure 10. Equal treatment of employees in senior positions

Another issue can be cultural differences in the work It is interesting to find that up to 26.7% of employees
environment, which can indeed lead to misunderstandings, said that they had no problem having a conversation on any
but it is important that organisations capture and address of the above topics. All topics and respondent results are
these situations early. An interesting finding is that up to presented in Figure 13. This suggests that a proportion of
87% of respondents reported that there were no significant employees are quite open and comfortable discussing a
misunderstandings related to cultural differences. This may variety of topics, including those that may be sensitive or
indicate that organisations are proactive in addressing these controversial. Other topics mentioned by respondents
issues and trying to prevent or resolve them. Nevertheless, include the COVID-19 pandemic situation and topics that
it is important that organisations continue to promote an might hurt or make some colleagues personally
inclusive working environment and raise awareness of uncomfortable. Respondents indicated that they are not
cultural differences to minimise the risk of preoccupied with having conversations about the topics
misunderstandings and conflict. mentioned below, such as national politics, religion, racial
issues, and the like, but they have to be careful who they
It is encouraging to see the results in figure 11 showing say what in front of. It is possible that the workplace is
that most respondents (81) reported no misunderstandings populated by people with different views and values, and
related to cultural differences. This may indicate that the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 35

therefore some topics could become controversial or beneficial for both the organization and its employees. The
occasionally confrontational. In such cases, it is important answers of the respondents are illustrated in Figure 14.
that employees can communicate openly, respecting the Several important points can be gleaned from the results.
views of others and creating an atmosphere of tolerance and Only 3.23% of the employees said that they did not see a
mutual understanding. direction that the trading company should be more focused
on. The rest of the respondents voted for different directions
which are mentioned in the chart below. Tribal employees
50
44;(26,7%) and freelances (28), gender balance in management (19),
45 generational change in the company (17), and other topics
40 were identified as the most common responses.
35
28;(17%)
According to the answers of the respondents and their
30 initial data, we can find a certain correlation concerning
14,5;(14,5%)
25 16;(9,7%) their education as well as their job function. The business
20
company in question is known for its training processes that
17;(10,3%)
14;(8,5%)
are used across the company. Although the respondents
15 6;(3,6%) showed that they are less familiar with the concept of
9;(5,5%) diversity itself, the training of the employees leads to the
10 7;(4,2%)
employees themselves noticing many situations and being
5
able to make better decisions based on their knowledge.
0
IV. OBSERVATION AND DISCUSSION
Nowadays, employees have a big advantage in that they
are the ones who choose where to work, as there are so
many options nowadays. Companies therefore need to be
responsive and able to attract potential candidates with their
organisational culture. The younger generation that is
entering the workforce today is relationship-oriented and
they are also looking at what the values of the organisation
are.
Figure 13. Topics that employees avoid in workplace conversation.
The 93 employees of an international company operating
in Slovakia voluntarily participated in our survey. Based on
30 28; (24,3%) the questions we asked the respondents; we agreed on three
main recommendations concerning training in the
25 organisation.
17; (14,8%)
19; (16,5%)
The first recommendation is that the company should
16; (13,9%)
20 place greater importance on educating employees, and
16; (13,9%) especially managers, about the benefits of diversity in the
15 13; (11,3%) working relationship. Companies cannot assume that all
employees will be actively informed about this issue and
10 understand what it means for their job role. There is a need
6; (5,2%) to empower employees in leadership roles with the skills to
5 be able and, more importantly, willing to lead a diverse
team. Employee training has a huge impact on productivity
0
and the very success of companies. It is therefore necessary
to consider what forms of training are appropriate for
employees and the work environment. Which points are
critical, and which need to be addressed more so that all
employees feel that their workplace is a safe place for them.
One possible form of training could be team building
focused on cultural diversity. Tailor-made teambuilding
sessions are now offered by various companies that are
making a profit specifically around educating employees of
other companies.
Our second recommendation is the use of blind
recruitment, where the recruiter selects employees based on
experience. However, it is worth mentioning that nowadays
Figure14. An area of diversity in which a business company can
it is also a good idea to focus on the soft skills of the job
improve.
candidate. At the same time, when advertising job positions
The last question asked on the topic of diversity in the or mass recruitments, it is necessary to be open to possible
new employees. Finally, make sure that the job
workplace focused on the subjective opinion of the specifications are as specific as possible so that people
respondents, which can provide valuable recommendations know what they can expect from the position.
for the business company itself. These opinions can be

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 36

 Certainly, the last point is very interesting, which is the to the commercial company in which the survey was
exchange of employees within the Slovak market. The next conducted and to all the participants.
level could be an international exchange, so that two
employees from different countries are exchanged for a REFERENCES
certain period. Of course, such cooperation would require a
lot of effort, but ultimately it would promote diversity in the [1] ARMACHE, J. Diversity in the Workplace: Benefits and
company in practice. Challenges [online]. 2012. s. 60 At:
http://files.transtutors.com/cdn/uploadassignments/1968642_5_evi
Thus, the business company surveyed in this research dence-article-2.pdf
still can develop in different areas of diversity in the [2] Balcová A, Bakalárska práca Diverzita v obchodných
workplace, where it will underpin its norms and standards spoločnostiach [online]. (2021). At:
for its employees. https://opac.crzp.sk/?fn=detailBiblioFormChildS1AUO4&sid=4A
Diversity is an element that came with globalisation in 7927334F9373E9274CDD99785B&seo=CRZP-detail-kniha
the 20th century and this process cannot be stopped in the [3] HUBBARD, Edward E. The Manager´s Pocket Guide to Diversity
modern world we live in. It is a challenge that can be an Management. Amherst: Human Resource Development HRD
Press, 2004, s 27-32. ISBN 0-87425-761-1.
opportunity for many companies, and it is up to them and
[4] Charta diverzity Slovensko [online]. (2023). At:
their leadership to embrace the new realities. However, the https://www.chartadiverzity.sk/charta-diverzity-sr/
sooner companies start to react to the emerging changes, [5] Kreitz, P. A. (2008). Best practices for managing organizational
the sooner they will get a quality and modern environment diversity. The Journal of Academic Librarianship, s 3-6.
for their employees. [6] Mezzanotte, C. (2022), The social and economic rationale of
inclusive education: An overview of the outcomes in education for
CONCLUSION diverse groups of students (Sociálne a ekonomické odôvodnenie
inkluzívneho vzdelávania: prehľad výsledkov vzdelávania pre
Diversity is perceived as a very sensitive topic in society, rôzne skupiny študentov), OECD Education Working Papers, č.
but few managers are aware of its added value for the 263, OECD Publishing, Paríž
development of the organization. The presented survey [7] O’Donovan, D. (2018). Diversity and inclusion in the workplace. In
Organizational Behaviour and Human Resource Management s 4-
and subsequent analysis opened space for discussion as 11
well as other studies necessary to confirm or modify our [8] Prieskum v oblasti diversity a inklúzie na Slovesnku 2018 [online].
findings. (2018). At: diversity-inclusion-survey-sk.pdf (pwc.com)
[9] VELÍŠKOVÁ, H. Víc (různých) hlav víc ví. Diversity Management
ACKNOWLEDGMENT – přínosy rozmanitých pracovních týmů (2007) Praha: Nový
Prostor, o.s., s 16. ISBN: 978-80-903990-0-6.
[10] The Radical Transformation of Diversity and Inclusion: The
We would like to thank all interested parties for their Millennial Influence [online]. (2015). At: us-inclus-millennial-
cooperation and support in this project. The main thanks go influence-120215.pdf (deloitte.com)

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 37

 Asynchronous Motor Speed Servo Drive
I. Bélai* and I. Bélai**
Institute of Automotive Mechatronics,
Faculty of Electrical Engineering and IT,
Bratislava, Slovakia
*
igor_belai@stuba.sk, **igor.belai@stuba.sk

Abstract— In order to improve the quality of education about rated power Pn = 250 W, rated rotational speed nn = 1350
electric servo drives at the IAM FEI STU in Bratislava, a rpm and rated torque Mn = 1.8 Nm.
workstation was built. It contains an electric drive that can
operate in torque and speed modes. The workstation contains
The SINAMICS G120 frequency inverter (Siemens) is
an asynchronous motor, an industrial electrical inverter, a designed for standard asynchronous motor applications
programmable logic controller and two personal computers. with lower precision and dynamic control requirements. It
Students can perform experiments with a speed servo drive consists of two essential units:
using Matlab and Starter software.  power module,
 control unit.
I. INTRODUCTION The power module supplies three-phase AC voltage to
Algorithms for controlling electric servo drives using a the asynchronous motor. The control unit controls and
torque generator are being taught at the Institute of monitors the power module and the motor connected to it.
Automotive Mechatronics (IAM) FEI STU in Bratislava. It also supports communication with the higher-level
The torque generator consists of an electric motor and an control unit or control elements via the serial
electric inverter with an appropriate control algorithm to communication bus (PROFInet or PROFIBUS) or via
control the motor torque. In the case of a DC motor, its analog and digital inputs/outputs (I/O).
torque is directly proportional to the rotor current [1], but The inverter can, from a motion control point of view,
the situation is more complex for AC motors and therefore
vector control or direct torque control (DTC) methods are operate in three modes [3]:
used to control the torque [1]-[2]. 1. V/F control – when the inverter controls the magnitude
It is then necessary to implement a speed controller and frequency of the motor voltage [4]. This mode is
which generates a torque setpoint at the input of the torque primarily used in systems without feedback control,
generator. The torque setpoint is calculated from the such as fan or pump drives.
difference between the speed setpoint and the actual speed. 2. Torque control – the inverter controls the motor torque
In practice, a PI speed controller is usually used. using the vector control method [1]. This operating
In practical exercises focused on the topic of speed servo mode is used in applications with higher demands on
drives, students derive formulae to calculate the parameters the dynamics of the motion control.
of the speed controller for a given electric drive. They then
use a simulation to verify the characteristics of the control 3. Speed control – in this operating mode, the inverter
loop. controls the rotational speed, while vector torque
control is also applied.
In order to verify the characteristics of the speed control
loop on a real asynchronous motor (AM) drive, a The PM 240 inverter power module is powered by a
workstation has been set up at the IAM FEI containing an three-phase 400 VAC voltage. To limit interferences to the
AM powered from a three-phase grid through an electrical power grid, an EMC filter is installed at the input of the PM
inverter with implemented vector control to control the 240 module. The asynchronous motor stator windings are
motor torque. The speed can be controlled by a controller connected to the output of the PM 240 module. An
implemented in the inverter or by an external control unit. incremental rotary encoder (IRC) of the type 1XP8001-1
[5] is attached to the AM shaft and it is used for the
The paper describes the workstation hardware in Section
evaluation of the position of the rotor. The rotor position is
II, then discusses the speed control loop structures in
used for motor torque control by vector control method and
Section III, followed by the description of the identification
for motor rotational speed estimation while controlling the
process of the moment of inertia of the motor in Section IV,
motor speed.
the speed controller tuning in Section V, the
implementation of the speed controller in an external The CU250S-2 PN control unit implements algorithms
controller, and illustrative experimental results in Section to control the stator power supply and the rotor motion.
VI. Setpoints (supply voltage frequency, motor torque, or rotor
rotation speed) and control signals (e.g., fault
II. DESCRIPTION OF THE WORKSTATION acknowledgement, power module enable, etc.) can enter
the inverter control unit via two channels: 1) via the analog
Fig. 1 shows a block diagram of the workstation with and digital inputs of the inverter (AI, DI), or 2) via the
the SINAMICS G120 inverter that powers the PROFInet interface. The same channels can also be used to
asynchronous motor (AM) type 1LA70704AB60-Z with

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 38

 Figure 1. Schematic diagram of the workstation

transfer status information and the actual value of the

controlled variable from the inverter to the higher-level
control module.
In the presented workstation, the torque or speed
setpoints can be generated in the CPU1512C central
processing unit representing the programmable logic
controller (PLC) of the SIMATIC S7-1500 series, which is
connected via PROFinet to the inverter control unit and the
programming/configuration computer PC 1. The actual
motor speed can also be transferred from the inverter to the
PLC via PROFInet.
The reference torque or speed can also be generated by
a PC 2 personal computer with the MF634 multifunction
I/O card inserted, containing A/D and D/A converters to
evaluate and generate voltage signals within ±10 V range.
When the drive is controlled by the PC 2, the analog output
of the MF634 card is connected to the analog input of the
inverter control unit. On the other hand, the actual torque
and speed values are transmitted from the analogue outputs
of the CU250S-2 PN inverter control unit to the analog
inputs of the MF634 card.
The user can enter the speed setpoint on the control Figure 2. The layout of the control and display elements on the control
panel shown in Fig. 2, which consists of the toggle panel
switches Sw 1 - Sw 11, the pushbutton switches PB 1, PB
2, the potentiometer P 1, and a digital voltmeter to display to acknowledge a fault if a fault occurs during inverter
the output voltage of the potentiometer. The toggle operation.
switches and pushbuttons are connected to the digital The USB interface is used to commission and configure
inputs of the PLC. The potentiometer output and the the inverter, connecting it to a PC 1 personal computer with
voltmeter are connected to the analog input and output of the necessary software installed. The Starter and TIA Portal
the PLC. The user can enter the speed setpoint in binary (Siemens) software can be used to configure the
code using the switches Sw 4 -Sw 11 or using the multi- SINAMICS G120 inverter. The Starter software [3] has
turn potentiometer P 1. Pushbutton switch PB 1 is used to been used to configure the described workstation, as it
features a clear structure of the inverter setup elements and
acknowledge the speed setpoint. Pushbutton PB 2 is used

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 39

 contains all the necessary tools for its efficient window shown in Fig. 4. For the speed controller gain Kp,
commissioning. Kp = P gain factor/1000, where P gain factor is the value
entered in the Speed controller window.
III. SPEED CONTROL LOOP STRUCTURE
If the SINAMICS G120 is operating in Torque control
A speed control loop with an asynchronous motor and a mode (see Fig. 3b), the speed controller is implemented in
SINAMICS G120 inverter can generally be implemented PC 2. The values of the torque setpoint Mm* and the actual
in two ways, which differ in the speed controller speed n are transferred to/from the inverter via its analog
implementation: input and output. The speed controller runs on PC 2 and it
1. The speed controller is implemented in the inverter, is implemented using Matlab and Simulink Desktop Real-
i.e., the inverter is configured in Speed control mode Time. The values of the run enable and fault acknowledge
and the speed setpoint n* enters the inverter. signals (Enable, Err_Ack) are set by the user on the control
2. The speed controller is implemented in an external panel (CP) and sent to the inverter via PROFInet. With this
control unit, the torque setpoint Mm* enters the inverter method of implementing the speed controller, the user can
and the inverter is configured in Torque control mode. implement virtually any type of speed controller and
possibly use a different control structure. From a teaching
Fig. 3 shows the block diagrams with the
interconnection of the workstation elements for the two point of view, it is also advantageous that the controller is
methods of implementing the speed controller mentioned implemented in Simulink, with which the FEI STU
above. students already have experience.
When the SINAMICS G120 is in Speed control mode Fig. 5 shows the structure of the speed control loop for
(see Fig. 3a), then the user enters the speed setpoint n* at the above two speed controller implementations when
the control panel (CP). The PLC user program sends this neglecting the dynamics of the torque generator, where:
value to the inverter via PROFInet, together with the run B – viscous friction coefficient,
enable (Enable) and fault acknowledge (Err_Ack) signals. J – moment of inertia,
The actual motor speed n is estimated in the inverter
Kp – speed controller gain,
control unit from the incremental IRC position sensor
signals and sent to the PLC via PROFInet. The actual ML – load torque,
speed value n is then transmitted via OPC UA to PC 1, n – rotational speed in revolutions per minute,
with the PLC acting as the OPC UA server and the Matlab Td – transport delay,
OPC UA client running on PC 1. A certain disadvantage is Tf – time constant of the speed setpoint filter,
the fact that the speed controller must be of the PI type and
Ti – time constant of the speed controller integration
the control loop has a fixed structure. The user can only set
component,
the values of the speed controller parameters, but for this
it is necessary to use the Starter program, where the ω – angular velocity in radians per seconds.
parameter values are entered in the Speed controller The variable PC/G120 is used to select the
implementation method of the speed controller. If
PC/G120 = 0, then the inverter operates in Speed control
mode, the controller is implemented in the inverter and the
actual speed signal is filtered by a filter with time constant
TN = 4 ms.
If PC/G120 = 1, then the speed controller is
implemented in an external controller (PC 2), the actual
speed is not filtered, but there is a transport delay Td =

a)

b)
Figure 3. Block diagram of the speed control loop with
SINAMICS G120 inverter: a) the speed controller is implemented Figure 4. The Starter software window for entering the parameter
in the inverter, b) the speed controller is implemented in a personal values of the PI speed controller implemented in the SINAMICS G120
computer inverter

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 40

 Figure 5. Speed control loop structure

12 ms at the inverter input, which delays the torque

setpoint at the inverter input and hence the actual motor
torque Mm. The speed setpoint is filtered by a filter with
time constant Tf ≥ 0 s.

IV. MOMENT OF INERTIA IDENTIFICATION

Before implementing the speed controller using
Simulink Desktop Real-Time, the motor moment of inertia
J was identified according to (1). Where ε is the angular
acceleration and a Δω is the change in angular velocity
over a time interval of length T.
ߝ ߂߱
‫ܬ‬ൌ ൌ (1)
‫ܯ‬௠ െ ‫ܯ‬௅ ܶሺ‫ܯ‬௠ െ ‫ܯ‬௅ ሻ
The inverter was configured in Torque control mode
and a rectangular waveform motor torque setpoint was
generated at its input. Its magnitude varied from −0.05 Nm Figure 6. Speed response when identifying the moment of inertia
to +0.45 Nm. The period of the torque setpoint signal was
TM* = 0.15 s.
݊ሺ‫ݏ‬ሻ ͵ͲȀߨ
Fig. 6 shows the motor speed response to the rectangular ‫ܨ‬௦ ሺ‫ݏ‬ሻ ൌ ‫ כ‬ሺ‫ݏ‬ሻ
ൌ ݁ ି்೏௦ (3)
waveform motor torque setpoint. The moment of inertia has ‫ܯ‬௠ ‫ ݏܬ‬൅ ‫ܤ‬
been calculated according to (1), when the motor torque In deriving the closed-loop transfer function, the
Mm = 0.45 Nm is applied at a time interval of length transport delay has been replaced by the transfer function
T = TM*/2 = 0.075 s and at a constant load torque of a first-order system with unit gain and time constant Td
ML = 0.23 Nm. The load torque has Coulomb friction according to (4). This substitution is based on the equality
characteristics. Ten moments of inertia were calculated for of the integrals of the difference of the steady-state value
ten speed changes over the time interval 1.5 s < t < 3 s, and the instantaneous value of the output for a unit step [7].
from which an arithmetic mean was calculated, giving the
identified value of J = 0.65×10-3 kgm2. ͳ
݁ ି்೏௦ ൎ (4)
ܶௗ ‫ ݏ‬൅ ͳ
V. SPEED CONTROLLER TUNING
If the transport delay in (3) is replaced by its equivalent
The speed controller is of PI type with transfer function first-order transfer function in (4), then (5) holds for the
Fc(s) in (2). The derivation of the formulae to calculate its equivalent transfer function of the system Fsn(s), and in (6)
parameters when the controller is implemented in an it is the closed-loop transfer function F(s) for Tf = Ti, where
external control unit, i.e., PC 2, is given below. for a polynomial N(s) in the denominator of F(s), (7) holds.
‫ܭ‬௣ ሺܶ௜ ‫ ݏ‬൅ ͳሻ ͵ͲȀߨ
‫ܨ‬௖ ሺ‫ݏ‬ሻ ൌ (2) ‫ܨ‬௦௡ ሺ‫ݏ‬ሻ ൌ (5)
ܶ௜ ‫ݏ‬ ሺܶௗ ‫ ݏ‬൅ ͳሻሺ‫ ݏܬ‬൅ ‫ܤ‬ሻ
The Pole Placement Method (PPM) was chosen to tune
the controller parameters. PPM allows the controller ͵Ͳ‫ܭ‬௣
parameter values to be calculated if the transfer function of ݊ሺ‫ݏ‬ሻ ߨ‫ܶܬ‬௜ ܶௗ (6)
‫ܨ‬ሺ‫ݏ‬ሻ ൌ ‫כ‬ ൌ
the controlled system and its parameter values are known ݊ ሺ‫ݏ‬ሻ ܰሺ‫ݏ‬ሻ
[6]. By selecting the closed-loop poles, the user can tune
the desired dynamics of the control loop. ‫ ܬ‬൅ ‫ܶܤ‬ௗ ߨ‫ ܤ‬൅ ͵Ͳ‫ܭ‬௣ ͵Ͳ‫ܭ‬௣
In this case, the transfer function of the controlled system ܰሺ‫ݏ‬ሻ ൌ ‫ ݏ‬ଷ ൅ ‫ ݏ‬ଶ ൅‫ݏ‬ ൅ (7)
‫ܶܬ‬ௗ ߨ‫ܶܬ‬ௗ ߨ‫ܶܬ‬௜ ܶௗ
Fs(s) is composed of the transfer functions of the transport
delay Td and of the mechanical subsystem of the motor The values of the parameters J, B, Td are known. The
according to (3). values of the controller parameters Kp, Ti have to be

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 41

 determined so that the transfer function F(s) has the desired
poles. The triple real pole −ω0 has been chosen, with a
corresponding characteristic polynomial N3(s) in (8).
ܰଷ ሺ‫ݏ‬ሻ ൌ ‫ ݏ‬ଷ ൅ ͵߱଴ ‫ ݏ‬ଶ ൅ ͵߱଴ଶ ‫ ݏ‬൅ ߱଴ଷ (8)
The equation N(s) = N3(s) gives the formulae for
calculating ω0, Kp, Ti in (9).
‫ ܬ‬൅ ‫ܶܤ‬ௗ
ɘ଴ ൌ
͵‫ܶܬ‬ௗ
ͳ
‫ܭ‬௣ ൌ ሺ͵ߨ‫ܶܬ‬ௗ ߱଴ଶ െ ߨ‫ܤ‬ሻ (9)
͵Ͳ
͵Ͳ‫ܭ‬௣
ܶ௜ ൌ
ߨ‫߱ܬ‬଴ଷ ܶௗ
For J = 0.65×10-3 kgm2, B = 0 Nms/rad, Td = 12 ms, the
following values are obtained: ω0 = 27.78 rad/s,
Kp = 0.0019 kgm2/s, Ti = 0.108 s.
Figure 7. Speed control algorithm structure in Simulink
VI. SPEED CONTROL LOOP IMPLEMENTATION
As mentioned above, the speed controller was
implemented on a PC 2 personal computer (see Fig. 1)
using Simulink Desktop Real-Time, which allows real-time
control algorithms to be implemented.
Simulink Desktop Real-Time includes a library of I/O
blocks that connect Matlab to I/O devices. As I/O devices,
I/O cards inserted into a PC motherboard slot can be used.
The control algorithm is created in Simulink using standard
blocks and its inputs and outputs can be connected to the
I/O card [8]. Figure 8. Structure of the Speed observer block
The motor speed control algorithm consists of the
following parts:
 Actual speed and torque signal processing.
 Calculation of the torque setpoint Mm* using the PI
controller algorithm.
 Adjustment of the torque setpoint signal that is to be
sent to the inverter.
Fig. 7 shows the structure of the speed control algorithm
implemented in Simulink using Simulink Desktop Real- Figure 9. Structure of the Torque observer block
Time. The Speed observer and Torque observer blocks read
the actual values from the analog inputs and convert them
into the corresponding velocity and torque values,
respectively.
The Speed observer block, whose structure is shown in
Fig. 8, reads the voltage magnitude at the analog input AI0
of the MF634 card and converts it to the actual speed value
nact according to (10), where
Figure 10. Structure of the Reference torque output block
‫ܭ‬௡ – experimentally determined constant for A/D and
D/A converter gain correction, ‫ܭ‬௡ = 1.00462,
݊௠௔௫ – maximum motor speed, ݊௠௔௫ ൌ ͳͷͲͲ‘–Ȁ‹, ʹ
‫ܯ‬௔௖௧ ൌ ‫ܯ‬ ൫ܷ ൅ ܷ௢௙௙ െ ͷ൯ (11)
ܷ௡ – voltage at analog input AI0, Ͳܸ ൑ ܷ௡ ൑ ͳͲܸ, ͳͲ ௠௔௫ ெ
ܷ௢௙௙௡ – analog input voltage offset, ܷ௢௙௙௡ ൌ ͲǤͲͳͺܸ. The Reference torque output block (see Fig. 10) converts
the motor torque setpoint into an output voltage that is sent
ʹ݊௠௔௫
݊௔௖௧ ൌ ‫ܭ‬௡ ൫ܷ௡ ൅ ܷ௢௙௙௡ െ ͷ൯ (10) to the inverter via the analog output of the MF634 card.
ͳͲ Fig. 11 shows the structure of the discrete PI speed
The Torque observer block in Fig. 9 reads the voltage controller with anti-windup reset. The controller output is
magnitude at the analog input AI1 of the MF634 card and limited to ±MN. The gain Kb determines the effectivity of
converts it to the actual motor torque value Mact according the oversaturation suppression of the integration
to (11), where ܷெ ‫Ͳۃ א‬ǡͳͲ‫ ۄ‬V is the voltage at the analog component when the output is limited. In case of output
input AI1 that is proportional to the actual motor torque, limitation, the feedback is closed through Kb and for the
Mmax = 3.39 Nm, Uoff = 0.018 V. controller output at the limiter input, expressed in

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 42

 Figure 11. Structure of the discrete PI speed controller (the Speed controller block)

continuous form, (12) holds, where ‫ܯ‬௟௜௠ ‫ א‬ሼെ‫ܯ‬ே ǡ ‫ܯ‬ே ሽ is

the limit of the controller output and e is the control error.
‫ܭ‬௣ ͳ
‫ܭ‬௕ ቀ‫ ݏ‬൅ ܶ௜ ቁ ͳ
(12)
‫ݑ‬ை ሺ‫ݏ‬ሻ ൌ ݁ሺ‫ݏ‬ሻ ൅ ‫ܯ‬௟௜௠
ͳ ͳ
‫ݏ‬൅ͳ ‫ݏ‬൅ͳ
‫ܭ‬௕ ‫ܭ‬௕
To illustrate, Fig. 12 and Fig. 13 show the results of an
experiment with a step change in the speed setpoint from
680 rpm to 1270 rpm at time t = 1 s. In the experiment, the
sampling period of the speed controller was set to Ts = 1
ms. The torque setpoint and actual motor torque waveforms
in Fig. 13 show the effect of the transport delay Td on the
delay of the actual motor torque signal Mm with respect to
the setpoint Mm*.

VII. CONCLUSION Figure 12. Motor speed response to the setpoint step

Working with the inverter in torque control mode and

entering the torque setpoint at the analogue input is the
primary educational use of the workstation. The inverter
must be connected to a computer with Matlab installed and
a multi-function I/O card inserted. A speed controller is
implemented in Matlab using Simulink Desktop Real-
Time. The controller is implemented, and its parameter
values are set using common Simulink blocks and Matlab
commands. The students do not come into contact with the
configuration tools for the PLC and the SINAMICS G120
inverter. They are only concerned with setting up and
verifying the characteristics of the speed control loop of the
servo drive with torque generator.
Another way of using the workstation in education is to
work with the inverter configured in a speed control mode,
where students customize the inveter configuration, while
entering the desired speed from the control panel. The
Figure 13. Actual torque and torque setpoint responses
control panel is used to enter the value of the speed setpoint.
The speed controller parameter values are set using the
Starter program. The speed response can be monitored MF634 multifunction I/O card, but the disadvantage is that
using the Starter program or in Matlab via OPC UA. In this the controller will have a fixed structure and the students
way, students will gain skills in inverter configuration and will not be able to modify it.
learn the principles of speed servo drive parameter tuning.
ACKNOWLEDGMENT
Motor moment of inertia identification is another
possible application of the workstation. The inverter This work has been supported by the projects APVV-21-
operates in the torque control mode. During experiments, 0125 and KEGA 039STU-4/2021.
students specify the magnitude and frequency of the
rectangular waveform of the torque setpoint. In Matlab, REFERENCES
they observe the motor speed response. They calculate the [1] W. Leonhard, “Control of Electric Drives,” Springer Verlag, Berlin,
moment of inertia from the measured motor speed response 1985, ISBN: 3642976468.
waveform. The Matlab - PLC connection is implemented [2] D. Casadei, F. Profumo, G. Serra and A. Tani, "FOC and DTC: two
via OPC UA. viable schemes for induction motors torque control," in IEEE
Transactions on Power Electronics, vol. 17, no. 5, pp. 779-787,
Future plans are to implement a PI speed controller in the Sept. 2002.
PLC (CPU1512C), where students will set the controller [3] SINAMICS G120 low voltage inverters – chassis devices with
parameters and enter the speed setpoint value in Matlab, CU250S-2 Control Units and encoder evaluation. Operating
which will communicate with the PLC via OPC UA. The instructions. 04/2015. Siemens AG, Nürnberg, Germany.
advantage of this approach is that there is no need to use the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 43

 [4] T.A. Hussein, „V/F Control of Three Phase Induction Motor Driven [7] I. Bélai and M. Huba, “Simulation tool for tuning of the speed servo
by VSI Based on SVPWM,“ in 2021 International Conference on drive,” in Proceeding of the 29th International conference 2018
Advance of Sustainable Engineering and its Application Cybernetics and Informatics (C&I). Lazy pod Makytou, Slovakia,
(ICASEA), Wasit, Iraq, pp. 49-53, July 2022. Jan. 2018.
[5] Rotary pulse encoder 1XP8001-1, 1XP8001-2. Mounting [8] “Simulink Desktop Real-Time.”
Instructions. 5/2006. Siemens AG, Erlangen, Germany. https://www.mathworks.com/products/simulink-desktop-real-
[6] K. J. Åström and T. Hägglund, PID Controllers: Theory, Design, time.html.
and Tuning, 2nd ed.. ISA - The Instrumentation, Systems and
Automation Society, 1995.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 44

      
  
  $\^  `$ |~
€   ‚ƒ   †  $‡ ˆ ‰‰‹Œˆ†ƒ † $
’$‰\“”•˜™•$‰ ~›‚
 ‹ œ‹ ~$~‹‚ œ‹ ~$ ‹ ~ˆœ‹ ~

              †       ƒ   ‚  †    
                      ƒ¨‚  †¨‚ ©®ª‹
                        €          
              ƒ‚   ‚  ‚  † †
                        ƒ            ‚ ƒ    ƒ  ƒ
             ¢  †ˆ‹  ’ƒ  ‚    † ¥
                            †ƒ  † ‚
     ƒƒ   ƒ ƒ † ©¯ª$©°ª‹
     " #       $%#            
              ­  ~ˆ             ƒ
  *       *           ††‚‚ ƒ  † ‹’ƒˆ †
                          † ¥‚ † †‚ ‹’ƒ  
          +    ƒ  ƒ    ¨‚     †  ƒ  †¥
                          ±   ƒ  †  ~  $     ƒ
                 /     ¨‹
                      ˆ  † “ †‚ƒ $
1             ƒ‚ †   ˆ† † 
                  ‚   ‹’ƒ $ †  ƒƒƒˆ
             ‚ˆ    ƒ     †  †  ƒ  ƒ
               ©«ª‹
      $  Ÿ‰‚$  ¡‰‚$ €         ‚‚    ~
‹¢ $ƒ‚ †  $ƒˆ‚‹  ‚ $    †          
†   ~  †  ƒ     ±    ‹
€‹ €£’›¤¡‡Œ’€¤£   $  †     †      ƒ  ƒ
ƒ$  ƒƒ   †ˆ  ‚‚   ƒ  † ‚  
€                    ~ ©˜ª‹
   † ¥         ˆ     ‚  †
’ƒ  ƒ    ƒƒˆ
    ƒ  ‹‰‚
†  †  †  ˆ † ‚ †
       ˆ    ƒƒ  ƒˆ  
† ¥† $  ƒ †  †
  ƒ ƒ   ƒ
†    †    ƒ    ‚  ‹  ­     †
‚  †ƒ ˆ   ‹
  ‚‚ ƒ      ~       
¤ ƒ~ˆ ‚     ƒ
  † ‚ ©¬ª‹
ˆ    † ‹’ƒˆ ƒƒ  
  ‚       ² ƒ  ˆ¨‚ ƒ‚  
‚‚ ƒ ‚  ‹€$           †         ‚
            ƒ  †  †  ƒ $  ƒˆ  ƒ †    †      ‹  ’ƒ 
$  ƒƒ           ‹ †ƒ±ˆ  †  $  ƒ
€     ‚ †  ‚  ‚       ¢ $  †          ƒ
ƒ  ¦‚‚ƒ  §‚  ‹’ƒ    ƒ †  ˆ†
†ƒ †ƒ †  ƒ ‚  ‹      †  ‚ ˆ  ƒ  † ¥
¡ˆ       ˆ  ƒ      †     † ƒ   
ƒ  $ƒƒƒ‚  † † † ©³ª‹
ˆ ‚  ‚ ‹ˆ‚ †  
‚       ‚¨   ‚  ƒ     ƒ $ €€‹ ›µ¶­’µ¡·¤›\‰
†  † †  $‹$ †   ˆ  † ƒ  ƒ ƒ‚ 
† † $ƒˆ†   † ‚    $  † ¦ 
    ©•ª$©«ª$©¬ª‹ † ƒ  ‚ ‹
­ ƒ          ‚ †  † €  †ˆ$ƒƒ  †† †ƒ±  ¸·ƒ
†~‹€   “  ƒ ˆ  ƒ  ‚  ‚
  † ¥  †ƒ  ‹’ƒ  †~   † ¹’ƒˆ   † ¦   ˆ
‚ˆ            †    ˆ ‚‚  † ‹’ƒ‚‚  ƒ  †ˆ 
‚‚ƒ ‚  ‹’ƒ    †   ƒ‚  ¦    

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 45

 ‚ ‹’ƒ   ƒ †ƒ  † 
» ¦     ¸     †  
 ¦ †  ˆ††  ˆ † ‚ˆ†$ ‚   † ‹
     ‚ ‹  º $       †
‡€§‡¾  ¸  †‚ 
   †    † ¥  †   †   ‚ ƒ ¼‡€½† ¨‚¼‡¾½
‚  ‹   ‚‚   †    ‹  ’ƒ 
’ƒ           ƒ  †    ˆ‚ˆ  ‚     $
†    ƒ      †     †ˆ   ‹
 ƒ† ‚  ƒ    $
ƒˆ          †
     $  ‚ “   ~ $    ‚    ƒ    †  ‚ƒˆ 
ƒ~ $  †     ˆ   ‹    $ ‚‚  †  ‚ƒ $   ‚    ‚
‚  †  †  ƒ  ƒ   ‚     ~       $   ˆ$  ‚  †
       ƒ      † †      $      $  †  †
†  ¢ ‹­  †   ‚ƒ ‹
‚†    $  ƒ        
­ †›ˆ¼­›½†»›ˆ¼»›½
‚  ƒ† ¢ ˆ©³ª$©”ª‹   ¸     †     ­›  †  »›
‰  †  ‚ƒ‚ †ˆ†    ƒ $ †
 ¦ ƒ ‚  ‹º $ƒ  ¨‚  †  †‹
†† ‚  ƒ ƒˆƒ   ©•™ª‹
€    ¸     ‚  
     ƒ        ‚   ‚    †ˆ †    ƒ           $
 ¦       ‚†   †     ~  ~ $  † $    $  †
    ¢ ©•™ª‹    ‹‰ƒ  ƒƒ
­††  ƒ†    ¥ ‚  †ˆ   †  ˆ    ƒ     $
  ƒ ¦ ‚  $            ¨‚‹
    ˆ     ƒ   ‚
 ‚ƒ  ‹’ƒ†   †ƒ¦ €»‹ ’ºµ ›¤Œµ‰‰¤²Œ›µ­’€£ ­£€­’€¤£‰
     †ˆ  ~ † †  †   ††    †    ƒ  †      †  ƒ
‚ ˆ‹  ’ƒ                    ƒ  ‚ ††      $    
†    ¨        ‚ƒ       ~ †   ƒ  ƒ  † ‚  
† †    ‚$  ƒ$  †  ‚†   †   ‚  †ƒ   ‚
†   ‹      †         ƒ† ± †  † †~ ¸
   ƒ    $ ˆ$†¨
    ‹  ’ƒ   ‚     ƒ   ­‹ †  $
† ‚†ƒ ‚‚ $ ‚  †   ‹ ‚ ‚† $
†‚ ‹ †  ‚    Œ‹  ˆ † $
 ƒ‚†††  ¢ ©••ª‹ ¡‹ ††   ˆ†   $
µ‹     $
€€€‹ €£’µ›­Œ’€»µ­£€­’€¤£µ¶µµ£’‰
²‹  
£ †ˆ $†ˆ‚     †ƒ ‹ †   ‹
 ‚   †ƒƒ†  
 ƒ†‹­  ƒ‚        ƒ    ‚‚   ±     ƒ †
   †  ƒ  ‚          ƒ        $     †   ˆ  
ƒ $ ¢    $†  ƒ   $  ƒ ‹
 ††ˆ ‹€ ƒ†ƒ±    
 ˆ  ‚      ƒ  ƒ    ±  
  $ $ †  † ©•«ª‹  ƒ  ƒ†  $ † 
  ƒ    “†‚ƒ  ˆ  $  ƒƒ      ƒ
 †ˆ$   ¦†  †     ƒ          $  ƒ
†ˆ  ˆ‚ ‹  ­  ƒ$    ƒ  $    
ƒ‚‚ $ƒ ¨‚ ƒ  ƒ
$ƒ‚  ¸
 $ƒ  ¨‚  $ƒ† 

«¡ ¸“ˆ“ $ƒƒ †† †   ƒ    ƒ ‚  ‹
†  ˆ  ˆ  †      ˆ
€     $               
±ˆ  †      ‹  µƒ  
 ˆ      ƒ  ††        † ˆ“
ƒ  ˆ  ƒˆ ƒ‚   $
ƒ† $‚†Œ‹² ƒ ‚‚ $
ƒ    ‹
 ‚    ƒ   

Œ ‚“ †     ¼Œ €½¸   ‚“ ƒ †     ¸
†        ‚   
 ƒ    ‚“ †  
ˆ† ‚ˆ   $
 † ƒ † ‹
  $ƒ  ¨¼$ ‚  $
 ˆ†† ½‚  

¡ «¡ ¸ †ƒ† 
  † $   ‚   ‚ƒ
       ¼ ƒ½$
 ‹   †ˆ†  ƒ ‹
 † ‚ˆ   †‚† 

¬¡  ­ ¸         ƒ“†   † “  † 
 ƒ  ¢ $~  † $†
  ƒƒ††  ‹

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 46

      
       
’ƒ    ‚    † 
 †ƒ  ‹  ƒ
†ˆ    ƒ    $    ƒ †    ~†  



  
 
    †  ƒ †  ¨‚  ƒ     †   ƒ   
‚‚ ‚  ˆ ƒ‚‚  ƒ  †    
ƒƒ  ‹
’ †† ƒ ‚ $†††  †~
   
ˆ  ƒ‚ ‚ ƒ††ˆ†† †
†  ˆ  ƒ  ‚‚ ‚      ƒ   ƒ†
†  ‹
     
ˆ‚  ƒ ‚‚   †  ¦
†¨‚    †Œ$†††      
 ~ ‚‚ $  ƒ ‚ ‹­  †
$    ‚‚  ƒ    †      ¦  ƒ  Œ
  $  $ ¨‚$‚‚ˆº     
¼«™•®½©•¬ª$ƒ † ‚†‚   ¦ƒŒ
    ‚  ‚   ~ $† †
   ƒ  †          †     ƒ        
‚ $          ‚‚  ˆ  €¦ƒ¦  ¼«™«™½  †
»Œ‹¢  ©•®ª$      Ÿ‰‚$  ƒƒ  ¨‚  ƒ        
    ƒ  Œ        †‹  ’ƒ  ††
‚     ˆ       ƒ  ‚       ƒ
 †¿  ƒˆ  †   ƒ      ƒ       ² •‹ ‰ ˆ †  
†$ƒˆ†     ¨‚ †ƒ 
  ƒ  †  ‚    ƒ   †$  ƒ    ††
¨‚ †  ±  ¦  ƒ †          
±  $  ƒƒ   †    ~†    ƒ  ‚    ƒ ­ƒ ‚ $ƒ††  ¦†
‹  ²  ƒ   $  †††       ¨    †  ƒ  ˆ    †††      ƒ   
  $ ˆ  ƒ‚   ˆ‹ ± ‹’ƒ † ƒ †ˆ††$
  †~ ‚ƒ ‚‚  ƒ ‚ƒ $ƒ    ††ƒ  ƒ
 ƒ    †  $    †††    ‚  «¡   ƒ     ¼  ½    ‚ƒˆ
    Ÿ‰‚  ‹ ‚ †‹’ƒ  ƒ ‚ ƒ ƒ†
 ƒ ‚ ‚ƒ ‹
     ² ˆ†  ‚  ƒ ˆ†† 
­  ˆ † $ ƒ $ ‚    ƒ     ƒ‚  $
      ‹€ ƒ  ƒ~ˆ  ˆ     †ˆ  ƒ  ††  ‚ƒ
ƒ  ‹ ƒ   ‚† ‚   ƒ  ƒ‚$ƒ 
† ‚  ƒ$      ƒ ƒƒ  ƒ ² «‹
ƒ ±  ‹€   ƒ ‚ ‚
ƒ  $ ƒ ƒ   
     ¦  ƒ      ˆ$        
 ˆƒ‚$    ƒ~ƒ
 † † ~ ‹    
     
€  ²   •$    ƒ      ˆ †            
        
†† ƒ†  ‹

² «‹ ¡ ‚ˆ   ˆ

  
­     ƒ     ‚    ƒ      
‚  $    ƒ  ‚ƒ           
 ‚ƒ± † ƒ‚‚ †‚ †
ƒƒ‚ † ‹’ƒ  ƒ‚ƒ ƒ
   ˆ  ƒ  †    ‚ƒ     †  ƒ  
  ‚ † ƒ    ƒ ‹‰
    $ † 
      ƒƒ ˆ‹

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 47

»‹ ’µŒº£€Œ­¶­‰ µŒ’‰¤²­£€­’€¤£€ ¶µµ£’­’€¤£ †   µ ‹  ’ƒ    †  ¨  ƒ  $
  $  ‹
’  ƒ  ~$   ˆ 
ƒƒ ‚ ƒ  $ƒ $ƒƒ
Ÿ‰‚   † 
‚     † ƒ $       †  ‚‚     ƒ
ƒƒ        ˆ          ‹  † µ ¿ ¨‚$ƒ † †
² ˆ$ ‚    ††‹¡   ƒ   ƒ‚‚ $
‚ˆ  ˆ$ƒ      
Ÿ‰‚  † ¢ ƒ
  ††¸ ¡‰‚      ¦  †  ¨†  ƒ   
 ‚‚ $  ‚ˆƒ
          $   ƒ‚‚ ƒ
Ÿ‰‚  ƒ   †    ƒ  †         ©•¯ª‹
† ‚    ‚‚   †  «¡       ƒ
 †  ‹  ¤‚      ƒ ƒ“  ‚  £ƒ $      ƒ    ‚ $  
  $‚   ƒˆ †ˆˆ               ‚‹  ¤
  ‚  $  ƒƒ    $ † ‚   ¡‰‚†ƒ   ‚ƒˆ
†    ƒ  ‹ † ‚ˆ  ‚    ƒ  ƒ    ¨$      
’ƒ          †  ˆ       † ‚     ƒ †  ¨   
 ˆ  †    ¨          Ÿ‰¤£‹  ¡ ‚   ˆ        
  †   ~ $       ¨  † Ÿ‰¤£  $‚  ƒ  ‚  †$
† ‚ ††‚ˆƒƒ†‹ ‚ˆ † ¨   ƒƒ  ¨†
ƒ† ‚ˆ    ‹› ˆ$ƒ  
¡‰‚$ ‚  ƒ †  µ $‚    † ˆ††$‚ ‚   
 †  ‚      ˆ  †     ƒ    Ÿ‰‚‹
 $ ‚¦   ‚      ˆ    
† ‚  ‚‚  ‹’ †        
   ˆ ƒ ƒ $ ¢ƒ $ Ÿ‰‚  ƒ†“‚ˆ   ƒ«¡
  $†    $ ¡‰‚¨ƒ †¬¡  $ƒƒ ƒˆ ~
 ˆ¨~  ˆƒ $     ‚ƒ    ~ƒ   ˆ †  ¸
†   ‹ ’ƒ “ƒ
    ‚   † ‚        †ˆ
‰ ~­   ¼ ‰­ ½¸‚ 
  †ƒ  ˆ$ ††ˆ †Ÿ‰‚ˆƒ  ˆ 
~                 †      ƒ  †   ‚¨        
‚‚ † ‚‹ ‚  $†   ‹
­ ‰‚  ¼­†   ­½      ‚ 
­‹¢ ¸     ƒ ƒ  †  ¨  Ÿ‰‚
  †  †  ­† ­¼ ˆ  ˆƒ‚ †  ˆˆ 
~     ­†   ² ƒ½‹  ’ƒ         ˆ       †  ‹€¥  ‚ˆ
      $  †  ‚‚  $   ~ ƒ«¡ ‚ƒ †
†  ‹­ ‰‚ƒ ƒ†     †        ƒ     $  †
 †    † ‚     †  ‚‚           $     
   ‹ ƒ  $‹
¡ ƒƒƒ† ‚ ­† ² ƒ$
» ˆ‹¢ ¸  Ÿ‰‚  ˆ$     ƒ   
   ¢Àˆ†Œ‰‰   ‹€  
ƒ  ‚†     ­†   ­$  ƒ    ˆ
†  ‹ ‹  ·ƒ ­‚‚$  †  ƒ‚‹  €  ƒ  
†  †† ‚† ˆ † ƒ  
 ~$ƒƒ‚  ‚ $ ‚ˆ
¼  Ÿ‰‚ ½ ƒ† ‚ 
      †     † ‹  €  †
 †    ‚‚  $    †††      ƒ
    $          $
‚    ‹€ ‚ ƒŸ‰‚
ƒ† § ƒ    $  ƒ  
ƒ‚ˆ   † ‚$
ƒ‚ ‚ †ƒ† ‹
†††   ¡‰‚

¨Ÿ‰¸Ÿ‰‚ ˆ  
’ƒ        Ÿ‰‚  †  ¡‰‚  
† «¡  $ ‚ˆ† 
† ‚ †     ƒƒ† †
†‚‚  ‹
 ˆ†ˆ  †‚‚  ‹
€ ‚  ˆ† †ƒ ƒ   
 ‹¢ ¸  Ÿ‰‚  ˆ        ƒ
 ~   ƒ  †  ƒ        ƒ    ‚    †  ‚ƒ    ƒ  ‚‚    
‚ ¢  ††‹ ˆ©•°ª‹
 ²   ¨‚$  ƒ          ¡‰‚   ’          $    ƒ 
    ƒ  Ÿ‰‚      ††   ƒ Ÿ‰‚  ƒ  ƒ  ¢Àˆ  ˆ      ‚ˆ
    ¸ ‚     $     ƒ  ƒ  » ˆ‹¢  †

€     ¸  ƒ  ‚ ˆ   ­‹¢   ‹’ƒ   ƒ ƒ 
                ¸
Ÿ‰‚  †           ‚‚ 
ƒ       ¢Àˆ$  ƒƒ      Ÿ‰‚  ˆ$
†ƒ † µ $    ƒ  ­‹¢            

†ˆ   ¸  Ÿ‰‚       †   ˆ     †     ¡¤  ‚   †
†ˆ  ‚ $ƒƒ ‚  $
ƒ  † ‚ˆ†      ‚‚       

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 48


¢Àˆ   ˆ¨ƒ  †    † ’ƒ  ƒ     ƒ¼½
ƒ † $ƒƒ ‚        ¢Àˆƒ  ††† ˆ
‚ $ ƒº’¶‹

­‹¢       ‚  †    ˆ¨    ’ƒ  †        ƒ  ­‹¢  ˆ    ƒ
†   $ ¨   †    ƒ  ‚‚   ¨‚  ƒ

ƒ   ¢Àˆ†­‹¢    ~ ˆ  ‹­ †      
   ƒ  ˆ‚     $    ˆ  ƒ  †   $  ƒ  ˆ      ƒ        «¡
‚    ‚¨  ±     ‹  €  ††       ††
  $  $  ˆ  ˆ  †

¢Àˆƒ ¨  ˆ†  $ ƒ  ‚            ‚    
ƒ  ƒ    ‚ ‚ˆ    †        ‹  €    ¨‚$    †    †  ƒ  ƒ
 †‚‚ ƒ ‹ † ‚ ††    ‚  
‰  ƒ  » ˆ‹¢  ˆ    †ˆ  †  †   «‹
 ~  ƒ  ¢Àˆ  †  †ˆ  ‚  ƒ    ‚`†‡^
 ¢Àˆ$†††  ƒ ‚  ƒ  ƒ&`$~%&$=*  = *+€^
   ‹
   ` ƒ&^ {<^ ++=>
$% &' *+ + <=$=%  >   @'$QXZ$\=^  ‚*}% &' *+ + } }=}==^
\^
º $ƒ   †  ‚‚ ‚
   ƒ  ‚‚  ƒ  » ˆ‹¢  ˆ    ƒ    ‚€XX=^
† ƒ† ‚‚ ƒ‡€ˆ† ƒ       ƒ  =>
† † ‹   >
_ `$%   z${_| {  ' 
_ `$%    z${_| {    @' ƒ  
  '}  „ Z }
²   ƒ   $     ††  ƒ  †ˆ     'QXXX
¢Àˆ¬‹°¼½~    ƒ  ƒ \=^
ƒˆ‹º ƒ†    † \
 †$‹ ‹¸
}~<}=  > ’ƒ  ƒ    ˆ   ‚‚ ƒ  ††      
'$`€XX$   $  ƒ   ˆ †  
\X=^ ‚   ƒ‚ †ƒ ‹’ƒ‚ 
 ††      ‚ $  ˆ  †  ƒ
€‚  †$ ‚    ƒ   ‚¨ˆ    ƒ  ‚   ƒ  ƒ     
 ‚   ƒ †±†¸
‚  ‹  ’ƒ       ˆ  †ƒ†  
 * ‚    Z=> ‚‚ †  † ‚   $
* =  >  ‚    ƒ     †  ƒ  †   ‹
'$`€XX$ Œ  † †   ƒ  †
\X=^    ‹  ’ƒ        ¼² ‹  ¬½
\=^   ˆ      †  $   
†ˆ $ˆ$†‚ˆ   
’ƒ            †††    ˆ ‚ ‹
†ˆƒº’¶¸
_ %  `$<$   `$% &  €$   `$  '
   @€XXZ=^${
’ƒ   †  ‚             ƒ  ­‹¢
ˆ‹’ƒ ‚ †  ƒƒ†ƒ
   †     †  ƒ  ±  †    ƒ  
  $ ˆƒ $† †
‚‚ ‚      ‚‚  ƒ ‹ ’ƒ $ ƒ ² ¬‹ ¡ ‚ˆƒ    ®
  ƒ ‚  ¼  «¡ ‚½
  ‚      ƒ     ƒ“ “ˆ‚¼½$     »€‹ Œ¤£Œ¶‡‰€¤£
‚¨‚    ¸ €  ƒ  ‚‚$    ƒ  ‚ †    ƒ †   
  $% &' *+ + <=$€XX=^ ‚  †ˆˆ  †  †    $  ƒƒ
  ‚     †     ˆ   ƒ   ˆ  †
      ƒ  => ƒˆ‹  ²   ƒ     †$       ‚‚ ‚
  > ‚ƒ  ¨‚    $  ƒ        ƒ
 '  ‚ † ‚ $  ƒ“†‚ƒˆ  †
   @' ƒ   † ƒƒ  ‹’ƒƒ ˆ
  '}  „ Z }   ƒ   †  ‚$  ƒƒ    †ˆ  †    ƒ
 'QXXX †ˆ   ƒ       †  ƒ  ƒ     ƒ
\=^ ‚‚ ‚ ‹€ ƒ †  †ƒƒ 
\           ˆ    ¢      ƒ  ‚ƒ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 49

†    ƒ‚ ‹   ˆ  ‚  †  ¨   †ˆ‹  Œ ‚ ˆ  µ† 
‚    †         † †         ˆ ˆƒ  ˆ‹  ˜¬‹  •™«•¯•‹  ƒ‚ ¸§§† ‹  §•™‹•™•°§¢‹†‚ ˆƒ‹
«™«¬‹•™«•¯•‹
‚‚ ‚ ¨‚   ƒ ¢‹·ƒ ‚ƒ
ƒ  ƒ  ‚‚    ƒ   †    ˆ  ƒ ©¯ª » `‹$È~ ‹«™«¬‹­   ¡   
’ƒ  ˆ  µ† ‹  €   Ÿ     µ 
†ˆ ‚ ‚ ± ƒ‚    † ˆ$  •¬  ¼«½$  ‚‚‹  ¬°  “  °¬$  ƒ‚ ¸§§† ‹  §•™‹¬””•§
   ˆ ‹ ¢‚‹•¬«‹¬¯”˜•
€     ‚ˆ    ~ˆ        ‚ ©°ª  ~$  ‹¿  º  ~ˆ$  ‹¿  ‰ †~$  ‹¿  º¢$  ¶‹  «™«•‹
† †ƒƒ ¨‚ˆ ‰‚  ­ˆ     µ ‚ƒ ‚ƒ  ¡    ‰ 
        † ‹  ’ƒ      ‹  ­‚‚‹  ‰‹  «™«•$  ••$  «®³™‹
ƒ‚ ¸§§† ‹  §•™‹¬¬”™§‚‚••™°«®³™
‚ †       †  ƒ         †
©˜ª É $  Ÿ ˆ  Å  ­†$  µ† ‹  «™«¬‹  €  »† “
 ‚¨   ‚     ‚  $  ƒ   ­††  €      †  ¶  Œ ‚  
 ƒ $†  † ˆ ƒ $ ƒˆ   µ† ‹  €   ¢       
 $  † †  ‚   ‹  ’ƒ  ƒ ƒˆ  Å      ƒ‹  «‹  ƒ‚ ¸§§† ‹  §•™‹¯³³™°§
    $  †      †‚ ¢ ƒ‹«™«¬‹«˜™°‹
† †  ƒ  ‚¨ ¢  ƒˆˆ ©³ª Œ  ­‹$  ²   ­‹  «™«¬‹  ’ƒ  ‚    ƒ  †
    ƒ  †  ‚      †    ‚‚  ‚   †‚ †   † †
ƒ† “  ‹       ‹  ­   µ† $  ƒ‚ ¸§§† ‹  §
•™‹•™³™§™”°¬”«³®‹«™«¬‹««¯˜°˜°
€  $       ƒ  ƒ   ©”ª € ¡$Ÿ  Ɂƒ$’ ² ƒ‹«™«¬‹» ¦ 
¢ ˆ     ‚    ˆ    †   †   †     $‰ 
 ‚ ƒ‚    $ ‰ ź ¤‚$» ³$€ •$«™«¬$•™™°™•$
††~$†  ‹’ƒ   €‰‰£«¯”™“«”••$ƒ‚ ¸§§† ‹  §•™‹•™•°§¢‹ ƒ ‹«™«¬‹•™™°™•‹
      ~    ƒ   ~ †   † ©•™ª » Ç`$  ~  Å  †ƒ~ $  ­~ †‹  «™««‹  ­   
‚  ‚‚ $     †  † ‚   ‰‚à  “  µ ‚  ›        »›‹  €   Ÿ   
‚  †   ‚ƒ         ‚  ƒ €     ’ƒ    ¼Ÿ€½‹  •°‹  «™“¬°‹
ƒ‚ ¸§§† ‹  §•™‹¬””•§¢‹•°•˜‹¬«¯˜•‹
      †    †  † 
  ˆ †‚ ‹ ©••ª Ƀ $¾‹$Ƀ $‹ÅɃ $²‹«™«¬‹‰†“† “
 †ƒ † Ê  †   ‚  †
     ‚ ‹€Ÿµ†’ƒ º ƒµ†«™$°
­Œ\£¤·¶µ¡ µ£’ ¼«™«¬½‹ƒ‚ ¸§§† ‹  §•™‹••³°§ ®•«¬”“™««“™™¬˜®“«
’ƒ   ƒ    ††  ˆ  ƒ    \µ ­ ©•«ª ˆ$  ‰†  Å  Ë $  ‹  «™•°‹  ¡    
™•˜‡Œ“®§«™««¡ ‚ “   ƒ ¹­“ˆ  ‹Œ ‚ ŵ† ‹•™•‹
   ‰­›’à  ƒ       ƒ  † ‚   ƒ‚ ¸§§† ‹  §•™‹•™•°§¢‹ ‚†‹«™•°‹™°‹™™¯‹
 ƒƒ~ †‚   ~ ‹ ©•¬ª º   µ $  ƒ  Å  † †$  Œƒ ‹  «™•®‹   
 ¦   †  ¨‚         Œ  ‚  ‹
 †  ƒ‰¨ƒ­  Œ ‚ µ† 
›µ²µ›µ£Œµ‰ Œ $­Œµ¥•®‹¯•“¯˜‹
©•ª \ă$  ’  Å  £$  ‰  Å  Ħ$  ‰‹  «™•³‹ ©•®ª ›ˆ ~  € ƒ¦$  \¦   ‰~  $  º    · ƒ¦~$
­  † ‚ ¸’ƒ ‚ ‚ † Æ ƒ~²~¦$«™«™‹ »Œ‹¢ ¸ ¦ Œ‚   
      ‹  ¶  †  €  ‹  ¯³‹  «™•“«™”‹       $ºˆ $» °$€ ®$«™«™$™¬³™°$
ƒ‚ ¸§§† ‹  §•™‹•™•°§¢‹ ‹«™•³‹™˜‹™™°‹ €‰‰£«®™¯“³®®™$ƒ‚ ¸§§† ‹  §•™‹•™•°§¢‹ƒˆ ‹«™«™‹™¬³™°‹
©«ª ¶¾‹$’ ·‹$·‹$¶Ƌ$¡ ²‹«™«¬‹’ƒ‚  ©•¯ª º†ˆ$\‹«™•”‹· ‰ƒ†  † ‹€¸ ‰ƒ†
†  † ˆ  †     ˆ      ¨‚   ¡ ‚‹­‚ $~ˆ$Œ­‹ƒ‚ ¸§§† ‹  §•™‹•™™˜§”˜³“
‚   †  ¸  ­    ˆ   ¼«™«¬½ •“®³®«“®®¯˜“”Ì•³
µ†   †  €    ’ƒ   $  «³  ¼³½$  ‚‚‹  •™°•˜  “
©•°ª ¡   ‰‹  «™«™‹  ­  Œ ‚     ­   ’ƒ   ‹
•™°¬˜$ƒ‚ ¸§§† ‹  §•™‹•™™˜§ •™°¬”“™«¬“••°™³“”
­  ¸  ƒ‚ ¸§§ “~ ‹ § ‚ “ “
©¬ª Ÿ `$  Ÿ‹$  Ç$  Ÿ‹  «™««‹  ¡ ‚      † ~ ‚ ƒ   §
‚‚         ‚¨  ƒ       ˆ ‹  €
«™««  «™ƒ  €   Œ     µ   ¶
’ƒ   †­‚‚  ¼€Œµ’­½¼‚‚‹«”®“«””½‹€µµµ‹
©®ª µƒƒ$  ’   Å  ¶†$  ‹  «™«¬‹  Œ ‚“ †
†  ¸µ   ††‚  

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 50

 



 




 

 
Î  




 

 



Ï

 

Ð=;3<7CA
‡<7D3@A7BG ÐÑ(
@/B7A:/D/
':=D/97/

3:3=<=@/03<=D/4;C<70/A9

/:H03B/8C6/A=D/C<70/A9

9/B/@7</9/<=D/C<70/A9

9/B/@7</>7ACB=D/C<70/A9

9/B/@7</@3D/G=D/C<70/A9





2
 
$
!!


!%

7<B3@4/13
E7B6=CB
6/D7<5
9<=E:3253
/0=CB
E6/B
7A
5=7<5

% "!%


!!
$



!!
=<
7<
B63
0/195@=C<2

$!


&

" 
!
!
"!
O
Ð=;>CB3@
CA3@A
037<5
C<A=>67AB71/B32
%3=>:3

#"
!
#"
 !#
!

$
E6=
/@3
5==2
/B
4=@
7<AB/<13
A>@3/2A633BA
/<2
E=@2

$
!

"%
!


2=1C;3<BA
=4B3<
4/:A3:G
03:73D3
B63G
9<=E
3D3@GB67<5

"
# !%


#
$

/0=CB
1=;>CB3@A

!
!

!!

"
"!%

 !!
"!%
!

!"! 

!
O
Ð=;>CB3@A
037<5
1@3/B32
E7B6=CB
/
B6=C56B
B=

 
"
 !
!
!
"
# !%

!

A31C@7BG
Ð@3/B=@A
=4
3/@:G
>3@A=</:
1=;>CB3@A
E3@3

!$
% "!%
$ 
!
 
!
4=1CA32
=<
>3@4=@;/<13
<=B
A31C@7BG
'31C@7BG
7AAC3A

"

!!

"
 !
% "!%
1/;3
:/B3@
/<2
>@=B31B7=<A
/@3
AB7::
A=;36=E
:/197<5

$

!

 
 "
"


!
O
(63
1C@@3<B
B@3<2
037<5
B=
A6/@3
<=B
>@=B31B

"!"

Ñ<B3@<3B
/<2
A=17/:
<3BE=@9A
;/23
7B
D3@G
3/AG
B=
/<2

3<1=C@/532
B63
A6/@7<5
=4
7<4=@;/B7=<
"/<G
>3=>:3
A6/@3

%$ 
% "!%
$
!
 
:/@53
/;=C<BA
=4
7<4=@;/B7=<
E7B6=CB
B67<97<5
/0=CB



A31C@7BG

O
/B/
037<5
/113AA70:3
4@=;
/<GE63@3
Ñ<
B=2/GMA

Ñ ÑÒ'%#('#Ò
E=@:2
7<
=@23@
B=
03
>@=2C1B7D3
CA3@A
E3
@3?C7@3
/<

=>B7=<
B=
/113AA
2/B/
4@=;
D/@7=CA
>:/13A
/<2
D/@7=CA

A
E3
/::
@31=5<7H3
ÐG03@
'31C@7BG
7A
/<
3FB@3;3:G
23D713A
(67A
/:A=
=>3<A
;=@3
=>>=@BC<7B73A
4=@
A31C@7BG

7;>=@B/<B
/<2
A3<A7B7D3
7AAC3
B63A3
2/GA
ÑB
7A
7;>=@B/<B
B=
0@3/163A

7<27D72C/:A
1=@>=@/B7=<A
A;/::
/<2
:/@53
/<2
B=
O
'31C@7BG
<=B
037<5
/0=CB
6/@2E/@3
/<2
A=4BE/@3

5=D3@<;3<BA
:;=AB
3D3@G
2/G
E3
63/@
=4
A=;3
"/<G
7<AB7BCB7=<A
03:73D3
B6/B
/
A31C@3
47@3E/::

KA31C@7BG
0@3/16L
4@=;
=C@
4@73<2A
4/;7:G
/<2
1=::3/5C3A
/CB63<B71/B7=<
/<2
=B63@
A=4BE/@3
/<2
6/@2E/@3
;/93

E7B67<
=C@
B=E<
=@
1=C<B@G
B63;
A31C@3
Ó=E3D3@
7<
;=AB
1/A3A
7B
7A
>3=>:3MA

(67A
6/A
1=;3
/0=CB
2C3
B=
B63
@313<B
/<2
D3@G
4/AB
036/D7=@
B6/B
1/CA3A
A31C@7BG
0@3/163A

7<4:CF
=4
Ñ<4=@;/B7=<
(316<=:=5G
/<2
Ñ<B3@<3B
7<
=C@
O
"/</53;3<B
A33A
A31C@7BG
/A
/
2@/7<
=<
B63

A=173BG
+7B6
B63
/>>3/@/<13
=4
D/@G7<5
B316<=:=5G
0=BB=;
:7<3
$4B3<
B63
7<AB7BCB7=<A
:3/23@A67>
63A7B/B3A
B=

;/:717=CA
>3=>:3
/>>3/@32
B@G7<5
B=
CA3
7B
4=@
B637@
5/7<

7<D3AB
B7;3
/<2
344=@B
7<B=
1G03@
A31C@7BG
J
CAC/::G
C<B7:

K+63<
Ñ
AB/@B32
;G
1/@33@
7<
B63
:/B3
3756B73A
74
B63@3
A=;3
A3@7=CA
A31C@7BG
0@3/163A
=11C@

E/A
/
0/<9
@=003@G
B63
>==:
=4
ACA>31BA
E/A
:7;7B32
B=
ÐC@@3<B
@3A3/@16
4=@
7<AB/<13
- .
AC553ABA
B6/B
=<3
=4

B63
>3=>:3
E6=
E3@3
7<
B63
D717<7BG
/B
B63
B7;3
#=E
E63<
B63
;=AB
7;>=@B/<B
4/1B=@A
7<
1G03@A31C@7BG
7AAC3A
/@3

/
0/<9
7A
@=0032
B63
>==:
=4
ACA>31BA
7A
:7;7B32
B=
B63
3;>:=G33A
J
<=<;/:717=CA
7<A723@A
E7B6
5==2
7<B3<B7=<A

<C;03@
=4
>3=>:3
7<
B63
E=@:2
E7B6
/113AA
B=
/
47D3 E6=
A7;>:G
;/93
/
;7AB/93

6C<2@322=::/@
:/>B=>
/<2
/<
Ñ<B3@<3B
1=<<31B7=<L
J

'6/E<
Ó3<@G
4=@;3@
F31CB7D3
AA7AB/<B
7@31B=@
=4
B63
%C0:7A67<5
/
A31C@7BG
>=:71G
7A
CAC/::G
<=B
3<=C56
J
B67A

ÑMA
ÐG03@
7D7A7=<- .
7A
E63@3
1G03@A31C@7BG
/E/@3<3AA
/<2
B@/7<7<5
>@=5@/;A

1=;3
7<B=
>:/G
'=;3
ABC273A
1:/7;
B6/B
1G03@A31C@7BG

(63
;/7<
=0AB/1:3A
B=
1G03@
A31C@7BG
- .
/@3
/E/@3<3AA
B@/7<7<5
7A
B63
;=AB
7;>=@B/<B
>/@B
=4
933>7<5

O
'31C@7BG
037<5
7<1=<D3<73<B
"/<G
3;>:=G33A
/<
=@5/<7H/B7=<
A31C@3
-.

>3@137D3
A31C@7BG
;3/AC@3A
/A
3FB@/
E=@9
3FB@/
B7;3
B63G

/@3
@3?C7@32
B=
A>3<2
=<
B=>
=4
B63
K@3/:
E=@9L
B63G
/@3
ÑÑ Ð,%&(%',
(%ÒÒ
Ð#(%&&

>/72
B=
>3@4=@;

ÐG03@A31C@7BG
B@/7<7<5
1=C@A3A
/@3
037<5
1@3/B32
/<2

O
Ð=;>CB3@A
037<5
>=E3@4C:
/<2
1=;>:3F
#=B
=443@32
B=
3;>:=G33A
=4
0CA7<3AA3A
C<7D3@A7B73A
/<2
=B63@

:=<5
/5=
1=;>CB3@A
E3@3
B/97<5
C>
E6=:3
@==;A
/<2
=<:G
7<AB7BCB7=<A
/@=C<2
B63
E=@:2
'=;3
/@3
/D/7:/0:3
4=@
4@33

1=;>CB3@
A>317/:7ABA
E3@3
E=@97<5
=<
B63;
+3
B3<2
B=
A=;3
4=@
/
433
11=@27<5
B=
-.
B63
03AB
/@3
B6=A3
=443@32

4=@53B
B=2/G
B6/B
=C@
6=;3
:/>B=>A
/@3
/A
1=;>:3F
/<2
D7/
B63
Ð=C@A3@/
>:/B4=@;
/
ÐG03@A31C@7BG
A>317/:7H/B7=<

1=;>:71/B32
/A
B63
=:2
1=;>CB3@A
E3@3
E3
=<:G
CA3
B63

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 51

1@3/B32
0G
‡<7D3@A7BG
=4
"/@G:/<2
‡23;G
/<2
/BB/19A
B=
A6=E
B63;
6=E
B=
@31=5<7H3
B63
2/<53@
6=E
B=

CBC@3!3/@<
>:/B4=@;A
/D=72
/BB/19A
/<2
6=E
B=
>@=B31B
B63;A3:D3A

(63=@73A
=4
:3/@<7<5
:3/2
B=
B63
1=<1:CA7=<
B6/B

1=<<31B7<5
E7B6
=B63@
>3=>:3
/<2
E7B6
:3/@<7<5
;/B3@7/:A
Ñ* Ð#(%&
Ð%'#Ò

7A
B63
93G
4/1B=@
7<
5/7<7<5
<3E
9<=E:3253
-.
Ð:3/@:G
(63
1=C@A3
E/A
1@3/B32
4@=;
"/G
B=
'3>B3;03@


7<B3@/1B7D7BG
/<2
ABC23<B
3<5/53;3<B
7A
/:A=
7;>=@B/<B
7<
+3
AB/@B32
E7B6
D723=
@31=@27<5A
/<2
327B7<5
CA7<5

ÐG03@A31C@7BG
B@/7<7<5
>@=5@/;A
-.
Ð/;B/A7/
A=4BE/@3
+3
1@3/B32
/
A3@73A
=4

D723=A

(63@3
/@3
/
D/@73BG
=4
;3B6=2A
AC553AB32
/<2
CA32
B=
1=<<31B32
B=
B63
;/7<
1=C@A3
B=>71A
$C@
5=/:
E/A
B=
933>

;/93
1G03@A31C@7BG
/E/@3<3AA
>@=5@/;A
;=@3
7<B3@3AB7<5
B63
B=>71A
7<B3@3AB7<5
/<2
C<23@AB/<2/0:3
4=@
3<2
CA3@A
A=

=@
7<AB/<13
5/;7471/B7=<
-.
>@/1B71/:
;/:E/@3
B@/7<7<5
E3
327B32
3/16
D723=
B=
/
:3<5B6
=4
/>>@=F7;/B3:G


/<2
@3/::743
A13</@7=A
-.
A3:423B3@;7<32
;=B7D/B7=<- .
;7<CB3A

1=<B3FB0/A32
;71@=B@/7<7<5
- .
/<2 =@
>CHH:30/A32
+3
;/23
B63
D723=A
/D/7:/0:3
D7/
B63
"==2:3
AGAB3;

/AA75<;3<BA
- .
E6716
1=<B/7<A
B63
E6=:3
B@/7<7<5
(=
933>
CA3@A
3<5/532

%@=0:3;A
E7B6
ÐG03@A31C@7BG
/E/@3<3AA
>@=5@/;A
- .

/<2
03BB3@
7<4=@;32
E3
CA32
B63
7<B3@/1B7D3
"==2:3

O
;>:=G33A
E7::
433:
0=@32
74
B63
>@=5@/;
=<:G
>:C57<
Ó%
(67A
3</0:3A
CA3@A
B=
B3AB
B637@
C<23@AB/<27<5

1=<B/7<A
D723=
:31BC@3A
/<2
/
;C:B7>:3
16=713
B3AB
/B
B63
=4
D723=
1=<B3<B
AB@/756B
/E/G
E67:3
E/B167<5
B63
D723=

3<2
(63
CA3@
1/<
1:719
=<
/<
7B3;
B6/B
A6=EA
=<
B63
D723=

A1@33<
(63
D723=
E7::
>/CA3
/<2
/
?C3AB7=<
/>>3/@A
(63

O
Ñ4
B63@3
/@3
<=
7<13<B7D3A
/<2
@3E/@2A
3;>:=G33A

CA3@
<332A
B=
/<AE3@
B63
?C3AB7=<
1=@@31B:G
1=<<31B
B63

433:
<=
;=B7D/B7=<
7<
>/@B717>/B7<5
7<
B63
>@=5@/;

1=@@31B
>71BC@3
=@
B3FB
B=
/
B3@;
/<AE3@
/
B@C3 4/:A3

O
;>:=G33A
1=;>:/7<
B6/B
B@/7<7<5
>@=5@/;A
/@3
?C3AB7=<
3B1
4=@
B63
D723=
B=
@3AC;3

=4B3<
<=B
B/7:=@32
B=E/@2A
B637@
7<AB7BCB7=<
/<2
B637@

=@
<=E
Ó%
7<B3@/1B7D3
?C3AB7=<A
/@3
D=:C<B/@G
/<2

>/@B71C:/@
<332A

<=B
/
>/@B
=4
B63
=44717/:
1=C@A3
1=<B3<B
‡A3@A
1/<
E/B16

O
(@/7<7<5
>@=5@/;A
=4B3<
2=<MB
/11=;;=2/B3
D723=A
/<G
B7;3
E7B6
/<
=>B7=<
B=
A3:31B
':=D/9
=@
<5:7A6

27443@3<B
:3/@<7<5
ABG:3A
AC0B7B:3A
(63
1=C@A3
3<2A
E7B6
/
47</:
B3AB

O
ÐG03@A31C@7BG
B@/7<7<5
>@=5@/;A
<332
4@3?C3<B

C>2/B3A
/<2
@3D7A7=<A
E6716
2=3A
<=B
/:E/GA
6/>>3<
* % Ò&
'#
''%'
'
Ð#(%&

&3A3/@16
/<2
3F>3@73<13
/:A=
A6=E
B6/B
1G03@A31C@7BG
Ñ<
B63
47@AB
>6/A3
E3
>:/<
B=
=443@
B63
1=C@A3
B=
/::

B@/7<7<5
7A
<=B
34431B7D3
74
7B
7A
8CAB
/
=<3B7;3
3D3<B
7B
Ð=;3<7CA
‡<7D3@A7BG
3;>:=G33A
;=D7<5
B=
%6
ABC23<BA

<332A
B=
03
1G1:71/:
-.

/<2
/::
ABC23<BA
:/B3@
(63
1=C@A3
E7::
03
=443@32
4=@
3/16

D3<
E7B6
/::
7AAC3A
/<2
16/::3<53A
1G03@A31C@7BG
4/1C:BG
A3>/@/B3:G

B@/7<7<5
A33;A
B=
03
34431B7D3

:/@53
ABC2G
7<
'=CB6
Ñ<
=@23@
B=
03
/0:3
B=
A33
E63B63@
B63
1=C@A3
;/23
/<G

=@3/
1=<47@;32
B6/B
7<AB7BCB7=<A
B6/B
/>>:732
27443@3<13
034=@3
AB/@B7<5
B=
=443@
B63
1=C@A3
E3
E7::
A3<2

1G03@A31C@7BG
B@/7<7<5
23/:B
E7B6
:3AA
A31C@7BG
7<1723<BA
=CB
/
4/13
>67A67<5
3;/7:
B=
Ð=;3<7CA
‡<7D3@A7BG

- .
3;>:=G33A
4B3@
3;>:=G33A
5=
B6@=C56
B63
1=C@A3
E3

E7::
@3>3/B
B63
3F>3@7;3<B
E7B6
/
4/93
>67A67<5
3;/7:
B=

ÑÑÑ Ð,%&(%',
*%Ò&&
Ð#(%&
'
A33
E63B63@
B63
1=C@A3
;/23
/
27443@3<13

Ð#!Ò(&
‡Ò)%&',
4B3@
/::
B6/B
E3
E7::
9<=E

ÐG03@A31C@7BG
7<
':=D/97/
7A
1=D3@32
0G
1B
#=
O
(63
>3@13<B/53
=4
CA3@A
E6=
43::
4=@
B63
7<7B7/:

 
=<
ÐG03@
'31C@7BG
!/19
=4
>@=B31B7=<
/<2
>67A67<5
3;/7:

1==@27</B7=<
1@3/B3A
DC:<3@/07:7BG
Ð=;3<7CA
‡<7D3@A7BG
O
(63
>3@13<B/53
=4
CA3@A
E6=
>/AA32
B6@=C56
B63

7AAC32
/<
7<B3@</:
27@31B7D3
1=<13@<7<5
1G03@
A31C@7BG
/::
1=C@A3
=@
4/7:32
3D3<BC/::G

B63
E/G
0/19
7<

- .
(67A
27@31B7D3
A>3174732

A31C@7BG
=4
E=@9AB/B7=<A
2CB73A
=4
CA3@A
/2;7<7AB@/B=@A
O
(63
>3@13<B/53
=4
CA3@A
E6=
E=C:2
4/::
4=@
B63

3B1

>67A67<5
3;/7:
3D3<
/4B3@
>/AA7<5
B63
1=C@A3

'B7::
E3
4/132
/
A3@7=CA
>67A67<5
/BB/19
B=
3;/7:
*Ñ Ð#(%&
Ð#Ò'Ò'

/22@3AA3A
E6716
;/23
7B
1:3/@
B6/B
/
27@31B7D3
7A
<=B

3<=C56
A
/<
/2232
A31C@7BG
;3/AC@3
4@=;
30@C/@G
(63
1=C@A3
1=<B/7<A

B=>71A A31B7=<A
3/16
4=1CA7<5
=<


/::
Ð=;3<7CA
‡<7D3@A7BG
CA3@A
:=557<5
7<
4@=;
A3>/@/B3
7AAC3A
/<2
/
47</:
B3AB

=CBA723
=4
':=D/97/
ÐH316
&3>C0:71
CAB@7/
/<2
ÓC<5/@G

Ð=C@A3
Ñ<B@=2C1B7=<

6/D3
B=
CA3
"C:B74/1B=@
CB63<B71/B7=<
"

/A71
(3@;7<=:=5G
7<
ÐG03@A31C@7BG
&7A9A

"
@3?C7@3A
A31=<2
:3D3:
=4
CA3@
/CB63<B71/B7=<
(67A

+63@3
Ð/<
+3
<1=C<B3@
/<
BB/19

;3/<A
B6/B
03A723A
B63
>/AAE=@2
B63
CA3@
<332A
B=
D3@74G

B637@
723<B7BG
0G
=<3
E/G
4=@
7<AB/<13
D7/
13::
>6=<3


%67A67<5
7<
;/7:A
/<2
Ó=E
B=
&31=5<7H3
7B

#332
4=@
1G03@
A31C@7BG
/E/@3<3AA
=4
CA3@A
03G=<2
B63

%67A67<5
=<
/
+30A7B3

27@31B7D3
E/A
/:A=
D3@G
1:3/@

+6/BMA
‡>
E7B6
%/AAE=@2A

ÑB
7A
/:A=
1:3/@
B6/B
B63
9<=E:3253
1/<MB
03
/D/7:/0:3

$B63@
$<:7<3
/<2
$44:7<3
(6@3/BA

=<:G
4=@
/
43E
3F>3@BA
7B
<332A
B=
03
E723A>@3/2
/;=<5

&31=;;3<2/B7=<A
=<
Ó=E
B=
%@=B31B
,=C@A3:4

/::
=C@
CA3@A
J
/2;7<7AB@/B=@A
B3/163@A
/A
E3::
/A
ABC23<BA


(3AB

'=
E3
1@3/B32
/
0/A71
1=C@A3
B7B:32
KÐG03@A31C@7BG
‡A3@

E/@3<3AA
/A71A
/B
Ð=;3<7CA
‡<7D3@A7BGL
(63
5=/:
7A
B=

7<B@=2C13
Ð=;3<7CA
‡<7D3@A7BG
CA3@A
B=
@7A9A
=4
1G03@ 


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 52

 " !"!
(67A
>/@B
>@=D723A
7<7B7/:
7<4=@;/B7=<
=<
1G03@A31C@7BG

B6@3/BA
+3
3;>6/A7H3
B6/B
7B
1=<13@<A
/::
=4
CA
+3
1/<MB

@3:G
=<
/<
/<B7D7@CA
>@=5@/;
7<
=C@
1=;>CB3@
=@
>@=B31B7=<

0C7:B
7<B=
B63
Ð=;3<7CA
‡<7D3@A7BG
AGAB3;A
ÑB
7A
C>
B=

3/16
/<2
3D3@G
CA3@
E63B63@
/<
/BB/19
7A
AC113AA4C:
=@
<=B

97DA5

 &1=@<5
6A?=
C85
E945?
*85A5
1>
*5
>3?D>C5A
1>
CC13;

+3
>@3A3<B
B63
4=@;A
1G03@/BB/193@A
CA3
B=
5/7<
=C@

A3<A7B7D3
7<4=@;/B7=<
CA7<5
3;/7:
B3FB
;3AA/53A
>6=<3

1/::A
=@
A=17/:
<3BE=@9A
BB/193@A
CA3
B6@3/BA
=@
=B63@


4=@;A
=4
>@3AAC@3
B=
53B
>3=>:3
B=
/1B
4/AB
/<2
E7B6=CB

B67<97<5
+3
/:A=
>=7<B
=CB
6=E
7;>=@B/<B
7B
7A
B=
933>

97DA5

 &1=@<5
6A?=
C85
E945?
?DAB5
>CA?4D3C9?>
G=C@
AGAB3;A
C>
B=
2/B3
B=
<=B
CA3
AB@/<53
‡'
93GA
/<2

B=
<=B
2=E<:=/2
7::35/:
A=4BE/@3

  $$ "!$   
  #! %!
(67A
A31B7=<
7<B@=2C13A
27443@3<B
BG>3A
=4
1G03@/BB/19A

B63
CA3@A
1/<
4/13
(63
A31B7=<
7A
27D7232
7<B=
BE=
>/@BA
"'
$CB:==9
/B
Ð=;3<7CA
‡<7D3@A7BG
6/A
/
0C7:B7<

AGAB3;
B=
A3:31B
'%"
B=
0:=19
ACA>717=CA
/BB/16;3<BA


/A71
97<2A
=4
;/:E/@3
D7@CA3A
B@=G
6=@A3A
=@
B=
;/@9
3;/7:A
1=;7<5
4@=;
=CBA723
=4
‡<7D3@A7BG

93G:=553@A
3B1
'GAB3;A
Ó=E3D3@
B67A
AGAB3;
7A
<=B
/0:3
B=
1=<A723@


$B63@
97<2A
B6/B
1=C:2
03
4=C<2
7<
3;/7:A
=@
3;/7:
1=<B3<B
A=
7B
7A
C>
B=
B63
CA3@
B=
23B3@;7<3
E63B63@

>6=<3
1/::A
A>/;
A1/;
>67A67<5
D7A67<5
3B1
7B
7A
/
>67A67<5
/BB3;>B
=@
<=B
(67A
A31B7=<
4=1CA3A
=<

+3
/:A=
7<1:C23
6=/F3A
63@3
(63A3
/@3
<=B
27@31B
>67A67<5
3;/7:A

1G03@/BB/19A
0CB
B637@
@7A9A
4=@
1/CA7<5
>@=0:3;A
B=2/G

/@3
5@=E7<5
@/>72:G

97DA5

 &1=@<5
6A?=
C85
E945?
$89B89>7
9>
=19<B



97DA5

 &1=@<5
6A?=
C85
E945?
1B93
'5A=9>?<?7H
9>
H25AB53DA9CH
+3
:7AB
BG>71/:
A75<A
=4
>67A67<5
3;/7:
4=@
7<AB/<13
/

%9B;B
AB@/<53
A3<23@
AB@/<53
5@/;;/B71A
=@
ABG:3
ACA>717=CA

:7<9A
B7;3
C@53<1G
=@
B6@3/BA

(63
D723=
7BA3:4
1=<B/7<A
3F/;>:3A
=4
/BB/19A
0CB
CA3@A
 ! !
1/<
B3AB
B63;A3:D3A
CA7<5
B63
7<B3@/1B7D3
Ó%
7B3;A

(67A
A31B7=<
7A
:793
B63
>@3D7=CA
=<3
6=E3D3@
7B
4=1CA3A

"!!! =<
>67A67<5
=<
E30A7B3A
(63
BG>71/:
A75<A
/@3
4=@

(67A
A31B7=<
A6=EA
27443@3<B
E/GA
7<
E6716
/
7<AB/<13
/
AB@/<53
:7<9
BG>=A?C/BB7<5
/
;7AA7<5
:=19
A75<

1G03@/BB/19
1/<
@3/16
B63
CA3@
/<2
7<
E6716
E/GA
7B
1/<
/<2
13@B7471/B3
/
;7AA7<5
>6GA71/:
/22@3AA
4=@
/
0CA7<3AA

7<4:C3<13
=C@
3;=B7=<A
/<2
2317A7=<
;/97<5
3B1I

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 53



97DA5

 &1=@<5
6A?=
C85
E945?
#C85A
#><9>5
1>4
#66<9>5
'8A51CB

! #! !!" 

(67A
7A
/
A6=@B
AC;;/@G
=4
/::
>@3D7=CA
B=>71A
+3
/:A=

>@=D723
7<4=@;/B7=<
=<
E6=
B=
1=<B/1B
74
B63@3
7A
/

>@=0:3;
/<2
6=E
/<2
E63@3
B=
@3>=@B
A31C@7BG
0@3/163A

 !
(63
47</:
B3AB
1=<B/7<A

?C3AB7=<A
=<
B=>71A
7<
B63

1=C@A3
(63
CA3@
>/AA3A
B63
1=C@A3
74
B63G
/<AE3@

=4

B63
?C3AB7=<A
1=@@31B:G
‡A3@A
1/<
B/93
B63
B3AB
;=@3
B6/<


=<13

97DA5

 &1=@<5
6A?=
C85
E945?
$89B89>7
1C
1
*52B9C5

+3
>@=D723
3F/;>:3A
=4
4/93
E30A7B3A
/<2
1=;>/@3

B63;
B=
@3/:
=<3A

!&  #!  # 
(67A
A31B7=<
3F>:/7<A
27443@3<B
E/GA
/<
/BB/193@
1/<

5/7<
/113AA
B=
>/AAE=@2A
/<2
6=E
3/AG
7B
1/<
03
B=
A7;>:G

5C3AA
7B
+3
3F>:/7<
B63
;=AB
=4B3<
CA32
>/AAE=@2A
/<2

E6/B
;7AB/93A
B=
/D=72
E63<
1@3/B7<5
G=C@
=E<
>/AAE=@2

+3
:7AB
E/GA
=4
>@=B31B7<5
G=C@
>/AAE=@2
/<2
>@=D723
/<

3F/;>:3
=4
6=E
B=
1@3/B3
/<2
@3;3;03@
/
AB@=<5

>/AAE=@2


97DA5

 &1=@<5
6A?=
C85
E945?
'5BC

*ÑÑ Ð Ò&
#%
'
('(%
Ò
Ð#Ò (&#Ò

(63
1=C@A3
7A
8CAB
/0=CB
B=
AB/@B
A=
=4
1=C@A3
/4B3@
B63

47@AB
@C<
E3
E7::
5/B63@
43320/19
/<2
03
/0:3
B=
;/93

<313AA/@G
7;>@=D3;3<BA
034=@3
6/D7<5
;=@3
CA3@A
B/93

B63
1=C@A3

+3
@3/:7H3
6=E
7;>=@B/<B
7B
7A
B=
933>
CA3@A
3<5/532
Ñ<

/
A7BC/B7=<
:793
/B
Ð=;3<7CA
‡<7D3@A7BG
E7B6
B6=CA/<2A

/<2
B6=CA/<2A
=4
CA3@A
<3327<5
B=
B/93
B63
1=C@A3
B67A
7A
/


07B
=4
/
16/::3<53
+3
6/D3
1@3/B32
/<2
7<1:C232
Ó%

7<B3@/1B7D3
/1B7D7B73A
B=
933>
B63
1=C@A3
;=@3
7<B3@3AB7<5

97DA5

 &1=@<5
6A?=
C85
E945?
*81CPB
(@
F9C8
$1BBF?A4B
/<2
3<5/57<5
+3
/@3
:==97<5
4=@E/@2
B=
G=C@
43320/19

/<2
B=
>=B3<B7/:
7;>@=D3;3<BA

!   ! 
+3
/:A=
@3/:7H3
B6/B
1G03@A31C@7BG
/E/@3<3AA
B@/7<7<5

(67A
A31B7=<
4=1CA3A
=<
6=E
B=
A31C@3
G=C@
2=1C;3<BA
A6=C:2
<=B
03
8CAB
/
=<3B7;3
B67<5
Ñ<
=@23@
B=
23D3:=>

7<
0=B6
2757B/:
/<2
>/>3@
4=@;A
/<2
E6/B
;7AB/93A
B=
1G03@A31C@7BG
/E/@3<3AA
1C:BC@3
E63@3
A/43BG
;3/AC@3A

/D=72

E7::
031=;3
/
<=@;
B63
B@/7<7<5
A6=C:2
03
1G1:71
/<2
4=@

(67A
B=>71
A33;A
D3@G
A7;>:3
/<2
A3:43F>:/</B=@G
/<2
A=;3
CA3@A
/:A=
;C16
;=@3
7<
23>B6
23>3<27<5
=<
B637@

;/G03
B6/B
7A
E6G
;/<G
>3=>:3
2=<MB
/>>:G
/<G
A31C@7BG
E=@9
>=A7B7=<A
/<2
/AA75<;3<BA
(67A
7A
1:3/@:G
=<:G
B63

;3/AC@3A
+3
3F>:/7<
6=E
7;>=@B/<B
7B
7A
B=
<=B
:3/D3
=CB
47@AB
AB3>
=<
/
:=<5
@=/2

A3<A7B7D3
2=1C;3<BA
B=
A6@32
2=1C;3<BA
E3
<332
B=
B@/A6

B=
:=5
=CB
=4
AGAB3;A
034=@3
:3/D7<5
G=C@
23A9
B=
CA3
B63
&%Ò&

‡<7D3@A7BG
1:=C2
AB=@/53
7<AB3/2
=4
/
>C0:71
=<3
/<2
B=
. / 9779>B
!

%571>
!
 
-&')&(-
#

>/AAE=@2
>@=B31B
G=C@
>3@A=</:
23D713A

. / )1331

%
 
$"%)(& # #$&"($# )&(-
#$$ 
!?A71>
1D6=1>
$D2<9B85AB
DA<9>7C?>
!
(&

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 54

 . / D>4D
'
 
52AD1AH
 #$+!# # &)# ( -&')&(- (&$) *#' "!+& (&##
?DA>1<

#$+##$#%#"%!$--&')&(-$"%!#
>
?6
9>C5<<53CD1<
31@9C1<
 
 

*&

C8
>C5A>1C9?>1<
?>65A5>35
?>
H25A
*1A61A5
. / 1=


#A=?>4


!5>1A4
$

A?BB<5A
%

 

1>4
&53DA9CH
@@
  
(' #(&'(#  # ,"#($# $ #(&'( ($&- #
. / &C561>9D;
'
 
&## # '%# "%!$- #$&"($# '!6(&"#($# # $&#'($#! -&')&(- (&##

')&(- +&#''
>CA5@A5>5DAB89@
1>4
&DBC19>129<9CH
BBD5B
>6?A=1C9?>
&HBC5=B
?DA>1<
 
 

 
 
. / KEA5BC14

DA>5<<
&

Ò?8<25A7
!
 
'&%&%($#
./ 1A5H

 
'( #!# -&')&(- $)&' $ 
$ $#(,( ' &$&##1 "($ $& -&')&(-
'538A141A3?=
.?><9>5/
1335BB54
&5@C5=25A


1C
(&##
>6?A=1C9?>
&53DA9CH
?DA>1<

<?21<
$5AB@53C9E5

8CC@BFFFC538A141A3?=25BC25BC?><9>53H25AB53DA9CH 

3?DAB5B
. / DAB;J


%DBSJ;
)

#R<5¢R5;
%
 
(&*#
./ A??;B

1>4
A??;B
!
  
>
&51A38
?6
#'(#($(%)..!'-&')&(-(&##
?=@DC5AB


(>45ABC1>49>7
 ' $& $#'(&)(*'( !''&$$"'
A1@893B

  

<5G1>4A91
)
BB?391C9?>
6?A
&D@5AE9B9?>
1>4
DAA93D<D=
. / -81>7
-
5
*
9
*

24?DB
!

 

5E5<?@=5>C

-&')&(- +&#'' (&## %&$&"'
1
3?BCO25>569C

./ &129<<?>
%
&5AA1%D9I


1E1<<5A
)
 
# (* 1>1<HB9B
6A1=5F?A;
>4DBCA91<
!1>175=5>C

1C1
&HBC5=B

-&')&(- (&## "$! ($ ')%%$&( # $&#.($#!  
  

+&#'' %&$&"
'85
H25AB53DA9CH
F1A5>5BB
'%9>9>7
. / F5?>

55

819
&

,??

 
 )(!(- $
!?45<
'%!

1B5
&CD4H
9>
1>141
>
%5B51A38
#$&"($# ')&(- (&## # )($# $# -&')&(-
>C8?<?7H
?>
AC969391<
>C5<<975>35
@@<931C9?>B
9>
&53DA9CH
@@
##('
>
5=@9A931<
5E945>35
>6?A=1C9?>
&HBC5=B
A?>C95AB

  

<?21<

   

./ 41=B
!

!1;A1=1<<1
!
 
-&')&(- ' !!' . / #/($&#4 %&%' 5 
&=5A>931
A5;C?A1
(>9E5AI9CH

(&##
>
1CC13;5A35>CA93
71=96954
1@@A?138
'538>?<?7H
?=5>B;L8?
E
A1C9B<1E5

>>?E1C9?>
!1>175=5>C
%5E95F
 
8CC@BD>921B;69<514=9>AD;<579B<1C9E1 )@0 0 @46

./ 5
*
B8

>F1A
!
9

,D1>
+
+D


'91>
+

 
"%&$*# "%!$-'2 #(!!()! %(- $&

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 55

 Simulating Vehicle-to-Vehicle (V2V)
Communication in Urban Traffic Scenarios
Dávid Bilik∗ , Peter Lehoczký ∗, Ivan Kotuliak ∗
∗ Faculty of Informatics and Information Technologies STU in Bratislava, Slovakia
xbilikd2@stuba.sk, peter.lehoczky@stuba.sk, ivan.kotuliak@stuba.sk

Abstract—Communication among various traffic participants sections II and III, respectively. We describe their features
continues to play an important role, including future intelligent and their availability. Since technology is always evolving,
transportation systems. Sharing important information between not every simulator offers the same new features that imitate
vehicles and other related entities helps enable new services that
can improve road safety. However, any new technologies need to recent technologies like 5G networks or self-driving cars.
be thoroughly tested before deployment. Simulation tools are After checking out VANET simulators, we set up simulation
important instruments that allow researchers to test various scenarios using the Veins simulator in section IV. Finally, we
aspects of new technologies quickly. Such tools are especially present simulation results in section V and conclude the paper
important for vehicular networks because real testing is costly, in section VI.
requiring specialized networking hardware and big test fields.
We focus on simulating vehicle-to-vehicle communication (V2V) II. M OBILITY SIMULATORS
based on vehicular ad hoc networks (VANETs). We study several
existing VANET simulators and their features. We demonstrate A critical aspect in a simulation study of VANETs is the
their functionalities by setting up multiple simulation scenarios need for a mobility model which reflects, as close as possible,
in an urban environment in Bratislava city. The Veins simulator the real behavior of vehicular traffic. When dealing with
and related tools were utilized for the task. We report the effects vehicular mobility modeling, we distinguish between macro-
of repeated re-broadcasts of messages on network performance
mobility and micro-mobility [3].
and maximum message travel distance.
Index Terms—Network simulation, VANET, V2V, Veins, Macro-mobility aspects influence vehicular traffic, which
SUMO, OMNeT++ means rules of the traffic, road topology, car movement
options, overtaking, and intersection rules. Micro-mobility
I. I NTRODUCTION refers to individual driver behavior and interaction with other
In today’s world, equipment and technology in modern drivers or road infrastructure, such as criteria for acceleration,
cars are constantly evolving. Communication between various overtaking, traffic signs, and general driving conditions.
traffic participants already plays an important role in various The advantage of macroscopic models is normally their fast
intelligent systems installed in vehicles. Vehicular ad hoc execution speed. However, the detailed simulation of micro-
networks (VANETs) are one type of network that supports scopic or submicroscopic models is more precise, especially
information sharing between traffic participants or road in- when car emissions or individual routes should be simulated
frastructure. Deploying such networks for testing purposes [4].
is impractical because it involves many vehicles equipped For the VANETs simulation to be similar as close as
with special modules and specific infrastructure installed along possible to the real world, both macro-mobility and micro-
roads. mobility options should be considered and used together when
Simulators allow researchers to test their assumptions with- modeling vehicular movements.
out using real hardware, which is costly in large-scale instal- A. SUMO
lations. They make it possible to discover design flaws before
SUMO (Simulation of Urban Mobility)1 is a free and
investing time and money into a prototype or even a final
open source, highly portable, microscopic traffic simulation
product. They allow to test different configurations multiple
suite that allows the modeling of intermodal traffic systems,
times effectively.
including road vehicles, public transport, and pedestrians with
Multiple network simulators focused on simulating traffic
a large set of tools for scenario creation.
scenarios have been available for many years [1], allowing
SUMO offers a large variety of tools that prepare the traffic
network engineers to test the behavior of their designs. Addi-
simulation. It requires specifically generated representations
tionally, researchers focusing on the detailed working of driver
of road networks and traffic conditions, where both can be
assistance systems inside the car alone have been using other
imported or generated using different sources. Examples of
specialized tools to simulate vehicle movements and other
included tools are supporting tools that can automate core tasks
physical aspects [2].
and ease the creation, execution, and evaluation of the traffic
This paper focuses on vehicular network simulation and
simulations, such as visualization and emission calculation [4].
global vehicular mobility simulation. First, we analyze the
workings of different mobility and network simulators in 1 https://www.eclipse.org/sumo/

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 56

 SUMO is a purely microscopic traffic simulation. Each vehi- multiple paid versions. VANET library is available in standard
cle is given explicitly, defined at least by an identifier (name), and pro versions.2
the departure time, and the vehicle’s route through the road VANET simulation is achieved by interfacing NetSim with
network. Simulation supports different vehicle types, multi- SUMO traffic simulation software. NetSim handles the WAVE
lane streets, lane changing, different rules for intersections, wireless network communication simulation between the ve-
interoperability with other applications at runtime, person- hicles, and SUMO is responsible for modeled road traffic
based inter-modal trips, and much more [5]. conditions.
A SUMO simulation consists of the main simulation file The connection between NetSim and SUMO is made out of
with settings of simulation and additional files with informa- pipes, which are sections of shared memory that are used by
tion about edges or lanes, traffic data of individual vehicles processes for communication. SUMO operates vehicles based
over the whole simulation, and other additional files based on on road conditions, then writes the positional information of
the use of the simulation. These files offer high interoperability vehicles while NetSim reads these data in runtime as input for
thanks to the XML file format. SUMO is highly portable with vehicle mobility.
libraries written in C++ and packages for Windows and main NetSim does the network simulation along with RF prop-
Linux distributions [6]. agation modeling in the physical layer. NetSim simulates
the communication occurring between the vehicles. As the
III. N ETWORK SIMULATORS network is wireless, RF channel loss is probable. This loss
Network simulators offer researchers to study how a net- is configurable together with other channel characteristics to
work behaves under different conditions. They can customize ensure a more realistic simulation.
their needs to make analyses based on specific scenarios. The NetSim also offers a wide range of libraries, including
simulation is an extremely fast and inexpensive solution to applications for 5G NR, various 802.11 wireless protocols,
prototyping and getting to know the behavior of the network. 802.22 Cognitive radio, LTE and LTE advanced, Mobile
This allows testing different settings and making modifications Adhoc Networks, Military Radio, IoT, Routing/Switching,
in controlled and reproducible ways. In this work, we focus Software Defined Networks or Satellite communication.3
on VANET simulators, i.e. complex simulators that combine 2) Veins: Veins (Vehicles in Network Simulation)4 is an
at least mobility and network simulation to provide a tool for open-source vehicular network simulation framework and
studying vehicular networks. There are several different net- ships as a suite of simulation models for vehicular networking.
work simulators, but ns-3 and OMNeT++ are most commonly Veins combines network simulator OMNeT++ and road
utilized in VANET simulators. traffic simulator SUMO into one framework. Veins extends
The network simulator 3 (ns-3) is made after its pre- SUMO to allow communication with OMNeT++ through a
decessor ns-2 with different implementations aiming to be TCP connection. This communication uses a standardized
more extensible and scalable. It is an open discrete-event Traffic Control Interface (TraCI) protocol. Vehicle movement
simulation environment developed in C++ with the ability in the SUMO mobility simulator is seen as the movement of
to use Python scripts. The main feature of this simulator is nodes in OMNeT++ simulation.
object aggregation, Being able to communicate with different Veins is using a manager module that is synchronized with
applications, nodes, and protocol stacks during runtime. both simulators. At regular intervals, the simulation triggers a
OMNeT++ is an open-source simulation environment dis- step in traffic simulation, receives the resulting mobility trace,
tributed under an academic license for non-commercial use. It and triggers updates for all modules [1].
offers functionality such as wireless ad-hoc networks, sensor Veins and OMNeT++ have a few different options for the
networks, and performance modeling. OMNeT++ comprises simulation output. OMNeT++ can output a single value that
its own IDE, simulation kernel, graphical runtime environ- can be used to describe defining characteristics of the simu-
ment, and other tools that integrate several other functions lation. This allows for automation and comparing simulation
[7]. results. Veins can run a test to see if the interactions between
vehicles in simulation lead to the expected results of the model
A. VANET simulators [8].
VANET simulators act as a form of middleware between For making simulation in Veins, both road traffic simulator
mobility simulators and network simulators. They integrate SUMO and network simulator OMNeT++ have to be present.
them to allow communication with each other. Based on All libraries and modules must be compatible and everything
the course of the simulation, they allow users to visualize, have to be compiled and configured to ensure valid simulation.
analyze and change the parameters of the simulation. In this There is a high chance of error while setting up the simulation
subsection, we describe some of the well-known VANET environment. An Instant Veins virtual machine image allows
simulators. using an already prebuilt package by running inside a virtual
1) NetSim: NetSim is a simulation tool to model, simu- environment.
late and analyze information flow. It has the capabilities of 2 https://tetcos.com/vanets.html
an end-to-end, full-stack, packet-level network simulator and 3 https://tetcos.com/help/v13.2/Technology-Libraries/VANETs.html

emulator. NetSim is a commercial product distributed under 4 https://veins.car2x.org/

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 57

 3) Eclipse MOSAIC: Eclipse MOSAIC5 , formerly known while some use ns-3. VANET simulators constantly improve
as VSimRTI (V2X Simulation Runtime Infrastructure), is a with new features, but not every simulator supports the newest
multi-scale and multi-domain co-simulation framework, cou- technologies. Veins and Eclipse MOSAIC are one of the
pling different simulators based on their use and specification. actively maintained open-source simulators.
It uses the concept of ambassadors and federates to connect
different simulators. Different simulators are under a federate IV. S IMULATION SETUP
object assigned to the ambassador in the MOSAIC runtime A. Simulation scenarios
infrastructure (RTI). This infrastructure controls all tasks in the
We implement multiple simulations of a scenario represent-
simulation, the interaction between simulators, the exchange of
ing a traffic situation where some accident happens, and the
data, the synchronization of event processing in all simulators,
involved car must stop relatively quickly. In real life, such
and more.
situations include a car crash or a technical failure preventing
There is already a set of supported simulators coupled
further driving. Accordingly, a certain vehicle in the simulation
with Eclipse MOSAIC. For mobility simulators, there are
is set up to stop at a specific time and to send a specific
VISSIM and SUMO, networks simulators ns-3, OMNeT++,
message informing other traffic participants about the accident.
JiST/SWANS, and different tools for visualization and anal-
The simulation environment is based on multiple locations
ysis. Eclipse MOSAIC specializes in the simulation of V2X
in Bratislava city, i.e. Karlova Ves, Bus station Nivy and
applications. It offers an interface to interact with commu-
Bratislava main railway station. All map data are taken from
nication modules and sensory data. All data from traffic,
the OpenStreetMap project.6 For every selected location, we
communication networks, and other connected simulators is
evaluate and compare the effect of different vehicle counts
available for vehicles [9].
present in the area (ranging from 50 to 300 vehicles).
Eclipse MOSAIC is available as open source with a precon-
Moreover, we implement a simple message dissemination
figured bundle that offers pre-modeled simulators and com-
process. Vehicles in the simulation are instructed to retransmit
mercial MOSAIC Extended with a different set of additional
received messages about an accident. The retransmission is
features that can be customized as needed.
done only in case of the maximum retransmission limit (ap-
4) EstiNet: EstiNet is a commercial network simulation
plicable for the specific received message) is not reached. Such
environment with an add-on for VANET simulation. Its simu-
a re-broadcast is called a hop. For every simulation scenario,
lation engine is based on its own simulation clock that protects
we experiment with a different number of hops.
it from the unpredictability of the operating system clock.
Message about the same accident can be re-broadcasted by
EstiNet’s objective is also an optimal speed and memory
the same vehicle multiple times (the only constraint is that
consumption [10].
the value of the remaining hop count decreased during each
The simulator is equipped with a built-in mobility generator
retransmission). We decided on such an approach allowing
with a microscopic traffic model. It has different traffic flow
duplicated retransmissions to increase message delivery while
modes, support for traffic lights, and speed models. EstiNet
observing impacts on overall network performance.
supports only IEEE 802.11p, IEEE 1609.3, and IEEE 1609.4
In summary, we evaluate simulation scenarios for multiple
protocols for simulation of V2V/V2I networking [11].
geographical locations, vehicle counts, and message hops. In
5) VENTOS: VENTOS is an open-source simulator de-
total, we prepared 27 different simulations. See table I for all
signed for vehicular traffic flow analysis. It integrates the
possible values of simulation parameters.
OMNeT++ network simulator and vehicular traffic simulator
SUMO. VENTOS supports improved models for automated
TABLE I
cruise control and other different algorithms. The TraCI in- S IMULATION SCENARIOS
terface implements a close connection with SUMO. Interac-
tion between infrastructure and vehicles is implemented as a Parameter Values
simulated DSRC wireless connection. It is possible to extend Number of vehicles [50a , 100, 200, 300b ]
the simulation by connecting real hardware and emulating the Geographical locations [Bratislava main railway station,
network [12]. Bus station Nivy, Karlova Ves]
Number of hops [3, 5, 7]
B. Summary a Only for the main railway station location
Countless different VANET simulators have been available. b Only for Bus station Nivy and Karlova Ves locations
They are using different network and vehicular mobility
simulators. SUMO is the de facto standard for simulating
vehicular mobility, as used in multiple VANET simulators
with a connection through the TraCI interface. It also has B. Mobility simulation
extensive documentation. Regarding network simulators, open- For the simulation of vehicle movement, we utilize the
source VANET simulation solutions mainly use OMNeT++, SUMO simulator. It is long-established and widely used in
5 https://www.dcaiti.tu-berlin.de/research/simulation/mosaic/ 6 https://www.openstreetmap.org/

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 58

many experiments providing enough functionality and micro- edge IDs for each vehicle. Vehicle movement created this way
scopic simulation. is completely random. Another method is configuring flows
The simulator requires two main inputs: specific information manually, i.e., defining a specific route for every car. It can be
extracted from maps and the definition of vehicle movements. done by listing the number of vehicles and their distance from
OpenStreetMap (OSM) was used as a source of maps data, edge A to edge B. Different parameters can be used to change
i.e. roads, streets, junctions, traffic rules, buildings (obstacles) each flow. It is also possible to create a group of vehicles
and related data. Using the online tool map editor provided by and configure all cars in the group at once. In our work, we
OpenStreetMap, we cut out areas for every simulation with all used the first approach to generate movements of cars in the
the needed data. Final changes (and small fixes if necessary) simulation randomly by using the netconvert tool.
to the exported map data were done by the JOSM editor (open We get the final data for SUMO simulation by merging the
source editor for OSM data). converted road networks data, polygon data and car movement
Next, the extracted map data needs to be converted in order data.
to be usable by SUMO. Using official tools distributed with the
C. Network simulation
SUMO simulator, we converted the map files to an XML file
format compatible with SUMO. We extracted data describing Following our analysis of VANET simulators, we decided
the roads, circumstances, specifications, rules, and other things to use the Veins simulator. Therefore, the simulation of
using the netconvert7 . We focused on our requirements using network communication between vehicles is implemented in
specific variables, ensuring that the converted file contained OMNeT++ with the INET module. Network simulation needs
only the needed data. Moreover, the buildings on the map to be tightly integrated with a mobility simulation, so that both
were transformed into polygons in order to be used for parts can influence further course of each other. To cooperate
our simulation. We achieved this by using the utility called with our SUMO simulation, we set the network simulation
polyconvert8 to extract the polygon data from the map and configuration file’s primary settings.
provide us with the locations and dimensions of the buildings Since we simulate direct V2V communication, the IEEE
so we can subsequently merge them with the roads data. The 802.11p network standard is used as a base. Simulated com-
network simulator uses the polygon data later when modeling munication uses channel bandwidth of 10 MHz in the 5.9
physical signal propagation. GHz band. See table II for an overview of selected network
Figure 1 shows the SUMO visualisation of all the converted parameter values.
map data. We also utilize a module for obstacles that uses information
from the SUMO configuration file - the PhysicalEnvironment
module. It enables computation of the propagation of signals
across various real world objects in our simulation. This
module records every obstacle, along with its type of material
and its attributes. To use the polygon data we created using
the SUMO conversion tool, it is required to transform it to an
INET-compatible format. Unofficial script available on GitHub
was used for the conversion.10 As an input, the script takes
boundaries of the vehicular network file and the polygons that
are gonna be converted into an INET-style format.
The DielectricObstacleLoss11 model was chosen for our
simulations. This obstacle loss function calculates the power
loss based on precise dielectric and reflection losses along a
straight path while taking into account the shape, location,
direction, and material of any obstructions.
The whole simulation scenario is set up to stop a chosen
Fig. 1. Converted map in SUMO. vehicle at a specific time and to send a specifically formatted
message informing other participants about an accident. After
Finally, we insert vehicles into the simulation. SUMO offers the vehicle stops (managed by the SUMO simulation), new
several techniques for integrating cars into the simulation. One instance of the accident message is created and broadcasted.
approach is to generate cars and their paths randomly. This The broadcasted message also includes a hop count field
can be achieved using the randomTrips.py9 generator. The that is initialized with one of the specific values tested in
generator allows one to specify the total number of cars, their our simulations. When any other vehicle successfully receives
speed, time, and other details in the arguments. The basic and decodes the message (depending on the simulated signal
output is a file including timings, IDs, and start and finish propagation model), our custom function is called to process
7 https://sumo.dlr.de/docs/netconvert.html 10 https://github.com/panagis/ConvertOsmToInetObstacles
8 https://sumo.dlr.de/docs/polyconvert.html 11 https://doc.omnetpp.org/inet/api-current/neddoc/inet.physicallayer.
9 https://sumo.dlr.de/docs/Tools/Trip.html wireless.common.obstacleloss.DielectricObstacleLoss.html

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 59

the message. Its main responsibility is to decrease the hop
count value recorded in the message and re-broadcast a copy
of the message in case the hop count is still a positive number.
The function also records and saves statistical information for
later processing, e.g. delay of the received message.

TABLE II
N ETWORK SIMULATION PARAMETERS

Parameter Value
Network standard IEEE 802.11p
Frequency 5.9 GHz
Radio bandwidth 10 MHz
Transmitter power 20 mW
Antenna type IsotropicAntenna
Obstacle loss model DielectricObstacleLoss

Fig. 2. Visualisation of message travel distance. Scenario with 3 hops.

V. E VALUATION AND RESULTS

The amount of data successfully transmitted within a spe-
cific time frame of simulation is measured as throughput.
Our throughput computation uses the throughput data for each
node (based on received packets) during each simulated time
interval, which is reported directly by Veins. We calculate an
average throughput for every 0.1 second.
One-hop delay is the time needed to transmit a message
from sender to receiver successfully. The whole time can be
divided into multiple smaller parts that contribute to the total
delay, i.e., processing, queue, transmission, and propagation
delays. We calculate the delay by subtracting the timestamp
when the packet was created (in simulation time) from the
current simulation time.
We compare the maximum distance traveled by our accident
message for various message hop counts. In general, increased
hop count results in greater delivery distance. Figures 2 and 3
Fig. 3. Visualisation of message travel distance. Scenario with 7 hops.
show a visual representation of such a scenario with different
hop counts. There are 200 vehicles in the scenarios. Green
cars represent vehicles that successfully received the accident
message transmitted by the vehicle in the turquoise circle.
However, duplicated retransmissions combined with higher
hop counts lead to increased total network traffic, delays, and
packet collisions. In the case with 7 hops, the latest message
(after all retransmissions) was delivered more than 2 seconds
after the initial accident message was broadcasted, meaning
that the whole network was handling only retransmissions of
the information about a single accident during the whole time.
See figure 5, which depicts an average one-hop delay as it
increases when messages are repeatedly re-broadcasted. It is
also for the same scenario with 200 vehicles. Values for three
hops are not reported because they are negligible compared to
the 5-hop and 7-hop scenarios.
VI. C ONCLUSION
In this paper, we described current well-known VANET
simulators, their features, and principles. We chose to utilize Fig. 4. Average throughput based on received packets by every vehicle

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 60

 R EFERENCES
[1] Spaho, E., Barolli, L., Mino, G., Xhafa, F. & Kolici, V. VANET Simu-
lators: A Survey on Mobility and Routing Protocols. 2011 International
Conference On Broadband And Wireless Computing, Communication
And Applications. (2011,10), https://doi.org/10.1109/bwcca.2011.11
[2] Ľuboš, M., Ján, D., Tomáš, M., Igor, K., Marek, G., Jozef, B. & Robert,
S.(3921). Virtual Simulation of Overtaking Maneuver of Autonomous
Vehicle. Strojnı́cky časopis - Journal of Mechanical Engineering, 71(2)
179-188. https://doi.org/10.2478/scjme-2021-0028
[3] Fiore, M., Harri, J., Filali, F. & Bonnet, C. Vehicular Mobility Simulation
for VANETs. 40th Annual Simulation Symposium (ANSS’07). (2007),
https://doi.org/10.1109/anss.2007.44
[4] Lopez, P., Behrisch, M., Bieker-Walz, L., Erdmann, J., Flötteröd, Y.,
Hilbrich, R., Lücken, L., Rummel, J., Wagner, P. & Wießner, E.
Microscopic Traffic Simulation using SUMO. The 21st IEEE Inter-
national Conference On Intelligent Transportation Systems. (2018),
https://elib.dlr.de/124092/
[5] Behrisch, M., Bieker, L., Erdmann, J. & Krajzewicz, D. SUMO –
Simulation of Urban MObility: An Overview. SIMUL 2011. (2011,10),
https://elib.dlr.de/71460/
[6] Lopez, P., Wiessner, E., Behrisch, M., Bieker-Walz, L., Erdmann, J.,
Flotterod, Y., Hilbrich, R., Lucken, L., Rummel, J. & Wagner, P.
Fig. 5. Average one-hop delay Microscopic Traffic Simulation using SUMO. 2018 21st International
Conference On Intelligent Transportation Systems (ITSC). (2018,11),
https://doi.org/10.1109/itsc.2018.8569938
[7] Weingartner, E., Lehn, H. & Wehrle, K. A Performance Comparison of
the Veins simulator for implementing our own simulation sce- Recent Network Simulators. 2009 IEEE International Conference On
narios. We have successfully created simple V2V simulation Communications. (2009,6), https://doi.org/10.1109/icc.2009.5198657
[8] Sommer, C., Eckhoff, D., Brummer, A., Buse, D., Hagenauer, F.,
scenarios based on different locations in Bratislava city. Joerer, S. & Segata, M. Veins: The Open Source Vehicular Network
We demonstrated that map data with a level of detail Simulation Framework. Recent Advances In Network Simulation. pp.
up to the positions of buildings are available under open 215-252 (2019), https://doi.org/10.1007/978-3-030-12842-5 6
[9] Sommer, C., Härri, J., Hrizi, F., Schünemann, B. & Dressler, F.
source license. Converting available map data from Open- Simulation Tools and Techniques for Vehicular Communications
StreetMap to formats acceptable by SUMO and OMNeT++ and Applications. Vehicular Ad Hoc Networks. pp. 365-392 (2015),
simulators is straightforward and official tools are relatively https://doi.org/10.1007/978-3-319-15497-8 13
[10] Balamurugan, A., Priya, M., Malar, A. & Janakiraman, S. Raccoon
well-documented. Data about buildings can be easily used optimization algorithm-based accurate positioning scheme for reliable
in network simulation, where they represent obstacles during emergency data dissemination under NLOS situations in VANETs.
the computation of signal propagation. Although it is still Journal Of Ambient Intelligence And Humanized Computing. 12, 10405-
10424 (2021,1), https://doi.org/10.1007/s12652-020-02839-6
only an abstraction of real signal propagation, the obstacle [11] Mussa, S., Manaf, M., Ghafoor, K. & Doukha, Z. Simula-
model provided by the OMNeT++ module considers the shape, tion tools for vehicular ad hoc networks: A comparison study
position, orientation, and material of every defined obstacle. and future perspectives. 2015 International Conference On Wire-
less Networks And Mobile Communications (WINCOM). (2015,10),
However, the material specification of every building might not https://doi.org/10.1109/wincom.2015.7381319
be accurate in map data sources, and various parts of buildings [12] Amoozadeh, M., Ching, B., Chuah, C., Ghosal, D. & Zhang, H.
might have different densities according to their architecture. VENTOS: Vehicular Network Open Simulator with Hardware-in-the-
Loop Support. Procedia Computer Science. 151 pp. 61-68 (2019),
According to our simulation scenarios, we compared the https://doi.org/10.1016/j.procs.2019.04.012
effects of different hop counts on maximum message travel
distance, network throughput, and delay. In our simulated envi-
ronment, where duplicated re-broadcasts of the same message
by the same vehicle were allowed, increased hop count led
to a situation known as a broadcast storm. While maximum
message distance was increased, the negative effects of the
broadcast storm on the whole network were not negligible,
and the worst one-hop delay was more than 500 ms.

ACKNOWLEDGMENT

This article was written thanks to the generous support

under the Operational Program Integrated Infrastructure for
the project: ”Support of research activities of Excellence
laboratories STU in Bratislava”, Project no. 313021BXZ1, co-
financed by the European Regional Development Fund.” The
research was also supported by the APVV-19-0401 and the
KEGA 025STU-4/2022 projects.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 61

 'HYHORSPHQWRID0RGXOHIRU&ROOHFWLQJ6WXGHQWV¶
)HHGEDFNLQDQ,QWHUDFWLYH0DQQHU
/LQGD%ODKRYD -R]HI.RVWROQ\
)DFXOW\RI0DQDJHPHQW6FLHQFHDQG,QIRUPDWLFV )DFXOW\RI0DQDJHPHQW6FLHQFHDQG,QIRUPDWLFV
8QLYHUVLW\RI=LOLQD 8QLYHUVLW\RI=LOLQD
=LOLQD6ORYDNLD =LOLQD6ORYDNLD
OLQGDEODKRYD#IULXQL]DVN MR]HINRVWROQ\#IULXQL]DVN

$EVWUDFW²7KH SDSHU LQWURGXFHV DQ HQJDJLQJ ZD\ RI FROOHFWLRQDQGDOORZVWKHVWXGHQWVWRSURYLGHWKHLUIHHGEDFN

FROOHFWLQJIHHGEDFNDQGWKHDWWHQGDQFHRIVWXGHQWVLQOHFWXUHVRU RQDUHJXODUEDVLV
VHPLQDUV7KHIHHGEDFNLVRIWHQFROOHFWHGWRZDUGVWKHHQGRIWKH
VHPHVWHUZKLFKLVUHVWULFWLYHLQHIILFLHQWDQGLQVXIILFLHQW2XU ,, 67$7(2)7+($57
VWXG\IRFXVHVRQHVWDEOLVKLQJDQLQIRUPDWLRQV\VWHPPRGXOHWKDW
JLYHV VWXGHQWV IUHTXHQW LQTXLULHV RQ DQ\WKLQJ RWKHU WKDQ $ &ROOHFWLQJIHHGEDFN
WHDFKLQJ (QWHUWDLQLQJ FXUUHQWWRSLF TXHVWLRQV PLJKW DOVR &XUUHQWO\WKHIHHGEDFNIURPVWXGHQWVLVFROOHFWHGRQO\DW
LQVSLUH VWXGHQWV WR SDUWLFLSDWH LQ WKH VXUYH\V 6WXGHQWV PD\ WKHHQGRIWKHVHPHVWHU$IWHUFRPSOHWLQJWKHVHPHVWHUDQGWKH
YROXQWDULO\DQVZHUDQRIIWRSLFTXHVWLRQLH 'RHVDSLQHDSSOH ILQDOH[DPRIWKHVXEMHFWWKHVWXGHQWVKDYHWKHRSSRUWXQLW\WR
ILW RQ D SL]]D"  )XUWKHU RQ WKH\ KDYH WKH RSSRUWXQLW\ WR JLYH ILOO RXW DQ HYDOXDWLRQ IRUP LQ ZKLFK WKH\ FDQ H[SUHVV WKHLU
OHFWXUH IHHGEDFN DQG E\ ILOOLQJ RXW WKH IRUP WKH\ SURYLGH D SRLQW RI YLHZ RQ KRZ WKH OHFWXUHV DQG VHPLQDUV ZHUH KHOG
UHFRUG RI WKHLU DWWHQGDQFH DW WKH OHFWXUH RU VHPLQDU ZKDW LPSURYHPHQWV ZRXOG WKH\ VXJJHVW DQG ZKDW WKH\
$QVZHULQJ WKH RIIWRSLF TXHVWLRQ DQG SUHVHQWLQJ DQRQ\PRXV
DSSUHFLDWHG DQG JDLQHG IURP DEVROYLQJ WKH VXEMHFW 7KLV
UHVXOWV RI WKH VXUYH\ DW WKH EHJLQQLQJ RI WKH FRXUVH PLJKW
DSSURDFK KDV VHYHUDO GLVDGYDQWDJHV 7KH IHHGEDFN LV RQO\
PRWLYDWH VWXGHQWV WR UHVSRQG DQG GHYHORS JURXS XQLW\ ZKLOH
DOORZLQJWKHOHFWXUHUWRFROOHFWWKHDWWHQGDQFHFRPSOHWLRQ
FROOHFWLQJDIWHUWKHFRXUVHLVILQLVKHGWKHUHIRUHZKLOHLWFDQ
KHOSRWKHUVWXGHQWVLQWKHIXWXUHLWGRHVQRWFKDQJHDQ\WKLQJ
.H\ZRUGV²IHHGEDFN LPSURYHPHQW HGXFDWLRQ SURFHVV IRUWKHVWXGHQWWKDWLVFXUUHQWO\ILOOLQJRXWWKHIRUP7KLVLVWKH
PRGXOHPLFURVHUYLFHVTXL] PDLQUHDVRQZK\VWXGHQWVGRQRWZDQWWRSDUWLFLSDWHLQVXFK
IHHGEDFN 7KH OHFWXUHU PLJKW DOVR FRQVLGHU WKLV IRUP RI
, ,1752'8&7,21 IHHGEDFNDVLQVXIILFLHQWLQRUGHUWRLPSURYHWKHLUSUHVHQWLQJ
)HHGEDFN LV HVVHQWLDO LQ PRGHUQ OLIH 6HUYLFH SURYLGHUV VNLOOVRUDGMXVWWKHIRUPLQZKLFKWKHVHPLQDUVDUHKHOG
PDUNHWHUVDQGVRIWZDUHGHYHORSHUVZDQWIHHGEDFN)HHGEDFN % 0DQDJLQJWKHVWXGHQWDWWHQGDQFH
LVPRVWO\XWLOLVHGWRHQKDQFHDVHUYLFHRUSURGXFWDQGDGDSWWR )LOOLQJ RXW WKH DWWHQGDQFH VKHHW LV XVXDOO\ GRQH DW WKH
WKH FXVWRPHU 7KH WHDFKLQJ SURFHVV LV LQFOXGHG LQ WKLV EHJLQQLQJRIHYHU\VHPLQDU7KHOHFWXUHUJRHVWKURXJKWKHOLVW
PHWKRG0RGHUQVFKRROLQJUHTXLUHVIHHGEDFN,WPD\EHXVHG RIWKHVWXGHQWVWKDWVKRXOGEHDWWHQGLQJWKHVHPLQDUDQGPDUNV
WR DVVHVV VWXGHQW FRPSUHKHQVLRQ RU LQVWUXFWRU SHUIRUPDQFH GRZQ WKH SUHVHQFH RU DEVHQFH RI HDFK OLVWHG VWXGHQW 7KH
:KHQWHDFKLQJLQWKHIRUPRIOHFWXUHVDQGSUHVHQWDWLRQVZH DWWHQGDQFH VKHHW LV RIWHQ LQ D SULQWHG SDSHU IRUP RU D
RIWHQHQFRXQWHUDQLPDJLQDU\EDUULHUEHWZHHQWKHOHFWXUHUDQG VSUHDGVKHHW7KHPDLQLVVXHRIWKLVDSSURDFKLVWKHWLPHWKDWLW
WKH DXGLHQFH 2QH RI WKH PDLQ IDFWRUV FRQWULEXWLQJ WR WKLV FRQVXPHV,Q HYHU\ VHPLQDUFROOHFWLQJ WKHDWWHQGDQFH WDNHV
LPDJLQDU\ EDUULHU LV WKH IUHTXHQW VK\QHVV RI WKH OLVWHQHUV DURXQGILYHWRWHQPLQXWHV1RWRQO\LWLVUHGXFLQJWKHDYDLODEOH
ZKLFK OHDGV WR WKH IDFW WKDW WKH OLVWHQHU GRHV QRW DVN WKH WLPH IRU WKH VHPLQDU EXW LW DOVR DOORZV WKH VWXGHQWV WR GR
OHFWXUHUWRUHSHDWRUH[SODLQWKHOHFWXUHGWRSLFLQPRUHGHWDLO GLIIHUHQW WKLQJV OLNH WDONLQJ ZLWK RWKHU VWXGHQWV RU VFUROOLQJ
,WFDQDOVRKDSSHQWKDWWKHVSDFHIRUTXHVWLRQVDQGGLVFXVVLRQ WKURXJKVRFLDOPHGLD:KHQWKHDWWHQGDQFHVKHHWLVGRQHWKH
LVUHVHUYHGRQO\DWWKHHQGRIWKHOHFWXUHDQGWKHOLVWHQHUZLWK OHFWXUHU VWDUWV WKH VHPLQDU $W WKDW SRLQW VWXGHQWV DUH RIWHQ
WKHDPRXQWRILQIRUPDWLRQUHFHLYHGIRUJHWVWKHTXHVWLRQKH GLVWUDFWHGDQGLWFDQWDNHHYHQVHYHUDOPRUHPLQXWHVWRJHW
ZDVJRLQJWRDVNWKHOHFWXUHU$QRWKHUIDFWRUREVHUYHGRQWKH WKHLU IXOO DWWHQWLRQ DQG FRQFHQWUDWLRQ WR WKH VXEMHFW RI WKH
SDUWRIWKHOHFWXUHULVWKHODFNRIIHHGEDFNIURPWKHDXGLHQFH OHFWXUH
$ IUHTXHQW FRQVHTXHQFH RI WKLV EDUULHU LV D ODFN RI & &RPPRQDYDLODEOHVROXWLRQV
XQGHUVWDQGLQJRIWKHWRSLFEHLQJOHFWXUHGZKLFKFDQOHDGWR 7KHUH DUH VHYHUDO RSHQVRXUFH RU OLFHQVHG VRIWZDUH
D ODFN RI NQRZOHGJH DQG FRQVHTXHQWO\ DIIHFW WKH OHDUQLQJ RSWLRQVWKDWFRXOGEHJHQHUDOO\XVHGIRUWKHVHSXUSRVHV7KH
RXWFRPHV7KDW¶VZK\LWLVEHQHILFLDOLIQRWFUXFLDOWRFROOHFW PRVW FRPPRQ VROXWLRQ LV 6OLGR >@ 6OLGR RIIHUV PDQ\
WKHIHHGEDFNIURPWKHSDUWLFLSDQWV IHDWXUHV LQFOXGLQJ DQRQ\PRXV TXHVWLRQV IRU WKH OHFWXUHU
:LWKLQ WKH OHFWXULQJ SURFHVV WKHUH LV DQRWKHU HVVHQWLDO TXL]]HVRUDIWHUHYHQWIHHGEDFNVXUYH\DPRQJRWKHUV2WKHU
DVSHFWZKLFKLVFROOHFWLQJWKHDWWHQGDQFHDWWKHOHVVRQ7KH VLPLODU VROXWLRQV LQFOXGH .DKRRW >@ 3LJHRQ+ROH /LYH >@
OHFWXUHUPXVWPDQXDOO\IROORZDOLVWRIVWXGHQWVµQDPHVDQG DQG4XL]L]]>@7KHUHDUHOHFWXUHUVWKDWXVH6OLGRIRUWKHVH
PDNH VXUH ZKHWKHU WKH\ DUH DWWHQGLQJ WKH VHPLQDU RU QRW SXUSRVHVDWRXUXQLYHUVLW\EXWLQRXURSLQLRQLWKDVVHYHUDO
:KLOHLWLVWLPHFRQVXPLQJVWXGHQWVRIWHQJHWGLVWUDFWHGDQG GLVDGYDQWDJHVRUUHVWULFWLRQV)LUVWRIDOOWKHIXQFWLRQDOLW\RI
LW FDQ EH FKDOOHQJLQJ WR JHW WKHLU DWWHQWLRQ EDFN DW WKH WKH IUHH YHUVLRQ LV OLPLWHG ZKHWKHU LW LV OLPLWDWLRQ RI
EHJLQQLQJRIWKHOHVVRQ SDUWLFLSDQWVRUOLPLWHGQXPEHURITXHVWLRQVWKDWFDQEHILOOHG
7KHPHQWLRQHGDVSHFWVOHGXVWRWKLQNDERXWKRZZHFRXOG RXWE\WKHPWKHUHIRUHLQRUGHUWRXVHWKHSURSHUIXQFWLRQDOLW\
LPSURYH WKH RYHUDOO SURFHVV DQG ZH GHVLJQHG WKH VROXWLRQ D VXEVFULSWLRQ PXVW EH SDLG 2WKHU GLVDGYDQWDJH LV WKH
WKDW LV HQJDJLQJ IRU VWXGHQWV LQYROYHV WKH DWWHQGDQFH FRPSOH[LW\ RI WKHVH V\VWHPV DV LW LQFOXGHV PDQ\ IHDWXUHV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 62

WKDWFDQVHHPWRRFRPSOH[IRUVRPHOHFWXUHUVRUSDUWLFLSDQWV  7KHDWWHQGHHVPXVWORJLQLQWRWKHV\VWHPDWILUVWDQG
WRXVH7KHWKLUGPDLQGLVDGYDQWDJHLVWKDWDOOWKRVHV\VWHPV WKHQ ZULWH WKH DFFHVV FRGH WR DFFHVV WKH OHFWXUH¶V
DUH IRFXVHG PDLQO\ RQ WKH IHHGEDFN WKHUHIRUH LW LV PRUH URRP
GLIILFXOWWRLQFOXGHWKHDWWHQGDQFHVKHHWWKHUH7KHDWWHQGDQFH  7KH\FDQZULWHWKHLUTXHVWLRQVRUUHPDUNVIUHHO\LQWR
PXVWEHUHFRUGHGDVDQRSHQTXHVWLRQWKDWWKHVWXGHQWVZLOO WKHµPHVVDJHV¶TXHVWLRQDWDQ\WLPH
ILOO RXW ZLWK WKHLU QDPH ,Q RXU RSLQLRQ ZH VHH D URRP IRU
LPSURYHPHQW RI WKH XVHU H[SHULHQFH IRU WKH PHQWLRQHG  7KH\ KDYH WKHRSSRUWXQLW\ WRDQVZHU WKHTXHVWLRQV
SXUSRVHV DQG WKDW LV ZK\ ZH GHFLGHG WR GHVLJQ DQG JLYHQE\WKHOHFWXUHULQWKHµTXHVWLRQV¶VHFWLRQ6LQFH
LPSOHPHQWDQHZV\VWHPGHGLFDWHGIRUWHDFKLQJSXUSRVHVWR WKH\DUHORJJHGLQZKHQWKH\DQVZHUWKHTXHVWLRQ
REWDLQIHHGEDFNDQGDWWHQGDQFHDWWKHVDPHWLPHLQDVLPSOH QRW RQO\ WKHLU DQVZHU LV UHFRUGHG EXW DOVR DQRWKHU
UHFRUGLVPDGHWKDWWKHDQVZHUZDVVXEPLWWHGE\WKDW
DQGHQJDJLQJPDQQHU
DFFRXQW+RZHYHUWKHDQVZHUVDUHQRWOLQNHGWRWKH
,,, 352326('62/87,21'(6,*1 DFFRXQWVWKHUHIRUHWKHDQVZHUVUHPDLQDQRQ\PRXV
EXW WKHUH LV D UHFRUG WKDW WKH SHUVRQ DQVZHUHG WKH
 2YHUDOOWKHSURSRVHGVROXWLRQKDVVHYHUDOIXQFWLRQDODQG TXHVWLRQZKLFKFDQVHUYHIRUWKHDWWHQGDQFHVKHHW
QRQIXQFWLRQDOUHTXLUHPHQWV)LUVWRIDOOLWVKRXOGEHVLPSOH ZLWKRXW DFWXDOO\ GRLQJ WKH DWWHQGDQFH VKHHW
WR XVH IRU QRZ LW VKRXOG LQFOXGH RQO\ WKH GHVLUHG PDQXDOO\
IXQFWLRQDOLW\ )RU HYHU\ VHPLQDU OHFWXUH RU VXEMHFW WKH
OHFWXUHUFDQFUHDWHDVHSDUDWHVSDFHFDOOHGµURRP¶,QWKHURRP  7KHTXHVWLRQVDQGDQVZHUVFDQEHIXUWKHUDQDO\]HG
WKHUHZLOOEHWZRVHFWLRQV)LUVWVHFWLRQLVFDOOHGµPHVVDJHV¶ E\WKHOHFWXUHUDIWHUZDUGVGLUHFWO\LQWKHV\VWHPRU
,QWKLVVHFWLRQVWXGHQWVFDQIUHHO\DVNTXHVWLRQVDQRQ\PRXVO\ GRZQORDGHGLQWKHIRUPRIDVSUHDGVKHHW
RUVXSSRUWRWKHUTXHVWLRQVZLWKµOLNHV¶LIWKH\DJUHHRUUHODWH
7KHSURSRVHG VROXWLRQ VKRXOG KDYH WKH IRUP RI WKH ZHE
WRWKHTXHVWLRQ7KHVHµPHVVDJHV¶FDQEHXVHGPHDQZKLOHWKH
DSSOLFDWLRQEHFDXVHRILWVDFFHVVLELOLW\RQDYDULHW\RIGHYLFHV
VHPLQDU RU OHFWXUH LV EHLQJ KHOG VR WKDW WKH OHFWXUHU FDQ
DQGSODWIRUPVZKHWKHULWLVDQ\W\SHRIDPRELOHGHYLFHRU
DGGUHVVWKRVHTXHVWLRQVRUFRQFHUQVGLUHFWO\DQGH[SODLQWKH
DODSWRS 7KH V\VWHP ZRXOG DFW DV DVLQJOHSDJH DSSOLFDWLRQ
DQVZHUVGLUHFWO\LQWKHOHFWXULQJSURFHVVRUDWWKHHQGRIWKH
ZLWK D5(67 $3, 7KH JHQHUDO DUFKLWHFWXUH LV VKRZQ LQ WKH
OHFWXUH :H WKLQN WKDW WKLV RQOLQH VSDFH IRU DQRQ\PRXV
GHSOR\PHQW GLDJUDP RQ )LJXUH  6RPH RI WKH FRPSRQHQWV
TXHVWLRQVFDQLQFUHDVHWKHDXGLHQFHHQJDJHPHQWDQGEUHDNWKH
KDYHEHHQRPLWWHGIRUVLPSOLILFDWLRQ
EDUULHU RI VK\QHVV 7KH VHFRQG SDUW RI WKLV V\VWHP LV FDOOHG
µTXHVWLRQV¶ 7KLV IXQFWLRQDOLW\ LV DLPHG IRU WKH OHFWXUHUV WR ĚĞƉůŽLJŵĞŶƚĞƉůŽLJŵĞŶƚDŽĚĞů

REWDLQ JXLGHG IHHGEDFN VLQFH WKH OHFWXUHU FDQ DVN GLUHFW ƉƉůŝĐĂƚŝŽŶ^ĞƌǀĞƌ

TXHVWLRQVWRZKLFKWKHVWXGHQWVFDQDQVZHU7KHUHVKRXOGEH
DW OHDVW IRXU W\SHV RI WKH TXHVWLRQV ± TXHVWLRQ ZLWK RSHQ
   
KƉĞƌĂƚŝŽŶ^LJƐƚĞŵ

DQVZHUPXOWLSOHFKRLFHRQHFKRLFHDQGVOLGHULQUDQJH8VLQJ ^ƚƌĂŶĂƐĞƌǀĞƌĂ

WKHVH TXHVWLRQV WKH OHFWXUHUV FDQ JDLQ IHHGEDFN IRU LQVWDQFH ^ĞƌǀŝĐĞĨŽƌhƐĞƌ
^ĞƌǀŝĐĞĨŽƌZŽŽŵ ^ĞƌǀŝĐĞĨŽƌ

DERXWWKHFRQWHQWRIWKHOHFWXUHLILWZDVRYHUZKHOPLQJRUWKH YƵĞƐƚŝŽŶƐ

SDUWLFLSDQWVIHOWOLNHWKH\FRXOGJRGHHSHULQWKHWRSLF$QRWKHU
H[DPSOHFDQEHDQRSHQTXHVWLRQRQZKDWSDUWVRIWKHOHFWXUH
  

Z^dW/ŽŶƚƌŽůůĞƌ Z^dW/ŽŶƚƌŽůůĞƌ Z^dW/ŽŶƚƌŽůůĞƌ

WKHDXGLHQFHGRHVQRWXQGHUVWDQGSURSHUO\RUWRPDUNGRZQ ĨŽƌhƐĞƌ ĨŽƌZŽŽŵ ĨŽƌYƵĞƐƚŝŽŶƐ

WKH WLPHV WKDW ZRXOG VXLW WKHP WKH EHVW IRU DGGLWLRQDO
FRQVXOWDWLRQVLIQHFHVVDU\8VLQJWKLVDSSURDFKWKHOHFWXUHU
ZLOOJDLQPXFKEHWWHUDQGUHJXODUIHHGEDFNDQGFDQDGMXVWWKH
WHDFKLQJDSSURDFKLQRUGHUWRPHHWWKHVWXGHQWV¶QHHGV        

 ,QRUGHUWRLQFOXGHWKHDWWHQGDQFHVKHHWLQWKHSURFHVVDV /ŶƚĞƌŶĞƚ

ZHOO DV DGMXVWLQJ WKH VROXWLRQ IRU WKH HGXFDWLRQDO SXUSRVHV  

RQO\ ZH VXJJHVW WKDW WKH VROXWLRQ VKRXOG EH OLQNHG WR WKH DŽďŝůĞĚĞǀŝĐĞ WĞƌƐŽŶĂůĐŽŵƉƵƚĞƌŽƌůĂƉƚŽƉĚĞǀŝĐĞ

XQLYHUVLW\¶V /'$3 DFFRXQWV WKHUHIRUH ERWK VWXGHQWV DQG    

WHDFKHUV FRXOG EH DEOH WR ORJLQ LQWR WKH V\VWHP ZLWK WKHLU
   
KƉĞƌĂƚŝŽŶ^LJƐƚĞŵ
KƉĞƌĂƚŝŽŶ^LJƐƚĞŵ

XQLYHUVLW\FUHGHQWLDOVDQGWKHUHZRXOGEHQRQHHGIRUWKHPWR tĞďďƌŽǁƐĞƌ tĞďďƌŽǁƐĞƌ

FUHDWHDGGLWLRQDODFFRXQWVVSHFLILFDOO\IRUWKLVV\VWHPZKLFK hƐĞƌ/ŶƚĞƌĨĂĐĞ
LV D JUHDW DGYDQWDJH LQ RSSRVLWLRQ WR WKH RWKHU ZLGHO\
hƐĞƌ/ŶƚĞƌĨĂĐĞ

DYDLODEOHVROXWLRQV
7KHJHQHUDOSURFHVVRIXVLQJWKHSURSRVHGV\VWHPLVGHVLJQHG
DVIROORZV 
)LJ'HSOR\PHQWGLDJUDPRIWKHSURSRVHGVROXWLRQ
 7KHOHFWXUHUFUHDWHVDGHGLFDWHGURRPZLWKXQLTXH
DFFHVVFRGH
7KH VROXWLRQ VKRXOG EH LPSOHPHQWHG XVLQJ VHYHUDO
 7KHOHFWXUHUFDQFUHDWHVRPHIHHGEDFNTXHVWLRQVRU DYDLODEOHWHFKQRORJLHV7KHIURQWHQGVKRXOGXVHWKH$QJXODU
RWKHUTXHVWLRQVWKDWKHZDQWVWKHDWWHQGHHVWRDQVZHU IUDPHZRUNLWVDYDLODEOHOLEUDULHVVXFKDV0DWHULDODQGRWKHU
WKLUGSDUW\SDFNDJHVWKDWSURYLGHWKHDGGLWLRQDOIXQFWLRQDOLW\
 7KHDFFHVVFRGHLVWKHQSUHVHQWHGWRWKHDWWHQGHHVWR 7KH EDFNHQG ZLOO EH LPSOHPHQWHG XVLQJ -DYD ZLWK 6SULQJ
DFFHVVWKHURRP %RRW IUDPHZRUN DQG LW VKRXOG FRPPXQLFDWH ZLWK WKH
3RVWUH64/ GDWDEDVH 2WKHU WHFKQRORJLHV ZH ZRXOG OLNH WR
PDNH XVH RI LQFOXGH 'RFNHU IRU FRQWDLQHUL]DWLRQ -3$

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 63

+LEHUQDWHDQG6SULQJ6HFXULW\ZLWK-:7WRNHQV7KHVROXWLRQ GLDORJ ER[ DERXW WKH SDVVFRGH EHLQJ LQ XVH ZLWK WKH QHZ
DVDZKROHDFWVDVDPRGXOHWKDWFDQEHLQFRUSRUDWHGLQWRWKH DYDLODEOH SDVVFRGH SURSRVDO ,I WKH XVHU FRQILUPV WKH FRGH
PRGXODUHGXFDWLRQDOV\VWHPWKDWLVFXUUHQWO\EHLQJGHYHORSHG FKDQJHWKHURRPDFFHVVFRGHZLOOEHFKDQJHGLIWKHXVHUGRHV
DWRXUIDFXOW\>@ QRWFRQILUPWKHFKDQJHWKHURRPDFWLYLW\ZLOOQRWEHUHVWRUHG
,9 ,03/(0(17$7,21
 7KLVVHFWLRQSUHVHQWVVRPHRIWKHIXQFWLRQDOLWLHVWKDWKDYH
EHHQLPSOHPHQWHGLQWRWKHVROXWLRQ
$ 8VHU,QWHUIDFH
7KHXVHULVDVNHGWRORJLQXVLQJWKHUHJLVWHUHGHPDLODQG
WKH DVVRFLDWHG SDVVZRUG $IWHU VXFFHVVIXO ORJLQ WKH XVHU LV

UHGLUHFWHGIXUWKHUGHSHQGLQJRQWKHXVHUUROH,IWKHXVHUKDV
WKH UROH RI VWXGHQW WKH\ DUH UHGLUHFWHG GLUHFWO\ WR HQWHU WKH )LJ5RRPFUHDWLRQIRUPDQGDQRYHUYLHZRIXVHU VURRPV
URRP,IWKH XVHULVD OHFWXUHUKHLVUHGLUHFWHGWRWKHKRPH
SDJHIRUWKHOHFWXUHU,IWKHXVHULVDQDGPLQLVWUDWRUWKHXVHU 7RLQWURGXFHWKHDSSOLFDWLRQWRWKHDXGLHQFHWKHOHFWXUHUV
LVUHGLUHFWHGWRWKHDGPLQLVWUDWRUKRPHSDJH7KHORJLQIRUP FDQ XVH WKH SUHSUHSDUHG $FFHVV VHFWLRQ )LJXUH   ZKLFK
LVSUHVHQWHGLQ)LJXUH GLVSOD\V WKH DFFHVV FRGH DQG URRP QDPH ,W LV XVHIXO WR
GLVSOD\WKLVVHFWLRQDWWKHEHJLQQLQJRIDOHFWXUHRUDVHPLQDU
VR WKDW SDUWLFLSDWLQJ VWXGHQWV FDQ MRLQ WKH URRP XVLQJ WKH
DFFHVVFRGHDQGFDQYHULI\WKHURRPQDPHRIWKHURRPWKH\
MRLQHG7KH\FDQWKHQSDUWLFLSDWHLQWKHRQOLQHFRQYHUVDWLRQ
DQGDQVZHUTXHVWLRQVSRVWHGE\WKHOHFWXUHUV



)LJ/RJLQIRUPRIWKHLPSOHPHQWHGV\VWHP

2QFH ORJJHG LQ VHYHUDO PHQX LWHPV ZLOO DSSHDU

GHSHQGLQJ RQ WKH XVHU V UROH 7KH ,QIR VHFWLRQ LV RQO\
DYDLODEOHZKHQWKHXVHULVLQDVSHFLILFURRPDQGFOLFNLQJRQ
LWZLOOSXOOXSDVLGHEDUZLWKEDVLFLQIRUPDWLRQ URRPQDPH
DQG SDVVFRGH  7KH ORJRXW VHFWLRQ ORJV RXW WKH FXUUHQWO\ 
ORJJHGLQXVHUDQGUHGLUHFWVWKHPWRWKHORJLQIRUP7KHRWKHU
)LJ5RRPDFFHVVGHWDLOVIRULQWURGXFLQJWKHURRPWRWKH
VHFWLRQV ZLOO EH GHVFULEHG LQ PRUH GHWDLO LQ WKH IROORZLQJ DXGLHQFH
SDUDJUDSKV7KHH[DPSOHRIWKHQDYLJDWLRQPHQXLVSUHVHQWHG
LQWKH)LJXUH 7KHIRUPQDPHG3DUWLFLSDQWVKRZHGLQ)LJXUHLVXVHG
WRDFFHVVWKHURRPLQWKHUROHRIDSDUWLFLSDQW7KLVIRUPLV
XVHGWRORJLQWRWKHURRPE\VWXGHQWVWKDWDUHDWWHQGLQJWKH
HYHQWDVDQDXGLHQFH7KHOHFWXUHUVDOVRKDYHWKHRSWLRQWRORJ
LQWRWKHURRPDVDSDUWLFLSDQW7KHOHFWXUHUZKRLVWKHDXWKRU

RIWKHURRPWKXVKDVWZRRSWLRQVIRUHQWHULQJWKHURRP(LWKHU
)LJ1DYLJDWLRQPHQX WKURXJKWKHRYHUYLHZVHFWLRQZKHUHWKH\HQWHUWKHURRPDV
WKH DXWKRU RU WKURXJK WKH SDUWLFLSDQW VHFWLRQ ZKHUH WKH\
 7KH RYHUYLHZ VHFWLRQ LVRQO\DYDLODEOH WR XVHUV ZLWK WKH
HQWHUWKHURRPDVDSDUWLFLSDQW
UROHRIOHFWXUHU DVDKRPHSDJH ,WDOORZVWKHPWRFUHDWHDQG
YLHZVHYHUDOURRPVDQGXSGDWHWKHLUGDWD2QWKHOHIWVLGHRI
WKH)LJXUHLVWKHIRUPIRUFUHDWLQJDQHZURRPRQWKHULJKW
LVDOLVWRIXVHU¶VFUHDWHGURRPV)RUHDFKURRPLWVQDPH OHIW
VLGH DQGDFFHVVFRGH ULJKWVLGH DUHOLVWHG$IWHURSHQLQJWKH
SDQHOWKHDFWLRQVDWWULEXWHGWRWKHURRPDUHSRVVLEOHHQWHULQJ
WKH URRP DV LWV DXWKRU GLVSOD\LQJ WKH DFFHVV GDWD IRU WKH
DXGLHQFH $FFHVV DQGGHOHWLQJWKHURRP2QWKHULJKWVLGHRI
WKHSDQHOWKHUHLVDWRJJOHVZLWFKWKDWGLVSOD\VDQGFKDQJHVWKH
DFWLYLW\RIWKHURRP &OLFNWKHVZLWFKWRFKDQJHDQLQDFWLYH
URRPWRDQDFWLYHURRPDQGYLFHYHUVD 6LQFHWKHURRPDFFHVV 
FRGHPXVWEHXQLTXHWRWKHDFWLYHURRPLWLVSRVVLEOHWRXVHD
VLQJOH DFFHVV FRGH LQ GLIIHUHQW URRPV EXW RQO\ RQH RI WKH )LJ)RUPIRUSDUWLFLSDQWDFFHVVWRWKHURRP
URRPV FDQ EH DFWLYH DW DQ\ JLYHQ WLPH ,I WKH XVHU WULHV WR
HQDEOHWKHDFWLYLW\RIDURRPZKRVHSDVVFRGHLVWKHVDPHDV 7KHYLUWXDOURRPVKDYHWZRDOUHDG\PHQWLRQHGVHFWLRQV
WKHSDVVFRGHRIDQRWKHUURRP EHORQJLQJWRKLPRUDQ\RWKHU ,QWKHILUVWVHFWLRQLWLVSRVVLEOHWRMRLQWKHFRQYHUVDWLRQE\
XVHU WKDWLVFXUUHQWO\DFWLYHWKHXVHUZLOOEHSUHVHQWHGZLWKD PHVVDJLQJ DQG UHDFWLQJ WR PHVVDJHV ZULWWHQ E\ WKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 64

SDUWLFLSDQWV 7KH )LJXUH  VKRZV H[DPSOHV RI PHVVDJHV
TXHVWLRQV  GLUHFWHG IURP WKH VWXGHQWV WR WKH OHFWXUHU 7KH
FRQYHUVDWLRQLVSULPDULO\IRUTXHVWLRQVIURPWKHDXGLHQFHWR
WKHSUHVHQWHU7KHDXWKRURIWKHURRPDOVRKDVWKHDELOLW\WR
GHOHWHVSHFLILFPHVVDJHVDQGDOVRWKHDELOLW\WRGRZQORDGWKH
PHVVDJHVLQWKHFRQYHUVDWLRQ E\FOLFNLQJWKHEXWWRQRQWKH 
ERWWRP ULJKW  DORQJ ZLWK WKHLU UHVSRQVH FRXQWV LQ DQ
)LJ)RUPIRUTXHVWLRQFUHDWLRQE\WKHURRPDXWKRU OHFWXUHU 
VSUHDGVKHHW IRUPDW (DFK PHVVDJH FDQ EH JLYHQ D UHDFWLRQ
WKDWH[SUHVVHVDJUHHPHQWZLWKWKHFRQWHQWRIWKHPHVVDJH$Q 'HSHQGLQJRQWKHW\SHRIWKHTXHVWLRQWKHSDUWLFLSDQWZLOO
DGYDQWDJH RI WKH UHDFWLRQV WR PHVVDJHV LV WKH SRVVLELOLW\ WR EHSUHVHQWHGZLWKDVSHFLILFIRUPWRFRPSOHWH$IWHUILOOLQJLQ
VRUW WKH PHVVDJHV IURP PRVW SRSXODU LH IURP WKH WKH IRUP WKH SDUWLFLSDQW VXEPLWV WKH IRUP E\ SUHVVLQJ WKH
PHVVDJHV ZLWK WKH KLJKHVW QXPEHU RI UHDFWLRQV WR WKH 6XEPLWEXWWRQ$IWHUVXEPLWWLQJWKHDQVZHUWKHSDUWLFLSDQW
PHVVDJHV ZLWK WKH ORZHVW QXPEHU RI UHDFWLRQV 7KXV WKH QRORQJHUKDVWKHRSWLRQRIPDNLQJFKDQJHVWRWKHDQVZHURU
SUHVHQWHU FDQ DVVHVV ZKLFK TXHVWLRQ WKH ODUJHVW QXPEHU RI GHOHWLQJWKHDQVZHU'LIIHUHQWDQVZHURSWLRQVGHSHQGLQJRQ
OLVWHQHUV ZDQW WR EH DQVZHUHG ZLWKRXW VFDQQLQJ WKH ZKROH WKHW\SHRIWKHTXHVWLRQDUHSUHVHQWHGRQWKH)LJXUH
FRQYHUVDWLRQ


)LJ([DPSOHVRIPHVVDJHVIRUWKHZHEDQGPRELOHGHYLFHIURP
URRPDXWKRU OHIW DQGVWXGHQW ULJKW SHUVSHFWLYH

)LJXUH  SUHVHQWV D VDPSOH ILOH RI WKH GRZQORDGHG
)LJ([DPSOHVRIOHFWXUHUV TXHVWLRQVIURPVWXGHQWSHUVSHFWLYH
FRQYHUVDWLRQZKLFKWKHDXWKRURIWKHURRPFDQVDYHDQGWKXV
FDQGHOHWHWKHXQQHFHVVDU\URRPDWDQ\WLPHNQRZLQJWKDWKH 7KHUHVXOWVRIWKHTXHVWLRQDUHGLVSOD\HGDXWRPDWLFDOO\WR
ZLOOQRWORVHGDWDWKDWPD\EHRIXVHWRKLPLQWKHIXWXUHZKHQ WKHDXWKRURIWKHURRP)RUTXHVWLRQVZLWKSUHSDUHGDQVZHUV
SUHSDULQJ QHZ OHFWXUH SUHVHQWDWLRQV 7KH QDPH RI WKH ILOH DOOWKHDQVZHUVDQGWKHQXPEHURIYRWHVIRUWKDWDQVZHUDUH
GHSHQGV RQ WKH GDWH WKH ILOH ZDV GRZQORDGHG DQG LV GLVSOD\HG DORQJ ZLWK WKH SHUFHQWDJH RI YRWHV )RU D VOLGHU
DXWRPDWLFDOO\ JHQHUDWHG IRU H[DPSOH $XGLHQFH4XHVWLRQV TXHVWLRQRQO\WKRVHYDOXHVWKDWUHFHLYHGDWOHDVWRQHYRWHLQ
[OV[ WKH SROO DUH GLVSOD\HG )RU DQ RSHQHQGHG TXHVWLRQ DOO
UHVSRQVHV DUH VKRZQ LQ D OLVW %HORZ WKH SROO UHVXOWV WKH
DXWKRUKDVWKHRSWLRQWRWRJJOHWKHYLVLELOLW\RIWKHTXHVWLRQ
IRUWKHDXGLHQFHDQGDQRWKHUWRJJOHEXWWRQWRGLVSOD\RUKLGH
WKH UHVXOWV WR WKH DXGLHQFH 7KHUH LV DOVR WKH SRVVLELOLW\ WR
 GRZQORDGWKHUHVXOWVDJDLQDVDVSUHDGVKHHWGRFXPHQWRUWR
GHOHWHWKHTXHVWLRQSHUPDQHQWO\)LJXUHVKRZVWKHUHVXOWV
)LJ6DPSOHILOHFRQWDLQLQJDOLVWRIXVHUTXHVWLRQVLQPHVVDJHV
VHFWLRQ GLVSOD\HGIRUWKHDXWKRURQERWKPRELOHDQGODSWRSGHYLFH

7KHVHFRQGSDUWRIWKHYLUWXDOURRPDUHTXHVWLRQVIURPWKH
VSHDNHU IRU WKH DXGLHQFH $ TXHVWLRQ FDQ EH FUHDWHG E\ WKH
DXWKRURIWKHURRPLQWKHSROOVHFWLRQXVLQJWKHIRUPWRFUHDWH
D QHZ TXHVWLRQ 7KH H[DPSOHV RI FUHDWLQJ VRPH W\SHV RI
TXHVWLRQDUHVKRZQLQ)LJXUH6LQFHWKHTXHVWLRQZLOOQRWEH
GLVSOD\HGWRWKHSDUWLFLSDQWVLQWKHURRPLIYLVLELOLW\LVWXUQHG
RII YLVLELOLW\ LV WXUQHG RII E\ GHIDXOW DIWHU WKH TXHVWLRQ LV
FUHDWHG DQRWLILFDWLRQZLOOEHGLVSOD\HGWRWKHDXWKRURIWKH
TXHVWLRQUHPLQGLQJWKHDXWKRUWRWXUQYLVLELOLW\RQDVQHHGHG
WR SUHVHQW WKH TXHVWLRQ WR WKH DXGLHQFH DW DQ\ GHVLUHG WLPH 
H[FOXVLYHO\ )LJ([DPSOHRIWKHTXHVWLRQUHVXOWVIURPOHFWXUHUSHUVSHFWLYH

,QWKHFDVHRIDVWXGHQWRUDWWHQGHHWKHTXHVWLRQDQVZHUVZLOO
RQO\EHGLVSOD\HGDIWHUWKHDQVZHUWRWKHTXHVWLRQKDVEHHQ
VXEPLWWHG DVVXPLQJ RI FRXUVH WKDW WKH URRP DXWKRU KDV
HQDEOHGWKHµ6KRZDQVZHUVWRDOO¶VZLWFK,IWKHDXWKRUKDV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 65

QRWHQDEOHGWKHGLVSOD\RIUHVXOWV DQVZHUV IRUSDUWLFLSDQWV
WKH SDUWLFLSDQW ZLOO RQO\ UHFHLYH D 7KDQN \RX IRU \RXU
DQVZHUPHVVDJHZKLFKLVGHPRQVWUDWHGLQWKH)LJXUH

 
)LJ([DPSOHRIWKHTXHVWLRQUHVXOWVIURPVWXGHQWSHUVSHFWLYH )LJ([DPSOHRIWHVWLQJWKHEDFNHQGVHUYHU

)LJXUH  GHPRQVWUDWHV WKH SUHYLHZ RI WKH GRZQORDGHG 9, &21&/86,21
TXHVWLRQ UHVSRQVHV )RU RSHQHQGHG TXHVWLRQV LQGLYLGXDO  ,Q WKLV ZRUN ZH SUHVHQWHG D V\VWHP IRU FROOHFWLQJ
UHVSRQVHV WR WKH TXHVWLRQV DUH UHFRUGHG IRU TXHVWLRQV ZLWK IHHGEDFNIURPVWXGHQWVLQDQLQWHUDFWLYHPDQQHU1RZDGD\V
GHIDXOWDQVZHUVDOOUHVSRQVHVDQGWKHFRXQWVRIWKHYRWHVDUH LWLVLPSRUWDQWWREHLQWHUHVWHGLQFROOHFWLQJIHHGEDFNEHFDXVH
UHFRUGHG LW LV GLIILFXOW WR PRWLYDWH VWXGHQWV WR JLYH WKHLU RSLQLRQ
7KHUHIRUHWKHDLPZDVWRGHVLJQDQGLPSOHPHQWDPRGXOHIRU
DFRPPXQLFDWLRQWRROWRVXSSRUWOHDUQLQJ7KHLPSOHPHQWHG
PRGXOHPHHWVWKHEDVLFUHTXLUHGIXQFWLRQDOLW\LHLWDOORZV
FUHDWLQJSURILOHVIRUVHYHUDOW\SHVRIXVHUV,WDOORZVXVHUVRI
WKH OHFWXUHU W\SH WR FUHDWH YLUWXDO URRPV DQG PDNH WKHP
DFFHVVLEOHWRORJJHGLQSDUWLFLSDQWVXVLQJDQDFFHVVFRGH7KH
 DXWKRURIWKHURRPFDQDGGTXHVWLRQVRIGLIIHUHQWW\SHVLQWKH
SROOVHFWLRQDQGPDQDJHWKHLUGLVSOD\RUGLVSOD\WKHUHVXOWVRI
)LJ6DPSOHVSUHDGVKHHWZLWKWKHUHVXOWVRIWKHTXHVWLRQ WKHTXHVWLRQ5RRPSDUWLFLSDQWVFDQVXEPLWDQVZHUVWRWKHVH
SROOVDIWHUHQWHULQJWKHURRPRUYLHZWKHDYDLODEOHUHVXOWVDIWHU
$QRWKHU DYDLODEOH IXQFWLRQDOLW\ LV WKH DGPLQLVWUDWLRQ YRWLQJ %RWK WKH SDUWLFLSDQW DQG WKH LQVWUXFWRU FDQ MRLQ WKH
VHFWLRQ7KHDGPLQLVWUDWLRQUROHLVDERYHDOODQGKDVWKHULJKWV FRQYHUVDWLRQ LQ WKH URRP DOWKRXJK WKH SULPDU\ XVH RI WKLV
WRGHOHWHWKHXVHUSURILOHDORQJZLWKWKHLUURRPVRULVDEOHWR FRQYHUVDWLRQLVIRUTXHVWLRQVIURPSDUWLFLSDQWVWRWKHDXWKRURI
UHPRYHDURRPXSRQDUHTXHVWRUFRPSODLQW WKHURRP6HYHUDODGGLWLRQDOIXQFWLRQDOLWLHVKDYHEHHQDGGHG
 6LQFH WKH UHTXLUHPHQW RI WKH JDWKHULQJ DWWHQGDQFH WR WKH DSSOLFDWLRQ VXFK DV WKH SRVVLELOLW\ RI UHDFWLQJ WR
IXQFWLRQDOLW\ ZDV VSHFLILHG ODWHU LQ WKH LPSOHPHQWDWLRQ PHVVDJHV H[SUHVVLQJDJUHHPHQWZLWKWKHTXHVWLRQDVNHG DQG
SURFHVVWKLVIHDWXUHLVFXUUHQWO\EHLQJLPSOHPHQWHGLQWRWKH WKHSRVVLELOLW\RIFKRRVLQJWKHRUGHURIPHVVDJHV²IURPWKH
V\VWHP7KHIHDWXUHLVDYDLODEOHLQWKHTXHVWLRQYLHZIRUWKH PRVWUHFHQWRUIURPWKHPRVWUHDFWLRQV)RUWKHDXWKRURIWKH
DXWKRURIWKHURRPDVDQRWKHUEXWWRQ$IWHUFOLFNLQJWKHEXWWRQ URRP WKH SRVVLELOLW\ WR GRZQORDG WKH PHVVDJHV LQ WKH
WKHV\VWHPVKRZVWKHQXPEHURIWKHUHVSRQVHVDQGWKHOLVWRI FRQYHUVDWLRQRUWKHUHVXOWVRIWKHSROOLQVSUHDGVKHHWIRUPDW
WKHSDUWLFLSDQWVWKDWDQVZHUHGWKHTXHVWLRQ ZDVDGGHGIRUIXUWKHUSURFHVVLQJRUDUFKLYLQJ
9 9(5,)<,1*7+(,03/(0(17$7,21  $VPHQWLRQHGSUHYLRXVO\ZHZDQWWRJHWWKHLQWHUHVWRIWKH
DXGLHQFHWKDWLVZK\ZHGHFLGHGWRDLPWKHTXHVWLRQVLQDIXQ
 7KH VROXWLRQ KDV EHHQ YHULILHG XVLQJ PDQXDO WHVWLQJ
DQG HDV\ WRSLFV WKDW VWXGHQWV FDQ UHODWH ZLWK %\ DVNLQJ DQ
VWUDWHJ\7KHXVHULQWHUIDFHZDVWHVWHGE\XVLQJZURQJLQSXWV
LQWHUHVWLQJTXHVWLRQLWLVSRVVLEOHWRFUHDWHLQGLUHFWLQWHUHVWDQG
LQWKHIRUPV LHLQFRUUHFWHPDLOZURQJQXPEHUVSDVVFRGHV
FXULRVLW\ WR DQVZHU ,Q WKH VDPH ZD\ ZH FDQ JHW WKH
HWF  7KH EDFNHQG ZDV WHVWHG VHQGLQJ GLIIHUHQW NLQGV RI
DWWHQGDQFHRIWKHVWXGHQWVLQEHWZHHQ,QWKHV\VWHPKRZHYHU
UHTXHVWVZLWKWKHZURQJW\SHVYLDWKH3RVWPDQVRIWZDUH$Q
WKHDQVZHUVWRWKHTXHVWLRQVUHPDLQVHSDUDWHDQGDQRQ\PL]HG
H[DPSOHRIWKHYHULILFDWLRQIRUWKHXVHULQWHUIDFHLVVKRZQRQ
IURPWKHFROOHFWHGDWWHQGDQFH:KHQHYDOXDWLQJWKHUHVXOWVLW
WKH)LJXUHDQGH[DPSOHRIWHVWLQJWKHEDFNHQGLVSUHVHQWHG
LV SRVVLEOH WR SUHVHQW WKHVH UHVXOWV WR WKH DXGLHQFH ZKLFK
LQWKH)LJXUH
PRWLYDWHVWKHPWRSDUWLFLSDWH$QRWKHUJUHDWDGYDQWDJHLVWKH
OHFWXUHWLPHVDYHGVLQFHVWXGHQWVDUHTXLFNDQGLWXVXDOO\WDNHV
XSWRPLQXWHVWRJDWKHUDOOWKHUHVSRQVHV6RPHH[DPSOHVRI
WKHTXHVWLRQVWRSURPRWHWKHHQJDJHPHQWDUH
• 'RHVSLQHDSSOHEHORQJRQWKHSL]]D"
• 'R\RXSUHIHUDUUD\LQGH[LQJVWDUWLQJZLWKRU"
• 'R\RXSUHIHUFRIIHHRUWHD"
 • :KLFK WHDP GR \RX EHORQJ WR GHVNWRS VFUHHQ IHZ
LFRQRUIXOO\ORDGHG"
)LJ([DPSOHRIWHVWLQJWKHXVHULQWHUIDFH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 66

 7KH SURFHVVLQJ DQG HYDOXDWLRQ RI WKHVH UHVSRQVHV FDQ 5()(5(1&(6
VHUYHDGXDOSXUSRVH3ULPDULO\LWLVDVRXUFHRILQIRUPDWLRQ >@ ³6OLGRIHDWXUHV_6OLGR$XGLHQFH,QWHUDFWLRQ0DGH(DV\´$FFHVVHG
DERXWWKHVWXGHQWURVWHURUWKHVWDWXVRIIHHGEDFNVRQDJLYHQ 6HS   >2QOLQH@ $YDLODEOH
LVVXH $QRWKHU XVH RI WKLV GDWD LV IRU JHQHUDWLQJ VWDWLVWLFDO KWWSVZZZVOLGRFRPSURGXFW"H[SHULHQFHBLG D
DQDO\VHVRQWKHVWDWXVDQGRSLQLRQVRIVWXGHQWVRUVRFDOOHG >@ & & 6KHQ < 0 :DQJ - :X ) ' 0HQJ DQG 7 : 6KRX
VRFLDOJURXSVZKLFKFDQEHDQDO\]HGXVLQJPRUHVRSKLVWLFDWHG ³7HDFKLQJ (IIHFW RI ,QWHUDFWLYH 5HVSRQVH 6\VWHP LQ 1HZ 0HGLD
PHWKRGV7KHVHPHWKRGVPD\LQFRUSRUDWHPRUHVRSKLVWLFDWHG &RXUVHV´WK,QWHUQDWLRQDO&RQIHUHQFHRQ2UDQJH7HFKQRORJ\
,&27GRL,&27
ZRUNZLWKGHFLVLRQWUHHV>@RUUHOLDELOLW\DQDO\VLVWHFKQLTXHV
>@ ³+RZLWZRUNV3LJHRQKROH/LYH´$FFHVVHG6HS>2QOLQH@
>@ ZKLFK ZLOO DOORZ WKH FUHDWLRQ RI DQDO\VHV RI KLGGHQ $YDLODEOHKWWSVSLJHRQKROHOLYHFRPKRZLWZRUNVKRZ DXGLHQFH
LQIRUPDWLRQDQGUHODWLRQVKLSVEHWZHHQUHVSRQVHVZKHUHWKHVH >@ %/:XODQGDULDQG0+DUWRQR³7KHHIIHFWLYHQHVVRIXVLQJ4XL]L]]
UHVXOWVFDQEHXVHGWRDOHUWWKHWHDFKHURISRVVLEOHSLWIDOOVRU DQG 4XL]DOL]H PHGLD RQ PDWKHPDWLFV OHDUQLQJ RXWFRPHV RI JUDGH ,9
LPSHUIHFWLRQV LQ WKH WHDFKLQJ 7KHVH DQDO\VHV ZLOO EH GHDOW VWXGHQWV´ 3URFHHGLQJV   WK ,QWHUQDWLRQDO &RQIHUHQFH RQ
ZLWK LQ WKH QH[W VWXG\ ZKLFK FDQ EH FDUULHG RXW RQ WKH (GXFDWLRQDQG7HFKQRORJ\,&(7SS±6HSGRL
FROOHFWHGGDWDIURPWKLVV\VWHP ,&(7
>@ 0 .URPND 3 )LGHN DQG - .RVWROQ\ ³'HYHORSLQJ 0RGXODU
 ,QIRUPDWLRQ 6\VWHP ZLWK 0LFURVHUYLFH 7HFKQRORJ\´ LQ  WK
,QWHUQDWLRQDO &RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ 7HFKQRORJLHV DQG
$&.12:/('*0(17 $SSOLFDWLRQV ,&(7$  ,((( 1RY  SS ± GRL
,&(7$
7KLV ZRUN ZDV VXSSRUWHG E\ *UDQW RI 6\VWHP RI WKH >@ =DLWVHYD ( /HYDVKHQNR 9 &RQVWUXFWLRQ RI D UHOLDELOLW\ VWUXFWXUH
8QLYHUVLW\ RI =LOLQD 1R  DQG E\ 0LQLVWU\ RI IXQFWLRQ EDVHG RQ XQFHUWDLQ GDWD ,((( 7UDQ RQ 5HOLDELOLW\   
(GXFDWLRQ 6FLHQFHDQG 6SRUW RIWKH 6ORYDN 5HSXEOLF XQGHU SS'2,75
WKHJUDQW9(*$ >@ =DLWVHYD ( /HYDVKHQNR 9 5HOLDELOLW\ DQDO\VLV RI 0XOWL6WDWH
6\VWHPDQG0XOWLSOH9DOXHG/RJLF,QWHUQDWLRQDO-RXUQDORI4XDOLW\ 
 5HOLDELOLW\0DQDJHPHQW  SS


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 67

 'HYHORSPHQWRI2QOLQH$QDWRPLFDO$WODVIRU
%LRPHGLFDO,QIRUPDWLFV

'RPLQLN%REDN$QHWD*DEULVRYD0DWHM-XULFHN3HWHU.XODV$GULDQ/LVWYDQ$QGUHM-HOLQHN.DUROLQD
.DUGRVRYD0LURVODY.YDVVD\
8QLYHUVLW\RI=LOLQD)DFXOW\RI0DQDJHPHQW6FLHQFHDQG,QIRUPDWLFV=LOLQD6ORYDNLD
^EREDNJDEULVRYDMXULFHNNXODVOLVWYDQ`#VWXGXQL]DVNDQGUHMMHOLQHN#JPDLOFRP
NDUGRVRYDNDUROLQD#JPDLOFRPPLURVODYNYDVVD\#IULXQL]DVN


$EVWUDFW²%LRPHGLFDO DQG PHGLFDO LQIRUPDWLFV EHORQJV WR HGXFDWHG JUDGXDWHV LQ ,&7EDVHG VWXG\ SURJUDPV 7KHVH
UDSLGO\JURZLQJDUHDZKHUHLQIRUPDWLFVDQG PHGLFLQHPHHW JUDGXDWHV VKRXOG EH HGXFDWHG QRW RQO\ LQ ,&7 VRIWZDUH
WR LPSURYH TXDOLW\ RI KHDOWKFDUH DQG KXPDQ ZHOOEHLQJ DV HQJLQHHULQJ GDWD SURFHVVLQJ DQG DQDO\VLV EXW DOVR LQ WKH
ZHOO7KLVIDFWFDQEHREVHUYHGLQPDQ\VRIWZDUHFRPSDQLHV EDVLFV RI PHGLFDO DQG ELRPHGLFDO WHUPLQRORJ\ VR WKH\
WKDW KDYH VWDUWHG WR IRFXV RQ WKH GHYHORSPHQW RI VRIWZDUH DUH DEOH WR FRPPXQLFDWH ZLWK SK\VLFLDQV QXUVHULHV VWDII
VROXWLRQVIRUKHDOWKFDUHLQVWLWXWLRQVIRUDQDO\VLVDQGIXUWKHU RI ELRPHGLFDO ODERUDWRULHV DQG ELRPHGLFDO HQJLQHHUV ,I
SURFHVVLQJ RI PHGLFDO DQG ELRPHGLFDO GDWD DV ZHOO DV IRU 6ORYDNLD ZDQWV WR EH VXFFHVVIXO LQ WKLV DUHD WHFKQLFDOO\
DVVLVWDQFH LQ PHGLFDO GLDJQRVLV DQG WUHDWPHQW 7KLV WUHQG RULHQWHG XQLYHUVLWLHV KDYH WR SUHSDUH H[FHOOHQW VRIWZDUH
FDOOVIRUQHZJHQHUDWLRQRI,7H[SHUWVWKDWDUHHGXFDWHGQRW HQJLQHHUV RU GDWD DQDO\VWV ZKR DUH DEOH WR XVH WKHLU
RQO\LQVRIWZDUHHQJLQHHULQJEXWDOVRLQGDWDDQDO\VLVDQGLQ NQRZOHGJH DQG VNLOOV LQ WKH GHYHORSPHQW RI DGYDQFHG
EDVLFV RI PHGLFDO DQG ELRPHGLFDO WHUPLQRORJ\ VR WKH\ DUH PHGLFDOVRIWZDUHDQGLQFUHDWLRQRIFRPSOH[VROXWLRQVIRU
DEOH WR VSHDN WKH VDPH ODQJXDJH DV SK\VLFLDQV DQG RWKHU SURFHVVLQJ PHGLFDO DQG ELRPHGLFDO GDWD $PRQJ RWKHUV
VWDII RI KRVSLWDOV DQG ELRPHGLFLQH ODERUDWRULHV 7KHUHIRUH WKHVH QHHGV DUH UHIOHFWHG E\ WKH )DFXOW\ RI 0DQDJHPHQW
/DWLQWHUPLQRORJ\DQG EDVLF NQRZOHGJHRI KXPDQ DQDWRP\ 6FLHQFHDQG,QIRUPDWLFVRIWKH8QLYHUVLW\RI=LOLQDZKLFK
DQG SK\VLRORJ\ KDYH EHFRPH D FUXFLDO SRLQW LQ WKHLU RSHQHG WKH ILUVW PDVWHU VWXG\ SURJUDP RQ PHGLFDO DQG
HGXFDWLRQ +RZHYHU VWXGHQWV RI WHFKQLFDOO\RULHQWHG VWXG\ ELRPHGLFDO LQIRUPDWLFV LQ 6ORYDNLD LQ  7KH VWXG\
SURJUDPV XVXDOO\ GR QRW OLNH HGXFDWLRQ EDVHG RQ VHOIVWXG\ SURJUDP LV QDPHG %LRPHGLFDO ,QIRUPDWLFV >@ DQG LW LV
RI PDQ\ WHDFKLQJ PDWHULDOV DQG PHPRUL]LQJ WKHP 7KH\ FRPSRVHG RI FRXUVHV UHODWHG WR VRIWZDUH HQJLQHHULQJ
SUHIHUWROHDUQE\GRLQJSUDFWLFDODVVLJQPHQWVZKLFKIRUFH PDQDJHPHQWDQGDQDO\VLVRIPHGLFDODQGELRPHGLFDOGDWD
WKHP WR DFTXLUH QHZ WKHRUHWLFDO NQRZOHGJH DQG LQFUHDVH DV ZHOO DV FRXUVHV UHODWHG WR EDVLFV RI ELRPHGLFDO DQG
WKHLUSUDFWLFDOHQJLQHHULQJVNLOOV)RUWKLVSXUSRVHZHKDYH
PHGLFDOWHUPLQRORJ\KXPDQDQDWRP\DQGSK\VLRORJ\
LQLWLDWHG WKH GHYHORSPHQW RI DQ RQOLQH DQDWRPLFDO DWODV IRU
WHDFKLQJ KXPDQ DQDWRP\ 7KH DWODV GHYHORSPHQW KDV EHHQ %LRPHGLFDO ,QIRUPDWLFV DW WKH )DFXOW\ RI 0DQDJHPHQW
VWDUWHG E\ VWXGHQWV RI WKH )DFXOW\ RI 0DQDJHPHQW 6FLHQFH 6FLHQFHDQG,QIRUPDWLFVKDVFXUUHQWO\JUDGXDWHV)RXU
DQG,QIRUPDWLFVRIWKH8QLYHUVLW\RI=LOLQD,QWKLVSDSHUZH RI WKHP FRQWLQXHV LQ WKHLU GRFWRUDO VWXGLHV ZKHUH WKH\
SUHVHQWLWVEDVLFDUFKLWHFWXUHWRJHWKHUZLWKLWVFRUHIHDWXUHV IRFXV RQ DSSOLFDWLRQ RI PDFKLQH OHDUQLQJ LQ ELRPHGLFLQH
DQG PHGLFLQH 2WKHUV DUH PRVWO\ HPSOR\HG DV VRIWZDUH
.H\ZRUGV±DQDWRPLFDODWODVPHGLFDOLQIRUPDWLFVVRIWZDUH HQJLQHHUVDQGSURJUDPPHUVLQ,7FRPSDQLHVGHDOLQJZLWK
HQJLQHHULQJWKUHHWLHUDUFKLWHFWXUHZHE'DSSOLFDWLRQ WKH GHYHORSPHQW RI PHGLFDO VRIWZDUH 6RPH RI WKH
SUHYLRXVDQGFXUUHQWVWXGHQWVRIWKLVVWXG\SURJUDPKDYH
SURSRVHGWRVXSSRUWHGXFDWLRQRIPHGLFDOWHUPLQRORJ\DQG
, ,1752'8&7,21
EDVLFVRIKXPDQDQDWRP\E\WKHGHYHORSPHQWRIDQRQOLQH
$VUHIOHFWHGLQVHYHUDOUHVROXWLRQVRIWKH:RUOG+HDOWK DWODVRIKXPDQDQDWRP\7KHDWODVKDVEHHQGHYHORSHGE\
$VVHPEO\ >@ XVH RI ,QIRUPDWLRQ DQG &RPPXQLFDWLRQ WKHVWXGHQWVZLWKLQWKHLUSURMHFWHGXFDWLRQ
7HFKQRORJLHV ,&7  IRU KHDOWK ZKLFK DJUHHV ZLWK WHUP 3URMHFWHGXFDWLRQLVDVHULHVRIWKUHHREOLJDWRU\FRXUVHV
H+HDOWK EHORQJV FXUUHQWO\ WR WKH PRVW UDSLGO\ JURZLQJ WKDWKDYHWREHFRPSOHWHGE\VWXGHQWVGXULQJWKHLUPDVWHU
DUHDV LQ KHDOWK )RU H[DPSOH ,&7 DUH XVHG LQ WKH VWXG\,WVDLPLVWRDSSO\NQRZOHGJHREWDLQHGZLWKLQRWKHU
GHYHORSPHQW RI H[SHUW V\VWHPV VXSSRUWLQJ GHFLVLRQ FRXUVHV LQ VROYLQJ VFLHQWLILF RU SUDFWLFDO SUREOHPV
PDNLQJ LQ PHGLFDO GLDJQRVLV DQG WUHDWPHQW > @ LQ WKH 7\SLFDOO\ VWXGHQWV RI WKH %LRPHGLFDO ,QIRUPDWLFV WDNH
GHYHORSPHQWRIPHGLFDOVLPXODWRUVIRUWUDLQLQJRIVSHFLILF SDUWLQSURMHFWVGHDOLQJZLWKPRGHOOLQJDQGVLPXODWLRQRI
PHGLFDOSURFHGXUHV>@VXFKDVORFDODQHVWKHVLD>@RU SK\VLRORJLFDOSURFHVVHV HJEORRGIORZ GHYHORSPHQWRI
VXUJHULHV >@ RU LQ FUHDWLRQ RI SURVWKHWLFV DQG RUWKRWLFV PHGLFDO LQIRUPDWLRQ V\VWHPV HJ KRVSLWDO LQIRUPDWLRQ
XVLQJ'SULQWLQJ>@2WKHUH[DPSOHVRIDSSOLFDWLRQRI V\VWHP FUHDWLRQRIPHWKRGVIRUSURFHVVLQJRIELRPHGLFDO
,&7 LQ PHGLFLQH DQG ELRPHGLFLQH LQFOXGH WKHLU XVH LQ DQGPHGLFDOGDWDXVLQJPDFKLQHOHDUQLQJ HJSURFHVVLQJ
PRQLWRULQJ KHDOWK LQGLFDWRUV XVLQJ YDULRXV IRUPV RI RI ((* DQG(&*GDWD DQGDSSOLFDWLRQRI ' PRGHOOLQJ
ZHDUDEOHV > @ SURFHVVLQJ RI ELRPHGLFDO GDWD > DQG YLVXDOL]DWLRQ LQ PHGLFLQH ,Q WKH FDVH RI WKH ODVW
@RUWKHGHYHORSPHQWRIKXPDQPDFKLQHLQWHUIDFHV> GRPDLQWKHVWXGHQWVKDYHFRPHZLWKWKHLGHDWRFUHDWHWKH
@ $OO WKHVH DSSOLFDWLRQV RI ,&7 DV ZHOO DV IXWXUH RQOLQH DQDWRPLFDO DWODV 7KH DWODV VKRXOG VXSSRUW '
GLVFRYHULHV DQG EUHDNWKURXJKV LQFOXGLQJ FUHDWLRQ RI YLVXDOL]DWLRQRIKXPDQRUJDQVDQGRUJDQV\VWHPVVKRXOG
DUWLILFLDO RUJDQV > @ RU WKH GHYHORSPHQW RI DUWLILFLDO FRQWDLQ 6ORYDN /DWLQ DQG (QJOLVK PHGLFDO WHUPLQRORJ\
FRJQLWLYHV\VWHPV>@UHTXLUHLQFUHDVLQJQXPEHURIZHOO VKRXOGEHDEOHWRDQLPDWHVSHFLILFSK\VLRORJLFDOSURFHVVHV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 68

RU PRWLRQV RI KXPDQ ERG\ HJ EUHDWKLQJ KHDUWEHDW %LR'LJLWDO LV DFFHVVLEOH WKURXJK D ZHE LQWHUIDFH
IDVFLDO PXVFOH FRQWUDFWLRQV GXULQJ H[SUHVVLQJ HPRWLRQV  DOORZLQJ XVHUV WR DFFHVV LW IURP DQ\ LQWHUQHWFRQQHFWHG
DQG VKRXOG FRQWDLQ WHVWLQJ HQYLURQPHQW IRU FKHFNLQJ WKH GHYLFH,WLVDOVRHTXLSSHGZLWKWRROVIRUFUHDWLQJFXVWRP
DFTXLUHGNQRZOHGJH$IWHULWVILQLVKLQJLWVKRXOGEHXVHG OHVVRQV DQG WHVW TXHVWLRQV PDNLQJ LW HDVLHUWRFUHDWH DQG
LQ WKH REOLJDWRU\ FRXUVH QDPHG %DVLFV RI 7KHRUHWLFDO VKDUHHGXFDWLRQDOFRQWHQW
0HGLFLQH ZKLFK JRDO LV WR PDNH WKH DWWHQGHHV IDPLOLDU
ZLWKPHGLFDOWHUPLQRORJ\DQGKXPDQDQDWRP\ % &RPSOHWH$QDWRP\
,Q WKLV SDSHU ZH SUHVHQW EDVLF VWUXFWXUH RI WKLV RQOLQH &RPSOHWH $QDWRP\ )LJ   LV DQRWKHU SODWIRUP
DQDWRPLFDO DWODV )LUVWO\ ZH IRFXV RQ WKH FRPSDULVRQ RI GHGLFDWHG WR PHGLFDO HGXFDWLRQ /LNH %LR'LJLWDO LW
H[LVWLQJ DQDWRPLFDO DWODVHV %DVHG RQ WKHLU DQDO\VLV ZH SURYLGHV'PRGHOVRIWKHKXPDQERG\DQGDOORZVIRULQ
VSHFLI\WKHPDLQUHTXLUHPHQWVRQRXUDWODVDQGSURSRVHLWV GHSWKH[SORUDWLRQRIDQDWRP\DQGSK\VLRORJ\
NH\ FRPSRQHQWV $IWHU WKLV ZH SUHVHQW NH\ WHFKQRORJLHV
XVHG LQ LWV GHYHORSPHQW DQG LQWURGXFH LWV DUFKLWHFWXUH
)LQDOO\ZHSUHVHQWVFUHHQVKRWVRIWKHILUVWYHUVLRQRIWKLV
WRRO VXSSRUWLQJ HGXFDWLRQ RI %LRPHGLFDO ,QIRUPDWLFV DW
RXUIDFXOW\

,, &855(17('8&$7,21$/3/$7)2506
(GXFDWLRQDO SODWIRUPV IRU PHGLFLQH SOD\ D VLJQLILFDQW
UROHLQWKHHGXFDWLRQDQGWUDLQLQJRIIXWXUHSK\VLFLDQVDQG
QXUVHULHV 7KH\ FDQ EH XVHIXO IRU ELRPHGLFDO HQJLQHHUV
DQGELRPHGLFDOLQIRUPDWLFLDQVDVZHOO&XUUHQWO\WKHUHDUH
VHYHUDOSODWIRUPVXVHGLQWKLVVHQVH,QZKDWIROORZV ZH
GHDO ZLWK VL[ RI WKHP QDPHO\ %LR'LJLWDO &RPSOHWH
$QDWRP\ =\JRWH%RG\ $QDWRP\ /HDUQLQJ $QDWRP\ '
$WODV7HDFK0H$QDWRP\:HIRFXVRQWKHPDVZKROHDQG
WU\ WR LGHQWLI\ WKHLUDGYDQWDJHV DQGGLVDGYDQWDJHV%DVHG
RQWKHPZHJLYHDQRYHUYLHZRIZKLFKRIWKHVHSODWIRUPV 
EHVW PHHW WKH QHHGV RI LQIRUPDWLFLDQV GHDOLQJ ZLWK
DSSOLFDWLRQ RI LQIRUPDWLFV DQG VRIWZDUH HQJLQHHULQJ LQ )LJXUH &RPSOHWH$QDWRP\ KWWSVGPHGLFDOFRP 
PHGLFLQHDQGELRPHGLFLQH
2QH RI WKH DGYDQWDJHV LV LWV ZLGH UDQJH RI IHDWXUHV
$ %LR'LJLWDO 8VHUV FDQ EURZVH RSHQ ]RRP LQ DQG URWDWH RUJDQV DQG
ERG\ VWUXFWXUHV 7KH\ FDQ JHW XS FORVH WR GHWDLOV DQG
%LR'LJLWDO )LJ LVDQLQWHUDFWLYHSODWIRUPGHVLJQHG
IRU PHGLFDO HGXFDWLRQ ,W FRQWDLQV UHDOLVWLF DQG YLVXDOO\ H[SORUH LQWHUQDO VWUXFWXUHV WKH QHUYRXV V\VWHP EORRG
ULFK ' PRGHOV RI WKH KXPDQ ERG\ DOORZLQJ WR H[SORUH YHVVHOVDQGRWKHULPSRUWDQWDVSHFWV$QRWKHUDGYDQWDJHLV
WKDW LW RIIHUV VSHFLILF PRGXOHV DQG WUDLQLQJV IRU GLIIHUHQW
KXPDQDQDWRP\DQGSK\VLRORJ\
PHGLFDO GLVFLSOLQHV &RPSOHWH $QDWRP\ DOVR LQFOXGHV
LQWHUDFWLYH YLGHRV GHPRQVWUDWLQJ WKH PRYHPHQWV DQG
IXQFWLRQVRIRUJDQVDQGWKHPXVFXODUV\VWHP
&RPSDUHGWR%LR'LJLWDO&RPSOHWH$QDWRP\GRHVKDYH
VRPH GUDZEDFNV )RU H[DPSOH LW PD\ QRW EH DV YLVXDOO\
UREXVW DQG GHWDLOHG DV %LR'LJLWDO LWV ' PRGHOV PD\ EH
OHVV UHDOLVWLF DQG VRPH VWUXFWXUHV PD\ EH OHVV GHWDLOHG
$GGLWLRQDOO\ VRPH RI LWV DGYDQFHG IHDWXUHVDQG PRGXOHV
PD\UHTXLUHDSUHPLXPPHPEHUVKLSRUDQDGGLWLRQDOIHH
& =\JRWH%RG\
=\JRWH%RG\ )LJ LVDQRWKHUSODWIRUPIRFXVHGRQ'
YLVXDOL]DWLRQ RI WKH KXPDQ ERG\ DQG PHGLFDO HGXFDWLRQ
6LPLODUWR%LR'LJLWDODQG&RPSOHWH$QDWRP\LWSURYLGHV
LQWHUDFWLYHPRGHOVRIWKHKXPDQERG\
 2QHRIWKHDGYDQWDJHVRI=\JRWH%RG\LVLWVKLJKTXDOLW\
YLVXDO PRGHOV ,W RIIHUV UHDOLVWLF DQG GHWDLOHG '
)LJXUH %LR'LJLWDO KWWSVZZZELRGLJLWDOFRP 
UHSUHVHQWDWLRQV RI RUJDQV WLVVXHV DQG ERG\ VWUXFWXUHV
$QRWKHU DGYDQWDJH LV WKDW =\JRWH%RG\ RIIHUV LQWXLWLYH
%LR'LJLWDO SURYLGHV RSWLRQV IRU LQWHUDFWLQJ ZLWK WKH
FRQWUROV DQG D XVHUIULHQGO\ LQWHUIDFH ,W DOORZV IRU HDV\
PRGHOV 8VHUV FDQ PDQLSXODWH ERG\ SDUWV FKDQJH
PDQLSXODWLRQ RI PRYHPHQW ]RRPLQJ DQG URWDWLRQ RI WKH
SHUVSHFWLYHVDQGH[SORUHUHODWLRQVKLSVEHWZHHQRUJDQVDQG
PRGHOVHQKDQFLQJWKHXVHUH[SHULHQFH
V\VWHPV,WDOORZVGLVSOD\LQJPRYHPHQWVDQGIXQFWLRQLQJ
RILQGLYLGXDOERG\SDUWVZKLFKLVXVHIXOIRUVWXG\LQJWKH $ GLVDGYDQWDJH RI =\JRWH%RG\ LV LWV OLPLWHG UDQJH RI
PXVFXORVNHOHWDO V\VWHP DQG PXVFOH G\QDPLFV $QRWKHU IHDWXUHV DQG FDSDELOLWLHV &RPSDUHG WR %LR'LJLWDO DQG
DGYDQWDJH LV LWV DELOLW\ WR VLPXODWH YDULRXV GLVHDVHV DQG &RPSOHWH $QDWRP\ LW PD\ KDYH IHZHU LQWHUDFWLYH
FRQGLWLRQVRIWKHERG\ HOHPHQWV DQG IHZHU GLVHDVH VLPXODWLRQV 6RPH VSHFLILF
IHDWXUHVDQGGHWDLOVPD\EHODFNLQJ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 69

 ( $QDWRP\'$WODV
$QDWRP\ ' $WODV )LJ   LV D FRPSUHKHQVLYH PRELOH
DSSOLFDWLRQ WKDW SURYLGHV D GHWDLOHG ' UHSUHVHQWDWLRQ RI
WKH KXPDQ ERG\ IRU HGXFDWLRQDO SXUSRVHV ,W RIIHUV
LQWHUDFWLYHPRGHOVDQDWRPLFDOVWUXFWXUHVDQGLQIRUPDWLYH
FRQWHQWWRDLGLQWKHVWXG\RIDQDWRP\


)LJXUH =\JRWH%RG\ KWWSVZZZ]\JRWHERG\FRP 

:KHQFRPSDULQJ=\JRWH%RG\WRSUHYLRXVRQHVHDFKRI
WKHPKDVVRPHDGYDQWDJHVDQGGLVDGYDQWDJHV %LR'LJLWDO )LJXUH $QDWRP\'$WODV KWWSVDQDWRP\GDWODVFRP 
LVNQRZQIRULWVH[WHQVLYHLQWHUDFWLYHIHDWXUHVDQGGLVHDVH
VLPXODWLRQV ZKLOH &RPSOHWH $QDWRP\ VSHFLDOL]HV LQ 2QH DGYDQWDJH RI $QDWRP\ ' $WODV LV LWV H[WHQVLYH
YDULRXV PHGLFDO GLVFLSOLQHV DQG SURYLGHV WUDLQLQJV DQG FROOHFWLRQRIDQDWRPLFDOVWUXFWXUHV,WFRYHUVDZLGHUDQJH
PRGXOHV IRU VSHFLILFDUHDV =\JRWH%RG\ VWDQGVRXW IRULWV RI ERG\ V\VWHPV DOORZLQJ XVHUV WR H[SORUH DQG GLVVHFW
KLJKYLVXDOTXDOLW\DQGFRQYHQLHQWFRQWUROV YDULRXV RUJDQV DQG VWUXFWXUHV ZLWK JUHDW SUHFLVLRQ
$QDWRP\ ' $WODV LQFOXGHV VXSSOHPHQWDU\ HGXFDWLRQDO
' $QDWRP\/HDUQLQJ PDWHULDOVOLNHTXL]]HVDQGUHIHUHQFHUHVRXUFHV
$QDWRP\ /HDUQLQJ )LJ   LV D PRELOH DSSOLFDWLRQ ) 7HDFK0H$QDWRP\
GHVLJQHGIRUOHDUQLQJDQDWRP\,WSURYLGHVLQWHUDFWLYH'
PRGHOV RI WKH KXPDQ ERG\ DQG HGXFDWLRQDO PRGXOHV IRU 7HDFK0H$QDWRP\ )LJ LVDQRQOLQHDQGDOVRPRELOH
VWXGHQWVDQGSURIHVVLRQDOVLQWKHPHGLFDOILHOG SODWIRUP WKDW RIIHUV HGXFDWLRQDO UHVRXUFHV DQG PDWHULDOV
IRU VWXG\LQJ DQDWRP\ ,W SURYLGHV D ZLGH UDQJH RI
FRPSUHKHQVLYH FRQWHQW LQFOXGLQJ DUWLFOHV WXWRULDOV
TXL]]HVDQGLQWHUDFWLYHGLDJUDPV


)LJXUH $QDWRP\/HDUQLQJ KWWSVDQDWRP\OHDUQLQJFRP 

2QH DGYDQWDJH RI $QDWRP\ /HDUQLQJ LV LWV PRELOH
DFFHVVLELOLW\ 8VHUV FDQ DFFHVV WKH PRGHOV DQG DQDWRP\ )LJXUH 7HDFK0H$QDWRP\ KWWSVWHDFKPHDQDWRP\LQIR 
LQIRUPDWLRQDQ\WLPHDQ\ZKHUHPDNLQJLWFRQYHQLHQWDQG
IOH[LEOH $QRWKHU DGYDQWDJH LV WKDW WKH DSSOLFDWLRQ RIIHUV 2QH RI WKH DGYDQWDJHV RI 7HDFK0H$QDWRP\ LV LWV
YDULRXVLQWHUDFWLYHIHDWXUHVVXFKDVURWDWLRQ]RRPLQJDQG H[WHQVLYHDQGGHWDLOHGFRQWHQW,WFRYHUVDEURDGUDQJHRI
PDQLSXODWLRQ RI WKH PRGHOV 8VHUV FDQ LQWHUDFW ZLWK DQDWRPLFDO WRSLFV DQG GHOLYHUV LQGHSWK H[SODQDWLRQV
RUJDQV DQG ERG\ VWUXFWXUHV JDLQLQJ D GHHSHU PDNLQJLWDYDOXDEOHUHVRXUFHIRUVWXGHQWVDQGKHDOWKFDUH
XQGHUVWDQGLQJRIWKHLUIXQFWLRQDOLW\ SURIHVVLRQDOV0RUHRYHU7HDFK0H$QDWRP\RIIHUVDXVHU
IULHQGO\ LQWHUIDFH DQG ZHOORUJDQL]HG VWUXFWXUH 7KH
$ SRWHQWLDO GUDZEDFN RI $QDWRP\ /HDUQLQJ LV LWV LQIRUPDWLRQ LV SUHVHQWHG LQ D FOHDU DQG FRQFLVH PDQQHU
OLPLWHG FRQWHQW FRPSDUHG WR SODWIRUPV OLNH %LR'LJLWDO IDFLOLWDWLQJ HDV\ QDYLJDWLRQ DQG XQGHUVWDQGLQJ $QRWKHU
&RPSOHWH $QDWRP\ DQG =\JRWH%RG\ ,W PD\ KDYH OHVV DGYDQWDJH LV WKH DYDLODELOLW\ RI LQWHUDFWLYH GLDJUDPV DQG
H[WHQVLYHLQIRUPDWLRQGLVHDVHVLPXODWLRQVDQGVSHFLDOL]HG TXL]]HV ZKLFK HQDEOH XVHUV WR WHVW WKHLU NQRZOHGJH DQG
PRGXOHV &RPSDULQJ WKH SUHYLRXV RQHV $QDWRP\ HQJDJHLQDFWLYHOHDUQLQJ
/HDUQLQJVWDQGVRXWIRULWVPRELOHDFFHVVLELOLW\

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 70

 $ SRWHQWLDO GUDZEDFN LV LWV UHOLDQFH RQ WH[WEDVHG RZQ ' VFHQHV VSHFLILFDWLRQ RI RZQ FRORU
FRQWHQW LQ WKH EDVLF YHUVLRQ :KLOH WKH SODWIRUP RIIHUV VFKHPDVWH[WXUHVHWF 
ZULWWHQH[SODQDWLRQVDQGGLDJUDPVLWPD\QRWSURYLGHWKH • ULFKWH[WXDOGHVFULSWLRQVRIRUJDQV\VWHPV
VDPH OHYHO RI YLVXDO DQG LQWHUDFWLYH H[SHULHQFHV DV RWKHU • LQWHUDFWLYH DQLPDWLRQ RI VSHFLILF SK\VLRORJLFDO
'DQDWRP\DSSOLFDWLRQVRUSODWIRUPV,I\RXZDQWWRKDYH SURFHVVHV DQG PRYHPHQWV RI VSHFLILF SDUWV RI
VXFK NLQG RI H[SHULHQFH WKHQ \RX KDYH WR SD\ IRU WKH KXPDQERG\
SUHPLXPYHUVLRQ • WHVWLQJ HQYLURQPHQW DOORZLQJ WR SUDFWLFH WKH
DFTXLUHG NQRZOHGJH XVLQJ WH[WXDO EXW DOVR
* &RPSDULVRQRI([LVWLQJ3ODWIRUPV LQWHUDFWLYH'WHVWV
(GXFDWLRQDO SODWIRUPV IRU PHGLFLQH KDYH WKHLU • DFFHVVWRDOOWHDFKLQJPDWHULDOVIRUDOOVWXGHQWV
DGYDQWDJHVDQGGLVDGYDQWDJHV ZLWKRXWDQ\IHH
%LR'LJLWDORIIHUVLQWHUDFWLYHPRGHOVDQGYLUWXDOUHDOLW\ • PXOWLOLQJXDO LQWHUIDFH RI WKH DSSOLFDWLRQ WKH
EXWLWPD\ODFNGHWDLOHGWH[WXDOH[SODQDWLRQV DSSOLFDWLRQ VKRXOG VXSSRUW /DWLQ 6ORYDN DQG
&RPSOHWH$QDWRP\KDVDFRPSUHKHQVLYH'PRGHOZLWK (QJOLVKPHGLFDOWHUPLQRORJ\ 
LQWHUDFWLYHIHDWXUHVEXWVRPHRIWKHPDUHRQO\DYDLODEOHLQ • UHVWULFWHG DUHD WKDW DOORZV RUJDQL]LQJ ILQDO
SDLGYHUVLRQV H[DPVLQ PDQGDWRU\FRXUVHV WDXJKW ZLWKLQWKH
=\JRWH%RG\ SURYLGHV GHWDLOHG YLVXDOL]DWLRQV DQG %LRPHGLFDO,QIRUPDWLFVVWXG\SURJUDPRIIHUHG
E\ WKH )DFXOW\ RI 0DQDJHPHQW 6FLHQFH DQG
LQWHUDFWLYH ' PRGHOV EXW LW PD\ QRW DOZD\V KDYH
,QIRUPDWLFV
VXIILFLHQWDQDWRPLFDOFRYHUDJH
2WKHUUHTXLUHPHQWVRQWKHDSSOLFDWLRQVKRXOGLQFOXGH
$QDWRP\/HDUQLQJKDVDQH[WHQVLYHOLEUDU\RIOLWHUDWXUH
DQGH[SODQDWLRQVEXWPD\QRWLQFOXGH'PRGHOV • UREXVWQHVV ZKLFK PHDQV VLPSOH DQG LQWXLWLYH
XVHULQWHUIDFH
$QDWRP\'$WODVKDVDODUJHOLEUDU\RI'PRGHOVEXW • DFFHVVLELOLW\ IURP DQ\ZKHUH DQG LQ DQ\WLPH
PD\ODFNWH[WXDOH[SODQDWLRQV ZLWKRXWWKHQHHGWRLQVWDOODQ\WKLQJ
7HDFK0H$QDWRP\SURYLGHV VWUXFWXUHG H[SODQDWLRQVEXW • LQGHSHQGHQFH RQ WKH WDUJHW SODWIRUP LH
PD\EHOHVVLQWHUDFWLYH 0LFURVRIW :LQGRZV /LQX[ $SSOH L26
6RPH RI WKH SODWIRUPV¶ IHDWXUHV FRPSDUHG LQ WKLV $QGURLG 
VHFWLRQ DUH DOVR OLVWHG LQ 7DEOH , )URP 6ORYDN SRLQW RI %DVHGRQDOOWKHVHUHTXLUHPHQWVZHGHFLGHGWRUHDOL]H
YLHZ DQ LPSRUWDQW GUDZEDFN RI DOO WKHVH SODWIRUPV LV RXU RZQ DQDWRPLFDO DWODV DV D ZHE DSSOLFDWLRQ ZLWK
PLVVLQJ6ORYDNORFDOL]DWLRQ UHVSRQVLYHGHVLJQ>@

,,, 352326$/2)$&8672062/87,21 % 8VHG7HFKQRORJLHV

)RU PHGLFDO HGXFDWLRQ ZLWKLQ WKH VFKRRO HQYLURQPHQW 7KH FRQVLGHUHG ZHE DSSOLFDWLRQ IRU OHDUQLQJ KXPDQ
DQGEH\RQGLWZRXOGEHHIILFLHQWIRUVWXGHQWVDQGWHDFKHUV DQDWRP\ FDQ EH EXLOW XVLQJ D WKUHHWLHU FOLHQWVHUYHU
WRKDYHXQUHVWULFWHGDFFHVVWRHGXFDWLRQDOPDWHULDOV7KHVH DUFKLWHFWXUH )LJ   &RQFHUQLQJ WKH FOLHQWVLGH LW LV
PDWHULDOVLQFOXGHQRWRQO\VFULSWVEXWDOVRRQOLQHWRROVIRU LPSRUWDQW IRU WKH DSSOLFDWLRQ WR UHVSRQG DSSURSULDWHO\ WR
OHDUQLQJ ZKHUH VWXGHQWV FDQ HGXFDWH WKHPVHOYHV PRUH XVHU UHTXHVWV EH DEOH WR GLVSOD\ ' PRGHOV DQG DOORZ
HIIHFWLYHO\ XVLQJ LQWHUDFWLYHPRGHOV DQG WHVWV *LYHQ WKDW XVHUVWRLQWHUDFWZLWKWKHP)RUWKLVSXUSRVHWKH$QJXODU
QHDUO\ HYHU\RQH KDV D PRELOH SKRQH DQG LQWHUQHW DFFHVV DQG %DE\ORQMV IUDPHZRUNV FRXOG EH HPSOR\HG 7R
QRZDGD\VOHDUQLQJFDQWDNHSODFHSUDFWLFDOO\DQ\ZKHUH HIILFLHQWO\SURFHVVUHTXHVWVDQGJUDQWXVHUVDFFHVVWRGDWD
DQG PRGHOV DSSOLFDWLRQ VHUYHU RI WKH VHUYHUVLGH FRXOG
$ 6SHFLILFDWLRQRI5HTXLUHPHQWV XWLOL]H WKH 6SULQJ %RRW IUDPHZRUN 7KH GDWD VWRUDJH
%DVHG RQ WKH DQDO\VLV RI H[LVWLQJ DSSOLFDWLRQV ZH EHKLQG WKH DSSOLFDWLRQ VHUYHU FDQ EH IDFLOLWDWHG XVLQJ D
GHFLGHG WR FUHDWH RXU RZQ DQDWRPLFDO DWODV ZKLFK ZLOO GDWDEDVHV\VWHPVXFKDV3RVWJUH64/
FRQWDLQ VLPLODU IXQFWLRQV DV WKH PHQWLRQHG DSSOLFDWLRQV
  &OLHQW6LGH )URQWHQG 
+RZHYHUGXULQJGHVLJQLQJWKHVROXWLRQZHKDYHIRFXVHG
7KH FOLHQW VLGH UHSUHVHQWV IURQWHQG RI WKH DSSOLFDWLRQ
RQ WKH DUHDV DQG FRXUVHV WDXJKW LQ WKH %LRPHGLFDO
%DVHGRQWKHUHTXLUHPHQWVZHKDYHGHFLGHGWRLPSOHPHQW
,QIRUPDWLFV VWXG\ SURJUDP 7KH NH\ IHDWXUHV RI WKH
LWXVLQJWZRIUDPHZRUNVWKDWDUH$QJXODUDQG%DE\ORQMV
DSSOLFDWLRQVKRXOGEH
$QJXODU >@ LV DQ RSHQVRXUFH IURQWHQG ZHE
• ' OHDUQLQJ HQYLURQPHQW ZLWK KLJKTXDOLW\ DSSOLFDWLRQIUDPHZRUNSULPDULO\PDLQWDLQHGE\*RRJOH,W
PRGHOV RI KXPDQ RUJDQV DQG RUJDQ V\VWHPV LV ZULWWHQ LQ 7\SH6FULSW DQG SURYLGHV GHYHORSHUV ZLWK D
SDUWLDOO\ FXVWRPL]DEOH E\ XVHUV FUHDWLRQ RI SRZHUIXOVHWRIWRROVIRUEXLOGLQJG\QDPLFDQGLQWHUDFWLYH
7$%/(,
&203$5,6212)3/$7)2506)257($&+,1*+80$1$1$720<
)HDWXUHV
$SSOLFDWLRQ3ODWIRUP
'PRGHOV $QLPDWLRQV 0RGHOV¶TXDOLW\ 2QOLQH )UHHOLFHQVH 7HVWV4XL]]HV
%LR'LJLWDO <HV <HV +LJK <HV <HV 1R

&RPSOHWH$QDWRP\ <HV <HV +LJK <HV 1R <HV

=\JRWH%RG\ <HV <HV +LJK <HV 1R 1R

$QDWRP\/HDUQLQJ <HV 1R 0HGLXP <HV <HV <HV

$QDWRP\'$WODV <HV 1R 0HGLXP <HV 1R 1R

7HDFK0H$QDWRP\ <HV 1R ± <HV <HV <HV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 71

ZHE DSSOLFDWLRQV $QJXODU IROORZV WKH FRPSRQHQWEDVHG IUDPHZRUNIRUWKHGHYHORSPHQWRIWKHEDFNHQGORJLFDQG
DUFKLWHFWXUH ZKHUH WKH DSSOLFDWLRQ LV FRPSRVHG RI 3RVWJUH64/GDWDEDVHPDQDJHPHQWV\VWHPIRUVWRULQJDQG
UHXVDEOH DQG PRGXODU FRPSRQHQWV 7KH EDVLF SDUWV WKDW PDQDJLQJ DOO SHUVLVWHQW GDWD LQFOXGLQJ ' PRGHOV RI
PDNHXSDFRPSRQHQWLQ$QJXODUDUHWHPSODWHFODVVDQG KXPDQ DQDWRP\ ZLWK WKHLU GHVFULSWLRQV PHGLFDO
VW\OHVKHHW6XFKDGLYLVLRQRILQGLYLGXDOSDUWVDOORZVIRU WHUPLQRORJ\WHVWVIRUXVHUVDQGLQIRUPDWLRQDERXWXVHUV
WKHVHSDUDWLRQRIORJLFIURPGHVLJQZKLFKPDNHVWKHFRGH 6SULQJ %RRW >@ LV DQ RSHQVRXUFH -DYDEDVHG
PRUH WUDQVSDUHQW DQG DOORZV UHXVLQJ LQGLYLGXDO SDUWV LQ IUDPHZRUNWKDWVLPSOLILHVWKHGHYHORSPHQWRIVWDQGDORQH
GLIIHUHQWSODFHV SURGXFWLRQUHDG\ZHEDSSOLFDWLRQV,WLVSDUWRIWKHODUJHU
6SULQJ HFRV\VWHP DQG LV PDLQWDLQHG E\ 90ZDUH ,QF
6SULQJ %RRW LV EXLOW RQ WKH 0RGHO9LHZ&RQWUROOHU
09&  DUFKLWHFWXUH >@ ZKLFK GLYLGHV WKH DSSOLFDWLRQ
LQWRWKUHHSDUWV )LJ 


)LJXUH 7KUHHWLHUFOLHQWVHUYHUDUFKLWHFWXUH

%DE\ORQMV>@LVDQRSHQVRXUFH'UHQGHULQJHQJLQH )LJXUH 6FKHPHRI09&DUFKLWHFWXUH
DQGJDPHGHYHORSPHQWIUDPHZRUNZULWWHQLQ7\SH6FULSW
,W DOORZV GHYHORSHUV WR FUHDWH LPPHUVLYH DQG LQWHUDFWLYH 7KH ILUVW SDUW LV PRGHO 7KH PRGHO UHSUHVHQWV WKH
' DSSOLFDWLRQV WKDW FDQ UXQ GLUHFWO\ LQ ZHE EURZVHUV DSSOLFDWLRQ V GDWD DQGEXVLQHVV ORJLF,Q 6SULQJ %RRW WKH
ZLWKRXW WKH QHHG IRU DGGLWLRQDO SOXJLQV RU LQVWDOODWLRQV PRGHOW\SLFDOO\FRQVLVWVRI3ODLQ2OG-DYD2EMHFWV 32-2 
.H\ IHDWXUHV RI %DE\ORQMV ZKLFK DUH QHFHVVDU\ IRU RXU RUHQWLWLHV WKDW KROG WKHGDWDDQG VWDWHRI WKHDSSOLFDWLRQ
RQOLQHDQDWRPLFDODWODVLQFOXGH 7KHVHREMHFWVPD\EHVWRUHGLQDGDWDEDVHRUPDQDJHGE\
• SRZHUIXO UHQGHULQJ ± LW SURYLGHV D KLJK RWKHUVHUYLFHV
SHUIRUPDQFH UHQGHULQJ HQJLQH FDSDEOH RI 7KHVHFRQGSDUWLVYLHZ,WLVUHVSRQVLEOHIRUSUHVHQWLQJ
KDQGOLQJ FRPSOH[ VFHQHV ZLWK OLJKWLQJ WKHGDWDWRWKHXVHU,QWKHFRQWH[WRI6SULQJ%RRWYLHZV
VKDGRZVDQGVSHFLDOHIIHFWV DUH XVXDOO\ UHSUHVHQWHG E\ +70/ WHPSODWHV VXFK DV
• :HE*/EDVHG ± LW OHYHUDJHV :HE*/ 7K\PHOHDI )UHHPDUNHU RU -63  WKDW JHQHUDWH WKH XVHU
WHFKQRORJ\WRWDNHDGYDQWDJHRIFDSDELOLWLHVRI LQWHUIDFHDQGGLVSOD\GDWDUHFHLYHGIURPWKHPRGHO
JUDSKLFSURFHVVLQJXQLWVHQVXULQJVPRRWKDQG )LQDOO\ WKH WKLUG SDUW LV FRQWUROOHU 7KH FRQWUROOHU DFWV
HIILFLHQWUHQGHULQJ DV DQ LQWHUPHGLDU\ EHWZHHQ WKH PRGHO DQG WKH YLHZ ,Q
• FURVVSODWIRUP ± DSSOLFDWLRQV FUHDWHG XVLQJ 6SULQJ %RRW FRQWUROOHUV DUH LPSOHPHQWHG DV -DYD FODVVHV
%DE\ORQMV FDQ UXQ RQ YDULRXV GHYLFHV DQG WKDW KDQGOH LQFRPLQJ UHTXHVWV IURP WKH XVHU DQG SURFHVV
SODWIRUPV LQFOXGLQJ GHVNWRSV VPDUWSKRQHV WKHPDFFRUGLQJO\7KH\LQWHUDFWZLWKWKHPRGHOWRUHWULHYH
DQGGHYLFHVIRUYLUWXDODQGDXJPHQWHGUHDOLW\ GDWD RU XSGDWH LW DV QHHGHG ZKLFK WKHQ SDVV WR WKH
• DQLPDWLRQ DQG LQWHUDFWLYLW\ ± GHYHORSHUV FDQ DSSURSULDWHYLHZWREHUHQGHUHG
DQLPDWHREMHFWVDQGDGGLQWHUDFWLYLW\WRVFHQHV
2YHUDOO 6SULQJ %RRW LV ZLGHO\ XVHG IRU GHYHORSLQJ
XVLQJ NH\IUDPH DQLPDWLRQV VNHOHWRQV DQG
UREXVW DQG VFDODEOH -DYD DSSOLFDWLRQV ZLWK D IRFXV RQ
SDUWLFOHV\VWHPV
SURGXFWLYLW\ DQG GHYHORSHUIULHQGO\ IHDWXUHV 7KDQNV WR
• FXVWRPVKDGHUVIRUVSHFLDOL]HGYLVXDOHIIHFWV WKHVHSDUWVWKHDSSOLFDWLRQLVPRUHHDVLO\H[WHQGDEOHDQG
• DVVHWORDGLQJ±LWSURYLGHVWRROVIRUORDGLQJ' PRUHRUJDQL]HG,QWKHDQDWRPLFDODWODVZHXVHLWIRUWKH
PRGHOVWH[WXUHVDQGRWKHUDVVHWVHIILFLHQWO\ LPSOHPHQWDWLRQ RI WKH PRVW EDFNHQG ORJLF LQFOXGLQJ
• VFHQH PDQDJHPHQW ± LW RIIHUV D UREXVW VFHQH PDQDJHPHQWRI'PRGHOV'VFHQHV'DQLPDWLRQVDQG
JUDSK DQG FDPHUD FRQWUROV IRU PDQDJLQJ DOOGDWDUHODWHGWRKXPDQDQDWRP\DQGWHVWVIRUXVHUV
FRPSOH[VFHQHVDQGFDPHUDPRYHPHQW
3RVWJUH64/ >@ LV D SRZHUIXO RSHQVRXUFH UHODWLRQDO
7R FRQFOXGH %DE\ORQMV LV D SRSXODU FKRLFH IRU JDPH
GDWDEDVHPDQDJHPHQWV\VWHP,WZDVILUVWGHYHORSHGDWWKH
GHYHORSPHQW DUFKLWHFWXUDO YLVXDOL]DWLRQ VKRZURRPV DQG
8QLYHUVLW\ RI &DOLIRUQLD %HUNHOH\ LQ  DQG KDV VLQFH
RWKHU LQWHUDFWLYH ' ZHE H[SHULHQFHV GXH WR LWV
JURZQLQWRRQHRIWKHPRVWSRSXODUDQGDGYDQFHGGDWDEDVH
SHUIRUPDQFH HDVH RI XVH DQG H[WHQVLYH FDSDELOLWLHV
V\VWHPVLQWKHZRUOG'XHWRLWVYHUVDWLOLW\DQGIHDWXUHULFK
7KHUHIRUHZHKDYHVHOHFWHGLWDVWKHPDLQ'HQJLQHWKDW
QDWXUH QRZDGD\V LW LV ZLGHO\ XVHG LQ YDULRXV LQGXVWULHV
DOORZV FUHDWLQJ LQWHUDFWLYH VFHQHV IRU YLVXDOL]DWLRQ RI
DQG DSSOLFDWLRQV LQFOXGLQJ ZHE GHYHORSPHQW HQWHUSULVH
KXPDQ RUJDQV DQG RUJDQ V\VWHPV DQG IRU DQLPDWLRQ RI
VROXWLRQV VFLHQWLILF UHVHDUFK DQG ELJ GDWD DQDO\VLV ,WV
VSHFLILFSURFHVVHV
FRPELQDWLRQ RI SHUIRUPDQFH IOH[LELOLW\ DQG UHOLDELOLW\
  6HUYHU6LGH %DFNHQG  PDNHVLWDSRSXODUFKRLFHIRUERWKVPDOOVFDOHSURMHFWVDQG
7KH VHUYHU VLGH DJUHHV ZLWK WKH EDFNHQG RI WKH RQOLQH ODUJHHQWHUSULVHDSSOLFDWLRQV
DQDWRPLFDODWODV)RUWKHGHYHORSPHQWRILWVNH\SDUWVWZR .H\IHDWXUHVDQGFKDUDFWHULVWLFVRI3RVWJUH64/ZKLFK
WHFKQRORJLHV KDYH EHHQ VHOHFWHG 7KHVH DUH 6SULQJ %RRW DUHLPSRUWDQWIRURXUSURMHFWLQFOXGH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 72

 • UHODWLRQDO PRGHO ± 3RVWJUH64/ IROORZV WKH IUDPHZRUN ,Q WKLV ZD\ WKH XVHU UHFHLYHV PDQDJHV DQG
UHODWLRQDO PRGHO DOORZLQJ XVHUV WR GHILQH DQG VHQGVLQIRUPDWLRQWKURXJKWKHIURQWHQG FOLHQWVLGH WRWKH
PDQDJH VWUXFWXUHGGDWDXVLQJWDEOHVURZVDQG EDFNHQG VHUYHU VLGH  RI WKH DSSOLFDWLRQ LPSOHPHQWHG
FROXPQV 3RVWJUH64/ LV DOVR FRPSOLDQW ZLWK XVLQJ 6SULQJ %RRW 7KLV H[FKDQJH RI LQIRUPDWLRQ LV
64/VWDQGDUGV UHDOL]HGXQGHUWKHK\SHUWH[WWUDQVIHUSURWRFROSURYLGHGE\
• FURVVSODWIRUPFRPSDWLELOLW\±LWLVGHVLJQHGWR $QJXODU¶V +WWS&OLHQW VHUYLFH FODVV 7KLV FOLHQW VHUYLFH
UXQ RQ YDULRXV RSHUDWLQJ V\VWHPV LQFOXGLQJ VHQGVUHTXHVWVWKDWDUHLQWHUFHSWHGE\WKH6SULQJ%RRW5HVW
/LQX[ :LQGRZV PDF26 DQG 8QL[OLNH &RQWUROOHU $FFRUGLQJ WR WKH VSHFLILHG HQGSRLQW ZKLFK
V\VWHPV SURYLGLQJ IOH[LELOLW\ DQG HDVH RI LQGLFDWHV D SUHGHILQHG UHTXLUHG H[FKDQJH RI LQIRUPDWLRQ
GHSOR\PHQW WKHGDWDLVUHFHLYHGDQGSURFHVVHGRUDVSHFLDOVHUYLFHLV
• DGYDQFHGGDWDW\SHV±LWVXSSRUWVDZLGHUDQJH SHUIRUPHG 6XEVHTXHQWO\ WKH EDFNHQG VHQGV D UHVSRQVH
RI GDWD W\SHV LQFOXGLQJ QXPHULF VWULQJ EDFNWRWKHIURQWHQGZKHUHWKHFKDQJHVPDGHE\WKHXVHU
GDWHWLPH-621 -DYD6FULSW 2EMHFW1RWDWLRQ  DUHGLVSOD\HG
;0/ H;WHQVLEOH 0DUNXS /DQJXDJH  DUUD\V :LWKLQ $QJXODU WKH %DE\ORQMV IUDPHZRUN LV
%/2% %LQDU\/DUJH2%MHFW DQGXVHUGHILQHG LQFRUSRUDWHG 7KLV IUDPHZRUN YLVXDOL]HV ' REMHFWV LQWR
W\SHV VFHQHVDQGUHQGHUWKHZKROHVFHQHV7KHVH'REMHFWVDUH
• H[WHQVLELOLW\ ± 3RVWJUH64/¶V IXQFWLRQDOLW\ FDQ VWRUHG LQ WKH 3RVWJUH64/ UHODWLRQDO GDWDEDVH 7KH -'%&
EH H[WHQGHG E\ FUHDWLQJ FXVWRP GDWD W\SHV -DYD 'DWD%DVH &RQQHFWLYLW\  LQWHUIDFH WDNHV FDUH RI WKH
RSHUDWRUV DJJUHJDWH IXQFWLRQV DQG SURFHGXUDO LQWHJUDWLRQ EHWZHHQ WKH GDWDEDVH DQG RXU -DYD VHUYLFHV
ODQJXDJHV LPSOHPHQWHG LQ 6SULQJ %RRW :H FRQWLQXH WR XVH WKH
• UREXVWQHVV DQG VWDELOLW\ ± LW LV NQRZQ IRU LWV +LEHUQDWHLQWHUIDFHZKLFKSHUIRUPVWKHPDSSLQJRI-DYD
VWDELOLW\UHOLDELOLW\DQGDELOLW\WRKDQGOHKLJK REMHFWVWRWDEOHVDQGPDQDJHV64/FRPPDQGVVHQWWRRXU
WUDIILFDSSOLFDWLRQVDQGODUJHGDWDEDVHV UHODWLRQDOGDWDEDVH
• WULJJHUV DQG VWRUHG SURFHGXUHV ± LW DOORZV
GHYHORSHUV WR GHILQH WULJJHUV DQG ZULWH VWRUHG ,935(6(17$7,212)7+(&855(1762/87,21
SURFHGXUHV XVLQJ YDULRXV SURJUDPPLQJ %DVHG RQ WKH UHTXLUHPHQWV VSHFLILHG LQ VHFWLRQ ,,,$
ODQJXDJHVOLNH3/SJ64/3/3\WKRQ3/3HUO WKHDSSOLFDWLRQFDQEHVSOLWLQWRVHYHUDOORJLFSDUWVFDOOHG
7KDQNV WR DOO WKHVH IHDWXUHV ZH XVH 3RVWJUH64/ WR PRGXOHV7KHPRVWFUXFLDOSDUWRIWKHFUHDWHGDSSOLFDWLRQ
PDQDJHDOOSHUVLVWHQWGDWDXVHGE\WKHEDFNHQGGHYHORSHG LV FDOOHG WKH &RUH PRGXOH 7KLV PRGXOH UHSUHVHQWV WKH
LQWKH6SULQJ%RRW:HDOVRXVHLWIRUVWRULQJ'REMHFWVLQ FRUH RI WKH RQOLQH DQDWRP\ DWODV HQDEOLQJ WKH
WKHIRUPRI%/2%GDWD YLVXDOL]DWLRQ RI ' PRGHOV ZLWKLQ ZHE SDJHV 7KH &RUH
PRGXOH LQWHJUDWHV WKH %DE\ORQMV IUDPHZRUN UHVSRQVLEOH
& $UFKLWHFWXUHRIWKH6ROXWLRQ IRU UHQGHULQJ VSHFLILF PRGHOV LQWR WKH VFHQH ,W DOVR
7KHSUHYLRXVO\GHVFULEHGWHFKQRORJLHVUHSUHVHQWWKHNH\ LPSOHPHQWV PDQLSXODWLRQ RI WKHVH REMHFWV URWDWLRQ
HOHPHQWV RQ ZKLFK WKH DUFKLWHFWXUH RI WKH VROXWLRQ KDV ]RRPLQJ SDQQLQJ  DQG VFHQH PDQDJHPHQW $OO WKHVH
EHHQSURSRVHG7KLVDUFKLWHFWXUHLVGHSLFWHGLQ)LJDQG IHDWXUHVDUHXVHGE\WKLVPRGXOHZLWKLQWKHDGPLQLVWUDWLRQ
LVEDVHGRQWXWRULDO>@ RI'PRGHOV )LJ EXWDOVRE\RWKHUPRGXOHVRIWKH
DSSOLFDWLRQ


)LJXUH$GPLQLVWUDWLRQRI'PRGHOVZLWKLQWKH&RUHPRGXOH

$QRWKHU HVVHQWLDO PRGXOH LV WKH +XPDQ %RG\ $WODV

:LWKLQWKLVPRGXOHVFHQHVZLWK'PRGHOVRILQGLYLGXDO
KXPDQV\VWHPVDUHDYDLODEOH7KHVHVFHQHVVKRXOGLQFOXGH
VNHOHWDO V\VWHP PXVFXODU V\VWHP QHUYRXV V\VWHP DQG
RWKHUV ,Q WKLV PRGXOH WKH XVHU FDQ FKRRVH D VSHFLILF
RUJDQV\VWHPWKDWLVGLVSOD\HGLQWKH'VFHQH8VLQJWKH
 IXQFWLRQDOLWLHVRIWKH&RUHPRGXOHLWLVSRVVLEOHWRURWDWH
)LJXUH $UFKLWHFWXUHRIWKHVROXWLRQ ]RRPLQDQGRXWWKH'PRGHOZLWKLQWKHVFHQHDQGDIWHU
VHOHFWLQJ D VSHFLILF SDUW LWV 6ORYDN DQG /DWLQ QDPH DUH
&RPPXQLFDWLRQ EHWZHHQ WKH XVHU DQG WKH ZHE GLVSOD\HG WRJHWKHU ZLWK LWV GHVFULSWLRQ )LJ  
DSSOLFDWLRQWDNHVSODFHZLWKWKHKHOSRIWKHXVHULQWHUIDFH &XUUHQWO\WKLVPRGXOHVXSSRUWZRUNZLWKVNHOHWDOV\VWHP
IURQWHQG  ZKLFK LV PDGH E\ WKH $QJXODU DSSOLFDWLRQ DQGZHSUHSDUHGDWDIRURWKHURUJDQV\VWHPV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 73

  
)LJXUH0RGXOHRI+XPDQ%RG\$WODV )LJXUH0RGXOHRI,QWHUDFWLYH4XHVWLRQV

$VSHFLILFSDUWRIWKH+XPDQ%RG\$WODVLVDSRVVLELOLW\ 2WKHU PRGXOHV ZKLFK DUH FXUUHQWO\ LQ WKH SUHOLPLQDU\

WRYLVXDOL]HLQWHUDFWLYH'DQLPDWLRQVZLWKLQZKLFKDXVHU VWDJHV RI WKH GHYHORSPHQW LQFOXGH 8VHU 0DQDJHPHQW
FDQVHOHFWVSHFLILFSDUWVRIWKHDQLPDWLRQWKDWWKH\ZDQWWR PRGXOH DQG 9LUWXDO 5HDOLW\ PRGXOH 7KH IRUPHU ZLOO EH
VHH)RUH[DPSOHLQFDVHRIH[SODQDWLRQKRZ PXVFOHVRI UHVSRQVLEOH IRU PDQDJHPHQW RI XVHUV DQG UROHV LQ WKH
WKHKHDGDUHXVHGGXULQJH[SUHVVLQJKXPDQHPRWLRQVRQH DSSOLFDWLRQ 7KH ODWWHU VKRXOG EH D VSHFLDO YHUVLRQ RI WKH
FDQ VHOHFW PXVFOHV DQG PRYHPHQWV WKDW VKRXOG EH +XPDQ%RG\$WODV,WZLOOKDYHVLPLODUIXQFWLRQDOLWLHVDVLW
GLVSOD\HG )LJ 7KHUHIRUHDQRWKHUPRGXOHLQFOXGHGLQ EXW LQ WKH YLUWXDO HQYLURQPHQW 7KLV PRGXOH ZLOO EH
WKH DSSOLFDWLRQ LV $QLPDWLRQ PRGXOH 7KLV PRGXOH DJDLQ LQGHSHQGHQW IURP WKH UHVW RI WKH DSSOLFDWLRQ $FWXDOO\ LW
XVHV IXQFWLRQDOLWLHV RI WKH &RUH PRGXOH DQG DOORZV ZLOOVKDUHRQO\GDWDEDVHRI'PRGHOVZLWKWKHUHVWRIWKH
PDQDJLQJDQGSOD\LQJ'DQLPDWLRQVLQLQWHUDFWLYHZD\ DSSOLFDWLRQ EXW LWV DUFKLWHFWXUH ZLOO EH FRPSOHWHO\
GLIIHUHQWIURPRQHGHSLFWHGLQ)LJ&XUUHQWO\ZHKDYH
WKHILUVWSURRIRIFRQFHSWRIWKLVPRGXOH )LJ FUHDWHG
XVLQJ8QLW\'>@IRU2FXOXV4XHVWEXWLWUHTXLUHVD
ORWRIIXUWKHUZRUNIRUSUDFWLFDOXVHE\VWXGHQWV


)LJXUH$QLPDWLRQPRGXOH

$QRWKHU LPSRUWDQW SDUW RI WKH DSSOLFDWLRQ LV WHVWLQJ

HQYLURQPHQW ZKLFK LV FXUUHQWO\ XQGHU GHYHORSPHQW ,W 
VKRXOGDOORZFUHDWLQJFODVVLFDOWHVWVDVZHOODVLQWHUDFWLYH )LJXUH+XPDQ%RG\$WODVLQ9LUWXDO5HDOLW\
WHVWVEDVHGRQLGHQWLILFDWLRQRIVSHFLILFRUJDQVRUHOHPHQWV
RI KXPDQ DQDWRP\ LQ D GHSLFWHG ' PRGHO )RU WKLV 5HJDUGLQJWKHPRGXOHVSUHVHQWHGLQWKLVVHFWLRQH[FHSW
SXUSRVHWKLVHQYLURQPHQWLVVSOLWLQWRWZRSDUWV7KHILUVW WKHODVWWZRDOOPRGXOHVDUHLPSOHPHQWHGZLWKUHVSHFWWR
RQH QDPHG DV ([DP PRGXOH LV UHVSRQVLEOH IRU FODVVLFDO WKH DUFKLWHFWXUH GHSLFWHG LQ )LJ  7KH GHSHQGHQF\
WHVWVZKHUHVWXGHQWVKDYHTXHVWLRQVRIYDULRXVW\SHVVXFK EHWZHHQWKHVHPRGXOHVLVVKRZQLQ)LJ&XUUHQWO\DOO
DV RQH FKRLFH PXOWLSOH FKRLFHV RU RSHQ TXHVWLRQV )LJ WKHVH PRGXOHV DUH LQ WKH ODWHU VWDJH RI WKH GHYHORSPHQW
 7KHVHFRQGRQHVXSSRUWVDGPLQLVWUDWLRQRILQWHUDFWLYH DQG ZH PDLQO\ IRFXV RQ WKHLU LQWHJUDWLRQ DQG GDWD
TXHVWLRQV VR ZH QDPHG LW DV D PRGXOH RI ,QWHUDFWLYH SUHSDUDWLRQIRUWKHP7KHIDFWXDOGDWDLVFUHDWHGZLWKWKH
4XHVWLRQV $JDLQ WKLV PRGXOH FRPPXQLFDWHV ZLWK WKH KHOSRIDQDWRPLFDODWODVHVDQGSURIHVVLRQDOOLWHUDWXUHVXFK
&RUH PRGXOH DQG XVHV LWV IXQFWLRQDOLWLHV WR FUHDWH DQG DV>@DQGZLOOEHUHYLHZHGE\SK\VLFLDQV5HJDUGLQJ'
GLVSOD\LQWHUDFWLYHTXHVWLRQV )LJ  PRGHOV DQG DQLPDWLRQV WKH\ DUH SUHSDUHG XVLQJ %OHQGHU
>@:HXVHLWWRHGLWDQGFRUUHFWWKHGDWDDYDLODEOHWRXV

9&21&/86,21
%LRPHGLFDODQGPHGLFDOLQIRUPDWLFVEHORQJVWRUDSLGO\
JURZLQJLQGXVWU\7KLVLQGXVWU\FDOOVIRUHGXFDWLRQRI,&7
H[SHUWVZKRKDYHSHUIHFWNQRZOHGJHDQGVNLOOVRIVRIWZDUH
HQJLQHHULQJ SURJUDPPLQJ DQG GDWD SURFHVVLQJ DQG DUH
DEOHWRFRPPXQLFDWHZLWKSHRSOHHGXFDWHGDQGWUDLQHGLQ
PHGLFLQHRUELRPHGLFLQH)RUWKLVSXUSRVHWKH)DFXOW\RI
0DQDJHPHQW6FLHQFHDQG,QIRUPDWLFVRIWKH8QLYHUVLW\RI
 =LOLQD RSHQHG QHZ PDVWHU VWXG\ SURJUDP FDOOHG
)LJXUH3URSRVDORIWKH([DPPRGXOH %LRPHGLFDO ,QIRUPDWLFV LQ  6WXGHQWV RI WKLV VWXG\
SURJUDP KDYH FRPH ZLWK WKH LGHD WR GHYHORS RZQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 74

 DQDWRPLFDO DWODV WKDW VKRXOG KDYH D FKDUDFWHU RI RQOLQH >@ 05'HVVHOOH5$%URZQ$5-DPHV0-0LGZLQWHU6.
DSSOLFDWLRQDQGVKRXOGEHXVHGLQWHDFKLQJFRXUVHVUHODWHG 3RZHOO DQG 0 $ :RRGUXII ³$XJPHQWHG DQG YLUWXDO UHDOLW\ LQ
VXUJHU\´&RPSXWLQJLQ6FLHQFH (QJLQHHULQJYROQRSS
WRPHGLFDOWHUPLQRORJ\KXPDQDQDWRP\DQGSK\VLRORJ\ ±0D\
>@ ( 7 : 7DQ - 0 /LQJ DQG 6 . 'LQHVK ³7KH IHDVLELOLW\ RI
SURGXFLQJ SDWLHQWVSHFLILF DFU\OLF FUDQLRSODVW\ LPSODQWV ZLWK D
ORZFRVW'SULQWHU´-RXUQDORI1HXURVXUJHU\YROQRSS
±0D\
>@ - .RSUQLFNê 3 1DMPDQ DQG - âDIND ³' SULQWHG ELRQLF
SURVWKHWLF KDQGV´ LQ  ,((( ,QWHUQDWLRQDO :RUNVKRS RI
(OHFWURQLFV&RQWURO0HDVXUHPHQW6LJQDOVDQGWKHLU$SSOLFDWLRQ
WR0HFKDWURQLFV (&060 SS±
>@ 0 .UH\ ³:HDUDEOH WHFKQRORJ\ LQKHDOWK FDUH ± $FFHSWDQFH DQG
WHFKQLFDOUHTXLUHPHQWVIRUPHGLFDOLQIRUPDWLRQV\VWHPV´LQ
WK,QWHUQDWLRQDO&RQIHUHQFHRQ,QIRUPDWLRQ0DQDJHPHQW ,&,0 
SS±
>@ 0+DJKLDQG70'HVHUQR³*HQHUDOFRQFHSWXDOIUDPHZRUNRI
IXWXUH ZHDUDEOHV LQ KHDOWKFDUH 8QLILHG 8QLTXH 8ELTXLWRXV DQG
8QREWUXVLYH 8  IRU FXVWRPL]HG TXDQWLILHG RXWSXW´
&KHPRVHQVRUVYROQR
>@ 0 &KRFKOLN 3 0DUWLQFRYD DQG - .RVWROQ\ ³6HDUFKLQJ IRU
KLGGHQSDWWHUQVLQDODUJHJHQHWLFGDWDEDVH´LQ,QWHUQDWLRQDO
&RQIHUHQFH RQ ,QIRUPDWLRQ DQG 'LJLWDO 7HFKQRORJLHV  SS
 ±
)LJXUH0RGXOHVRIWKHDSSOLFDWLRQ >@ 2 &KRYDQFRYD $ 6WDIXULNRYD ' 0DFHNRYD 7 .LVNRYD -
5DEFDQDQG-.RVWROQ\³,PSDFWRIPHWDERORPLFVRQGHSUHVVLRQ
XVLQJ GDWD PLQLQJ WHFKQLTXHV´ LQ  WK ,((( ,QWHUQDWLRQDO
,QWKLVSDSHUZHVXPPDUL]HGNH\UHTXLUHPHQWVRQWKLV &RQIHUHQFH RQ ,QWHOOLJHQW 'DWD $FTXLVLWLRQ DQG $GYDQFHG
RQOLQHDQDWRPLFDODWODVDQGEURXJKWFORVHULWVDUFKLWHFWXUH &RPSXWLQJ6\VWHPV7HFKQRORJ\DQG$SSOLFDWLRQV ,'$$&6 6HS
WRJHWKHU ZLWK WKH XVHG WHFKQRORJLHV :H DOVR H[SODLQHG YROSS±
KRZ WKLV DWODV KDG EHHQ GHYHORSHG E\ LWV VHSDUDWLRQ LQWR >@ $ )HUUHLUD : & &HOHVWH ) $ &KHHLQ 7 ) %DVWRV)LOKR 0
PRUH RU OHVV LQGHSHQGHQW PRGXOHV 7KH NH\ PRGXOHV DUH 6DUFLQHOOL)LOKRDQG5&DUHOOL³+XPDQPDFKLQHLQWHUIDFHVEDVHG
RQ (0* DQG ((* DSSOLHG WR URERWLF V\VWHPV´ -RXUQDO RI
FXUUHQWO\ LQ WKH ILQDO VWDJH RI WKH GHYHORSPHQW DQG ZH 1HXUR(QJLQHHULQJDQG5HKDELOLWDWLRQYROQRS
ZRUNRQWKHLULQWHJUDWLRQDQGGDWDSUHSDUDWLRQIRUWKHP >@ / -LDQJ HW DO ³6PDUW5ROOLQJ $ KXPDQ±PDFKLQH LQWHUIDFH IRU
ZKHHOFKDLU FRQWURO XVLQJ ((* DQG VPDUW VHQVLQJ WHFKQLTXHV´
$&.12:/('*0(17 ,QIRUPDWLRQ3URFHVVLQJ 0DQDJHPHQWYROQRS

7KLV ZRUN ZDV VXSSRUWHG E\ WKH *UDQW 6\VWHP RI >@ $ )DXONQHU-RQHV HW DO ³%LRSULQWLQJ RI KXPDQ SOXULSRWHQW VWHP
8QLYHUVLW\RI=LOLQD1R7KLVZRUNZDVDOVR FHOOVDQGWKHLUGLUHFWHGGLIIHUHQWLDWLRQLQWRKHSDWRF\WHOLNHFHOOVIRU
VXSSRUWHG E\ SURMHFW 1R (336. WKHJHQHUDWLRQRIPLQLOLYHUVLQ'´%LRIDEULFDWLRQYROQR
S2FW
(33.$.$ ³8QLYHUVLW\,QGXVWU\ (GXFDWLRQDO &HQWUH LQ
>@ 1 3DQMD 6 0DML 6 &KRXGKXUL . $ $OL DQG & 0 +RVVDLQ
$GYDQFHG %LRPHGLFDO DQG 0HGLFDO ,QIRUPDWLFV´ FR ³' ELRSULQWLQJ RI KXPDQ KROORZ RUJDQV´ $$36 3KDUP6FL7HFK
IXQGHG E\ WKH (UDVPXV 3URJUDPPH RI WKH (XURSHDQ YROQRS0D\
8QLRQ DQG E\ WKH 0LQLVWU\ RI (GXFDWLRQ 6FLHQFH >@ : 6LQJHU ³'LIIHUHQFHV EHWZHHQ QDWXUDO DQG DUWLILFLDO FRJQLWLYH
5HVHDUFKDQG6SRUWRIWKH6ORYDN5HSXEOLFXQGHUWKHJUDQW V\VWHPV´ LQ 5RERWLFV $, DQG +XPDQLW\ &KDP &+ 6SULQJHU
9(*$ ,QWHUQDWLRQDO3XEOLVKLQJSS±
>@ 3RUWDO96VN ³%LRPHGLFDO ,QIRUPDWLFV´ 
KWWSVZZZSRUWDOYVVNHQVWXGLMQ\SURJUDPELRPHGLFLQVND
5()(5(1&(6 LQIRUPDWLND DFFHVVHG6HS 
>@ :+2 ³*OREDO 2EVHUYDWRU\ IRU H+HDOWK´  >@ (0DUFRWWH5HVSRQVLYH:HE'HVLJQ6HFRQG(GLWLRQ1HZ<RUN
KWWSVZZZZKRLQWREVHUYDWRULHVJOREDOREVHUYDWRU\IRUHKHDOWK 1<86$$%RRN$SDUW
DFFHVVHG6HS  >@ $QJXODU³:KDWLV$QJXODU"´KWWSVDQJXODULRJXLGHZKDW
>@ 9/HYDVKHQNRDQG(=DLWVHYD³)X]]\GHFLVLRQWUHHVLQPHGLFDO LVDQJXODU DFFHVVHG6HS 
GHFLVLRQ PDNLQJ VXSSRUW V\VWHP´ LQ  )HGHUDWHG&RQIHUHQFH >@ %DE\ORQMV ³(QJLQH 6SHFLILFDWLRQV´ 
RQ &RPSXWHU 6FLHQFH DQG ,QIRUPDWLRQ 6\VWHPV )HG&6,6   KWWSVZZZEDE\ORQMVFRPVSHFLILFDWLRQV DFFHVVHG 6HS 
SS±  
>@ 0 : / 0RUHLUD - - 3 & 5RGULJXHV 9 .RURWDHY - $O >@ 90ZDUH ³6SULQJ %RRW´  KWWSVVSULQJLRSURMHFWVVSULQJ
0XKWDGL DQG 1 .XPDU ³$ FRPSUHKHQVLYH UHYLHZ RQ VPDUW ERRW DFFHVVHG6HS 
GHFLVLRQVXSSRUWV\VWHPVIRUKHDOWKFDUH´,(((6\VWHPV-RXUQDO
YROQRSS± >@ 0 )RZOHU ' 5LFH 0 )RHPPHO ( +LHDWW 5 0HH DQG 5
6WDIIRUG3DWWHUQVRI(QWHUSULVH$SSOLFDWLRQ$UFKLWHFWXUH%RVWRQ
>@ 6 / 'DZVRQ DQG - $ .DXIPDQ ³7KH LPSHUDWLYH IRU PHGLFDO 0$86$$GGLVRQ:HVOH\
VLPXODWLRQ´3URFHHGLQJVRIWKH,(((YROQRSS±
>@ 7KH 3RVWJUH64/ *OREDO 'HYHORSPHQW *URXS ³3RVWJUH64/ ±
0DU
$ERXW´  KWWSVZZZSRVWJUHVTORUJDERXW DFFHVVHG 6HS
>@ & ' &RPEV ³0HGLFDO VLPXODWRUV &XUUHQW VWDWXV DQG IXWXUH  
QHHGV´LQWK,(((,QWHUQDWLRQDO:RUNVKRSVRQ(QDEOLQJ
7HFKQRORJLHV,QIUDVWUXFWXUHVIRU&ROODERUDWLYH(QWHUSULVHV >@ )URQW%DFNHQG ³$QJXODU   6SULQJ %RRW   3RVWJUH64/´
 KWWSVIURQWEDFNHQGFRPVSULQJERRWDQJXODUVSULQJ
SS±
ERRWSRVWJUHVTO DFFHVVHG$XJ 
>@ 7 0 'HVHUQR - ( ( G 2OLYHLUD DQG 2 *URWWNH ³5HJLRQDO
>@ 8QLW\ 7HFKQRORJLHV ³8QLW\ (QJLQH´ 
$QDHVWKHVLD 6LPXODWRU DQG $VVLVWDQW 5$6LP$V  0HGLFDO LPDJH
SURFHVVLQJ VXSSRUWLQJ DQDHVWKHVLRORJLVWV LQ WUDLQLQJ DQG KWWSVXQLW\FRPSURGXFWVXQLW\HQJLQH DFFHVVHG6HS 
SHUIRUPDQFH RI ORFDO EORFNV´ LQ  ,((( WK ,QWHUQDWLRQDO >@ 5ýLKiN$QDWRPLH7KLUG3UDJXH&=*UDGD3XEOLVKLQJDV
6\PSRVLXPRQ&RPSXWHU%DVHG0HGLFDO6\VWHPVSS± 
 >@ %OHQGHURUJ ³7KH )UHHGRP WR &UHDWH´ 
KWWSVZZZEOHQGHURUJDERXW DFFHVVHG6HS 


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 75

 3URSRVHGPRGLILFDWLRQVRIWKHFRPSXWHUVFLHQFH
VWXG\SURJUDPVWRDGGUHVVVSDFHVDIHW\LQGXVWU\
DQGDFDGHPLDUHTXLUHPHQWV

3HWHU%XWNDDQG0DUWLQ6DUQRYVNê
'HSDUWPHQWRI&\EHUQHWLFVDQG$UWLILFLDO,QWHOOLJHQFH
)DFXOW\RI(OHFWULFDO(QJLQHHULQJDQG,QIRUPDWLFV
7HFKQLFDO8QLYHUVLW\RI.RVLFH
/HWQi.RãLFH6ORYDNLD
SHWHUEXWND#WXNHVNPDUWLQVDUQRYVN\#WXNHVN


$EVWUDFW²,Q WKLV SDSHU ZH SUHVHQW WKH UHVXOWV RI WKH H[LVWLQJFXUULFXOXPVRQWZRLPSRUWDQWVORYDNXQLYHUVLWLHV
RQJRLQJ (6$ 3(&6 VXSSRUWHG SURMHFW 6.63(GX ZKLFK 7HFKQLFDO8QLYHUVLW\RI.RãLFHDQG&RPHQLXV8QLYHUVLW\
IRFXVHV RQ DQDO\VLV RI WKH VWXG\ EUDQFKHV RI FRPSXWHU LQ %UDWLVODYD DQG GHYHORS LPSOHPHQWDWLRQ SODQV IRU
VFLHQFHDQGDVWURSK\VLFVDQGSURSRVHWKHLUPRGLILFDWLRQVIRU LGHQWLILHGPRGLILFDWLRQVWRUHGXFHWKHJDSEHWZHHQDFWXDO
VWUHQJWKHQLQJ WKH NQRZOHGJH RI VWXGHQWV WRZDUG 6SDFH VLWXDWLRQDQGUHDOQHHGVRI63GRPDLQ
6DIHW\ 3URJUDP 63  :H IRFXVHG RQ WKH DQDO\VLV RI %\WKHSDUWLFLSDWLRQRILPSRUWDQWXQLYHUVLWLHVIURPWZR
FRPSXWHUVFLHQFHUHODWHGVWXG\SURJUDPVDW)(,78.(DQG PDLQ FLWLHV ORFDWHG LQ GLIIHUHQW SDUWV RI 6ORYDNLD WKH
LGHQWLILHGJDSVEHWZHHQWKHWRSLFVUHTXLUHGE\WKHDFDGHPLD RYHUDOO FRYHUDJH RI 6ORYDN 67(0 VWXGHQWV ZLOO EH
DQG LQGXVWU\ LQ WKLV GRPDLQ 7KHQ ZH SURSRVHG WKH DVVXUHG 7KH LPSRUWDQW UROH ZLOO SOD\ WKH H[SHUW
PRGLILFDWLRQV DLPHG WR PLQLPL]H LGHQWLILHG JDSV LQ IRUP RI
NQRZOHGJHIURPOHDGLQJHQWLWLHVLQVSDFHVFLHQFH ,QVWLWXWH
QHZO\ HVWDEOLVKHG FRXUVHV PRGLILFDWLRQV RI FRXUVH FRQWHQW
RI ([SHULPHQWDO 3K\VLFV 6ORYDN $FDGHP\ RI 6FLHQFHV 
RUDGGLWLRQRIH[LVWLQJFRXUVHVIURPRWKHUVWXG\SURJUDPV
DQG VSDFH LQGXVWU\ $VWURV 6ROXWLRQV  7KH\ UHSUHVHQW
SRWHQWLDO IXWXUH HPSOR\HUV DQG WKH\ ZLOO KHOS WKH
, ,1752'8&7,21 XQLYHUVLWLHV WR WXQHXS WKH 63 FXUULFXOXPV WRZDUG
(XURSHDQOHYHO$VDUHVXOWRIWKHSUHVHQWHGFRQVRUWLXP¶V
6ORYDNLDLVRQLWVZD\WREHFRPHDQ$VVRFLDWHPHPEHU FRRSHUDWLRQ VWXGHQWV ZLOO EH EHWWHU HTXLSSHG IRU WKHLU
RI(6$7REHVXFFHVVIXOLQWKLVLQWHUQDWLRQDOFRRSHUDWLRQ SRVLWLRQV LQ LQGXVWU\ RUUHVHDUFK LQ WKH 63 GRPDLQ7KH
6ORYDNLD QHHGV WR GHILQH LWV PDLQ VWUHQJWKV LQ WKH VSDFH GDWDDQDO\VWVDQGPDFKLQHOHDUQLQJHQJLQHHUVZLOOEHDEOH
VHFWRU DQG SODQ KRZ WR VXSSRUW LWV IXUWKHU GHYHORSPHQW WR EHWWHU XQGHUVWDQG WKH QHHGV DQG VSHFLILFV RI 63
7KDQNVWRRQJRLQJ6ORYDNQDWLRQDO6SDFH6DIHW\3URJUDP GRPDLQ DV ZHOO DV DVWURSK\VLFLVWV ZLOO REWDLQ H[FHOOHQW
VWXG\ 6.63  WKDW LV VXSSRUWHG E\ (6$ WKH 63 RU VFLHQWLILFNQRZOHGJHRI63GRPDLQDQGVROLGNQRZOHGJH
6SDFH6DIHW\ GRPDLQKDVEHHQUHFRJQLVHGDVDQLPSRUWDQW RI SRWHQWLDO DSSOLFDWLRQ RI PRGHUQ GDWD DQDO\VLV DQG
SDUWRIWKH6ORYDNVSDFHHFRV\VWHPZLWKKLJKSRWHQWLDOIRU PDFKLQH OHDUQLQJ WHFKQLTXHV %HWWHU SUHSDUHG JUDGXDWHV
WKHIXWXUH>@6ORYDNLDGHFLGHGWRVHOHFW63DVRQHRILWV IURPFDQOHDGWRVWURQJHULQGXVWU\DQGUHVHDUFKLQVWLWXWLRQV
SURJUDPVIRUSDUWLFLSDWLRQLQ(6$ LQWKH63MREVPDUNHWLQ6ORYDNLDDQGLQ(XURSH
,IZHZDQWWRGHILQH6SDFHVDIHW\ZHFDQVD\WKDWLWLV 7KLV VKRXOG EH EHQHILFLDO DOVR WR (6$ DQG LWV IXWXUH
V\VWHPDWLF PRQLWRULQJ RI RXWHU VSDFH WR HQVXUH WKH VDIH DFWLYLWLHV DV WKH SURSRVHG DFWLYLW\ LV LQ FRLQFLGHQFH ZLWK
H[SORUDWLRQDQGVXVWDLQDEOHXVHRIVSDFH7KHVDIHW\DVSHFW (6$$JHQGD>@7KHFKDOOHQJHLVWRRIIHU(XURSHDQ
LV LPSRUWDQW IRU VFLHQWLILF DQG FRPPHUFLDO DSSOLFDWLRQV WDOHQWDWWUDFWLYHRSSRUWXQLWLHVZLWKLQ(XURSHDQGWRDGDSW
DQGVHUYLFHVEDVHGRQVSDFHLQIUDVWUXFWXUHV,WLVWKHDLPRI KLJKHU HGXFDWLRQ FXUULFXOD WR WKH VNLOOV UHTXLUHG IRU WKH
(6$¶V 6SDFH 6DIHW\ 3URJUDPPH WR FRQWULEXWH WR WKH IXWXUH 0RUHRYHU (6$ LQ 'HFHPEHU  DSSURYHG WKH
SURWHFWLRQ RI RXU SODQHW RI KXPDQLW\ DQG RI 6SDFH IRU (GXFDWLRQ  6(   YLVLRQ DQG SODQ
LQIUDVWUXFWXUHVERWKRQ(DUWKDQGLQ VSDFHDJDLQVWWKUHDWV JLYLQJWKHLUJUHHQOLJKWIRUWKH(6$(GXFDWLRQ3URJUDPPH
IURPRXWHUVSDFH>@7KHSURJUDPPHLQFOXGHVDFWLYLWLHVLQ WR JR ZLGHU PRUH LQQRYDWLYH PRUH LQVSLUDWLRQDO DQG
WKH DUHDV RI VSDFH ZHDWKHU DQG QDYLJDWLRQ VHUYLFHV UHDFKLQJPXFKIDUWKHUWKDQEHIRUH>@>@
SURWHFWLQJWKHSODQHWDJDLQVWDVWHURLGVFROOLVLRQVEHWZHHQ
VDWHOOLWHV DQG VSDFH GHEULV DQG WKH VXVWDLQDEOH XVH RI 7KHPDLQDLPRIWKLVSDSHULVWRSURYLGHWKHUHVXOWVRI
VSDFH SURMHFW UHODWHG WR DQDO\VLV RI FRPSXWHU VFLHQFH SURJUDPV
DW RXU )DFXOW\ RI (OHFWULFDO (QJLQHHULQJ DQG ,QIRUPDWLFV
+RZHYHU WKH JURZWK RI 6ORYDN SDUWLFLSDWLRQ LQ 63 )(,  ZKHUH RXU JRDO ZDV WR DQDO\VH H[LVWLQJ JDSV IRU
PLJKWEHOLPLWHGE\DYDLODEOHKXPDQUHVRXUFHV2QHRIWKH IXWXUH,7HQJLQHHUV ZLWK WKH VSHFLDO IRFXVRQ WKHLU VNLOOV
VROXWLRQV IRU RYHUFRPLQJ WKLV OLPLWDWLRQ LV WR HTXLS WKH EHVLGHWUDGLWLRQDO,7VNLOOV LQGDWDDQDO\VLVDQGPDFKLQH
VWXGHQWV ZLWK WKH UHTXLUHG VNLOOV DQG VWHHU WKHP LQWR OHDUQLQJ
SDUWLFLSDWLRQRQ63
)RU WKLV UHDVRQ ZLWKLQ WKH 6.63(GX DFWLYLW\
VXSSRUWHGE\(6$ZHZRXOGOLNHWRUHYLHZSUHVHQWIXWXUH  
LQGXVWU\ DQG UHVHDUFK QHHGV LQ 63 GRPDLQ DQDO\VH 
67(0±6FLHQFH7HFKQRORJ\(QJLQHHULQJ0DWKHPDWLFV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 76

 7KHSDSHULVRUJDQL]HGDVIROORZVVHFWLRQ,,GHVFULEHV
WKH SURMHFW 6.63(GX DQG LWV EDVLF PHWKRGRORJ\ 7KH
GHVFULSWLRQRI)(,FRPSXWHUVFLHQFHLVSURYLGHGLQVHFWLRQ
,,,6HFWLRQ,9GHVFULEHVWKHDFDGHPLDDQGLQGXVWU\QHHGV
RQVNLOOVRIJUDGXDWHVRIFRPSXWHUVFLHQFHSURJUDPVDWWKH
)(, DQG WKH LGHQWLILFDWLRQ RI JDS EHWZHHQ FXUUHQW DQG
H[SHFWHGVWDWHRIWKHLUFXUULFXOXPVLVLQVHFWLRQ9)LQDOO\
VHFWLRQ9,LVGHGLFDWHGWRWKHGHVFULSWLRQRIWKHSURSRVHG
PRGLILFDWLRQ RI FRPSXWHU VFLHQFH SURJUDPV IROORZHG E\
FRQFOXVLRQ )LJXUH ,PSOHPHQWDWLRQSODQRIWKH6.63SURMHFW

,, %5,()'(6&5,37,212)6.63('8352-(&7 $FFRUGLQJ WR DGDSWDWLRQ RI )(, FRPSXWHU VFLHQFH
SURJUDPVWKHPDLQZRUNSDFNDJHVDUH
7KHPDLQREMHFWLYHRIWKH(6$3(&6SURMHFWHQWLWOHG
3URSRVDO IRU 6ORYDN XQLYHUVLWLHV FXUULFXOXP DGDSWDWLRQ • :3 ± )(, FXUULFXOXP UHYLHZ ZLWK WKH RXWSXW
WRZDUG63PDUNHW 6.63(GX LVWRDQDO\VHH[LVWLQJ GHOLYHUDEOH 71 FRQWDLQLQJ WKH DQDO\VLV RI WKH
FXUULFXOD RQ WZR XQLYHUVLWLHV 7HFKQLFDO 8QLYHUVLW\ RI FXUUHQWYHUVLRQRIVWXG\SURJUDPVDW)(,
.RãLFHDQG&RPHQLXV8QLYHUVLW\LQ%UDWLVODYD LQVWXG\ • :3 ± )(, FXUULFXOXP PRGLILFDWLRQV ZLWK WKH
EUDQFKHV RI FRPSXWHU VFLHQFH DQG DVWURSK\VLFV DQG RXWSXWGHOLYHUDEOH71FRQWDLQLQJDQDO\VLVRIJDSV
SURSRVH WKHLU PRGLILFDWLRQV IRU VWUHQJWKHQLQJ WKH EHWZHHQ FXUUHQW SURJUDPV DQG H[SHFWHG VNLOOV
LGHQWLILHG LQ 71 DQG 71 E\ H[SHUW DFDGHPLD
NQRZOHGJH RI VWXGHQWV WRZDUG 6SDFH 6DIHW\ 3URJUDP
DQG LQGXVWU\ SDUWQHUV  DV ZHOO DV SURSRVHG
63 7KHVSHFLILFJRDOVZKLFKDOVRSURYLGHRYHUYLHZRI PRGLILFDWLRQV DOVRUHYLHZHGE\H[SHUWV 
SURMHFWLPSOHPHQWDWLRQPHWKRGRORJ\DUHWR
• :3±,PSOHPHQWDWLRQSODQIRU)(,FXUULFXOXP
• UHYLHZ RI FXUUHQW LQGXVWU\ DQG DFDGHPLF 
ZLWK WKH RXWSXW GHOLYHUDEOH 71 ZKLFK ZLOO
UHVHDUFKQHHGVLQ63GRPDLQUHFRJQL]HJDSVLQ SURYLGH GHWDLOHG SODQ IRU LPSOHPHQWDWLRQ RI
WKH NQRZOHGJH RI JUDGXDWHV LQ DVWURSK\VLFV DQG SURSRVHGPRGLILFDWLRQV
FRPSXWHU VFLHQFH PDLQO\ IRU PDFKLQH OHDUQLQJ
DQGGDWDDQDO\VLVWRSLFV  ,,, )(,&20387(56&,(1&(678'<352*5$06
• DQDO\VH H[LVWLQJ FXUULFXOD RI FRPSXWHU VFLHQFH )(, 78.( VWXG\ SURJUDPV DUH GHILQHG E\ WKH VHW RI
DQG DVWURSK\VLFV VWXG\ EUDQFKHV ZLWK UHVSHFW WR VXEMHFWVSURYLGHGWRVWXGHQWVDVPDQGDWRU\DQGRURSWLRQDO
WKH 63 GRPDLQ DQG WKH NQRZOHGJH RI GDWD LQGLIIHUHQWOHYHOVRIHGXFDWLRQDQG\HDUVRIVWXG\7KHUH
DQDO\VLVDQGPDFKLQHOHDUQLQJDUHD DUH WKUHH OHYHOV RI VWXG\  %DFKHORU %F 
• SURSRVH PRGLILFDWLRQV LQ ERWK WHFKQLFDO DQG (QJLQHHULQJ0DVWHU (QJ06F  DQG 'RFWRUDO 3K' 
VFLHQWLILF VWXG\ EUDQFKHV WR UHGXFH WKH JDS %DFKHORU VWXG\ VW OHYHO  FRQVLVWV RI D \HDU HGXFDWLRQ
DFFRUGLQJWRFXUUHQWLQGXVWU\DQGUHVHDUFKQHHGV SURJUDP 0DVWHU VWXG\ QG OHYHO  LV  \HDUV ORQJ DQG
LQ63GRPDLQ 'RFWRUDOVWXG\LVRU\HDUV UGOHYHO 7KHPDLQIRFXV
• GHYHORS LPSOHPHQWDWLRQ SODQV IRU LGHQWLILHG RI RXU DQDO\VLV LV RQ FRPSXWHU VFLHQFH SURJUDPV WR
SURYLGHVWXGHQWVZLWKDOOWKHQHFHVVDU\VNLOOVWRZDUGVWKH
PRGLILFDWLRQV LQ ERWK WHFKQLFDO DQG VFLHQWLILF
HIIHFWLYHXVHRIPHWKRGVDQGWRROVIRUGDWDDQDO\VLVGDWD
VWXG\EUDQFKHVDWWKHLUXQLYHUVLWLHV SURFHVVLQJ DQG PDFKLQH OHDUQLQJ ZLWK DQ HPSKDVLV RQ
7KH SURMHFW FRQVRUWLXP FRQVLVWV RI OHDGLQJ SDUWQHU WKHGDWDDQDO\WLFDOWDVNVLQWKH63GRPDLQ
7HFKQLFDO 8QLYHUVLW\ RI .RãLFH )DFXOW\ RI (OHFWULFDO
)(,SURYLGHVGLIIHUHQWFRPSXWHUVFLHQFHSURJUDPVIRU
(QJLQHHULQJ DQG ,QIRUPDWLFV )(,  DQG VXEFRQWUDFWRUV VWXGHQWV ,Q %DFKHORU VWXG\ DOO VWXGHQWV VWDUW ZLWK WKH
&RPHQLXV 8QLYHUVLW\ LQ %UDWLVODYD )DFXOW\ RI VDPHVXEMHFWV LQFOXGLQJSURJUDPVLQ ZLWKEDVLFVRI
0DWKHPDWLFV 3K\VLFV DQG ,QIRUPDWLFV )03,  ,QVWLWXWH PDWKHPDWLFV VXEMHFW 0DWKHPDWLFV ,  ILUVW FRPSXWHU
RI ([SHULPHQWDO 3K\VLFV 6ORYDN $FDGHP\ RI 6FLHQFHV VFLHQFH SURJUDPPLQJ VXEMHFW )XQGDPHQWDOV RI
,(3  $VWURV 6ROXWLRQV VUR 8QLYHUVLW\ SDUWQHUV DUH $OJRULWKPV DQG 3URJUDPPLQJ  DV ZHOO DV EDVLFV RI
WDVNHG WR SURYLGH DQ DQDO\VLV RI WKHLU FXUUHQW FXUULFXOD HOHFWULFDODQGPDWHULDOHQJLQHHULQJ VXEMHFWV)XQGDPHQWDOV
ZKLOHH[SHUWSDUWQHUVDLPWRSURYLGHUHSRUWVRQH[SHFWHG RI HOHFWULFDO HQJLQHHULQJ DQG )XQGDPHQWDOV RI 0DWHULDOV
QHHGVIURPLQGXVWU\DQGDFDGHPLDRQJUDGXDWHVIRUWKHLU HQJLQHHULQJ DQG VRPH VKRUWLQWURGXFWLRQ RQRWKHU VWXG\
VXFFHVVIXOLQYROYHPHQWLQIXWXUHMRESRVLWLRQVDOOUHODWHG SURJUDPV LQFDVHWKH\ZDQWWRFKDQJHVWXG\SURJUDPDIWHU
WR 63 GRPDLQ DQG ERWK VFLHQWLILF DQG WHFKQLFDO VNLOOV ILUVW VHPHVWHU  2QHRI WKH DGYDQWDJHV LV WKDW DOO VWXGHQWV
KDYH VDPH VWDUWLQJ OLQH IRU WKH VHFRQG VHPHVWHU $OVR
7KHQSURMHFWFRQVRUWLXPPDWFKHVWKHQHHGVWRWKHFXUUHQW
GXULQJ WKH VHFRQG VHPHVWHU VHYHUDO VXEMHFWV DUH DJDLQ
SURJUDPV DQG SURSRVH WKH PRGLILFDWLRQV RI WKH H[LVWLQJ VKDUHGWKURXJKDOOSURJUDPV LQFOXGLQJVWXG\SURJUDPVLQ
SURJUDPV WR LPSURYH WKH FRYHUDJH RI WKH QHHGV 7KHQ  ,WLQFOXGHVWKUHHVXEMHFWV±0DWKHPDWLFV,,WKHILUVW
LPSOHPHQWDWLRQ SODQ LV SURSRVHG GHVFULELQJ QHFHVVDU\ EDVLF SK\VLFV VXEMHFW 3K\VLFV ,  DQG 3URJUDPPLQJ ,W
VWHSV WR SXW WKH SURSRVHG LQ SUDFWLFH 3URMHFW GXUDWLRQ LV PHDQV WKDW DOO )(, VWXGHQWV KDG DW OHDVW WKHVH VXEMHFWV
PRQWKVDQGWKHZRUNSODQIRUWKHSUHYLRXVO\GHVFULEHG ZLWK D EDVLF FRXUVH RQ SK\VLFV WZR LQSXW FRXUVHV RQ
VWHSVLVGHSLFWHGLQ)LJ PDWKHPDWLFVDQGSURJUDPPLQJ LQ&ODQJXDJH $VWKHVH
 DUHEDVLFOHYHOVXEMHFWVZHZLOOQRWLQFRUSRUDWHWKHPLQWR
GHHSHU DQDO\VLV ODWHU H[FHSW IRU D VKRUW VXPPDU\ RI WKH
EDVLFV RI3K\VLFV (YHU\ VWXG\ SURJUDP PLJKW KDYH WZR
DGGLWLRQDOVXEMHFWVWRLQWURGXFHWKHLUVSHFLILFWRSLFVZLWKLQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 77

WKHVHPHVWHU IRUH[DPSOH,QWHOOLJHQW6\VWHPVKDV3\WKRQ DJDLQVW F\EHU WKUHDWV 7KH JUDGXDWH LV DEOH WR
SURJUDPPLQJDQG)XQGDPHQWDOVRI,QWHOOLJHQW6\VWHPVDV LGHQWLI\ DQG VROYH VHFXULW\ LQFLGHQWV LPSOHPHQW
WKHLULQWURGXFWLRQIRUWKHUHVWRIWKHILUVW\HDURIVWXG\  VHFXULW\DUFKLWHFWXUHDQGGDWDVHFXULW\RSHUDWHDQG
7R FKDUDFWHUL]H VWXG\ SURJUDPV DQG WKHLU FRPSDULVRQ PDQDJH V\VWHPV ZLWK UHJDUG WR WKHLU VHFXULW\
ZHFDQVXPPDUL]HDOORIWKHPDVIROORZV SDUWLFLSDWH LQ WKH GHVLJQ DQG DXGLWV RI VHFXULW\
SURMHFWV &RPSDULVRQ WR RWKHUV 7KLV SURJUDP LV
• ,QIRUPDWLFV ,1)  ± &KDUDFWHULVWLF RI JUDGXDWHV VXSSRUWHGE\WKHVDPHGHSDUWPHQWDVWKH,1)DQG
7KHIRFXVLVRQIRXQGDWLRQVRILQIRUPDWLFVDQGGDWD LWDOVRVKDUHVPDQ\RILWVVXEMHFWVLQFOXGLQJVRPH
VWUXFWXUHV RSHUDWLQJ DQG GDWDEDVH V\VWHPV XVHU RI WKH VXEMHFWV UHODWHG WR GDWD SURFHVVLQJ DQG
LQWHUIDFHV ZHE DQG PRELOH WHFKQRORJLHV DQG PDFKLQH OHDUQLQJ %DVLFDOO\ ZLWKLQ WKH VFRSH RI
FRPSXWLQJ QHWZRUNV 7KH JUDGXDWH NQRZV RXU DQDO\VLV JUDGXDWHV FDQ JHW WKH VDPH VNLOOV DV
WHFKQRORJLHV IRU VRIWZDUH HQJLQHHULQJ DQG ,QIRUPDWLFV VWXGHQWV LQ WKHVH WRSLFV 7KHUHIRUH
LPSOHPHQWDWLRQ PDLQWHQDQFH RI LQIRUPDWLRQ VWXGHQWV RI WKLV SURJUDP DUH DOVR JRRG LQ
V\VWHPV &RPSDULVRQ WR RWKHUV *UDGXDWHV DUH SURJUDPPLQJDQG6:GHYHORSPHQW
VWURQJ LQ SURJUDPPLQJ VNLOOV DQG 6:
GHYHORSPHQW 'DWDUHODWHG WRSLFV DUH PDLQO\ RQ • &RPSXWHU 0RGHOLQJ &0  ± &KDUDFWHULVWLF RI
GDWD SURFHVVLQJ LQ GDWDEDVHV HYHQ VRPH PDFKLQH JUDGXDWHV$JUDGXDWHLQ&0KDVNQRZOHGJHRIWKH
OHDUQLQJLVDOVRSURYLGHGZLWKLQVSHFLILFVXEMHFWV PDWKHPDWLFDO DQG WKHRUHWLFDO FRQFHSWV RI
LQIRUPDWLFV DQG UHODWHG SUREOHPV 7KH\ KDYH
• ,QWHOOLJHQW V\VWHPV ,6  ± &KDUDFWHULVWLF RI H[SHULHQFHZLWKDFWLYHFRQWURORIPRGHUQVRIWZDUH
JUDGXDWHV  $ JUDGXDWH RI ,6 XQGHUVWDQGV WKH LQ VROYLQJ SUREOHPV DQG XVH PRGHUQ DQDO\WLFDO
GHSOR\PHQWRSHUDWLRQ RI F\EHULQIRUPDWLRQ PHWKRGV DQG VRIWZDUH IRU WKH DSSOLFDWLRQ RI
V\VWHPV ZLWK HOHPHQWV RI $, DQG PRGHUQ ,&7 GHVLJQHG PRGHOV IRU WKHLU VLPXODWLRQ *UDGXDWHV
WHFKQRORJLHV DUH VNLOOHG LQ LQWHOOLJHQW LQIRUPDWLRQ KDYH DQ DGYDQFHG XQGHUVWDQGLQJ RI QXPHULFDO
DQG QHWZRUN WHFKQRORJLHV IRU DXWRPDWLRQ DQG PRGHOLQJ DQG VLPXODWLRQ ZKLFK FDQ EH XVHG LQ
FRQWURORIYDULRXVW\SHVRIV\VWHPVLQGXVWULDODQG LGHQWLILFDWLRQ DQDO\VLV UHVHDUFK GHYHORSPHQW
VRFLDO URERWLFV PDQXIDFWXULQJ LQGXVWU\ WUDQVSRUW GHVLJQ DQG LPSOHPHQWDWLRQ RI FRPSXWHU V\VWHPV
RU KHDOWKFDUH 7KLV LQFOXGHV WKH VNLOOV IRU WKH IRU SUREOHPVROYLQJ &RPSDULVRQ WR RWKHUV 7KH
LPSOHPHQWDWLRQ RI F\EHUQHWLFV DXWRPDWLRQ SURJUDP KDV HPHUJHG ZLWKLQ WKH PDWKHPDWLFDO
LQIRUPDWLFV DQG $, IRU GHVLJQLQJ LQWHOOLJHQW GHSDUWPHQW WKHUHIRUH WKH PDLQ VWUHQJWK LV LQ
V\VWHPV XVLQJ VNLOOV UHODWHG WR GDWDEDVHV VHQVRUV QXPHULFDO PHWKRGV VLPXODWLRQ DQG RSWLPL]DWLRQ
WKH ,R7 DQG ,QGXVWU\  &RPSDULVRQ WR RWKHUV WKDW PLJKW EH XVHG IRU YDULRXV SUREOHPV 7KH
*UDGXDWHV *UDGXDWHV DUH VWURQJ LQ XQGHUVWDQGLQJ JUDGXDWHV DUH QRW WKDW VWURQJLQ SURJUDPPDWLF DQG
DQG DSSOLFDWLRQ RI $, FRQFHSWV 7KHLU VNLOOV DUH 6: GHYHORSPHQW VNLOOV DOVR WKH\ GR QRW VKDUH
YHU\ JRRG LQ WKH ODUJHU VFRSH RI $, WRSLFV IURP PDQ\RIGDWDDQDO\VLVDQGPDFKLQHOHDUQLQJWRSLFV
JHQHUDO $, WRSLFV ZLWK NQRZOHGJHEDVHG ZLWKH[FHSWLRQRQPHWKRGVIRUWLPHVHULHVDQDO\VLV
UHSUHVHQWDWLRQV IX]]\ PRGHOLQJ $,EDVHG ORJLF
LQIHUHQFH 7KHLU VFRSH RI PDFKLQHGHHS OHDUQLQJ • &RPSXWHU QHWZRUNV &1  ± &KDUDFWHULVWLF RI
DQGKHXULVWLFDSSURDFKHVLVZLGHUWKDQLQDOORWKHU JUDGXDWHV 7KH &1 JUDGXDWH ZLOO XQGHUVWDQG
VWXG\SURJUDPV FRQFHSWV UHODWHG WR FRPSXWHU HQJLQHHULQJ DQG
FRPSXWHU QHWZRUN GHVLJQ ZLWK DQ HPSKDVLV RQ
• %XVLQHVV ,QIRUPDWLFV %,   &KDUDFWHULVWLF RI
FRPPXQLFDWLRQ V\VWHPV 7KH\ FDQ VSHFLI\ GHVLJQ
JUDGXDWHV7KHPDLQIRFXVLVRQWKHGHVLJQDQGRU
DQG LPSOHPHQW FRPSXWHU QHWZRUNV DQG WKHLU
DSSOLFDWLRQRILQWHOOLJHQWPHWKRGVDQG,7WRROVIRU FRPSRQHQWV DQG LQWHJUDWH FRPSXWHU DQG
WKH VXSSRUW RI GHFLVLRQ SURFHVVHV LQ YDULRXV FRPPXQLFDWLRQWHFKQRORJLHVLQWKHGHYHORSPHQWRI
GRPDLQV PHGLFLQH LQGXVWU\ VFLHQFH HGXFDWLRQ
WHFKQLFDO DQG V\VWHP PHDQV ,PSRUWDQW SDUW RI
HWF  7KH JUDGXDWHV FDQ DQDO\]H DQG GHVLJQ
NQRZOHGJH DFTXLUHG LQ WKH &1 SURJUDP LV YHU\
LQIRUPDWLRQV\VWHPVIRUGHFLVLRQSURFHVVHVDVZHOO JRRG VNLOOV LQ VLJQDO SURFHVVLQJ &RPSDULVRQ WR
DVDQDO\]HGDWDIURPWKHPDQGSURYLGHNQRZOHGJH RWKHUV &1 SUREDEO\ SURYLGHV WKH EHVW +: VNLOOV
EDVHG GHFLVLRQV XVLQJ $,EDVHG PHWKRGV
UHODWHG WR VHQVRUV DQG VLJQDO SURFHVVLQJ IURP DOO
*UDGXDWHVRI06F/HYHOKDYHYHU\JRRGVNLOOVIRU
FRPSXWHU VFLHQFH SURJUDPV DW )(, 7KH JUDGXDWHV
LQWHOOLJHQW NQRZOHGJHEDVHG GDWD DQDO\VLV LQ DUHVWURQJLQWKHVHWRSLFVZKLOHWKHLUSURJUDPPDWLF
YDULRXV GRPDLQV XVLQJ GLIIHUHQW $, PHWKRGV DQG6:GHYHORSPHQWVNLOOVDUHUDWKHUDYHUDJH'XH
&RPSDULVRQWRRWKHUV*UDGXDWHVDUHVWURQJLQGDWD
WRWKHXVHRIHOHFWURQLFVLQVLJQDOSURFHVVLQJ+:
DQDO\VLV DQG PDFKLQH OHDUQLQJ WRSLFV DV WKHVH
WKH\KDYHEHWWHUHOHFWULFDOHQJLQHHULQJDQGSK\VLFV
WRSLFV DUH TXLWH VWURQJO\ VXSSRUWHG 2Q WKH RWKHU NQRZOHGJHWKDQRWKHUSUHYLRXVSURJUDPV
KDQG FRPSOH[ 6: LPSOHPHQWDWLRQ DQG UHODWHG
WRSLFV DUH QRW DV VWURQJO\ FRYHUHG DV LQ WKH
,9 5(48,5('6.,//6)25$&$'(0,$$1',1'8675<
,QIRUPDWLFV SURJUDP 7KH FORVHVW VWXG\ SURJUDP
ZLWKPDQ\VKDUHGVXEMHFWVLV,6 7KHJRDORIWKLVSKDVHZDVWRVXPPDUL]HWKHQHHGVDQG
• &\EHUVHFXULW\ &6  ± &KDUDFWHULVWLF RI UHTXLUHPHQWV RI DFDGHPLD LQVWLWXWLRQV DQG UHVHDUFK
JUDGXDWHV7KH$JUDGXDWHRI&6KDVNQRZOHGJHRI RUJDQL]DWLRQVDVZHOODVLQGXVWU\WRJUDGXDWHVZLWKUHVSHFW
WKH PDWKHPDWLFDO DQG WKHRUHWLFDO IRXQGDWLRQV RI WRSDUWLFLSDWLRQRQ6SDFH6DIHW\3URJUDPPH 63 
LQIRUPDWLFVZLWKDQDVSHFWRIF\EHUVHFXULW\.QRZV
$ $FDGHPLDQHHGV
DQG XQGHUVWDQGV SULQFLSOHV PHWKRGV DQG VHOHFWHG
WHFKQRORJLHV IRU VHFXULQJ FRPSXWHU V\VWHPV ,QWKLVDUHDWKHH[SHUWSDUWQHUIURP6ORYDN$FDGHP\RI
FRPSXWHU QHWZRUNV RUJDQL]DWLRQ DQG VRFLHW\ 6FLHQFHV SURYLGHG DQ RYHUYLHZ RI WKH QHHGV DQG

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 78

H[SHFWDWLRQV RQ JUDGXDWHV IURP SK\VLFV DQG HQJLQHHULQJ 7KH DELOLW\ WR FRPPXQLFDWH HIIHFWLYHO\ DQG VHUYH DV D
VWXGLHV IRU WKHLU VXFFHVVIXO LQYROYHPHQW LQ 63 EULGJH EHWZHHQ WKH GHYHORSPHQW DQG PDLQWHQDQFH WHDP
SURJUDPPHDQGUHODWHGWRSLFVDOOIURPWKHDFDGHPLDSRLQW DQG WKH FOLHQW LV DOVR LPSRUWDQW 7R VXP XS WKH PRVW
RI YLHZ $FDGHPLD QHHGV ZHUH FRYHUHG LQ WKUHH PDLQ IUHTXHQWO\ UHTXLUHG FRPSHWHQFLHV E\ WKH 63 UHODWHG
VHFWLRQV 6SDFH :HDWKHU 6SDFH GHEULV DQG &OHDQ 6SDFH LQGXVWU\ ZKLFK DUH PRVW UHODWHG WR WKH )(, FRPSXWHU
DQG 3ODQHWDU\ 'HIHQFH )RU HDFK VHJPHQW WKH PDLQ VFLHQFHVWXG\SURJUDPVDQGWKHLUJUDGXDWHVDUH
UHTXLUHPHQWVRIJUDGXDWHVJRRGSUDFWLFHLQ(6$0HPEHU • 3URJUDPPLQJ ODQJXDJHV VXFK DV 3\WKRQ &&
6WDWHV 06  DQG VXPPDU\ RI H[SHFWDWLRQV IRU JUDGXDWHV -DYD-DYD6FULSW0DWODE
ZHUH SURYLGHG LQFOXGLQJ VRPH LQWHUHVWLQJ FRXUVHV WR EH
FRQVLGHUHGIRUIXWXUHPRGLILFDWLRQV • 'DWDEDVHV 2UDFOH3RVWJUH64/ 
$FFRUGLQJWRWKHVFRSHRIRXUSURMHFWPDLQDLPRI)(, • 6RIWZDUHKDUGZDUH GHYHORSPHQW GHVLJQ
LV WR SURYLGH PRGLILFDWLRQV RI )(, FRPSXWHU VWXG\ LPSOHPHQWDWLRQYDOLGDWLRQWHVWLQJ 
SURJUDPV ZLWK DOO QHFHVVDU\ FKDQJHV WR VXSSRUW PDLQO\ • 6RIWZDUH LQWHJUDWLRQGHYHORSPHQW WRROV *LW
GDWD DQDO\VLV DQG PDFKLQH OHDUQLQJ '$0/  LQ 63 -HQNLQVHWF 
GRPDLQLQFOXGLQJRWKHU,7VNLOOVZKLFKDUHLPSRUWDQWIRU • $UWLILFLDO LQWHOOLJHQFH0DFKLQH /HDUQLQJ,PDJH
UHVHDUFKDFDGHPLDSRVLWLRQV.QRZOHGJHRIWRSLFVIURP SURFHVVLQJ
SK\VLFVZDVDOVRFRQVLGHUHGEXWLWVOHYHOPLJKWQRWEHDW
,Q WKH IROORZLQJ SKDVH WKH GHWDLOHG YHUVLRQ RI ERWK
VDPH GHSWK OHYHO DV LQ FDVH RI VWXGHQWV IURP )03, LQ
DFDGHPLDDQGPDUNHWQHHGVZHUHPDSSHGWRWKH)(,VWXG\
3K\VLFV VWXGLHV 7KHUHIRUH WKH H[SHFWHG OHDUQLQJ
SURJUDPV WR LGHQWLI\ WKH JDSV EHWZHHQ WKH UHTXLUHPHQWV
RXWFRPHVDUHPRUHGHWDLOHGLQFRPSXWHUVFLHQFHWRSLFV
DQGFXUUHQWO\FRYHUHGWRSLFV
5HJDUGLQJWKHNQRZOHGJHRUH[SHFWDWLRQVIRUJUDGXDWHV
ZH SURYLGHG DQ RYHUYLHZ LQ VHYHUDO JURXSV IURP ZKLFK 9 ,'(17,),&$7,21 2) *$36 %(7:((1
WKHPRVWLPSRUWDQWDUHWKRVHUHODWHGWRSURFHVVLQJRIGDWD 1(('6$1'&855(17678'<352*5$06
GDWDVFLHQFHVLPXODWLRQDQGVRPHLQWURGXFWRU\WRSLFVIRU
XQGHUVWDQGLQJ SULQFLSOHV RI SK\VLFV LQ 63 GRPDLQ $IWHU LGHQWLILFDWLRQ DQG XQLILFDWLRQ RI QHHGV IURP WKH
HVSHFLDOO\ H[SHUW SDUWQHUV RXU JRDO ZDV WR PDS UHTXLUHPHQWV ZLWK
WKH FXUUHQW VWDWXV:H KDYH VHOHFWHG VHYHUDO PDLQ JURXSV
• 0DFKLQHOHDUQLQJ'HHSOHDUQLQJ
RI WRSLFV DQG WKHQ LGHQWLI\ ZKLFK RI WKH WRSLFV IURP
• $GYDQFHGLPDJHSURFHVVLQJ UHTXLUHPHQWV SRRO DUH RU DUH QRW UHSUHVHQWHG LQ )(,
• 'DWDDQDO\WLFDOSURFHVVHVGDWDVFLHQFHDQGUHODWHG FRPSXWHU VFLHQFH SURJUDPV ,Q VRPH FDVHV WRSLF LV ZHOO
WRSLFV GDWDEDVHVELJGDWD  UHSUHVHQWHGLQFRXUVHVZLWKLQWKHDOOSURJUDPVVRPHWLPHV
• ,7WHFKQRORJLHVIRUVXSSRUWRIGDWDSURFHVVLQJDQG LW LV PLVVLQJ IRURQH RUWZR SURJUDPV DQG WKHUHDUHDOVR
GHYHORSPHQWLQ63GRPDLQ FDVHVZKHQWRSLFLVPLVVLQJLQDOOSURJUDPV$VH[SHFWHG
GDWDDQDO\VLVDQGPDFKLQHOHDUQLQJLVEHVWFRYHUHGE\%,
• ,QWURGXFWLRQWR6:(6SDFH'HEULVDQG3ODQHWDU\ DQG,6DVERWKDUHSURJUDPV IXOILOOHGE\WKH'HSDUWPHQW
'HIHQFH HQJLQHHULQJOHYHO  RI&\EHUQHWLFVDQG$UWLILFLDO,QWHOOLJHQFH2WKHUSURJUDPV
$QDO\VLV RI VXEJURXSV RI WRSLFV DQG UHODWHG FRXUVHV  KDVWKHLURZQVWUHQJWKVDQGWKHLUFRYHUDJHFRUUHODWHGZLWK
FXUULFXOD XVHG ZLWKLQ (6$ PHPEHU VWDWHV SURYLGHV WKHQ WKHFKDUDFWHULVWLFVSURYLGHGLQVHFWLRQ,,,:HDUHQRWDEOH
PRUHGHWDLOHGYLHZDQGGHILQLWLRQRIOHDUQLQJRXWFRPHV WRSURYLGHWKHIXOOVFRSHRIUHVXOWVZLWKDOOWRSLFVEXWZH
ZLOOVKRZVRPHRIWKHH[DPSOHVRQKRZWKHJDSVLQWRSLFV
% ,QGXVWU\QHHGV DQG LQ QH[W VHFWLRQ DOVR SURSRVHG PRGLILFDWLRQV  ZHUH
,Q WKLV DUHD ZH KDYH SURYLGHG DQ RYHUYLHZ RI QHHGV GRFXPHQWHG
UHTXLUHG IRU YDULRXV SRVLWLRQV LQ GRPDLQ RI VSDFH VDIHW\ )LUVWZHSURYLGHRYHUYLHZRIPDLQJURXSLQJRIWRSLFV
,QWKLVFDVHWKHUHTXLUHPHQWVZHUHREWDLQHGIURPDVHWRI ,QILQDOZHLGHQWLILHGPDLQJURXSVRIWRSLFVDFFRUGLQJ
YDFDQFLHVLQERWK6ORYDNDQG(8(6$SULYDWHHQWLWLHV WRUHTXLUHPHQWVGRFXPHQWIURPH[SHUWV(YHU\JURXSFDQ
)URP WKH FRPSXWHU VFLHQFH SHUVSHFWLYH PRUH WKDQ ? FRQWDLQ RQH RU VHYHUDO WRSLFV WR ZKLFK ZH WULHG WR ILQG
RI YDFDQFLHV UHTXLUH GHVLJQLQJ DQG LPSOHPHQWLQJ FRXUVHVWKDWFRYHULW+HUHE\³WRSLF´RUOHDUQLQJRXWFRPH
YLUWXDOL]HG KLJKO\ DYDLODEOH KDUGZDUH HQYLURQPHQWV ZLWK ZHPHDQRQHRUPRUHWKHPHVWKDWDUHFRPSLOHGWRJHWKHU
QHWZRUN FRPPXQLFDWLRQ WKDW UHTXLUH KLJK OHYHOV RI 7KHUHIRUH WRSLF FDQ EH GHVFULEHG VKRUWO\ HJ REMHFW
SHUIRUPDQFH DQG VHFXULW\ $Q XQGHUVWDQGLQJ RI VSDFH RULHQWHGSURJUDPPLQJ EXWLWDOVRPD\EHSURYLGHGXVLQJ
DSSOLFDWLRQ VWDQGDUGV DQG (&66 (XURSHDQ &RRSHUDWLRQ PRUHGHWDLOHGDQGORQJHUGHVFULSWLRQFRQWDLQLQJVHWRIWKH
IRU 6SDFH 6WDQGDUGL]DWLRQ  VWDQGDUGV DUH DOVR KLJKO\ PHWKRGV VRIWZDUH SDUDGLJPV HWF ,I ZH GLG QRW ILQG
UHTXLUHG6RPHVSHFLILFVNLOOVUHTXLUHGIRUWKHVHSRVLWLRQV FRYHUDJHLQVRPHVWXG\SURJUDPVWRSLFZDVLGHQWLILHGDV
LQFOXGH SURJUDPPLQJ LQ 3\WKRQ -DYD & DQG PLVVLQJ IRU WKRVH SURJUDPV DQG LQ QH[W VWHS LW ZDV
-DYD6FULSW DV ZHOO DV H[SHULHQFH ZLWK /LQX[ GDWDEDVHV DGGUHVVHG +HUH DUH PDLQ WRSLF JURXSV DQG VKRUW
VXFK DV 2UDFOH DQG 3RVWJUH64/ 2WKHU VNLOOV UHTXLUHG GHVFULSWLRQ RQ LGHQWLILHG WRSLFV QXPEHU RI WRSLFV ZLWKLQ
LQFOXGH H[SHULHQFH LQ V\VWHPV HQJLQHHULQJ LQWHJUDWLRQ JURXSLVLQEUDFNHW 
YHULILFDWLRQDQGYDOLGDWLRQ)DPLOLDULW\ZLWKVFULSWLQJDQG • 3URJUDPPLQJ ODQJXDJHV   ± PRVWO\ UHODWHG WR
/LQX[ 3\WKRQ VKHOO HWF  VRIWZDUH GHYHORSPHQW WRROV EDVLF SURJUDPPLQJ VNLOOV LQ GLIIHUHQW ODQJXDJHV
-HQNLQV *LW HWF  6RPH RI WKH SRVLWLRQV LQYROYH HJ3\WKRQ&0$WODEHWF DQGSDUDGLJPV HJ
UHYLHZLQJ WHFKQLFDO GRFXPHQWDWLRQ SUHSDULQJ WHVW REMHFWRULHQWHGSURJUDPPLQJ 
HQYLURQPHQWV DQG XQLWWHVW FRGHV DQG H[HFXWLQJ WHVW
FDPSDLJQV GXULQJ WKH GLIIHUHQW SKDVHV RI WKH YHULILFDWLRQ • 26 $GPLQLVWUDWLRQ 6FULSWLQJ   ± /LQX[ 26
DQGYDOLGDWLRQSURFHVV,QDGGLWLRQGLUHFWLQWHUDFWLRQZLWK H[SHULHQFHVZHUHLGHQWLILHGKHUH
FOLHQWVLVIUHTXHQWO\UHTXLUHGIRUWKHVHSRVLWLRQV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 79

 • 'HYHORSPHQW DQG LQWHJUDWLRQ WRROV   ± ODUJHU FRQVLGHUDWLRQRQKRZWKRVHDUHFRYHUHGE\SDUWLFXODU)(,
JURXS ZLWK HPSKDVLV RQ WKH YHUVLRQLQJ WRROV FRPSXWHUVFLHQFHSURJUDPV,WLVQRWSRVVLEOHWRVKRZDOO
LQWHJUDWLRQRIVRIWZDUHFRQWDLQHUL]DWLRQDQGFORXG RI WKHP EXW )LJ  VKRZV H[DPSOHV RI WZR JURXSV RI
WHFKQRORJLHVPLFURVHUYLFHVHWF WRSLFV OHDUQLQJ RXWFRPHV  ± VRIWZDUH GHYHORSPHQW DQG
• 6RIWZDUH GHYHORSPHQW   ± GHVLJQ PRGHOLQJ GDWDDQDO\VLV
*8,  DQG GHYHORSPHQW RI VRIWZDUH DQG ZHEDSSV
WHVWLQJDQGYDOLGDWLRQ
• 'DWDEDVHV   ± EDVLF VNLOOV IRU GDWDEDVHV 64/
1R64/GLVWULEXWHGGDWDEDVHVHWF
• 3URMHFWPDQDJHPHQW  ±PDQDJHPHQWRISURMHFWV
LQFO PHWKRGRORJLHV IRU VRIWZDUH GHYHORSPHQW DQG
LWVOLIHF\FOH$JLOH6&580PHWKRGRORJLHVHWF
• 6WDWLVWLFV DQG PXOWLYDULDWH DQDO\VLV   ± EDVLFV RI
VWDWLVWLFVPXOWLYDULDWHDQDO\VLVUHJUHVVLRQPHWKRGV
DQG IRUHFDVWLQJ GLVFULPLQDQW DQDO\VLV FRPSRQHQW
DQDO\VLV)RXULHUDQDO\VLVHWF
• 1XPHULFDO PRGHOLQJ DQG VLPXODWLRQ   ±
FRPSXWDWLRQDOPHWKRGVXVHGIRUSUREOHPVROYLQJLQ
DQDO\WLFDO WDVNV SUREDELOLVWLF PRGHOLQJ PHWKRGV
RSWLPL]DWLRQHWF
• 'DWD DQDO\VLV   ± DFFHVVLQJ DQG SURFHVVLQJ RI
GDWDRIGLIIHUHQWW\SHVODQJXDJHVIRUGDWDDQDO\VLV
'$  H[SORUDWLYH GDWD DQDO\VLV '$ DV D SURFHVV
LQFOGDWDPLQLQJGDWDYLVXDOL]DWLRQWHFKQLTXHV
• 0DFKLQH'HHS/HDUQLQJ  ±GXHWRRXUVFRSH
WKH ODUJHVW JURXS RI WRSLFV XQGHUVWDQGLQJ RI 0/
DSSOLFDWLRQ LQSXWV RXWSXWV HYDOXDWLRQ RI PRGHOV
EXLOGLQJRIPRGHOVIURPWUDGLWLRQDO0/WHFKQLTXHV
WR GHHS OHDUQLQJ DUFKLWHFWXUHV XVH RI VRIWZDUH IRU
0/'/WHFKQRORJLHVLQFO%LJGDWDHWF 
• ,PDJH SURFHVVLQJ   ± ZRUNLQJ ZLWK LPDJHV )LJXUH ([DPSOHRILGHQWLILFDWLRQRIPLVVLQJOHDUQLQJRXWFRPHV
VWDQGDUG WDVNV LQ LPDJH SURFHVVLQJ VHJPHQWDWLRQ 
GHWHFWLRQ FODVVLILFDWLRQ RI LPDJHV VSHFLILF '/ $V ZH FDQ VHH HYHU\ ³WRSLF´ OHDUQLQJ RXWFRPH  LV
DUFKLWHFWXUHVIRULPDJHSURFHVVLQJWDVNVHWF DQDO\VHGDFFRUGLQJWRFRYHUDJHLQ)(,VWXG\SURJUDPV,I
• 'RPDLQVSHFLILF GDWD KDQGOLQJ DQG SURFHVVLQJ   WRSLFV LV FRYHUHG FRXUVH LV PHQWLRQHG LQ ³&RYHUDJH LQ
± VSHFLILF VXEVHW RI WRSLFV IRU GDWD SURFHVVLQJ RI FRXUVHV´,IWRSLFLVQRWFRYHUHGE\VRPHVWXG\SURJUDP
63 UHODWHG GDWD VDWHOOLWH GDWD SURFHVVLQJ GDWD LWV DEEUHYLDWLRQ LV XVHG LQ ³0LVVLQJ  1RW FRYHUHG´7KLV
DQDO\VLV DQG YLVXDOL]DWLRQ VSHFLILF IRU GRPDLQ LQIRUPDWLRQZDVWKHQXVHGLQVWHSRIFUHDWLRQRUSURSRVLQJ
SDFNDJHV GDWD IRUPDWV DQG GDWDVHWV VSHFLILF IRU RIPRGLILFDWLRQV
GRPDLQXVHFDVHVRI'$0/LQGRPDLQHWF
9, 352326('02',),&$7,21672678'<352*5$06
• +:  (PEHGGHG 6\VWHPV   ± GHYHORSPHQW RI
HPEHGGHG V\VWHPV WRSLFV RQ ,R7 ,QGXVWU\  $IWHUDQDO\VLVRIWKHJDSVRXUJRDOZDVWRJRWKURXJK
VHQVRUV IRU VRPH RI WKH SURJUDPV DOVR GHVLJQ RI PLVVLQJ  QRW FRYHUHG SDUW RI WKH WDEOHV IRU JDS DQDO\VLV
VXFK V\VWHPV RQ KDUGZDUH OHYHO &$' V\VWHPV DQGVHOHFWDVROXWLRQIRUFRYHUDJHLQVRPHRIWKHSURJUDP
GHVLJQ RI HOHFWURQLF KDUGZDUH FLUFXLWERDUGVSOXV ,Q JHQHUDO WKHUH DUH VHYHUDO VROXWLRQV ZKLFK GHSHQG RQ
VSHFLILF NQRZOHGJH UHODWHG WR RSHUDWLRQ RI H[LVWHQFH RI VXLWDEOH FRXUVH LQ RWKHU VWXG\ SURJUDP ZH
HPEHGGHGV\VWHPDQGKDUGZDUHLQFRQWH[WRIVSDFH PD\ UHXVH H[LVWLQJ FRXUVH LQ SURJUDP ZKLFK GLG QRW
PLVVLRQVLQFOGDWDKDQGOLQJHWF FRYHUHG WRSLF  XSGDWH RI H[LVWLQJ FRXUVH LQ SURJUDP WR
HQKDQFH FRYHUDJH IRU VRPH PLVVLQJ WRSLFV WKLV LQFOXGHV
• 6WDQGDUGV    (&66 (XURSHDQ &RRSHUDWLRQ IRU UHXVHRIVXFKXSGDWHGFRXUVHLQRWKHUSURJUDP RUFUHDWH
6SDFH 6WDQGDUGL]DWLRQ  DQG UHODWHG VWDQGDUGV QHZ FRXUVH WKDW KHOS WR FRYHU XVXDOO\ YHU\ QHZ VHW RI
ZKLFKDUHXVHGLQVSDFHLQGXVWU\VHFWRU WRSLFV 7KLV OHDG XV WR FRPELQDWLRQ RI WRSLFV OHDUQLQJ
• 3K\VLFV ± EDVLF NQRZOHGJH   ± VHYHUDO WRSLFV RXWFRPHV  LQ RQH WDEOH DQG VHOHFWLRQ PRGLILFDWLRQ VHWXS
FRQVLGHUHG DV EDVLF FRXUVHV LQ SK\VLFV ZKLFK IRU HDFK FRPELQDWLRQ $FWXDOO\ ZH ZHUH DEOH WR KHOS
PLJKW EH LPSRUWDQW IRU EHWWHU XQGHUVWDQGLQJ RI FRYHUPRVWRIWKHWRSLFVE\UHXVHDQGXSGDWHRIH[LVWLQJ
VRPHSULQFLSOHVXVHGLQ63GRPDLQ FRXUVHV ZLWK RQO\ WZR QHZ FRXUVHV PDLQO\ IRU H[DFW
• 3K\VLFV  VHOHFWHG WRSLFV UHODWHG WR 63   ± FRQQHFWLRQWRSK\VLFVDQGXVHFDVHVLQ63GRPDLQ
VSHFLILFWRSLFVLQSK\VLFVUHODWHGWR63GRPDLQLQ ,QILQDOSURSRVDORIPRGLILFDWLRQVKHUHLVWKHVXPPDU\
VSDFHZHDWKHUVSDFHGHEULVDQGSODQWDU\GHIHQFH RI FRXUVHV LQYROYHG LQ SURSRVHG FKDQJHV RI SURJUDPV
 QXPEHURIFRXUVHVLQEUDFNHWV 
7KH ZKROH WDEOH ZKLFK PDSV WRSLFV LQ DOO  JURXSV • &RXUVHV ZKLFK DUH UHXVHG LQ RWKHU SURJUDPV LQ
KDYHWRJHWKHUWRSLFGHILQLWLRQV OHDUQLQJRXWFRPHV IRU WKHLU FXUUHQW IRUP   /DQJXDJHV IRU 'DWD

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 80

 $QDO\WLFV %XVLQHVV $QDO\WLFV 'DWD YLVXDOL]DWLRQ ³,QWURGXFWLRQWR6SDFH5HVHDUFKDQG7HFKQRORJ\´QHZ
&RPSXWHU9LVLRQ$GYDQFHGPHWKRGVRIFRPSXWHU FRXUVH ZLOO FRPELQH LQWURGXFWRU\ NQRZOHGJH RQ 6SDFH
YLVLRQ 7HFKQRORJLHV IRU %LJ 'DWD 3URFHVVLQJ 6DIHW\ GRPDLQ VSDFH ZHDWKHU VSDFH GHEULV HWF  ZLWK
)XQGDPHQWDOV RI FORXG V\VWHPV 3URMHFW VRPH EDVLF RYHUYLHZ RI SULQFLSOHV  SUREOHPV ZKLFK DUH
0DQDJHPHQW,QGXVWU\ VROYHG WKHUH DV ZHOO DV H[DPSOHV KRZ LW LV VROYHG
• &RXUVHV XSGDWHG DQG UHXVHG LQ VDPH RU RWKHU REVHUYDWLRQV LQVWUXPHQWV IRU REVHUYDWLRQ GDWD DQG WKHLU
SURJUDP   ,QWHJUDWLRQ DQG GHSOR\PHQW RI DQDO\VLV WR VROYH VRPH SUREOHP  7KH RWKHU SDUW RI WKH
VRIWZDUH DSSOLFDWLRQV 1XPHULFDO 0DWKHPDWLFV FRXUVH DSSUR[ ZLOOIROORZEDVLFSULQFLSOHVZKLFKDUH
3UREDELOLW\ 7KHRU\ DQG 0DWKHPDWLFDO 6WDWLVWLFV XVHIXO IRU WKH GHYHORSPHQW RI VDWHOOLWHV IURP VSDFH
3URFHVVLQJ RI 6SDFH 'DWD $GYDQFHG PHWKRGV RI HQYLURQPHQW FKDUDFWHUL]DWLRQ DQG QHHGV IRU VSDFH
GDWDDQDO\VLV3K\VLFV,, PLVVLRQVWRRSHUDWLRQRIPLVVLRQVDQGGDWDSURFHVVLQJ,Q
JHQHUDO WKLV FRXUVH ZLOO EH DW DQ LQWURGXFWRU\ OHYHO 2XU
• 1HZ FRXUVHV   ,QWURGXFWLRQ WR 6SDFH 5HVHDUFK H[SHFWDWLRQ LV WR DOZD\V SURYLGH VRPH WRSLF ZLWK EDVLF
DQG 7HFKQRORJ\ 'DWD DQDO\VLV LQ 6SDFH 5HVHDUFK WKHRU\ DQG FRPELQH LW ZLWK VRPH XVH FDVH )RU H[DPSOH
'RPDLQ WKHUH ZLOO EH VKRUW LQWURGXFWLRQ RQ VRPH SKHQRPHQD LQ
$JDLQ IRU WKH VDNH RI VLPSOLFLW\ ZH RQO\ VKRZ DQ VRODUSK\VLFVXVHFDVHRIWKHLUREVHUYDWLRQVDQGSURYLGHG
H[DPSOHRIWDEOHZKLFKVXPPDUL]HVRPHRIWKHSURSRVHG GDWD H[DPSOHV RI DQDO\VLV ZKLFK PLJKW EH XVHIXO IRU
FKDQJHV DFFRUGLQJ WR FRPELQDWLRQ RI WRSLFV ,Q )LJ EHWWHU XQGHUVWDQGLQJ RI SKHQRPHQD RU VRPH DSSOLFDWLRQ
VHYHUDOWRSLFVDUHFRPELQHWRKHOSVRPHRIWKHSURJUDPV OLNHZDUQLQJV\VWHP ,Q6RODUSK\VLFVIRUH[DPSOHVRODU
FRYHUWKRVHWKHPHVWKDWDUHPLVVLQJZLWKLQIRUPDWLRQRQ HUXSWLRQV REVHUYDWLRQV PLJKW EH GLVFXVVHG ZKDW DUH
ZKLFKWRSLFVDUHPHDQWZKDWW\SHRIFKDQJHLWLVQDPHRI FRQVHTXHQFHV IRU 6XQ(DUWK UHODWLRQV KRZ REVHUYDWLRQDO
WKH FRXUVH GHVFULSWLRQ UHODWHG WR FKDQJH DQG DOVR GDWD ORRNV OLNH IRU VHOHFWHG PLVVLRQ  LQVWUXPHQW  KRZ
SURSRVHGVHPHVWHUDQGSURJUDPVIRUFKDQJH,QWRWDOZH WKHVH GDWD PLJKW EH XVHIXO IRU SUHGLFWLRQ RU ZDUQLQJ IRU
KDYH SURSRVHG  FKDQJHV LQ FRXUVHV EXW LW RIWHQ DIIHFW SRVVLEOH JHRPDJQHWLF VWRUP 6LPLODUO\ H[DPSOH RI
WKHLU UHXVH LQ VHYHUDO SURJUDPV WKHUHIRUH QXPEHU RI VDWHOOLWH GHYHORSPHQW WRSLFV PLJKW EH FRYHUHG 2WKHU
FKDQJHVLVHYHQKLJKHUIRU)(,SURJUDPV SDUWQHUVLQWKLVDFWLYLW\PLJKWEHFRDXWKRUVRIVXFKFRXUVH
 FRQWHQW LQ WKH IXWXUH LPSOHPHQWDWLRQ 7KDQNV WR WKLV
FRPELQDWLRQ WKLV FRXUVH PLJKW EH XVHIXO IRU PRVW RI WKH
SURJUDPV ZLWKLQ WKH IDFXOW\XQLYHUVLW\ DOVR IRU RWKHU
HQJLQHHULQJWRSLFV 
7KH VHFRQG QHZ FRXUVH ³'DWD DQDO\VLV LQ 6SDFH
5HVHDUFK 'RPDLQ´ IROORZXS WR WKH LQWURGXFWRU\ FRXUVH
RQVSDFHUHVHDUFK PRVWO\IRU63GRPDLQ ZLWKSUDFWLFDO
XVHFDVHVRIGDWDDQDO\VLVDQGDSSOLFDWLRQRI0/PHWKRGV
7KLV FRXUVH ZLOO FRQVLVW RI XVH FDVH RI UHVHDUFK DOUHDG\
GRQH LQ 63 ZKHUH 0/ ZDV XVHG ,W DOVR PLJKW LQFOXGH
VRPH WRSLFV IURP RWKHU VSDFHUHODWHGDUHD ZKLFK DUH QRW
GLUHFWO\LQ63 DVWURQRP\WRSLFV 7KHELJDGYDQWDJHIRU
VHOHFWLRQ RI XVH FDVHV LV WKDW DOO IRXU SDUWQHUV KDG PDQ\
VXFKXVHFDVHVVRPHGRQHWRJHWKHU HJRQVRODUFRURQD
VHJPHQWDWLRQ LRQRVSKHUH VFLQWLOODWLRQV SUHGLFWLRQ
GHWHFWLRQ RI 7UDQVLHQW  /XPLQRXV (YHQWV GLIIHUHQW WDVNV
UHODWHGWRVSDFHGHEULVDQGSODQHWDU\GHIHQFHLQFORUELWDO
GDWDUHWULHYDOSURFHVVLQJYLVXDOL]DWLRQH[LVWLQJOLEUDULHV
WKHLUXVDJHHWF 7KHFRXUVHLVSUHIHUUHGWREHDWWKHHQG
RI 06F VWXG\ VKRUWO\ EHIRUH JUDGXDWHV DUH ORRNLQJ IRU
MREV
7R VXPPDUL]H WKH FKDQJHV IRU SDUWLFXODU )(, VWXG\
SURJUDPV ZH DOVR SUHSDUHG WDEOH ZKLFK FRPELQHV DOO
SURSRVHG PRGLILFDWLRQ IRU HYHU\ SURJUDP DQG SODFH LQ
EDFKHORU RU PDVWHU VWXG\ 7KLV PLJKW KHOS WR EHWWHU
XQGHUVWDQGWKHFKDQJHVLQWLPHGXULQJWKHVWXG\SURJUDPV
ZKLFK VHPHVWHU DUH WKH\ SURSRVHG 2I FRXUVH ZH DOVR
 WULHG DOZD\V WR ILQG VDPH SODFHPHQW IRU HYHU\ FRXUVH
)LJXUH ([DPSOHRIVRPHSURSRVHGPRGLILFDWLRQV WRJHWKHU ZLWK WKH FRQVLGHUDWLRQ RI LWV VWXG\ OHYHO
 %F06& SRVLWLRQLQZLQWHUYVVXPPHUWHUPHWF
5HJDUGLQJ WKH QHZ FRXUVHV RXU SURSRVDO LV WR VXSSRUW )LJ GHSLFWV FKDQJHV SHU SURJUDP +HUH ,QIRUPDWLFV
EHWWHU XQGHUVWDQGLQJ RI SK\VLFV UHODWHG WR 63 LQ FRXUVH DQG &\EHUVHFXULW\ SURJUDPV DUH YHU\ VLPLODU DQG WKHLU
³,QWURGXFWLRQWR6SDFH5HVHDUFKDQG7HFKQRORJ\´DVZHOO FKDQJHV DUH WKH VDPH WKHUHIRUH DUH UHSUHVHQWHG WRJHWKHU
DVXVHFDVHIRUGDWDDQDO\VLVGDWDSURFHVVLQJDQGPDFKLQH 7KHUHDUHQRFKDQJHVH[SHFWHGLQILUVW\HDURIVWXG\LQ%F
OHDUQLQJLQ63GRPDLQLQFRXUVH³'DWDDQDO\VLVLQ6SDFH OHYHO WKHUHIRUH WDEOH VWDUWV ZLWK QG \HDU ZLQWHU WHUP
5HVHDUFK 'RPDLQ´ 7KDQNV WR H[SHUWV LQ WKH SURMHFW ZH $OVR HQG RI PDVWHU VWXG\ VXPPHU WHUP RI QG \HDU  LV
KDYHJRRGRSSRUWXQLW\IRUIXWXUHVXSSRUWLQSUHSDUDWLRQRI IRFXVHGPDLQO\RQILQLVKLQJRIWKHVHV)RUWKHEHWWHUYLVXDO
WKHVHFRXUVHV GLVWLQFWLRQ EHWZHHQ W\SH RI FKDQJHV ZH KDYH XVHG
GLIIHUHQWFRORU5HG±IXOO\QHZFRXUVH2UDQJH±UHXVHRI

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 81

 XSGDWHGFRXUVHLQWKLVSURJUDP*UHHQ±UHXVHRIH[LVWLQJ DEOH WR VXSSRUW VXFK FKDQJHV ZLWKLQ RUJDQL]DWLRQ
FRXUVH LQ WKLV SURJUDP %OXH ± XSGDWH RI FRXUVH ZKLFK UHVRXUFHVRUZHZLOODVNIRUVRPHJUDQWIRULWVUHDOL]DWLRQ
H[LVWVDOUHDG\LQSURJUDP ZKLFKPLJKWLQFOXGHVRPHQHZ(6$JUDQWIRUVXSSRUWRI
HGXFDWLRQDO DFWLYLWLHV +RZHYHURXU VWXG\ PD\ EH XVHIXO
QRW RQO\ IRU WKH VXSSRUW RI FXUULFXOXP DGDSWLRQ RQ 63
GRPDLQ EXW LW PD\ DOVR VHUYH DV D XVH FDVH RI
PHWKRGRORJ\ IRU RWKHU UHVHDUFKHUV DQG HGXFDWRUV LQ WKHLU
HIIRUW IRU DGDSWDWLRQV RI WKHLU FXUULFXOD WR RWKHU
IDFXOWLHVXQLYHUVLWLHVDQGDOVRRWKHUGRPDLQV

$&.12:/('*0(17
7KHZRUNSUHVHQWHGLQWKLVSDSHUZDVVXSSRUWHGE\WKH
(XURSHDQ 6SDFH $JHQF\ (6$  ZLWKLQ WKH FRQWUDFW QR
 SURSRVDO 6.63(GX SURSRVDO ,'
6.B SURSRVDO WLWOH 3URSRVDO IRU 6ORYDN XQLYHUVLWLHV
FXUULFXOXP DGDSWDWLRQ WRZDUG 63 PDUNHW  ZKLFK ZDV
VXEPLWWHGUHYLHZHGUHFRPPHQGHGDQGQHJRWLDWHGIRUWKH
LPSOHPHQWDWLRQLQ(6$WK3(&6FDOO 3ODQIRU(XURSHDQ
&RRSHUDWLQJ6WDWHV IRU6ORYDNLDLQ

 5()(5(1&(6
)LJXUH 6XPPDU\RIFKDQJHGFRXUVHVWKURXJKWLPHIRU)(, >@ 6.63 &RQVRUWLXP ³&ROOHFWLRQ RI &DSDELOLWLHV DQG $VVHWV LQ
FRPSXWHUVFLHQFHSURJUDPV 6ORYDNLD UHODWHG WR (6$ 6SDFH 6DIHW\ 3URJUDPPH´ GHOLYHUDEOH
 71  7HFKQLFDO 1RWH  6.$' 6ORYDNLD 1DWLRQDO 6SDFH
6DIHW\3URJUDPPH 63 6WXG\LPSOHPHQWHGZLWKLQWKH(6$WK
7KH QH[W VWHS IRU WKH ILQDO PHWKRGRORJ\ VWHS LQ 6. 3(&6FDOO
63(GX LV WR SURYLGH LPSOHPHQWDWLRQ SODQ DV ILQDO >@ + .UDJ ³(6$ V 6SDFH 6DIHW\ 3URJUDPPH´ LQ 3URFHHGLQJV )LUVW
GHOLYHUDEOH ,W LQFOXGHV DQDO\VLV RI DGPLQLVWUDWLYH ,QWHUQDWLRQDO 2UELWDO 'HEULV &RQIHUHQFH 6XJDU /DQG 7H[DV /3,
SURFHVVHVIRUDSSOLFDWLRQRISURSRVHGPRGLILFDWLRQVLQ)(, &RQWULEXWLRQ 1R  +RXVWRQ 7; /XQDU DQG 3ODQHWDU\
FXUULFXOXP ILQDO GHFLVLRQ RQ PRGLILFDWLRQV ZKHWKHU ,QVWLWXWH
FKDQJHZLOOEHVXSSRUWHGDQGLQ ZKDWH[WHQWSUHSDUDWLRQ >@ (XURSHDQ6SDFH$JHQF\³(6$$*(1'$0DNHVSDFHIRU
RIQHFHVVDU\GRFXPHQWDWLRQIRUFKDQJHVLQVWXG\SODQVLQ (XURSH´ SXEOLF UHOHDVH GRFXPHQW DYDLODEOH RQOLQH DW
VHOHFWHG FRXUVHVSUHSDUDWLRQRI QHFHVVDU\ GRFXPHQWDWLRQ KWWSVHVDPXOWLPHGLDHVDLQWGRFV(6$B$JHQGDBBILQDOSGI
IRU SURSRVDO RI QHZ VXEMHFWV DQG EXGJHW SODQQLQJ IRU >@ 6 'DOOHGRQQH 09 3UHVW 0 +HUPHV DQG / 3RKO ³6SDFH
(GXFDWLRQ LQ (XURSH 6WDWXV DQG 3URVSHFWV´ WK 6\PSRVLXP RQ
FUHDWLRQ RI QHZ FRQWHQW DQG VXSSRUW RI DGPLQLVWUDWLYH 6SDFH(GXFDWLRQDO$FWLYLWLHV%DUFOHQDSS
SURFHVVHV )LQDOLPSOHPHQWDWLRQSODQIURPWKLVVWXG\FDQ >@ (XURSHDQ 6SDFH $JHQF\ (6$ HGXFDWLRQ ³:LWK VSDFH DW WKH
EH WKHQ XVHG DV D VWDUWLQJ GRFXPHQW IRU UHDOL]DWLRQ RI IRUHIURQW RI HGXFDWLRQ´ RQOLQH DYDLODEOH DW
SURSRVHGPRGLILFDWLRQV%XGJHWHVWLPDWLRQPLJKWEHYHU\ KWWSVZZZHVDLQW(GXFDWLRQ:LWKBVSDFHBDWBWKHBIRUHIURQWBRIBH
KHOSIXOLQGHFLVLRQRQKRZWRSURFHHGHJLIZHZLOOEH GXFDWLRQ


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 82

 ASM Modeling for Training System and
Handwritten Recognition

Pasquina Campanella
Department of Computer Science
University of Bari “Aldo Moro”
via Orabona, 4 – 70126 Bari – Italy
pasqua13.cp@libero.it

Abstract— In the area of Pattern Recognition, specifically

in the field of Handwriting Recognition, will be presented
modeling a system of training and recognition of
handwritten characters and figures by means an Abstract
State Machine (ASM). The processing operation is to
perform the thinning in the singular regions of the track
pattern handwritten character after determining the
thinning in the regular regions. In fact, it is considered a
character of the Latin alphabet, the letter “R” in capital
letters, or the digit “4”, chosen because they contain all
the singularities that are possible to find the other
characters or figures, and applies them to a technique Figure 1. Phases of training and recognition
structural decomposition intended for training and
recognition using Hidden Markov Model (HMM). Of
course, the same character written by different authors, For each letter of Latin alphabet, we construct a
or by the same author at different times, produces a Hidden Markov Model [9]. The system is initialized by
different result. So, the training activities are more the initial training of each HMM through a number of
properly configured with an operation for classifying patterns of the character [20], [21], [22]. The
handwritten characters more or less similar, based on processing can be made up of many different
slight differences in slope of the stroke. The strokes are operations, such as: segmentation, the extraction of the
the parts of the track, the character components contour, thinning, histogram, etc [19]. The operation
manuscript. The classification of characters allows that we take into consideration is the thinning, and in
obtaining a knowledge base to be used in the recognition particular operation in the depletion regions of singular
phase. track, having already determined the thinning in the
regular parts. Through the operation of thinning, which
is an operation of processing, it produces a path of a
Keywords: Pattern Recognition, Handwriting thickness of 1 pixel; also through a series of operations
Recognition, Abstract State Machine, Hidden Markov of pre-processing and processing of the singular points
Model, Database. are obtained for the track, those in red in the Fig. 2-3,
in particular are distinguished: end point, bend points
and branch point [16], [17]. It is designed and built a
database of handwritten characters: a sheet, a
I. INTRODUCTION considerable number of writers have shown his
manuscript of 26 Latin basic characters for 7 samples
of each [17], [18]. The basic characters are cut into the
A system of training and recognition of cursive database. The operation of modeling by the method
handwriting consists of the following steps in sequence ASM intended data is the operation of thinning in the
(Fig. 1): regions singular.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 83

 ground model can represent a contract between
customer and developer.

A. Formulation of Requirements

We specify the informal requirements of the

problem. The module consists of various processing
operations. Modeling by an ASM, the operation of
thinning in the regions singular [11], [12]. We have a
number of points containing the information of its
direction, given by the direction of the line which is the
prolongation of the segment (thin), wherein the
Figure 2. Fan exploration. Figure 3. Extension and cleaning
directional point belongs, corresponding to the region
adjacent to adjust [13], [14]. Consider a directional
The extension of the second directional point meets point any. Imagine that each of these points directional
move forward according to its direction in the plot
the extension of the first directional point, the point of
(foreground) of the pattern, within the boundary, going
intersection becomes the point of curvature, the to scrutinize, observe what happens before. In practice,
remaining part of the stroke (i.e., the semi-segment that observes the subsequent adjacent pixels, forward,
reaches the boundary) is cleaned. This will mean according to its direction, by means of a search
designing it in the refinement of level 2. The technique in a fan. If the pixel being analyzed is a
extensions that reach the boundary at the end, when foreground, look at the next adjacent pixel (always
there are more directional points to allow them to using a search range), until you see another directional
explore in the singular regions, these extensions should point; a point of contour; a thinned; a point of
be eliminated, and the points from which direction curvature; a branching point. If I meet another
matches the explorations become endpoints of thinned directional point, then traced the segment connecting
image. the two directional points, namely that from which they
are party to that found. If you encounter a boundary,
then carry out an extended line (thin) by drawing a
II. METHODOLOGY ASM BASED segment connecting the directional point from where I
started to edge point detected; after directional consider
another point and repeat the procedure from the
beginning. If you encounter a thinned, then carry out an
The method based on Abstract State Machine (ASM) extended line (thin) by drawing a segment connecting
is a practical design for the development of complex the directional point from where I started to point
software systems, based on the formalism of ASM [2], thinned identified. I am going to analyze both two
[22]. It allows the development of rigorous and formal semi-segments where I crossed the segment narrowed
software systems, for sequential processing, parallel at the point: if the opposite extreme of the first semi-
and distributed. The Abstract State Machine have been segment is an edge point, then delete the semi-segment
used effectively to model systems of any type. Allow by setting the pixels of the semi-segment points of the
to model systems with a top-down approach by foreground, and the imposed point identified thinned to
controlling the complexity of the project [3], [6]. To a point of curvature; if the opposite extreme of the
identify and manage risks from the early planning second semi-segment is an edge point, then delete the
stages. The objective of the ASM method, and that is semi-segment by setting the pixels of the semisegment
the objective that is intended to pursue in this paper is points of the foreground, and imposed point of the
to define a conceptual framework to support and identified thinned to a point of curvature; otherwise,
supplement the main activities of software the opposite extremes of both semi-segment points will
development and the main techniques of modeling and be directional or points of curvature or branch points,
of analysis [14], [15]. Here we apply the method of therefore, imposed on the thinned point identified in a
software development based on ASM to a problem in branch point of the 3rd order (the order will be only an
processing of the handwriting recognition. information for subsequent processing). After
directional consider another point and repeat the
procedure from the beginning. If you encounter a point
III. GROUND MODEL DEFINITION of curvature, then traced the segment connecting the
directional point from where I started with the point of
curvature, and the point of curvature becomes a
The ground model is intended to describe the branching point of the 3rd order (the order is only for
information processing later). After directional
system requirements in an evolving, in a consistent and
consider another point and repeat the procedure from
unambiguous, simple, concise and complete abstract the beginning. If you encounter a branch point, then
[4]. It is a formal method for representing traced the segment connecting the directional point
specifications [1]. The formalization process can from where I started with the branching point, and the
address the problems of language and communication branching point is that while your order is increased by
between the client and the developer, in fact, the 1 (the order will only information for further

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 84

processing). After directional consider another point δ(Cleaning,g) = Cleaning,
and repeat the procedure from the beginning. At the δ(Cleaning, h) = EndState
end of the analysis of each directional point, if there are
points directional whose extension is finished on the For other pairs of the cartesian product Qx∑
contour, and the extension has not yet been deleted, function is not defined, so δ in reality is a function
then that extension should be deleted, and the restricted to the subset of Qx∑. The state StartState
directional point becomes the terminal point. The corresponds to the image of the character, tapered only
termination condition is one in which each directional in the regular regions identified. The state Marking is
point has resolved his status as a directional point in the image of the character, marked at the ends of the
any of the other items required, which are each thinned segments of regular regions, and the term
directional point became another point provided, that marking is understood as marking the ends of the
there are more directional points. The output of the thinned segments of regular regions. The state
system is the thinned image pattern. Exploration is the image of the character, unique in the
regions explored, and the term exploration is
B. Finite State Machine (FSM) understood as exploration regions in the singular image
of the character. The state Thinning corresponds to the
image of the character, thinned regions singular, and
FSM = ¢Q, ∑, δ, q0, F² the term thinning is understood as thinning the image
of the singular character in the regions. The state
Q = ^StartState, Marking, Cleaning of the character corresponding to the image,
Exploration,Thinning,Cleaning, EndState` cleaned up by extensions of the segments that have
∑ = ^a,b, c, d, e, f , g, h,i`q0 = StartState F = gone on to slam on the boundary, and the term cleaning
^EndState` is understood as a clean image of the character. The
state EndState corresponds to the thinned character,
even in the singular regions (Fig. 4).
a = there are segments thinned regions of regular; b =
no thinning segments of regular regions there are C. Ground Model (livello 0)
points; c = directional; d = no directional points; e =
and you can draw a segment; f = you can not draw a The computational model of the ASM machine is a
segment; g = there are extensions on the boundary; h = state model. So, here we go FSM to ASM. In ASM
there are no extensions on the boundary; i = no states are associated with a set of values of any type,
directional points and there are no extensions on the stored in appropriate locations [6]. The environment of
boundary. ASM consists of the segments obtained by thinning of
regular regions adjacent regions singular (Fig. 5).
Marking in the state (image directional points), the
segments are those of the regular regions and their
references have been marked as directional points. This
state is the prime condition to start with. The state
Exploration (image candidate points) indicates the
situation in which from directional point has been
identified another point with which the two points are
candidates to be connected by a segment of the two
extreme candidate points. The images of pixels in the
matrix corresponding to states Marking (directional
image points), Exploration (image candidate points)
and thinning every time take a different representation.
An image of the pattern (ie the character manuscript) is
loaded into a data structure “matrix”, possibly
consisting of pointers to pointers, using a programming
language such as ANSI C. The different images that
Figure 4. States of transition are created successively represent the different
locations of the ASM. A state of the ASM is a different
δ: Q x∑o Q transition function configuration of the pixels in the array representing the
δ(StartState,a) = Marking, image of the pattern. The first example was to be one
δ(StartState,b) = EndState, in which the ends of the segments of the regular
δ(Marking,c) = Exploration, regions were marked with directional points (yellow
δ(Marking,g) = Cleaning, pixels, determining the coordinates of the point, the
δ(Marking,i) = EndState, direction of the pixel data from the direction of the line
δ(Exploration,e) = Thinning, which was detached from the segment where the
δ(Exploration,f) = Marking, extreme point belongs, and other data).
δ(Thinning,c) = Exploration,
δ(Thinning,d) = Marking,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 85

 Functions
dynamic controlled pxlType : NPxlInMatrix
PixelType
dynamic monitored pxlCoord : NPxlInMatrix
Integer x Integer
dynamic shared pxlColor : NPxlInMatrix
PixelColor
dynamic monitored pxlDir : NPxlInMatrix
Real
dynamic controlled pcLinkpd : ContourPoint
DirectionalPoint

The function pxlColor in the points is a function

directional shared, because the color of such points has
been written the first time by the agent outside, ie from
the environment, in fact it can also be considered to
function controlled by the ASM.
Controlled variables
dynamic controlled pxl : NPxlInMatrix
dynamic controlled pd : DirectionalPoint
dynamic controlled pc : ContourPoint
Ground Model Rules
macro rule r_Explore($pd in DirectionalPoint) =
skip
macro rule r_TraceSegment($pd in DirectionalPoint,
$pxl in NPxlInMatrix) =
skip
Figure 5. States of transition in Ground Model macro rule r_CleaningSegment($pc in ContourPoint,
$pd in DirectionalPoint) =
skip
Signature of Ground Model macro rule r_DisplayImage =
skip
main rule r_ThinningSingularRegion =
We plan to load a pattern into an array. Matrix seq
elements are numbered from 1 to size. For each forall pd in DirectionalPoint with true do
element of the matrix corresponds to a pixel of the seq
image. The numerical values of the contour points and r_Explore[pd]
directional points are awarded in previous processing
r_TraceSegment[pd, pxl]
operations, respectively, in the operation of research in
the operation of the contours and thinning of the endseq
regular regions. forall pc in ContourPoint with pxlType(pc) = PM do
par
pd:= pcLinkpd(pc)
domain NPxlInMatrix subsetof Integer r_CleaningSegment[pc,pd]
domain NPxlInMatrix = {1..p} endpar
enum domain PixelType = {PD | PF | PC | PT | PB | r_DisplayImage[]
PR | PE | PM} endseq
enum domain PixelColor = {yellow | black | blue |
fuchsia | green | lightgreen | gold | red} Initial State – The foreign agent, which constitutes
domain DirectionalPoint subsetof NPxlInMatrix the environment than the ASM, has set is the total
domain DirectionalPoint = {pd1, pd2, ..., pdn} number of points directional (ie how many points are
domain ContourPoint subsetof NPxlInMatrix directional been identified in the transaction processing
domain ContourPoint = {pc1, pc2, ..., pcm} above) both the number of directional point in the
matrix and values of each of the components which
The elements of DirectionalPoint and ContourPoint make up each directional point. Same setting for the
are integers corresponding to the numerical values of contour points.
the pixels or, equivalently, to the boxes in the matrix of
the image. In fact, it would be abstract domain in this default init s0:
context, in fact, the whole number values are assigned function pxlCoord($pd1 in DirectionalPoint) = (x, y)
in the previous processing operations. function pxlColor($pd1 in DirectionalPoint) = yellow
function pxlDir($pd1 in DirectionalPoint) = r
function pxlCoord($pd2 in DirectionalPoint) = (x, y)

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 86

function pxlColor($pd2 in DirectionalPoint) = yellow respective directions. Marking the state is the state in
function pxlDir($pd2 in DirectionalPoint) = r which the system is brought to be able to determine
……………………………… from which directional point to start from the
function pxlCoord($pdn in DirectionalPoint) = (x, y) exploration. After the first phase, in fact, the only
function pxlColor($pdn in DirectionalPoint) = yellow possible situation is to start from one of the keys to the
function pxlDir($pdn in DirectionalPoint) = r exploration. Keep in mind that when you return to this
state the number of points decreased directional.
At this level of detail, the rules for the tracking of Exploring the state is the state in which the system is
the segment and for the cleaning of finite segments on brought to be able to determine which pixels to take
the boundary, they do nothing but establish the point at into consideration on the basis of a search procedure in
which they will then be included in a more concrete a fan. For which each time a pixel is analyzed, if it is a
vision of the model, other specified actions from the foreground pixel, the system must pass through the
requirements. analysis of the next pixel, as this point is not of interest,
until it encounters, or a directional point, or a point of
contour, or thinned a point, or a point of curvature, or a
IV. HORIZONTAL REFINEMENT (LEVEL 1)
branching point. Thinning the state in which the system
is brought in order to draw a segment (the thin line)
from the directional point from which it started to the
From the previous machine to obtain a more abstract
we operate its most refined form (concrete) [9]. point which is considered the second end of the
Imagine that each of these points directional moves in segment. The state Exploration, when it returns to that
the path (foreground) of the pattern, within the state, is to move to a new directional point that has not
boundary, going to scrutinize, explore, and establish yet been considered. Cleanup the state is reached at the
the type of pixels that happen before following a search moment when the state is located in a Marking exit
path to find a fan or a another point directional, or a condition determined by the fact that there are several
point of contour, or thinned a point, or a point of points to be considered directional. Then, from the
curvature or a branching point (Fig. 6). state of performing a final cleanup before exiting,
which is to identify those segments that are over the
A. Description of the various states. edge and that during the previous processes have not
been treated, then this state is the deletion of those
segments from the point boundary up to its singular
Marking the state is a bound state by an initialization point, exclusive. In addition, the directional point that
phase that includes the signature of directional points at produced the prolongation of the segment is ended up
the ends of the segments (thin) regions of regular and on the contour, in the case where this had remained
singular regions adjacent to their directional point, becomes the terminal point.

B. Description of ASM rules

In this section we describe the rules for the ASM

refinement horizontal (level 1), in particular, we
describe the Condition, the Update, the Location and
establish the consistency of Update. An ASM is a set of
instructions, called ASM rules, of the form if Condition
then Updates, where Updates is a set of updates of
functions f (t1, t2 ,..., tn):t. The signature (or vocabulary)
of the ASM, denoted by, is the set of function names
ASM. The superuniverse ASM is the collection of
elements of a state. A state of a signature ∑ is the
superuniverse of that state and in the interpretation of
function names in ∑. The interpretation of a function f
of arity n over a universe X is a function from Xn to X.
In particular, the interpretation of a constant is an
element of X. The partial functions are aggregated as
putting undef as an interpretation for the points where
the function is not defined. In our case, a state of the
ASM is a different configuration of the color of the
pixels in the array representing the image of the pattern
and any other data related to each of the pixels, such as:
the coordinates of the pixels, the direction of a certain
pixel, the distance in pixels of a certain pixel to
another, the marking of a certain pixel by using a
symbol, and so on. The locations are couples
Figure 6. States of transition in Horizontal Refinement

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 87

(f,(v1,v2,…,vn )) with f a function name and v1,v2,…,vn
list of values X.
An ASM is thus a system that comprises a finite
number of transition rules of the form if Condition then
Updates that determine the state transitions in the
machine. Condition (or guard) is any first-order
formula without free variables, whose interpretation
may be true or false, serves as a sentinel, as only its
occurrence determines the application of the rule.
Updates is a finite set of function updates the execution
of which determines the establishment or change (in
parallel) values of a function. An update corresponds to
the pair ¢location, value² .

¢Q,∑,δ ,q0, F²

whereby Updates = {¢location,value²} =

{¢location,value²} = {¢(f ,(v1,v2,…,vn)),v²}.

We define a matrix “matrix” of the type of data

structure. The structure is thus shown:
struct pixel {
int x,y;
enum color {black, yellow, blue, fuchsia, green,
lightgreen, gold, red} pxlClr;
enum type { PD, PF, PC, PT, PB, PR, PE, PM} V. VERTICAL REFINEMENT (LEVEL 1)
pxlType;
float dir; When starting from a directional point, exploring
}; meets a thinned, it means that we have met with a
struct pixel matrix[n][m]; thinning of the track. It should however, check whether
struct pixel segment[ns][p]; one of two semisegments segment intersected, it was
not long until you see the outline, in one of the
The first state StartState is the matrix “matrix” previous explorations. As if it were, this semi-segment
containing the image of the pattern of segments in the be cleaned and the intersection becomes a point of
regular regions or have been determined or have not curvature, and not a branch point of the third order.
been determined.
State := StartState New function
There are SegmAssRegioRego := false dynamic controlled thereIsSegment : NPxlInMatrix x
if (There aren’t SegmAssRegioRego = true) then { ContourPoint Bool
save(matrix) := matrix, Vertical integration of refinement rules to the rules of
view(matrix) := matrix, level 0
State := EndState main rule r_ThinningSingularRegion =
} seq
There are not SegmAssRegioRego := false forall pd in DirectionalPoint with true do
seq
r_Explore[pd]
if (pxlType(pxl) = PT) then
forall pc in ContourPoint with
thereIsSegment(pxl,pc) then
par
r_CleaningSegment[pc, pxl]
pxlColor(pxl) := green
pxlType(pxl) := PB
endpar
endif
r_TraceSegment[pd, pxl]
endseq
forall pc in ContourPoint with (pxlType(pc) = PM)
do
par

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 88

 pd:= pcLinkpd(pc) // determines the pd point
connected to pc
r_CleaningSegment[pc, pd]
pxlColor(pc) := blue
pxlType(pc) := PC
pxlColor(pd) := gold // the directional point
becomes the terminal point
pxlType(pd) := PE
endpar
r_DisplayImage[]
endseq

VI. VERTICAL REFINEMENT (LEVEL 2)

We think that the system allows to have better results if

all the points within a region unique directional parallel
go for exploration in search of other keys to the
junction points in the same direction, or thinned to
determine points of intersection between segments, or
end up on the boundary [5].

VII. HORIZONTAL REFINEMENT (LEVEL 2)

The second horizontal level of refinement is that one Figure 7. States of transition in Horizontal Refinement
side is most useful in the other manuscript appears
dangerous because it could change the character of the
skeleton manuscript [10]. In this second aspect, with
the right devices on the parameters, about the direction VIII. CORRECTNESS PROOFS
and distance, in the programming phase, avoids this
danger. Is to identify a point of curvature (order 2) or a
branching point (order n) and another point of The ASM method allows for verification by proving
curvature (order 2) or another branching point (order to the level of detail desired. At each level of detail
m) which are very close together, say the maximum of should check the properties of correctness: in terms of
three pixels apart, and in merging the second point in Ground Model is correct with respect to requirements;
the first in order to obtain only one singular point of the at the next level is the accuracy compared to the more
two. In the case where it joins a branching point of abstract level [7], [8]. Once the correspondence of
order n with another branch point of order m, the new states and their equivalence, it must be acknowledged
order of the first point is given by the sum of the orders that M* is a correct refinement of M if and only if
of the individual points that are united, less their every run fine (infinite) execution simulates abstract
connection, namely: n + m - 1. At this level we are (infinite) between states corresponding equivalents.
going to insert the following refinement (Fig. 7): Thanks to the verification of these properties, the
model is consistent not only accurate, flexible, simple,
concise, etc.

IX. CONCLUSIONS AND FUTURE

DEVELOPMENTS

The most commonly encountered in the application

advantageous aspect of the Abstract State Machine to
the problem of image processing is the facilitation in
refining the system horizontally and especially
vertically up to several levels of detail, as the further
refinement identifies and resolves a sub-problem which
is inherent to the problem of directly higher level. The
programming task is inherently complex in this area,
but the design using ASM gives you an overview of the
problem, and allows the verification and validation at
different levels of detail. This approach is a new idea

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 89

that can be applied in various areas of Pattern [13] P. Campanella, Oracle i-Learning Platform: Un caso di Studio,
Atti Didamatica 2014 – Nuovi Processi e Paradigmi per la
Recognition, think: the Signature Verification System, Didattica, pp. 128-137, Napoli, Italy, AICA, 2014.
the face recognition, the recognition of the palm or [14] P. Campanella, Tecniche di Benchmark Learning User Based,
fingers of the hand, to biometrics, and so on. Note that Atti Didamatica 2014 – Nuovi Processi e Paradigmi per la
we have uploaded an image into an array and within Didattica, pp. 148-157, Napoli, Italy, AICA, 2014.
[15] P. Campanella, Piattaforme per l’Uso Integrato di Risorse
that we prepared in accordance with the requirements, Formative nei Processi di e-learning, Atti Didamatica 2015 –
so nothing prevents you from using the same approach Studio Ergo Lavoro – dalla società della conoscenza alla
for other various ASM processing of images. I think, società delle competenze, pp. 237-244, Genova, Italy, AICA,
for example, a satellite tracking system showing a 2015.
[16] P. Campanella, Piattaforme proprietarie: Un’analisi
picture of a situation to go on to perform: data metodologica, Atti Didamatica 2015 – Studio Ergo Lavoro –
extraction, analysis, processing, measurement, dalla società della conoscenza alla società delle competenze,
mapping and so on. pp. 757-758, Genova, Italy, AICA, 2015.
[17] P. Campanella, Flipped Teaching: Un Caso di Studio, Atti
Didamatica 2020 – Smarter School for Smart Cities, pp. 224-
232, Trieste, Italy, AICA, 2020.
[18] P. Campanella, “The Flipped Learning Model: A New
REFERENCES Paradigm”, In: 32ma International Conference SITE 2021,
AACE Publish, San Diego, California, pp. 1170-1175, 2021.
[1] P. Beccacece, E-learning: la scelta di un Learning [19] P. Campanella, Method of experimental evaluation of ICT in
Management System opensource e la creazione di pacchetti teaching, Atti del convegno Elearn 2011, World conference on
SCORM, University of Bologna, Tek Tank, 2005. E-learning in Corporate Governement, Healthcare e Higher
[2] P. Campanella, “Mobile Learning: New forms of education”, Education organized by AACE, 17-18-19-20-21/10/2011,
Proc. of the 10th International Conference on Emerging e- Honolulu, Hawaii, USA.
Learning Technologies and Applications (ICETA 2012), IEEE, [20] P. Campanella, “Neuro-Fuzzy Learning in Context Educative”,
Stará Lesná, The High Tatras, Slovakia, pp. 51-56, 08- Proc. of the 19th International Conference on Emerging e-
09/11/2012. Learning Technologies and Applications, ICETA 2021, Virtual
[3] P. Campanella, Content Management System Open Source: Conference – Technical University of Košice, IEEE, Slovakia,
Un’analisi comparativa, Atti Didamatica 2011, Informatica per pp. 58-69, 2021.
la didattica, Torino, Italy, AICA, 2011. [21] D. Colombo, Formazione a distanza – ambienti e piattaforme
[4] P.rd Campanella, “FAD Platforms: Proprietary Solutions”, In: telematiche a confronto, 2001.
3 International Conference on Computing Technology and [22] V. Di Lecce, A. Giove, M. Calabrese, “E-Learning Platform for
Information Management (ICCTIM2017), SDIWC, University Guidance”, In Proceedings of the International
Metropolitan College, Thessaloniki, Greece, pp. 15–20, 2017. Conference on Computer Aided Learning (ICL 08), Villach,
[5] P. Campanella, “Proprietary Platforms for FAD: A Austria, 2008.
Comparative Analysis”, IJEETDM, SDIWC, vol. 3, n. 4, [23] C. Donati, W. Marabini, D. Guidi, La scelta di un Learning
pp.153-160, Hong Kong, 2017. Management System OpenSource: Atutor vs Ilias vs Moodle,
[6] P. Campanella, S. Impedovo, IAPR Fellow, IEEE S. M., University of Bologna, Tek Tank, 2004.
“Innovative Methods for the E-learning Recommendation”, In: [24] G. Farrell, COL LMS Open Source, Vancouver:
5th International Conference on Digital Information Processing Commonwealth of Learning (COL), 2003.
and Communications (ICDIPC2015), IEEE, Sierre, [25] S. Fraccavento, L'e-learning inteso come fenomeno sociale e di
Switzerland, pp. 312-317, 2015. mercato (Le differenti piattaforme e tipologie di
[7] P. Campanella, S. Impedovo, IAPR Fellow, IEEE S. M. apprendimento), 2003.
“Techniques for Benchmark Learning User–Based”, [26] D. F. Garcia, C. Uria, J. C. Granda, F. J. Suarez, F. Gonzalez,
International Conference on Information System Security, “A Functional Evaluation of the Commercial Platforms and
Robotics Modeling, and E-Commerce Transactions Tools for Synchronous Distance e-Learning”, International
(ISSRMET2015), Islamic Azad University, Academic City, Journal of Education and Information Technologies, vol.1, n.2,
Dubai, UAE, 2015. pp. 95-104, 2007.
[8] P. Campanella, “A Comparative Assessment of e-Learning [27] S. Impedovo, P. Campanella, G. Facchini, G. Pirlo, R.
Platforms”, In: 2thInternational Conference on Digital Modugno, L. Sarcinella, Learning Management Systems:
Information Processing, Data Mining, and Wireless un’analisi comparativa delle piattaforme open-source e
Communications (DIPDMWC2015), Islamic Azad University, proprietarie, Atti Didamatica 2011 - Informatica per la
Academic City, Dubai, UAE, pp. 193-198, 2015. didattica, Torino, Italy, AICA, 2011.
[9] P. Campanella, “Learning Management Systems: A [28] M. Yamada, Y. Goda, H. Matsukawa, K. Hata and S.
comparative analysis of open-source and proprietary Yasunami, “A Computer-Supported Collaborative Learning
platforms”, In: 2thInternational Conference on Digital Design for Quality Interaction”, IEEE Multimedia, vol. 23, pp.
Information Processing, Data Mining, and Wireless 48-59, 2016.
Communications (DIPDMWC2015), Islamic Azad University, [29] S. Luciani, Caratteristiche tecniche e funzionalità didattiche
Academic City, Dubai, UAE, pp. 187-192, 2015. delle piattaforme per l’apprendimento on-line, 2005.
[10] P. Campanella, “Platforms for Use Integrated Resources [30] V. Mobilio, “Sperimentare piattaforme Open Source: idee per
Formative Processes in E-learning”, In: 2thInternational una metodologia d’analisi”, In eLearning & Knowledge
Conference on Digital Information Processing, Data Mining, Management, 2006.
and Wireless Communications (DIPDMWC2015), Islamic Azad [31] M. Ouadoud, K. Eddine and E. Kadiri, “Studying and
University, Academic City, Dubai, UAE, pp. 181-186, 2015. Comparing the Free E-learning Platforms”, In 4th International
[11] P. Campanella, “Platforms and methods for the integrated use Colloquium on Information Science and Technology (CiSt),
of educational resources in the processes of e-learning”, In: T., IEEE, pp. 581-586, 2016.
Bastiaens & M., Ebner (Eds.), Proc. of the World Conference [32] M. Pedroni, Dall’interoperabilità delle piattaforme
on Educational Multimedia, Hypermedia and all’integrabilità dei moduli interattivi, Atti Didamatica 2004,
Telecommunications 2011, Chesapeake, Lisbona, Portogallo, pp.731-735, Omniacom Editore, AICA, 2004.
AACE, pp. 2375-2384, 2011. [33] P. Robinson, J. Carroll, “An online learning platform for
[12] P. Campanella, NetLearn2.0: Piattaforma e-learning e teaching, learning, and assessment of programming”, In
metodologie integrative, Atti Didamatica 2013 –Tecnologie e Proceedings of Global Engineering Education Conference
Metodi per la Didattica del Futuro, pp. 999-1008, Pisa, Italy, (EDUCON), IEEE, Athens, Greece, 2017.
AICA, 2013. [34] M. Vercellotti. Griglia di valutazione delle piattaforme e-
learning open source, 2005.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 90

Exploring the Capabilities and Possible Applications
of Large Language Models for Education
Matúš Čavojský∗ , Gabriel Bugár† , Tomáš Kormanı́k‡ , Martin Hasin§
∗† Departmentof Electronics and Multimedia Telecommunications
ठDepartment
of Computers and Informatics
Faculty of Electrical Engineering and Informatics
Technical University of Košice, Letná 9, Košice, Slovakia
matus.cavojsky@tuke.sk∗ , gabriel.bugar@tuke.sk† , tomas.kormanik@tuke.sk‡ , martin.hasin@tuke.sk§

Abstract—This research looks into the possible applications summarization, and even creative writing, demonstrating a
of large language models (LLMs) in education, with a special major improvement in large language models’ capabilities [3].
emphasis on ChatGPT, an OpenAI-developed chat bot model. It When ChatGPT was launched on November 30, 2022, it was
demonstrates the usefulness of ChatGPT as a learning aid and
research assistant while addressing ethical and pedagogical con- considered an unprecedented technological revolution in NLP.
cerns such as cognitive offloading, academic integrity, and critical The chat bot model was developed by OpenAI and attracted
thinking. The study covers a wide range of educational settings, over one million users in just five days, causing site issues,
including content development, learning a foreign language, and generating widespread attention and discussion globally
problem solving, and literature review assistance. Furthermore, [4]. In February 2023, Microsoft unveiled the ChatGPT search
it presents ZeroGPT, a plagiarism detection tool for recognizing
GPT-generated content and summarizes the findings to highlight engine in the Edge browser, as a component of extensive AI-
LLMs’ ability to improve educational experiences. driven enhancements made to both its Bing search engine and
Index Terms—Educational Digital Transformation, Feynman Edge browser. This update allowed users to conduct online
Technique, Large Language Models, ChatGPT, Integration of searches utilizing the chat bot’s input, made possible by the
Educational Technology, Prompt Engeneering incorporation of OpenAI’s GPT-4 model into Bing, resulting
in a ChatGPT-like search experience. Bard is a large language
I. I NTRODUCTION model from Google AI, trained on a massive dataset of text and
code. It was released to the public on March 21th, 2023, and
Chatbots based on Artificial Intelligence-Generated Content can generate text, translate languages, write different kinds of
(AIGC) technology have a long history dating back to Eliza creative content, and answer questions in an informative way.
in the 1960s, which demonstrated the potential of text-based Google Bard’s core UI offers three answer options after every
human-computer interactions despite its simplicity by today’s question, whereas ChatGPT generates responses on-demand.
standards. Natural language processing (NLP) has undergone Regarding the response accuracy, Bard can sometimes provide
a dramatic transformation in recent decades as a result of the truth-looking but misleading information, especially regarding
development of large language models (LLMs). These models, specific dates that are readily available on the internet. The
which can learn directly from raw text, have revolutionized the rapid advancement of these chatbots became critical for effi-
field by allowing a deeper understanding of complex language cient human-computer communication.
structures and nuances that were previously difficult to model. This conference paper focuses on demonstrating ChatGPT’s
More recently, the introduction of the Bidirectional Encoder utility as a learning aid and research assistant. We showcase
Representations from Transformers (BERT) model in 2018 [1] its capabilities as an interactive conversational partner that
marked a significant milestone, establishing new benchmarks facilitates the application of the Feynman Technique, enhanc-
in various NLP tasks. BERT, which was built on Vaswani ing learners’ comprehension of complex concepts. Various
et al.’s [2] transformer architecture, represented a paradigm educational use cases of LLMs are presented, encompassing
shift in NLP by replacing traditional recurrent neural networks tasks like generating summaries, questions, tests, essays, and
(RNNs) with a self-attention mechanism, enabling parallel feedback across different educational levels. Additionally, the
computation of sequences and expanding the model’s ability to ethical and societal implications of these models, including
capture long-range dependencies and contextual understand- concerns related to privacy, bias, and transparency, are ana-
ing. BERT’s success encouraged further research into larger lyzed. We delve into the challenges and risks of employing
and more sophisticated models. GPT-3, the third iteration of LLMs for academic purposes, including plagiarism, cheating,
OpenAI’s Generative Pre-trained Transformer (GPT) model, and cognitive offloading, introducing ZeroGPT, a plagiarism
was released in 2020, boasting an impressive 175 billion detection tool for identifying LLM-generated text. The con-
parameters [3], making it the largest language model of its clusion summarizes our experiments with chat bots, evalu-
time. GPT-3 demonstrated remarkable proficiency across a ates their response generation and use cases across multiple
range of language-related tasks, such as question answering, prompts, and discusses the implications of our findings.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 91

 II. R ELATED WORK academic and professional exams in different disciplines, in-
cluding law, medicine, and business [9]. Roivainen [10] in his
The integration of LLMs into educational settings has wit-
article presented exact questions from the IQ test to ChatGPT
nessed a remarkable surge in interest in recent years. Table I by
and estimated its Verbal IQ to be 155, which puts it in the
the Yu [4] compares the significant advantages of ChatGPT’s
top 0.1% of 2,450 people. Gilson et al. [11] show that the
advancements to older chat bot models used for educational
ChatGPT model achieves the equivalent of a passing score
applications.
for a third-year medical student by performing at a greater
than 60% threshold on the NBME-Free-Step-1 (64.4%) and
TABLE I
C OMPARISON OF C HAT GPT TO TRADITIONAL CHAT BOTS FOR 57.8% NBME-Free-Step-2 data sets. As a reaction to such
EDUCATIONAL APPLICATIONS [4] excellent performance, universities and also K-12 schools have
frequently resorted to banning the use of ChatGPT. According
Traditional educational ChatGPT to Lung et al. [6] in an attempt to prevent the use of AI-
chat robots generated assignments, the New York City Department of Ed-
Search mode Keyword based Based on large-scale
ucation passed a rule in January 2023 that prohibited students
retrieval corpus learning from using such tools to conduct plagiarism. RV University in
Bangalore, India has implemented stringent measures which
Response quality Answering questions Similar manual feed-
mechanically back
explicitly forbid students from employing Chat GPT to fulfill
assignments, engage in examinations, or conduct laboratory
Answer scope Limited answerable Significantly tests. Following suit, the Australian state of New South Wales
questions expanding the scope of was the first area to place limitations on students’ use of
answerable questions
ChatGPT, after which measures were implemented in public
Understanding level Context understanding Ability to understand schools in Queensland, Tasmania, and Western Australia with
not supported context a similar goal in mind. Flinders University was among the first
Iterative ability Cannot iterate based Ability to iteratively Australian institutions to implement a policy against computer-
on user feedback optimize based on user generated cheating, which included the unauthorized use of AI
feedback tools such as ChatGPT, Bard, or DALL-E without authoriza-
tion from instructors as well as proper attribution or citations.
As stated in an OpenAI’s overview [5], ChatGPT is a power- Moreover, Australian universities announced the return of
ful language generation model, but it has some limitations. The closed book pen-and-paper exams and a new emphasis of in-
most significant one is that it can only generate text based on class assessment writing [7].
the input given to it; it has no access to external information or
the ability to browse the internet. This means it cannot provide III. L EARNING USING F EYNMAN T ECHNIQUE
accurate or up-to-date information on a wide range of recent The Feynman Technique is a cognitive learning strategy
topics, and it may be unable to generate responses to complex developed by physicist Richard Feynman. It aims to enhance
or unusual questions. The most significant difference between understanding and retention of knowledge through active en-
ChatGPT and Bing Chat/Google Bard lies in the latter two’s gagement and self-explanation. The technique involves a four-
access to internet resources. Access to the internet enables step process.
Bing Chat and Google Bard to be aware of current events 1) Based on knowledge gained from prior experience,
that occur after September 2021, which are not covered by research or lectures, writing down all the information
ChatGPT’s knowledge base. The differences between these about the selected or focused topic or concept.
tools (ChatGPT, Bing Chat, Google Bard) and their respective 2) Using simple language, explaining the concept as if
advantages can be applied in a variety of ways: Conducting the learner was teaching it to someone with no prior
literature review, data and article analysis and other ways knowledge of the subject.
not researched by us like Language translation and Question 3) While teaching the concept, attention should be directed
answering [6]. Rudolph et al. [7] used questions from a wide towards areas where struggles arise in explaining or
variety of academic disciplines to assess the models, including delving deeper into the subject matter. This makes it
a non-trivial inquiry: ”How can one combine eight instances of easier to identify gaps in understanding, indicating the
the digit 8 to obtain the value 1000?”. Surprisingly, almost all need for further research and clarification.
chatbots solved this problem successfully, with the exception 4) The final step is to revisit the learning materials; it is
of Bard, which humorously denied the possibility of reaching critical to thoroughly review the topic, paying special
1000 through addition. attention to areas where difficulties were encountered in
Moreover, it is becoming increasingly difficult to distinguish explaining. The process involves simplifying explana-
whether a text is machine-generated or human-generated, pre- tions, employing analogies, and establishing connections
senting an additional major challenge to teachers and educators with other relevant concepts to facilitate a deeper under-
[8]. Recently newly released version of ChatGPT, GPT-4, has standing, with steps 2-4 forming an iterative or looping
shown human-level performance on the majority of multiple sequence.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 92

 In a recent study on Feynman Technique as a Heutagogical fering contextual information, detailing the task at hand,
Learning Strategy for Independent and Remote Learning [12] and specifying the preferred presentation format.
researchers used a true experimental design with pretest and Prompt: For an upcoming assessment from computer
posttest to compare the learning outcomes of students who networking, create a multiple question multiple answer
applied the Feynman Technique (experimental group) and test with bulletpoints. Mark correct answers.
those who did not (control group) in three grade levels (4, 7, • Other, alternative approaches, to prompts can enhance
and 11). The results showed that the experimental group had the quality of outputs by providing detailed step-by-
significantly higher posttest scores and learning gains than the step instructions, specifying required components, and
control group in all grade levels, indicating that the Feynman employing prompt chaining through separate chat inter-
Technique was effective in improving students’ learning. actions. The prompt chaining can be seen in Table II,
Learners may utilize ChatGPT as an interactive conver- where the first prompt requires the ChatGPT to wait for
sational partner to help them explain complex concepts in second input. This methodology allows for the breakdown
their own words. This pedagogical process assists students of complex ideas into smaller tasks, facilitating better
in clarifying their understanding, identifying knowledge gaps, management of prompts and subsequent outputs.
and consolidating their knowledge. The example of ChatGPT
conversation using Feynman Technique is shown in Table In a recent development, ChatGPT has been updated to
II. ChatGPT correctly recognized a knowledge gap in which enable users to provide custom instructions for the model
the user missed to point out that layer one is in control of to consider in its responses. This update includes the ability
signal encoding/decoding, modulation/demodulation, and bit to use RTF as custom instructions, eliminating the necessity
synchronization. The prompt required the use of uppercase of manually writing RTF instructions for each prompt. Self-
”explain to YOU every” and the sentence ”Wait for my input.” directed learning methodologies have proven to be valuable
at the end, as without them, the model started explaining layers [13] during the response to the COVID-19 pandemic by
1-6 instead of waiting for next input. Learners can also use the empowering individuals to take control of their education in
interactive nature of ChatGPT to seek clarification and pose remote and flexible learning environments. This reflects the
probing questions. Engaging in a conversation with ChatGPT growing interest in large language models in the field of edu-
allows learners to obtain additional explanations and gain cation, where they have the potential to significantly enhance
additional insights that improve their overall understanding learning and teaching experiences across various educational
of the subject matter. ChatGPT can also be used for self- levels, from primary and secondary school to higher education
reflection and review by students. By going back and carefully and professional development environments.
examining previous conversations and the responses provided A. Elementary school students
by ChatGPT, learners can identify areas for improvement and
enhance their understanding of the subject. Large language models provide excellent assistance to stu-
dents in primary school in developing their reading and writing
IV. U SE OF LARGE LANGUAGE MODELS IN EDUCATION skills, including activities like identifying syntactic and gram-
When asking a general question without the use of frame- matical corrections. Furthermore, they help to improve writing
work, the ChatGPT responds in a structured answer, in the style and critical thinking skills. These methods encourage
first sentence repeating the question with slight use of syn- students to engage in analytical assessment of their reading and
onyms. Next, the core of the answer is written where most writing materials by producing thought-provoking questions
of the knowledge and typically the answer lies in. Finally and feedback. Furthermore, they assist in the development of
the summary layer is printed where the ChatGPT summarizes reading comprehension abilities by providing students with
it’s core answer in a two to three sentences. In the context brief overviews and elucidations regarding complicated texts,
of AI writing, it is widely acknowledged that the quality boosting their ability to comprehend and digest challenging
of prompts significantly influences the quality of generated subject matter.
outputs, which is also known as prompt engineering. As • Prompt: Check my paragraph for spelling, grammar, and
such, several frameworks have been developed to enhance the punctuation mistakes.
effectiveness of prompts. • Prompt: Provide basic information on the history of
• RTF (Role, Task, Format): An effective framework for steam machines and sources for further reading.
improving AI-generated outputs involves delineating the • Prompt: Suggest arguments for a debate on the merits
AI’s role, providing a clear outline of the task, and speci- and drawbacks of climate change.
fying the desired format. This approach has demonstrated
notable success in enhancing the quality of generated B. Middle and high school students
content. For middle and high school students, large language models
Prompt: As a university student, write an essay about are useful resources because they assist with language learning
large language models in a summary. and proficiency in a wide variety of courses, including math,
• CTF (Context, Task, Format): A framework that, while physics, language, and literature. Despite some limitations,
less effective, still yields significant results involves of- such as ChatGPT’s occasional issues with basic mathematical

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 93

 TABLE II
A N EXAMPLE OF USING F EYNMAN T ECHNIQUE WITH C HAT GPT 3.5

Using Feynman Technique, I will explain to YOU every layer of model OSI/ISO used in networking. Correct my mistakes in the explanation and suggest
me topics that I missed. Wait for my input.

Of course! I’d be happy to help you explain the layers of the OSI/ISO model and correct any mistakes in your explanation. Please go ahead and explain
each layer, and I’ll provide feedback and suggest any missing topics.

Start with Layer 1 (the Physical Layer) and work your way up to Layer 7 (the Application Layer). Whenever you’re ready, describe each layer’s purpose
and functions.

Physical layer consists of digital and analog signals sent thru the channel (be it a fibre-cable, metalic or wireless). This layer defines hardware equipment,
frequencies and pulses used to represent binary signals. A network disruption can be caused by e.g. a wire cut.
Data link layer is used to communicate via MAC addresses between devices, sending and receiving frames. Is responsible for error detection and
correction using for example a parity bit. Layer 2 can transfer data reliably and unreliably.
Network layer, also known as L3, uses IP addressing to forward network packets to their destination using one or more networks. Typical protocols are
IPv4 and IPv6.
Transport layer is mostly known for its TCP and UDP suites. L4 is responsible for connection-oriented communication. Transport layer uses port numbers
to establish and keep connection relations. Some of the well-known ports are 0 through 1023 which define common services.
Session layer is responsible for keeping and orderly release of connections. In case of a connection loss this protocol may try to recover the connection.
Presentation layer, also called syntax layer, is mainly for encryption and decryption of traffic and ensures the information that the application layer of
one system sends out is readable by the application layer of another system.
Application layer is used by programs to communicate on the internet. They use protocols such as HTTP, FTP or SMTP to deliver information to remote
systems.

You’ve provided a decent overview of the OSI/ISO model layers, but there are some inaccuracies and omissions in your explanation. Let’s correct those
and provide some additional details:
1) Physical Layer:
• You correctly mention that this layer deals with digital and analog signals.
• It defines not only hardware equipment, frequencies, and pulses but also the physical media (e.g., copper cables, fiber-optic cables, wireless
radio waves).
• Mention that this layer is responsible for signal encoding/decoding, modulation/demodulation, and bit synchronization.
• Network disruptions can indeed be caused by issues like wire cuts, interference, or signal degradation.
2) Data Link Layer:
• You’re right that it communicates using MAC (Media Access Control) addresses.
• It’s responsible for framing data into frames, error detection (often using CRC checks), and flow control.
• Mention that it ensures data integrity within a single link or hop.
• Emphasize that it operates on both reliable (e.g., Ethernet) and unreliable (e.g., Wi-Fi) networks.
3) Network Layer:
• You mention IP addressing, but clarify that it handles logical addressing, such as IPv4 and IPv6.
• It’s responsible for routing packets between different networks and subnets.
• Explain the concept of routers, which are devices that operate at this layer and make forwarding decisions based on the destination IP address.
4) Transport Layer:
• Mention that it provides end-to-end communication and is responsible for end-to-end error recovery.
• TCP provides reliable, connection-oriented communication, whereas UDP offers unreliable, connectionless communication.
• Clarify that it uses port numbers to identify specific services on a device.
• Mention that it also handles flow control and segmentation/reassembly of data.
5) Session Layer:
• Describe its role in establishing, maintaining, and terminating sessions between applications.
• Mention that it helps with checkpointing and recovery in case of a connection loss.
6) Presentation Layer:
• Mention that it is responsible for data translation, encryption/decryption, and data compression.
• Explain that it ensures that data sent by the application on one end can be properly understood by the application on the other end.
7) Application Layer:
• Highlight that this is the topmost layer and is where user applications and services directly interact.
• Give examples of common protocols used at this layer, such as HTTP for web browsing, FTP for file transfer, SMTP for email, and many
others.
• Mention that these protocols define how data is formatted, presented, and exchanged between applications.
These corrections and additions should provide a more comprehensive understanding of the OSI/ISO model and its layers. If you have any further questions
or need additional details, feel free to ask!

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 94

queries, these models excel at providing detailed step-by- and paper. If more specific input is given, the request and
step explanations, developing student’s abilities to solve prob- searches can be tailored to meet particular needs, making the
lems, providing detailed explanations, and interesting follow- bot accurate to a greater extent.
up questions. By using these models to create personalized
practice questions and tests, students can improve their un-
derstanding, contextualization, and memory of the material.
This approach fosters an understanding of the underlying rea-
soning behind solutions and cultivates analytical and creative
thinking, as it guides users without immediately revealing the
answers. An example of a practise test made by ChatGPT for
computer network student is shown in the Table III.
• Prompt: Suggest how to rework this short story (para-
graph) for clarity and coherence.
• Prompt: Identify key physicists in space exploration,
summarize their theories.
• Prompt: Analyze my essay for weak points, propose
alternative views, make a review.

C. Higher education students

ChatGPT requires users to input data in plain text format
within the chat interface, which may present challenges when
dealing with data that is not inherently structured for such
an interface, such as tabular data. In contrast, Google Bard
utilizes links to access data, facilitating easier retrieval and
summarization of scientific papers. This approach allows for
more comprehensive summaries than abstracts and enables
the explanation of specific points within the article. Our
experimentation with Bing Chat revealed the capability to Fig. 1. A proposal of interactive phishing training.
request in-depth analyses of the current browser tab, including
assessments of the approach’s advantages and disadvantages,
reviewer-like reviews, and potential future research directions. D. Professional development using phishing security trainings
Importantly, these inquiries can be posed for any open browser The utilization of the chat system through an application
window, expanding the scope of analysis beyond traditional programming interface integration offers the opportunity for
text-based inputs. ChatGPT can help to research a topic by interactive engagement of email service users with examples
generating article summaries or generating a list of relevant of phishing emails. With the growing prevalence of phishing
sources based on a specified topic or keyword as stated in attacks, organizations invest significant resources in training
Lund et al. [6]. However, our research indicates that ChatGPT users to mitigate the risks associated with such fraudulent
sometimes generates inaccurate references, unlike Bing Chat emails. Although machine learning-based tools can effectively
and Google Bard, which, due to internet connectivity, can detect anomalies in emails, they have thus far lacked the
perform comprehensive research, offer proper citations, and capability to effectively communicate these findings to users.
provide reliable academic references when requested. By leveraging the communication and explanatory capabili-
Universities should make clear to students the types and ex- ties of ChatGPT, the proposed approach holds potential to
tent of cognitive offloading permitted for specific assessments, enhance security measures and automate the evaluation of
with AI increasingly incorporated into higher level tasks. employee susceptibility to phishing attacks. In this study, an
• Prompt: This paragraph is a rough outline for my es- email sourced from the official website of CSIRT (Computer
say on windmills. Refine my arguments, restructure the Security Incident Response Team Slovakia) was provided as
paragraphs, and generate a draft of the essay. input to the Bing Chat tool, prompting an analysis of the
• Prompt: Develop a research question and hypothesis reasons underlying its fraudulent nature directly from the open
about the correlation of security and ease of use. website.
• Prompt: Propose novel ideas, arguments, theories for my
thesis on security of email agents. Prompt: Is the email on this web page legitimate or
• Prompt: Find relevant research papers for a specific
phishing? Explain why.
topic. Bing Chat effectively identified the provided email as a
An example of fully automatic search with Bing chat bot is phishing attack and proceeded to provide an explanatory
shown in Figure 1. Each link is redirecting to correct website analysis of its typical characteristics and objectives. Further-

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 95

 TABLE III
A N EXAMPLE OF A COMPUTER NETWORKING PRACTISE TEST GENERATED BY THE C HAT GPT 3.5

Create a test for computer networking student with 3 questions each being multiple-answer (from 1 to all correct) type from EIGRP Cisco course. Do
not write answers, grade my next answer.

Certainly, here’s a test with three multiple-answer questions related to the Enhanced Interior Gateway Routing Protocol (EIGRP) in a Cisco networking
course. Please provide your answers, and I will grade them.

Question 1: Which statements about EIGRP (Enhanced Interior Gateway Routing Protocol) are true? (Select all that apply)
1) EIGRP is a distance-vector routing protocol.
2) EIGRP uses the DUAL (Diffusing Update Algorithm) to calculate the best path.
3) EIGRP supports both IPv4 and IPv6 routing.
4) EIGRP uses hop count as its metric by default.
5) EIGRP exchanges periodic full routing table updates.
Question 2: What are the main benefits of EIGRP? (Select all that apply)
1) Rapid convergence
2) Support for VLSM (Variable Length Subnet Masking)
3) Scalability through route summarization
4) Compatibility with OSPF (Open Shortest Path First)
5) Native support for IPv6
Question 3: In EIGRP, what is the purpose of the ”hello” packets? (Select all that apply)
1) To establish and maintain neighbor adjacencies.
2) To advertise network reachability information.
3) To calculate the composite metric for route selection.
4) To detect and recover from link failures.
Please provide your answers for these questions, and I’ll grade them accordingly.

Question 1 - A, B, C

Question 2 - A, B, C

Question 3 - A, D

Let’s grade your answers:

Question 1: Correct answers: B, C

• B is correct: EIGRP uses the DUAL (Diffusing Update Algorithm) to calculate the best path.
• C is correct: EIGRP supports both IPv4 and IPv6 routing.
Question 2: Correct answers: A, B, C
• A is correct: EIGRP offers rapid convergence.
• B is correct: EIGRP supports VLSM (Variable Length Subnet Masking).
• C is correct: EIGRP allows scalability through route summarization.
Question 3: Correct answers: A, D
• A is correct: ”Hello” packets in EIGRP are used to establish and maintain neighbor adjacencies.
• D is correct: ”Hello” packets also play a role in detecting and recovering from link failures.
You got all the correct answers for these questions! Well done. If you have any more questions or need further clarification, feel free to ask.

Why in Question 1 the A answer was not correct?

In Question 1, option A (”EIGRP is a distance-vector routing protocol”) is not correct because EIGRP is not a pure distance-vector routing protocol. It’s
considered an advanced distance-vector routing protocol or a hybrid routing protocol.

EIGRP does incorporate some features typically associated with distance-vector protocols, such as the use of distance (metric) for route selection, but it
also includes features commonly found in link-state protocols, like maintaining a topology table.

The main reason EIGRP is not categorized as a pure distance-vector protocol is that it uses the DUAL (Diffusing Update Algorithm) to calculate the best
path, which is a significant departure from the simple distance-vector algorithms like RIP. The DUAL algorithm allows EIGRP to converge quickly and
provide loop-free paths, making it more advanced than traditional distance-vector protocols.

So, while EIGRP shares some characteristics with distance-vector protocols, it’s more accurately described as an advanced or hybrid routing protocol.
That’s why option A is not considered correct in this context.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 96

more, the system provided a comprehensive breakdown of the
indicators that signal a phishing attack within the given email:
1) The sender’s address is not a valid domain name for
Microsoft Outlook, but for gmail.sk, which is suspicious.
The correct domain name should be “outlook.com” or
“microsoft.com”.
2) The email claims that the recipient has received a voice
mail, but the attachment is a zip file, not a wav file. This
could be a way to hide malware or ransomware.
3) The link to download the attachment is not from out-
look.com, but from outlook.gmail.sk, which is also
suspicious and could lead to a fake or compromised
website.
4) The email does not address the recipient by name, but
by email address, which is impersonal and generic.
5) The email does not provide any details about the caller,
Fig. 2. A proposal of interactive phishing training.
the message, or the reason for the voice mail, which is
vague and unprofessional.
The initial indicator of a phishing email was accurately
et al. [14] define ”Cognitive Offloading” as any physical
identified, revealing that the email purportedly originated
action to alter the information processing requirements of a
from the Miscorsoft Outlook system as a system message
task to reduce cognitive demand, which can be as simple as
to the email box user, while the actual sender was ”no-
writing something down to avoid the need to remember it later.
reply@gmail.sk”. This scenario assumes the legitimacy of the
While these models can provide vast amounts of information
gmail.sk domain as an email service provider. Changing the
quickly, they may inadvertently encourage users to rely too
Bing Chat setting from ”Precise” to ”Balanced” resulted in
heavily on them for cognitive tasks. This overreliance can
an erroneous classification of the recipient’s email address,
lead to a passive consumption of information without the
”chucknorris@gmail.sk,” as an attack targeting a larger user
engagement of critical thinking and problem-solving skills.
population. Consequently, Bing Chat was solely utilized in
Students in educational environments, for example, may be
”Precise” mode.
The second significant indicator correctly flagged the chat tempted to use these models as a shortcut for learning,
tool and subsequently marked the attachment as a zip file, lowering their active cognitive involvement in the learning
resembling a voice mail. The third indicator of a suspicious process. Therefore, understanding the impact of LLMs on
email was the domain to which the link directed, as a legiti- cognitive offloading and developing strategies to balance their
mate system email should not point to another domain such as use with active cognitive engagement is essential to maximize
outlook.gmail.sk, which may indicate a fake or compromised their benefits while mitigating excessive dependence by using
website. The final two indicators, namely the absence of the anti-GPT tools where student’s critical thinking or problem-
email address and contextual message details, were deemed solving is required.
less significant since it is assumed that a genuine system- ZeroGPT [15] is a plagiarism detection tool that differ-
generated email would not contain such information. However, entiates content created by AI tools from human-authored
a legitimate email might include additional specifics and the text, using perplexity as a metric to assess the generalization
purpose of the message. capabilities of the text’s creator, which can be particularly
Utilizing keywords, the large language model can serve as beneficial for university teachers and educators. To test the
an effective detector and explainer of phishing emails. This accuracy of ZeroGPT, we used the RTF (Role, Task, Format)
capability allows for the identification and highlighting of framework to reduce the likelihood of ChatGPT’s test being
specific keywords, enabling users to comprehend the ratio- detected by the ZeroGPT. However, the essay about proactive
nale behind marking an email as illegitimate. Consequently, and reactive cybersecurity received a score of 62.22% nev-
users can assess the email’s trustworthiness based on the ertheless, as shown in Figure 3. Moreover, the detection of
provided output and offer feedback, thereby facilitating the text translated into foreign language but generated by LLMs
enhancement of the detection and reasoning of the model. An poses a significant challenge for anti-GPT tools, reducing
example of the output transformed to the proposed approach the probability of detection even more. These tools have
is illustrated in Figure 2. the primary goal to identify and flag content generated in
the source language, and they frequently fail to differentiate
V. D ISTINGUISHING BETWEEN TEXT GENERATED BY THE machine-generated translations from human-authored text. The
LLM AND STUDENT reason is that large language models have high linguistic
The proliferation of large language models poses a potential proficiency, allowing them to produce translations that closely
threat in the context of excessive cognitive offloading. Risko resemble human-written content in terms of fluency, grammar,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 97

and context. For example, a text generated by ChatGPT information. Learners should critically evaluate the responses
is translated into Slovak using an online translator, and if provided by ChatGPT and seek additional resources or expert
ZeroGPT detects a significant match, a paraphraser tool can guidance (preferably from humans) when necessary.
be used to reduce the likelihood of raising a suspicion.
ACKNOWLEDGMENT
This work was supported by the Ministry of Education,
Science, Research and Sport of the Slovak Republic, and the
Slovak Academy of Sciences under Grant VEGA 1/0685/23
and by the Slovak Research and Development Agency under
Grant APVV SK-CZ-RD-21-0028.
R EFERENCES
[1] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training
of deep bidirectional transformers for language understanding,” in
Proceedings of the 2019 Conference of the North American Chapter
of the Association for Computational Linguistics: Human Language
Technologies, Volume 1 (Long and Short Papers). Minneapolis,
Minnesota: Association for Computational Linguistics, jun 2019, pp.
4171–4186. [Online]. Available: https://aclanthology.org/N19-1423
[2] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez,
L. Kaiser, and I. Polosukhin, “Attention is all you need,” 2023.
[3] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal,
A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-
Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler,
J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray,
B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever,
and D. Amodei, “Language models are few-shot learners,” 2020.
[4] H. Yu, “Reflection on whether chat gpt should be banned
by academia from the perspective of education and teaching,”
Frontiers in Psychology, vol. 14, 2023. [Online]. Available:
https://www.frontiersin.org/articles/10.3389/fpsyg.2023.1181712
[5] J. Deng and Y. Lin, “The benefits and challenges of chatgpt:
Fig. 3. An essay about proactive and reactive cybersecurity in a scientfic An overview,” Frontiers in Computing and Intelligent Systems,
researcher words inspected by ZeroGPT. vol. 2, no. 2, p. 81–83, Jan. 2023. [Online]. Available:
https://drpress.org/ojs/index.php/fcis/article/view/4465
[6] B. D. Lund and T. Wang, “Chatting about chatgpt: how may ai and gpt
Despite the precise and relevant information provided by impact academia and libraries?” Library Hi Tech News, vol. 40, no. 3,
large language models, student assessments frequently place pp. 26–29, 2023.
more emphasis on evaluating students’ ability to access and [7] J. Rudolph, S. Tan, and S. Tan, “War of the chatbots: Bard, bing chat,
chatgpt, ernie and beyond. the new ai gold rush and its impact on higher
organize predefined knowledge than on promoting creativity, education,” Journal of Applied Learning and Teaching, vol. 6, no. 1,
critical thinking, and problem-solving abilities. 2023.
[8] D. R. E. Cotton, P. A. Cotton, and J. R. Shipway, “Chatting and
VI. C ONCLUSION cheating: Ensuring academic integrity in the era of chatgpt,” Innovations
in Education and Teaching International, vol. 0, no. 0, pp. 1–12, 2023.
In recent years, with the advent of deep learning techniques, [Online]. Available: https://doi.org/10.1080/14703297.2023.2190148
LLMs have emerged as a powerful tool for NLP. These models [9] OpenAI, “Gpt-4 technical report,” 2023.
can learn directly from raw text, allowing them to capture [10] E. Roivainen, “I gave chatgpt an iq test. here’s
what i discovered,” March 28 2023. [Online]. Avail-
complex language structures and nuances that were previously able: https://www.scientificamerican.com/article/i-gave-chatgpt-an-iq-
difficult to model. The GPT models, in particular, have pushed test-heres-what-i-discovered/
the boundaries of what is possible with LLMs, with the largest [11] A. Gilson, C. W. Safranek, T. Huang, V. Socrates, L. Chi, R. A.
Taylor, and D. Chartash, “How does chatgpt perform on the united
models reaching up to hundreds of billions of parameters. The states medical licensing examination? the implications of large
application of LLMs in education is an interesting field for language models for medical education and knowledge assessment,”
research, offering a wide range of opportunities to enhance JMIR Med Educ, vol. 9, p. e45312, Feb 2023. [Online]. Available:
http://www.ncbi.nlm.nih.gov/pubmed/36753318
students’ educational experiences as well as assist teachers [12] E. P. Reyes, R. M. F. L. Blanco, D. R. L. Doroon, J. L. B. Limana, and
in achieving their objectives. In response to the COVID-19 A. M. A. Torcende, “Feynman technique as a heutagogical learning strat-
pandemic, educational institutions’ paradigm shift toward dis- egy for independent and remote learning,” Recoletos Multidisciplinary
Research Journal, vol. 9, no. 2, pp. 1–13, 2021.
tance learning has highlighted the importance of self-directed [13] L. Gerard, K. Wiley, A. H. Debarger, S. Bichler, A. Bradford, and M. C.
learning methodologies such as ChatGPT along with the Linn, “Self-directed science learning during covid-19 and beyond,”
Feynman Technique. Although LLMs can provide information Journal of science education and technology, pp. 1–14, 2022.
[14] E. F. Risko and S. J. Gilbert, “Cognitive offloading,” Trends in Cognitive
with accuracy and relevance, they cannot replace the essential Sciences, vol. 20, no. 9, pp. 676–688, 2016. [Online]. Available:
human traits of creativity, critical thinking, and problem- https://www.sciencedirect.com/science/article/pii/S1364661316300985
solving, hence teachers should utilize them as supplementary [15] E. Tian, “Zerogpt,” Jan. 2023. [Online]. Available:
https://www.zerogpt.com
tools rather than replacements. While ChatGPT can serve as
a valuable learning aid, it should not be the sole source of

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 98

 7HDFKLQJ6\QFKURQL]DWLRQLQ3DUDOOHO&RPSXWLQJ

0LFKDOýHUĖDQVNê
,QVWLWXWHRI&RPSXWHU7HFKQRORJLHVDQG,QIRUPDWLFV8QLYHUVLW\RI66&\ULODQG0HWKRGLXV7UQDYD6ORYDN5HSXEOLF
PLFKDOFHUQDQVN\#XFPVN

$EVWUDFW²6\QFKURQL]DWLRQ LV DQ LPSRUWDQW FRQFHSW LQ LQWXLWLYH PDNLQJ LW KDUGHU WR FRQYH\ WKHVH FRQFHSWV
SDUDOOHO FRPSXWLQJ 6WXGHQWV DUH XVXDOO\ LQWURGXFHG WR WKLV HIIHFWLYHO\
WRSLFGXULQJWKHLUXQGHUJUDGXDWHVWXGLHVLQFRXUVHVGHYRWHG 6\QFKURQL]DWLRQ LV EHVW OHDUQHG WKURXJK KDQGVRQ
WR RSHUDWLQJ V\VWHPV RU SDUDOOHO FRPSXWLQJ ,Q WKLV SDSHU H[SHULHQFHZKLFKPHDQVVWXGHQWVQHHGWRZULWHDQGGHEXJ
ZH SUHVHQW VRPH DVSHFWV RI WHDFKLQJ D V\QFKURQL]DWLRQ SDUDOOHO SURJUDPV WR IXOO\ JUDVS WKH FRQFHSWV 7KLV
PRGXOH LQ D SDUDOOHO FRPSXWLQJ FRXUVH DQG GLVFXVV WKH SUDFWLFDO DVSHFWRI WHDFKLQJ V\QFKURQL]DWLRQFDQEH WLPH
DGYDQWDJHV RI DVVLJQLQJ VWXGHQWV SUDFWLFDO SURJUDPPLQJ FRQVXPLQJ DQG LQ VRPH VSHFLILF VLWXDWLRQ PD\ UHTXLUH
SUREOHPV LQVSLUHG E\ FODVVLFDO V\QFKURQL]DWLRQ FKDOOHQJHV DFFHVV WR SDUDOOHO FRPSXWLQJ UHVRXUFHV ZKLFK QRW DOO
VXFKDVWKH³'LQLQJ3KLORVRSKHUV´SUREOHP HGXFDWLRQDOLQVWLWXWLRQVKDYHUHDGLO\DYDLODEOH
7R RYHUFRPH DERYHPHQWLRQHG FKDOOHQJHV WHDFKHUV
, ,1752'8&7,21 RIWHQ XVH D FRPELQDWLRQ RI WHDFKLQJ PHWKRGV LQFOXGLQJ
OHFWXUHV SUDFWLFDO SURJUDPPLQJ DVVLJQPHQWV DQG
6\QFKURQL]DWLRQ LV HVVHQWLDO LQ SDUDOOHO FRPSXWLQJ IRU
VLPXODWLRQV 3URYLGLQJ VWXGHQWV ZLWK DPSOH RSSRUWXQLWLHV
SUHVHUYLQJ GDWD LQWHJULW\ SUHYHQWLQJ UDFH FRQGLWLRQV
WR SUDFWLFH V\QFKURQL]DWLRQ LQ D FRQWUROOHG HQYLURQPHQW
PDQDJLQJ UHVRXUFH DFFHVV RSWLPL]LQJ SHUIRUPDQFH DQG
DQG HPSKDVL]LQJ WKH LPSRUWDQFH RI SURSHU
JXDUDQWHHLQJ WKH FRUUHFWQHVV DQG UHOLDELOLW\ RI SDUDOOHO
V\QFKURQL]DWLRQ WHFKQLTXHV FDQ KHOS VWXGHQWV EHWWHU
SURJUDPV3URSHUO\GHVLJQHGV\QFKURQL]DWLRQPHFKDQLVPV
XQGHUVWDQGWKHVHFRQFHSWV
DUHFULWLFDOIRUKDUQHVVLQJWKHIXOOSRWHQWLDORISDUDOOHODQG
FRQFXUUHQW V\VWHPV ZKLOH DYRLGLQJ SLWIDOOV DQG HQVXULQJ
WKDWSURJUDPVSURGXFHDFFXUDWHDQGFRQVLVWHQWUHVXOWV ,,, &/$66,&$/6<1&+521,=$7,21352%/(06
,Q WKH QH[W VHFWLRQ ZH ZLOO H[SORUH WKH FKDOOHQJHV ³7KH /LWWOH %RRN RI 6HPDSKRUHV´ >@ LV DQ H[FHOOHQW
DVVRFLDWHG ZLWK WHDFKLQJ V\QFKURQL]DWLRQ LQ SDUDOOHO UHVRXUFH IRU DQ\RQH ORRNLQJ WR JDLQ D GHHSHU
FRPSXWLQJDQGZHZLOOSURYLGHDEULHIRYHUYLHZRIVRPH XQGHUVWDQGLQJ RI V\QFKURQL]DWLRQ DQG FRQFXUUHQF\ LQ
ZHOONQRZQ FODVVLFDO V\QFKURQL]DWLRQ SUREOHPV VXFK DV FRPSXWHUVFLHQFH,WSURYLGHVFOHDUH[SODQDWLRQVSUDFWLFDO
WKH ³'LQLQJ 3KLORVRSKHUV´ 6XEVHTXHQWO\ ZH ZLOO H[DPSOHVDQGKDQGVRQH[HUFLVHVWKDWKHOSVWXGHQWVJUDVS
LQWURGXFHFRQGLWLRQYDULDEOHDVV\QFKURQL]DWLRQSULPLWLYH WKHLQWULFDFLHVRISDUDOOHOSURJUDPPLQJXVLQJVHPDSKRUHV
ZKLFK ZH UHFRPPHQG IRU SUDFWLFDO DVVLJQPHQWV LQ WKH 7KH ERRN VWDUWV E\ LQWURGXFLQJ WKH FRQFHSW RI
3DUDOOHO &RPSXWLQJ FRXUVH :H ZLOO WKHQ GHOYH LQWR WKH VHPDSKRUHV >@ ZKLFK DUH V\QFKURQL]DWLRQ SULPLWLYHV
V\QFKURQL]DWLRQ PRGXOH ZLWKLQ WKH FRXUVH SURYLGLQJ D XVHG WR FRQWURO DFFHVV WR VKDUHG UHVRXUFHV LQ D PXOWL
PRUH FRPSUHKHQVLYH GHVFULSWLRQ RI WKH V\QFKURQL]DWLRQ WKUHDGHG RU PXOWLSURFHVV HQYLURQPHQW ,W H[SODLQV WKH
WDVNDVVLJQHGWRVWXGHQWVDVDSUDFWLFDODVVLJQPHQW IXQGDPHQWDORSHUDWLRQVRIVHPDSKRUHV>@VXFKDVZDLW
3  DQG VLJQDO 9  DQG KRZ WKH\ FDQ EH XVHG WR VROYH
,, 7($&+,1*6<1&+521,=$7,21 V\QFKURQL]DWLRQSUREOHPV
6WXGHQWV DUH XVXDOO\ LQWURGXFHG WR WKH WRSLFV RI 7KH PDLQ IRFXV RI WKH ERRN LV WR SUHVHQW DQG VROYH D
V\QFKURQL]DWLRQ GXULQJ WKHLU XQGHUJUDGXDWH VWXGLHV LQ YDULHW\RIFODVVLFDOV\QFKURQL]DWLRQSUREOHPV,WSURYLGHV
FRXUVHV GHYRWHG WR RSHUDWLQJ V\VWHPV RU SDUDOOHO GHWDLOHG H[SODQDWLRQV DQG VROXWLRQV IRU SUREOHPV OLNH WKH
FRPSXWLQJ >@ 7HDFKLQJ SDUDOOHO FRPSXWLQJ DQG ³3URGXFHUV&RQVXPHUV´ ³'LQLQJ 3KLORVRSKHUV´
V\QFKURQL]DWLRQLVFKDOOHQJLQJIRUVHYHUDOUHDVRQV>@ ³5HDGHUV:ULWHUV´ DQG PRUH (DFK SUREOHP LV SUHVHQWHG
ZLWK D FOHDU GHVFULSWLRQ FRQVWUDLQWV DQG VROXWLRQV XVLQJ
6\QFKURQL]DWLRQLQYROYHVPDQDJLQJWKHLQWHUDFWLRQVDQG
VHPDSKRUHV7KHVHFODVVLFDOV\QFKURQL]DWLRQSUREOHPVDUH
FRRUGLQDWLRQEHWZHHQPXOWLSOHWKUHDGVRUSURFHVVHVZKLFK
ZHOONQRZQ FKDOOHQJHV LQ WKH ILHOG RI SDUDOOHO DQG
FDQ EH KLJKO\ DEVWUDFW DQG FRPSOH[ 8QGHUVWDQGLQJ WKH
FRQFXUUHQWFRPSXWLQJ7KH\VHUYHDVLPSRUWDQWH[DPSOHV
LQWULFDFLHV RI ORFNV VHPDSKRUHV FRQGLWLRQ YDULDEOHV DQG
WR VWXG\ DQG XQGHUVWDQG V\QFKURQL]DWLRQ WHFKQLTXHV DQG
RWKHU V\QFKURQL]DWLRQ SULPLWLYHV FDQ EH FKDOOHQJLQJ IRU
GHPRQVWUDWH FRPPRQ LVVXHV WKDW FDQ DULVH LQ SDUDOOHO
VWXGHQWVHVSHFLDOO\WKRVHQHZWRSDUDOOHOSURJUDPPLQJ
SURJUDPPLQJ
3DUDOOHO SURJUDPV RIWHQ H[KLELW QRQLQWXLWLYH EHKDYLRU
GXH WR WKH FRQFXUUHQW H[HFXWLRQ RI WKUHDGV RU SURFHVVHV $ 7KH'LQLQJ3KLORVRSKHUV3UREOHP
5DFH FRQGLWLRQV DQG GHDGORFNV ZKLFK DUH FRPPRQ ,Q WKLV SUREOHP WKHUH DUH VHYHUDO SKLORVRSKHUV VLWWLQJ
V\QFKURQL]DWLRQUHODWHG LVVXHV FDQ EH GLIILFXOW WR JUDVS DURXQG D GLQLQJ WDEOH HDFK RI ZKRP DOWHUQDWHV EHWZHHQ
DQG SUHGLFW PDNLQJ LW FKDOOHQJLQJ WR WHDFK VWXGHQWV KRZ WKLQNLQJ DQG HDWLQJ >@ 7R HDW D SKLORVRSKHU PXVW
WRSUHYHQWDQGWURXEOHVKRRWWKHP SLFN XS WZR IRUNV RQH RQ HDFK VLGH RI WKHLU SODWH
6\QFKURQL]DWLRQLVVXHVRIWHQLQYROYHVXEWOHWLPLQJDQG +RZHYHUWKHUHLVDOLPLWHGQXPEHURIIRUNVDYDLODEOHDQG
RUGHURIH[HFXWLRQ SUREOHPV ZKLFK DUH QRW HDVLO\ WKH\ VKDUH WKH IRUN RQ WKHLU VLGH ZLWK WKHLU IHOORZ
YLVXDOL]HG 8QOLNH VRPH SURJUDPPLQJ FRQFHSWV WKDW FDQ SKLORVRSKHU $V D UHVXOW QR WZR SKLORVRSKHUV FDQ HDW
EH GHPRQVWUDWHG ZLWK VLPSOH GLDJUDPV RU YLVXDO DLGV VLPXOWDQHRXVO\7KHFKDOOHQJHLVWRSUHYHQWGHDGORFNVDQG
V\QFKURQL]DWLRQ FKDOOHQJHV DUH RIWHQ QRW DV YLVXDOO\ HQVXUHWKDWSKLORVRSKHUVFDQHDWVDIHO\ZLWKRXWFRQWHQWLRQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 99

% 7KH3URGXFHU&RQVXPHU3UREOHP PRQLWRUV WKH\ KHOS HQVXUH WKDW WKUHDGV FDQ VDIHO\ VKDUH
,Q WKLV SUREOHP WKHUH DUH WZR W\SHV RI SURFHVVHV UHVRXUFHVDQGDYRLGFRQWHQWLRQZKLOHZDLWLQJIRUHYHQWVRU
SURGXFHUV DQG FRQVXPHUV 3URGXFHUV SURGXFH LWHPV DQG FKDQJHVLQWKHVKDUHGVWDWH
DGG WKHP WR D VKDUHG EXIIHU ZKLOH FRQVXPHUV UHPRYH &RQGLWLRQ YDULDEOHV DUH XVHG IRU VLJQDOLQJ DQG
LWHPV IURP WKH EXIIHU 7KH FKDOOHQJH LV SURWHFW DFFHVV WR QRWLILFDWLRQ EHWZHHQ WKUHDGV 7KUHDGV FDQ ZDLW IRU VRPH
WKHEXIIHUDQGWRHQVXUHWKDWSURGXFHUVGRQ WDGGLWHPVWRD FRQGLWLRQWREHFRPHWUXH±VRPHVKDUHGUHVRXUFHWREHLQD
IXOO EXIIHU DQG FRQVXPHUV GRQ W UHPRYH LWHPV IURP DQ GHVLUHGVWDWH7KUHDGVFDOOLQJZDLWRQDFRQGLWLRQYDULDEOH
HPSW\EXIIHUZKLOHPLQLPL]LQJFRQWHQWLRQ DUH SXW LQWR D ZDLWLQJ VWDWH DQG WKH\ GR QRW FRQVXPH
SURFHVVRU WLPH XQWLO WKH\ DUH DZDNHQHG :KHQ DQRWKHU
& 7KH5HDGHUV:ULWHUV3UREOHP WKUHDG FKDQJHV WKH VWDWH RI WKH VKDUHG UHVRXUFH LQ D ZD\
,Q WKLV SUREOHP WKHUH DUH WZR W\SHV RI SURFHVVHV WKDWVDWLVILHVWKHFRQGLWLRQLWFDQXVHVLJQDORUEURDGFDVW
UHDGHUVDQGZULWHUV0XOWLSOHUHDGHUVFDQDFFHVVDVKDUHG WRQRWLI\ZDLWLQJWKUHDGVWRZRNHWKHPXS
UHVRXUFHVLPXOWDQHRXVO\EXWZULWHUVQHHGH[FOXVLYHDFFHVV
WR PRGLI\ WKH UHVRXUFH 7KH FKDOOHQJH LV WR DOORZ IRU & 6SXULRXVZDNHXSV
FRQFXUUHQWUHDGLQJZKLOHHQVXULQJH[FOXVLYHZULWLQJ &RQGLWLRQ YDULDEOHV DUH SURQH WR VSXULRXV ZDNHXSV
6RPHRWKHUIDPRXVFODVVLFDOV\QFKURQL]DWLRQSUREOHPV ZKHUH D WKUHDG FDQ EH DZDNHQHG IURP D ZDLWLQJ VWDWH
DUH ³6OHHSLQJ %DUEHU´ ³%ULGJH &URVVLQJ´ ³&LJDUHWWH ZLWKRXW WKH DVVRFLDWHG FRQGLWLRQ EHLQJ H[SOLFLWO\ PHW RU
6PRNHUV´³+23UREOHP´>@ VLJQDOHG 7KLV PHDQV D WKUHDG ZDNHV XS XQH[SHFWHGO\
HYHQWKRXJKQRRWKHUWKUHDGKDVH[SOLFLWO\LVVXHGDVLJQDO
,9 &21',7,219$5,$%/(6 RUEURDGFDVWRSHUDWLRQRQWKHFRQGLWLRQYDULDEOH
6SXULRXV ZDNHXSV FDQ RFFXU GXH WR YDULRXV IDFWRUV
:H ZHUH VLJQLILFDQWO\ LQIOXHQFHG E\ WKH ³7KH /LWWOH LQFOXGLQJ WKH LPSOHPHQWDWLRQ RI WKH RSHUDWLQJ V\VWHP V
%RRN RI 6HPDSKRUHV´ >@ 7KH DXWKRU RI WKH ERRN WKUHDG VFKHGXOHU DQGWKH XQGHUO\LQJ KDUGZDUH:KLOH WKH
HPSOR\V VHPDSKRUHV DV V\QFKURQL]DWLRQ SULPLWLYHV IRU H[DFW FDXVH RI VSXULRXV ZDNHXSV PD\ YDU\ WKH\ DUH
VROYLQJ D YDULHW\ RI V\QFKURQL]DWLRQ WDVNV :KLOH W\SLFDOO\ D UHVXOW RI WKH FRPSOH[LWLHV LQYROYHG LQ
VHPDSKRUHV VHUYH DV JHQHUDOSXUSRVH V\QFKURQL]DWLRQ PDQDJLQJ WKUHDG V\QFKURQL]DWLRQ DQG WKH QHHG IRU
SULPLWLYH FRQVWUXFWLRQV RIWHQ XVHG IRU V\QFKURQL]DWLRQ RI HIILFLHQF\LQWKUHDGVFKHGXOLQJ
SURFHVVHVORFNVDQGFRQGLWLRQYDULDEOHV>@DUHPRUH
VXLWHGIRUWKHV\QFKURQL]DWLRQIRUPXOWLWKUHDGHGSURJUDPV 9 6<1&+521,=$7,2102'8/(2)3$5$//(/
$OWKRXJKXVLQJFRQGLWLRQYDULDEOHVPLJKWLQLWLDOO\DSSHDU
PRUH FRPSOH[ DQG D VSHFLDO VSXULRXV ZDNHXSV TXLUN &20387,1*&2856(
PDNHVLWHYHQ PRUHSHFXOLDUWKH\SURYLGHPRUHLQWXLWLYH ,Q WKLV FRQWULEXWLRQ ZH SURPRWH WKH WHDFKLQJ RI
V\QFKURQL]DWLRQ PHFKDQLVP 0RUHRYHU FRQGLWLRQ V\QFKURQL]DWLRQ XVLQJ VLPSOH SUREOHPV WKDW UHVHPEOH
YDULDEOHV DUH ZLGHO\ UHFRJQL]HG LQ YDULRXV SURJUDPPLQJ FODVVLFDOV\QFKURQL]DWLRQFKDOOHQJHVDQGDUHLPSOHPHQWHG
HQYLURQPHQWV 3URJUDPPLQJ ODQJXDJHV VXFK DV & & DV PXOWLWKUHDGHG SURJUDPV :H DGYRFDWH WKH XVH RI
*RODQJ -DYD 3\WKRQ 5XVW HWF RIIHU VXSSRUW IRU FRQGLWLRQ YDULDEOHV DQG ORFNV RU PRQLWRUV  DV HVVHQWLDO
FRQGLWLRQ YDULDEOHV DV SDUW RI WKHLU FRQFXUUHQF\ DQG V\QFKURQL]DWLRQSULPLWLYHV7KLVDSSURDFKLVDSSOLHGLQWKH
SDUDOOHOSURJUDPPLQJFDSDELOLWLHV 3DUDOOHO &RPSXWLQJ FRXUVH VSHFLILFDOO\ ZLWKLQ D PRGXOH
GHGLFDWHGWRV\QFKURQL]DWLRQ
$ 7KUHDGVYV3URFHVVHV 7KHV\QFKURQL]DWLRQPRGXOHFRQVLVWVRIDSSUR[LPDWHO\
$ WKUHDG LV WKH VPDOOHVW XQLW RI H[HFXWLRQ ZLWKLQ D KRXUVRIOHFWXUHWLPHDQGKRXUVRISUDFWLFDOH[HUFLVHV
SURFHVVLQDPXOWLWDVNLQJRSHUDWLQJV\VWHP,WUHSUHVHQWVD LQFOXGLQJ KDQGVRQ SURJUDPPLQJ WDVN WR EH FRPSOHWHG
VLQJOH VHTXHQFH RI LQVWUXFWLRQV WKDW FDQ EH VFKHGXOHG IRU GLUHFWO\GXULQJSUDFWLFDOH[HUFLVH
H[HFXWLRQ E\ WKH RSHUDWLQJ V\VWHP V VFKHGXOHU 7KUHDGV 6WXGHQWV DWWHQGLQJ WKH FRXUVH DOUHDG\ KDYH VRPH
ZLWKLQ D SURFHVV VKDUH WKH VDPH PHPRU\ VSDFH ILOH NQRZOHGJH RQ SDUDOOHO H[HFXWLRQ DQG SURFHVVHV DFTXLUHG
GHVFULSWRUV DQG RWKHU UHVRXUFHV ZKLFK DOORZV WKHP WR RQ 2SHUDWLQJ 6\VWHPV FRXUVH $GGLWLRQDOO\ VWXGHQWV
ZRUN WRJHWKHU DQG VKDUH GDWD PRUH HIILFLHQWO\ WKDQ PLJKW KDYH EHHQ LQWURGXFHG WR VHPDSKRUHV
VHSDUDWHSURFHVVHV V\QFKURQL]DWLRQ6WXGHQWVDOVRKDYHJHQHUDOSURJUDPPLQJ
$ SURFHVV LV DQ LQGHSHQGHQW SURJUDP ZLWK LWV RZQ H[SHULHQFH LQ & SURJUDPPLQJ ODQJXDJH DFTXLUHG RQ
PHPRU\ VSDFH ZKLOH D WKUHDG LV D OLJKWZHLJKW XQLW RI PXOWLSOH FRXUVHV VXFK DV 3URJUDPPLQJ DQG $OJRULWKPLFV
H[HFXWLRQ ZLWKLQ D SURFHVV 7KUHDGV ZLWKLQ WKH VDPH DQG'DWD6WUXFWXUHV
SURFHVVFDQFRPPXQLFDWHDQGVKDUHGDWDPRUHHDVLO\WKDQ 7KHOHFWXUHWRSLFVLQFOXGH
SURFHVVHV ZKLFK W\SLFDOO\ UHTXLUH LQWHUSURFHVV
 3URFHVVHVDQG7KUHDGV
FRPPXQLFDWLRQ ,3& PHFKDQLVPV
 6HPDSKRUHV DQG WKH 326,; VHPDSKRUHV SDUDOOHO
0XOWLWKUHDGLQJLVWKHSUDFWLFHRIXVLQJPXOWLSOHWKUHDGV
SURJUDPPLQJPRGHO
ZLWKLQ D VLQJOH SURFHVV WR SHUIRUP FRQFXUUHQW WDVNV
0XOWLWKUHDGLQJ DOORZV IRU EHWWHU XWLOL]DWLRQ RI PXOWLFRUH  &RQGLWLRQYDULDEOHVDQGWKH326,;WKUHDGVSDUDOOHO
SURFHVVRUV DQG FDQ LPSURYH WKH UHVSRQVLYHQHVV RI SURJUDPPLQJPRGHO
DSSOLFDWLRQV E\ GLYLGLQJ WDVNV LQWR VPDOOHU FRQFXUUHQWO\  )XQGDPHQWDO V\QFKURQL]DWLRQ FRQFHSWV PXWXDO
H[HFXWLQJWKUHDGV H[FOXVLRQDQGVLJQDOLQJ
 %DVLF V\QFKURQL]DWLRQ FRQVWUXFWV EDUULHU
% &RQGLWLRQ9DULDEOHV V\QFKURQL]DWLRQ UHDGHUVZULWHUV SURGXFHUV
&RQGLWLRQYDULDEOHVSURYLGHDVWUXFWXUHGDQGRUJDQL]HG FRQVXPHUV
ZD\ WR FRRUGLQDWH WKUHDGV WKDW DUH ZDLWLQJ IRU VSHFLILF
FRQGLWLRQV WR EH VDWLVILHG :KHQ XVHG ZLWK ORFNV RU

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 100

  7KH 'LQLQJ 3KLORVRSKHUV SUREOHP ZLWK PXOWLSOH 

VROXWLRQV 6LPXODWHWKHIROORZLQJVLWXDWLRQ

7KHUHDUHJLUOVDQGER\VLQWKHKROLGD\FDPS(DFKER\RUJLUOILUVWSOD\V ER\VDQGJLUOV 
7KHSUDFWLFDOH[HUFLVHVLQYROYHVROYLQJWDVNVRIYDU\LQJ DQGWKHQJRHVWRWKHUHVWURRP V DQGWKHQHDWVLQWKHGLQLQJURRP V 6LPXODWLRQHQGVDIWHU
URXQGV JDPHUHVWURRPHDW 
FRPSOH[LW\ SULPDULO\ XWLOL]LQJ WKH 326,; WKUHDGV 

SURJUDPPLQJPRGHO7KHVHH[HUFLVHVLQFOXGH >@$GGFRXQWHUVWRWKHVLPXODWLRQWRWUDFNKRZPDQ\WLPHVWKHJLUOVDWH DWWKHHQGRI 

DQGKRZPDQ\WLPHVWKHER\VDWH DWWKHHQGRI 

 &UHDWLQJDQGPDQDJLQJWKUHDGV >@(QVXUHWKDWWKHUHDUHPD[JLUOVRUPD[ER\VLQWKHUHVWURRPQRWDER\DQGDJLUODW
WKHVDPHWLPHJLUOVKDYHSULRULW\ZKLOHZDLWLQJIRUWKHUHVWURRPWREHIUHH
 ,PSOHPHQWLQJ PXWXDO H[FOXVLRQ XVLQJ VHPDSKRUHV 
>@(QVXUHWKDWDOOFKLOGUHQZDLWIRUHDFKRWKHULQWKHGLQLQJURRPILUVWDQGRQO\WKHQWKH\VWDUW
DQGPXWH[HV HDWLQJ

 ,PSOHPHQWLQJ VLJQDOLQJ XVLQJ VHPDSKRUHV DQG 1RWHV
XVHRQO\PXWH[HVDQGFRQGLWLRQYDULDEOHV
PXWH[HVFRQGLWLRQYDULDEOHV GRQWUHO\RQJLYHQWLPHVVLPXODWLRQVKRXOGZRUNDOVRZLWKGLIIHUHQWWLPHVRUFRXQWV
EXLOG FRQVROH JFFFDPSBHQFRFDPSBHQOSWKUHDG
 ([SORULQJ VHPDQWLFDOO\ HTXLYDOHQW V\QFKURQL]DWLRQ 


FRQVWUXFWV LQ RWKHU SURJUDPPLQJ ODQJXDJHV LQFOXGHVWGLRK!

LQFOXGHVWGOLEK!
LQFOXGLQJWKUHDGVFRQGLWLRQYDULDEOHVDQGPRQLWRUV LQFOXGHSWKUHDGK!
LQFOXGHXQLVWGK!
 ([DPLQLQJ VDPSOH VROXWLRQV WR PRUH FRPSOH[ 

V\QFKURQL]DWLRQSUREOHPV SWKUHDGBPXWH[BWPXW 37+5($'B087(;B,1,7,$/,=(5
SWKUHDGBFRQGBWFRQG 37+5($'B&21'B,1,7,$/,=(5

 6ROYLQJ D SUDFWLFDO SURJUDPPLQJ DVVLJQPHQW 
SWKUHDGBPXWH[BORFN PXW 
W\SLFDOO\ LQYROYLQJ D VLPSOH V\QFKURQL]DWLRQ SWKUHDGBPXWH[BXQORFN PXW 

SUREOHP 7KH OHFWXUHU SUHVHQWV D FRUUHFW VROXWLRQ SWKUHDGBFRQGBZDLW FRQG PXW 
SWKUHDGBFRQGBVLJQDO FRQG 
IROORZHGE\DJURXSGLVFXVVLRQ SWKUHDGBFRQGBEURDGFDVW FRQG 

 ,QGLYLGXDO GLVFXVVLRQV ZLWK VWXGHQWV DERXW WKHLU 
GHILQH&2817B%2<6
VXEPLWWHGVROXWLRQVDGGUHVVLQJDQ\PLVWDNHVPDGH GHILQH&2817B*,5/6
GHILQH&2817B5281'6
DQGPDNLQJQHFHVVDU\DVVHVVPHQWDGMXVWPHQWV GHILQH7,0(B*$0(B%2<
GHILQH7,0(B*$0(B*,5/
GHILQH7,0(B:&
GHILQH7,0(B($7
9, 6,03/(6<1&+521,=$7,21352%/(0$6 
YRLGJDPHBER\ YRLG ^
$66,*10(17 VOHHS 7,0(B*$0(B%2< 
`
$ VLPSOH V\QFKURQL]DWLRQ WDVN LV WKH FORVXUH RI WKH 
YRLGJDPHBJLUO YRLG ^
PRGXOH GHGLFDWHG WR WKH V\QFKURQL]DWLRQ 7KH WDVN VOHHS 7,0(B*$0(B*,5/ 
`
UHVHPEOHV WKH FODVVLFDO V\QFKURQL]DWLRQ SUREOHPV 
YRLGUHVWURRPBER\ YRLG ^
SUHVHQWHGLQWKH6HFWLRQ,,, VOHHS 7,0(B:& 
`

$ 3UHSDUDWLRQ YRLGUHVWURRPBJLUO YRLG ^
VOHHS 7,0(B:& 
`
7KHWDVNLVDVVLJQHGWRWKHVWXGHQWVGLUHFWO\GXULQJWKH 
SUDFWLFDO H[HUFLVH DQG WKH\ KDYH SUHVFULEHG WLPH WR YRLGHDWBER\ YRLG ^
VOHHS 7,0(B($7 
FRPSOHWH LW 7R VROYH WKH WDVN VWXGHQWV HQJDJH LQ SULRU `

H[HUFLVHV DQG VROYH VLPSOH WDVNV DVVLJQHG IRU LQGLYLGXDO YRLGHDWBJLUO YRLG ^
VOHHS 7,0(B($7 
SUHSDUDWLRQ3UHSDUDWLRQDOVRLQYROYHVSUHVHQWLQJWZRYHU\ `

VLPLODU V\QFKURQL]DWLRQ SUREOHPV IROORZHG E\ D GHWDLOHG YRLG JLUO YRLG SWU ^

DQDO\VLV RI H[HPSODU\ VROXWLRQV E\ WKH WHDFKHU 6WXGHQWV IRU LQWU U&2817B5281'6U ^
PD\ DOVR LQGLYLGXDOO\ WDFNOH VHYHUDO VLPLODU WDVNV IRU JDPHBJLUO 
UHVWURRPBJLUO 
ZKLFK H[HPSODU VROXWLRQV DUH QRW SURYLGHG ZLWK WKH HDWBJLUO 
`
RSSRUWXQLW\ WR VHHN JXLGDQFH DQG GLVFXVV WKHLU VROXWLRQV UHWXUQ18//
`
DQGSRWHQWLDOLVVXHVZLWKWKHWHDFKHU 
YRLG ER\ YRLG SWU ^

% 6WUXFWXUH IRU LQWU U&2817B5281'6U ^
JDPHBER\ 
UHVWURRPBER\ 
$VVLJQPHQW LV SURYLGHG WR VWXGHQWV DV D VLQJOH & HDWBER\ 
`
SURJUDPPLQJODQJXDJHILOHXWLOL]LQJ326,;WKUHDGV7H[W UHWXUQ18//

SRUWLRQ RI WKH DVVLJQPHQW LV LQFOXGHG DV D PXOWLOLQH `


FRPPHQW DW WKH EHJLQQLQJ RI WKH ILOH ,W FRPSULVHV D PDLQI
LQWPDLQ YRLG ^
SDUDJUDSK LQWURGXFLQJ WKH SUREOHP DQG D OLVW RI XVXDOO\ SWKUHDGBWER\V>&2817B%2<6@
SWKUHDGBWJLUOV>&2817B*,5/6@
WKUHH QXPEHUHG VXEWDVNV )HZ QRWHV UHOHYDQW WR WKH 
LQWPD[BFRXQW &2817B%2<6!&2817B*,5/6"&2817B%2<6&2817B*,5/6
H[SHFWHG VROXWLRQ GHVLJQ DQG LPSOHPHQWDWLRQ DUH DOVR IRU LQWL LPD[BFRXQWL ^
LI L&2817B%2<6 SWKUHDGBFUHDWH ER\V>L@18// ER\18// 
LQFOXGHG $Q H[DPSOH RI WKH VSHFLILHG V\QFKURQL]DWLRQ LI L&2817B*,5/6 SWKUHDGBFUHDWH JLUOV>L@18// JLUO18// 
`
WDVNLVSUHVHQWHGLQ)LJ 
IRU LQWL L&2817B%2<6L SWKUHDGBMRLQ ER\V>L@18// 
IRU LQWL L&2817B*,5/6L SWKUHDGBMRLQ JLUOV>L@18// 

H[LW (;,7B68&&(66 
`

)LJXUH ([DPSOHRIVLPSOHV\QFKURQL]DWLRQSUREOHPWREH
VROYHGE\VWXGHQWVGXULQJSUDFWLFDOH[HUFLVH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 101

 7KH DVVLJQPHQW LV D PRGHUDWHO\ GHPDQGLQJ VLQJOHWKUHDGWRPRGLI\WKHPDWWLPHWKXVSUHYHQWLQJUDFH
V\QFKURQL]DWLRQ SUREOHP IRUPXODWHG DV D VLPSOH VWRU\ FRQGLWLRQV
JLYHQ LQ WKH ILUVW SDUDJUDSK ,W LQWURGXFHV DFWRUV XVXDOO\ :KLOH WKLV VXEWDVN DFFRXQWV IRU  RI WKH RYHUDOO
KXPDQVRUDQLPDOV DQGDFWLRQVSHUIRUPHGE\WKHVHDFWRUV DVVHVVPHQW LW DORQH LV LQVXIILFLHQW WR DFKLHYH D SDVVLQJ
7KH VWRU\ VKRXOG GHSLFW D VLWXDWLRQ WKDW LV YHU\ JUDGH 7R SDVV VWXGHQWV PXVW DOVR WDFNOH DW OHDVW RQH RI
FRPSUHKHQVLEOHDQGFDQEHHDVLO\PHPRUL]HG WKHVXEWDVNVWKDWUHTXLUHVLJQDOL]DWLRQ
7KH ILUVW VXEWDVN UHTXLUHV VWXGHQW WR XQGHUVWDQG WKH   6ROXWLRQIRU6XEWDVN5HXVDEOH%DUULHU
LPSRUWDQFHRIDFULWLFDOVHFWLRQWRSURWHFWJOREDOYDULDEOHV 7KHH[HPSODUVROXWLRQIRUWKHWKLUGVXEWDVNLVVKRZQLQ
DQG SUHYHQW UDFH FRQGLWLRQV XVLQJ PXWH[HV 7KH VHFRQG )LJ  7KLV VXEWDVN LQYROYHV WKH XVH RI D ZHOONQRZQ
DQG WKH WKLUG VXEWDVNV UHTXLUH VLJQDOL]DWLRQ EHWZHHQ V\QFKURQL]DWLRQFRQVWUXFWWKHEDUULHU6LQFHWKUHDGVQHHG
WKUHDGVQHFHVVLWDWLQJWKHXVHRIFRQGLWLRQYDULDEOHV WRV\QFKURQL]HUHSHDWHGO\WKHEDUULHUPXVWEHUHXVDEOH
3URJUDPJLYHQWRVWXGHQWVFRQVLVWVRIDVNHOHWRQRIWKH $UHXVDEOHEDUULHUFRQVLVWVRIWZRVLPSOHEDUULHUVZKHUH
VROXWLRQ WKDW FDQ EH FRPSLOHG DQG H[HFXWHG 7KUHDGV WKUHDGV PXVW ZDLW XQWLO DOO WKUHDGV KDYH UHDFKHG WKH
UHSUHVHQWLQJ DFWRUV LQ WKH VWRU\ DUH DOUHDG\ FUHDWHG DQG UHQGH]YRXV SRLQW EHIRUH SURFHHGLQJ WR WKH FULWLFDO SRLQW
VWXGHQWV VKRXOG DGG V\QFKURQL]DWLRQ LQWR SUHSDUHG ,QFRPLQJ WKUHDGV XVH WKH LQFRPPLQJB DQG
IXQFWLRQV UHSUHVHQWLQJ DFWLRQV SHUIRUPHG E\ DFWRUV LQFRPPLQJBYDULDEOHVWRFRXQWKRZPDQ\RIWKHPKDYH
6WXGHQWV KDYH WKH IOH[LELOLW\ WR QHZ IXQFWLRQV RU PRGLI\ DOUHDG\DUULYHGDWWKHUHQGH]YRXVSRLQWVRIWKHILUVWRUWKH
H[LVWLQJ IXQFWLRQV DV WKH\ QHHG $QRWKHU PXOWLOLQH VHFRQGVLPSOHEDUULHUUHVSHFWLYHO\
FRPPHQWJLYHVVWXGHQWVKLQWVWR326,;WKUHDGV$3,W\SHV
DQGIXQFWLRQVXVHGIRUFUHDWLQJDQGRSHUDWLQJPXWH[HVDQG 7KH ODVW LQFRPLQJ WKUHDG WR WKH VLPSOH EDUULHU
FRQGLWLRQDOYDULDEOHV EURDGFDVWV D VLJQDO WR DOO RWKHU ZDLWLQJ WKUHDGV ZKLFK
ZHUH ZDLWLQJ RQ WKH ZDLWBDOOBFRQG FRQGLWLRQ YDULDEOH
3URJUDP VWUXFWXUH SURYLGHG WR VWXGHQWV VHUYHV DV WKH XQWLOWKHODVWWKUHDGUHDFKHVWKHUHQGH]YRXV)XUWKHUPRUH
EDVLFIUDPHKRZWKHVROXWLRQVKRXOGEHLPSOHPHQWHGDQG WKHODVWLQFRPLQJWKUHDGUHVHWVWKHRWKHUEDUULHUE\VHWWLQJ
KRZ DFWRUV DQG DFWLRQV DUH H[SHFWHG WR EH PRGHOOHG ± LQFRPPLQJB RU LQFRPPLQJB WR ]HUR GHSHQGLQJ RQ
XVLQJ 326,; WKUHDGV RU VLPLODU SURJUDPPLQJ PRGHOV ZKLFK VLPSOH EDUULHU ZDV XVHG %RWK VLPSOH EDUULHUV FDQ
:KLOHRWKHUDSSURDFKHVVXFKDVGLVFUHWHHYHQWVLPXODWLRQ HIILFLHQWO\VKDUHWKHVDPHZDLWBDOOBPXWH[PXWH[DQGWKH
FRXOG EH FRQVLGHUHG DV D EHWWHU DSSURDFK IURP FHUWDLQ ZDLWBDOOBFRQGFRQGLWLRQYDULDEOH
SHUVSHFWLYHVWKH\ZRXOGQRWDOLJQZLWKWKHFRQWH[WRIWKH
FRXUVH PRGXOH $OVR WKH SURYLGHG VWUXFWXUH VLJQLILFDQWO\ 
ZDLWIRUDOO
KHOSV HYDOXDWLRQ RI WKH DVVLJQPHQW VLQFH LPSOHPHQWHG SWKUHDGBPXWH[BWZDLWBDOOBPXWH[ 37+5($'B087(;B,1,7,$/,=(5
SWKUHDGBFRQGBWZDLWBDOOBFRQG 37+5($'B&21'B,1,7,$/,=(5
V\QFKURQL]DWLRQ VWUXFWXUHV DUH H[SHFWHG WR EH ORFDWHG LQ LQWLQFRPPLQJB 
LQWLQFRPPLQJB 
GHILQHG SODFHV ZLWKLQ WKH FRGH VSHFLILFDOO\ ZLWKLQ WKH 
YRLGZDLWBIRUBDOO ^
SURYLGHGIXQFWLRQV 
SWKUHDGBPXWH[BORFN ZDLWBDOOBPXWH[ 

& ([SHFWHG6ROXWLRQDQG*UDGLQJ LQFRPPLQJB
LI LQFRPPLQJB &2817B%2<6&2817B*,5/6 ^
SWKUHDGBFRQGBEURDGFDVW ZDLWBDOOBFRQG 
([SHFWHG VROXWLRQV IRU WKH WKUHH VXEWDVNV RI WKH LQFRPPLQJB 
SURYLGHGH[HPSODUV\QFKURQL]DWLRQDVVLJQPHQWDUHVKRZQ `
ZKLOH LQFRPPLQJB &2817B%2<6&2817B*,5/6 
LQ )LJ 7KHVH VROXWLRQV DUH SURYLGHG WR RIIHU LQVLJKW SWKUHDGBFRQGBZDLW ZDLWBDOOBFRQG ZDLWBDOOBPXWH[ 

LQWR WKH GLIILFXOW\ RI HDFK VXEWDVN DQG WKH QDWXUH RI LQFRPPLQJB
LI LQFRPPLQJB &2817B%2<6&2817B*,5/6 ^
UHDVRQLQJZHDQWLFLSDWHIURPVWXGHQWV SWKUHDGBFRQGBEURDGFDVW ZDLWBDOOBFRQG 
LQFRPPLQJB 
`
 ZKLOH LQFRPPLQJB &2817B%2<6&2817B*,5/6 
HDWLQJ SWKUHDGBFRQGBZDLW ZDLWBDOOBFRQG ZDLWBDOOBPXWH[ 
YRLGZDLWBIRUBDOO  
 SWKUHDGBPXWH[BXQORFN ZDLWBDOOBPXWH[ 
SWKUHDGBPXWH[BWHDWBPXWH[ 37+5($'B087(;B,1,7,$/,=(5 `
LQWER\VBDWH 
LQWJLUOVBDWH 
)LJXUH 6ROXWLRQIRUVXEWDVNUHXVDEOHEDUULHU 

YRLGHDWBJLUO ^ 7KH UHXVDEOH EDUULHU FRQVWUXFWLRQ ZDV H[SODLQHG WR
ZDLWBIRUBDOO 
 VWXGHQWV GXULQJ WKH OHFWXUH GHGLFDWHG WR V\QFKURQL]DWLRQ
VOHHS 7,0(B($7 

WKHUHIRUHWKHDVVHVVPHQWIRUWKLVVXEWDVNDFFRXQWVRQO\IRU
SWKUHDGBPXWH[BORFN HDWBPXWH[ 
JLUOVBDWH
RIWKHRYHUDOODVVHVVPHQW6ROYLQJRQO\VXEWDVNDQG
SWKUHDGBPXWH[BXQORFN HDWBPXWH[ 
`
PHDQVWKDWVWXGHQWVHDUQRIWKHRYHUDOODVVHVVPHQW
 ZKLFKLVWKHPLQLPXPUHTXLUHGWRDFKLHYHDSDVVLQJJUDGH
YRLGHDWBER\ ^
ZDLWBIRUBDOO  IRUWKHDVVLJQPHQW

VOHHS 7,0(B($7 

  6ROXWLRQIRU6XEWDVN±&XVWRP6\QFKURQL]DWLRQ
SWKUHDGBPXWH[BORFN HDWBPXWH[ 
ER\VBDWH
)LJGHSLFWVWKHVROXWLRQIRUWKHVHFRQGVXEWDVNZKLFK
SWKUHDGBPXWH[BXQORFN HDWBPXWH[  LV FRQVLGHUHG WKH PRVW GLIILFXOW DQG DFFRXQWV IRU  RI
`
 WKHRYHUDOODVVHVVPHQW7KHVROXWLRQIRUWKLVVXEWDVNEHDUV

PDLQI
 VLJQLILFDQW VLPLODULWLHV WR WKH VROXWLRQ RI WKH UHDGHUV
LQWPDLQ YRLG ^
ZULWHUV SUREOHP ZLWK ZULWHU V SUHIHUHQFH DV H[SODLQHG WR
  6ROXWLRQIRU6XEWDVN0XWXDO([FOXVLRQ


VWXGHQWVGXULQJWKHOHFWXUHGHYRWHGWRV\QFKURQL]DWLRQ
7KHILUVWVXEWDVNLVWKHOHDVWFRPSOH[DQGLQYROYHVWKH

SULQWI *LUOVDWHGDQGER\VDWHGWLPHVJLUOVBDWHER\VBDWH 
XVHRIPXWH[ HV WRFUHDWHDFULWLFDOVHFWLRQ V WRSURWHFWD
 +RZHYHU WKHUH DUH D IHZ FUXFLDO GLIIHUHQFHV ± LQ WKH
H[LW (;,7B68&&(66 
VKDUHG UHVRXUFH WKDW LV DFFHVVHG E\ PXOWLSOH WKUHDGV
` UHDGHUVZULWHUVSUREOHPWKHUHLVQROLPLWWRWKHQXPEHURI

VKDUHG YDULDEOHV DFWLQJ DV FRXQWHUV ,Q WKH SURYLGHG UHDGHUV ZKR FDQ DFFHVV WKH VKDUHG UHVRXUFH ZKLOH RQO\
H[DPSOH)LJXUH 6ROXWLRQRIVXEWDVNPXWXDOH[FOXVLRQ
VKRZQ LQ )LJ  WKH JOREDO YDULDEOH ER\VBDWH RQHZULWHUFDQDFFHVV PRGLI\ WKHVKDUHGUHVRXUFH,QWKH
DQGJLUOVBDWHDUHSURWHFWHGE\HDWBPXWH[DOORZLQJRQO\D VHFRQG VXEWDVN OLPLWV DUH LPSRVHG RQ ERWK DFWRU JURXSV
DQGWKHVHOLPLWVFDQEHDUELWUDU\

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 102

  ' ([HFXWLRQ
UHVWURRP
GHILQH0$;B%2<6
GHILQH0$;B*,5/6
$ VLPSOH V\QFKURQL]DWLRQ WDVN LV WR EH FRPSOHWHG

SWKUHDGBPXWH[BWUHVWURRPBPXWH[ 37+5($'B087(;B,1,7,$/,=(5
GXULQJ SUDFWLFDO H[HUFLVHV ZLWKLQ D VSHFLILHG WLPH IUDPH
SWKUHDGBFRQGBWUHVWURRPBFRQG 37+5($'B&21'B,1,7,$/,=(5
LQWJLUOVBUHVWURRP 
7KH WDVN LV FDUULHG RXW RQ FRPSXWHUV LQ D SUHSDUHG
LQWER\VBUHVWURRP  HQYLURQPHQW DOWKRXJK WHDFKHU FDQ DOVR UHTXHVW D SDSHU
LQWJLUOVBZDLWLQJ 
 EDVHG VROXWLRQ RU FDQ DOORZ VWXGHQWV WR XVH WKHLU RZQ
YRLGUHVWURRPBJLUO YRLG ^
 FRPSXWHUV ,QWHUQHW DFFHVV LV W\SLFDOO\ QRW DOORZHG EXW
SWKUHDGBPXWH[BORFN UHVWURRPBPXWH[ 
JLUOVBZDLWLQJ
VWXGHQWV PD\ EH SHUPLWWHGWRXVH PDWHULDOV IURP OHFWXUHV
ZKLOH ER\VBUHVWURRP!__JLUOVBUHVWURRP!0$;B*,5/6 
SWKUHDGBFRQGBZDLW UHVWURRPBFRQG UHVWURRPBPXWH[ 
RUH[HUFLVHV
JLUOVBZDLWLQJ
JLUOVBUHVWURRP 6DPSOHVROXWLRQVLQSUDFWLFDOH[HUFLVHVDQGWKHSUHSDUHG
SWKUHDGBPXWH[BXQORFN UHVWURRPBPXWH[ 

DVVLJQPHQW DUH EDVHG RQ WKH 326,; 7KUHDGV
VOHHS 7,0(B:& 

SURJUDPPLQJ PRGHO +RZHYHU VWXGHQWV DUH DOVR
SWKUHDGBPXWH[BORFN UHVWURRPBPXWH[ 
JLUOVBUHVWURRP
LQWURGXFHG WR VHPDQWLFDOO\ HTXLYDOHQW FRQVWUXFWLRQV LQ
SWKUHDGBFRQGBEURDGFDVW UHVWURRPBFRQG  YDULRXV RWKHUSURJUDPPLQJ ODQJXDJHV DQGWKH\ KDYH WKH
SWKUHDGBPXWH[BXQORFN UHVWURRPBPXWH[ 
` RSWLRQWRLPSOHPHQWWKHWDVNLQVXFKODQJXDJH

YRLGUHVWURRPBER\ YRLG ^
 7KHWLPHDOORFDWHGIRUVROYLQJWKHWDVNLVW\SLFDOO\VHWDW

$ VROXWLRQ FDQ EH DFKLHYHG E\ LPSOHPHQWLQJ VLPSOH
SWKUHDGBPXWH[BORFN UHVWURRPBPXWH[  PLQXWHVWRHQVXUHWKDWVWXGHQWVDUHQRWXQGHUWKHWLPH
ZKLOH ER\VBUHVWURRP!0$;B%2<6__JLUOVBUHVWURRP!__JLUOVBZDLWLQJ 
SDWWHUQV IRU XVLQJ FRQGLWLRQ YDULDEOHV :KHQ D WKUHDG
SWKUHDGBFRQGBZDLW UHVWURRPBFRQG UHVWURRPBPXWH[  VWUHVV7KHDVVLJQPHQWLVUHDGDORXGWRWKHVWXGHQWVZLWK
ER\VBUHVWURRP
QHHGVWRFKHFNZKHWKHUWKHVKDUHGJOREDOVWDWHPHHWVWKH
SWKUHDGBPXWH[BXQORFN UHVWURRPBPXWH[  WKH FRQGLWLRQV IRU VROYLQJ WKH WDVN UHLWHUDWHG DQG DQ\

GHVLUHGFRQGLWLRQWKHIROORZLQJSDWWHUQFDQEHHPSOR\HG
VOHHS 7,0(B:& 
TXHVWLRQVDUHDGGUHVVHG

)LUVWWKHFRUUHVSRQGLQJPXWH[UHVWURRPBPXWH[PXVW
SWKUHDGBPXWH[BORFN UHVWURRPBPXWH[  $IWHUVHVVLRQDVROXWLRQWRWKHSUREOHPLVSUHVHQWHGWR
ER\VBUHVWURRP
EH ORFNHG WR DOORZ WKH WKUHDG WR DFFHVV VKDUHG YDULDEOHV
SWKUHDGBFRQGBEURDGFDVW UHVWURRPBFRQG  WKHVWXGHQWVDQGRQWKHQH[WVHVVLRQDIWHUUHYLHZLQJWKHLU
SWKUHDGBPXWH[BXQORFN UHVWURRPBPXWH[ 
DQG GHWHUPLQH ZKHWKHU WKH GHVLUHG FRQGLWLRQ LV PHW
`
VXEPLVVLRQVHDFKVWXGHQWUHFHLYHVLQGLYLGXDOIHHGEDFNRQ
/RFNLQJ WKH 6ROXWLRQIRUVXEWDVNPRGLILHG5HDGHUV±:ULWHUV
)LJXUH PXWH[ HQVXUHV WKDW WKH FRQGLWLRQ FDQQRW EH WKHLUHUURUVDQGDMXVWLILHGDVVHVVPHQW
DOWHUHG E\ RWKHU WKUHDGV ZKLOH WKH PXWH[ LV DFTXLUHG E\
SUREOHPZLWKSULRULW\IRURQHWKUHDGW\SH 6WXGHQWV DUH JLYHQ WKH RSSRUWXQLW\ WR SDUWLFLSDWH LQ D
WKLVWKUHDG VHFRQG UHSODFHPHQW VHVVLRQ IRU WKH V\QFKURQL]DWLRQ WDVN
7KHQWKHFRUUHFWFRQGLWLRQVXFKDVER\VBUHVWURRP! ZKLFK FDQ EH KHOSIXO LQ FDVHV RI IDLOXUH RU DEVHQFHV
__JLUOVBUHVWURRP!0$;B*,5/6VKRXOGEHFKHFNHG,I $OORZLQJ DOO VWXGHQWV UHJDUGOHVV RI UHDVRQ WR DWWHQG WKH
WKHFRQGLWLRQLVQRWPHWWKHWKUHDGLVVXVSHQGHGE\FDOOLQJ UHSODFHPHQWVHVVLRQKDVSURYHQWREHEHQHILFLDO
SWKUHDGBFRQGBZDLW UHVWURRPBFRQG UHVWURRPB
PXWH[  7KH WKUHDG UHPDLQV VXVSHQGHG XQWLO ( $LPVRIWKHDVVLJQPHQW
UHVWURRPBFRQGLVVLJQDOHGDQGWKHQLWLVDZDNHQHGZKHQ 7KH VLPSOH V\QFKURQL]DWLRQ WDVN VHUYHV VHYHUDO
LWVXFFHHGVLQDFTXLULQJUHVWURRPBPXWH[ SXUSRVHV
'XH WR WKH SRVVLELOLW\ RI VSXULRXV ZDNHXSV DQG WKH :HDLP WRSURYLGH VWXGHQWVZLWK SUDFWLFDO H[SRVXUH WR
FKDQFH WKDW DQRWKHU WKUHDG PD\ LQYDOLGDWH WKH FRQGLWLRQ VROYLQJDV\QFKURQL]DWLRQSUREOHPD³FKDOOHQJH´ZLWKD
EHIRUH UHVWURRPBPXWH[ LV DFTXLUHG WKH WKUHDG PXVW VLPLODU FKDUDFWHU WR WKH FODVVLFDO ZHOONQRZQ
LPPHGLDWHO\ UHHYDOXDWH WKH FRQGLWLRQ ,I WKH FRQGLWLRQ LV V\QFKURQL]DWLRQ SUREOHPV DV GHVFULEHG LQ 6HFWLRQ ,,, 
QRWVDWLVILHGSWKUHDGBFRQGBZDLWPXVWEHFDOOHGDJDLQWR 6\QFKURQL]DWLRQ LV D FUXFLDO DVSHFW WKDW GLVWLQJXLVKHV
VXVSHQGWKHWKUHDG SDUDOOHO SURJUDPPLQJ IURP FODVVLFDO VHTXHQWLDO
:KHQWKHFRQGLWLRQLVILQDOO\PHWWKHWKUHDGSURFHHGV SURJUDPPLQJ DQG VXFFHVVIXOO\ WDFNOLQJ VXFK D SUREOHP
WKURXJK WKH V\QFKURQL]DWLRQ FRQVWUXFW SRWHQWLDOO\ FDQ DOVR VWUHQJWKHQ WKH VWXGHQWV¶ FRQILGHQFH LQ WKHLU
PRGLI\LQJ VRPH JOREDO YDULDEOHV DQG HYHQWXDOO\ UHOHDVHV DELOLWLHV
UHVWURRPBPXWH[VRWKDWRWKHUWKUHDGVFDQDWWHPSWWRXVH 7KH WDVN DOVR VHUYHV DV DQ DVVHVVPHQW WRRO WR HYDOXDWH
V\QFKURQL]DWLRQFRQVWUXFWVSURWHFWHGE\WKHPXWH[ WKH DELOLWLHV RI VWXGHQWV LQ D FRQWUROOHG HQYLURQPHQW
,IDWKUHDGLVJRLQJWRPRGLI\DVKDUHGJOREDOYDULDEOH HQVXULQJWKHDXWKHQWLFLW\RIWKHLUZRUNDQGVROXWLRQV
WKDW FRXOG IXOILOO D FRQGLWLRQ RWKHU WKUHDGV PLJKW EH &RQGLWLRQ YDULDEOHV DQG PRQLWRUV DUH V\QFKURQL]DWLRQ
ZDLWLQJ IRU DQRWKHU ZHOOHVWDEOLVKHG SDWWHUQ FDQ EH FRQVWUXFWV WKDW DUH FRPPRQ FRPSRQHQWV RI WKH VWDQGDUG
DSSOLHG OLEUDULHV LQ PRVW PRGHUQ SURJUDPPLQJ ODQJXDJHV 7KH
7KHWKUHDGPXVWILUVWDFTXLUHWKHFRUUHVSRQGLQJPXWH[ NQRZOHGJH FDQ EH DSSOLHG E\ PDQ\ VWXGHQWV LQ WKHLU
2QFH DFTXLUHG LW KDV H[FOXVLYH DFFHVV WR WKH VKDUHG SUHIHUUHG SURJUDPPLQJ ODQJXDJHV PDNLQJ LW D SUDFWLFDO
YDULDEOH SUHYHQWLQJ DQ\ RWKHU WKUHDG IURP UHDGLQJ RU DQGYDOXDEOHVNLOOWRDFTXLUH
ZULWLQJ WR LW EHFDXVH WKH\ FDQQRW RZQ WKH PXWH[
) &KDOOHQJHV
VLPXOWDQHRXVO\ :LWK WKH PXWH[ DFTXLUHG WKH WKUHDG FDQ
VDIHO\ PRGLI\ WKH VKDUHG YDULDEOH DV QHHGHG $IWHU WKH 7KHUHDUHIHZFKDOOHQJHVDVVRFLDWHGZLWKWKHH[HFXWLRQRI
PRGLILFDWLRQ WKH WKUHDG FDQ VLJQDO RU EURDGFDVW WKH WKHDVVLJQPHQW
FRUUHVSRQGLQJ FRQGLWLRQ YDULDEOH V  WR QRWLI\ ZDLWLQJ (DFK DFDGHPLF \HDU D XQLTXH V\QFKURQL]DWLRQ WDVNV
WKUHDGV WKDW WKH VKDUHG VWDWH PD\ KDYH FKDQJHG )LQDOO\ PXVW EH GHVLJQHG IRU HDFK VWXGHQW WHDFKLQJ JURXS 7KH
WKHWKUHDGUHOHDVHVWKHPXWH[7KUHDGVWKDWZHUHVLJQDOHG WDVN VKRXOG SRVVHVV DQ DSSURSULDWH OHYHO RI GLIILFXOW\
WKURXJKFRQGLWLRQYDULDEOHVVWDUWWRFRPSHWHIRUWKHPXWH[ UHTXLULQJVWXGHQWVWRGHVLJQDQGLPSOHPHQWVROXWLRQVWKDW
DQG ZKHQ WKH\ VXFFHHG LQ DFTXLULQJ WKH PXWH[ WKUHDG LV DUH QRW PHUHO\ GLUHFW DSSOLFDWLRQV RI ZHOONQRZQ
DZDNHQDQGUHHYDOXDWHVWKHFRQGLWLRQ V\QFKURQL]DWLRQ FRQVWUXFWV VXFK DV UHDGHUVZULWHUV RU
SURGXFHUVFRQVXPHUV7KHV\QFKURQL]DWLRQSUREOHPVVXFK
DV H[DPSOH VKRZQ LQ )LJ  VKRXOG QRW EH D UHDO

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 103

 V\QFKURQL]DWLRQ FKDOOHQJHV VXFK DV WKH ³'LQLQJ WKDW VWXGHQWV VROYH GLUHFWO\ DV SDUW RI WKHLU SUDFWLFDO
3KLORVRSKHUV´ ZKHUH PXOWLSOH VROXWLRQV RI GLIIHUHQW H[HUFLVH
FKDUDFWHULVWLFV H[LVW ,QVWHDG VWXGHQWV VKRXOG EH 7KURXJK WKLV PRGXOH VWXGHQWV ZLOO QRW RQO\ DFTXLUH
HQFRXUDJHG WR PDVWHU FRUUHFW SDWWHUQV IRU FUHDWLQJ NQRZOHGJH EXW DOVR JDLQ SUDFWLFDO H[SHULHQFH ZLWK WKH
V\QFKURQL]DWLRQFRQVWUXFWLRQVDQGDSSO\WKHPSURSHUO\ LPSOHPHQWDWLRQ RI V\QFKURQL]DWLRQ LQ PXOWLWKUHDGHG
9DULRXV V\QFKURQL]DWLRQ WDVNV QDWXUDOO\ KDYH GLIIHUHQW SDUDOOHO SURJUDPV $GGLWLRQDOO\ VWXGHQWV ZLOO GHYHORS
OHYHOVRIFRPSOH[LW\7KLVUDLVHVTXHVWLRQVRIIDLUQHVVIRU SUREOHPVROYLQJ VNLOOV UHODWHG WR RUGHURIH[HFXWLRQ
VWXGHQWVRIGLIIHUHQW WHDFKLQJJURXSV7R HQVXUH IDLUQHVV SUREOHPVZLWKXQSUHGLFWDEOHWLPLQJZKLFKDUHYDOXDEOHLQ
LW V GHVLUHG WKDW DOO VROXWLRQV DUH DVVHVVHG E\ WKH VDPH VFHQDULRV LQYROYLQJ PXOWLSOH H[HFXWLRQ IORZV DQG
WHDFKHU DV\QFKURQRXVHQYLURQPHQWV
:KLOHXVLQJDXWRPDWHGWHVWLQJWRROVPLJKWEHWHPSWLQJ
LW V FUXFLDO WR DWWULEXWH DVVHVVPHQW WR VWXGHQWV HYHQ IRU $&.12:/('*0(17
SDUWLDO LQFRPSOHWH RU QRQZRUNLQJ VROXWLRQV 7KHUHIRUH 7KH SDSHU ZDV IXQGHG E\ WKH JUDQW .(*$ 8&0
HYHU\ VXEPLWWHG DVVLJQPHQW LV PDQXDOO\ DGGUHVVHG DQG  'HYHORSPHQW RI DQ LQWHUDFWLYH HFRXUVH XVLQJ
FKHFNHGIRUFRUUHFWQHVV)LQGLQJV\QFKURQL]DWLRQHUURUVLQ ³60$57´ WHFKQRORJLHV IRU WKH GHYHORSPHQW RI
SURJUDPV FUHDWHG E\ RIWHQ LQH[SHULHQFHG SURJUDPPHUV DOJRULWKPLFWKLQNLQJDQGSURJUDPPLQJVNLOOV
FDQ EH FKDOOHQJLQJ DQG SURQH WR HUURUV 6WXGHQWV PXVW
KDYHWKHRSSRUWXQLW\WRH[SODLQDQGGHIHQGWKHLUVROXWLRQV 5()(5(1&(6
VR WHDFKHU¶V HUURUV RU LQDELOLW\ WR XQGHUVWDQG VWXGHQW
>@ $ 6 7DQHQEDXPDQG + %RV0RGHUQ 2SHUDWLQJ 6\VWHPV WK
VROXWLRQLQUHDVRQDEOHWLPHLVFRUUHFWHG HG%RVWRQ0$3HDUVRQ
$VVLJQPHQWVFRQVLVWRIVLPSOHVWRULHVLQZKLFKWKUHDGV >@ 7$QGHUVRQDQG0'DKOLQ2SHUDWLQJ6\VWHPV3ULQFLSOHVDQG
DUH XVHG WR PRGHO DFWRUV RIWHQ KXPDQV 6RPH DWWHQWLRQ 3UDFWLFH5HFXUVLYHERRNV
VKRXOG EH SDLG WR SROLWLFDO FRUUHFWQHVV DQG QHXWUDOLW\ >@ 0 +HUOLK\ DQG 1 6KDYLW 7KH $UW RI 0XOWLSURFHVVRU
ZKHUHYHUDSSURSULDWH 3URJUDPPLQJ0RUJDQ.DXIPDQQ
>@ 9.XPDU$*UDPD$*XSWDDQG*.DU\SLV,QWURGXFWLRQWR
* 5HFHSWLRQDQGRWKHU2EVHUYDWLRQV 3DUDOOHO&RPSXWLQJ%HQMDPLQ&XPPLQJV
7KHDVVLJQPHQWLVJHQHUDOO\ZHOOUHFHLYHGE\VWXGHQWV >@ * :ROIIH DQG &  7UHIIW] 7HDFKLQJ SDUDOOHO FRPSXWLQJ QHZ
SRVVLELOLWLHV-RXUQDORI&RPSXWLQJ6FLHQFHVLQ&ROOHJHVQR
7KH PDMRULW\ RI VWXGHQWV FDQ DW OHDVW SDUWLDOO\ VROYH WKH 
SUREOHP ZLWKLQ WKH DOORFDWHG WLPH OLPLW 7KH\ DSSUHFLDWH >@ 75*URVV%UHDGWKLQGHSWKDVW\HDULQWURGXFWLRQWRSDUDOOHO
WKH LQGLYLGXDOL]HG DSSURDFK DQG ILQG WKH H[HPSODU SURJUDPPLQJ 3URFHHGLQJV RI WKH QG $&0 WHFKQLFDO
VROXWLRQVSURYLGHGE\WKHWHDFKHUSDUWLFXODUO\WKHFRUUHFW V\PSRVLXPRQ&RPSXWHUVFLHQFHHGXFDWLRQ
VROXWLRQ IRU WKH DVVLJQPHQW KLJKO\ EHQHILFLDO &RPSDUHG >@ &'5DIIDHOH66PLWKDQG2*HPLNRQDNOL([SODLQLQJPXOWL
WR WKH PRUH WUDGLWLRQDO IRUP RI ODUJHU SURMHFWV LQ WKH WKUHDGHGWDVNVFKHGXOLQJXVLQJWDQJLEOHXVHULQWHUIDFHVLQKLJKHU
3DUDOOHO&RPSXWLQJFRXUVHZKHUHVWXGHQWVQHHGWRGHIHQG HGXFDWLRQDOFRQWH[WV('8&21SS±
WKHLU ZRUN WKLV DVVLJQPHQW VWDQGV RXW GXH WR LWV XQLTXH >@ $/D]DU191LFXOHVFXDQG'3%XQGH3HDFK\3DUDOOHO
H[HFXWLRQIRUP $VVLJQPHQWV,(((,QWHUQDWLRQDO3DUDOOHODQG'LVWULEXWHG
3URFHVVLQJ6\PSRVLXP:RUNVKRSV ,3'36: 
0DQ\ VWXGHQWV ZKR GR QRW DGHTXDWHO\ SUHSDUH IRU WKH >@ 71HZKDOO.&:HEE9&KDJDQWLDQG$'DQQHU
DVVLJQPHQWWHQGWRIDLORUVLJQLILFDQWO\XQGHUSHUIRUP7KH ,QWURGXFLQJ3DUDOOHO&RPSXWLQJLQD6HFRQG&6&RXUVH,(((
FRQFHSWVUHTXLUHGIRUWKHDVVLJQPHQWDUHQRWW\SLFDOO\SDUW ,3'36:SS
RI VWXGHQWV  SULRU NQRZOHGJH DQG WKH\ PD\ QRW EH HDVLO\ >@ $%'RZQH\7KHOLWWOHERRNRIVHPDSKRUHV
JUDVSHG XSRQ LQLWLDO H[SRVXUH 6RPH WUDLQLQJ DQG >@ ( : 'LMNVWUD &RRSHUDWLQJ VHTXHQWLDO SURFHVVHV LQ
XQGHUVWDQGLQJ DUH QHHGHG WR H[FHO LQ WKLV WDVN 3URJUDPPLQJ /DQJXDJHV 1$72 $GYDQFHG 6WXG\ ,QVWLWXWH
&RQVHTXHQWO\ HYHQ FDSDEOH VWXGHQWV ZKR XQGHUHVWLPDWH $FDGHPLF3UHVVSS
WKH LPSRUWDQFH RI SUHSDUDWLRQ DQG UHO\ VROHO\ RQ WKHLU >@ ( : 'LMNVWUD +LHUDUFKLFDO RUGHULQJ RI VHTXHQWLDO SURFHVVHV
SURJUDPPLQJ DQG DQDO\WLFDO VNLOOV PD\ QRW DFKLHYH WKH $FWD,QIRUPDWLFD  SS
H[SHFWHG UHVXOWV 7R DGGUHVV WKLV LVVXH LW LV KLJKO\ >@ ' - /HKPDQQ DQG 0 2 5DELQ 2Q WKH DGYDQWDJHV RI IUHH
DSSURSULDWHWRRIIHUHYHU\VWXGHQWZKRVKRZVLQWHUHVWWKH FKRLFHDV\PPHWULFDQGIXOO\GLVWULEXWHGVROXWLRQWRWKHGLQLQJ
SKLORVRSKHUV SUREOHP $&06,*$&7 6\PSRVLXP RQ
RSSRUWXQLW\ WR SDUWLFLSDWH LQ WKH VHFRQG UHSODFHPHQW 3ULQFLSOHVRI3URJUDPPLQJ/DQJXDJHV
VHVVLRQ >@ & $ 5 +RDUH 0RQLWRUV $Q RSHUDWLQJ V\VWHP VWUXFWXULQJ
FRQFHSW&RPPXQLFDWLRQVRIWKH$&0SS
9,, &21&/86,21 >@ ' 5 %XWHQKRI 3URJUDPPLQJ ZLWK 326,; WKUHDGV $GGLVRQ
,QWKLVFRQWULEXWLRQZHSURYLGHGDEULHIRYHUYLHZRIWKH :HVOH\3URIHVVLRQDO
PRGXOHZLWKLQWKH3DUDOOHO&RPSXWLQJFRXUVHGHGLFDWHGWR >@ %%DUQH\326,;7KUHDGV3URJUDPPLQJ
KWWSVFRPSXWLQJOOQOJRYWXWRULDOVSWKUHDGV
V\QFKURQL]DWLRQ :H DOVR RIIHUHG D PRUH GHWDLOHG
GHVFULSWLRQ RI WKH V\QFKURQL]DWLRQ SUREOHP ± DVVLJQPHQW 


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 104

 Application of PID controller and CNN to control
Duckiebot robot
1st Marek Długosz 2nd Paweł Skruch
IEEE Senior Member IEEE Senior Member
AGH University of Science and Technology AGH University of Science and Technology
Cracow, Poland Cracow, Poland
mdlugosz@agh.edu.pl pawel.skruch@agh.edu.pl
0000-0001-6827-9149 0000-0002-8290-8375

3rd Marcin Szelest 4th Artur Morys-Magiera

IEEE Senior Member AGH University of Science and Technology
AGH University of Science and Technology Cracow, Poland
Cracow, Poland amorys@student.agh.edu.pl
mszelest@agh.edu.pl 0000-0002-2137-8841
0000-0002-0522-1270

Abstract—The paper presents the design and practical imple- (hardware, software, simulator, and teaching materials) is the
mentation by students of a control system using a classic PID Duckietown project. The Duckietown project was created in
controller and a controller using artificial neural networks. The 2016 at the Massachusetts Institute of Technology (MIT)
control object is a Duckiebot robot, and the task it is to perform
is to drive the robot along a designated line (line follower). The as part of a class for students [3]. It is an open project
purpose of the proposed activities is to familiarize students with that aims to promote education and research related to robot
the advantages and disadvantages of the two controllers used autonomy, especially in the context of autonomous cars. As
and for them to acquire the ability to implement control systems part of the project, the authors designed entities like Duckiebot
in practice. The article briefly describes how the two controllers robots, mock-ups of the Duckietown cities the robots need
work, how to practically implement them, and how to practically
implement the exercise. to navigate through, and additional infrastructure, such as the
Index Terms—autonomouse robot, PID controller, convolu- Watchtowers, to support the navigation process. The project
tional neural network, line follower also includes the development of simulation software called
Duckiebot-GYM, which allows the simulation of robots in a
I. I NTRODUCTION virtual environment [4]. The most recent, second generation
Recent years have seen a renaissance of sorts in the use of of Duckiebot robots, is using the Jetson Nano single-board
artificial neural networks. Various types of network architec- computer for control. Using the Jetson Nano platform opens
tures have been created and made generally available, as well the door to using advanced neural networks with dedicated
as their training methods have been improved. All this has GPU support to control the Duckiebot robot.
led to the application of artificial neural networks being on The article is organized as follows: Section II defines the
a rise, particularly successfully, in automatic control systems problem to be solved, Section III describes the student activ-
[1], [2]. The use of neural networks in automation systems ities that need to be carried out to solve the problem. Section
is not their typical application because very often control IV contains a brief design of Duckietown, the structure and
can be implemented using well-known and described classical control of the Duckiebot robot, sections V and VI describe how
control algorithms (PID, LQR, MPC, etc.). This provides an to solve the problem using a PID controller and a controller
opportunity to compare the performance of two different tech- using artificial neural networks. The VII section gives some
nologies and find their advantages and disadvantages. In the practical tips to help with the task, and finally the VIII section
process of educating students, the design and implementation provides conclusions.
by them of a control system using classical control algorithms
and artificial neural networks is very interesting from both a II. P ROBLEM DEFINITION
scientific and didactic point of view. As part of the didactic process, students have the goal
An important element of student education that increases of designing an automatic control system for a differentially
the attractiveness of the classes themselves, providing the driven robot that performs the functionality of moving along
opportunity to solve practical problems, is the use of real a designated route. The route of the robot is marked with a
robots. One project that provides a complete robotics platform colored line (e.g. red, yellow, etc.) distinguishable from the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 105

ground (usually black), on which the robot moves. The shape
of the trajectory is a closed contour. The control system should
generate a control signal to minimize the total deviation error
of the robot from the preset trajectory. This type of task is a
typical scenario implemented in various types of autonomous
robots and is often referred to as the line follower.

III. S TUDENT ACTIVITIES

Within the problem defined in the II section, students
will have to demonstrate adequate theoretical preparation and
practical skills such as: knowledge of the principle of operation
of the PID controller, knowledge of the principle of operation
of artificial neural networks, knowledge of operations related
to image transformations, ability to implement the discrete
version of the PID controller, skills to implement selected
structures of neural networks, ability to implement operations
of image transformations, knowledge of issues related to the
selection of parameters of the PID controller, knowledge of
issues related to training neural networks and preparing data
for their training, evaluation of the correctness of the training Fig. 1: Duckiebot robot (http://duckietown.org).
process of the network. A seemingly simple algorithm to
control an uncomplicated robot that realizes the functionality
of line follower generates a number of tasks, performance of
which requires adequate theoretical and practical preparation
from students. It should also be noted that the problems
that students have to solve during the project are typical
problems that also occur in the programming of other types
of autonomous robots.

IV. D UCKIEBOT ROBOT

The Duckiebot robot is a small two-wheeled robot that is
capable of moving on flat surfaces. Two modules containing
an electric motor, a gearbox and an encoder were used to drive
the robot. The Duckiebot robot is equipped with the following
sensors: an RGB camera, a distance sensor and an IMU inertial Fig. 2: Duckiebot robot kinematic system.
sensor [5]. A Jetson NANO single-board computer is used to
control the robot. In addition, the Duckiebot robot is equipped
with four RGB LEDs which can be used, for instance, to signal
the state the robot is currently in. The Duckiebot robot is v − Lω
ωL = (1)
designed for autonomous control and can move without human R
intervention, based on camera data and appropriate control v + Lω
algorithms. The figure 1 shows the Duckiebot robot in three ωR = (2)
R
projections.
A consequence of using two independent electric motors This type of Duckiebot robot control system is very commonly
in the design of the Duckiebot robot is the way in which the used in robotics and is called differential drive [6]. The
direction of the robot’s movement is controlled. The Duckiebot advantage of such a kinematic system for the motion of the
robot makes turns by changing the speeds of the electric Duckiebot robot is the simplicity of the design (no torsion axis
motors depending on which direction it wants to turn and mechanisms of the wheels), while the disadvantage is the need
how large the turning radius should be. Two signals are used to constantly control the rotational speed of the robot’s wheels
to control the robot: v-value of the linear speed and omega- (ωR , ωL ) so that it moves along the specified route.
value of the rotational speed along the robot’s vertical Z axis,
see Fig. 2. V. C LASSIC IMPLEMENTATION WITH A PID CONTROLLER
Given the values of (v, ω) (determined, for example, by The classic Duckiebot robot control system uses a PID
the controller), we can calculate the rotational speeds of the controller. The task of the control system is to generate such a
robot’s wheels from the formulas (1) and (2): control signal ui to minimize the value of the error signal ei ,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 106

where i corresponds to consecutive time moments. Figure 3 A. PID controller
shows a block diagram of the Duckiebot robot control system One of the best known and most widely used controllers
that uses the PID controller and the robot camera as a sensor. in automatic control systems is the PID controller. This is
primarily due to its versatility, ease of implementation in both
analog and digital form, and the relatively simple and intuitive
way of tuning the PID controller parameters. The discrete
version of the PID controller that was used in the Duckietown
robot control system is given by the equation (3):


k
ei − ei−1
ui = Kp ei + Ki ei Δt + Kd (3)
i=0
Δt
where: Kp amplification of the proportional part, Ki am-
plification of the integrating part, Kd amplification of the
differentiating part, ei error at the i-th instant of time, Δt
time step [7].
B. Error measurement
The value of the error signal ei is determined from the im-
ages transmitted by the Duckiebot robot camera. To determine
the value of the ei signal, it is necessary to perform several
operations on the transmitted images (see Figure 3). The first
operation is to convert the image from the RGB color space
to the HSV space (Hue, Saturation, Value). This conversion of
Fig. 3: Schematic of Duckiebot robot control system using between color spaces of images is mainly aimed at better color
PID controller. discrimination and recognition. Another operation is to find a
line of certain color in the image, along which the Duckiebot
The error signal ei is defined as the difference between the robot should move. Such an operation is to leave only the line
center of the horizontal axis of the camera image and the point marking the robot’s route in the image and remove all other
in the analyzed image region contained within the line along elements (e.g. background) from the image. The next operation
which the Duckiebot robot should move; see Fig. 4. is to reduce the area being analyzed by performing a cropping
operation. This operation aims to find the exact center of the
line along which the Duckiebot robot will move. The center
of the line is found as the coordinates of the center of gravity
obtained, as a result of the previous transformations, of the
line that determines the route of the Duckiebot robot. Figure
5 shows the result of the specific transformations performed
on the robot’s camera image to determine the value of the
position error ei .
Finally, the position error value ei is normalized to the
interval ei ∈ [−1; 1], so as to make the result independent
of the resolution (width, height) of the images captured from
the camera.
VI. I MPLEMENTATION USING NEURAL NETWORKS
The second method of generating a control signal ui uses
Fig. 4: Error determination for the control system implement- a convolutional neural network (convolutional neural network
ing the line follower functionality for the Duckiebot robot. CNN). The block diagram of the control system using neural
networks is shown in Fig. 6.
The control system of the Duckiebot robot is a discrete Convolutional neural networks are very effective in the
system (implemented on Jetson NANO computer), so the dis- tasks of pattern recognition and image processing. CNNs
crete version of the PID controller has to be used. Taking into repeatedly use the convolution operation, which allows for
account the drive type of the Duckiebot robot differential drive extracting characteristic features (edges, textures, shapes) from
robot, the control signal ui generated by the PID controller is input images on the basis of which a classical neural network
equivalent to the rotational speed ui along the Z axis of the can judge on "what is in the image". Convolutional neural
robot. networks are used in various fields, such as image recognition,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 107

Fig. 5: Operations performed on robot camera images to
determine position error ei .

Fig. 7: Structure of the convolutional neural network used to

control the Duckiebot robot.

The network consists of the first convolutional layer, the

Fig. 6: Schematic of Duckiebot robot control system using
next layer is the MaxPooling layer. Its purpose is to reduce
convolutional neural networks.
the size of the input data. The next layers of the network
are the convolution layer and the MaxPooling layer, and
text analysis, speech processing, facial recognition, control of this arrangement of layers is repeated twice. Each time, the
vehicles, diagnosing diseases from medical images, analyzing convolution layers extract further features of the input image
satellite images, etc. [8], [9], [10]. relevant to the control of the Duckiebot robot, while the
When a CNN is used to control a Duckiebot robot, such a MaxPooling layers reduce the size of the feature maps. The
network receives images from the camera as input (after per- function of the next layer Dropout is randomly turning off
forming several transformations) based on which it generates individual neurons of the network, to prevent the phenomenon
a ui control signal which is interpreted as the ωi speed along of overfitting and to further support the generalization ability
the robot’s Z-axis; see Fig. 6. of the neural network. Before the data is sent to the input
of the actual neural network, it must be flattened, which is
A. Convolutional Neural Network the reasoning behind placing a Flatten layer: to transform
The proposed structure of the convolutional neural network multidimensional data into a one-dimensional input vector to
is shown in Fig. 7 and it should be noted that this is one of pass to the next, Dense layer. The Dense layer is responsible
the possible network structures that can be used. for performing abstract data representation tasks, e.g. may be

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 108

used for classification, regression. The output of the CNN is
the value of the signal ω in the interval [−8; −8], therefore,
the activation function from the Dense output layer is the tanh
function.

B. Data preparation
A neural network is only as good as the data with which
it was trained. Therefore, a very important step in working
with neural networks is to prepare a good set of training data.
In case of the Duckiebot robot and the task it is supposed
to perform (line follower), the training data for the network
consist of pairs: the image from the camera and the corre-
sponding correct ω value, see Fig. 6. The data necessary to
train the network can be obtained by controlling the Duckiebot
robot manually (e.g., with a gamepad) or by using a previously
developed control system with a PID controller by adding a Fig. 9: Operations performed on images from the camera of
script that writes the data into a log file. the Duckiebot robot controlled before the application of the
In addition to the appropriate amount of logged data for CNN controller.
training the neural network, it is important that the data is
sufficiently diverse, that is, that it covers the largest possible
number of cases, situations, events in which the Duckiebot VII. P RACTICAL ADVICE
robot may find itself. Figure 8a shows a histogram of an
example dataset that has been logged. The Duckietown robot software was implemented using the
Robot Opertion System framework (ROS) [11]. It is currently
one of the most popular frameworks for programming robots
of various kinds. The consequence of choosing the ROS
framework is the choice of programming language, and in
the case of Duckiebot robots it is Python. The Duckietown
project also defines how to develop custom software for
robots and enforces the use of containerization technology.
This is a very good practice in accordance with the current
software development trends. A detailed description together
with examples of creating software for Duckie robots is very
(a) (b) well described in the project documentation [12]. The use
Fig. 8: Histogram of data recorded before analysis (a) and of the ROS framework also makes it possible to efficiently
after analysis (b). create and test software without having to constantly upload
new versions to the Duckiebot robot. By configuring the host
It can be clearly seen that the dominant value in this data computer appropriately so that it connects to the ROS server
set for the variable ω is close to 0, and if you use such data to running on the selected Duckiebot robot, you can implement
train a CNN that is supposed to control the Duckiebot robot, it and test your own software conveniently and efficiently, see
is likely that the robot would drive straight very well, while it Fig. 10.
would have problems turning. In such a case, after analyzing
the data, samples with values that significantly dominate other
values should be discarded from the training set, obtaining a
more representative training data set, see Fig. 8b.
Similarly to control of the Duckiebot robot using a PID
controller, it is necessary to adequately prepare the camera
images when controlling the Duckiebot robot using neural
networks. The various stages of processing Duckiebot robot
camera images are shown in Figure 6 (yellow blocks) and are: Fig. 10: Setting up a development environment for the Duck-
cropping the image to the area of interest, conversion to YUV iebot robot.
color space (Y-luminance, UV color, chrominance), filtering
using the Gauss filter, resizing the image and normalizing the The proposed controllers should be implemented as ROS
values of individual image channels to the interval [0; 1]. The nodes. Nodes should read images from the robot’s camera
result of the various operations can be seen in Figure 9. (sent as ROS messages) and generate a ω control signal as

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 109

output. To simplify the control of the Duckiebot robot, it can the advantages and disadvantages of each. Students, in the
be assumed that it moves at a constant linear speed v = const. course of solving the problem presented, must demonstrate
Convolutional neural networks can be created, trained, and theoretical knowledge of the issues and - which is particularly
tested using the keras and tensorflow libraries in the Python important for engineers - acquire the ability to practically
programming language. The advantage of the keras and ten- apply theoretical knowledge.
sorflow modules is also a very good documentation with a
R EFERENCES
very large number of examples, as well as a large community
of developers gathered around them. [1] Y. Huang, “Advances in artificial neural networks – methodological
development and application,” Algorithms, vol. 2, no. 3, pp. 973–1007,
It is also necessary to pay attention to the correct preparation 2009. [Online]. Available: https://www.mdpi.com/1999-4893/2/3/973
of data for training neural networks, each camera image must [2] M. Hagan and H. Demuth, “Neural networks for control,” in Proceedings
correspond to a single, correct value of the ω signal. of the 1999 American Control Conference (Cat. No. 99CH36251), vol. 3,
1999, pp. 1642–1656 vol.3.
[3] L. Paull, J. Tani, H. Ahn, J. Alonso-Mora, L. Carlone, M. Cap, Y. F.
VIII. C ONCLUSIONS Chen, C. Choi, J. Dusek, Y. Fang et al., “Duckietown: an open,
The PID controller is used successfully in many control inexpensive and flexible platform for autonomy education and research,”
in 2017 IEEE International Conference on Robotics and Automation
systems, and its implementation is relatively simple. There are (ICRA). IEEE, 2017, pp. 1497–1504.
also a number of methods and algorithms for adjusting con- [4] M. Chevalier-Boisvert, F. Golemo, Y. Cao, B. Mehta, and L. Paull,
troller parameters for this type of controller. PID controllers, “Duckietown environments for openai gym,” https://github.com/
duckietown/gym-duckietown, 2018.
on the other hand, are not free of disadvantages. One of [5] A. Gupta and A. Easwaran, “A low-cost lane-following algorithm for
them is the requirement of prior knowledge of, even roughly, cyber-physical robots,” arXiv preprint arXiv:2208.10765, 2022.
the model of the process one wants to control. Thus, it is [6] R. Siegwart, I. R. Nourbakhsh, and D. Scaramuzza, Introduction to
autonomous mobile robots. MIT press, 2011.
necessary to identify both the structure of the process model [7] K. J. Åström and R. M. Murray, Feedback systems: an introduction for
and its parameters. Identification tasks are complex tasks, scientists and engineers. Princeton university press, 2021.
requiring a great deal of knowledge about the nature of the [8] Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convolutional
neural networks: analysis, applications, and prospects,” IEEE transac-
process itself. There are also methods for identifying process tions on neural networks and learning systems, 2021.
models based on the results of practical experiments, however [9] W. Rawat and Z. Wang, “Deep convolutional neural networks for image
sometimes it may not be possible to conduct such experiments. classification: A comprehensive review,” Neural computation, vol. 29,
no. 9, pp. 2352–2449, 2017.
When using a PID controller, one should also be aware that [10] P. Almási, R. Moni, and B. Gyires-Tóth, “Robust reinforcement
it was developed for processes, operation of which can be learning-based autonomous driving agent for simulation and real world,”
described by linear models. Unfortunately, the behavior of the in 2020 International Joint Conference on Neural Networks (IJCNN).
IEEE, 2020, pp. 1–8.
vast majority of dynamic systems is described by non-linear [11] M. Quigley, B. Gerkey, and W. D. Smart, Programming Robots with
models. The consequence of this fact is that, in such cases, the ROS: a practical introduction to the Robot Operating System. "
PID controller works using linear approximations of nonlinear O’Reilly Media, Inc.", 2015.
[12] A. F. Daniele. Developer manual. [Online]. Available: https://docs.
systems, which can lead to various errors, inaccuracies, etc. duckietown.com/daffy/devmanual-software/intro.html
Unlike the classic PID controller, controllers using artifi-
cial neural networks do not need to know the mathematical
model of the process they control and its parameters. The
ability to design different neural network architectures, such
as convolutional, recurrent, or deep neural networks, makes it
possible to adapt the neural regulator to the specific process it
is supposed to control. On the other hand, the multiplicity of
neural network architectures and their design means that we
can never be sure whether a given neural network structure is
optimal. The selection of neural controller parameters is done
automatically using appropriate network training algorithms.
The key element influencing the accuracy of neural regulator
operation is the data used for training the neural network. The
disadvantage of regulators using neural networks is the inabil-
ity to demonstrate the stability of operation of the systems
they control. In case of the PID regulator, despite the use of
approximate models of the process, it is very often possible
to prove that a closed control system will operate stably in
any or a certain range of values of variables. Unfortunately,
such an analysis cannot be carried out in the case of neural
regulators.
In summary, the implementation of two different controllers
to perform the same task provides an opportunity to learn

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 110

Usage of workshop programming for turning with
live tools
T. Dodok *, N. ÔXERÕRYÖ* and M. ÔLHUÕDYD*
* 8QLYHUVLW\RI×LOLQD'HSDUWPHQWRI$XWRPDWLRQDQG3URGXFWLRQV\VWHPV×LOLQD6ORYDNLD
tomas.dodok@fstroj.uniza.sk
QDGH]GDFXERQRYD#IVWURMXQL]DVN
FLHUQDYD@stud.uniza.sk

Abstract— The article deals with application of workshop RI YHULILFDWLRQ LV FRPSDULVRQ RI WKH HIIHFWLYHQHVV IRU WKH
programming in manufacturing process. The main goal was SURGXFWLRQ RI H[SHULPHQWDO FRPSRQHnts also at turning
to analyze the possibilities of application and use of the with usage of live tools.
workshop programming system at turning. The solution 7KH PRWLYDWLRQ IRU VROYLQJ WKH JLYHQ SUREOHP ØDV WR
includes verification of the control system Sinumerik FRPSDUH WKH HIILFLHQF\ RI WKH SUHSDUDWLRQ RI WKH 1&
Operate 840D SL for the production of components through SURJUDP IRU GLIIHUHQW W\SHV RI FRPSRQHQWV ØLWK VLPSOH
experimental components. The experimental components VKDSHV DQG YHU\ FRPSOH[ VKDSHV &RPSDULVRQ RI WKH
were divided into 3 groups from the simplest to the most effHFWLYHQHVV RI SUHSDULQJ WKH 1& SURJUDP XVLQJ WKH
complex. The article concludes a comparison of the programming workshop for turning driven tools
effectiveness between workshop programming Sinumerik
PHQWLRQHG LQ WKLV DUWLFOH LV RQO\ SDUWLDO DV ØHOO DV WKH
Operate 840D SL and ISO programming in control system
LQLWLDO VROXWLRQ WR WKHSUREOHPRI WXUQLQJ ,Q WKHSDVW WKH
Sinumerik840D. Part of verification is also comparison of
DXWKRUV KDYH DGGUHVVHG WKH LVVXH RI FRPSDUison of the
the effectiveness of the use this method for creation of NC
HIIHFWLYHQHVV EHWØHHQ ØRUNVKRS SURJUDPPLQJ ,62
programs.
SURJUDPPLQJDQG&$0V\VWHPSURJUDPPLQJLQWKHILHOG
RIPLOOLQJ7KHUHVHDUFKUHVXOWVØHUHSXEOLVKHGLQIRUHLJQ
I. INTRODUCTION DQGGRPHVWLFSXEOLFDWLRQV.
$W SUHVHQW there are used following types of II. D(6&5,37,212)(;3(5,0(17
SURJUDPPLQJPHWKRGVIRUWKHSUHSDUDWLRQDQGFUHDWLRQRI
1& SURJUDPV PDQXDO SURJUDPPLQJ E\ WKH KHOS RI ,62
Machine tool, control panel and software for creating
FRGeVWHDFKSURJUDPPLQJDXWRPDWLFSURJUDPPLQJXVLQJ
SURJUDPPLQJ ODQJXDJHV DXWRPDWLF SURJUDPPLQJ XVLQJ NC program
&$'&$0V\VWHPVDQGØRUNVKRSSURJUDPPLQJ>@$OO ([SHULPHQWV ØDV PDGH DW WUDLQLQJ PDFKLQH (PFR
these types impose requirements for operators at a &RQFHSW 7851  $ )LJ   D SRØHUIXO  D[HV &1&
PDFKLQH RU SURJUDPPHUV LQ WKH &$0 V\VWHP >@ WXUQLQJPDFKLQHØLWKLQWHUFKDQJHDEOHFRQWUROWKDWRIIHUVD
Writing program manually requires a good knowledge of ODUJHYDULHW\RIWUDLQLQJSRVVLELOLWLHVIRUH[DFWLQJWXUQLQJ
WKH,62FRGH)RUFRPSOLFDWHGVKDSHVFDQEHXVHG&$0 and milOLQJSURGXFWLRQV
V\VWHP EXW SURJUDPPHU PXVW NQRØ KRØ WR ØRUN ØLWK 7KH-IROG9',WRROWXUUHWFDQVWRUHXSWRVL[GULYHQ
FKRVHQV\VWHP>@7KHPRGHRIWKH1&SURJUDPFUHDWLRQ tools. Thus, drilling, milling and thread-FXWWLQJRSHUDWLRQV
IRU D FRQFUHWH FRPSRQHQW LV DIIHFWHG E\ WKH VKDpe of a
FRPSRQHQW LWVHOI KDUGØDUH DQG VRIWØDUH OHYHO IRU DUH DOVR SRVVLEOH RXWVLGH RI WKH WXUQLQJ FHQWHU $Q\
SURJUDPPLQJ RSHUDWRUÙV SULRU H[SHULHQFH DQG VNLOOV WKH FRQWRXUV DW WKH IURQW HQG DQG DW WKH ODWHUDO DUHDV FDQ EH
W\SH RI WKH FRQWURO V\VWHP RI D &1& PDFKLQH WKH SURGXFHG with the C-axis.
WHFKQLFDO HTXLSPHQW RI D &1& PDFKLQH >@ :RUNVKRS :RUN DUHD DOORØV WR XVH VØLQJ RYHU EHG Û  PP
SURJUDPPLQJ SURJUDPLQJ DOORØV WR FUHDWH 1C programs VØLQJ RYHU FURVV VOLGH Û  PP WUDYHO RI ;-D[LV 
IRU FRPSOH[ EXW VLPSOH FRPSRQHQWV :RUNVKRS PP WUDYHO RI = D[LV  PP & D[LV DOORØV UHVROXWLRQ
SURJUDPLQJLVDOVRFDOOHGÚFRQ-versational programming” Ü 0DFKLQH FDQ EH FRQWUROOHG E\ VHYHUDO FRQWURO
>@ V\VWHPV HJ 6LQXPHULN '' +HLGHQKDLQ 71&
/LYH7RROLQJDVWKHQDPHLPSOLHVLVVSHFLILFDOO\GULYHQ 6LQXPHULN2SHUDWH'6/DQGRWKHUV:LQ1&
E\WKH&1&FRQWURODQGWKHWXUUHWRIYDULRXVVSLQGOHDQG VHUYHVOLNHKRVWIRUHPXODWLRQFRQWUROV\VWHPVDQGDOORØV
SRØHUHG VXE-VSLQGOH FRQfigurations on CNC lathes to WR LQVWDOO XS WR QLQH FRQWURO V\VWHP $OO V\VWHPV DUH
SHUIRUP YDULRXV RSHUDWLRQV ØKLOH WKH ØRUNSLHFH UHPDLQV FRQWUROOHG ØLWK LQWHUFKDQJHDEOH FRQWURO SDQHOV )LJ  
in orientation to the main spindle. The CNC lathe live DQGFRQWURONH\-ERDUGV
tooling option allows to drive axial or radial driven tools 7KH 6LQXPHULN 2SHUDWH ' 6/ LV SDUW RI WKH
WR SHUIRUP VXFK VHFRQGDU\ RSHUDWLRQV DV GULOOLQJ RU
FKDQJHDEOH FRQWURO :LQ1& :LQ1& DOORØV WKH XVHU WR
tappinJ ERWK RQ WKH IDFH RI WKH SDUW DQG DURXQG WKH
GLDPHWHU 7KH PDLQ VSLQGOH SURYLGHV LQGH[LQJ LQ SUHFLVH OHDUQ DOO &1& LQGXVWU\ FRQWUROV WKDW DUH FRPPRQ RQ WKH
LQFUHPHQWVIRUSDUWSRVLWLRQLQJDQGUHSHDWDELOLW\>@ PDUNHWRQDVLQJOHPDFKLQH2QHRIWKHVHFRQWUROVLVWKH
7KH DUWLFOH GHDOV ØLWK WKH DSSOLFDWLRQ RI ØRUNVKRS 6LQXPHULN2SHUDWH'6/
SURJUDPPLQJ LQ PDQXIDFWXULQJ SURFHVV 7KH VRlution ,W FRUUHVSRnGV LQ KDQGOLQJ DQG IXQFWLRQ WKH SDUWLFXODU
LQFOXGHV D FRPSDULVRQ RI WKH HIIHFWLYHQHVV EHWØHHQ RULJLQDOFRQWURODQGLWFDQEHXVHGRQDFRPPHUFLDO3&
ØRUNVKRSSURJUDPPLQJ6LQXPHULN2SHUDWH'6/DQG 7KHXVHUKDVWKHSRVVLELOLW\WRØRUNRQWKH3&DVØHOODV
,62SURJUDPPLQJLQFRQWUROV\VWHP6LQXPHULN'3DUW RQWKHPDFKLQHFRQWURODQGJHWVIDPLOLDUØLWKDOODVSHFWV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 111

RI FRQWURO SURJUDPPLQJ LQ GHWDLO. )LJ shows layout of FRQWURO V\VWHP &KDQJH FDQ EH PDGH TXLFNO\ DQG LW WDNH
FRQWUROV\VWHP6LQXPHULN2SHUDWH'6/. RQO\ IHØ VHFRQGV ,QWHU-FKDQJHDEOH SDQHO FDQ EH
FRQQHFWHGWRDQ\&RQFHSWPDFKLQHRU3&,Q)LJXUH are
shown FRQWURO NH\ERDUG XWLOL]H E\ WKH H[SHULPHQW $ -
6LQXPHULN'%- 6LQXPHULN2SHUDWH'6/(DFK
NH\ERDUG KDV VSHFLILF NH\-ERDUG OD\RXW DQG NH\V ØLWK
VSHFLILFIXQFWLRQV.

)LJXUH(PFR&RQFHSW7851$ )LJXUH. &RQWURONH\ERDUGVIRULQWHUFKDQJHDEOHFRQWrol panel

:LWK YDULRXV SURJUDPPLQJ PHWKRGV 6LQXPHULN

supports all NC programming methods - from single part
SURGXFWLRQWRPDVVSURGXFWLRQ,QH[SHULPHQWDOSDUWØDV
XVHG WØR DSSOLFDWLRQ RI 1& SURJUDPV FUHDWLRQ XVLQJ
FRQWROV\VWHP6LQXPHULN- the editor of FRQWUROV\VWHPIRU
ØRUNVKRSSURJUDPPLQJ6LQXPHULN2SHUDWH'6/DQG
,62 SURJUDPPLQJ LQ HGLWRU RI FRQWURO V\VWHP
6LQXPHULN'.

III. $33/,&$7,21$1'9(5,),&$7,212):25.6+23
352*5$00,1*$7(;3(5,0(17$/3$57352'8&7,21
,Q WKLV SDUW DUH GHVFULEHG RQO\ H[DPSOHV of
SURJUDPPLQJ RI FRPSRQHQWV LQ WKH 6LQXPHULN 2SHUDWH
' 6/ V\VWHP DUH JLYHQ 3URJUDPPLQJ LQ 6LQXPHULN
' FRQWURO V\VWHP FRQWDLQV WKH EDVLF ,62 FRGH
SURJUDPPLQJNQRØOHGJH>@
)LJXUH. ,QWHUFKDQJHDEOHFRQWUROpanel. 7KH H[SHULPHQWDO FRPSRQHQWV ØHUH GLYLGHG LQWR 
groups from the simplest to WKH PRVW FRPSOH[ )RU
YHULILFDWLRQØDVXVHGDJURXSRIFRPSRQHQWVØLWKVLPLODU
VKDSHV)URPWKLVJURXS ØHUHVHOHFWHGWKUHHFRPSRQHQWV
for further experimentation - FRPSRQHQWQR7KH
H[SHULPHQWDO FRPSRQHQWV ØHUH GHVLJQHG WR YHULI\ WKH
DFFXUDF\ DQG HIIHFWLYHQHVV RI XVLQJ GLIIHUHQW F\FOHV LQ
WKHLUSURGXFWLRQ2QHSXUSRVHRIYHULILFDWLRQØDVDOVRWR
YHULI\IXQFWLRQDOLW\RIWKHDYDLODEOHF\FOHV
7KHILUVWFRPSRQHQW )LJ UHSUHVHQWVDJURXSRIWKH
VLPSOHVWFRPSRQHQWVFRQWDLQLQJWKHEDVLFUDGLXVVXUIDFes,
JUDGXDWHG VXUIDFHV FRQLFDO VXUIDFHV DQG FKDPIHUV 7KH
FUHDWLRQ RI WKH SURJUDP LQFOXGHG PDFKLQLQJ F\FOHV RI
FRQWRXULQJ ILQLVKLQJ DQG JURRYLQJ FRQWRXULQJ 7KH
FRPSRQHQW ØDV PDQXIDFWXUHG IRU WØR FODPSV )LJXUH 
VKRØV D SDUW RI WKH FUHDWHG SURJUDP IRU WKH VHFRQG
)LJXUH. LD\RXWRIFRQWUROV\VWHP6LQXPHULN2SHUDWH'6/ FODPSLQJ
)LJXUH  VKRØV WKH VLPXODWLRQ RI WKH PDFKLQLQJ
,QWHUFKDQJHDEOH FRnWURO SDQHO )LJ   DOORØV XVHU WR SURFHVV IRU WKH SURGXFWLRQ RI WKH ILUVW FRPSRQHQW in the
FKDQJH WKH FRQWURO NH\ERDUGV DFFRUGLQJ WR WKH VHOHFWHG 6LQXPHULN2SHUDWH'6/V\VWHP.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 112

 )LJXUH. )LUVWW\SHRIFRPSRQHQW.

)LJXUH. 'LDORJIRUJURRYLQJF\FOHØLWKGHILQLQJSDUDPHWHUV.

)LJXUH  shows the simulation of the PDFKLQLQJSURFHVV

for the SURGXFWLRQRIWKHVHFRQGW\SHRI FRPSRQHQW in the
6LQXPHULN2SHUDWH'6/V\VWHP.

)igure . 1&SURJUDPLQ6LQXPHULN2SHUDWH.

)LJure  6LPXODWLRQ RI WKH PDFKLQLQJ SURFHVV LQ WKH 6LQXPHULN
2SHUDWH'6/V\VWHP
)LJure . 6LPXODWLRQRIWKHPDFKLQLQJSURFHVVLQWKH6LQXPHULN2SHUDWH
'6/V\VWHP
The third type of FRPSRQHQWV )LJ  FRQVLVWHG RI
FRPSOH[ FRPSRQHQWV PDGH XVDJH WKH OLYH WRROV &\FOHV
7KH VHFRQG W\SH RI FRPSRQHQW )LJ   LQFOXGHV D
IRUWXUQLQJJURRYLQJPLOOLQJDQGIDFHGULOOLQJØHUHXVHG
PDFKLQLQJF\FOHIRUJURRYLQJGULOOLQJDQGWKUHDGLQJ7KH
LQWKHFUHDWLRQRIWKHSURJUDP
PDQXIDFWXULQJ SURFHVV ØDV FUHDWHG WR YHULI\ WKH
7KH SURJUDPPLQJ RI WKH VSLQGOH SRVLWLRQLQJ &-axis
IXQFWLRQDOLW\RItKHF\FOHV)LJXUH displays a dialog for
URWDWLRQ LQWKHGLDORJERx LVVKRØQLQ)LJXUH.
JURRYLQJF\FOHØLWKGHILQLQJSDUDPHWHUV.

)LJXUH. 6HFRQGW\SHRIFRPSRQHQW. )LJXUH. 7KLUGW\SHRIFRPSRQHQW.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 113

 generated NC programV IURP ERWK FRQWURO V\VWHPV ØDV
UHDOL]HGRQWKH&1&ODWKH(PFR&RQFHSW7851$
7KHHIILFLHQF\RISUHSDUDWLRQRILQGLYLGXDOFRPSRQHQWVDV
ØHOO DV WKH FRPSDULVRQ RI WKH JUDSKLFDO HQYLURQPHQW RI
the systems was evaluated.
IV. C203$5,6212)()),&,(1&<%(7:((N 7<3(62)
352*5$00,1*,16,180(5,.O3(5$7(' 6/ $1'
6,180(5,.'
,W LV QHFHVVDU\ WR FRQVLGHU IHØ IDFWV DW FRPSDULVRQ
EHWØHHQV\VWHPV>@
x the appropriateness of the system - shape
)LJXUH. 6SLQGOHSRVLWLRQLQJ. FRPSOH[LW\RIFRPSRQHQWV
x knowledge and skills of the operator or
The ILUVW FODPSLQJ of WKLUG FRPSRQHQW LQ )LJXUH  programmer
simulate the positioning of the holes of the PDFKLQLQJ x the limits of system - FDQ LQIOXHQFH WKH
SURFHVVduring in WKH6LQXPHULN2SHUDWH'6/V\VWHP IXQFWLRQDOLW\RIWKHILQDO1&SURJUDP
x the step of generating NC program
x %DVHG RQ WKHVH IDFWV ØHUH VHOHFWHG WØR
FRPSDUDWLYHPHWKRGV
x The time required for preparation of the NC
program.
x 7KH SURGXFWLRQ WLPH UHTXLUHG WR SURGXFH
FRPSRQHQWV RQ WKH &1& ODWKH &RQFHSW 7XUQ
$.

The time required for preparation of the NC program

In the first method, the time required for the preparation
RI 1& SURJUDPV JHQHUDWHG LQ 6LQXPHULN 2SHUDWH '
6/DQG6LQXPHULN'ØaVFRPSDUHG.

)LJXUH. The simulation of the positioning of the holes

)LJXUH VKRØVWKHVLPXODWLRQRIWKHPDFKLQLQJSURFHVV
RI WKH VHFRQG FODPSLQJ IRU WKH SURGXFWLRQ RI WKH third
FRPSRQHQWLQWKH6LQXPHULN2SHUDWH'6/system.

)LJXUH . &RPSDULVRQ EHWØHHQ ,62 SURJUDPPLQJ DQG ØRUNVKRS

SURJUDPPLQJRQWKUHHH[SHULPHQWDOFRPSRQHQWV.

7KH JUDSK )LJ  VKRØV WKH WLPH VDYHG DW

SURJUDPPLQJ WKH H[SHULPHQWDO FRPSRQHQWV )URP WKe
SRLQW RI YLHØ RI SURJUDPPLQJ WKH FRQWURO V\VWHP
6LQXPHULN 2SHUDWH ' 6/ LV PRUH DGYDQWDJHRXV IRU
SURJUDPPLQJ FRPSRQHQWV 7KLV FRQWURO V\VWHP LV 
PRUHHFRQRPLFDOWKDQWKH6LQXPHULN'.

)LJXUH  7KH VLPXODWLRQ RI WKH PDFKLQLQJ SURFHVV RI WKH VHFRQG The production time required to produce components
FODPSLQJ on the CNC lathe Concept Turn 250 A
,Q WKH H[SHULPHQWDO SDUW WKUHH FRPSRQHQWV ØHUH ,QWKHSURGXFWLRQRIDOOFRPSRQHQWVRQWKH&1&ODWKH
SURGXFHG WR FRPSDUH WKH 6LQXPHULN 2SHUDWH ' 6/ &RQFHSW 7XUQ  $ WKH WLPH UHTXLUHG IRU SURGXFWLRQ
DQG6LQXPHULN'FRQWUROV\VWHPV7KHSURGXFWLRQRI ØDVDOPRVWWKHVDPH )LJ %DVHGRQWKHFRPSDULVRQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 114

LW ØDV IRXQG WKDW WKH SURGXFWLRQ WLPH IRU WKH SURGXFWLRQ 0RVW VXLWDEOH LV IRU D ØLGH UDQJH RI FRPSRQHQWV ØLWK
of the part LQERWKFRQWUROV\VWHPVLVYHU\VLPLODU. VLPSOHVKDSHVLQSLHFHVRUVPDOOVHULHVSURGXFWLRQ

%DVLF IHDWXUHV DQG FKDUDFWHULVWLFV ØRUNVKRS

SURJUDPPLQJFDQEHVHSDUDWHGLQWRVHYHUDOJURXSV

D  (IILFLHQF\ DQG Lntuitiveness - GLDORJXH EHWØHHQ

operator and system
E  8VHU IULHQGO\ LQWHUIDFH – LQSXW ILHOGV LQFOXGLQJ
animated elements
F 6LPXODWLRQRIWKHPDFKLQLQJSURFHVV- simulation of
WKHPDFKLQLQJSURFHVVWRROSDWKDQG
FROOLVLRQEHWØHHQWRRODQGØRUNSLHFH

$IWeU YHULILFDWLRQ RI ØRUNVKRS SURJUDPLQJ 6LQXPHULN

)LJXUH . &RPSDULVRQ EHWØHHQ ,62 SURJUDPPLQJ DQG ØRUNVKRS 2SHUDWH ' 6/ GXULQJ SURGXFLQJ RI H[SHULPHQWDO
SURJUDPPLQJRQWKUHHH[SHULPHQWDOFRPSRQHQWV. FRPSRQHQWVØHUHGHILQHGWKHFXUUHQWV\VWHPFDSDELOLWLHV
7KH\ ØHUH VSHFLILHG FXUUHQW OLPLWV RI V\VWHP DQG WKHUH
7KH FRPSDULVRQ EHWØHHQ WKH SURJUDPPLQJ RI WKH were determinent further steps for experimentaOUHVHDUFK
6LQXPHULN 2SHUDWe ' 6/ DQG WKH 6LQXPHULN ' for the more FRPSOH[ DQG WHFKQRORJLFDO GLIIHUHQW
VHUYHVWRLOOXVWUDWHWKHGLIIHUHQFHLQWKHSUHSDUDWLRQRIWKH FRPSRQHQWVLQFOXGLQJDOVRWKHFUHDWLRQRI1&SURJUDPLQ
NC program in terms of the degree and level of work &$'&$0V\VWHPV.
LQWHQVLW\ DQG WKH XVHG HQYLURQPHQW RI ERWK FRQWURO
systems.
V. C21&/86,216 $&.12:/('*0(17
7KH UHVXOWV LQFOXGH YHULILFDWLRQ RI WhH FRQWURO V\VWHP 7KLV ØRUN ØDV VXSSRUWHG E\ WKH 6ORYDN 5HVHDUFK Dnd
6LQXPHULN 2SHUDWH ' 6/ IRU WKH SURGXFWLRQ RI 'HYHORSPHQW $JHQF\ XQGHU WKH FRQWUDFW 1R ×8-
FRPSRQHQWVLQFOXGLQJWKHOLYHWRROLQJ'XULQJFUHDWLRQRI  3URMHFW WLWOH ,QQRYDWLRQ RI WKH VWUXFWXUH DQG
FRQWHQW RI VXEMHFWV LQ WKH ILHOG RI FRPSXWHU VXSSRUW IRU
1& SURJUDPV E\ XVLQJ ØRUNVKRSSURJUDPLQJ 6LQXPHULN SURGXFWLRQ LQ YLHØ RI WKH QHØ DFFUHGLWDWLRQ RI WKH $96
2SHUDWH ' 6/ KDYH EHHQ IRXQG VRPH GUDØEDFNV ,W engineering study program.
ØDV FDXVHG E\ WKDW WKH FXUUHQW YHUVLRQ RI 6LQXPHULN
2SHUDWH ' 6/ GRHV QRW VXSSRUW VDPH F\FOHV DV R()(5(1&(6
6LQXPHULN 2SHUDWH  E\ 6LHPHQV $*  7KH OLPLWHG >@ M. )LFNR, , 3DKROH - %DOLF. Ú$XWRPDWHG SURJUDPPLQJ RI 1&-
QXPEHU RI DYDLODEOH F\FOHV PD\ KDYH DQ LPSDFW RQ WKH PDFKLQHV (YROXWLRQ RI VKRSIORRU SURJUDPPLQJ.” $QQDOV RI
SUHSDUDWRU\ WLPHV PHQWLRQHG LQ FKDSWHU &RPSDULVRQ RI '$$$0 DQG 3URFHHGLQJV RI '$$$0 6\PSRVLXP )RFXV RQ
HIILFLHQF\ EHWØHHQ W\SHV RI SUogramming. These &UHDWLYLW\ 5HVSRQVLELOLW\ DQG (WKLFV RI (QJLQHHUV SS -
,661
GUDØEDFNV FDQ EH HOLPLQDWHG ØLWK DGGLWLRQDO V\VWHP
>@ -. 3HWHUND, 3. 3RNRUQÞ, ß 9ÖFODY Ú&$0 VWUDWHJLHV DQG VXUIDFHV
updates. DFFXUDF\Ú $QQDOV RI '$$$0 DQG 3URFHHGLQJV RI '$$$0
7KH FRPSDULVRQ EHWØHHQ GLIIHUHQW W\SHV RI 6\PSRVLXP 9RO  1R  )RFXV RQ 1H[W *HQHUDWLRQ RI
SURJUDPPLQJ DQG V\VWHPV VKRØV WKH GLIIHUHQFHV LQ ,QWHOOLJHQW6\VWHPVDQG6ROXWLRQVSS-,661-,
.
FUHDWLRQ RI 1& SURJUDPV ØLWK WKHLU DGYDQWDJHV DQG
>@ 3. 3RNRUQÞ, V. ŠimnaÚThe proposal of a system for determining
GLVDGYDQWDJHVDWWKHSURGXFWLRQRIFRPponents. WKH FKDUDFWHULVWLFV RI PRGHOHG VXUIDFHVÚ 7HKQLàNL Vjesnik -
,W ØDV VKRØQ WKDW WKH WLPH UHTXLUHG WR FUHDWH DQ 1& 7HFKQLFDO *D]HWWH 9RO  1R  SS - ,661 -,
SURJUDPLQ6LQXPHULN2SHUDWH'6/ØDVVLJQLILFDQWO\ .
VKRUWHU DQG PRUH HIILFLHQW IRU WKH W\SH RI FRPSRQHQW >@ L.I. Martinova, N.N. )RNLQÚ$QDSSURDFKWRFUHDWLRQRIDXQLILHG
SURGXFHG E\ OLYH WRROV 7KH WLPH GLIIHUHQFH LQ WKH system of progrDPPLQJ &1& PDFKLQHV LQ WKH GLDORJ PRGHÚ
MATEC Web of Conferences. Vol. 224, art. no. 01101. ISSN
preparation of the NC program also afIHFWVWKHOHQJWKRI 2261- 236X.
WKH1&SURJUDP7KHGLIIHUHQFHLVLQWKHRUGHURIWHQVRI >@ N. ÔXERÕRYÖ, „ &RPSXWHUDLGHGSURJUDPPLQJRI&1&PDFKLQHVÚ
OLQHV LQFOXGLQJ VXEURXWLQHV 7KLV KDG DQ HIIHFW RQ WKH (',6×8×LOLQD,6%1----S.
FRPSRQHQWSURGXFWLRQRQWKHODWKH&RQFHSW7XUQ$ >@ 3. 0LFKDOLN, M. +DWDOD, -. DXSOÖN, „ &UHDWLRQRISURJUDPVE\VKRS
LWVHOI ØKHQ WKH GLIIHUHQFH ØDV  VHFRQGV LQ WKH IORRUSURJUDPPLQJIRU&1&PDFKLQHVXVLQJFDPVRIWØDUHÚ,1(6
 - ,((( WK ,QWHUQDWLRQDO &RQIHUHQFH RQ ,QWHOOLJHQW
SURGXFWLRQ RI RQH pDUW ØKLFK LQ WKH SURGXFWLRQ RI (QJLQHHULQJ6\VWHPVSS-,661, .
KXQGUHGVRISLHFHVFDQEULQJKLJKHUVDYLQJVLQSURGXFWLRQ >@ L.I. Martinova, R.L. 3XVKNRY, N.N. )RNLQ ÚDevelopment of
time. standardized tools for shopfloor programming of turning and turn-
PLOOLQJ PDFKLQHV,” ICMTMTE IOP Conf. Series: Materials
Science and Engineering 709 (2020) 044064, 2020.
Workshop programming allows users to easily
>@ 3. +DQVHQ Ú7KH%DVLFVRI/LYH7RROLQJ,” 7RROLQJ60(, .
SURJUDP FRPSOH[ FRPSRQHQWV HYHQ ØLWKRXW NQRØOHGJH
>@ T 'RGRN 1ÔXERÕRYÖ , .XULF Ú:RUNVKRSSURJUDPPLQJ DV D
RI ,62 FRGHV :LWK WKLV W\SH RI SURJUDPPLQJ LV SDUW RI WHFKQRORJLFDO SUHSDUDWLRQ RI SURGXFWLRQã $GYDQFHV LQ
HQFRXQWHUHG PDLQl\ IRU ODWKHV DQG PLOOLQJ PDFKLQHV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 115

 VFLHQFH DQG WHFKQRORJ\-UHVHDUFK MRXUQDO YRO    '2,
SS-.
>@ I. .XULF ÚNew methods and trends LQ SURGXFW GHYHORSPHQW DQG
planningÚVW,QWHUQDWLRQDO&RQIHUHQFHRQ4XDOLW\DQG,QQRYDWLRQ
LQ (QJLQHHULQJ DQG 0DQDJHPHQW 4,(0  &OXM 1DSRFD SS -
,6%1---- .
>@ &äVDU 0 'LDJQRVLV RI HTXLSPHQWV ,Q 0HFKDQL]DWLRQ DQG
automation eTXLSPHQW IRU SURFHVVLQJ - Cluj-1DSRFD 3XEOLVKLQJ
+RXVH $OPD 0DWHU  - ,6%1 ---- SS -

>@ 0,&+$/,. 3 '83/å. - .2502ß 0 1XPHULFDOO\
FRQWUROOHG PDFKLQH WRROV DQG WKHLU SURJUDPPLQJ 8QLYHUVLW\
WH[WERRNVWHG)9778LQ.RæLFe , 3UHæRY,6%1--
--

>@ 0$'åÔ.)$%,$10.5åç-0(/.2-Optimization
RI 1& SURJUDPPLQJ VW HG 7HFKQLFDO 8QLYHUVLW\ RI .RæLFH
)DFXOW\RI0HFKDQLFDO(QJLQHHULQJ ,6%1---


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 116

 Crimes in Slovak Republic - Statistical Analysis of
Trends
Dominik Dubovec Marek Kvet
University of Žilina University of Žilina
Univerzitná 8215/1 Univerzitná 8215/1
010 26 Žilina, Slovakia 010 26 Žilina, Slovakia
dubovec9@stud.uniza.sk marek.kvet@uniza.sk

Abstract—The purpose of this study was to examine the Finally, an analysis of seasonality will be performed to clarify,
trends in criminal activity in Slovak republic and its higher whether there are any predictable changes in the number of
territorial units between the years of 2011 and 2020. The study crimes across the year.
was focused on felonies in various stages of completion across
both the individual higher territorial units and the country. The The main intention of this study was to identify the time
study was created as a part of project education at the periods, where criminal activity waned, or where there was a
University of Žilina. The dataset, provided to the authors by the descending trend in this activity. These time periods will then
Police Force of the Slovak Republic, contained only be analyzed thoroughly with the help of legal experts in
administrative information about the crimes, such as date of another study, which will focus on identifying the laws and
filing a complaint, the department responsible for resolving the regulations responsible for the positive influence on criminal
complaint, state of the complaint, etc., but did not include activity.
information about the region, where the crime took place. This
information was replaced by the location of the department that II. ANALYSIS OF TRENDS IN CRIMINAL ACTIVITY IN SLOVAK
processed the individual complaints. In order to clarify the REPUBLIC - TOTAL NUMBER OF CRIMES
development in the number of crimes in Slovak republic, an
analysis of trends in the number of crimes was performed on
The total number of reported crimes has been declining
both the national level, and on the level of higher territorial steadily over the observed period, starting with 110015
units. In this part of study, different stages of completion were reported crimes in the year 2011 and ending with 64080
also taken into account. After the analysis of trends, an analysis (58,25% of the number of crimes in 2011) in 2020 as can be
of seasonality of total number of crimes was performed, on both seen in Fig. 1.
the national level and on the level of higher territorial units.

Keywords—Data analytics, crime monitoring, information

systems, Slovak regions

I. INTRODUCTION

Low-code programming is one of the fastest growing

technique of the application development, because it can
incorporate not only IT experts, but also other roles and
professions into the development. Oracle APEX is a popular
low-code application platform offering building scalable
applications deployable in the Oracle Cloud. The web and Fig. 1. Total number of crimes in Slovak Republic
mobile solutions can be built and deployed. In addition to the
applications themselves, it allows you to create an analytical Crimes were assigned to the HTUs (higher territorial unit)
interface and monitor individual parameters over time with the according to the location of the police department that was
support of machine learning and artificial intelligence. responsible for the resolution of the corresponding complaint.
The number of crimes in a HTU was not adjusted to the
Even though there are plenty of available software population of the HTU in question. There were three distinct
environments, Oracle APEX proved to be a perfect tool to do categories of trends identified amongst the total number of
presented research, in which large amount of data needs to be crimes in the higher territorial units:
stored and analyzed similarly to the cases reported in [1, 2, 3,
4, 5]. 1. Decreasing trend during the entire observed
period.
Crimes in Slovakia are a subject of increasing concern for
the country's citizens in recent years. Ranging from driving 2. Stagnating in the first three or four years
under influence [6] to cases of economic crime [7], reports followed by decrease in number of crimes in the
concerning various crimes have become an inherent part of remainder of the observed period.
news in various media. 3. Decreasing in general, with minor increases
This article will explore the distribution of crimes both observed in some years.
across the higher territorial units in Slovakia and the country
as a whole. Subsequently, this article will explore the trends
across different stages of completion of the reported crimes.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 117

 The first group consists of the HTU (Higher territorial III. ANALYSIS OF TRENDS - CRIMES WITH REGARDS TO STATE
unit) of Bratislava and Žilina, where a steady decline over the OF COMPLETION
observed time period was identified, as can be seen in Fig. 2.
There are three states of reported crimes in the dataset
provided:
1. preparation to commit the crime,
2. attempt to commit the crime,
3. and accomplished crime.
Vast majority of crimes included in the dataset (882512
individual crimes) were in the accomplished crime category,
followed by attempts to commit the crime (8956 attempts)
with the amount of preparations to commit the crime being the
smallest category, at only 957 crimes reported falling into this
category over the entire observed period, as can be seen in the
following Fig. 5.
Fig. 2. Number of crimes in Bratislava and Žilina HTU

The second group of territorial units consist of Banská

Bystrica, Košice and Trenčín HTU, where the number of
crimes was almost equal through the years 2011 – 2013 after
which a steady decline followed as evidenced in Fig. 3.

Fig. 5. Number of reported crimes across various stages of completion

The development of accomplished crimes, which can be

seen in the Fig. 6, follows the trend of crimes regardless of the
state of completion in the second group of territorial units, i.e.,
stagnation in the first three years that is followed by decline in
the rest of the observed period.
Fig. 3. Number of crimes in Košice, Banská Bystrica and Trenčín HTU

The last group of territorial units contains the HTUs of

Nitra, Prešov and Trnava. In these territorial units an increase
was observed around the year 2013, which was followed by a
steady decline in the following years, as can be seen in Fig. 4.

Fig. 6. Number of accomplished crimes per year

Attempts to commit the crime follow the declining trend

of the first group of crimes regardless of the state. Apart from
Fig. 4. Number of crimes in Prešov, Trnava and Nitra HTU
the years 2012 and 2013 when there was a minor increase of
8 attempts between these years, the number of attempts to
commit a crime was steadily declining over time as can be
seen in Fig. 7.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 118

 the highest average number in January. This substantial
increase is then followed by a decrease in February (7575,7
crimes per month) after which an increase in average number
of crimes per month can be seen in March (7973,9 crimes per
month). In December, there is also an abnormal difference
(5136 crimes) between the smallest observed number of
crimes (2726 crimes, December 2020) and the highest number
of crimes per month (7862 crimes, December 2011) present.
This difference can be attributed to the Covid-19 pandemic,
which, in Slovakia, began in the March of 2020 [11] and was
still emerging during the December of 2020 [12]. During this
pandemic, there were many restrictions in place, which,
although designed to slow down the spread of the respiratory
disease, probably influenced the number of crimes committed
during the time they were in effect too.
Fig. 7. Number of attempts to commit a crime per year.
11000

The number of preparations to commit a crime does not 10000

follow any of the previously described trends, with the trend

being a periodic one as is evident in Fig. 8. However, this 9000

might be caused by a small sample size with the highest 8000

number of preparations to commit a crime in 2012.

7000

6000

5000

4000

3000

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Fig. 9. Boxplot of number of crimes per month in Slovak republic in

individual months

After identifying seasonality in the number of crimes in

the Slovak republic as a whole, an analysis of seasonality on
higher territorial units was conducted, to confirm whether the
seasonality appears on the level of HTUs too, or whether it is
only a feature of the superset of nationwide crimes. The first
HTU that was examined for seasonality was the HTU of
Fig. 8. Number of preparations to commit a crime
Banská Bystrica. Boxplot of crimes in individual months over
the observed period (Fig. 10.) is very similar to the boxplot
IV. IDENTIFYING SEASONALITY IN MONTHLY DATA
created on the national level, with the exception of the
In order to deepen the understanding of trends present in December of 2020 which is considered to be an outlier, which
the development of crimes, an analysis of seasonality was is a symptom of smaller ranges of number of crimes in every
conducted, with the aim of clarifying whether seasonality is month, which is caused by focusing on a smaller geographical
present in the data, and if that is the case, identify in which area in this part of the analysis of seasonality. The decrease in
months it is present, and how significant it is. number of crimes in December of 2020 is consistent in
Fig. 9. shows, that there is indeed seasonality present in magnitude with the decrease on national level.
the development of crime as has been demonstrated in
previous studies focusing on the seasonality of crime in the
US [8], UK [9] and Canada [10] with the difference (3246,2
crime per month) between means of the month with highest
crime rate (January, 9091,6 crime per month) and the month
with the lowest crime rate (December, 5845,4 crime per
month) being more than half of the average number of crimes
per month in December.
However, the average number of crimes throughout the
year is rather stable most of the year, mainly between the
months of April and October with the average number of
crimes per month being within the range of 7310 crimes per
month +/- approx. 270 crimes per month. After October, the
average number of crimes dwindles throughout November
(6635,5 crimes per month) until the lowest average number of Fig. 10. Total number of crimes in individual months in HTU of Banská
crimes is reached in December, with the crime rates soaring to Bystrica for the years 2011-2020

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 119

 Afterwards, the HTU of the capital city Bratislava was of these decreases can be linked to the COVID-19 restrictions
examined, with results similar to those observed on the [12, 13].
national level. No outliers were identified, unlike in the
Banská Bystrica HTU, with the difference between extreme Another outlier can be observed in December of 2013 with
values of individual months being similar for the most of the 916 crimes reported, a number that exceeds the median
year, with the spring months (March, April and May) having number of crimes for January, which is the month with highest
a slightly greater difference between minimum and maximum average number of crimes in Nitra (941,8 crimes per month
than these months had when analyzed on the national level. on average). The cause of this increase however could not be
deduced in this study, as the underlying causes are presumably
of a origin different than purely statistical.

Fig. 11. Total number of crimes in individual months in HTU of Bratislava

for the years 2011-2020
Fig. 13. Total number of crimes in individual months in HTU of Nitra for the
The following HTU examined was the territorial unit of years 2011-2020
the second largest city of Slovak Republic, Košice.
Seasonality of crimes in this HTU is more evident, than in the Another HTU consistent with the findings in Banská
other previous territorial units, with the boxplot (Fig. 12.) Bystrica and Košice HTUs was the HTU of Prešov. There was
being more similar to the boxplot of the Banská Bystrica HTU a single outlier, the December of 2022. Seasonality in this
(Fig. 10.) than the boxplot of Bratislava HTU (Fig. 11.). HTU is consistent with the findings on both the national level,
Number of crimes in December 2020 is again considered to and in all of the previously analyzed territorial units.
be an outlier with 365 crimes, being significantly lower than
the mean of 909,8 crimes per month.

Fig. 14. Total number of crimes in individual months in HTU of Prešov for
the years 2011-2020

Fig. 12. Total number of crimes in individual months in HTU of Košice for
the years 2011-2020

Subsequently the analysis of seasonality was performed on

the HTU of Nitra, which confirmed the results of previous
seasonality analyses, but in this case, there were more outliers
present, as can be seen in Fig. 13., than observed in the
previously analyzed territorial units. Two of those outliers
were observed in the year 2020 in April, and December. Both

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 120

Fig. 15. Total number of crimes in individual months in HTU of Trenčín for Fig. 17. Total number of crimes in individual months in HTU of Žilina for
the years 2011-2020 the years 2011-2020

HTU of Trenčín was more similar to the HTU of V. CONCLUSIONS

Bratislava, with no identified outliers, but the presence of Data analytics plays an important role in data science and
seasonality is slightly subtler, with the average number of advanced applications in many areas of human life. In this
crimes reported in February being similar to spring and paper, we have focused on specific dataset, which comes from
summer months, however, the decrease observed during the crime and police environment. The main goal of this study
October through December and increase in January is still was to provide the reader with a basic statistical analysis of
present. available data.
The penultimate territorial unit analyzed was the territorial The analysis identified that the year 2013 is presumably
unit of Trnava. This territorial unit was very similar to the the year in which a change in laws with positive impact on
HTU of Trenčín, with some months, having the values in criminal activity took place, which should be further analyzed
February similar to spring and summer months and the same with the help of legal experts. The importance of the year 2013
decrease in October through December followed by an with regards to the criminal activity is further emphasized by
increase in January, which was on average higher than the one the fact, that in this year, there also was observed an increase
in Trenčín. in criminal activity in several higher territorial units across the
country.
There were other positive effects observed, which in all
probability originate in the time period prior to the year 2011,
however the data for this time period did not include the
information for the department that was responsible for the
resolution of crime in all observations, which complicated the
identification of the higher territorial unit in which the crime
took place, therefore keeping this assertion a theory. The
analysis of seasonality proved, that there is seasonality present
in the data, both on the national level and on the level of higher
territorial units, with decrease in total number of crimes during
the months of October, November and December, followed by
a surge of crimes in January. This surge is then, in majority of
higher territorial units, followed by a slight decrease in
number of crimes in February, which leads to the number of
crimes stabilizing during the spring and summer months.

Fig. 16. Total number of crimes in individual months in HTU of Trnava for ACKNOWLEDGMENT
the years 2011-2020
This publication was supported by the Erasmus+ project:
The last territorial unit examined was the HTU of Žilina. Better Employability for Everyone with APEX (2021-1-
This territorial unit showed the same increase in number of SI01-KA220-HED-000032218).
crimes in January as all of the other territorial units, with the
decrease in number of crimes in February being consistent
with the decrease on the national level. The decrease in
October through December was however much less evident,
than in all the other territorial units. The December of 2022
was again identified as an outlier with 318 crimes per month REFERENCES
being slightly less than the average number of crimes in [1] Hrínová, Durneková, M., Kvet, M. (2022). Information system for
December (646,10). asset management in buildings. In: ICETA 2022 : 20th Anniversary

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 121

 of IEEE International Conference on Emerging eLearning
Technologies and Applications : proceedings : 20th Anniversary of
IEEE International Conference on Emerging eLearning
Technologies and Applications: proceedings, Piscataway: Institute
of Electrical and Electronics Engineers, 2022, pp. 141-146.
[2] Kvet, M., Papán, J. (2021). The complexity of the data retrieval
process using the proposed index extension. In IEEE Access:
practical innovations, open solutions: practical innovations, open
solutions, Piscataway: Institute of Electrical and Electronics
Engineers, Vol. 10, pp. 46187-46213.
[3] Kvet, M., Matiaško, K. (2021). Classification rules for temporal
databases. In: IPSI Transactions on Internet Research, Vol. 16, pp.
31-38.
[4] Kvet, M., Toth, Š. & Kršák, E. (2020). Concept of temporal data
retrieval: undefined value management. In: Concurrency and
computation-practice and experience, Vol. 32. No. 13, John Wiley
& Sons, pp. 1-12.
[5] Kvet, M., Kršák, E. & Matiaško, K. (2020). Locating and accessing
large datasets using Flower Index Approach. In: Concurrency and
computation-practice and experience, Vol. 32, No. 13, John Wiley
& Sons, pp. 1-13.
[6] Kuruc, Roman & Sidlo, Jozef & Valent, D & Mlynar, Juraj &
Murarikova, B. (2009). The responsibility of alcohol-impaired road
users in fatal road traffic accidents. Bratislavské lekárske listy. 110.
802-6.
[7] Suhányi, Ladislav & Suhányiová, Alžbeta & Martina, Mokrisova &
Jarmila, Horvathova. (2015). ANALYSIS OF ECONOMIC
CRIME IN SLOVAKIA.
10.5593/SGEMSOCIAL2015/B23/S7.006.
[8] Mcpheters, Lee & Stronge, William. (1973). Testing for seasonality
in reported crime data. Journal of Criminal Justice. 1. 125-134.
10.1016/0047-2352(73)90093-7.
[9] Hird, Celia & Ruparel, Chandni. (2007). Seasonality in recorded
crime: preliminary findings. Research Development and Statistics
Directorate, Home Office. ISBN: 978 1 84726 159 5.
[10] Andresen, Martin A. Malleson Nicolas. Crime seasonality and its
variations across space. Applied Geography, Volume 43. (2013).
Pages 25-35. ISSN 0143-6228,
[11] Ministerstvo zdravotníctva Slovenskej republiky. Slovensko
zaznamenalo prvý prípad ochorenia COVID-19. 2020-03-06, [cit.
2023-03-22]
[12] Ministerstvo zdravotníctva Slovenskej republiky. COVID-19:
Spoločnou zodpovednosťou pomôžeme zvládnuť Vianoce v
nemocniciach. 2020-12-23, [cit. 2023-03-18]
[13] Ministerstvo zdravotníctva Slovenskej republiky. COVID-19:
Buďme zodpovední. Voľný pohyb osôb počas veľkonočných
sviatkov je obmedzený. 2020-04-06, [cit. 2023-03-18]

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 122

 Preparation and implementation of educational
games in the teaching of Controlling in the
construction industry
A. Ďuriš*, J. Smetanková*, R. Ručinský*, P. Mésároš** and K. Krajníková**
* Technical University of Kosice, Faculty of Civil Engineering, Expert´s Institute in Construction, Vysokoskolska 4,
042 00 Kosice
** Technical University of Kosice, Faculty of Civil Engineering, Institute of Technology, Economics and Management

in Construction, Vysokoskolska 4, 042 00 Kosice

adrian.duris@tuke.sk, jana.smetankova@tuke.sk, rastislav.rucinsky@tuke.sk,
peter.mesaros@tuke.sk, katarina.krajnikova@tuke.sk

Abstract— This study aims to investigate the effects of using Educational games require students’ active participation
scientific educational games in teaching on Technical in the learning process. Educational games are, at the
university of Kosice, Faculty of Civil Engineering on same time, considered an effective alternative to
students’ academic achievement and retention of supporting traditional teaching approaches in terms of
knowledge. The study also assessed students’ perceptions of educators’ responsibility, such as inspiring students to
these educational games. the study group for the subject learn [2]. These responsibilities are often neglected in
Controlling in construction consists of a total of 27 students, education because students’ motivation towards and
of which 4 are part-time students and 23 are full-time participation in lessons is usually a challenging task for in-
students. Controlling is one of the subjects taught at the class teaching [3].
engineering degree in the last year of study. The
Educational games are considered an effective
implementation of the study consisted of the preparation of
alternative to supporting traditional teaching approaches in
a new teaching concept of Controlling in the construction
terms of educators’ responsibility, such as inspiring
industry with the implementation of educational games. The
finished draft of the concept was presented at the beginning
students to learn and making learning fun [4]. These
of the Controlling subject, right after the presentation of the
responsibilities are often neglected in education because
standard used teaching style. After the discussion, a survey students’ motivation towards and participation in lessons
was created in the form of a questionnaire, which consisted is usually a challenging task for in-class teaching, as is
of 15 questions with options. Each student answered still discussed in many studies [3]. However, students
individually. Based on the answers from the questionnaire, become more enthusiastic to learn when learning takes
several interesting evaluations of teaching at the Faculty of place in a fun and interactive way [5]. Games have an
Civil Engineering of the Technical University in Kosice were important role in realizing active learning since they
obtained. The results indicated that educational games include both interactive and distinctive elements [6].
improve motivation, interest and are an effective tool in Based on several sources it is possible to state that
providing retention of new knowledge. Students described educational games are an interactive approach to boosting
educational games as informative, fun, and reinforcing in active learning and motivation and encouraging teamwork
their learning, and reported that they were motivating and [7]. Students are motivated and included in the teaching
engaging in the classroom. method can have a more successful learning outcome,
yielding a permanent body of knowledge which can later
Keywords— Educational games, construction, controlling. be recalled [7]. Educational games make learning more
entertaining but also encourage students’ in-class
participation and foster their attitudes towards learning
I. INTRODUCTION [8]. Educators or teachers can use games for various
Traditional didactic teaching is a teacher-centered purposes such as reinforcing a previously learned topic,
method carried out without any interaction between the teaching new concepts, and motivating students to
teacher and students or among students themselves, and participate [9]. Educational games can be used at the
generally leads to boring and ineffective lessons [1]. lesson or exercises, at the beginning and end of a lesson.
Students on Faculty of Civil engineering are prone to Applicated educational games can foster students’ interest
memorizing instead of learning by thorough and motivation towards lessons, or review, reinforce and
comprehension. In order for students to achieve such a assess the learned topics. Educational games promote
deep comprehension, educators generally favor classroom students’ active participation in lessons and thus assist
activities that encourage active learning [1]. Students can with their learning [10]. There is a great deal of research
learn more effectively through active learning, a process revealing the influence of educational games on learning
whereby students directly participate in their own learning [11]. It was not possible to find similar research studies
by interacting with other students to think critically and with an orientation to construction subjects, therefore
bolster the learning of new concepts. During these there was an interest in such a study.
interactions, students might explain a concept to each
other in different ways or express unnoted issues. This is
not only a method of review but also a process requiring
analysis and critical thinking [2].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 123

 II. METHOD of the level of own costs included in the budget
prices of construction production for controlling
The Faculty of Civil Engineering in Košice was
purposes. Selected problems of overhead cost
founded in 1976, so it has been operating for more than 47
years. The Faculty is one of the best educational management.
institutions in the Slovak Republic. Over the years of its x Internal accounting as a tool for the company's
operation it has more than 9 000 graduates in all three controlling activity. Basic differences between
levels of study. Graduates have been employed in various financial and internal accounting. The
positions as construction managers, managers, general relationship between internal (cost) and
managers, successful designers, scientific researchers and managerial accounting.
teachers. During its existence, the faculty has undergone a x Functions and subject of internal accounting.
development, thanks to which it is recognized not only Organization and methods of displaying
within Slovakia but also abroad. With its results in accounting information in internal accounting.
scientific research and pedagogical field, it is currently x Determining actual own costs according to
ranked among the stable faculties of the Technical performance - performance accounting.
University of Košice, which is also evidenced by the Accounting display of aggregate methods of
certificate of international accreditation EUR-ACE. Based determining costs according to performances.
on the above-mentioned facts, selected educational games Influence of production conditions on procedures
are gradually implemented in the educational process [14]. for recording actual costs.
A. Current concept of Controlling in construction x Comparison of assumptions/reality function and
The subject of Controlling in construction deals with all procedure. Methodology for calculating
phases of a construction project - from planning, deviations. 0 breaths. Calculation of deviations.
preparation, implementation to handover. All these phases Analysis of deviations. Proposal of measures to
involve cost tracking as one of the most important aspects eliminate deviations.
for stakeholders. Students discuss and analyse cost Exercises consist of solving a semester assignment
planning, preparation of bills of quantities, preparation of focused on planning and managing costs in a specific
construction firm's input costs to performance tracking on construction company. The evaluation of the assignment
site. The result of the subject Controlling in construction is consists of continuous checks and the following four areas
compared to plan vs actual consumption of resources, are evaluated:
calculation and analysis of qualitative and quantitative x Selection of a construction company and analysis
deviation. of approaches to planning and management in
The aim of the subject is to convey knowledge from the the company.
field of controlling as an effective tool for profit x Planning, management and control of costs of
management and cost reduction, increasing the construction production - Approaches to pricing
competitiveness of construction companies and to (price policy, method of determining prices of
understand the particularities of controlling the production, subcontracting), Analysis of planning
construction industry (analysis, planning, setting goals and and cost management in a construction company,
strategies [15]. Analysis of control of costs of construction
The subject brings knowledge and practical procedures production, calculation of direct and indirect
focused on the application of selected controlling tools for costs, profit.
operational management of construction company costs x Operational controlling of a specific order -
and construction process costs. Lecture topics: Specifics of the order (description of the specific
x Introduction to the subject matter - content and order), comparison of planned and actual costs
objective of the subject. Basic terms. Concept and Determination of deviations
and concept of controlling. Theoretical definition x Evaluation of planned and actual costs -
and definition of controlling. Basic functions and identification of the causes of deviations,
tasks of controlling. The position of controlling analysis of deviations and proposal of a possible
in the organizational structure of a construction solution to improve the functioning of the
company. construction company.
x Strategic and operative controlling. Cost The study was divided from the preparation and
controlling. Operational controlling tools. implementation of educational games in the teaching of
x The current state of the issue of planning own Controlling in the construction industry. The team of
costs in construction production enterprises. teachers prepared a new teaching concept including
Defining the content of own cost planning. Own educational games. This new concept was presented to
costs and efficiency. Current practice and second-year engineering students at the Faculty of Civil
problems of own cost planning in the business- Engineering at the Technical University in Kosice.
economic sphere of construction.
B. Participants
x Starting points and bases for calculating the level
of own costs included in the budget prices of The study was attended by students of the second year
construction production. of full-time and part-time studies of the engineering
degree of the Faculty of Construction of the Technical
x The procedure for calculating the level of own University in Kosice. A total of 27 students participated,
costs included in the budget prices of including four part-time students and twenty-three full-
construction production. The use of calculations

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 124

time students in the field of Technology, Management and proposal for measures for the construction
Economics in Construction. company.
x For all successes – a correctly created
C. Preparation of a new teaching concept assignment, a tender budget with a defensible
The preparation of the new teaching concept of price, the correct calculation of deviations and
Controlling in the construction industry took place in a the selection of the causes of deviations, he
workshop of the subject guarantor together with the receives credit points. With this educational
teachers. The result of the meetings is a proposal for a new game, students gain knowledge and experience
teaching concept of Controlling, which was presented to from various positions - construction manager,
the students. Students were introduced to the new concept construction supervisor, investor, subcontractor,
and evaluated its benefits and obstacles in a questionnaire. contractor, which will be beneficial to them in
practice.
D. Educational game
The new concept of teaching controlling does not E. Data analysis
interfere with modifying the content and scope of the The survey was created in the form of a questionnaire,
subject, it adds fun educational components to the lecture which consisted of 15 questions with options. The survey
and assignments. At the end of the lecture, the student was focused on educational games improve motivation,
takes a quiz called "memory challenge". The quiz will interest and are an effective tool in providing retention of
consist of questions from the lesson and will verify the new knowledge. The interest of students of the Faculty of
students' knowledge and findings from the teacher's Civil Engineering in lectures, learning and evaluation of
explanation. These memory challenges after each lesson education was also investigated. The questions of the
aim to contribute to a better knowledge of the students and survey were to verify whether they consider knowledge
a higher interest in the lectures, which they sometimes games to be informative, entertaining and strengthening
describe as boring and unattractive. The assignment their learning. Total of 27 students filled out the survey.
entitled Planning and cost management in a construction Each student answered anonymously and individually.
company has undergone fundamental changes. The Based on the answers from the questionnaire, several
student establishes and invents his own construction interesting evaluations of teaching at the Faculty of Civil
company. He will manage his own fictitious construction Engineering of the Technical University in Kosice were
company during all phases of controlling (task preparation obtained.
phase, creation of tender budget, creation of production
costing, monitoring and recording of work lists and actual III. RESULTS
consumption of resources, calculation and analysis of
deviations, proposal of improvement measures. Based on The results of the survey revealed and brought different
these phases, he will involve his own fictitious company opinions of students on teaching at the Faculty of Civil
to processes: Engineering, they emphasized the need to innovate and
involve students more in teaching and also to modernize
x Each student creates an assignment (blind the form of teaching, e.g. o educational games to make
budget) for a selected part of construction or studying more interactive. The survey was created in the
renovation. The student must defend his form of a questionnaire, which consisted of 15 questions
assignment, other students analyze the with options. The questionnaire was aimed at evaluating
assignments and look for flaws and errors in the the teaching of subjects and evaluating the new concept of
assignment. Processing the assignment and Controlling teaching. Student responses are analyzed in
possibly detecting errors helps the student earn more detail below.
credit points.
The first question of the survey was the inclusion of a
x These assignments will be shared among full-time or part-time student. The result of this question
classmates and their task will be to process bid divides the answers into 23 full-time students and 4 part-
budgets and apply for contracts. time students. No different answers were recorded in the
x In cooperation with the teacher, an analysis of overall evaluation, for this reason the evaluation is not
bid budgets will take place, the winners of divided for full-time and part-time students. The second
individual tasks will be selected (acquiring credit question was to find out whether they noted or
points) and bid price analyzes (calculation of bid experienced the completed lectures and exercises at the
prices and profit rate). If the student does not Faculty of Civil Engineering as boring. The stories are
succeed in the competition for the order, he has presented in “Fig. 1”, where 22 students requested that
the opportunity to apply for the next order that only some were boring, 4 students marked them as boring,
will be offered. As a result of the competitions, and 1 student is satisfied with the lectures.
each student will succeed or be a subcontractor
of another classmate who wins 2 or more
competitions.
x Monitoring of actual consumption and costs
takes place according to predetermined values
entered by the teacher, and students calculate
quantitative and qualitative deviations.
x Based on the detected deviations, the student
selects from the list of causes only those that are
directly related to his bid budget and proposes a

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 125

 Figure 1 Boredom during lectures and exercises according to students

Figure 3 Educational games enable faster learning of the lectured topic

The third question was the use of educational games
during their studies at the Faculty of Civil Engineering of
the Technical University in Kosice. The results of the The sixth question examined the opinion of the
students, whether educational games support the retention
question on "Fig. 2" brought the following conclusions.
of knowledge. Almost all said yes - 24 students, 2 students
The majority – 20 students, students did not record do not agree with this opinion, 1 student cannot evaluate.
educational games during their studies, a minority of 7
The seventh question examined the opinion of students,
students assessed that they completed and had games
whether educational games support interest in the lesson.
during lessons and exercises. The majority - 22 students said yes, 5 students could not
evaluate and no one marked no.

Figure 4 Opinion on faster learning, retention and support interest

Figure 2 Educational games during the previous studies
The eighth question examined the opinion of students,
The fourth question was whether students can imagine whether educational games strengthen and increase their
the implementation of educational games in teaching at motivation during lectures and exercises. The majority -
the Faculty of Civil Engineering. No one marked the 22 students said yes, 5 students disagree and did not
answer no, 8 students cannot imagine it, 19 students can increase their motivation, two could not evaluate.
imagine it, and in the empty line they also gave tips on
which subjects it would work - Controlling in
construction, Optimization methods, Construction
technology project, Technologies of built processes and
environmental subjects.
The fifth question checked the opinion of students
whether educational games enable faster learning of the
lectured topic (see “Fig.3”). 25 students dominantly
agreed, 1 student marked I don't know, 1 student
disagreed. As a result of the survey, it can be stated that
the prevailing opinion among students is that educational
games help in learning.

Figure 5 Opinion on motivation, construction knowledge

and reduce boredom

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 126

 The ninth question examined the opinion of students, among students. No one fundamentally disagreed with the
whether educational games strengthen construction new concept, which is very encouraging and beneficial for
knowledge and self-confidence. The majority - 19 the research team consisting of the guarantor and teachers
students said yes, 2 students disagreed, six could not for further research.
evaluate.
The tenth question examined the opinion of students, IV. CONCLUSION
whether educational games reduce boredom in lessons and
exercises. The majority - 25 students said yes, two could This study investigated the effect of the use of
not evaluate and no one marked no. entertaining educational games to review and reinforce
students’ knowledge of the Controlling in construction.
The eleventh question examined the students' opinion, The study was divided from the preparation and
whether educational games improve the teacher's implementation of educational games in the teaching of
approach and behavior. The majority - 25 students said Controlling in the construction industry. The team of
yes, two could not evaluate and no one marked no. teachers prepared a new teaching concept including
Educational games clearly improve the cooperation educational games. The preparation of the new teaching
between the teacher and the students. concept of Controlling in the construction industry took
The twelfth question checked the opinion of the place in a workshop of the subject guarantor together with
students, whether learning with the help of educational the teachers. The result of the meetings is a proposal for a
games is not monotonous and boring. The majority - 21 new teaching concept of Controlling, which was presented
students said yes, four could not evaluate and two to the students. Students were introduced to the new
disagreed. concept and evaluated its benefits and obstacles in a
The thirteenth question was directed at students, survey. The study was attended by students of the second
whether they would accept more practical games in year of full-time and part-time studies of the engineering
subjects taught at the Faculty of Civil Engineering. degree of the Faculty of Civil Engineering of the
Almost everyone said yes - 26 students, one student could Technical University in Kosice. A total of 27 students
not evaluate and no one marked no. These conclusions participated. The students commented on the use of
clearly confirm the implementation of practical games in educational games during lessons. Their responses and
teaching. interpretations revealed that the use of educational games
in the teaching process helps students to learn and
improve certain skills, as well as increase their academic
success and knowledge retention. The students agreed the
benefits of educational games:
x they make the lesson more entertaining,
x they promote collaboration among students,
x they encourage active student participation and
teachers,
x they increase motivation towards the lesson and
exercises,
x they make lessons enjoyable and attract attention
to the topic,
x they facilitate learning and provide visual
learning,
x they help students to reinforce their knowledge
Figure 6 The influence of educational games on improving the teacher's by reviewing the topics,
approach, reducing monotony and implementing practical games in
teaching. x they keep students engaged in the material,
x they empower students to correct their own
The last questions were directly related to the subject of mistakes,
Controlling in the construction industry and the x they enable to overcome challenges,
implementation of a new teaching concept. The fourteenth
question was directed at the students, whether they would x they boost students’self-confidence,
accept the change of the concept and the inclusion of the x they improve the evaluation of the teacher by
"Memory challenge" at the end of the lecture. The students and he is more popular.
overwhelming majority of 22 students would gladly
accept such a change, 4 students cannot evaluate, 1 Many studies in the related literature on using games to
student does not consider it necessary and no one chose teach various topics report that teaching with games
the option no. The fifteenth question asked whether the enhances students’ motivation as well as increases their
students would accept the draft of the new exercise motivation, promotes active participation in lessons and
concept (Fictitious company and its functioning). the development of positive attitudes, and makes lessons
Nineteen students agreed, 8 could not say - they were more entertaining and pleasant. Educational games can be
afraid of more complicated getting credit, no one marked used as an effective and powerful tool for students.
the answer no and I don't think it is necessary. The results
of the implementation of educational games on the subject
of Controlling in construction have potential and interest

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 127

 Acknowledgment [7] Patel, J. (2008). Using game format in small group classes for
pharmacotherapeutics case studies. American Journal of
The paper presents partial research results of project Pharmaceutical Education, 72(1), 21.
KEGA 009TUKE-4/2022" An interactive tool for [8] Jones, J. S., Tincher, L., Odeng-Otu, E., & Herdman, M. (2015).
designing a safe construction site in an immersive virtual An educational board game to assist PharmD students in learning
reality. The paper presents partial research results of autonomic nervous system pharmacology. American Journal of
project APVV-22-0576 “Research of digital technologies Pharmaceutical Education, 79(8), 114.
and building information modeling tools for designing and [9] Odenweller, C. M., Hsu, C. T., & DiCarlo, S. E. (1998).
evaluating the sustainability parameters of building Educational card games for understanding gastrointestinal
structures in the context of decarbonization and circular physiology. Advances in Physiology Education, 275(6), S78.
construction”. This article was created thanks to support [10] Persky, A. M., Stegall-Zanation, J., & Dupuis, R. E. (2007).
Students’ perceptions of the incorporation of games into
under the Operational Program Integrated Infrastructure classroom instruction for basic and clinical pharmacokinetics.
for the project: National infrastructure for supporting American Journal of Pharmaceutical Education, 71(2), 21.
technology transfer in Slovakia II – NITT SK II, co- [11] Barclay, S. M., Jeffres, M. N., & Bhakta, R. (2011). Educational
financed by the European Regional Development Fund. card games to teach pharmacotherapeutics in an advanced
pharmacy practice experience. American Journal of
Pharmaceutical Education, 75(2), 33.
[12] Pereira, E., Thom, N. Teaching Construction Project Planning
REFERENCES and Control Using a Management Game Approach, Proceedings
[1] Su, T., Cheng, M. T., & Lin, S. H. (2014). Investigating the of the Canadian Society of Civil Engineering Annual Conference
effectiveness of an educational card game for learning how human 2021 (pp.59-69) [Online]
immunology is regulated. CBE-Life Sciences Education, 13(3), https://www.researchgate.net/publication/360874971_Teaching_C
504-515. onstruction_Project_Planning_and_Control_Using_a_Managemen
[2] Odenweller, C. M., Hsu, C. T., & DiCarlo, S. E. (1998). t_Game_Approach
Educational card games for understanding gastrointestinal [13] Castronovo, F., Stepanik, N., Van Meter, P.N., Messner, J.I.:
physiology. Advances in Physiology Education, 275(6), S78. Problem-solving processes in an educational construction
[3] Glynn, S. M., Aultman, L. P., & Owens, A. M. (2005). Motivation simulation game, Advanced Engineering Informatics, Volume 52,
to learn in general education programs. The Journal of General April 2022, 101574 [Online],
Education, 54(2), 150-170. https://www.sciencedirect.com/science/article/pii/S147403462200
[4] Lujan, H. L., & DiCarlo, S. E. (2006). Too much teaching, not 0465?via%3Dihub
enough learning: what is the solution? Advances in Physiology [14] Mésároš, P., Krajníková, K., Smetanková, J., Čabala, J.: Use of
Education, 30(1), 17-22 interactive tools and support of M-learning in educational
[5] Spiegel, C. N., Alves, G. G., Cardona, T. D. S., Melim, L. M., institutions in the construction industry, INTED2021 Proceedings,
Luz, M. R., Araújo-Jorge, T. C., & Henriques-Pons, A. (2008). 15th International Technology, Education and Development
Discovering the cell: An educational game about cell and Conference, 2021,
molecular biology. Journal of Biological Education, 43(1), 27-36. [15] Information list of Controlling in construction,
[6] Selby, G., Walker, V., & Diwakar, V. (2007). A comparison of https://maisportal.tuke.sk/portal/report.htm
teaching methods: interactive lecture versus game playing.
Medical Teacher, 29(9-10), 972-974.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 128

 Early Detection of Cyber Grooming in Online
Conversations: A Dynamic Trust Model and Sliding
Window Approach
Thomas Nyrem Eilifsen, Bhanu Shrestha, Patrick Bours
Department of Information Security and Communication technology
Norwegian University of Science and Technology (NTNU)
Gjøvik, Norway
patrick.bours@ntnu.no

Abstract—Detecting predatory behavior in online conversa- The objective of this study can be divided into three: (1) to
tions is crucial for ensuring the safety of its users, in particular utilize pre-trained models available for early cyber grooming
minors. This study investigates the performance of various detection, (2) to implement a sliding window approach for
machine learning models in identifying predatory conversations
using a sliding window approach, that looks at a set number of analyzing message sequences in a conversation, and (3) to
messages in an online conversation. The models are evaluated implement a method for updating the risk-score based on the
based on their accuracy, precision, recall, and F1 scores. The content of the messages. To achieve this, a sigmoid function
results showed that certain models performed differently with is used which takes into consideration the previous messages
different window sizes. The study has some limitations, such within the conversation. By constantly updating the risk score
as small dataset sizes and fixed sliding window sizes. This
warrants further investigation, and future research should focus throughout the conversation, the goal is to improve the early
on evaluating the models on a bigger dataset, exploring other detection of cyber grooming activities and enable potential
window sizes, and considering different hyper-parameters for the perpetrators to be identified as early as possible.
evaluation of the risk-scoring. This report is organized as follows: Section II presents
Index Terms—Cyber Grooming Detection, Early Detection, the state of the art of early cyber grooming detection, and
Online Conversations, Sexual Predator, Sliding Window
describes a set of different approaches that have been applied
with the goal of early cyber grooming detection. This includes
I. I NTRODUCTION
machine learning techniques, natural language processing, and
In an increasingly interconnected digital world, the arrival of behavioral analysis. Section III outlines the methodology,
social media platforms and online communities has redefined including the dataset and the pre-trained models that are used.
the way people communicate, learn, and build relationships Section IV describes the analysis that was done in order to get
with each other. While the benefits of these technological the results, followed by Section V which presents the results
advancements are great, they also have their drawbacks. One obtained, including the performance of the models. Section
of the biggest concerns in this domain is the alarming rise VI provides a discussion of the results and their limitations,
of cyber grooming. Cyber grooming can be defined as a as well as suggestions for future research. Finally, Section VII
predatory behavior in which an individual establishes a trust- concludes the report with a summary of the findings and their
based relationship with a minor through online communication implications for cyber grooming detection and prevention.
with the intent of exploiting them for sexual purposes [1].
The devastating effects of cyber grooming on victims and II. S TATE OF A RT
their families have prompted the need for a more proactive Since cyber grooming has increased in recent years, de-
approach to combat this phenomenon. Early detection of cyber tecting and preventing grooming behavior has thus become
grooming activities is crucial for the timely intervention and an important topic in research, with researchers using various
prevention of further harm. techniques to identify predatory conversations.
In recent years, machine learning techniques have been The current state of the art in early cyber grooming detection
employed to identify and detect cyber grooming in online incorporates several techniques, including machine learning,
conversations. By analyzing the textual content and patterns natural language processing, behavioral analysis, and biomet-
of communication, these algorithms can help flag suspicious ric evaluation. Researchers have explored different approaches
interactions between two or more subjects, and hence achieve to identify potential predators and detect grooming behavior
early intervention. In this study, the aim is to utilize pre-trained in online environments.
models based on the PAN2012 dataset [2] and implement a A study by Bours and Kulsrud [3] examined three dis-
sliding window approach that generates risk scores for a given tinct methods (message-based, author-based, and conversation-
set of messages in a conversation. based) combined with multiple classification algorithms and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 129

feature sets. Their findings indicated that an author-based chat logs from Omegle [9] which contains sexual adult con-
approach using a Neural Network classifier with a TF-IDF versations and other abusive language. The next two sources
feature set, or a conversation-based approach with Ridge or are ICR chat logs, and the last source for the dataset is chat
Naive Bayes classifier and TF-IDF feature set was the most conversations from the Perverted Justice (PJ) website [10],
effective. which features transcripts of chat conversations between sexual
The PAN2012 competition [2], was a competition that predators and individuals posing as minors with the goal of
focused on identifying sexual predators in chats. In this com- collecting evidence for potential court cases.
petition, Villatoro-Tello et al. [4] presented a 2-stage approach
with Neural Networks and SVM. This approach achieved a B. Pre-trained Models
precision of 0.9804 and a recall of 0.7874, placing them in first For the analysis, a set of pre-trained models based on the
place in the competition. The highest recall in the competition work of Fauzi and Bours [6] are utilized. These models have
however was achieved by Eriksson and Karlgren’s [5], who been trained on full conversations from the PAN2012 dataset.
achieved a recall of 0.8937. Their method involved modeling There are a total of 24 models.
the attributes of a conversation, such as a chatter’s specific
IV. A NALYSIS
vocabulary usage in comparison to others, the length of the
conversation, and the number of individuals that were involved The following Section gives insight into the activities and
in the conversation. Their submission reached 5th place in the techniques that were employed in order to implement a sliding
competition. window approach with the goal of early cyber grooming
In 2020, Fauzi and Bours [6] conducted a study where detection.
they investigated various machine learning classifiers such as The first step in the analysis was to load and pre-process
Naive Bayes, SVM, Neural Networks, K-Nearest Neighbors, the data from the PAN2012 dataset. The data was structured
Logistic Regression, Random Forest, and Decision Tree. Their in XML files, where each file contained the messages in a
study employed Bag of Words features coupled with an array conversation between two authors. These files were prepared
of weighting methods. The study introduces two ensemble for further analysis by labeling each conversation as either
techniques that enhance the classification process. Their results predatory or non-predatory based on the author IDs of con-
proved to be very promising, and their best approach which firmed predatory authors.
utilized a soft-voting-based ensemble for the first stage, and The next step of the analysis involved the implementation
later Naive Bayes-based method, achieved a F0.5-score of of the sliding window approach. This approach involved ex-
0.9348, which would have placed them in first place in the amining a fixed-size window of messages (e.g., five messages
PAN2012 competition. at a time) and incrementally moving the window throughout
Ebrahimi et al. [7] were the first to utilize Convolutional the conversation. For each window, the pre-trained models
Neural Networks (CNN) for predator detection. They use the were used to generate a risk score based on the content of
same dataset as for the PAN2012 competition, and their most the messages. The idea behind this approach was to capture
effective approach achieved a precision of 0.9157 and a recall how online conversations evolve and provide a more dynamic
of 0.7241, which outcomes would have placed them in third representation of the risk as the conversation carries on.
position in the PAN2012 competition. To update the risk score for each new sliding window in
Vogt et al. [8] explored early sexual predator detection a conversation, the Dynamic Trust Model (DTM) introduced
(eSPD) in chats, by analyzing running chats from the be- by Mondal [11] was adopted. The DTM algorithm calculates
ginning. The authors examined existing datasets and created the change in risk using a sigmoid function, which takes into
a new dataset called PANC for realistic evaluations. Using account the classification score of the current window and four
BERT, they achieved state-of-the-art results for conventional parameters (A, B, C, and D). By tuning the four parameters,
SPD. The study also investigated how limited computational one can tailor the sigmoid function to significantly increase
resources affect detection. the risk score when risky messages are detected, and at the
same time require more messages to decrease the score. By
III. M ETHODOLOGY using this approach one can ensure that a sudden spike in
A. Dataset risky behavior has a considerable impact on the risk score,
This study uses data from the PAN2012 dataset [2], as and at the same time make it more difficult to reduce the
well as a subset of it, containing only 1-on-1 conversations score without a consistent pattern of non-risky messages. The
where a number of conversations where no real chatting took Equation 1 below, adopted by Mondal [11] shows the dynamic
place are removed. The PAN2012 dataset includes records of calculation of risk change (ΔR ), based on the parameters and
a large number of chat conversations. These conversations are the classification score of the current window.
   
a valuable resource for researchers studying the differentia- D × (1 + C1
tion between “normal” and “suspicious” chat conversations, ΔR (sci ) = min −D + 1 sci −A
,C (1)
particularly those involving potential sexual predators. C + exp(− B )

The dataset is put together using transcripts from a total The parameter A represents the threshold that distinguishes
of four different sources. The first source is a selection of between an increase or decrease of the risk. If the classification

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 130

score for the current window (sci ) is equal to A, no change positive cases while minimizing false positives. The Receiver
is made to the risk score. If sci > A, the risk score increases, Operating Characteristic (ROC) curve is used, which balances
and if sci < A, the risk decreases. The parameter A controls the false positive rate (FPR) and the true positive rate (TPR).
the balance between increase and decrease. The threshold with the highest index value was considered
Parameter B dictates the width of the sigmoid function, optimal, representing the best compromise between accurately
while the parameters C and D represent the maximum in- detecting potential predatory conversations and avoiding false
crease and decrease, respectively. These parameters set the positives.
boundaries for the degree of change to the risk score, allowing
for the control of the maximum rate of risk change. V. R ESULTS
To assess the effectiveness of the approach, the performance The performance of the different models was evaluated
of the pre-trained models and sliding window approach was using various values for the parameters A, B, C, and D,
evaluated using various evaluation metrics, including preci- as well as for different sliding window sizes. A total of 500
sion, recall, and F1 -score. These metrics provided insights conversations of each type (predatory and non-predatory) were
into the models’ ability to accurately identify cyber grooming randomly selected from the PAN2012 dataset, and it was
instances, and the trade-offs between false positives and false ensured that all conversations had at least 10 windows. During
negatives. These are also the metrics used in the PAN2012 the evaluation, the A and B parameters were set to fixed
competition [2], meaning that the performance of the approach values, while the C and D parameters were changed in order to
in this paper can be compared to the performance of the find a ratio that would best succeed in significantly increasing
submissions for the competition. the risk score when risky messages were detected, but at the
Precision is an indication of the correct positive predictions same time require a set of normal messages to decrease the
made. It is defined as the ratio of the true positives (the score. During the manual evaluation, the best results were
number of predatory messages correctly classified) to the sum obtained using the parameters A = 0.4, B = 0.1, C = 3,
of true positives and false positives (the number of non- and D = 1, as can be seen in Tables I and II. The tables
predatory messages incorrectly classified as predatory) [12]. show which model was used, the accuracy, precision, recall,
In the context of this study, a higher precision would indicate and the F1 -score, as well as the average number of messages
a lower rate of “false alarms”, meaning the model has made it took before the threshold was reached. The average number
fewer errors in wrongly classifying a non-predatory message of messages before a threshold is reached is calculated by
as predatory. keeping track of the number of messages as the sliding window
Recall, measures the model’s ability to correctly identify the algorithm processes the conversations. The average length of
positive cases. It is calculated as the ratio of the true positives the 1000 randomly picked conversations was 134 messages.
to the sum of true positives and false negatives (the number
of predatory messages incorrectly classified as non-predatory) TABLE I
[12]. A high recall rate would mean the model is successful S IGMOID PARAMETERS (0.4, 0.1, 3, 1) AND A SLIDING WINDOW OF 5
in detecting most of the predatory messages from the dataset. MESSAGES .

The F1 -score provides a single score which is the weighted Model Acc. Prec. Rec. F1 -score Avg.Msgs
combination of precision and recall. An F1 -score is a better bern-nb-binary 0.3884 0.3884 1.0000 0.5594 46.08
measure than accuracy, especially for imbalanced class distri- bern-nb-tf 0.3884 0.3884 1.0000 0.5594 46.08
bution [12]. A higher F1 -score indicates a more balanced and bern-nb-tfidf 0.3884 0.3884 1.0000 0.5594 46.08
dt-binary 0.7125 0.9342 0.2795 0.4302 51.13
better performance of the model in terms of both precision dt-tf 0.6896 0.9474 0.2126 0.3472 44.61
and recall. Equation 2 shows the formula for the F1 -score. dt-tfidf 0.7049 0.9296 0.2598 0.4061 50.65
knn-binary 0.3884 0.3884 1.0000 0.5594 46.08
P recision × Recall knn-tf 0.3884 0.3884 1.0000 0.5594 46.08
F1 = 2 × (2) knn-tfidf 0.7966 0.7814 0.6614 0.7164 46.99
P recision + Recall
lr-binary 0.8410 0.7568 0.8701 0.8095 46.54
For the PAN2012 competition, there was a larger emphasis lr-tf 0.8486 0.7881 0.8346 0.8106 45.45
lr-tfidf 0.8104 0.7407 0.7874 0.7633 46.17
on precision compared to recall, which would indicate that
mult-nb-binary 0.8425 0.7938 0.8031 0.7984 45.03
they deemed it more important that potential suspects were in mult-nb-tf 0.8899 0.8669 0.8465 0.8565 46.04
fact sexual predators, in order to reduce the work of police mult-nb-tfidf 0.8058 0.7138 0.8346 0.7694 45.50
investigators who would have to go through large lists of nn-binary 0.3884 0.3884 1.0000 0.5594 46.08
nn-tf 0.7859 0.6900 0.8150 0.7473 47.01
potential suspects. However, as Bours and Kulsrud suggest [3], nn-tfidf 0.3884 0.3884 1.0000 0.5594 46.08
one could argue that it is equally important to make sure that as rf-binary 0.6621 0.5438 0.8071 0.6497 47.29
many sexual predators are identified as possible, meaning that rf-tf 0.7110 0.6087 0.7165 0.6582 47.91
rf-tfidf 0.6682 0.5654 0.6299 0.5959 46.66
there would be a stronger focus on recall rather than precision.
Since the goal is detecting potential cyber grooming as
early as possible, an optimal threshold was calculated for each In Table I, which presents results for a sliding window size
of the models using Youden’s Index [13]. Youden’s Index is of 5 messages, the model Multinomial Naive Bayes with Term
utilized to determine the ability of a model to correctly identify Frequency (mult-nb-tf) is the top performer. It has the highest

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 131

accuracy of 0.8899, precision of 0.8669, and a F1 -score of Figures 1 and 2 show how two different model’s (Multi-
0.8565. The average number of messages before the risk nomial Naive Bayes with TF and Multinomial Naive Bayes
threshold was reached was 46.04, indicating that the model with TFIDF) scores the risk of each sliding window in a
was able to detect potential predatory conversations relatively randomly chosen conversation. The horizontal axis represents
early. It is however beaten by the Decision Tree with Term the messages in the conversation (grouped by the window
Frequency (dt-tf), Logistic Regression with Term Frequency size), and the vertical axis represents the risk score for each
(lr-tf), Multinomial Naive Bayes with Binary (mult-nb-binary), window. The risk score is the result of the dynamic calculation
and Multinomial Naive Bayes with Term Frequency-Inverse as described in Equation 1, and the values are limited between
Document Frequency (mult-nb-tfidf), when it comes to the 0 and 1. There is also a threshold line indicating the optimal
speed of detecting predatory conversations. risk score threshold for determining predatory behavior based
Some models like the Bernoulli Naive Bayes, Neural Net- on Youden’s index in Section IV.
work, and the K-Nearest Neighbors show a recall score The figures show how the models distinguish between
of 1.0000, implying that they could correctly identify all predatory and non-predatory conversations, and give a visual
actual positive cases. However, their overall performance was insight into how the conversation evolves during a conversa-
compromised by lower accuracy, precision, and F1 -scores, tion.
suggesting a higher number of false positives. In Figure 1 one can see that the predatory conversation
For Table II, where a sliding window size of 10 messages has a high risk-score already from the beginning, indicating
was used, the Multinomial Naive Bayes with Term Frequency that the first messages in the conversation have predatory
(mult-nb-tf) model again achieved the highest performance. characteristics. This continues throughout the conversation,
This time, its accuracy further increased to 0.9500, and its and the score just barely goes below the threshold set for this
recall score was 0.9800, indicating a high capability to detect model.
positive cases. The F1 -score, which balances precision and Figure 2 on the other hand starts with a fairly low-risk
recall, reached 0.9514. The model maintained its ability to score on the predatory conversation. Towards the end, the risk
detect potential predatory conversations relatively early, with increases significantly, indicating that the author of the mes-
an average of 53.74 messages before the risk threshold was sages used some time before they started with their predatory
reached. behavior.
VI. D ISCUSSION
TABLE II
S IGMOID PARAMETERS (0.4, 0.1, 3, 1) AND A SLIDING WINDOW OF 10 In this section an interpretation of the result is given, as well
MESSAGES .
as some limitations of this study, and suggestions for future
Model Acc. Prec. Rec. F1 -score Avg.Msgs research.
bern-nb-binary 0.5000 0.5000 1.0000 0.6666 53.65
bern-nb-tf 0.5000 0.5000 1.0000 0.6666 53.65 A. Interpretation of the results
bern-nb-tfidf 0.5000 0.5000 1.0000 0.6666 53.65
dt-binary 0.6825 0.9398 0.3900 0.5512 53.36 Among the models evaluated, the Multinomial Naive Bayes
dt-tf 0.6750 0.9375 0.3750 0.5357 49.42 with Term Frequency (mult-nb-tf) consistently performed the
dt-tfidf 0.6875 0.9121 0.4150 0.5704 54.63 best for both sliding window sizes of 5 and 10 messages. It
knn-binary 0.5000 0.5000 1.0000 0.6666 53.65
knn-tf 0.5000 0.5000 1.0000 0.6666 53.65
achieved the highest accuracy (0.8899 and 0.9500), showing
knn-tfidf 0.8500 0.8763 0.8150 0.8445 53.30 a strong overall performance across different classifications.
lr-binary 0.8700 0.8978 0.8350 0.8652 53.96 Interestingly, the accuracy of this model improved as the
lr-tf 0.8550 0.8660 0.8400 0.8528 50.71
lr-tfidf 0.8475 0.8971 0.7850 0.8373 52.74
sliding window size increased, indicating that a larger con-
mult-nb-binary 0.9050 0.9355 0.8700 0.9015 52.86 text of messages may improve the performance. The most
mult-nb-tf 0.9500 0.9245 0.9800 0.9514 53.74 accurate model was not the fastest in detecting potentially
mult-nb-tfidf 0.8450 0.8026 0.9150 0.8551 53.17 predatory conversations. For example, both the Decision Tree
nn-binary 0.5000 0.5000 1.0000 0.6666 53.65
nn-tf 0.8875 0.8894 0.8850 0.8871 53.51 with Term Frequency (dt-tf) and the Logistic Regression with
nn-tfidf 0.5000 0.5000 1.0000 0.6666 53.65 Term Frequency (lr-tf) were among the models that detected
rf-binary 0.7525 0.7919 0.6850 0.7345 54.28 predatory conversations faster than the top performing model,
rf-tf 0.7000 0.6695 0.7900 0.7247 53.94
rf-tfidf 0.7175 0.7062 0.7450 0.7250 54.48 despite having lower overall scores. This shows the need for a
balance between detection speed and accuracy when doing
early cyber grooming detection, as faster detection can be
The best performing model, which in this case was the crucial in preventing predatory behavior.
Multinomial Naive Bayes with Term Frequency (mult-nb-tf) Certain models, including the Bernoulli Naive Bayes, Neu-
was also evaluated on the full PAN2012 dataset that was ral Network, and the K-Nearest Neighbors, achieved a perfect
provided for this study. There is a total of 32064 conversa- recall score of 1.0000, indicating their ability to identify all
tions, and the results after the evaluation were the following: positive cases. However, their overall performance showed
Accuracy: 0.9085, Precision: 0.361, Recall: 0.8724, F1 -score: lower accuracy, precision, and F1 -scores. This suggests a high
0.5106. number of false positives, implying that these models labeled

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 132

 Fig. 1. Risk change using Multinomial Naive Bayes with TF.

Fig. 2. Risk change using Multinomial Naive Bayes with TFIDF.

many non-predatory conversations as predatory. Such high B. Limitations and further work
false positive rates can lead to unnecessary intervention which
The analysis presented in this study offers a starting point
highlights the importance of precision in early cyber grooming
into the performance of various machine learning models
detection.
in detecting predatory behavior in online conversations with
the use of sliding windows. However, the study has clear
The evaluation of the full PAN2012 dataset confirms the limitations that must be acknowledged and addressed in future
capability of the Multinomial Naive Bayes with TF model. research. Among the limitations are:
While its precision decreased due to the larger scale of data The analysis was done using a relatively small number of
and possible class imbalance, its high recall and F1 -scores conversations. This limited sample size may not represent the
suggest its applicability to larger datasets, maintaining its diversity that exists in online conversations well enough, and
capability to detect most predatory conversations. as such, the generalizability of the results, and the evaluation

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 133

of the models on this particular approach may not be accurate. not be optimal across all conversation types or models. Further
Future studies should therefore consider testing this approach research should explore adaptive or dynamic window sizes,
on a larger sample to see how the models would perform. which could adjust according to the conversation characteris-
Furthermore, the analysis looked at the impact of different tics. In addition, the sigmoid parameters used in calculating
sliding window sizes on model performance. However, these risk scores were chosen based on the Dynamic Trust Model,
window sizes may not be optimal for all types of conversations without specific tuning for the individual models or the dataset
or for all models. The length of a conversation would greatly used. Future research should look into the possibility of
impact which window size would be the most effective. parameter optimization using techniques like cross-validation
Further investigation is needed to determine the most suitable or grid search, to further improve model performance.
sliding window sizes for various conversation types and mod-
R EFERENCES
els, as well as to evaluate the impact of adaptive or dynamic
window sizes that can adjust based on the characteristics of [1] S. Wachs, K. D. Wolf, and C.-C. Pan, “Cybergrooming: Risk factors,
coping strategies and associations with cyberbullying,” Psicothema,
the conversation. pp. 628–633, 2012.
The study utilized a fixed set of sigmoid parameters inspired [2] G. Inches and F. Crestani, “Overview of the international sexual predator
by the Dynamic Trust Model for calculating risk scores. The identification competition at pan-2012.,” in CLEF (Online working
notes/labs/workshop), vol. 30, Citeseer, 2012.
chosen parameters may not be ideal for all situations or for all [3] P. Bours and H. Kulsrud, “Detection of cyber grooming in online
models, and the performance of the models might be improved conversation,” in 2019 IEEE International Workshop on Information
by tuning these parameters using methods like cross-validation Forensics and Security (WIFS), pp. 1–6, IEEE, 2019.
[4] E. Villatoro-Tello, A. Juárez-González, H. J. Escalante, M. Montes-
or grid search. Further investigation is therefore needed to y Gómez, and L. V. Pineda, “A two-step approach for effective de-
determine the optimal sigmoid parameters for various models tection of misbehaving users in chats.,” in CLEF (Online Working
and situations. Notes/Labs/Workshop), vol. 1178, 2012.
[5] G. Eriksson and J. Karlgren, “Features for modelling characteristics of
conversations: Notebook for pan at clef 2012,” in CLEF 2012 Evaluation
VII. C ONCLUSION Labs and Workshop, Rome, Italy, 17-20 September 2012, 2012.
This study investigated the performance of various machine [6] M. A. Fauzi and P. Bours, “Ensemble method for sexual predators
identification in online chats,” in 2020 8th international workshop on
learning models in detecting early signs of predatory behavior biometrics and forensics (IWBF), pp. 1–6, IEEE, 2020.
in online conversations using a sliding window approach. [7] M. Ebrahimi, C. Y. Suen, and O. Ormandjieva, “Detecting predatory
The performance of these models was evaluated using a conversations in social media by deep convolutional neural networks,”
Digital Investigation, vol. 18, pp. 33–49, 2016.
range of metrics, including accuracy, precision, recall, and F1- [8] M. Vogt, U. Leser, and A. Akbik, “Early detection of sexual predators in
score. The impact of different sliding window sizes on model chats,” in Proceedings of the 59th Annual Meeting of the Association for
performance was also examined. Computational Linguistics and the 11th International Joint Conference
on Natural Language Processing (Volume 1: Long Papers), pp. 4985–
The Multinomial Naive Bayes model with Term Frequency 4999, 2021.
(mult-nb-tf) demonstrated the best performance across multi- [9] “Omegle: Talk to strangers!.” http://www.omegle.com/ [Accessed:
ple metrics and window sizes, with the accuracy improving 14/09/2023].
[10] “Perverted-justice.com - the largest and best anti-predator organization
as the sliding window size increased. This finding suggests online.” http://www.perverted-justice.com/ [Accessed: 14/09/2023].
that the model’s performance can be improved when provided [11] S. Mondal, Continuous user authentication and identification: Combi-
with a larger context of messages. Despite not being the fastest nation of security & forensics. PhD thesis, Norwegian University of
Science and Technology, 2016.
in detection, the model’s balanced accuracy and precision [12] M. Sokolova, N. Japkowicz, and S. Szpakowicz, “Beyond accuracy,
indicate its usefulness in early cyber grooming detection with f-score and roc: a family of discriminant measures for performance
an approach like this. evaluation,” in Australasian joint conference on artificial intelligence,
pp. 1015–1021, Springer, 2006.
When evaluating the Multinomial Naive Bayes model with [13] R. Fluss, D. Faraggi, and B. Reiser, “Estimation of the youden index
Term Frequency on the full PAN2012 dataset, a drop in and its associated cutoff point,” Biometrical Journal: Journal of Math-
precision was observed due to the larger scale of data and ematical Methods in Biosciences, vol. 47, no. 4, pp. 458–472, 2005.
potential class imbalance. However, its strong recall and F1 -
scores suggest the model’s capability to generalize to larger
datasets and effectively detect most predatory conversations.
The Bernoulli Naive Bayes, Neural Network, and K-Nearest
Neighbors models achieved perfect recall scores but suffered
from a higher number of false positives, which can lead to
unnecessary intervention in non-predatory conversations. This
underlines the role precision plays in the context of early cyber
grooming detection.
The study presented here, has its limitations. The limited
sample size of conversations used for the analysis may not
sufficiently represent the diversity in online conversations,
potentially affecting the generalizability of the results. Further,
the study focused on fixed sliding window sizes, which may

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 134

 Embodiment and body awareness and their
impact on student interactions in higher education
M. Filipová*, P. Balco*
* Comenius University Bratislava, Faculty of Management, Department of Information Systems, Bratislava, Slovakia
Filipova20@uniba.sk, Peter.Balco@fm.uniba.sk

Abstract— Higher education in Slovakia is primarily

oriented on academic knowledge, memorization of theory II. EMBODIMENT AND BODY AWARENESS IN HIGHER
and lessons. Based on our practical experience, most of the EDUCATION
education takes place in classrooms, where students sit most
of the time behind desks and listen to the teacher, follow the Embodiment
teaching process, and in the case of practically oriented In the field of health and psychology, the term
subjects, sit in laboratories, at computers or at various embodiment is often explained as the experience of
machines. While the use of information technology has experiencing and living in the body [5]. Wilde [6]
brought innovations to education and some benefits, it has
explains embodiment as "how we live in and experience
often resulted in students spending a large part of the
the world through our bodies, especially through
learning process in a sedentary position. Several studies
perception, emotion, language, movement in space, time,
over the last decade have confirmed the positive impact of
movement and body engagement in the educational process
and sexuality".
not only in learning but also on students' overall well-being. Piran et al. published in 2020 the Experience of
The aim of this paper is to share the experience of using Embodiment Scale (EES) [5] - a fully structured measure
embodiment and body awareness in Project Management of the construct of embodiment experience that allows the
teaching at the Faculty of Management, Comenius range of either positive or negative experiences
University Bratislava. individuals have with life in their bodies to be captured
not only by scientists but also by clinicians.
The importance of the body in the educational
I. INTRODUCTION environment is emphasized by researchers from many
Already during the COVID-19 pandemic, the WHO disciplines - from linguistics and neuroscience to natural
stressed the importance of physical activities and exercise. sciences, telecommunications, education, sport, and
Among the reported benefits of regular physical activity leisure [7].
are improvements in thinking, learning, and reasoning Despite the fact that almost thirty years ago the first
skills, and overall well-being [1]. initiatives came up with the intention that education
More and more attention is being paid to how to should not only take place on a cognitive level but that the
incorporate more movement into the educational process, body should also be involved in learning, and although
and this is especially based on the results of various there are several research pointing to the benefits of
scientific researches. This does not have to be directly in embodiment in education, embodiment is only rarely part
physical education classes, for example, it is of higher education [8].
recommended to encourage students to walk or cycle to
school or for educators to use different movement
activities in different lessons - to break the ice, reduce
stress or even increase curiosity [2].
An experiment with undergraduate students who
physically engaged in an experiment related to a specific
area of physics being discussed showed that there was
increased engagement of the sensorimotor brain systems,
and students who physically engaged in the experiment
performed better on a test related to that area than students
who just sat and listened while the topic was being
presented [3].
“Neuroscience research over the past 10 years has
produced significant evidence that movement and
cognition are favorably linked” [4].

Figure 1 – A model of embodied intelligence by Mark Walsh

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 135

Body Awareness Balance - awareness of one's balance, whether they are
We live in a very busy time, we are daily exposed to a pointing their body more forward or backward, and using
lot of demands, information, tasks. If our mindset is gentle micro-movements forward, backward, and side to
oriented towards processing or fulfilling them, we can side to find their balance; C stands for Core relaxation -
often end up using our body as just a "taxi for the brain" - awareness of their core and stomach and their gradual
ignoring or suppressing our perception of pain, feelings, contraction and relaxation), or work with visualization -
emotions and sometimes even our physiological needs. imagine a loved one we love and care about [8].
Meanwhile, a signal that we are, or we were Upregulation
disconnected from the perception of our own body can be
when we realize in the evening that we've only had one Upregulation is a process in which the nervous system
glass of water all day. Alternatively, whether we and our body are activated, the level of attention and the
consciously consumed nutritionally valuable foods level of energy at the level of the physical body is
regularly during the day or whether the all-day increased [12].
unawareness of hunger results in a peak in the evening in Upregulation techniques can be used during hypo-
front of the refrigerator. arousal, when we need to "wake up" our system or reduce
Awareness of one's own body and what is going on in it the freeze or fawn state. Techniques that can be used for
has multiple benefits. If we can perceive our body's this purpose are, for example, prolonged inhalation
physiological needs such as hunger, thirst, pain, fatigue, or (compared to the length of exhalation), jumping up and
the need to go to the toilet, we can satisfy them. If we can down (which speeds up our heartbeat and breathing),
perceive the emotions in our body, we can understand and massaging the ears with the fingers of the hands, and
process them, which can help us avoid an accumulation of stretching the muscles of the face by using various facial
emotions in the body, which in turn can lead to a situation expressions - as if while yawning, etc. [13]
when our emotions explode, and we can't handle them
(called ‘flooding’) or we may feel as if we are stuck, III. METHODOLOGY
holding our breath, feeling a heaviness of limbs, it may This quasi-experimental study was conducted at the
seem impossible to take any action (called freezing) [9]. Faculty of Management of Comenius University
Awareness of one's own body and what is happening in Bratislava in the academic year 2022/2023 during the
it, including the emotions experienced, is very important winter semester.
in the educational process. Emotions are a central All the subjects were second year Masters students who
motivator of the student experience, research has shown a were enrolled in a Project Management course and
close link between emotions, learning and well-being [10]. belonged to the classes led by authors of this paper. They
Even in the corporate world, some companies are were divided into two groups, control (n=34) and test
already recognizing the importance of individuals (n=49).
connecting to their bodies because it improves skills such The focus was on exploring the inclusion of
as leading others, resilience to stress, improved embodiment and selected body awareness techniques,
relationships within teams, and overall well-being [8]. upregulation and downregulation, and their impact on
One of the basic conditions of mental and physical students’ engagement during teaching as well as students’
health is the ability to regulate emotional reactions to feedback on these elements in the classroom. Two
various events. In order to do so, we need to be aware of research questions were asked in this quasi-experiment:
our emotional states, and this is related to awareness of RQ1: Which of the areas of Body Awareness,
our bodily signals [11]. Knowledge Application, and Request to Share an Opinion
will have the highest engagement score through tracking
Downregulation in the polling platform?
Downregulation is the process by which an individual In the academic year 2022/2023 in the winter semester,
becomes aware of the state they are in, aware of the bodily we used a commercially available, easy-to-use Q&A and
processes going on in their body, and the more they can polling platform for live questionnaires, quizzes, and
become aware of these bodily processes, the more anonymous interaction with students to teach the Project
successful they will be at regulating their emotions in Management course in the second year of the Master's
response to a negative situation [11]. degree.
If we are stressed and tense, we are unlikely to be able In the test group, we asked students questions in three
to think logically or concentrate. Downregulation is a way main areas: Body Awareness, Knowledge Application and
to calm the nervous system, for example by making our Request to Share an Opinion. The control group did not
exhalation longer than our inhalation. It can be used have these interactions during the semester and were
before an exam or presenting in class [8]. taught without the use of polling platforms and the below
Sensitivity to the perception of one's own body and mentioned questions.
what is going on in it supports the downregulation of
bodily arousal and may also support effective emotion 1. Body Awareness
regulation within our social interactions, contributing to At the beginning of the first class of the semester, we
higher levels of experienced positive emotions [11]. asked students the question: "What is your energy level
Techniques for down-regulation include, for example, right now?"
prolonged exhalation (compared to the length of the Students chose their answer from a scale of 1 to 5, with
inhalation), ABC Centering (where A stands for 1 being very low, almost no energy and 5 being a high
Awareness - awareness of one's own body; B stands for level of energy and alertness. After sharing the results in

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 136

each category and the average of the group, we explained questions were part of the questionnaire in which we also
why a sufficient level of energy is necessary for learning, collected students' feedback on the Project Management
thinking, listening, and expressing one's thoughts. course:
Then, we asked students the question: "How do you
know what your energy level is? Where did you go for TABLE I.
the answer?" QUESTIONS REGARDING COURSE EVALUATION
We then had students explore what they used to assess Nr. Description
what their energy level was and share their opinions How do you rate the quality of the Project Management
1.
within the group. exercises from the point of view of content?
For the rest of the semester, we asked students at the How do you rate the teachers from the point of view of
2.
professional skills for Project Management?
beginning of each subsequent class a question about their
energy level at that moment and let them anonymously
reply by using the polling platform. After that we offered The questionnaire was emailed to students after the end
several different techniques for upregulation and of the semester, was accessible online and was not
downregulation for them to experience. mandatory to answer.
We then offered the students a different technique for In this questionnaire we used scale from 1 to 5 as
downregulation each lesson so that they could perceive grades in Slovak republic – 1 for A grade and 5 for E
the difference between the state they were in at the grade.
beginning of the lesson and the state they were in after the
technique. After the time needed for introspection, we IV. RESEARCH RESULTS
offered the students a technique for upregulation and
again allowed them to perceive the difference in their RQ1
bodies. Those who were interested were able to share their For RQ1, we used a polling platform to evaluate the
insights publicly. engagement score, which provides information on the
number of students who scanned QRs on their mobile
2. Knowledge Application phones for relevant questions on a given class day, and the
During the semester, we also used the polling platform number of students who also entered their answer to the
to ask questions related to the application of knowledge question into the platform.
from the theoretical or practical lessons to different areas Each group had their own poll in which they had
of life - whether applying the acquired knowledge to questions and could answer them. We labeled those
projects in personal life (e.g., if they imagine that writing groups Class 1 and Class 2, then calculated the average
their thesis is a project, or their university study is a engagement score for each area:
project, what can be a measurable benefit of that project,
or what are risks or dis-benefits of that project). 1. Body Awareness
At the same time, we asked them questions related to Table II shows that Class 1 had an engagement score on
the application of their knowledge to other projects in questions related to Body Awareness of 80.63%, Class 2
another field of practice that is part of the specialization had this score at 88.15%. On average, the engagement
that the students studied (e.g., how marketing, and digital score for both groups involved in this research was at
marketing students can use the acquired knowledge from 84.39%.
this specialization to deal with the risk that the public will
have a negative attitude towards a construction or TABLE II.
renovation project in their neighborhoods). ENGAGEMENT SCORE FOR BODY AWARENESS

3. Request to Share an Opinion Group Engagement Score

We also used this platform for questions or requests to Class 1 80.63%
express their opinion on topics related to their
Class 2 88.15%
collaboration, the practical techniques we used during
semester, or the use of human skills during practical Class 1 + Class 2 84.86%
sessions (e.g., when using the start-stop-continue
technique, students had the opportunity to express in the
middle of the semester which of the activities we do in 2. Knowledge Application
practical classes they want to stop doing, which they want
to continue doing, and which they want to start doing, or Table III shows that Class 1 had an engagement score
we asked them how they would define trust and what they of 70.59% on the Knowledge Application questions,
think why it is important for teamwork, what are their key Class 2 had this score at 70.34%. The engagement
takeaways from the semester and so on). score in both groups that were engaged in this
research averaged at 70.43%.
RQ2: What will be the course evaluation by students TABLE III.
who had Embodiment and Body Awareness as part of the ENGAGEMENT SCORE FOR KNOWLEDGE APPLICATION
Project Management course compared to students who did
not have this topic in the course? Group Engagement Score
For RQ2, we used a questionnaire that was distributed Class 1 70.59%
to students after the end of the semester. The following Class 2 70.34%

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 137

 Group Engagement Score The results of our quasi-experiment showed that the
Class 1 + Class 2 70.43%
highest engagement scores during classroom interactions
with students were for questions related to Body
Awareness.
3. Request to Share an Opinion Positive views on Body Awareness were also reported
by students in the section of the questionnaire in which
For Request to Share an Opinion, Class 1 had an they were given the opportunity to indicate their views
engagement score of 70.54% and Class 2 had an about Project Management. This field in the questionnaire
engagement score of 72.32%. The engagement score was not mandatory.
in both groups that were engaged in this research was
at an average of 72.51% (see Table IV). RQ2
As a part of the exploration of RQ2, we sent
TABLE IV. satisfaction questionnaires to each group of students,
ENGAGEMENT SCORE FOR REQUEST TO SHARE AN OPINION
which included questions about the evaluation of the
Group Engagement Score course as they had taken it in that semester. The
questionnaire was distributed to both test group and
Class 1 70.54%
control group.
Class 2 72.32% Test group had 49 students in two different classes –
Class 1 + Class 2 72.51% Class 1 and Class 2.
Control group had 34 students in two different classes -
Class 3 and Class 4. The numbers of students in each
4. Summarized Results group, and the response rates in each group, are shown in
the Table VI below.
In general, we observed that engagement scores were
higher for Class 2 than for Class 1 in two of three areas.
TABLE VI.
Fig. 2 shows that in the case of both Knowledge RESPONSE RATE FROM STUDENTS’ GROUPS
Application and Request to Share an Opinion, the
Number of Received
differences were minimal, while in the case of Body Class Nr..
Students Responses
Response Rate
Awareness, the difference in engagement scores was on
Class 1+2 49 17 34.69%
the order of 8%.
Class 3+4 34 19 59.38%

The test group where students had the possibility to

explore Embodiment and Body Awareness techniques as
part of their practical lessons during the semester had a
response rate 34.69% and the response rate of the control
group was 59.38%.

Fig. 3 shows that in the test group, the evaluation of the

questionnaire shows that 85% of the students rated this
subject with a grade of 1 (A) and another 15% with a
grade of 2 (B). No students rated the grades 3 (C), 4 (D)
and 5 (E).

Figure 2 – Engagement Score – Summarized Results

Taking the results for both classes together, Table V

shows that the highest engagement score was in the area
of Body Awareness at 84.86%, following with Request
to Share and Opinion with 72.51% and Knowledge
Application with 70.43%.

TABLE V.
FINAL RESULTS OF ENGAGEMENT SCORE IN ALL AREAS

Nr. Area Engagement Score

1 Body awareness 84.86%
Figure 3 – Evaluation of the quality of the course’s content in test group
2 Request to Share an Opinion 72.51%

3 Knowledge Application 70.43% Fig. 4 shows that Teachers' professional skills were
rated by 100% of students with a grade of 1 (A). No
students rated the grades 2 (B), 3 (C), 4 (D) and 5 (E).

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 138

 Classes that were taught in the test group with
embodiment and body awareness elements had better
ratings. Both test and control groups rated the Teachers'
professional skills equally.

V. CONCLUSION
Although embodiment experts report that students
experience feelings of nervousness when asked to do
something physical (Hrach, 2021), our experience has
shown that when an embodiment is coupled with a brief
explanation of its importance, and modern information
technology is used with the option of anonymous
expression, students positively respond to such activities.
We can agree with Lisa Clughen, that embodied learning
Figure 4 – Evaluation of the Teachers’ professional skills in test group can be a humane and enjoyable approach to modern
academic problems [8], and in doing so can contribute to
The control group had standard curricula with no students' overall well-being - especially in the final years
embodiment and body awareness components. of their university studies, when they have to prepare for
the state examinations and the defense of their theses.
Our intention was both to investigate whether students
Fig. 5 shows that in the control group, the evaluation of are interested in the topic of embodiment and body
the questionnaire shows that 42% of the students rated this awareness within higher education and to obtain their
subject with a grade of 1 (A), 53% with a grade of 2 (B) feedback on this area. To also draw their attention to their
and 5% of students gave it grade 3 (C). No students rated body perception, to be aware of what their energy level is
the 4 (D) and 5 (E). and to know some techniques to both influence their
energy level but also to calm their nervous system. An
interesting finding was that engagement scores when
completing the questionnaire for the course were higher
with just the test group and the feedback was positive.
Although student feedback on lessons that included
body awareness, upregulation and downregulation
techniques was positive even outside of the implemented
experiment, we are aware that this experiment has its
limitations. Students' response rate could have been
influenced by other factors, not only the content of the
practical lessons. Whether the relationship between the
areas covered by the questions and the engagement score
of students engaging in polling platform interactions will
be shown to be significant needs to be further investigated.
Figure 5 – Evaluation of the quality of the course’s content in control Another interesting area of research may be how
group activities related to body awareness, upregulation, and
downregulation affect students' perceived stress during
their university studies.
Also, in this case 100% of students rated Teachers'
professional skills with a grade of 1 (A). No students rated ACKNOWLEDGMENT
the grades 2 (B), 3 (C), 4 (D) and 5 (E) (see Fig. 6).
M. Filipová would like to acknowledge the financial
support provided by Comenius University Bratislava –
Grant mladých UK nr. UK/306/2023.

REFERENCES
[1] World Health Organisation (WHO). (2020). WHO GUIDELINES
ON PHYSICAL ACTIVITY AND SEDENTARY BEHAVIOUR.
https://www.who.int/publications/i/item/9789240015128.
Available at:
https://www.who.int/publications/i/item/9789240015128
[accessed 10 August 2023].
[2] Doherty, A., & Forés Miravalles, A. (2019). Physical Activity and
Cognition: Inseparable in the Classroom. Frontiers in Education,
4. https://doi.org/10.3389/feduc.2019.00105 Available at:
https://www.frontiersin.org/articles/10.3389/feduc.2019.00105/full
[accessed 10 August 2023].
Figure 6 – Evaluation of the Teachers’ professional skills in control [3] Kontra, C., Lyons, D. J., Fischer, S. M., & Beilock, S. L. (2015).
group Physical Experience Enhances Science Learning. Psychological
Science, 26(6), 737–749.
https://doi.org/10.1177/0956797615569355

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 139

[4] Doherty, A., & Forés Miravalles, A. (2019). Physical Activity and individual-techniques-1/awareness/body-awareness/ [accessed 10
Cognition: Inseparable in the Classroom. p.2 Frontiers in August 2023]
Education, 4. https://doi.org/10.3389/feduc.2019.00105, [10] White, W., & Ingram, R. (2023). Reconceptualising postgraduate
[5] Piran, N., Teall, T. L., & Counsell, A. (2020). The experience of taught student experience through the lens of emotions and well-
embodiment scale: Development and psychometric evaluation. being: Moving from explanatory methodology to revelatory.
Body Image, 34, 117–134. International Journal of Educational Research, 117, 102135.
https://doi.org/10.1016/J.BODYIM.2020.05.007 https://doi.org/10.1016/J.IJER.2022.102135
[6] Wilde, M. H. (1999). Why Embodiment Now? Advances in [11] Füstös, J., Gramann, K., Herbert, B. M., & Pollatos, O. (2013). On
Nursing Science, 22(2), 25–38. https://doi.org/10.1097/00012272- the embodiment of emotion regulation: interoceptive awareness
199912000-00004 facilitates reappraisal. Social Cognitive and Affective
[7] Clughen, L. (2014). ‘Embodied writing support’: The importance Neuroscience, 8(8), 911–917. https://doi.org/10.1093/scan/nss089
of the body in engaging students with writing. Journal of Writing [12] Cortina, L. M., Sandy Hershcovis, M., & Clancy, K. B. H. (2022).
in Creative Practice, 7(2), 283–300. The Embodiment of Insult: A Theory of Biobehavioral Response
https://doi.org/10.1386/jwcp.7.2.283_1 to Workplace Incivility. Journal of Management, 48(3), 738–763.
[8] Clughen, L. (2023). ‘Embodiment is the future’: What is https://doi.org/10.1177/0149206321989798
embodiment and is it the future paradigm for learning and [13] Walsh, M. (2017). Centring why mindfulness alone isn’t enough
teaching in higher education? Innovations in Education and (pp. 45–48). Copyright Licensing Agency Ltd.
Teaching International, 1–13. https://embodiedfacilitator.com/product/centring-why-
https://doi.org/10.1080/14703297.2023.2215226 mindfulness-alone-isnt-enough-e-book/
[9] University of Virginia. (n.d.). Body Awareness. Retrieved August [14] Hrach, S. (2021). Minding bodies: How physical space, sensation,
12, 2023, from and movement affect learning. West Virginia University Press.
https://www.medicalcenter.virginia.edu/wwp/positive-practices-
to-enhance-resilience-and-improve-interpersonal-communication-

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 140

 Artificial intelligence in education, issues and
potential of use in the teaching process
D. Gabriska*, K. Pribilova**
* University of SS. Cyril and Methodius, Department of Applied Informatics and Mathematics, Trnava, Slovakia
darja.gabriska@ucm.sk
katarina.pribilova@ucm.sk

Abstract—These instructions give you basic guidelines for learning. Algorithms themselves are procedures that
preparing camera-ready papers for conference proceedings. perform a certain task [3].
In the modern educational space, artificial intelligence (AI) In practice, artificial intelligence is used as a support
is increasingly explored. The application of generative tool for specialists. For example, the AI-Pathway
artificial intelligence in the learning process is a current Companion software supports clinical decision-making
trend. It has a high potential for prospective solutions in the and facilitates the determination of diagnostic and
field of educational development. The development and therapeutic procedures. Artificial intelligence in certain
spread of technology is transforming teaching methods and processes can evaluate either images or find suspicious
resources. However, it inevitably brings about a
areas. Based on this they determine the probability of
transformation of the approach to the teaching process from
tumour development [4].
the point of view of both students and teachers. The
introduction of new technologies into the field of education Developments in the field of artificial intelligence have
provides space for professional discussions. These made it possible to replace experts in the educational field
discussions are mainly based on the analysis of the use of to a certain extent. The algorithm can perform complex
artificial intelligence in the educational environment or the tasks that contribute to the improvement of the
scientific community. Artificial intelligence is getting more educational process. For example, an extension of natural
and more into learning processes. At the same time, it is language processing includes an algorithm that analyzes
oriented towards both the student and the teacher. and interprets human speech. These algorithms are, for
Identification and comparison of expert opinions and example, often used in teaching foreign languages. Input
scientific approaches are underway. This makes it possible data based on natural language provides the possibility of
to present a promising vision of the processes of integration authentic communication within speech practices.
of generative artificial intelligence into education This Artificial intelligence brings various advantages. One of
makes it possible to push a new direction in the processes of them is adaptive education. The teaching method works
integrating generative artificial intelligence into education. on interactive mechanisms and considers the individual
The forecast and analysis of the use of generative artificial needs of each student. Adaptive learning options based on
intelligence in the university environment is carried out on artificial intelligence move students forward based on
the basis of current preferences. From this, it follows that feedback [5]. This feedback enables the identification of
artificial intelligence is a functional tool that allows to weak points in the student's knowledge. And getting such
optimization of many different operations. These operations feedback is almost immediate. In the case of a problem
contribute to the organization of an effective educational area, the algorithm analyzes the data and offers suitable
process. The increasing speed of introducing new materials to improve knowledge.
technologies into education or research determines the
direction of future education in the world. Many educational programs are using NLP (Natural
Language Processing) technologies. Grammarly is an
online platform with a text processing function, capable of
I. INTRODUCTION analyzing not only spelling and punctuation but also the
rhetorical positioning of the text based on a built-in
Intensive changes in the educational system are brought
database of contextual sentences. Other applications such
about by the dynamic development of the scientific and
as Babbel, Virtual Talk, Mondly, Quilet or Duolingo.
technical direction. The current stage of technological
These applications use a multimedia interface that
development of society is characterized by the dynamic
provides an opportunity for students to practice their
integration of artificial intelligence into various spheres of
chosen language.
human life, including the fields of science and education
[1,2,3]. The rapid increase in the volume of data, and the ChatGPT is currently the most discussed representative
improvement of technologies condition the ability to of artificial intelligence. Chat GPT is a large-scale
effectively navigate in different areas. Modern language model capable of generating human text, trained
technologies make it possible to efficiently acquire, on a huge dataset. The program uses a deep learning
process or store information. Among the tools offered, for technique to search several terabytes of data. This data
example, predictive analysis, natural language processing contains billions of words to create answers to questions
and machine learning are used. One of the most common [6].
programs created based on artificial intelligence ChatGPT can play an important role in teaching. The
technologies such as natural language and machine usefulness of the chatbot in adaptive education is mainly
learning is chatbots. The algorithm is trained using deep supported using intellectual data analysis. ChatGPT is

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 141

based on a learning neural network that allows it to adapt in the educational process. In the same way, the potential
to the wishes and individual preferences of the user. degree of depth of the introduction of changes that are
Another advantage of ChatGPT is the ability to formulate gradually entering education due to the influence of
a response without waiting. In this way, communication artificial intelligence is also addressed. In terms of
represents the maximum similarity of a real dialogue with opinions, we can single out two main groups:
a teacher. • Total ban – a complete ban on the use of technology
ChatGPT developed based on artificial intelligence such as ChatGPT. Competent representatives of
technologies, can significantly change the research universities refer to inaccuracy, plagiarism or cheating in
activity of scientists and students. Can perform routine the use of artificial intelligence in seminar or final theses.
work in searching and processing literary sources, based Restrictions on the use of ChatGPT have already spread to
on current requirements. While selecting the most relevant several educational organizations in different countries.
sources for the query or keyword. It allows us to help As mentioned above, these countries include, for example,
researchers analyze and understand huge volumes of the USA, Great Britain, France and Japan. Various
textual data [7]. solutions are being sought to prevent the use of artificial
The use of artificial intelligence in various countries has intelligence. The main ones include, for example,
raised concerns and debates. Several universities have blocking access from school networks, which, however,
offered a way out in tightening restrictions on the use of does not allow control of use in the home environment.
artificial intelligence during the educational process. Another solution is to use artificial intelligence itself to
Moreover, the uncertainty in the correctness of the recognize texts that have been generated using artificial
information content offered by the neural network also intelligence. Intelligent systems are being developed that
serves as a basis for concern. For example, the Institute of use neural networks to figure out how the text was
Political Studies (Sciences Po) sent an email to students, created. The GPTZero service belongs to this technology.
where they banned the use of not only ChatGPT but all The app can recognize mixed text and highlight sentences
tools based on artificial intelligence [8]. Other universities most likely to be generated by AI. Another online tool is
around the world are gradually being added, for example, ContentDetector.AI. It is a free tool for identifying content
some universities in Japan, Great Britain, or public generated by artificial intelligence, including AI models
schools in New York or Seattle. [9,10,11,12] On the such as Chat GPT, GPT-3 or GPT-4. It contains advanced
contrary, some universities incorporate these new algorithms that analyze the text in detail. The application
technologies into the educational process. They encourage evaluates an estimated percentage score based on the
educators to be proactive in using artificial intelligence. probability by which tools the text was generated.
Rather, educators should be facilitators who offer an Probability represents the possibility of content generated
understanding of strengths or limitations in the use of by AI tools or software.
artificial intelligence. • Cooperation – the possibility of cooperation, which,
Artificial intelligence in the higher education process, however, means the need for changes and the
from the point of view of traditional assessment, has not development of new approaches to the evaluation of
yet created a risk of replacing student activities. The students' works. It is also necessary to change the ways of
emergence of the generative model and its increased use organizing the educational process. This process should
by students to generate content while completing tasks has support various modern technologies and options. Using
significantly changed this situation [13]. The future of ChatGPT and AI-based technologies in the learning
traditional education in the context of the expansion of process allows all stakeholders to set routine activities to
generative artificial intelligence is a current topic. an automated process. With such a setting, it is possible to
ensure an increase in the quality and speed of preparation
II. ANALYSIS OF THE PERCEPTION OF ARTIFICIAL of educational materials, analysis of student works or
INTELLIGENCE IN THE EDUCATIONAL ENVIRONMENT
objective evaluation of results. Innovations in education
are affecting the way we teach, learn, and assess. The
Artificial intelligence in the form of a text-processing entire educational system must adapt to new technologies.
algorithm has aroused great interest, especially in the field Students should be evaluated based on the quality of
of education. ChatGPT began to be used in masse for text work, understanding of the topic and demonstration of
generation. These texts represented various essays, knowledge in a specific area. That is, it is not enough for
scientific articles as well and students' final theses. the student to know the answer, he should also be able to
According to experts, a paper or essay written using answer how he came up with it. Support for scholastic
ChatGPT technology is sufficient for evaluation. This work that belongs to the skill of students. In this area, e.g.
shows the possibility of completing subjects that assume dr. Boris Steipe, professor emeritus at the Department of
skills in the field of content and text formulation. Biochemistry at the University of Toronto, considers the
Despite these facts, there are different views on the emergence of artificial intelligence to be a revolutionary
issue. These views make it possible to identify the basic step in the field of education. He recommends teaching
distribution of groups. Groups are identified mainly based students to think with artificial intelligence instead of
on opinions or procedures that are applied in the teaching banning its use. However, work created using artificial
process. Stakeholders are an integral part of education intelligence is still considered plagiarism. This means that
[14,15,16,17]. Among these groups, we can include, for students must research the area and provide sources and
example, the management of the university, which sets the references [17]. The professor created a project for
rules for the use of artificial intelligence. Other groups that teachers, Sentient Syllabus, which seeks to provide
are part of it are various educational agencies, experts in educators with guidance on how to teach students to use
the educational environment, scientists and, last but not ChatGPT to accelerate academic work. ChatGPT
least, students. The question arises of using generative AI OpenAI's spokesperson sees artificial intelligence as a

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 142

useful tool in the field of education. However, it is also Grammarly uses artificial intelligence to analyze English
necessary to keep in mind the possible misuse of artificial text and suggest changes to remove errors and improve
intelligence, therefore the entire school system must adapt quality. Carnegie Learning - a service providing
to new modern technologies [18]. In this regard, it is educational courses using AI to achieve better results.
important to develop appropriate regulations, codes of Artificial intelligence technology together with cognitive
ethics and academic integrity policies in universities. It is research provides personalized education. Quizlet - an
equally important to establish definitions of plagiarism interactive educational platform for creating content in a
when using generative AI. Another task in the educational playful way. Duolingo - a platform for teaching languages
environment is to teach students to work with new and translating foreign language text. ThinkerMath -
artificial intelligence tools. applications for children helping children learn
mathematics. Various games are used in the teaching
III. POSSIBILITIES OF USING ARTIFICIAL INTELLIGENCE process. Netex Learning – tools for teachers to create
IN THE FIELD OF EDUCATION discussions, and personalized tasks, while they can
provide a visual representation of the student's progress.
The integrated development of computer and intelligent Gradescope AI - a digital tool for educators providing
information technologies made it possible to approach the clear assessment of students. It is based on a combination
creation of strong artificial intelligence [19]. Artificial of machine learning and artificial intelligence to simplify
intelligence uses models to create various information evaluation. The teacher can use it to evaluate exams, do
materials. On-demand, it can generate texts, images, online homework, or prepare projects. Fetchy – a
videos, etc. It represents a digital tool that works with a lot generative platform designed for educators. It provides
of information. Technologies based on artificial streamlining of tasks such as creating lessons, newsletters,
intelligence can be widely used in the educational emails, etc. It provides the possibility to improve teaching
environment. Exist many tools to create helpful methods, optimize time management or generate results
educational content. Applications that make it possible to based on specific requirements. Nuance's Dragon - a
create an effective learning environment for interaction speech recognition application for both students and
between students and teachers. Other applications help to educators. It helps students who have a writing problem to
increase, for example, the productivity of learning foreign rewrite texts. The tool also supports voice commands,
languages. which makes it possible to work for students with special
The latest development in the field of artificial needs. Ivy – chat bot specially designed for universities
intelligence uses an algorithm for data analysis, the and colleges. They help with many parts of the university
collection of information is used in the field of education, process such as applications, admissions, tuition,
e.g. how: deadlines and more. It provides students with various
• Adaptive learning – offers appropriate materials based information such as loans, scholarships, grants, etc. This
on students' knowledge and goals. The system can identify approach can be incorporated into different departments
insufficient knowledge in a certain part. It will then offer based on the ability to develop specialized chatbots for
the necessary materials to reach the required level. each of them. Cognii - Cognii is a virtual educational
• Interactive teaching systems – involvement in the assistant and is one of the basic tools of artificial
educational process of various technologies, such as intelligence. It uses conversational technology to help
virtual reality. students with open-ended responses and improve critical
thinking skills. It also provides personalized coaching as a
• Intelligent teaching system – interaction between the
virtual assistant. Feedback is tailored to each student in
student and the computer system
real time. Knowji – an application designed for language
• Testing – topics and questions are generated based on learners. Based on research, it can help students learn
the student's level of knowledge faster. Artificial intelligence monitors the learning of each
• Automatic creation of the educational process. word. Based on this, he can then predict when students are
Processing of materials that are necessary to create a likely to forget him. These skills are achieved through a
course. Artificial intelligence uses data analysis, drawing graded repetition algorithm that allows students to learn
conclusions and making decisions when creating courses. better over time. Plaito - acts as a coach, providing
AI-based systems assess the student's level of prompts and suggestions to move students forward in
knowledge, provide feedback, identify weaknesses, and writing, discussion, and collaboration in new and exciting
provide guidance to improve understanding. Applications ways. Using artificial intelligence, this tool delivers the
have a wide range of uses. benefits of personalized learning – deep understanding,
These applications include, for example: confidence, clarity, and empowerment – for all learners.
Querium - artificial intelligence that enables students to
Pearson WriteToLearn is used to improve spelling learn critical skills. It is based on the adaptation of the
skills. It is a system that processes natural language. Third lessons, which is progressed gradually. It has possibilities
Space Learning – a system based on teaching mathematics of use for educators as well. It helps teachers gain insight
with the support of artificial intelligence. They provide into students' study habits. Based on the analysis of the
interactive classrooms, and personalized education answer and the time spent on tutoring. It offers areas in
focused on the individual. Little Dragon - an educational which the student must improve. Century Tech - Creates
application combining shape tracking, machine learning personalized learning plans for students based on the use
and emotional perceptions, which adapts the teaching of cognitive neuroscience. These plans reduce the work of
content after evaluating them. Educational games using educators. Provides feedback and recommendations for
methods of reading emotions are being created. tutoring, in case of gaps in knowledge. Carnegie Learning
Grammarly - an online platform based on artificial – offers many interesting solutions in the field of
intelligence that helps to communicate in English. mathematics or foreign languages. It won the "Best

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 143

AI/Machine Learning Application" award at the Tech • Creating an essay outline – the possibility to make a
Edvocate Awards. basic outline for an outline for a defined topic, which the
student then completes.
IV. CHATGPT AND POSSIBILITIES OF USE • Individual approach - ChatGPT enables the immediate
creation of individual tasks adapted for the student.
A. Possible uses from the student's point of view • Creating interesting lessons – within the development
In the previous chapter, applications based on artificial of activities, it is possible to use artificial intelligence to
intelligence were listed that can help in the education create fun activities in the classroom.
process. As already mentioned, technologies are gradually • Generating discussion questions or creating review
getting more and more into the field of education. It is questions - ChatGPT can be used as a discussion partner.
necessary to teach students to properly use a tool that can
subsequently be helpful, such as Excel. Modern • Creating critical thinking through feedback - Critical
technologies in the form of artificial intelligence are thinking applied to a real person may not be easy.
becoming part of the teaching process. Various educators • Grammar and vocabulary - improving grammar and
encourage their students to use artificial intelligence in a vocabulary, as well as sentence structure.
creative way to study the issue [21, 22]. For example, as a • Generating titles, abstracts or ideas – ChatGPT can
stand-in for a specific person who will act as a partner in a read and remember the given topic as well.
discussion on a given topic. The student receives feedback • Organizing and managing time - artificial intelligence
on the arguments used and thus creates a new perspective. can help organize time and create study procedures,
ChatGPT itself has a huge range of uses for students curricula, and lesson plans.
[23,24]. It is possible to mention only a part of the
possible use of artificial intelligence. Basic possibilities of V. QUESTIONS OF ADVANTAGES AND DISADVANTAGES
using artificial intelligence from the student's point of OF USING ARTIFICIAL INTELLIGENCE
view:
• Summarizing Long Essays and Articles – For long A. The advantages of AI technology.
articles or essays, students may benefit from a summary. As can be seen from the applications mentioned in the
• Plot ideas and creative plots – in the case of tasks previous chapter, the possibilities of use are great.
aimed at creating creative texts or stories, the GPT chat is Artificial intelligence can write essays, and create and
a useful tool. answer questions on a given topic. Create interesting
• Generating ideas - in this case, it is possible to use it content based on the stated requirements. For educators
to get a framework for the assigned work or to gather and students, there are tools based on artificial intelligence
information and ideas. that help organize the educational process. They make it
• Writing messages and e-mails – assists in writing e- possible to monitor the progress of students and prepare
mails. Formal communication is often required in a the following lessons based on this. A big advantage of
university environment. using artificial intelligence is for disadvantaged students
who have trouble fitting into the regular education system.
• Summary of notes - arrangement of notes from
lectures in a meaningful and concise way. The tool is It gradually became a part of the software used up to
useful for summarizing notes based on, for example, a now. Microsoft has integrated ChatGPT into the Teams
word limit. communication program. A neural network captures the
main points of the meeting in text during calls, marks
• Practicing is learning for exams and tests - in terms of important places on the video timeline, stores timecodes
the possibility of using GPT chats, one of the best features divide the recording into discussed topics, and enters
is the ability to understand the topics and create different scheduled meetings into the calendar [27]
questions accordingly.
Applications with artificial intelligence technologies
• Use as a spelling and grammar checker – using can help researchers analyze and interpret large volumes
artificial intelligence to assess grammar and spelling. of information.
B. Possible use from the point of view of the teacher B. Disadvantages of AI
Educators, as well as students, have a huge scope for Applications based on artificial intelligence are based
using artificial intelligence in the teaching process [25,26]. on information that someone has given them. The amount
From creating review questions, and lesson plans to of data based on which the applications are trained does
creating the course itself. Artificial intelligence gives the not offer the possibility of explaining the origin of the
possibility to adapt teaching for a specific area or cultural conclusion or judgment. Likewise, there is no guarantee
environment. Basic possibilities of using artificial that the training data set has passed validation and that the
intelligence from the point of view of the educator: information that is the basis for the algorithm is of high
• Intelligent search engine – in the case of the amount quality and relevant. Various experiments have shown that
of information that exists on various topics, artificial some statements are not true, or artificial intelligence can
intelligence is a simple solution. adapt a certain statement to a question or result. This
• Various assignments / Creating examples / means that artificial intelligence can generate fake content.
mathematical problems - if the teacher needs to provide an The unreliability of data is one of the main
interesting example, this is an easy way to generate them. disadvantages of artificial intelligence. It is also the result
of transferring the characteristics of people to the digital
environment. Since the data on which artificial
intelligence is based is provided by humans, it may be

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 144

subject to bias, subjectivity, or inaccuracy. A system that students how to search for information sources, verify
is created by human resources repeats mistakes made by facts and draw conclusions. Under this influence,
humans. Another limitation can be outdated data. The education will move to a gradual reorientation towards the
technological process is constantly accelerating and formation of a creative personality. Creating new
complicated to follow the changes. knowledge or solving complex tasks should remain a
The question of authorship is a very important topic. If person's priority. Artificial intelligence should remain as
the author generates an abstract based on the research an auxiliary tool for solving routine tasks. This will most
done, is it the work of the author himself or is it already likely lead to a higher value of author texts.
the work of artificial intelligence? As mentioned above, Replacement of routine work - due to the number of
there are various text editing tools such as Grammarly or different functions offered by artificial intelligence, it can
GPT chat. In this case, the application can edit the text in effectively contribute to increasing the possibility of
such a way that it does not contain grammatical errors or replacing routine work. Organization of the educational
incorrect word order. Currently, there is no ethical or legal process, preparation of teaching materials, and curricula,
framework for the use of artificial intelligence. creation of a virtual environment, or interactive courses
supported by modern technologies.
VI. PREDICTION OF THE INTEGRATION OF GENERATIVE
ARTIFICIAL INTELLIGENCE VII. CONCLUSION
There are currently many studies investigating the Artificial intelligence makes it possible to analyze large
impact of artificial intelligence on education [28, 29, 30]. volumes of data in real-time, to provide new materials
The attitude of individual institutions, educators and based on requirements. Students have a continuous
experts is diverse. Some institutions ban the use of opportunity for education. All these approaches to
artificial intelligence en masse, while others see benefits education can provide an effective and tailored learning
in new modern technologies. However, the process.
implementation of artificial intelligence can provide new Developing artificial intelligence technologies are
insights into the field of education. The development of increasingly entering the field of education. The digital
technology is shifting ideas about traditional education. educational area with the use of artificial intelligence
Based on the current situation, several main directions in technology creates space for more efficient adaptation of
the field of education can be predicted. processes. These processes help, for example, to plan the
New opportunities in the field of education - artificial education process itself create appropriate programs or
intelligence is an effective tool in the field of personalized analyze solutions. The use of artificial intelligence gives
education. It can also be considered as an individual the possibility to improve or automate routines. It allows
assistant. It will be able to be used in the search for you to create new assignments based on feedback from
adequate content, which will be adapted to, for example, students' correct or incorrect practices. The use of artificial
the limited capabilities of the student. In this direction, it intelligence exponentially increases the ability to analyze
is necessary to find the right level of use and integration information. Therefore, in the case of using artificial
into the student's normal activities. Automatic generation intelligence, it requires the implementation of new
of documents for students who have gaps in a specific educational activities. These activities assume a high level
area. Education can adapt to the student, becoming a of expertise of the teaching staff. Process efficiency can be
digital teacher. achieved by setting tasks correctly. These settings require
In recent years, the trend of written review or the training and training of specialists to achieve effective
automated assessment of answers has entered education. solutions in the educational field.
This trend with the expansion of artificial intelligence may On the other hand, rapid technological development is
put us in a position where it will be necessary to focus on associated with many risks or problems. This requires
an individual approach to students. Technical progress has applying new standards to areas of education. In the case
brought various conveniences that have been successfully of the use of artificial intelligence, the question of solving
integrated into the field of education. Computer ethical problems is equally important. Space is opening
technology has made it possible to replace real for the creation of formal requirements such as legal
experiments in technical and natural sciences and thus regulatory acts.
shorten research time. Creating a model of a real system, However, from the point of view of the usability of
e.g. in the Matlab environment, on which the research is artificial intelligence in the educational process, AI brings
conducted, does not reflect negatively on the contribution modern methods and solutions. New technologies open
of the author's idea itself. Therefore, the creation and space for improving the quality of education.
analysis of texts will become an integrated part of Artificial intelligence supports the transformation of
education. There is a need to reevaluate the goals and education and plays a big role in the introduction of new
methodologies of higher education and the way to achieve technologies in the field of education. Based on the
them. Artificial intelligence is already excellent at scientific and public attitude, artificial intelligence has
teaching foreign languages and correcting spelling and great potential in the field of education. It provides space
punctuation. It is likely to contribute to the development for the introduction of new innovative methods that
of research skills and provide support in further education. support technological progress.
Involvement of students in the actual process of
research or verification of information. The idea of ACKNOWLEDGMENT
introducing reverse research is the fact-checking that
artificial intelligence provides [31]. Evaluating the The work was funded by the grant KEGA 012UCM-
reliability of the content offered is an opportunity to teach 4/2021 “Modern technologies and innovations in network
security education”.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 145

 REFERENCES [15] Tlili A., Shehata B., Adarkwah M.A., Bozkurt A., Hickey D.T.,
Huang R., Agyemang B. What if the devil is my guardian angel:
[1] Weng Marc Lim, Asanka Gunasekara, Jessica Leigh Pallant, Jason ChatGPT as a case study of using chatbots in education // Smart Learn.
Ian Pallant, Ekaterina Pechenkina, Generative AI and the future of Environ. 2023. 10.. DOI: 10.1186/s40561-023-00237-x.
education: Ragnarök or reformation? A paradoxical perspective
[16] Opara E.C. Tolorunleke C. et al. ChatGPT for Teaching, Learning
from management educators, The International Journal of
and Research: Prospects and Challenges. Global Academic
Management Education, Volume 21, Issue 2, 2023, 100790, ISSN
Journal of Humanities and Social Sciences. Dubai, 2023; Vol. 5
1472-8117, https://doi.org/10.1016/j.ijme.2023.100790.
[17] Boris Steipe, My students are using ChatGPT to write papers and
[2] Laupichler M.C., Aster A., Schirch J., Raupach T. Artificial
answer exam questions—and I support it. Available at:
intelligence literacy in higher and adult education: A scoping
https://macleans.ca/society/technology/chatgpt-ai-university-
literature review // Computers and Education: Artificial
students-professor-exams/, accessed on 20th September 2023
Intelligence. 2022. Vol. 3. 100101. DOI: 10.1016/j.
caeai.2022.100101. [18] Sentient Syllabus, https://sentientsyllabus.substack.com/
[3] Lund B., Wang T. Chatting about ChatGPT: How may AI and [19] Susnjak T. ChatGPT: The End of Online Exam Integrity? /
GPT impact academia and libraries? Library Hi Tech News. 2023; Available at: https://arxiv. org/pdf/2212.09292. accessed on 20th
[4] Umelá inteligencia v službách zdravotníctva Available at> [20] Firat, M. How chat GPT can transform autodidactic experiences
https://www.trend.sk/technologie/umela-inteligencia-sluzbach- and open education? https://doi.org/10.31219/osf.io/9ge8m
zdravotnictva, accessed on 20th September 2023 [21] Gheisari M., Becerik-Gerber B., Dossick C.S. Emerging learning
[5] K. Hammond, “Learning to fish for a lifetime: Personalised technologies for future of work and education in engineering.
mobile academic phrase libraries – project proposal,” In B. Advanced Engineering Informatics. 2022. 101775.
Hegarty, J. McDonald, & S.-K. Loke (Eds.), Rhetoric and Reality: https://www.sciencedirect.
Critical perspectives on educational technology. Proceedings com/science/article/abs/pii/S1474034622002336. , accessed on
ascilite Dunedin 2014, pp. 495–499. 20th September 2023
[6] Thorp H. ChatGPT is fun, but not an author // Science. 2023. Vol. [22] Brady L., Wang T. Chatting about ChatGPT: How May AI and GPT
379. № 6630. P. 313-314. https://doi.org/10.1126/science.adg7879 Impact Academia and Libraries? // Library Hi Tech News. 2023.
DOI: 10.2139/ssrn.4333415.
[7] Atlas S. ChatGPT for Higher Education and Pro- fessional
Development: A Guide to Conversational AI. 2023. URL: [23] M.-L. Tsai, C. W. Ong, and C.-L. Chen, “Exploring the use of
https://digitalcommons.uri.edu/ cba_facpubs/548 (дата large language models (LLMs) in chemical engineering education:
обращения: 11.05.2023). Building core course problem models with Chat-GPT,” Educ.
Chem. Eng., vol. 44, pp. 71–95, 2023
[8] ABC News. Queensland to join NSW in banning access to
ChatGPT in state schools. Available at: [24] K. Fuchs, “Exploring the opportunities and challenges of NLP
https://www.abc.net.au/news/2023-01-23/queensland-to-join-nsw- models in higher education: is Chat GPT a blessing or a curse?,”
in-banning-access-to/101884288, accessed on 20th September Front. Educ., vol. 8, 2023.
2023 [25] Drapkin A. Microsoft Teams Premium Launched With ChatGPT
[9] Schools Ban ChatGPT amid Fears of Artificial Intelligence- for Meetings. Tech. co. 2023. Available at:
Assisted Cheating. Available at: https://tech.co/news/microsoft-teams-premium-chatgpt.
https://www.voanews.com/a/schools-ban-chatgpt-amid-fears-of- [26] OpenAI. (2022). OpenAI about page. Retrieved from
artificial-intelligence-assisted-cheating/6949800.html, accessed on https://openai.com/about/.
20th September 2023 [27] Pavlik, J. V. (2023). Collaborating With ChatGPT: Considering
[10] The wall street journal. ChatGPT Banned in New York City the Implications of Generative Artificial Intelligence for
Public Schools Over Concerns About Cheating, Learning Journalism and Media Education. Journalism & Mass
Development. Available at: https://www.wsj.com/articles/chatgpt- Communication Educator, 78(1), 84–93.
banned-in-new-york-city-public-schools-over-concerns-about- https://doi.org/10.1177/10776958221149577
cheating-learning-development-11673024059, accessed on 20th [28] Huallpa, Jorge & Flores-Arocutipa, Javier & Panduro, Walker &
September 2023 Chauca Huete, Luis & Limo, Fernando & Herrera, Edward &
[11] Reuters. Top French university bans use of ChatGPT to prevent Callacna, afael & Ariza Flores, Victor & Medina-Romero, Miguel
plagiarism. Available at: Ángel & Quispe, Isaac & Hernández, Fredy. (2023). Exploring the
https://www.reuters.com/technology/top-french-university-bans- ethical considerations of using Chat GPT in university education.
use-chatgpt-prevent-plagiarism-2023-01-27/, accessed on 20th Periodicals of Engineering and Natural Sciences (PEN). 11. 105-
September 2023 115.
[12] Sciences po bans the use of chatGPT without transparent [29] Bai, Long & Liu, Xiangfei & Su, Jia-Can. (2023). ChatGPT: The
referencing Available at: https://newsroom.sciencespo.fr/sciences- cognitive effects on learning and memory. Brain‐X. 1.
po-bans-the-use-of-chatgpt/ accessed on 20th September 2023 10.1002/brx2.30.
[13] Lucas Mearian, Schools look to ban ChatGPT, students use it [30] I. Dergaa, K. Chamari, P. Zmijewski, and H. Ben Saad, “From
anyway. Available at: human writing to artificial intelligence ‰‡‡”ƒ–‡† –‡š–ǣ
https://www.computerworld.com/article/3694195/schools-look-to- ‡šƒining the prospects and potential threats of ChatGPT in
ban-chatgpt-students-use-it-anyway.html, accessed on 20th academic writing,”‹‘ŽǤ’‘”–ǡ vol. 40, no. 2, pp. 615Ȃ622, 2023.
September 2023 [31] Halaweh M. ChatGPT in education: Strategies for responsible
[14] Srinivasan V. AI & learning: A preferred future // Computers and implementation //Cont. Ed. Technology. 2023. Vol. 15. Iss. 2. Art.
Education: Artificial Intelligence. 2022. Vol. 3. 100062. DOI: Well. er421. https://doi.org/10.30935/cedtech/13036
10.1016/j. caeai.2022.100062.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 146

 Chain Collision Avoidance Using
Vehicle-to-Everything (V2X) Communication
1st Marek Galinski 2nd Jozef Juraško 3rd Peter Trúchly 4th Lukáš Šoltés
FIIT STU FIIT STU FIIT STU FIIT STU
Bratislava, Slovakia Bratislava, Slovakia Bratislava, Slovakia Bratislava, Slovakia
marek.galinski@stuba.sk jozef.jurasko@stuba.sk peter.truchly@stuba.sk lukas.soltes@stuba.sk

Abstract—This paper presents a solution for chain collision de- hicles. V2X (Vehicle-to-Everything) communication plays a
tection using V2X (Vehicle-to-Everything) communication, which crucial role in enabling real-time data exchange among vehi-
is a critical aspect of road safety, especially in the context cles and infrastructure for effective chain collision detection.
of connected and autonomous vehicles. It describes the com-
munication with other modules, which are part of the same In this paper, we present a solution design and implementa-
project and show design and implementation of our own solution. tion for chain collision detection using V2X communication.
This described solution consist of the two main processes - Our solution is part of a larger project, which main goal is to
Initialization and Evaluation. The paper presents a detailed increase safety on the road. Our solution consists of two main
description of the communication protocol and the processes processes - Initialization and Evaluation.
involved in the solution, and discusses the potential benefits
of using V2X communication for chain collision detection. The The Initialization process establishes the connection with
solution is part of a larger project with the main goal of increasing the integration module, fetches and prepares necessary data
safety on the road, and it has the potential to contribute to the from the digital twin of the infrastructure.
development of safer road transportation systems, especially in The Evaluation process continuously evaluates the situation
the era of connected and autonomous vehicles. on the road, calculates distances between vehicles, and sends
Index Terms—Vehicle-to-Everything, V2X, Automated Mobil-
ity, Intelligent Safety, 5G Networks notifications to the integration module in case of potential
collisions. Our solution aims to improve road safety by en-
abling timely and efficient chain collision detection using V2X
I. I NTRODUCTION
communication.
Road safety is one of the main priorities of all new vehicles. The rest of this paper is organized as follows: In the Section
Many systems that are mandatory across new vehicles today 2 we describe other work published in recent years related
are designed to prevent accidents. These systems often use to this discussed topic. In the Section 3 we describe the
elements of vehicle automation. Most often it involves the overall system architecture, followed by solution design and
evaluation of data obtained by sensors and in the event of implementation in Section 4. In the Section 5 we conclude our
detecting dangerous situation, these systems intervene through observations and discuss potential future work.
acceleration, braking and steering. In the same way, au-
tonomous vehicles today operate mainly on the principle of II. R ELATED WORK
Advanced Driver Assistance Systems (ADAS), the task of Neither the idea of connected safety, nor the problem of
which is to collect and evaluate data from sensors. The biggest avoiding chain collisions are new by themselves. One of the
disadvantage of such a system is the limitation of perception oldest mentions of this topic in when discussing research
of what is happening on the road only to the immediate done during this century can be traced back to 2005 where
surroundings of the vehicle, i.e. the vehicle only reacts to in the paper [1] authors ”propose a broadcast based packet
what it ”sees” with the help of visual sensors such as camera, forwarding mechanism for intra-platoon cooperative collision
radar or lidar. This solution faces a number of shortcomings, avoidance (CCA) using dedicated short range communication
as there are various obstacles or weather conditions that can (DSRC) links.” This paper according to authors serves as
significantly reduce visibility distance of the vehicle. a motivation for the needs for broadcast forwarding instead
Increasing the vehicle’s perception of the entire traffic, of using unicast routing for transporting inter-vehicle data
which would be functional in all weather conditions, could for safety-critical applications. However, the approach based
lead to an increase in safety. One of the possibilities of creating on DSRC, which would currently involve ITS-G5 European
such a system of collecting and evaluating data on the road Standard in 5.9 GHz band would be limited to relatively small
is the use of V2X communication (vehicle communication radius. IEEE 802.11p technology, that is essential to ITS-G5
with the surrounding world). Proper use of this communication is known for providing range within 1km radius from the
could lead to a significant increase in road safety. vehicle capable of communicating using this technology. [2]
Chain collision detection is a critical aspect of road safety, However, 1 km distance on a highway with vehicles driving
especially in the context of connected and autonomous ve- 130km/h provides timeframe of 27 seconds before the vehicle,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 147

that received the emergency message arrives to location of
where emergency situation occured. To address this, we either
use some flooding protocol (which would work only if there
will be other vehicles on the highway equipped with the same
technology, able to serve as packet forwarding hops), or we
need to make use of cellular data network (i.e. LTE or 5G).
However, using the approach in [2], authors conclude that
”Using an implicit acknowledgement strategy it is shown that
with inter-vehicle spacing of nearly one second, the proposed
mechanism is capable of saving up to 90% of vehicles in a
platoon from chain crashes following emergency events at the
front of the platoon.” If we had to stay using 802.11 based
technology, we might want to consider a strategy for effective
management of 802.11 based communication using Software-
Defined Networking (SDN) approach, i.e. the one proposed in
[8].
Similar problem but with more open possibilities of the
V2X communication (DSRC, WAVE, 5G) is discussed in
[3] by Ji, Wang and Ren from Southwest University and
Tsinghua University in China published in 2021. One year
earlier in 2020, Muzahid et al. in [4] examined the possibility
of utilizing Learning-Based Conceptual framework to avoid Fig. 1. Architecture overview.
multiple vehicle collision on the road. However, their work
”proposes a conceptual framework to investigate the causes
of multiple vehicle collisions in autonomous driving systems Second external module, which communicates with our so-
and in-depth investigation on the aspect of lane change” lution is Integration module. This module provides information
and they encourage safety engineers and automated driving about connected vehicles, which are situated on monitored
systems (ADS) developers to deploy this framework to explore road section. The communication protocol used in this com-
”strategies to improve the autonomous systems”. In 2022, basi- munication is UDP, which was chosen due to its properties
cally these same authors propose deep reinforcement learning- such as low latency. However, to ensure proper communication
based driving strategy for avoidance of chain collisions in [5], between these modules, we needed to define our own message
where they ”consider the problem of chain collision avoidance protocol.
as a Markov Decision Process problem in order to propose
a reinforcement learning-based decision-making strategy and A. Communication Protocol
analyse the safety efficiency of existing methods in driving A detailed description of the communication protocol be-
security”. Authors conclude, that ”results of the study show tween the modules was defined by Bc. Ivan Jatz [10] in his
that the agent vehicle effectively performed the avoidance diploma thesis. The following message definitions are based
of multiple-vehicle collisions.” Another study performed by on his unpublished work.
Muzahid et al. regarding this topic can be found at [6] or [7]. The message that is sent at the beginning of a communi-
Last but not least, when dealing with chain collisions, we cation connection is the “Connect” message. The author of
must be aware of the fact, that on every road there may be the mentioned work defines this message based on his own
vulnerable road users present and we shall not forget about analysis and he refers to this message as “a superstructure
their presence, when discussing connected vehicles. The safety over the UDP protocol that creates a simple connection.” [10]
of vulnerable road users and the issue on how to detect and Another defined message that is used after a successful
track them is a matter of many research papers, i.e. [9]. connection using the “Connect” message is the “RequestArea”
message. This message is sent from our application to the
III. OVERALL S YSTEM A RCHITECTURE integration module, which in response sends the coordinates of
Our main software module communicates with several dif- the boundary of the monitored road section. These are further
ferent modules involved in overall solution. Diagram shown used by our application to obtain data about the road in the
in (Fig. 2) shows all necessary modules for our module manner described in the previous section.
to work correctly. The Digital Twin Map module provides After receiving the “Area” message with the coordinates of
information about the road. These information are represented the road section boundary, the processes necessary for the road
by road points, which are grouped in road fragments. Received preparation are carried out. The complete details of procedure
fragments are in random order, so before using them, this data will be described later in this section. After the preliminary
needs to be preprocessed. Communication with this module is preparation of all the necessary data, the actual evaluation of
ensured through REST API endpoint. the current vehicle data follows. The “Subscribe” message

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 148

 Road surface Road condition Adhesion coefficient
is used to obtain such data. This message is sent from our Dry 0.7 - 0.8
application to the integration module. The message provides Wet 0.5 - 0.6
Asphalt, concrete
the integration module with a request for the information Dusty 0.25 - 0.45
Snow (less than 5cm) 0.2 - 0.4
that the application wants to receive (in our case - vehicles), Stone paving, Dry 0.6 - 0.7
the interval in which the integration module should send interlocking paving Wet 0.4 - 0.5
the data and the identifier of the road section monitored by Dry 0.5 - 0.6
Dusty road Wet 0.2 - 0.4
our application, so that the integration module knows which Dusty 0.15 - 0.30
vehicles to send. Wet 0.4 - 0.5
Sand
Since the UDP protocol does not provide the possibility of Dry 0.2 - 0.3
Dry 0.4 - 0.5
verifying the permanence of the connection, a “KeepAlive” Clay Wet, plastic 0.2 - 0.4
message was defined, which is periodically sent to the inte- Wet, liquid 0.15 - 0.25
gration module. To confirm successful communication, we use Meadow Grassland 0.1 - 0.4
Fresh 0.2 - 0.4
our custom implemented “Acknowledge” messages. Snow
Driven 0.3 - 0.5
After sending the “Subscribe” message, the application Ice Smooth (below 0 °C) 0.05 - 0.1
starts receiving “UpdateVehicles” messages from the integra- TABLE I
G RIP COEFFICIENT BASED ON SPEED AND ROAD SURFACE [11]
tion module at predefined regular intervals. These messages
consist of a list of vehicles connected to the integration
module. For each vehicle, information about its type, speed,
acceleration, direction and current location are received.
The main task of our application is to detect potential
impending accidents based on the information obtained in
this message and prepare warnings for vehicles that are at
risk of these collisions. The application subsequently send
these warnings to the integration module in the “Notify”
message. These messages are sent for each vehicle separately
and contain information about the vehicle that is at risk of a
collision, the time for which the accident is likely to occur, Fig. 2. Safe distance definition.
the type of accident (in our application it is chain collision)
and the severity of the warning, which is determined based on
Braking deceleration or deceleration (a) can be calculated as
the details of the potential threat.
the product of gravitational acceleration (g - value 9.81 m/s2 )
Communication with the integration module can be termi- and adhesion coefficient (μ).
nated with the “Unsubscribe” message. This message is sent
by the application to the integration module. After sending it,
our application stops receiving messages from the integration a = g.μ (2)
module. The coefficient of adhesion (μ), the values of which are
shown in the table I, is defined depending on the type of road
B. Safe braking distance evaluation and its condition.
The simplest principle compares the distance between two
When dealing with the detection and prevention of chain vehicles (sv ) with the braking distance of the vehicle behind
collisions, one of the main necessary factors is the braking (sb1 ). If the braking distance is greater from these two dis-
distance. Braking distance is defined as the distance that the tances, a warning will be sent to the vehicle. This warning
vehicle is able to travel until it comes to a complete stop. is further transmitted to the other vehicles behind it, so that
There are a number of algorithms for calculating the braking they are warned that an accident is imminent. This principle
distance, which differ mainly in the parameters that are taken was used only in the first prototype of the application and
into account and from which the accuracy of the final result was subsequently expanded by several levels of severity. This
also depends. extension created a second evaluation principle, which, in
One of the ways to calculate the braking distance is the addition to the distance between two vehicles and the braking
calculation depending on the vehicle speed and braking decel- distance of the rear vehicle, also takes into account the relative
eration. The basic formula for calculating the distance traveled speed of the vehicles (the difference between the speeds of the
by the vehicle until the moment of stopping (Sb ) defines it as given vehicles). If the first vehicle slows down with respect to
the quotient of the square of the current vehicle speed (v) and the second vehicle, a Danger message is sent to the integration
twice the braking deceleration (a). module. If the vehicle is traveling at the same or higher speed,
a warning message is sent.
v2 Another principle of evaluating the situation compares the
Sb = (1) braking distance (sb1 ) increased by the distance (sr ) that
2.a

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 149

the vehicle travels during the driver’s reaction time with the the aforementioned work defines this message based on his
distance from the vehicle in front (sv ), from which the braking own analysis, and refers to this message as “a superstructure
distance is subtracted of the front vehicle (sb2 ). The basic over the UDP protocol that creates a simple connection.” [10]
condition according to which it is evaluated looks like this: The author further defines and explains the assignment of the
index to the message - “The message will always contain a
sb1 + sr > sv − sb2 (3) zero index. In this way, the integration module will be notified
that a new connection has been created, whose messages will
If this condition is met, the situation is evaluated as risky start from index 1.” [10]
and, as in the previous principle, it is subsequently evaluated
according to the relative speed whether a notification of the
{
danger type (Danger) or only a warning (Warning) is sent. In
"index": "0",
all principles, a warning is also sent to vehicles behind vehicles "type": "connect",
that are at risk of an accident, so that drivers can prepare in "timestamp": "YYYY-MM-DDTHH:MM:SS.SSSZ"
advance for the situation ahead. }
In all cases, the evaluation takes place until all the vehicles
on the section have been passed. Notifications are sent to the
Another defined message that is used after a successful
integration module continuously, whenever a risk is detected,
connection using the “Connect” message is the “RequestArea”
so that the vehicle at risk of an accident is notified as quickly
message. This message is sent from our application to the
as possible.
integration module, which in response sends the coordinates of
IV. S OLUTION D ESIGN AND I MPLEMENTATION the boundary of the monitored road section. These are further
For implementation of our solution, we choose framework used by our application to obtain data about the road in the
.NET of programming language C#. This decision was made manner described in the previous section. The coordinates of
based on results of the study UML Modeling and Performance the border are two - the upper left corner and the lower right
Evaluation of Multithreaded Programs on Dual Core Processor corner of the slice, over which the given modules work. The
[12]. Our solution consist of two main processes - Initialization form of the “RequestArea” message is the same as the form
and Evaluation. First mentioned, Initialization is used only of the previous “Connect” message, with a different index and
once, at the beginning of the application run and it prepares message type. The format of the “Area” message returned by
necessary data. Second process - Evaluation is the main the integration module is as follows:
process which runs in regular intervals and it is triggered by
receiving message “UpdateVehicles”. This process evaluates {
the actual situation on the road and sends notifications to the "index": "integer",
"type": "area",
integration module.
"timestamp": "YYYY-MM-DDTHH:MM:SS.SSSZ",
A. Setup data acquisition "topLeft": {
"lat": "float", "lon": "float"
1) Acquisition of road section data: The module will obtain },
data about the road section from the infrastructure’s digital "bottomRight": {
twin. The connection with this component will take place "lat": "float", "lon": "float"
through the REST API interface defined in the documentation }
for the given project available online.1 Data on the road section }
is obtained from the /roads/ endpoint, where a filter is applied
based on the geographic coordinates of the selected section After receiving the “Area” report with the coordinates of
and the identification number of the road. The application will the section boundary, the processes necessary for the road
obtain this information from the configuration files or from the preparation are carried out. The complete detailed procedure
integration module. will be described in the later parts of this chapter. After pre-
2) Vehicle Data Acquisition: Current vehicle data will be preparation of all the necessary data, the actual evaluation of
obtained from the integration module to which the application the current vehicle data follows. The “Subscribe” message is
will be connected. The integration module was created as part used to obtain such data. This message is sent from our appli-
of the diploma thesis of a B.Sc. Stefan Schindler. The main cation to the integration module. The message will provide the
activity that the integration module will deal with is obtaining integration module with a request for the information that the
from all the cars connected to the system that are located in application wants to receive (in our case, it is vehicles), the
the selected section and then sending this data to the collision interval and the identifier of the road. The interval defines time
detection module. period in which the integration module should send the data.
The message that is sent at the beginning of a communi- The identifier describes the road on which our application is
cation connection is the “Connect” message. The author of working, so that the integration module knows which vehicles
1 https://github.com/milosgaleta/digital to send.
twin map

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 150

 a row, the connection is terminated and the application tries
{ to establish the entire connection anew.
"index": "integer",
After receiving the “Subscribe” message, the integration
"type": "subscribe",
module starts sending current data about all vehicles connected
"timestamp": "YYYY-MM-DDTHH:MM:SS.SSSZ",
"content": "enum('vehicles', 'warnings')", to the system to the application at pre-defined time intervals.
"interval": "float(seconds)", The task of the application is therefore to receive this message
"road": "string" and evaluate the situation on the road in the shortest possible
} time and identify potential accidents. The application waits for
the “UpdateVehicles” message asynchronously in a separate
thread and after its delivery it starts calculations and at the
B. Initialization same time open a new thread where it asynchronously waits
After the initial launch, it is necessary to establish a for the another “UpdateVehicles” message. As part of the
connection with the integration module. The module sends a calculations, the vehicles are at first sorted based on data about
Connect message and waits for a response from the integration the road section. After alignment, the distance of the vehicle
module. Subsequently, a RequestArea message is sent and the from the nearest vehicle in front of it is calculated, as well as
module waits for a response. In case of receiving a response the braking distance of each vehicle. The braking distance is
from the integration module, the application sends a request calculated based on the formula defined in the article Adhesion
to the digital twin of the infrastructure based on the obtained of car tires to the road surface during reconstruction of road
coordinates. accidents [11]. During the calculations, it is necessary to pay
After obtaining the data about the road section from the attention to the time efficiency of the selected algorithms. The
digital twin of the infrastructure, it is necessary to modify this given distances are then compared (braking distance and the
data so it can be used for our purposes. As already mentioned, distance from the nearest vehicle) and if the braking distance
the data about the road section obtained from the digital twin is less than the distance from the other vehicle, a notification
of the infrastructure is in the form of a list of road fragments, about a potential collision that could occur is sent to the
which are composed of sorted road points. However, this list of integration module, which sends this message to the vehicle.
fragments is in random order and needs to be sorted correctly. The notification will also be transmitted to the vehicles behind
All fragments that connect to each other share one common this vehicle, as long as there is not too long distance between
point. In one fragment this common point is the first one in the vehicles. In this way, all lined-up vehicles are passed one
the sequence and in the other fragment it is the last one. by one, and if a potential accident is evaluated, the “Notify”
For correct sorting, we need to recursively go through all message with the right severity is sent to integration module.
fragments and find the previous or next fragment. The severity of potential accident is determined by these terms:
After sorting all the fragments, it is necessary to go through • Danger - the braking distance is greater than the distance
the entire list of sorted fragments and get a list of all points to the closest vehicle ahead and the speed of the closest
from it, with the fact that the duplicate ones that were used vehicle ahead is lower
to sort the fragments are omitted. After obtaining a complete • Warning - the braking distance is greater than the distance
sorted list of points of the road section, it is necessary to cal- to the closest vehicle ahead and the speed of the closest
culate the distances between each consecutive pair of points. vehicle ahead is equal/greater
These calculated distances will be used in the evaluation of • Notification - this message is sent to whole vehicle chain
vehicle data. behind the vehicle with Danger/Warning message
Subsequently, after obtaining and preparing all the necessary In the future work, it is planned to extend the evaluation
data, a Subscribe message is sent to the integration module, process of the solution with including driver reaction time.
which starts process of receiving the data about vehicles from
D. Performance evaluation
the integration module. This step completes the initialization
process of the module and moves to the data evaluation Data processing time increases linearly depending on the
process. number of vehicles. In order to verify whether this hypothesis
obtained from testing is true, we calculated the time complex-
C. Functional Evaluation ity of our solution. When calculating the complexity, we went
The process of data evaluation and chain collision detection through the entire calculation algorithm from receiving data
begins at the moment of sending and verifying the “Subscribe” from the integration module to finishing the last calculations.
message. After receiving the message confirming the deliv- Two basic variables m and n are used in the calculations, where
ery of the “Subscribe” message, one thread starts sending m is the number of road points on the selected section and
“KeepAlive” messages at regular intervals. These messages n is the number of received vehicles in all lanes. Since the
are, as defined in the previous part of this chapter, sent peri- calculations for individual lanes are carried out separately one
odically and expect a confirmation of receipt, which indicates after the other, and therefore the same calculation is always
that the communication link with the integration module is performed for n vehicles in the sum, we will not take this fact
active. If confirmation is not delivered for several messages in into account when determining the complexity. This division

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 151

into individual lanes happens in the CollisionDetector class, chain collision detection in connected and autonomous vehicle
where an object of the VehiclesInfo class is created in the environments.
Detect method. In this object, a list of vehicles of type Vehi-
ACKNOWLEDGMENT
cleInfo will be created, where for each vehicle a pair of closest
points in front and behind it will be found. This search for This article was written thanks to the generous support
the nearest points is represented by a cycle that goes through under the Operational Program Integrated Infrastructure for
the waypoints until it finds the nearest one. So, in the worst the project: ”Support of research activities of Excellence
case, the complexity of this operation is O(m). In the best laboratories STU in Bratislava ”, Project no. 313021BXZ1,
case (the vehicle is right at the first point) the complexity of co-financed by the European Regional Development Fund.”
this operation is O(1). After creating a list of vehicles of type The research was also supported by the APVV-19-0401 and
VehicleInfo, the SortVehiclesBasedOnPointAhead method of the KEGA 025STU-4/2022 projects.
the VehiclesInfo class is executed. In this method, all vehicles R EFERENCES
are sorted based on their nearest points. Sorting takes place in
[1] R. Tatchikou, S. Biswas and F. Dion, ”Cooperative vehicle collision
the form of passing all road points and all vehicles are passed avoidance using inter-vehicle packet forwarding,” GLOBECOM ’05.
for each. The complexity of this operation is therefore always IEEE Global Telecommunications Conference, 2005., St. Louis, MO,
O(m.n). After lining up all the vehicles on the road section, USA, 2005, pp. 5 pp.-2766, doi: 10.1109/GLOCOM.2005.1578262.
[2] Wang, Y., Duan, X., Tian, D., Lu, G. and Yu, H., 2013. Throughput
the distance between two consecutive vehicles is calculated. and Delay Limits of 802.11 p and its Influence on Highway Capacity.
This calculation takes place by traversing all lined-up vehicles, Procedia-Social and Behavioral Sciences, 96, pp.2096-2104.
where for each vehicle all points located between the vehicle [3] Ji, J., Wang, H. and Ren, Y., 2021. Collision Avoidance in Longitudinal
Direction With/Without V2X Communication. In Path Planning and
and the nearest vehicle in front of it are traversed. In the Tracking for Vehicle Collision Avoidance in Lateral and Longitudinal
best case (all vehicles are located between the same points), Motion Directions (pp. 91-115). Cham: Springer International Publishing.
this operation has complexity O(n) - passing all vehicles. In [4] A. J. Md Muzahid, S. F. Kamarulzaman and M. A. Rahim, ”Learning-
Based Conceptual framework for Threat Assessment of Multiple Vehicle
the worst case, it is necessary to calculate the complexity Collision in Autonomous Driving,” 2020 Emerging Technology in Com-
of O(n.m), since for each vehicle it is necessary to pass all puting, Communication and Electronics (ETCCE), Bangladesh, 2020, pp.
the points of the road. After adding up all the complexities 1-6, doi: 10.1109/ETCCE51779.2020.9350869.
[5] A. J. M. Muzahid, S. F. Kamarulzaman, M. A. Rahman and A. H.
(we take into account the worst cases), we get the following Alenezi, ”Deep Reinforcement Learning-Based Driving Strategy for
equation: Avoidance of Chain Collisions and Its Safety Efficiency Analysis in
Autonomous Vehicles,” in IEEE Access, vol. 10, pp. 43303-43319, 2022,
doi: 10.1109/ACCESS.2022.3167812.
O(n, m) = O(n ∗ m) + O(n) + O(n ∗ m) = [6] A. J. M. Muzahid, M. A. Rahim, S. A. Murad, S. F. Kamarulzaman
(4) and M. A. Rahman, ”Optimal Safety Planning and Driving Decision-
= O(n ∗ (3 + 2m)) = O(n + n ∗ m) Making for Multiple Autonomous Vehicles: A Learning Based Ap-
proach,” 2021 Emerging Technology in Computing, Communication
As we can see, the resulting complexity depends on the and Electronics (ETCCE), Dhaka, Bangladesh, 2021, pp. 1-6, doi:
number of vehicles (n) and the number of road points of 10.1109/ETCCE54784.2021.9689820.
the road section (m). Since the number of road points is [7] Muzahid, A.J.M., Kamarulzaman, S.F., Rahman, M.A. et al. Multiple
vehicle cooperation and collision avoidance in automated vehicles: survey
constant (but not negligible) for each selected section, we can and an AI-enabled conceptual framework. Sci Rep 13, 603 (2023).
consider the resulting dependence to be linearly dependent on https://doi.org/10.1038/s41598-022-27026-9
the number of vehicles on the given road section. [8] V. Bernolák, P. Lehoczký, R. Bencel and I. Kotuliak, ”Efficient dynamic
management of IEEE 802.11 networks using SDN,” 2022 IEEE Zooming
V. C ONCLUSION AND F UTURE W ORK Innovation in Consumer Technologies Conference (ZINC), Novi Sad,
Serbia, 2022, pp. 170-175, doi: 10.1109/ZINC55034.2022.9840549.
In this paper, we presented a solution for chain collision [9] Markiewicz, P., Długosz, M. and Skruch, P., 2017, June. Review of
tracking and object detection systems for advanced driver assistance and
detection using V2X communication. The Initialization pro- autonomous driving applications with focus on vulnerable road users
cess establishes the connection with the integration module sensing. In Polish Control Conference (pp. 224-237). Cham: Springer
and prepares necessary data, while the Evaluation process International Publishing.
[10] Jatz, Ivan and Galinski, Marek. “Head Collision Avoidance Using V2X
continuously evaluates the situation on the road and sends Communication”. Unpublished.
notifications to the integration module in case of potential col- [11] Evtukov, Stanislav and Golov, Egor. “Adhesion of car tires to the road
lisions. The solution has been designed to be time-efficient and surface during reconstruction of road accidents”. In: E3S Web Conf. 164
(2020), s. 03022. doi: 10.1051/e3sconf/202016403022. url: https://doi.org/
scalable, allowing for real-time detection of chain collisions 10.1051/e3sconf/202016403022
and timely notifications to the relevant parties. Future work [12] Saxena, Vipin and Shrivastava, Manish. “UML Modeling and Perfor-
includes the extension of the evaluation process to include mance Evaluation of Multithreaded Programs on Dual Core Processor”.
In: International Journal of Hybrid Information Technology 2 (aug. 2009)
driver reaction time for even more accurate and effective
chain collision detection. Our solution can also serve in
the future works focused on chain collision prevention. The
whole project, that includes our solution alongside with others
(e.g. above mentioned Integration module and Digital twin
of road infrastructure) contributes to improving road safety
by leveraging the power of V2X communication for efficient

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 152

 The Good, the Bad and the Ugly ethics of
automated vehicles
Z. Gyurász*, P. Dražová**
* Institute of Information Technology Law and Intellectual Property Law, Faculty of Law, Comenius
University Bratislava, Bratislava, Slovakia
** Institute of Information Technology Law and Intellectual Property Law, Faculty of Law, Comenius
University Bratislava, Bratislava, Slovakia
zoltan.gyurasz@flaw.uniba.sk
petra,drazova@flaw.uniba.sk

Abstract — The advent of automated driving has provided accidents will reduce healthcare3 costs as well as vehicle
fertile ground for discussions not only about legal, insurance costs.4
economic, and philosophical issues but also about the ethics Approximately 1.3 million lives are lost on the
of autonomous vehicles. Consumers expect automated roads every year.5 Studies have shown that up to 94% of
technology to be perfectly secure and are extremely
unforgiving of any errors or flaws such systems may have.
accidents are caused by human error. Therefore, the
we believe that society will not accept the implementation of implementation of autonomous vehicles that can help
automated vehicles that do not meet our ethical expectations prevent many of these accidents, as automated vehicles
of whom and how to protect when harm is inevitable. are not susceptible to the most common human errors that
Therefore, not addressing this issue could lead to people cause accidents, such as distraction, aggressive driving,
avoiding automated vehicles, nullifying all of their promised intoxication, or fatigue.6
benefits this technology offers. Despite these enormous advantages, the
implementation of automated vehicles is not expected to
I. INTRODUCTION
be easy.
In recent years, the advent of automated driving Recent surveys show that as many as three out of
has provided fertile ground for discussions not only about four consumers are skeptical about driving in an
legal, economic, and philosophical issues but also about automated vehicle.7 Consumers expect automated
the ethics of autonomous vehicles.1 Automated vehicles technology to be perfectly secure and are extremely
have the potential to positively transform our lives as well unforgiving of any errors or flaws such systems may
as our society. Reducing environmental pollution, have. A growing body of algorithm aversion is
reducing emissions, faster, more reliable, and safer
transport as well as a significant reduction in traffic 3
Rojas-Rueda, D., Nieuwenhuijsen, M. J., Khreis, H., & Frumkin, H.
accidents are the advantages that automated vehicles offer Autonomous vehicles and public health. Annual Review of Public
us.2 At the same time, the reduced number of traffic Health, 41(1), s329–345. 2020. Available
at: <https://doi.org/10.1146/annurev-publhealth-040119-094035>. cit.
2023-08-20.
4
Light, D. A scenario: The end of auto insurance. Celent | Experts in
financial services., 2012. Available
at: <https://www.celent.com/insights/121822340.>. cit. 2023-08-20.
5
Wolrd health organization: Road traffic injuries. 2022. Available at:
1
Etzioni, A., Etzioni, O. Incorporating ethics into artificial <https://www.who.int/news-room/fact-sheets/detail/road-traffic-
intelligence. The Journal of Ethics, 21(4), s403–418. 2017. Available at: injuries.>. cit. 2023-08-20.
6
<https://philpapers.org/archive/ETZIEI.pdf>. cit. 2023-08-20. or Miller, J. Self-Driving Car Technology’s Benefits, Potential Risks,
Hengstler, M., Enkel, E., Duelli, S. Applied artificial intelligence and and Solutions. 2014 Available at:
trust – The case of autonomous vehicles and medical assistance <https://web.archive.org/web/20150508091217/theenergycollective.co
devices. Technological Forecasting and Social Change, s105–120. m/jemillerep/464721/self-driving-car-technology-s-benefits-potential-
2016. Available risks-and-solutions. >. cit. 2023-08-20. or Deng, B. Machine ethics:
at:<https://www.sciencedirect.com/science/article/abs/pii/S0040162515 The robot’s dilemma. Nature, 523(7558), s24–26. 2015. Available at:
004187>. cit. 2023-08-25. or Lin, P., Abney, K., Jenkins, R. Robot ethics <https://doi.org/10.1038/523024a>. cit. 2023-08-20. or Woodyard, C.
2.0: From autonomous cars to artificial intelligence. Oxford: Oxford McKinsey study: Self-driving cars yield big benefits. USA TODAY.
University Press. 2017. Available at: 2015. Available at:
<https://books.google.sk/books?hl=en&lr=&id=y2EwDwAAQBAJ&oi <https://www.usatoday.com/story/money/cars/2015/03/04/mckinsey-
=fnd&pg=PP1&ots=8aH9pQcVfK&sig=6Pr1F3ULv5Tbd05ocikTspL_ self-driving-benefits/24382405/>. cit. 2023-08-20. or Saunders, J. How
Aqg&redir_esc=y#v=onepage&q&f=false>. Cit 2023-08-20 can autonomous cars help reduce accidents? London Business News
2
Waldrop, M. M. Autonomous vehicles: No drivers Londonlovesbusiness.com. 2020. Available at:
required. Nature, 518(7537), s20–23. 2015. Available <https://londonlovesbusiness.com/how-can-autonomous-cars-help-
at: <https://doi.org/10.1038/518020a>. cit. 2023-08-20. or Pyper, J. reduce-accidents/>. cit. 2023-08-20.
7
ClimateWire, J. Self-driving cars could cut greenhouse gas pollution. Edmonds, E. Three in four Americans remain afraid of fully self-
Scientific American. 2014. Available driving vehicles. AAA NewsRoom. 2019. Available
at: <https://www.scientificamerican.com/article/self-driving-cars-could- at:<https://newsroom.aaa.com/2019/03/americans-fear-self-driving-
cut-greenhouse-gas-pollution/>. cit2023-08-202023-08-20. cars-survey/.> cit. 2023-08-20.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 153

emerging.8 Meanwhile, every fatal accident involving an such as the cost-benefit analysis of transport projects or
automated vehicle instills fear and doubt in consumers, the fairness of pricing.13
and such a negative public reaction will be amplified by The ethical debates associated with automated
the accompanying moral outrage, which will most likely vehicles, which are documented in the scientific
have a knock-on effect on consumer confidence and literature, have mainly focused on extreme traffic
significantly reduce people's trust in this new technology. situations that will present moral dilemmas.14 One such
However, if automated vehicles can save ethical dilemma that has received the most attention
millions of lives every year, then their implementation among academic researchers is the "trolley problem." It is
will have to be considered a categorical imperative! a moral question in which a person has to decide to save
one side or the other in a situation where there is danger
II. ETHICAL FRAMEWORK OF AUTOMATED VEHICLES that is inevitable.15
Already John Stuart Mill noted that: “[Even in There are many variations and extensions of this
natural philosophy, there is always some other thought experiment, but the core of it can be defined as a
explanation possible of the same facts; some geocentric moral choice between actions in traffic that will result in
theory instead of heliocentric, some phlogiston instead of various combinations of lives saved and lives sacrificed.
oxygen; and it has to be shown why that other theory Most of this research has been modeled as a thought
cannot be the true one: and until this is shown, and ] until experiment based on some form of this legendary moral
we know how it is shown, we do not understand the psychology dilemma.16 17
grounds of our opinion. But when we turn to subjects These dilemmas were formulated on the
infinitely more complicated, to morals, religion, politics, following questions:
social relations, and the business of life, three-fourths of i. How should an automated vehicle be
the arguments for every disputed opinion consist in programmed to make good decisions in such situations?
dispelling the appearances which favor some opinion ii. What ethical and moral rules and principles
different from it.“9 We believe that this thought aptly should an automated vehicle follow?
illustrates the controversy and obscurity concerning the iii. Should the decisions of automated vehicles
very essence issues of the debate on the ethical dilemmas reflect the specific characteristics of potential harm
of autonomous vehicles. On this issue, Baldwin holds the subjects?
opinion that there are different views that just cannot be Of course, it can also be assumed that there is no
reduced to some Platonic essence or a unified concept administrative authority that should decide on the ethics
that would help to understand the essence of the given of autonomous vehicles. And that the decision must
dilemma.10 Orbach then adds on this topic that if we look remain in the hands of the driver, just as it has been in the
deeper into the issue, psychological studies confirm the context of traditional means of transport. Simply because
unpleasant truth that people tend to reject information and most decisions made by an automated vehicle during its
arguments that contradict their own beliefs.11
Unfortunately, too often today we can encounter the fact
that ideologies dictate the perception of what regulation 13
Van Wee, B. Transport and ethics: Ethics and the evaluation of
means and whether regulation is necessary at all. Such a transport policies and projects. Cheltenham: Edward Elgar Publishing
basis consequently leads us to inconsistent preferences 2011. Available at:
<https://books.google.sk/books?hl=en&lr=&id=rUiNNCGb_60C&oi=f
for regulation and consequently obscures and distorts the nd&pg=PR1&ots=CXtdE3tBKV&sig=O3pnMkA69-
true meaning and purpose of regulation.12 6zWty7N0iQTQiszF4&redir_esc=y#v=onepage&q&f=false> cit. 2023-
The amount of attention paid to the ethics of 08-20.
14
automated technologies in recent years is quite new to the Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A.,Rahw
an, I. The moral machine experiment. Nature, s59–64. 2018. Available
field of transportation. Traditionally, ethical debates in at:< https://www.nature.com/articles/s41586-018-0637-6.>. cit. 2023-
this area have revolved around less sensational issues, 08-20. or Bonnefon, J. F., Shariff, A., Rahwan, I. The social dilemma of
autonomous vehicles. Science, 352(6293), 1573–1576. 2016. Available
at:< https://www.science.org/doi/abs/10.1126/science.aaf2654>. cit.
2023-08-20.
8 15
Dietvorst, B. J., Simmons, J. P., & Massey, C. Algorithm aversion: However, this dilemma is not entirely new. We had this same debate
People erroneously avoid algorithms after seeing them err. Journal of in the context of counter-terrorism measures in 2001. September 11,
Experimental Psychology: General,144(1), s114–126. 2015. Available 2001 showed us what a huge security risk a hijacked means of transport
at:<https://repository.upenn.edu/cgi/viewcontent.cgi?article=1392&cont can be. I dare to note that even in this context we still believe that
ext=fnce_papers>. cit. 2023-08-20. or Burton, J. W., Stein, M.-K., & sacrificing the smaller to save the majority is a classic military strategy.
Jensen, T. B. A systematic review of algorithm aversion in augmented Therefore, the question arises to what extent is it possible in a civil
decision making. Journal of Behavioral Decision Making, 33(2), s220– advanced democratic state to accept such practices if we want to call
239. 2020. Available at: ourselves a state governed by the rule of law?
16
<https://onlinelibrary.wiley.com/doi/abs/10.1002/bdm.2155>. cit. 2023- Thomson, J. J. The trolley problem. Yale Law Journal, 94, 1985.
08-20. Available at: <http://jonathonklyng.com/wp-
9
Mill, J. S.: O slobode, Iris, 1995 s 67. content/uploads/2016/08/Thomson-The-trolley-problem.pdf.> cit. 2023-
10
Baldwin, R. Scott, C. Hood, C.: A Reader on Regulation Oxford: 08-20.
17
Oxford University Press, 1998. Of course, there are other ethical issues regarding personal data
11
Mercier, H. Sperber, D.: Why Do Humans Reason? Arguments for an protection, cyber security or the potential for mass surveillance.
Argumentative Theory, 2011. or Nickerson, R. S.: Confirmation Bias: Automated vehicle sensors collect vast amounts of data on the location,
A Ubiquitous Phenomenon in Many Guises, 1998. traffic and behavior of various actors. While this data could help with
12
Ron, P.: Liberty Defined, grand central publ, Little, Brown Book navigation and monitoring of illegal activity, there is potential for
Group, 2012 s352. people tracking or privacy violations.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 154

normal operation, such as lane keeping or general traffic through 13 scenarios, through 2.3 million respondents
compliance, will pose very few potential risks to users. from over 200 countries, how automated vehicles should
However, there may be decisions that, although be programmed in the context of these ethical dilemmas.
extremely rare, will have a huge risk of negative impact. Results from the MME revealed several strong public
Despite widespread media attention, automated vehicle preferences in light of the ethics of automated vehicles. A
manufacturers have largely avoided such dilemmas, significant finding of the survey was also that the moral
devoting limited resources to designing appropriate principles that guided drivers' decisions differed from
solutions.18 country to country. Another significant finding was that
That is why the discussion about the ethics of women and men perceived ethical and moral situations
automated vehicles is mainly focused on two questions: differently. However, without consensus on a universal
i. Is the ethics of the automated at all important moral code, it would be nearly impossible to develop
to the implementation of this new technology? automated decision-making that would universally meet
ii. Is steering public opinion the best way to the ethical frameworks expected by populations
offer a solution to the ethical dilemmas of automated worldwide.
vehicles?
IV. THE INSIGNIFICANCE OF THE ETHICS OF
AUTOMATED VEHICLES
III. THE NEED FOR AUTOMATED VEHICLE ETHICS Approaches that espouse the ethical value of
Researchers have discussed in detail the automated vehicles have their opponents. Some
relevance of the moral dilemmas of automated vehicles, prominent academics argue that the ethics of automated
the benefits of using different ethical frameworks such as vehicles is essentially an engineering and political
deontology or utilitarianism as the basis for algorithms distraction rather than a pressing issue for automated
that make decisions in automated vehicles.19 They dealt vehicles. They believe that the ethics of automated
with moral preferences and societal expectations vehicles is merely a distraction and has little practical
regarding ethical rules that should be encoded into value for the implementation of automated vehicles in our
automated vehicles.20 Several large-scale behavioral transportation.23
studies modeled after the classic trolley problem in moral They believe that ethical dilemmas are of
psychology have been published in prestigious academic negligible value compared to questions such as:
media.21 Bonnefon's pioneering work offered us findings i. How to solve the technical problems of
that while humans expect automated vehicles to be automated vehicles?
programmed with a utilitarian ethic, which would mean ii. How to resolve liability for damage caused by
that in the event of irreversible harm, the automated automated vehicles?
vehicle would sacrifice the driver to save five pedestrians. iii. How to solve the protection of personal data
However, these same people hesitate to use such a in automated vehicles?
vehicle for their own use. iv. How to solve the cyber security of automated
The most complex research in this direction is vehicles?
the moral machine experiment (MME).22 MME used Compared to these questions, the "trolley
many variations of the "tram dilemma" and sought problem" scenarios are too abstract, incredibly rare and
unlikely to occur, and even if they do, automated systems
18
To prioritize the life of a child over an adult, or to prioritize the life of may not even "realize" they are in such a situation. In
a woman over a man? addition, they argue that other technological and legal
19
Bonnefon, J. F., Shariff, A., Rahwan, I. The social dilemma of
issues that we have also mentioned are a much more
autonomous vehicles. Science, 352(6293), 1573–1576. 2016. Available
at:< https://www.science.org/doi/abs/10.1126/science.aaf2654>. cit. pressing problem.
2023-08-20. or The manufacturers of automated vehicles also
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A.,Rahwa support these views and do not consider the ethics of
n, I. The moral machine experiment. Nature, 563(7729), s59–64. 2018. automated vehicles to be an issue that needs to be
Available at:< https://www.nature.com/articles/s41586-018-0637-6.>.
cit. 2023-08-20. or Etzioni, A., Etzioni, O. Incorporating ethics into urgently addressed. While the scientific literature has
artificial intelligence. The Journal of Ethics, 21(4), s403–418. 2017. largely been preoccupied with deep reflections on
Available at: <https://philpapers.org/archive/ETZIEI.pdf>. cit. 2023-08-
20.
20
23
Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A.,Rahwa De Freitas, J., Censi, A., Smith, B. W., Di Lillo, L., Anthony, S. E.,
n, I. The moral machine experiment. Nature, 563(7729), s59–64. 2018. Frazzoli, E. From driverless dilemmas to more practical commonsense
Available at:< https://www.nature.com/articles/s41586-018-0637-6.>. tests for automated vehicles. 2021 Available
cit. 2023-08-20. or Bonnefon, J. F., Shariff, A., Rahwan, I. The social at:<https://www.pnas.org/doi/full/10.1073/pnas.2010202118> cit. 2023-
dilemma of autonomous vehicles. Science, 352(6293), 1573–1576. 08-20. or Dewitt, B., Fischhoff, B., & Sahlin, N. ‘Moral machine’
21
Thomson, J. J. The trolley problem. Yale Law Journal, 94, 1985. experiment is no basis for policymaking. Nature, 567, 2019 Available
Available at: <http://jonathonklyng.com/wp- at:<https://go.gale.com/ps/i.do?id=GALE%7CA577251215&sid=googl
content/uploads/2016/08/Thomson-The-trolley-problem.pdf.> cit. 2023- eScholar&v=2.1&it=r&linkaccess=abs&issn=00280836&p=HRCA&sw
08-20. =w&userGroupName=anon%7E297fcce4> cit. 2023-08-20.or Nyholm,
22
Bonnefon, J. F., Shariff, A., Rahwan, I. The social dilemma of S., Smids, J. The ethics of accident-algorithms for self-driving cars: An
autonomous vehicles. Science, 352(6293), 1573–1576. 2016. Available applied trolley problem? Ethical Theory and Moral Practice, 19(5).
at:< https://www.science.org/doi/abs/10.1126/science.aaf2654>. cit. 2016 Available at:<https://link.springer.com/article/10.1007/s10677-
2023-08-20. 016-9745-2> cit. 2023-08-20.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 155

abstract moral dilemmas, industry reports use of a much preferences and societal expectations regarding ethical
more pragmatic, optimistic narrative.24 rules that should be encoded into automated vehicles.
Similarly, Nyholm & Smids25 argue that while Approaches that place a high value on the
the ethics of automated vehicles is designed based on ethics of automated vehicles have their detractors. Some
results from theoretical work, real-life scenarios for prominent academics argue that the ethics of automated
automated vehicles will not be nearly the same. Instead of vehicles is essentially an engineering and political
focusing narrowly on ethical dilemmas, it is distraction rather than a more pressing issue for
recommended that the development of automated systems automated vehicles. They believe that ethical dilemmas
focus on the general principle of minimizing total harm. 26 are of negligible value compared to other, more practical
Also, the European Commission's recent report issues.
on the Ethics of Connected and Automated Vehicles However, the development of ethical rules
states that "moral dilemmas in accident prevention are applicable to AI-based machines is still in its early stages.
not the only, or even the most pressing, ethical and Such a discussion requires interdisciplinary research,
societal issue."27 which should take place between experts from several
scientific fields, especially ethics, technical engineering
CONCLUSION in the field of artificial intelligence, and law.
The amount of attention paid to the ethics of However, we believe that society will not accept
automated technologies in recent years is quite new to the the implementation of automated vehicles that do not
field of transportation. Traditionally, ethical debates in meet our ethical expectations of whom and how to protect
this area have revolved around less sensational topics, when harm is inevitable. Therefore, not adequately
such as cost-benefit analysis of transport projects or addressing this issue could lead to people avoiding
fairness in pricing. automated vehicles, nullifying all their promised benefits.
However, it is not surprising that our attention
has focused precisely on the topic of automated vehicles, REFERENCES
when we know that new technologies carry with them [1] Anderson, M., Anderson, S. Machine Ethics, Cambridge
University Press. Ai Magazine, Volume 28 number 4 s. 15-26
risks that can potentially conflict with existing social, 2016, Available at:<
cultural, or ethical values. https://www.researchgate.net/publication/220605213_Machine_Et
Consumers expect automated technology to be hics_Creating_an_Ethical_Intelligent_Agent>.
perfectly secure and are extremely unforgiving of any [2] Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A.,R
errors or flaws such systems may have. Research shows ahwan, I. The moral machine
experiment. Nature, 563(7729), s59–64. 2018. Available at:<
that people expect extreme safety from automated https://www.nature.com/articles/s41586-018-0637-6.>.
vehicles and will only consider purchasing them when [3] Baldwin, R., Scott, C., Hood, C.: A Reader on Regulation Oxford:
automated vehicles prove to be significantly safer than Oxford University Press, 1998.
conventional vehicles. And we agree that stability and [4] Banzhaf, J.: When a Computer Needs a Lawyer, Communications
predictability are therefore not only a desirable element, of the ACM, 1966. Available at:
<https://dl.acm.org/doi/10.1145/363567.363573,>.
but they are also a necessity. But if automated vehicles
[5] Bernstein, G.: The Socio-Legal Acceptance of New Technologies:
can save millions of lives every year, then their A Close Look at Artificial Insemination, 2002, Available at:
implementation will have to be considered a categorical <https://pubmed.ncbi.nlm.nih.gov/15212080/>.
imperative? [6] Black, J.: Decentring Regulation: Understanding The Role Of
The ethical debates associated with automated Regulation And Self-Regulation In A 'Post- Regulatory' World,
2001.
vehicles, which are documented in the scientific
[7] Bonnefon, J. F., Shariff, A., Rahwan, I. The social dilemma of
literature, have mainly focused on extreme traffic autonomous vehicles. Science, 352(6293), 1573–1576. 2016.
situations that will present moral dilemmas. The Available at:<
researchers discussed in detail the relevance of the moral https://www.science.org/doi/abs/10.1126/science.aaf2654>. cit.
2023-08-20.
dilemmas of automated vehicles, the advantages of using
[8] Bonnefon, J.-F., Cerny, D., Danaher, J., Devillier, N., Johansson,
different ethical frameworks. They dealt with moral V., Kovacikova, T., Martens, M., Mladenovic, M. N., Palade, P.,
Reed, N., Santoni de Sio, F., Tsinorema, S., Wachter, S.,
24
Olson, E. Trolley Folly. Medium 2018. Available Zawieska, K. Ethics of connected and automated vehicles:
at:< https://medium.com/may-mobility/trolley-folly-fcbd181b7152.>. Recommendations on road safety, privacy, fairness, explainability
cit. 2023-08-20. and responsibility. EU Commission. 2020 Available at:<
25
Nyholm, S., Smids, J. The ethics of accident-algorithms for self- https://aaltodoc.aalto.fi/handle/123456789/61824>.
driving cars: An applied trolley problem? Ethical Theory and Moral [9] Bostrom, N.: Ethical Issues in Advanced Artificial Intelligence,
Practice, 19(5). 2016 Available Cognitive, Emotive and Ethical Aspects of Decision Making in
at:<https://link.springer.com/article/10.1007/s10677-016-9745-2> cit. Humans and in Artificial Intelligence, Vol. 2, ed. I. Smit et al., Int.
2023-08-20. Institute of Advanced Studies in Systems Research and
26
Nyholm, S., Smids, J. The ethics of accident-algorithms for self- Cybernetics, 2003. s12-17, Available
driving cars: An applied trolley problem? Ethical Theory and Moral at:<https://nickbostrom.com/ethics/ai.html>
Practice, 19(5). 2016 Available [10] Brown, J.: Electronic Brains and the Legal Mind: Computing the
at:<https://link.springer.com/article/10.1007/s10677-016-9745-2> cit. Data Computer’s Collision with Law, yale, L.J. 1961
2023-08-20. [11] Burton, J. W., Stein, M.-K., & Jensen, T. B. A systematic review of
27
Bonnefon, J. F., Shariff, A., Rahwan, I. The social dilemma of algorithm aversion in augmented decision making. Journal of
autonomous vehicles. Science, 352(6293), 1573–1576. 2016. Available Behavioral Decision Making, 33(2), s220–239. 2020. Available at:
at:< https://www.science.org/doi/abs/10.1126/science.aaf2654>. cit. <https://onlinelibrary.wiley.com/doi/abs/10.1002/bdm.2155>.
2023-08-20.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 156

[12] De Freitas, J., Censi, A., Smith, B. W., Di Lillo, L., Anthony, S. E., [29] Isaac, R.: A practical treatise upon the law of railways. Boston,
Frazzoli, E. From driverless dilemmas to more practical Little, Brown and company; 2d ed. edition, 1858.
commonsense tests for automated vehicles. 2021 Available [30] Kundu, S.: Ethics in the Age of Artificial Intelligence, 2019.
at:<https://www.pnas.org/doi/full/10.1073/pnas.2010202118> Available at:
[13] Deng, B. Machine ethics: The robot’s https://blogs.scientificamerican.com/observations/ethics-in-the-
dilemma. Nature, 523(7558), s24–26. 2015. Available at: age-of-artificial-intelligence/
<https://doi.org/10.1038/523024a>.. [31] Lassen, J., Jamison, A.: Genetic Technologies Meet the Public:
[14] Dewitt, B., Fischhoff, B., & Sahlin, N. ‘Moral machine’ The Discourses of Concern, SCI., TECH. & HUMAN VALUES,
experiment is no basis for policymaking. Nature, 567, 2019 2006 Available at:
Available <https://www.researchgate.net/publication/6952632_Genetic_Tec
at:<https://go.gale.com/ps/i.do?id=GALE%7CA577251215&sid= hnologies_Meet_the_Public_The_Discourses_of_Concern>.
googleScholar&v=2.1&it=r&linkaccess=abs&issn=00280836&p= [32] Light, D. A scenario: The end of auto insurance. Celent | Experts
HRCA&sw=w&userGroupName=anon%7E297fcce4> in financial services., 2012. Available
[15] Dietvorst, B. J., Simmons, J. P., & Massey, C. Algorithm aversion: at: <https://www.celent.com/insights/121822340.>.
People erroneously avoid algorithms after seeing them err. Journal [33] Lin, P., Abney, K., Jenkins, R. Robot ethics 2.0: From autonomous
of Experimental Psychology: General,144(1), s114–126. 2015. cars to artificial intelligence. Oxford: Oxford University Press.
Available 2017. Available at:<
at:<https://repository.upenn.edu/cgi/viewcontent.cgi?article=1392 https://books.google.sk/books?hl=en&lr=&id=y2EwDwAAQBAJ
&context=fnce_papers>. &oi=fnd&pg=PP1&ots=8aH9pQcVfK&sig=6Pr1F3ULv5Tbd05o
[16] Edmonds, E. Three in four Americans remain afraid of fully self- cikTspL_Aqg&redir_esc=y#v=onepage&q&f=false>.
driving vehicles. AAA NewsRoom. 2019. Available [34] Lowry, P. H.: Does Computer Stored Data Constitute a Writing for
at:<https://newsroom.aaa.com/2019/03/americans-fear-self- the Purposes of the Statute of Frauds and the Statute of Wills?,
driving-cars-survey/.> RUTGERS COMP. & TECH. L.J. 1982, Available at:
[17] Etzioni, A., Etzioni, O. Incorporating ethics into artificial http://houstonputnamlowry.com/Articles/Computer_Stored_Data_
intelligence. The Journal of Ethics, 21(4), s403–418. 2017. Article.pdf
Available at: <https://philpapers.org/archive/ETZIEI.pdf>. [35] Ludow, K., Bowman, M.D., Gafot, J., Bennett, G. M.: Regulating
[18] Fábry, B., Kasinec, R., Turčan, M.: Teória práva. 1. vydanie. Emerging and Future Technologies in the Present, Springer, 2015.
Bratislava: Wolters Kluwer, 360 strán. ISBN: 9788081685958. [36] Mandel, G.: Technology Wars: The Failure of Democratic
[19] Fernández, A.G.: how to regulate the future of technology Discourse, Michigan Telecommunications and Technology Law
challenges and principles, Tallinn University, 2020. Available Review, 2005. Available at:
at:<http://publications.tlu.ee/index.php/eastwest/article/view/900>. <https://repository.law.umich.edu/cgi/viewcontent.cgi?article=111
cit. 2023-08-20. 5&context=mttlr>.
[20] Gahér, F., Mrva, M., Števček, M,. Turčan, M.: Open Texture of [37] Mercier, H., Sperber, D.: Why Do Humans Reason? Arguments
Concepts and Rules – Fuel for the Fire of Subjectivism in Applied for an Argumentative Theory, 2011.
Semantics? FILOZOFIA, 75, 2020, No 4, s309–323 Available at: [38] Mesthene, G. E.: The Role of Technology in Society, in
<https://www.sav.sk/journals/uploads/04211152ilozofia.2020.75.4 Technology and the future, 1997.
.4.pdf>. [39] Mill, J. S.: O slobode, Iris, 1995.
[21] Geist, M: Is There a There There? Toward Greater Certainty for [40] Miller, J. Self-Driving Car Technology’s Benefits, Potential
Internet Jurisdiction, 2001. Available at: Risks, and Solutions. 2014 Available at:
<https://www.jstor.org/stable/24116973?seq=1> <https://web.archive.org/web/20150508091217/theenergycollectiv
[22] Gill T. Ethical dilemmas are really important to potential adopters e.com/jemillerep/464721/self-driving-car-technology-s-benefits-
of autonomous vehicles. Ethics and Information Technology potential-risks-and-solutions>.
volume. 2021. Available
[41] Moses, L: Recurring Dilemmas: The Law’s Race to Keep Up with
at:<https://link.springer.com/article/10.1007/s10676-021-09605-
Technological Change.. University of Illinois Journal of Law,
y.>
Technology & Policy, 2007 Available at:
[23] Gyurász, Z., Mihálik, S., Dražová, P.: Právne aspekty zavádzania <https://www.researchgate.net/publication/228183058_Recurring_
autonómnych vozidiel ComenIUS - Roč. 6, č. 2. 2021. Available Dilemmas_The_Law%27s_Race_to_Keep_Up_With_Technologic
at: al_Change>.
<https://www.researchgate.net/publication/358046983_Pravne_as [42] Moses, L.: Legal Responses to Technological Change: The
pekty_zavadzania_autonomnych_vozidiel.> Example of In Vitro Fertilization, Minnesota Journal of Law,
[24] Gyurász, Z.: Duševné vlastníctvo vo svetle umelej inteligencie In Science & Technology, 2005. Available at:
COMENIUS časopis, 2/2020, Právnická fakulta, Univerzita https://core.ac.uk/download/pdf/217199479.pdf.
Komenského v Bratislave, s. 6-14. [43] Nickerson, R. S.: Confirmation Bias: A Ubiquitous Phenomenon in
[25] Gyurász, Z.: Ethics in the age of AI, Bratislava legal forum 2020 Many Guises, 1998.
[elektronický dokument] : disruptive technologies: regulatory and
[44] Nyholm, S., Smids, J. The ethics of accident-algorithms for self-
ethical challenges. - : 1. vyd. ISBN 978-80-7160-567-6. -
driving cars: An applied trolley problem? Ethical Theory and
Bratislava : Právnická fakulta UK, 2020. - S. 63-68
Moral Practice, 19(5). 2016 Available
[26] Gyurász, Z.: From principles to practice: regulating artificial at:<https://link.springer.com/article/10.1007/s10677-016-9745-2>.
intelligence, Míľniky práva v stredoeurópskom priestore 2020 [45] Olson, E. Trolley Folly. Medium 2018. Available
[elektronický dokument] : zborník z online medzinárodnej at:< https://medium.com/may-mobility/trolley-folly-
vedeckej konferencie doktorandov a mladých vedeckých fcbd181b7152.>..
pracovníkov. - : 1. vyd. ISBN 978-80-7160-576-8. - Bratislava:
Právnická fakulta UK, 2020. - S. 683-688. 2020 [46] Pyper, J. ClimateWire, J. Self-driving cars could cut greenhouse
gas pollution. Scientific American. 2014. Available
[27] Hengstler, M., Enkel, E., Duelli, S. Applied artificial intelligence at: <https://www.scientificamerican.com/article/self-driving-cars-
and trust – The case of autonomous vehicles and medical could-cut-greenhouse-gas-pollution/>..
assistance devices. Technological Forecasting and Social
Change, s105–120. 2016. Available [47] Rojas-Rueda, D., Nieuwenhuijsen, M. J., Khreis, H., & Frumkin,
at:<https://www.sciencedirect.com/science/article/abs/pii/S004016 H. Autonomous vehicles and public health. Annual Review of
2515004187>. Public Health, 41(1), s329–345. 2020. Available
at: <https://doi.org/10.1146/annurev-publhealth-040119-094035>.
[28] Holmes, O: The Path of the Law, 1897. Available at:
<https://warwick.ac.uk/fac/soc/sociology/staff/sfuller/social_theor [48] Ron, P.: Liberty Defined, grand central publ, Little, Brown Book
y_law_2015- Group, 2012 s352.
16/oliver_wendell_holmes_oliver_holmes_the_path_of_law.pdf>. [49] Saunders, J. How can autonomous cars help reduce
accidents? London Business News Londonlovesbusiness.com.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 157

 2020. Available at: <https://londonlovesbusiness.com/how-can- [57] Waldrop, M. M. Autonomous vehicles: No drivers
autonomous-cars-help-reduce-accidents/>. required. Nature, 518(7537), s20–23. 2015. Available
[50] Shariff, A., Bonnefon, J. F., Rahwan, I. Psychological roadblocks at: <https://doi.org/10.1038/518020a>.
to the adoption of self-driving vehicles. Nature Human [58] Wolrd health organization: Road traffic injuries. 2022. Available
Behavior, 1, s694–696. 2017. Available at: <https://www.who.int/news-room/fact-sheets/detail/road-
at:<https://dspace.mit.edu/bitstream/handle/1721.1/125064/Shariff traffic-injuries.>.
EtAl_NHB_revision.pdf?sequence=1&isAllowed=y.>. [59] Woodyard, C. McKinsey study: Self-driving cars yield big
[51] Schön, D.: Technology and change;: The new Heraclitus, benefits. USA TODAY. 2015. Available at:
Delacorte Press, 1967. <https://www.usatoday.com/story/money/cars/2015/03/04/mckins
[52] Števček, M. a kol.: Občiansky zákonník, Komentár. 2. vydanie. ey-self-driving-benefits/24382405/>.
Praha: C.H.Beck, 2015. [60] G. Eason, B. Noble, and I. N. Sneddon, “On certain integrals of
[53] Thomson, J. J. The trolley problem. Yale Law Journal, 94, 1985. Lipschitz-Hankel type involving products of Bessel functions,”
Available at: <http://jonathonklyng.com/wp- Phil. Trans. Roy. Soc. London, vol. A247, pp. 529–551, April
content/uploads/2016/08/Thomson-The-trolley-problem.pdf.>. 1955.
[54] Tranter, K.: The History of the Haste-Wagons’: The Motor Car [61] J. Clerk Maxwell, A Treatise on Electricity and Magnetism, 3rd
Act 1909, Emergent Technology and the Call for Law, 2005. ed., vol. 2. Oxford: Clarendon, 1892, pp.68–73.
[55] Van Wee, B. Transport and ethics: Ethics and the evaluation of [62] I. S. Jacobs and C. P. Bean, “Fine particles, thin films and
transport policies and projects. Cheltenham: Edward Elgar exchange anisotropy,” in Magnetism, vol. III, G. T. Rado and H.
Publishing 2011. Available at: Suhl, Eds. New York: Academic, 1963, pp. 271–350.
<https://books.google.sk/books?hl=en&lr=&id=rUiNNCGb_60C [63] K. Elissa, “Title of paper if known,” unpublished.
&oi=fnd&pg=PR1&ots=CXtdE3tBKV&sig=O3pnMkA69- [64] R. Nicole, “Title of paper with only first word capitalized”, J.
6zWty7N0iQTQiszF4&redir_esc=y#v=onepage&q&f=false> cit. Name Stand. Abbrev., in press.
2023-08-20. [65] Y. Yorozu, M. Hirano, K. Oka, and Y. Tagawa, “Electron
[56] Waismann, F.: Verifiability, Proceedings of the Aristotelian spectroscopy studies on magneto-optical media and plastic
Society, Supplementary Volume XIX, 1965. Available at: substrate interface,” IEEE Transl. J. Magn. Japan, vol. 2, pp. 740–
<http://users.umiacs.umd.edu/~horty/courses/readings/waismann- 741, August 1987 [Digests 9th Annual Conf. Magnetics Japan, p.
verifiability.pdf>. 301, 1982].
[66] M. Young, The Technical Writer's Handbook. Mill Valley, CA:
University Science, 1989.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 158

 ([DPSOHRI)LUH'HWHFWLRQ8VLQJ8$9DQG
&HOOXODU1HWZRUNV
5HQiW+$/8â.$ 6WDQLVODY0$5&+(96.é 0DW~ã3/(9$ 6ODYRPtU3,//È5 
'HSDUWPHQWRI(OHFWURQLFVDQG0XOWLPHGLD&RPPXQLFDWLRQV)DFXOW\RI(OHFWULFDO(QJLQHHULQJDQG
,QIRUPDWLFV7HFKQLFDO8QLYHUVLW\RI.RãLFH/HWQi6ORYDN5HSXEOLF
7(/(6(59,6VSROVUR-+DJDUX%UDWLVODYD5DþD
(PDLO UHQDWKDOXVND#WXNHVN VWDQLVODYPDUFKHYVN\#WXNHVN PDWXVSOHYD#WXNHVN VODYRPLUSLOODU#JPDLOFRP

$EVWUDFW²7KHDUWLFOHLQFOXGHVWKHXVHRI8$9HTXLSPHQWLQ WR$3WKDWKHOSVPRELOHXVHUVFRQQHFWWRQHWZRUNV
FULWLFDOVLWXDWLRQV2QHRIWKHFULWLFDOVLWXDWLRQVLVILUHVZKLFK 7KHUHLVDQDLULQWHUIDFHRISK\VLFDOSURWRFRODQGOLQNOD\HU
DUH RQH RI WKH GHDGOLHVW DQG PRVW GDQJHURXV QDWXUDO SURWRFRO EHWZHHQ WKH PRELOH DQG WKH EDVH VWDWLRQ
GLVDVWHUV LQ WKH ZRUOG LQ WKLV ZRUN QHXUDO QHWZRUNV KDYH 7RJHWKHU WKHVH FHOOV SURYLGH UDGLR FRYHUDJH RI ODUJHU
EHHQGHYHORSHGWRGHWHFWILUHVDQGWKHQHYDOXDWHWKHLPDJHV
JHRJUDSKLF DUHDV 8VHU (TXLSPHQW 8(  VXFK DV PRELOH
WDNHQ E\ 7HOOR ('8 8$9 GHYLFH 7KH UHYLHZ LV IRFXVHG RQ
WKH GHVFULSWLRQ RI SURJUDPPDEOH 8$9 DV DQ HOHPHQW RI D SKRQHVGURQHVDUHWKHUHIRUHDEOHWRFRPPXQLFDWHHYHQLI
GURQH QHWZRUN 7KH VHOHFWHG 8$9 DUH VXLWDEOH IRU WKH GHYLFH PRYHV EHWZHHQ GLIIHUHQW FHOOV GXULQJ
WKHRUHWLFDO DV ZHOO DV SUDFWLFDO WHDFKLQJ RI GURQH QHWZRUN WUDQVPLVVLRQ>@
GHVLJQ
.H\ZRUGV² &HOOXODU QHWZRUNV 'URQHV )LUH GHWHFWLRQ
,, &(//8/$51(7:25.6
8$9V &HOOXODUQHWZRUNVRIIHUH[WHQVLYHKLJKVSHHGDQGVHFXUH
 ZLUHOHVV FRQQHFWLYLW\ ZLWKLQ OLFHQVHG VSHFWUXP ZKLFK
, ,1752'8&7,21 FDQVLJQLILFDQWO\LPSURYHWKHFRQWURODQGVDIHW\RIVPDOO
8$9 RSHUDWLRQV DQG HQDEOH EH\RQG WKH YLVXDO OLQH RI
8QPDQQHG DHULDO YHKLFOHV 8$9V  DOVR NQRZQ DV VLJKW %9/26  RSHUDWLRQV 7KH FHOOXODU DUFKLWHFWXUH KDV
GURQHV KDYH JDLQHG VLJQLILFDQW DWWHQWLRQ LQ YDULRXV VHYHUDODGYDQWDJHV
PLOLWDU\DQGFLYLOVHUYLFHGLVFLSOLQHVGXHWRWKHLUHQKDQFHG ‡ FRYHUDJHFDQEHH[WHQGHGWRODUJHDUHDVE\PXOWLSOH
VWDELOLW\ DQG UHVLOLHQFH LQ VHYHUDO RSHUDWLRQV 8$9 *&6V ZKLFKSURYLGHQDWXUDO UHGXQGDQF\LQVXFKD
DSSOLFDWLRQV DUH H[SDQGLQJ H[FHSWLRQDOO\ GXH WR WKHLU ZD\ WKDW LI RQH FRQQHFWLRQ LV ZHDN DQRWKHU
DGYDQFHGXVHLQWKH,QWHUQHWRIWKLQJV ,R7 *DQG%* FRQQHFWLRQFDQSHUIRUPEHWWHU
8$9VKDYHEHHQXVHGLQDYDULHW\RIDSSOLFDWLRQVRYHUWKH ‡ FHOOXODUQHWZRUNVDUHFXUUHQWO\DEOHWRVXSSRUWKLJK
SDVW GHFDGH LQFOXGLQJ REMHFW GHWHFWLRQ DQG WUDFNLQJ WKURXJKSXW
SXEOLF VDIHW\ WUDIILF VXUYHLOODQFH PLOLWDU\ RSHUDWLRQV ‡ WKH HQGXVHU HTXLSPHQW 8(  LV VPDOO FRQVXPHV
UHFRQQDLVVDQFH RI KLGGHQ RU GDQJHURXV DUHDV LQGRRU RU ORZ SRZHU DQG EHQHILWV IURP PDVV SURGXFWLRQ
RXWGRRU QDYLJDWLRQ DWPRVSKHULF VHQVLQJ SRVWGLVDVWHU ZKLFK FDQ LQFUHDVH LWV XVH LQ SUDFWLFDO 8$9
RSHUDWLRQV KHDOWKFDUH GDWD VKDULQJ LQIUDVWUXFWXUH LPSOHPHQWDWLRQV
PDQDJHPHQW HPHUJHQF\ DQG FULVLV PDQDJHPHQW IUHLJKW ‡ UHGXFHVLQWHUIHUHQFHIURPRWKHUVLJQDOV>@>@>@
WUDQVSRUW IRUHVW ILUH PRQLWRULQJ DQG ORJLVWLFV 8$9V DUH
QRGHVRIWKHRYHUDOOV\VWHPZKLFKLVPDQDJHGWKURXJKDQ 
LQWHUQDOPHFKDQLVPDQG$G+RFFRPPXQLFDWLRQEHWZHHQ $ 8$9EDVHGFHOOXODUQHWZRUNV
QRGHV>@7KH\FDQDOVREHXVHGLQKHDOWKFDUHIRU 0RELOH 1HWZRUN 2SHUDWRUV 012V  GHVLJQ WKHLU
‡SUHKRVSLWDOHPHUJHQF\FDUH QHWZRUNV IRU FRYHUDJH DQG FDSDFLW\ RSWLPL]DWLRQ DQG
‡DFFHOHUDWLRQRIODERUDWRU\GLDJQRVWLFVDQGWHVWLQJ WKHUHIRUH FHOOXODU QHWZRUNV DUH RSWLPL]HG IRU WHUUHVWULDO
EURDGEDQG ZKHUH WKH %6 %DVH 6WDWLRQ  DQWHQQDV DUH
‡PRQLWRULQJ
WLOWHGGRZQZDUGVWRUHGXFHLQWHUIHUHQFHIURPRWKHUFHOOV
$W WKH VDPH WLPH GURQHV DUH XVHG LQ WKH KHDOWKFDUH 7LOWLQJ LV SRVVLEOH ERWK PHFKDQLFDOO\ DQG HOHFWULFDOO\
LQGXVWU\ WR GHOLYHU YDFFLQHV KHPDWRORJ\ SURGXFWV DQG EXWLVXVXDOO\XVHGWRFRYHUDQGRSWLPL]HFDSDFLW\
DXWRPDWHGH[WHUQDOGHILEULOODWRUV>@
+RZHYHU H[LVWLQJ 8$6 8QPDQQHG $LUFUDIW 6\VWHPV 
/HIW DQG ULJKWMXVWLI\ \RXU FROXPQV 8VH WDEOHV DQG PDLQO\UHO\RQVLPSOHSRLQWWRSRLQWFRPPXQLFDWLRQYLD
ILJXUHVWRDGMXVWFROXPQOHQJWK2QWKHODVWSDJHRI\RXU DQXQOLFHQVHGEDQG HJ,60*+] ZKLFKKDVDORZ
SDSHUDGMXVWWKHOHQJWKVRIWKHFROXPQVVRWKDWWKH\DUH GDWD UDWH LV XQUHOLDEOH LQVHFXUH VXVFHSWLEOH WR
HTXDO 8VH DXWRPDWLF K\SKHQDWLRQ DQG FKHFN VSHOOLQJ LQWHUIHUHQFH GLIILFXOW WR OHJLWLPDWHO\ PRQLWRU DQG
'LJLWL]HRUSDVWHGRZQILJXUHV PDQDJHDQGFDQRQO\IXQFWLRQWRDYHU\OLPLWHGH[WHQW
$FHOOXODUQHWZRUNLVDUDGLRQHWZRUNGLVWULEXWHGRYHU $V WKH QXPEHU RI 8$9V DQG WKHLU DSSOLFDWLRQV LV
WKH JURXQG WKURXJK FHOOV ZKHUH HDFK FHOO FRQWDLQV D H[SHFWHGWRFRQWLQXHWRJURZLQWKHFRPLQJ \HDUVLWLV
WUDQVPLWWHUUHFHLYHU ZLWK D IL[HG ORFDWLRQ NQRZQ DV D
JURXQGFRQWUROVWDWLRQ *&6 KDVDEDVHVWDWLRQDQDORJRXV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 159

 LPSHUDWLYH WR GHYHORS QHZ ZLUHOHVV WHFKQRORJLHV WKDW & $GYDQWDJHVRI8$9QHWZRUNV
ZLOO HQDEOH D VLJQLILFDQW LPSURYHPHQW LQ 8$9 JURXQG 7KDQNV WR WKH QHDUXELTXLWRXV DYDLODELOLW\ RI FHOOXODU
WRJURXQGFRPPXQLFDWLRQV QHWZRUNV DURXQG WKH ZRUOG FHOOXODUFRQQHFWHG 8$9V
&HOOXODUFRQQHFWHG 8$9 LV D SURPLVLQJ WHFKQRORJ\ WR DOORZWKHJURXQGSLORWWRUHPRWHO\FRPPDQGDQGFRQWURO
DFKLHYH WKH DERYH JRDO ZLWK 8$9V IRU YDULRXV & &  WKH 8$9 ZLWK HVVHQWLDOO\ XQOLPLWHG RSHUDWLRQDO
DSSOLFDWLRQV EHLQJ LQWHJUDWHG LQWR H[LVWLQJ DQG IXWXUH UDQJH ,W DOVR SURYLGHV DQ HIIHFWLYH VROXWLRQ WR PDLQWDLQ
FHOOXODU QHWZRUNV DV QHZ DLU XVHU HTXLSPHQW 8(  ZLUHOHVV FRQQHFWLYLW\ EHWZHHQ 8$9V DQG YDULRXV RWKHU
&RPSDUHG WR WUDGLWLRQDO JURXQG8$9 FRPPXQLFDWLRQ VWDNHKROGHUVVXFKDVHQGXVHUVDQGDLUWUDIILFFRQWUROOHUV
WKURXJK SRLQWWRSRLQW OLQNV 8$9V ZLWK FHOOXODU OLQNV UHJDUGOHVVRIWKHLUORFDWLRQ)RUH[DPSOHXVLQJDFHOOXODU
KDYHVHYHUDODWWUDFWLYHDGYDQWDJHV>@>@ QHWZRUNOLYHYLGHRFDQEHGLUHFWO\VHQWIURPD8$9WRD
UHPRWHDXGLHQFHDURXQGWKHZRUOG
% 8$91HWZRUNV
:LWK DGYDQFHG FHOOXODU WHFKQRORJLHV DQG
8QPDQQHG DHULDO YHKLFOHV DUH FRPSOH[ DHULDO DXWKHQWLFDWLRQ PHFKDQLVPV FHOOXODUFRQQHFWHG 8$9V
SODWIRUPVGHVLJQHGIRUZLUHOHVVVHQVLQJDQGFRQWURO7KH KDYH WKH SRWHQWLDO WR DFKLHYH VLJQLILFDQW SHUIRUPDQFH
XVH RI 8$9V HVSHFLDOO\ TXDGURWRUV KDV SHUPHDWHG DOO LPSURYHPHQWV RYHU VLPSOH GLUHFW JURXQGWR8$9
DVSHFWV RI KXPDQ OLIH 8$9V FDQ EH HIIHFWLYHO\ XVHG WR FRPPXQLFDWLRQ LQ WHUPV RI UHOLDELOLW\ VHFXULW\ DQG
VXUYH\ XUEDQ DUHDV LQFOXGLQJ EXLOGLQJV LQIUDVWUXFWXUH FRPPXQLFDWLRQWKURXJKSXW
QDWXUDO DUHDV DQG HQYLURQPHQWDO KHDOWK &KDOOHQJHV VXFK
7UDGLWLRQDO 8$9 QDYLJDWLRQ PDLQO\ UHOLHV RQ D VDWHOOLWH
DVEDGZHDWKHUFRQGLWLRQVOLPLWHGRUFRPSOHWHDEVHQFHRI
QHWZRUN VXFK DV WKH *36 *OREDO 3RVLWLRQLQJ 6\VWHP 
QHWZRUNFRQQHFWLYLW\OLPLWHGYLVXDOUDQJHXQDYDLODELOLW\
EXW LW LV SURQH WR LQWHUIHUHQFH ZLWK VDWHOOLWH VLJQDOV IRU
RI*36VLJQDODQGRWKHUVLPLODUSUREOHPVDUHW\SLFDOIRU
H[DPSOHGXHWREORFNLQJE\WDOOEXLOGLQJVRUEDGZHDWKHU
WKH XVH RI 8$9V HVSHFLDOO\ D ODUJHU JURXS VZDUP  RI
FRQGLWLRQV 0RELOH 8$9V RIIHU RQH HIIHFWLYH PHWKRG
8$9VIRUVHDUFKDQGVXUYHLOODQFHWDVNV>@
DPRQJ RWKHUV VXFK DV GLIIHUHQWLDO *36 '*36  WR
 DFKLHYH PRUH UREXVW 8$9 QDYLJDWLRQ E\ XVLQJ FHOOXODU
7$%/(,
&203$5,6212)',))(5(17&(//1(7:25.6 VLJQDOVDVDVXSSOHPHQWWR*36QDYLJDWLRQ
/DVW EXW QRW OHDVW FHOOXODUFRQQHFWHG 8$9V DUH DOVR
 DEd sEd hsŶĞƚǁŽƌŬƐ
FRVWHIIHFWLYH 2Q WKH RQH KDQG WKH\ FDQ UHXVH WKH
 
      
  ! 
PLOOLRQVRIFHOOXODUEDVHVWDWLRQV %6V DOUHDG\GHSOR\HG
    
  DURXQG WKH ZRUOG ZLWKRXW WKH QHHG WR EXLOG QHZ
   ! 

    "   
"    LQIUDVWUXFWXUHGHGLFDWHGRQO\WR8$6VLJQLILFDQWO\VDYLQJ
ĞƐĐƌŝƉƚŝŽŶ 
   ! 
! #$
QHWZRUNGHSOR\PHQWFRVWV2QWKHRWKHUKDQGLWFDQDOVR
    
    KHOSVDYHRSHUDWLRQDOFRVWVE\FRQQHFWLQJ8$9& &DQG
  !  
    
  RWKHU QXPHURXV W\SHV RI SD\ORDG FRPPXQLFDWLRQV LQWR

FHOOXODU V\VWHPV FUHDWLQJ QHZ EXVLQHVV RSSRUWXQLWLHV IRU

 % 
* % +  ERWKPRELOHDQG8$9RSHUDWRUV>@
&+,+'  /++'
&'
 )-./+' 0   !
DŽďŝůŝƚLJ
(  
  !   &   ,
,,, 5(9,(:2)&+($3$1'352*5$0(%/('521(6
  
" ) )
  -!   - 3URJUDPPDEOH GURQHV DUH EHFRPLQJ LQFUHDVLQJO\
       ) 
 
       SRSXODUEHFDXVHWKH\RIIHUDJUHDWQXPEHURIEHQHILWVIRU
ERWK FRQVXPHUV DQG EXVLQHVVHV )RU H[DPSOH
 %   %   

(  -        - SURJUDPPDEOHGURQHVFDQEHXVHGIRUSKRWRJUDSK\YLGHR
dŽƉŽůŽŐLJ
  !  '0!  UHFRUGLQJ DQG HYHQ PDSSLQJ ,Q DGGLWLRQ SURJUDPPDEOH
 #$ GURQHV FDQ EH SURJUDPPHG WR DXWRQRPRXVO\ IROORZ D
 1    
    0    
VSHFLILF IOLJKW SDWK PDNLQJ WKHP LGHDO IRU XVH LQ VHDUFK
ŚĂŶŐĞƐŝŶ   02 3   %  )-   DQGUHVFXHPLVVLRQVRURWKHUKD]DUGRXVHQYLURQPHQWV
ƚŽƉŽůŽŐLJ   !   2   3RSXODU PRGHOV LQFOXGH '-, 7HOOR 5\]H 7HOOR DQG
2      )
  ) 3DUURW 0DPER 7KHVH GURQHV DUH FKHDS SRUWDEOH DQG
 0     %#$  SURYLGH SURJUDPPLQJ FDSDELOLWLHV WKURXJK WKHLU RZQ
dLJƉĞ ŽĨ   ! 1  ! !  ) VRIWZDUH LQWHUIDFH RU RWKHU SODWIRUPV VXFK DV 3\WKRQ ,Q
ĞŶĞƌŐLJ
ƵƐĞĚ
! - 
  )!
 ) 
 ! 
4 
) )
DGGLWLRQWKHUHDUHDOVRSURJUDPPDEOHGURQHVVXFKDV'-,
    0DYLF 0LQL DQG '-, 3KDQWRP WKDW DUH HTXLSSHG ZLWK
 
 KLJKTXDOLW\ FDPHUDV DQG VHQVRUV 7KHVH GURQHV DUH
 
dLJƉŝĐĂůƵƐĞ 1 !  
(   
(   
)     
FDSDEOHRIIO\LQJDWORQJHUGLVWDQFHVDQGDWKLJKDOWLWXGHV
ŝŶƉƵďůŝĐ    
   -
  5!   DOORZLQJ IRU D ODUJHU DUHD WR EH FRYHUHG $OO RI WKHVH
 )  
    GURQHV DOVR KDYH YDULRXV VDIHW\ IHDWXUHV WR KHOS SUHYHQW
DFFLGHQWV VXFK DV REVWDFOH VHQVRUV DQG *36 QDYLJDWLRQ
)RU WKHLU XVH LQ D GURQH VHQVRU QHWZRUN WKH\ FDQ EH
HTXLSSHG ZLWK YDULRXV VHQVRUV DQG FDPHUDV ZKLFK

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 160

LQFUHDVHV WKHLU DELOLW\ WR FROOHFW GDWD DQG XVH WKHP LQ FRQWURO,WFDQSHUIRUPYDULRXVWULFNVVXFKDVIOLSVEDUUHO
YDULRXVDSSOLFDWLRQV>@ UROOVDQGGHJUHHVSLQVPDNLQJLWLGHDOIRUWKRVHZKR
ZDQWWROHDUQKRZWRIO\GURQHV,WDOVRKDVDQLQWHJUDWHG
$ '-,0DWUL[57.
FDPHUD WKDW FDSWXUHV S +' YLGHR DQG SKRWRV 7KH
7KH'-,0DWULFH57.LVDTXDOLW\GURQHIRUWKRVH 3DUURW 0DPER GURQH FDQ EH SURJUDPPHG XVLQJ VHYHUDO
ZKR QHHG D ORQJ IOLJKW WLPH DQG D JRRGUDQJH ,W LV DOVR SURJUDPPLQJ ODQJXDJHV LQFOXGLQJ 6FUDWFK 3\WKRQ DQG
LGHDO IRU DQ\RQH ZKR ZDQWV WR XVH WKH GURQH IRU 6ZLIW 7KLV SURJUDPPLQJ DELOLW\ DOORZV XVHUV WR FUHDWH
SKRWRJUDSK\ RU YLGHR UHFRUGLQJ 7KH RQO\ GRZQVLGH LV WKHLURZQIOLJKWSDWWHUQVDXWRPDWHWDVNV>@>@
WKDW LW LV TXLWH ODUJH DQG KHDY\ ZKLFK FDQ PDNH LW
GLIILFXOW WR WUDQVSRUW 7KH '-, 0DWULFH  57. KDV D ' '-,0DYLF0LQL
IOLJKW WLPH RI  PLQXWHV 7KH 7% EDWWHU\ KDV D 7KH'-,0DYLF0LQLLVDVPDOOOLJKWZHLJKWDQGSRUWDEOH
FDSDFLW\RIP$KZKLFKLVVLJQLILFDQWO\KLJKHUWKDQ GURQH GHVLJQHG IRU EHJLQQHUV DQG DQ\RQH ZKR ZDQWV D
PRVWGURQHEDWWHULHVRQWKHPDUNHW7KLVDOORZVWKHGURQH FRPSDFW DQG HDV\WRXVH GURQH ,W LV PDQXIDFWXUHG E\
WR VWD\ LQ WKH DLU PXFK ORQJHU :KHQ FRQVLGHULQJ LWV '-, D &KLQHVH WHFKQRORJ\ FRPSDQ\ WKDW VSHFLDOL]HV LQ
UHDFK ZH QHHG WR WKLQN DERXW WZR WKLQJV FRQWURO UHDFK WKH SURGXFWLRQ RI GURQHV DQG FDPHUDV 7KH '-, 0DYLF
DQG UHDOWLPH YLGHR VWUHDPLQJ UHDFK 7KH GURQH KDV D 0LQLZHLJKVRQO\JUDPVPDNLQJLWRQHRIWKHOLJKWHVW
FRQWURO UDQJH RI  NP DQG D UHDOWLPH YLGHR VWUHDPLQJ GURQHV RQ WKH PDUNHW 'HVSLWH LWV VPDOO VL]H LW LV
UDQJHRINP6RZHZLOOEHDEOHWRIO\WKHGURQHTXLWH HTXLSSHGZLWKDD[LVVWDELOL]HGFDPHUDWKDWFDQFDSWXUH
IDUDQGVWLOOKDYHFRQWURORYHULW)DVWHUGURQHVDUHXVXDOO\ .YLGHRDQGPHJDSL[HOSKRWRV7KHFDPHUDLVDOVR
XVHGIRUFRPSHWLWLRQVRUWRJHWDPD]LQJVKRWV+RZHYHU HTXLSSHGZLWKDFPVHQVRUDQGFDQVKRRWDWIUDPHV
WKH '-, 0DWULFH  57. KDV D WRS VSHHG RI  PV SHUVHFRQG7KHGURQHKDVD PD[LPXP IOLJKWWLPHRI
ZKLFK LV IDVW HQRXJK IRU PRVW XVHV 7KLV GURQH LV PRUH PLQXWHV DQG FDQ IO\ XS WR  NLORPHWHUV IURP WKH
IRFXVHG RQ VWDELOLW\ DQG IOLJKW WLPH WKDQ VSHHG 7KH FRQWUROOHU GHSHQGLQJ RQ ORFDO UHJXODWLRQV ,W DOVR KDV
GURQH VFDPHUDKDVDUHVROXWLRQRISDQGFDQUHFRUGDW VHYHUDO VPDUW IOLJKW PRGHV LQFOXGLQJ 4XLFN6KRW ZKLFK
 ISV IUDPHV SHU VHFRQG  ,W DOVR KDV D ƒ ILHOG RI DXWRPDWLFDOO\ FUHDWHV FLQHPDWLF VKRWV DQG &LUFOH ZKLFK
YLHZ ZKLFK LV YHU\ JRRG IRU JHWWLQJ D ZLGH VKRW ,W RUELWVDVSHFLILFREMHFW
ZHLJKVNJ>@
,9 &20081,&$7,21%(7:((17+('521($1'
% '-,7HOOR *5281'67$7,21
7KH 7HOOR GURQH LV PDQXIDFWXUHG E\ 5\]H 7HFK DQG &HOOXODU QHWZRUNV RIIHU H[WHQVLYH KLJKVSHHG DQG
SRZHUHGE\'-,,WKDVDZHLJKWRIJGLPHQVLRQVRI VHFXUH ZLUHOHVV FRQQHFWLYLW\ ZLWKLQ OLFHQVHG VSHFWUXP
PPîPPîPPDQGDPPSURSHOOHU7KH ZKLFKFDQVLJQLILFDQWO\LPSURYHWKHFRQWURODQGVDIHW\RI
GURQH LV HTXLSSHG ZLWK D QXPEHU RI VHQVRUV RQ ERDUG VPDOO8$9RSHUDWLRQVDQGHQDEOHEH\RQGWKHYLVXDOOLQH
LQFOXGLQJ DQ RSWLFDO VHQVRU DQ DFFHOHURPHWHU D RI VLJKW %9/26  RSHUDWLRQV ,Q RUGHU IRU WKH 8$9 DQG
J\URVFRSHDEDURPHWHUDQGD7LPHRI)OLJKW 7R) UDQJH WKHDSSOLFDWLRQWRFRPPXQLFDWHDOO7HOOR6'.FRPPDQGV
VHQVRU &RQQHFWLQJ WR WKH GURQH FDQ EH GRQH YLD :L)L PXVWEHZUDSSHGLQ3\WKRQIXQFWLRQV$IWHUWKDWVSHFLDO
XVLQJ WKH DFFRPSDQ\LQJ 6'. SURYLGHG E\ '-,  $Q FRPPDQGV DUH PDGH DYDLODEOH IRU FRQWUROOLQJ WKH 8$9
RSWLFDO IORZ VHQVRU ORFDWHG DW WKH ERWWRP RI WKH GURQH ZKLFKFDQEHXVHGZLWKDQ\DOJRULWKPLFFRQVWUXFWLRQVDQG
WRZDUGVWKH JURXQGFDQFDOFXODWHWKHFKDQJHRI;DQG= GDWDSURJUDPPLQJODQJXDJH7KHJHQHUDOFRPPXQLFDWLRQ
FRRUGLQDWHVLQUHDOWLPH VFKHPH RI RQH 8$9 7HOOR ('8 ZLWK WKH DSSOLFDWLRQ LV
7KLV VHQVRU LV HVVHQWLDOO\ D ORZUHVROXWLRQ LPDJH VKRZQLQ)LJ>@
VHQVRU FDPHUD  WKDW WDNHV WLPHODSVH LPDJHV FRPSDUHV 
FRQVHFXWLYH LPDJHV DQG LQIHUV WKH FKDQJH LQ HDFK
FRRUGLQDWH7KHRSWLFDOIORZVHQVRUKHOSVWKHGURQHKRYHU
RYHUDJLYHQORFDWLRQZLWKRXWGULIWLQJDQGLVDOVRXVHGWR
FDOFXODWHWKHUHODWLYHVSHHGRIWKHGURQH7KHRSWLFDOIORZ
VHQVRU LV RQO\ XVHG LQWHUQDOO\ E\ WKH GURQH WR FDOFXODWH
VSHHG DQG FDQQRW EH DFFHVVHG GLUHFWO\ 7KH 7R) UDQJH
VHQVRU ORFDWHG DW WKH ERWWRP RI WKH GURQH WRZDUGV WKH
JURXQG LV XVHG WR GHWHUPLQH WKH UHODWLYH KHLJKW RI WKH
GURQHWRDQREMHFWEHORZZKLFKLVHVVHQWLDOIRUFOLPELQJ
VWDLUV DQG DYRLGLQJ REVWDFOHV IURP EHORZ 8VLQJ D
EDURPHWHU WKH GURQH FDQ GHWHFW WKH UHODWLYH KHLJKW )LJ7HOOR('8FRPPXQLFDWLRQZLWKFRPSXWHUDSSOLFDWLRQ
UHODWLYHWRWKHWDNHRIISRLQW>@>@ 
'XHWRWKHIDFWWKDWLWLVQRWSRVVLEOHWRGLUHFWO\FRQQHFW
& 3DUURW0DPER PXOWLSOH7HOOR('8VWRDFRPSXWHULWLVQHFHVVDU\WRXVH
7KH 3DUURW 0DPER )O\ PLQLGURQH LV VWDEOH VDIH VPDOO DQDX[LOLDU\FRQQHFWLRQWKDWFRQQHFWVWKHJURXSRIGURQHV
DQG HDV\ WR FRQWURO ,W LV HTXLSSHG ZLWK DQ XOWUDVRXQG DQG WKH DSSOLFDWLRQ 6XFK D FRQQHFWLRQ SRLQW LV D URXWHU
VHQVRUDGRZQZDUGIDFLQJFDPHUDDSUHVVXUHVHQVRUDQG 7KHSURJUDPVHQGVFRPPDQGVWRWKHURXWHUZKLFKLQWXUQ
DQ ,08 VHQVRU ,W LV DOVR HTXLSSHG ZLWK D D[LV VHQGV WKRVH FRPPDQGV WR DOO 7HOOR ('8V LQ WKH VZDUP
VWDELOL]DWLRQ V\VWHP ZKLFK PDNHV LW HDV\ WR IO\ DQG $QVZHUVZLOOEHVHQWLQWKHVDPHZD\RQO\LQWKHRSSRVLWH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 161

 RUGHUIURPWKH8$9WRWKHFRPSXWHUWKURXJKWKHURXWHU 7RWUDLQWKHQHXUDOQHWZRUNWKHGDWDVHWLVGLYLGHGLQWR
&RPPXQLFDWLRQLVVKRZQLQ)LJ WUDLQLQJLPDJHVDQGWUDLQLQJYDOLGDWLRQLPDJHVLQD
 UDWLR 6R ZH XVHG  LPDJHV IRU WUDLQLQJ WKH QHXUDO
 QHWZRUN DQG  IRU YDOLGDWLRQ 9DOLGDWLRQ LPDJHV DUH
XVHGLQQHXUDOQHWZRUNVWRHYDOXDWLQJPRGHOSHUIRUPDQFH
GXULQJ WKH WUDLQLQJ SURFHVV DQG IRU WXQLQJ
K\SHUSDUDPHWHUV :KHQ WUDLQLQJ D QHXUDO QHWZRUN WKH
JRDO LV WR PLQLPL]H WKH ORVV RI WUDLQLQJ GDWD EXW WKH
XOWLPDWH JRDO LV WR FUHDWH D PRGHO WKDW FDQ JHQHUDOL]H
DSSURSULDWHO\
,Q)LJZHFDQVHHWKDWWKHQHXUDOQHWZRUNKDVJRQH
WKURXJK WHQ HSRFKV RI WUDLQLQJ (DFK HSRFK ODVWHG
DSSUR[LPDWHO\  WR  VHFRQGV XVLQJ HLJKW ILOWHUV ,I
)LJ7HOOR('8VTXDGURQFRPPXQLFDWLRQZLWKFRPSXWHU PRUH ILOWHUV ZHUH XVHG HJ    VR LW ZRXOG
VLJQLILFDQWO\LQFUHDVHWKHWUDLQLQJWLPH%\XVLQJGLIIHUHQW

ILOWHU VL]HV LQ HDFK EORFN WKH QHXUDO QHWZRUN FDQ OHDUQ
9 ;&(37,211(7:25.)25),5('(7(&7,21 IHDWXUHV DW GLIIHUHQW VFDOHV (DUO\ EORFNV XVH VPDOOHU

ILOWHUV WR OHDUQ ORZOHYHO IHDWXUHV VXFK DV HGJHV DQG
$ ;&HSWLRQ1HWZRUN FRUQHUVZKLOHODWHUEORFNVXVHODUJHUILOWHUVWROHDUQKLJK
;&HSWLRQ LV D GHHS FRQYROXWLRQDO QHXUDO QHWZRUN OHYHOIHDWXUHVVXFKDVSDUWVDQGREMHFWSDWWHUQV
DUFKLWHFWXUH WKDW LQFRUSRUDWHV GHSWKVHSDUDEOH
FRQYROXWLRQV ,W LQWHUSUHWV ,QFHSWLRQ PRGXOHV LQ
FRQYROXWLRQDO QHXUDO QHWZRUNV DV DQ LQWHUPHGLDWH VWHS
EHWZHHQ UHJXODU FRQYROXWLRQ DQG D GHSWK VHSDUDEOH
FRQYROXWLRQ RSHUDWLRQ GHSWK FRQYROXWLRQ IROORZHG E\ 
SRLQWZLVHFRQYROXWLRQ >@ )LJ/LVWRIHSRFKVRIWUDLQLQJ
7KH GDWD ILUVW SDVVHV WKURXJK WKH HQWU\ IORZ WKHQ 
WKURXJK WKH PLGGOH IORZ ZKLFK LV UHSHDWHG HLJKW WLPHV & (YDOXDWLRQRIDQHXUDOQHWZRUNRQDQLPDJH
DQGILQDOO\WKURXJKWKHH[LWIORZ$VWDQGDUGFRQYROXWLRQ FDSWXUHGIURPDGURQH
OD\HU RI D QHXUDO QHWZRUN LQFOXGHV WKH SDUDPHWHUV
LQSXW RXWSXW ZLGWK KHLJKW ZKHUH ZLGWK DQG KHLJKW DUH :HXVHGDVHWRILPDJHVDVLQSXWLPDJHVWRHYDOXDWHWKH
QHXUDOQHWZRUN$OOLPDJHVZHUHPDGHXVLQJ7HOOR('8
WKHZLGWKDQGKHLJKWRIWKHILOWHU)RULQSXWFKDQQHODQG 7KH SURFHVV RI PDNLQJ D QXPEHU RI LPDJHV ZDV DV
RXWSXW ZLWKD ILOWHULWZLOOKDYHSDUDPHWHUV IROORZV
6R PDQ\ SDUDPHWHUV LQFUHDVH WKH SRVVLELOLW\ RI RYHU
‡DYLGHRKDVEHHQUHFRUGHG
ILWWLQJ
‡LWZDVWKHQHGLWHGLQWRLPDJHVE\WKHSURJUDP
2YHUILWWLQJLVDQXQGHVLUDEOHPDFKLQHOHDUQLQJEHKDYLRU
‡HGLWHGLPDJHVZHUHHYDOXDWHGE\DQHXUDOQHWZRUN
WKDW RFFXUV ZKHQ D PDFKLQH OHDUQLQJ PRGHO PDNHV
DFFXUDWHSUHGLFWLRQVIRUWUDLQLQJGDWDEXWQRWIRUQHZGDWD ,Q)LJZHFDQVHHWKHLPDJHRIWKHILUHGXULQJOXQFK
RQWKHOHIW DQGWKHLPDJHRIWKHILUHLQWKHHYHQLQJ RQ
7KHVH FDWHJRULHV LQFOXGH GHSWK FRQYROXWLRQ DQG GHSWK
WKH ULJKW  1H[W ZH ZLOO WHVW WKHVH LPDJHV RQ GLIIHUHQW
VHSDUDEOHFRQYROXWLRQ WUDLQHG QHXUDO QHWZRUNV :H WUDLQHG  GLIIHUHQW QHXUDO
% 1HXUDOQHWZRUNWUDLQLQJ QHWZRUNV ZLWK ILOWHU VL]H  DQG  7KH WKLUG QHXUDO
QHWZRUNZHWUDLQHGRQDODUJHUQXPEHURILPDJHV
7R WUDLQ WKH QHXUDO QHWZRUN LW ZDV QHFHVVDU\ WR 
GRZQORDGDVHWRILPDJHV GDWDVHW RQZKLFKWKHUHZDVD
ILUHILUH DQG LPDJHV ZKHUH WKHUH ZDV QR ILUHILUH ,Q RXU
QHXUDO QHWZRUN  ZDV DVVLJQHG IRU LPDJHV ZKHUH WKHUH

ZDVILUHDQGIRULPDJHVZKHUHWKHUHZDVQRILUH )LJ 

)LJ,QSXWLPDJHVIRUQHXUDOQHWZRUNHYDOXDWLRQ

7KHJUDSKV )LJDQG)LJ VKRZWKHGHSHQGHQFHRIWKH

RFFXUUHQFH RI ILUH RQ WKH HYDOXDWHG SKRWRV ,Q )LJ  LV
XVHG ILOWHU VL]H  DQG LQ WKH )LJ  LV XVHG ILOWHU VL]H 
7KH [D[LV GHVFULEHV WKH QXPEHU RI LPDJHV WKDW ZHUH
HYDOXDWHGDQGWKH \D[LVGHVFULEHVWKHSHUFHQWDJHRI ILUH
RFFXUUHQFHLQLQGLYLGXDOLPDJHV$SDLURIJUDSKVGHSLFWV

)LJ([DPSOHRISLFWXUHVZLWKDQGZLWKRXWWKHILUH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 162

 

WKHGHSHQGHQFHRILPDJHVWDNHQLQWKHDIWHUQRRQ $V ZH

FDQVHHWKHVL]HRIWKHILOWHUDIIHFWHGWKHSHUIRUPDQFHRI
WKHQHXUDOQHWZRUN$QHXUDOQHWZRUNZLWKDILOWHUVL]HRI
 HYDOXDWHG LPDJHV ZLWK JUHDWHU DFFXUDF\ WKDQ D ILOWHU
VL]HRI



)LJ1HXUDOQHWZRUNUHVXOWVIRULPDJHVWDNHQLQWKHDIWHUQRRQE\XVLQJ
DODUJHUVDPSOHLQWUDLQLQJDQGILOWHUVL]H  



)LJ1HXUDOQHWZRUNUHVXOWIRULPDJHVWDNHQLQWKHDIWHUQRRQ
XVLQJILOWHUVL]HV  



)LJ1HXUDOQHWZRUNUHVXOWVIRULPDJHVWDNHQLQWKHDIWHUQRRQE\XVLQJ
DODUJHUVDPSOHLQWUDLQLQJDQGILOWHUVL]H  


:HXVHGDQHXUDOQHWZRUNWKDWZDVWUDLQHGRQDODUJHU
QXPEHU RI LPDJHV 7KH JUDSK RQ WKH OHIW UHSUHVHQWV WKH
 HYDOXDWLRQRILPDJHVWKDWZHUHWDNHQLQWKHDIWHUQRRQDQG
)LJ1HXUDOQHWZRUNUHVXOWIRULPDJHVWDNHQLQWKHDIWHUQRRQ
XVLQJILOWHUVL]HV  
WKHJUDSKRQWKHULJKWUHSUHVHQWVWKHHYDOXDWLRQRILPDJHV
WKDW ZHUH WDNHQ LQ WKH HYHQLQJ +RZ ZH FDQ VHH WKH
2QWKHJUDSKV )LJDQG)LJ ZHFDQVHHWKHXVHG LPDJHVWKDWZHUHWDNHQLQWKHHYHQLQJWKHQHXUDOQHWZRUN
ILOWHUVL]HDQGILOWHUVL]H7KHVDPHLPDJHVZHUHDOVR DOVR HYDOXDWHG WKHP DOPRVW  WKDW WKHUH LV D ILUH RQ
HYDOXDWHGQDPHO\LPDJHVWDNHQLQWKHHYHQLQJ$VZHFDQ WKHP7KHLPSURYHPHQWFRPSDUHGWRWKHQHXUDOQHWZRUN
VHHERWKQHXUDOQHWZRUNVHYDOXDWHGWKHRFFXUUHQFHRIILUH ZLWK WKH DSSOLHG ILOWHU VL]H RI  DQG  RFFXUUHG ZKHQ
ZLWKDOPRVWVXFFHVV HYDOXDWLQJLPDJHVWKDWZHUHWDNHQLQWKHDIWHUQRRQ6LQFH
WKH ILUH LV QRW VR YLVLEOH LQ WKH DIWHUQRRQ WKH QHXUDO
QHWZRUNKDVGLIILFXOW\HYDOXDWLQJLWZLWKJUHDWHUDFFXUDF\
2Q WKH JUDSKV )LJ  DQG )LJ   ZH FDQ VHH D
FRPSDULVRQRIWKHHYDOXDWLRQRIWKHQHXUDOQHWZRUN ZLWKD
ODUJHUWUDLQLQJVDPSOH DQGXVLQJDILOWHUVL]HRI$VZH
FDQ VHH WKH QHXUDO QHWZRUN ZLWK D ILOWHU VL]H RI 
HYDOXDWHG WKH LPDJHV PRUH DFFXUDWHO\ LI ZH FKDQJHG WKH
LQSXW LPDJHV WKDW FRQWDLQHG ILUH WR LQSXW LPDJHV ZKLFK
GLGQRWFRQWDLQILUH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 163

 $&.12:/('*0(17
7KLV ZRUN ZDV VXSSRUWHG E\ WKH 6ORYDN 5HVHDUFK DQG
'HYHORSPHQW $JHQF\ XQGHU *UDQWV $399
$3996.7:DQGWKH6FLHQWLILF*UDQW$JHQF\
RIWKH0LQLVWU\RI(GXFDWLRQ6FLHQFH5HVHDUFKDQG6SRUW
RI WKH 6ORYDN 5HSXEOLF DQG WKH 6ORYDN $FDGHP\ RI
6FLHQFHV XQGHU *UDQWV 9(*$  9(*$
 9(*$

5()(5(1&(6
>@ 5$ %HFNHU 5 &DFHUHV . +DQVRQ -0 /RK 6 8UEDQHN $
9DUVKDYVN\ DQG & 9ROLQVN\  $ WDOH RI RQH FLW\ 8VLQJ
FHOOXODU QHWZRUN GDWD IRU XUEDQ SODQQLQJ ,((( 3HUYDVLYH
&RPSXWLQJ  SS
 >@ 6$+0RKVDQ0$.KDQ)1RRU ,8OODK 0$OVKDULI
  7RZDUGV WKH XQPDQQHG DHULDO YHKLFOHV 8$9V  $
)LJ5HVXOWVRIGLIIHUHQWQHXUDOQHWZRUNIRUFKDQJLQJLPDJHV FRPSUHKHQVLYHUHYLHZ'URQHV  
 >@ .'K\DQL7*XKDQ3*XSWD6 %KDFKDZDW**DQDSDWK\ 
.6ULQLYDVDQ  $SSOLFDWLRQVRI,R7DQG&ORXG&RPSXWLQJ
$ &29,' 'LVDVWHU 3HUVSHFWLYH ,Q 1HZ )URQWLHUV LQ &ORXG
&RPSXWLQJDQG,QWHUQHWRI7KLQJV SS &KDP6SULQJHU
,QWHUQDWLRQDO3XEOLVKLQJ
>@ 05RXVH ³&HOOXODU 1HWZRUN´  'RVWXSQp QD LQWHUQHWH
KWWSVZZZWHFKRSHGLDFRPGHILQLWLRQFHOOXODUQHWZRUN
>@ + 0DUTXHV 30DUTXHV -5LEHLUR 7$OYHV/3HUHLUD LQ 
,(((WK,QWHUQDWLRQDO:RUNVKRSRQ&RPSXWHU$LGHG0RGHOLQJ
DQG 'HVLJQ RI &RPPXQLFDWLRQ /LQNV DQG 1HWZRUNV &$0$' 
([SHULPHQWDO(YDOXDWLRQRI&HOOXODU1HWZRUNVIRU8$92SHUDWLRQ
DQG6HUYLFHV2NWyEHU,6%1
>@ &HOOXODU 1HWZRUN  $YDLODEOH RQ LQWHUQHW
KWWSVZZZJHHNVIRUJHHNVRUJFHOOXODUQHWZRUNV
>@ :KDWLV&HOOXODUQHWZRUNRU0RELOHQHWZRUN"$YDLODEOHRQ
LQWHUQHWKWWSVZZZVDPVXQJFRPLQVXSSRUWPRELOH
GHYLFHVZKDWLVDFHOOXODUQHWZRUNRUPRELOHQHWZRUN
>@ <=HQJ -/\X 5 =KDQJ ³ ,((( :LUHOHVV &RPPXQLFDWLRQV
&HOOXODU&RQQHFWHG 8$9 3RWHQWLDO &KDOOHQJHV DQG 3URPLVLQJ
 7HFKQRORJLHV´YROQR6HSWHPEHU±V
)LJ5HVXOWVRIGLIIHUHQWQHXUDOQHWZRUNIRUFKDQJLQJLPDJHV >@ /*XSWD5-DLQ*9DV]NXQ³6XUYH\RI,PSRUWDQW,VVXHVLQ8$9
&RPPXQLFDWLRQ 1HWZRUNV´ ,((( &RPPXQLFDWLRQV 6XUYH\V 
&21&/86,21 7XWRULDOVYROQR1RYHPEHU±V
7KHILUVWQHXUDOQHWZRUNWKDWXVHGDILOWHUVL]HRIKDG >@ 5-*URVV ³ %HVW 3URJUDPPDEOH 'URQHV´  'RVWXSQp QD
LQWHUQHWHKWWSVZZZSURSHOUFFRPEHVWSURJUDPPDEOHGURQHV
WURXEOH UHFRJQL]LQJ ILUH LPDJHV GXULQJ OLJKW DIWHUQRRQ  >@ 63LNDORY ($]DULD 66RQQHQEHUJ %%HQ0RVKH $$]DULD
EXW HYDOXDWHG LPDJHV LQ WKH GDUN HYHQLQJ  ZLWK DOPRVW ³9LVLRQ/HVV 6HQVLQJ IRU $XWRQRPRXV 0LFUR'URQHV´ 
 VXFFHVV UDWH 7KH VHFRQG QHXUDO QHWZRUN ZKLFK 'RVWXSQp QD LQWHUQHWH KWWSVZZZPGSLFRP
XVHGDILOWHUVL]HRIZDVVOLJKWO\EHWWHUWKDQWKHQHXUDO 
QHWZRUNZLWKDILOWHUVL]HRILQHYDOXDWLQJLPDJHVLQWKH >@ % 0DVKDULSRY ³6OHGRYiQt GURQX 5\]H 7HOOR´ ýHVNp Y\VRNp
XþHQtWHFKQLFNpY3UD]H%DNDOiUVNDSUiFD+
OLJKW DIWHUQRRQ  DQG DOVR HYDOXDWHG LPDJHV LQ WKH GDUN
>@ 30&DVHUR³$ 9LVLRQEDVHG /LQH)ROORZLQJ0HWKRG IRU 0LFUR
ZLWK DOPRVW  VXFFHVV UDWH 7KH EHVW HYDOXDWLRQ RI $LU9HKLFOHV´
ILUH LPDJHV GXULQJ OLJKW ZDV HYDOXDWHG E\ D WKLUG QHXUDO >@ 3 -DQGD ³1iYUK NRPSRQHQWRY EXQNRYêFK VLHWL QD Ei]H
QHWZRUN WKDW ZDV WUDLQHG RQ D ODUJHU VDPSOH RI LPDJHV EH]SLORWQêFK OLHWDM~FLFK SURVWULHGNRY 8$9  SUH PRQLWRURYDQLH
7KHQHXUDOQHWZRUNZLWKDILOWHUVL]HRIHYDOXDWHGWKH PLPRULDGQ\FK XGDORVWL´ 7HFKQLFNi XQLYHU]LWD Y .RãLFLDFK
'LSORPRYiSUiFD
LPDJHV PRUH DFFXUDWHO\ LI ZH FKDQJHG WKH LQSXW LPDJHV
>@ 23RKXGLQD0.RYDOHYVN\L 03\YRYDU 6HSWHPEHU 
WKDWFRQWDLQHGILUHWRLQSXWLPDJHVZKLFKGLGQRWFRQWDLQ *URXS IOLJKW DXWRPDWLRQ XVLQJ 7HOOR ('8 XQPDQQHG DHULDO
ILUH YHKLFOH,Q,(((WK,QWHUQDWLRQDO&RQIHUHQFHRQ&RPSXWHU
7KH ILHOG RI XQPDQQHG DHULDO YHKLFOHV KDV JUHDW 6FLHQFHV DQG ,QIRUPDWLRQ 7HFKQRORJLHV &6,7  9RO  SS 
 ,(((
SRWHQWLDODQGDSSOLFDWLRQLQYDULRXVILHOGV,WWXUQVRXWWKDW
>@ ' 3DQ 6XQ 5 / :DQJ =KDQJ ;  5 2 6LQQRWW 
WKHUH ZLOO EH D ORW RI LQWHUHVW LQ WKLV GHYHORSPHQW LQ WKH 'HFHPEHU  'HHSIDNH GHWHFWLRQ WKURXJK GHHS OHDUQLQJ ,Q 
QHDU IXWXUH &UHDWLQJ FRPPHUFLDO ORZFRVW VROXWLRQV IRU ,((($&0 ,QWHUQDWLRQDO &RQIHUHQFH RQ %LJ 'DWD &RPSXWLQJ
PDQ\ LQGXVWULHV LV LPSRUWDQW :H DOVR EHOLHYH WKDW LW LV $SSOLFDWLRQVDQG7HFKQRORJLHV %'&$7  SS ,(((
QHFHVVDU\ WR SD\ DWWHQWLRQ LQ WKH WHDFKLQJ RI >@ $ 3DQGH\   'HSWKZLVH FRQYROXWLRQ DQG GHSWKZLVH
FRPPXQLFDWLRQVXEMHFWVLQWKH&RPSXWHU1HWZRUNVVWXG\ VHSDUDEOHFRQYROXWLRQ
SURJUDP WR WKH LVVXH RI DGKRF QHWZRUN GHVLJQ XVLQJ 
XQPDQQHGDHULDOYHKLFOHV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 164

 0RELOH(GXFDWLRQDO$SSOLFDWLRQ 
IRU.H\VWURNH'\QDPLFV,GHQWLILFDWLRQ6\VWHPV
5HQiW+$/8â.$ 'RPLQLN02/ý$1<, 0DW~ã3/(9$ 'DQLHO+/È'(. -iQ67$â 0LQJ+VLDQ68 
<XDQ)X/,$2 
'HSDUWPHQWRI(OHFWURQLFVDQG0XOWLPHGLD&RPPXQLFDWLRQV)DFXOW\RI(OHFWULFDO(QJLQHHULQJDQG,QIRUPDWLFV 
7HFKQLFDO8QLYHUVLW\RI.RãLFH/HWQi6ORYDN5HSXEOLF
(PDLOUHQDWKDOXVND#WXNHVNGRPLQLNPROFDQ\L#VWXGHQWWXNHVNPDWXVSOHYD#WXNHVNPDWXVSOHYD#WXNHVN
GDQLHOKODGHN#WXNHVNMDQVWDV#WXNHVN7HO
6RRFKRZ8QLYHUVLW\7DLZDQ
(PDLOPLQJKVLDQJVX#JPVFXHGXWZ
1DWLRQDO<DQJ0LQJ&KLDR7XQJ8QLYHUVLW\7DLZDQ
(PDLO\IOLDR#Q\FXHGXWZ

$EVWUDFW²7KLVSDSHUGHVFULEHVDFKRVHQELRPHWULFPHWKRG
XVHGLQELRPHWULFV\VWHPVWRLGHQWLI\WKHXVHUGXULQJRQOLQH , ,1752'8&7,21
HGXFDWLRQLQWHUDFWLRQ7KHFXUUHQWGHPRSDJHRIWKHV\VWHP 7KH SUHVHQWHG PRELOH GHPRQVWUDWLRQ DSS LV D VRIWZDUH
LV WH[WGHSHQGHQW DQG LW LV SODQQHG WR LPSURYH WKH DSSOLFDWLRQ WKDW UXQV RQ D ZHE $SDFKH VHUYHU DQG LV
LGHQWLILFDWLRQ DQG DXWKHQWLFDWLRQ PHFKDQLVP WR VXSSRUW FUHDWHGXVLQJ+70/&66-DYD6FULSWDQG3+3>@8VHUV
FRQWLQXRXV DXWKHQWLFDWLRQ DQG EH DEOH WR GHWHFW FKHDWLQJ FDQDFFHVVWKLVDSSOLFDWLRQWKURXJKDZHEEURZVHUZLWKD
GXULQJ RQOLQH ZULWWHQ H[DPV 7KH DUWLFOH GHVFULEHV WKH FRQQHFWLRQ WR WKH XQLYHUVLW\ ORFDO VHUYHU 7KLV PRELOH
NH\VWURNH G\QDPLFV PHWKRG DQG LQIRUPV WKH UHDGHU RI GHPRQVWUDWLRQDSSOLFDWLRQLVXVHGIRULQWHUDFWLYHOHDUQLQJ
FXUUHQWWUHQGVXVLQJWKLVPHWKRGDQGWKHLPSOHPHQWDWLRQRI EHWZHHQWHDFKHUVDQGVWXGHQWVDQGLVDLPHGDWHGXFDWLRQ
NH\VWURNH G\QDPLFV ZLWKLQ D ZHE DSSOLFDWLRQ XVHG LQ DQ LQWKHVXEMHFWRI%LRPHWULF6HFXULW\6\VWHPV>@'XHWR
HGXFDWLRQDOELRPHWULFVFODVV WKHODUJHDPRXQWRILQIRUPDWLRQWKDWLVDFTXLUHGRQDGDLO\
.H\ZRUGV ² NH\VWURNH G\QDPLFV HGXFDWLRQDO DSSOLFDWLRQ EDVLVLWLVLPSRUWDQWQRWRQO\WRWDNHLQWKLVLQIRUPDWLRQ
ELRPHWULFVPRELOHDSSOLFDWLRQXVHULGHQWLILFDWLRQ EXW DOVR WR XQGHUVWDQG LW >@ 7KHUHIRUH WKLV ZHEEDVHG

)LJ   +RPHSDJHRIWKHZHEDSSOLFDWLRQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 165

OHDUQLQJ DSSOLFDWLRQ DOORZV VWXGHQWV WR OHDUQ DERXW
ELRPHWULF PHWKRGV VXFK DV IDFH UHFRJQLWLRQ YRLFH
UHFRJQLWLRQRUPRUHUHFHQWO\UHFRJQLWLRQXVLQJNH\VWURNH
G\QDPLFV 7KH LPSOHPHQWDWLRQ RI WKH ODWWHU ELRPHWULF
PHWKRG NH\VWURNH G\QDPLFV LV WKH NH\ HOHPHQW RI WKH
SDSHU DQG WKH IROORZLQJ VHFWLRQV >@ 7KH NH\VWURNH
G\QDPLFV LV D YHU\ HIIHFWLYH PHWKRG RI LGHQWLI\LQJ D
UHPRWH XVHU ZRUNLQJ LQ WKH ZHE DSSOLFDWLRQ ZKHUH DOVR
PRXVHPRYHPHQWVFRXOGEHWDNHQLQWRDFFRXQW>@XVLQJ )LJ   3DJHIRUNH\VWURNHG\QDPLFVEDVHGXVHULGHQWLILFDWLRQ
PXOWLVHQVRU\IXVLRQDSSURDFK>@
$ ,GHQWLILFDWLRQRIWKHXVHUEDVHGRQKLVW\SLQJ
,, ('8&$7,21$/:(%6,7(
EHKDYLRU
7KH ZHEVLWH LV ORFDWHG RQ WKH XQLYHUVLW\ KDUGZDUH
 7KHXVHURIWKHHGXFDWLRQDOZHEVLWHFDQEHUHFRJQL]HG
UXQQLQJ WKH $SDFKH ZHE VHUYHU )LJ  VKRZV WKH KRPH
EDVHG RQ KLVKHU W\SLQJ RQ WKH NH\ERDUG KHQFH WKH
SDJHZKHQWKHXVHUHQWHUVWKHVLWH7KHXVHULVSUHVHQWHG
.H\VWURNH'\QDPLFV .' ELRPHWULFPHWKRGZLOOEHXVHG
ZLWK D ZHOFRPH VFUHHQ FXUUHQWO\ LQ 6ORYDN ODQJXDJH 
IURPNH\ERDUGWLPLQJGDWDFDSWXUHG>@7KLVIXQFWLRQZLOO
:KHQWKHXVHUFOLFNVRQWKH*HW6WDUWHGEXWWRQWKHSDJH
EH XVHG LI WKH XVHU VHOHFWV WKH RSWLRQ FDOOHG 3HUVRQ
GLVSOD\VWKHRUHWLFDOLQIRUPDWLRQDERXWELRPHWULFV2QWKH
LGHQWLILFDWLRQ EDVHG RQ W\SLQJ ZKLFK LV ORFDWHG LQ WKH
ZHOFRPHVFUHHQLWLVDOVRSRVVLEOHWRQRWLFHWKHPHQXRI
PHQX7KHDSSHDUDQFHRIWKHKRPHSDJHDIWHUVHOHFWLQJWKH
FKRLFHV7RXVH.H\VWURNH'\QDPLFV .' >@LWLVRQO\
PHQWLRQHGRSWLRQFDQEHVHHQLQ)LJ 
QHFHVVDU\ WR PRYH WKH PRXVH WR WKH 3HUVRQ
2QWKHLGHQWLILFDWLRQ KRPHSDJHWKH XVHUFDQVHHWKH
,GHQWLILFDWLRQ VHFWLRQ DQG WKHQ FOLFN WKH %DVHG RQ
ZRUG WKDW ZLOO EH XVHG DV WKH SDWWHUQ IRU UHFRJQLWLRQ LQ
W\SLQJRSWLRQ
WKLVFDVHLWLVSDVVZRUGDQGDWH[WER[ZLWKDFRXQWHU

RQWKHULJKW,QWKHWH[WER[WKHXVHUW\SHVWKHVSHFLILHG

ZRUGDQGDIWHUW\SLQJWKHZRUGFRUUHFWO\DQGSUHVVLQJWKH

)LJ   7DEOHZLWKNH\ERDUGWLPLQJGDWD

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 166

 

)LJ   ([DPSOHRIQDUUDWLYHUHVXOWV

(QWHU NH\ WKH WH[W ER[ LV FOHDUHG DQG WKH FRXQWHU LV LVGLVSOD\HG%RWKRSWLRQVDUHGHVFULEHGEHORZ$UDQGRP
LQFUHPHQWHG E\ RQH 7KH FRXQWHU VKRZV WKH QXPEHU RI LGHQWLILHULVJHQHUDWHGIRUHDFKXVHUZKHQWKH\HQWHUWKH
FRUUHFWO\W\SHGZRUGV$IWHUHDFKFKDUDFWHULVW\SHGLQWR ZHEVLWH ZKLFK LV VWRUHG LQ FRRNLHV :KHQ \RX FKDQJH
WKH WH[W ER[ WKH XVHU FDQ YLHZ WKH UHODWLYH WLPH LQ FRPSXWHUVRUGHOHWHFRRNLHVWKHLGHQWLILHUZLOOFKDQJH
PLOOLVHFRQGV IRU HDFK FKDUDFWHU LQ WKH WDEOH EHORZ WKH ,IWKHXVHUHQWHUVWKHZRUGSDVVZRUGDWOHDVWRQFHDQG
KRPH SDJH )LJ   (DFK FRUUHFWO\ W\SHG FKDUDFWHU WKDW FOLFNV RQ WKH 5HFRJQLWLRQ EXWWRQ WKH UHFRJQLWLRQ
FRUUHVSRQGV WR D VSHFLILHG ZRUG LV PDUNHG LQ JUHHQ IXQFWLRQ VWDUWV LQ WKH EDFNJURXQG ZKLFK FROOHFWV DOO
2WKHUZLVHLWLVPDUNHGLQZKLWH JUHHQ LH FRUUHFWO\ W\SHG FKDUDFWHUV WKHLU WLPHVWDPSV
7KH7DEOHWKDWFDQEHVHHQLQ)LJXUHKDVFROXPQV DQGFRPSDUHVWKHPZLWKDOOWKHGDWDLQWKHGDWDEDVH:KHQ
‡ &ROXPQ 1R   GHWHUPLQHV WKH RUGHU RI WKH WKHIXQFWLRQFRPSOHWHVDQGWKHUHFRJQLWLRQLVVXFFHVVIXO
FKDUDFWHU WKHXVHULVSUHVHQWHGZLWKDJUDSKDQGDWDEOHRIWKHUHVXOWV
‡ &ROXPQ1RLQGLFDWHVZKLFKFKDUDFWHULWZDVDQG DVVHHQLQ)LJ
LWVUHODWLYHSUHVVWLPHLQPLOOLVHFRQGV ,Q)LJDJUDSKFDQEHVHHQZKHUHWKH\D[LVVKRZV
‡ &ROXPQ1RLQGLFDWHVZKLFKFKDUDFWHULWZDVDQG WKH VLPLODULW\ SHUFHQWDJH FRPSXWHG IURP WKH FKRVHQ
LWVUHODWLYHVWURNHWLPHLQPLOOLVHFRQGV 6FDOHG0DQKDWWDQ'LVWDQFHDOJRULWKP>@DQGWKH[D[LV
‡ &ROXPQ1RLQGLFDWHVWKH'ZHOOWLPHYDOXH VKRZV WKH ,' RI WKH XVHUV ZKR KDYH EHHQ VWRUHG LQ WKH
‡ &ROXPQ1RLQGLFDWHVWKH/DWHQF\WLPHYDOXH  GDWDEDVHDQGZLWKZKRPWKHUHLVWKHKLJKHVWPDWFKXSWR
,IWKHXVHUKDVW\SHGWKHZRUGSDVVZRUGDWOHDVWRQFH DPD[LPXPRIWKHWRSWHQ7KH60'ZDVFKRVHQDVTXLFN
WKHUHFRJQLWLRQIXQFWLRQFDQEHXVHG,IWKHXVHUKDVW\SHG RQOLQH FRPSXWDEOH DOJRULWKP FRPSDUDEOH ZLWK QHXUDO
WKHZRUGPRUHWKDQWHQWLPHVWKHVDYHWRGDWDEDVHEXWWRQ QHWZRUNVROXWLRQV>@%HORZWKHJUDSKWKHUHLVDWDEOH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 167

 

)LJ   7KHFXPXODWLYHPDWFKFXUYHIRU5DQNWR5DQN

IURP ZKLFK ZH FDQ DGGLWLRQDOO\ UHDG WKH 7UDLQHG YDOXH 7KH &0& FXUYH VKRZV WKH FXPXODWLYH LGHQWLILFDWLRQ
ZKLFKVKRZVKRZPDQ\ZRUGVWKHXVHUKDVHQWHUHGLQWR DFFXUDF\ IRU GLIIHUHQW ORFDWLRQVUDQN 7RS.  ZLWK
WKHGDWDEDVHDQGWKHUHIRUHKRZPDQ\ZRUGVKLVPRGHOZDV GLIIHUHQWLGHQWLILFDWLRQSUREDELOLWLHV%DVHGRQWKHDERYH
EXLOWIURP YDOXHV LW FDQ EH VHHQ WKDW WKH V\VWHP FDQ UHFRJQL]H WKH
XVHUZLWKWKHEHVWVFRUHFRPSDUHGWRWKHPRGHOVLQWKH
,,, (9$/8$7,21 GDWDEDVH ZLWK D SUREDELOLW\ RI  SHUFHQW 7KLV UHVXOW
7KHPRELOHZHEGHPRQVWUDWLRQDSSOLFDWLRQZDVWHVWHG LQGLFDWHV WKDW WKH V\VWHP LV FDSDEOH RI UHFRJQL]LQJ DQ
RQGLIIHUHQWVXEMHFWVLQWKHDJHJURXSRIWR\HDUV LQGLYLGXDO SHUVRQ EDVHG RQ WKHLU XQLTXH NH\VWURNH
(DFKVXEMHFWWHVWHGWKHDSSRQRQHDQGWKHVDPHGHYLFH G\QDPLFV ZLWK UHODWLYHO\ KLJK DFFXUDF\ 7KHSUREDELOLW\
D'HOO/DWLWXGHODSWRSZLWKDEXLOWLQNH\ERDUG(DFK RILGHQWLILFDWLRQFRXOGKDYHEHHQKLJKHULIWKHVXEMHFWVKDG
6XEMHFW ZDV LQVWUXFWHG WR W\SH DV QDWXUDOO\ DV SRVVLEOH VWRUHG PRUH GDWD LQ WKH GDWDEDVH RU W\SHG WKH LQWHQGHG
ZKLOH ZULWLQJ D PLQLPXP RI WHQ ZRUGV $ WRWDO RI  ZRUGPRUHWLPHV 
LGHQWLILFDWLRQVHVVLRQVZHUHFROOHFWHG 5DQN1SUREDELOLW\ 7DEOH  FRQWDLQV WKUHH FROXPQV 7KH ILUVW FROXPQ
DFKLHYHG EHFDXVHVRPHSHRSOHWULHGLWPXOWLSOHWLPHVIRU LQGLFDWHVWKHSHUVRQ VILUVWQDPHZLWKDJHRUWKHILUVWOHWWHU
WKHIROORZLQJUHDVRQV7KHVXEMHFWDIWHUZULWLQJGRZQ RI WKH VXUQDPH LI WKH QDPHV DUH UHSHDWHG 7KH VHFRQG
ZRUGVZDVQRWIRXQGLQWKHILUVWSODFHDQGWKXVHQWHUHG FROXPQOLVWVWKHORFDWLRQLQWKHUHVXOWLQJWDEOHDQGWKHODVW
DQGVWRUHGWKHQH[WZRUGVZKHUHWKHUHZDVDOUHDG\D FROXPQ OLVWV WKH QXPEHU RI ZRUGV ZULWWHQ EHIRUH
PDWFK$OWHUQDWLYHO\SHRSOHWULHGWKHDSSOLFDWLRQPXOWLSOH LGHQWLILFDWLRQ
WLPHVEHFDXVHDPDWFKZDVIRXQGWKHILUVWWLPHLQWKHILUVW %DVHGRQ7DEOHZHFDQDUJXHWKDWLIWKHVXEMHFWW\SHV
SODFHDQGWKH\ZDQWHGWRWU\LWDJDLQ PRUHWKDQZRUGVWKHDSSOLFDWLRQLVDEOHWRUHFRJQL]H
7RHYDOXDWHWKHVXFFHVVRIWKHLGHQWLILFDWLRQZDVFUHDWHG WKHVXEMHFWDW5DQN,QRUGHUWRLQFUHDVHWKLVSUREDELOLW\
D&0&FXUYHGHSLFWHGLQ)LJ ZHQHHGWRZULWHDQGVWRUHPRUHZRUGVLQWKHGDWDEDVH,I

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 168

WKHVXEMHFWKDVZULWWHQOHVVWKDQZRUGVLQWKHGDWDEDVH GDWDVHWZKHUHWKHGDWDZRXOGEHFROOHFWHGRYHUDORQJHU
WKHV\VWHP LV QRORQJHUDEOHWRLGHQWLI\ KLPRQ 5DQN WLPH LQWHUYDO ,W ZRXOG DOVR EH LQWHUHVWLQJ WR LQFOXGH
)URP)LJXUHLWFDQEHVHHQWKDWRIWKHVXEMHFWV GLIIHUHQW VFHQDULRV DQG FRQGLWLRQV LQ ZKLFK VXFK WHVWLQJ
ZHUH FRUUHFWO\ LGHQWLILHG VR WKH V\VWHP ZDV DEOH WR FRXOG\LHOGEHWWHUDQGPRUHDFFXUDWHUHVXOWV 
LGHQWLI\VXEMHFWVRXWRIWHVWLQJRQ5DQNQXPEHU 7KHDSSOLFDWLRQFRXOGEHLPSURYHGZLWKDIXQFWLRQRI
723 7KH5DQNSUREDELOLW\ZDVDQGIRU WKH GHOHWLQJ SHUVRQDO GDWD IURP WKH GDWDEDVH DIWHU XVHU
5DQN  7KLVUHVXOWPD\EHGXHWRYDULRXVIDFWRUVVXFK UHTXHVW RU XVH GLIIHUHQW PRGHUQ PHWKRGV RI PDFKLQH
DVHUURUVLQGDWDFDSWXUHYHU\VLPLODUNH\VWURNHG\QDPLFV OHDUQLQJ WR LPSURYH WKH LGHQWLILFDWLRQ UHVXOWV ,Q D ODWHU
EHWZHHQGLIIHUHQWVXEMHFWVRUVXEMHFWVKDGDVPDOOQXPEHU VWDJH ZH EHOLHYH WKH DSSOLFDWLRQ ZLOO EH FDSDEOH RI
RIGDWDHQWHUHGIRUFRPSDULVRQ UHFRJQL]LQJWKHXVHUW\SLQJDIUHHWH[WGXULQJWKHUHPRWH
HYDOXDWLRQLQWKHHGXFDWLRQDOSODWIRUPGXULQJH[DPV>@
7$%/($6$03/(2)7+(86(567$7,67,&62%7$,1('
'85,1*(;3(5,0(176 $&.12:/('*0(17
7KLVZRUNZDVVXSSRUWHGE\WKH6ORYDN5HVHDUFKDQG
'HYHORSPHQW $JHQF\ XQGHU *UDQWV $399
$3996.7: $399 DQG WKH
6FLHQWLILF *UDQW $JHQF\ RI WKH 0LQLVWU\ RI (GXFDWLRQ
6FLHQFH5HVHDUFKDQG6SRUWRIWKH6ORYDN5HSXEOLFDQG
WKH 6ORYDN $FDGHP\ RI 6FLHQFHV XQGHU *UDQW 9(*$


5()(5(1&(6
>@ 1 /LWD\HP % 'KXSLD  6 5XEDE   5HYLHZ RI
FURVVSODWIRUPV IRU PRELOH OHDUQLQJ DSSOLFDWLRQ
GHYHORSPHQW,QW-RXUQDORI$GYDQFHG&RPSXWHU6FLHQFH
DQG$SSOLFDWLRQV   
>@ $. -DLQ HW DO   $Q LQWURGXFWLRQ WR ELRPHWULF
 UHFRJQLWLRQ,(((7UDQV&LUFXLWV6\VW9LGHR7HFKQRO  

,9 &21&/86,216 >@ 05DM 70DKDMDQ  6WXG\RI2SLQLRQV5HODWHGWR
7KH SURSRVHG VROXWLRQ EDVHG RQ DSSURSULDWH WKH8VHRI%LR(QDEOHG$WWHQGDQFH6\VWHPIRU,PSURYLQJ
WHFKQRORJLHVZLWKUHVSHFWWRDYDLODEOHKDUGZDUHVROXWLRQV 3XQFWXDOLW\$PRQJ7HDFKHUV0,(5-RXUQDORI(GXFDWLRQDO
ZDVYHULILHGLQH[SHULPHQWDORSHUDWLRQRQDVDPSOHRIDW 6WXGLHV7UHQGVDQG3UDFWLFHV
OHDVWXVHUV7KHUHVXOWVVKRZWKDWWKHELRPHWULFV\VWHP >@ +7 'DYHQSRUW  -& %HFN   7KH DWWHQWLRQ
SHUIRUPHG YHU\ ZHOO RQ WKH &0& FXUYH ZLWK YDOXHV IRU HFRQRP\8ELTXLW\ 0D\ HV
5DQNRI2XUH[SORUDWRU\VWXG\VKRZVWKDWKDOI >@ 6 0RQGDO  3 %RXUV   $ VWXG\ RQ FRQWLQXRXV
RI WKH UHVSRQGHQWV DUH IDPLOLDU ZLWK WKH ELRPHWULF DXWKHQWLFDWLRQXVLQJDFRPELQDWLRQRINH\VWURNHDQGPRXVH
NH\VWURNH G\QDPLFV PHWKRG DQG PRVW RI WKHP FRQVLGHU ELRPHWULFV1HXURFRPSXWLQJSS
WKHPHWKRGDVDVHFXUHVROXWLRQ7KHUHVXOWVREWDLQHGDUH >@ / 9RNRURNRV HW DO 6HFXULW\ RI 'LVWULEXWHG ,QWUXVLRQ
YHU\HQFRXUDJLQJDQGVKRZWKDWELRPHWULFV\VWHPVEDVHG 'HWHFWLRQ6\VWHP%DVHGRQ0XOWLVHQVRU)XVLRQWK
RQNH\VWURNHG\QDPLFVKDYHWKHSRWHQWLDOWREHDVHFXUH ,QWHUQDWLRQDO6\PSRVLXPRQ$SSOLHG0DFKLQH,QWHOOLJHQFH
DQG HIIHFWLYH VROXWLRQ IRU XVHU LGHQWLILFDWLRQ LQ YDULRXV DQG ,QIRUPDWLFV +HUODQ\  SS  GRL
VHFWRUVVXFKDVKHDOWKFDUH>@GLVWDQWHGXFDWLRQSXEOLF 6$0,
DGPLQLVWUDWLRQDQGLQGXVWU\ >@ 36 7HK $%- 7HRK  6 <XH   $ VXUYH\ RI
,QWKHIXWXUHLW ZRXOGEHLQWHUHVWLQJWRWHVWNH\VWURNH NH\VWURNH G\QDPLFV ELRPHWULFV 7KH 6FLHQWLILF :RUOG
G\QDPLFVRQDODUJHUVDPSOHRIUHVSRQGHQWVZLWKDODUJHU -RXUQDO

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 169

>@ %DXHU0  8VLQJ0HWULFVDQG.H\VWURNH'\QDPLFV
LQ*DPHVWRPHDVXUH3OD\HU([SHULHQFH
>@ 0 3OHYD HW DO   ,PSURYLQJ VWDWLF DXGLR NH\VWURNH
DQDO\VLV E\ VFRUH IXVLRQ RI DFRXVWLF DQG WLPLQJ GDWD
0XOWLPHG 7RROV $SSO  ± GRL
V
>@.6.LOORXUK\DQG5$0D[LRQ&RPSDULQJDQRPDO\
GHWHFWLRQ DOJRULWKPV IRU NH\VWURNH G\QDPLFV 
,(((,),3 ,QWHUQDWLRQDO &RQIHUHQFH RQ 'HSHQGDEOH
6\VWHPV  1HWZRUNV /LVERQ  SS  GRL
'61
>@6 2QGiã HW DO  3HGLDWULF VSHHFK DXGLRPHWU\ ZHE
DSSOLFDWLRQIRUKHDULQJGHWHFWLRQLQWKHKRPHHQYLURQPHQW
(OHFWURQLFV  SGRLHOHFWURQLFV
>@1$JDUZDO1)'DQLHOVHQ3.*UDYGDODQG3%RXUV
&RQWUDFW &KHDW 'HWHFWLRQ XVLQJ %LRPHWULF .H\VWURNH
'\QDPLFV  WK ,QWHUQDWLRQDO &RQIHUHQFH RQ
(PHUJLQJ H/HDUQLQJ 7HFKQRORJLHV DQG $SSOLFDWLRQV
,&(7$ 6WDU\6PRNRYHF6ORYDNLDSSGRL
,&(7$

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 170

 8VHU5HFRJQLWLRQLQ0RELOH$SSOLFDWLRQV 
8VLQJ)LQJHUSULQW6HQVRUV
5HQiW+$/8â.$ (ULNâ9(1. 0DW~ã3/(9$ 6WDQLVODY21'Èâ 0LQJ+VLDQ68 <XDQ)X/,$2 
'HSDUWPHQWRI(OHFWURQLFVDQG0XOWLPHGLD&RPPXQLFDWLRQV 
)DFXOW\RI(OHFWULFDO(QJLQHHULQJDQG,QIRUPDWLFV 
7HFKQLFDO8QLYHUVLW\RI.RãLFH/HWQi6ORYDN5HSXEOLF
(PDLOUHQDWKDOXVND#WXNHVNHULNVYHQN#VWXGHQWWXNHVNPDWXVSOHYD#WXNHVNVWDQLVODYRQGDV#WXNHVN 
7HO
6RRFKRZ8QLYHUVLW\7DLZDQ
(PDLOPLQJKVLDQJVX#JPVFXHGXWZ
1DWLRQDO<DQJ0LQJ&KLDR7XQJ8QLYHUVLW\7DLZDQ
(PDLO\IOLDR#Q\FXHGXWZ  
7HOH[W

$EVWUDFW ² :LWK WKH ZLGHVSUHDG DGRSWLRQ RI VPDUWSKRQHV FRPSDULVRQLVUHTXLUHGLQWKLVFDVH
DQG WKH LQFUHDVLQJ QHHG IRU VHFXUH DQG SHUVRQDOL]HG XVHU $ W\SLFDO ELRPHWULF LGHQWLILFDWLRQ V\VWHP RSHUDWHV LQ
H[SHULHQFHVXVHUUHFRJQLWLRQKDVEHFRPHDFUXFLDODVSHFWRI WZRSKDVHVLHDQHQUROPHQWSKDVHDQGDQLGHQWLILFDWLRQ
PRELOH DSSOLFDWLRQ GHYHORSPHQW ,Q UHPRWH DXGLRPHWU\ SKDVH 'XULQJ WKH HQUROPHQW SKDVH D GLJLWDO
PRELOH DSSOLFDWLRQV LW LV UHTXHVWHG WR KDYH D TXLFN XVHU UHSUHVHQWDWLRQ RI WKH ELRPHWULF GDWD LV REWDLQHG E\
LGHQWLILFDWLRQ RU DXWKHQWLFDWLRQ IHDWXUH IRU SURWHFWLQJ WKH VFDQQLQJ WKH ELRPHWULF FKDUDFWHULVWLFV ZLWK D VSHFLDO
PHGLFDO SULYDWH GDWD )LQJHUSULQW VHQVRUV QRZ FRPPRQO\ ELRPHWULFVFDQQHU>@7KHGLJLWDOUHSUHVHQWDWLRQLVIXUWKHU
LQWHJUDWHG LQWR PRELOH GHYLFHV RIIHU D FRQYHQLHQW DQG SURFHVVHGWRREWDLQDQH[SUHVVLYHUHSUHVHQWDWLRQFDOOHGD
UHOLDEOH ELRPHWULF DXWKHQWLFDWLRQ PHWKRG 7KLV SDSHU WHPSODWH 'XULQJ WKH LGHQWLILFDWLRQ SKDVH WKH VDPH
H[SORUHVXVLQJIXOOVL]HGH[WHUQDOILQJHUSULQWVHQVRUVIRUXVHU SURFHVV LV UHSHDWHG WR FUHDWH WKH WHPSODWH DQG WKHQ
UHFRJQLWLRQ LQ PRELOH DSSOLFDWLRQV :H GLVFXVV WKH FRPSDUHG ZLWK WKH WHPSODWHV LQ WKH GDWDEDVH XVLQJ D
XQGHUO\LQJ WHFKQRORJ\ RI ILQJHUSULQW VHQVRUV WKHLU PDWFKHUWRGHWHUPLQHWKHLGHQWLW\>@$QLGHDOELRPHWULF
LQWHJUDWLRQLQWRPRELOHGHYLFHVDQGWKHYDULRXVDOJRULWKPV VKRXOG KDYH WKH SURSHUWLHV RI XQLYHUVDOLW\ XQLTXHQHVV
DQGWHFKQLTXHVXVHGIRUXVHUUHFRJQLWLRQ:HDOVRSUHVHQWD VWDELOLW\ FROOHFWDELOLW\ DQG DFFHSWDELOLW\ >@ $ JHQHUDO
FDVH VWXG\ ZKHUH ZH GHYHORS D PRELOH DSSOLFDWLRQ WKDW ELRPHWULFV\VWHPFRQWDLQVIRXUPRGXOHVVXFKDVVHQVRUV
XWLOL]HV ILQJHUSULQW VHQVRUV IRU XVHU DXWKHQWLFDWLRQ DQG TXDOLW\ DVVHVVPHQW DQG IHDWXUH H[WUDFWLRQ PRGXOH
GLVFXVVWKHSHUIRUPDQFHDQGXVDELOLW\RIWKHV\VWHP PDWFKLQJDQGGHFLVLRQPRGXOHDQGGDWDEDVHPRGXOH
.H\ZRUGV ² PRELOH DSSOLFDWLRQ XVHU LGHQWLILFDWLRQ
ILQJHUSULQWELRPHWULFVXVDELOLW\ ,, 02%,/($33$1'&203$5,1*(;7(51$/ 
),1*(535,176(16256 
, ,1752'8&7,21
7KHSURSRVHGDSSOLFDWLRQKDVPXOWLSOHXVHVLQELRPHWULF
:H FRQVLGHU ELRPHWULF LGHQWLILFDWLRQ IRU WZR SXUSRVHV VHFXULW\ EHFDXVH LW FDQ SURFHVV PXOWLSOH ELRPHWULF
LGHQWLILFDWLRQ DQG YHULILFDWLRQ ,Q LGHQWLILFDWLRQ V\VWHPV PRGDOLWLHVQDPHO\ILQJHUSULQWVIDFHLULVDQGYRLFH:H
WKHLGHQWLW\RIDSHUVRQLVGHWHUPLQHGIURPDGDWDEDVHRI PRGLILHGWKH1HXURWHFKQRORJ\PXOWLPRGDO0HJD0DWFKHU
NQRZQ UHFRUGV >@ $ SHUVRQ PDNHV QR FODLP DERXW 6'. DSSOLFDWLRQ WULDO
LGHQWLW\DQGELRPHWULFLQIRUPDWLRQLVVHDUFKHGLQWKHHQWLUH KWWSVZZZQHXURWHFKQRORJ\FRPGRZQORDGKWPO  IRU
GDWDEDVHRIWHPSODWHV,QYHULILFDWLRQV\VWHPVDSHUVRQ V RXU HGXFDWLRQDO XVDJH >@ 2XU RSWLRQV IRU QDYLJDWLQJ
ELRPHWULF WHPSODWH LV FRPSDUHG DJDLQVW WKH WHPSODWH WKURXJKWKHDSSOLFDWLRQDUHVKRZQLQ)LJ7KHXVHUKDV
VWRUHG LQ WKH LGHQWLW\ GDWDEDVH >@ 2QO\ D RQHWRRQH ILYH RSWLRQV WR FKRRVH IURP LQ WKH PDLQ VFUHHQ RI WKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 171

DSSOLFDWLRQ$EXWWRQWKDWLVORFDWHGRQWKHKRPHVFUHHQLV ,,, &2//(&7,1*7(679$/8(6
WKH1HZEXWWRQZKLFKZLOOFOHDUWKHORDGHGVDPSOHVDQG 7KHFROOHFWLRQRIWHVWYDOXHVLVGLYLGHGLQWRWZREDVLF
WKHHQWHUHG,GHQWLILFDWLRQ ,' DQGDOORZXVWRORDGDQRWKHU SDUWVQDPHO\
XVHU7KHUHDUHDOVRRSWLRQVRQWKHPDLQVFUHHQWRPHDVXUH ‡ (QUROOLQJILQJHUSULQWVLQWRWKHGDWDEDVH
WKUHHRWKHUELRPHWULFV7KH*UDGOHWRROLVXVHGWRFUHDWHD ‡ &RPSDULVRQRIWKHILQJHUSULQWVZLWKWKHVDPSOHVLQ
QHZPRGLILHGYHUVLRQRIWKHDSSOLFDWLRQ  WKHGDWDEDVH
:HKDGILYHGLIIHUHQWILQJHUSULQWVHQVRUVDYDLODEOHIRU 
WHVWLQJDQGFRPSDWLEOHZLWKWKHLQFOXGHGGULYHUV 
‡ %LR0LQL6OLP6
‡ *UHHQ%LW'DFW\,'
‡ 'LJLWDO3HUVRQD8DUH8
‡ (LNRQ7RXFK
‡ (LNRQ7RXFK

2IWKHVHVHQVRUVWKHILUVWWKUHHDUHRSWLFDOVHQVRUVDQG
WKH UHPDLQLQJ WZR DUH FDSDFLWLYH VHQVRUV 7R PDNH WKH
FRPSDULVRQHDVLHUDQGFOHDUHUWZRVWHSVQHHGWREHGRQH
WRKHOSZLWKWKHPHDVXUHPHQWDQGHYDOXDWLRQRIWKHUHVXOWV
$VWKHILUVWVWHSZKHQPHDVXULQJZLWKPXOWLSOHVHQVRUVRQ
PXOWLSOH ILQJHUV LW LV QHFHVVDU\ WR GHWHUPLQH WKH
DEEUHYLDWLRQV IRU WKH ILQJHUV VR WKDW ZH FDQ GLVWLQJXLVK
ZKLFK ILQJHU LV ZKLFK 6LQFH ZH KDYH ILYH VHQVRUV WR )LJ)LQJHUODEHOVIRULQGLYLGXDOVHQVRUV
PHDVXUH ZH ZLOO DOVR FKRRVH ILYH ILQJHUV WR WDNH WKHLU 
ILQJHUSULQWV )LJ  2QO\WZRRUPRUHFDSWXUHVFRXOGEHFRPSDUHG,QWKLV
VWHSZHDFKLHYHWKDWZHKDYHWKHILUVWFDSWXUHLQRXUFDVH
WKH VDPSOH LQ WKH GDWDEDVH 7KH JLYHQ SURFHVV FDQ EH
DFKLHYHGE\WDNLQJRQHRIWKHVHQVRUVDQGIRUDSDUWLFXODU
ILQJHUWKDWLVDVVRFLDWHGZLWKDJLYHQH[WHUQDOILQJHUSULQW
VHQVRUWKDWZHHQUROOLWWRWKHGDWDEDVH
7KH QH[W VWHS ZKLFK FDQ EH FRQVLGHUHG DV WKH PRVW
LPSRUWDQW LV WKH DFWXDO FRPSDULVRQ RI WKH ILQJHUSULQWV
6LQFH ZH DOUHDG\ KDYH WKH EDVLF ILQJHUSULQWV HQUROOHG LQ
RXUGDWDEDVHHYHU\WKLQJLVUHDG\ IRUWKLVQH[WVWHS7KH
SURFHGXUH LQ WKLV VWHS ZLOO EH DOPRVW WKH VDPH DV LQ WKH
SUHYLRXVVWHSEXWZHZLOOQRWVWRUHWKHILQJHUSULQWVDPSOH
LQWKHGDWDEDVH:HZLOOFRPSDUHLWZLWKWKHVDPSOHWKDW
KDV DOUHDG\ EHHQ GDWDEDVH KDV EHHQ XSORDGHG LQ WKH
SUHYLRXVVWHS:HFDQFRPSDUHLWLQRXUPRELOHDSSDIWHU
WDNLQJWKHILQJHUSULQWXVLQJWKH,GHQWLI\EXWWRQ7KHDSS
ZLOOGLVSOD\XVDUHSRUWZKHUHWKHHVVHQWLDOLQIRUPDWLRQIRU
XVLVZKRZHPDWFKHGZLWKZKHWKHUWKH,'LVRXUVDQGWKH
YDOXHRIWKHPDWFKLQJVFRUHZKLFKLVWKHPRVWLPSRUWDQW
IRURXUPHDVXUHPHQWV7KHQDIWHUWHVWLQJZHZULWHHDFK
PHDVXUHGYDOXHLQWKHFRUUHFWFHOOLQWKHWDEOHDVVKRZQLQ
7DEOH 
)LJ0DLQYLHZRIWKHPRGLILHGPRELOHDSSOLFDWLRQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 172

 7$%/(   7DEOHZLWKPHDVXUHGVFRUHYDOXHV SULPDULO\IRFXVRQWKHVHIHDWXUHVWRILQGRXWZKLFKVHQVRU
%   ( ( ( 3, 3/ LV WKH EHVW DQG WR FRPSDUH WZR ILQJHUSULQW VHQVLQJ
ŝŽDŝŶŝ^ůŝŵϮ^ ,.: &7, ,9. &8. &.+ WHFKQRORJLHVZKLFKDUHRSWLFDODQGFDSDFLWLYHILQJHUSULQW
'ƌĞĞŶŝƚĂĐƚLJ/ϮϬ &,7 ,.: ,97 ,+& ,+& VHQVLQJ
h͘ĂƌĞ͘hϰϱϬϬ &,& /8+ ,&: &9/ &,:
$ 0HDVXUHPHQWZLWKWKH/RZHVW$YHUDJH0DWFKLQJ
ŝŬŽŶdŽƵĐŚϱϭϬ &77 &79 ,./ &7, &99

ŝŬŽŶdŽƵĐŚϳϭϬ &98 &67 &96 ,:/ ,/&

6FRUH

 2QH RI WKH LQWHUHVWLQJ YDOXHV LV WKH ORZHVW DYHUDJH

7KHILQDOPDWFKLQJVFRUHYDOXHVDUHWKHPRVWLPSRUWDQW PDWFKLQJVFRUH7KHDYHUDJHPDWFKLQJVFRUHRIWKHZKROH
IRURXUZRUNEHFDXVHZHFDQILQGRXWWKHUHOLDELOLW\DQG WDEOHZDVFDOFXODWHGE\DGGLQJDOOWKHYDOXHVDQGGLYLGLQJ
TXDOLW\RIWKHVHQVLQJ7KH\HOORZILHOGVPHDQWKDWDJLYHQ E\WKHQXPEHURIPHDVXUHPHQWV7KHORZHVWDYHUDJHYDOXH
ILQJHUKDVEHHQHQUROOHGLQWKHGDWDEDVHE\DJLYHQVHQVRU IRUWKHHQWLUHWDEOHZDVDQGWKHUHZHUHWKUHHRXWOLHU
7KHWDEOHFRQWDLQVDWRWDORIPHDVXUHGYDOXHVVLQFHZH YDOXHVLQWKHHQWLUHWDEOH6XFKDVPDOODYHUDJHPDWFKLQJ
KDYH ILYH H[WHUQDO ILQJHUSULQW VHQVRUV DQG ZH WRRN ILYH VFRUH PHDQV WKDW D JLYHQ WHVW VXEMHFW KDV UHODWLYHO\ IHZ
PHDVXUHPHQWVXVLQJHDFKH[WHUQDOILQJHUSULQWVHQVRU PDUNHUVRQKLVRUKHUILQJHUV 7DEOH 
 
7KHYDOXHVLQWKLVWDEOHFDQEHIRUH[DPSOH 7$%/(7DEOHZLWKWKHORZHVWDYHUDJHPDWFKLQJVFRUH  

‡ 0LQLPXPYDOXH  %   ( ( ( 3, 3/

‡ 0D[LPXPYDOXH  ŝŽDŝŶŝ^ůŝŵϮ^ 6& /78 ,+7 /,/ ,&+

'ƌĞĞŶŝƚĂĐƚLJ/ϮϬ
‡ 0HDQYDOXH  89 /8& ,,+ /99 &9:

h͘ĂƌĞ͘hϰϱϬϬ

/.: /&7 &.7 &+/ &76

ŝŬŽŶdŽƵĐŚϱϭϬ 99 79 /9& /7: ,6+

,Q RUGHU WR EH DEOH WR GUDZ FRQFOXVLRQV IURP WKH
ŝŬŽŶdŽƵĐŚϳϭϬ /96 &,+ /,+ &/. &+:
PHDVXUHPHQWV ZH FDQ RQO\ FRPSDUH VSHFLILF FROXPQV

ZLWK HDFK RWKHU LH WKH PHDVXUHG YDOXHV IURP RQH
VSHFLILF ILQJHU XVLQJ  GLIIHUHQW H[WHUQDO ILQJHUSULQW % 0HDVXUHPHQWZLWKWKH+LJKHVW$YHUDJH0DWFKLQJ
VHQVRUV 6FRUH
:H IRFXV RQ WKH YDOXHV LQ WKH FROXPQV WKDW ZLOO EH
$QRWKHU LQWHUHVWLQJ YDOXH LV WKH UHYHUVH RI WKH ORZHVW
VLJQLILFDQWO\ GLIIHUHQW IURP WKH RWKHU YDOXHV IRU D JLYHQ
DYHUDJHYDOXHWKHKLJKHVWDYHUDJHYDOXHRIWKHPDWFKLQJ
ILQJHUSULQW WKHVH YDOXHV DUH FDOOHG RXWOLHUV >@ 7KHVH
VFRUH WDEOH7KH DYHUDJH YDOXH RI WKH PDWFKLQJ VFRUH RI
RXWOLHUVPD\EHGXHWRPHDVXUHPHQWHUURULQFRUUHFWGDWD
WKHZKROHWDEOHZDVFDOFXODWHGE\VXPPLQJDOOWKHYDOXHV
UHFRUGLQJ RU RWKHU IDFWRUV WKDW DIIHFW WKH PHDVXUHPHQWV
DQGGLYLGLQJE\WKHQXPEHURIPHDVXUHPHQWV
>@$VDQH[DPSOHZHFDQLOOXVWUDWHWKH7DEOHRQ)LJ
7KH KLJKHVW DYHUDJH YDOXH IRU WKH ZKROH WDEOH ZDV
DQGLIZHIRFXVRQO\RQWKHVHFRQGFROXPQZHZLOOVHH
DQGLQWKHZKROHWDEOHWKHUHZHUHRXWOLHUYDOXHV
WKDWRQHRIWKHYDOXHVLVTXLWHORZFRPSDUHGWRWKHRWKHUV
6LQFHWKHUHZHUHXSWRRXWOLHUYDOXHVLQWKHWDEOHZKLFK
DQG ZH FDQ FDOO LW DQ RXWOLHU 7KH JLYHQ YDOXH ZDV
DUH VLJQLILFDQWO\ ORZHU WKDQ WKH RWKHU YDOXHV WKH WUXH
PHDVXUHG ZLWK WKH 8DUH8  VHQVRU DQG ZH DGG 
DYHUDJH YDOXH FRXOG EH HYHQ KLJKHU WKDQ RXU FDOFXODWHG
YDOXH RI RXWOLHUV WR WKH ILQDO HYDOXDWLRQ IRU WKH JLYHQ
YDOXH6XFKDKLJKDYHUDJHYDOXH PDWFKLQJVFRUH PHDQV
VHQVRU7KHVHQVRUZLWKWKHORZHVWRXWOLHUYDOXHVKRXOGEH
WKDW WKH SHUVRQ EHLQJ WHVWHG KDV PDQ\ PDUNHUV RQ WKHLU
WKHPRVWUHOLDEOHIRUSUDFWLFDOXVDJH
ILQJHUVDQGWKHSHUVRQ VILQJHUVDUHKDUGHUWRPLPLFDQG
,9 (9$/8$7,212)7+(3(5)250('7(676 IRUJH 7DEOH 

7KHSXUSRVHRIWKHH[SHULPHQWZLWKFRPSDULQJH[WHUQDO
ILQJHUSULQWVHQVRUVLVWRVHHKRZYDULRXVIDFWRUVVXFKDV
WKHVHQVLQJWHFKQRORJ\XVHGWKHTXDOLW\RIWKHVHQVRUWKH
UHOLDELOLW\ DQG DFFXUDF\ RI WKH VHQVLQJ DQG ILQJHUSULQW
UHFRJQLWLRQ DIIHFW WKH ILQJHUSULQW >@ ,Q WKLV SDSHU ZH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 173

 7$%/(7DEOHZLWKWKHKLJKHVWDYHUDJHPDWFKLQJVFRUH   ,QWKHUHVXOWLQJWDEOHZHORRNHGIRURXWOLHUYDOXHVZKLFK
%   ( ( ( 3, 3/
ZHPDUNHGDQGILQDOO\VXPPHGXSIRUHYHU\VLQJOHVHQVRU
ŝŽDŝŶŝ^ůŝŵϮ^ :/& :7/ ,&. :,6 ::7
ZKLFK OHG XV WR WKH FRQFOXVLRQ WKDW RSWLFDO VHQVRUV DUH
'ƌĞĞŶŝƚĂĐƚLJ/ϮϬ ::: :&. ,// 6,7 6:& EHWWHU WKDQ FDSDFLWLYH VHQVRUV IURP WKH XVHU H[SHULHQFH
h͘ĂƌĞ͘hϰϱϬϬ ,.. :6+ 6,6 :,6 &7: SHUVSHFWLYH DQG WKDW WKH EHVW SHUIRUPLQJ VHQVRUV LQ WKLV
ŝŬŽŶdŽƵĐŚϱϭϬ ::6 ,67 &6/ ,,7 :77 W\SHRIPHDVXUHPHQWVHWXSZHUHWKH%LR0LQL6OLP6DQG
ŝŬŽŶdŽƵĐŚϳϭϬ ::6 :.: //, :&/ .,. WKH'DFW\,'

$&.12:/('*0(17
& 0HDVXUHPHQWZLWKWKH+LJKHVW$YHUDJH0DWFKLQJ
7KLVZRUNZDVVXSSRUWHGE\WKH6ORYDN5HVHDUFKDQG
6FRUH 'HYHORSPHQW $JHQF\ XQGHU *UDQWV $399
7KH QH[W LPSRUWDQW YDOXH LV WKH KLJKHVW QXPEHU RI $3996.7: $399 DQG WKH
RXWOLHUVYDOXHV2XWOLHUYDOXHVDUHVKRZQLQWKHWDEOHDV 6FLHQWLILF *UDQW $JHQF\ RI WKH 0LQLVWU\ RI (GXFDWLRQ
WKHRUDQJHEDFNJURXQGFHOOVLQ7DEOH$WWKHVDPHWLPH 6FLHQFH5HVHDUFKDQG6SRUWRIWKH6ORYDN5HSXEOLFDQG
ZHFDQVHHWKDWWZRRIWKHILYHRXWOLHUYDOXHVDUHORFDWHG WKH 6ORYDN $FDGHP\ RI 6FLHQFHV XQGHU *UDQWV 9(*$
DWWKHSRVLWLRQRIWKHVHQVRUZLWK ZKLFKWKHVDPSOH ZDV 
HQUROOHG LQ WKH GDWDEDVH $V D UHVXOW ZH FDQ VD\ WKDW 
YDOXHV RXW RI  PHDVXUHPHQWV DUH FRQVLGHUHG WR EH 5()(5(1&(6
RXWOLHUVZKLFKLV:HZLOOXVHWKHVHYDOXHVIRUWKH >@ $.-DLQ$5RVVDQG63UDEKDNDU$QLQWURGXFWLRQWR
ILQDO HYDOXDWLRQ PHDVXUHPHQWV ZKHUH ZH HYDOXDWH WKH ELRPHWULF UHFRJQLWLRQ LQ ,((( 7UDQVDFWLRQV RQ &LUFXLWV
LPSUHFLVLRQUHOLDELOLW\DQGRWKHUSURSHUWLHV7KHVHYDOXHV DQG6\VWHPVIRU9LGHR7HFKQRORJ\YROQRSS
ZRXOG OLNHO\ EH UHPRYHG E\ UHPHDVXUHPHQW EXW LQ WKLV -DQGRL7&697
ZRUN ZH DUH FRQFHUQHG ZLWK WKHVH YDOXHV LQ WKH VW >@ ) /HKPDQQ ' %XVFKHN +HDUWEHDWV LQWKH ZLOG D ILHOG
PHDVXUHPHQW VWXG\ H[SORULQJ (&* ELRPHWULFV LQ HYHU\GD\ OLIH ,Q
 3URFHHGLQJVRIWKH&+,&RQIHUHQFHRQ+XPDQ)DFWRUV
LQ &RPSXWLQJ 6\VWHPV SS   GRL
7$%/(7DEOHZLWKWKHKLJKHVWQXPEHURIRXWOLHUVYDOXHV  

%   ( ( ( 3, 3/
>@ / 2 *RUPDQ &RPSDULQJ SDVVZRUGV WRNHQV DQG
ŝŽDŝŶŝ^ůŝŵϮ^ &6. :+8 :88 ,&8 :96
ELRPHWULFV IRU XVHU DXWKHQWLFDWLRQ LQ 3URFHHGLQJV RI WKH
'ƌĞĞŶŝƚĂĐƚLJ/ϮϬ ,7: ,.+ :79 &77 ,8+

h͘ĂƌĞ͘hϰϱϬϬ ,,, ,7& ,8: &.+ :7&

,((( YRO  QR  SS  'HF  GRL
ŝŬŽŶdŽƵĐŚϱϭϬ ,&9 &9/ :,8 &/: &9&
-352&
ŝŬŽŶdŽƵĐŚϳϭϬ &76 ,8+ ::8 &+& ,8. >@ / /X HW DO /LS 5HDGLQJ%DVHG 8VHU $XWKHQWLFDWLRQ
7KURXJK$FRXVWLF6HQVLQJRQ6PDUWSKRQHVLQ,((($&0

7UDQVDFWLRQV RQ 1HWZRUNLQJ YRO  QR  SS 
)HEGRL71(7
9 &21&/86,216
>@ 6 3UDEKDNDU 6 3DQNDQWL DQG $ . -DLQ %LRPHWULF
7KH DLP RI WKLV ZRUN ZDV WR FDUU\ RXW DQ DQDO\VLV RI
UHFRJQLWLRQ VHFXULW\ DQG SULYDF\ FRQFHUQV LQ ,(((
UHTXLUHPHQWV IRU WKH LPSOHPHQWDWLRQ RI WKH VHOHFWHG
6HFXULW\ 3ULYDF\ YROQRSS0DUFK$SULO
LGHQWLILFDWLRQ DQG YHULILFDWLRQ V\VWHP RI WKH XVHU IURP
GRL06(&3
ILQJHUSULQWVRQWKH $QGURLGSODWIRUPDQGWRSURSRVHWKH
>@ - +XUWXN 0 &KRYDQHF DQG 1 ÈGDP 7KH $UGXLQR
FRQFHSW RI LWV HQKDQFHPHQWV ZLWK HPSKDVLV RQ XVHU
SODWIRUPFRQQHFWHGWRHGXFDWLRQSURFHVV,(((VW
H[SHULHQFH DYDLODEOH KDUGZDUH VROXWLRQV DQG H[SORULQJ
,QWHUQDWLRQDO &RQIHUHQFH RQ ,QWHOOLJHQW (QJLQHHULQJ
WKHKXPDQIDFWRULQWKHXVHRIWKHV\VWHP
6\VWHPV ,1(6  /DUQDFD &\SUXV  SS  GRL
7R WHVW WKHVH VHQVRUV ZH XVHG D PHWKRG ZKHUH ZH
,1(6
PHDVXUHGWKHYDOXHVRIWKHPDWFKLQJVFRUHIRUILQJHUSULQW
>@ . &KDOOLWD + )DUKDW DQG . .KDOGL %LRPHWULF
PDWFKLQJDQGZHUHFRUGHGWKHYDOXHVLQWRDVSUHDGVKHHW
$XWKHQWLFDWLRQIRU,QWUXVLRQ'HWHFWLRQ6\VWHPV)LUVW

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 174

 ,QWHUQDWLRQDO &RQIHUHQFH RQ ,QWHJUDWHG ,QWHOOLJHQW
&RPSXWLQJ %DQJDORUH ,QGLD  SS  GRL
,&,,&
>@ % 0LOOHU 9LWDO VLJQV RI LGHQWLW\ >ELRPHWULFV@ LQ ,(((
6SHFWUXP YRO  QR  SS  )HE  GRL

>@ -:D\PDQ$-DLQ'0DOWRQL'0DLR$QLQWURGXFWLRQ
WR ELRPHWULF DXWKHQWLFDWLRQ V\VWHPV %LRPHWULF V\VWHPV
7HFKQRORJ\'HVLJQDQGSHUIRUPDQFHHYDOXDWLRQ6SULQJHU
/RQGRQSSGRLB

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 175

 English language teaching through social media
and digital tools
Tatiana Havlaskova*, Tomas Javorcik* and Katerina Kostolanyova*
* University of Ostrava, Faculty of Education, Department of Information and Communication Technologies, Ostrava,
Czech Republic
tatiana.havlaskova@osu.cz
tomas.javorcik@osu.cz
katerina.kostolanyova@osu.cz

Abstract – Media plays an important role in foreign various topics, debate and exchange experiences [4]. At
language teaching. In general it optimises learning, serves the same time social media is a public relations tool for
the function of visualisation, makes new information raising issues, influencing the opinions of specific target
accessible and facilitates the learning process. In foreign groups, building an image [5]. Therefore, we can say that
language classes it can be used when working with audio its main task is to connect groups of people and allow
recordings, presenting life and institutions or practising them to share information with each other in various
various grammatical phenomena. At the same time, it can forms.
motivate pupils to learn, speed up the memorisation process The number of social media users grows every year.
or help to hold their attention. Therefore, in addition to
Kepios’ Digital 2023 Global Overview Report [6] reveals
traditional media such as textbooks, worksheets or
that the total number of social media users worldwide has
dictionaries, it is important to include the so-called new
increased by nearly 30% since the start of the pandemic,
media – the realm of the Internet, apps and social media.
which amounts to more than 1 billion new users in the last
After all, as the most recent reports inform us, social media
itself is today a frequently used platform. In recent years,
3 years. In April 2023 there were 4.8 billion social media
communication through this type of media has expanded so
users worldwide, equivalent to 59.9% of the global
much that it has become the most important communication population. With the growing number of users, the time
tool, not only for adolescents. The aim of this paper is to we spend on social media has also increased. The typical
present the results of a questionnaire survey carried out working-age internet user spends more than 2.5 hours a
across all regions of the Czech Republic, looking at the day using social media platforms.
options of using social networks, digital tools and Ranking of social media platforms by global active user
applications in English language classes – the extent of figures (in millions) that the top four most used social
involvement, the tools, the impact on teaching or its use in platforms are Facebook (2.958 million), YouTube (2.514
personal life. million), WhatsApp and Instagram (2 million). The main
reasons users gave for using social networking sites were
Keywords: social media, digital tools, English language to keep in touch with family and friends (48.2%), to fill
teaching their free time (36.8%), to read the latest news (34.5%)
and to find content in its various forms (29.2%) [7].
I. SOCIAL MEDIA IN THE CONTEXT OF SOCIETY
Focusing specifically on the Czech Republic, 8.07
million people use social media, where they spend an
Mass media and its use is a typical feature of average of almost 2 hours of their time, with Facebook
contemporary society. It has grown from a simple tool of and Messenger being the most used platforms. In terms of
mass communication at the beginning of the last century activities carried out on social platforms, the Czech
into a powerful system, without which the functioning of a Republic does not stand out from the global average.
globalised information society is now unimaginable [1]. Users mostly follow accounts focused on entertainment
Historically, we can look at media in two ways: old media and leisure. It is certainly worth mentioning that only a
(print media, radio, television, film, recordings) and new third of users focus on education and activities related to
media, based on digital data encoding, computers their work [8].
(hardware, software), tablets, mobile phones, apps, In the infographic below (Figure 1) we can see a
computer networks (internet), social media, blogs, internet representation of the use of social media, online payments
search engines and other web services [2]. The phrase new or the use of streaming services – this is the number of
media is, however, largely problematic as its meaning these actions per minute (valid for 2022). Thanks to the
changes over time. New media that was already based on visualisation we find that in just 60 seconds, for example,
digital computer technology has been introduced and is 231.4 million e-mails were sent, 66,000 photos were taken
referred to by an alternative term, in addition to new on Instagram, 347.2 thousand tweets were sent on Twitter
media, namely network digital media [3]. or 500 hours of videos were uploaded to YouTube.
Social media is one of the internet services on which
users can create different profiles. Through these profiles
users can share videos, photos and other content. They can
form groups, discussion forums, etc., where they discuss

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 176

 The Covid pandemic has been an acid test for all areas
66
of life and education, including language learning.
231
500
347 Teachers had to adapt very quickly to teaching online
using technologies that many were unfamiliar with, and to
preparing lessons that reflected the required way of
teaching. Another concern was that such a form of

thousand

thousand
million

hours
language learning inevitably entails a decline in the
204 standard of learning. On the other hand, many teachers
100 found that distance learning offered them the opportunity
48
to experiment with a range of technologies and tools that
3,6
led to enriched learning and therefore decided to continue
Emails Emails YT YT Insta Insta Twitter Twitter
using them even after the pandemic ended.
2013 2022 2013 2022 2013 2022 2013 2022 Over a two-year period (2020 to 2022), in-depth
Figure 1. One minute of the day [9]
research was carried out [17] across EU member states
under the umbrella of the ECML (European Centre for
Modern Languages) in cooperation with the European
II. MEDIA IN FOREIGN LANGUAGE TEACHING Commission. The research involved 1,735 teachers from
According to the Czech Statistical Office [10] the 40 countries and not only examined how the pandemic
equipment in schools is improving every year, especially had affected methodology, educational objectives and
after the Covid-19 pandemic, in which many teachers assessment, but asked respondents to describe their
were forced to use information technology even if they experiences, positives, negatives, etc. Among other things,
had little experience with it themselves. the research found that nearly 41% of respondents had
appropriate equipment for effective online learning before
We will use data from 2018 and 2022 to reflect the the pandemic began, 31% said that equipment was
situation before and after the pandemic to outline the purchased in the early days of the pandemic, and the
current and recent situation. This is illustrated by the remaining 28% lacked appropriate digital equipment and
figures, where the total number of computers in Czech software.
elementary schools in 2018 was 142,700, but in 2022 it
was already 243,600 [11]. These figures not only include What was recommended in conclusion, among other
desktop computers, but also laptops and tablets. Portable things, was that it is important for language teachers to
devices account for a significantly larger share of the have digital skills and the ability to use a range of general
increase in numbers. The Information Society in Figures and specific educational software and applications, and
report [12] confirms that there has been a clear increase in that teachers should also develop their pupils’ digital skills
the number of schools allowing pupils to use their own and expose them to a wider range of activities and
devices in lessons, with an increase between 2020 and resources, including games, videos, etc.
2022 from 14.4% to 21.2% for the first stage of
elementary schools, from 25% to 35.7% for the second III. THE USE OF SOCIAL MEDIA FOR ENGLISH
stage of elementary schools and from 52.1% to 57.2% for LANGUAGE (SELF-)LEARNING
secondary schools. The issue of using digital technology in education
In 2022 the Ministry of Education, Youth and Sports always raises the question “why use it?”. The first of
has promised schools (kindergartens, elementary and many reasons is to increase pupils’ interest in learning
secondary schools) a subsidy of CZK 4.3 billion for the and the topic. With the use of digital technologies,
prevention of the digital divide – the purchase of digital various types of materials (video material, audio
teaching aids, to be drawn from the European Union’s
recordings, etc.) can be introduced into the classroom,
National Recovery Plan Fund. The purpose of this subsidy
is to develop children’s and pupils’ digital thinking and which enables teaching to be more varied and helps
digital competences and to purchase mobile digital pupils understand even complex topics. Another reason is
devices for disadvantaged pupils in order to prevent the to provide access to more information, which is today
digital divide [13]. Purchased mobile digital devices (e.g. something that we not only need to know how to work
laptops, ultrabooks, chromebooks, tablets, smartphones) with, but also how to find relevant information. As a
and accessories for these devices (e.g. webcams, mouses, result pupils also learn how to use online resources and
headphones, keyboards, etc.) and mobile digital improve their digital literacy.
technology for pupils with special educational needs will An equally important reason is to improve
be available for lending to pupils so that they can use them communication between pupils and teachers. Thanks to
for regular and, where appropriate, distance learning [14]. technology, the teacher is able to better provide feedback
Various digital technologies are increasingly being used to the pupils. Teachers, on the other hand, can get
in foreign language education, both e-learning [15] – as a interesting input from pupils on how to conduct lessons
form of education that uses modern information and and explain different topics.
communication technologies to deliver learning content, Another reason for using them is the opportunity for
to allow communication among learners and to manage
the learning process, and m-learning [16] – a specific form pupils to progress at their own pace. In the event a pupil
of e-learning that uses mobile communication is not able to attend school, e-learning courses can be
technologies and does not bind learning to a single used so that the pupil does not miss the topic taught in
location. their absence. With the help of these technologies pupils

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 177

can be more easily guided to grow in confidence and Another way to use this platform is for so-called
motivated to learn. brainstorming. By using the #hashtags feature, interesting
ideas and problem solving can be discussed with others
A. Online courses
and thus not only get quick feedback from classmates.
One way to educate yourself is by using online
courses. This is education that takes place online (on the F. YouTube
internet) and can take various forms. Some forms allow Deepening discussed topics and then making the lesson
downloading of the exercises or learning materials and more attractive by using a variety of videos and thereby
can, therefore, be used later without further access to the increasing interest in the subject. Educational videos can
internet. Some of these courses are offered free of charge, be used to explain the topic. YouTube can also be used to
but in most cases they are provided for a fee. watch, for example, various talk shows, documentaries
with English subtitles, stand-up comedy (to lighten up the
B. Podcasts
lesson) or listening to songs to practise listening and
Another way to improve your English is to use understanding the spoken word in English. Another
podcasts. The duration of a podcast is around 5–60 example of the use of this platform is the demonstration
minutes. This is an audio recording of a programme that of native speakers’ accents and dialect differences in
can be listened to on the internet. Podcasts offer a variety different localities of English-speaking countries.
of topics, be it literature, science, gaming news, cooking
tips, linguistics, etc. IV. RESEARCH STUDY

C. Social media accounts A. Description of the research

User profiles or accounts on social media such as Our main objective was to map the extent and impact of
Facebook or Instagram can be used to some extent for the use of social media, apps and digital tools in English
self-education. There are many accounts focused at language teaching. For our purposes we have evaluated
improving different parts of the English language skills. the questionnaire survey method, which is currently used
Sometimes they offer short advice or tips on how to in the field of quantitative research, as the best way to
improve your pronunciation, for example, but you can obtain data. We chose a questionnaire in electronic form.
also find instructional videos focusing on English The reasons behind the choice of electronic data collection
language issues. were obtaining a larger number of responses, time
flexibility and 100% respondent anonymity. As a tool for
D. Facebook the implementation of the survey we chose the Google
This platform offers many interesting features that can Forms platform.
be used for the benefit of English language learning. One The questionnaire was designed to best cover the
of these features is the ability to share documents using selected issue. It contained 23 questions divided into
class groups, which can easily be created separately for several sections logically linked to each other. The first
part contained functional items – questions on gender,
each grade and class. In these groups, for example,
length of experience, type of school, etc. Next came the
worksheets can be shared, both with pupils and with content items, which first focused on social media issues,
teachers in other schools who can help or advise each then on the use and engagement of digital tools, and the
other through this platform. It is also possible to share last part was dedicated to questions on working with apps.
interesting videos, which can then be discussed in The nature of the items was varied. We used closed,
English. This platform can also be used to gather tips and semi-closed and open-ended questions. Of the closed
ideas from other English teachers, for example, to make items we incorporated dichotomous, single-choice and
the lesson more interesting or to explain a topic in a multiple-choice items where respondents had
different way. Teacher groups on Facebook often gather, predetermined response alternatives. Semi-closed items
for example, interesting teaching materials or worksheets offered the respondent a choice of response, while also
that can be used for teaching English. Teachers can also having the option of an open-ended answer. Open-ended
create private groups for conversations among pupils in items gave respondents the opportunity for free response.
English. The advantage is that a new and interesting perspective on
the issue may emerge, the disadvantage is that it is more
E. X (formerly Twitter) laborious to evaluate the answers with a large sample of
X, with its limit of 280 characters per post, offers respondents [18].
various ways in which it can be used to teach English and The survey was launched in early December 2022 and
develop communication skills. One of the first ran for 3 months. A link to the prepared questionnaire was
possibilities is to search for news from abroad, translate sent to the e-mail addresses of the contacted schools
and analyse text, explain unfamiliar words. Another (elementary and secondary schools, including grammar
schools). After the data collection was completed it was
option is to send posts in English, which their classmates found that 174 respondents – English language teachers –
can then respond to. The limited number of characters in participated in the questionnaire survey.
this platform prompts pupils to think about how to
communicate accurately and concisely while maintaining
the main form of the idea. This limited writing space can
therefore be used to develop expressive skills in English.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 178

B. Analysis – functional part length of teaching experience of 10 to 20 years with 58
All the collected information was transformed from the respondents (24 elementary school teachers, 34 secondary
Google Forms web interface into Microsoft Excel, where school teachers). The second largest group were teachers
it was reworked into clear graphs. with 20 to 30 years of teaching experience with 50
respondents (13 elementary school teachers, 37 secondary
The gender ratio of the respondents (136 women and 38 school teachers). The last group of over 30 years of
men) reflects the fact that the Czech education system is experience consisted of 18 respondents (5 elementary
still dominated by women in the long term, which is also school teachers, 13 secondary school teachers).
illustrated by the publication of the Czech Statistical
Office [19], which focuses on comparing the differences
between women and men in various areas of life. The ratio
of female teachers in the regional education system over 30 years 18
depends on the grade and level of expertise – the higher
the level, the more equal the ratio of women to men in the 20 to 30 years 50
teaching staff.
10 to 20 years 58
Most teachers were from secondary schools (107),
followed by elementary school teachers (67). In terms of 5 to 10 years 18
the type of schools contacted, the largest group was
teachers who teach in a public school (139) – 52 2 to 5 years 21

elementary school teachers and 87 secondary school

under two years 9
teachers. There were significantly fewer teachers from
private schools (35) – 14 elementary school teachers and 0 10 20 30 40 50 60
21 secondary school teachers.
Figure 3. Length of teaching experience
We are pleased that we were able to obtain responses
from teachers representing all regions. The largest number
of respondents was from the Central Bohemia region,
while the smallest number of respondents was in the Plzeň C. Analysis – content part: social media
and South Moravia regions (Figure 2). The following is the content part of the questionnaire,
specifically the items focused on social media. We asked
South Moravian Region 7 about what social media they use and for what purpose.
Pilsen Region 7
Liberec Region 8
We were also interested in any difference between use in
Hradec Králové Region 8 personal life and use directly in teaching. We used semi-
Karlovy Vary Region 8 closed and open-ended items.
Olomouc Region 9
Zlín Region 11 The use of three social media sites – YouTube (140),
Moravian-Silesian Region 12 Facebook (130) and Instagram (73) – was predominant in
South Bohemian Region 12
Prague 16
personal life (Figure 4), with 20 respondents stating that
Usti Region 17 they did not use any social media sites in their personal
Vysočina Region 18
life.
Pardubice Region 18
Central Bohemian Region 23 Respondents also indicated what they use the
0 5 10 15 20 25 mentioned social media for in their personal life. The
largest group stated that they use these platforms to
Figure 2. Respondents by regions
communicate with friends or their contacts abroad.
Furthermore, respondents reported looking for and
sharing new information as inspiration for their personal
The next item, which was in the functional section, life, sharing news, inspiration for their education or for
asked about other subjects that the respondents teach at interesting information about life.
their respective schools besides the English language. 55
respondents answered that they do not teach any other A large number of respondents also reported the use of
subject (13 elementary school teachers, 42 secondary these platforms for entertainment. Some respondents also
school teachers). Of the rest of the respondents (119), 34 reported using social media to educate themselves in
of them said they were teaching other languages: Czech, various areas, such as advice on technology or improving
German, French, Russian and Spanish. 85 respondents their foreign language skills.
most often teach one or more of the following subjects:
History, Music Education, Computer Science,
Mathematics, and Physical Education.
The last data that was collected in the functional part of
the questionnaire was the length of the respondents’
teaching experience (Figure 3). There were 9 respondents
in the under two years of experience group (4 elementary
school teachers, 5 secondary school teachers). The 2 to 5
years group contained 21 respondents (10 elementary
school teachers and 11 secondary school teachers). In the
third group with 5 to 10 years of experience we found 18
respondents (8 elementary school teachers, 10 secondary
school teachers). The largest group were teachers with a

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 179

 teaching, time required for preparation or resistance to
new practices on the part of the teacher. One of the
Signal 1
Viner 2
reasons may also be the prohibition of the use of platforms
Bereal 2 other than YouTube by the school management.
Snapchat 3
Messenger 3
We were also interested in the advantages and
MS Teams 4 disadvantages in the possible integration of social media
Discord 4
into teaching. For the sake of clarity we have presented
Pinterest 5
Tiktok 6 the most frequent answers in tables (Table 1 and 2):
Twitter 17
WhatsApp 19
any social media 20
Instagram 73 TABLE I.
Facebook 130 ADVANTAGES OF THE USE OF SOCIAL MEDIA IN TEACHING
Youtube 140
0 20 40 60 80 100 120 140
Current and updated information
Interactive
Figure 4. Use of social media in personal life
Easily accessible
In terms of the use of social media in the learning
environment (Figure 5), YouTube was clearly the Materials in one place; their variety and diversity
predominant platform (143), followed by Facebook and Communication with pupils from other countries, between pupil
Instagram (16) and Teams (11) by an equal margin, with and teacher
24 respondents stating that they do not use any social Listening to real English spoken in different dialects
media in their learning environment.
Arousing interest in the subject
Respondents also indicated how they incorporate social
media into their teaching. The largest group said that they Pupils process audio-visual content better
use various teaching videos, video tutorials,
Engaging most senses
documentaries or different videos in their lessons to
enliven the lesson and engage pupils. They use videos to
expand pupils’ vocabulary, as suggestions for discussion
or as a supplement for certain topics in English. In TABLE II.
addition, respondents mentioned the use of authentic DISADVANTAGES OF THE USE OF SOCIAL MEDIA IN TEACHING
language recordings, listening to songs in English. A
smaller group of respondents specifically reported using Technical issues User difficulties
the Instagram platform to share materials and Advertisements Personal data safety
assignments, which could primarily be used by pupils who
are ill. Some of the respondents said that they use social Technical issues of equipment Time-consuming preparation
media sites such as Facebook and Microsoft Teams to Dependence on internet
Unwanted content
communicate with pupils and also to give quick feedback connection
Pupils who do not use social Necessity of basic knowledge
on assignments or material covered. Respondents also networks for use
reported using various social media groups as a source of
interesting, unusual or time-consuming materials or Distraction
worksheets that can be used in lessons. Using social media for other
than intended purposes
Ted talk 1 Social media addiction
streaming platforms 1
Quizziz 1 Loss of concentration
Pinterest 1
Netflix 1
Pupils do not want to go back
iSLCollective 1 to textbooks
Internet 1
Etwinning 1
Bakaláři 1
Messenger 2 D. Analysis – content part: digital tools and
G-suite 2
WhatsApp 3 applications
Kahoot! 3
Twitter 5
The following section of the questionnaire focuses on
MS Teams 11 digital tools and applications. We asked about what digital
Instagram 16
Facebook 16
tools and applications they incorporate into their teaching
any social media 24 and what the positives or negatives are. We used semi-
Youtube 143 closed and open-ended items.
0 20 40 60 80 100 120 140 160
In Figure 6 we can see that the largest grouping (141)
Figure 5. Use of social media in teaching was the group of respondents who use a computer in
teaching (by computer here we mean a desktop computer
The results are consistent with Kepios’ findings, where that is not portable). Respondents who use a mobile phone
Facebook, YouTube and Instagram ranked among the for teaching were the second largest group (121). The
most used social platforms. The use of these platforms is group that uses an interactive whiteboard and a laptop for
also very similar – networking, entertainment and content teaching was equal in number (114).
search. The lack of use of platforms other than YouTube
may be due to many factors, such as unsuitability for

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 180

 any digital tools 1 TABLE III.
Robot classroom 1 ADVANTAGES AND DISADVANTAGES OF INTEGRATING DIGITAL TOOLS
Robots 1 AND APPLICATIONS IN TEACHING
Camera 1
Classroom presentation tool 1 Advantages Disadvantages
E-textbook 3
Data projector 4 Demands on teacher’s time and
Creativity
Tablet 63 preparation
E-book 70
E-journals 70
Flexibility Level of effectiveness...?
Player (CD, DVD, MP3) 89
Laptop 114 Motivation Possible addiction
Interactive whiteboard 114
Adaptation of lesson to Internet access or installation
Mobile phone 121
Computer 141
teaching style sometimes required
Not suitable for practising
0 20 40 60 80 100 120 140 160 Quick feedback
certain skills – writing
Figure 6. Use of digital tools in teaching Gamification of learning

Individualisation
The results correspond with the reports of the Czech
Statistical Office, where desktop computers still dominate, Multimediality
but we can also see an increase in the number of portable
devices.
One of the last questions asked about the use of specific V. CONCLUSION
applications to support teaching (Figure 7). An app that
has become popular with many educators in recent years Social media and digital technologies are changing the
is the Kahoot! app, and not surprisingly, this is the most way we look at education as a whole, including English
used app among respondents (59). Other so-called language teaching. It is changing approaches to teaching
feedback apps such as Quizlet or Quizizz also appear in and learning a foreign language and to the role of a
the list (20, 15). A lot of respondents also use applications teacher as well. The combination of standard teaching
for creating interactive exercises – Wordwall (24) or with e-learning represents a comprehensive change in the
interactive worksheets – Liveworksheets (15). approach to foreign language teaching, especially in terms
of the combination of learning objectives and content.
ESL Kidsgames 2
The questionnaire survey shows that teachers see
itools 2 potential advantages in the inclusion of social media and
Lyricstraining 2 other digital tools in English language teaching –
Moodle 2
Umimeto.org 2 attractiveness for the learner, interactivity, instant
WhatsApp 2 feedback, infinite range of resources, multimedia content,
Wizerme 2
BBC english 3
focus on multiple senses, support for self-study, pace
Cambridge dictionary 3 control, etc. For the positives to outweigh the negatives, it
Canva 3
iSLCollective 3
is essential that these changes go hand in hand with a
Messenger 3 transformation of teaching, didactics and pedagogy.
Pinterest 3
Socrative 3 The position of the teacher still plays an important role
Spotify 3 in teaching. Their task is, among other things, to teach
OneNote 4
Padlet 4
pupils how to use these tools effectively and meaningfully
Blooket 5 and to not only engage them in school, but also in their
streaming platforms 6
other activities. In order to use and pass on the
Duolingo 7
Nearpod 7 possibilities offered by new media, teachers need to be
Wocabee 8 digitally literate and know how to use the technology and
Oxford English Bookshelf 9
Google apps 15 applications and how to implement them effectively in the
Liveworksheets 15 classroom. Only a digitally literate teacher can “create”
Quizizz 15
Quizlet 20
digitally literate pupils.
MS Teams 21 The correct use of social media and apps can enrich
Wordwall 24
Youtube 27 English language learning and support the development of
Kahoot! 59 language skills. It is important to monitor trends, respond
0 10 20 30 40 50 60 70 to them and integrate them appropriately into the learning
process to make English language teaching effective,
Figure 7. Use of apps in teaching
motivating and relevant to the needs of modern learners.

We were also interested in the advantages and REFERENCES

disadvantages of incorporating digital tools and apps into [1] J. Jirák and B. Köpplová, Masová Média, 2rd ed. Praha: Portál,
teaching. For better clarity, we have presented the most 2015.
frequent answers in a table (Table 3): [2] B. Osvaldová and J. Halada, Praktická Encyklopedie Žurnalistiky
a Marketingové Komunikace, 3rd ed. Praha: Libri, 2007.
[3] V. Moravec, Média v Tekutých Časech: Konvergence
Audiovizuálních Médií v ČR. Praha: Academia, 2016.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 181

[4] M. Kožíšek and V. Písecký, Bezpečně na Internetu. Praha: Grada, [online] Available:
2016. https://www.msmt.cz/ministerstvo/novinar/ministerstvo-skolstvi-
[5] O. Jurášková and P. Horňák, Velký Slovník Marketingových posle-skolam-penize-na-digitalizaci.
Komunikací. Praha: Grada, 2012. [14] L. Gregůrková, “Národní Plán Obnovy - Prevence Digitální
[6] We Are Social and Meltwater, “Digital 2023: Global Overview Propasti - Pořízení Mobilních Digitálních Technologií Určená pro
Report”, DataReportal, 2023, [online] Available: Soukromé a Církevní Základní Školy, Střední Školy a
https://datareportal.com/reports/digital-2023-global-overview- Konzervatoře pro Kalendářní Rok 2022”, Ministerstvo školství,
report. mládeže a tělovýchovy, 2022, [online] Available:
https://www.msmt.cz/vzdelavani/skolstvi-v-cr/narodni-plan-
[7] We Are Social and Meltwater, “Global Social Media
obnovy-porizeni-mobilnich-
Statistics”, DataReportal, 2023, [online] Available:
digitalnich?fbclid=IwAR1lYaba0gtOd215JU4OpmuUVyaCTjTKe
https://datareportal.com/social-media-users.
Ss2v6f3q8cFZ03oGVEOxc0aSg8.
[8] We Are Social and Meltwater, “Digital 2023:
Czechia”, DataReportal, 2023, [online] Available: [15] V. Maněna, Moderně s Moodlem: Jak Využít E-learning ve Svůj
https://datareportal.com/reports/digital-2023-czechia. Prospěch. Praha: CZ.NIC, z.s.p.o., 2015.
[9] Domo, Inc., “Data Never Sleeps”, Domo.com, 2022, [online] [16] J. Havlová, "M-learning", in KTD: Česká terminologická databáze
Available: https://www.domo.com/data-never-sleeps. knihovnictví a informační vědy, [online] Available:
https://aleph.nkp.cz/F/?func=direct&doc_number=000015163&lo
[10] J. Wichová, “Informační Technologie ve Školách v České cal_base=KTD.
Republice - 2021”, Český statistický úřad, 2022, [online]
[17] F. Heyworth and R. Rossner, Rethinking language education after
Available: https://www.czso.cz/csu/czso/cri/informacni-
the experience of Covid - Final report. Strasbourg: Council of
technologie-ve-skolach-v-ceske-republice-2021.
Europe Publishing, 2023, [online] Available:
[11] J. Wichová, “Informační Technologie ve Školách”, Český https://www.ecml.at/Resources/ECMLresources/tabid/277/ID/141/
statistický úřad, 2023, [online] Available: language/en-GB/Default.aspx.
https://www.czso.cz/csu/czso/informacni_technologie_ve_skolach
[18] P. Gavora, Elektronická učebnica pedagogického výskumu.
[12] Informační Společnost v Číslech: Česká Republika a Svět. Praha: Bratislava: Univerzita Komenského, 2010.
Český statistický úřad, 2023, [online] Available:
https://www.czso.cz/csu/czso/informacni-spolecnost-v-cislech- [19] Zaostřeno na Ženy a Muže - 2022. Praha: Český statistický úřad,
2023. 2022, [online] Available: https://www.czso.cz/csu/czso/zaostreno-
na-zeny-a-muze-2022.
[13] P. Kubas, “Ministerstvo Školství Pošle Školám Peníze na
Digitalizaci”, Ministerstvo školství, mládeže a tělovýchovy, 2022,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 182

 Puzzle-Driven Learning: Developing and Assessing
IT Challenges for Varied Experience Levels

Marek Horváth Emı́lia Pietriková

Department of Computers and Informatics Department of Computers and Informatics
Technical University of Košice Technical University of Košice
Košice, Slovakia Košice, Slovakia
marek.horvath@tuke.sk emilia.pietrikova@tuke.sk

Abstract—This research explores the development and applica- impact of this approach varies among individuals based on
tion of IT-related puzzles as a tool for enhancing problem-solving factors such as age and experience.
skills and stimulating interest in the Information Technology (IT) Given the rapid advancements in the IT field and the
field. It thoroughly examines the cognitive impact of solving
such puzzles and how they serve as both entertaining and growing demand for skilled professionals, enhancing problem-
educational mediums. The study involves the crafting of twelve solving skills has become more important than ever. The find-
diverse and complex puzzles, incorporating various IT disciplines, ings of this study provide valuable insights and contribute to
ensuring they are challenging, engaging, and cater to a wide the expanding knowledge base regarding the potential benefits
demographic with varying levels of experience. The aim is to of using puzzles and riddles as engaging learning tools. These
provide practical knowledge applicable to real-world IT sce-
narios while maintaining an entertaining and engaging learning tools not only aid in the development of problem-solving skills
environment. The puzzles are subsequently tested on a group of but also play a crucial role in maintaining interest and active
individuals, shedding light on their accessibility and engagement participation in the field of IT.
levels. The findings of this research are crucial, offering valuable Moreover, the insights gained from this research have the
insights into the role of puzzles as innovative learning tools. potential to inform educational strategies and training pro-
They inform educational strategies and professional development
programs, highlighting the potential of such interactive tools in grams, shaping the way we learn and refine skills in IT. By
the comprehensive development of individuals in the IT sector showcasing the versatility and utility of puzzles and riddles,
and fostering a sustained interest in the field. the study aims to foster increased adoption and application
Index Terms—IT Problem-Solving, Puzzle Design, Interactive of these tools in both educational and professional settings,
Learning, Educational Tool, Skill Enhancement supporting the continuous development of individuals in the
IT field.
I. I NTRODUCTION
II. T HE C OGNITIVE I MPACT OF P UZZLES
In recent times, there’s been a noticeable increase in the Engaging in puzzles and brain teasers activates the brain and
interest in using puzzles and riddles to enhance problem- is crucial in fostering cognitive development. These activities
solving skills, particularly in the field of Information Tech- are renowned for their ability to mitigate the effects of aging
nology (IT). These engaging challenges are more than just on memory and perception [1]. Nevertheless, the extent to
entertainment; they serve as a practical means for individuals which puzzles contribute to enhancing our cognitive abilities
to sharpen their skills, regardless of their level of experience remains a topic of ongoing conversation. Tackling puzzles
in the discipline. By combining creativity with logic, puzzles strengthens vital brain functions like visual-spatial skills and
and riddles promote a culture of curiosity and critical thinking, information synthesis, which are valuable across diverse fields
which are essential for effective problem-solving in IT. such as education and cognitive therapy [2]. Taking on these
These playful yet challenging activities stimulate the brain challenges helps build a willingness to take risks, which is
in different ways, introducing a refreshing approach to skill de- really important for learning, especially in respected schools
velopment in the IT sector. This article introduces an in-depth where being trusted and believed in is key. With changes
study focused on examining the varied effects and overall in teaching methods and more teaching assistants helping
effectiveness of using puzzles and riddles to improve problem- out [1], it’s really important to look for real improvements
solving skills in IT. The investigation seeks to understand the in how we teach and learn. The Prospect theory suggests
correlation between an individual’s problem-solving ability, that individuals are more inclined to take risks when they
professional experience, and the time invested in tackling chal- perceive potential gains, underscoring the importance of this
lenging problems. Additionally, the study explores whether the mindset in educational contexts [2]. Focusing on making

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 183

strong connections in schools, through education that values
relationships, shows how important it is to build friendships
for helping our brains grow [3].
Furthermore, puzzles serve as a practical tool for educators
and learners to explore new learning strategies and cognitive
processes. They can be particularly beneficial in academic en-
vironments that are continually seeking innovative approaches
to enhance learning outcomes. By integrating puzzles into the
learning experience, educators can facilitate a more interactive
and stimulating learning environment, encouraging students to
think critically and solve problems effectively. This integration
not only supports academic achievement but also contributes to
the development of life-long learning skills. Additionally, the
exploration of the impact of puzzles on cognitive development
provides a foundation for future research and discussions in
this area. By looking more into how puzzles work on the
brain, people who study and teach can find new ways to
use this knowledge in different places where we learn. This
continuous exploration and understanding of the human brain
and its response to puzzles create avenues for developing more
effective and personalized learning experiences, catering to the
diverse needs of learners. Fig. 1. Vigenère Square - A tool used for encrypting and decrypting text
using a simple form of polyalphabetic substitution.
In conclusion, while the debate on the full extent of the
benefits of puzzles continues, their role in enhancing vari-
ous cognitive functions and fostering a risk-taking mindset Also, giving clear instructions and helpful feedback is key to
is undeniable. The evolving educational landscape and the making sure that solvers understand the problem and learn
insights from theories and research in this field underline the from their mistakes, creating a good environment for learning
potential of puzzles as valuable tools for learning and cognitive new skills.
development. The ongoing exploration of these activities and
their impact offer promising prospects for furthering our un- IV. D EVELOPING D IVERSE AND F UN T ECH -C ENTRIC
derstanding of the brain and enriching the learning experience P UZZLES
in education. Creating puzzles is all about diving into different parts of
information technology, making sure there’s a cool and varied
III. C RAFTING P UZZLES FOR E FFECTIVE P ROBLEM mix for everyone. They come in all shapes and sizes – from
S OLVING questions and tasks to brain teasers. Take one tricky puzzle,
In a time where everyone is looking for quick and efficient for example, where you have to figure out a sentence where
solutions, being able to solve problems is a must-have skill. each word is hidden within its own puzzle. This requires
Solving a problem means figuring out what the issue is, combining knowledge of codes and creative thinking. The first
coming up with a way to fix it, and then making that solution word is like a puzzle within a puzzle – starting with an XOR
work. So, finding new and creative ways to learn these skills encryption key and a ciphertext, then moving on to Morse
is super important, and that’s where solving puzzles comes in code and a Karnaugh map to show a Boolean function.
as a excellent strategy. The next word is all about using an Adjacency Matrix –
Creating puzzles means thinking carefully about a bunch a basic idea in graph theory. A little hint, “xirtam ycneca-
of things, like how hard they are, what skills are needed to jdA” (which is “Adjacency Matrix” backward), helps guide
solve them, and what the main goal of the puzzle is. Logic the solver. Solving this is a hands-on way to get better at
puzzles, which make you use deductive reasoning and critical problem-solving, critical thinking, and really understanding
thinking, are especially good tools for building problem- graph theory.
solving skills. Some well-known ones are Sudoku, Kakuro, The third puzzle steps into image processing, asking the
and Nonograms. On the other hand, word puzzles, which need solver to spot the differences between two images by looking
a good vocabulary and recognizing patterns, are also great for closely at each pixel and noting the different color values.
helping the brain grow. This group has crossword puzzles, The result is a unique image highlighting the different areas
word searches, and anagrams. – a technique often used in computer vision. How tough this
Thinking about who will be solving the puzzles and how puzzle is depends on how complex the images are and how
experienced they are is crucial when making them. Puzzles different they are from each other.
that are too easy might not be interesting, while super hard Another puzzle, called “Where where true equals nothing,”
ones might just make people frustrated and less motivated. is inspired by the quirks of SQL (Structured Query Language)

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 184

 Fig. 2. Puzzle Adjacency Matrix
Fig. 4. Four images showcasing different aspects of the Golden Ratio or
Fibonacci Sequence for analytical comparison.

VI. D EVELOPING A U SER -C ENTRIC P UZZLE P LATFORM

Selecting an appropriate tool to present our puzzles is
critically important. The ideal platform should be functional,
versatile, compatible with various devices, and above all, user-
friendly, allowing us to bring our innovative concepts to life
effectively. A website emerges as the ideal candidate, ticking
Fig. 3. Puzzle Illustrating Difference Between Two Images all these boxes and facilitating the creation, resolution, and
sharing of puzzles seamlessly. Thus, choosing a tool that aligns
with these requirements is fundamental to the success of a
programming. It’s a reminder to developers about keeping delightful puzzle platform.
code clean and easy to read. There are more puzzles mixing
Our design philosophy centers on clarity, user-friendliness,
different subjects, like one matching important dates with
and aesthetic appeal. The platform has been carefully or-
number systems, showing how versatile math can be for
ganized and features a soothing color palette to improve
creating interesting challenges.
user enjoyment. Ensuring cross-device compatibility, from
In total, twelve diverse puzzles were made, each offering a
phones to tablets, enables users to indulge in puzzles at their
unique mix of challenge and fun, aimed at improving problem-
convenience, regardless of location or device. By actively
solving, critical thinking, and IT know-how. The variety in
incorporating user feedback, we have continually refined the
puzzle types ensures a diverse and exciting experience, satis-
platform, striving to meet the diverse needs of users across
fying the curiosity of the audience.
different age groups and proficiency levels in puzzles.
V. H ANDPICKED C OURSES FOR M ASTERING TASKS For the development of our platform, we opted for Re-
The list of courses we’re talking about is part of the act, a renowned JavaScript library, renowned for its efficacy
Computer Science program offered by the Faculty of Electrical in crafting user-centric web and mobile applications with
Engineering and Informatics. Solving the puzzles requires a a unified codebase. This choice facilitated the design of a
wide range of knowledge, and these courses lay the ground- platform that is universally compatible and adaptable. Utilizing
work for tackling them. This includes a bunch of different React required the organization of reusable UI components
areas like image processing, graph theory, cryptography, the and meticulous state management to maintain synchronicity
Fibonacci sequence, programming languages, color theory, and between the user interface and underlying data, ensuring a
number systems. reliable and uniform user experience. This approach also made
For example, to solve the ”Decipher a Sentence” puzzle, it easier to add new features and make adjustments based on
you need a basic understanding of cryptography, covered user preferences.
in the Computer Systems Security course. Also, knowing In conclusion, choosing to use React was a wise decision,
about algorithms and programming is key, which is what you as it facilitated the development of a responsive, user-oriented
get from courses like Fundamentals of Algorithmization and website and made the development process more adaptable
Programming and Data Structures and Algorithms. The course and efficient. The resulting platform is intuitive, promising an
on Formal Languages is super helpful for figuring out the enjoyable and enriching experience for puzzle enthusiasts of
structure and syntax of the encrypted sentence. all levels.
These courses act like a guide, helping students learn the
VII. I NSIGHTS FROM F EEDBACK
various knowledge and skills they need to get through the
puzzles. By taking these courses, students are set to boost Getting feedback is super important when doing research.
their problem-solving skills and face each puzzle challenge It helps us figure out how well our solution works and what
confidently. people think about it. For this study, we used an Evaluation

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 185

 Fig. 5. List of Courses Covering Task-Related Topics

Form with 17 questions and shared it with people who 10) Did you learn something new in the process? If so, what
tried out our solution. The main goal of this form was to exactly?
understand how well our solution was working, find ways 11) Did you need help from others to solve the problem?
to make it better, and get the opinions of the people using 12) What did you like about solving the problem?
it. We included questions about participants’ education, job 13) What did you dislike about solving the problem?
experience, problem-solving skills, and what they thought of 14) Do you have any suggestions for improvement?
the role. These questions were carefully made to give us a clear idea
We split the Evaluation Form into two parts. The first of how effective the solution was and what parts might need
part, with five questions, was all about getting to know the some tweaking.
participants. The questions in this section included: From the responses, we were able to gather diverse insights.
For example, participants held different education levels and
1) What is your level of education?
job positions in the IT field, reflecting a variety of expe-
2) Is your education related to the IT field?
riences and perspectives. The feedback on time spent and
3) Are you currently studying? If so, please specify the
task difficulty helped us assess the complexity of the task,
year and field.
while the answers on learning outcomes and suggestions for
4) How many years of experience do you have in IT?
improvement provided valuable directions for refining our
5) Do you have a job? If so, what position do you hold?
solution.
The answers to these questions gave us a picture of who All in all, the Evaluation Form looked at every aspect of our
was taking part and if they had any special characteristics that solution. The responses helped us figure out if we met our
might affect how they interacted with our solution. goals and what areas might need some work. The feedback
The second part, with the remaining 12 questions, focused was really helpful in making decisions about the solution and
on the task itself. The questions were designed to help us learn gave us lots of info for future improvements.
about the time spent, how hard it was, what new knowledge VIII. E VALUATIVE INSIGHTS
was gained, and possible improvements. Some of the key
Looking at the data, it’s not easy to find a clear link between
questions included:
the number of problems solved, how long someone’s been in
6) Were you able to solve the task? If not, please specify the profession, and the time spent. But, we did make some
the problem you encountered. interesting observations.
7) How much time did you spend on the task? The first participant, with five years of experience, didn’t
8) How challenging was the task for you? solve any problems and spent just ten minutes testing. This
9) What areas/subjects did you utilize your knowledge might mean a lack of hands-on experience or maybe not
from? much interest in solving IT problems. On the other hand,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 186

the second participant, with two years of experience, solved offers valuable MDSD teaching guidelines and concludes with
five problems in five hours, showing a strong dedication to positive results from a student survey.
solving problems. The third person, similar to the first in Another noteworthy contribution is by [5], focusing on pro-
experience, did really well by solving nine problems in just gramming assignment assessment. Unlike most works centered
1.5 hours. But the fourth, with nine years of experience, only on the evaluation of final submissions, this study introduces
solved two problems in an hour, which might suggest a gap an evaluation process and a developing system for tracking
in dedication or focus. The fifth and eighth participants, with students’ behavior throughout the semester, offering insights
different levels of experience, showed similar problem-solving into various characteristics like commit and code activity, and
skills. The sixth and seventh, despite having good experience, workload. The tool is based on the free repository management
didn’t show much problem-solving skill, possibly indicating system Gitlab.
a shift in career interests or a lack of engagement with such Several notable studies in the field have explored various as-
tasks. The ninth, with only 1.5 years of experience, solved pects of puzzle games, educational tools, and therapy methods.
all 12 problems, but took seven days, hinting at the task’s [6] introduced the Collaborative Puzzle Game (CPG), a unique
complexity. The last participant, with a whopping 30 years of technology-supported activity designed to foster collaboration
experience, was super efficient, solving 10 problems in just in children with Autistic Spectrum Disorder. The study found
two hours. that the ”enforced collaboration” rules made interaction more
A table summarizing the participant data is provided below: complex but positively impacted children’s collaboration.
In another therapeutic application, [7] developed a movie-
TABLE I based VR therapy system specifically for treating anthro-
PARTICIPANT DATA OVERVIEW
pophobia, or the fear of public speaking. Using immersive
Participant Yrs of No. of Problems Time Invested technology like a head-mounted display and 3-dimensional
No. Exp. Solved (hrs) sound system, the system created a virtual seminar room to
1 5 0 0.17 help patients overcome their fear in a controlled environment.
2 2 5 5 Addressing the challenges of cyberlearning, [8] contributed
an open-source, knowledge-driven web tutoring system frame-
3 5 9 1.5
work designed for adaptive and assessment-driven learning.
4 9 2 1 This framework allows for better knowledge representation
Comp. to and customization to cater to students with different back-
5 2 N/S
8th
grounds, enhancing the overall learning experience.
6 Consid. 0 N/S
[9] explored the application of Puzzle methods to improve
7 Consid. 0 N/S various tools in SAS Enterprise Miner. This study focused on
Comp. to the integration of different algorithms and constraints to reveal
8 21 N/S
5th
new types of non-implicative causal relations and aimed at
9 1.5 12 168 creating more universal applications of logical and statistical
10 30 10 2 methods.
Note: ”Comp. to” denotes the level of the problem solved, Diving into the realm of Computer Science education, [10]
”Consid.” is an abbreviation for considerable years of experience, examined the integration of a puzzle game project focused
and ”N/S” indicates data not specified. on introductory data structures. This approach showcased how
game-based projects can be adapted for students with different
One key observation is that the most experienced person skill levels, providing a versatile tool in the computer science
showed great problem-solving skills, highlighting that having curriculum.
more experience can help in solving problems quickly and
[11] offered a cognitive perspective on the age-related
effectively. The least experienced participant managed to solve
puzzle of language learning. The study suggested that higher
all problems but took much longer, suggesting a possible link
cognitive development in adults could constrain implicit sta-
between experience and tackling complex challenges. The lack
tistical learning processes essential for learning language pat-
of a clear pattern between the number of problems solved
terns, providing insight into why children learn languages
and time spent indicates that solving more problems doesn’t
more easily than adults.
necessarily mean spending more time.
Lastly, [12] focused on adapting various algorithms such
as Bayesian Knowledge Tracing (BKT), Performance Factor
IX. R ELATED W ORK
Analysis (PFA), Elo, and Deep Knowledge Tracing (DKT)
In addition to the studies mentioned below, [4] sheds light to measure geometry competencies through a puzzle game
on the myths and misunderstandings surrounding Model- named Shadowspect. The study found the Elo algorithm to
driven software development (MDSD), presenting an MDSD be the most effective in predicting students’ performance,
course designed to challenge these misconceptions and mo- highlighting the potential of such tools in enhancing formal
tivate students to understand MDSD principles. The paper education.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 187

 These diverse studies provide valuable insights and method- of Modern Methods and Education Forms in the Area of
ologies that can be considered and built upon in the develop- Cybersecurity towards Requirements of Labour Market.”
ment and evaluation of IT-related puzzles aimed at enhancing
R EFERENCES
problem-solving skills and interest in the field.
[1] G. Pleschova and E. Simon, “What we know and fail to know about
the impact of teacher development,” in TEACHER DEVELOPMENT IN
X. C ONCLUSION HIGHER EDUCATION: EXISTING PROGRAMS, PROGRAM IMPACT,
AND FUTURE TRENDS, ser. Routledge Research in Education, 2013,
This project led us on a revealing journey into the world of pp. 1–16.
IT-related puzzles, aiming to measure their effectiveness across [2] G. Pleschova, T. Roxa, K. E. Thomson, and P. Felten, “Conversa-
different experience levels. The variety in problem-solving tions that make meaningful change in teaching, teachers, and aca-
demic development,” INTERNATIONAL JOURNAL FOR ACADEMIC
methods we saw was enlightening, showing how individual DEVELOPMENT, vol. 26, no. 3, SI, pp. 201–209, 2021, dOI:
experience in IT affects one’s approach to challenges. The 10.1080/1360144X.2021.1958446.
reactions from participants were mostly positive, showing a [3] G. Pleschova, “Review of relationship-rich education. how human con-
nections drive success in college,” INTERNATIONAL JOURNAL FOR
good mix between enjoyment and learning value of the puz- ACADEMIC DEVELOPMENT, vol. 27, no. 3, pp. 299–300, 2022, dOI:
zles, which aligns well with our initial hopes. Looking at the 10.1080/1360144X.2021.2019040.
details, we noticed that having more experience didn’t always [4] J. Poruban, M. Bacikova, S. Chodarev, and M. Nosal, “Teaching
pragmatic model-driven software development,” COMPUTER SCIENCE
mean solving more problems or spending more time, showing AND INFORMATION SYSTEMS, vol. 12, no. 2, pp. 683–705, 2015, dOI:
the complex nature of problem-solving. The differences in 10.2298/CSIS140107022P.
approach and results gave us helpful insights, showing the [5] M. Novak, M. Binas, M. Michalko, and F. Jakab, “Student’s progress
tracking on programming assignments,” in 10TH IEEE INTERNA-
many sides of individual engagement and learning in IT. TIONAL CONFERENCE ON EMERGING ELEARNING TECHNOLO-
This observation was especially important as it helped us GIES AND APPLICATIONS (ICETA 2012), 2012, pp. 279–282, dOI:
understand the relationship between experience, commitment, 10.1109/ICETA.2012.6418321.
[6] A. Battocchi, F. Pianesi, P. Venuti, A. Ben-Sasson, E. Gal, and P. L.
and flexibility in dealing with IT-related tasks. Weiss, “Collaborative puzzle game: Fostering collaboration in children
With these findings in mind, we looked at other research with autistic spectrum disorder (asd) and with typical development,” in
in the field to better understand our results. The study done 2009 Virtual Rehabilitation International Conference, 2009, pp. 204–
204.
at Usmanu Danfodiyo University, Sokoto, Nigeria, shared a [7] H. Jo, J. Ku, D. Jang, B. Cho, H. Ahn, J. Lee, Y. Choi, I. Kim, and
similar message, highlighting the positive effect of puzzle- S. Kim, “Movie-based vr therapy system for treatment of anthropopho-
based learning on understanding basic programming [13]. bia,” in 2001 Conference Proceedings of the 23rd Annual International
Conference of the IEEE Engineering in Medicine and Biology Society,
The MobileEdu-Puzzle app used in the study was well-liked, vol. 4, 2001, pp. 3788–3791 vol.4.
showing the increasing importance and usefulness of such [8] M. Sette, L. Tao, and N. Jiang, “A knowledge-driven web tutoring system
interactive learning tools. In the same way, another study that framework for adaptive and assessment-driven open-source learning,” in
2017 IEEE 41st Annual Computer Software and Applications Confer-
added puzzle-based learning to a basic computer programming ence (COMPSAC), vol. 2, 2017, pp. 712–717.
course showed an improvement in the students’ learning expe- [9] V. Jotsov, P. Petrova, and E. Iliev, “Puzzle methods for data science
rience [14]. Using puzzles seemed to create more interest and applications,” in 2016 IEEE 8th International Conference on Intelligent
Systems (IS), 2016, pp. 322–332.
participation among students, supporting the possible benefits [10] L. Opyrchal, J. Van Boxtel, and S. A. Wallace, “A puzzle game for
of this learning method. These studies, along with our findings, introductory data structures,” J. Comput. Sci. Coll., vol. 26, no. 1, p.
make a strong argument for more research and use of puzzle- 133–139, oct 2010.
[11] E. Smalle and R. Mottonen, “Cognitive development as a piece of the
based learning in IT education. language learning puzzle,” Wiley interdisciplinary reviews. Cognitive
Looking back on the project, we’re happy with the progress science, vol. 47, 04 2023.
and the knowledge we gained. The puzzles we created did [12] S. Strukova, J. A. Ruipérez-Valiente, and F. Gomez Marmol, “Adapting
knowledge inference algorithms to measure geometry competencies
their job, providing practical knowledge for different levels through a puzzle game,” ACM Transactions on Knowledge Discovery
of experience. The positive feedback and the similarities with from Data, vol. 18, 08 2023.
other studies suggest a bright future. However, we realize there [13] S. S. Oyelere, F. J. Agbo, I. T. Sanusi, A. A. Yunusa, and K. Sunday,
“Impact of puzzle-based learning technique for programming education
are things to improve, like giving better information with each in nigeria context,” in 2019 IEEE 19th International Conference on
puzzle and making the answer checking process better. Advanced Learning Technologies (ICALT), vol. 2161-377X, 2019, pp.
In conclusion, it’s clear that we’ve started exploring some- 239–241.
[14] K. E. Merrick, “An empirical evaluation of puzzle-based learning as
thing important. Bringing together fun and learning through an interest approach for teaching introductory computer science,” IEEE
puzzles offers new ways for learning and participation. We’re Transactions on Education, vol. 53, no. 4, pp. 677–680, 2010.
happy with the progress made and the things we’ve learned,
but we’re also ready for the next steps, keen to learn more,
improve our method, and find out everything puzzles can offer
in IT education and more.

ACKNOWLEDGMENT
This work was supported by KEGA Agency of the Slovak
Republic under Grant no. ”002TUKE-4/2021 Implementation

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 188

 Developing a User-Centric Queueing Simulation
Engine for Educational Purposes
Roman Horváth
Faculty of Education, Trnava University in Trnava, Slovakia
roman.horvath@truni.sk

Abstract—This article details the development and features mathematical rigour, perhaps with a greater focus on
of a highly customizable simulation engine for logistics and theoretical models. These methods were relatively new and
production systems. Utilizing the GRobot framework, the challenging to understand at the time. In the West, research
system is comprised of 13 core classes that focus on key is often more focused on applied aspects. The scientists
aspects like visual adaptability, mode and type flexibility, and used statistical methods such as Monte Carlo simulation
time management. These core classes facilitate various and Markov chains in their calculations. These methods
functionalities such as object pooling, visualization, naming, were already relatively well-developed and easy to
and link association, making the engine suitable for the understand at that time.
variability of applications. Two key features under
In addition, they mentioned the general context. For
consideration for near future implementation or
example, the connection with the different economic
reimplementation include an undo/redo system and an
situations in the countries at the time and, therefore, the
algorithmic redesign of the internal simulation engine for
focus on solving current problems in these countries. Or
reaching more deterministic behaviour. The article serves as
a comprehensive case study, laying the groundwork for
differences in the availability of publication resources at
future improvements in simulation systems.
that time. And, of course, differences in terminology were
a direct result of what languages the researchers spoke.

I. INTRODUCTION III. THE CORE OF THE SIMULATION ENGINE

In 2021, we commenced the implementation of The core of the system is implemented in thirteen classes
a queueing system specifically designed for instructional whose utilise the GRobot framework. The framework is too
purposes within courses on modelling and simulation [1]. complex, so we will not describe it here in detail. It is well
The primary objective was to establish an environment that documented in other sources – you can read about it, for
facilitates effective simulation and analysis of various example, in [11–20], or on the pages with the framework’s
logistics and production scenarios for students and documentation [21–23]. Not all the thirteen core classes are
educators. While the system was initially conceived for important to be described here. The most fundamental are
educational settings, we entertain the possibility that its those three: Linka (the production line), Zákazník (the
potential applications could extend beyond the classroom. customer), and Systém (which represents the simulation
system). Since the original names of all classes and their
II. BACKGROUND AND LITERATURE REVIEW elements are in Slovak, we will state them in original
wording with the translation in parentheses.
When searched for relevant information in this area in
Slovak and in English, distinct approaches to the subject
matter were observed. They were, in principle, the same, The Linka (Link) class is designed to model a production
but they style was different. This divergence suggests that line or link in a logistics system. The line represents one
research in the field evolved somewhat independently in stop in the simulation process (within the production
the former Czechoslovakia compared to Western countries. system). Each line can be connected to other lines. The
When two do the same thing, it is not the same thing. We customer can also be perceived as goods processed (served)
needed to find a simple way to implement the internal in the line, which takes some time. This might represent
simulation engine, but since we could not identify with any a conveyor belt, a queue of products, or a station where
of the described principles in the studied resources [2–10] robots perform specific tasks. The class implements the
(we did not consider any approach to be elementary Činnosť (Activity) interface that is common for more
enough), we decided to invent our own way. classes of the system and helps implement mass processing.
Recently, automated literature reviews were conducted The two key areas of features include:
using two AI systems—ChatGPT and Bard—to • Visual adaptability – the class can change its visual
supplement traditional background research methods. representation through methods like zmeňNaElipsu
These AI systems corroborated the observed (changeToEllipse) and zmeňNaObdĺžnik (changeTo-
methodological differences. Specifically, research from Rectangle), which is helpful for intuitive graphical
former Czechoslovakia often employed mathematical representation of the simulation elements. Methods like
methods; in contrast, Western research favoured statistical spojnica (connector), zrušSpojnicu (removeConnector),
methods. and aktualizujSpojnice (updateConnectors) represent
As they explained, in former Czechoslovakia, the the visual site of how the line connects to other elements
scientists used mathematical methods such as analytical in the system, allowing for a dynamic and flexible
probability theory and stochastic differential equations in production line. With kopíruj (copy) and blikni (blink),
their calculations. Emphasis was often placed on

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 189

 the class can replicate its state or indicate status changes processing, etc. By doing so, it offers a centralized platform
through visual feedback like blinking. to simulate, manage, and interact with a logistics or
• Mode and type flexibility – the class supports various production line system. In essence, it serves as the “control
“modes” or “states” like režimVýberuZákazníkov room” for the simulated world, providing a structured yet
(customersSelectionMode) and režimVýberuLiniek flexible environment where all objects can interact and
(linesSelectionMode), allowing it to adapt to different function according to defined rules and conditions.
scenarios or requirements. Methods that start with Considering its importance, its description requires
zmeňNa (like zmeňNaEmitor – changeToEmitter or dividing the features into more fine areas.
zmeňNaZásobník – changeToStorageBin) will enable First, let us mention the tools aiding the visualization part
the line to transform its role in the production system, control:
like becoming a source of customers (emitter) or a place • Information display – methods like zobrazInformácie
having specified capacity and able to “store” the (displayInfo) and prepniZobrazenieInformácií (toggle-
customers for an indefinite amount of time (storage Info) control the visibility of system-wide information,
bin). aiding in debugging or performance analysis.
• Grid layout management – mriežkaX (gridX), mriež-
The Zákazník (Customer) class represents a customer or kaY (gridY), and other similar methods manage the
an item in a logistics or production line system. It offers grid-based layout of the production lines. The grid does
a variety of methods for accessing several areas of the key not affect the objects of customers.
features: Then the simulation control:
• Object pooling – static methods like nový (new), počet • Pause and resume – the repauzuj (repause – neologism
(count), and daj (get) are part of the object pooling, derived from pause the same way as reset is derived
allowing management of instances and saving memory from set), pauza (pause), and similar methods are used
demands. Instance method reset helps initialize and to pause and resume the simulation, offering control
reinitialize the objects consistently. over the simulation.
• Visualization – methods like upravCieľPodľaLinky • Timer and events – setTimer (exceptionally, some
(adjustTargetByLink) adjust the visual part of the methods are named in English; this applies also to next
simulation. The target of a customer should stay aligned feature areas), tik (tick), klik (click), and other event-
to its current link (production line) so the visual related methods allow to manage time-sensitive
representation can match the current internal simulation activities and user interactions.
state. Handling events like dosiahnutieCieľa (target- And last the data management control:
Acquired) helps animate instances that got to a specific
target area, which is presumably the target production • System operations – methods like newSystem, open-
line. System, saveSystem, undo, and redo unlock the
capability to create, open, save, and modify the state of
• Naming – with meno (short for getName) and pomenuj the entire system.
(setName) methods, it is possible to get and set the
name of the customers’ instances. This can be crucial • Selection management – methods like selectAll,
for identification, but it depends on the use case. deselectAll, selectNext, and selectPrevious allow
Unnamed objects cannot be distinguished one from the interactive selection capabilities, possibly for Linka
other; however, this might not be a problem in most (Link) and Zákazník (Customer) objects.
simulations. Naming might help or might be even • Connector management – newConnector, delete-
necessary in some use cases, like the distribution of Connectors, jeZačiatokKonektora (isStartOfConnector)
students into groups. mean that the class can manage connections between
• Time management – methods like pridajInterval (add- the production links of the current setup of the
Interval), nastavInterval (setInterval), and čas (time) simulation.
influence the internal clock of the customer. This clock
determines the time remaining to change the state of the IV. FEATURES AND FUNCTIONALITIES
customer. It says, for example, how long a customer When creating a new line, the user can optionally enter
stays in a particular part of the line chain. its name. After the creation, the line can be further widely
• Link association – priraďKLinke (attachToLink) and configured. The usual process is as follows: designate the
vyraďZLinky (detachOfLink) are used to attach or line type (emitter, conveyor, some kind of “processor,” or
detach the customer/item from a specific link in the a “consumer” that is denoted as releaser in this system),
system. configure the line parameters (timer – for example,
processing time, generation time or other telling interval,
line capacity, customer selection mode, etc. – specific
The Systém (System) class serves as the central hub that parameters depend on the line type), modify line visual
orchestrates the simulation environment. It interacts closely parameters (dimension, rotation, shape…) and possibly
not only with both the Linka (Link) and Zákazník modify the next line selection mode for customers leaving
(Customer) classes but also utilises the functionality of all the line. During this configuration process, connections
remaining ten classes. (It interconnects everything between lines can be added or removed, too. Customers can
together.) only transit between lines that are connected.
It can control the visibility and presentation of the Customers are generated automatically in the emitter
simulation objects, manage their lifecycle, provide lines. The emitter can generate named or unnamed
capabilities for pausing and resuming the simulation customers. Cyclic rotation of names can be configured, but
process, mediate the user interface, provide tools for data if we need, for example, to shuffle the list of names, we

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 190

 Figure 1. Simulation chain of lines in inactive and active state. (The image contains translations of selected Slovak texts.)

need to find another way, for example, to generate line in the defined time interval (with an optional dispersion
customers at once, send them to the waiting room, and then that is configurable for almost every type of line), and after
set a random customer selection mode. Scripting of this time, they are released at the exit. The passing through
simulation events is not allowed yet. a conveyor is indicated visually by the animation of
The simulator currently supports the following seven transporting the customers along the entire length of the
types of lines: emitter, storage bin, waiting room, halting conveyor (in contrast with the halting room where they just
room, conveyor, converter, and releaser. Although the wait at random positions). Also, the difference between the
theory notices that node types can be significantly reduced, conveyor and the halting room is in the unlimited capacity
from a functionality and user-friendliness point of view, it of the conveyor.
was easier to implement multiple line types that have The last type of line – converter – represents any type of
certain pre-programmed behaviour. This simplifies the customer service or processing of a product. In Figure 1,
setting up of the simulation system. The following two the same simulation chain of lines in inactive and active
types of lines are particularly substantial: the emitter and states is shown one above the other. All parameters of the
the releaser. The emitter generates customers; the releaser, lines are set to default. The chain contains an emitter,
on the contrary, correctly releases them from the a conveyor, a halting room, and a releaser. The halting
simulation. Correct release means that the customer has room represents here some type of machine, which is not
been served correctly. In addition, a situation may arise a typical use of this line type. Usually, the converter fulfils
during the simulation when a customer attempts to exit this task. The difference is that the halting room has
a line, but none of the connected (output) lines of the a capacity.
current are ready for further processing. Then, the customer For use within this simulation system, we explored the
leaves as “unserved,” and this is recorded as an incorrect possibilities and implemented our own pseudo-random
release. It is necessary to include appropriate line types, number generator, as reported in the article [24].
such as storage bins, halting rooms, waiting rooms, or
conveyors, into the simulation system to prevent such V. VISUAL AND USER EXPERIENCE ASPECTS
situations.
Applicable line configuration parameters depend on the
In the storage bin, customers may wait until the next line type. Figure 2 shows the dialogues for setting the basic
connected line is ready to serve them. In the waiting room, properties of different line types. On the left is a dialogue
the customers also wait, but there is a restriction. When the for a line that still has no purpose set (the most general
customer arrives at the waiting room, a time interval is case); in the middle is a dialogue for a waiting room; and
assigned to him. During the interval, he is willing to wait; on the right are dialogues for an emitter and a release, one
otherwise, he leaves as “unserved.” If any of the next
above the other. Then, Figure 3 shows a dialogue for
connected lines become free earlier than the interval modifying the mode of selecting one of the following lines
expires, the customer leaves the waiting room properly. (for the customers leaving the given line). This setting can
The halting room also assigns an interval to the customer, be well combined with the customer selection mode setting,
but during it, the customer just waits, even if there is a free
which can be seen in Figure 2 – at the bottom of the general
line in the queue. All these lines have a limited capacity of version of the dialogue (on the left) and of the dialogue for
customers they can accept. After any link reaches the the waiting room (in the middle).
maximum capacity, it becomes unavailable until the count
of customers inside the line is lowered. The conveyor Since there is insufficient space in Figures 2 and 3 to
seems to have similar behaviour as the halting groom. It translate the strings, we present them here with a short
accepts customers who stay inside the line until the time explanation. In Figure 2, it makes the most sense to
runs out, but the staying is perceived as passing through the translate the texts from the general variant of the dialogue

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 191

 Figure 2. Dialogues for setting the basic properties of lines. (Strings translations are in the article text.)

because, in the other variants, the strings do have the same prechádzanie spojení […] – sequential (cyclic) switching
or similar meaning (at most just with subtle meaning shifts of connectors – the cyclic counter determines at which line
according to the line types). will start (this is important – the start of search shifts each
• Časovač – timer – indicates the time interval of the time; this significantly increases the probability that the
operation of this line. The true meaning differs. For lines would cycle) the search for the next available line;
instance, for the emitter it is the interval of customer náhodné prechádzanie spojení vyvážené pravdepo-
generation, for the conveyor the transport time, and for dobnosťami […] – random traversal of links balanced by
the releaser this parameter is not applicable. probabilities – each connection (connector) has a value that
determines the weight of the probability that the line on the
• Rozptyl – variance – determines the random spread of
other side will be selected; podľa priorít – uprednostňujúce
the timer values (in both directions). This approach was
linky s vyššou prioritou […] – according to priorities –
chosen due to a more straightforward interpretation
giving priority to lines with a higher priority (weight) –
than entering an interval in the form of parameters
each search for a free line always starts in the same order,
representing the lower and upper bounds and due to the
which is determined by the connector priorities.
possibility of implementing more distribution methods
of pseudo-random values in the future. Currently, only To demonstrate some of the visual settings, we also
uniform distribution is implemented. include Figure 4 in this text, where on the left is the
definition of a simple system that can represent, for
• Počiatočný čas (emitora) – (emitter) initial time – has
example, a computer cafe. The appearance of the lines is
meaning only for the emitter and indicates the initial
adjusted by the dialogue shown in Figure 4 on the right. We
delay of this emitter before it starts generating
do not translate the texts on these images in this article
customers.
because, from our point of view, it (the visualization) is
• Limit (emitora) – (emitter) limit – again has meaning a less important aspect of the simulation system.
only for the emitter and indicates the number of (Although, from the system users’ point of view, this is the
customers that this emitter can generate before it is most important aspect, it is a marginal issue from the point
“exhausted.” (Utilising this feature, it is possible (for of view of data processing.)
instance) to simulate “waves of customers” – you can,
for example, set up two emitters with different time
delays (emitter initial times), any customers limits and
connect them to the same line, through which the
customers will continue further into the system. This
creates two waves of customers of specified numbers at
specified time intervals.)
• Kapacita – capacity – is applicable for line types such
as a storage bin, waiting room, or halting room and
determines the maximum possible capacity of
customers that the line “is able to” store at once.
• Režim výberu zákazníkov – customer selection mode –
there are three modes to choose from: prvý […] – first –
the first customer to arrive will be the first to leave;
posledný […] – last – the last coming customer leaves
first; náhodný […] – random – a random customer is
Figure 3. Dialogue for selecting the mode of the next line choice
chosen (with an even distribution). from the connected ones. (Strings translations are in the article text.)
In Figure 3, there is a dialogue allowing the current line
to change the selection mode of the next line (for outcoming
customers) from the connected ones, i.e., where the VI. FUTURE WORK
customer will go next according to priorities after being The system is practically done. There are just two parts
served. There are three options: postupné (cyklické) that are in consideration: adding the undo/redo feature and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 192

 Figure 4. An example of the definition of a slightly more complex simulation using customization
of the visual side of the lines (left) and the dialogue for choosing a custom line shape (right).

rebuilding the internal simulation engine in the context of Some features still need to be implemented. While the
its principle. The undo/redo is not as crucial since it is undo/redo features just enhance user comfort, the crux of
mainly about user comfort while using the software. The our future development focuses on the internal simulation
rebuilding of the engine is crucial. The question is, what engine. Our tests have pinpointed discrepancies while
makes us consider doing it? adjusting the speed of the simulation (i.e., its timer), leading
Currently, the internal simulation engine is based on the us to reconsider the engine’s architecture. As we’ve ruled
computer operating system nanosecond clock. The out algorithmic issues, the problem seems to be the
application has its own timer, and it measures the amount juxtaposition of different timers, which necessitates a more
of system nanoseconds passed between the application deterministic approach in future iterations.
ticks. Then it slices the nanoseconds into smaller intervals Our system employs its own pseudo-random number
according to the simulation setup and computes the time of generator and offers a variety of line types (emitter, storage
simulation events to make the advancements of the bin, waiting room, halting room, conveyor, converter, and
simulation elements’ internal states. We supposed this releaser), each with specific behaviours and configurations.
principle is universal and will not be affected by speeding This practical utility makes the system not only
up the simulation by multiples of the original time. Since theoretically sound but also user-friendly.
that is not the case, as the tests proved, something must be In summary, this project serves as a case study that
going wrong during the process, and thus the engine must underscores the importance of adaptability and user-centric
be rewritten. design in simulation systems. As we move forward, we aim
We tested the engine using a simple deterministic to further align this balance of functionality, user-
system. If the engine would not be affected by the time -friendliness, and theoretical rigour, setting the stage for
acceleration factor, the results of simulations running at future improvements that will make these systems even
normal speed had to be the same as the results running at more robust and reliable.
multiples of the original speed. But the results began to
change slightly at quadruple speed and were slightly ACKNOWLEDGMENT
different as the factor raised. This result did not please us at The work has been supported by the Cultural and
all. Since we did not find any bug in the engine algorithm
Educational Grant Agency of the Ministry of Education,
as it was designed, it must be a problem using two different Science, Research and Sport of the Slovak Republic
timers. Therefore, we want to dispose of the system timer (KEGA) and the contribution was elaborated as part of the
in the future and advance the simulation states in following KEGA projects: KEGA 013TTU-4/2021 entitled
a different, more deterministic way.
Interactive animation and simulation models for deep
learning and KEGA 012TTU-4/2021 entitled Integration
VII. CONCLUSION of the usage of distance learning processes and the creation
We started with the ambition to create a system that of electronic teaching materials into the education of future
could adapt to various scenarios (primarily intended for use teachers.
in teaching), and the end product has largely achieved that
aim, albeit with some identified limitations. Although the REFERENCES
system we’ve designed is essentially complete, it is [1] Horváth, Roman. (2023). Simulátor systémov hromadnej obsluhy –
continuously evolving. It integrates complex SHO; Simulator of Queuing Networks – QN. Available at: ‫ۦ‬https://
functionalities, from visual adaptability to mode and type github.com/raubirius/SHO/ۧ. Last accessed: 2019-12-05.
flexibility, providing a holistic environment for simulating [2] Dvořák, Jiří. (2002). Systémy hromadné obsluhy. Available at:
complex logistics or production line systems. ‫ۦ‬http://www.uai.fme.vutbr.cz/~jdvorak/vyuka/tsoa/PredO11.pptۧ.
Last accessed: 2019-12-05.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 193

[3] Ullrich, Oliver – Lückerath, Daniel. (2017). An Introduction to PdF TU. In DidInfo 2012. Banská Bystrica : Univerzita Mateja
Discrete-Event Modeling and Simulation. In Simulation Notes Bela, Fakulta prírodných vied, Katedra informatiky, pp. 83–86.
Europe SNE 27(1), 9–16. ‫ۦ‬https://doi.org/10.11128/sne.27.on. ISBN 978-80-557-0342-8.
10362ۧ. Last accessed: 2019-12-05. [15] Horváth, Roman. (2011). Nový model vyučovania programovania
[4] Sundarapandian, Vaidyanathan. (2009). Probability, Statistics and na Pedagogickej fakulte TU. In Acta Facultatis Paedagogicae
Queuing Theory. India : Prentice Hall India Pvt., Limited. Universitatis Tyrnaviensis. Trnava : Trnavská univerzita,
[5] Rábová, Zdeňka – Zendulka, Jaroslav – Češka, Milan – Peringer, Pedagogická fakulta, pp. 98–102. ISBN 978-80-8082-514-0.
Petr – Janoušek, Vladimír. (1992). Modelování a simulace. Brno : Retrieved from: ‫ۦ‬http://pdf.truni.sk/actafp/2011/c/#HORVATHۧ
Vysoké učení technické v Brně. ISBN 80-214-0480-9. (access date: 21/08/2023).
[6] Law, Averill M. – Kelton, W. David. (1991). Simulation Modeling [16] Horváth, Roman. (2011). Teaching one Language in More Depth is
and Analysis. 2nd Edition. New York : McGraw-Hill. ISBN 0-07- better than Many Languages Superficially. In ICETA 2011. Stará
-100803-9. Lesná, The High Tatras, Slovakia : Óbuda University, IEEE,
[7] Dorda, Michal. (2009). Modelování systému hromadné obsluhy pp. 71–74. ISBN 978-1-4577-0050-7. ISBN 978-1-4577-0051-4.
M/M/1/1. In Perner’s Contacts, 4(1), 50–58. Available at: ‫ۦ‬https://doi.org/10.1109/ICETA.2011.6112588ۧ.
‫ۦ‬https://pernerscontacts.upce.cz/index.php/perner/article/view/109 [17] Horváth, Roman. (2009). Inovácia vyučovania predmetov
1ۧ. Last accessed: 2023-09-15. programovania na Pedagogickej fakulte Trnavskej univerzity. In
[8] Špendla, Lukáš – Tanuška, Pavol – Štrbo, Milan. (2013). Model Acta Facultatis Paedagogicae Universitatis Tyrnaviensis. Trnava :
proposal for performance testing of safety-critical systems. In Trnavská univerzita v Trnave, Pedagogická fakulta, pp. 64–68.
Proceedings of the Third International Conference on Control, ISBN 978-80-8082-320-7.
Automation and Systems Engineering (CASE-13). Paris (France) : [18] Horváth, Roman – Fialová, Jana. (2020). The Creation of
Atlantis Press. Advances in Intelligent Systems Research, pp. 42– Simulation with an Algorithm Optimisation in Java for the
45. ISSN 1951-6851. ISBN 978-90786-77-81-9. Teaching Process. In ICETA 2020. Danvers : IEEE, pp. 160–166.
ISBN 978-0-7381-2366-0.
[9] Stoffová, Veronika – Gabaľová, Veronika. (2023). Interactive
simulation models for teaching and learning of computer science. [19] Horváth, Roman. (2019). Empirical Study of Minkowski–
In INTED2023 – International Technology, Education and Bouligand and Hausdorff–Besicovitch Dimensions of Fractal
Development Conference 2023. ‫ۦ‬https://doi.org/10.21125/inted. Curves. In ICETA 2019. Danvers : IEEE, pp. 231–238. ISBN 978-
2023.1050ۧ. Barcelona (Spain) : IATED, pp. 3945–3953. ISSN -1-7281-4966-0.
2340-1079. ISBN 978-84-09-49026-4. [20] Stoffová, Veronika – Horváth, Roman. (2017). Didactic Computer
[10] Važan, Pavel – Križanová, Gabriela – Červeňanská, Zuzana – Games in Teaching and Learning Process. In eLSE 2017 –
Znamenák, Jaroslav. (2017). Modelovanie a simulácia systémov. eLearning and Software for Education Conference. Bucharest :
Simulátor Witness : Návody na cvičenia. Trnava : AlumniPress. Carol I National Defence University Publishing House, pp. 310–
ISBN 978-80-8096-252-4. 319. ISSN 2066-026X, ISSN 2066-026X, ISSN 2343-7669.
[11] Horváth, Roman. (2013). Hľadanie nových postupov vo výučbe [21] Horváth, Roman. (2014). Dokumentácia skupiny tried grafického
programovania na VŠ. Dizertačná práca. Bratislava : Fakulta robota pre Javu – 2. prepracované vydanie. Trnava : Faculty of
matematiky, fyziky a informatiky Univerzity Komenského. 162 s. Education, Trnava University in Trnava. [27,67 AH]. ISBN 978-
-80-8082-796-0. Retrieved from: ‫ۦ‬http://cec.truni.sk/horvath/
[12] Horváth, Roman. (2018). The Past Seven Years of Development of
GRobot/ۧ (access date: 17/08/2023).
the Framework for Teaching Programming and the Students’
Results. In ICETA 2018. Danvers : IEEE, pp. 185–189. ISBN 978- [22] Horváth, Roman. (2012). Dokumentácia skupiny tried grafického
-1-5386-7912-8. robota pre Javu. Trnava : Faculty of Education, Trnava University
[13] Horváth, Roman – Javorský, Stanislav. (2014). New Teaching in Trnava. [20,55 AH]. ISBN 978-80-8082-537-9. Retrieved from:
Model for Java Programming Subjects. In Procedia social and ‫ۦ‬http://cec.truni.sk/horvath/Robot/ۧ (access date: 17/08/2023).
behavioral sciences. Vol. 116(2014). 5th World Conference on [23] Horváth, Roman. (2023). Dokumentácia programovacieho rámca
Educational Sciences – WCES 2013 (konferencia), WCES 2013, GRobot. Retrieved from: ‫ۦ‬https://pdfweb.truni.sk/horvath/
pp. 5188–5193. ISSN 1877-0428. ‫ۦ‬https://doi.org/10.1016/j.sbspro. GRobot/ۧ (access date: 17/08/2023).
2014.01.1098ۧ. Retrieved from: ‫ۦ‬http://www.sciencedirect.com/ [24] Horváth, Roman. (2022). Optimisation of Algorithms Generating
science/article/pii/S187704281401115Xۧ (access date: Pseudorandom Integers with Binomial Distribution. In ICETA
18/08/2023). 2022. Danver : Institute of Electrical and Electronics Engineers,
[14] Horváth, Roman. (2012). Porovnanie hodnotenia študentov pp. 197–201. ISBN 979-8-3503-2032-9.
z predmetu PA1 s bodovaním získaným v prijímacom konaní na

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 194

 Serious games: The Possibilities
of Data Mining and Data Analysis through
Neuro and Biofeedback
M. Hosťovecký*
* Institute of Computer Technologies and Informatics, Faculty of Natural Sciences, University of SS. Cyril and
Methodius, Trnava, Slovakia
marian.hostovecky@ucm.sk

Abstract — Serious games have emerged as valuable tools the primary objectives of developing SG in the field of
for training in various domains, making them an essential education is to enhance the understanding of the
component of training and coaching programs aimed at curriculum and provide more comprehensive explanations
enhancing skills and knowledge. This study aims to explore of content-intensive topics through simplified
the potential of SGs in collecting and analyzing data using explanations and increased interactivity among learners.
neuro and biofeedback techniques. Specifically, we focus on The multi-platform synergy of SG allows for collaboration
selected methods for data analysis, including eye-tracking, among different platforms, eliminating the need for new
electroencephalography (EEG), heart rate variability hardware or platform conversion.
(HRV), and electro-dermal activity (EDA). The objective is Another appealing attribute of SG is the increasingly
to investigate how playing SGs can facilitate the collection immersive environment it provides, facilitated by constant
and analysis of these types of data. technological advancements. This leads to a more targeted
motivation for learners and allows for a fuller integration
I. INTRODUCTION into the learning content. Advancements in hardware, such
as the ability to develop SGs with support for virtual,
Serious games (SGs) are now an essential component of augmented, or mixed reality, exemplify the potential for
the training in technologically advanced societies where creating highly immersive environments. Therefore, we
digital education is prevalent. SGs are interactive digital believe that SG and virtual reality resources offer broader
applications primarily designed for educational purposes, horizons and the potential for a more interactive and
rather than entertainment [1]. The number of registered immersive perception of the applications developed. On
SGs and their popularity continues to grow, with an the basis of the above, it is worth mentioning a few
increasing number of applications targeting various areas aspects of virtual reality in relation to games and training.
of focus. The objective of any SG should be to facilitate
Virtual reality (VR) is a technology that allows for the
the acquisition of knowledge, skills, or expertise in a
simulation of characters, objects, storylines, activities,
specific area.
space, and time in order to create and evoke a realistic
Advancements in digital technology allow impression [3]. Coates defines VR as electronic
programmers to tailor SGs to effectively develop the simulations of environments that are experienced through
required skills among their target audience. These digital head-mounted eye goggles and wired clothing, enabling
applications are no longer restricted to primary pupils, users to interact within immersive three-dimensional
secondary school students or university students. They are situations [4]. Another definition describes VR as an
now widely applicable to diverse professions and specific alternate world, populated by computer-generated images
target groups. SGs can help individuals acquire the that respond to human movements. These simulated
specific knowledge and skills necessary for particular job environments are typically accessed using an expensive
positions. Noteworthy target groups that can benefit from data suit, equipped with stereophonic video goggles and
SGs include doctors, soldiers, teachers, pilots, athletes, fiber-optic data gloves [5]. According to authors Hamad
actors, and many others. Similarly, SGs can be and Jia, while VR initially targeted the gaming industry,
implemented in various fields, including engineering, there are numerous potential applications within various
science, humanities, and professional domains such as sectors including education, training, simulations, and
engineering, veterinary medicine, medicine, economics, even exercise and healthcare [6]. Consequently, it can be
etc. Therefore, it is evident that SGs can be designed to argued that games designed for training, acquiring
cater to different age groups, areas of focus, and specific knowledge or skills, or serving as simulations can benefit
needs or objectives [2]. greatly from the integration of virtual reality technology.
One of the strengths of SGs (SG) lies in their SGs can be also designed specifically for physicians,
continuous innovation of game design, and architecture e.g. for surgeons. These games allow surgeons to develop
for training purposes, using various forms of graphics, their skills and gain experience in a virtual environment
sounds and animations to make them more appealing and before engaging in real-world procedures. The purpose of
desirable to the target audience. The integration of this targeted training is to enhance professional expertise,
artificial intelligence has brought about a significant prepare for the sequential nature of surgical procedures,
change in the meaning and nature of interactivity, navigate non-standard situations that may arise during
enabling wider and more interactive applications. One of operations, and support the development of existing

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 195

knowledge, skills, and overall readiness. By practicing II. DATA ACQUISITION
both general and procedure-specific tasks within the There are numerous methodologies available for data
simulated space, surgeons can thoroughly prepare and collection during SGs, accompanied by a diverse array of
improve themselves. It is important to note that while the
analytical approaches. Consequently, we have opted to
virtual environment cannot fully substitute real-life
focus solely on a subset of specialized data acquisition
experience, it serves as a valuable platform for practicing, techniques in this study. From the myriad of options, we
coordinating, and reinforcing procedural habits. have chosen to explore eye-tracking technology,
Another example of SG implementation is evident in electroencephalography, heart rate variability, and electro-
the field of patient rehabilitation. These SGs provide an dermal skin activity as our primary data sources.
excellent environment for interactive practice for patients
who have experienced injuries or require physical or A. Eye-tracking data mining
cognitive rehabilitation. Rehabilitation exercises extend This method offers new possibilities for data collection
beyond software-based visual perception tasks and can through the tracking of eye movements during serious
incorporate hardware solutions for a comprehensive gaming. It is a field of research primarily concerned with
approach. identifying patterns of human attention while individuals
In addition to the above, another value added by serious view content on a monitor. The technology utilizes
gaming is the verification and testing of acquired or advanced algorithms to gain valuable insights into the
trained procedures. These assessments can take the form behavior of respondents as they interact with the
of quizzes and tests, providing feedback and data for application. Some eye-tracking technologies can also
further processing, analysis, and evaluation. Simulated detect and analyze eye errors made by respondents. For
training through SG aims to improve procedures, skills, or accurate eye movement measurements during application
knowledge. usage, calibration must be performed prior to data
Regarding the acquisition of real data or feedback from collection.
user behavior, it is a current trend in software solutions. In the context of research on the relationship between
Data can be collected in various ways, methods, and by games/SGs and eye-tracking, the following studies are
various devices. However, it remains a question of how to noteworthy. In work of [7], authors discuss the potential
acquire and validate this data in SG. The purpose of for enhancing the gaming experience through peripheral
collecting feedback data during serious gaming is to devices and describe the integration of eye-tracking with
ensure that developers understand and comprehend player other virtual reality technologies in a course setting. They
behaviours, their quick reactions in certain situations, and also provide an overview of eye-tracking and interaction
identify problematic situations or tasks that players may analysis in video games and virtual reality applications.
struggle with or misunderstand. This data allows Silva et al. [8] offer a comprehensive overview of typical
developers to identify and modify complex, visual analytics (VA) systems, discussing their structure
misinterpreted, or generally incomprehensible parts of a and scope. They present five detailed examples that cover
serious game scene. Data mining can also be observed in various application scenarios and demonstrate how the
rehabilitation-focused SGs, where the goal is to make the VA model can be extended to incorporate eye-tracking,
patient understand the importance of adhering to enabling the development of supportive and adaptive
rehabilitation principles and evaluate the correctness of analytics systems. In [9] focus their research on visual
exercise methodology. This data provides insights into attention in the context of eye-tracking. They examine
player behavior, exercise performance speed, and exercise improved transfer performance and reduced cognitive load
duration adequacy. for challenging topics, as well as increased judgment
Exploring a different area, SGs can also be utilized in about learning and satisfaction for both challenging and
the collection and analysis of data related to soft skills non-challenging topics.
development, which has gained prominence in recent As seen, there are extensive possibilities for the
years. This includes skills such as communication, critical utilization of eye-tracking. In relation to SGs, the
thinking, creativity, imagination, teamwork, conflict following research focuses can be pursued:
resolution, problem-solving, and decision-making. Data
x identifying the most/least engaging parts of a
gathered from modeling situations in these areas provides
serious game/application on screen:
a prospective understanding of which aspects should
receive more or less attention. For instance, when with a section designed in this manner, it becomes feasible
addressing conflict between colleagues, the focus should to accurately identify the specific area of the screen within
lie on interacting with individuals on the verge of job the application that is most or least likely to engage a
burnout, experiencing heightened emotions, or physical particular player. Consequently, the development team is
distress. Analyzing the data collected through SGs helps able to evaluate the degree of engagement associated with
determine specific areas that require broader or narrower each element such as graphics, sound, and animation, and
attention to improve the game. These modeled situations discern which elements are more or less engaging.
provide insight for interpreting and drawing conclusions, Conversely, it also helps identify elements that have
enabling the development of more advanced scenes and minimal or no impact on the player's attention or
enhanced negotiation or communication approaches. concentration;
x identification of the player's search areas:
this option primarily deals with the exploration of visual
patterns or decision-making processes employed by
players when searching for specific information or
elements in a game. Each player employs their own
judgment in seeking out particular elements. Therefore,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 196

eye-tracking technology proves to be highly effective in Heart rate variability
analyzing, evaluating, and generalizing the results of such Heart rate variability (HRV) is a device or sensor that
analysis. It aids in understanding the disparities in the captures and analyzes changes in the time intervals
search patterns of elements or information between male between consecutive heartbeats. It functions by
and female players, as well as differences among monitoring the interaction between the sympathetic and
respondents of varying age groups and fields of study like parasympathetic branches of the autonomic nervous
technology, education, and economics; system. The HRV sensor is capable of detecting subtle
x understanding the meaning of the variations in heart rate caused by factors such as
texts/instructions in the classical play: physiological responses to stress, emotions, and cognitive
in the event that the game provides instructions to the processes. From a design standpoint, these devices can
player either at the outset or during gameplay through incorporate other techniques like photo-plethysmography
textual or graphical means, it is possible to analyze and (PPG) and electrocardiography (ECG). Moreover, the
collect data related to these instructions. This data HRV sensor is non-invasive and operates in real-time. To
provides insights into determining which instructions are ensure accuracy, a high-quality HRV sensor must display
more effective and which ones may be confusing. exceptional sensitivity in real-time detection of
Accordingly, any confusing instructions can be improved physiological changes in individuals.
and reformulated. Additionally, it is possible to identify HRV sensors can be utilized by a broad range of
specific words or groups of words that the player may individuals including researchers, athletes, healthcare
completely skip over during the instruction display. Based professionals, and seniors. In the field of SGs, HRV
on these findings, correlations can be established between sensors can be employed as integral tools for:
thoroughly read instructions and successful completion of x stress response,
serious game levels within a designated timeframe, among x emotional regulation,
other factors;
x mental stress.
x identification of colors and color visual
processing of the game:
by leveraging eye-tracking technology, it becomes It is possible to combine the target group and integrate
feasible to monitor how players perceive colors in a it through the application of SGs, such as employing them
serious game. This monitoring enables the identification as training simulators for athletes. In this scenario, athletes
of colors that may be particularly distracting or, can utilize a sensor to record their heart rate, which can be
conversely, colors that yield consistent matches. used to optimize training procedures, monitor recovery
progress, prevent overtraining, and achieve emotion
x exploring empathic perception: regulation during long-term training. By undergoing this
through eye-tracking, the empathic reactions of players in type of training process, athletes will enhance their ability
simulated scenarios within SGs can be assessed. This to handle similar situations encountered in real
analysis facilitates the identification of differences in the competitions. A suitable approach would be to integrate
understanding of a given situation, such as whether more SGs with an HRV sensor into a virtual or mixed-reality
players focus on objects or on subjects (characters). environment.
B. Biometric and neuro data mining options
Galvanic skin response data mining
This area of interest has witnessed steady growth over Serious gaming has emerged as a prominent means of
the past decade, not only in terms of the increasing imparting specific knowledge, expertise, and skills to
number of published articles but also in the heightened gamers. Software development in this domain prioritizes
overall interest within the professional community, interactivity and aims to create an immersive experience
particularly regarding its connection to games. Biosensors, for players. Consequently, the field of human-computer
designed specifically for this purpose, can be defined as interaction research is witnessing a rising trend in
devices that enable the acquisition of specific or selected popularity. One particular measurement approach in this
physiological signals (of various types) from the human area involves the use of a galvanic skin response (GSR)
body. Typically, each sensor is engineered to capture data sensor to analyze physiological reactions. The primary
from a single variable, such as stress, heart rate, or brain objective of this sensor is to quantify the changes in skin's
activity. Once appropriately calibrated, these sensors can electrical conductivity, specifically in response to
be configured to capture data from the human body over a emotional arousal or psychological stress, which are
specific time period. primarily affected by sweat gland activity.
For instance, in the case of capturing human body stress The underlying principle involves the production of
signals, a signal can be recorded every millisecond, sweat by the human body, resulting in moisture on the
resulting in 1,000 records per second. During a few skin. When individuals encounter emotionally or
minutes of gameplay, we can amass data ranging from a psychologically impactful stimuli, their nervous system
few hundred thousand to several million records from a activity intensifies, consequently increasing sweat gland
single participant. Biometric devices currently available activity and lowering skin resistance.
for data collection and analysis encompass such
instruments as: When measuring skin conductance, it is crucial that the
respondent's skin is free from impurities such as food
x heart rate variability, residue, dye or pigments, chemicals or detergents, dust,
x galvanic skin response (GRS), and grease. Thus, it is necessary for respondents to
x electroencephalography (EEG). thoroughly wash and dry their hands before each
measurement.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 197

 The sensors used for conducting these measurements x delta (0.5 - 4 Hz)
are generally compatible with various data collection the delta brainwaves are slow, loud brainwaves (low
devices, although there can be exceptions. This frequency and deeply penetrating, like a drum beat). They
specialized hardware amplifies signals and allows are generated in deepest meditation and dreamless sleep.
researchers to modify and adjust them according to their Delta waves suspend external awareness and are the
requirements. Similarly, when measuring skin source of empathy. Healing and regeneration are
conductance, researchers have the option to create stimulated in this state, and that is why deep restorative
"markers" that serve as reliable indicators of changes sleep is so essential to the healing process [10,11],
during the measurement process. These markers can
include transitions between game levels, task completion x theta (4 – 8 Hz)
or solving, and initiation of new tasks, among others. By theta brainwaves occur most often in sleep but are also
carefully noting these changes, researchers can compare dominant in deep meditation. Theta is our gateway to
them through data analysis among the observed learning, memory, and intuition. In theta, our senses are
respondents. Additionally, software solutions are available withdrawn from the external world and focused on signals
to evaluate the recorded data and generate graphical originating from within. It is that twilight state that we
representations, providing an easily understandable normally only experience fleetingly as we wake or drift
interpretation based on advanced algorithms. off to sleep. In theta we are in a dream; with vivid
However, biometric devices may also have some imagery, intuition, and information beyond our normal
conscious awareness [10,11].
shortcomings. The most common shortcomings include:
x alpha (8 – 12 Hz)
x Signal noise: the alpha activity is best seen in the posterior regions
the reliability and accuracy of captured biometric data and is typical for relaxed and reflecting states of mind. It
can be influenced by various external factors, including emerges with the closing of the eyes.
ambient light and devices operating on the same x beta (12 – 30 Hz)
transmission frequency, which can lead to mutual the beta activity can be divided into sub-bands: low
interference. This interference can result in the incorrect beta waves (12 – 15 Hz), mid-range beta waves (15 - 20
capture of data and subsequently, misinterpretation of Hz), and high beta waves (18 – 40 Hz). Mid-range beta
analysis. activity is associated with increases in energy, anxiety,
performance, and concentration. The beta activity is most
x Measurement accuracy:
evident in frontal regions [12].
is affected by the deployment of the device, its
functionality, etc.. An illustrative research study conducted by the authors
[13] aimed to measure the brain activity of the
x User comfort: participants. In our own research, our objective was to
in the case of measurements and deployment of determine the brain activity of university students while
individual biometric sensors, there is a certain limitation engaged in a serious game focused on English
of comfort. The sensors are implemented on different prepositions, specifically analyzing the beta waves. Two
parts of the human body i.e. the respondent is aware that versions of the serious game were used: one presented in a
he/she has an additional device ready. 2D format and the other in 3D. The only distinction
x Multi-platform limitations: between the two versions was the visual environment.
some devices do not work on the basis of multi-platform Within the conducted research, we employed the
synergy, making their openness to implementation and standard 10-20 EEG system, consisting of 19 electrodes,
deployment more limited. and involved a total of 24 participants in our experiment.
x Incorrect calibration: The content and other components of the game (such as
English sentences, sounds, and animations) were identical
non-spreadsheet calibration can also result in incorrectly
for both 2D and 3D versions. The gameplay of the SG
recorded data. Predictive modeling algorithms can be entailed the participants selecting the correct missing
helpful for this purpose, which can increase measurement English preposition within a sentence. Each sentence had
accuracy. three options to choose from, with only one being the
correct answer. In total, the game included 20 different
Electroencephalography (EEG) data mining examples of English sentences, which were spread across
Among the other possibilities of acquiring data through three different rooms.
biosensors is the monitoring of brain wave activity during In relation to the brain activity measurement, the focus
play. The measurement of brain activity can be achieved was on monitoring beta waves. These waves fall within
by different technologies (in terms of device mobility) the frequency range of 13 Hz to 30 Hz. The resulting
whether they are wired or wireless devices. Within the measurements were visually represented using
above, several types of brainwaves can be measured spectrograms, as depicted in Figure 1. The spectrogram
depending on what the researcher is currently monitoring. was generated using the Short-Time Fourier Transform
The EEG activity can be divided into bands by the (STFT), which analyzes the frequency spectrum of signal
frequency. The frequency bands are extracted from EEG segments over time. By dividing the longer signal into
signal using spectral methods such as PSD, FFT, or STFT. equal-length segments and computing the Fourier
The main energy of the EEG signal measured on the scalp transform on each segment, the STFT provides a graphical
falls in the range of 0.5 – 30 Hz. Four basic bands are representation of the changing spectra as they correlate
recognized in this range: with time [13].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 198

 will result in desired outcomes and interpretations that
address the central research questions. It is important to
note that a large quantity of data does not always
guarantee satisfactory answers to research inquiries.
Therefore, the key parameters for success will likely
include high-quality sensing sensors capable of capturing
various aspects of human activity, a well-designed
research methodology, prepared respondents, and pre-
surveys, among other factors. These parameters will
greatly influence the measurement outcomes, obtained
results, and subsequent interpretations.

ACKNOWLEDGMENT
This article was financially supported and funded by the
grant VEGA 2/0070/21 LOW-D-MATTER.
Figure 1. Spectrogram of channel
Source: [13] REFERENCES
[1] C. Abt, “SGs”. 1987. University press of America.
There are several different options available for [2] T. Susi, M. Johanesson, and P. Backlund, “SGs - An
brainwave measurements, from wired to wireless. Overview (Technical Report)”. Skövde, Sweden: University of
The basic setup is just as important when preparing the Skövde, 2007.
data acquisition and subsequent analysis. This is the [3] M. Host’ovecky, L. Huraj, and K. Pribilova, “Virtual reality in
amount of data captured per unit of time (e.g., mathematics: Design of the serious game improves spatial
milliseconds). Within the EEG, consistent calibration is imagination performance,” in 2019 17th International Conference
important, as well as the setup and configuration of the on Emerging eLearning Technologies and Applications (ICETA),
device. There may be issues during implementation such 2019
as one of the connectors not working properly. [4] G. Coates, “Program from Invisible site-a virtual show a
multimedia performance work”, 1992
In terms of the possibility of collecting data through [5] P. Greenhum, “The lawnmower man”, in Film and video, 9(3),
EEG measurements, it is possible to carry out studies such 1992, pp. 58–62
as:
[6] A. Hamad and B. Jia, “How virtual reality technology has changed
x differences in the perception of the game in our lives: An overview of the current and potential applications
2D and/or 3D, and limitations,” Int. J. Environ. Res. Public Health, vol. 19, no.
18, p. 11278, 2022.
x training simulators of SGs,
[7] V. Sundstedt, D. Navarro, and J. Mautner, “Possibilities and
x measurement of brain wave activity in various challenges with eye tracking in video games and virtual reality
different SGs, applications,” in SIGGRAPH ASIA 2016 Courses, 2016.
x possibilities for monitoring brain beta waves [8] N. Silva et al., “Eye tracking support for visual analytics systems:
Foundations, current applications, and research challenges,” in
in serious gaming, Proceedings of the 11th ACM Symposium on Eye Tracking
x coping with challenging stressful situations, Research & Applications, 2019.
[9] J. Wang, P. Antonenko, and K. Dawson, “Does visual attention to
x measuring a variety of post-traumatic the instructor in online video affect learning and learner
disorders, perceptions? An eye-tracking analysis,” Comput. Educ., vol. 146,
x monitoring social interactions and changes in no. 103779, p. 103779, 2020.
the brain, [10] What are brainwaves? 2017. In: brainworksneurotherapy.com
[online]. Available: http://www. brainworksneurotherapy.com
x attention during gaming, /what-are-brainwaves
x memory and concentration, and many others. [11] The basics of Brain Waves. 2017. In Radarsign.com [online].
Available: https://www.radarsign. com/wp-content/uploads/
CONCLUSION 2016/01/TheBasicsofBrainWaves. RS.pdf
[12] M. Hosťovecký and B. Babušiak, “Brain activity: beta wave
In conclusion, SGs are gaining an increasingly larger analysis of 2D and 3D SGs using EEG,” J. Appl. Math. Stat.
audience and application across various age groups. Inform., vol. 13, no. 2, pp. 39–53, 2017.
Additionally, there is a growing range of SGs in terms of [13] S. HEATHER, 2017. Brain Waves. Available:
content and the acquisition of specific skills, whether they http://www.simonheather.co.uk/pages/articles/brain waves.pdf
be professional or otherwise. To identify specific elements
or behaviors of respondents, it is often necessary to utilize
sensors that measure physiological variables related to
human activity. The data obtained from these recordings
must then undergo analysis and data mining. The goal of
this analysis is to provide an understanding and answer the
researcher's pertinent questions.
Data analysis will be an important but also a critical
aspect of Sgs in the future. We expect an increase in the
rate and volume of data acquisition and analysis within the
realm of serious gaming. However, it is essential to
determine whether such a vast amount of collected data

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 199

 On the border between automatic control and
artificial intelligence
1st Mikuláš Huba 2rd Pavol Bisták
Fac. El. Eng. and Inf. Tech. Fac. El. Eng. and Inf. Tech.
Slovak University of Technology Slovak University of Technology
Bratislava, Slovakia Bratislava, Slovakia
mikulas.huba@stuba.sk pavol.bistak@stuba.sk

Abstract—The contribution deals with the discussion of points beings. Instead of the simple “ability of ruling something“,
of contact and historical differences between automatic control “the intellectual processes characteristic of humans, such as
and artificial intelligence. It shows that many problems concern- the ability to reason, discover meaning, generalize, or learn
ing automatic regulation, that is, keeping process quantities at a
constant level, try to be owned by both disciplines and that the from past experience” is highlighted here.
given fact does not have to be harmful if we can learn from the At this place, however, one has to ask whether it is possible
resulting consequences. This can be illustrated, for example, by to do without above listed abilities to rule something. Does
analyzing concerns about the impact of artificial intelligence on
not the control process require any intelligence, or each
the development of human society, when parallel problems can
also be shown in the development of automatic control. And this ruling, therefore also automatic control, implicitly requires
is not only related to problems that will ”asymptotically” touch certain abilities of artificial intelligence. In the second case,
us sometime in the distant future, but also problems related to since the history of automatic control is at least a couple
solving current problems of university education. of decades longer than the history of digital computers and
Index Terms—PID control, automatic reset, intelligent control
computer-controlled robots, something would probably have
to be omitted in the previous definition of AI. That is,
I. I NTRODUCTION the problem could be overcome by omitting the adjective
“digital”. What some colleagues from the field of digital
When trying to delineate the boundaries between automatic
computers may not like...
control and artificial intelligence, we cannot avoid the fact
that both disciplines are relatively young, rapidly developing The advent of digital computers was actually preceded
and, moreover, related. Thus, it can happen that even if in by the use of analog computers, and the given issue was
our reasoning we start from the definition of basic terms generally referred to as self-acting computers, or as math-
from world-renowned publications, our conclusions might be ematical machines. If we were to include Wolfgang von
outdated or ideologically biased. For example, according to Kempelen’s chess automaton (about 1770), the boundaries
[1] is the Automatic Control (AC) briefly defined as “Control of artificial intelligence would move significantly further into
in which regulating and switching operations are performed the past. Although in this case it was only a trick and the first
automatically in response to predetermined conditions. Also real chess machine, capable of playing simple endings, was
known as automatic regulation.” In some publications, the built only in 1912. However, history is full of various other
differences between regulation and control are detailed more mechanisms that could provoke discussions and the need to
consistently. In the case of regulation, it is about maintaining move the beginnings of artificial intelligence deeper into the
the selected quantities of processes at required constant levels past, let’s just mention the mechanism from Antikythera, or
or close to them despite disturbances. In the case of control, Heron’s automaton, which filled the inserted glass with wine
it is already a matter of changing the defined quantities of after a coin was thrown.
processes according to more complex schedules. Regulation In the field of automatic control, Watt’s centrifugal regula-
is therefore the simplest case of control. And, if we add to this tor (1765) is usually referred to as the first modern controller.
that according to https://dictionary.cambridge.org/dictionary/ It was just a modernized version of older windmill regulators,
english/control the verb “control” means “to order, limit, or but let’s conclude our trip to the past with this date. Its
rule something, or someone’s actions or behavior”, the range mission was to maintain constant revolutions even with a
of possibilities for the meaning of automatic control will variable load on the machine, which, however, was not per-
probably continue to expand. fectly ensured in the original version. Its basic “intelligence”
In contrast, artificial intelligence (AI) according to https:// was limited to acting against changes leading to deviations
www.britannica.com/technology/artificial-intelligence means of the output of the process from its desired (setpoint) value.
the ability of a digital computer or computer-controlled Therefore, its activity was improved later by W.Siemens [2]
robot to perform tasks commonly associated with intelligent and numerous other inventions.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 200

 equivalent to such solutions in some (frequently narrow) sit-
uations. The difference may seem insignificant, but improper
presentation of proven historical solutions during the transi-
tion to digital technologies gave rise to a significant, previ-
ously unknown problem of unintended integration (windup)
and the need for a special solution to prevent it (anti-windup).
However, the fact that the scientific community was busy
solving tasks that did not have to be at all is only one aspect
of the overall problem.
Another, perhaps even more important aspect is that no
attention was paid to the true nature of the controller us-
ing automatic reset. Automatic reset represents the simplest
known disturbance observer based on steady states of integral
process models. Disturbance observer is able to supply in-
formation regarding the controlled process and thus be used,
for example, for fault diagnosis. Moreover, its design can be
interpreted as an intelligent control imitating the procedure
of an experienced process operator. The latter identified the
input disturbance by waiting for the controller output to
be established in steady state and using the obtained data
to correct the output of the stabilizing controller. If we
want to simplify such a procedure as much as possible and
avoid in general a more complicated decision regarding the
achievement of steady states, it was enough to insert a simple
low-pass filter with a sufficiently long time constant into
the positive feedback channel transforming the established
Fig. 1. Watt’s centrifugal governor (1765)
value of the overall controller output to the stabilizing
controller offset (see e.g. [4]). Thus, an intelligent solution
was introduced, invariant to the controller output constraints
II. F IRST INDUSTRIAL CONTROLLERS causing windup. Automatic reset was shown to be expandable
also for the case of stabilizing controllers of increasing
complexity, such as hyper-reset from 1938. Only later it was
Although the Watt’s centrifugal regulator is usually men- (confusingly) renamed to proportional-integrative-derivative
tioned as the first known AC application, it was not some- (PID) controller.
thing used independently from the controlled process. It
represented just a fixed part of the steam engine. The steam Then in the 1950s, the “artificial intelligence” (AI) had
engine was also offering the energy needed for the governor been used to describe machines that mimic and display
in implementation of action interventions. Equivalent, but “human” cognitive skills associated with the human mind,
independently existing controllers, which would be useful for such as “learning” and “problem- solving” (from Wikipedia,
various other processes from practice, appeared around the the free encyclopedia). The heuristic explanation of automatic
time of the First World War [2]. In the next development, reset and hyper reset (PI and PID controllers) with the help
mainly pneumatic regulators gained ground, which used of an integration component obscured the true essence of
compressed air supply. This was an advantage especially in intelligent industrial solutions that existed and were widely
the explosive environment typical of the chemical industry. used much earlier. Although, their impact on society was
Even though the first solutions with the function of a non- enormous, when only in enriching uranium to prepare the
linear amplifier (designated as a proportional (P) controller) first atomic bomb, more such controllers were needed than
were not yet able to completely eliminate impact of inter- had been produced up to that time.
fering disturbances (such as load changes), around 1930 a However, the danger of using inadequate explanations of
pneumatic solution capable of full compensation of piece- already discovered and widely used solutions is not only
wise constant disturbances also appeared. And although this related to the fact that their real inventors have disappeared
solution, labeled as automatic reset, is still the basis of widely from textbooks. Far greater consequences were caused by
used industrial controllers, in standard textbooks it is not slowing down progress in solving newer, more difficult tasks,
explained and presented in accordance with its essence [3], which require the expansion of already known solutions in
[4]. a more complex and more demanding context. And this
Indeed, standard references talk instead about proportional- is where older (almost forgotten) principles become more
integrative (PI) controllers, although real industrial con- effective than their replacements with integrative controllers
trollers do not include an explicit integrator - they are only extended by anti-windup [5]–[10].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 201

 III. AI AS A REBELLION AGAINST MATHEMATICS IN AC? [21]–[24], the basic problem of the approaches inspired
If you look at the newly cited contributions introducing by [20] was the application of optimization methods to
automatic reset controllers with higher-order derivatives, you any system models, without looking to models with wider
will find that they are full of mathematical apparatus, which is applicability to broader classes of dynamic systems. The
often disliked by a significant part of our young community, apparent simplicity of finding an optimal solution for any
which is much more fascinated by the interactive possibilities system required consequently repeating the optimization for
of numerical computers. Although, in reality, already these each new set of identified system parameters.
works already represent a certain rebellion against the ill- Avoiding the analytical solution of the problem of optimal
conceived use of mathematics. But, from the first view of control by numerical optimization brought to the controller
the non-participants, it does not seem so. design a boom of genetic and bio-inspired optimization
Revolts against used mathematical methods in automatic algorithms with various exotic names. Let’s just briefly
control and efforts to revise existing automatic control design mention several approaches to optimization of controllers
methods is far from new. However, at the end, each such a using the 2nd output derivative denoted as the PIDD2 , i.e.,
rebellion leads regularly to even more complex mathematical proportional-integrative-derivative-accelerative (PIDA) con-
apparatus, which should be balanced by its (proclaimed) sig- troller. For the automatic voltage regulator (AVR) of a
nificantly higher relevance and effectiveness - let’s mention synchronous generator, [25] was minimizing the integral of
only the biggest initiatives associated with the appearance the time-weighted absolute error (ITAE) using particle swarm
of modern control theory (MCT) [11], [12], internal model optimization. The optimal design of a higher order nonlinear
control (IMC) [13] and dead- time compensators (DTC) as time-delay system using a modified butterfly optimization
generalized Smith predictor [14], active disturbance rejection algorithm was treated in [26]. [27] deals with improving
control (ADRC) [15], [16], or model-free control (MFC) the flight stability of a quad copter in a noisy environment
[17]. Despite the amount of new interesting knowledge they by means of a heuristic genetic filter design. In [28], the
have brought, the key question of, which of these approaches output voltage control of a dc-dc converter was calculated
gives better results and when, is still not answered in general. using a modified Gray-Wolf optimizer. [29] uses combination
At the same time, the huge inflation of optimal control was of the flower pollinated algorithm (FPA) and pathfinder
already created in the PID control itself. Already decades algorithm (PFA) to regulate combined load frequency and
ago O’Dwyer [18] collected hundreds of optimal settings of terminal voltage control regulation systems. Broader review
PI and PID controllers, and the increase in their number has of classical and intelligent PID tuning methods are discussed,
not slowed down in the meantime, rather the opposite. And for example, by [30], etc.
if there is more than one optimal solution, the researcher But, of course, the overview of various methods of intelli-
should start dealing with the question of which one is the gent setting of algorithms cannot be considered closed with
right one. And because with the growing number of optimal these examples (see e.g. [31], [32]). One can only state a huge
solutions, it may not be an easy task, it prompts him to decide amount of new approaches obtained by applying artificial
to actually find an own “optimal” solution. intelligence methods to find optimal controller parameters, as
At the beginning of the 1990s, after the climax of booms well as a number of new controllers created by implementing
of MCT, IMC and other postmodern approaches, there was the principles of new control approaches as fractional-order
a return to the issue of “traditional” PID control, manifested PID, fuzzy, or neural control. However, the disproportionate
by the publication [19]. Soon, however, another contribution emphasis on new aspects of work in scientific publications
[20] indicated that it is not just a return, but rather a new and the small emphasis on the separation of essential and
rebellion, a transition to a new dimension with the use non-essential aspects of the solutions, not noticing the in-
of numerical optimization. If O’Dwyer already signaled in terrelationships of different approaches, did not lead to the
the older editions of his collection [18] the stagnation of creation of a more concentrated and more effective theory
a unifying theory expressed by the excessive number of of automatic control. And so, even if, for example, PID
“optimal” settings of PI and PID controllers, the emergence controllers were “innovated” with fuzzy control procedures,
of numerical optimization only accelerated this inflation. It PID still only represents hardware at the level of the inter-
is a fact that in AC you have to adjust the controller de- war period. Similarly, AI methods did not lead to higher
pending on a wide range of different practical requirements, automation of setting controllers in practice, which can be
which could seemingly explain at least part of this inflation. considered one of the important reasons for the formulation
However, if we are unable to overcome this problem with a of the Industry 4.0 initiative and its newer versions.
clear abstraction relating the multiplicity of existing methods
with the multiplicity of the corresponding requirements and IV. P ERFORMANCE P ORTRAIT M ETHOD
not able to separate the essential and non-essential factors Among the large number of different approaches to op-
of the overall solution, this inflation will increase and the timally setting the parameters of controllers, we would like
work space of the scientific community will be overwhelmed to draw attention to a method that is fundamentally different
by a number of non-essential works. While, on the one from the above mentioned methods. The performance por-
hand, it must be noted that the development of numerical trait method (see e.g. [5], [33]–[40]) is based on mapping
optimization has also brought a large number of useful results the properties of a closed loop control circuit using set

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 202

of monotonicity-based and integral of absolute error (IAE) more than a decade ago, several details must be worked out
performance measures. Over a grid of all possible relevant for its broader acceptance.
settings of the controller the performance data achieved by By preferentially using simple integral models, PPM is
simulation or real-time control are stored in the so-called close to the so-called model-free methods of designing
performance portrait and archived for a future use. From the controllers such as ADRC [15], [16], or model-free control
point of view of the principles of artificial intelligence applied (MFC) [17]. The accuracy of control based on the approx-
until then, such an approach requiring the application of all imation of processes using “coarse”, ultra-local models (eg
relevant settings (i.e. optimization inputs) was considered integrator plus dead-time, IPDT) increases with the growing
as a principal disadvantage and therefore referred to as a range of controller derivations [5], [7], [10], [42]. However,
”brute force approach” [22]. However, this ignored the usual this also increases the number of unknown controller param-
practice of “smart” operators in practice, applying frequently eters and the required dimension of the generated PP, which
“trial and error” approach. They first systematically map the exponentially increases the number of points at which it must
properties of the control circuit corresponding to various con- be generated, the memory requirements increase, the search
troller settings and just then, based on such “identification” for appropriate settings takes longer, etc. From a practical
(i.e. gained experience), they can easier orientate themselves point of view, it would also be appropriate to solve the
when solving the currently given task. possibilities of expansion and refinement of the existing PP,
When designing an optimal control, it is not just about possibly also compression (especially in the case of growing
finding the best control for one set of specifically given PP dimensions), use of parallel processing for PP calculation,
requirements. Such control may not even exist. A signif- etc.
icantly more effective procedure is therefore to find the One of the interesting features of PPM is that it allows
optimal control for arbitrarily specified requirements. Solving to complement the analytical design of optimal nominal
a given task based on the previously achieved results of controllers for IPDT and double integrator plus dead-time
the initial analysis recorded in the database denoted as the (DIPDT) systems considered so far using the analytical
performance portrait (PP) can then be significantly simpler multiple real dominant poles (MRDP) method. From this
and more effective and reliable. The search itself avoids point of view, it is useful that the possibility of combining
the optimization problem associated with finding the global the numerical search for optimal parameters with the MRDP
extreme, and only the resulting accuracy of finding the method and the possibility of modifying this method for more
optimum depends on the quantization step when defining general groupings of dominant poles have recently appeared
the grid of input values. Alternatively, problems related to [8], [10], [39]. Using MRP, it is also possible to significantly
the stability of the closed loop system for controller settings speed up browsing around the calculated optimal nominal
lying between the evaluated points are solved in a different setting [8], [10]. Further research will therefore also focus
way for the given type of controller and process (see e.g. on examining other modifications of these two methods.
[41]).
V. T HE DANGERS OF NEW TECHNOLOGIES
It turns out that from the point of view of the problem set
up in this way, the key role is played by: A. Safety of automatic control
In connection with the use of PP defined over a grid of
1) Approximation of the controlled process by a suitable
discrete points, we briefly stopped at the question of whether
model (with the possibly low number of parameters);
hidden instability of the system cannot be expected between
2) Calculation of performance portrait for specially se-
the considered points. And we immediately answered that
lected simplified models with standardized parameters.
independent methods and the existence of a certain continuity
In other words, the use of PPM avoids the calculation of PP of properties when changing parameters are used to verify
for any existing process model and the search for an optimal stability. However, this does not apply in general, because
setting satisfying some set of specified parameters. When the essence of new solutions can be completely different.
solving a specific task, it is enough to limit yourself to a In connection with the increasingly frequent use of neural
process approximation corresponding to an already existing networks, as if the safety and stability aspects recede into the
PP, select the best suitable standardized controller parameters background. These controllers can actually be trained in such
from it and recalculate real controller parameters to corre- a way that they ensure high control quality in many tested
spond to real model parameters. And from this point of view, situations. However, even with a large number of tests, it is
it is precisely the optimization carried out repeatedly for the not possible to guarantee that they will behave safety like
arbitrary process and looking for a solution corresponding to this absolutely always. And problems can occur already in
one specific set of control requirements given that appears to such simple systems as, for example, used for switching high-
be such a ”brute force approach” and therefore will probably beam and dim lights in cars, which are not absolutely reliable
continue to remain in the position of an academic exercise. and can cause traffic accidents due to faulty operation. In the
On the other hand, even PPM, representing an automatized case of the unexplained crashes of planes with completely
trial and error approach, cannot yet be considered a finished contrived systems providing various degrees of support to
and ripe method used widely in practice. Although its prin- pilots, the search for the cause of the failure can then be
cipals and basic procedures for its use were demonstrated significantly more complicated.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 203

B. Various applications of conservatism, which would lead to over-damping, too
slow transients. In case of insufficient respect for changes
If we want to continue discussing the overlap between the
in dynamic properties, however, it may occur to exceed the
two fields and explaining the importance of understanding the
specified speed limits, which will be more and more punished
relationship between AC and AI, in addition to the history
in the future, for example, by the installation of stationary
of this relationship, which we illustrated using early work
radars and other types of speed sensors in the streets.
on feedback control systems, it would be useful to say more
about the convergence of automatic control and AI in recent VI. C ONCLUSIONS
years. In doing so, it would be interesting to mention some
Comparing the definitions of automatic control and artifi-
basic applications of automatic control and AI in
cial intelligence available to the public via the Internet shows
• Car driving their interrelationships but certainly showed also doubts about
• Electromobility their accuracy. This fact is surprising, because there are
• Industrial automation, or strong communities of researchers working in both areas,
• Financial trading who would be expected to place more importance on the
With respect to the participants of the conference, it would definition of their mission.
be appropriate to limit ourselves to the most general areas of Developments in the field of automatic control, as well as
their use, with which they may actually have experience. AI, also document the adage that when someone knows how
The media like to deal with this topic very often, and to do something, he does it. If he doesn’t know how to do it,
it might seem that the dangers come from some artificial he teaches it, or even directs it. This fully reflects the position
world. At this point, however, it would be necessary to of automatic reset and hyper reset controllers, which form the
emphasize that the greatest danger from the point of view basis of industrial controllers, but remain misunderstood in
of artificial intelligence results from its stronger and more both communities. Their excellent robustness can, at least
frequent combination with human intelligence. And this is to some degree, deal with this misunderstanding. However,
not only about its conscious abuse through the development questions arise regarding the further development of this
of more and more effective and sophisticated weapons, but exceptional structure towards more demanding tasks and
also about unintended consequences. implementation with the use of new technologies. As an
In automatic control, from the beginning of development, example, let’s just mention nuclear fusion, the successful
great attention was paid to the analysis and ensuring the implementation of which is still beyond the edge of current
stability of the built systems. This was sometimes to the detri- possibilities and the successful mastery of which would mean
ment of the given discipline, which, thanks to its reliability, a huge yield. And what about, if it will finally be achieved
compiled it hidden from the public. with help of analog circuits such as can be built on Field
A lot has changed with the advent of computer networks. Programmable Analog Arrays (FPAA), [43]?
While in 1990 the computer was able to work a few seconds Last but not least, the whole matter has an impact on the
after starting, today it may happen that the computer reports education of a new generation of researchers. We live in a
the need to install an operating system upgrade and it may time when the space for teaching the basics of automatic
be unusable for minutes. Other problems arise as a result of control is constantly shrinking [44]–[46] and the readiness
upgrades of individual user programs. of students rather decreases. But, since automatic control
Both compared disciplines can be found to participate in overlaps to a considerable extent with the basics of artificial
the automatic maintenance of the selected driving speed of a intelligence, this applies even more generally. At the same
car while keeping the distance from the vehicles in front. It time, the demands on the performance of the proposed new
requires high robustness to changes in system parameters, control systems are constantly growing. So if we are to talk
evaluating the position of several vehicles, but also high about the dangers of new technologies, we are here at their
accuracy. In the future, we can expect improvements to be roots.
achieved through mutual communication between vehicles, ACKNOWLEDGMENT
which will make it easier to coordinate the maintenance of
a safe distance between them. In this area, for example, the This research was supported by the grant KEGA 030STU-
control algorithms mentioned above can be used [5]–[10]. 4/2021
The corresponding system is usually referred to as adaptive R EFERENCES
cruise control.
[1] “Automatic control,” in McGraw-Hill Dictionary of Scientific Tech-
We are talking about the requirement of high robustness nical Terms, 6Ed. New York: The McGraw-Hill Companies, Inc.,
because the system must work sufficiently precisely when the 2003.
vehicle is full and empty, when driving on a straight road, [2] S. Bennet, “A brief history of automatic control,” IEEE Control
Systems, vol. Volume 16, Number 3, pp. 17–25, 1996.
uphill and downhill and must also adapt to the movement [3] M. Huba and Z. Gao, “Uncovering Disturbance Observer and Ultra-
of another vehicle(s). All these changes lead to considerable Local Plant Models in Series PI Controllers,” Symmetry, vol. 14, 4, p.
variations in the dynamic characteristics of the vehicle. At 640, 2022.
[4] M. Huba and P. Bisták, “Should we forget the PID control?” in 2022
the same time, it is not possible to use traditional automatic 20th International Conference on Emerging eLearning Technologies
control approaches for the design because of their high degree and Applications (ICETA), Oct 2022.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 204

 [5] M. Huba, P. Bistak, and D. Vrancic, “Series PIDA Controller Design optimisation,” International Journal of Systems Science, vol. 53, no. 1,
for IPDT Processes,” Applied Sciences, vol. 13, no. 4, p. 2040, 2023. pp. 168–181, 2022.
[6] P. Bisták, M. Huba, S. Chamraz, and D. Vrančić, “IPDT Model-Based [32] S. Bansal, “Nature-inspired hybrid multi-objective optimization algo-
Ziegler-Nichols Tuning Generalized to Controllers with Higher-Order rithms in search of near-ogrs to eliminate fwm noise signals in optical
Derivatives,” Sensors, vol. 23, p. 3787, 2023. wdm systems and their performance comparison,” J. Inst. Eng. India
[7] M. Huba, P. Bisták, and D. Vrančić, “Series PID Control with Higher- Ser. B, vol. 102, 2021.
Order Derivatives for Processes Approximated by IPDT Models,” IEEE [33] M. Huba, “Setpoint versus disturbance responses of the IPDT plant,”
TASE, vol. accepted, 2023. IFAC Conf. Advances in PID Control PID’12, 2012.
[8] ——, “Optimizing constrained series PIDA controller for speed loops [34] ——, “Performance measures, performance limits and optimal PI
inspired by Ziegler-Nichols,” in EDPE, High Tatras, Slovakia, 2023. control for the IPDT plant,” Journal of Process Control, vol. 23, 4,
[9] ——, “Designing automatic-reset controllers with higher-order deriva- pp. 500–515, 2013.
tives,” in EDPE, High Tatras, Slovakia, 2023. [35] ——, “Comparing 2DOF PI and Predictive Disturbance Observer
[10] M. Huba, P. Bistak, and D. Vrancic, “Parametrization and Optimal Based Filtered PI Control,” Journal of Process Control, vol. 23, 10,
Tuning of Constrained Series PIDA Controller for IPDT Models,” pp. 1379–1400, 2013.
Mathematics, vol. 11, 20, p. 4229, 2023. [36] ——, “Performance Portrait Method: a new CAD Tool,” in 10th
[11] W. L. Brogan, Modern control theory. Pearson education india, 1991. Symposium on Advances in Control Education (ACE). Sheffield, UK:
IFAC, 2013.
[12] K. Ogata, Modern Control Engineering. 3rd Ed. Marcel Dekker, 1997.
[37] M. Huba, J. Škrinárová, and P. Bisták, “Higher order pd and ipd
[13] M. Morari and E. Zafiriou, Robust Process Control. Englewood Cliffs, controller tuning,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 8808–8813,
N. Jersey: Prentice Hall, 1989. 2020, 21th IFAC World Congress.
[14] O. Smith, “Closser control of loops with dead time,” Chemical [38] M. Huba, J. Škrinárová, A. Dudáš, and P. Bisták, “Optimal PID
Engineering Progress, vol. 53, pp. 217–219, 1957. controller tuning for the time delayed double integrator,” in 21st Int.
[15] J. Han, “From PID to Active Disturbance Rejection Control,” Industrial Carpathian Control Conference - ICCC, Slovak Rep., 2020.
Electronics, IEEE Transactions on, vol. 56, no. 3, pp. 900–906, march [39] M. Huba and D. Vrančić, “Tuning of PID Control for the Double
2009. Integrator Plus Dead-Time Model by Modified Real Dominant Pole
[16] Z. Gao, “On the centrality of disturbance rejection in automatic and Performance Portrait Methods,” Mathematics, vol. 10, 6, p. 971,
control,” ISA transactions, vol. 53, pp. 850–857, 10 2014. 2022.
[17] M. Fliess and C. Join, “An alternative to proportional-integral and [40] ——, “Performance Portrait Method: an Intelligent PID Controller
proportional-integral-derivative regulators: Intelligent proportional- Design Based on a Database of Relevant Systems Behaviors,” Sensors,
derivative regulators,” International Journal of Robust and Nonlinear vol. 22, no. 10, p. 3753, 2022.
Control, pp. 1–13, 2021. [41] M. Huba, P. Bistak, and D. Vrancic, “Robust Stability Analysis of
[18] A. O’Dwyer, Handbook of PI and PID controller tuning rules. 3rd Ed. Filtered PI and PID Controllers for IPDT Processes,” Mathematics,
Imperial College Press, 2009. vol. 11, 7, 2023.
[19] K. J. Åström and T. Hägglund, PID controllers: Theory, design, and [42] M. Huba and D. Vrančič, “Comparing filtered pi, pid and pidd2 control
tuning - 2nd ed. NC: Instrument Society of America, Research for the fotd plants,” IFAC-PapersOnLine, vol. 51, no. 4, pp. 954–
Triangle Park, 1995. 959, 2018, 3rd IFAC Conference on Advances in Proportional-Integral-
[20] K. J. Åström, H. Panagopoulos, and T. Hägglund, “Design of PI Derivative Control PID 2018.
controllers based on non-convex optimization,” Automatica, vol. 34, [43] J. Hasler, “Large-scale field-programmable analog arrays,” Proceedings
pp. 585–601, 1998. of the IEEE, vol. 108, no. 8, pp. 1283–1302, 2020.
[21] P. Mercader, K. J. Åström, A. Banos, and T. Hägglund, “Robust [44] J. Rossiter, K. Zakova, M. Huba, A. Serbezov, and A. Visioli, “A first
PID Design Based on QFT and Convex/Concave Optimization,” IEEE course in feedback, dynamics and control: findings from an online pilot
Transactions on Control Systems Technology, vol. 25, no. 2, pp. 441– survey for the ifac community,” IFAC-PapersOnLine, vol. 52, no. 9,
452, 2017. pp. 298 – 305, 2019, 12th IFAC Symposium on Advances in Control
[22] P. Mercader and A. Banos, “A PI tuning rule for integrating plus dead Education ACE 2019.
time processes with parametric uncertainty,” ISA Transactions, vol. 67, [45] J. A. Rossiter, K. Zakova, M. Huba, A. Serbezov, and A. Visioli, “A
pp. 246 – 255, 2017. first course in feedback, dynamics and control: findings from 2019
[23] P. Mercader, K. Soltesz, and A. Banos, “Robust pid design by chance- online survey of the international control community,” in IFAC WC,
constrained optimization,” Journal of the Franklin Institute, vol. 354, Berlin, Germany, July 2020.
no. 18, pp. 8217–8231, 2017. [46] A. Rossiter, A. Serbezov, A. Visioli, K. Zakova, and M. Huba, “A
[24] S. Veinović, D. Stojić, and L. Ivanović, “Optimized pidd2 controller for survey of international views on a first course in systems and control
avr systems regarding robustness,” International Journal of Electrical for engineering undergraduates,” IFAC Journal of Systems and Control,
Power and Energy Systems, vol. 145, p. 108646, 2023. vol. 13, p. 100092, 2020.
[25] M. A. Sahib, “A novel optimal PID plus second order derivative
controller for AVR system,” Engineering Science and Technology, an
International Journal, vol. 18, no. 2, pp. 194 – 206, 2015.
[26] J. Arulvadivu, S. Manoharan, R. Lal Raja Singh, and S. Giriprasad,
“Optimal design of proportional integral derivative acceleration con-
troller for higher-order nonlinear time delay system using m-mboa
technique,” International Journal of Numerical Modelling: Electronic
Networks, Devices and Fields, 2022.
[27] S. M. Zandavi, V. Chung, and A. Anaissi, “Accelerated control
using stochastic dual simplex algorithm and genetic filter for drone
application,” IEEE Transactions on Aerospace and Electronic Systems,
vol. 58, no. 3, pp. 2180–2191, 2022.
[28] M. Kumar and Y. V. Hote, “PIDD2 Controller Design Based on Internal
Model Control Approach for a Non-Ideal DC-DC Boost Converter,” in
2021 IEEE Texas Power and Energy Conference, TPEC 2021, 2021.
[29] S. Oladipo, Y. Sun, and Z. Wang, “An effective hFPAPFA for a PIDA-
based hybrid loop of Load Frequency and terminal voltage regulation
system,” in 2021 IEEE PES/IAS PowerAfrica, 2021, pp. 1–5.
[30] R. Borase, D. Maghade, S. Sondkar, and S. N. Pawar, “A review of
pid control, tuning methods and applications,” Int. J. Dynam. Control,
vol. 9, pp. 818–827, 2021.
[31] S. Saxena and S. Biradar, “Fractional-order imc controller for high-
order system using reduced-order modelling via big-bang, big-crunch

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 205

 Selected Issues Causing Meta-stability in Digital
Circuits and Approaches to Avoiding Them
1st Adam Hudec 2nd Richard Ravasz
Institute of Electronics and Photonics Institute of Electronics and Photonics
Slovak University of Technology in Bratislava Slovak University of Technology in Bratislava
adam.hudec@stuba.sk richard.ravasz@stuba.sk

3rd Robert Ondica 4th Viera Stopjakova

Institute of Electronics and Photonics Institute of Electronics and Photonics
Slovak University of Technology in Bratislava Slovak University of Technology in Bratislava
robert.ondica@stuba.sk viera.stopjakova@stuba.sk

Abstract—This paper presents selected issues that may occur ing circuits (e.g. resistor and capacitor banks) controlled by
in digital circuit design flow. The first problem deals with register memory [2]. It could be written or read out by an
reset signal. The digital circuits become more robust and their external device, for example by a Microprocessor (MCU), a
design requires considering the pros and cons of different reset
architectures to apply a proper one. The second issue is related Field Programmable Gate Array (FPGA), or a computer via
to meta-stability, which is frequently caused by either setup time standard or non-standard communication protocols. It is not
or hold time violations. Closely related with this issue is Clock a rule but the frequency of filling tuning registers is usually
Domain Crossing (CDC) that happens when the output of a flip- different than the frequency of capturing these states, which
flop is fed to the input of another flip-flop and these flip-flops change behaviour of the tuned circuit. This situation can lead
do not share the same clock domain. The last section offers
information about a clock multiplexer or a clock switch. to a phenomenon called the Clock Domain Crossing. [3] With
increasing complexity of digital circuits, it is no longer even
Index Terms—Reset, Setup and Hold Time, Clock Domain possible that there would not be more than one clock domain.
Crossing, Clock Switching, Glitch-Free
If CDC issue is not identified, it will primarily bring the entire
digital circuit to fail.
I. I NTRODUCTION
A synchronous digital circuits must have a common syn-
Advanced research in the field of microelectronics and chronization clock signal for proper functioning. Mostly, only
related modern technologies give an option to Application- one common clock signal is present in simple proposals which
Specific Integrated Circuit (ASIC) designers to create chips is distributed over the clock net to all elements of the digital
with growing representation of discrete circuits. In addition, it system. More complex proposals use multiple clock domains
has a positive effect on increasing the scale of integration that with different frequencies. In some cases, a certain block of
in practice means an increase in computing rate and a decrease a circuit needs to be triggered by different clock frequency
in power consumption on the same area. On the other hand, than the others. As an example, consider a system in idle
high density and longer signal or data paths bring problems, state and clock frequency could be lower to save energy
which cannot be neglected in digital circuit design flow. or a system which use standard communication protocol as
The first problem could be observed as an issue of Reset. the Serial Peripheral Interface (SPI) or inter-integrated circuit
The reset is very important for circuits based on sequential (I2C), where the domain frequency must be switched to a
logic because this signal causes that digital hardware cells ca- proper value. This can be realized by a clock multiplexer, also
pable hold their previous logic state are initialized to a known known as a clock switch. Hardware structure of a clock switch
state. There are two types of reset, namely: synchronous and must be carefully designed because it can easily generates
asynchronous one what means that it can be either dependent glitches at its output, which may bring the system to a meta-
or independent on the active edge of clock signal present. [1] stable state. There are several techniques to prevent such a
Each of these reset types represents its pros and cons which situation, and in this paper, some of them will be presented.
will be mentioned in the next section.
Complex circuits or mixed-signal (both and digital signals
II. R ESET S IGNAL
are used) circuit design, especially for research purposes, often
require their parameters to be tuned. This can be usually The reset and clock signals are essential part of sequential
realized by laser trimming method or by fuses destruction. circuits because they do not have any option to perform self
Nowadays, more modern and suitable way how to trim in- initialization. In general, hence at the beginning, the reset must
tegrated circuit parameters is implementing additional tun- always be able to force the system into a known state for

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 206

correct operation. [4], [5] Additionally, during processing tasks 2) Disadvantages:
failure, it returns the circuit to its normal operation without any • It requires the presence of an active edge of clock
false triggers, which could lead to system corruption. In our signal to reset system.
proposal, one of two reset types, namely Synchronous reset • Synchronous reset might be a great problem for
and Asynchronous reset, can be implemented. Each of these systems operating with clock gating cells (to reduce
types is suitable for a specific application and therefore, it is power consumption).
useful to take a closer look at the reset signal properties for • Since, the clock works as a kind of reset glitches
better selection, and also to show how designing these reset filter, it is necessary to stretch the reset signal.
types in Verilog language can be carried out.
B. Asynchronous Reset
A. Synchronous Reset The asynchronous reset does not require active clock signal
The synchronous reset is fully-dependent on the event to get flip-flops to a known state. That means, if the reset
(active edge) of clock signal. Thus, when the reset signal is is enabled or activated, flip-flops bring their output Q to the
enabled, any sequential circuits will not be really initialized known state immediately, as shown in Figure 2. DFF with
to a known state till the active edge of clock comes. The asynchronous reset can also be designed using Verilog. The
demonstration of signals flow and behaviour of D Flip-Flop code in both cases is almost the same except for the sensitivity
(DFF) with synchronous reset is depicted in Figure 1 and list of always block, where asynchronous reset is secured by
in Listing 1, where DFF circuit was described using Verilog adding N RST signal as shown in Listing 2.
language.

CLK
CLK
N_RST
N_RST
D
D
Q
Q
Fig. 2. D Flip-Flop with asynchronous reset.
Fig. 1. D Flip-Flop with synchronous reset.

Listing 1. D Flip Flop with synchronous reset in Verilog Listing 2. D Flip-Flop with asynchronous reset in Verilog
module DFF( module DFF(
input wire CLK, input wire CLK,
input wire N_RST, input wire N_RST,
input wire D, input wire D,
output wire Q output wire Q
); );

reg q; reg q;

always @(posedge CLK) begin always @(posedge CLK or negedge N_RST) begin
q <= (N_RST == 1’b0) ? 1’b0 : D; q <= (N_RST == 1’b0) ? 1’b0 : D;
end end

assign Q = q; assign Q = q;

endmodule endmodule

1) Advantages: 1) Advantages:
• The synchronous reset can only be occurred at • Asynchronous reset has the highest priority among
active edge of clock signal. all signals.
• Waiting for active clock edge can also be interpreted • If the reset is activated, all flip-flops will be initial-
as a reset glitches filter. ized promptly.
• The flip-flop circuit is simpler. • Reset is independent of the active clock signal.
• The whole proposed digital circuit is fully- • The data path is guaranteed not to be limited by
synchronous. reset.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 207

 2) Disadvantages: Remaining time of the aperture interval - thold is necessary
• Asynchronous reset signal is high sensitive to time to keep launched logical state absolutely stable. The
glitches at any time. hold time observance prevents to reach meta-stable state.
• Activating asynchronous reset near an active edge These time parameters are always provided in data-sheet by
of clock could cause meta-stablility of DFFs. manufacturer and have to be kept by designer.
B. Clock Domain Crossing
III. M ETA - STABILITY I SSUE Increasing demands on chip design often bring the issue
Standard structure of designed digital circuits usually, con- called Clock Domain Crossing. [9] It is the process of passing
sists of both combination logic and sequential logic. However, the signal from one clock domain to the other clock domain.
the sequential logic can be categorized into asynchronous and If we look at the digital chip from the global perspective,
synchronous parts. the system is globally asynchronous but locally works
Asynchronous sequential circuits allow changes of logic synchronously. [10] Moreover, inside the chip, more than just
states regardless to the active edge of clock, and outputs one clock domain can be used. As mentioned above, the clock
swap at different times. The most significant advantage of signal is used to synchronize flip-flops within the system. In
asynchronous circuit is the speed, but on the other hand, the Figure 4 and Figure 5, there are depicted situations, where two
process of circuit design is difficult and the behavior is not flip-flops are triggered but each with different clock frequency.
always predictable.
More reliable and predictable digital circuits, so-called D_FF1 D_FF2
synchronous sequential circuits, are based on triggering the
DIN Q1 Q2
transition from the previous state to the current state on active D Q D Q

edge of governing synchronization signal called clock. In these CLK1 CLK2

systems, every single synchronous block controlled by the Q Q

clock signal is driven at the same time that solves many

problems common in asynchronous circuits. Synchronization
seems to make a sequential logic flawless but the opposite Fig. 4. Two DFFs synchronized with different clock domains.
is true, and some issues linked with the clock signal will be
presented below.
A. Setup and Hold time CLK1
In digital circuit design, flip-flops are very often used and
CLK2
implemented. These are electronic elements and each flop-flop
is able to keep information of one bit width. The output state
Q1
varies due to input and with respect to active edge of the
clock domain but the data and clock transitions might not be
Q2
executed exactly at the same time because it could cause the
output remain unresolved for an infinity, in other words it will
reach so-called meta-stable state. [6], [7] Fig. 5. Occurring meta-stability due to clock domains crossing.
There is a kind of time window called aperture, which is
attached to the active edge of clock. It consists of setup time
tsetup and hold time thold as depicted in Figure 3. [8] Setup In order to avoid propagation of the meta-stability state due
to the crossing of time domains, we can use an additional
circuit to synchronize them. [11] One of the most applied
CLK synchronizing circuits is a dual flip-flop synchronizer (2-FF).
aperture A dual flip-flop synchronizer is a circuit based on two DFFs
that are connected back to back in the receiver domain as
tsetup thold displayed in Figure 6.
If the first DFF in receiver domain goes into the meta-
D stable state because of setup time or hold time violation, the
second DFF stays in a known state as depicted in Figure 7.
It helps getting some time to acquire the known state again.
Fig. 3. Setup and Hold time. The following logic use data only from the second DFF in the
receiver domain.
time represents the minimum amount of time before a flip-flop
will be triggered with the active edge of clock signal. During
this interval, the input data must be stable for being captured
correctly by DFF.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 208

 Sender Domain Receiver Domain Listing 3. 2FF synchronizer in Verilog
module D_2FF(
D_FF1 D_FF2 D_FF3 input wire CLK,
DIN
D Q
Q1
D Q
Q2
D Q
Q3 input wire N_RST,
input wire D,
CLK1 output wire Q
Q Q Q
);
CLK2
reg q1, q2;
Synchronization chain
always @(posedge CLK) begin
if (N_RST == 1’b0) begin
Fig. 6. Two-stages D flip-flop synchronizer. q1 <= 1’b0;
q2 <= 1’b0;
end else begin
Dual synchronizer is used in many designs, and works q1 <= D;
q2 <= q1;
quite good until it reaches its frequency limitations. If the end
clock speed is very high, it could cause malfunctions of end
digital system because the first DFF in synchronization chain
may not have time to recover its output to well-defined assign Q = q2;
state and thus, the second DFF will capture and transfer this
endmodule
meta-stable state to the logic which follows. The way how
to increase parameter called Mean Time Between Failures
(MTBF), is using more than 2 stages of flip-flops in the
C. Clock Switching
synchronization chain.
Growing use and integration of standard communication
protocols like I2C, SPI and others on a chip requires the
CLK1 presence of multiple clock domains. Moreover, it is necessary
to switch between them during operation. This is usually
CLK2 secured by a digital multiplexer, which is able to switch
between several input lines to which the clock domains
Q1 are connected. Selection/control of the multiplexer inputs is
possible by its input labelled as select as shown in Figure 8.
Q2 Clock frequencies provided through the multiplexer could be

Q3 CLK1

SELECT

Fig. 7. Waveforms of dual flip-flop synchronizer.

OUTPUT CLK

If we want to synchronize a single-bit control signal, a

common way to design a synchronization circuit is to use a CLK0

two-stage flip-flop chain ( see Listing 3). This solution can

only be used where the frequency of sender domain is less
than the frequency of receiver domain. If we do not want Fig. 8. Standard digital multiplexer.
the information to be lost, this difference in frequency must
be at least 1.5 times. On the other hand, if the frequency of absolutely unrelated to each other or they may be multiples of
the receiver domain is almost equal or less than the sender each other. Switching of the output clock signal can generate a
domain, the sender must hold its state stable for receiver to glitch because the selector is switched by asynchronous logic
capture it. In other words, the sender must stretch or hold or driven one of the input clock domains. A glitch occurred
its output in stable state for more than one clock cycle. In in the clock signal (see Figure 9) can cause meta-stability to
such a case, there are synchronizers based on handshaking. the whole system because some registers could break the rules
A sender sends a request and receives acknowledgement from about setup time and hold time.
a receiver. However, this handshake technique can be used
instead of 2 stages flip-flop synchronizer. If more than just
1-bit information is synchronized with 2-FF, we have to use
a 2-FF synchronizer to every single bit of data, which has
negative impact to the chip area.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 209

 CLK0
frequency, the output of clock switch can still reach hazardous
state.
CLK1
Listing 4. Glitch-free clock switching in Verilog
SELECT module GFCS(
input wire CLK0,
OUTPUT CLK input wire CLK1,
input wire N_RST,
input wire SEL,
Fig. 9. A glitch caused by output clock switching. output wire OUT_CLK
);

The way to avoid hazards caused by glitches at the output reg q0, q1;
of clock multiplexer is to employ additional flip-flops in the always @(negedge CLK0) begin
multiplexer circuit, as presented in Figure 10. [12] DFFs q0 <= (N_RST == 1’b0) ?
serve as enabling gates triggered on negative edge of every 1’b0 : ˜SEL & q1;
provided clock source. Capturing the select signal at negative end
(falling) edge guarantees that no changes occur at the switch
always @(negedge CLK1) begin
output while both domains are at high level, which secures a q1 <= (N_RST == 1’b0) ?
soft continuing clock signal without any unwanted artefacts. 1’b0 : SEL & q0;
The connection between feedback from one clock’s selection end
to the other is enabled by a switch, which ensures the output
without glitches. The wave diagram depicted in Figure 11 assign OUT_CLK = (q1 & CLK1) | (q0 & CLK0);
shows behaviour of the glitch-free clock switch. When the endmodule
selector is switched from 0 to 1 during CLK0 interval, the
output waits for the positive edge of the CLK1 domain. In Figure 12, a glitch-free clock switch which middle part
resemble synchronizer is depicted. [13], [14] An extra D flip-
SELECT D_FF1 flop for each clock source is added and triggered by the
D Q
positive edge of clock signal. This is the way how to avoid
getting the whole circuit to unknown state if both domains are
Q
CLK1
unrelated as it is shown in Figure 13.

SELECT D_FF1

D_FF0 OUTPUT CLK

D Q D Q

D Q Q Q
CLK1
Q
CLK0
D_FF0 OUTPUT CLK

D Q D Q

Fig. 10. Glitch-free clock switch for related clocks. Q Q

CLK0

Fig. 12. Glitch-free clock switching for unrelated clocks.

CLK0

CLK1
CLK0
SELECT
CLK1
OUTPUT CLK
SELECT

Fig. 11. Switching clock domains without glitch. OUTPUT CLK

Fig. 13. Switching unrelated clock domains without glitch.

The glitch-free clock switch hardware structure is depicted
in Figure 10, and its description in Verilog is shown in
Listing 4. However this solution is still not ideal and there IV. C ONCLUSION
is some space for enhancement. Even if the clock domain In this paper, we addressed selected issues linked with
with higher frequency has a multiplied value of one with lower the control/synchronizing signals, which may appear during

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 210

digital circuits design flow. The first issue was related to reset [7] H. Aparicio and P. Ituero, “A 40nm critical path monitor for the detection
signal and we tried to point out that before the digital circuit of setup and hold time violations,” in 2018 28th International Sym-
posium on Power and Timing Modeling, Optimization and Simulation
design begins, it is necessary to determine the proper type (PATMOS), 2018, pp. 13–18.
of reset that depends on circuit operation and specifications. [8] L. Jayashree, B. Suveditta, and K. Vasundhara Patel, “Setup and hold
The most applied reset type is synchronous one because it can time analysis of d flip-flop based on cmos and gnrfet,” in 2023 IEEE
3rd International Conference on Technology, Engineering, Manage-
solve many problems linked with asynchronous reset such as ment for Societal impact using Marketing, Entrepreneurship and Talent
glitches or meta-stability. (TEMSMET), 2023, pp. 1–5.
The meta-stability is closely related to flip-flops and the [9] Y. Feng, Z. Zhou, D. Tong, and X. Cheng, “Clock domain crossing
fault model and coverage metric for validation of soc design,” in 2007
clock domain present. The active edge of clock signal is Design, Automation Test in Europe Conference Exhibition, 2007, pp.
surrounded by setup time and hold time. These intervals are 1–6.
critical parameters requiring a stable input data state until the [10] A. B. Chong, “Clock domain crossing verification challenges,” in
2021 2nd International Conference on Electronics, Communications and
time passed these intervals. Information Technology (CECIT), 2021, pp. 383–387.
Then, we discussed Clock Domain Crossing issue that might [11] H. S. Poornima and C. Nagaraju, “Functional verification of clock
be present in multi-domain clock systems, where passing the domain crossing in register transfer level,” in 2023 International Con-
ference on Recent Trends in Electronics and Communication (ICRTEC),
signal from one clock domain to the other domain takes place, 2023, pp. 1–6.
which without additional flip-flop in the receiver domain can [12] R. Mahmud, “Techniques to make clock switching glitch free,” Captured
bring a system into hazardous state. A clock circuit where in http://www. eetimes. com/news/design/showArticle. jhtml, 2003.
[13] H. Wang, Y. Zhang, X. Li, L. Chen, Z. Wen, K. Zhang, and M. Wang, “A
flip-flops are chained is called the clock synchronizer. This configurable fault-tolerant glitch-free clock switching circuit,” in 2013
method is helpful for passing the signal from domain with IEEE 56th International Midwest Symposium on Circuits and Systems
lower frequency to domain with higher frequency. On the other (MWSCAS), 2013, pp. 537–540.
[14] S. Zeidler, O. Schrape, A. Breitenreiter, and M. Krstić, “A glitch-free
hand, signal generated by the sender domain must be stretched. clock multiplexer for non-continuously running clocks,” in 2020 23rd
The last discussed issue was Clock Switching that occurs in Euromicro Conference on Digital System Design (DSD), 2020, pp. 11–
systems working with multiple clock domains, where switch- 15.
ing between domains uses a standard multiplexer. This kind of
clock switch works very good, however, some glitches at the
multiplexer output can occur that consequently, may lead to
the setup/hold time violation. The option how to avoid glitches
is enhancing the standard multiplexer by implementation flip-
flops inserted into each clock domain.
All digital system design issues described and discussed
in this paper, and presented schematics or Verilog source-
codes can be efficiently included and used in curricula of
related subject, and also provided as educational material for
students/designers in the related study programs.

ACKNOWLEDGEMENT
This work was supported in part by the Slovak Research
and Development Agency under grant APVV-19-0392, and by
grants VEGA 1/0760/21 and VEGA 1/0731/20.

R EFERENCES
[1] C. E. Cummings, D. Mills, and S. Golson, “Asynchronous & syn-
chronous reset design techniques-part deux,” SNUG Boston, vol. 9, 2003.
[2] A. Hudec, R. Ravasz, D. Arbet, and V. Stopjakova, “A control system
for automated evaluation and tuning of asic parameters,” in 2021
International Conference on Applied Electronics (AE), 2021, pp. 1–4.
[3] S. Verma and A. S. Dabare, “Understanding clock domain crossing
issues,” EE Times, 2007.
[4] X. Wei, “Asynchronous reset design architecture,” in 2014 12th IEEE
International Conference on Solid-State and Integrated Circuit Technol-
ogy (ICSICT), 2014, pp. 1–3.
[5] S. Athreya, H. Hedayati, S. Kazemkhani, Y. Chen, S. Vats, M. D. Scott,
B. Zeydel, P. Keller, J. Wang, B. Avula, B. Murmann, and E. Iroaga,
“Clock synchronous reset and skew calibration of 65gs/s adcs in a multi-
lane coherent receiver,” in ESSCIRC 2018 - IEEE 44th European Solid
State Circuits Conference (ESSCIRC), 2018, pp. 250–253.
[6] T. Iwagaki and K. K. Saluja, “Indirect detection of clock skew induced
hold-time violations on functional paths using scan shift operations,”
in 14th IEEE International Symposium on Design and Diagnostics of
Electronic Circuits and Systems, 2011, pp. 175–178.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 211

 /HDUQLQJHQKDQFHPHQWZLWK$,)URPLGHDWRLPSOHPHQWDWLRQ 

/+XUDM-3RVStFKDODQG,'LUJRYi/XSWiNRYi
8QLYHUVLW\RI66&\ULODQG0HWKRGLXV,QVWLWXWHRI&RPSXWHU7HFKQRORJLHVDQG,QIRUPDWLFV7UQDYD6ORYDNLD
ODGLVODYKXUDM#XFPVNMLULSRVSLFKDO#XFPVNLYHWDGLUJRYDOXSWDNRYD#XFPVN


$EVWUDFW7KHLQWHJUDWLRQRIDUWLILFLDOLQWHOOLJHQFH $, LQWRWKH • 0DFKLQH OHDUQLQJ WKH SURFHVV E\ ZKLFK D V\VWHP
OHDUQLQJHQYLURQPHQWUHSUHVHQWVDIXQGDPHQWDOFKDQJHLQWKH OHDUQVIURPH[SHULHQFHDQGLPSURYHVLWVSHUIRUPDQFH
WUDQVIRUPDWLRQRIHGXFDWLRQDOSUDFWLFHV,QWKHFRQWH[WRIWKLV RYHUWLPH
DUWLFOH D ZLGH UDQJH RI SURPLVLQJ SRVVLELOLWLHV WKDW DUWLILFLDO
• 'HHSOHDUQLQJDSDUWLFXODUW\SHRIPDFKLQHOHDUQLQJ
LQWHOOLJHQFH EULQJV WR WKH PRGHUQL]DWLRQ RI WKH HGXFDWLRQDO
WKDWXVHVGHHSQHXUDOQHWZRUNVWRH[WUDFWSDWWHUQVDQG
HQYLURQPHQWLVH[DPLQHG,WVSULPDU\SXUSRVHLVWRSHUIRUPD
NQRZOHGJHIURPGDWD
WKRURXJK DQDO\VLV RI $, DSSOLFDWLRQV ZLWKLQ HGXFDWLRQDO
FRQWH[WV ZLWK DQ HPSKDVLV RQ VSHFLILF H[DPSOHV RI • 1DWXUDO /DQJXDJH 3URFHVVLQJ WKH DELOLW\ RI D
DSSOLFDWLRQV DQG WKHLU HIIHFWLYHQHVV FDWHJRUL]HG DFFRUGLQJ WR FRPSXWHU WR SURFHVV XQGHUVWDQG DQG JHQHUDWH
WKHEDVLFW\SHVRI$,SURFHVVLQJ,QRUGHUWRDFKLHYHDKROLVWLF KXPDQODQJXDJH
YLHZRIWKHWRSLFWKHSDSHUV\VWHPDWLFDOO\FROOHFWVDQGSUHVHQWV • $UWLILFLDO QHXUDO QHWZRUN $ PRGHO LQVSLUHG E\ WKH
UHOHYDQW UHVHDUFK SDSHUV IURP WKH GRPDLQV RI $, DQG VWUXFWXUH RI ELRORJLFDO QHXUDO QHWZRUNV LV HPSOR\HG
HGXFDWLRQ (DFK DSSOLFDWLRQ SUHVHQWHG LV VXSSRUWHG E\ WRWDFNOHLQWULFDWHSUREOHPV
FRQFUHWHUHVXOWVIURPH[SHULPHQWVRQJURXSVRIUHDOVWXGHQWV
KLJKOLJKWLQJ WKH UHDO LPSDFW DQG HIIHFWLYHQHVV RI WKHVH • 7H[W DQG GDWD PLQLQJ LGHQWLI\LQJ SDWWHUQV DQG
WHFKQRORJLHV ,Q WKLV ZD\ WKH SDSHU KLJKOLJKWV WKH HYHU LQIRUPDWLRQIURPWH[WDQGGDWDVRXUFHV
HYROYLQJ SRWHQWLDO DQG SRVVLELOLWLHV RI DUWLILFLDO LQWHOOLJHQFH • $OJRULWKPLFWUDGLQJWKHXVHRIDUWLILFLDOLQWHOOLJHQFH
ZLWKLQ WKH HGXFDWLRQDO HQYLURQPHQW SURYLGLQJ D VLJQLILFDQW DOJRULWKPV IRU DXWRPDWHG WUDGLQJ LQ ILQDQFLDO
FRQWULEXWLRQ WR WKH ILHOG RI HGXFDWLRQ DQG HQFRXUDJLQJ PDUNHWV
VWLPXODWLQJGHEDWHDERXWWKHIXWXUHRIHGXFDWLRQ • 8VLQJIX]]\ORJLFDPRGHOIRUSURFHVVLQJIX]]\DQG
YDJXHLQIRUPDWLRQ
, ,1752'8&7,21 LL  $,EDVHGRQLQGXVWU\DSSOLFDWLRQV
$UWLILFLDOLQWHOOLJHQFH $, LVQRZDGD\VEHFRPLQJDNH\ 7KLV FDWHJRU\ IRFXVHV RQ WKH XVH RI DUWLILFLDO
SOD\HU LQ WKH WUDQVIRUPDWLRQ DQG LPSURYHPHQW RI YDULRXV LQWHOOLJHQFHLQLQGXVWULDOHQYLURQPHQWVDQGWKHFUHDWLRQRI
DUHDV RI RXU OLYHV ,W LV D VLJQLILFDQW FDWDO\VW WKDW LV LQWHOOLJHQWSURGXFWV7KLVLQFOXGHVWKHIROORZLQJ
FKDQJLQJ WKH ZD\ ZH SHUFHLYH DQG LQWHUDFW ZLWK • ,QGXVWU\DFRQFHSWUHODWHGWRWKHDXWRPDWLRQDQG
WHFKQRORJ\ $UWLILFLDO LQWHOOLJHQFH LV FUHGLWHG ZLWK WKH GLJLWDOL]DWLRQRISURGXFWLRQSURFHVVHV
DELOLW\WROHDUQDQDO\]HGDWDVROYHFRPSOH[SUREOHPVDQG
• ,QGXVWULDO ,QWHUQHW RI 7KLQJV ,,R7  UHDOWLPH
HYHQVLPXODWHKXPDQWKLQNLQJ7KHVHFKDUDFWHULVWLFVDOORZ
DUWLILFLDO LQWHOOLJHQFH V\VWHPV WR JR EH\RQG WUDGLWLRQDO LQWHJUDWLRQDQGPRQLWRULQJRISK\VLFDOREMHFWV
WKLQNLQJ DQG FRQWULEXWH WR QHZ ZD\V RI UHDVRQLQJ DQG • 5RERWLFV WKH XVH RI URERWV DQG DXWRPDWHG V\VWHPV
GHFLVLRQPDNLQJ IRUYDULRXVWDVNV
2YHUWKHSDVWGHFDGHVDUWLILFLDOLQWHOOLJHQFHKDVEHFRPH • :HDUDEOHWHFKQRORJ\GHYLFHVWKDWDUHZRUQGLUHFWO\
D IRFDO SRLQW IRU UHVHDUFK DQG LQQRYDWLRQ OD\LQJ WKH RQWKHERG\DQGFROOHFWELRPHWULFGDWD
IRXQGDWLRQIRUFRXQWOHVVDSSOLFDWLRQVLQDYDULHW\RIILHOGV • 6PDUW SURGXFWV SURGXFWV ZLWK LQWHJUDWHG DUWLILFLDO
,WVUHDFKH[WHQGVIURPPHGLFLQHWRLQGXVWU\WRHGXFDWLRQ LQWHOOLJHQFH WR LPSURYH WKHLU IXQFWLRQDOLW\ DQG
$OWKRXJK VRPH $, FRQFHSWV EHJDQ WR HYROYH LQ WKH ODVW HIILFLHQF\
FHQWXU\ LW LV RQO\ ZLWK WKH DGYHQW RI PRGHUQ FRPSXWLQJ
SRZHUDQGGDWDVRXUFHVWKDWWKHVHFRQFHSWVKDYHEHFRPHD LLL $,EDVHGRQV\VWHPDSSOLFDWLRQV
UHDOLW\ 7RGD\ D QHZ GLPHQVLRQ LV EHLQJ DGGHG WR 7KLVFDWHJRU\GHDOVZLWKWKHXVHRIDUWLILFLDOLQWHOOLJHQFH
FODVVLFDO $, WKURXJK PDFKLQH OHDUQLQJ PHWKRGV WKDW LQV\VWHPVDQGWHFKQRORJLHV7KLVLQFOXGHVWKHIROORZLQJ
LQFOXGHGHHSQHXUDOQHWZRUNVSDWWHUQUHFRJQLWLRQQDWXUDO • $XWRPDWLRQ DXWRPDWHG FRQWURO RI SURFHVVHV DQG
ODQJXDJHSURFHVVLQJDQGPDQ\RWKHUWHFKQLTXHVDOORZLQJ V\VWHPV
LW WR LQFUHDVLQJO\ DVVHUW LWVHOI LQ FRPSOH[ WDVNV WKDW
SUHYLRXVO\VHHPHGLPSRVVLEOH>@ • ,QWHUQHW RI 7KLQJV ,R7  WKH LQWHUFRQQHFWLRQ RI
SK\VLFDOGHYLFHVDQGREMHFWVXVLQJWKH,QWHUQHW
7KHDXWKRUV%DKRRHWDOLQ>@EDVHGRQDUHYLHZRIWKH
OLWHUDWXUHRQ$,EHWZHHQDQGGLYLGHDQGFRYHU • 6RIW &RPSXWLQJ XVLQJ XQFHUWDLQW\ DQG
GHILQLWLRQV RI DUWLILFLDO LQWHOOLJHQFH EDVHG RQ WKUHH PDLQ DSSUR[LPDWLRQWRVROYHFRPSOH[SUREOHPV
FDWHJRULHVVHH)LJ • &ORXG &RPSXWLQJ WKH SURYLVLRQ RI FRPSXWLQJ
UHVRXUFHVYLDWKH,QWHUQHWFORXG
L  $,EDVHGRQPHWKRG
• %LRPHWULFV WKH XVH RI ELRORJLFDO FKDUDFWHULVWLFV IRU
7KLV FDWHJRU\ IRFXVHV RQ WKH YDULRXV PHWKRGV DQG
LGHQWLILFDWLRQDQGDXWKHQWLFDWLRQ
WHFKQLTXHV WKDW IRUP WKH EDVLV IRU WKH GHYHORSPHQW RI
DUWLILFLDO LQWHOOLJHQFH V\VWHPV 7KHVH LQFOXGH WKH • *HRWDJJLQJ DGGLQJ JHRJUDSKLF LQIRUPDWLRQ WR WKH
IROORZLQJ GDWD

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 212

 • 'LJLWDO WHFKQRORJLHV WKH XVH RI GLJLWDO PHGLD DQG LQWHUDFWLYLW\ SHUVRQDOL]DWLRQ DQG HIIHFWLYHQHVV RI WKH
WHFKQRORJLHVIRUDYDULHW\RISXUSRVHV HGXFDWLRQDO SURFHVV GHSHQGV RQ WKH DELOLW\ WR LQWHJUDWH
• ,QIRUPDWLRQ DQG &RPPXQLFDWLRQ 7HFKQRORJ\ WKHVH WHFKQRORJLHV LQWR OHDUQLQJ HQYLURQPHQWV WR FUHDWH
V\VWHPV IRU SURFHVVLQJ VWRULQJ DQG WUDQVPLWWLQJ VLJQLILFDQWLPSURYHPHQWVLQWKHOHDUQLQJHQYLURQPHQWVRI
LQIRUPDWLRQ IXWXUHJHQHUDWLRQV
 ,Q WKLV DUWLFOH ZH VHHN WR JR EH\RQG WKH JHQHUDO
FRQFHSWV RI DUWLILFLDO LQWHOOLJHQFH DQG IRFXV RQ VSHFLILF
DSSOLFDWLRQV RI WKLV WHFKQRORJ\ LQ WKH FRQWH[W RI VWXGHQW
OHDUQLQJ SUHVHQWLQJ FRQFUHWH UHVXOWV 7R WKLV HQG ZH
KDYH IROORZHG D PHWKRGLFDO DSSURDFK LQ VHOHFWLQJ
H[DPSOHV )URP WKH DEXQGDQFH RI OLWHUDWXUH LQGH[HG LQ
WKH :R66FRSXV GDWDEDVHV RYHU WKH ODVW ILYH \HDUV ZH
VHOHFWHG RQH VSHFLILF DSSOLFDWLRQ H[DPSOH IRU HDFK
FDWHJRU\ RI $, LQ ZKLFK D JLYHQ $, PHWKRG KDV EHHQ
HIIHFWLYHO\ LPSOHPHQWHG LQ WKH FRQWH[W RI HGXFDWLQJ
VWXGHQWVLQDVSHFLILFVXEMHFWRUVXEMHFWDUHD
$FUXFLDOFKDUDFWHULVWLFRIWKHVHVHOHFWHGVWXGLHVLVWKDW
WKH\ GR QRW MXVW VWLFN WR WKHRUHWLFDO FRQVLGHUDWLRQV EXW
DFWXDOO\FRQGXFWH[SHULPHQWVRQJURXSVRIVWXGHQWV(DFK
RI WKHVH VWXGLHV UHSUHVHQWV D XQLTXH H[SHULPHQWDO
DSSURDFK LQ ZKLFK DUWLILFLDO LQWHOOLJHQFH PHWKRGV ZHUH
XVHG DQG WHVWHG RQ D OLYH VDPSOH RI VWXGHQWV 7KHVH
H[SHULPHQWV VKRZ XV WKH FRQFUHWH LPSDFW DQG
HIIHFWLYHQHVV RI $, DSSOLFDWLRQV LQ WKH HGXFDWLRQDO
SURFHVV
,QWKLVZD\ZHKDYHEHHQDEOHWRRIIHUDGHHSHULQVLJKW
LQWR WKH SUDFWLFDO DSSOLFDWLRQ RI $, LQ HGXFDWLRQ DQG
SURYLGH FRQFUHWH H[DPSOHV WKDW VKRZ LWV UHDO EHQHILWV
2XU DLP LV WR SURYLGH DQ RYHUYLHZ RI KRZ WKHVH
LQQRYDWLYHWHFKQRORJLHVLQIOXHQFHDQGPRGLI\HGXFDWLRQDO
SURFHVVHV WR EHWWHU HTXLS VWXGHQWV IRU WKH FRQWHPSRUDU\
FKDOOHQJHVDQGUHTXLUHPHQWVRIVRFLHW\

,, $,,1('8&$7,21
:LWKLQ WKH ILHOGRI $, LQ HGXFDWLRQ $, WHFKQLTXHV DUH
 RIWHQ DSSOLHG WR WUDGLWLRQDO HGXFDWLRQDO DSSURDFKHV 7KLV
)LJXUH 9LVXDOL]DWLRQ RI WKH EUHDNGRZQ RI WKH FRQFHSW RI DUWLILFLDO DSSURDFK IRFXVHV RQ WKH DOLJQPHQW RU DXWRPDWLRQ RI
LQWHOOLJHQFHDFFRUGLQJWR>@LQWRPDLQFDWHJRULHV DOUHDG\ H[LVWLQJ HGXFDWLRQDO SUDFWLFHV DQG SURFHGXUHV
0XFK $, LQ HGXFDWLRQ KDV EHHQ GHYHORSHG ZLWK WKH
7KHUROHRIDUWLILFLDOLQWHOOLJHQFHLQHGXFDWLRQLVEDVHG LQWHQWLRQRIUHSODFLQJWHDFKHUVRUOLPLWLQJWKHLUUROHZKLFK
RQ WKH NQRZOHGJH DQG SHUVSHFWLYHV RIIHUHG E\ WKH ZLGH LV QRW DOZD\V FRQGXFLYH WR HIIHFWLYH OHDUQLQJ $OWKRXJK
VSHFWUXPRIPHWKRGVDQGDSSOLFDWLRQVRIWKLVWHFKQRORJ\ VXFK DQ DSSURDFK PD\ EH XVHIXO LQ VLWXDWLRQV ZKHUH
$, RSHQV XS D ZLGH UDQJH RI SRVVLELOLWLHV LQ HGXFDWLRQ WHDFKHUV DUH VKRUWVWDIIHG RU ZKHUH VWXGHQWV DUH
WKURXJK LWV DELOLW\ WRDQDO\]HGDWD LGHQWLI\ SDWWHUQV DQG JHRJUDSKLFDOO\ GLVWDQW LW PD\ XQGHUHVWLPDWH WKH XQLTXH
H[WUDFW YDOXDEOH LQIRUPDWLRQ IURP WKHP ,Q WKLV ZD\ LW VNLOOV DQG H[SHULHQFHV RI WHDFKHUV DQG QHJOHFW WKH VRFLDO
HQDEOHVWKHSHUVRQDOL]DWLRQRIOHDUQLQJH[SHULHQFHVEDVHG OHDUQLQJDQGOHDGHUVKLSQHHGVRIVWXGHQWV>@
RQ WKH QHHGV RI LQGLYLGXDO OHDUQHUV $, DOORZV WR +RZHYHU WKH UHDO YDOXH RI $, LQ HGXFDWLRQ LV WKDW LW
FXVWRPL]H FRQWHQW DQG SDFH OHDUQLQJ WKXV SURPRWLQJ VKRXOG QRW MXVWSDVVLYHO\ DXWRPDWLFDOO\ WHDFK VWXGHQWV RQ
LQGLYLGXDO GHYHORSPHQW DQG D EHWWHU XQGHUVWDQGLQJ RI FRPSXWHUV 5DWKHU LW VKRXOG EH D PHDQV RI RSHQLQJ XS
FRQFHSWV,QDGGLWLRQ$,FDQKHOSVWUHDPOLQHDVVHVVPHQW QHZ SRVVLELOLWLHV IRU WHDFKLQJ DQG OHDUQLQJ WKDW ZRXOG
SURFHVVHVDQGSURYLGHUDSLGIHHGEDFNLQFUHDVLQJVWXGHQW RWKHUZLVH EH GLIILFXOW WR DFKLHYH 7KHVH DSSURDFKHV FDQ
HQJDJHPHQW DQG PRWLYDWLRQ ,QWHJUDWLQJ $, LQWR WKH FKDOOHQJHWUDGLWLRQDOSHGDJRJLFDOPHWKRGVRUKHOSWHDFKHUV
OHDUQLQJ HQYLURQPHQW FDQ DOVR IRFXV RQ FUHDWLQJ EHFRPHPRUHHIIHFWLYH$,LQHGXFDWLRQFDQEHDWRROWKDW
LQWHUDFWLYH DQG HQJDJLQJ OHDUQLQJ WRROV WKDW LPSURYH KHOSVWHDFKHUVFXVWRPL]HLQVWUXFWLRQEDVHGRQWKHQHHGVRI
PHPRUDELOLW\ DQG XQGHUVWDQGLQJ RI WKH FXUULFXOXP ,Q LQGLYLGXDO VWXGHQWV DQG FUHDWH VWLPXODWLQJ DQG LQWHUDFWLYH
JHQHUDO DUWLILFLDO LQWHOOLJHQFH LQ HGXFDWLRQ RIIHUV QHZ OHDUQLQJ HQYLURQPHQWV WKDW WDNH LQWR DFFRXQW WKH XQLTXH
GLPHQVLRQVRIFXVWRPL]DWLRQLQWHUDFWLYLW\DQGHIILFLHQF\ DVSHFWV RI VRFLDO OHDUQLQJ DQG OHDGHUVKLS ,Q WKLV ZD\
WKDWDUHLPSDFWLQJWKHZD\ZHOHDUQDQGWHDFK DUWLILFLDO LQWHOOLJHQFH LQ HGXFDWLRQ GRHV QRW UHSODFH
7KLV DUWLFOH VHHNV WR SURYLGH DQ RYHUYLHZ RI KRZ WHDFKHUV EXW VXSSRUWV WKHP DQG DOORZV WKHP WR DFKLHYH
DUWLILFLDO LQWHOOLJHQFH FDQ FRQWULEXWH WR LPSURYLQJ WKH WKHLUHGXFDWLRQDOJRDOVPRUHHIIHFWLYHO\>@
HGXFDWLRQDOSURFHVV (DFK RIWKH PDLQ JURXSV RI $, KDV $ UHYLHZ RI $, OHDUQLQJ VFHQDULRV DQG UHODWHG
LWVRZQXQLTXHEHQHILWVDQGSRWHQWLDOLQHGXFDWLRQVRLWLV WHFKQLTXHV SUHVHQWHG LQ >@ PDNHV LW SRVVLEOH WR VHH KRZ
LPSRUWDQWWRXQGHUVWDQGKRZ WRFRPELQHDQGLPSOHPHQW WKHVH WHFKQLTXHV FDQ EH DSSOLHG HIIHFWLYHO\ WR GLIIHUHQW
WKHP DSSURSULDWHO\ IRU RSWLPDO UHVXOWV ,PSURYLQJ WKH OHDUQLQJSUDFWLFHV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 213

 L $VVHVVPHQWRIVWXGHQWVDQGVFKRROV V\QHUJHWLF FRPELQDWLRQ RI GLIIHUHQW WHFKQLTXHV PDNHV LW
,Q WKH DUHD RI VWXGHQW DQG VFKRRO HYDOXDWLRQ DGDSWLYH SRVVLEOH WR DFKLHYH FRPSOH[ DQG VRSKLVWLFDWHG VROXWLRQV
OHDUQLQJ PHWKRGV DUH EHFRPLQJ D WRRO IRU WDLORULQJ WKDW DGDSW WR WKH VSHFLILF QHHGV RI WKH OHDUQLQJ
HGXFDWLRQDO FRQWHQW WR WKH LQGLYLGXDO QHHGV DQG SDFH RI HQYLURQPHQWDQGLPSURYHWKHRYHUDOOHIIHFWLYHQHVVRIWKH
VWXGHQWV $ SHUVRQDOL]HG DSSURDFK WR OHDUQLQJ DOORZV OHDUQLQJSURFHVV
FXUULFXODWREHGHYHORSHGEDVHGRQDQDQDO\VLVRIVWXGHQW L  $,EDVHGRQPHWKRG
DFKLHYHPHQW DQG SUHIHUHQFHV HQVXULQJ WKDW OHDUQLQJ LV
HIIHFWLYH DQG HQJDJLQJ $FDGHPLF DQDO\WLFV WKHQ KHOSV • 0DFKLQH /HDUQLQJ ,Q HGXFDWLRQ PDFKLQH OHDUQLQJ
WUDFN VWXGHQW SHUIRUPDQFH EDVHG RQ KLVWRU\ DQG SDWWHUQV FDQEHXVHGWRFUHDWHSHUVRQDOL]HGOHDUQLQJSODQVIRU
SURYLGLQJ DQ LPSRUWDQW WRRO WR LPSURYH DVVHVVPHQW DQG VWXGHQWV EDVHG RQ WKHLU LQGLYLGXDO SHUIRUPDQFH DQG
SHUVRQDOL]DWLRQRILQVWUXFWLRQ SUHIHUHQFHV )RU H[DPSOH WKH V\VWHP FDQ LGHQWLI\
SDWWHUQV LQ VWXGHQW SHUIRUPDQFH DQG UHFRPPHQG
LL *UDQGLQJDQGHYDOXDWLRQRISDSHUVDQGH[DPV VSHFLILF H[HUFLVHV RU PDWHULDOV WR LPSURYH WKHLU
7HFKQLTXHVVXFKDVFRPSXWHUYLVLRQLPDJHUHFRJQLWLRQ ZHDNQHVVHV $V DQ LOOXVWUDWLYH H[DPSOH D VWXG\ >@
DQG SUHGLFWLRQ V\VWHPV DUH ILQGLQJ DSSOLFDWLRQV LQ XVHGPDFKLQHOHDUQLQJWRSUHGLFWWKHILQDOJUDGHVRI
DXWRPDWHG DVVHVVPHQW RI ZULWWHQ ZRUN DQG H[DPLQDWLRQV VWXGHQWV DW DQ (FXDGRULDQ XQLYHUVLW\ EDVHG RQ WKHLU
7H[W LGHQWLILFDWLRQ DQG YLVXDO FRQWHQW DQDO\VLV DOORZ IRU KLVWRULFDO SHUIRUPDQFH DQG JURXSLQJ VWXGHQWV ZLWK
UDSLG DQG FRQVLVWHQW DVVHVVPHQW IUHHLQJ XS WLPH IRU VLPLODUSDWWHUQVRIDFDGHPLFSHUIRUPDQFHLQDVLPLODU
LQGLYLGXDOPHQWRULQJE\WHDFKHUV ZD\ 6XEVHTXHQWO\ WKH PRVW VXLWDEOH VXSHUYLVHG
LLL 3HUVRQDOL]HGLQWHOOLJHQWWHDFKLQJ OHDUQLQJ DOJRULWKP ZDV FKRVHQ XVLQJ WKH LGHQWLILHG
'DWD PLQLQJ DQG %D\HVLDQ NQRZOHGJH LQWHUYHQWLRQV SDWWHUQVDVDEDVLV$VDUHVXOWRIVXFKUHVHDUFKWKH
DOORZ RQH WR DQDO\]H SDWWHUQV LQ WKH GDWD DQG XQFRYHU XQLYHUVLW\ DXWKRULWLHV DSSURYHG WKH FKDQJHV DQG WKH
VWXGHQWQHHGVDQGSUHIHUHQFHV,QWHOOLJHQWWXWRULQJV\VWHPV UHVXOW ZDV DQ LPSURYHPHQW LQ VWXGHQWV  JUDGHV DQG
FUHDWH LQWHUDFWLYH OHDUQLQJ PDWHULDOV EDVHG RQ VWXGHQW DFDGHPLF SHUIRUPDQFH DV ZHOO DV D UHGXFWLRQ LQ
UHVSRQVHV LPSURYLQJ LQWHUHVW DQG HQJDJHPHQW /HDUQLQJ VWXGHQW DEVHQWHHLVP UDWHV DW WKH HQG RI HDFK
DQDO\WLFV SURYLGH WUDFNLQJ RI OHDUQLQJ SURJUHVV DQG DFDGHPLFWHUP
LGHQWLI\DUHDVZKHUHVWXGHQWVPD\EHVWUXJJOLQJ • 'HHS OHDUQLQJ ,Q WKH FRQWH[W RI HGXFDWLRQ GHHS
LY 6PDUW6FKRRO OHDUQLQJFDQEHXVHGIRUH[DPSOHWRDQDO\]HDXGLR
UHFRUGLQJV GXULQJ FODVV WR LGHQWLI\ SDWWHUQV DQG
)DFLDO DQG VSHHFK UHFRJQLWLRQ $XJPHQWHG 5HDOLW\
HIIHFWLYHO\GHWHUPLQHVWXGHQWQHHGVDQGLQWHUHVWV$V
$5 9LUWXDO 5HDOLW\ 95  YLUWXDO ODEV DQG RWKHU
DQ H[DPSOH LV WKH VWXG\ >@ WKDW GHVFULEHV WKH
WHFKQRORJLHV DUH FRQWULEXWLQJ WR WKH FUHDWLRQ RI PRGHUQ GHYHORSPHQWRIDPRGHOXVLQJGHHSQHXUDOQHWZRUNV
DQG LQWHUDFWLYH OHDUQLQJ HQYLURQPHQWV 7KHVH WHFKQLTXHV
WR FODVVLI\ $UDELF VKRUW YRZHOV LQ HGXFDWLRQ 7KH
SURPRWH H[SHULHQWLDO OHDUQLQJ DQG DOORZ VWXGHQWV WR
UHVXOWLQJPRGHODFKLHYHGDWHVWLQJSUHFLVLRQRIPRUH
H[SORUH FRPSOH[ FRQFHSWVWKURXJK KDQGVRQ H[SHULHQFHV
WKDQ  LQ LGHQWLI\LQJ WKH H[DFW SURQXQFLDWLRQ RI
7KHVH DSSURDFKHV DUH YHU\ YDOLG LQ IRU H[DPSOH IOLJKW WKH $UDELF DOSKDEHW 7KLV UHVHDUFK PD\ KDYH
VLPXODWRUV RU QXFOHDU SRZHU SODQW FRQWURO VLPXODWLRQV
LPSOLFDWLRQV IRU WKH GHYHORSPHQW RI DSSOLFDWLRQV
ZKHUHVWXGHQWVFDQQRWEHHIIHFWLYHO\WUDLQHGLQUHDOZRUOG
DLPHG DW OHDUQLQJ $UDELF DQG WR DVVLVW VWXGHQWV
HQYLURQPHQWV
SUDFWLFLQJ SURQXQFLDWLRQ HYHQ ZKHQ D WHDFKHU LV
Y 2QOLQHDQGPRELOHUHPRWHHGXFDWLRQ XQDYDLODEOH
(GJH FRPSXWLQJ UHGXFHV GDWD SURFHVVLQJ GHOD\V DQG • 1DWXUDO ODQJXDJH SURFHVVLQJ 1DWXUDO ODQJXDJH
DOORZV DQ LPPHGLDWH UHVSRQVH WR VWXGHQW DFWLRQV 9LUWXDO SURFHVVLQJ 1/3  DSSOLFDWLRQV FDQ KHOS FUHDWH
SHUVRQDOL]HG DVVLVWDQWV DQG UHDOWLPH DQDO\WLFV DGG LQWHUDFWLYHOHDUQLQJHQYLURQPHQWVZKHUHOHDUQHUVFDQ
LQWHUDFWLYLW\ WR RQOLQH OHDUQLQJ SURYLGLQJ VWXGHQWV ZLWK D LQWHUDFWZLWKFKDWERWVRUYLUWXDOWXWRUVEDVHGRQWKHLU
SHUVRQDOL]HGDQGG\QDPLFOHDUQLQJH[SHULHQFH TXHVWLRQVDQGQHHGV$QH[DPSOHRIDVXEVWLWXWHIRU
IDFHWRIDFHLQWHUDFWLRQVEHWZHHQVWXGHQWVDQGWXWRUV
,,, 86(&$6(6,17+(('8&$7,216(&725 LVWKHVWXG\>@ZKLFKIRFXVHVRQFUHDWLQJFKDWERWV
:KHQ ORRNLQJ DW WKH UHODWLRQVKLS EHWZHHQ DUWLILFLDO ZLWK D QDWXUDO ODQJXDJH SURFHVVLQJ DSSURDFK 7KH
LQWHOOLJHQFH DQG HGXFDWLRQ WKH IRFXV LV RIWHQ RQ KRZ $, PRGHO KHOSV WR VXSSRUWKRXU RQOLQH OHDUQLQJ DQG
FDQHQULFKWKHHGXFDWLRQDOSURFHVV,QWKLVSDSHUKRZHYHU SURYLGH TXLFN DQVZHUV WR VWXGHQWV  TXHVWLRQV 7KH
ZH XVH WKH RSSRVLWH SHUVSHFWLYH ZKLFK VHHNV WR H[SORUH UHVXOWVVKRZHGWKDWWKHFKDWERWV\VWHPFUHDWHGEDVHG
KRZWKHGLIIHUHQWFDWHJRULHVRI$,DQGWKHLUVXEFDWHJRULHV RQ1/3LVHIIHFWLYHIRULQWHUDFWLYHOHDUQLQJDQGFDQ
FDQ EH HIIHFWLYHO\ DSSOLHG LQ HGXFDWLRQ 7KLV DSSURDFK EH LQWHJUDWHG LQWR GLIIHUHQW FRXUVHV 7KH HPEHGGHG
DOORZV XV WR H[SORUH WKH GLYHUVH ZD\V LQ ZKLFK $, FKDWERW V\VWHP NHHSV D FRPSOHWH KLVWRU\ RI
WHFKQRORJLHVFDQEHLQFRUSRUDWHGLQWRGLIIHUHQWDVSHFWVRI FRQYHUVDWLRQV ZLWK VWXGHQWVDOORZLQJ WKH XWLOL]DWLRQ
WKHHGXFDWLRQDOSURFHVV RIWKLVGDWDIRURWKHUVWXGHQWVDVVXSSOHPHQWDU\VWXG\
,QJHQHUDOWKLVDSSURDFKWRWKHUHODWLRQVKLSEHWZHHQ$, PDWHULDO
DQGHGXFDWLRQIRFXVHVRQLGHQWLI\LQJVSHFLILFPHWKRGVDQG • $UWLILFLDO QHXUDO QHWZRUN 7HDFKLQJ ZLWK DUWLILFLDO
WHFKQLTXHV WKDW FDQ EH DSSOLHG LQ HGXFDWLRQDO VHWWLQJV ,Q QHXUDOQHWZRUNVFDQLQYROYHFUHDWLQJVLPXODWLRQVDQG
WKLV ZD\ ZH FDQ JDLQ D EHWWHU XQGHUVWDQGLQJ RI KRZ $, PRGHOV WR KHOS VWXGHQWV EHWWHU XQGHUVWDQG FRPSOH[
FDQ FRQWULEXWH WR FUHDWLQJ D VWLPXODWLQJ HIIHFWLYH DQG FRQFHSWV $Q H[DPSOHRI WKHXVH RI DUWLILFLDO QHXUDO
SHUVRQDOL]HG OHDUQLQJ HQYLURQPHQW WKDW LV UHVSRQVLYH WR QHWZRUNV LQ HGXFDWLRQ LV WKH VWXG\ >@ ZKLFK
VWXGHQWV QHHGVDQGPRGHUQSHGDJRJLFDODSSURDFKHV,WLV IRFXVHV RQ LGHQWLI\LQJ VWXGHQWV  OHDUQLQJ VW\OHV LQ
QDWXUDOWKDWFDVHVWXGLHVLQDGGLWLRQWRWKHPDLQFDWHJRU\ 67(0 6FLHQFH 7HFKQRORJ\ (QJLQHHULQJ DQG
PD\LQFRUSRUDWHWHFKQLTXHVIURPRWKHUVXEFDWHJRULHV7KLV 0DWKHPDWLFV  XVLQJ DUWLILFLDO QHXUDO QHWZRUNV 7KH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 214

 (GXFDWLRQ EDODQFH JDPH ZLWK WKH )HOGHU6LOYHUPDQ IX]]\ORJLFLQHYDOXDWLQJFRPSOH[IDFWRUVWKDWDIIHFW
OHDUQLQJVW\OHPRGHOKHUHFROOHFWVGDWDIURPVWXGHQWV  VWXGHQWSODFHPHQWLQWKHZRUNHQYLURQPHQW
SRUWIROLRVDQGXVHVLWWRLGHQWLI\OHDUQLQJVW\OHVLQD
LL  $,EDVHGRQLQGXVWU\DSSOLFDWLRQV
VXSSRUW YHFWRU PDFKLQH 5HVHDUFK IRFXVHV RQ
SRVVLEOH GLIIHUHQFHV LQ PRWLYDWLRQ HPRWLRQV • ,QGXVWU\,QWKHILHOGRIHGXFDWLRQWKHFRQFHSWRI
OHDUQLQJ RXWFRPHV DQG SUREOHP VROYLQJ EHWZHHQ ,QGXVWU\  FDQ EH XVHG WR FUHDWH LQWHUDFWLYH
VWXGHQWVZLWKGLIIHUHQWOHDUQLQJVW\OHV OHDUQLQJSURJUDPVWKDWDGDSWWRVWXGHQWQHHGVDQGDUH
IXOO\GLJLWDODQGDXWRPDWHG$QH[DPSOHLVWKHVWXG\
• 7H[WDQGGDWDPLQLQJ7H[WDQGGDWDPLQLQJFDQ
>@ZKLFKVKRZVKRZWKH,QGXVWU\FRQFHSWFDQ
EH XVHG WR DQDO\]H VWXGHQW WH[WV DQG JDLQ LPSRUWDQW
EH XVHG LQ WRXULVP HGXFDWLRQ 7RXULVP VWXGHQWV FDQ
LQVLJKWV LQWR WKHLU SHUIRUPDQFH DQG SURJUHVV $V DQ
H[SORUHWHFKQRORJLHVVXFKDVDXWRPDWLRQYLUWXDODQG
H[DPSOH D VWXG\ >@ SURSRVHV DQ LQWHOOLJHQW
DXJPHQWHG UHDOLW\ URERWLFV DQG ,&7 LQ WKH FRQWH[W
GLVWDQFH HGXFDWLRQ VHUYLFH PRGHO EDVHG RQ ZHE
RIVRFDOOHG7RXULVP7HDFKLQJIRFXVHVRQVNLOOV
PLQLQJWRVROYHWKHSUREOHPRIPRGHRIOHDUQLQJLQ
VXFK DV VRFLDO PHGLD GDWD PLQLQJ PDFKLQH OHDUQLQJ
QHWZRUNHG GLVWDQFH HGXFDWLRQ 7KH SDSHU GHVFULEHV
GDWD DQDO\VLV DQG GDWD GHVLJQ IRU WRXULVP
WKH GDWD SURFHVVLQJ SURFHGXUH LQFOXGLQJ WKH
FRPPXQLFDWLRQ V\VWHPV 6WXGHQWV OHDUQ WR FUHDWH
VHOHFWLRQ DQG LPSOHPHQWDWLRQ RI WKH PLQLQJ
LQWHOOLJHQW V\VWHPV WKDW FRPELQH WHFKQRORJLFDO
DOJRULWKP WR REWDLQ D VHW RI UXOHV IRU SURYLGLQJ
NQRZOHGJH ZLWK WRXULVP HQDEOLQJ WKHP WR UHVSRQG
SHUVRQDOL]HG JXLGDQFH WR VWXGHQWV ,Q WKLV ZD\ WKH
WR QHZ FKDOOHQJHV DQG DGDSW WR WHFKQRORJ\GULYHQ
V\VWHP FDQ G\QDPLFDOO\ JHQHUDWH SDJHV RI LQWHUHVW
FRQVXPHUV 7KLV DSSURDFK WR HGXFDWLRQ KHOSV
IRU XVHUV &OXVWHULQJ WHFKQRORJ\ DQG WKH DVVRFLDWLRQ
VWXGHQWV DFTXLUH WKH QHFHVVDU\ VNLOOV DQG SURPRWH
UXOH DOJRULWKP DUH XVHG WR PLQH DQG DQDO\]H WKH
WHFKQRORJLFDO FKDQJH LQ WRXULVP DQG KRVSLWDOLW\
OHDUQLQJ FKDUDFWHULVWLFV RI VWXGHQWV LQ WKH
HGXFDWLRQ
UHFRPPHQGDWLRQ VWUDWHJ\ IRU QHWZRUN OHDUQLQJ 7KH
H[SHULPHQWDO DQDO\VLV YDOLGDWHV WKDW WKH GLVWDQFH • ,QGXVWULDO ,QWHUQHW RI 7KLQJV ,,R7  ,Q HGXFDWLRQ
OHDUQLQJ PRGHO ZKLFK LV EDVHG RQ ZHE PLQLQJ ,,R7GHYLFHVFDQEHXVHGWRFUHDWHVLPXODWLRQVRIUHDO
VXSSRUWV VWXGHQWV LQ JDLQLQJ NQRZOHGJH DQG LQGXVWULDOSURFHVVHVDQGDOORZVWXGHQWVWRH[SHULPHQW
IRUHFDVWLQJWKHGLVWDQFHOHDUQLQJWUDMHFWRU\EDVHGRQ DQGH[SORUHWKHPLQUHDOWLPH$QH[DPSOHLVDVWXG\
LQGLYLGXDO VWXGHQW FKDUDFWHULVWLFV 7KLV LQ WXUQ >@ H[SORULQJ WKH XVH RI ,,R7 LQ HGXFDWLRQ WR
HQKDQFHVWKHHIIHFWLYHQHVVRIWKHOHDUQLQJSURFHVV LPSURYHWKHWHDFKLQJRIDGYDQFHGFRQWUROWHFKQLTXHV
VXFK DV PRGHO SUHGLFWLYH FRQWURO 03&  WR
• $OJRULWKPLF WUDGLQJ $OJRULWKPLF WUDGLQJ FDQ EH
XQGHUJUDGXDWH VWXGHQWV ,,R7 DOORZV VWXGHQWV WR
XVHG LQ HFRQRPLF HGXFDWLRQ WR VLPXODWH YDULRXV
H[SHULPHQW ZLWK UHDOWLPH PRQLWRULQJ RI SK\VLFDO
ILQDQFLDO VLWXDWLRQV DQG GHFLVLRQV (GXFDWLRQ DQG
REMHFWVDQGWKHLUGLVWULEXWHGFRQWURO6WXGHQWVFDQXVH
WUDLQLQJLQDOJRULWKPLFWUDGLQJFDQLQFOXGHVLPXODWLQJ
VLQJOHERDUG FRPSXWHUV WR LPSOHPHQW RSWLPL]DWLRQ
UHDO WUDGLQJ HQYLURQPHQWV LQWHUDFWLQJ ZLWK OLYH
VROXWLRQV ZLWKLQ WKH ,,R7 EURDGHQLQJ WKHLU
ILQDQFLDO H[FKDQJHV DQG OHDUQLQJ GLIIHUHQW WUDGLQJ
H[SHULHQFH LQ FRQWUROOLQJ FRPSOH[ SURFHVVHV 7KH
VWUDWHJLHV7KH%ULVWRO6WRFN([FKDQJH %6( >@LV
FRPELQDWLRQRIWHDFKLQJ03&ZLWKLQ,,R7LPSURYHV
D VLPXODWRU WKDW DOORZV VWXGHQWV WR VWXG\ WKH
OHDUQLQJ RYHU FRQYHQWLRQDO PHWKRGV LQYROYLQJ
ZRUNLQJV RI ILQDQFLDO H[FKDQJHV DQG H[SHULPHQW
VLPXODWLRQV DQG VLPSOH PLFURFRQWUROOHUV 7KLV
ZLWK DOJRULWKPLF WUDGLQJ V\VWHPV 7KLV WRRO LV D
PHWKRGDOVRDOORZVIRUFRGHWHVWLQJDQGH[SHULPHQWV
XVHIXOWRROIRUHGXFDWLRQDQGUHVHDUFKRQDOJRULWKPLF
ZLWKGLIIHUHQWVFHQDULRV)XUWKHUPRUH,,R7WHDFKLQJ
WUDGLQJDQGKHOSVJUDGXDWHVJDLQSUDFWLFDOH[SHULHQFH
LV PRUH HIILFLHQW DQG FRVWHIIHFWLYH FRPSDUHG WR
LQWKHVWRFNPDUNHWV%6(DOORZVIRUGLIIHUHQWRUGHU
WUDGLWLRQDO DSSURDFKHV DOORZLQJ IRU ZLGHU
W\SHV DQG HQDEOHV VWXGHQWV WR H[SHULPHQW ZLWK
DYDLODELOLW\DQGKDQGVRQH[HUFLVHV
GLIIHUHQW WUDGLQJ VWUDWHJLHV 7KLV VLPXODWRU DOORZV
\RX WR VWXG\ DQG H[SHULPHQW ZLWK FRPSOH[ WUDGLQJ • 5RERWLFV FXUULFXOD FDQ LQWHJUDWH URERWLFV IRU
VFHQDULRVDQGDQDO\]HYDULRXVDVSHFWVRIDOJRULWKPLF KDQGVRQ H[HUFLVHV DQG GHPRQVWUDWLRQV DOORZLQJ
WUDGLQJHIIHFWLYHO\ VWXGHQWV WR JDLQ KDQGVRQ H[SHULHQFH ZLWK
SURJUDPPLQJ DQG FRQWUROOLQJ URERWV EXW LW LV DOVR
• )X]]\ ORJLF DQDO\VLV )X]]\ ORJLF FDQ EH XVHG LQ
DSSOLFDEOH WR RWKHU LQGXVWULHV $Q H[DPSOH LV WKH
HGXFDWLRQ WR VROYH SUREOHPV ZKHUH WKHUH LV
VWXG\ >@ RI WKH 6PDUW 7UDVKFDQ %URWKHUV
XQFHUWDLQW\ DQG D ODFN RI DFFXUDWH GDWD VXFK DV
HGXFDWLRQDOURERWFRQFHSW7KH6PDUW7UDVKFDQURERW
DVVHVVLQJ VWXGHQW SHUIRUPDQFH LQ PXOWLSOH DVSHFWV
LV XVHG WR WHDFK HQYLURQPHQWDO HGXFDWLRQ DQG
$Q H[DPSOH LV D VWXG\ >@ WKDW LQYHVWLJDWHV WKH
SURJUDPPLQJ 7KH URERW DQDO\]HV LWHPV IRU
UHODWLRQVKLS EHWZHHQ DFDGHPLF DQG QRQDFDGHPLF
UHF\FODELOLW\ SURYLGHV IHHGEDFN WR VWXGHQWV DQG
SHUIRUPDQFH RI PDQDJHPHQW VWXGHQWV DQG WKHLU
DOORZV SURJUDPPLQJ WKURXJK D YLVXDO LQWHUIDFH %\
SODFHPHQW LQ WKH MRE PDUNHW XVLQJ D IX]]\ ORJLF
DQDO\]LQJ REMHFWV LW GHWHUPLQHV ZKHWKHU WKH\ DUH
DSSURDFK %\ DQDO\]LQJ KLVWRULFDO SODFHPHQW GDWD
UHF\FODEOH RU QRW SURYLGLQJ IHHGEDFN WR VWXGHQWV
WKH LPSDFW RI YDULRXV SHUIRUPDQFH IDFWRUV RQ WKH
7KHYLVXDOSURJUDPPLQJLQWHUIDFHDOORZVWKHURERWWR
ILQDO SODFHPHQW RI VWXGHQWV LV GHWHUPLQHG 7KLV
EHSURJUDPPHGHYHQIRUROGHUVWXGHQWV7KHV\VWHP
DSSURDFK UHYHDOV WKDW DQ LQFUHDVH LQ QRQDFDGHPLF
LV DIIRUGDEOH DQG DFWV DV D WRRO IRU HFRORJ\ DQG
SHUIRUPDQFH PD\ KDYH D JUHDWHU LPSDFW RQ
67(0 HGXFDWLRQ 7KLV DSSURDFK DOORZV IRU WKH
SODFHPHQW WKDQ DQ LQFUHDVH LQ DFDGHPLF
LQWHJUDWLRQ RI HQYLURQPHQWDO YDOXHV IURP DQ HDUO\
SHUIRUPDQFH7KHFRQFOXVLRQRIWKHVWXG\KLJKOLJKWV
DJH
WKH VLJQLILFDQW DGYDQWDJHV RI IX]]\ ORJLF LQ
HYDOXDWLQJ VWXGHQW SHUIRUPDQFH FRPSDUHG WR WKH • :HDUDEOH WHFKQRORJ\ (GXFDWLRQDO DSSOLFDWLRQV FDQ
WUDGLWLRQDO PHWKRG DQG HPSKDVL]HV WKH SRWHQWLDO RI XVH ZHDUDEOH WHFKQRORJ\ WR PRQLWRU VWXGHQWV 

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 215

 SK\VLRORJLFDO SDUDPHWHUV DQG SURYLGH IHHGEDFN ,R705SODWIRUPKDVDQDGYDQWDJHLQXQGHUVWDQGLQJ
EDVHGRQWKHLUSK\VLFDOFRQGLWLRQ$QH[DPSOHLVWKH ERWK WKH GHVLJQ SULQFLSOHV DQG WKH WLPH VSHQW RQ
VWXG\ >@ IRFXVHG RQ HGXFDWLRQ XVLQJ ZHDUDEOH WDVNV &RPSDULVRQ ZLWK WKH RWKHU WZR SODWIRUPV
WHFKQRORJLHV LQ EDGPLQWRQ 7KH VWXG\ VKRZHG WKDW FRQILUPHG WKH HIIHFWLYHQHVV RI XVLQJ WKH ,R705
WKHSURSRVHG:,6(5PRGHOZLWK PXOWLSOHIHHGEDFN SODWIRUP 7KH WUDGLWLRQDO PHWKRG XVHV SHUIRUPDQFH
ORRSVFDQVLJQLILFDQWO\LPSURYHVWXGHQWSHUIRUPDQFH VLPXODWLRQ VRIWZDUH WR VKRZ VWXGHQWV WKH LPSDFW RI
7KLV PRGHO SURYLGHV VWXGHQWV ZLWK YLVXDO WKHLU GHVLJQ VWUDWHJLHV 8VLQJ WKH ,R705 SODWIRUP
LQIRUPDWLRQDO DQG YHUEDO IHHGEDFN RQ WKH DOORZVUHDOVHQVRUGDWDWREHYLVXDOL]HGLPPHGLDWHO\
FRUUHFWQHVV RI WKHLU PRYHPHQWV 7KH H[SHULPHQWDO LQ UHDO WLPH 6WXGHQWV FDQ WUDFN KRZ WKHLU GHVLJQ
JURXS WKDW XVHG WKLV PRGHO SHUIRUPHG EHWWHU LQ WKH FKDQJHVDIIHFWLQGRRUOLJKWLQJ
DFTXLVLWLRQ RI EDGPLQWRQ VNLOOV FRPSDUHG WR WKH • 6RIW &RPSXWLQJ WKH XVH RI VRIW FRPSXWLQJ FDQ
FRQWURO JURXS WKDW XVHG WUDGLWLRQDO PHWKRGV 7KH HQDEOH WKH VROXWLRQ RI FRPSOH[ SUREOHPV LQ
PRGHO HVSHFLDOO\ KHOSHG VWXGHQWV ZLWK ORZHU LQLWLDO HGXFDWLRQDO HQYLURQPHQWV VXFK DV D SHUVRQDOL]HG
VNLOO OHYHOV 7KLV VWXG\ KLJKOLJKWV WKH EHQHILWV RI DSSURDFKWROHDUQLQJEDVHGRQ PXOWLIDFWRUDQDO\VLV
YLVXDO IHHGEDFN REWDLQHG E\ ZHDUDEOH WHFKQRORJLHV $JDLQDVSHFLILFH[DPSOHLVJLYHQIURPWKHILHOGRI
DQGWKHLULPSRUWDQFHLQLPSURYLQJPRWRUVNLOOV7KH VWXGHQW WHVWLQJ ZKHUH WKH VWXG\ >@ IRFXVHV RQ
:,6(5 PRGHOSURYLGHV D YDOXDEOH WRRO IRU FRDFKHV DQDO\]LQJ WKH SHUIRUPDQFH RI HQJLQHHULQJ VWXGHQWV
DQGSK\VLFDOHGXFDWLRQWHDFKHUVLQSDUWLFXODU RQ H[DPV ZKHQ DQVZHULQJ TXHVWLRQV IURP GLIIHUHQW
• 6PDUW SURGXFWV FUHDWLQJ DQG H[SORULQJ VPDUW SDUWV RI WKH FXUULFXOXP ,W LGHQWLILHV ZKLFK SDUWV RI
SURGXFWV DOORZV VWXGHQWV WR H[SORUH KRZ DUWLILFLDO WKH V\OODEXV DUH IUHTXHQWO\ VNLSSHG E\ VWXGHQWV DQG
LQWHOOLJHQFH LQWHJUDWHV LQWR HYHU\GD\ WKLQJV DURXQG ZKLFK SDUWV DUH IRFXVHG RQ $V D QRYHO DSSURDFK
XV$QH[DPSOHLVWKHVWXG\>@ZKLFKIRFXVHVRQ VRIW FRPSXWLQJ LV XVHG WR JURXS VWXGHQWV DFFRUGLQJ
WKHLQQRYDWLYHXVHRIVPDUWSURGXFWVLQWKHHGXFDWLRQ WRWKHLUSHUIRUPDQFHDQGOHDUQLQJXQLWV7KHUHVHDUFK
RI IDPLO\ KHDOWK 7KH DLP LV WR FUHDWH DQ LQWHJUDWHG ZDV FDUULHG RXW RQ  VWXGHQWV IURP ILYH VXEMHFWV
V\VWHP WKDW RIIHUV YDULRXV VHUYLFHV DQG SRVVLELOLWLHV WKDWLVUHFRUGV7KHDSSOLFDWLRQRIWKLVDSSURDFK
LQIDPLO\KHDOWKFDUHDQGFRPELQHV=LJ%HHDQG5),' LQYROYHG D FRPELQDWLRQ RI PDQXDO SURFHGXUHV DQG
WHFKQRORJ\ LQ ,R7 WHFKQRORJ\ ZLWK DUWLILFLDO VRIW FRPSXWLQJ WHFKQLTXHV HPSOR\LQJ D VLPSOH
LQWHOOLJHQFH EDVHG RQ WKH FKDUDFWHULVWLFV RI $, QHXUDOQHWZRUNIRULPSOHPHQWDWLRQ7KHUHVXOWVZHUH
WHFKQRORJLHV 7KH UHVXOWV RI WKH H[SHULPHQW VKRZ D FRPSDUHG DQG SRVLWLYH GHYLDWLRQV DUH DQ LQGLFDWLRQ
 LPSURYHPHQW LQ WKH SHUIRUPDQFH RI VPDUW RIWKHVXFFHVVRIWKHVRIWFRPSXWLQJDSSURDFK
SURGXFWV RYHU WKH WUDGLWLRQDO KHDOWK HGXFDWLRQ • &ORXG &RPSXWLQJ 5HPRWH DFFHVV WR FRPSXWLQJ
PHWKRG 7KLV UHVHDUFK KLJKOLJKWV WKH SRWHQWLDO RI UHVRXUFHVDQGVWRUDJHDOORZVVWXGHQWVDQGWHDFKHUVWR
VPDUW SURGXFWV WR LPSURYH WKH HIIHFWLYHQHVV RI DFFHVV HGXFDWLRQDO PDWHULDOV IURP GLIIHUHQW GHYLFHV
IDPLO\KHDOWKFDUHHGXFDWLRQ DQG HQDEOHV FROODERUDWLRQ DQG FRQWHQW VKDULQJ
LLL $,EDVHGRQV\VWHPDSSOLFDWLRQV EHWZHHQ VWXGHQWV DQG WHDFKHUV $V DQ H[DPSOH WKH
VWXG\>@IRFXVHVRQDQRQOLQHOHDUQLQJPRGHOIRUD
• $XWRPDWLRQ LQ HGXFDWLRQ DXWRPDWLRQ FDQ KHOS PDUNHWLQJ GLVFLSOLQH7KHRQOLQH OHDUQLQJ PRGHO IRU
PDQDJHUHSHWLWLYHWDVNVVXFKDVJUDGLQJWHVWVIUHHLQJ WKH PDUNHWLQJ PDMRU ZDV RSWLPL]HG E\ XVLQJ FORXG
XS WLPH IRU WHDFKHUV WR LQWHUDFW ZLWK VWXGHQWV
VHUYHUV DQG UHOHYDQW KDUGZDUH GHYLFHV 7KH RQOLQH
LQGLYLGXDOO\ $Q H[DPSOH LV D VWXG\ E\ >@ WKDW
OHDUQLQJPHWKRGZDVGHWHUPLQHGDQGWKHFRQWHQWIRU
ORRNVDWWKHLPSOHPHQWDWLRQRIDQDXWRPDWHG(QJOLVK RQOLQH OHDUQLQJ ZDV VHOHFWHG 2QOLQH OHDUQLQJ
ODQJXDJH DVVHVVPHQW DQG FRUUHFWLRQ V\VWHP LQ DQ UHVRXUFHV IRU WKH PDUNHWLQJ VSHFLDOL]DWLRQ ZHUH
RQOLQH HQYLURQPHQW 7KH V\VWHP DOORZV RQOLQH FROOHFWHG DQG VKDULQJ RI WKHVH UHVRXUFHV ZDV GRQH
(QJOLVK WHVWV DFFRUGLQJ WR KLJKHU SURILFLHQF\
XVLQJ FORXG FRPSXWLQJ WHFKQRORJ\ 2QOLQH OHDUQLQJ
VWDQGDUGVDQGLQFOXGHVVFRUHPDQDJHPHQWJURXSWHVW DFWLYLWLHV DQG LQWHUDFWLYH IHDWXUHV ZHUH GHVLJQHG IRU
PDQDJHPHQWDQGRQOLQHWHVWDGPLQLVWUDWLRQIHDWXUHV PDUNHWLQJ VSHFLDOL]DWLRQ 'XULQJ WKH PRGHO WHVW WKH
([SHULPHQWVLQFOXGHDJHQHWLFDOJRULWKPWRRSWLPL]H HIIHFWLYHQHVV DQG SHUIRUPDQFH RI WKH PRGHO ZHUH
WHVW SDSHUV 7KLV VWXG\ DQDO\]HV WKH ZULWWHQ SDSHUV
GHPRQVWUDWHG ZLWK WKH VXFFHVV UDWH RI WKH
DQG H[HUFLVHV RI WKLUG\HDU FROOHJH VWXGHQWV 7KH IXQFWLRQDOLW\UHDFKLQJZKLFKPHWWKHPRGHO V
ILQGLQJV VKRZ WKDW HOHFWURQLFDVVHVVPHQW KDV D KLJK UHTXLUHPHQWV DQG WKH UHVSRQVH WLPHV DQG PHPRU\
FRUUHODWLRQZLWKPDQXDODVVHVVPHQWDQGWKHSURSRVHG FRQVXPSWLRQ ZHUH ZLWKLQ DFFHSWDEOH YDOXHV ,Q WKLV
V\VWHP SURYLGHV YDOXDEOH LQIRUPDWLRQ IRU VWXGHQWV
ZD\FORXGFRPSXWLQJWHFKQRORJ\FRQWULEXWHGWRWKH
DQGWHDFKHUVWRDQDO\]HDQGLPSURYH
GHYHORSPHQW RI DQ HIIHFWLYH RQOLQH OHDUQLQJ PRGHO
• ,QWHUQHW RI 7KLQJV ,R7  XVLQJ WKH ,R7 LQ WKH IRUWKHPDUNHWLQJPDMRU
FODVVURRP FDQ PHDQ FUHDWLQJ DQ LQWHUDFWLYH • %LRPHWULFV WKH XVH RI ELRPHWULFV FDQ HQKDQFH
HQYLURQPHQW ZKHUH GHYLFHV DQG WHDFKLQJ DLGV VHFXULW\ DQG DXWKHQWLFDWLRQ ZKHQ DFFHVVLQJ OHDUQLQJ
LQWHUDFW DQG HQKDQFH WKH OHDUQLQJ H[SHULHQFH 7KH
V\VWHPVDQGLQIRUPDWLRQ&29,'KDVDFFHOHUDWHG
FDVHVWXG\>@IRFXVHVRQFRPELQLQJ,R7DQGPL[HG
WKH QHHG IRU VHFXUH PHFKDQLVPV IRU DXWKHQWLFDWLQJ
UHDOLW\LQWKHWHDFKLQJRIDUFKLWHFWXUDOGHVLJQZLWKD DQG PRQLWRULQJ RQOLQH OHDUQHUV  7KH VWXG\ >@
IRFXVRQEXLOGLQJSHUIRUPDQFH7KLVSODWIRUPDOORZV GHWDLOV D VROXWLRQ EDVHG RQ WKH DXWKHQWLFDWLRQ RI
VWXGHQWV WR LQWXLWLYHO\ REVHUYH UHDOWLPH VHQVRU\
GLIIHUHQW ELRPHWULF WHFKQRORJLHV DQG DXWRPDWHG
DQDO\VLV RI OLJKWLQJ LQ LQWHUDFWLYH +ROR/HQV JODVVHV
PRQLWRULQJ ERWK V\VWHP ZRUNIORZ DQG $,
6WXGHQWV FDQ REVHUYH OLJKWLQJ IOXFWXDWLRQV ZKLOH DOJRULWKPV  7KH DXWKRUV SURSRVHG D UHOLDEOH DQG
PDQXDOO\ DGMXVWLQJ WKH HTXLSPHQW PRGHOV IRU VFDODEOH ELRPHWULF DXWKHQWLFDWLRQ DQG DXWRPDWLF
VKDGLQJ 7KH UHVXOWV DQG DQDO\VLV VKRZHG WKDW WKH
PRQLWRULQJ V\VWHP 7KH WHVW UHVXOWV VKRZHG KLJK

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 216

 DFFXUDF\ ,Q DGGLWLRQ VWXGHQWV FRQVLGHU LW IDLU WR FRQWULEXWLRQRIWKHZRUNZDVWRUHILQHWKHDVVLJQPHQW
KDYHDQ\W\SHRIELRPHWULFUHFRJQLWLRQWHFKQRORJ\WR RI WKH OHDUQLQJ VW\OH WR HDFK VWXGHQW 7KH UHVHDUFK
PRQLWRU FKHDWLQJ 7KH SULPDU\ UHDVRQ ZK\  RI DOVR DGGUHVVHG WKH LGHQWLILFDWLRQ RI GRPLQDQW
WKH VXUYH\HG VWXGHQWV ILQG WKH LQFRUSRUDWLRQ RI OHDUQLQJ VW\OHV DQG WKHDSSOLFDWLRQ RI WKHVH ILQGLQJV
ELRPHWULFDXWKHQWLFDWLRQDQGLQVWUXFWLRQDOPRQLWRULQJ WRWKHSODQQLQJRIGLVWDQFHHGXFDWLRQFRXUVHV
KLJKO\ IDYRUDEOH LV WKHLU UHFRJQLWLRQ RI WKH LVVXH RI
FKHDWLQJDPRQJVRPHVWXGHQWVZKLFKWKH\FRQVLGHU 7KH H[DPSOHV DERYH LGHQWLI\ GLIIHUHQW ZD\V LQ ZKLFK
XQIDLUWRRWKHUV DUWLILFLDOLQWHOOLJHQFHFDQFRQWULEXWHWRWKHGHYHORSPHQWRI
PRGHUQOHDUQLQJHQYLURQPHQWV7KHVHDSSOLFDWLRQVHQDEOH
• *HRWDJJLQJ $Q H[DPSOH LV DGGLQJ JHRJUDSKLF
SHUVRQDOL]HG OHDUQLQJ LPSURYHG DQDO\VLV RI VWXGHQW
LQIRUPDWLRQ WR WKH GDWD DOORZLQJ VWXGHQWV WR EHWWHU
SHUIRUPDQFH DQG WKH FUHDWLRQ RI LQWHUDFWLYH DQG
XQGHUVWDQG VSHFLILF ORFDWLRQV DQG WKHLU LPSDFW
VWLPXODWLQJ OHDUQLQJ HQYLURQPHQWV $UWLILFLDO LQWHOOLJHQFH
*HRWDJJLQJ FDQ DOVR EH XVHG IRU RWKHU HGXFDWLRQDO
KDVWKHSRWHQWLDOWRWUDQVIRUPWKHZD\OHDUQLQJWDNHVSODFH
SXUSRVHV 7KH VWXG\ >@ IRFXVHV RQ LPSURYLQJ
DQGFRQWULEXWHWRLWVHIILFLHQF\DQGHIIHFWLYHQHVV
VFKRRO RXWFRPHV E\ FRPELQLQJ D GXDO DQDO\VLV
LGHQWLI\LQJ VFKRROV DQG DUHDV EHVW VXLWHG IRU 7KH SRWHQWLDO RI DUWLILFLDO LQWHOOLJHQFH PHWKRGV LQ
LQWHUYHQWLRQVDQGLGHQWLI\LQJNH\IDFWRUVLQIOXHQFLQJ HGXFDWLRQ LV QRW RQO\ LQ OHDUQLQJ EXW DOVR LQ WKH
OHDUQLQJ ,Q WKLV UHVHDUFK ERWK GHVFULSWLYH DQG DGPLQLVWUDWLRQRIHGXFDWLRQDOLQVWLWXWLRQVDQGLQWKHZRUN
LQIHUHQWLDO VWDWLVWLFDO DQDO\VHV DUH XVHG WR UHFRJQL]H RIWHDFKHUV>@
DUHDV WKDW KDYH SRWHQWLDO IRU HQKDQFHPHQW ,W XVHV ,QWKHILHOGRIDGPLQLVWUDWLRQDUWLILFLDOLQWHOOLJHQFHFDQ
JHRWDJJLQJ ER[ DQDO\VLV FKLVTXDUH WHVW DQG EULQJ D QXPEHU RI EHQHILWV WKDW FDQ VLJQLILFDQWO\
PXOWLSOHOLQHDUUHJUHVVLRQDQDO\VLVRQGDWDIURP VWUHDPOLQH WKH HQWLUH HGXFDWLRQ SURFHVV )DVWHU H[HFXWLRQ
SXEOLFVFKRROVLQ'HOKL,QGLD7KHUHVXOWVVKRZWKDW RI DGPLQLVWUDWLYH WDVNV WKDW WDNH XS D ORW RI LQVWUXFWRUV 
WKHSHUIRUPDQFHRIDVFKRROGHSHQGVRQLWVORFDWLRQ WLPHLVRQHRIWKHP$UWLILFLDOLQWHOOLJHQFHFDQEHDVVLJQHG
DQG WKH DQDO\VLV LGHQWLILHV VLJQLILFDQW IDFWRUV7KHVH WRJUDGHH[DPVDQGSURYLGHIHHGEDFNDOORZLQJWHDFKHUVWR
IDFWRUV LQFOXGH WKH H[LVWHQFH RI D SOD\JURXQG IRFXV PRUH RQ WKH DFWXDO WHDFKLQJ DQG GHYHORSPHQW RI
LQVSHFWLRQVODQJXDJHWHDFKLQJVW\OHOLEUDU\VWXGHQW VWXGHQWV
DWWHQGDQFH DQG WKH SURSRUWLRQ RI IHPDOH WHDFKHUV $QRWKHUDGYDQWDJHDVPHQWLRQHGDERYHLVWKHDELOLW\WR
7KLVDSSURDFKLQFOXGHVERWKRSHUDWLRQDODQGVWUDWHJLF LGHQWLI\WKHOHDUQLQJVW\OHVDQGSUHIHUHQFHVRIHDFKRIWKH
DVSHFWVWKDWDIIHFWHGXFDWLRQ VWXGHQWV ,Q WKLV ZD\ D SHUVRQDOL]HG FXUULFXOXP FDQ EH
• 'LJLWDO WHFKQRORJ\ LQWHJUDWLQJ GLJLWDO PHGLD DQG GHVLJQHG WKDW PRUH HIIHFWLYHO\ WDNHV LQWR DFFRXQW WKH
WRROVLQWR WHDFKLQJHQDEOHVLQWHUDFWLYH DQGHQJDJLQJ LQGLYLGXDOQHHGVRIWKHVWXGHQWV$UWLILFLDOLQWHOOLJHQFHFDQ
OHDUQLQJH[SHULHQFHV$QH[DPSOHLVDVWXG\>@WKDW DOVR DVVLVW LQVWUXFWRUV ZLWK GHFLVLRQ VXSSRUW DQG GDWD
IRFXVHV RQ SLDQR HGXFDWLRQ DQG WKH UHVHDUFK GULYHQ ZRUN %\ DQDO\]LQJ GDWD RQ VWXGHQW SHUIRUPDQFH
LQFOXGHV D VXUYH\ RQ WKH SRWHQWLDO RI GLJLWDO DQGSURJUHVVLQIRUPDWLRQFDQEHJHQHUDWHGWKDWFDQJXLGH
WHFKQRORJ\ LQ SLDQR HGXFDWLRQ $FFRUGLQJ WR WKH WHDFKHUV LQ PDNLQJ GHFLVLRQV DERXW QHFHVVDU\
VXUYH\ UHVXOWV D VXEVWDQWLDO  RI WKH SDUWLFLSDQWV LQWHUYHQWLRQVDQGDGMXVWPHQWVWRLQVWUXFWLRQ
DFNQRZOHGJHG WKDW WKH FXUULFXOXP SOD\V D FUXFLDO 2I FRXUVH $, GRHV QRW ODFN WKH DELOLW\ WR SURYLGH
UROH LQ LQIOXHQFLQJ WKH SDFH RI OHDUQLQJ DQG SOD\V D IHHGEDFN DQG ZRUN ZLWK VWXGHQWV LQ D WLPHO\ DQG GLUHFW
UROH LQ IDFLOLWDWLQJ OHDUQLQJ EH\RQG WKH FRQILQHV RI PDQQHU7KLVLPPHGLDWHIHHGEDFNLVDQLPSRUWDQWWRROLQ
WKH FODVVURRP HQYLURQPHQW 7KH VWXG\ ILQGLQJV GHYHORSLQJ VNLOOV DQG FRUUHFWLQJ GHILFLHQFLHV 6XFK DQ
VKRZHG WKDW GLJLWDO WHFKQRORJ\ VLJQLILFDQWO\ LQWHUDFWLRQ FDQ EH HYHQ PRUH SHUVRQDO DQG HIIHFWLYH
LQIOXHQFHVWKHOHDUQLQJSURFHVVIRUSOD\LQJWKHSLDQR 8OWLPDWHO\WKLVFDQWUDQVODWHLQWRDQLQFUHDVHLQWKHRYHUDOO
DQG FRQWULEXWHV WR HIIHFWLYH OHDUQLQJ 7KH XVH RI TXDOLW\ RI OHDUQLQJ DQG VWXGHQW LQWHUHVW LQ WKHLU RZQ
YDULRXV GLJLWDO SODWIRUPV LQFUHDVHV VWXGHQWV  LQWHUHVW SURJUHVV
LQ LQGHSHQGHQW SUDFWLFH DQG DW WKH VDPH WLPH ,Q DGGLWLRQ WR WKH DGPLQLVWUDWLYH DVSHFWV $, FDQ EH
IDFLOLWDWHVHUURUFKHFNLQJDQGFRUUHFWLRQ DVVLJQHGWKHUROHRIWHDFKLQJDVVLVWDQW)RUH[DPSOHLQWKH
• ,QIRUPDWLRQDQGFRPPXQLFDWLRQWHFKQRORJLHV7KHVH DUHD RI LQVWUXFWLRQ $, KDV WKH DELOLW\ WR SUHGLFW KRZ D
V\VWHPV VXSSRUW WKH UDSLG H[FKDQJH RI LQIRUPDWLRQ VWXGHQWZLOOH[FHHGH[SHFWDWLRQVLQSURMHFWVDQGH[HUFLVHV
EHWZHHQWHDFKHUVDQGVWXGHQWVZKLOHHQDEOLQJRQOLQH DQGKRZOLNHO\WKH\DUHWRGURSRXWRIVFKRRO,QWKLVZD\
OHDUQLQJ DQG FROODERUDWLRQ 7KH FDVH VWXG\ >@ LW FDQ SURYLGH WHDFKHUV ZLWK UHOHYDQW LQIRUPDWLRQ VR WKDW
VKRZV WKDW DOWKRXJK WKH XVH RI ,&7 DQG $, LQ WKH\ FDQ LQWHUYHQH HDUOLHU LI SUREOHPV DULVH >@ %\
HGXFDWLRQ LV UHDVRQDEOH LW VWLOO KDV OLPLWDWLRQV 7KH DQDO\]LQJ FXUULFXOXP DQG PDWHULDOV LW FDQ GHVLJQ
VWXG\ IRFXVHV RQ WKH UHODWLRQVKLS EHWZHHQ DUWLILFLDO FXVWRPL]HG FRQWHQW WKDW EHWWHU PHHWV VWXGHQWV  QHHGV ,W
LQWHOOLJHQFHDQGWKHFRQFHSWRIOHDUQLQJVW\OHVZKLFK DOVRHQDEOHVOHDUQLQJRXWVLGHWKHFODVVURRPDQGSURPRWHV
LGHQWLI\VWXGHQWV OHDUQLQJSUHIHUHQFHV7KHDLPZDV FROODERUDWLRQ EHWZHHQ VWXGHQWV DW D KLJKHU OHYHO RI
WR GHWHUPLQH ZKHWKHU LW LV SRVVLEOH WR DXWRPDWLFDOO\ OHDUQLQJ
LGHQWLI\ VWXGHQWV  OHDUQLQJ VW\OHV IURP WKHLU 8VLQJ $, LQ LQVWUXFWLRQ DOVR PHDQV WDLORULQJ WKH
LQWHUDFWLRQV ZLWK D OHDUQLQJ PDQDJHPHQW V\VWHP WHDFKLQJPHWKRGVIRUHDFKVWXGHQWEDVHGRQWKHLUSHUVRQDO
$UWLILFLDO LQWHOOLJHQFH DOJRULWKPV EDVHG RQ D GDWD7KLVDFKLHYHVDSHUVRQDOL]HGDSSURDFKWKDWFDQOHDG
PXOWLOD\HU QHXUDO QHWZRUN DUH XVHG WR LGHQWLI\ WKH WR EHWWHU UHVXOWV DQG JUHDWHU VWXGHQW SDUWLFLSDWLRQ LQ WKH
UHODWLRQVKLS EHWZHHQ OHDUQLQJ VW\OHV DQG VWXGHQW OHDUQLQJ SURFHVV 7KLV SHUVRQDOL]DWLRQ FDQ DOVR EH
EHKDYLRU LQ WKH V\VWHP 7KH UHVXOWV VKRZ D ZHDN HIIHFWLYHO\LPSOHPHQWHGE\FUHDWLQJSHUVRQDOL]HGOHDUQLQJ
UHODWLRQVKLS EHWZHHQ OHDUQLQJ VW\OHV DQG VWXGHQW SODQV PDNLQJ LW HDVLHU IRU LQVWUXFWRUV WR FUHDWH GLYHUVH
LQWHUDFWLRQV ,Q DGGLWLRQ WR WKH UHVXOWV RI WKH /06
/HDUQLQJ 0DQDJHPHQW 6\VWHP  GDWD DQDO\VLV WKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 217

OHDUQLQJ WUDMHFWRULHV EDVHG RQ WKH QHHGV RI LQGLYLGXDO >@ 6%&RúNXQDQG07XUDQOL³&UHGLWULVNDQDO\VLVXVLQJERRVWLQJ
VWXGHQWV PHWKRGV´ -RXUQDO RI $SSOLHG 0DWKHPDWLFV 6WDWLVWLFV DQG
,QIRUPDWLFV  
$OWKRXJK OLWWOH ZLOO FKDQJH LQ WKH SULQFLSOHV RI WKH >@ 6 %DKRR 0 &XFFXOHOOL DQG ' 4DPDU ³$UWLILFLDO LQWHOOLJHQFH
DSSURDFKHV SUHVHQWHG KHUH LQ WKH QHDU IXWXUH ZLWK WKH DQG FRUSRUDWH LQQRYDWLRQ $ UHYLHZ DQG UHVHDUFK
DGYHQWRIJHQHUDWLYHDUWLILFLDOLQWHOOLJHQFHRQHFDQH[SHFW DJHQGD´7HFKQRORJLFDO )RUHFDVWLQJ DQG 6RFLDO &KDQJH
DQRUGHURIPDJQLWXGHLQFUHDVHLQWKHXVHRIWKHVHWRROVLQ 
WHDFKLQJ >@ $ $ODP 6KRXOG 5RERWV 5HSODFH 7HDFKHUV" 0RELOLVDWLRQ RI $,
DQG /HDUQLQJ $QDO\WLFV LQ (GXFDWLRQ ,QWHUQDWLRQDO
,9 &21&/86,216 &RQIHUHQFH RQ $GYDQFHV LQ &RPSXWLQJ &RPPXQLFDWLRQ DQG
&RQWURO ,&$&  0XPEDL ,QGLD  SS  GRL
7KLVDUWLFOHSURYLGHGDFRPSUHKHQVLYHRYHUYLHZRIKRZ ,&$&
DUWLILFLDO LQWHOOLJHQFH FDQ FRQWULEXWH WR LPSURYLQJ WKH >@ : +ROPHV 0 %LDOLN DQG & )DGHO $UWLILFLDO LQWHOOLJHQFH LQ
HGXFDWLRQDOSURFHVV(DFKRIWKHPDLQFDWHJRULHVRI$,KDV HGXFDWLRQ*OREHWKLFV3XEOLFDWLRQV
LWV RZQ XQLTXH EHQHILWV DQG SRWHQWLDO LQ WKH ILHOG RI >@ -4X<=KDRDQG<;LH$UWLILFLDOLQWHOOLJHQFHOHDGVWKHUHIRUP
HGXFDWLRQ KLJKOLJKWLQJ WKH LPSRUWDQFH RI XQGHUVWDQGLQJ RI HGXFDWLRQ PRGHOV6\VWHPV 5HVHDUFKDQG %HKDYLRUDO 6FLHQFH
YROQRSS
KRZ WR SURSHUO\ FRPELQH DQG LPSOHPHQW WKHVH
>@ /&KHQ3&KHQDQG=/LQ$UWLILFLDO,QWHOOLJHQFHLQ(GXFDWLRQ
WHFKQRORJLHV IRU RSWLPDO UHVXOWV ,PSURYLQJ WKH $5HYLHZ,((($FFHVVYROSS
LQWHUDFWLYLW\ SHUVRQDOL]DWLRQ DQG HIIHFWLYHQHVV RI WKH
>@ ' %XHQDxR)HUQiQGH] ' *LO DQG 6 /XMiQ0RUD $SSOLFDWLRQ
HGXFDWLRQDO SURFHVV OLHV LQ WKH DELOLW\ WR LQWHJUDWH WKHVH RI PDFKLQH OHDUQLQJ LQ SUHGLFWLQJ SHUIRUPDQFH IRU FRPSXWHU
WHFKQRORJLHV LQWR OHDUQLQJ HQYLURQPHQWV WKXV FUHDWLQJ HQJLQHHULQJVWXGHQWV$FDVHVWXG\6XVWDLQYROQRSS
VLJQLILFDQW LPSURYHPHQWV LQ WKH HGXFDWLRQDO HQYLURQPHQW 
IRUIXWXUHJHQHUDWLRQV >@ $ $VLI + 0XNKWDU ) $OTDGKHHE + ) $KPDG DQG $
6HHNLQJWRJREH\RQGWKHJHQHUDOFRQFHSWVRIDUWLILFLDO $OKXPDP $Q DSSURDFK IRU SURQXQFLDWLRQ FODVVLILFDWLRQ RI
FODVVLFDO $UDELF SKRQHPHV XVLQJ 'HHS /HDUQLQJ$SSOLHG
LQWHOOLJHQFHWKLVDUWLFOHIRFXVHVRQVSHFLILFDSSOLFDWLRQVRI 6FLHQFHVYROQRSS
WKLV WHFKQRORJ\ LQ WKH FRQWH[W RI VWXGHQW OHDUQLQJ >@ ')0XUDG$*,VNDQGDU()HUQDQGR762FWDYLDDQG'(
SUHVHQWLQJ FRQFUHWH UHVXOWV DFKLHYHG 0HWKRGRORJLFDOO\ 0DXUHG 7RZDUGV VPDUW /06 WR LPSURYH OHDUQLQJ RXWFRPHV
ZH KDYH VHOHFWHG H[DPSOHV IURP WKH EURDG OLWHUDWXUH VWXGHQWV XVLQJ /HQR%RW ZLWK QDWXUDO ODQJXDJH SURFHVVLQJ3URF
LQGH[HG LQ WKH :R66FRSXV GDWDEDVHV RYHU WKH ODVW ILYH WK,QW&RQIRQ,QIRUPDWLRQ7HFKQRORJ\&RPSXWHUDQG(OHFWULFDO
\HDUV )RU HDFK $, FDWHJRU\ ZH VHOHFWHG D VSHFLILF (QJLQHHULQJ ,&,7$&(( SS
H[DPSOH RI DQ DSSOLFDWLRQ LQ ZKLFK DQ $, PHWKRG KDV >@ &+:X$&RQFHSW)UDPHZRUNRI8VLQJ(GXFDWLRQ*DPHZLWK
EHHQ LPSOHPHQWHG HIIHFWLYHO\ LQ VWXGHQW OHDUQLQJ LQ D $UWLILFLDO 1HXUDO 1HWZRUN 7HFKQLTXHV WR ,GHQWLI\ /HDUQLQJ
6W\OHV3URF,QW&RQI0DFK/HDUQ&\EHUQYRO-XO\SS
SDUWLFXODUVXEMHFWRUGRPDLQ 
$NH\FKDUDFWHULVWLFRIWKHVHVHOHFWHGVWXGLHVLVWKDWWKH\ >@ 3<DQURQJ5HVHDUFKRQ,QWHOOLJHQW(GXFDWLRQ0HWKRG%DVHGRQ
DUH QRW OLPLWHG WR WKHRUHWLFDO FRQVLGHUDWLRQV EXW DFWXDOO\ 'DWD 0LQLQJ RI 'LVWDQFH (GXFDWLRQ 6\VWHP  ,QWHUQDWLRQDO
FRQGXFW H[SHULPHQWV ZLWK JURXSV RI VWXGHQWV (DFK RI &RQIHUHQFHRQ ,QWHOOLJHQW7UDQVSRUWDWLRQ%LJ'DWD 6PDUW&LW\
WKHVHVWXGLHVUHSUHVHQWVDXQLTXHH[SHULPHQWDODSSURDFKLQ ,&,7%6  ;L DQ &KLQD  SS  GRL
,&,7%6
ZKLFK $, PHWKRGV ZHUH WHVWHG RQ UHDO VWXGHQW VDPSOHV
7KHVH H[SHULPHQWV VKRZ XV FRQFUHWH UHVXOWV DQG WKH >@ ' &OLII $Q 2SHQ6RXUFH /LPLW2UGHU%RRN ([FKDQJH IRU
7HDFKLQJ DQG 5HVHDUFK  ,((( 6\PSRVLXP 6HULHV RQ
HIIHFWLYHQHVVRI$,DSSOLFDWLRQVLQWKHHGXFDWLRQDOSURFHVV &RPSXWDWLRQDO ,QWHOOLJHQFH 66&,  %DQJDORUH ,QGLD  SS
,QWKLVZD\ZHKDYHSURYLGHGDGHHSHULQVLJKWLQWRWKH GRL66&,
SUDFWLFDO DSSOLFDWLRQ RI $, LQ HGXFDWLRQ DQG SURYLGHG >@ $ 0HKUD 2 *XSWD DQG 6 $YLNDO)LQGLQJ WKH FRPELQHG HIIHFW
FRQFUHWH H[DPSOHV WKDW KLJKOLJKW LWV UHDO EHQHILWV %\ RI DFDGHPLF DQG QRQDFDGHPLF SHUIRUPDQFH RQ PDQDJHPHQW
GHPRQVWUDWLQJ WKH FRQFUHWH EHQHILWV WKDW DUWLILFLDO VWXGHQWV¶ SODFHPHQW $ IX]]\ ORJLF DSSURDFK7KH ,QWHUQDWLRQDO
-RXUQDORI0DQDJHPHQW(GXFDWLRQ  
LQWHOOLJHQFHEULQJVWRHGXFDWLRQWKLVDUWLFOHDLPVWRLQVSLUH
>@ (%LORWWD)%HUWDFFKLQL/*DEULHOH6*LJOLR363DQWDQRDQG
HGXFDWRUV DGPLQLVWUDWRUV DQG SROLF\PDNHUV WR HPEUDFH 7 5RPLWD ,QGXVWU\   WHFKQRORJLHV LQ WRXULVP HGXFDWLRQ
WKHVH WUDQVIRUPDWLYH WRROV DQG H[SORUH WKH PDQ\ 1XUWXULQJVWXGHQWVWRWKLQNZLWKWHFKQRORJ\-+RVS/HLV6SRUW
SRVVLELOLWLHVWKH\SUHVHQWWRVKDSHWKHIXWXUHRIHGXFDWLRQ 7RXU(GXFYROSS
*LYHQWKHHYHUFKDQJLQJGRPDLQVRI$,DQGHGXFDWLRQWKH >@ 1 0XWKXNXPDU 6 6ULQLYDVDQ % 6XEDWKUD DQG . 5DPNXPDU
PHUJLQJ RI WKHVH WZR GRPDLQV SURPLVHV WR FUHDWH D 7HDFKLQJ LQGXVWULDO LQWHUQHWRIWKLQJVEDVHG PRGHOSUHGLFWLYH
G\QDPLFDQGDGDSWLYHHGXFDWLRQDOHFRV\VWHPWKDWZLOOWDNH FRQWUROOHU,((( 7UDQVDFWLRQV RQ (GXFDWLRQ YRO  QR  SS
LQWRDFFRXQWWKHGLYHUVHQHHGVDQGDVSLUDWLRQVRIOHDUQHUV 
LQWKHGLJLWDODJH >@ 0 $UQHWW 6 0DKPXG 5+ 6RXUDYH DQG -+ .LP ³6PDUW
7UDVKFDQ %URWKHUV (DUO\ &KLOGKRRG (QYLURQPHQWDO (GXFDWLRQ
7KURXJK *UHHQ 5RERWLFV´ ,Q,QWHUQDWLRQDO &RQIHUHQFH RQ
$&.12:/('*0(17 ,QWHOOLJHQW +XPDQ &RPSXWHU ,QWHUDFWLRQ SS   &KDP
7KH VWXG\ ZDV HODERUDWHG ZLWKLQ WKH UHVHDUFK SURMHFW 6SULQJHU,QWHUQDWLRQDO3XEOLVKLQJ
VXSSRUWHGE\ 6ORYDN 5HVHDUFK DQG 'HYHORSPHQW $JHQF\ >@ .& /LQ +& +XQJ DQG 16 &KHQ ³7KH HIIHFW RI ZHDUDEOH
WHFKQRORJ\ RQ EDGPLQWRQ OHDUQLQJ SHUIRUPDQFH D PXOWLSOH
$399  1R $399 $399 5RDGPDS RI D IHHGEDFN :,6(5 PRGHO LQ SK\VLFDO HGXFDWLRQ´6PDUW /HDUQLQJ
GLJLWDO SODWIRUP SURYLGLQJ $, $UWLILFLDO ,QWHOOLJHQFH  (QYLURQPHQWV  
DXWRPDWLRQ RI GHFLVLRQPDNLQJ SURFHVVHV LQ WKH ILHOG RI >@ - =KDR DQG * )X ³$UWLILFLDO LQWHOOLJHQFHEDVHG IDPLO\ KHDOWK
FRPPXQLFDWLRQVWUDWHJ\ HGXFDWLRQ SXEOLF VHUYLFH V\VWHP´)URQWLHUV LQ 3V\FKRORJ\

5()(5(1&(6 >@ & 'DQ 5HVHDUFK RQ WKH /HYHO 7HVW $SSOLFDWLRQ 6\VWHP RI
&RPSXWHU $UWLILFLDO ,QWHOOLJHQFH 7HFKQRORJ\ LQ (QJOLVK
>@ < -LDQJ ; /L + /XR 6 <LQ DQG 2 .D\QDN 4XR YDGLV
(GXFDWLRQ  ,QWHUQDWLRQDO 6\PSRVLXP RQ $GYDQFHV LQ
DUWLILFLDOLQWHOOLJHQFH"'LVFRYHU$UWLI,QWHOOYROQRSS
,QIRUPDWLFV (OHFWURQLFV DQG (GXFDWLRQ ,6$,((  )UDQNIXUW
'HF

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 218

 *HUPDQ\  SS  GRL 0XOWLPRGDO%LRPHWULFV7HFKQRORJ\LQ,((($FFHVVYROSS
,6$,(( GRL$&&(66
>@ 6 =KDR 4 3DQ ' *DR DQG - &KHQJ ,QWHJUDWLQJ LQWHUQHW RI >@ $%KDWQDJDU$-DLVZDO$-DLQDQG1%%ROLD³$QDQDO\VLVRI
WKLQJVDQGPL[HGUHDOLW\WRWHDFKSHUIRUPDQFHEDVHGDUFKLWHFWXUDO NH\LQGLFDWRUVIRUHQKDQFLQJVFKRROSHUIRUPDQFH(YLGHQFHVIURP
GHVLJQ D FDVH VWXG\ RI VKDGLQJ GHYLFHV (GXFDWLRQ DQG ,QGLD´6RFLR(FRQRPLF3ODQQLQJ6FLHQFHV
,QIRUPDWLRQ7HFKQRORJLHVSS >@ - 3HQJ ³'LJLWDO WHFKQRORJLHV SRWHQWLDO IRU SLDQR
>@ & %KDQXSUDNDVK < 6 1LMDJXQDU\D DQG 0 $ -D\DUDP HGXFDWLRQ´,QWHUDFWLYH/HDUQLQJ(QYLURQPHQWV
3HUIRUPDQFHDQDO\VLVRIVWXGHQWVE\HYDOXDWLQJWKHLUH[DPLQDWLRQ >@ 5'GD&RVWD*)GH6RX]D7%GH&DVWUR5$GH0HGHLURV
DQVZHU VFULSWV E\ XVLQJ VRIW FRPSXWLQJ WHFKQLTXHV LQ (PHUJLQJ 9DOHQWLPDQG$GH3LQKR'LDV,GHQWLILFDWLRQRI/HDUQLQJ6W\OHV
5HVHDUFK LQ (OHFWURQLFV &RPSXWHU 6FLHQFH DQG 7HFKQRORJ\ LQ'LVWDQFH(GXFDWLRQ7KURXJKWKH,QWHUDFWLRQRIWKH6WXGHQW:LWK
6SULQJHUSS D /HDUQLQJ 0DQDJHPHQW 6\VWHP LQ,((( 5HYLVWD
>@ 0DR/ ;X= -XO\ &RQVWUXFWLRQRI2QOLQH(GXFDWLRQ ,EHURDPHULFDQD GH 7HFQRORJLDV GHO $SUHQGL]DMH YRO  QR 
0RGHO RI 0DUNHWLQJ 6SHFLDOW\ %DVHG RQ &ORXG &RPSXWLQJ SS$XJGRL5,7$
,Q,QWHUQDWLRQDO &RQIHUHQFH RQ (/HDUQLQJ ((GXFDWLRQ DQG >@ ' %DxHUHV 0 ( 5RGUtJXH] $ ( *XHUUHUR5ROGiQ DQG $
2QOLQH 7UDLQLQJ SS   &KDP 6SULQJHU 1DWXUH .DUDGHQL]$QHDUO\ZDUQLQJV\VWHPWRGHWHFWDWULVNVWXGHQWVLQ
6ZLW]HUODQG RQOLQHKLJKHUHGXFDWLRQ$SSO6FLYROQRSS-XQ
>@ 0/DED\HQ59HD-)OyUH]1$JLQDNRDQG%6LHUUD2QOLQH 
6WXGHQW $XWKHQWLFDWLRQ DQG 3URFWRULQJ 6\VWHP %DVHG RQ


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 219

 'HYHORSLQJ3UHUHTXLVLWH.QRZOHGJHIRU'
0RGHOOLQJDQG'3ULQWLQJ
-+YRUHFNêDQG$6FKPLG
)DFXOW\RI(GXFDWLRQ8QLYHUVLW\RI2VWUDYD&]HFK5HSXEOLF
MR]HIKYRUHFN\#RVXF]DQJHOLNDVFKPLG#RVXF]


$EVWUDFW²' SULQWLQJ LV DFRQWHPSRUDU\ PHWKRG RI WKUHH DGGUHVVHV SUHVHUYLFH WHDFKHUV WKH VWXGHQWV RI
GLPHQVLRQDO REMHFWV¶ SURGXFWLRQ 3ULRU WR WKH SULQWLQJ WKH SDUWLFLSDWLQJXQLYHUVLWLHV
REMHFW PXVW EH GHVLJQHG LQ WKH IRUP RI LWV ' PRGHO ± D ,Q67($0WKHOHWWHU0VWDQGVIRU0DWKHPDWLFV0DQ\
YLUWXDO VSDWLDO ERG\ 7KH PRGHO LV WKHQ WUDQVIHUUHG LQWR D VWXGHQWV VHHP 0DWKHPDWLFV ERULQJ >@ $V WKHUH DUH ILYH
FRPSXWHU SURJUDP ZKLFK ZLOOFRQWURO WKH SULQWLQJ SURFHVV RWKHU OHWWHUV LQ WRWDO WKH VWXGHQWV OHVV LQWHUHVWHG LQ
)LQDOO\WKHREMHFWLVFRPSRVHGXVLQJDVWHSZLVHDGGLWLRQRI PDWKHPDWLFV FDQ DVN WKHPVHOYHV ,V WKHUHDQ\ UHDVRQIRU
PLFURSDUWLFOHV JUDLQV  LQFOXGLQJ0"(YHU\RQHZKRZDQWVWRIROORZWKH67($0
SKLORVRSK\ PXVW EH FDSDEOH PDNH WKLV VXEMHFW PRUH
' SULQWLQJ LV YHU\ DSSURSULDWH IRU WKH LQGLYLGXDOL]HG
DWWUDFWLYH 7KHUH DUH VHYHUDO ZD\V RI GRLQJ VR $ UHFHQW
SURGXFWLRQLHIRUIDEULFDWLQJXQLTXHREMHFWVRUWKRVHGRQH
RQHLVSODFLQJWKHFRXUVHLQWRYLUWXDOUHDOLW\HQYLURQPHQWV
LQVPDOOVHULHV$W\SLFDOOLIHF\FOHRIWKHVHREMHFWVLV³'HVLJQ
±VHHIRUH[DPSOH>@,QWKLVSDSHUZHGHPRQVWUDWH
LW±3URGXFHLW±0DNHXVHRILW´$JURZLQJLQWHUHVWLQWKLV
H[WHQWRISURGXFWLRQUHTXLUHVSUHSDULQJHQRXJK'SULQWLQJ
DQRWKHUDSSURDFKPDNLQJLWPRUHDWWUDFWLYHE\HQULFKLQJ
VSHFLDOLVWV LQ D YDULHW\ GLVFLSOLQHV ,Q RXU SUHVHQWDWLRQ ZH
WKHFRXUVHPDWHULDOE\UHODWLRQVEHWZHHQWKH0DQGWKH6
GLVFXVV ZKDW SURIHVVLRQDOV DUH QHHGHG DQG ZKDW 7(DQG$,QSDUWLFXODUWKH0UHIHUVWRVSDWLDOJHRPHWU\
PRGLILFDWLRQV RI FRQWHPSRUDU\ VFKRROLQJ ZRXOG IXOILO WKHVH DQGWKH67(DQG$UHIHUWRDQ\LPDJLQDEOHVLWXDWLRQLQ
UHTXLUHPHQWVWRDPD[LPXPGHJUHH ZKLFK WKH VSDWLDO JHRPHWU\ SOD\V UROH UHOHYDQW WR DQ\ RI
WKHP
2XU SURMHFW LV WKHUHIRUH RULHQWHG WR GHVLJQLQJ DQG
, ,1752'8&7,21 GHYHORSLQJWHDFKLQJDQGOHDUQLQJDFWLYLWLHVFRUUHVSRQGLQJ
7R JXDUDQWHH HFRQRPLF GHYHORSPHQW LQGXVWULDO WR WKHLU DVVRFLDWLRQV IDU RXW RI FODVVURRPV 7KH DFWLYLWLHV
LQQRYDWLRQV KDYH WR EH UHIOHFWHG E\ LQQRYDWLRQV RI ZLOO EH SULPDULO\ KDQGV RQ DQG VKRXOG IDFLOLWDWH WKH
HGXFDWLRQ7KHVHUHIOHFWLRQVPXVWDIIHFWDOOLWVFRQWHQWLWV VWXGHQWV¶ FUHDWLYLW\ $V DOPRVW QR RQH LV VNLOOHG LQ DOO
PHWKRGRORJ\ DQG LWV IRUPDW 2QH RI FRQWHPSRUDU\ FRPSRQHQWV RI WKH 67($0 TXLQWHW WKH DSSURDFK
LQGXVWULDO LQQRYDWLRQV LV WKH LQGLYLGXDOL]HG SURGXFWLRQ ± FRPELQHDFWLYHDQGFROODERUDWLYHOHDUQLQJ
DQ LQWHOOLJHQW KLJKO\ DXWRPDWHG SURGXFWLRQ V\VWHP WKDW %HORZZHDOVRGHVFULEHWKHURDGPDSWRDQHGXFDWLRQDO
DOORZV KLJK YDULDQFHDQGG\QDPLVPLQWKHSURGXFWUDQJH HQYLURQPHQW RIIHULQJ WR DFKLHYH WKH DERYH JRDOV ,W LV
ZLWKSURGXFWLRQFRVWVDWWKHOHYHORIPDVVSURGXFWLRQ>@ EDVHG RQ DFWLYH OHDUQLQJ > @ ZKLFK HQJDJHV VWXGHQWV
,QWKLVSDSHUZHORRNDWLQGLYLGXDOL]HGSURGXFWLRQIURP DFWLYHO\ ZRUN ZLWK WKH FRXUVH PDWHULDO WKURXJK
WZRSRLQWVRIYLHZ GLVFXVVLRQVSUREOHPVROYLQJFDVHVWXGLHVUROHSOD\VDQG
RWKHU PHWKRGV $W &]HFK XQLYHUVLWLHV DV ZHOO DV DW PRVW
• $V D VSHFLPHQ RI LQGLYLGXDOL]HG DSSURDFK WR &HQWUDO DQG (DVWHUQ RQHV SDVVLYH OHDUQLQJ EDVHG RQ
FXVWRPHUV QRW RQO\ LQ LQGXVWU\ EXW DOVR WR WKRVH LQ OHFWXULQJ SUHYDLOV 2XU FHQWUDO TXHVWLRQLV KRZ WR HQJDJH
RWKHUDUHDV±LQHGXFDWLRQLQRXUFDVH WKHP LQWR DFWLYH LQWHUGLVFLSOLQDU\ OHDUQLQJ DURXQG '
• $V DQ RXWFRPH DQG SURFHVVRULHQWHG V\VWHPLF PRGHOOLQJ DQG ' SULQWLQJ $FFRUGLQJ WR >@ ³WKH NH\
PHWKRGRORJ\ ZKLFK FDQ EH DIWHU QHFHVVDU\ VWUDWHJ\ RI FKDQJLQJ WKH VWXGHQW¶V UROH IURP SDVVLYH WR
PRGLILFDWLRQV  UHSOLFDWHG LQ WKHVH DUHDV $JDLQ LQ DFWLYH LV FRRSHUDWLYH OHDUQLQJ´ :H WU\ WR FRPELQH WKH
HGXFDWLRQ VXFK DQDORJ\ ILWV ZHOO WR FRQVWUXFWLYH EHVW RI WKUHH FRQWHPSRUDU\ DSSURDFKHV FRRSHUDWLYH
OHDUQLQJ OHDUQLQJ FROODERUDWLYH OHDUQLQJ DQG SUREOHPEDVHG
:KDWKDVLQVSLUHGXVWRWKHVHFRQVLGHUDWLRQV" OHDUQLQJ>@
0DQ\ VWXGHQWV FDQ KDUGO\ LPDJLQH WKH FRPSOH[LW\ RI )LUVW ZH GLVFXVV SUHVXPSWLRQV DQG SUHUHTXLVLWHV ZH
SURGXFWLRQ SURFHVVHV 7KH\ UDUHO\ ZDWFK D SURGXFW¶V OLIH H[SHFW IURP WKH LQVWLWXWLRQ LWV HGXFDWRUV DQG OHDUQHUV WR
F\FOHIURPWKHLQLWLDOLGHDWRLWVSURGXFWLRQDQGVDOH,I' PDNH EXLOGLQJ VXFK HQYLURQPHQWV SRVVLEOH 7KHQ ZH
SULQWLQJ ZRXOG EHSUHVHQWHG WR WKHPMXVW DVDSURGXFWLRQ DQDO\VH RSWLPDO ZD\V RI GHPRQVWUDWLQJ DOO DERYH
RIFRPSXWHURXWSXWWKRVHZKRDUHQRWWHFKQLFDOO\RULHQWHG PHQWLRQHG FRQQHFWLRQV DPRQJ WKH 67($0 FRPSRQHQWV
FRXOG DVN WKHPVHOYHV :K\ DUH ZH OHDUQLQJ LW" 3RVLQJ HQKDQFH WKH VWXGHQWV¶ PRWLYDWLRQ WR SDUWLFLSDWH RQ WKHLU
VXFKTXHVWLRQLQGLFDWHVDEVHQFHRILQWHUHVWDQGPRWLYDWLRQ UHGLVFRYHU\ $IWHU LW ZH RXWOLQH DFWLYLWLHV ZH DUH QRZ
7KH 8QLYHUVLW\ RI 2VWUDYD LV WKH FRRUGLQDWRU RI DQ GRLQJ WR PRYH IURP WKH WKHRU\ WR SUDFWLFH )LQDOO\ ZH
(UDVPXV SURMHFW $FFHOHUDWLQJ 67($0UHODWHG GLVFXVV WKH DGYDQWDJHV DQG GUDZEDFNV RI RXU SURSRVHG
.QRZOHGJHDQG6NLOOVYLD'0RGHOOLQJDQG'3ULQWLQJ DSSURDFK
ZKLFKLQFOXGHVRWKHUXQLYHUVLWLHVIURP$XVWULD)LQODQG
/X[HPERXUJ DQG 6ORYDNLD 7KH SURMHFW SULPDULO\

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 220

 WKHLUH[SORUDWLRQRIPDWHULDOV7KLVZD\RIOHDUQLQJRIWHQ
EHJLQV ZLWK TXHVWLRQV SUREOHPV RU VFHQDULRV ,Q WKLV
PHWKRGWHDFKHUVQRWRQO\LPSDUWLQIRUPDWLRQEXWIDFLOLWDWH
VWXGHQWV WUDQVLWLRQIURPPHUHFXULRVLW\WRFULWLFDOWKLQNLQJ
DQG GHHSHU XQGHUVWDQGLQJ 7KH\ VXSSRUW VWXGHQWV E\
JXLGLQJ WKHLU MRXUQH\ RI LQTXLU\ DQG HQFRXUDJLQJ
VWUXFWXUHGTXHVWLRQLQJDFWLYLWLHV
7RLOOXVWUDWHWKLVOHDUQLQJSURFHVV7HMHUDHWDO>@JLYH
DQ H[DPSOH RI KRZ WR LPSOHPHQW WKLV FRQFHSW LQ WKH
FODVVURRP7KHRULJLQDO WDVNZDV WR PRGHOWKHRXWOLQHRI
WKH *KHUNLQ EXLOGLQJ LQ /RQGRQ 7KH SURFHVV RI WKH
PDWKHPDWLFDOPRGHOOLQJSKDVHLVHYLGHQWLQ)LJXUHDQG
 ZKHQ WKH EXLOGLQJ LPDJH ZDV XSORDGHG LQWR WKH
 *HR*HEUDSURJUDPDQGUHSODFHGZLWKSLHFHZLVHIXQFWLRQV

)LJXUH5HYHUWHG*KHUNLQEXLOGLQJ>@ %\ XWLOLVLQJ SUHYLRXVO\ OHDUQHG PDWKHPDWLFDO FRQFHSWV
VXFK DV SDUDERODV DQG HOOLSVHV ZDV FUHDWHG DQ LQWHJUDWHG
QHWZRUN RI PDWKHPDWLFDO DFWLYLWLHV >@ 6XEVHTXHQW
GLVFXVVLRQ RI %ULWLVK YHUVXV ORFDO DUFKLWHFWXUH VXSSRUWHG
WKHFXOWXUDODVSHFWRIPDWKHPDWLFDOPRGHOOLQJ>@
8VLQJ SXSLOV  H[LVWLQJ NQRZOHGJH RI IXQFWLRQV
WUDQVIRUPDWLRQVHTXDWLRQVDQGRWKHUQXPHUDF\WRSLFVWKH
FODVV VHW DERXW FUHDWLQJ D ' PRGHO RI WKH VWUXFWXUH WR
GHWHUPLQHLWVYROXPHDQGFUHDWHDPDWHULDOUHSUHVHQWDWLRQ
XVLQJ'SULQWLQJ )LJXUH 7KLVH[SHULHQFHKLJKOLJKWHG
WKHYDOXHRIRSHQHQGHGWDVNVDQGH[SORUDWRU\OHDUQLQJDV
 VWXGHQWOHG H[SORUDWLRQ IDFLOLWDWHG GHHSHU PDWKHPDWLFDO
 HQJDJHPHQW>@8VLQJWKH*HR*HEUDFRPPDQG6XUIDFH
)LJXUH0RGHOOLQJWKH*KHUNLQEXLOGLQJ>@

WR FUHDWH D URWDWLQJ VXUIDFH DURXQG WKH [D[LV FRPSOHWHG
WKH PRGHOLQJ SURFHVV 7KH YROXPH FDOFXODWLRQ LQYROYHG
LQWHJUDOLQDUHDOOLIHFRQWH[W
' SULQWHG PRGHO ZDV WKHQ XVHG IRU WHVWLQJ WKH
FRUUHFWQHVV RI WKH UHVXOW E\ GHWHFWLQJ WKH YROXPH XVLQJ
ZDWHUZKLFKUHSUHVHQWVWKHLQWHUSUHWDWLRQDQGYHULILFDWLRQ
LQ WKH PRGHOOLQJ F\FOH >@ 7KLV WDVN QRW RQO\ EULGJHV
DUFKLWHFWXUH DQG PDWKHPDWLFV EXW DOVR LOOXVWUDWHV
PRGHOOLQJ F\FOH )LJXUH   ZKLFK HPSKDVLVHV T
VLPSOLILFDWLRQRIWKHUHDOVLWXDWLRQWRPDNHLWVROYDEOHE\D
PDWKHPDWLFDODSSDUDWXV>@,WVWDUWVZLWKWKHIRUPXODWLRQ
 RIWKHSUREOHPDQGHQGVZLWKWHVWLQJLWVFRUUHFWQHVV,WDOVR
)LJXUH*KHUNLQEXLOGLQJDVD'SULQWRXW>@ GHPRQVWUDWHVWKHUHODWLRQEHWZHHQWKHPRGHOPDGHE\WKH
 VWXGHQWV DQG WKH YROXPH RI WKH EXLOGLQJ $V D UHVXOW WKH
OHDUQHUVFDQXQGHUVWDQGWKHSURSRUWLRQEHWZHHQWKHP
,, '02'(//,1*$6$&25(2)7+(352-(&7
$W WKH VDPH WLPH WKRVH VWXGHQWV OHVV LQWHUHVWHG LQ
7KH ' SULQWLQJ LV LPSRVVLEOH ZLWKRXW FUHDWLQJ D ' PDWKHPDWLFVKDYHQRWJRWWKHIXOOUHVSRQVHWRWKHLULQLWLDO
PRGHORIWKHREMHFWZHZDQWWRSURGXFH7HMHUDHWDO>@ TXHVWLRQ:K\DP,OHDUQLQJLW")URPWKHLUSRLQWRIYLHZ
VHHV WKH LQWHJUDWLRQ RI WHFKQRORJ\ LQWR PDWKHPDWLFV ' 0RGHOOLQJ DQG ' SULQWLQJ VWLOO UHPDLQ FRPSOH[
HGXFDWLRQDVDZD\RILQWURGXFLQJD67($0DSSURDFKDQG PDWKHPDWLFDOO\EDVHG FRQFHSWV SRVVLEO\ XVHIXO IRU
PDNLQJ PDWKHPDWLFV KDQGVRQ DQG HQJDJLQJ $OWKRXJK DUFKLWHFWVEXWKDUGO\IRUDQ\RQHHOVH7KHSUREOHPVZKLFK
LPSOHPHQWLQJ WHFKQRORJ\ FDQ EH FKDOOHQJLQJ DQG H[SODLQLWWRWKHP PXVWEHVRPHWKLQJLQWHUHVWLQJDOVRIRU
LQWLPLGDWLQJ IRU WHDFKHUV WKH\ KLJKOLJKW WKH SRWHQWLDO HYHU\SURIHVVLRQDOLQFOXGLQJDUWLVWVDQGHFRQRPLVWV
EHQHILWV
7HDFKLQJPDWKHPDWLFVLVRIWHQLQWHUWZLQHGZLWKUHDOOLIH
DUFKLWHFWXUHVXFKDVEULGJHVDQGEXLOGLQJV(O%HGHZ\HW
DO>@VXJJHVWWKDWHGXFDWRUVXVHWKHVHUHDOOLIHH[DPSOHV
LQ WKH FODVVURRP WR KHOS VWXGHQWV VHH WKH FRQQHFWLRQ
EHWZHHQ PDWKHPDWLFDO FRQFHSWV DQG WKH ZRUOG DURXQG
WKHP 3XSLOV FRXOG EULQJ SHUVRQDO RU FXOWXUDO REMHFWV WR
PRGHODQGLQFOXGHFXOWXUDODQGKLVWRULFDOFRQWH[W6XFKDQ
DSSURDFKQRWRQO\UHLQIRUFHVPDWKHPDWLFVDQGWHFKQRORJ\
EXW DOVR FRQQHFWV FXOWXUH LQFUHDVLQJ VWXGHQW LQWHUHVW DQG
PRWLYDWLRQ E\ OLQNLQJ NQRZOHGJH WRSHUVRQDO H[SHULHQFH
DV'XLWHWDO>@SRLQWRXW 

$FFRUGLQJWR>@,QTXLU\%DVHG7HDFKLQJLVDQDFWLYH )LJXUH0RGHOOLQJF\FOH>@
DSSURDFK WR OHDUQLQJ LQ ZKLFK VWXGHQWV  FXULRVLW\ IXHOV 

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 221

 ,,, '35,17,1*352-(&7$6$5($//,)($&7,9,7< LY 7KHUHVXOWZLOOEHWKHFKRLFHRIWKHILODPHQWDQG
LWVDPRXQWIRUHDFKRIWKHWKUHHPHGDOW\SHV7KH
0RGHOOLQJLVDXVHIXODFWLYLW\EXWLWLVLVRODWHGIURPWKH
PDQDJHUPXVWHQVXUHLWVGHOLYHU\
UHDO ZRUOG LQ PRUH GLUHFWLRQV WKDQ )LJXUH  LQGLFDWHV
1DPHO\LWGRHVQRWVROYHRXULQLWLDOTXHVWLRQ:K\DP, Y 7KH PHGDO GHVLJQ ZLOO EH WUDQVIRUPHG WR EH
OHDUQLQJ LW" ,Q RWKHU ZRUGV ZKDW ± H[SHFW RI KLVKHU SUHSDUHGIRU'SULQWLQJ
WHDFKHU¶V RUGHU ± ZLOO OHDG WKH VWXGHQW WR PRGHO WKH YL 7KHILUVWVDPSOHVZLOOEHVKRZHGDVWKHSDWWHUQWR
*KHUNLQ EXLOGLQJ" ,I KH LV QRW D IXWXUH DUFKLWHFW RU D WKHRUJDQL]HUVIRUWKHLUILQDODSSURYDO$IWHUWKHLU
PDWKHPDWLFDO HQWKXVLDVW KDUGO\ DQ\WKLQJ 3URGXFLQJ LWV FRQVHQWWKHUHTXHVWHGQXPEHURIPHGDOVZLOOEH
PLQLDWXUH LV D QLFH H[DPSOH RI ' PRGHOOLQJ DQG ' SULQWHGRXW
SULQWLQJEXWKDUGO\DQ\WKLQJPRUH 7KHWDVNGLYLVLRQLVDSDUWRIWKHSUREOHPVROXWLRQDQG
:K\FDQZHFODLPWKLV" VKRXOGEHGRQHE\WKHVWXGHQWV1RWLFHWKDWPDWKHPDWLFVLV
)URPWKH67($0TXLQWHW7DQG0DUHWRXFKHG0 QHYHU PHQWLRQHG LQ DQ\ RI DERYH VXEWDVNV 1HYHUWKHOHVV
VWURQJO\ 7 PXFK OHVV 7KUHH UHPDLQLQJ FRPSRQHQWV VWD\ ZLWKRXW LW WKH ZKROH SUREOHP LV QRW VROYDEOH ,W LV QRW
EHKLQG)RUWKLVUHDVRQPRVWVWXGHQWVRI6(DQG$ZLOO VXEVWDQWLDO ZKHWKHU WKH VWXGHQWV UHDOL]HG WKLV IDFW QRZ RU
OLNHO\VHHVLPLODUWDVNVDVDQ³DFWLYLW\IRUDFWLYLW\´$VWKH ODWHU6RRQHURUODWHUWKH\UHDOL]HWKDWLWLVDSDUWRI,WHP
SUREOHPDVVLJQHGE\WKHLUWHDFKHULVQRWPRWLYDWLQJWKH\ 9,QDQ\FDVHWKHVWXGHQWVZLOOFRQVLGHULWDVDQHFHVVDU\
PLVVDSURSHULQLWLDOSRLQWRIWKHF\FOHLQ)LJXUH±DUHDO EXWPLQRUSDUWRIWKHLUWDVNVRLWZLOOKDUGO\UHSHOWKHP
OLIH VLWXDWLRQ OHDGLQJ WR D FDSWLYDWLQJ SUREOHP DSSHDOLQJ
WKHLUDWWHQWLRQ,WPHDQVWKDWSUREOHPVFRPELQLQJ67($0 9&2//$%25$7,9(/($51,1*$6$3$572)
LGHDV KDV WR H[WHQW LWV ³OHIW ZLQJ´ WR VWDUW ZLWK DQ 62/87,21
LQWHUHVWLQJ QHHG DQG HQGLQJ LWV XVHIXO UHVXOW 7KH 2XU PDLQ DLP LV WR XVH WKH ' SULQWLQJ SURMHFW IRU
DSSURSULDWHSUREOHPVKDYHWRFRQWDLQDWOHDVWRQHHOHPHQW GHPRQVWUDWLQJ UHODWLRQVKLSV DPRQJ WKH TXLQWHW
UHSUHVHQWLQJHYHU\OHWWHURITXLQWHW FRPSRQHQWV/HWXVGLVFXVVWKHPLQWKHVXEWDVNVRUGHU
7KHUHLVDQH[DPSOH$WRXUVFKRROZHSODQWRRUJDQL]H L 7KH QXPEHU RI PHGDOV DQG WKHLU SURSHUWLHV
DVSRUWFRPSHWLWLRQ7RDZDUGWKHZLQQHUVWKHRUJDQL]LQJ FRPELQH PDQDJHULDO DQG HFRQRP\ LVVXHV )LUVW
FRPPLWWHH QHHGV JROG VLOYHU DQG EURQ]H PHGDOV 'HVLJQ WKHLUQXPEHUPXVWEHVXIILFLHQWIRUDOOZLQQHUV,W
DQGSURGXFHDVXIILFLHQWQXPEHURIWKHP PXVWHYHQH[FHHGLWEHFDXVHLQVRPHGLVFLSOLQHV
1RWLFH WKDW WKH IRUPXODWLRQ GRHV QRW PHQWLRQ DQ\ PRUH FRPSHWLWRUV FDQ EH SRVLWLRQHG DW WKH VDPH
PDWKHPDWLFV ,W UDWKHU VRXQGV OLNH D PDQDJHULDO WDVN SODFH $OVR WKH WRWDO FRVW RI PHGDOV FDQ EH D
UHTXLULQJDWHDPZRUN$WWKHILUVWJODQFHWKHWHDPPXVW GHFLVLYH IDFWRU )RU H[DPSOH DV XVXDO ³JROG
FRQVLVW IURP LWV PDQDJHU DQ DUWLVW DQG D ' SULQWLQJ PHGDOV´DUHQRWPDGHIURPJROG7KH\DUHJROG
SURIHVVLRQDO,QUHDOLW\WKHWDVNLVPXFKPRUHFRPSOH[DV SODWHGRUMXVWJLOGHGXVLQJDQRWKHU\HOORZPHWDO
ZHGHPRQVWUDWHLQWKHQH[WVHFWLRQ 6ROYLQJ WKLV SUREOHP FDQ EH DQ H[FHOOHQW
PDWKHPDWLFDO FKDOOHQJH HYHQ LI ZH DUH
,9 ',9,',1*7+(7$6.,17268%7$6.6 FRQVLGHULQJPHGDOVPDGHIURP\HOORZJUH\DQG
)LUVW WKH PDLQ DLP RI WKH SURMHFW PXVW EH VSHFLILHG EURZQ'SULQWHUILODPHQW
+RZ PDQ\ PHGDOV KDYH WREH SURGXFHG DQG XQWLO ZKHQ" LL 7KH RUJDQL]LQJ FRPPLWWHH SUHVXPHV VRPH
7KH7KHQWDVNPXVWEHVSOLWLQWRVXEWDVNVWKHPDQDJHULDO H[SHQVHWRWKHPHGDOV1RWWRH[FHHGWKLVEXGJHW
DFWLYLWLHV PHGDO GHVLJQ ' SULQWLQJ HWF 7KH WHDP WKH GHVLJQHU KDV WR FDOFXODWH WKH DSSUR[LPDWH
PHPEHUUROHVDQGWKHZD\VRIWKHLUFROODERUDWLRQKDYHWR YROXPHRIPHGDOVDQGGHVLJQWKHPLQDFFRUGDQFH
EHGHILQHG WR WKH UHVXOWLQJ YDOXH ,W LV D IDFWRU OLPLWLQJ
7KH ' SULQWLQJ LV D SDUW RI D FROODERUDWLYH SURMHFW KLVKHUFUHDWLYLW\
DLPHGWRSURGXFHWKHVH'REMHFWVIRUVWXGHQWV¶SHUVRQDO LLL ,WWKHPHGDOZRXOGEHWRRPLQLDWXUHWKHUHFRXOG
XVDJH7KHREMHFW¶VORRNDQGVL]HDUHH[FOXVLYHO\XSRQWKH EH HDVLO\ EURNHQ 7KH ³WHFKQLFLDQ´ LQ WKH WHDP
JURXS GHFLVLRQ )LUVW WKH JURXS KDV WR PDNH D GHFLVLRQ VKRXOG GHFLGH ZKHWKHU WKH FDOFXODWHG DPRXQW RI
ZKDW REMHFW WR FUHDWH DQG WR VSHFLI\ LWV SXUSRVH DQG PDWHULDO LV VXIILFLHQW IRU WKH PHGDOV +HVKH
QXPEHU RI SLHFHV 7KHQ WKH REMHFW LV GHVLJQHG VKRXOG DOVR GLVFXVV WKH PHGDO VWUXFWXUH ± WKH
$IWHUZDUGV LW LV PRGHOOHG XVLQJ D VSDWLDO JHRPHWU\ ERG\GRHVQRWQHHGWREHVROLGLWFDQEHSRURXV
FRQFHSWV DQG WHFKQLTXHV ,Q WKH QH[W VWHS WKH PRGHO LV LHOLJKWHU,QRSSRVLWHLIWKHPHGDOLVIUDJLOHWKH
H[SUHVVHGDVDFRPSXWHUSURJUDPIRUWKH'SULQWHU7KH VWDQGDUG ILODPHQW FDQ EH UHSODFHG E\ D PRUH
ILQDO RXWFRPH LV H[SORLWHG E\ WKH JURXS PHPEHUV GXUDEOHRQH$JDLQLWVSULFHVKRXOGEHFDOFXODWHG
DFFRUGLQJWRWKHLUSODQ7KHUHLVDQRSWLRQDOSODQ DQGWKHUHVXOWGLVFXVVHGZLWKWKHVSRQVRU
L 7KH PDQDJHU PXVW OHDUQ WKH QXPEHU RI PHGDOV LY $IWHU WKH ILQDO FKRLFH RI WKH ILODPHQW WKH
DQGRWKHUUHTXLUHPHQWVRIHYHQWRUJDQL]HUV PDQDJHU VWDUWV RUGHULQJ LW +HUH KHVKH VKRXOG
LL 7KH PHGDO GHVLJQHU WKHQ SUHSDUHV VHYHUDO QHJRWLDWH LWV SULFH IRU H[DPSOH JHWWLQJ D
VNHWFKHVZKLFKWREHDSSURYHGE\WKHRUJDQL]LQJ UHGXFWLRQ LQ H[FKDQJH IRU SRVWLQJ ORJRV RI WKH
FRPPLWWHH VHOOHUZLWKLQWKHHYHQWSUHPLVHV$JDLQWKRVHDUH
WDONV EHORQJLQJ WR WKH HFRQRPLF FDWHJRU\ RI
LLL 7KH WHFKQRORJLVW ZLOO DQDO\VH WKH IHDVLELOLW\ RI ³YDOXH IRU PRQH\´ ,Q DGGLWLRQ WR WKH SUHYLRXV
SURGXFLQJ WKH VHOHFWHG SDWWHUQ DQG VSHFLI\ WKH ³SXUH´FDOFXODWLRQVQRZZHHVWLPDWHWKHYDOXHRI
PRVWDSSURSULDWHPDWHULDOWKHPHGDOGLPHQVLRQV LPPDWHULDO REMHFWV VSDFH IRU DGYHUWLVHPHQW
ZHLJKW DQG GXUDELOLW\ DV ZHOO DV RWKHU HVWLPDWHG SURILW IURP WKH DGYHUWLVHPHQW
WHFKQRORJ\IDFWRUV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 222

 UHDGLQHVVRIWKHVHOOHUWRFRQWULEXWHWRWKHVFKRRO FRPSRVLWLRQ RI D VWURQJ WHDPV LV D FULWLFDO
DFWLYLWLHVHWF  SUHVXPSWLRQ ,WV PHPEHUV KDYH WR EH TXDOLILHG DQG
Y 7KHGHVLJQHGPHGDOKDVWREHVSHFLILHGDVD' VNLOOHG WR DFFRPSOLVK DOO DERYH VWHSV $V D WHDP QRW
REMHFW XVLQJ 6SDWLDO *HRPHWU\ DQG WKHQ SULQWHG LQGLYLGXDOO\ EHFDXVH WKH\ DUH VXSSRVHG WR
XVLQJD'SULQWHU FRPPXQLFDWHDQGWRFROODERUDWH
YL 7KH ' SULQWLQJ LV D UDWKHU VORZ DFWLYLW\ 7KH • 7KHLQVWLWXWLRQPXVWDFFHSWFROODERUDWLYHHGXFDWLRQDV
WLPH IRU SULQWLQJ RQH SLHFH SUHGHWHUPLQHV WKH DQHTXLYDOHQWWRDOOWUDGLWLRQDORQHV,QSDUWLFXODUWKLV
SHULRG QHFHVVDU\ IRU FRPSOHWLQJ WKH SURFHVV ,Q DSSOLHV WR HYDOXDWLRQJUDGLQJ DQG LWV UHFRJQLWLRQ E\
IDFWWKLVWLPHVKRXOGEHFDOFXODWHGLQDGYDQFH± RIILFLDO ERGLHV 7KH VFKRRO VKRXOG DOVR FRQVLGHU
DPRQJWKHILUVWWKLQJVLQWKHSURMHFW LQWHUWZLQLQJ LWV VWXG\ SURJUDPV DOORZLQJ WR FRPSOHWH
1RWLFH WKDW HYHU\ RI WKH DERYH SRLQW UHTXLUH VRPH DSSURSULDWH SURMHFW WHDPV ,W VKRXOG LQYLWH WKH VWXG\
PDWKHPDWLFV ,Q3RLQW , WKHSULFH RI PHGDO LV DUHVXOWRI SURJUDPOHDGHUVWRFRRSHUDWHRQWKHLULPSOHPHQWDWLRQ
VHYHUDO FDOFXODWLRQV FRQVLGHULQJ LWV YROXPH DQG XVHG ± DW OHDVW RQH SDUWQHU IURP HYHU\ 67($0 ILHOG 
PDWHULDO,WLVDW\SLFDORSWLPL]DWLRQ OLQHDUSURJUDPPLQJ  7KHVH ZRXOGEHFRPH WKH HGXFDWRU WHDP ± WKH SDUWQHU
SUREOHP,Q3RLQW,,RQHVHHVWKDWWKHSULFHRIWKHRXWFRPH RI VWXGHQW WHDPV´ ,WV PHPEHUV ZRXOG EH JXDUDQWRUV
LVDOLPLWDWLRQRIDUWLVW¶VIUHHGRP7KLVDOVRLQGLFDWHVZK\ FRRUGLQDWLQJ DQG PDQDJLQJ WKLV LQQRYDWLYH FRXUVH
VRPH DUWLVW GLHG IURP VWDUYDWLRQ WKH\ ZHUH QRW UHDG\ WR VXSSRUWLQJ LWV LQWHUGLVFLSOLQDU\ FKDUDFWHU DQG
ORVH WKHLU IUHHGRP ± WKH\ UDWKHU ORVH WKHLU OLIH :H FRQWUROOLQJ LWV FRQWULEXWLRQ WR HYHU\ SDUWLFLSDWLQJ
FHUWDLQO\GRQRWH[SHFWWKLVIURPRXUVWXGHQWV GLVFLSOLQH
,Q 3RLQW ,,, RQH KDV WR RSHQ WKH LVVXH RI SURMHFW WLPH • 7KHHGXFDWRUVPXVWFRPSUHKHQGWKHTXDOLWLHVRIDFWLYH
VFKHGXOH$VZHVHHIURP3RLQW9,WKHVDIHVWPHWKRGRI OHDUQLQJ LWV DGYDQWDJHV DQG GUDZEDFNV 7KH WHDP
LWV FUHDWLRQ LV WR VWDUW IURP WKH HQG WKHSURMHFWGHDGOLQH JXDUDQWHHLQJ WKH LQQRYDWLYH FRXUVH VKRXOG PDNH
&RXQWLQJEDFNRQHFDQVHWXSWKHGHDGOLQHVRIWKHSURMHFW DJUHHPHQWV RQ RSWLPDO JURXSV¶ FRPSRVLWLRQ DQG VL]H
VWDJHV WKH SULQFLSOHV RI DVVLJQPHQW WR WHDP UROHV SURYLGLQJ
PDWHULDO WR JURXSV FRQWUROOLQJ WKHLU SURJUHVV HWF ,W
3RLQW ,9 GHPRQVWUDWHV DQRWKHU IDFH RI 0DWKHPDWLFV PXVW EH FDSDEOH RI LJQLWLQJ JURXS DFWLYLW\ DQG RI
2IWHQ LW FDQQRW EH SXQFWXDO EHFDXVH WKHUH DUH QR H[DFW FRQWUROOLQJ LWV SURJUHVV ZLWKRXW D GLUHFW JXLGDQFH DV
LQSXWYDOXHV,WDSSOLHVSULPDULO\WRLPPDWHULDOVLWXDWLRQV± ZHOO DV RI UHFRJQL]LQJ ZKHWKHU WKH WHDP GRHV QRW
-XGD¶V WKLUW\ VLOYHU DUH WKH H[DPSOH :KDW LV ³WKH ULJKW EOXQGHU ZLWKRXW FOHDU JRDOV DQGRU OHDGHUVKLS 7R GR
SULFH IRU WUHDVRQ´" 7KH VWXGHQWV VKRXOGGLVFXVV VRPH RI VR LW KDV WR EH DZDUH WR HYDOXDWH ZKHWKHU WKH JURXS
VLPLODU³LPSHUIHFWLRQV´UHODWHGWRWKHLUSURMHFW DSSURDFKHV HDFK RI LWV 67($0 JRDOV DQG LI
3RLQW9LVWKHRQO\RQHILWWLQJWRWKHWUDGLWLRQDOLPDJHRI QHHGHG RIIHU FRQVXOWDWLRQV7KH\ VKRXOG DOVR PDNH D
VFKRRO PDWKHPDWLFV 1DWXUDOO\ LQ VXFK FDVH WKH PHGDO FRQVHQVXV RQ WKH OHYHO RI WKHLU LQYROYHPHQW 1R RQH
GHVLJQVKRXOGEHGRQHWRVWXGHQWVDQGWKH\VKRXOGOLWHUDOO\ VKRXOG DGYLVH WKH JURXS ZLWK PXFK KLJKHU RU PXFK
IROORZ LW 7KDW¶V ZK\ ZH LQVLVW RQ WKH VWXGHQWV¶ ORZHU LQWHQVLW\WKDQWKHUHPDLQLQJHGXFDWRUV
LQGLYLGXDOO\ SURSRVHG GHVLJQ 1RW RQO\ HYHU\ JURXS ZLOO
• 7KH VWXGHQWV PXVW KDYH VXIILFLHQW SUHUHTXLVLWH
SURGXFHGLIIHUHQWGHVLJQEXWLWDOVRZLOOKDYHWRPDNHLWV
NQRZOHGJH ± WKH VXP RI WKHLU NQRZOHGJH VNLOOV DQG
LQGLYLGXDO ' PRGHO 2QO\ WKH JURXS PHPEHUV ZLOO EH
H[SHULHQFHVKRXOGH[FHHG DWOHDVWWREHVXIILFLHQW IRU
FDSDEOHWRH[SODLQZKDWWUDQVIRUPDWLRQZDVGRQHKRZDQG
VROYLQJ WKH WDVN %DVLFDOO\ LW PHDQV WKDW WKH JURXS
ZK\ 1DWXUDOO\ ZH SUHVXPH WKDW WKH ³PDWKHPDWLFLDQ´ DV
PHPEHUV VKRXOG FRYHU DOO QHFHVVDU\ 67($0
WKHOHDGLQJDFWRUZLOOH[SODLQWRWKHRWKHUVWKHPRGHOOLQJ
EUDQFKHV $V D JURXS WKH\ PXVW XQGHUVWDQG WKH JRDO
SULQFLSOHV DQG VWHSV ,Q UHWXUQ WKH RWKHU WHDP VSHFLDOLVWV
DQG EH FDSDEOH WR GLYLGH LW WR VXEWDVN (YHU\
ZLOO HOXFLGDWH KLPKHU WKHLU VSHFLILF DUWLVWLF WHFKQRORJ\
³VSHFLDOLVW´VKRXOGEHFDSDEOHWRVROYHWKHSUREOHPVRI
HFRQRP\DQGVFLHQWLILFUHDVRQLQJ
KLVKHU ILHOG EH UHDG\ WR FROODERUDWH DQG WR EH
7KHSURMHFWVROXWLRQSUHVHQWVWKHXELTXLWRXVSUHVHQFHRI DFFRXQWDEOH IRU KLVKHU SDUWLDO RXWFRPH ,Q DGGLWLRQ
PDWKHPDWLFVLQLWVYDULRXVIDFHV±VWDUWLQJIURPWUDGLWLRQDO HYHU\RQH VKRXOG EH FDSDEOH RI H[SODLQLQJ KLVKHU
SUREOHPV OLNH FDOFXODWLRQ RI D YROXPH WKURXJK DFWLYLW\ WR WKH UHVW RI JURXS DQG ± LQ WKLV ZD\ ± WR
RSWLPL]DWLRQV WR FRPSOH[ XVLQJ VSDWLDO JHRPHWU\ ,Q FRQWULEXWHQRWRQO\WRWKHSURGXFWFRPSOHWLRQEXWDOVR
DGGLWLRQ WR WKLV LW DOVR VKRZV WKH QHFHVVLW\ WR DSSO\ WKH IRUWKHLUPHQWDOVKLIW
UHVXOWVZLVHO\IRUH[DPSOHSXUFKDVLQJH[DFWO\FDOFXODWHG
QXPEHU RI ILODPHQWV FRXOG OHDG WR WKHLU VKRUWDJH GXH WR 9,, &21&/86,21
VPDOO ORVHV DV D UHVXOW RI WHFKQRORJ\ WROHUDQFH 7KH
³WHFKQLFLDQ´ PD\ H[SODLQ WR LWV WHDP WKDW SXUFKDVLQJ 7KHWLWOHRIWKLVSDSHUVSHDNVDERXW'0RGHOOLQJDQG
DGGLWLRQDO  RI PDWHULDO ZLOO SUHYHQW LWV GHILFLHQF\ 'SULQWLQJ'XHWRWKHLULQGLUHFWUHODWLRQWRPDWKHPDWLFV
ZKLFK FRXOG UHVXOW LQ D GHOD\ FDXVLQJ PLVVLQJ WKH WKHVH DQG VLPLODU ZRUGV PD\ LQVWDQWO\ JHQHUDWH FRQFHUQV
GHDGOLQH LQ VWXGHQWV¶ PLQGV 7KDW¶V ZK\ WKH IRUPXODWLRQ RI WKH
SUREOHP WU\ WR DYRLG PHQWLRQLQJ WKHP >@ $V ZH KDYH
9, 0$.,1*$&7,9(/($51,1*3266,%/( VHHQ WKH WDVN FRPSOHWLRQ UHTXLUHV D ORW RI PDWKHPDWLFDO
UHDVRQLQJDQGFDOFXODWLRQVDQGGHPRQVWUDWHVLWVUROHLQRXU
7R LPSOHPHQW WKLV W\SH RI HGXFDWLRQ D VHULHV RI HYHU\GD\ OLIH +RZHYHU WKH\ DSSHDU LQ WKH FRXUVH RI
SUHUHTXLVLWHVPXVWEHIXOILOOHG VROXWLRQ DQG WKH OHDUQHUV XQGHUVWDQG WKHLU QHFHVVLW\ IRU
• $ VXIILFLHQW QXPEHU RI ' SULQWHUV WR VWXGHQWV¶ DFKLHYLQJ WKHLU JRDO .QRZLQJ LW EXW DOVR D ORW RI RWKHU
GLVSRVDO LV D QHFHVVDU\ FRQGLWLRQ ± EXW QRW WKH FRPSRQHQW DV HFRQRPLFV PDQDJHPHQW PDWHULDO VFLHQFH
VXIILFLHQW $V WKH SULQWLQJ LV XVHG LQ WKH HQG RI WKH HWF  LV WKH ³SUHUHTXLVLWH NQRZOHGJH´ PHQWLRQHG LQ WKH
SURMHFW ZKHQ DOO RWKHU REVWDFOHV ZHUH RYHUFRPH WKH WLWOH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 223

 $W WKH VDPH WLPH HGXFDWRUV KDYH WR GHPRQVWUDWH >@ -RKQVRQ 5 7  -RKQVRQ ' :   $FWLYH OHDUQLQJ
SUHUHTXLVLWH NQRZOHGJH RI RWKHU VRUW 7R DSSO\ WKH &RRSHUDWLRQ LQ WKH FODVVURRP7KH DQQXDO UHSRUW RI HGXFDWLRQDO
SV\FKRORJ\LQ-DSDQ
GHVFULEHG HGXFDWLRQDO DSSURDFK SURSHUO\ WKH KDYH WR
>@ 'DYLGVRQ 1  0DMRU & +   %RXQGDU\ FURVVLQJV
LQYLWHWKHLUVWXGHQWVWRGHPRQVWUDWHPXFKPRUHWKDQWKHLU &RRSHUDWLYH OHDUQLQJ FROODERUDWLYH OHDUQLQJ DQG SUREOHPEDVHG
PHPRUL]HGNQRZOHGJH&XUUHQWO\VXFKH[DFWIRUPXODWLRQV OHDUQLQJ-RXUQDO RQ H[FHOOHQFH LQ FROOHJH WHDFKLQJ    
DUHFRPPRQSUDFWLFH>@ 
$W\SLFDOOLIHF\FOHRIWKHVHREMHFWVVKRXOGEH³'HVLJQLW >@ 7HMHUD0$JXLODU*  /DYLF]D=  0RGHOOLQJDQG'
± 3URGXFH LW ± 0DNH XVH RI LW´ ,Q WKLV ZD\ DOO SDUWQHUV 3ULQWLQJ $UFKLWHFWXUDO 0RGHOV $ :D\ WR'HYHORS 67($0
ZLOOEHQHILWRILW'XHWRWKHVORZSULQWLQJDW'SULQWHUV 3URMHFWVIRU0DWKHPDWLFV&ODVVURRPV,Q/HDUQLQJ0DWKHPDWLFVLQ
WKH&RQWH[WRI'3ULQWLQJSS±6SULQJHU1DWXUH
WKH VL]H RI RXWFRPHV VKRXOG EH TXLWH VPDOO GHVLJQHU
>@ (O%HGHZ\6&KRL./DYLF]D=)HQ\YHVL. +RXJKWRQ
EXWWRQV MHZHOOHU\ ULQJV DQG VLPLODU 2Q WKH RWKHU KDQG 7   67($0 3UDFWLFHV WR ([SORUH $QFLHQW $UFKLWHFWXUHV
JUDWHUREMHFWVFDQEHSUHSDUHGDVVHWRIFRPSRQHQWVWREH 8VLQJ$XJPHQWHG5HDOLW\DQG'3ULQWLQJZLWK*HR*HEUD2SHQ
DVVHPEOHGODWHU6XFKDWDVNDOVRUHSUHVHQWVDWHFKQRORJ\ (GXFDWLRQ6WXGLHV  
FKDOOHQJHDQGFDQEHZHOFRPHE\VRPHJURXSV >@ 'XLW50LNHOVNLV6HLIHUW6 :RG]LQVNL&7  3K\VLFV
7KH IDFXOW\ DQG VFKRRO PDQDJHPHQW PXVW XQGHUVWDQG LQ &RQWH[W ± $ SURJUDP IRU ,PSURYLQJ 3K\VLFV ,QVWUXFWLRQ LQ
*HUPDQ\ (Q 5 3LQWy  ' &RXVR (GV  &RQWULEXWLRQV IURP
WKHVSLULWRIFROODERUDWLYHOHDUQLQJDQGLQFOXGHLWLQWRWKH 6FLHQFH(GXFDWLRQ5HVHDUFK SS 6SULQJHU1HWKHUODQGV
VFKRRODWPRVSKHUHWRSUHVHQWDSRVLWLYHDWWLWXGHWRLWDQG
>@ /LHEDQ '   ([SORULQJ RSSRUWXQLWLHV IRU FRQQHFWLQJ
WR SURYLGH WKHLU VXSSRUW 2IIHULQJ SURMHFW LGHDV LV D ZD\ SK\VLFDO DQG GLJLWDO UHVRXUFHV IRU PDWKHPDWLFV WHDFKLQJ DQG
GHVLJQLQJ DQG SURGXFLQJ PHGDOV LV LWV H[DPSOH ,Q WKLV OHDUQLQJ>'RFWRUDOWKHVLV@-RKDQQHV.HSOHU8QLYHUVLWlW/LQ]
ZD\ WKH\ GHPRQVWUDWH WR WKHVWXGHQWV WKDW WKH LQVWLWXWLRQ >@ %OXP: /HL‰'  +RZGR6WXGHQWVDQG7HDFKHUV'HDO
EHOLHYHVLQWKHLUFDSDELOLWLHV,QYHUVHO\WKHVFKRROZLOOEH ZLWK 0RGHOOLQJ 3UREOHPV" ,Q 0DWKHPDWLFDO 0RGHOOLQJ SS 
UHZDUGHGE\RULJLQDOLWHPVVWXGHQW¶VDGYDQFHGNQRZOHGJH (OVHYLHU
DQGVNLOOV±DQGDSXEOLFUHODWLRQVKLSSRWHQWLDO >@ 9LOOD2FKRD - $  %HUUtR 0 -   0DWKHPDWLFDO
0RGHOOLQJ DQG &XOWXUH $Q (PSLULFDO 6WXG\ (Q * $ 6WLOOPDQ
:KDW IHDWXUHV LQ WKLV ZD\ RI WHDFKLQJ DUH QRW MXVW : %OXP  0 6DOHWW %LHPEHQJXW (GV  0DWKHPDWLFDO
PHPRUL]HG SLHFHV RI NQRZOHGJH" 7KRVH W\SLFDO IRU 0RGHOOLQJ LQ (GXFDWLRQ 5HVHDUFK DQG 3UDFWLFH &XOWXUDO 6RFLDO
FROODERUDWLYH OHDUQLQJ OHDGHUVKLS DELOLW\ WR FROODERUDWH DQG &RJQLWLYH ,QIOXHQFHV ± 6SULQJHU ,QWHUQDWLRQDO
SDWLHQFH H[FHOOHQW FRPSOHWLRQ RI GXWLHV LQQRYDWLYHQHVV 3XEOLVKLQJ
UHDGLQHVV WR KHOSDQG RWKHUV >@7KHOHDUQHUV JHW JURXS >@ +YRUHFNê-  0DQDJHULDO,VVXHVRI7HDFKLQJ0DWKHPDWLFV
PHPEHUVKLS IHHOLQJ DQG VWDUW XQGHUVWDQGLQJ WKDW 3URFHHGLQJVRIWKHWZHOIWK$7&07DLSHL7DLZDQ
WHDPZRUNLVDQHIIHFWLYHPHWKRGRIRYHUFRPLQJLQGLYLGXDO >@ 8OEULFK( $QGMLF%  /DYLF]D=  3RVVLELOLWLHV IRU
GHILFLHQFLHV 7KHLU WHDPZRUN SUDFWLFHV ZLOO PDNH WKHP 67($0WHDFKHUVXVLQJ'PRGHOOLQJDQG'SULQWLQJ0,1786±
PRUH UHVLVWDQW WR VWUHVV EHFDXVH WKHLU LQGLYLGXDO %HLWUlJH]XUPDWKHPDWLVFKQDWXUZLVVHQVFKDIWOLFKHQ%LOGXQJ
FRQWULEXWLRQV WR WKH ILQDO RXWFRPH ZLOO WHDFK WKHP WKDW >@ *RQVRXOLQ-U:%:DUG5( )LJJ&  /HDUQLQJE\
OHDGLQJ8VLQJEHVWSUDFWLFHVWRGHYHORSOHDGHUVKLSVNLOOVLQDWULVN
ZLWK JRRG ZLOO DQG PXWXDO HIIRUWV DOO WURXEOHV FDQ EH DQGJLIWHGSRSXODWLRQV(GXFDWLRQ  
RYHUFRPH >@ 'HZL\DQWL 6 %UDQG*UXZHO 6 -RFKHPV :  %URHUV 1 -
,QDGGLWLRQFROODERUDWLYHOHDUQLQJLVDUREXVWPHWKRGRI   6WXGHQWV¶ H[SHULHQFHV ZLWK FROODERUDWLYH OHDUQLQJ LQ
HGXFDWLRQ DQG FDQ EH VXFFHVVIXOO\ FRPSOHWHG DOVR LQ DV\QFKURQRXV FRPSXWHUVXSSRUWHG FROODERUDWLYH OHDUQLQJ
EOHQGHG DQG FRPSXWHUVXSSRUWHG IRUPDWV >  @ ,W HQYLURQPHQWV&RPSXWHUVLQ+XPDQ%HKDYLRU  
PHDQV WKDW DSSO\LQJ LW LV D SUHYHQWLYH PHWKRG GXULQJ >@ 0XxR]&DUULO 3 & +HUQiQGH]6HOOpV 1 )XHQWHV$EHOHGR (
IXWXUHSDQGHPLFDQGRWKHUXQH[SHFWHGLQWHUUXSWLRQVRIRQ -  *RQ]iOH]6DQPDPHG 0   )DFWRUV LQIOXHQFLQJ
VWXGHQWV¶ SHUFHLYHG LPSDFW RI OHDUQLQJ DQG VDWLVIDFWLRQ LQ
VLWHHGXFDWLRQ &RPSXWHU 6XSSRUWHG &ROODERUDWLYH /HDUQLQJ&RPSXWHUV 
(GXFDWLRQ
5()(5(1&(6 >@ 6R + -  %UXVK 7 $   6WXGHQW SHUFHSWLRQV RI
>@ ,R721( ,QGLYLGXDOL]HG 3URGXFWLRQ ± 2YHUYLHZ FROODERUDWLYH OHDUQLQJ VRFLDO SUHVHQFH DQG VDWLVIDFWLRQ LQ D
KWWSVZZZLRWRQHFRPWHUPLQGLYLGXDOL]HGSURGXFWLRQW EOHQGHG OHDUQLQJ HQYLURQPHQW 5HODWLRQVKLSV DQG FULWLFDO
IDFWRUV&RPSXWHUV HGXFDWLRQ  
>@ <HDJHU ' 6 +HQGHUVRQ 0 ' 3DXQHVNX ' :DOWRQ * 0
' 0HOOR66SLW]HU%- 'XFNZRUWK$/  %RULQJEXW
LPSRUWDQW $ VHOIWUDQVFHQGHQW SXUSRVH IRU OHDUQLQJ IRVWHUV
DFDGHPLF VHOIUHJXODWLRQ-RXUQDO RI 3HUVRQDOLW\ DQG 6RFLDO
3V\FKRORJ\  ±
>@ +YRUHFNê- 5R]HKQDO3  9LUWXDO5HDOLW\(QYLURQPHQW
DV D 8QLYHUVLW\ 2QOLQH (GXFDWLRQ 3ODWIRUP ,Q WK
,QWHUQDWLRQDO &RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ 7HFKQRORJLHV
DQG$SSOLFDWLRQV ,&(7$ SS,(((
>@ /DLQH 7 +  /HH :   &ROODERUDWLYH 9LUWXDO 5HDOLW\ LQ
+LJKHU(GXFDWLRQ6WXGHQWV 3HUFHSWLRQVRQ3UHVHQFH&KDOOHQJHV
$IIRUGDQFHV DQG 3RWHQWLDO ,((( 7UDQVDFWLRQV RQ /HDUQLQJ
7HFKQRORJLHV
>@ )XFKVRYD 0  .RUHQRYD /   9LVXDOLVDWLRQ LQ %DVLF
6FLHQFH DQG (QJLQHHULQJ (GXFDWLRQ RI )XWXUH 3ULPDU\ 6FKRRO
7HDFKHUV LQ +XPDQ %LRORJ\ (GXFDWLRQ 8VLQJ $XJPHQWHG
5HDOLW\(XURSHDQ MRXUQDO RI FRQWHPSRUDU\ HGXFDWLRQ   

>@ %UDPH &   $FWLYH OHDUQLQJ9DQGHUELOW 8QLYHUVLW\ &HQWHU
IRU7HDFKLQJ
>@ 7RQJ 6  $FWLYH OHDUQLQJ WKHRU\ DQG DSSOLFDWLRQV
6WDQIRUG8QLYHUVLW\

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 224

 Small and affordable platform for research and
education in Connected, Cooperative and
Automated Mobility
1st Matej Janeba 2nd Samuel Bohumel 3rd Jakub Hrnčár 4th Dr. Marek Galinski 5th Prof. Ivan Kotuliak
FIIT STU FIIT STU FIIT STU FIIT STU FIIT STU
Bratislava, Slovakia Bratislava, Slovakia Bratislava, Slovakia Bratislava, Slovakia Bratislava, Slovakia
matej.janeba@stuba.sk xbohumel@stuba.sk xhrncar@stuba.sk marek.galinski@stuba.sk ivan.kotuliak@stuba.sk

Abstract—In the recent, massive effort has been put into re- systems [3]. Data collection is one of the most crucial steps for
search and education on connected, collaborative and automated future improvements of systems. Furthermore, the improve-
mobility to improve road safety and the affordability of mobility. ment in the education of young scientists in the field could
New data sets must be created to maintain and enhance the speed
of innovations in the field. Therefore, a simple and affordable potentially speed up innovations and improvements in road
solution for data collection from mobility scenarios is crucial to safety applications. The authors propose small and affordable
future improvements in the field. This paper introduces the design evaluation and prototyping platform design for research and
of small and affordable platform for research and education in education in Connected Cooperative and Automated Mobility
the field of connected, collaborative, and automated mobility. The (CCAM).
solution provides real-time and historical data analysis in various
mobility use cases. The solution offers efficient and reliable data In Section Related work, authors describe actual state-of-
capture, processing, and analysis, making it a valuable tool for the-art in context of prototyping, research, and evaluation
research and education on smart mobility. The mobile interface platforms for CCAM. In the Section Proposed solution, design
allows for live data collection, while the desktop interface offers for the solution is described. The Section Verification and
data visualization for historical data analysis. With its ability to performance evaluation, simple verification, user experience
capture data on traffic conditions, vehicle speed, network latency,
and bandwidth, the smart mobility device provides valuable testing and user interface performance is discussed. In the Sec-
insights into mobility operations that can be used to optimize tion Conclusion, final thoughts and future work are described.
transportation efficiency.
Index Terms—CCAM, mobility data collection, data collection II. R ELATED WORK
device In the article Roads infrastructure digital twin: A step
toward smarter cities realization [4] the authors proposed to
I. I NTRODUCTION use a NVIDIA Jetson as a data collection unit together with
Mobility is of critical importance in large cities and is a appropriate sensors; however, the data are being fed into a
key aspect of urbanization. In 2050, more than 20 billion Machine Learning (ML) model and applied in the detection of
trips can be made each day, therefore we should be interested vital road infrastructure. This is a slightly different use case
in safety during transport. The fundamental goal of smart than the solution proposed by this paper. As with any ML,
mobility research is to examine current circumstances, process the authors have encountered challenges in the performance
data, and develop new strategies that can be used to make bottlenecks of the system. Another downside is the lack
transportation more efficient, safe, and affordable of built-in visualization solution - the researchers need to
Cities are utilizing the Internet and communication tech- generate his own graphs.
nology to build a contemporary, environmentally friendly and In paper Testing the Readiness of Slovak Road Infras-
efficient environment while providing essential services to the tructure for the Deployment of Intelligent Transportation, an
population. Numerous aspects including smart streets, parking, illustration of real-world use case for this tool is described.
pedestrian management, traffic management, navigation sys- The authors proposed a solution to test the conditions of the
tems, or transportation services are included in smart mobility. road infrastructure [5] such as vertical signs and visibility of
Some of these options, like cooperative collision avoidance horizontal lanes, as well as network connectivity, which will
or intelligent safety mechanisms for venerable road users, be crucial in terms of CCAM.
are rarely used. Therefore, there is definitely room for future In the paper Evaluation of qualitative parameters of LTE and
research and advancement of these aspects [2] . 5G for V2X use cases, the author proposed stream-based mea-
To improve research and education in the field, many surement of the network [6]. The authors compared the long-
technologies are being developed, including digital twins , term evolution cellular network with the non-standalone 5G
automated driving systems , and intelligent transportation cellular network. The results showed that in terms of response

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 225

time and jitter, non-standalone 5G was slightly better than every scenario. To enhance simulations, one needs to acquire
long-term evolution. However, the results, authors proposed data sets that could inform simulations to possibly improve
in conclusion a better way to measure network parameters in simulation results.
the future.
The importance of conducting research and experiments III. P ROPOSED SOLUTION
for network connectivity in real-life scenarios can be seen in
A. Device modules
the paper Evaluating Slovak Roads for Vehicle-to-Everything
Communication 3GPP Requirements [7]. The authors of the The proposed solution is divided into three separate logical
paper conducted multiple repeated experiments on different blocks, as depicted in Figure 2.
classes of Slovak roads and compared the results with 3GPP In the center of the design is the On Board unit (OBU),
requirements for different automation scenarios. Thus, the which in the case is represented by the Nvidia Jetson TX2
requirements in many cases have not been met, and further with the JN30D expansion card [11]. The Nvidia Jetson was
research and evaluations needs to be conducted. Importantly, chosen due to its capabilities of providing high computational
the authors used the ping and Iperf [8] utilities for the network power together with relatively low energy consumption. The
speed and response time to the server. To ensure further OBU can be easily expanded with storage capacity to provide
evaluation of different roads, the proposed solution needs to more space for data collection.
be low cost and has to be equipped with a network connection As sensors, a camera, GNSS sensor, and mobile internet
module. In addition, the convention of measuring two different access modem have been utilized for data collection. The
response times will be implemented in the solution. The authors chose Logitech C920 Pro [12] as the video source for
the data set. The camera is able to provide frame rates around
30 frames per second in FullHD resolution. As a GNSS sensor,
the Ublox ZED-F9P module has been chosen [13]. The module
has been chosen due to the ability to process Real-Time kine-
matics correction data to achieve precision of measurement of
14,1 millimeters. As depicted in figure the solution provides

Sensors On Board Unit User web Interface

REST API
Camera Logitech
C920 Pro
WebSocket Tablet Application

Fig. 1. SUMO simulation U-blox GNSS Jetson TX2 NX

ZED - F9P REST API

Simulation of Urban Mobility (SUMO) project is a great case WebSocket Desktop Application
of how to use data collected under actual settings to address
Teltonika RUTX50
difficulties in daily life. Large cities struggle with heavy
traffic, mainly because the number of automobiles is increasing
while the capacity of the roadways stays the same. SUMO
is a realistic traffic simulator with the primary objective of Fig. 2. Block diagram of proposed solution
estimating the volume of traffic in particular places. It is based
on ITS, an information and communication technology (ICT) two separate user interfaces. Two separate use cases have been
communication system. Using this technology can increase the considered for user interfaces. First, recording and viewing
efficiency and security of traffic delivery. Individual vehicle actual data in real time was considered. Second, viewing
behaviors, which are explicitly modeled to move in the system, already pre-recorded data and doing fast overview over the
can be processed using simulations [9]. Also, data collected measurements in terms of plots and features showing maps
from a real world environment can be used as a base for has been implemented. Even though both applications can be
simulation to achieve higher precision in the simulated result. used in each of the mentioned scenarios, tablet applications
Therefore, the proposed solution can collect positioning data are optimized for life measurements recording because of
by GNSS sensor. the nature of the task. In the other hand, the desktop web
Moreover, the authors of the virtual simulation environment application is optimized towards the second scenario.
of the autonomous A-EVE vehicle [10] were able to simulate
the mobility environment and vehicles, as well as the sensors B. On board unit as a server
itself. Unfortunately, the authors were unable to compare On board unit can be decoupled into two separate logical
the results with those of the real-life A-EVE vehicle. Also, blocks. At first,an application server, which provides applica-
there are infinite possible situations that could happen on tion programming interface for user interface and managing of
the road; therefore, there is no possible way to simulate parallel data capture. The second logical block is composed

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 226

from multiple simple data acquisition programs - one for each
sensor. Live measurement
For the purpose of serving the data, Django framework for
Python [14] has been chosen as a base of the server. Due to the Client Server
versatility of the framework, which is well-suited for building
complex applications, further expansion of design could be
done with ease. Django is capable of communicating with
other components of a smart mobility system, such as IoT
devices and mobile applications. Django channels are used for SYN
live measurements and playback history measurements, which
is a library for building real-time applications using Websock- SYN ACK
ets, allowing for efficient and immediate data transmission. ACK
As mentioned, the logic of capturing data from sensors was
moved to a compiled language C++. As the code is translated WebSocket Upgrade Request
into machine code during the compilation process, small
multiple applications written in the C++ language provides WebSocket Upgrade Response
better performance than the interpreted language, particularly
message: start
for applications that require computation or real-time data
processing. Moreover, C++ offers better low-level control over message: ...
the hardware, which is important for applications that require
direct access to sensors or other hardware components. ....
Due to decoupled logic, solution can work in two modes, message: ...
as measurement and evaluation device, and as only-recorded-
data-provider platform. For instance, an application server can message: stop
be deployed in the cloud, without the ability to perform life
measurements, only as a highly available server for serving
already recorded data at any time. This mode provides great
availability of recorded data, which can be accessed at any Fig. 3. Live data sequence diagram
time, even if the OBU is turned off. Second mode is a
standalone mode, where OBU acts as described before as a
server and data acquisition platform at the same time. The 4) Data Transfer: A JSON containing all the measured
combined regime was also tested in the development process, values is parsed into string and sent to the client. What
where the application server was deployed in the cloud as is not shown in this diagram is that we provide only
well as in the OBU. In the user interface, two endpoints were numerical data for this connection. Photographs are sent
specified, first for live measurements and second for history over a separate WebSocket, which mimics the behavior
data. The combined mode has not been verified, therefore, of this parent WebSocket.
future work needs to be done in this mode. In the near future, 5) Stop message: When the client decides to stop the
the author will focus on the topic. measurement, this message is relayed to the server,
where the entry in the database has been given the
C. Client-Server communication ending timestamp.
As previously stated, communication is handled by REST
API and WebSockets. However, WebSocket communication D. Data acquisition
is considered to be the primary method, since the primary As mentioned above, at the time, the solution can acquire
function of the application is being handled by them. To better data from three sensors. Arguably, the most important data
illustrate this, we provide a sequence diagram for live data: 3 source in terms of future mobility solutions will be the
camera due to its cost efficiency and value of data. For
1) TCP Three-way handshake: Standard connection estab- efficient processing and time consumption, built-in Nvidia
lishment procedure for TCP. libraries together with FIFO queues have been used. Built-in
2) WebSocket handshake: WebSocket connection is only libraries provide native support of Jetson’s powerful on-board
an upgrade over the existing HTTP. GPUs, which can process images in significantly lower time
3) Start message: In a live measurement, we command the than CPU. Queues have been used to utilize CPU time and
server to start the measurement by this message. On the to counteract occasional CPU core congestion. Queues also
server side, it creates a new entry in the database and enable the solution to utilize multiple cores to process images
saves the timestamp, after which the data transfer can and not lose track of the order in which image frames have
begin. been captured.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 227

 The U-blox ANN-MB-00-00, Multiband Active GNSS an- suitable for real-time systems. Despite the advantages, parallel
tenna together with U-blox ZED F9P GNSS module is used to processing also introduces multiple challenges that need to
provide accurate positioning and timing information essential be addressed. The biggest challenge is to synchronize data
for tracking applications in the field of smart mobility. The from multiple parallel processes with multiple different data
following information is collected from a sensor along with refresh rates. As depicted in Figure 4, a solution similar to
the acronyms that are used in measurements. the solution used in the paper Testing the Readiness of Slovak
• Latitude (lat): The angular distance of a location north or Road Infrastructure has been used [?].
south of the Earth’s equator, measured in degrees.
• Longitude (lon): The angular distance of a location east Data
or west of the prime meridian, which is an imaginary line Primary data source synchronization
proces
that passes through the Royal Observatory in Greenwich,
London, England, measured in degrees.
• Height above mean sea level (hmsl): The vertical distance Small volume
Auxiliary data source Data
of a location above the average level of the sea, measured data - csv file

Data set record

1 buffer 1
in meters.
• GPS Ground Speed (ggs): Speed at which an object is
moving over the ground, measured in meters per second Auxiliary data source Data
2 buffer 2
or kilometers per hour.
• GPS Course (gc): It is the direction in which an object is

......

......
moving over the ground, measured in degrees clockwise High volume
data, camera
from the true north. Auxiliary data source Data frames
• Horizontal accuracy (hacc): It is the measure of the n buffer N
accuracy of the GPS position in the horizontal plane. It is
expressed in meters and represents the radius of a circle Legend:
centered on the reported position within which the true Synchronized data High volume data
location is expected to lie with a certain probability. Asynchronous data Small volume data
Data sync request
As mentioned, Teltonika RUTX50 has been utilized to pro-
vide connection to the network. Recently, great effort has been
put into researching and advancing the performance of wireless Fig. 4. Diagram of synchronizing data
technologies. Fundamentally, decreasing the response time
and increasing the transmission bandwidth is of paramount As Figure 4 shows, in the approach there are two types
importance in terms of CCAM. Currently, in Slovakia two of data sources considered. The primary data source needs
main technologies for cellular communication are accessi- to be one with the highest data refresh rate in the system.
ble by public, long-term evolution (LTE) network and new Furthermore, the refresh rate of the primary data source will
5G radio (5G-NR) which can be also called non-standolone become the refresh rate of the system itself. Auxiliary data
5G network. The network modem was chosen accordingly; sources update its data buffers at their own independent refresh
Teltonika RUTX50 provides capabilities to connect to both rates. The data synchronization process is trigger each time,
networks. Despite the fact that Teltonicka’s devices could be when the primary data source obtains new data from the
identified as industrial-grade devices, the team encountered environment. Afterwards data synchronization process sends a
unidentified issues with connection to the cellular network sync request to pull the data from auxiliary data source buffers.
and, in times, the device needed to be restarted to solve the Synchronized data are then transformed into data set record in
problem. Two network parameters have been measured, the form of small volume data which can be represented in form
response time and the network speed on the application layer. of text or numbers, and in form of high volume data such as
Response time has been measured to two servers - edge server camera image which are represented by png files f.i. . Small
and towards the cloud. For the measurements ping utility was volume data are then stored into a csv formatted file which
used. Network speed has been measured consequently, first in could be easily imported into any data processing tool.
the upload direction and then in the download direction. Both
measurements have been conducted for 5 seconds and then the F. User interface
mean of the measurements was used as a measurement result.
For continued measurement, at least three separate network As mentioned before, the user interface is split into 2
devices could be utilized. applications: a desktop and a tablet application. The main
building blocks are HTML, CSS and TypeScript with Vue.js
E. Data synchronization 3 framework. This technological stack can be considered an
In the solution, the paradigm of paraller procesing has been industry standard; they are all stable and tested technologies
utilized. Parallel processing offers faster data processing due to with committed support for the foreseeable future, which was
better CPU utilization, which provide improved performance the main reason behind choosing this stack.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 228

 In particular, Vue.js offers simplicity and ease of use be-
cause its syntax and structure were designed to be intuitive
and straightforward. The framework also offers a flexible and
modular architecture, which was a feature that was used to a
considerable degree. However, a key benefit of Vue.js is its
impressive performance. Its headline feature, called ’Virtual
DOM’, enables fast and efficient rendering of components,
even for complex applications. Server communication is han-
dled by REST API client Axios and native WebSocket API.
Web sockets provide the application with the test data, while
the REST API is used by common requests that do not require
full-duplex communication. This solution and approach have
been tested by researchers [15]. The analysis will be carried
out mostly on numerical and geospatial data. The former is
visualized in multiple charts that display vital test data (e.g.
ping). The open source library Chart.JS has been chosen as
simplistic and easily imported into any application. Another
well-used and established solution, called OpenLayers, has
been used to display the data. A screenshot of the user
interface can be seen in Figure 5.

Fig. 6. Comparison of idle and scripting time, with different rendering

intervals.

can shorten the delay between capturing data and displaying

data in plots. Based on acquired information from the test,
authors decided to update data every 600 milliseconds to
maintain data relative fresh and also to ensure responsiveness
of the application. Furthermore, more idle time means less
energy consumption which needs to be addressed as well
according to the fact that the application utilizes tablet as
deployment platform.
As for the user experience test, results showed two common
Fig. 5. Screenshot of desktop application. problems with tablet applications:
• Lack of action awareness. Users have to guess where to
click, and some actions had to be guessed. This has been
IV. V ERIFICATION AND PERFORMANCE EVALUATION resolved by adding additional buttons to the actions to
Solution has been verified in terms of correctly displaying make the user interface more intuitive.
the data into plots. For the verification, pre-recorded data • Plot interactivity - Users complained about the lack of
have been used. The data was displayed as raw text on the interactivity with the plots. Complains was resolved by
server side and then plotted on the front-end side. The values an additional feature, the ability to zoom and pan across
were then compared and showed no difference, and hence the the plots.
fronted showed the data as it should. Both of the main complaints from the user experience test
In terms of evaluation, the front end rendering perfor- have been addressed and resolved accordingly. In the future,
mance test, and user experience test of tablet app have been energy consumption measurement of the whole system needs
conducted. During the development of the system, the team to be performed to ensure that the system as a whole is
encountered performance issues when the applications were sufficient in terms of energy consumption.
used for displaying and recording data in real time. Therefore,
front end performance evaluation tests have been conducted to V. C ONCLUSION
find render time which satisfies the need of showing actual data The paper proposes a small affordable solution for collecting
and ensures responsiveness of interface. The results of the test data from mobility scenarios. The proposed solution can
are depicted in Figure 6. collect, record, and replay recorded data, which could then be
As plotted in the figure, the dependency between idle time used for future data science work. The solution also provides
and scripting time can be seen. More idle time improves the an interface to display recorded data to enhance the ability
responsiveness of the app. On the other hand, scripting time of students and researchers to analyze data collected by the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 229

device faster. The solution is designed as a vendor-independent [9] Dian Khumara, M., Fauziyyah, L. & Kristalina, P. Estimation of Urban
system, which can utilize different hardware with a small code- Traffic State Using Simulation of Urban Mobility(SUMO) to Optimize
Intelligent Transport System in Smart City. 2018 International Electron-
base modification. The solution is designed in a way that can ics Symposium On Engineering Technology And Applications (IES-ETA).
be easily expanded with additional sensors to enhance research pp. 163-169 (2018)
in the field of connected, cooperative, and automated mobility. [10] Długosz, M., Roman, M., Wegrzyn, P. “Virtual simulation environment
of anautonomous A-EVE vehicle”, In 2021 25th International Confer-
The device is also designed as a prototyping device and can ence on Methods andModels in Automation and Robotics (MMAR) (pp.
be used as a vehicle’s on-board unit in connected mobility 168 - 172), IEEE, 2021.
scenarios due to its ability to connect into a cellular network. [11] JNX30D for NVIDIA® Jetson Xavier NXTM – Auvidea.
(n.d.). JNX30D for NVIDIA® Jetson Xavier NXTM – Auvidea.
The solution have been verified, and properly tested in terms https://auvidea.eu/product/70879/
of user experience. [12] Logitech C920 Pro HD webcam, 1080p video with Stereo
Audio (no date) , 1080p Video with Stereo Audio. Available
A. Next steps at: https://www.logitech.com/en-eu/products/webcams/c920-pro-hd-
webcam.960-001055.html (Accessed: 20 August 2023).
Next steps identified by the authors are testing the device [13] Zed-F9P module (2023) u. Available at: https://www.u-
in various scenarios to verify its functionality, performance, blox.com/en/product/zed-f9p-module (Accessed: 20 August 2023).
[14] Django Software Foundation. (2019). Django. Retrieved from
and user interface. The device should be tested under various https://djangoproject.com
network conditions to ensure that it can handle different [15] Wessels A., Purvis M., Jackson J. & Rahman S. Remote Data Visual-
levels of traffic and network latency. Additionally, real-life ization through WebSockets 2011 Eighth International Conference on
Information Technology: New Generations. pp. 1050-1051 (2011)
tests together with researchers and students need to be held.
Once the testing is complete, any issues that arise should be
addressed and improvements should be made as necessary.
This will ensure that the smart mobility device is a reliable
and effective tool for capturing and analyzing mobility data,
ultimately improving the ability of researchers to fast track
tendencies and dependencies in mobility scenarios.
ACKNOWLEDGEMENT
This article was written thanks to the generous support
under the Operational Program Integrated Infrastructure for
the project: ” Support of research activities of Excellence
laboratories STU in Bratislava ”, Project no. 313021BXZ1,
co-financed by the European Regional Development Fund. ”
R EFERENCES
[1] European Climate, Infrastructure and Environment Executive Agency.
EU Road Safety: Towards “Vision Zero.” 2022b, November 7.
https://cinea.ec.europa.eu/publications/eu-road-safety-towards-vision-
zero en
[2] Savithramma, R., Ashwini, B. & Sumathi, R. Smart Mobility Imple-
mentation in Smart Cities: A Comprehensive Review on State-of-art
Technologies. 2022 4th International Conference On Smart Systems And
Inventive Technology (ICSSIT). pp. 10-17 (2022)
[3] B. Wang, C. Zhang, M. Zhang, C. Liu, Z. Xie and H. Zhang, ”Digital
Twin Analysis for Driving Risks Based on Virtual Physical Simulation
Technology,” in IEEE Journal of Radio Frequency Identification, vol. 6,
pp. 938-942, 2022, doi: 10.1109/JRFID.2022.3203694.
[4] El Marai O., Taleb T. & Song J. Roads infrastructure digital twin: A step
toward smarter cities realization. IEEE Network. pp. 136-143 (2020).
[5] Lehoczký P.,Janeba M.,Galinski M. & Šoltés L. Testing the Readiness of
Slovak Road Infrastructure for the Deployment of Intelligent Transporta-
tion. Strojnı́cky časopis - Journal of Mechanical Engineering, Vol.72
(Issue 2). pp. 103-112 (2022)
[6] Janeba M., Lehoczký P., Galinski M., Milesich T., Danko J. & Kotuliak
I. Evaluation of LTE and 5G qualitative parameters for V2X use cases.
2022 IEEE Zooming Innovation in Consumer Technologies Conference
(ZINC). pp. 165-169 (2022)
[7] Janeba, M., Galinski, M., Lehoczky, P., Milesich, T., Danko, J., Mag-
dolen, L. (2023). Evaluating Slovak Roads for Vehicle-to-Everything
Communication 3GPP Requirements. In: Pawelczyk, M., Bismor, D.,
Ogonowski, S., Kacprzyk, J. (eds) Advanced, Contemporary Control.
PCC 2023. Lecture Notes in Networks and Systems, vol 709. Springer,
Cham. https://doi.org/10.1007/978-3-031-35173-0 39
[8] GUEANT, V. (no date) Iperf - the ultimate speed test tool for TCP,
UDP and SCTPTEST the limits of your network + internet neutrality
test, iPerf.fr. Available at: https://iperf.fr/ (Accessed: 20 August 2023).

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 230

 $QDO\VLVRIPHWKRGVDQGWHFKQRORJLHVXVHGWR
SUHGLFWDQGGHWHFWILUHVLQIRUHVWU\
0LFKDO-DQRYHF-R]HI3DSiQ
8QLYHUVLW\RIäLOLQDäLOLQD6ORYDNLD
0LFKDO-DQRYHF#IULXQL]DVN-R]HI3DSDQ#IULXQL]DVN


$EVWUDFW² 7R HQVXUH HDUO\ GHWHFWLRQ RI ILUHV YDULRXV WHPSHUDWXUH UHODWLYH KXPLGLW\ JHRJUDSKLF ORFDWLRQ
PHWKRGV LQFOXGLQJ DUWLILFLDO LQWHOOLJHQFH $,  :LUHOHVV ODQGVFDSHFRQGLWLRQVDQGYDULRXVRWKHUV$ILUHFDQDOVREH
6HQVRU 1HWZRUNV :61  DQG PDFKLQH OHDUQLQJ DUH FDXVHGE\QHJOLJHQWKXPDQDFWLRQRULQWHQWLRQDOLJQLWLRQE\
UHFRPPHQGHG7KHPRVWHIIHFWLYHDSSURDFKIRUVDIHJXDUGLQJ YDQGDOV 7KLV W\SH RI ILUH RFFXUUHQFH LV YHU\ GLIILFXOW WR
IRUHVWU\LVWKURXJKSUHGLFWLRQDQGWKHHDUO\LGHQWLILFDWLRQRI LQIOXHQFHDQGZHFDQQRWHIIHFWLYHO\GHIHQGDJDLQVWLWVLQFH
YXOQHUDEOH DUHDV ZLWKLQ IRUHVWV ZKHUH LJQLWLRQ LV OLNHO\ WR WKHKXPDQIDFWRULVLQYROYHG>@
RFFXU7KLVDUWLFOHSURYLGHVDQRYHUYLHZRIYDULRXVPHWKRGV :H NQRZ WZR EDVLF W\SHV RI IRUHVW ILUHV ZKLFK DUH
DOJRULWKPVDQGWKHRULHVHPSOR\HGIRUHDUO\ILUHGHWHFWLRQLQ FKDUDFWHUL]HGE\WKHVSHHGRIWKHLUVSUHDG7KHILUVWRQHLV
IRUHVWU\DVZHOODVILUHSUHGLFWLRQ,WRIIHUVLQVLJKWVLQWRWKH WKHVXUIDFHW\SHRISURSDJDWLRQDQGWKHVHFRQGRQHLVWKH
XWLOL]DWLRQ DQG DFFXUDF\ RI LQGLYLGXDO WHFKQLTXHV PHWKRGV FURZQW\SHRISURSDJDWLRQ$VXUIDFHILUHLVFKDUDFWHUL]HG
DQG DOJRULWKPV GHYHORSHG IRU SUHGLFWLQJ DQG PRQLWRULQJ
E\ WKH IDFW WKDW LW RFFXUV PDLQO\ LQ PHDGRZV DQG IRUHVWV
IRUHVW ILUHV ,Q WKLV DUWLFOH ZH SUHVHQW YDULRXV XVHG
ZLWK ORZ YHJHWDWLRQ VPDOO WUHHV EXVKHV KHUEV JUDVVHV
PHWKRGVDOJRULWKPV RU WKHRULHV WKDW DUH XVHG IRU HDUO\ ILUH
URRWV\VWHPV 7KHVSHHGRIVSUHDGLQJRIWKLVW\SHRIILUHLV
GHWHFWLRQLQIRUHVWU\DVZHOODVLWVSUHGLFWLRQ:HZLOOGHVFULEH
WKH XVH DV ZHOO DV WKH DFFXUDF\ RI WKH DYDLODEOH LQGLYLGXDO
DURXQGPPLQ,WLVFODVVLILHGDVPLQRU&RURQDILUHLV
RSWLRQVPHWKRGVDQGDOJRULWKPVGHVLJQHGIRUWKHSUHGLFWLRQ
YHU\ GDQJHURXV DQG VHULRXV ,W LV FKDUDFWHUL]HG E\ GHQVH
DQGUHFRUGLQJRIIRUHVWILUHV YHJHWDWLRQZKLFKPHDQVIRUHVWV7KLVW\SHRIILUHVSUHDGV
WKURXJK WKH FURZQV RI WUHHV ZKLFK DOVR UHVXOWV LQ D KLJK
.H\ZRUGV)LUH3UHGLFWLRQ)LUH'HWHFWLRQ:611HWZRUN VSHHGRIIODPHWUDQVPLVVLRQ7KHWUDQVPLVVLRQVSHHGUDQJHV
6HQVRUV3UHGLFWLRQ$OJRULWKPV0DFKLQH/HDUQLQJ$UWLILFLDO IURP  PPLQ WR  PPLQ 7KLV VSHHG FDQ EH
PXOWLSOLHGE\WKHZLQGVSHHGLQWKHDUHDZKLFKPHDQVWKDW
,QWHOOLJHQFH
WKHILUHZLOODIIHFWDODUJHDUHD7KLVW\SHRIILUHLVGLIILFXOW
, ,1752'8&7,21 WRH[WLQJXLVKDQGEULQJXQGHUFRQWURO>@
,QUHFHQW\HDUVWKHUHKDYHEHHQPDQ\LVVXHVUHODWHGWR ,QWKLVDUWLFOHZHZLOOPDLQO\GHDOZLWKWKHSRVVLELOLWLHV
FOLPDWHFKDQJH)RUHVWVDQGPHDGRZVDUHRQHRIWKHDUHDV RIIRUHVWILUHSUHGLFWLRQZKLFKZLOOKHOSXVWRGHWHFWFULWLFDO
WKDW DIIHFW DLU TXDOLW\ DV ZHOO DV ODQGVFDSH GLYHUVLW\ SODFHVLQIRUHVWVWDQGVLQWLPH7KHVHSRVVLELOLWLHVLQFOXGH
)RUHVWU\LVPDGHXSRIDODUJHQXPEHURIGLYHUVHHOHPHQWV WKHXVHRIYDULRXVSUHGLFWLRQDOJRULWKPVWKDWFDQGHWHFWZLWK
ZKLFKLQFOXGHWUHHVEXVKHVDQLPDOVDQGYDULRXVRWKHUV,W DUHODWLYHO\KLJKSUREDELOLW\WKHSRWHQWLDOORFDWLRQRIDILUH
LVQHFHVVDU\WKDWZHWDNHFDUHRIWKHIRUHVWVDQGNHHSWKHP WKDWFDQDULVHQDWXUDOO\$QHFHVVDU\FRQGLWLRQIRUDFRUUHFW
LQ WKH EHVW SRVVLEOH FRQGLWLRQ 9DULRXV FULWLFDO VLWXDWLRQV DQGDVDFFXUDWHDVSRVVLEOHILUHIRUHFDVWLVWKHUHFRUGLQJRI
FDQ RFFXU GXULQJ IRUHVWU\ PDQDJHPHQW LQFOXGLQJ IORRGV FULWLFDO TXDQWLWLHV 7KHVH TXDQWLWLHV QHHG WR EH PHDVXUHG
ZLQGVWRUPVGLVHDVHVRUIRUHVWILUHV>@>@ DQGUHFRUGHGLQUHDOWLPHZKLFKZLOOKHOSXVUHDFWIOH[LEO\
WR WKH VLWXDWLRQ 7KH ORFDWLRQ RI PHDVXULQJ GHYLFHV LQ
$PRQJWKHPRVWVHULRXVDUHIRUHVWILUHVZKLFKFDXVHWKH IRUHVWU\ KDV D JUHDW LQIOXHQFH RQ WKH WLPHO\ DQG DFFXUDWH
ORVV RI QDWXUDO KDELWDWV IRU WKH DQLPDOV WKDW OLYH LQ WKHP UHFRUGLQJRIFULWLFDOSDUDPHWHUVIRUSUHGLFWLRQDVZHOODVWKH
7KH\ DOVR FDXVH JUHDW ILQDQFLDO DQG HFRORJLFDO GDPDJH RFFXUUHQFHRIWKHILUHLWVHOI>@>@
ZKLFKPXVWEHUHPRYHGDVVRRQDVSRVVLEOH$IRUHVWWKDWLV
DIIHFWHGE\DIRUHVWILUHQHHGVDYHU\ORQJWLPHWRUHVWRUH ,, &203$5,6212)35(',&7,210(7+2'6
DQGIRUPXODWHWKHRULJLQDOYDULHW\RIIDXQDDQGIORUD6XFK
DUHVWRUDWLRQSURFHVVWDNHVGHFDGHV 9DULRXV PHWKRGV DQG SULQFLSOHV FDQ EH XVHG WR GHWHFW
IRUHVWU\ ILUHV $PRQJ WKH PRVW FRPPRQ GDWD WKDW FDQEH
7KH QXPEHU RI IRUHVW ILUHV UHFRUGHG LQ ,QGLD EHWZHHQ XVHG WR SUHGLFW WKH RFFXUUHQFH RI D ILUH LV WKH XVH RI
1RYHPEHU  DQG-XQH ZDV  IRUHVW ILUHV JHQHUDOO\DYDLODEOHGDWDIURPPHWHRURORJLFDOVLWHV7KHGDWD
EDVHGRQUHFRUGHGGDWDIURPWKH,QGLDQ)RUHVW6HUYLFHDQG UHFRUGHGLQWKLVZD\DUHKRZHYHUYHU\LPSUHFLVHDVWKH\
)RUHVW6XUYH\RI,QGLD,WLVQRWHZRUWK\WKDWPRUHWKDQ DUHRIWHQQRWIRXQGLQWKHVSHFLILFIRUHVWPDQDJHPHQWIRU
RIWKHIRUHVWKROGLQJVORFDWHGLQ,QGLDDUHFODVVLILHGDVDUHDV ZKLFKZHZDQWWRSUHGLFWWKHRFFXUUHQFHRIDILUH
ZLWKIUHTXHQWRFFXUUHQFHRIIRUHVWILUHV7KHZRUU\LQJWKLQJ
LVWKDWDERXWRIWKHVHIRUHVWIDUPVDUHYHU\OLNHO\WR $PRQJWKHRWKHUIUHTXHQWO\XVHGPHWKRGVLVWKHDQDO\VLV
VWDUWDILUH>@ RI DHULDO SKRWRJUDSKV ZKLFK DUH FUHDWHG XVLQJ DLUSODQHV
'DWD UHFRUGHG LQ WKLV ZD\ ZLOO KHOS XV WR GHWHFW FULWLFDO
6WDWHVDQGWHUULWRULDOXQLWVWU\WRSUHYHQWVXFKGDPDJHVRU SODFHVLQWKHIRUHVWZKHUHWKHUHLVDKLJKHUSUREDELOLW\RI
WRUHFRUGWKHPLQWLPHVRWKDWWKHOHDVWSRVVLEOHGDPDJHV ILUH
RFFXU2QHRIWKHEHVWZD\VWRSUHYHQWZLOGILUHVLVWRSUHGLFW
WKHP7KHFRQGLWLRQVQHFHVVDU\IRUDILUHWRVWDUWDUHNQRZQ 2QHRIWKHRWKHUDYDLODEOHPHWKRGVLVWRFUHDWHDYLGHR
+RZHYHU WKHVH FRQGLWLRQV GLIIHU IURP WKH JHRJUDSKLF UHFRUGLQJIURPWKHSODQH$FDPHUDLVXVHGWRUHFRUGWKH
ORFDWLRQ RI IRUHVWU\ 7KH FRQGLWLRQV WKDW PXVW EH PHW LQ YLGHR UHFRUGLQJ ZKLFK FDQ GHWHFW WKH LQIUDUHG VSHFWUXP
RUGHU IRU D ILUH WR RFFXU QDWXUDOO\ LQFOXGH D VSHFLILF IURP ZKLFK LW FUHDWHVD WKHUPDO YLGHRUHFRUGLQJ 9DULRXV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 231

DOJRULWKPV FDQEH XVHGLQ WKH WKHUPDOUHFRUGLQJ PDGH LQ 2QHRIWKHRWKHUSRVVLEOHHOHPHQWVWKDWFDQEHDSSOLHGLQ
WKLVZD\7KHDOJRULWKPFDQIRUH[DPSOHHYDOXDWHSODFHV WKH GHWHFWLRQ RI IRUHVWU\ ILUHV LV WKH XVH RI LPDJHV DQG
ZKHUH WKH KLJKHVW FRQFHQWUDWLRQ RI WHPSHUDWXUH RFFXUV YLGHRV RI IRUHVWU\ ZKLFK DUH FUHDWHG ZLWK WKH KHOS RI
ZKLFKPHDQVUHYHDOLQJDSRWHQWLDOSODFHZKHUHDILUHFDQ GURQHV +DQK 'DQJ1JRF DQG +LHX 1JX\HQ7UXQJ XVHG
VWDUW WKHVHLPDJHVDQGWKHLUHDUO\DQDO\VLVFDQKHOSLQWKHHDUO\
2QH RI WKH SRVVLEOH PHWKRGV LV WKH DQDO\VLV RI LPDJHV GHWHFWLRQ RI WKH ILUH VFHQH DV ZHOO DV LQ WKH GHWHFWLRQ RI
WKDW DUH FUHDWHG XVLQJ VDWHOOLWHV :LWK WKH KHOS RI LPDJHV FULWLFDOSODFHVZKHUHWKHUHLVDULVNRIILUH$JHQHUDOIRUHVW
FDSWXUHG LQ WKLV ZD\ ZH FDQ DOVR GHWHUPLQH SRWHQWLDOO\ ILUHGHWHFWLRQPRGHOLVWHVWHGZKLFKZLOOXVHWKHDQDO\VLVRI
FULWLFDOSODFHVLQIRUHVWU\MXVWDVZLWKDHULDOLPDJHV,WLV YLGHRV WKDW ZHUH FUHDWHG XVLQJ GURQHV 6XFK D PRGHO LV
DOVRSRVVLEOHWRXVHWKHPWRGHWHFWDILUHWKDWKDVDOUHDG\ WHVWHGLQRUGHUWREHSXWLQWRUHDORSHUDWLRQ7KHV\VWHPLV
VWDUWHG ZKLFK ZLOO HQDEOH WKH UDSLG LQWHUYHQWLRQ RI WKH EDVHGRQSL[HOGHWHFWLRQ)LUHSL[HOVDUHH[WUDFWHGXVLQJWKH
UHOHYDQWUHVFXHVHUYLFHV FKDUDFWHULVWLFFRORUVDQGPRWLRQRIWKHILUH7KHGDWDEDVHRI
2WKHU SRVVLEOH VRXUFHV RI GDWD LQFOXGH D QHWZRUN RI FROOHFWHG YLGHRV RQ ZKLFK WKH DOJRULWKP ZDV WHVWHG
FRQWDLQV  DHULDO UHFRUGLQJV IURP ZKLFK  VKRWV
VHQVRUVWKDWZLOOUHFRUGWKHPRQLWRUHGTXDQWLWLHVGLUHFWO\LQ
ZHUHDQDO\]HG7KHDFFXUDF\DQGUHOLDELOLW\RIWKHV\VWHP
IRUHVWU\ 7KH GDWD PHDVXUHG LQ WKLV ZD\ DUH DFFXUDWH
WKDWGHWHFWVIRUHVWILUHVLV7KHGDWDLQZKLFKDIDOVH
EHFDXVHWKH\UHIHUWRVSHFLILFIRUHVWU\
ILUHZDVUHFRUGHGLVDWWKHOHYHORI7KHSHUFHQWDJH
7KHUH DUH RWKHU SRVVLEOH PHWKRGVDOJRULWKPV DQG RIFDVHVLQZKLFKWKHV\VWHPGLGQRWUHFRUGDQDFWXDOILUH
SULQFLSOHVWRVROYHWKLVSUREOHPLQWKHIROORZLQJVHFWLRQV UHDFKHV D YDOXH RI  $PRQJ WKH PDLQ IDFWRUV WKDW
ZHZLOOGHVFULEHVRPHRIWKHP>@ FDXVHG WKH V\VWHP WR IDLO LQ IRUHVW ILUH SUHGLFWLRQ LV WKH
 SUHVHQFH RI WKLFN VPRNH WKDW DFFRPSDQLHV HYHU\ ILUH 7R
LPSURYHWKHIXQFWLRQDOLW\RIWKHV\VWHPLWLVDGYLVDEOHWR
$ )RUHVWILUHGHWHFWLRQXVLQJLPDJHVDQGDUWLILFLDO LPSOHPHQWWKHXVHRIVPRNHSL[HOVZKLFKZLOOVLJQLILFDQWO\
LQWHOOLJHQFH LPSURYH WKH GHWHFWLRQ RI ILUH DQG WKH UHOLDELOLW\ RI WKH
)RU WKHSUHGLFWLRQRI IRUHVW ILUHV LW LVDGYLVDEOHWR XVH SUHGLFWLRQPRGHO7KHGHWHFWLRQDQGUHFRUGLQJRIWKHSODFH
DUWLILFLDOLQWHOOLJHQFHRUGHHSQHXUDOOHDUQLQJZKLFKRSHQV ZKHUHWKHILUHVWDUWHGDIWHUWKHDSSOLFDWLRQRIWKHGHFLVLRQ
XS WKH SRVVLELOLWLHV RI IRUHVW ILUH SUHGLFWLRQ %DVHG RQ DOJRULWKPFDQEHVHHQLQ)LJ>@
UHVHDUFKGRQHE\:RQMDH/HHHWDO>@>@WKHXVHRIGHHS 
QHXUDO OHDUQLQJ WR SUHGLFW IRUHVW ILUHV LV DSSURSULDWH 7KH
DXWKRUVFUHDWHGDV\VWHPWKDWFRQVLVWHGRIDQDO\]LQJLPDJHV
RIIRUHVWILUHVXVLQJGHHSQHXUDOOHDUQLQJIRUWKHDQDO\VLV
)XUWKHUPRUHWKHV\VWHPFRQVLVWHGRIDGLVDVWHUSUHGLFWLRQ
DOJRULWKP DQG D ZDUQLQJ V\VWHP 7KH UHVHDUFK JURXS
SRLQWHG WR D SUREOHP DUHD ZKLFK LV WKH SURFHVVLQJ RI
LPDJHVWKDWZHUHFUHDWHGDWORZDOWLWXGH7KHSUREOHPOLHV
LQWKHPLVLGHQWLILFDWLRQRIWKHILUHLQWKHLPDJHVGXHWRWKH
EOXUULQJ RI WKH LPDJH 7R VROYH WKLV DVSHFW WKH UHVHDUFK
WHDPDSSOLHGGLIIHUHQWPRGHOV7KHPRGHOVXVHGIRUWUDLQLQJ
LQFOXGH *RRJOH/H1HW $OH[1HW 9** DQG WKHLU
PRGLILFDWLRQV $ GDWDVHW ZDV XVHG IRU WUDLQLQJ ZKLFK
FRQWDLQHG  LPDJHV LQ ZKLFK D ILUH ZDV YLVLEOH DQG
 LPDJHV LQ ZKLFK WKHUH ZDV QR ILUH 7KH DSSOLHG
PRGHOV RQ WKLV GDWDVHW ZHUH WKHQ UDQNHG DQG HYDOXDWHG
EDVHG RQ ILUH GHWHFWLRQ DFFXUDF\ WUDLQLQJ WLPH DQG
FODVVLILFDWLRQ WLPH WR ILQG WKH PRGHO ZLWK WKH EHVW
SHUIRUPDQFH,WWXUQHGRXWWKDW*RRJOH/H1HWDQGPRGLILHG
9** DFKLHYHG KLJK DFFXUDF\ RI IRUHVW ILUH SUHGLFWLRQ
*RRJOH/H1HWZDVDEOHWRGHWHFWDILUHLQWLPHIURPVWDWLF
DHULDOLPDJHVZLWKDQDFFXUDF\RIXSWR>@>@

ůĂƐƐŝĨŝĐĂƚŝŽŶ dƌĂŝŶŝŶŐ
EE ĐĐƵƌĂĐLJ dŝŵĞ dŝŵĞ
2 8:7 99:, ,6   
€ )32 88+ //.69 ,/   )LJXUH)LUHGHWHFWLRQXVLQJILUHDUHDFRUUHFWLRQEDVHGRQGHFLVLRQ
UXOHV>@
0  
8.8 /++/9 /6  
€ )32 $QRWKHUUHVHDUFKWHDPWKDWDSSOLHGPDFKLQHOHDUQLQJWR
0  ILUHGHWHFWLRQZDV'L\DQD.LQDQHYDHWDO7KH\DUHZRUNLQJ
8.& 986/ .+6  
$€€/, RQ D GDWD FROOHFWLRQ V\VWHP YLGHR  XVLQJ GURQHV DQG
DLUFUDIW WR HQVXUH WKH ORZHVW SRVVLEOH UDWH RI IDOVHO\
$€€/ , 7.& /+&/6 /99  
UHFRUGHG ILUHV 9LGHRV DUH UHFRUGHG DW WZR IOLJKW OHYHOV
 QDPHO\PPDQGPP7KHVHDLUFUDIWDUH
)LJXUH&RPSDULVRQRIFRQYROXWLRQDOQHXUDOQHWZRUNV>@ HTXLSSHG ZLWK D FDPHUD WKDW FDQ FUHDWH ZLGHVSHFWUXP
 YLGHR UHFRUGLQJ %RWK DLUFUDIW DQDO\]H VSHFLILHG DUHDV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 232

ZKLFKDUHGHWHUPLQHGXVLQJ*36FRRUGLQDWHV,IDQDLUFUDIW
RSHUDWLQJDWDORZHUIOLJKWOHYHOGHWHFWVDILUHLWFUHDWHVDQ
DOHUWWKDWLWVHQGVWRWKHRWKHUDLUFUDIW7KHSODQHZKLFKLV
DWDKLJKHUIOLJKWOHYHOZLOODUULYHDWWKHUHSRUWHGORFDWLRQ
DQGZLOODOVRDQDO\]HWKHPHDVXUHGGDWD,IWKHVHFRQGSODQH
HYDOXDWHV WKH SUHVHQFH RI D ILUH LW VHQGV WKH GDWD WR WKH
GHVLJQDWHG DXWKRULWLHV 7KH FUHDWHG YLGHR UHFRUGLQJ LV
DQDO\]HG XVLQJ PDFKLQH OHDUQLQJ 7KH OHDUQLQJ SURFHVV
FRQVLVWV RI LPDJH FODVVLILFDWLRQ FODVVLILFDWLRQ DVVRFLDWHG
ZLWKORFDOL]DWLRQREMHFWILQGLQJDQGLQVWDQFHVHJPHQWDWLRQ
$ FRPSDULVRQ RI WKH LQGLYLGXDO VWHSV FDQ EH VHHQ LQ
)LJ>@



)LJXUH:DVSPRWHVHQVRUIRUHDUO\IRUHVWILUHGHWHFWLRQXVLQJ:61
IURP/LEHOLXP>@
7KLV GHYLFH UHFRUGV IRXU FULWLFDO SDUDPHWHUV QDPHO\
UHODWLYHKXPLGLW\DPELHQWWHPSHUDWXUHFDUERQGLR[LGHDQG
FDUERQ PRQR[LGH ,W VHQGV PHDVXUHG YDOXHV DW UHJXODU 
PLQXWHLQWHUYDOVWRWKHFRUUHVSRQGLQJJDWHZD\$VVRRQDV
VRPHYDOXHVDUHPHDVXUHGWKHV\VWHPDQDO\]HVWKHVHYHULW\
RIWKHVLWXDWLRQ,IWKHV\VWHPHYDOXDWHVWKHRFFXUUHQFHRID
 IRUHVWILUHLWVHQGVDQDODUPPHVVDJHWRWKHILUHGHSDUWPHQW
)LJXUH'LIIHUHQFHEHWZHHQLPDJHFODVVLILFDWLRQREMHFWGHWHFWLRQ RU D FRPSHWHQW LQVWLWXWLRQ WKDW GHDOV ZLWK IRUHVW ILUH
DQGLQVWDQFHVHJPHQWDWLRQ>@
PRQLWRULQJ 7KDQNV WR WKH LQWHJUDWHG *36 PRGXOH WKH\

LPPHGLDWHO\NQRZDERXWWKHILUHDQGLWVH[DFWORFDWLRQ
7KHSURSRVHGVROXWLRQLVVWLOOLQWKHGHYHORSPHQWSKDVH 7KLVW\SHRIGHYLFHXVLQJ:61QHWZRUNVLVDYHU\JRRG
7KHUHVHDUFKWHDPFRQGXFWHGVLPXODWLRQWHVWVWKDWZHQWDV VRXUFH RI GDWD ZKLFK ZLOO EH DSSUHFLDWHG E\ UHVFXH
H[SHFWHG)RUWKHLPSOHPHQWDWLRQRIWKHUHVHDUFKLQ VHUYLFHV EXW WKH W\SH RI ILUH WKDW RFFXUUHG UHPDLQV
SUDFWLFHWKHLPSOHPHQWDWLRQWHDPLVZDLWLQJIRUWKH XQLGHQWLILHG'HWHUPLQLQJWKHW\SHRIILUHLVRQHRIWKHNH\
SXUFKDVHRIWKHQHFHVVDU\HTXLSPHQW DVSHFWVRIFRUUHFWO\DVVHVVLQJWKHVHYHULW\RIWKHVLWXDWLRQ
% )RUHVWILUHGHWHFWLRQXVLQJ:61QHWZRUN 7RGHWHUPLQHWKHW\SHRIILUHLWLVQHFHVVDU\WRLPSOHPHQW
DGGLWLRQDOVHQVRUVWKDWZLOOUHFRUGWKHQHFHVVDU\GDWDDQG
$QRWKHUSRVVLEOHVROXWLRQIRUIRUHVWILUHGHWHFWLRQLVWR
DOVRDSSO\QHZDOJRULWKPVWKDWZLOOHYDOXDWHWKHPHDVXUHG
XVH:LUHOHVV6HQVRU1HWZRUNV :61V $:61QHWZRUN
GDWD
LVPDGHXSRIVPDOOGHYLFHVWKDWDUHFKHDS7KHGHYLFH
FRQVLVWVRIDGHWHFWLRQHOHPHQW VHQVRUVHQVRUV ZKLFKKDV 7KH UHVHDUFK JURXS RI $$ .KDPXNKLQ DQG WKH WHDP
DORZSRZHUFRQVXPSWLRQDQGLVXVHGWRUHFRUGFULWLFDO FUHDWHGDXQLTXHDSSURDFKWRIRUHVWILUHGHWHFWLRQXVLQJD
YDOXHVLQWKHIRUHVWU\HQYLURQPHQW7KHUHDUHDODUJH :61 QHWZRUN 7KH\ IRFXVHG RQ LGHQWLI\LQJ WKH W\SH RI
QXPEHURIDUWLFOHVGHDOLQJZLWKWKHXVHRIGHWHFWLRQ:61 IRUHVWILUH)RUGHWHFWLRQWKH\XVHGGDWDQRWPHDVXUHGE\D
QHWZRUNV7KHGLVDGYDQWDJHRIDODUJHQXPEHURIVROXWLRQV VHQVRUWKDWUHFRUGVVRXQGLQDIRUHVWU\HQYLURQPHQW7KH\
LVWKDWGHYLFHVDUHFUHDWHGIRUWKHHDUO\GHWHFWLRQRIIRUHVW XVHG VSHFWUDO DQDO\VLV RQ UHFRUGHG DXGLR $FFRUGLQJ WR
ILUHVEXWWKH\FDQQRWGHWHUPLQHZKDWW\SHRIILUHLWLV VFLHQWLILFOLWHUDWXUHWKHSRZHUVSHFWUXPLVFDOFXODWHGLQWZR
EDVHGRQWKHPHDVXUHGSDUDPHWHUV2QHRIWKHH[DPSOHV DUHDVZLGH +]+] DQGQDUURZ +]
RIWKLVW\SHRI:61QHWZRUNXVHLVDGHYLFHIURP +] >@7KHUHVHDUFKVKRZVWKDWWKHVRXQGUHFRUGLQJRID
/LEHOLXP<RXFDQVHHWKHGHWHFWLRQGHYLFHLQ)LJ>@ VXUIDFHILUHFDQEHLGHQWLILHGZLWKWKHVSHFWUXPRIUHGQRLVH
7KH VRXQG UHFRUGLQJ FUHDWHG E\ D FURZQ ILUH FDQ EH
LGHQWLILHGZLWK*DXVVLDQQRLVH7KHW\SHRIILUHWKDWJRHV
IURP VXUIDFH WR FURZQ LV GHILQHG E\ WKH WUDQVLWLRQ
FKDUDFWHULVWLF LQ WKH VSHFWUXP EHWZHHQ UHG DQG *DXVVLDQ
QRLVH7KLVSDUDPHWHUZKLFKWKHDXWKRUVDQDO\]HGVKRXOG
EHDGGHGWRH[LVWLQJGHYLFHVWKDWDUHDSSOLHGLQIRUHVWU\>@
& ,PSOHPHQWDWLRQRIPDFKLQHOHDUQLQJWRSUHGLFW
IRUHVWILUHV

0DFKLQHOHDUQLQJLVDVXEVHWRIDUWLILFLDOLQWHOOLJHQFHWKDW
HQDEOHV PDFKLQHV WR FRPSUHKHQG DQG DQDO\]H GDWD LQ D
KXPDQOLNH PDQQHU ZLWKRXW WKH QHHG IRU H[SOLFLWO\

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 233

SUHGHILQHG LQVWUXFWLRQV 0DFKLQH OHDUQLQJ HQFRPSDVVHV 7KH5DQGRP)RUHVWDSSURDFKLVLQGHHGYHUVDWLOHDQGFDQ
ERWKVXSHUYLVHGDQGXQVXSHUYLVHGOHDUQLQJDSSURDFKHV,Q EHDSSOLHGWRERWKUHJUHVVLRQDQGFODVVLILFDWLRQSUREOHPV,W
VXSHUYLVHGOHDUQLQJWKHUHDUHLQSXWDQGRXWSXWQRGHVDQG RSHUDWHV RQ WKH SULQFLSOH RI HQVHPEOH OHDUQLQJ ZKLFK
WKHUHODWLRQVKLSEHWZHHQLQSXWDQGRXWSXWLVGHWHUPLQHGE\ LQYROYHVFRPELQLQJPXOWLSOHFODVVLILHUVWRDGGUHVVFRPSOH[
WKHSURYLGHGLQSXWGDWD2QWKHRWKHUKDQGXQVXSHUYLVHG SUREOHPVDQGHQKDQFHDOJRULWKPSHUIRUPDQFH7RHQKDQFH
OHDUQLQJDOJRULWKPVDUHHPSOR\HGLQWDVNVVXFKDVJURXSLQJ SUHGLFWLYHDFFXUDF\WKH5DQGRP)RUHVWDOJRULWKPHPSOR\V
LQIRUPDWLRQEDVHGRQVKDUHGFKDUDFWHULVWLFVQDWXUDOHYHQWV DPXOWLWXGHRIGHFLVLRQWUHHVWUDLQHGRQYDULRXVVXEVHWVRI
EHKDYLRUVRULQWHUDFWLRQV WKH RULJLQDO GDWDVHW 7KLV GLYHUVLW\ LQ WUHHV FRQWULEXWHV WR
$QRYHODSSURDFKWRSUHGLFWLQJZLOGILUHVZDVLQWURGXFHG LPSURYHGRYHUDOOSHUIRUPDQFH>@
E\$-D\DNXPDUDQGKLVFROOHDJXHV>@ZKRKDUQHVVHGWKH 
SRZHU RI FODVVLILFDWLRQ DQG FOXVWHULQJ WHFKQLTXHV 7KH\
XWLOL]HG )X]]\ FOXVWHULQJ DQG WKH $GDSWLYH QHXUR)X]]\
,QIHUHQFH 6\VWHP $1),6  FODVVLILFDWLRQ DOJRULWKPV WR
EXLOG DQ DGYDQFHG FODVVLILFDWLRQ V\VWHP 7KLV LQQRYDWLYH
PHWKRG IROORZHG D WZRVWHS SURFHGXUH VWDUWLQJ ZLWK
FHQWURLGEDVHG FOXVWHULQJ DQG WKHQ SURFHHGLQJ WR $1),6
FODVVLILFDWLRQ 7KH RXWFRPHV UHYHDOHG WKDW WKLV SURSRVHG
IUDPHZRUNDFKLHYHGDQLPSUHVVLYHURRWPHDQVTXDUHHUURU
506( VFRUHRI
,QD UHVHDUFK VWXG\ FRQGXFWHGE\ $/ /DWLIDK DQG KHU
WHDP ZLWKLQ WKH JHRJUDSKLFDO UHJLRQV RI %RUQHR ,VODQG
VLWXDWHG LQ ,QGRQHVLD WKH\ HPSOR\HG WKH 5DQGRP )RUHVW
DOJRULWKPWRIRUHFDVWIRUHVWILUHV7KLVSUHGLFWLRQPRGHOZDV 
EDVHG RQ D FRPELQDWLRQ RI KLVWRULFDO DQG FRQWHPSRUDU\ )LJXUH7KHGLIIHUHQFHEHWZHHQWKHGHVFULEHGDOJRULWKPV>@
ZLOGILUH GDWD DORQJ ZLWK FOLPDWH GDWD VRXUFHG IURP WKH 
JOREDOILUHHPLVVLRQVGDWDEDVH7KHGDWDVHWXQGHUVFUXWLQ\ 2QHRIWKHRWKHUW\SHVRIPDFKLQHOHDUQLQJWKDWZHFDQ
VSDQQHG IURP  WR WKH SUHVHQW GD\ $GGLWLRQDOO\ WKH XVHWRSUHGLFWIRUHVWILUHVLVWKH([WUHPH/HDUQLQJ0DFKLQH
UHVHDUFK WHDP XWLOL]HG FOLPDWH LQIRUPDWLRQ IURP WKH (/0  (/0 RYHUFRPHV WKH SUREOHPV DVVRFLDWHG ZLWK
(XURSHDQ &HQWHU IRU 0HGLXP5DQJH :HDWKHU )RUHFDVWV XVLQJDQDOJRULWKPEDVHGRQJUDGLHQWGHVFHQW8VLQJ(/0
(&0:)  GDWDEDVH WR DQDO\]H ORFDO IDFWRUV VXFK DV FDQVLJQLILFDQWO\UHGXFHWKHDPRXQWRIWLPHLWWDNHVWRWUDLQ
WHPSHUDWXUH GHZ SRLQW UHODWLYH KXPLGLW\ DQG ZLQG DQHXUDOQHWZRUNIRUDYDULHW\RIDSSOLFDWLRQV$XWKRUV%.
VSHHG>@>@ 6LQJK DQG WKH WHDP>@SHUIRUPHG WKH H[SHULPHQW RQ
,W LV DOVR SRVVLEOH WR XVH YDULRXV RWKHU DOJRULWKPV IRU GLIIHUHQW GDWDVHWV DQG PDLQO\ XVLQJ GLIIHUHQW WUDLQLQJ
DQDO\]LQJ UHFRUGHG GDWD XVLQJ PDFKLQH OHDUQLQJ $PRQJ IXQFWLRQVIRU(/0
WKH PRVW IDPRXV DUH ORJLVWLF UHJUHVVLRQ .1HDUHVW ,QH[SHULPHQWVFRQGXFWHGIRUIRUHVWILUHSUHGLFWLRQWKUHH
QHLJKERU VXSSRUW YHFWRU PDFKLQH DOJRULWKP DQG UDQGRP GLIIHUHQW GDWD SDUWLWLRQV ZHUH XVHG HDFK ZLWK YDU\LQJ
IRUHVWDOJRULWKP QXPEHUVRIWUDLQLQJDQGWHVWLQJLQVWDQFHV7KHUHVXOWVIURP
/RJLVWLF 5HJUHVVLRQ LV RQH RI WKH PRVW ZLGHO\ XVHG WKHVH H[SHULPHQWV LQGLFDWH WKDW WKH KLJKHVW DFFXUDF\ ZDV
PDFKLQHOHDUQLQJDOJRULWKPVDQGIDOOVXQGHUWKHFDWHJRU\RI DFKLHYHG ZKHQ XVLQJ WKH VLJPRLG DFWLYDWLRQ IXQFWLRQ
VXSHUYLVHG OHDUQLQJ 7KLV PHWKRG LV XVHG WR SUHGLFW WKH 6SHFLILFDOO\
GHSHQGHQWYDULDEOHEDVHGRQDVHWRILQGHSHQGHQWIDFWRUV,W 6LJPRLG $FWLYDWLRQ )XQFWLRQ 7KH PRGHO XVLQJ WKH
LVSDUWLFXODUO\ZHOOVXLWHGIRUSUHGLFWLQJGLVFUHWHRXWFRPHV VLJPRLGDFWLYDWLRQIXQFWLRQDWWDLQHGWKHEHVWDFFXUDF\ZKHQ
ZKLFKPHDQVWKHRXWSXWLVELQDU\W\SLFDOO\7UXHRU)DOVH WKH GDWDVHW KDG  WUDLQLQJ LQVWDQFHV DQG  WHVWLQJ
$QDOWHUQDWLYHDSSURDFKLQYROYHVPRGLI\LQJWKHLQLWLDOWZR LQVWDQFHVDFKLHYLQJDQDFFXUDF\RI
FRQYROXWLRQDOOD\HUVRIWKH,QFHSWLRQ1HWZLWKDFXVWRPL]HG
YHUVLRQRIWKH6]HJHG\LQFHSWLRQPRGXOH7KHREMHFWLYHRI 6LQ& $FWLYDWLRQ )XQFWLRQ :LWK WKH 6LQ& DFWLYDWLRQ
WKLVFKDSWHULVWRHQKDQFHZRUNVWDWLRQVE\RSWLPL]LQJWKH IXQFWLRQWKHDFFXUDF\ZDVVOLJKWO\ORZHUDW
PRGHO WKURXJK WKH LQWHJUDWLRQ RI PXOWLSOH IHDWXUH 5DGLDO%DVLV)XQFWLRQ7KHPRGHOXVLQJWKHUDGLDOEDVLV
UHSUHVHQWDWLRQV>@ IXQFWLRQDFKLHYHGDQDFFXUDF\RI
:KLOH.1HDUHVW1HLJKERU .11 FDQEHDSSOLHGWRERWK +DUGOLP $FWLYDWLRQ )XQFWLRQ +RZHYHU WKH DFFXUDF\
FODVVLILFDWLRQ DQG UHJUHVVLRQ WDVNV LW LV SUHGRPLQDQWO\ VLJQLILFDQWO\ GHFUHDVHG WR  ZKHQ WKH PRGHO ZDV
XWLOL]HG IRU FODVVLILFDWLRQ SUREOHPV ,W EHORQJV WR WKH WUDLQHGZLWKWKHKDUGOLPDFWLYDWLRQIXQFWLRQ
VXSHUYLVHGPDFKLQHOHDUQLQJDOJRULWKPVFDWHJRU\.11LV 7KHUHVXOWVIURPWKLVUHVHDUFKGHPRQVWUDWHWKDW([WUHPH
RIWHQ UHIHUUHG WR DV D OD]\ OHDUQHU DOJRULWKP DQG LW LV /HDUQLQJ 0DFKLQH (/0  FODVVLILHUV ZKHQ WUDLQHG ZLWK
UHODWLYHO\VWUDLJKWIRUZDUGFRPSDUHGWRPDQ\RWKHUPDFKLQH ILYH GLIIHUHQW DFWLYDWLRQ IXQFWLRQV DFKLHYHG WKH KLJKHVW
OHDUQLQJDOJRULWKPV>@>@ DFFXUDF\ ZKHQ WKH GDWDVHW FRQVLVWHG RI  WUDLQLQJ
6XSSRUW9HFWRU0DFKLQH 690 FDQLQGHHGEHXVHGIRU LQVWDQFHV DQG  WHVWLQJ LQVWDQFHV $PRQJ WKH ILYH
ERWKUHJUHVVLRQDQGFODVVLILFDWLRQWDVNV,WIDOOVZLWKLQWKH DFWLYDWLRQ IXQFWLRQV WHVWHG WKH VLJPRLG IXQFWLRQ
FDWHJRU\RIVXSHUYLVHGOHDUQLQJDOJRULWKPV690RSHUDWHV FRQVLVWHQWO\ RXWSHUIRUPHG WKH RWKHUV ZKLOH WKH KDUGOLP
E\LGHQWLI\LQJWKHGDWDSRLQWVWKDWKDYHWKHPRVWVLJQLILFDQW IXQFWLRQ H[KLELWHG WKH SRRUHVW SHUIRUPDQFH LQ DOO WKUHH
LQIOXHQFH LQ GHILQLQJ D K\SHUSODQH ZKLFK VHUYHV DV WKH SDUWLWLRQ VFHQDULRV   DQG 
RSWLPDO GHFLVLRQ ERXQGDU\ 7KHVH FULWLFDO GDWD SRLQWV DUH WUDLQLQJDQGWHVWLQJLQVWDQFHVUHVSHFWLYHO\
NQRZQDVVXSSRUWYHFWRUV>@>@ %DVHGRQWKHVHH[SHULPHQWDOILQGLQJVLWLVUHFRPPHQGHG
WRXWLOL]HWKHVLJPRLGDFWLYDWLRQIXQFWLRQIRUIXUWKHUIRUHVW

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 234

ILUH SUHGLFWLRQ WDVNV DV LW FRQVLVWHQWO\ GHPRQVWUDWHG
VXSHULRUSHUIRUPDQFHDFURVVYDULRXVGDWDVHWSDUWLWLRQV>@

^ĂŵƉůĞ ZĂĚŝĂů dƌŝĂŶŐƵůĂƌ
,ĂƌĚůŝŵ ^ŝŐŵŽŝĚ ^ŝŶ
ĂƚĂƐĞƚƐ ĂƐŝƐ ĂƐŝƐ
.:7:,& 96,9 76/8 7:., 7:7/ 7,/6

6:+6:+ 99+7 7.69 76:& 7:86 7/&6

96.,&: 9.6: 7:&. 7:69 7,.: 98.,


7DEOH&RPSDULVRQRIWUDLQLQJPHWKRGV>@

 )LJXUH)ORZFKDUWRI*HQHWLFDOJRULWKP>@
2QH RI WKH RWKHU SRVVLEOH PHWKRGVRISUHGLFWLQJ IRUHVW  
ILUHVXVLQJDUWLILFLDOLQWHOOLJHQFHLV)X]]\ORJLFRUJHQHWLF
DOJRULWKP ,,, &21&/86,21
)X]]\ ORJLF LV DFRPSXWDWLRQDO DSSURDFK WKDW UHOLHVRQ ,QWKLVUHVHDUFKSDSHUZHFRQGXFWHGDQLQGHSWKDQDO\VLV
WKH FRQFHSW RI WUXWK UDWKHU WKDQ FRQYHQWLRQDO WUXH RU DQG H[SORUDWLRQ RI YDULRXV DSSURDFKHV IRU IRUHVW ILUH
IDOVH ELQDU\ ORJLF UHSUHVHQWHG DV  RU   ,Q DUWLILFLDO SUHGLFWLRQ DQG GHWHFWLRQ 2XU LQYHVWLJDWLRQ FDWHJRUL]HG
LQWHOOLJHQFHV\VWHPVIX]]\ORJLFLVXVHGWRUHSOLFDWHKXPDQ SRWHQWLDO PHWKRGV LQWR WKUHH JURXSV LPDJH DQDO\VLV
WKLQNLQJ DQG FRJQLWLRQ 8QOLNH WUDGLWLRQDO ELQDU\ ORJLF HPSOR\LQJDUWLILFLDOLQWHOOLJHQFHWKHXWLOL]DWLRQRI:LUHOHVV
ZKLFKRQO\GHDOVZLWK VDQG VWRH[SUHVVDEVROXWHWUXWKV 6HQVRU1HWZRUNV :61 DQGWKHDSSOLFDWLRQRIPDFKLQH
IX]]\ORJLFLQYROYHVGLIIHUHQWGHJUHHVRIWUXWKDOORZLQJIRU OHDUQLQJWHFKQLTXHV
D ILQHU UHSUHVHQWDWLRQ RI UHDOLW\ )X]]\ ORJLF PRGHOLQJ LV 7KH XWLOL]DWLRQ RI DUWLILFLDO LQWHOOLJHQFH RQ UHFRUGHG
HVSHFLDOO\ XVHIXO IRU PDQDJLQJ XQFHUWDLQW\ DQG FODULI\LQJ SKRWRV RU YLGHRV KDV HPHUJHG DV D KLJKO\ EHQHILFLDO
FRPSOH[ FRQQHFWLRQV DQG FDXVDO UHODWLRQVKLSV EHWZHHQ DSSURDFK DV LW FDQ FRYHU H[WHQVLYH IRUHVWHG DUHDV DQG
YDULDEOHV 7R LOOXVWUDWH WKLV FRQFHSW D VFKHPDWLF GLDJUDP OHYHUDJH DGYDQFHG $, PRGHOV VXFK DV *RRJ/H1HW RU
IRUIX]]\ORJLFLVSURYLGHGLQ)LJ>@ $OH[1HWIRUILUHGHWHFWLRQ7KLVPHWKRGKDVGHPRQVWUDWHG
 H[FHOOHQWUHVXOWV+RZHYHURQHGUDZEDFNLVWKHLQLWLDOKLJK
FRVWDVVRFLDWHGZLWKSXUFKDVLQJDLUFUDIWRUGURQHVDQGWKH
QHFHVVDU\HTXLSPHQW
2QWKHRWKHUKDQGHPSOR\LQJ:61QHWZRUNVIRUGDWD
FROOHFWLRQ LQ IRUHVW ILUH SUHGLFWLRQ DQG GHWHFWLRQ RIIHUV D
PRUH FRVWHIIHFWLYH VROXWLRQ :KLOH WKH GHYLFHV XVHG LQ
:61 QHWZRUNV DUH JHQHUDOO\ DIIRUGDEOH VHWWLQJ XS DQG
PDQDJLQJDQHWZRUNRIVXFKGHYLFHVFDQEHFRPSOH[DQG
 UHTXLUHVDVLJQLILFDQWTXDQWLW\RIWKHP
)LJXUH7KHSULQFLSOHRIIX]]\ORJLF>@ 0DFKLQH OHDUQLQJ SURYHV WR EH D VXLWDEOH WRRO IRU
 DQDO\]LQJ DQG LQWHUSUHWLQJ WKH FROOHFWHG GDWD 1HXUDO
7KLVLVDKHXULVWLFVHDUFKDOJRULWKPHPSOR\HGIRUVROYLQJ QHWZRUNV LQ SDUWLFXODU H[KLELW WKH FDSDELOLW\ WR VZLIWO\
VHDUFK DQG RSWLPL]DWLRQ SUREOHPV ,W RSHUDWHV DV DQ SUHGLFWDQGGHWHFWIRUHVWILUHVZLWKDKLJKOHYHORIDFFXUDF\
HYROXWLRQDU\ SURJUDPPLQJ WHFKQLTXH WR IDFLOLWDWH 7KH VSHFLILF WHFKQLTXHV DQG PHWKRGV XVHG IRU WUDLQLQJ
UHLQIRUFHPHQW OHDUQLQJ 7KH DOJRULWKP DVVHVVHV QHXUDO QHWZRUNV VLJQLILFDQWO\ LQIOXHQFH WKHLU HYDOXDWLRQ
SHUIRUPDQFH WKURXJK D ILWQHVV IXQFWLRQ ZKLFK WDNHV DQ SHUIRUPDQFH
LQGLYLGXDODVLQSXWDQGSURGXFHVDUHDOQXPEHUDVRXWSXW $OO RI WKHVH PHQWLRQHG PHWKRGV DQG SULQFLSOHV SOD\ D
7KH SUREDELOLW\ RI DQ LQGLYLGXDO V VHOHFWLRQ LV RIWHQ YLWDO UROH LQ VDIHJXDUGLQJ IRUHVWU\ DQG HQDEOLQJ WKH HDUO\
UDQGRPL]HG DQG LQYHUVHO\ UHODWHG WR WKHLU ILWQHVV )RU GHWHFWLRQRISRWHQWLDOO\GHVWUXFWLYHILUHV
LQVWDQFHLI3HUVRQ;UHFHLYHVDILWQHVVVFRUHWKDWLVWZLFHDV
KLJK DV 3HUVRQ < 3HUVRQ ; PLJKW EH FKRVHQ DV D ,9 $&.12:/('*0(17
UHSURGXFWLYH WDUJHW WZLFH DV RIWHQ DV 3HUVRQ < <RX FDQ 7KLV SDSHU LV VXSSRUWHG E\ SURMHFWV .(*$ =8
REVHUYHWKHIORZFKDUWLOOXVWUDWLQJWKLVPHWKRGLQ)LJ>@  ³7HFKQRORJLHV RI SULYDWH FORXG HQYLURQPHQWV LQ
 XQLYHUVLW\HGXFDWLRQ´

5()(5(1&(6
>@ 73UHHWL6.DQDNDUDGGL$%HHODJL60DODJL
DQG $ 6XGL ³)RUHVW )LUH 3UHGLFWLRQ 8VLQJ 0DFKLQH
/HDUQLQJ7HFKQLTXHV´,QWHUQDWLRQDO&RQIHUHQFHRQ
,QWHOOLJHQW 7HFKQRORJLHV &21,7  -XQ  GRL
&21,7
>@ .9LVKDO9DVHHNDUDQ3.LVKRUH66UHHYDUVKDQ
66DUDQ\D*6XGKDDQG66XEELDK³&ORXG$VVLVWHG
,27 (QDEOHG )RUHVW )LUH 'HWHFWLRQ 6\VWHP´ UG
,QWHUQDWLRQDO&RQIHUHQFHRQ3RZHU(QHUJ\&RQWURODQG

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 235

 7UDQVPLVVLRQ 6\VWHPV ,&3(&76   3URFHHGLQJV &RQIHUHQFH RQ &RQWURO DQG &RPPXQLFDWLRQV 6,%&21
GRL,&3(&76   3URFHHGLQJV -XQ  GRL
>@ ³,QGLD 'HIRUHVWDWLRQ 5DWHV  6WDWLVWLFV _ *):´ 6,%&21
$FFHVVHG 6HS   >2QOLQH@ $YDLODEOH >@ ³:DVSPRWHௗª 7KH VPDUW VHQVRU SODWIRUP WR
KWWSVZZZJOREDOIRUHVWZDWFKRUJGDVKERDUGVFRXQWU\, GHYHORS ,R7 SURMHFWV  /LEHOLXP´ $FFHVVHG 6HS 
1'"PDS H\-M<:&E9X=&,G+-=;' >2QOLQH@$YDLODEOHKWWSVZZZOLEHOLXPFRPLRW
>@ $ $ .KDPXNKLQ DQG 6 %HUWROGR ³6SHFWUDO SURGXFWVZDVSPRWH
DQDO\VLV RI IRUHVW ILUH QRLVH IRU HDUO\ GHWHFWLRQ XVLQJ >@ : *URVVKDQGOHU DQG 0 -DFNVRQ ³$FRXVWLF
ZLUHOHVV VHQVRU QHWZRUNV´  ,QWHUQDWLRQDO 6LEHULDQ HPLVVLRQRIVWUXFWXUDOPDWHULDOVH[SRVHGWRRSHQIODPHV´
&RQIHUHQFH RQ &RQWURO DQG &RPPXQLFDWLRQV 6,%&21 )LUH6DI- YRO  QR SS± -DQGRL
  3URFHHGLQJV -XQ  GRL   
6,%&21 >@ $ -D\DNXPDU $ 6KDML DQG / 1LWKD ³:LOGILUH
>@ - 0DFKDM 3 %ULGD DQG , %ULGRYD ³,PSDFW RI IRUHFDVW ZLWKLQ WKH GLVWULFWV RI .HUDOD XVLQJ )X]]\ DQG
5DGLR 0DS RQ WKH 3HUIRUPDQFH RI )LQJHUSULQWLQJ $1),6´ 3URFHHGLQJV RI WKH WK ,QWHUQDWLRQDO
$OJRULWKPV´ /HFWXUH 1RWHV LQ &RPSXWHU 6FLHQFH &RQIHUHQFH RQ &RPSXWLQJ 0HWKRGRORJLHV DQG
LQFOXGLQJ VXEVHULHV /HFWXUH 1RWHV LQ $UWLILFLDO &RPPXQLFDWLRQ,&&0&SS±0DU
,QWHOOLJHQFH DQG /HFWXUH 1RWHV LQ %LRLQIRUPDWLFV  YRO GRL,&&0&,&&0&
 /1$, SS ±  GRL  >@ $//DWLIDK$6KDEULQD,1:DK\XQLDQG5
B),*85(6 6DGLNLQ³(YDOXDWLRQRI5DQGRP)RUHVWPRGHOIRUIRUHVW
>@ , %ULGRYD 3 %ULGD - 3DSDQ DQG 2 .UHMFDU ILUHSUHGLFWLRQEDVHGRQFOLPDWRORJ\RYHU%RUQHR´
³1HZ$GYDQFHG$SSURDFKIRU'DWD)ORZV3ULRULWL]DWLRQ ,QWHUQDWLRQDO &RQIHUHQFH RQ &RPSXWHU &RQWURO
DWDQ2XWSXWRID8VHU7HUPLQDO´,((($FFHVVYRO ,QIRUPDWLFVDQGLWV$SSOLFDWLRQV(PHUJLQJ7UHQGVLQ%LJ
SS ±  GRL 'DWDDQG$UWLILFLDO,QWHOOLJHQFH,&,1$SS±
$&&(66 2FWGRL,&,1$
>@ $6&KDQG.6%KDUJDYL56.LUDQ0. >@ $$'HVKPXNK6'%6RQDU59,QJROH5
.DXVKLN ' 5 3UDVKDQWKL DQG 6 6 .XPDU $JUDZDO & 'KXOH DQG 1 & 0RUULV ³6DWHOOLWH ,PDJH
³6$05$.6+$ௗ'HYHORSLQJDUHDOWLPHDQGDXWRPDWLF 6HJPHQWDWLRQ IRU )RUHVW )LUH 5LVN 'HWHFWLRQ XVLQJ
HDUO\ZDUQLQJV\VWHPIRUIRUHVWILUH´,QWHUQDWLRQDO *DXVVLDQ0L[WXUH0RGHOV´SS±-XQGRL
&RQIHUHQFHRQ&LUFXLWVDQG6\VWHPVLQ'LJLWDO(QWHUSULVH ,&$$,&
7HFKQRORJ\ ,&&6'(7  'HF  GRL >@ 9%KDUDWKLDQG&3HGGD5HGG\³&RPSDULVRQRI
,&&6'(7 )RUHVW )LUH 3UHGLFWLRQ 6\VWHP XVLQJ 0DFKLQH /HDUQLQJ
>@ : /HH 6 .LP < 7 /HH + : /HH DQG 0 $ORJULWKPV´  UG ,QWHUQDWLRQDO &RQIHUHQFH RQ
&KRL³'HHSQHXUDOQHWZRUNVIRUZLOGILUHGHWHFWLRQZLWK $GYDQFH &RPSXWLQJ DQG ,QQRYDWLYH 7HFKQRORJLHV LQ
XQPDQQHG DHULDO YHKLFOH´  ,((( ,QWHUQDWLRQDO (QJLQHHULQJ ,&$&,7( SS±0D\GRL
&RQIHUHQFH RQ &RQVXPHU (OHFWURQLFV ,&&(  SS ,&$&,7(
±0DUGRL,&&( >@ 7 6DUDYDQDQ 7 6DWKLVK DQG . .HHUWKLND
>@ $$'HVKPXNK6'%6RQDU59,QJROH5 ³)RUHFDVWLQJ (FRQRP\ XVLQJ 0DFKLQH /HDUQLQJ
$JUDZDO & 'KXOH DQG 1 & 0RUULV ³6DWHOOLWH ,PDJH $OJRULWKP´ WK ,QWHUQDWLRQDO &RQIHUHQFH RQ (PHUJLQJ
6HJPHQWDWLRQ IRU )RUHVW )LUH 5LVN 'HWHFWLRQ XVLQJ 5HVHDUFK LQ (OHFWURQLFV &RPSXWHU 6FLHQFH DQG
*DXVVLDQ0L[WXUH0RGHOV´SS±-XQGRL 7HFKQRORJ\ ,&(5(&7   GRL
,&$$,& ,&(5(&7
>@ $$'HVKPXNK6'%6RQDU59,QJROH5 >@ 7 6DUDYDQDQ DQG 6 6DUDYDQDNXPDU ³0RGHOLQJ
$JUDZDO & 'KXOH DQG 1 & 0RUULV ³6DWHOOLWH ,PDJH DQ(QHUJ\(IILFLHQW&OXVWHULQJ3URWRFROZLWK6SLGHU&DW
6HJPHQWDWLRQ IRU )RUHVW )LUH 5LVN 'HWHFWLRQ XVLQJ 6ZDUP 2SWLPL]DWLRQ IRU :61´ 3URFHHGLQJV RI ,(((
*DXVVLDQ0L[WXUH0RGHOV´SS±-XQGRL 9/6,'&6UG,(((&RQIHUHQFHRQ9/6,'HYLFH
,&$$,& &LUFXLW DQG 6\VWHP SS ±  GRL
>@ + 'DQJ1JRF DQG + 1JX\HQ7UXQJ ³$HULDO 9/6,'&6
)RUHVW )LUH 6XUYHLOODQFH(YDOXDWLRQ RI )RUHVW )LUH >@ %.6LQJK1.XPDUDQG37LZDUL³([WUHPH
'HWHFWLRQ 0RGHO XVLQJ $HULDO 9LGHRV´ ,QWHUQDWLRQDO /HDUQLQJ 0DFKLQH $SSURDFK IRU 3UHGLFWLRQ RI )RUHVW
&RQIHUHQFH RQ $GYDQFHG 7HFKQRORJLHV IRU )LUHV XVLQJ 7RSRJUDSKLFDO DQG 0HWURORJLFDO 'DWD RI
&RPPXQLFDWLRQV YRO 2FWREHU SS ± 2FW 9LHWQDP´ 3URFHHGLQJV   :RPHQ ,QVWLWXWH RI
GRL$7& 7HFKQRORJ\ &RQIHUHQFH RQ (OHFWULFDO DQG &RPSXWHU
>@ ' .LQDQHYD * +ULVWRY - 5D\FKHY DQG 3 (QJLQHHULQJ :,7&21 (&(  SS ± 1RY
=DKDULHY ³(DUO\ IRUHVW ILUH GHWHFWLRQ XVLQJ GURQHV DQG GRL:,7&21(&(
DUWLILFLDO LQWHOOLJHQFH´  QG ,QWHUQDWLRQDO >@ $*DLNZDG1%KXWD7-DGKDY3-DQJDOHDQG
&RQYHQWLRQ RQ ,QIRUPDWLRQ DQG &RPPXQLFDWLRQ 6 6KLQGH ³$ 5HYLHZ 2Q )RUHVW )LUH 3UHGLFWLRQ
7HFKQRORJ\ (OHFWURQLFV DQG 0LFURHOHFWURQLFV 0,352 7HFKQLTXHV´  WK ,QWHUQDWLRQDO &RQIHUHQFH RQ
  3URFHHGLQJV SS ± 0D\  GRL &RPSXWLQJ &RPPXQLFDWLRQ &RQWURO DQG $XWRPDWLRQ
0,352 ,&&8%($   GRL
>@ $ $ .KDPXNKLQ DQG 6 %HUWROGR ³6SHFWUDO ,&&8%($
DQDO\VLV RI IRUHVW ILUH QRLVH IRU HDUO\ GHWHFWLRQ XVLQJ
ZLUHOHVV VHQVRU QHWZRUNV´  ,QWHUQDWLRQDO 6LEHULDQ


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 236

  $QDO\VLVRIDYDLODEOHWHFKQRORJLHVIRUWKHGHVLJQ
RIDWUDFNLQJGHYLFHIRUPRQLWRULQJHQGDQJHUHG
DQLPDO
0LFKDO-DQRYHF-R]HI3DSiQ0LFKDO+UDãND
8QLYHUVLW\RIäLOLQDäLOLQD6ORYDNLD
0LFKDO-DQRYHF#IULXQL]DVN-R]HI3DSDQ#IULXQL]DVN0LFKDO+UDVND#IULXQL]DVN


$EVWUDFW² +LJK PRELOLW\ RI SHRSOH WKLQJV DQLPDOV LV DQ 3HW7UDFNLQJ/RFDWLRQGHYLFHVFDQEHHPSOR\HGWRWUDFN
HVVHQWLDOSDUWRIWRGD\ VOLIH*LYHQWKHIXQGDPHQWDOUROHRI WKHZKHUHDERXWVRISHWVLQWKHHYHQWWKH\DUHORVWDLGLQJLQ
PRELOLW\LQRXUOLYHVLWLVQHFHVVDU\WRGHWHUPLQHWKHORFDWLRQ WKHLUUHFRYHU\
RILQGLYLGXDOVDQGYDULRXVREMHFWV/RFDWLRQWUDFNLQJHQDEOHV )RUHVWU\$GPLQLVWUDWLRQ/RFDWLRQGHYLFHVDUHYDOXDEOH
EHWWHUQDYLJDWLRQ PDQDJHPHQWDQGRSWLPL]DWLRQRIYDULRXV WRROV IRU IRUHVWU\ DGPLQLVWUDWLRQ SDUWLFXODUO\ LQ WKH
SURFHVVHV ZKHWKHU LQ LQGXVWULDO SURGXFWLRQ VHUYLFHV SURWHFWLRQRIHQGDQJHUHGDQLPDOVSHFLHV
WUDQVSRUW DQLPDO WUDFNLQJ RU HYHQ LQ SHUVRQDO OLIH 7R
IDFLOLWDWH WKH WDVN RI GHWHUPLQLQJ WKH ORFDWLRQ RI PRQLWRUHG 1DYLJDWLRQ 6\VWHPV 1DYLJDWLRQ V\VWHPV D SUHYDOHQW
REMHFWV LW LV DGYLVDEOH WR XVH D GHGLFDWHG GHYLFH ZKLFK ZH DSSOLFDWLRQRIORFDWLRQGHYLFHVDVVLVWZLWKZD\ILQGLQJDQG
UHIHUWRDVDORFDWLRQGHYLFH,QWKLVDUWLFOHZHZLOOGHOYHLQWR RULHQWDWLRQLQXQIDPLOLDUHQYLURQPHQWVDLGLQJERWKGULYHUV
WKHFRPPXQLFDWLRQWHFKQRORJ\DQGWKHWHFKQRORJ\QHFHVVDU\ DQGSHGHVWULDQV
IRU GHWHUPLQLQJ WKH ORFDWLRQ RI WKH WDUJHW SRLQW :H ZLOO 7KHVH GHYLFHV SOD\ D SLYRWDO UROH LQ HQKDQFLQJ VDIHW\
H[SORUH FRPPXQLFDWLRQ WHFKQRORJLHV WKDW IDFLOLWDWH WKH HIILFLHQF\ DQG FRQYHQLHQFH DFURVV D ZLGH UDQJH RI
WUDQVIHURILQIRUPDWLRQEHWZHHQWKHORFDOL]DWLRQPRGXOHDQG VFHQDULRVDQGLQGXVWULHV
WKH RXWSXW FRPPXQLFDWLRQ PRGXOH FRQQHFWHG WR D VHUYHU 7KH GHYHORSPHQW DQG FRQVWUXFWLRQ RI ORFDOL]DWLRQ
$GGLWLRQDOO\ZHZLOOFRQVLGHUWKHVHOHFWLRQRIWHFKQRORJ\WKDW GHYLFHV KDYH OHG WR YDULRXV VROXWLRQV EDVHG RQ GLIIHUHQW
VXSSRUWV ORFDWLRQ GHWHUPLQDWLRQ YLD VDWHOOLWH SRVLWLRQLQJ WHFKQRORJLHV DQG SULQFLSOHV 7KH IXQGDPHQWDO SULQFLSOH
V\VWHPV2XUDQDO\VLVZLOODOVRHQFRPSDVVH[LVWLQJVROXWLRQV XQGHUO\LQJDOOORFDWLRQGHYLFHVLVWKHFROOHFWLRQRIVHQVRU\
WHFKQLTXHVDQGDOJRULWKPVUHODWHGWRORFDWLRQGHYLFHV GDWD W\SLFDOO\ UHFHLYHG IURP VDWHOOLWHVRUELWLQJWKH(DUWK
7KLV GDWD FRQWDLQV FULWLFDO LQIRUPDWLRQ DERXW WKH SUHFLVH
.H\ZRUGV   /RFDOL]DWLRQ *166 /R5D 3HW 7UDFNLQJ ORFDWLRQRIWKHWUDFNHGREMHFW
(QGDQJHUHG$QLPDOV/RFDOL]DWLRQ'HYLFH*60,R7
$Q HVVHQWLDO FRPSRQHQW RI D ORFDOL]DWLRQ GHYLFH LV WKH
, ,1752'8&7,21 FRPPXQLFDWLRQ LQWHUIDFH :KHQ XVLQJ D ORFDWLRQ GHYLFH
VROHO\ IRU SHUVRQDO ORFDWLRQ GHWHUPLQDWLRQ DQG QRW IRU
,QWRGD\ V ZRUOGWKH KLJK PRELOLW\ RI ERWKSHRSOHDQG FRQQHFWLQJ WR WKH ,QWHUQHW RI 7KLQJV ,R7  QHWZRUN WKLV
HYHU\GD\ REMHFWV VXFK DV FDUV VPDUWSKRQHV NH\V DQG HOHPHQW PD\ QRW EH QHFHVVDU\ 7KH FRPPXQLFDWLRQ
YDULRXVRWKHULWHPVDVZHOODVOLYLQJFUHDWXUHVLVDGHILQLQJ LQWHUIDFH HQFRPSDVVHV RQH RU PRUH FRPPXQLFDWLRQ
FKDUDFWHULVWLF7KLVPRELOLW\LVFORVHO\DVVRFLDWHGZLWKWKH SHULSKHUDOV ZKLFK FDQ EH EXLOW XVLQJ YDULRXV ZLUHOHVV
QHFHVVLW\RISLQSRLQWLQJWKHSUHFLVHSRVLWLRQVRILQGLYLGXDOV FRPPXQLFDWLRQ WHFKQRORJLHV 7KHVH WHFKQRORJLHV UHO\ RQ
DQG REMHFWV 7KH DELOLW\ WR DFFXUDWHO\ GHWHUPLQH ORFDWLRQ GLIIHUHQWWUDQVPLVVLRQPHGLDDQGWUDQVPLVVLRQDOJRULWKPV
VXEVHTXHQWO\IDFLOLWDWHVLPSURYHGQDYLJDWLRQFRQWURODQG
,W V LPSRUWDQW WR QRWH WKDW ZKHQ LPSOHPHQWLQJ D
RSWLPL]DWLRQ RI YDULRXV SURFHVVHV ZKHWKHU LQ LQGXVWULDO
FRPPXQLFDWLRQ WHFKQRORJ\ EDVHG RQ SK\VLFDO PHWDOOLF RU
SURGXFWLRQ VHUYLFHV SURWHFWLRQ RI HQGDQJHUHG DQLPDOV
WUDQVSRUWDWLRQRUSHUVRQDOOLIH RSWLFDO PHGLXPV IRU GDWD WUDQVIHU WKH PRELOLW\ RI WKH
ORFDOL]DWLRQ GHYLFH PD\ EH OLPLWHG GXH WR WKH QHHG IRU
7RHQDEOHLQGLYLGXDOVWRGHWHUPLQHWKH ZKHUHDERXWVRI SK\VLFDOFRQQHFWLRQV
WKHVH REMHFWV WKH\ UHTXLUH D GHYLFH GHVLJQHG IRU WKLV
$ FUXFLDO FRQVLGHUDWLRQ ZKHQ VHOHFWLQJ D ZLUHOHVV
SXUSRVHFRPPRQO\UHIHUUHGWRDVDORFDWLRQGHYLFH
FRPPXQLFDWLRQ WHFKQRORJ\ LV WKH VHFXULW\ RI GDWD
/RFDWLRQ GHYLFHV KDYH LQGHHG IRXQG ZLGHVSUHDG XVH WUDQVPLVVLRQ (QVXULQJ WKH VHFXUH WUDQVPLVVLRQ RI
DFURVV YDULRXV DVSHFWV RI OLIH +HUH DUH VRPH QRWDEOH PHDVXUHG GDWD LV YLWDO WR SUHYHQW SRWHQWLDO PLVXVH RU
H[DPSOHVRIWKHLUDSSOLFDWLRQV XQDXWKRUL]HGDFFHVV
,QGXVWULDO3URGXFWLRQ/RFDWLRQGHYLFHVDUHHPSOR\HGLQ ,QWKHDUWLFOHZH ZLOOGHOYHLQWRWKHEDVLFFRPSRQHQWV
LQGXVWULDO VHWWLQJV WR PRQLWRU WKH PRYHPHQWV RI QHHGHG LQ WKH GHYHORSPHQW RI D WUDFNLQJ GHYLFH
FRPSRQHQWV SHUVRQQHO DQG SDUWV GXULQJ WKH SURGXFWLRQ 6SHFLILFDOO\ ZH ZLOO H[DPLQH WKH QHFHVVDU\ WUDQVPLVVLRQ
SURFHVV 7KLV KHOSV HQKDQFH ZRUN HIILFLHQF\ RSWLPL]H WHFKQRORJLHVIRUFRPPXQLFDWLRQZLWKLQWKH,R7QHWZRUNDV
SURGXFWLRQ SURFHVVHV DQG LPSURYH ZDUHKRXVH ZHOODVWKHORFDWLRQSURWRFROVDQGWHFKQLTXHVXVHGLQGHYLFH
PDQDJHPHQW RSHUDWLRQ
$VVHW 6HFXULW\ 7KHVH GHYLFHV DUH XVHG WR WUDFN DQG
VHFXUH HYHU\GD\ HVVHQWLDOV OLNH PRELOH SKRQHV DQG ,, $1$/<6,62)7+(,668(
FRPSXWHUVVDIHJXDUGLQJWKHPDJDLQVWWKHIWRUXQDXWKRUL]HG  7KH QHHG IRU REMHFW ORFDOL]DWLRQ VSDQV YDULRXV
UHORFDWLRQ GRPDLQV LQFOXGLQJ LQGXVWULDO DSSOLFDWLRQV DQG IRUHVWU\

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 237

PDQDJHPHQW ,Q WKHVH FRQWH[WV WKHUH DUH VWULQJHQW */21$66*/21$66W\SLFDOO\FRQVLVWVRIDURXQG
UHTXLUHPHQWV IRU ERWK WKH DFFXUDF\ RI ORFDWLRQ WRRSHUDWLRQDOVDWHOOLWHV
GHWHUPLQDWLRQ DQG WKH UHOLDELOLW\ RI LQIRUPDWLRQ &RYHUDJH
WUDQVPLVVLRQ UHJDUGLQJ DQ REMHFW V SRVLWLRQ 0RUHRYHU *36*36SURYLGHVJOREDOFRYHUDJHDQGLVZLGHO\XVHG
VHFXULW\ FRQVLGHUDWLRQV DUH EHFRPLQJ LQFUHDVLQJO\ FUXFLDO ZRUOGZLGHIRUYDULRXVDSSOLFDWLRQV
LQWKHHQWLUHORFDOL]DWLRQSURFHVVDVGDWDSHUWDLQLQJWRDQ
REMHFW V ORFDWLRQ LV RIWHQ FDWHJRUL]HG DV VHQVLWLYH *DOLOHR*DOLOHRRIIHUVJOREDOFRYHUDJHDQGLVGHVLJQHG
LQIRUPDWLRQ WR ZRUN LQ FRQMXQFWLRQ ZLWK RWKHU *166 V\VWHPV IRU
LPSURYHGDFFXUDF\DQGUHOLDELOLW\
)RUWKLVUHDVRQUHO\LQJRQFRPPHUFLDOFRPPXQLFDWLRQ
FKDQQHOV IRU REMHFW ORFDOL]DWLRQ LV QRW DOZD\V VXLWDEOH $ %HLGRX %'6 %'6RIIHUVUHJLRQDODQGJOREDOFRYHUDJH
PRUHDSSURSULDWHDOWHUQDWLYHLQYROYHVWKHXWLOL]DWLRQRIQRQ ZLWKDQHPSKDVLVRQWKH$VLD3DFLILFUHJLRQ
WUDGLWLRQDOFRPPXQLFDWLRQWHFKQRORJLHVWKDWGRQRWUHO\RQ */21$66*/21$66SURYLGHVJOREDOFRYHUDJHEXWLV
WKH,QWHUQHWDQGRIIHUEHWWHUFRQWURORYHUDFFHVVWRVHQVLWLYH SDUWLFXODUO\VWURQJLQQRUWKHUQODWLWXGHV
GDWD 7KLV DSSURDFK HQVXUHV JUHDWHU VHFXULW\ DQG )UHTXHQF\%DQGV
FRQILGHQWLDOLW\ZKHQKDQGOLQJORFDWLRQUHODWHGLQIRUPDWLRQ *36 *36 RSHUDWHV LQ PXOWLSOH IUHTXHQF\ EDQGV
IRUREMHFWV LQFOXGLQJ//DQG//LVFRPPRQO\XVHGIRUFLYLOLDQ
7KH PDUNHW RIIHUV D UDQJH RI ORFDOL]DWLRQ GHYLFHV DQG DSSOLFDWLRQV
WHFKQRORJLHVVXLWDEOHIRUDGGUHVVLQJWKHVSHFLILHGSUREOHP *DOLOHR *DOLOHR XVHV PXOWLSOH IUHTXHQF\ EDQGV
+RZHYHU PDQ\ RI WKHVH VROXWLRQV WHQG WR EH ILQDQFLDOO\ LQFOXGLQJ ( (D (E ( DQG ( ( LV HTXLYDOHQW WR
GHPDQGLQJ 2QH VLJQLILFDQW GUDZEDFN RI FRPPHUFLDOO\ *36/DQGLVXVHGIRUFLYLOLDQSXUSRVHV
DYDLODEOHVROXWLRQVOLHVLQWKHFKRLFHRIWHFKQRORJLHVXVHG
IRU WUDQVIHUULQJ VHQVLWLYH GDWD EHWZHHQ WKH PHDVXULQJ %HLGRX %'6 %'6XWLOL]HVPXOWLSOHIUHTXHQF\EDQGV
GHYLFH DQG WKH HQG GHYLFH 7KHVH WHFKQRORJLHV RIWHQ LQFOXGLQJ%%DQG%ZLWK%HTXLYDOHQWWR*36/
LQYROYHVWRULQJGDWDDERXWWKHORFDWLRQRIDWUDFNHGREMHFW */21$66 */21$66 DOVR RSHUDWHV LQ PXOWLSOH
LQ D GDWDEDVH DQG JUDSKLFDOO\ SUHVHQWLQJ LW WR WKH FOLHQW IUHTXHQF\EDQGVLQFOXGLQJ/DQG/
8QIRUWXQDWHO\ WKLV GDWD LV VRPHWLPHV XVHG IRU RWKHU $FFXUDF\DQG3UHFLVLRQ
SXUSRVHVZKLFKFDQSRVHFKDOOHQJHVLQPDLQWDLQLQJDKLJK *36*36LVNQRZQIRULWVKLJKDFFXUDF\DQGSUHFLVLRQ
OHYHORIVHFXULW\ZKHQZRUNLQJZLWKVHQVLWLYHLQIRUPDWLRQ HVSHFLDOO\LQRSHQVN\HQYLURQPHQWV
$ *166WHFKQRORJLHV *DOLOHR *DOLOHR DLPV WR SURYLGH VLPLODU OHYHOV RI
7KH ORFDOL]DWLRQ GHYLFH FDQ FRQVLVW RI VHYHUDO DFFXUDF\ WR *36 PDNLQJ LW VXLWDEOH IRU D ZLGH UDQJH RI
FRPPXQLFDWLRQPRGXOHV$WLWVFRUHWKHORFDWLRQGHYLFHLV DSSOLFDWLRQV
XVXDOO\ HTXLSSHG ZLWK D ORFDWLRQ PRGXOH WKDW FDQ XVH %HLGRX %'6 %'6RIIHUVJRRGDFFXUDF\DQGSUHFLVLRQ
YDULRXV WHFKQRORJLHV VXSSRUWLQJ VDWHOOLWH QDYLJDWLRQ IRU FLYLOLDQ XVHUV DQG FDQ SURYLGH LPSURYHG DFFXUDF\ LQ
1RWDEOH H[DPSOHV RI VDWHOOLWH QDYLJDWLRQ V\VWHPV LQFOXGH GXDOIUHTXHQF\PRGH
*36 GHYHORSHGLQWKH86 *DOLOHR GHYHORSHGLQ(XURSH  */21$66 */21$66 SURYLGHV JRRG DFFXUDF\
*/21$66 GHYHORSHG LQ 5XVVLD  %HLGRX GHYHORSHG LQ HVSHFLDOO\LQQRUWKHUQUHJLRQV
&KLQD DPRQJRWKHUV7KHVHWHFKQRORJLHVDUHWKHEDVLVRI 6LJQDO&RPSDWLELOLW\
DFFXUDWHORFDWLRQGHWHUPLQDWLRQLQVXFKGHYLFHV,QRUGHUWR
VHOHFWVXLWDEOHFDQGLGDWHVIRURXUDSSOLFDWLRQLWLVQHFHVVDU\ *36 *DOLOHR %'6 DQG */21$66 VLJQDOV DUH RIWHQ
WR FRPSDUH WKH DIRUHPHQWLRQHG WHFKQRORJLHV LQ GLIIHUHQW FRPSDWLEOH ZLWK PXOWL*166UHFHLYHUV DOORZLQJ XVHUVWR
DUHDV>@>@ EHQHILWIURPPXOWLSOHFRQVWHOODWLRQVWRLPSURYHSRVLWLRQLQJ
DFFXUDF\DQGUHOLDELOLW\
&RPSDULVRQRIWKHVH*166V\VWHPV
&LYLOLDQDQG0LOLWDU\8VH
2ULJLQDQG2ZQHUVKLS
*36*36ZDVLQLWLDOO\GHYHORSHGIRUPLOLWDU\XVHEXWLV
*36 *36 LV GHYHORSHG DQG RSHUDWHG E\ WKH 8QLWHG QRZZLGHO\XVHGIRUFLYLOLDQDSSOLFDWLRQV
6WDWHV'HSDUWPHQWRI'HIHQVH 'R' 
*DOLOHR *DOLOHR LV SULPDULO\ LQWHQGHG IRU FLYLOLDQ XVH
*DOLOHR *DOLOHR LV GHYHORSHG DQG RSHUDWHG E\ WKH DOWKRXJKLWKDVVRPHVHFXULW\IHDWXUHV
(XURSHDQ 8QLRQ (8  DQG WKH (XURSHDQ 6SDFH $JHQF\
(6$  %HLGRX %'6  %'6 RIIHUV FLYLOLDQ DQG DXWKRUL]HG
PLOLWDU\VHUYLFHV RSHQDQGDXWKRUL]HGDFFHVV 
%HLGRX %'6  %HLGRX LV GHYHORSHG DQG RSHUDWHG E\
&KLQD VSHFLILFDOO\ WKH &KLQD 1DWLRQDO 6SDFH */21$66 */21$66 SURYLGHV FLYLOLDQ DQG PLOLWDU\
$GPLQLVWUDWLRQ &16$  VLJQDOV RSHQDQGUHVWULFWHGDFFHVV 
*/21$66 */21$66 LV GHYHORSHG DQG RSHUDWHG E\ 6LJQDO$XWKHQWLFDWLRQ
5XVVLD PDQDJHG E\ WKH 5XVVLDQ )HGHUDO 6SDFH $JHQF\ *36DQG*DOLOHRGRQRWSURYLGHDXWKHQWLFDWLRQIHDWXUHV
526&26026  IRUWKHLUVWDQGDUGVLJQDOV
1XPEHURI6DWHOOLWHV %HLGRX %'6 DQG*/21$66RIIHUDXWKHQWLFDWLRQDQG
*36*36KDVDPLQLPXPFRQVWHOODWLRQRIVDWHOOLWHV HQFU\SWLRQIHDWXUHVIRUVHFXUHDSSOLFDWLRQV
ZLWKDGGLWLRQDOVDWHOOLWHVLQUHVHUYH 7DEOHSUHVHQWVGHWDLOVDERXWWKHFRPPXQLFDWLRQVLJQDOV
*DOLOHR *DOLOHR DLPV WR KDYH D WRWDO RI  VDWHOOLWHV DQGWKHIUHTXHQF\UDQJHVXWLOL]HGE\VSHFLILFSURWRFROVIRU
LQFOXGLQJRSHUDWLRQDODQGVSDUHVDWHOOLWHV WKHSXUSRVHRIORFDWLRQGHWHUPLQDWLRQ
%HLGRX %'6 %'6DLPVWRKDYHDWRWDORIVDWHOOLWHV 
LQ LWV FRQVWHOODWLRQ LQFOXGLQJ RSHUDWLRQDO DQG EDFNXS 
VDWHOOLWHV 

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 238

 %: 0+] VKDUHGZLWK*36/ (EDW0+](DW
6\VWHP %DQG &HQWHU)UHT 0+]  0+]DQG(DW0+]>@>@
0+] 

/  
*36 /  
/  
/  
*/21$66
/  
(  
*DOLOHR (D  
(E   
)LJXUH7UDMHFWRU\RIVDWHOOLWHVHHDQGH6DWHOOLWHHLQ
%   RUELWLVWDNHQDVDUHIHUHQFH>@

%HLGRX OO  %  
)LJXUH  GLVSOD\V WZR VDWHOOLWHV ( DQG ( ZKLFK
%   HQFRXQWHUHGODXQFKLVVXHVLQDQGFRQVHTXHQWO\GLGQRW
6EDQG UHDFKWKHLULQWHQGHGRUELWV3UHVHQWO\IROORZLQJDVHULHVRI
  PDQHXYHUV WKHVH WZR VDWHOOLWHV DUH LQ RUELW DW DOWLWXGHV
5; 
%HLGRX O  UDQJLQJIURPNPWRNPZLWKDQLQFOLQDWLRQ
/EDQG  %36. RIƒ7KHLUUHYROXWLRQSHULRGLVDSSUR[LPDWHO\KRXUV

7;  0RG  DQG  PLQXWHV 5HPDUNDEO\ GHVSLWH WKHLU LQLWLDO
 FKDOOHQJHV WKHVH VDWHOOLWHV FDQ QRZ EH XWLOL]HG IRU
7DEOH)UHTXHQFLHVDQGVLJQDOVIRUHDFK*166WHFKQRORJ\>@ SRVLWLRQLQJSXUSRVHV DQGSUHFLVHRUELWGDWD IRUERWK(
 DQG(LVDFFHVVLEOHIRUXVH>@>@
,Q FRQFOXVLRQ *36 *DOLOHR %HLGRX %'6  DQG % :LUHOHVV7HFKQRORJLHV
*/21$66 DUH LPSRUWDQW *166 V\VWHPV ZLWK WKHLU RZQ :LUHOHVV FRPPXQLFDWLRQ LV D WHFKQRORJ\ WKDW UHOLHV RQ
FKDUDFWHULVWLFV DQG VWUHQJWKV0DQ\ PRGHUQUHFHLYHUV FDQ WUDQVPLWWLQJ GDWD XVLQJ D PHGLXP RWKHU WKDQ PHWDOOLF
XWLOL]H VLJQDOV IURP PXOWLSOH V\VWHPV WR HQKDQFH PDWHULDOV$SDUWIURPPHWDOOLFPHGLDWKHUHDUHWKUHHRWKHU
SRVLWLRQLQJDFFXUDF\DQGDYDLODELOLW\7KHFKRLFHEHWZHHQ WUDQVPLVVLRQWHFKQRORJLHVUDGLRZDYHVVRXQG VRQLF DQG
WKHVH V\VWHPV PD\ GHSHQG RQ IDFWRUV VXFK DV UHJLRQDO OLJKW RSWLFDO  7KH FRPPXQLFDWLRQ UDQJH RI D JLYHQ
SUHIHUHQFHV VLJQDO DYDLODELOLW\ DQG VSHFLILF DSSOLFDWLRQ WHFKQRORJ\GHSHQGVRQYDULRXVIDFWRUVLQFOXGLQJZHDWKHU
UHTXLUHPHQWV FRQGLWLRQVEXWSULPDULO\RQWKHW\SHRIWHFKQRORJ\FKRVHQ
)RURXUUHIHUHQFHDQGDVWKHSULPDU\*166V\VWHPVWR )RULQVWDQFH%OXHWRRWKWHFKQRORJ\DOORZVFRPPXQLFDWLRQ
FRQVLGHUIRUWKHORFDOL]DWLRQGHYLFHZHZLOOIRFXVRQ*36 RYHU VKRUW GLVWDQFHV W\SLFDOO\ MXVW D IHZ WHQV RI PHWHUV
DQG *DOLOHR 7KHVH WZR V\VWHPV DUH KLJKO\ VXLWDEOH ZKLOH KLJKIUHTXHQF\ UDGLR WUDQVPLVVLRQ LQ RXWHU VSDFH
FDQGLGDWHV>@>@ HQDEOHVFRPPXQLFDWLRQRYHUYDVWGLVWDQFHV>@±>@
7KH PRVW ZLGHO\ XVHG *166 V\VWHP LV WKH $PHULFDQ ,QWKHFRQWH[WRIWKLVZRUNZHZLOOIRFXVRQUDGLREDVHG
*362ULJLQDOO\FRPSULVLQJVDWHOOLWHVWKHFRQVWHOODWLRQ FRPPXQLFDWLRQ ZKLFK LV FRPPRQO\ XVHG IRU
ZDV H[SDQGHG LQ  E\ DQ DGGLWLRQDO WKUHH VDWHOOLWHV LPSOHPHQWLQJ,R7 ,QWHUQHWRI7KLQJV GHYLFHV,R7GHYLFHV
NQRZQ DV WKH ([SDQGDEOH  FRQVWHOODWLRQ 7KHVH RIWHQ XWLOL]H UDGLR FRPPXQLFDWLRQ WHFKQRORJLHV VXFK DV
VDWHOOLWHV DUH SRVLWLRQHG DFURVV VL[ RUELWDO SODQHV DW DQ :L)L*60/R5DDQG%OXHWRRWK>@±>@
LQFOLQDWLRQRIƒDQGRUELWDWDQDOWLWXGHRINP>@ %OXHWRRWKWHFKQRORJ\RSHUDWHVXQGHUWKH,(((
>@ VWDQGDUG DQG EHORQJV WR WKH FDWHJRU\ RI SHUVRQDO DUHD
2Q WKH RWKHU KDQG WKH *DOLOHR V\VWHP LV D (XURSHDQ QHWZRUNV 3$1V  2YHU WKH \HDUV VHYHUDO YHUVLRQV RI
QDYLJDWLRQV\VWHPH[SHFWHGWREHIXOO\RSHUDWLRQDOE\ %OXHWRRWK KDYH EHHQ GHYHORSHG ZLWK WKH ODWHVW UHOHDVH
,WZLOOFRQVLVWRIVDWHOOLWHVZLWKRSHUDWLRQDOVDWHOOLWHV EHLQJ YHUVLRQ ZKLFK LV LQWHJUDWHGLQWR PRVWSHUVRQDO
DQGVL[VSDUHV*DOLOHRVDWHOOLWHVDUHSRVLWLRQHGDWDQDOWLWXGH GHYLFHV7KLVWHFKQRORJ\RSHUDWHVLQWKHVDPH*+],60
RI  NP GLVWULEXWHG DFURVV WKUHH FLUFXODU PHGLXP ,QGXVWULDO 6FLHQWLILF DQG 0HGLFDO  EDQG DV :L)L ,W
(DUWK RUELW SODQHV ZLWK DQ LQFOLQDWLRQ RI ƒ 7KH ILUVW HPSOR\VDWHFKQLTXHNQRZQDV)UHTXHQF\+RSSLQJ6SUHDG
*DOLOHRVDWHOOLWHVZHUHODXQFKHGLQDQGE\ 6SHFWUXP )+66  ZKLFK DOORZV LW WR VZLWFK EHWZHHQ
VDWHOOLWHVKDGEHHQGHSOR\HG*DOLOHRXWLOL]HVLWVRZQWLPH PXOWLSOHIUHTXHQF\EDQGVIRUGDWDWUDQVPLVVLRQ7KLV)+66
V\VWHP NQRZQ DV *DOLOHR 6\VWHP 7LPH *67  ZKLFK PHWKRG HQKDQFHV UHVLVWDQFH WR LQWHUIHUHQFH GXULQJ GDWD
EHJDQ RQ $XJXVW   DW  87& 6LPLODU WR WUDQVPLVVLRQ>@>@
*36 *DOLOHR GRHV QRW DFFRXQW IRU OHDS VHFRQGV 'XULQJ WKH WUDQVPLVVLRQ SURFHVV GDWD LV GLYLGHG LQWR
$GGLWLRQDOO\ *DOLOHR HPSOR\V LWV RZQ UHIHUHQFH IUDPH SDFNHWVDQGWUDQVPLWWHGDFURVVDYDLODEOHFKDQQHOVHDFK
FDOOHG *DOLOHR 7HUUHVWULDO 5HIHUHQFH )UDPH *75)  7KH ZLWKDEDQGZLGWKRI0+],QSUDFWLFDOWHUPVWKLVPHDQV
UHYROXWLRQ SHULRG IRU *DOLOHR VDWHOOLWHV LV  KRXUV DQG  WKDW DSSUR[LPDWHO\  VXFK IUHTXHQF\ KRSVRFFXULQD
PLQXWHV>@>@±>@ VLQJOHVHFRQGRILQIRUPDWLRQWUDQVPLVVLRQ7KLVH[WHQVLYH
*DOLOHRXWLOL]HVVHYHUDOVLJQDOIUHTXHQFLHVLQFOXGLQJ(O IUHTXHQF\KRSSLQJHQKDQFHVWKHWHFKQRORJ\ VLPPXQLW\WR
DW  0+] VKDUHG ZLWK *36 /  (D DW 

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 239

LQWHUIHUHQFHDWWKHVDPHIUHTXHQF\PDNLQJLWPRUHUREXVW /R5D 33  ,Q /R5D 3HHUWR3HHU FRPPXQLFDWLRQ
LQQRLV\ZLUHOHVVHQYLURQPHQWV>@>@ PRGH WZR FOLHQWV FRPPXQLFDWH GLUHFWO\ ZLWK HDFK RWKHU
:L)LDOVRDGKHUHVWRWKHRSHQVWDQGDUGVRXWOLQHGLQWKH DQG WKH\ DUH FRQVLGHUHG HTXDOV GXULQJ WKH WUDQVPLVVLRQ
,((( VWDQGDUG DQG LVRIWHQUHIHUUHG WR DV:/$1 SURFHVV 7R HQDEOH WKLV FRPPXQLFDWLRQ RQH FOLHQW LV
:LUHOHVV/RFDO$UHD1HWZRUN ,WVIXQGDPHQWDOPRGHRI GHVLJQDWHGDVWKHWUDQVPLWWHUZKLOHWKHRWKHUEHFRPHVWKH
WUDQVPLVVLRQUHOLHVRQXWLOL]LQJDOLFHQVHIUHHEDQGZKLFK UHFHLYHU%RWKSDUWLHVLQYROYHGLQWKHFRPPXQLFDWLRQPXVW
HQDEOHVWKHFUHDWLRQRIH[WHQVLYHQHWZRUNVZLWKRXWWKHQHHG KDYHSUHHVWDEOLVKHGFRPPXQLFDWLRQSDUDPHWHUVLQFOXGLQJ
IRUOD\LQJSK\VLFDOPHWDOOLFLQIUDVWUXFWXUH6LQFHWKHUHLVD WKH WUDQVPLVVLRQ IUHTXHQF\ FRGLQJ VFKHPH DQG RWKHU
OLPLWHGQXPEHURIXVDEOHIUHTXHQFLHVIRUFRPPXQLFDWLRQ UHOHYDQWVHWWLQJV>@>@
DQG PDQ\ RI WKHP DUH SURQH WR LQWHUIHUHQFH UHJXODWRU\ /R5D:$1/R5D:$1RQWKHRWKHUKDQGLVDSURWRFRO
DXWKRULWLHV RYHUVHH WKH DOORFDWLRQ RI WKHVH IUHTXHQFLHV WKDW DOORZV IRU FRPPXQLFDWLRQ DPRQJ PXOWLSOH SRLQWV
8VHUV W\SLFDOO\ QHHG OLFHQVHV WR RSHUDWH ZLWKLQ WKHVH FUHDWLQJ D FRPSOH[ ZLUHOHVV QHWZRUN 7KLV SURWRFRO
UHJXODWHGEDQGV%DQGVWKDWZHUHRQFHSULPDULO\UHVHUYHG RSHUDWHVRQDFOLHQWVHUYHUPRGHOZLWKWKHVHUYHUDFWLQJDV
IRUFRPPHUFLDOSXUSRVHVVXFKDVWKH*+]EDQGKDYH WKHFHQWUDOQRGHUHVSRQVLEOHIRUSURFHVVLQJDQGIRUZDUGLQJ
EHHQPDGHDYDLODEOHIRUEXVLQHVVHVVFLHQWLILFDSSOLFDWLRQV GDWD WR WKH GDWD VHUYHU 7KLV DSSURDFK HQDEOHV D VLQJOH
DQGSXEOLFXVH>@ VHUYHUWRVHUYHPXOWLSOHFOLHQWVVLPXOWDQHRXVO\/R5D:$1
,Q SUDFWLFDO WHUPV WKLV WHFKQRORJ\ LV XELTXLWRXV DQG QHWZRUNV DOVR VXSSRUW WKH GHSOR\PHQW RI PXOWLSOH
ZLGHO\ XVHG ,W JDLQHG SRSXODULW\ GXH WR LWV FDSDELOLW\ WR FRPPXQLFDWLRQ QRGHV FDOOHG JDWHZD\V FUHDWLQJ DQ
SURYLGH ,QWHUQHW DFFHVV LQ KRXVHKROGV DQG EXVLQHVVHV H[WHQVLYH QHWZRUN ZKHUH HDFK FOLHQW FRPPXQLFDWHV ZLWK
SDUWLFXODUO\IRUPRELOHGHYLFHVWKDWODFNDSK\VLFDOZLUHG WKH JDWHZD\ WKDW SURYLGHV WKH EHVW FRQQHFWLRQ TXDOLW\ ,Q
FRQQHFWLRQ 7R GLIIHUHQWLDWH EHWZHHQ YDULRXV QHWZRUNV D  WKH /R5D:$1 $OOLDQFH ZDV HVWDEOLVKHG WR
6HUYLFH6HW,GHQWLILHU 66,' LVXVHGZKLFKFDQFRQWDLQXS VWDQGDUGL]H DQG SURPRWH WKLV FRPPXQLFDWLRQ SURWRFRO
WR  $6&,, FKDUDFWHUV HQVXULQJ WKH GLVWLQFWLRQ EHWZHHQ IDFLOLWDWLQJFRPSDWLELOLW\EHWZHHQGHYLFHV>@>@>@
LQGLYLGXDO QHWZRUNV 7KLV QHWZRUN ODEHO LV SHULRGLFDOO\ 7KH*60 *OREDO6\VWHPIRU0RELOH&RPPXQLFDWLRQV 
EURDGFDVW WR WKH QHWZRUN XVLQJ D EURDGFDVW DGGUHVV V\VWHPRSHUDWHVXVLQJWZRIUHTXHQF\EDQGV±0+]
HQVXULQJFRYHUDJHIRUDOOSRWHQWLDOFOLHQWV7KHSURFHVVRI IRU WUDQVPLVVLRQ DQG ±0+] IRU UHFHSWLRQ ,W
FRQQHFWLQJDFOLHQWWRWKHQHWZRUNLVNQRZQDVDVVRFLDWLRQ HPSOR\V D FRPELQDWLRQ RI )'' )UHTXHQF\ 'LYLVLRQ
ZLWK WKH FRUUHVSRQGLQJ DFFHVV SRLQW $3  7KHUH DUH WZR 'XSOH[ DQG7'0$ 7LPH'LYLVLRQ0XOWLSOH$FFHVV IRU
W\SHVRIVXFKQHWZRUNV>@ FRPPXQLFDWLRQ 7KH UHFHSWLRQ EDQG LV GLYLGHG LQWR 
$GKRF 1HWZRUNV 7KHVH QHWZRUNV IDFLOLWDWH GLUHFW FKDQQHOV HDFK ZLWK D EDQGZLGWK RI  .+] 7KHVH
FRPPXQLFDWLRQ EHWZHHQ WZR SHHU FOLHQWV ZLWK WKHLU FKDQQHOVDUHVKDUHGDPRQJDVPDQ\DVHLJKWXVHUV>@
LGHQWLILFDWLRQ EDVHG RQ WKH 66,' $GKRF QHWZRUNV DUH 7KH*60V\VWHPLVSULPDULO\FRPSRVHGRIWKUHHPDLQ
VXLWDEOH IRU VPDOO DUHDV DQG RFFDVLRQDO FRPPXQLFDWLRQ SDUWV
ZLWKLQVPDOOQHWZRUNV>@ 1HWZRUNDQG6ZLWFKLQJ6XEV\VWHP 166 $WWKHFRUHRI
,QIUDVWUXFWXUH 1HWZRUNV ,QIUDVWUXFWXUH QHWZRUNV DUH 166 LV WKH 0RELOH6HUYLFHV 6ZLWFKLQJ &HQWHU 06& 
GHVLJQHG IRU ODUJHU QHWZRUNV DQG ODUJHU DUHDV ,Q WKHVH ZKLFKVHUYHVDVWKHFHQWUDOVZLWFKLQJHQWLW\UHVSRQVLEOHIRU
QHWZRUNV FOLHQWV FRQQHFW WR DFFHVV SRLQWV $3V  $FFHVV HVVHQWLDO VZLWFKLQJ IXQFWLRQV 06& FDQ HVWDEOLVK
SRLQWVEURDGFDVWWKH66,'FRGHRIWKHQHWZRUNWKH\RIIHU FRQQHFWLRQVZLWKRWKHUQHWZRUNV
FRQQHFWLYLW\WR2QHVLJQLILFDQWDGYDQWDJHRILQIUDVWUXFWXUH %DVH 6WDWLRQ 6XEV\VWHP %66  %66 FRQVLVWV RI WKH
QHWZRUNVLVWKHSUHVHQFHRIQXPHURXVDFFHVVSRLQWVZKLFK %DVLF6\VWHP&RQWUROOHU %6& %DVLF7UDQVFHLYHU6\VWHP
FDQ EH XVHG WR FUHDWH D VLQJOH H[WHQVLYH QHWZRUN 7KLV %76  DQG 0RELOH 6WDWLRQ 06  %6& DQG %76 DUH
HQVXUHV WKDW FOLHQWV DOZD\V KDYH WKH EHVW SRVVLEOH UHVSRQVLEOH IRU FRQWUROOLQJ DQG PDQDJLQJ WKH UDGLR
FRQQHFWLRQWRWKHQHWZRUN>@ FRPPXQLFDWLRQEHWZHHQPRELOHGHYLFHVDQGWKHQHWZRUN
/R5D /RQJ5DQJH WHFKQRORJ\LVDPRQJWKHQXPHURXV 06UHSUHVHQWVWKHPRELOHGHYLFHVWKHPVHOYHV>@
VROXWLRQVDYDLODEOHIRUGHYLFHVGHVLJQHGIRUWKH,QWHUQHWRI 2SHUDWLRQ6XSSRUW6XEV\VWHP 266 266LVLQFKDUJH
7KLQJV ,R7  ,Q ,R7 DSSOLFDWLRQV GHYLFHV W\SLFDOO\ RI WKH PDQDJHPHQW DQG PRQLWRULQJ RI WKH HQWLUH *60
FRPPXQLFDWHRYHUDZLGHDUHDQHWZRUN :$1 1HWZRUNV V\VWHP ,W FRQVLVWV RI WKH 2SHUDWLRQ 0DLQWHQDQFH &HQWHU
XWLOL]LQJ /R5D WHFKQRORJ\ DUH NQRZQ IRU WKHLU 20& DQGV\VWHPVRIWZDUH
FKDUDFWHULVWLFV RI KDQGOLQJ VPDOO GDWD SD\ORDGV DQG 7KLVV\VWHPDUFKLWHFWXUHIRUPVWKHIRXQGDWLRQRI*60
VLPXOWDQHRXVO\ FRQVXPLQJ ORZ DPRXQWV RI SRZHU >@ WHFKQRORJ\HQDEOLQJPRELOHFRPPXQLFDWLRQRYHUFHOOXODU
>@ QHWZRUNV>@
7KH IXQGDPHQWDO SULQFLSOH EHKLQG /R5D WHFKQRORJ\ LV
VSUHDG VSHFWUXP PRGXODWLRQ ZKLFK VKDUHV VRPH & $QDO\VLVRIH[LVWLQJVROXWLRQV
VLPLODULWLHV ZLWK )UHTXHQF\ 6KLIW .H\LQJ )6.  ;LDR\XH+RXDQGWKHLUWHDP>@LQWURGXFHGDWHFKQLTXH
PRGXODWLRQ +RZHYHU WKH SULPDU\ GLVWLQFWLRQ OLHV LQ LWV WKDWXWLOL]HV0&/ 0RQWH&DUOR/RFDOL]DWLRQ HPSOR\LQJ
H[WHQGHG FRPPXQLFDWLRQ UDQJH 6SUHDG VSHFWUXP WZRVHQVRUVVSHFLILFDOO\DQDFFHOHURPHWHUDQGDFRPSDVV
PRGXODWLRQWHFKQRORJ\KDVEHHQLQXVHIRUVHYHUDOGHFDGHV LQFRQMXQFWLRQZLWKFRPPRQO\GHSOR\HG%/(EHDFRQVIRU
LQLWLDOO\ JDLQLQJSURPLQHQFH LQ PLOLWDU\ DSSOLFDWLRQV7KH LQGRRUORFDOL]DWLRQRILQGLYLGXDOVZLWKPRELOHGHYLFHV7ZR
SULPDU\GULYHUVIRULWVDGRSWLRQLQWKHVHFRQWH[WVZHUHLWV LQGRRU H[SHULPHQWV ZHUH FRQGXFWHG HQFRPSDVVLQJ /LQH
KLJKUHVLVWDQFHWRLQWHUIHUHQFHDQGLWVFDSDELOLW\WRSURYLGH RI6LJKW /26  DQG QRQ/LQHRI6LJKW QRQ/26 
H[WHQGHGUDQJHFRYHUDJHDFURVVYDVWDUHDV>@>@>@ VFHQDULRVWRFRPSDUHWKHSHUIRUPDQFHRI0&/ZLWKRWKHU
:KHQ LW FRPHV WR /R5D WHFKQRORJ\ WKHUH DUH WZR LQGRRUSRVLWLRQLQJDOJRULWKPV
SULPDU\RSWLRQVWRFKRRVHIURP ,Q D UHVHDUFK SDSHU DXWKRUHG E\ <XQZRR /HH DQG
FROODERUDWRUV>@ WKH\ LQWURGXFHG DQ HQHUJ\HIILFLHQW

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 240

DGDSWLYH ORFDWLRQ PLGGOHZDUH WKDW FRPELQHV *36 DQG ZLUHOHVV GDWD WUDQVPLVVLRQ FRQFHUQLQJ WKH PHDVXUHG GDWD
HPEHGGHG VHQVRUV LQFOXGLQJ DFFHOHURPHWHUV DQG SRLQWV
JHRPDJQHWLF VHQVRUV IRU VPDUW PRELOH GHYLFHV ,Q SUDFWLFDO WHUPV WKHUH H[LVW PXOWLSOH *166 V\VWHPV
6PDUWSKRQHVRIIHUQHWZRUNRU*36EDVHGORFDWLRQVHUYLFHV VXFKDV*36*DOLOHR*/21$66%HLGRXDQGRWKHUV,Q
IRUYDULRXVDSSOLFDWLRQV1HWZRUNEDVHGORFDOL]DWLRQZKLOH RXUUHVHDUFKZHFRQFHQWUDWHGRQGLVFHUQLQJWKHGLVSDULWLHV
QRWKLJKO\DFFXUDWHFRQVXPHVPLQLPDOSRZHU,QFRQWUDVW EHWZHHQ *36 DQG *DOLOHR )RU WKH SURSRVHG ORFDOL]DWLRQ
*36EDVHG ORFDOL]DWLRQ SURYLGHV KLJK DFFXUDF\ EXW LV GHYLFH ZH KDYH RSWHG WR XWLOL]H WKH *166 *36 V\VWHP
OLPLWHGWRRXWGRRUHQYLURQPHQWVDQGFRQVXPHVVLJQLILFDQW 7KLV FKRLFH SULPDULO\ VWHPV IURP WKH DYDLODELOLW\ RI
SRZHU/RFDWLRQEDVHGVHUYLFHVRIWHQSULRULWL]H*36IRULWV QXPHURXVVHQVRUVWKDWVXSSRUWWKLVWHFKQRORJ\DQGDUHFRVW
DFFXUDF\EXWSURORQJHG*36XVDJHFDQTXLFNO\GHSOHWHD HIIHFWLYH
VPDUWSKRQH VEDWWHU\FDSDFLW\7KHDXWKRUV DSSURDFKDLPV
2XU UHVHDUFK HQFRPSDVVHG DQ DVVHVVPHQW RI YDULRXV
WR PLWLJDWH WKLV LVVXH E\ SDUWLDOO\ UHSODFLQJ *36EDVHG
ZLUHOHVVWUDQVPLVVLRQWHFKQRORJLHVLQFOXGLQJ:L)L*60
SRVLWLRQLQJZLWKDQHQHUJ\HIILFLHQWVHQVRUEDVHGPHWKRG /R5DDQG%OXHWRRWK$PRQJWKHVHRSWLRQVZHLGHQWLILHG
*LYHQ WKH WUDGHRII EHWZHHQ DFFXUDF\ DQG SRZHU
/R5D DQG *60 WHFKQRORJLHV DV VXLWDEOH FDQGLGDWHV :H
HIILFLHQF\WKHLUV\VWHPUHGXFHVSRZHUFRQVXPSWLRQZKLOH
XOWLPDWHO\VHOHFWHGERWKRIWKHVHWHFKQRORJLHVUHVXOWLQJLQ
PDLQWDLQLQJ WKH UHTXLUHG OHYHO RI DFFXUDF\ IRU WKH
UHGXQGDQW GDWD WUDQVPLVVLRQ WR HQVXUH D KLJK OHYHO RI
DSSOLFDWLRQVLQXVH DYDLODELOLW\/R5DZDVFKRVHQDVWKHSULPDU\WUDQVPLVVLRQ
0DUFR 6WUDQQHU DQG WKHLU WHDP>@ KDYH GHYHORSHG D SURWRFROEHFDXVHLWUHSUHVHQWVDQRQWUDGLWLRQDOWHFKQRORJ\
VHQVRU FXEH FDSDEOH RI GHOLYHULQJ DFFXUDWH DQG UHOLDEOH IRU WKLV SDUWLFXODU DSSOLFDWLRQ SRWHQWLDOO\ HQKDQFLQJ WKH
'2) 6L[ 'HJUHHV RI )UHHGRP  ORFDOL]DWLRQ 7KHLU VHFXULW\ RI GDWD WUDQVPLVVLRQ $GGLWLRQDOO\ /R5D ERDVWV
UHVHDUFKGHPRQVWUDWHGWKDWWKHVHFRVWHIIHFWLYHVHQVRUVFDQ H[WHQVLYHFRYHUDJHVSDQQLQJVHYHUDONLORPHWHUVZKLFKLV
FRPSHWH ZLWK WKH PRUH H[SHQVLYH KLJKSUHFLVLRQ VHWXSV LGHDO IRU HVWDEOLVKLQJ D SULYDWH QHWZRUN RI ORFDOL]DWLRQ
XVHGLQSUHYLRXVPHWKRGV1HYHUWKHOHVVUHO\LQJVROHO\RQ GHYLFHV ,Q FDVHV ZKHUH D ORFDOL]DWLRQ GHYLFH YHQWXUHV
VHQVRUEDVHG WUDFNLQJ PD\ QRW DOZD\V HQVXUH UREXVWQHVV EH\RQGWKHUHDFKRIWKH/R5DQHWZRUNFRPPXQLFDWLRQZLOO
3DUWLFXODUO\ LQ XUEDQ VHWWLQJV *36 DFFXUDF\ FDQ EH EH LQLWLDWHG WKURXJK WKH *60 SURWRFRO HYHQ WKRXJK LWV
FRPSURPLVHGE\VDWHOOLWHVKDGRZLQJDQGUHIOHFWLRQVIURP FRYHUDJHLVOLPLWHGHVSHFLDOO\LQOHVVGHYHORSHGDUHDVGXH
EXLOGLQJV6LPLODUO\WKHPDJQHWRPHWHU VSHUIRUPDQFHFDQ WRUHOLDQFHRQPRELOHQHWZRUNV
EH LQIOXHQFHG E\ QHDUE\ PHWDOOLF REMHFWV OLNH FDUV 7R
,QWKHLPPHGLDWHIXWXUHRXULQWHQWLRQLVWRSURFHHGZLWK
DGGUHVV WKHVH FKDOOHQJHV WKH\ KDYH GHYLVHG D K\EULG
WKH GHVLJQ DQG GHYHORSPHQW RI D ORFDOL]DWLRQ GHYLFH WKDW
WUDFNLQJV\VWHPWKDWFRPELQHVWKHVWUHQJWKVRIYLVLRQDQG ZLOO LQFRUSRUDWH WKH DIRUHPHQWLRQHG WHFKQRORJLHV 7KLV
WUDFNLQJ VHQVRUV 7KLV K\EULG DSSURDFK DOUHDG\ RIIHUV GHYLFHZLOOEHHPSOR\HGIRUWUDFNLQJDQGLGHQWLI\LQJQHZ
YDOXDEOHHQKDQFHPHQWV WR WKH VHQVRU VHWXS+RZHYHU LW V
LQGLYLGXDOVZLWKLQSURWHFWHGDQLPDOVSHFLHV
LPSRUWDQW WR QRWH WKDW WKH FXUUHQW 6/$0 6LPXOWDQHRXV
/RFDOL]DWLRQ DQG 0DSSLQJ  FRQILJXUDWLRQ LV VWLOO LQ ,9 $&.12:/('*0(17
GHYHORSPHQWDQGQRW\HWFDSDEOHRIUHDOWLPHRSHUDWLRQRQ
PRELOHGHYLFHV3URWRW\SHRI/RFDOL]DWLRQGHYLFHE\0DUFR 7KLV SDSHU LV VXSSRUWHG E\ SURMHFWV .(*$ =8
6WUDQQHU\RXFRXOGVHHLQ)LJXUH  ³7HFKQRORJLHV RI SULYDWH FORXG HQYLURQPHQWV LQ
XQLYHUVLW\HGXFDWLRQ´

5()(5(1&(6

>@ 6 /L HW DO ³5HFRQILJXUDEOH DOOEDQG 5) &026 WUDQVFHLYHU IRU
*36*/21$66JDOLOHREHLGRXZLWKGLJLWDOO\DVVLVWHGFDOLEUDWLRQ´
,(((7UDQV9HU\/DUJH6FDOH,QWHJU9/6,6\VWYROQRSS
±6HSGRL79/6,
>@ 60DKDWR0*RVZDPL6.XQGXDQG$%RVH³*36*/21$66
FRPELQHG SRVLWLRQ SHUIRUPDQFH DFFXUDF\ IRU ,QGLDQ GHIHQFH
DSSOLFDWLRQ´QG,QWHUQDWLRQDO&RQIHUHQFHRQ5DQJH7HFKQRORJ\
,&257  $XJ  GRL
,&257
 >@ =/XR4=KDQJ::DQJDQG7-LDQJ³6LQJOH$QWHQQD'HYLFH
)LJXUH3URWRW\SHRIFLUFXLWIRUORFDOL]DWLRQGHYLFHE\0DUFR WR'HYLFH/RFDOL]DWLRQLQ6PDUW(QYLURQPHQWV:LWK%DFNVFDWWHU´
6WUDQQHUDQGWKHLUWHDP>@ ,(((,QWHUQHW7KLQJV-YROQRSS±-XQ
 GRL-,27
>@ ³'HVLJQ VPDUW DQWHQQD IRU *36*/21$66 XVLQJ DGDSWLYH
,,, &21&/86,21 EHDPIRUPLQJ _ ,((( &RQIHUHQFH 3XEOLFDWLRQ _ ,((( ;SORUH´
$FFHVVHG 6HS   >2QOLQH@ $YDLODEOH
,QWKLVUHVHDUFKSDSHUZHKDYHFRQGXFWHGDQH[WHQVLYH KWWSVLHHH[SORUHLHHHRUJGRFXPHQW
DQDO\VLVDQGLQYHVWLJDWLRQLQWRYDULRXVPHWKRGRORJLHVDQG >@ ' .ZDĞQLDN 6 &HOOPHU DQG . 1RZHO ³3UHFLVH SRVLWLRQLQJ LQ
WHFKQRORJLHV UHTXLUHG IRU WKH GHYHORSPHQW RI D WUDFNLQJ (XURSHXVLQJWKH*DOLOHRDQG*36FRPELQDWLRQ´SS±
GHYLFH 2XU SULPDU\ IRFXV LQ UHVHDUFKLQJ WKH WHFKQRORJ\ GRLHQYLUR
QHFHVVDU\ WR FRQVWUXFW D ORFDOL]DWLRQ GHYLFH ZDV RQ WKH >@ -3D]LHZVNLDQG3:LHOJRV]³$VVHVVPHQWRI*36*DOLOHRDQG
*166 V\VWHP ZKLFK ZLOO EH HPSOR\HG WR SLQSRLQW WKH PXOWLIUHTXHQF\ *DOLOHR VLQJOHHSRFK SUHFLVH SRVLWLRQLQJ ZLWK
WUDFNHG ORFDWLRQ $GGLWLRQDOO\DV DQ HVVHQWLDO FRPSRQHQW QHWZRUNFRUUHFWLRQV´*366ROXWLRQVYROQRSS±
2FWGRL6),*85(6
RI WKH GHYLFH V FRQVWUXFWLRQ DQG GHVLJQ ZH H[SORUHG
>@ '.ZDĞQLDN6&HOOPHUDQG.1RZHO³3UHFLVHSRVLWLRQLQJXVLQJ
WKHPRGLILHGDPELJXLW\IXQFWLRQDSSURDFKZLWKFRPELQDWLRQRI*36

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 241

 DQG *DOLOHR REVHUYDLRQV´ WK 6DLQW 3HWHUVEXUJ ,QWHUQDWLRQDO DQG (QWUHSUHQHXUVKLS ,&:,7(   3URFHHGLQJV  GRL
&RQIHUHQFH RQ ,QWHJUDWHG 1DYLJDWLRQ 6\VWHPV ,&,16   ,&:,7(
3URFHHGLQJV SS ± -XO  GRL >@< /DOOH 0 )RXUDWL / & )RXUDWL DQG - 3 %DUUDFD ³5RXWLQJ
,&,16 6WUDWHJLHVIRU /R5D:$10XOWL+RS1HWZRUNV$6XUYH\DQGDQ
>@ $KPHG(O5DEEDQ\³,QWURGXFWLRQWR*36ௗWKH*OREDO3RVLWLRQLQJ 6'1%DVHG6ROXWLRQIRU6PDUW:DWHU*ULG´,((($FFHVVYRO
6\VWHP´S SS±GRL$&&(66
>@ ³$NRIXQJXMH*36 *OREDOSRVLWLRQLQJV\VWHP "´ $FFHVVHG6HS >@: &KDQZDWWDQDSRQJ 6 +RQJGXPQXHQ % .XPNKHW 6 -XQRQ
  >2QOLQH@ $YDLODEOH KWWSVLQYLYRPDJD]LQVNDNR DQG 3 6DQJPDKDPDG ³/R5D 1HWZRUN %DVHG 0XOWL:LUHOHVV
IXQJXMHJSV JOREDOSRVLWLRQLQJV\VWHP BKWPO 6HQVRU 1RGHV DQG /R5D *DWHZD\ IRU $JULFXOWXUH $SSOLFDWLRQ´
>@6-DNRYOMHY06XERWLFDQG,3DSS³5HDOLVDWLRQRID6PDUW3OXJ 3URFHHGLQJV5HVHDUFK,QYHQWLRQDQG,QQRYDWLRQ&RQJUHVV
GHYLFH EDVHG RQ :L)L WHFKQRORJ\ IRU XVH LQ KRPH DXWRPDWLRQ ,QQRYDWLRQ(OHFWULFDOV DQG (OHFWURQLFV 5,&SS±
V\VWHPV´  ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ &RQVXPHU 6HSGRL5,&
(OHFWURQLFV ,&&(  SS ± 0DU  GRL >@<&&KDQJ7:+XDQJDQG1)+XDQJ³$0DFKLQH/HDUQLQJ
,&&( %DVHG 6PDUW ,UULJDWLRQ 6\VWHP ZLWK /R5D 33 1HWZRUNV´ 
>@6-'DQEDWWDDQG$9DURO³&RPSDULVRQRI=LJEHH=:DYH:L WK $VLD3DFLILF 1HWZRUN 2SHUDWLRQV DQG 0DQDJHPHQW
)LDQGEOXHWRRWKZLUHOHVVWHFKQRORJLHVXVHGLQKRPHDXWRPDWLRQ´ 6\PSRVLXP 0DQDJHPHQW LQ D &\EHU3K\VLFDO :RUOG $31206
WK ,QWHUQDWLRQDO 6\PSRVLXP RQ 'LJLWDO )RUHQVLFV DQG 6HFXULW\ 6HSGRL$31206
,6')6-XQGRL,6')6 >@596HQ\XYD³&RPSDULVRQRI/R5D%DVHG0RGXODWLRQV´
>@2 +RU\DFK\\ ³&RPSDULVRQ RI :LUHOHVV &RPPXQLFDWLRQ WK 6LJQDO 3URFHVVLQJ DQG &RPPXQLFDWLRQV $SSOLFDWLRQV
7HFKQRORJLHVXVHGLQD6PDUW+RPHௗ$QDO\VLVRIZLUHOHVVVHQVRU &RQIHUHQFH 6,8   GRL
QRGH EDVHG RQ $UGXLQR LQ KRPH DXWRPDWLRQ VFHQDULR´  6,8
$FFHVVHG 6HS   >2QOLQH@ $YDLODEOH >@' - & 0DFND\ ³,QIRUPDWLRQ 7KHRU\ ,QIHUHQFH DQG 1HXUDO
KWWSVXUQNEVHUHVROYH"XUQ XUQQEQVHEWK 1HWZRUNV´S
>@6 'DV 6 *DQJXO\ 6 *KRVK 5 6DUNHU DQG ' 6HQJXSWD ³$ >@**XDQG*3HQJ³7KHVXUYH\RI*60ZLUHOHVVFRPPXQLFDWLRQ
EOXHWRRWK EDVHG VRSKLVWLFDWHG KRPH DXWRPDWLRQ V\VWHP XVLQJ V\VWHP´ 3URFHHGLQJV RI ,&&,$    ,QWHUQDWLRQDO
VPDUWSKRQH´  ,QWHUQDWLRQDO &RQIHUHQFH RQ ,QWHOOLJHQW &RQIHUHQFH RQ &RPSXWHU DQG ,QIRUPDWLRQ $SSOLFDWLRQ SS ±
&RQWURO 3RZHU DQG ,QVWUXPHQWDWLRQ ,&,&3, SS± GRL,&&,$
)HEGRL,&,&3, >@; +RX DQG 7 $UVODQ ³0RQWH &DUOR ORFDOL]DWLRQ DOJRULWKP IRU
>@66LQJKDQG..XPDU5D\³+20($8720$7,216<67(0 LQGRRU SRVLWLRQLQJ XVLQJ %OXHWRRWK ORZ HQHUJ\ GHYLFHV´ 
86,1* ,17(51(7 2) 7+,1*6´ $FFHVVHG 6HS   ,QWHUQDWLRQDO &RQIHUHQFH RQ /RFDOL]DWLRQ DQG *166 ,&/*166
>2QOLQH@$YDLODEOHZZZLMFHDFRP SS±-XQGRL,&/*166
>@. 6DQDGD + +DWDQR DQG . 0RUL ³6LPSOH 5HLQIRUFHPHQW >@</HH-/HH'6.LPDQG+&KRR³(QHUJ\HIILFLHQWDGDSWLYH
/HDUQLQJEDVHG&RQWHQWLRQ:LQGRZV$GMXVWPHQWIRU,((( ORFDOL]DWLRQPLGGOHZDUHEDVHGRQ*36DQGHPEHGGHGVHQVRUVIRU
1HWZRUNV´ 3URFHHGLQJV  ,((( &RQVXPHU &RPPXQLFDWLRQV DQG VPDUW PRELOHV´ ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ &RQVXPHU
1HWZRUNLQJ &RQIHUHQFH &&1& YRO -DQXDU\ SS ± (OHFWURQLFV  %HUOLQ ,&&(%HUOLQ YRO )HEUXDU\ QR
GRL&&1& )HEUXDU\ SS ± )HE  GRL ,&&(
>@+6$JKGDVL17RUDEL$5DKPDQ]DGHK0$PLQLD]DUDQG0 %(5/,1
$EEDVSRXU³8VHIXOQHVVRIPXOWLFDVWURXWLQJSURWRFROVIRUYHKLFXODU >@0 6WUDQQHU & $UWK ' 6FKPDOVWLHJ DQG 3 )OHFN ³$ KLJK
$GKRF QHWZRUNV´  WK ,QWHUQDWLRQDO 6\PSRVLXP RQ SUHFLVLRQ ORFDOL]DWLRQ GHYLFH IRU RXWGRRU DXJPHQWHG UHDOLW\´
7HOHFRPPXQLFDWLRQV ,67  SS ±  GRL $GMXQFW3URFHHGLQJVRIWKH,(((,QWHUQDWLRQDO6\PSRVLXPRQ
,67(/ 0L[HG DQG $XJPHQWHG 5HDOLW\ ,60$5$GMXQFW  SS ±
>@5.9DNDDQG$56KDQNDU³,PSOHPHQWDWLRQDQG$QDO\VLVRI 2FWGRL,60$5$'-81&7
:L)L1HWZRUNVOLFHVEDVHGRQ*1HWZRUN6OLFLQJ´,((( 
,QWHUQDWLRQDO &RQIHUHQFH IRU :RPHQ LQ ,QQRYDWLRQ 7HFKQRORJ\




979-8-3503-7069-0/23/$31.00 ©2023 IEEE 242

 Diagnostics of affective components of digital
competences in elementary school pupils – a pilot
study
Tomas Javorcik* and Tatiana Havlaskova*
*University of Ostrava, Faculty of Education / Department of Information and Communication Technologies, Ostrava,
Czech Republic
tomas.javorcik@osu.cz, tatiana.havlaskova@osu.cz

Abstract – This paper describes the results of a pilot study information and education. The digitally literate thus
focused on measuring the affective components of individual benefit from new technologies more than others and the
digital competences defined by the DigComp digital literacy gap between them is growing [3]. The level of digital
concept. Even if pupils are gradually developing digital literacy is influenced by the socio-demographic
literacy in elementary schools and are introduced to the characteristics of a person including age, education, socio-
potential of digital technologies and their possibilities of use, economic status and geographical factors (urban or rural
they may still lack some of the important affective areas) [4]. In the international context, digital literacy is
components such as motivation, desire and willingness to currently mainly mentioned in the context of the global
engage (appropriately) with digital technologies in their COVID-19 pandemic. The results of such targeted studies
professional and private lives. A prerequisite for valid show significant differences in digital literacy between
measurement is a well-designed diagnostic tool including the teachers [5] and their pupils, but also in the general
identification and assessment of emotional and motivational population [6]. The definition of digital literacy has
factors that are directly related to the development of digital undergone significant changes since it was first mentioned
competences. The article describes a pilot study with the in 1997 [7], [8], reflecting the evolution and availability of
objective of designing a diagnostic tool to measure the technology. We can consider the following definition as
affective components of digital literacy. The pilot study was up-to-date: “Digital literacy is the ability to access,
conducted on a sample of second stage elementary school manage, understand, integrate, communicate, evaluate and
pupils. create information safely and appropriately through digital
technologies for employment, decent jobs and
I. INTRODUCTION entrepreneurship. It includes competences that are
variously referred to as computer literacy, ICT literacy,
The inevitable process of digitalisation, which is information literacy and media literacy” [9]. The Global
beginning to affect most fields of human activity, places Standard on Digital Literacy, Digital Skills and Digital
new demands on individuals in the field of education and Readiness [10], which introduces the concept of Digital
learning. As the importance of digital technologies in Intelligence (DQ), takes a somewhat different view of this
society will continue to grow, digital skills must be seen issue. This is defined as a comprehensive set of technical,
as one of the essential components of a person’s cognitive, meta-cognitive and socio-emotional
functional literacy. Despite the constant growth of the competences that are grounded in universal moral values
share of the population actively using digital technologies, and that enable individuals to face the challenges and
the Czech Republic is among the countries with an harness the opportunities of digital life. Reference [11]
average or below-average percentage according to the understands digital literacy as the ability to successfully
Digital Economy and Society Index (DESI) [1]. Digital perform digital activities (the ability to work effectively
literacy is a key concept currently resonating across with digital technologies) in a variety of life situations,
different sectors. A digitally literate person has a set of which may include work, learning, leisure and other
competences universally applicable in the digital world. aspects of everyday life. Reference [12] states that digital
These competences are not only the result of learning, literacy involves more than just knowledge of using
acquiring knowledge and skills, but are also the potential devices and applications. The sensible and healthy use of
for further personal development. The objective of today’s ICT requires specific knowledge and attitudes regarding
society is to develop digital literacy across all age and legal and ethical aspects, privacy and security as well as
social groups, as a new social divide can emerge based on an understanding of the role of ICT in society and a
the level of digital literacy. The so-called digital exclusion balanced approach to technology.
partly replicates the previous line of inequalities (in
income, assets, opportunities, employment, etc.) and
might deepen them. Given the dominant role of the
Internet, it is expected to have a greater impact than the
telephone, television and computers. Moreover, the digital
divide is reproduced and the so-called second-degree
digital divide can occur [2]. This means that people with a
lack of digital literacy find it difficult to access

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 243

 of psychodiagnostic methods called projective methods,
where the subject responds according to their needs,
motives, perceptions, attitudes and other personality traits.
The use of projective methods in pedagogical research has
been addressed, for example, by authors Catterall and
Ibbotson [17]. They also mention the advantages of these
kind of research methods.
Versatility – a projective test can be designed to be
directly tailored to a specific research project. In most
cases the test is composed of multiple methods and is also
often combined with another qualitative or quantitative
method.
Comprehensiveness – the test records feelings,
Figure 1. The concept of digital literacy – in the middle of core perceptions and attitudes, among other things. In other
competences, these are reinforced by blocks on the sides (blue and research methods the determination of these personality
green). The purple and grey blocks represent “supporting” competences traits is more complex (a larger number of questions, etc.).
[12].
User-friendliness – the respondent finds the projective
test more pleasant than filling out lengthy questionnaires
The individual competence areas can then be further
and scales. It allows respondents to answer regardless of
divided into sub-competences. Considering the general
the frame of reference.
definition of competence as a set of knowledge, skills,
abilities, attitudes and values, we can state that each In practice, one can encounter a large number of
competence consists of a cognitive (knowledge) and projective methods, which are usually divided into three
affective (attitude) component [13]. groups – verbal, graphical and choice methods (also called
manipulation methods) [18]. In some cases most
From the perspective of evaluating the effectiveness of
projective techniques allow respondents some freedom
the mentioned measures related to digital literacy and the
and a large number of responses [19].
possibility of identifying possible shortcoming in its
development in different target groups and comparing For each test tool there arises the question of
them with each other, it is convenient to summarise the objectivity, reliability and validity. Psychological test
level of digital competence. A large number of tools have procedures may be questioned when it comes to their
been used in studies to express the level of digital literacy, objectivity. For example, questionnaires, so often used in
whether standardised (e.g. ECDL/ICDL) or non- psychology, are based on subjective responses. In
standardised [14] and [15]. However, most of these tools projective techniques the outcome is sensitive to the
emphasise the measurement of skills and attitudes as an interpretive skills of the administrator or the quality of the
important aspect of competence remain unnoticed relationship between client and psychologist. Thus,
[16](…). Therefore, if we want to measure digital literacy complete objectivity is never fully achieved. The problem
objectively, we need to focus on its affective component primarily tends to be a lack of objectivity in scoring. The
in addition to the cognitive dimension. In this case we final steps in scoring raw data depend on the skill and
primarily mean the degree of an individual’s motivation to experience of the psychologist. With projective techniques
use digital technology, the individual’s attitude towards there is the problem of inconsistency in scoring the results,
this kind of technology, etc. As a result of the i.e., the reliability of the evaluator. In practice we may
insufficiently acquired affective component of digital encounter variability in the conclusions of different
literacy, the individual will not use the knowledge and clinicians quite often. However, for most of the basic
skills in practice. We are convinced that for the needs of projective methods, reliability is confirmed. For the
further development of education and precise targeting of validity of projective tests, the interpretive factor, i.e., the
the pupil with regard to their specific level of digital person making the conclusions, again becomes extremely
literacy, it is necessary to know their current level as a important. The outcome is highly dependent on how the
starting point for its further development and also for the psychologist structures the material acquired via
needs of comparison of its progress. projective techniques. It depends on the psychologist’s
readiness, their system of concepts, ability to handle them,
The aim of our research was to design a pilot test and
solution procedures, etc.
tool for diagnosing the affective components of digital
literacy sub-competences in elementary school pupils. The Given the focus of the research, two verbal methods
research will enable us to answer the following questions: were chosen – the incomplete sentences test and the word
association experiment. Another criterion for selection
a) Do older pupils show better results in digital was the accuracy of these methods, which for the
literacy than younger pupils? incomplete sentences test reaches split-half reliability
b) Which of the digital literacy sub-components are r=0.73 to r=0.98 and the concurrence of multiple
most developed in pupils? judgements tends to be 92% [20], [21]. Based on the two
mentioned methods, a model of a diagnostic tool for
c) Are there differences between girls and boys in detecting the level of the affective component of digital
the level of digital literacy? literacy will be developed.
In this research a projective test consisting of two parts
II. METHODS – incomplete sentences test and word association
In developing a tool to measure the affective experiment – was designed. The first part of the test
components of digital literacy, we want to use the findings consisted of 29 incomplete sentences, which corresponded

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 244

in their assignment to each sub-competency of digital points in security, and an average score of 1 point in
literacy according to the DigComp 2.1 model [22]. Each problem solving.
sub-competency was thus captured by an average of 5 to 6 Older pupils had an average score of 4.7 points in
incomplete sentences. The second part of the test was information and data literacy, 4.1 points in communication
based on a word association experiment where 8 words or and cooperation, -1.2 points in digital content creation, 8
phrases were chosen. Pupils could react to each given points in security and 0.8 points in problem solving.
word or phrase for 1 minute. Each response in both parts
of the test was given a score from a scale of -2 to +2
points according to the degree of correspondence with the
content of the specific digital literacy sub-competency.
The sum of the points thus awarded determined the total
score achieved.
A. Research sample
A total of 38 pupils participated in the pilot study. The
research sample was purposefully focused on pupils in the
6th and 9th grade of elementary school, so that it was
possible to compare whether there is any difference in the
development of digital literacy in the individual grades.
The test to measure the affective component of digital
literacy was anonymous. Pupils were required to indicate
the grade they attend and their gender. Figure 2. Comparison of incomplete sentences test results
Care was taken during testing to ensure that pupils were In the second part of the test with the word association
not distracted, that they followed the test instructions and experiment method, 9th grade pupils performed better than
that testing was conducted in a classroom with good 6th graders, despite the smaller number of respondents.
acoustics to avoid potential distractions from ambient Especially in the area of digital content creation and
sounds. security, where the differences are most noticeable.
The respondents were pupils of the Dr. E. Beneš 1 Younger pupils on average scored 3.9 points in
Elementary School in Šumperk. The measurement of the information and data literacy, 4.6 points in communication
affective component level of digital literacy took place and cooperation, -6.4 points in digital content creation, 2
from 13 March 2023 to 14 March 2023. points in security and an average of 1 point in problem
solving.
III. RESULTS
Older pupils scored an average of 8.5 points in
A total of 23 sixth grade pupils and 15 ninth grade information and data literacy, 6.8 points in communication
pupils participated in the pilot testing of the sample test. and cooperation, −2.2 points in digital content creation,
The PAST 4.03 tool was used for statistical data 8.1 points in security and an average of 1.9 points in
processing. problem solving.

TABLE I.
BASIC CHARACTERISTICS OF THE OBTAINED DATA

Score
Girls Boys
Min. Max. Avg. Med.
A* 12 11 -42 46 19.65 24
(n=23) (52.17%) (47.83%)
B** 6 9 -10 113 39.53 39
(n=15) (40%) (60%)
Sum 18 20 -42 113 29.59 30
(n=38) (47.37%) (52.63%)
*) Younger pupils – 6th grade of elementary school
**) Older pupils – 9th grade of elementary school

A. Difference in results between older and younger

pupils
In the first part of the test, namely the incomplete Figure 3. Comparison of the results of the word association
sentences method, it can be seen that the pupils of the 6th experiment
grade on average scored better than the pupils of the 9th
grade. This may partially be attributable to the number of There is a statistically significant difference between
respondents in each grade, as the responses of 23 younger and older pupils. Calculated p-value = 0.0015 at
respondents in the 6th grade and 15 respondents in the 9th a set level of α=0.05. A t-test was used to calculate the p-
grade are included, i.e., 8 fewer. value.
On average, younger pupils scored 2.2 points in The graph shows that the 9th grade pupils performed
information and data literacy, 4.5 points in communication better than 6th grade pupils.
and cooperation, 0.4 points in digital content creation, 6.6

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 245

 29.3 written words. Boys had an average of 8.2 points and
an average of 19.9 written words.

Figure 4. Comparison of the overall results of younger and older

pupils
Figure 6. Gender comparison of differences in digital literacy levels
among younger pupils.
B. Most developed areas of digital literacy
A comparison of the results of the two pupil groups in The difference was statistically verified for 9th grade
the different areas of digital literacy is illustrated in Figure pupils and came out as statistically significant with p-
5. The 6th grade pupils showed the best results in the area value = 0.0156 at the specified level of α=0.05.
of communication and cooperation, while the 9th grade In the first part of the test, i.e., the incomplete
pupils showed the best results in the area of security. sentences method, 9th grade girls scored an average of 10
From the results it can be seen that pupils are either points. Boys showed an average of 20.6 points.
unaware of the content they create through digital The lowest score the girls got in the incomplete
technologies or are more likely to be consumers of digital sentences test was in the problem solving area (-5 points).
content. The girls scored the most points in the area of security (38
points in total).
Boys were least successful in the area of digital
content creation (-15 points in total). The highest number
of points achieved by boys was in the area of security (82
points in total).
In the second part of the test, the word
association experiment, girls had an average of 11 points
and 28.7 written words. Boys had an average of 31.3
points and an average of 29.2 written words.

Figure 5. Comparison of the overall results of younger and older

pupils by each area of digital literacy

C. Gender differences in overall scores

Differences between boys and girls are also
differentiated by grade, due to differences in age. For 6th
grade pupils no statically significant difference in digital
literacy levels between girls and boys was detected.
According to the Mann-Whitney test, the p-value =
0.9754 at the set level of α=0.05. Figure 7. Gender comparison of differences in digital literacy levels
In the test of incomplete sentences, the 6th grade among older pupils.
female pupils had an average of 15.2 points per pupil.
Boys had an average of 14.1 points.
In the test of incomplete sentences, 6th grade girls IV. SUMMARY OF RESULTS
scored the lowest in problem solving (7 points in total) The pilot study described above provided the
and the highest in security (95 points). opportunity to answer the following research questions:
The 6th grade boys scored the lowest in the a) Do older pupils show better results in digital
incomplete sentences test in the area of digital content literacy than younger pupils?
creation (-2 points overall), and the highest in the area of
security (56 points overall). b) Which of the digital literacy sub-components are
In the second part, the word association most developed in pupils?
experiment, the 6th grade girls averaged 2.1 points and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 246

 c) Are there differences between girls and boys in digital literacy, our proposed test identified two
the level of digital literacy? competences in which pupils of both groups performed
First research question: “Do older pupils show better significantly worse than in other competences. These were
results in digital literacy than younger pupils?” was the content creation and problem solving competences.
answered as follows. Yes, older pupils perform much Our findings are supported by other studies focusing on
better than younger pupils. These results have been digital literacy sub-competences [30]. Both groups
statistically verified using the Past Statistics tool. There is performed best when responding to items related to the
a statistically significant difference between the compared security competency. This can be attributed to the quality
groups. In terms of the individual parts of the test, the of preventive action by the school and other institutions
younger pupils were more successful in the incomplete dealing with this issue.
sentences test. Older pupils did better in the second part of What posed a limitation of our research was mainly the
the test – word associations. small size of the research sample. After some adjustments
While dealing with the second research question to the test, we intend to apply the test to a larger research
concerning the level of acquisition of the sub-areas of sample.
digital literacy, we concluded that the most developed area
of digital literacy for both groups is the area of Security. VI. CONCLUSION
This is not surprising given the number of preventive It is necessary to look at the issue of developing digital
actions related to the safe use of digital technologies by competences in a comprehensive way, to deal with the
children. The second most developed area was development of all its dimensions, to not only strengthen
Communication and Cooperation. This is where the the cognitive part but to focus on the affective component
current influence of social media and the tendency of as well. To diagnose the affective components, we
today’s generation to primarily communicate via the decided to use different options of projective methods. For
Internet has become apparent. Based on our results, the the purpose of our research (for the time being it was a
area of content creation can be identified as problematic pilot study) a projective test was created, which consisted
for both groups. of two parts: an incomplete sentences test and a word
In the third research question, we investigated whether association experiment. The projective test was piloted on
there are differences between girls and boys in the a selected sample of 6th and 9th grade elementary school
acquisition of digital literacy. The difference did not turn pupils. However, it is still in the process of being modified
out to be statistically significant for younger pupils. In and fine-tuned, and a revision is already being prepared
contrast, for older pupils, the differences between boys for the needs of other target groups. This diagnostic tool
and girls were statistically significant. Given the small could in the future be used by teachers to assess emotional
size of the research sample, it cannot be clearly factors related to digital competences – identification and
established here whether our proposed tool is able to subsequent intervention. However, we must not forget that
distinguish differences between girls and boys. For this any diagnostic is not a one-off event, but should be
reason it is important to replicate the research on a larger repeated regularly – with the intention of monitoring
research sample. pupils’ ongoing development and providing the necessary
support where appropriate.
V. DISCUSSION
REFERENCES
The aim of this study was to pilot a proposed test aimed
at diagnosing the affective components of digital [1] Digital Economy and Society Index (DESI) 2022: Thematic
chapters. European Commission, 2022.
competence in elementary school pupils. The test was
[2] P. Attewell, “Comment: The First and Second Digital
based on projective techniques – incomplete sentence test Divides”, Sociology of Education, vol. 74, no. 3, pp. 252-259,
and word association experiment. A similar approach 2001.
using psychodiagnostic methods to measure the affective [3] M. Pietrass, “Digital Literacy Research from an International and
dimension of competence has been used by other authors Comparative Point of View”, Research in Comparative and
[23]. International Education, vol. 2, no. 1, pp. 1-12, 2007.
The pilot testing provided us with a number of new [4] P. Lupač, A. Chrobáková, and J. Sládek, Internet v České
findings, which we now interpret in more detail. It is republice 2014. Praha, 2014.
evident from the data that older pupils have developed the [5] S. Tejedor, L. Cervi, A. Pérez-Escoda, and F. T. Jumbo, “Digital
Literacy and Higher Education during COVID-19 Lockdown:
affective components of digital competence better than the Spain, Italy, and Ecuador”, Publications, vol. 8, no. 4, pp. 1-17,
younger pupils. This result suggests that digital literacy 2020.
and relationships with digital technologies develop over [6] A. J. A. M. van Deursen, “Digital Inequality During a Pandemic:
time, and experiences with digital technologies in early Quantitative Study of Differences in COVID-19–Related Internet
and middle adolescence may pose important elements of Uses and Outcomes Among the General Population”, Journal of
this development [24]. Our findings are also confirmed by Medical Internet Research, vol. 22, no. 8, pp. 1-13, 2020.
another study [25]. Focusing on the gender differences in [7] P. Gilster, Digital Literacy. New York: Wiley Computer Pub.,
the results of the pilot test, we find that the difference was 1997.
not significant for younger pupils, however, there was a [8] K. Stopar and T. Bartol, “Digital competences, computer skills
statistically significant difference between girls and boys and information literacy in secondary education: mapping and
visualization of trends and concepts”, Scientometrics, vol. 118, no.
among the older pupils. This result may point to the fact 2, pp. 479-498, 2019.
that gender differences in digital literacy do not occur [9] A Global Framework of Reference on Digital Literacy Skills for
until later in life. Existing studies dealing with this topic Indicator 4.4.2. Montreal: Unesco Institute for Statistics, 2018.
are inconsistent regarding gender differences [26], [27], [10] “IEEE Standard for Digital Intelligence (DQ)--Framework for
[28], [29]. Focusing on the individual competences of Digital Literacy, Skills, and Readiness.” 2021.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 247

[11] A. Martin and , Eds., “Digital Literacy and the “Digital Society””, [23] F. M. Morales-Rodríguez, M. Martí-Vilar, M. A. N. Peláez, J. M.
in Digital Literacies: Concepts, Policies and Practices, 2008, pp. G. Lozano, J. P. Martínez-Ramón, and A. Caracuel,
151-170. “Psychometric Properties of the Affective Dimension of the
[12] J. Janssen, S. Stoyanov, A. Ferrari, Y. Punie, K. Pannekeet, and P. Generic Macro-Competence Assessment Scale: Analysis Using
Sloep, “Experts' views on digital competence: Commonalities and Rasch Model”, Sustainability, vol. 13, no. 12, 2021.
differences”, vol. 68, pp. 473-481, 2013. [24] A. W. Lazonder, A. Walraven, H. Gijlers, and N. Janssen,
[13] K. Ala-Mutka, Mapping Digital Competence:Towards a “Longitudinal assessment of digital literacy in children: Findings
Conceptual Understanding. Luxembourg: European Union, 2011. from a large Dutch single-school study”, vol. 143, 2020.
[14] K. -Y. Jin, F. Reichert, L. P. Cagasan, J. de la Torre, and N. Law, [25] K. -Y. Jin, F. Reichert, L. P. Cagasan, J. de la Torre, and N. Law,
“Measuring digital literacy across three age cohorts: Exploring test “Measuring digital literacy across three age cohorts: Exploring test
dimensionality and performance differences”, vol. 157, 2020. dimensionality and performance differences”, vol. 157, 2020.
[15] Recommendations on Assessment Tools for Monitoring Digital [26] T. M. Link and R. Marz, “Computer literacy and attitudes towards
Literacy within UNESCO’s Digital Literacy Global Framework. e-learning among first year medical students”, BMC Medical
Montreal: UNESCO Institute for Statistics, 2019. Education, vol. 6, no. 1, pp. 1-8, 2006.
[16] E. Godaert, K. Aesaert, J. Voogt, and J. van Braak, “Assessment [27] N. A. M. Zin, H. B. Zaman, H. M. Judi, N. A. Mukti, H. M. Amin,
of students’ digital competences in primary school: a systematic S. Sahran, K. Ahmad, M. Ayob, S. Abdullah, and Z. Abdullah,
review”, Education and Information Technologies, vol. 27, no. 7, “Gender Differences In Computer Literacy Level Among
pp. 9953-10011, 2022. Undergraduate Students In Universiti Kebangsaan Malaysia
(UKM)”, The Electronic Journal of Information Systems in
[17] M. Catterall and P. Ibbotson, “Using Projective Techniques in
Developing Countries, vol. 1, no. 1, pp. 1-8, 2000.
Education Research”, British Educational Research Journal, vol.
26, no. 2, pp. 245-256, 2000. [28] E. C. Maxwell and E. M. Maxwell, “Gender Differences in Digital
[18] M. Svoboda, Psychologická diagnostika dospělých. Praha: Portál, Literacy Among Undergraduate Students of Faculty of Education,
2010. Kogi State University”, Advances in Social Sciences Research
Journal, vol. 1, no. 7, pp. 96-108, Nov. 2014.
[19] S. O. Lilienfeld, J. M. Wood, and H. N. Garb, “The Scientific
Status of Projective Techniques”, Psychological Science in the [29] R. Rizal, D. Rusdiana, W. Setiawan, P. Siahaan, and I. M.
Public Interest, vol. 1, no. 2, pp. 27-66, 2000. Ridwan, “Gender differences in digital literacy among prospective
physics teachers”, Journal of Physics: Conference Series, vol.
[20] B. Murstein, Handbook of Projective Techniques. New York: 1806, no. 1, pp. 1-6, 2021.
Basic Books, 1965.
[30] T. Linde-Valenzuela, F. D. Guillen-Gamez, and A. Devitt,
[21] S. Morávek, Úvod do psychodiagnostiky dospělých, Vyd. 2. “Digital Literacy of Teachers, Families, and Students for Virtual
Olomouc: Rektorát Univerzity Palackého, 1991. Participation in School: A Multiple Comparison Analysis”, IEEE
[22] S. Carretero Gomez, R. Vuorikari, and Y. Punie, DigComp 2.1: Revista Iberoamericana de Tecnologias del Aprendizaje, vol. 17,
The Digital Competence Framework for Citizens with eight no. 1, pp. 1-8, 2022.
proficiency levels and examples of use. Luxembourg: Publications
Office of the European Union, 2017.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 248

 Low-Code Languages in IT Education: Integrating
Theory and Practice
Gabriel Juhás∗ , Ana Juhásovᶠ, Luboš Petrovič∗
∗ Faculty
of Informatics
Pan-European University, Bratislava, Slovakia
gabriel.juhas@paneurouni.com
¶ BIREGAL s.r.o., Bratislava, Slovakia
ana.juhasova@biregal.sk
 NETGRIF, s.r.o., Bratislava, Slovakia
juhas@netgrif.com, petrovic@netgrif.com

Abstract—Programming is basically about humans telling Typically, the necessary knowledge is obtained in a course
computers what to do in programming languages that computers devoted to Object-oriented programming.
understand but humans do not as much. Because of that, educa- In order to master presentation layer, one has to understand
tion in IT focuses on teaching students principles of programming
together with concrete programming languages needed to develop appropriate technologies to develop a static part of web-pages,
different parts of applications. For example, to develop web such as Hyper Text Markup Language or HTML and to
enterprise applications, such as e-commerce applications, using learn appropriate technologies to develop a dynamic part of
standard technology stack, students first need to learn principles web-pages, such as JavaScript language. In order to develop
of databases and database languages, such as entity-relationship modern user interfaces one usually uses some frameworks,
diagrams and SQL, principles of programming application layer
such as object-oriented programming and related languages, such such as Angular or React. To do it, one has to master
as Java, and programming languages used in presentation layer language TypeScript that extends JavaScript. Typically, the
such as HTML and JavaScript. Only then are students able to necessary knowledge is obtained in a course devoted to Web
develop web applications. Low-code languages’ objective is to technologies.
offer a tool that both computers and humans understand. When After a student gains these knowledge, student is prepared
using a suitable low-code language, such as Petriflow based on
object-centric processes, by learning principles of object-centric to take a course on Web-application development, where
processes and just one language, students can quickly learn how student learns how to connect these three layers, how to
to develop web process-based applications. We illustrate such send data from presentation layer to application layer and
application development using low code language on practical vice-versa using Hypertext Transfer Protocol (HTTP) get and
examples. post commands, how to use appropriate design pattern such
as Model–view–controller or MVC, how to add a service
I. I NTRODUCTION layer, integration layer and data access layer, i.e. how to
persist the Java objects into a repository such as a relational
One of the main motivation of students, when starting database using Java Persistence Application Programming
to study Computer Science and Information Technology, is Interface or Java Persistence API as a specification and some
to be able to develop a web-application, such as e-shops. implementation such as Hibernate. The student has to master
However, in last decades in order to be able to develop a writing queries, working with Dependency Injection, For better
web-applications, which typically have multi-layer architecture illustration of the complexity of the full stack sorce code see
including a data layer, application layer and presentation layer, Figure 1.
one has to master development of these layers. If several different users are working with the web-
In order to master data layer, one has to understand prin- application one often need to understand business processes
ciples of designing data layer, such as entity-relationship and to learn an appropriate language to model processes such
diagrams and to learn a language to implement data layer, as Business Process Model and Notation or BPMN and to
such as Sequential Query Language or SQL to implement adopt a workflow engine that control processes on application
a relational database. Typically, the necessary knowledge is layer, such as Camunda workflow engine.
obtained in a course devoted to Databases. If a student develop the application, in order to deploy the
In order to master application layer, one has to understand application and run it, student has to learn at least basics about
principles of programming, such as object-oriented design and (virtual) servers, application and web servers.
to learn a programming language to implement application Altogether, a student needs to take several courses prior to
layer such as Java. To develop a object oriented program be able to develop, deploy and run a web-application such as
in Java one usually uses some framework, such as Spring e-shop, which can take a relatively longer time in comparison
Boot and an Integrated Development Environment or IDE. to the time needed to develop and run a simple desktop

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 249

 Fig. 1. An example of full stack source code of a multi layer web application storing text input field name into a database

application. This is basically caused by fact, that students Netgrif application engine [5] available at etask.netgrif.cloud
have to take bottom up approach as there were no higher-level simply by uploading file with the generated Petriflow source
languages that abstract from the multi-layer architecture that code. Netgrif application engine is an interpreter of low-code
would enable to develop applications without above described language Petriflow.
knowledge. Up to our experience during lecturing courses In this way, one could teach computer science and infor-
covering the full stack described above, it takes at least four mation technologies in a different way, implementing a top
semesters to teach students the necessary fundamentals before down approach, starting to explain the principles of computer
they are able to implement multi layer web applications. This science and information technologies using higher-level low-
can decrease the motivation of students and even cause that code languages, such as Petriflow, and using cloud installments
they interrupt the study of computer science and information of a web-based Integrated Development Environment for such
technologies. a language and cloud installment of an interpreter of such
With the cloud technology and new generation of higher a language. Following this pattern, student would be able
level languages that abstract from multi layer architecture, to implement web applications within their first semester of
such as low-code language Petriflow [1], [2], [3], [4], based on study together with gathering knowledge of the theoretical
the concept of object-centric processes, everybody can develop fundamentals behind such language. This approach could
web-applications, even those controlled by business processes, substantially increase the motivation of students to understand
without any prior knowledge of databases, programming, and computer science and information technology more deeply,
web technologies. Developing web-applications in Petriflow even by understanding the implementation of the used in-
language from scratch, one can master principles of mod- terpreter of the low-code language used in the first phase
eling entities data, modeling entities life cycle in form of their study along with the standard courses such as databases,
workflow processes consisting of tasks associated with forms object oriented programming or web technologies. It is worth
containing input fields. Learning just one language, one is able mentioning, that for this approach it is crucial that the used
to develop a web-application within few lectures. Students interpreter of the low-code language has source code available
can use cloud installment of Petriflow builder available at for the students as it is in the case of Netgrif application
builder.netgrif.com, which is a web-based Integrated Devel- engine. For example, Netgrif application engine itself is a
opment Environment for dragging and dropping low-code ap- multi layer web application consisting of a data layer (using
plications and generating Petriflow source code. The generated non-relational database MongoDB indexed in Elasticsearch),
Petriflow applications can be deployed in cloud installment of application layer written in Java using Spring Boot framework

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 250

 Fig. 2. An example of the workflow process of conference paper submission processing in low-code language Petriflow

and with presentation layer assembled from Angular compo- and columns represented attributes of entities. Using special
nents. Therefore, alongside with course on object oriented pro- columns storing identity of entities and foreign keys, i.e.
gramming in later semesters, lecturer can use implementation special columns storing identity of different entities, relational
and source code of Netgrif application engine, that student databases implemented relationships between entities. The
used in their first semester, to illustrate a complex object- algorithmic part in database systems often reduces to just
oriented application written in Java. In that way, students can create, read, update delete operations and the most focus
still deeper understand why the web application that they wrote was put into efficient searching of records based on queries,
in a low-code language in first semester works. At the end, resulting in Sequential Query Language or SQL. We simply
the students will even be able to modify the interpreter of the call this approach data-centric. In programming focus was on
low-code language by extending its functionality. data structures, such as different types of collections, e.g. sets,
In following section, we will describe principles of low- lists, maps, linked lists, binary trees and on algorithms over
code language Petriflow based on object-centric processes in those data structures, that enable to put an entity into the
order to illustrate, how a low-code language can be used data structure and to find an entity in the data structure or
to teach students about basic concepts such as entities and to sort the data structure, yielding to searching and sorting
their data attributes, scope of attributes, relationships between algorithms over data structures [6]. Development in program-
entities, workflow processes, tasks, event-driven programming, ming languages answered with object-oriented programming,
relationships between events of the workflow processes, forms where types of entities were called classes and entities were
built using components and resulting in user interfaces, role called objects. A class defined data attributes of objects, which
based access, users. We will illustrate these principles devel- could be pointers to other objects, i.e. the attributed with values
oping a concrete web application using Netgrif application storing computer memory address of other objects. Class also
builder step by step and deploying the resulting source code defined methods, that are basically functions that can be called
of the application in low-code lagnuage Petriflow into Netgrif and that manipulate with data attributes. The life cycle of
application engine. objects in object-oriented programming languages was rather
trivial, with explicit constructor and often implicit destructor
II. P RINCIPLES OF LOW- CODE LANGUAGE P ETRIFLOW managed by garbage collection [7].
AND OBJECT- CENTRIC PROCESSES
Thus, in recent history IT departments were focused on stor-
Primarily, computers were used to solve computing tasks. ing data resulting in data-centric applications while business
In other words, computers were programmed to transform an departments were focused more on executing tasks to reach
input set of data to the output set of data. Programs were business objectives. Those tasks are organized or causally
basically algorithms, often first visualizes via flow charts, ordered into processes consisting of tasks and their causal flow.
written in a language computers understand. We can call this Business uses different diagrams or flow charts to visualize
approach algorithm-centric. Later, computers were started to processes, such as Bussiness Process Modelling and Notation
primary store data using relational databases that organized or BPMN, Event-driven Process Chain of EPC. This brings a
data into tables, where rows represented records or entities requirement on IT departments to deliver process based or

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 251

 Fig. 3. Designing form of task Paper of the object-centric process Paper in Netgrif Application Builder

process oriented applications resulting in so called process data attributes of the entity, life-cycle of the entity in form of
automation a part of broader term of digital transformation. a workflow process consisting of tasks, where each task may
Let us call this approach process-centric. Apparently, the have attached a form.
data-centric and process-centric approaches are in conflict. Take for example this paper, that you just read as the
Answer of the departments was that the data layer is designed, entity or object. On one hand, it can be understood or can
modeled, and implemented separately, application layer is be characterized by its data attributes and their values, for
separately designing, modelling and implementing processes example by a title, list of authors, corresponding author’s
and presentation layer is separately designing and imple- name, surname, email, affiliation, by abstract of the paper and
menting user interface in forms of screens via web pages by a file containing full version of the paper in PDF format
containing forms. Using this traditional approach, it may stored in a database of an information system.
results in one database table accessed directly by several At the same time, the same paper can be characterized by
workflow processes, one process accessing directly several a workflow process representing the life cycle of the paper,
tables, one task accessed or an event on a task triggered by a starting with the task called Paper submission. The task should
request from several different forms or one form triggering have its presentation layer consisting of a form, where the cor-
events in different workflow processes, or even one form responding authors first name, surname, email and affiliation
containing attributes of different tables. This may result in can be filled and a field where the submitter version of the
quite chaotic relationships between the data layer, process paper in PDF paper can be uploaded. In Petriflow language,
layer and presentation layer. this is done by so called dataRefs, which basically determine
However, data-centric and process-centric approach does the data fields that are displayed on the task data form, how
not need to be in conflict, when they are understood just and where on the form grid they are displayed, whether one
as different aspects of an entity. This approach, also recently can change their value or just view their value and whether
known in literature as object-centric processes, is based on the their value is required, If the corresponding information system
premise, that any entity or any object is a data entity consisting requires registration, one could suppose that the first name, the
of a set of data attributes and at the same time it is a workflow surname and the email of the corresponding author would be
process, where different tasks with attache user interface can pre-filled using the corresponding data of signed user.
change values of the data attributes of the entity or object. After the paper is submitted, that means after the task
Instead of separately modeling data, processes and forms and Paper submission is finished, the workflow process become to
then interconnect them freely, the object-centric processes uses the state submitted. Then the Review assignment task can be
vertically defined holistic all-layer-in-one entities, that include assigned to the program committee chair. In this task, program

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 252

 Fig. 4. Assigning and finishing a task in Petriflow process describing conference paper submission processing

committee chair should associate reviewers to the paper. whenever value of the field is changes and as we will see later
As we stated above, we will use this example as a running also an event Get triggered whenever it is sent from the server
example explaining how low-code language Petriflow imple- to the client user interface to be displayed.
ments the principles of object-centric processes. Similarly as Now to be more precise, in order to achieve that only the
we distinguish between a class and its objects, we distinguish user that created the instance of the paper can submit the paper,
between object-centric process and its instances also called one need to add a data field, say corrensponding_author of the
cases. Thus when speaking about paper as an object-centric type userList and in action of Create event change the value of
process, we mean general description of the paper, describing corrensponding_author by adding the logged user into the user
which attributes a paper has and what is their life-cycle list. Then, one need to set permission corrensponding_author
described by a workflow process, i.e. we consider paper as on events of the task Paper submission.
a type of objects, while a concrete paper from its submission In action of the Assign event of Submit paper task, we
through reviewing and acceptance/rejection till publishing the can trigger Set event of data fields name, surname and email
paper represent an instance also called case of the object- of the corresponding author by changing their values to
centric process paper. logged user name, surname and email. After logged user,
Important concept of Petriflow language are users and who is corresponding author fill other required attributes, such
roles, that implements role access control. Another important as title and upload the file as the value of data attribute
concept are events and actions, Actions of an event are pieces subbmitedPaper, task Paper submission can be finished. To
of code written in Groovy language that are executed whenever avoid repetitive dataRefs at different tasks, as you can see on
the event is triggered by a user. Users with a role of an object- Figure 2, where the workflow of the object centric process
centric process can trigger a set of events in any instance of is depicted with rectangular elements representing tasks and
that object-centric process. Thus, the scope of the role is the circles representing states, we use a special task called Paper,
process. If one wants to control access to events for specific which in fact has dataRefs to attributes name, surname, email,
instance, one can use special data attribute or data field type, affiliation, submittedPaper, title, abstract and authors. It also
called userList, which has the same functionality as role, but has a dataRef to attribute of enumeration type called status,
its scope is the single instance of the process. value of which will contains a string informing about status
Construction of an new instance of an object-centric process of the instance. This dataref will initially be hidden and will
is triggering an event Create. Each task has an event Assign be made visible and set to string Submitted by action of
which assign a user to the task, an event Finish and an Finish event of the task Paper submission. We use the fact, that
event Cancel. Each data field can have an event Set triggered Petriflow has special type of data attribute called taskRef, that

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 253

 Fig. 5. Creating instances of object-centric process Review in event Finish of the task Reviewer assignment of the object-centric process Paper

can point to a list of tasks, respectively their forms. Whenever produces for each arc starting in the transition the number of
there is a dataRef of a taskList attribute in a task, then at tokens given by the arc weight to the place connected by the
the respective place of the grid the forms of the tasks in the arc.
taskList attribute value will be displayed as sub-forms. Here For example, the task transition Paper submisssion can only
we use a data attribute called paper of type taskRef with be assigned to a user if in the place init is at least one token,
initial value set to the list of tasks containing just one task its assignment will consume one token from place init and
Paper. In this way, when displaying the form of the task Paper lead to a situation that Paper submisssion is in progress. After
submission it will display the form of the task Paper. The task finishing the task transition Paper submisssion a token will be
Paper will be performed only by system role, so no real user produced in places sumbitted1 and submitted2, as is visualized
can directly approach it. in Figure 4, where green transitions are enabled. Using this
The forms of tasks can easily be designed using form editor construction a parallel gateway is modeled.
of Netgrif application builder. An example of designing form The task Submission info contains just two dataRefs, one
of the task Paper is given on Figure 3. dataRef to the taskRef paper pointing to the form of the task
To determine workflow of object-centric processes, Petri- Paper as its sub-form in viewable non-editable node and a
flow uses extended Petri nets [8], [9], [10], [11], [12]. As we initially hidden dataRef to file field finalPaper. Once the task
already mentioned above, tasks are depicted as rectangles, also Submission info is enabled it remains enabled.
called transitions, and state variables are depicted as circles The task Reviewer assignment can only be assigned to a user
called places. Places and transitions are connected by directed with role PC Chair. It contains two dataRefs, a multichoice
weighted arcs with the default value of weight equal to one. map with all users that have role Reviewer filtered in action
Places can hold tokens. To assign a task transition to a user, of its Assign event, and a dataRef to the taskRef paper pointing
each place connected with the transition by an arc outgoing to the form of the task Paper as its sub-form in viewable non-
from the place has to hold at least so much token as is the editable node.
weight of the arc, then we say that the transition is enabled. Here we can illustrate how relationship between object-
If such an the arc finishing in the transition is finished by a centric processes are established. What we need is that for
bullet (so called red arc or test), then the assignment of the each user with role Reviewer, that was chosen by PC chair
task does not change the number of tokens in the place. If to review given paper, we want to create a new instance
the arc finishing in the transition is finished by an arrow, then of different object-centric process called Review in action of
assignment of the task consumes the number of tokens given Finish event of the task Reviewer assignment. This process
by the arc weight from the place. A task assigned to a user is depicted in Figure 5. It has the task Review that has form
can be eventually finished. Finish event of a task transition of the task form as a sub-form via a taskRef. Task form of

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 254

 Fig. 6. Action of the Set event of dataRef decision in task Approval of the object-centric process Paper

the process Review contains several dataRefs, including pre- members.

filled name and surname of the reviewer, the text of the review This example illustrates that relationships between object-
and the enumeration where reviewer chose one of the five centric processes can be established at level of data attributes
predefined options ranging from Accept to Reject. Task form in similar ways to foreign keys in databases of references
of the process Review contains also a taskRef pointing to to other objects in object oriented programming, but it also
the task Paper of the instance process Paper from which the enable synchronization of events in different instances and also
Review process instance was created. Thus, each reviewer has allow to use task forms from different instances of different
the details about the paper as a sub-form of the task Review. processes as sub-forms in a task form [13], [14], [?], [15].
The task View of the process Review is enabled after the Let us continue with task Approval in the process Paper. In
task Review is finished. It has only one dataRef to the taskRef this task a user with role PC chair may access the enumeration
form pointing to the form of the task form as its sub-form in data attribute named decision with just two options to choose:
viewable non-editable node. accept and reject. Based on the value of this enumeration, in
The task formRefed contains name, surname, text of the action of the Set event of the enumeration the values of number
review and the enumeration with recommendation whether to data attributes accept and reject are changed to one and zero
accept the paper. This task is used to serve as a sub-form whenever the option accept was chosen or to the values zero
in the instance of the process Paper. It is added to the list of and one whenever the option reject was chosen by the PC
tasks in taskRef called all_reviews, which is a data attribute of chair.
the process Paper, already in action of Finish event of the task Here we use the fact, that the arc weights in the underlying
Reviewer assignment of the process Paper, where the instances Petri net do not need to be constant, but can be variable.
of Review process were created. Namely, any data attribute with the type number can be
In the action of Finish event of the task Review of the used as a weight of an arc. The the respective weight will
process Review, it is tested whether this is the last review of take the value of the data attribute. Then, by finishing task
the paper instance missing. If yes, the actions trigger automatic Approval in the process Paper, the number of tokens equal
Assign event and Finish event of the task Reviews in the to the value of attribute accept will be produced in places
instance of the process Paper. This Finish event produces accepted1 and accepted2 and the number of tokens equal to
tokens in both places labeled as reviewed, making a parallel the value of attribute reject will be produced in place labeled as
split. Task All reviews of the process Paper shows all reviews, rejected and connected with the Approval task. If the accepted
i.e. all tasks formRefed of the related instances of the process attribute is one and rejected attribute is zero, then tasks Submit
Review. Task All reviews is accessible by users with role PC final version and Info about acceptance are enabled for the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 255

Fig. 7. Filling form of Paper submission task of the object-centric process Paper written in low-code language Petriflow deployed in Netgrif Aplication
Engine

corresponding author. If the accepted attribute is zero and processes are uploaded to Netgrif Application , PC chair can
rejected attribute is one, then task Info about rejection is invite PC members and Reviewer, associate them with their
enabled for the corresponding author. roles and the web application is ready to use for management
Using variable arc weight we can model any kind of of the process from paper submission to paper publishing.
decision, such as or-splits and xor-splits.
After the corresponding author upload the file with the III. S UMMARY
final version of its submission into the value of the data
attribute finalPaper during performing the task Submit final This paper discussed the possible alternative to traditional
version, the task can be finished and the token is produced causal flow of education in IT, which is usually built bottom
in the place submittedFinal. Now the PC chair can assign up. The presented alternative advocates the top down approach
the task Publishing, where he decides using enumeration using the next generation low-code languages such as low-
attribute decision2 with three options, whether the attribute code language Petriflow based on object-centric processes as
accept, reject or correction should be equal to one and the start of the educational process. The hypothesis is that using
remaining two attributes should be equal to zero. Based on such top down approach where students could really quickly
the decision, finish event of the task will produce token in develop relatively complex process-based application could
place finished or place accepted1 or place rejected connected encourage and motivate them to understand the necessary
with the Publishing task. If the token will be produced in place concept and technologies behind such a low-code language.
accepted1, a loop is realized allowing the corresponding author In order to motivate the reader and also to illustrate the
to resend the final version of the paper. If the token will be complexity of principles that can be explained using low-
produced in place rejected connected with the Publishing task, code language, such as complex relationships between object-
the corresponding author can read rejection reasons in task centric processes, we discussed an example of process based
Final version rejected. Finally, if the token will be produced web application for processing the submission of papers to a
in place accepted1, a loop is realized and any user can read conference, including review process. As a further research,
the final version of the paper in task View. we encourage to evaluate the presented hypothesis on the
After generating the Petriflow source code of both processes top down educational process staring with low-code language
Paper and Review designed using Netgrif Application Builder, using two groups of student, one following the presented top
PC chair can now register to etask.netgrif.cloud, make the down approach and the other group following the traditional
created group public and upload both processes. After the bottom up curricula.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 256

 R EFERENCES
[1] G. Juhás, T. Kovácik, J. Kovár, M. Kranec, and L. Petrovic, “Petri-
flow language and netgrif application builder,” in Proceedings of the
Demonstration Resources Track co-located with the 19th International
Conference on Business Process Management, BPM 2021, vol. 2973.
CEUR Workshop Proceedings, 2021, pp. 171–175.
[2] G. Juhas, L. Molnar, A. Juhasova, M. Ondrisova, M. Mladoniczky, and
T. Kovacik, “Low-code platforms and languages: the future of software
development,” 20th Anniversary of IEEE International Conference on
Emerging eLearning Technologies and Applications, ICETA 2022 -
Proceedings, pp. 286–293, 2022.
[3] G. Juhás, T. Ková?ik, J. Ková?, and M. Kranec, “Collecting structured
logs of instances as process nets in petriflow language,” CEUR Workshop
Proceedings, vol. 2907, pp. 105–122, 2021.
[4] M. Riesz, M. Se?kár, and G. Juhás, “Petriflow: A petri net based
framework for modelling and control of workflow processes,” CEUR
Workshop Proceedings, vol. 827, pp. 191–205, 2010.
[5] G. Juhás, T. Kovácik, J. Kovár, M. Kranec, and L. Petrovic, “Netgrif ap-
plication engine.” in Proceedings of the Demonstration Resources Track
co-located with the 19th International Conference on Business Process
Management, BPM 2021, vol. 2973. CEUR Workshop Proceedings,
2021, pp. 166–170.
[6] D. E. Knuth, The Art of Computer Programming, Vol. 1: Fundamental
Algorithms (3rd. ed.). Addison Wesley Longman Publishing Co., Inc.,
1997.
[7] A. Juhasova, G. Juhas, L. Molnar, M. Ondrisova, J. Mazari, and
M. Mladoniczky, “It induced innovations: Digital transformation and
process automation,” ICETA 2019 - 17th IEEE International Conference
on Emerging eLearning Technologies and Applications, Proceedings, pp.
322–329, 2019.
[8] W. M. Van der Aalst, “The application of petri nets to workflow
management,” Journal of circuits, systems, and computers, vol. 8, no. 01,
pp. 21–66, 1998.
[9] J. Desel, G. Juhás, R. Lorenz, and C. Neumair, “Modelling and vali-
dation with viptool,” in International Conference on Business Process
Management. Springer, 2003, pp. 380–389.
[10] A. Juhásova, I. Kazlov, G. Juhas, and L. Molnár, “How to model
curricula and learnflows by petri nets - a survey,” ICETA 2016 - 14th
IEEE International Conference on Emerging eLearning Technologies
and Applications, Proceedings, pp. 147–152, 2016.
[11] G. Juliás, F. Lehocki, and R. Lorenz, “Semantics of petri nets: A
comparison,” Proceedings - Winter Simulation Conference, pp. 617–628,
2007.
[12] J. Desel and G. Juhas, “"what is a petri net?" informal answers for the
informed reader,” Unifying Petri Nets, vol. 2128, pp. 1–25, 2001.
[13] J. Desel, H.-M. Hanisch, G. Juhás, R. Lorenz, and C. Neumair, “A guide
to modelling and control with modules of signal nets,” Lecture Notes
in Computer Science (including subseries Lecture Notes in Artificial
Intelligence and Lecture Notes in Bioinformatics), vol. 3147, pp. 270–
300, 2004.
[14] G. Juhás, R. Lorenz, and C. Neumair, “Modelling and control with
modules of signal nets,” Lecture Notes in Computer Science (including
subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics), vol. 3098, pp. 585–625, 2004.
[15] G. Juhas and R. Lorenz, Modelling with Petri modules, ser. Synthesis
and Control of Discrete Event Systems, B. Caillaud, P. Darondeau,
L. Lavagno, and X. Xie, Eds., 2002.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 257

2SHQERRNYHUVXVFORVHGERRNH[DPVDVDSDUWRI
K\EULGHGXFDWLRQLQFRPSXWHUVFLHQFHDQG
KXPDQLWLHVLQWKHSRVW&29,'HUD
-DQD-XULQRYi DQG-XUDM0LãWLQD 
,QVWLWXWHRI&RPSXWHU7HFKQRORJLHVDQG,QIRUPDWLFV 'HSDUWPHQWRI(QJOLVKDQG$PHULFDQ6WXGLHV
8QLYHUVLW\RI6V&\ULODQG0HWKRGLXV7UQDYD6.6ORYDN5HSXEOLF
MDQDMXULQRYD#XFPVNMXUDMPLVWLQD#XFPVN



$EVWUDFW²7KLV SDSHU GHOYHV LQWR WKH ORQJVWDQGLQJ GHEDWH
VXUURXQGLQJ WKH HIILFDF\ RI RSHQERRN YHUVXV FORVHGERRN , ,1752'8&7,21
H[DPLQDWLRQV LQ WKH FRQWH[W RI KLJKHU HGXFDWLRQ ZLWK D &XUUHQWO\WKHUHDUHVHYHUDOGLIIHUHQWZD\VRIDVVHVVLQJ
VSHFLILFIRFXVRQFRXUVHVZLWKLQWKHGRPDLQVRI,QIRUPDWLRQ VWXGHQWV HDFK RI ZKLFK KDV LWV RZQ VSHFLILFV DV ZHOO DV
6FLHQFH DQG +XPDQLWLHV 7KH FKRLFH EHWZHHQ WKHVH WZR DGYDQWDJHV DQG GLVDGYDQWDJHV ,Q WKH XQLYHUVLW\
H[DPLQDWLRQ IRUPDWV KDV VLJQLILFDQW SHGDJRJLFDO HQYLURQPHQWWKHHYDOXDWLRQRIVWXGHQWVLQWKHH[DPSHULRG
LPSOLFDWLRQV DV LW VKDSHV WKH OHDUQLQJ H[SHULHQFH DVVHVVHV LQ WKH IRUP RI D ZULWWHQ H[DP VWLOO SUHYDLOV 7KLV FDQ EH
WKH GHSWK RI XQGHUVWDQGLQJ DQG LQIOXHQFHV VWXGHQW GHVLJQHG DV D WHVW ZLWK PRVWO\ FORVHG TXHVWLRQV RU VKRUW
RXWFRPHV 7R HOXFLGDWH WKH DGYDQWDJHV DQG GUDZEDFNV RI DQVZHUTXHVWLRQV6XFKDWHVWFDQEHDSSOLHGHOHFWURQLFDOO\
HDFK DSSURDFK WKLV VWXG\ HPSOR\V D PL[HGPHWKRGV ZLWK DXWRPDWLF HYDOXDWLRQ ZKLFK VLJQLILFDQWO\ LQFUHDVHV
UHVHDUFK GHVLJQ LQFRUSRUDWLQJ TXDQWLWDWLYH GDWD IURP
WKH REMHFWLYLW\ RI WKH HYDOXDWLRQ 7KLV PHWKRG LV PDLQO\
VWXGHQW SHUIRUPDQFH PHWULFV DQG TXDOLWDWLYH LQVLJKWV IURP
XVHGLQWKHFDVHRIWKHQHFHVVLW\RIWHVWLQJDODUJHQXPEHU
VXUYH\V DQG LQWHUYLHZV 7KH DXWKRUV H[DPLQH WKHVH WZR
RI VWXGHQWV GXH WR LWV ORZ WLPH FRQVXPSWLRQ VLPSOH
SKHQRPHQD LQ D K\EULG OHDUQLQJ HQYLURQPHQW ZKLFK KDV
SURYHQWR EH DUHVXOWLQJHIIHFWLYHWRROIRUWKHLPSDFW RIWKH
GLVWULEXWLRQDQGHYDOXDWLRQ$VPHQWLRQHGE\3RQGHOtNRYi
&29,' SDQGHPLF 7KH\ H[SORUH KRZ WKH FRJQLWLYH
HW DO >@ WKDQNV WR WKH FXUUHQW SRVVLELOLWLHV RI VRIWZDUH
GHPDQGV DQG DVVHVVPHQW FULWHULD GLIIHU EHWZHHQ WKHVH WZR
VROXWLRQVLWFDQDOVREHDSSOLHGRQOLQHZLWKPLQLPDOULVN
PRGHV RIH[DPLQDWLRQFRQVLGHULQJWKH GLVWLQFWLYH QDWXUHRI RI IUDXG RQ WKH SDUW RI VWXGHQWV +RZHYHU ZLWK VXFK DQ
GLVFLSOLQHV ZLWKLQ ,QIRUPDWLRQ 6FLHQFH DQG +XPDQLWLHV ,Q HYDOXDWLRQ PHWKRG LW LV RIWHQ FRPSOLFDWHG WR WHVW KLJKHU
,QIRUPDWLRQ 6FLHQFH RSHQERRN H[DPV DUH RIWHQ UHJDUGHG FRJQLWLYHSURFHVVHVHLWKHUDFFRUGLQJWR%ORRP VWD[RQRP\
DV DOLJQHG ZLWK UHDOZRUOG VFHQDULRV UHIOHFWLQJ WKH >@ RU 1LHPHUNR V WD[RQRP\ >@ $FFRUGLQJ WR 6]ĘN|O HW
DFFHVVLELOLW\ RI LQIRUPDWLRQ UHVRXUFHV LQ SURIHVVLRQDO DO>@DVZHOODV$GDPHFHWDO>@LWLVDOVRQRWSRVVLEOH
FRQWH[WV LQ WKH SRVW&29,' HUD ,Q FRQWUDVW FORVHGERRN WRWHVWDQGDWWKHVDPHWLPHGHYHORSWKHVWXGHQWV FUHDWLYLW\
H[DPV HPSKDVL]H PHPRU\ UHFDOO DQG DQDO\WLFDO UHDVRQLQJ DQGWKHLUXQLTXHDSSURDFKWRVROYLQJSUREOHPVLQWKLVZD\
VNLOOV :H LQYHVWLJDWH ZKHWKHU WKHVH GLVWLQFWLRQV WUDQVODWH ,Q VXFK D ZD\ RI WHVWLQJ DGHHS LQVLJKW LQWR WKH DELOLWLHV
LQWR YDULHG OHDUQLQJ RXWFRPHV VWXGHQW VDWLVIDFWLRQ DQG DQG NQRZOHGJH RI VWXGHQWV DQG WKHLU DSSOLFDWLRQ LQ
ORQJWHUP NQRZOHGJH UHWHQWLRQ :LWKLQ WKH +XPDQLWLHV VSHFLILFVLWXDWLRQVDQGLQVROYLQJVSHFLILFSUREOHPVFDQEH
VXEMHFWV ZKHUH FULWLFDO WKLQNLQJ DQDO\VLV DQG ORVW+RZHYHULWLVVXLWDEOHIRUWHVWLQJVSHFLILFNQRZOHGJH
LQWHUSUHWDWLRQ DUH SDUDPRXQW WKH GHEDWH EHWZHHQ RSHQ DQG LQIRUPDWLRQ DV FRQILUPHG E\ $QD\D HW DO >@
ERRN DQG FORVHGERRN H[DPV JDLQV IXUWKHU FRPSOH[LW\ :H ³SUHSDULQJ IRU D FORVHGERRN H[DP FDQ HQFRXUDJH
H[DPLQH KRZ HDFK IRUPDW LQIOXHQFHV WKH GHYHORSPHQW RI FUDPPLQJZKLFKOHDGVWRVKRUWWHUPPHPRUL]DWLRQUDWKHU
FULWLFDO WKLQNLQJ VNLOOV WKH DELOLW\ WR V\QWKHVL]H DQG DSSO\ WKDQ UHWDLQLQJ NQRZOHGJH ORQJWHUP´ :ULWWHQ WHVWV FDQ
NQRZOHGJH DQG WKH FXOWLYDWLRQ RI D GHHS XQGHUVWDQGLQJ RI DOVR FRQVLVW RI RSHQHQGHG TXHVWLRQV RU SUREOHPEDVHG
FRPSOH[ WH[WV DQG FRQFHSWV $GGLWLRQDOO\ WKLV SDSHU WDVNV EXW VXFK D IRUP LV QRW VXLWDEOH IRU DXWRPDWLF
H[SORUHV WKH UROH RI WHFKQRORJ\ DQG GLJLWDO UHVRXUFHV LQ HYDOXDWLRQDQGWKHHYDOXDWLRQUHPDLQVZLWKWKHH[DPLQHU
VKDSLQJWKHG\QDPLFVRIRSHQERRNH[DPVDQGZKHWKHUWKLV ZKHUH WKHUH ZLOO DOZD\V EH D ULVN RI ELDVHG HYDOXDWLRQ
HYROXWLRQ LQ WKH DFFHVVLELOLW\ RI LQIRUPDWLRQ KDV DOWHUHG WKH HYHQ ZLWK WKH PD[LPXP HIIRUW WR PLQLPL]H LW 7KHUH LV
RXWFRPHV LQ ,QIRUPDWLRQ 6FLHQFH FRXUVHV FRPSDUHG WR DOVR WKH ULVN RI D WLPHFRQVXPLQJ GXUDWLRQ RI WKH H[DP
+XPDQLWLHV FRXUVHV 7KH ILQGLQJV IURP WKLV VWXG\ SURYLGH LWVHOIDVZHOODVLWVHYDOXDWLRQ,QWKHSDVWZULWWHQH[DPV
YDOXDEOH LQVLJKWV IRU HGXFDWRUV FXUULFXOXP GHVLJQHUV DQG DIWHU WKHLU VXFFHVVIXO FRPSOHWLRQ ZHUH RIWHQ
DGPLQLVWUDWRUV LQ ,QIRUPDWLRQ 6FLHQFH DQG +XPDQLWLHV VXSSOHPHQWHGZLWKDQRUDOH[DPWKDQNVWRZKLFKVWXGHQWV
SURJUDPVKHOSLQJWKHPPDNHLQIRUPHGGHFLVLRQVDERXWWKH FRXOGGHIHQGWKHLUVROXWLRQRUFRUUHFWPLVWDNHVFDXVHGE\
PRVW DSSURSULDWH H[DPLQDWLRQ IRUPDW IRU WKHLU FRXUVHV %\ LQDWWHQWLRQDQGFRQYLQFHWKHH[DPLQHURIWKHLUNQRZOHGJH
FRQVLGHULQJ WKH XQLTXH FKDUDFWHULVWLFV RI WKHVH GLVFLSOLQHV %XWRQWKHFRQWUDU\WKHH[DPLQHUKDGWKHRSSRUWXQLW\WR
WKH UHVHDUFK FRQWULEXWHV WR WKH RQJRLQJ FRQYHUVDWLRQ YHULI\ WKH UHDO NQRZOHGJH RI WKH VWXGHQW DQG WKXV
VXUURXQGLQJ DVVHVVPHQW PHWKRGV LQ KLJKHU HGXFDWLRQ DQG PLQLPL]H WKH ULVN RI IUDXG RQ WKH SDUW RI WKH VWXGHQW
RIIHUV D QXDQFHG SHUVSHFWLYH RQ WKH RSHQERRN YHUVXV +RZHYHUWKLVFRPELQHGPHWKRGLVWLPHFRQVXPLQJDQGLV
FORVHGERRNH[DPLQDWLRQGLOHPPD FXUUHQWO\GLVDSSHDULQJ
(VSHFLDOO\ LQ WKH ILHOGV RI KXPDQLWLHV DQG VRFLDO
VFLHQFHV WKH ZULWWHQ RU RUDO H[DPLQDWLRQ GXULQJ WKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 258

H[DPLQDWLRQ SHULRG LV XVXDOO\ UHSODFHG E\ WKH SURFHVVLQJ ,, 0$7(5,$/6$1'0(7+2'6
RI D VHPLQDU SDSHU 6WXGHQWV ZULWH H[WHQVLYH ZRUNV RU )URP WKH DFDGHPLF \HDU  DV SDUW RI WKH
HVVD\V RQ D FHUWDLQ WRSLF ZKLFK IRUFHV VWXGHQWV WR PDVWHU V GHJUHH SURJUDP ³DSSOLHG LQIRUPDWLFV´ DW WKH
FRPSUHKHQVLYHO\ GHOYH LQWR DQG WKLQN DERXW WKH WUHDWHG ,QVWLWXWHRI&RPSXWHU7HFKQRORJLHVDQG,QIRUPDWLFVDWWKH
WRSLFEDVHGRQWKHVWXG\RIYDULRXVVRXUFHVZKLFKOHDGVWR )DFXOW\RI1DWXUDO6FLHQFHVRIWKH8QLYHUVLW\RI6V&\ULO
DEHWWHUXQGHUVWDQGLQJRIWKHWRSLF$VPHQWLRQHGLQ>@LQ DQG0HWKRGLXVLQ7UQDYD6ORYDNLDWDXJKWWKHFRPSXOVRU\
WKLV ZD\ WKH\ FDQ GHPRQVWUDWH WKHLU NQRZOHGJH DQG RSWLRQDO VXEMHFW ³7KHRU\ DQG GHYHORSPHQW RI FRPSXWHU
DQDO\WLFDOVNLOOVVKRZFUHDWLYLW\LQZULWWHQH[SUHVVLRQDQG JDPHV´7KHGHVLJQDQGGHYHORSPHQWRIFRPSXWHUJDPHV
ZRUNZLWKUHVRXUFHV(YDOXDWLQJVXFKZRUNVFDQRIWHQEH FDQEHFRQVLGHUHGDVXIILFLHQWO\FRPSOH[LVVXH,QDGGLWLRQ
WLPHFRQVXPLQJ IRU HGXFDWRUV WKH HYDOXDWLRQ FDQ EH WR WKH SURJUDPPLQJ DQG GHYHORSPHQW RI WKH JDPH LWVHOI
VXEMHFWLYHLIWKHUHDUHQRVWULFWO\GHILQHGHYDOXDWLRQUXOHV ZKLFKLVWKHHVVHQWLDOFRQWHQWRIWKHPHQWLRQHGVXEMHFWLW
DQGWKHUHLVDKLJKULVNRISODJLDULVPZKLFKLQPDQ\FDVHV DOVRLQFOXGHVVXFKFRPSUHKHQVLYHWRSLFVDVJDPHGHVLJQ
LV GLIILFXOW WR GHWHFW DV ZHOO DV SURYH 7KHVH DUH RIWHQ PRGHOOLQJ RI ' REMHFWV LPSOHPHQWDWLRQ RI DUWLILFLDO
VXSSRUWHGE\ D SUHVHQWDWLRQZKHUH VWXGHQWV SUHVHQW WKHLU LQWHOOLJHQFH LQ JDPHV SURFHVVLQJ RI DXGLRYLVXDO JDPH
NQRZOHGJHDQGILQGLQJVLQIURQWRIDJURXSRIFODVVPDWHV HOHPHQWV JDPH PDUNHWLQJ DQGGLVWULEXWLRQRI JDPHVEXW
DQG D WHDFKHU WKXV GHYHORSLQJ WKH DELOLW\ WR VSHDN DQG DOVR WKH JDPLQJ LQGXVWU\ DQG OHJDO DVSHFWV &RQVLGHULQJ
FRPPXQLFDWH6WXGHQWVPXVWGHPRQVWUDWHWKHLUNQRZOHGJH WKH DERYH ZH FRXOG FODLP WKDW WKH LVVXH LV VR H[WHQVLYH
DQGXQGHUVWDQGLQJLQIURQWRIDJURXSIRUZKLFKWKH\FDQ WKDWLWZRXOGDOVREHVXLWDEOHIRUWKHFRQFHSWRIDVHSDUDWH
EHDVRXUFHRIQHZLQIRUPDWLRQ7KLVIRUPRIWHVWLQJFDQ VWXG\SURJUDPWKHSRWHQWLDORIZKLFKFRXOGEHLQWHUHVWLQJ
OHDG WR SURIHVVLRQDO GLVFXVVLRQV DQG LQWHUDFWLRQ EHWZHHQ ZKLFKLVLQGLUHFWO\FRQILUPHGE\WKHH[LVWHQFHRIWKHVWXG\
VWXGHQWV DQG WKH WHDFKHU +RZHYHU VRPH VWXGHQWV PD\ SURJUDP³WKHRU\RIGLJLWDOJDPHV´DWWKH)DFXOW\RI0DVV
KDYH SUREOHPV ZLWK SXEOLF VSHDNLQJ VWDJH IULJKW VR LW LV 0HGLD&RPPXQLFDWLRQRIWKH8QLYHUVLW\RI6V&\ULODQG
DGYLVDEOH WR GHYHORS WKHVH VNLOOV GXULQJ WKH VHPHVWHU DV 0HWKRGLXV LQ 7UQDYD 6ORYDNLD 6WXGHQWV RI DSSOLHG
ZHOO 3UHVHQWLQJ LWVHOI FDQ WDNH D ORW RI WLPH DV ZHOO DV LQIRUPDWLFV ZKR DUH LQWHUHVWHG LQ WKLV LVVXH DUH RIIHUHG
RUJDQL]LQJ WKHP FDQ FDXVH FRQVLGHUDEOH SUREOHPV ,Q VHYHUDOFRPSXOVRU\RSWLRQDOVXEMHFWV%HĖRHWDO>@DOVR
WHFKQLFDO ILHOGV ZH DUH PRUH OLNHO\ WR HQFRXQWHU WKH GHDO ZLWK WKH LVVXH LQ WKHLU UHVHDUFK 7KH FRQFHSW RI WKH
SURFHVVLQJRISURMHFWVRQZKLFKVWXGHQWVRIWHQZRUNIRUD VXEMHFW ³7KHRU\ DQG GHYHORSPHQW RI FRPSXWHU JDPHV´
ORQJWLPH7KHVHDUHVXEVHTXHQWO\HYDOXDWHGDFFRUGLQJWR GHWHUPLQLQJWKHFRQWHQWDQGWKHJRDORIHGXFDWLRQZDVQRW
WKHLUFRPSOH[LW\DQGTXDOLW\ DQHDV\WDVN:HZHUHIDFHGZLWKWKHTXHVWLRQRIKRZWR
$VVHVVPHQWRIVWXGHQWVGXULQJWKHH[DPSHULRGFDQDQG LPSOHPHQWWKHFRQWHQWRIWKHHGXFDWLRQLQYLHZRIWKHVHW
RIWHQLVVXSSOHPHQWHGE\RQJRLQJDVVHVVPHQWGXULQJWKH JRDOVRIWKHVXEMHFWZLWKWKHDLPRI PD[LPXPTXDOLW\RI
VHPHVWHU(GXFDWRUVWKXVHYDOXDWHVWXGHQWVRQWKHEDVLVRI HGXFDWLRQ PRWLYDWLQJ HQYLURQPHQW DQG FRYHUDJH RI WKLV
WKHLU SHUIRUPDQFH WKURXJKRXW WKH SHULRG QRW MXVW RQ WKH EURDGWRSLF7KHPRWLYDWLRQIRUGHYHORSPHQWDQGWKHPDLQ
EDVLV RI RQHWLPH H[DP ZKLFK FDQ EH VWUHVVIXO IRU JRDO IRU WKH JDPH GHYHORSHU LV WKH SURGXFW LWVHOI
VWXGHQWV DQG FDQ DIIHFW WKHLU SHUIRUPDQFH 7KH IRUP RI 7KHUHIRUH WKH PRVW VXLWDEOH VROXWLRQ VHHPHG WR EH WR
FRQWLQXRXV HYDOXDWLRQ FDQ EH GLIIHUHQW ,W FDQ FRQVLVW RI IRFXVRQWKHGHYHORSPHQWRIDFRPSOHWHO\IXQFWLRQDOJDPH
PLGWHUP WHVWV ZULWWHQ H[DPV VPDOOHU SURMHFWV VHPLQDU RU LWV SURWRW\SH :H DOVR FRQVLGHUHG RWKHU DOWHUQDWLYHV
SDSHUV SUDFWLFDO WDVNV HWF EXW WKH WHDFKHU FDQ DOVR VXFK DV LOOXVWUDWLQJ GLIIHUHQW FRQFHSWV DQG JDPH
HYDOXDWH WKH DFWLYH SDUWLFLSDWLRQ DQG SURIHVVLRQDO PHFKDQLVPVRQVXEVLWXDWLRQVRFFXUULQJLQGLIIHUHQWW\SHV
GLVFXVVLRQ RI VWXGHQWV LQ FODVV RU LQYROYH WKH VWXGHQWV RI JDPHV 7KLV LQWHQWLRQ DOVR SURYHG WR EH VXLWDEOH DQG
WKHPVHOYHVLQWKHHYDOXDWLRQZKLFKXQGRXEWHGO\LQFUHDVHV ZRXOG FHUWDLQO\ ILQG LWV SRSXODULW\ DPRQJ WKH VWXGHQWV
WKHREMHFWLYLW\RIWKHHYDOXDWLRQ $IWHUDWKRURXJKDQDO\VLVLWEHFDPHFOHDUWKDWLWZRXOGEH
,Q UHFHQW \HDUV DVVHVVPHQW XVLQJ D SRUWIROLR ZKLFK XVHIXOLIVHYHUDOVXEMHFWVZHUHGHYRWHGWRWKHJLYHQLVVXH
UHSUHVHQWV WKH GRFXPHQWDWLRQ RI VWXGHQWV  ZRUN DV DQG FRXOG EXLOG RQ WKH DOUHDG\ H[LVWLQJ NQRZOHGJH DQG
HYLGHQFH RI WKH GHYHORSPHQW RI WKHLU NQRZOHGJH RYHU H[SHULHQFH RI WKH VWXGHQWV DQG GHYHORS WKHLU RZQ JDPHV
WLPHKDVDOVRFRPHWRWKHIRUH,WVKRXOGHQDEOHWKHPRVW DQGLGHDV(DFKW\SHRIJDPHKDVLWVRZQVSHFLILFVVRLW
FRPSUHKHQVLYH YLHZ RI KRZ VWXGHQWV OHDUQ KRZ WKH\ ZDV LPSRUWDQW WR GHFLGH ZKLFK JDPHV WR LQFOXGH LQ
FRPPXQLFDWH ZLWK RWKHUV DQG ZKDW LQWHUHVWV DQG WDOHQWV HGXFDWLRQ ,W VHHPHG LGHDO WR ILQG RQH UHSUHVHQWDWLYH
WKH\ KDYH $VVHVVPHQW XVLQJ D SRUWIROLR DOORZV VWXGHQWV PRGHO RQ ZKLFK DOO IDFWV FRXOG EH LOOXVWUDWHG +RZHYHU
ZLWK GLIIHUHQW OHDUQLQJ VW\OHV WR GHPRQVWUDWH NQRZOHGJH WKLVLVQRWSRVVLEOHGXHWRWKHLVVXHEHFDXVHHDFKW\SHRI
DQG VNLOOV > @ ,W HQDEOHV FRQWLQXRXV DVVHVVPHQW RI JDPHKDVLWVRZQVSHFLILFV7KHHIIRUWZDVWRFKRRVHVXFK
VWXGHQWVEDVHGRQWKHLUZRUNRYHUDORQJHUSHULRGRIWLPH W\SHV RI JDPHV VR WKDW WKH\ FRXOG LOOXVWUDWH WKH ZLGHVW
,W DOORZV IRU DXWKHQWLF DVVHVVPHQW DV LW FRQWDLQV UHDO SRVVLEOHQXPEHURIFRQFHSWVDQGJDPHPHFKDQLFVDQGDW
VDPSOHVRIVWXGHQWZRUN WKH VDPH WLPH WKH\ ZRXOG KDYH VXIILFLHQW SRWHQWLDO IRU
,W LV LPSRUWDQW WR QRWH WKDW QR DVVHVVPHQW PHWKRG LV H[SDQVLRQVRWKDWWKHVHIXQFWLRQDOLWLHVFRXOGEHGHYHORSHG
SHUIHFWDQGWKHFKRLFHRIDQDSSURSULDWHPHWKRGGHSHQGV E\VWXGHQWVZLWKLQWKHVHPHVWHUDVVLJQPHQWV/DVWEXWQRW
RQWKHDVVHVVPHQWREMHFWLYHVWKHQDWXUHRIWKHVXEMHFWDQG OHDVW ZH KDG WR FRQVLGHU WKH WLPHFRQVXPLQJ QDWXUH RI
WKH SUHIHUHQFHV RI WKH HGXFDWRU DQG VWXGHQWV $ JDPH GHYHORSPHQW LQ RUGHU WR EH DEOH WR IXOILO WKH
FRPELQDWLRQRIGLIIHUHQWDVVHVVPHQWPHWKRGVFDQRIWHQEH HGXFDWLRQDO JRDOV ZLWKLQ WKH VFRSH RI WKH VXEMHFW V WLPH
WKHPRVWHIIHFWLYHZD\WRJHWDEDODQFHGYLHZRIVWXGHQW DOORZDQFH :H DUUDQJHG WKH VXEMHFW LQ VXFK D ZD\ WKDW
SHUIRUPDQFHZKLFKLVDOVRFRQILUPHGE\>@$VUHSRUWHG VWXGHQWV ZLOO HQFRXQWHU WKH GHYHORSPHQW RI WKUHH JDPH
E\>@WKHUHDUHDYDULHW\RIWHDFKLQJVWUDWHJLHVWKDWFDQ SURWRW\SHV ,Q RUGHU WR IXOO\ SXEOLVK WKH SURGXFW LWVHOI LW
IRVWHU FULWLFDO WKLQNLQJ WKDW LQFOXGHUHIOHFWLRQ VLPXODWLRQ ZRXOG VWLOO EH QHFHVVDU\ WR ZRUN H[WHQVLYHO\ RQ
FDVHVWXGLHVH[DPLQDWLRQVDQGFROODERUDWLYHOHDUQLQJ´ GHYHORSPHQW VR WKDW WKH JDPHV FDQ DWWUDFW WKH SOD\HU V
DWWHQWLRQIRUDORQJHUSHULRGRIWLPH+RZHYHUWKHEDVLVLV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 259

WRWUDQVIHUNQRZOHGJHWRWKHVWXGHQWVWRVXFKDQH[WHQWWKDW GXULQJ GHYHORSPHQW UHJDUGOHVV RI WKH UHDGLQHVV DQG
WKH\ DUH LQGLYLGXDOO\ FDSDEOH RI LW 'XULQJ WKH VHPHVWHU VRSKLVWLFDWLRQRIWKHOHVVRQFRQFHSWDQGGXHWRWKHWLPH
ZH GHYHORS IXQFWLRQDO SURWRW\SHV RI WKHVH JDPHV ZKLFK FRQVXPLQJ QDWXUH RI WKH GHYHORSPHQW LWVHOI IXOO
VWXGHQWVIXUWKHUGHYHORSDVSDUWRIWKHVXEMHFWHYDOXDWLRQ LPSOHPHQWDWLRQRIWKHVHWHGXFDWLRQDOJRDOVZRXOGQRWEH
'XULQJWKHLUHGXFDWLRQVWXGHQWVFUHDWHWKHLURZQSRUWIROLR SRVVLEOH ,Q VXFK D FDVH WKHUH ZRXOG EH QR SRLQW LQ
RIPDWHULDOVWKDWWKH\FDQXVHWRSUHVHQWWKHPVHOYHVZKHQ WKLQNLQJDERXWWKHLUHIIHFWLYHDQGKLJKTXDOLW\IXOILOPHQW
DSSO\LQJ IRU D MRE7KH VXEMHFW GRHV QRW RPLW DQ\ RI WKH )RUWKHUHDVRQVPHQWLRQHGZHFRQVLGHUHGK\EULGWHDFKLQJ
DUHDVFRQQHFWHGZLWKWKHGHYHORSPHQWRIJDPHVEXWLQWKH EDVHG RQ WKH SULQFLSOH RI WKH IOLSSHG FODVV DV DQ LGHDO
FRQWH[WRIWKHVFRSHRIWKHJLYHQLVVXHLWGRHVQRWFRYHU VROXWLRQ )RU VWXGHQWV ZH KDYH SUHSDUHG LQVWUXFWLRQDO
DOODUHDVLQGHWDLO7KHJRDOLVWRSUHSDUHWKHVWXGHQWWREH YLGHRV DV ZHOO DV VFULSWV DQG HOHDUQLQJ PDWHULDOV
DEOHWRFRQWLQXHWRGHYHORSLQWKHJLYHQILHOGWKURXJKVHOI GHGLFDWHGWRWKHGHYHORSPHQWRIVSHFLILFJDPHSURWRW\SHV
VWXG\:HVXSSRUWWKLVJRDOZLWKYDULRXVSUREOHPVROYLQJ ZULWWHQLQWKHIRUPRIDWXWRULDO
WDVNV WKDW VWXGHQWV ZRUN RQ GXULQJ WKH VHPHVWHU $PRQJ ,Q WKH DFDGHPLF \HDUV  DQG  ZH
RWKHU WKLQJV WKLV VXEMHFW EXLOGV RQ WKH NQRZOHGJH WKDW FDUULHG RXW TXDOLWDWLYH UHVHDUFK ZLWKLQ WKHVH FRXUVHV LQ
VWXGHQWV KDYH DFTXLUHG LQ RWKHU VXEMHFWV 7KH FRQWHQW RI ZKLFKZHPDLQO\IRFXVHGRQYHULI\LQJWKHHIIHFWLYHQHVVRI
WKHVXEMHFWLVWKHUHIRUHFRQVWUXFWHGLQVXFKDZD\WKDWLWV WKH FUHDWHG PDWHULDOV LQ WKH LPSOHPHQWDWLRQ RI YDULRXV
FRQWHQW FDQ EH DGDSWHG DQG GHYHORSHG DFFRUGLQJ WR WKH IRUPV RI HGXFDWLRQ >@ 7KH PDWHULDOV ZHUH FUHDWHG LQ
VWXGHQWV¶ FXUUHQW QHHGV JLYHQ WKH VWXGHQWV¶ LQSXW VXFK D ZD\ DV WR UHIOHFW WKH FXUUHQW WUHQGV RI LQQRYDWLYH
NQRZOHGJH 7KH JRDO LV QRW WR IRFXV RQ VSHFLILF JDPH HGXFDWLRQZKLOHDGKHULQJWRWKHVHWJRDOVDQGDSSO\LQJWKH
PHFKDQLVPV XVHG LQ VSHFLILF ' RU ' JDPHV ORFDO RU UHTXLUHGGLGDFWLFSULQFLSOHV2XUHIIRUWLVQRWWRHOLPLQDWH
PXOWLSOD\HUJDPHVEXWLQVXFKDZD\WKDWWKHVXEMHFWFDQ SHUVRQDO FRQWDFW ZLWK WKH WHDFKHU DV ZHOO DV ZLWK
EHJXLGHGDFFRUGLQJWRWKHSUHIHUHQFHVRIWKHVWXGHQWVDQG FODVVPDWHVZKRDUHWKHVRXUFHRIPDQ\RWKHUVWLPXOLWKDW
WKHLUFXUUHQWOHYHORINQRZOHGJHZLWKWKHDLPRIDFTXLULQJ KHOSHGXFDWLRQ7KHUHIRUHWKHPDWHULDOLVFUHDWHGLQVXFKD
QHZSUHGHILQHGFRPSHWHQFLHVDQGVNLOOV7KHEDVLFLGHDLV ZD\ WKDW LW FDQ EH LQFRUSRUDWHG LQWR WKH HGXFDWLRQDO
WRFRQGXFWHGXFDWLRQZLWKLQWKLVFRXUVHLQVXFKDZD\WKDW SURFHVVLQGLIIHUHQWZD\V,WFDQEHXVHGERWKIRUIDFHWR
HDFKVWXGHQWJRHVWKURXJKDOOVWDJHVRIJDPHGHYHORSPHQW IDFHWHDFKLQJDQGIRULQGLYLGXDOVWXGHQWZRUNZLWKRXWWLPH
DQGEHFRPHVIDPLOLDUZLWKXQLYHUVDOJDPHFRQFHSWVXVHG DQG VSDFH OLPLWDWLRQV $W WKH VDPH WLPH RXU HIIRUW LV WR
LQJDPHVWKDWGHSHQGRQWKHLUW\SHV/DVWEXWQRWOHDVWLW QDWXUDOO\ DIIHFW WKH GLJLWDO OLWHUDF\ RI VWXGHQWV WKH
LV DOVR DERXW WKH VWXGHQWV EHLQJ DEOH WR EXLOG RQ WKHLU GHYHORSPHQWRIVRIWVNLOOVDOJRULWKPLFORJLFDODQGFULWLFDO
VWUHQJWKV DQG XVH WKHP DQG IXUWKHU GHYHORS WKHP WR D WKLQNLQJ ZKLFKLVUHTXLUHGE\ZRUNLQJZLWKPDWHULDOVDQG
VDWLVIDFWRU\ OHYHO :H VXSSRUWHG WKLV E\ LQWURGXFLQJ D WKH RYHUDOO ZD\ RI FRQGXFWLQJ DQG LPSOHPHQWLQJ WKH
FRPSOH[ KHWHURJHQHRXV HYDOXDWLRQ V\VWHP >@ WKH VXEMHFW  DQG IRUPLQJ D SRVLWLYH UHODWLRQVKLS WR OLIHORQJ
HIIHFWLYHQHVVRIZKLFKZHKDYHDOUHDG\YHULILHGLQVHYHUDO OHDUQLQJ LWVHOI WKH ZD\ PDWHULDOV DUH SURFHVVHG DQG WKHLU
VXEMHFWV:HKDYHSURYHQWKDWVXFKDQHYDOXDWLRQV\VWHP XVHLQWKHHGXFDWLRQDOSURFHVV
PRWLYDWHV VWXGHQWV WR VHOIUHJXODWH OHDUQLQJ ZKLOH 7KH UHVHDUFK VDPSOH FRQVLVWHG RI  VWXGHQWV RI
PDLQWDLQLQJ DQG GHYHORSLQJ WKH VWXGHQWV¶ RZQ LQGLYLGXDO $SSOLHG,QIRUPDWLFVDQGVWXGHQWVRIWKHVWXG\SURJUDP
DELOLWLHVDQGLQGLVSXWDEO\VXSSRUWVWKHEXLOGLQJRIDJRRG (QJOLVK /DQJXDJH LQ &XOWXUH DQG 6SHFLDOL]HG
VWXGHQWWHDFKHUUHODWLRQVKLS &RPPXQLFDWLRQ7KHLUJHQGHUUHSUHVHQWDWLRQZDV
,QRUGHUWRREWDLQDVPXFKGDWDDVSRVVLEOHIURPDQRWKHU PHQDQGZRPHQLQWKHFRPSXWHUVFLHQFHSURJUDP
SRLQW RI YLHZ ZH DOVR LQYHVWLJDWHG WKH DWWLWXGHV DQG DQG PHQ DQG  ZRPHQ LQ WKH KXPDQLWLHV VWXG\
H[SHULHQFHVRIPDVWHU VGHJUHHVWXGHQWVLQWKHKXPDQLWLHV SURJUDP7KHDJHUDQJHRIWKHSDUWLFLSDQWVZDVGLIIHUHQW
0DVWHU¶VVWXG\SURJUDP(QJOLVK/DQJXDJHLQ&XOWXUHDQG IURPWR\HDUV LQFOXGLQJSDUWWLPHVWXGHQWV ,QWKH
6SHFLDOL]HG &RPPXQLFDWLRQ ,W LV D VWXG\ SURJUDP WKDW FRQWH[W RI WKH UHVHDUFK ZH FRQVLGHU WKLV D SRVLWLYH
UHTXLUHV GLUHFW FRQWDFW EHWZHHQ WKH WHDFKHU DQG VWXGHQWV SKHQRPHQRQEHFDXVHGXHWRWKHDJHUDQJHRIWKHUHVHDUFK
GXH WR WKH DSSOLHG WHDFKLQJ PHWKRGV VXFK DV JURXS SDUWLFLSDQWV ZH H[SHFWHG PRUH UHOLDEOH DQVZHUV DQG D
WHDFKLQJ SUREOHPEDVHG WDVNV FRPPXQLFDWLYH ODQJXDJH GLYHUVH YLHZ RI WKH LQYHVWLJDWHG LVVXH EDVHG RQ WKH
WHDFKLQJ FRQWHQWEDVHG WDVNEDVHG DQG SDUWLFLSDWRU\ SHUVRQDODQGSURIHVVLRQDOPDWXULW\RIWKHSDUWLFLSDQWV7KH
DSSURDFKHV HWF 7KHUHIRUH ZH LQYHVWLJDWHG WKH LPSOHPHQWDWLRQ RI WKUHH FRPSOH[ FDVH VWXGLHV WRJHWKHU
SRVVLELOLWLHV RI DSSO\LQJ WKH PRGHUQ SULQFLSOH RI RSHQ ZLWK H[SHULPHQWV DQG DFWLRQ UHVHDUFK DOORZHG XV WR
ERRN H[DPV DQG VWXGHQW VDWLVIDFWLRQ DV ZHOO DV WKH LQYHVWLJDWH WKH REVHUYHG SKHQRPHQD PRUH GHHSO\ LQ
HIIHFWLYHQHVV RI VXFK DQ DSSURDFK :H FRQVLGHUHG WKH LQWHUFRQQHFWHG FRQWH[WV DQG ZH FRXOG DOVR SD\ DWWHQWLRQ
LQWHJUDWLRQRIWKHWZRVHPHVWHUFRXUVH³(QJOLVK/DQJXDJH WR WKH LQYHVWLJDWLRQ RI XQLTXH SKHQRPHQD DQG VLWXDWLRQV
LQ 6FLHQFH DQG 7HFKQRORJ\´ LQWR WKH UHVHDUFK WR EH D WKDWRFFXUUHGGXULQJWKHUHVHDUFK7KHVHKDYHEURXJKWXV
VXLWDEOH DQG HIIHFWLYH HOHPHQW LQ PHDVXULQJ RXWSXWV LQ VHYHUDO ILQGLQJV ZKLFK ZH DUH JUDGXDOO\ H[DPLQLQJ LQ
FRPSDULQJWKHHIIHFWLYHQHVVRIDSSO\LQJRSHQERRNH[DPV GHWDLO SURSRVLQJ QHZ VROXWLRQV DQG YHULI\LQJ WKHLU
LQLQIRUPDWLFVDQGKXPDQLWLHVVXEMHFWV HIIHFWLYHQHVV:HZHUHDEOHWRFRQILUPWKDWHGXFDWLRQVHW
$QRWKHU GLOHPPD ZH IDFHG ZDV WKH IRUP RI WHDFKLQJ XS LQ WKLV ZD\ ZKHWKHU XVLQJ LQVWUXFWLRQDO YLGHRV
WKH VXEMHFW ³7KHRU\ DQG 'HYHORSPHQW RI &RPSXWHU HOHDUQLQJPDWHULDOVRUVFULSWVLQDQRQOLQHRUK\EULGIRUP
*DPHV´ 7KHUH ZDV D FRQFHUQ WKDW DSSO\LQJ WKH XVXDO XVLQJWKHIOLSSHGFODVVURRPFRQFHSWOHDGVWRWKHLQWHQGHG
PHWKRG RI IDFHWRIDFH WHDFKLQJ ZLWK LPSOHPHQWDWLRQ UHVXOWV
GXULQJIDFHWRIDFHWHDFKLQJDQGSUDFWLVLQJDQGFODULI\LQJ :HDOVRGHPRQVWUDWHGWKDWLQWKLVFDVHWUDGLWLRQDOIDFH
IDFWVGXULQJKRPHSUHSDUDWLRQ ZRXOGQRWEHSRVVLEOHDQG WRIDFH HGXFDWLRQ ZRXOG QRW OHDG WR WKH VHW JRDOV RU DW
HIIHFWLYH IURP D WLPH SRLQW RI YLHZ %DVHG RQ P\ RZQ OHDVW QRW WR WKH H[WHQW WKDW ZH DFKLHYHG ZLWK WKH K\EULG
H[SHULHQFH ZLWK WKH GHYHORSPHQW RI XSFRPLQJ JDPH DQG RQOLQH IRUP LQ FRPELQDWLRQ ZLWK WKH IOLSSHG
SURWRW\SHV DQG RQ WKH EDVLV RI SUREOHPV WKDW PD\ DULVH FODVVURRP FRQFHSW ,Q DQ\ FDVH GXULQJ WKH FRQWHQW

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 260

DQDO\VLV RI WKH WDVNV WKDW WKH VWXGHQWV ZRUNHG RQ WKHFRPELQDWLRQRIDOOWKHDFWLYLWLHVDQGZRUNRQWKH
LQGHSHQGHQWO\ ZH QRWHG D EHWWHU WHUPLQRORJLFDO ILQDO WKHVLV LW ZDV VRPHWLPHV QRW SRVVLEOH 7KHLU
H[SUHVVLRQRIWKHVWXGHQWVZKRZRUNHGZLWKVFULSWV,WZDV FODLPV ZHUH DOVR VXSSRUWHG E\ WKH IDFW WKDW ZKHQ
WKH VDPH ZKHQ SURFHVVLQJ WKH GRFXPHQWDWLRQ RI WKH SURFHVVLQJRQJRLQJWDVNVWKH\RIWHQSURFHVVHGWKHP
VHPHVWHUDVVLJQPHQWLQWKHIRUPRIDWXWRULDO,WLVREYLRXV DERYH VWDQGDUG +RZHYHU WKLV SURFHVV ZDV QRW
WKDWLIWKHPDWHULDOXVHGLQHGXFDWLRQLVLQWKHIRUPWKDWLV
V\VWHPDWLF EXW VXGGHQ ,QWHUDFWLRQ LQ
H[SHFWHG IURP WKH VWXGHQWV DQG WKHLU RXWSXW LW KDV D
SRVLWLYH HIIHFW RQ WKH VWXGHQWV :H DOVR PDQDJHG WR FRPPXQLFDWLRQ ZKLFK LV FKDUDFWHULVWLF RI WKH IDFH
GHPRQVWUDWH WKDW ZKHQ DSSO\LQJ YDULRXV IRUPV RI WRIDFHIRUPRIHGXFDWLRQZDVDEVHQWIRUKXPDQLWLHV
HGXFDWLRQLQZKLFKWKHVWXG\LVEDVHGRQWKHSULQFLSOHVRI VWXGHQWV
WKHIOLSSHGFODVVURRPHLWKHUZLWKWKHKHOSRILQVWUXFWLRQDO • 3RRU VWXGHQW WLPH PDQDJHPHQW DQG SURFUDVWLQDWLRQ
YLGHRV RU D VFULSW LW DOORZV WKH HGXFDWLRQDO JRDOV WR EH 1RQHRIWKHSDUWLFLSDQWVZDQWHGWRDGPLWWKLVIDFWLQ
VDWLVIDFWRULO\PHWHYHQLQWKHDEVHQFHRIWKHVWXGHQWIURP WKHEHJLQQLQJEXWIRUVHYHUDOLWWXUQHGRXWWREHDQ
WKHOHVVRQ REMHFWLYH IDFW $SSUR[LPDWHO\  RI WKH VWXGHQWV
RIWKHJLYHQ\HDURIDSSOLHGLQIRUPDWLFVDQGRI
,,, 5(68/76$1'',6&866,21 WKH KXPDQLWLHV VWXGHQWV ZHUH QRW DGPLWWHG WR WKH
,Q WKH ILUVW UHVHDUFK \HDU   DIIHFWHG E\ WKH VWDWHH[DPRQWLPHEHFDXVHWKH\GLGQRWKDYHWLPHWR
RQVHWRIWKHSDQGHPLFWKHWHDFKLQJRIVXEMHFWVWRRNSODFH FRPSOHWH WKHLU ILQDO ZRUN 7KLV RQO\ FRQILUPV WKH
ZLWKRXW SUREOHPV 2QO\ LQ WKH ILQDO TXHVWLRQQDLUH GLG IDFW WKDW IRU WKH HIIHFWLYH DFWLYLW\ RI VWXGHQWV LW LV
VRPHSDUWLFLSDQWVH[SUHVVWKDWWKHDPRXQWRIFRQWHQWDQG LPSRUWDQW WR VHW D VFKHGXOH IRU WKH IXOILOPHQW RI
WDVNV WKDW ZHUH LQFOXGHG LQ WKH JLYHQ VXEMHFW ZDV LQWHULP JRDOV WRJHWKHU ZLWK WKHLU FRQVFLHQWLRXV
FRQVLGHUDEOH+RZHYHUDVWKH\VWDWHGGXHWRWKHFRPSOH[ FRQWURO
KHWHURJHQHRXVHYDOXDWLRQV\VWHPLWZDVQRWDSUREOHPWR ,Q WKH FRQWH[W RIWKHVH ILQGLQJV ZH FDQ FODLP WKDW WKH
PDVWHU WKH VXEMHFW 7KLV RQH ZDV NQRZQ IURP WKH SURFHVVRIHGXFDWLRQDQGHYDOXDWLRQZDVVHWFRUUHFWO\DQG
EHJLQQLQJ RI WHDFKLQJ ,Q WKLV ZD\ VWXGHQWV FRXOG OHG WR WKH VHW JRDOV GHVSLWH WKH SURYHQ WLPHFRQVXPLQJ
GLVWULEXWH WKHLU VWUHQJWKV DQG FRQVLGHU WKHLU QDWXUH RI WKH VXEMHFW ZKLFK OHG XV WR D QHZ LGHD LQ WKH
SUHGLVSRVLWLRQV 'XULQJ WKH ILQDO JURXS LQWHUYLHZ VHYHUDO LPSOHPHQWDWLRQ RI HGXFDWLRQ :LWK D ODUJHU QXPEHU RI
SDUWLFLSDQWVFRQILUPHGWKHVHILQGLQJVEXWVWDWHGWKDWLWGLG VWXGHQWV LW ZRXOG EH YHU\ GLIILFXOW WR LPSOHPHQW VXFK D
QRWERWKHUWKHPDWDOO:HDOVRSHUFHLYHGWKHGLIILFXOW\RI VXEMHFWFRQFHSW%HIRUHWKHLPSOHPHQWDWLRQRIWKHVXEMHFW
LPSOHPHQWDWLRQ DQG HYDOXDWLRQ RI LQGLYLGXDO DFWLYLWLHV LQWKHDFDGHPLF\HDUZHUHYLHZHGWKHVXEMHFW
+RZHYHU WR SURYLGH VWXGHQWV ZLWK DV PXFK DV SRVVLEOH HYDOXDWLRQ FRQFHSW DQG DSSOLHG WKH 8QGHUVWDQGLQJ E\
ZH VWDUWHG WHDFKLQJ LQ WKH VDPH ZD\ ZLWK VOLJKW 'HVLJQŠ 8E' FRQFHSW8E'LVDIOH[LEOHIUDPHZRUNIRU
DGMXVWPHQWV EDVHG RQ WKH H[SHULHQFH RI WKH SLORW \HDU LQ GHVLJQLQJ DQG FUHDWLQJ FXUULFXOD DQG OHDUQLQJ SODQV WKDW
WKH DFDGHPLF \HDU  +RZHYHU ZH UDQ LQWR HPSKDVL]H GHHS XQGHUVWDQGLQJ DQG VXVWDLQDEOH OHDUQLQJ
FRQVLGHUDEOH SUREOHPV WKHUH ,Q WKH EHJLQQLQJ ZH FRXOG QRW MXVW VXSHUILFLDO PHPRUL]DWLRQ RI LQIRUPDWLRQ 7KLV
QRWLGHQWLI\WKHFDXVHV6WXGHQWVGLGQRWKDYHWLPHWRZRUN FRQFHSW ZDV GHYHORSHG E\ *UDQW :LJJLQV DQG -D\
RQWKHDFWLYLWLHVWKDWZHUHDVVLJQHGWRWKHPFRQWLQXRXVO\ 0F7LJKHEDFNLQWKHV7KHHQWLUH8E'IUDPHZRUNLV
,WDOWHUQDWHGGLIIHUHQWO\IURPRQHVWXGHQWWRDQRWKHUIURP RIWHQ GHSLFWHG DV EDFNZDUGV GHVLJQ VWDUWLQJ ZLWK
RQH DFWLYLW\ WR DQRWKHU $W ILUVW JODQFH ZH FRXOG QRW GHILQLQJ RXWFRPHV DQG REMHFWLYHV DQG PRYLQJ WRZDUGV
LGHQWLI\DQ\SDWWHUQLQWKLVVRZHEHJDQWRLQYHVWLJDWHDQG FUHDWLQJ WHDFKLQJ PDWHULDOV DQG DVVHVVPHQWV WKDW DFKLHYH
PRQLWRUWKHVLWXDWLRQLQGHWDLO:HLGHQWLILHGWKHSUREOHPV WKHVHREMHFWLYHV
RI WKLV VLWXDWLRQEDVHGRQ LQWHUYLHZV REVHUYDWLRQ DV ZHOO
DVLQIRUPDWLRQIURPFROOHDJXHVDQGE\VWXG\LQJH[DPSOHV 7KHEDVLFSULQFLSOHVRI8E'LQFOXGH
RIJRRGSUDFWLFH>@DVIROORZV • &ODULI\7KHILUVWVWHSLQ8E'LVWRFOHDUO\GHILQHWKH
• 7KH RQVHW RI WKH SDQGHPLF FRQILQHG SHRSOH WR WKHLU JRDOV DQG RXWFRPHV RI HGXFDWLRQ (GXFDWRUV PXVW
KRPHVDQGVWXGHQWVZHUHZLOOLQJWRVSHQGPRUHWLPH DVN ZKDW VWXGHQWV VKRXOG XQGHUVWDQGDQGEH DEOH WR
VWXG\LQJ WKDQ ZKHQ PHDVXUHV ZHUH VOLJKWO\ UHOD[HG GRDWWKHHQGRIWKHOHDUQLQJSURFHVV
$VVRPHVWDWHGWKLVVXEMHFWZDVDOVRDUHVWIRUWKHP • 'HWHUPLQH 7KH QH[W VWHS LV WR GHWHUPLQH WKH
DQGDQRSSRUWXQLW\WRFRPHXSZLWKRWKHULGHDVDQG PHWKRGV RI DVVHVVPHQW DQG HYLGHQFH WKDW ZLOO EH
WKHUHIRUH WKH\ RIWHQ GHYRWHG WLPH WR LPSURYLQJ WKH XVHGWRPHDVXUHZKHWKHUVWXGHQWVKDYHDFKLHYHGWKH
JLYHQ SURWRW\SHV ZLWK WKH DLP RI WKHLU ORQJHU VWDWHG REMHFWLYHV 7KLV LQFOXGHV FUHDWLQJ DVVHVVPHQW
SOD\DELOLW\DVZHOODVH[SDQGLQJWKHIXQFWLRQDOLW\RQ WRROVDQGDVVHVVPHQWFULWHULD
WKHLURZQLQLWLDWLYH • 'HVLJQ ,Q WKLV VWHS HGXFDWRUV GHVLJQ WHDFKLQJ
• 7KH FRQVWDQWO\ FKDQJLQJ FRQGLWLRQV FDXVHG E\ WKH DFWLYLWLHVOHVVRQVDQGPDWHULDOVWKDWZLOOOHDGWRWKH
RQJRLQJSDQGHPLF>@WKHWUDQVLWLRQIURPDQRQOLQH DFKLHYHPHQWRIWKHVHWJRDOV7KHVHDFWLYLWLHVVKRXOG
HQYLURQPHQWWRDIDFHWRIDFHHQYLURQPHQWDQGEDFN DLP WR SURPRWH GHHS XQGHUVWDQGLQJ DQG VNLOOV >
WKH FRQVWDQW UHRUJDQL]DWLRQ RI WKH HGXFDWLRQDO @
SURFHVV WKH DVVRFLDWHG DGDSWDWLRQ RI VWXGHQWV  :HSUHFLVHO\VSHFLILHGWKHHGXFDWLRQDOJRDOVEDVHGRQ
H[WHUQDODFWLYLWLHV DODUJHUSHUFHQWDJHRIJUDGXDWLQJ ZKLFK ZH PRGLILHG WKH HYDOXDWLRQ PHWKRG )URP WKH
VWXGHQWVDOVRZRUNLQDGGLWLRQWRWKHLUVWXGLHV PHDQW DFDGHPLF \HDU  ZH DSSO\ D K\EULG PHWKRG RI
HGXFDWLRQ :H VWLOO XVH D IOLSSHG FODVVURRP DSSURDFK WR
WKDW VWXGHQWVGLG QRW KDYH WLPH RU SXUSRVHIXOO\ GLG
PHHW RQJRLQJ OHDUQLQJ JRDOV LQ WKH FRQWH[W RI JDPH
QRWZDQWWRGHYRWHDVPXFKWLPHRIWKHLVVXH6HYHUDO SURWRW\SH GHYHORSPHQW ZKHUH VWXGHQWV FDQ XVH YLGHRV
RIWKHPH[SUHVVHGWKDWWKH\ZRXOGEHZLOOLQJEXWLQ HOHDUQLQJPDWHULDOVRUVFULSWVWRFRPSOHWHWDVNVDFFRUGLQJ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 261

WR WKHLU SUHIHUHQFHV $PRQJ RWKHU WKLQJV ZH SUHVHQW WKH 7KH VWXGHQWV WKHPVHOYHV DUH DOVR LQYROYHG LQ WKH
GHYHORSHG JDPH SURWRW\SHV WR WKH VWXGHQWV LQ WKH ILUVW HYDOXDWLRQ SURFHVV HVSHFLDOO\ WKH FUHDWLYH SDUWV LQ RUGHU
OHVVRQ DQG DOORZ WKHP WR FKRRVH ZKLFK SURWRW\SH WKH\ WRREMHFWLI\WKHHYDOXDWLRQRIWKHVHRIWHQVXEMHFWLYHSDUWV
ZDQWWRSD\DWWHQWLRQWRGXULQJWKHVHPHVWHU7KHVWXGHQW %DVHG RQ WKH UHVXOWV ZH DUULYHG DW RQ WKH EDVLV RI
WKXV FRQFHQWUDWHV KLV HIIRUWV RQO\ RQ WKH GHYHORSPHQW RI TXDOLWDWLYH UHVHDUFK ZH DOVR IRUPXODWHG D FRPSDUDWLYH
RQH SURWRW\SH RI WKH JDPH 6LQFH HDFK JDPH V DQDO\VLV RI WKH GLIIHUHQFHV LQ WKH SHUFHSWLRQ RI WKH
GHYHORSPHQWGHSHQGVRQLWVFKDUDFWHUQRWDOOVWXGHQWVZLOO DSSOLFDWLRQ RI RSHQERRN H[DPV LQ DSSOLHG LQIRUPDWLFV
HQFRXQWHU DOO WKH JDPH PHFKDQLFV WKDW DUH GHYHORSHG DQG LQ KXPDQLWLHV VXEMHFWV $FFRUGLQJ WR UHIHUHQFH >@
LQGLYLGXDOO\ LQ WKHVH JDPHV 6LPLODUO\ LQ WKH (QJOLVK LQ RSHQERRN H[DPV LQ WHFKQLFDO DQG KXPDQLWLHV VWXGLHV
6FLHQFHDQG7HFKQRORJ\VXEMHFWVWXGHQWVZLOOUHFHLYHDQ VKDUH VRPH VLPLODULWLHV EXW WKHUH DUH DOVR VLJQLILFDQW
H[DFW VFKHGXOH DQG VWUDWHJ\ IRU HYDOXDWLQJ LQGLYLGXDO GLIIHUHQFHVLQKRZWKH\DUHVWUXFWXUHGDQGDVVHVVHGGXHWR
RXWSXWVULJKWDWWKHILUVWVHPLQDU+RZHYHUVRWKDWWKH\GR WKH QDWXUH RI WKH VXEMHFWV +HUH V D FRPSDULVRQ RI RSHQ
QRWORVHWKLVNQRZOHGJHRQHRIWKHVWXGHQWVLVGHVLJQDWHG ERRNH[DPVLQERWKILHOGVEDVHGDQGIRUPXODWHGXSRQRXU
HYHU\ ZHHN WR SUHVHQW DQG FODULI\ WKH IDFWV 7KH UHVHDUFK
SUHVHQWDWLRQ RI WKHLU UHVXOWV LV FKHFNHG HYDOXDWHG 6LPLODULWLHV
H[SODLQHG LQ GHWDLO RU H[SDQGHG GLUHFWO\ GXULQJ IDFHWR
IDFHWHDFKLQJ7KHRWKHUVWXGHQWVZKRDUHLQYROYHGLQWKH  $FFHVVWR5HIHUHQFH0DWHULDOV,QERWKWHFKQLFDO
GHYHORSPHQWRIWKHSUHVHQWHGSURWRW\SHDLPWRFKHFNWKH DQG KXPDQLWLHV RSHQERRN H[DPV VWXGHQWV DUH
IDFWVSUHVHQWHGE\WKLVVWXGHQWDQGLIQHFHVVDU\WRFRUUHFW DOORZHG WR XVH WH[WERRNV QRWHV DQG RWKHU
RU VXSSOHPHQW WKHP ZKLFK OHDGV WR SURIHVVLRQDO UHIHUHQFH PDWHULDOV GXULQJ WKH WHVW 7KLV DFFHVV LV
GLVFXVVLRQV :KHQ FKRRVLQJ D GLIIHUHQW VROXWLRQ WR WKH LQWHQGHG WR HQFRXUDJH WKH DSSOLFDWLRQ RI
VDPH SUREOHP WKH\ FDQ DFWLYHO\ SDUWLFLSDWH LQ WKH NQRZOHGJHUDWKHUWKDQURWHPHPRUL]DWLRQ
SUHVHQWDWLRQDQGWDNHRYHUWKHUROHRIWKHSUHVHQWHU7KXV  (PSKDVLV RQ &ULWLFDO 7KLQNLQJ 2SHQERRN
WKH SHGDJRJXH FDQ DFW DV D IDFLOLWDWRU DQG PRGHUDWH H[DPVLQERWKILHOGVRIWHQSODFHDVWURQJHPSKDVLV
FRQWURO JXLGH VXSSOHPHQW DQG H[SDQG WKLV SURFHVV 7KH RQ FULWLFDO WKLQNLQJ DQG SUREOHPVROYLQJ VNLOOV
RWKHUVWXGHQWVZKRGHYHORSWKHVHFRQGSURWRW\SHDUHWKXV 6WXGHQWV DUH H[SHFWHG WR DQDO\]H DQG V\QWKHVL]H
QRWGHSULYHGRIDQ\IDFWVDQGKDYHDGHWDLOHGRYHUYLHZRI LQIRUPDWLRQ IURP WKHLU UHIHUHQFH PDWHULDOV WR
ZKDW LV LQYROYHG LQ WKH GHYHORSPHQW RI WKH JDPH DQVZHUFRPSOH[TXHVWLRQV
SURWRW\SH ZKLFK WKH\ DUH QRW LQYROYHG LQ 7KH\ FDQ DVN  5HGXFHG6WUHVV2SHQERRNH[DPVFDQSRWHQWLDOO\
WKHLUFODVVPDWHVZKDWLQWHUHVWVWKHPLQWKHFRQWH[WRIWKH UHGXFH WHVWUHODWHG VWUHVV DQG DQ[LHW\ FRPSDUHG WR
FRQWHQW 7KLV LV KRZ WKH SUHVHQWDWLRQ RI ERWK SURWRW\SHV FORVHGERRNH[DPVVLQFHVWXGHQWVKDYHWKHFRPIRUW
DOWHUQDWHVHYHU\ZHHN RIUHIHUHQFHPDWHULDOV
2WKHUHGXFDWLRQDODFWLYLWLHV VROXWLRQDQGHYDOXDWLRQRI  5HDO:RUOG 6LPXODWLRQ %RWK WHFKQLFDO DQG
SURIHVVLRQDOO\ RULHQWHG WDVNV VHPHVWHU DVVLJQPHQWV DQG KXPDQLWLHVILHOGVUHFRJQL]HWKHYDOXHRIVLPXODWLQJ
HYDOXDWLRQ RI VWXGHQWV¶ SURIHVVLRQDO DFWLYLW\  DUH FDUULHG UHDOZRUOGVFHQDULRV,QPDQ\SURIHVVLRQDOVHWWLQJV
RXW LQ WKH VDPH ZD\ DV LW ZDV LPSOHPHQWHG LQ SUHYLRXV LQGLYLGXDOVKDYHDFFHVVWRUHIHUHQFHPDWHULDOVDQG
\HDUV :LWKLQ WKH VHPHVWHU DVVLJQPHQW ZKLFK IRUPV WKH RSHQERRNH[DPVPLPLFWKLVVLWXDWLRQ
PDLQ SDUW RI WKHLU HYDOXDWLRQ WZR DUHDV DUH VSHFLILHG 'LIIHUHQFHV
IXQFWLRQDO DQG JUDSKLF SDUW 7KH SRLQW YDOXH RI WKH WDVNV
WKDWEHORQJWRWKHIXQFWLRQDOSDUWLVKLJKHUWKDQWKDWRIWKH  1DWXUHRI5HIHUHQFH0DWHULDOV
WDVNVIURPWKHJUDSKLFSDUW7KHWDVNVDUHGHVLJQHGIUHHO\ • 7HFKQLFDO6WXGLHV,QWHFKQLFDORSHQERRNH[DPV
ZLWKVWULFWO\GHILQHGUXOHVDQGSRLQWHYDOXDWLRQVRWKDWWKH HJ HQJLQHHULQJ VFLHQFH PDWKHPDWLFV 
VWXGHQWVNQRZZKDWFRQFHSWVJDPHPHFKDQLVPVDQGZKDW UHIHUHQFH PDWHULDOV W\SLFDOO\ LQFOXGH WH[WERRNV
OHYHO RI TXDOLW\ LV H[SHFWHG RI WKHP ZKLOH PDLQWDLQLQJ OHFWXUH QRWHV IRUPXOD VKHHWV DQG VSHFLILF GDWD
DEVROXWHIUHHGRPRIFUHDWLYHSURFHVVLQJDQGFRQFHSW7KH WDEOHV7KHVHH[DPVRIWHQUHTXLUHWKHDSSOLFDWLRQ
DVVLJQPHQW DOVR LQFOXGHVERQXV WDVNV WKDW DIWHU D PXWXDO RIPDWKHPDWLFDODQGVFLHQWLILFSULQFLSOHVWRVROYH
GLVFXVVLRQ EHWZHHQ VWXGHQWV DQG WKH WHDFKHU FDQ UHSODFH SUREOHPV
VRPH WDVNV IURP WKH IXQFWLRQDO RU JUDSKLF SDUW ,Q • +XPDQLWLHV 6WXGLHV ,Q KXPDQLWLHV RSHQERRN
LQGLYLGXDO FDVHV D FRPSOHWH FKDQJH RI WKH VHPHVWHU H[DPV HJ OLWHUDWXUH KLVWRU\ SKLORVRSK\ 
DVVLJQPHQW PD\ DOVR RFFXU 7KLV KDSSHQV HVSHFLDOO\ LQ UHIHUHQFH PDWHULDOV FDQ EH EURDGHU DQG PD\
FDVHV ZKHUH VWXGHQWV DOUHDG\ KDYH H[SHULHQFH ZLWK JDPH HQFRPSDVVDUDQJHRIWH[WVDUWLFOHVDQGHVVD\V
GHYHORSPHQW DQG ZDQW WR IRFXV RQ PRUH DGYDQFHG JDPH 6WXGHQWV DUH H[SHFWHG WR DQDO\]H DQG LQWHUSUHW
FRQFHSWV6RIDUKRZHYHURQO\RQHVWXGHQWKDVXVHGWKLV LQIRUPDWLRQ GHYHORS DUJXPHQWV DQG VXSSRUW
RSWLRQ 3UHVHQWDWLRQ RI WKH UHVXOWV RI ZRUN RQ LQGLYLGXDO WKHLU DQVZHUV ZLWK HYLGHQFH IURP WKH SURYLGHG
WDVNVDQGVHPHVWHUDVVLJQPHQWVWDNHVSODFHH[FOXVLYHO\LQ WH[WV
SHUVRQ,WKDVEHHQSURYHQWKDWVXFKSUHVHQWDWLRQVOHDGWR
 4XHVWLRQ7\SHV
PRUH ODVWLQJ NQRZOHGJH IRU WKH SUHVHQWLQJ VWXGHQWV DQG
DUHDVWLPXODWLQJVRXUFHRILQIRUPDWLRQIRURWKHUV3DUWRI • 7HFKQLFDO 6WXGLHV 4XHVWLRQV LQ WHFKQLFDO RSHQ
WKH VHPHVWHU DVVLJQPHQW LV WKH SURFHVVLQJ RI ERRN H[DPV RIWHQ LQYROYH VROYLQJ SUREOHPV
GRFXPHQWDWLRQLQWKHIRUPRIDWXWRULDO7KLVLVHYDOXDWHG FRQGXFWLQJ FDOFXODWLRQV DQG DSSO\LQJ VSHFLILF
E\WKHWHDFKHUHYHQEHIRUHWKHSUHVHQWDWLRQVRWKDWKHFDQ PHWKRGRORJLHV 7KH\ UHTXLUH D GHHS
HIIHFWLYHO\ PRGHUDWH WKH SUHVHQWDWLRQ LWVHOI 7KH IRUP RI XQGHUVWDQGLQJ RI WHFKQLFDO FRQFHSWV DQG WKHLU
WKH SUHVHQWDWLRQ WKH ZD\ RI XVLQJ WKH SUHSDUHG PDWHULDOV SUDFWLFDODSSOLFDWLRQ
RU WKH FRPSOHWLRQ RI QHZ RQHV IRU WKH SXUSRVH RI WKH • +XPDQLWLHV 6WXGLHV 4XHVWLRQV LQ KXPDQLWLHV
SUHVHQWDWLRQLVIXOO\ZLWKLQWKHFRPSHWHQFHRIWKHVWXGHQW RSHQERRN H[DPV PD\ UHTXLUH FULWLFDO DQDO\VLV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 262

 LQWHUSUHWDWLRQHVVD\ZULWLQJDQGWKHV\QWKHVLVRI >@ 6]ĘN|O , DQG , 3ãHQiNRYi &RQWLQXDO ,PSURYHPHQW RI WKH
LQIRUPDWLRQ IURP YDULRXV VRXUFHV 6WXGHQWV DUH (GXFDWLRQDO 3URFHVV $G $OWD -RXUQDO RI ,QWHUGLVFLSOLQDU\
5HVHDUFK
DVVHVVHGRQWKHLUDELOLW\WRWKLQNFULWLFDOO\DUJXH
>@ $GDPHF 3 DQG 5 +UPR (QWUHSUHQHXUVKLS (GXFDWLRQ RI
SHUVXDVLYHO\DQGHQJDJHZLWKFRPSOH[LGHDV 8QLYHUVLW\ 6WXGHQWV([DPSOHV RI *RRG 3UDFWLFH 7(0 -RXUQDO
 7LPH0DQDJHPHQW   
• 7HFKQLFDO 6WXGLHV 7LPH PDQDJHPHQW LV FUXFLDO >@ $QD\D / (YDQJHORSRXORV 1 DQG 8 /DZDQL 2SHQERRN YV
LQ WHFKQLFDO RSHQERRN H[DPV DV VWXGHQWV QHHG FORVHGERRN WHVWLQJ $Q H[SHULPHQWDO FRPSDULVRQ $6(( $QQXDO
&RQIHUHQFH DQG ([SRVLWLRQ &RQIHUHQFH 3URFHHGLQJV /RXLVYLOOH
WRHIILFLHQWO\QDYLJDWHUHIHUHQFHPDWHULDOVWRILQG .HQWXFN\,661GRL²
UHOHYDQW IRUPXODV DQG FRQFHSWV 6ROYLQJ >@ 6PHNDORYD / &KDORXSNRYD 3 1HPHMF . HW DO 6DWLVIDFWLRQ
SUREOHPV ZLWKLQ WLPH FRQVWUDLQWV FDQ EH ZLWKDFTXLUHGWUDQVIHUDEOHFRPSHWHQFHVDPRQJXQLYHUVLW\VWXGHQWV
FKDOOHQJLQJ LQ &DPERGLD $VLD 3DFLILF (GXFDWLRQ 5HYLHZ 
KWWSVGRLRUJV
,9 &21&/86,21 >@ /DP 5 :ULWLQJ SRUWIROLR DVVHVVPHQW LQ SUDFWLFH LQGLYLGXDO
LQVWLWXWLRQDO DQG V\VWHPLF LVVXHV 3HGDJRJLHV $Q ,QWHUQDWLRQDO
7KH JRDO RI WKH UHVHDUFK ZDV WR LQWURGXFH DQ -RXUQDO   
H[DPLQDWLRQ LQ VXFK D IRUP WKDW VWXGHQWV FDQ XVH DQ\ KWWSVGRLRUJ;
OLWHUDWXUH QRWHV RU DLGV RSHQERRN H[DP  FRPSDUHG WR >@ %DUQRYi6HWDOâNROVNiSHGDJRJLND'XEQLFDQDG9iKRP9â
WKH WUDGLWLRQDO DVVHVVPHQW DSSURDFK FORVHGERRN H[DP  '7,,6%1
7KH FRQFOXVLRQVRI VRPH DXWKRUV> @SRLQW WR WKH >@ 6WUX\YHQ.HWDO7KHRYHUDOOHIIHFWVRIHQGRIFRXUVHDVVHVVPHQW
IDFWWKDWWKHUHZDVQRVLJQLILFDQWGLIIHUHQFHLQWKHREWDLQHG RQ VWXGHQW SHUIRUPDQFH D FRPSDULVRQ EHWZHHQ PXOWLSOH FKRLFH
HYDOXDWLRQ VFRUH GXH WR WKH DSSOLHG IRUP RI VWXGHQW WHVWLQJ SHHU DVVHVVPHQW FDVHEDVHG DVVHVPHWQ DQG SRUWIROLR
DVVHVVPHQW ,Q 6WXGLHV LQ (GXFDWLRQDO (YDOXDWLRQ 9ROXPH 
HYDOXDWLRQ5HIHUHQFH>@DOVRGRHVQRWUHSRUWVLJQLILFDQW ,VVXH   SS  ,661 ;
GLIIHUHQFHVLQWKHFRQWH[WRIGHYHORSLQJFULWLFDOWKLQNLQJLQ KWWSVGRLRUJMVWXHGXF
QXUVLQJ VWXGHQWV 7KH\ FRQVLGHU WKH DGYDQWDJHV RI >@ %HWK-'LQNHQV$DQG-0RRUH$V\VWHPDWLFUHYLHZFRPSDULQJ
LQGLYLGXDOIRUPVRIDVVHVVPHQWWREHFRPSDUDEOH:HDOVR RSHQERRN DQG FORVHGERRN H[DPLQDWLRQV (YDOXDWLQJ HIIHFWV RQ
PDQDJHGWRSURYHWKDW6WXGHQWHYDOXDWLRQVGLGQRWVKRZD GHYHORSPHQW RI FULWLFDO WKLQNLQJ VNLOOV 1XUVH (GXFDWLRQ LQ
VLJQLILFDQW GLIIHUHQFH +RZHYHU DV UHIHUHQFH >@ VKRZV 3UDFWLFH 9ROXPH   3DJHV  ,661 
WKHUHZDVDFRQVLGHUDEOHGLIIHUHQFHLQWKHDQ[LHW\VFRUHV KWWSVGRLRUJMQHSU
7KH ILQGLQJ VKRZV WKDW WKRVH ZKR DSSHDUHG LQ WKH H[DP >@ %HĖR 0 DQG 0 gOYHFNê 7KH DSSOLFDWLRQ RI NQRZOHGJH LQ
W\SLFDOSUREOHPVLWXDWLRQVXVLQJFRPSXWHU,Q,&(7$WK
ZLWKRXW ERRNV VKRZHG PRUH DQ[LHW\ WKDQ WKRVH ZKR ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ
DSSHDUHG LQ WKH H[DP ZLWK ERRNV 6LPLODU ILQGLQJV ZHUH 7HFKQRORJLHVDQG$SSOLFDWLRQVSURFHHGLQJV'HQYHU,QVWLWXWHRI
DOVRGHPRQVWUDWHGLQRXUUHVHDUFK$VWKHELJJHVWSRVLWLYH (OHFWULFDO DQG (OHFWURQLFV (QJLQHHUV  ,6%1 
RIWKHVHWHYDOXDWLRQPHWKRGZHHYDOXDWHWKHSRVVLELOLW\RI SS
YHULI\LQJ KLJKHU OHYHOV RI FRJQLWLYH JRDOV DFFRUGLQJ WR >@ -XULQRYi-DQG-0LãWLQD'HYHORSPHQWRIDGHVNWRSDSSOLFDWLRQ
%ORRP V WD[RQRP\ ZKLFK ZRXOG QRW EH IHDVLEOH ZLWKRXW IRU D FRPSOH[ KHWHURJHQHRXV HYDOXDWLRQ V\VWHP ,Q ,&(7$ 
VXFK D PHWKRG RI VHWWLQJ WKH VXEMHFW DQG LWV HYDOXDWLRQ  WK $QQLYHUVDU\ RI ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ
(PHUJLQJH/HDUQLQJ7HFKQRORJLHVDQG$SSOLFDWLRQVSURFHHGLQJV
7KH VWXGHQWV DUH IDFHG ZLWK WKH VROXWLRQ RI YDULRXV UHDO  2FWREHU  ±   *UDQG +RWHO 6WDUê 6PRNRYHF +LJK
SUREOHPV IURP SUDFWLFH IURP YDULRXV WKHPDWLF DUHDV 7DWUDV 6ORYDNLD 'DQYHU  ,QVWLWXWH RI (OHFWULFDO DQG (OHFWURQLFV
ZKLFK UHTXLUH QRW RQO\ WKH SURIHVVLRQDO NQRZOHGJH DQG (QJLQHHUV,6%1SS
VNLOOV RI VWXGHQWV EXW DOVR D ORQJHU WLPH VSDQ IRU WKHLU >@ -XULQRYi - 9\XåLWLH GLJLWiOQ\FK WHFKQROyJLt D GLJLWiOQHKR
VROXWLRQDVZHOODVHYDOXDWLRQE\WKHWHDFKHUVWKHPVHOYHV HGXNDþQpKR REVDKX Y WHFKQLFNRP Y]GHOiYDQt 'XEQLFD QDG
/DVW EXW QRW OHDVW LW HQDEOHV D PRUH FRPSUHKHQVLYH 9iKRPSS+DELOLWDþQiSUiFD9\VRNiãNROD'7,
DVVHVVPHQWRIWKHVWXGHQW >@ $QGUHV 3 'REURYVNi ' 9DQČþHN ' DQG - 0LãWLQD 7KH
,PSDFW RI WKH 3DQGHPLF &ULVLV RQ 7HFKQRORJ\ 6WDQGDUG RI
7UDGLWLRQDO 8QLYHUVLW\ (GXFDWLRQ ,Q ,&/   0RELOLW\ IRU
$&.12:/('*0(17 6PDUW &LWLHVDQG5HJLRQDO 'HYHORSPHQW  &KDOOHQJHV IRU +LJKHU
7KH SDSHU ZDV IXQGHG E\ WKH JUDQW .(*$ 8&0 (GXFDWLRQ /HFWXUH 1RWHV LQ 1HWZRUNV DQG 6\VWHPV YRO 
 'HYHORSPHQW RI DQ LQWHUDFWLYH HFRXUVH XVLQJ 6SULQJHU &KDP  KWWSVGRLRUJ
B
³60$57´ WHFKQRORJLHV IRU WKH GHYHORSPHQW RI
>@ 6LPRQRYD , )DOW\QNRYD / .RVWRODQ\RYD . 1HZ %OHQGHG
DOJRULWKPLFWKLQNLQJDQGSURJUDPPLQJVNLOOV /HDUQLQJ (QULFKHG DIWHU WKH &29,' ([SHULHQFH" 6WXGHQWV¶
2SLQLRQV 6XVWDLQDELOLW\    
5()(5(1&(6 KWWSVGRLRUJVX
>@ 3RQGHOtNRYi,DQG77|N|O\RYi:RUN)URP+RPHDQG6WUHVVLQ >@ 0F7LJKH%-DQG*:LJJLQV  8QGHUVWDQLQJE\GHVLJQŠ
WKH6ORYDN$FDGHPLF(QYLURQPHQW,Q+DQGERRNRI5HVHDUFKRQ )UDPHZRUN >RQOLQH@  >FLW @ $YDLODEOH
WKH&RPSOH[LWLHVDQG6WUDWHJLHVRI2FFXSDWLRQDO6WUHVV+HUVKH\ DWKWWSVZZZJRRJOHFRPXUO"VD W UFW M T HVUF V VRXUFH
,*, *OREDO  $GYDQFHV LQ +XPDQ 5HVRXUFHV 0DQDJHPHQW ZHE FG FDG UMD XDFW  YHG DK8.(ZM<L3M6RX%$[9TBE
DQG2UJDQL]DWLRQDO'HYHORSPHQW $+502' %RRN6HULHV,661 V,+8HI'QV4)QR(&%04$4 XUO KWWSV$))ILOHVDVFGR
  ,6%1  V  UJ)VWDWLFILOHV)DVFG)SGI)VLWH$6&')SXEOLFDWLRQV
KWWSVGRLRUJFK )8E'B:KLWH3DSHUSGI XVJ $2Y9DZP267ZJ5
>@ $UPVWURQJ 3 %ORRP¶V 7D[RQRP\ 9DQGHUELOW 8QLYHUVLW\ &HQWHU HJ+6FXJR0 RSL 
IRU7HDFKLQJ$YDLODEOHDWKWWSVFIWYDQGHUELOWHGXJXLGHV >@ :LJJLQJV * 3 DQG - 0F7LJKH 8QGHUVWDQGLQJ E\ 'HVLJQ QG
VXESDJHVEORRPVWD[RQRP\ ([SDQGHG (GLWLRQ  $VVRFLDWLRQ IRU 6XSHUYLVLRQ  &XUULFXOXP
>@ 1LHPLHUNR % 'LDJQRVW\ND HGXNDF\MQD 3RGUĊF]QLN DNDGHPLFNL 'HYHORSPHQW,6%1
(GXFDWLRQDOGLDJQRVWLFV$WH[WERRN@3:1:DUV]DZD >@ 9\DV * DQG - 9\DV $ FRPSDUDWLYH VWXG\ RI RSHQ ERRN H[DP
DQGFORVHGERRNH[DP,Q6KRGK6DPLNVKD$XU0XO\DQNDQ9RO
,,,VVXH,661


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 263

 1HWZRUN6\VWHP+HDOWKFKHFN

'DOLERU.DIND-DURVODY6D[D3DYHO6HJHþ$GULiQ6WUDND7RPiã-XUtN
)DFXOW\RI0DQDJHPHQW6FLHQFHDQG,QIRUPDWLFV8QLYHUVLW\RI=LOLQD8QLYHU]LWQD=LOLQD
HPDLO^GDOLERUNDINDSDYHOVHJHF`#IULXQL]DVN^MDURVD[DVWUDNDDGULDQMH\WHH`#JPDLOFRP

$EVWUDFW² :LWK WKH GHYHORSPHQW RI LQIRUPDWLRQ IXQFWLRQLQJ FRUUHFWO\ LW GRHVQ W PHDQ LW LV DSSURSULDWHO\
FRPPXQLFDWLRQ DQG FORXG WHFKQRORJLHV DQG WKHLU FRQILJXUHG:KDW VPRUHDGPLQLVWUDWRURXJKWWREHVXUH
FRQYHUJHQFH WRZDUGV JOREDO FRQQHFWLYLW\ TXHVWLRQV DULVH WKDWFRQILJXUDWLRQLVDFFHSWHGDQGGHYLFHVLVXVLQJLW)RU
UHJDUGLQJWKHLUHIILFLHQWPDQDJHPHQWHVSHFLDOO\FRQFHUQLQJ H[DPSOHYDULRXVSURWRFROVDQGVHUYLFHVWKDWDUHQRWXVHG
WKH VXVWDLQDELOLW\ RI FRQVLVWHQW DGPLQLVWUDWLYH DQG VHFXULW\
PD\ EH HQDEOHG RQ LW 2U XVLQJ SDVVZRUGV EXW ZLWKRXW
SROLFLHV 6LJQLILFDQW SURJUHVV FDQ FHUWDLQO\ EH H[SHFWHG LQ
WKH DGRSWLRQ RI VRIWZDUHGHILQHG DSSURDFKHV 6'1  LQ WKLV VXIILFLHQW FRPSOH[LW\ $OO WKHVH DWWDFNHUV FDQ XVH LW WR
ILHOG +RZHYHU JLYHQ WKH YDVW QXPEHU RI OHJDF\ QHWZRUN WKHLU DGYDQWDJH 6LQFH QHWZRUN DGPLQLVWUDWRUV DUH RQO\
VROXWLRQVVWLOOLQRSHUDWLRQZLWKOLPLWHGRUQR6'1VXSSRUW KXPDQ VRPH LQDGYHUWHQW HUURUV FDQ RFFXU HVSHFLDOO\
WKHLU HIIHFWLYH PDQDJHPHQW DQG FRQWURO UHPDLQ D FKDOOHQJH ZLWK PDQ\ GHYLFHV 7KHUHIRUH LW LV LPSRUWDQW WR XVH
IRU FXUUHQW ,&7 VROXWLRQ DGPLQLVWUDWRUV 7KH SUHVHQWHG YDULRXV VRSKLVWLFDWHG WRROV WKDW ZLOO PDNH WKHLU ZRUN
DUWLFOH LQWURGXFHV D VROXWLRQ QDPHG 1HWZRUN 6\VWHP HDVLHURQWKHRQHKDQGDQGVDYHWKHPWLPHRQWKHRWKHU
+HDOWKFKHFN 16+  WKDW VLPSOLILHV WKH YHULILFDWLRQ RI 7KH QHHG IRU UHPRWH GHYLFH FRQILJXUDWLRQ LV KXJH
QHWZRUNGHYLFHVE\DXWRPDWLQJWKHDQDO\VLVRIWKHLUFXUUHQW WKHVHGD\V&RQVLGHULQJWKDWPRVWGHYLFHVDUHLQGLIIHUHQW
FRQILJXUDWLRQ EDVHG RQ WKH XVHU¶V SUHGHILQHG LQSXWV :H
WKHQORDGDQGSURFHVVLQSXWGDWDDERXWQHWZRUNGHYLFHVDQG
SDUWVRIWKHZRUOGWKHLUUHPRWHFRQILJXUDWLRQLVRIWHQWKH
FRPPDQG RXWSXWV H[HFXWHG RQ WKHP $V D UHVXOW RI RQO\ VROXWLRQ >@ $W WKH WLPH WKLV SURJUDP ZDV FUHDWHG
FRQILJXUDWLRQ DVVHVVPHQW WKH VROXWLRQ SURYLGHV SRVLWLYH WKHZRUOGZDVLQDJOREDOSDQGHPLF,WDOVRSXVKHGXVWR
IHHGEDFN RU UHSRUWV LGHQWLILHG GHILFLHQFLHV WKDW WKH WKH QHHG IRU UHPRWH GHYLFH FRQILJXUDWLRQ DV PRVW
DGPLQLVWUDWRU FDQ VXEVHTXHQWO\ FROOHFWLYHO\ DGMXVW XVLQJ HPSOR\HHVZHUHIRUFHGWRZRUNIURPKRPH
SUHGHILQHGFRPPDQGV7KHVROXWLRQVXSSRUWVVHFXUHUHPRWH ,Q WKLV DUWLFOH ZH SUHVHQW WKH RXWFRPH RI VXFFHVVIXO
FRPPXQLFDWLRQ ZLWK KXQGUHGV RI YDULRXV QHWZRUN GHYLFHV SURMHFWEDVHG OHDUQLQJ DW WKH )DFXOW\ RI 0DQDJHPHQW
GHSOR\HGRQWKHQHWZRUN7KHVROXWLRQZDVFUHDWHGZLWKWKH 6FLHQFH DQG ,QIRUPDWLFV DW 81,=$ ZKHUH WKH UHVXOW LV
DVVLVWDQFH DQG DFFRUGLQJ WR WKH UHTXLUHPHQWV RI QHWZRUN WKHGHYHORSPHQWDQGLPSURYHPHQWRIDVRIWZDUHVROXWLRQ
HQJLQHHUV
IRU DXWRPDWHG UHPRWH FRQILJXUDWLRQ VHWXS YHULILFDWLRQ
.H\ZRUGV²1HWZRUN GHYLFHV 5HPRWH FRPPXQLFDWLRQ DQGFRQWURORIQXPHURXVQHWZRUNGHYLFHVZLWKWKHDLPRI
([SHFW66+UHJH[S\WKRQ1HWZRUN6\VWHP+HDOWKFKHFN DFKLHYLQJ RSHUDWLRQDO DQG VHFXULW\ FRQILJXUDWLRQ
FRQVLVWHQF\ LQ OLQH ZLWK HVWDEOLVKHG FRUSRUDWH SROLFLHV
:HDOVRVKRZFDVHWKHGHFLVLRQVPDGHGXULQJLWVFUHDWLRQ
, ,1752'8&7,21 DQGWKHUHDVRQEHKLQGDFKLHYLQJVXFKDUHVXOW
&KHFNLQJWKHFRQILJXUDWLRQRI PDQ\ QHWZRUNGHYLFHV
LV PRUH DQG PRUH FRPSOH[ 7KHUHIRUH YDULRXV ,, 63(&,),&$7,212)352*5$05(48,5(0(176
WHFKQRORJLHVVHUYLFHVRUDSSOLFDWLRQV KDYHDOUHDG\EHHQ 7R EH DEOH WR FUHDWH D VRIWZDUH DSSOLFDWLRQ DFFRUGLQJ
FUHDWHG WKDW KHOS DGPLQLVWUDWRUV ZLWK WKHLU PDQDJHPHQW WR WKH FXVWRPHU V ZLVKHV ZH QHHG WR VSHFLI\ WKH
DQGYHULILFDWLRQ>@ UHTXLUHPHQWV >@ 7KLV LV D YHU\ LPSRUWDQW VWHS LQ WKH
0DQ\DSSOLFDWLRQVRQWKH PDUNHWDOORZ \RXWRYHULI\ FUHDWLRQ RI WKH VRIWZDUH LWVHOI 3RRUO\ VSHFLILHG
WKH FRUUHFW IXQFWLRQDOLW\ RI QHWZRUN GHYLFHV LQ D FHUWDLQ UHTXLUHPHQWV FDQ OHDG WR LQFRQVLVWHQFLHV LQ ZKDW WKH
ZD\EXWHDFKRIWKHPLVRULHQWHGLQDGLIIHUHQWGLUHFWLRQ FXVWRPHUH[SHFWVIURPWKHILQLVKHGSURGXFWDQGZKDWZH
ZKHQLWFRPHVWRFKHFNLQJGHYLFHV>@ ZLOO RIIHU KLP DV D UHVXOW 5HTXLUHPHQWV DUH REWDLQHG
)RUWKLVUHDVRQZHGHFLGHGWRGHYHORSRXUVROXWLRQDV WKURXJK PXWXDO FRPPXQLFDWLRQ EHWZHHQ WKH VRIWZDUH
D GHGLFDWHG DSSOLFDWLRQ WKH PDLQ JRDO RI ZKLFK ZDV WR DSSOLFDQW DQG WKH VRIWZDUH GHYHORSHU ,W LV QHFHVVDU\ WR
KDYH D VPDOO VL]H VLPSOH XVDELOLW\ DQG DSSOLFDELOLW\ QRW FRYHU DOO DVSHFWV DQG H[SHFWHG IXQFWLRQDOLWLHV $
RQO\LQDVFKRROHQYLURQPHQWEXWSULPDULO\HIILFLHQF\IRU GRFXPHQW WKDW FRQWDLQV VRIWZDUH UHTXLUHPHQWV LV D IRUP
ODUJHFRUSRUDWHXVH$PRQJWKHPDLQUHDVRQVIRUFUHDWLQJ RIZDUUDQW\IRUERWKSDUWLHV
VXFK DQ DSSOLFDWLRQ LV WKH HYHULQFUHDVLQJ QXPEHU RI 5HTXLUHPHQWVDUHGLYLGHGLQWRWZRPDLQFDWHJRULHVWKH
GHYLFHVRQWKHQHWZRUNWKDWQHWZRUNDGPLQLVWUDWRUVQHHG IXQFWLRQDODQGWKHQRQIXQFWLRQDOUHTXLUHPHQWV
WR FRQWURO &KHFNLQJ QHWZRUNLQJ GHYLFHV GRHV QRW PHDQ
$ )XQFWLRQDOUHTXLUHPHQWV
RQO\ D YLVXDO FKHFN WR VHH LI WKHUH DUH DQ\ PHFKDQLFDO
GDPDJHV EXW PDLQO\ UHDOWLPH UXQQLQJ VHWXS QHJRWLDWHG 2XU IXQFWLRQDO UHTXLUHPHQWV LQFOXGH IHDWXUHV DQG
SURSHUWLHV DQG LWV FRUUHFW DQG VHFXUH FRQILJXUDWLRQ >@ IXQFWLRQV WKDW HQDEOH WKH V\VWHP WR SHUIRUP DV H[SHFWHG
7KLV LV LPSRUWDQW IRU LWV SURSHU DQG VDIH IXQFWLRQLQJ >@7KH\DUHWKHIROORZLQJ
,QFRUUHFW RU LQDSSURSULDWH FRQILJXUDWLRQ FDQ OHDG WR   6RXUFHFRQWURO
LQFRUUHFWRSHUDWLRQRULWPD\HYHQUHSUHVHQWVHFXULW\JDSV 7KHVRIWZDUH PXVWHQVXUHWKHFRQWURORIWKH QHFHVVDU\
LQ QHWZRUN VHFXULW\ 6LPLODUO\ MXVW EHFDXVH D GHYLFH LV GHSHQGHQFLHV IRU LWV RSHUDWLRQ ,W VKRXOG QRW EH SRVVLEOH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 264

WRUXQLWZLWKRXWWKHP5XQQLQJWKHSURJUDPZLWKRXWWKH 2QFH FRQQHFWHG LW ZLOO H[HFXWH SUHGHILQHG FRPPDQGV
UHTXLUHGUHVRXUFHVFRXOGUHVXOWLQXQSUHGLFWDEOHEHKDYLRU RQ WKH GHYLFH $IWHU H[HFXWLQJ WKH FRPPDQGV LW ZLOO
RU HYHQ D FUDVK ,I FHUWDLQ GHSHQGHQFLHV DUH IRXQG WR EH UHWULHYHWKHRXWSXWVIURPWKHGHYLFHVSURFHVVHVWKHPDQG
PLVVLQJ WKH XVHU VKRXOG EH DOHUWHG DQG WKH SURJUDP VDYHV WKH UHVXOWV LQWR LQGLYLGXDO ILOHV $Q\ HUURUV WKDW
VKRXOGQRWSURFHHGZLWKWKHH[HFXWLRQ RFFXUGXULQJH[HFXWLRQZLOOEHUHSRUWHGWRWKHXVHU
  ,QVWDOOLQJUHVRXUFHV   /RJJLQJLQWRGHYLFHV
7KHXVHUZLOOEHSURYLGHGZLWKWKHRSWLRQRIDXWRPDWLF 7KHXVHUZLOOEHDOORZHGWRXVHWKUHHGLIIHUHQWPHWKRGV
LQVWDOODWLRQRIQHFHVVDU\UHVRXUFHVHOLPLQDWLQJWKHQHHGWR IRUHQWHULQJFUHGHQWLDOVRQUHPRWHGHYLFHV
VKXWGRZQWKHSURJUDPLQVWDOOWKHUHTXLUHGUHVRXUFHVDQG 7KH ILUVW PHWKRG XVHV WKH ORJLQ GDWD IURP WKH LQSXW ILOH
WKHQ UHODXQFK WKH SURJUDP DJDLQ 7KH UHVRXUFH (DFKGHYLFHZLOOKDYHLWVVSHFLILFFUHGHQWLDOV
LQVWDOODWLRQ SURFHVV ZLOO VWDUW DXWRPDWLFDOO\ $IWHU ,I WKH XVHU ZLVKHV WR XVH WKH VDPH FUHGHQWLDOV IRU DOO
LQVWDOODWLRQ WKH SURJUDP ZLOO FRQWLQXH WR UXQ ,W PD\ GHYLFHVKHZLOOEHDEOHWRORDGWKHPLQDGYDQFH
KDSSHQ WKDW WKH XVHU GRHV QRW KDYH DGPLQ ULJKWV IRU WKH 7KH ILQDO DSSURDFK LV WR DFTXLUH FUHGHQWLDOV GLUHFWO\
LQVWDOODWLRQ ,Q VXFK FDVH WKH SURJUDP ZLOO QRW SURFHHG GXULQJWKHGHYLFHFRQQHFWLRQ,ILQFRUUHFWORJLQGDWDZDV
ZLWKWKHH[HFXWLRQDQGQRWLI\WKHXVHURIWKHHUURU,WZLOO SURYLGHGIRULQGLYLGXDOGHYLFHVWKHSURJUDPZLOOLQIRUP
EH WKH XVHU V UHVSRQVLELOLW\ WR UHVROYH WKH LQVWDOODWLRQ WKHXVHURIWKLVHUURU
HUURU DV WKH SURJUDP ZLOO QRW DWWHPSW WR FRUUHFW LW   'HYLFHFRQILJXUDWLRQ
DXWRPDWLFDOO\ ,Q DGGLWLRQ WR JHWWLQJ WKH FRQILJXUDWLRQ WKH XVHU ZLOO
  +HOSHU DOVR EH DEOH WR FRQILJXUH WKH GHYLFH 7KH QXPEHU RI
%HIRUH VWDUWLQJ WKH SURJUDP WKH XVHU ZLOO KDYH WKH FRQILJXUDWLRQ FRPPDQGV IRU RQH GHYLFH ZLOO QRW EH
RSWLRQ WR DFFHVV KHOS 7KLV DVVLVWDQW ZLOO LQIRUP DERXW OLPLWHG7KHSURJUDPPXVWHQVXUHWKDWWKHFRPPDQGVDUH
SRVVLEOH SURJUDP LQSXWV DQG DOVR SURYLGH D VLPSOH H[HFXWHG V\VWHPDWLFDOO\ RQH DIWHU WKH RWKHU DV WKH XVHU
H[DPSOHRILWVH[HFXWLRQXVLQJDOODYDLODEOHLQSXWV VSHFLILHV WKHP 7KH RUGHU RI H[HFXWHG FRPPDQGV LV
  /RDGLQJLQSXWV GHWHUPLQHGE\WKHXVHUVWKHPVHOI7KHSURJUDPGRHVQRW
5HWULHYLQJ XVHU LQSXWV PXVW EH DYDLODEOH LQ PXOWLSOH QHHGWRH[DPLQHDQGFKDQJHWKHLURUGHU
ZD\V   6WDUWIURPWKHEDFNXS
)LUVW DQG IRUHPRVW LW LV HVVHQWLDO WR HQVXUH WKDW LQSXWV ,WLVSRVVLEOHWKDWGXULQJWKHH[HFXWLRQRIFRPPDQGVRQ
DUH ORDGHG XVLQJ LQSXW DUJXPHQWV EHIRUH VWDUWLQJ WKH GHYLFHVWKHSURJUDPPD\XQH[SHFWHGO\WHUPLQDWHEHIRUH
SURJUDP7KLVPHWKRGRIORDGLQJLQSXWVVKRXOGEHRUGHU FRPSOHWLQJ WKH FRQILJXUDWLRQ UHWULHYDO IURP DOO GHYLFHV
DJQRVWLFWRSUHYHQWDFFLGHQWDOPLVRUGHULQJWKDWFRXOGOHDG 7KLVFRXOGRFFXUIRUH[DPSOHGXHWRSRZHURXWDJHVORVV
WR XQSUHGLFWDEOH EHKDYLRU GXH WR RYHUVLJKW ,W LV DOVR RIFRQQHFWLYLW\RURWKHUXQIRUHVHHQHYHQWV,QVXFKFDVHV
DGYLVDEOHWRSURYLGHWKHXVHUZLWKWKHSRVVLELOLW\RIXVLQJ WKH XVHU ZLOO KDYH WKH RSWLRQ WR UHVXPH IURP WKH SRLQW
WKH WDE NH\ ZLWK ZKLFK WKH LQSXW ZLOO EH DXWRPDWLFDOO\ ZKHUHWKHSURJUDPFUDVKHGDWWKHQH[WVWDUWXS7RSUHYHQW
FRPSOHWHG DFFLGHQWDOGDWDRYHUZULWLQJWKHXVHU ZLOOEHLQIRUPHGLQ
$QRWKHU RSWLRQ LV WR ORDG LQSXWV DIWHU VWDUWLQJ WKH DGYDQFH WKH GDWD E\ PLVWDNH LQ DOO ZD\V RI VWDUWLQJ WKH
SURJUDP EXW EHIRUH HQWHULQJ WKH PHQX ,I WKH XVHU GRHV H[HFXWLRQRIFRPPDQGV
QRW ZDQW WR VHW DQ\ LQSXW DUJXPHQWV KH ZLOO EH DEOH WR
% 1RQIXQFWLRQDOUHTXLUHPHQWV
VNLS LW 6XEVHTXHQWO\ SUHGHILQHG LQSXW YDOXHV DUH XVHG
ZKLFKWKHXVHUPXVWEHLQIRUPHGDERXW$IWHUHQWHULQJWKH 1RQIXQFWLRQDO UHTXLUHPHQWV GHVFULEH WKH IHDWXUHV DQG
LQSXWLWVFRUUHFWQHVVZLOOEHFKHFNHG,IWKHFRUUHFWLQSXW OLPLWDWLRQV RI WKH SURJUDP WKDW LW PXVW UHVSHFW>@ 7KH\
LVQRWHQWHUHGWKHSURJUDPWKURZVDQHUURU DUHWKHIROORZLQJ
  ([HFXWDEOHILOHV   3HUIRUPDQFH
7KH SURJUDP ZLOO DXWRPDWLFDOO\ FRQILJXUH WKH 7KH SURJUDP VKRXOG HVWDEOLVK FRQQHFWLRQV WR GHYLFHV
QHFHVVDU\ SHUPLVVLRQV WR H[HFXWH DOO ILOHV WKDW DUH TXLFNO\H[HFXWHWKHUHTXLUHGFRPPDQGVDQGUHWULHYHWKH
UHTXLUHGIRULWVIXQFWLRQDOLW\3URYLGLQJWKLVIXQFWLRQWKH DSSURSULDWHRXWSXWV
XVHU LV UHOLHYHG RI WKH EXUGHQ RI VSHFLI\LQJ ILOH ULJKWV   &RPSDWLELOLW\
LQGLYLGXDOO\ ,W DOVR SUHYHQWV WKH RFFXUUHQFH RI DQ HUURU 7KH SURJUDP PXVW EH FRPSDWLEOH ZLWK GLIIHUHQW
WKDWZRXOGOHDGWRWKHFUDVKRIWKHSURJUDP YHUVLRQVRI/LQX[RSHUDWLQJV\VWHPV 26 LQFOXGLQJIRU
  6WDUWLQJWKHSURJUDPZLWKRXWHQWHULQJWKHPHQX H[DPSOH'HELDQ5HG+DWDQGRWKHUV
7KH SURJUDP SURYLGHV WKH RSWLRQ RI VWDUWLQJ ZLWKRXW   0RGXODULW\
HQWHULQJWKHSURJUDPPHQX,WLVQHFHVVDU\WRHQVXUHWKDW &RQVLGHULQJWKDWYDULRXVH[WHQVLRQVDUHH[SHFWHGLQWKH
DOO LPSRUWDQW LQSXWV IRU WKH GHVLUHG FRPPDQG H[HFXWLRQ IXWXUH LW LV LPSRUWDQW WR HQVXUH LWV VLPSOLFLW\ DQG
DUHORDGHGEHIRUHVXFKH[HFXWLRQ,IVXFKDQLQSXWLVQRW PRGXODULW\ 7KLV ZLOO PDNH WKH ZRUN RI IXWXUH VRIWZDUH
HQWHUHGWKHXVHUZLOOEHQRWLILHGRIWKLV GHYHORSHUVHDVLHU
  8VHULQWHUIDFH   6HFXULW\
7KHXVHULQWHUIDFHPXVWEHFUHDWHGLQ(QJOLVKLWVKRXOG 7KH DSSOLFDWLRQ FDQ EH XVHG E\ PXOWLSOH XVHUV RQ D
EHWUDQVSDUHQWDQGHDV\WRXQGHUVWDQG VLQJOH PDFKLQH :H DOVR KDQGOH ORJLQ GDWD ZKLFK LV
  5HWULHYLQJFRQILJXUDWLRQIURPUHPRWHGHYLFHV FRQVLGHUHG VHQVLWLYH LQIRUPDWLRQ>@ 7KHUHIRUH LW LV
7KH SURJUDP ZLOO HVWDEOLVK FRQQHFWLRQ WR UHPRWH QHFHVVDU\ WR HQVXUH WKDW WKLV LQIRUPDWLRQ UHPDLQV VHFXUH
GHYLFHV XVLQJ WKHLU ,3 DGGUHVVHV DQG ORJLQ FUHGHQWLDOV DQG GRHV QRW JHW H[SRVHG >@ 8QGHU QR FLUFXPVWDQFHV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 265

VKRXOG SDVVZRUGV EH GLVSOD\HG LQ WKH XVHU LQWHUIDFH RU ,QWKH)LJXUHZHFDQVHHWKHDYHUDJHWLPHVDFKLHYHG
ZULWWHQLQILOHV>@ ZKHQH[HFXWLQJWZRFRPPDQGVRQWKHGHYLFHLQSDUDOOHO
7KH3DUDPLNRDQG3\66+OLEUDULHVDFKLHYHGYHU\VLPLODU
,,, 7(67,1*$1'$1$/<6,62)/,%5$5,(6 UHVXOWV WLPHV ,W WRRN WKHP DSSUR[LPDWHO\  PLQXWHV WR
7R SUHVHUYH WKH QRQIXQFWLRQDO UHTXLUHPHQWV IRU WKH H[HFXWH WKH RUGHUV :H PHDVXUHG DQ DYHUDJH WLPH RI
SURJUDP¶V SHUIRUPDQFH RXU LQLWLDO GHFLVLRQ OHG WR DURXQG  PLQXWHV IRU WKH )DEULF OLEUDU\ DQG WKH ([SHFW
FRQGXFWWHVWVRQWKHOLEUDULHVXVHGIRUGHYLFHFRQQHFWLYLW\ H[WHQVLRQ 7KH IDVWHVW OLEUDU\ ZDV 66+3\WKRQ ZKLFK
%DVHG RQ WKH UHVXOWV ZH ZRXOG HYDOXDWH ZKLFK RI WKH ZDV DEOH WR H[HFXWH WKH VDPH QXPEHU RI FRPPDQGV RQ
OLEUDULHV VKRXOG EH LQWHJUDWHG LQ WKH SURJUDP )RU WKH WKH GHYLFHV LQ DERXW  PLQXWHV 6R VKH H[HFXWHG WKH
VDPH UHDVRQ SHUIRUPDQFH WHVWV ZHUH DOVR SHUIRUPHG IRU FRPPDQGVWRPLQXWHVIDVWHUWKDQWKHRWKHUV
ORDGLQJDQGZULWLQJGDWDWRIURPWKHILOH&RQQHFWLRQDQG
UHDGZULWH WHVWV ZHUH PDGH LQ 'HFHPEHU  7KH
VXEVHTXHQW DQDO\VLV RI WKH H[HFXWLRQ RI VHYHUDO
FRPPDQGV LQ RQH FRQQHFWLRQ WRRN SODFH LQ WKH SHULRG
IURP-DQXDU\WR)HEUXDU\
$ &RQQHFWLRQWHVWV
)RUWKHWHVWZHKDYHXVHGWZRW\SHVRIGHYLFHV)LUVW
ZDV WKH /LQX[ VHUYHU ZKRVH SXUSRVH ZDV WR VWDUW WKH
FRQQHFWLRQVFULSWVDQGVHFRQGZDVWKHQHWZRUNLQJGHYLFH
,Q RXU FDVH ZH XVHG WKH VHUYHU V\VWHP 'HELDQ 
VHUYHU XVHG &38 ,QWHO 5  ;HRQ 5  &38 ( )LJXUH5HVXOWVRIVHFRQGVFHQDULR
*+]DQG5$0PHPRU\RIN% 
:H SHUIRUPHG WHVWV RQ 3DUDPLNR )DEULF 3\66+   3UREOHPRISDUDOOHOFRPPDQGH[HFXWLRQ
66+3\WKRQDQG([SHFWS\WKRQOLEUDULHV )URPWKHSUHYLRXVWZRWHVWV ZH IRXQGWKDWWKH66+
,QRXU FDVH ZH GLG WZR W\SHV RI WHVWV 7KH ILUVW W\SH 3\WKRQOLEUDU\DQGWKH([SHFWH[WHQVLRQDFKLHYHGWKHEHVW
FRQVLVWHGRIFRQQHFWLQJDWKRXVDQGWLPHVZLWKWKHGHYLFH UHVXOWV
DQG H[HFXWLQJ WKH FRPPDQG VKRZ YHUVLRQ 7KH VHFRQG 7KH SUREOHP LV WKDW WKH XVHU GRHV QRW KDYH WR H[HFXWH
ZDV WKH VDPH FRPPDQG EXW ZH PDGH WZR SDUDOOHO RQO\ RQH FRPPDQG RQ D UHPRWH QHWZRUN GHYLFH :H
FRQQHFWLRQVWRWKHGHYLFH WKHUHIRUHGHFLGHGWRFKHFNWKHVLWXDWLRQZKHQZHQHHGWR
)RU WKH DQDO\VLV RI WKH RXWSXWV ZH UHSHDWHG HDFK RI H[HFXWHVHYHUDOGLIIHUHQWFRPPDQGVRQRQHGHYLFH:H YH
WKHVFHQDULRVILYHWLPHVDQGDYHUDJHGWKHUHVXOWLQJWLPHV PRGLILHG WKH SUHYLRXV VFULSWV VR WKDW ZH GRQ W FORVH WKH
  )LUVWVFHQDULR FRQQHFWLRQ DIWHU H[HFXWLQJ RQH FRPPDQG DQG JHWWLQJ
$VZHFDQVHHLQWKH)LJXUHWKH3DUDPLNRDQG)DEULF RXWSXW IURP WKH 66+ GHYLFH :H WKHQ VHQG DQRWKHU
OLEUDULHVDFKLHYHGDOPRVWWKHVDPHUHVXOWV:HUHFRUGHGD FRPPDQGWRWKHGHYLFHLQWKHVDPH66+FRQQHFWLRQ
WLPHRIMXVWRYHUPLQXWHVIRUERWKOLEUDULHV7KHUHVXOW 8QIRUWXQDWHO\ WKH VFULSW IHOO LQWR DQ HUURU 7KH HUURU
LV QRW EDG EXW FRPSDUHG WR RWKHU OLEUDULHV WKH\ WXUQHG RFFXUUHG ZLWK DOO WKH 3\WKRQ OLEUDULHV ZH XVHG LQ RXU
RXW WKH ZRUVW 7KH 3\66+ OLEUDU\ LV DOVR YHU\ FORVH WR VROXWLRQ 8SRQ IXUWKHU LQYHVWLJDWLRQ DQG DQDO\VLV RI WKH
WKHVHWLPHVUHDFKLQJDWLPHRIDURXQGPLQXWHV([SHFW LVVXH ZH OHDUQHG WKDW WKLV LV WKH GHVLUHG EHKDYLRU 7KH
LV PXFKEHWWHU,WUHFRUGHGDWLPHRIDURXQG PLQXWHV OLEUDULHV DUH SURJUDPPHG WR DXWRPDWLFDOO\ FORVH WKH
ZKLFK LV  PLQXWHV EHWWHU WKDQ RWKHU OLEUDULHV 7KH FRQQHFWLRQWRWKHGHYLFHDIWHUH[HFXWLQJRQHFRPPDQG,I
66+3\WKRQOLEUDU\ZDVWKHIDVWHVW,WDFKLHYHGDWLPHRI ZH ZDQWHG WR H[HFXWH DQRWKHU FRPPDQG RQ WKH GHYLFH
DURXQG  PLQXWHV ZKLFK LV  PLQXWHV IDVWHU WKDQ WKH ZHZRXOGKDYHWRXVHDQHZ66+FRQQHFWLRQ
VHFRQGEHVWWLPHDFKLHYHGE\([SHFW 7KHUHIRUHZHGHFLGHGWRFRPSDUHWKHH[HFXWLRQWLPHV
RI WHQ FRPPDQGV RQ RQH GHYLFH XVLQJ WKH 66+3\WKRQ
OLEUDU\DQGWKH([SHFWH[WHQVLRQ:HZRQGHUHGLI([SHFW
GRHVQRWDFKLHYHEHWWHUUHVXOWVLQWKLVFDVH
7KHVHZHUHFRPPDQGVVKRZYHUVLRQVKRZLSLQWEULHI
VKRZ LQWHUIDFHV VKRZ ORJ VKRZ UXQ VKRZ FORFN VKRZ
6103 VKRZ VVK VKRZ WRSRORJ\ DQG VKRZ DUS 7KHUH
ZHUH D KXQGUHG GHYLFHV WKDW ZHUH SURFHVVHG JUDGXDOO\
7KLV PHDQVWKDWRQO\DIWHUFRPSOHWLQJDOOWHQFRPPDQGV
RQ RQH GHYLFH LW ZDV SRVVLEOH WR VZLWFK WR WKH QH[W RQH
:H UHSHDWHG WKLV WHVW D KXQGUHG WLPHV WR DYRLG HUURUV LQ
WKHUHVXOWV:HFRXQWHGRQWKHH[LVWHQFHRIDPD[LPXPRI
 ILYHSDUDOOHO66+FRQQHFWLRQV
)LJXUH5HVXOWVRIILUVWVFHQDULR ,Q WKH 7DEOH  ZH FDQ VHH WKH PHDVXUHG GXUDWLRQ RI
LQGLYLGXDO WHVWV UHFRUGHG LQ VHFRQGV *LYHQ WKDW ([SHFW
 FDQ H[HFXWH DOO FRPPDQGV LQ D VLQJOH 66+ FRQQHFWLRQ
  6HFRQGVFHQDULR ZHRQO\QHHGHGWRUXQWHVWVIRURQHFDVH$VZHFDQVHH
ZLWK 66+3\WKRQ ZH FKDQJHG WKH QXPEHU RI

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 266

FRQFXUUHQWO\ H[HFXWHG FRPPDQGV 7KHLU QXPEHU LV LV WKDW LW LV HDV\ WR GHSOR\ DQG JLYHQ WKH UHODWLYHO\
GHWHUPLQHGE\WKHVHFRQGFROXPQ ZKLFKDOVRUHSUHVHQWV EDODQFHGWLPHVLW ZDVRXUFKRLFH)RUWKLV ZHDQDO\]HG
WKHQXPEHURIFRQFXUUHQW66+FRQQHFWLRQV WKH OLEUDU\ DQG PDGH D VPDOO H[HFXWLRQ VFULSW ZKHUH ZH
WHVWHGVRPHRIWKHIXQFWLRQDOLWLHVRIWKHOLEUDU\
7DEOH&RPSDULVRQRIWKHVSHHGRIH[HFXWLRQRIWHQFRPPDQGVRQ
WKHGHYLFH

/LEUDU\ 1XPEHURIFRQFXUUHQW $YHUDJHH[HFXWLRQ

66+FRQQHFWLRQV WLPH>V@
([SHFW  
66+  
3\WKRQ  
  &RQFOXVLRQRIFRQQHFWLRQWHVWV
$FFRUGLQJWRWKHFROOHFWHGUHVXOWVDQGDIWHUFRQVLGHULQJ
RWKHULQIOXHQFHVZHGHFLGHGWRXVHERWKOLEUDULHV 66+
3\WKRQDQG([SHFW WRFRQQHFWWRQHWZRUNGHYLFHV%RWK
OLEUDULHV KDYH EHHQ GHSOR\HG WR WKH SURJUDP LQ VXFK D 
ZD\WKDWLIRQHFRPPDQGLVH[HFXWHGRQWKHGHYLFHVWKH )LJXUH$YHUDJHUHDGVSHHGRIOLQHVWLPHV
66+3\WKRQ OLEUDU\ LV XVHG DQG LI PRUH FRPPDQGV LV
H[HFXWHGWKH([SHFWOLEUDU\LVDXWRPDWLFDOO\XVHG & $QDO\VLVRIPXOWLFRPPDQGFRQQHFWLRQ
%HFDXVH RI WKH SUREOHPV ZKLFK ZHUH VDG LQ FKDSWHU
% 5HDG:ULWHSHUIRUPDQFHWHVWV ,,,$  DIWHU WKH FUHDWLRQ RI WKH SURJUDP ZH IRFXVHG RQ
7RSHUIRUPWKLVW\SHRIWHVWZHQHHGHGRQHGHYLFHRQ EULQJLQJ XVHUV EHWWHU H[SHULHQFH DQG SHUIRUPDQFH IURP
ZKLFKWKHVFULSWZRXOGEHH[HFXWHG,QRXUFRQGLWLRQVZH WKH SURJUDP )RU WKLV SXUSRVH ZH WULHG WR DQDO\]H VRPH
XVHGWKHVDPH/LQX[VHUYHUDVLQWKHFRQQHFWLYLW\WHVWV RIWKHQHZDQGROGWHVWHGOLEUDULHVLIWKH\FDQVHQGPRUH
)RU WKH SHUIRUPDQFH WHVWLQJ ZH PDGH WZR W\SHV RI FRPPDQGVYLDRQHFRQQHFWLRQVHVVLRQ>@
VFHQDULRV ZKHUH ZH WHVWHG WKUHH 3\WKRQ OLEUDULHV ,Q WKLV ,QDIHZ\HDUVZHIRXQGRXWWKDWIRURXUSXUSRVHZDV
VFHQDULR ZH WHVWHG WKH EDVLF 3\WKRQ OLEUDU\ QDPHG &69 QHHGHGWKDWWKHOLEUDU\LVVXSSRUWHGLQ3\WKRQLVXVDEOH
DQGH[WHUQDOOLEUDULHV3DQGDVDQG&OHYHU&69 LQWKH/LQX[RSHUDWLQJV\VWHPVXSSRUWVDXWKHQWLFDWLRQYLD
,QWKH ILUVWVFHQDULR ZH ZDQWHGWRUHDGOLQHVRQH XVHUQDPHSDVVZRUG DQG SULYDWH NH\ DQG LV QHHGHG WR
WLPH DQG LQ WKH VHFRQG VFHQDULR ZDV WR UHDG  OLQHV VHQG PRUH WKDQ RQH FRPPDQG LQ RQH FRQQHFWLRQ WR WKH
 WLPHV %RWK VFHQDULRV ZHUH PDGH  WLPHV DQG GHYLFH>@ )RU WKLV ZH DQDO\]HG OLEUDU\ IXQFWLRQV DQG
DIWHUZDUGZHPDGHDYHUDJHWLPHVIURPWKHP PDGHDVPDOOVFULSWIRUHDFKRIWKHPZKHUHZHWULHGWKH
  )LUVWVFHQDULR IXQFWLRQV
(YHQ ZLWK D VPDOO VDPSOH LQ WHUPV RI VSHHG WKH )URPWKHUHVXOWVLQWKH7DEOHRQO\WKH([SHFWOLEUDU\
3DQGDVOLEUDU\FDPHRXWDVWKHZRUVWFDQGLGDWHDPRQJWKH PHHWVDOOWKHFRQGLWLRQVGHILQHGE\XV'XULQJWHVWLQJWKH
WKUHH WHVWHG OLEUDULHV 7KH VSHHGV RI WKH UHPDLQLQJ $VV\QF66+ OLEUDU\ DOVR ORRNHG TXLWH LQWHUHVWLQJ EXW LW
OLEUDULHV ZHUH URXJKO\ DW WKH VDPH OHYHO LQ WHUPV RI KDGWZRSUREOHPV7KHILUVWZDVWKDWLQLWVFXUUHQWIRUP
H[HFXWLRQWLPHVR ZHGHFLGHGWRSHUIRUP IXUWKHUWHVWLQJ WKH OLEUDU\ GRHV QRW KDYH VXSSRUW RQ /LQX[ RSHUDWLQJ
RQDODUJHUVDPSOH V\VWHPV DQG WKH VHFRQG ZDV WKH LPSRVVLELOLW\ RI XVH LQ
RXU FRQGLWLRQV GXH WR WKH LPSRVVLELOLW\ RI PDWFKLQJ WKH
OLVW RI VHQW FRPPDQGV ZLWK WKH UHVSRQVH WR WKH GHYLFH
7KLV LV FDXVHG E\ KLV DV\QFKURQRXV ZD\ RI
FRPPXQLFDWLRQ
,9 $5&+,7(&785($1'&2'(
7KH SULPDU\ 16+ GLUHFWRU\ FRQWDLQV D VFULSW FDOOHG
UXQ ZKLFK LV HVVHQWLDO ZKHQ UXQQLQJ DSSOLFDWLRQV 7KH
SULPDU\ IXQFWLRQ RI WKH VFULSW LV WR ORDG LQSXW YDULDEOHV
DQG QHFHVVDU\ UHVRXUFHV DQG UXQ WKH DSSOLFDWLRQ
VPRRWKO\ PHHWLQJ DOO UHTXLUHPHQWV ,Q WKLV FDVH WKH
 DSSOLFDWLRQ ZLOO KDYH DOO WKH LQIRUPDWLRQ DERXW WKH
)LJXUH$YHUDJHUHDGVSHHGRIOLQHVWLPHV GHYLFHV LW LV FRQQHFWLQJ WR WKH LQIRUPDWLRQ LW KDV WR
  6HFRQGVFHQDULR SHUIRUPRQWKHGHYLFHVDQGDOOWKHSDUDPHWHUVQHHGHGWR
+HUH ZLWKD KXQGUHGOLQHVEHLQJ ORDGHGWKHVLWXDWLRQ HVWDEOLVK D FRQQHFWLRQ ZLWK WKH GHYLFHV )XUWKHUPRUH LQ
ZLWKWKH&69DQG&OHYHU&69OLEUDULHVKDVWXUQHGDURXQG WKH PDLQ IROGHU WKHUH DUH WKH ILOHV 5($'0(PG
&OHYHU&69KDGDVOLJKWHGJHRYHUWKHEDVLF&69OLEUDU\ ZKLFK LV QRW QHFHVVDU\ DQG WKH ILOH UHTXLUHPHQWVW[W
+RZHYHUDIWHUGRLQJVRPHPRUHWHVWLQJORDGLQJDQG ZKLFKLVXVHGWRLQVWDOOWKHFRUUHFWYHUVLRQRIWKH66+
ZULWLQJ LQ GLIIHUHQW IRUPV ZH GHFLGHG WR JR ZLWK WKH 3\WKRQ OLEUDU\ GXULQJ WKH ILUVW UXQ 7KH GLUHFWRU\ DOVR
EDVLF&69OLEUDU\ IRUXVHLQ RXUDSSOLFDWLRQ7KHUHDVRQ FRQWDLQV DQRWKHU VXEIROGHU QDPHG QVK ZKLFK LV WKH
PDLQ IROGHU RI WKH SURMHFW ,W FRQWDLQV WKH 1HWZRUN

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 267

 WZR VXEIROGHUV IRU VHQGLQJ LQIRUPDWLRQ WR GHYLFHV 2QH
7DEOH/LEUDU\DQDO\VLVUHVXOWV XVHV WKH 66+3\WKRQ OLEUDU\ DQG WKH RWKHU WKH ([SHFW
 $VV\QF 6FUDSOL 66+ 3DUDPLNR OLEUDU\7KHIROORZLQJDUHWZRSDFNDJHVRQHRIZKLFKLV
66+ 3\WKRQ XVHGIRU ZRUNLQJ ZLWKGRFXPHQWVORDGLQJWKHP LQWRWKH
7HVWYHUVLRQ     SURJUDP RU VDYLQJ LQIRUPDWLRQ IURP WKH SURJUDP LQWR D
 ILOH7KHVHFRQGLVXVHGIRUSURFHVVLQJDQGHYDOXDWLQJWKH
6XSSRUWRI3\WKRQ <HV <HV <HV <HV
WH[WREWDLQHGIURPWKHGHYLFHV
$V\QFKURQRXV <HV <HV 1R 1R 7KH UHPDLQLQJ WZR IROGHUV FRQWDLQ WH[WV XVHG IRU
FRPPXQLFDWLRQ RXWSXW WR WKH FRQVROH VXFK DV WKH LQLWLDO PHQX SURJUDP
6XSSRUWRQ/LQX[ 1R 1R <HV <HV
EDQQHU DQG SURJUDP KHOS DQG IXUWKHU FRQWDLQ VDPSOH
$XWKYLD <HV <HV <HV <HV &69GRFXPHQWVWKDWWKHSURJUDPXVHVWRORDGGHYLFHVRU
XVHUQDPHSDVVZRUG WRREWDLQWDVNVWREHSHUIRUPHGRQGHYLFHV
$XWKYLDSULYDWH <HV <HV <HV <HV
NH\ ,QWKH)LJXUHZHFDQVHHWKHGLDJUDPRIWKHSURJUDP
0XOWLFRPPDQG <HV 1R 1R 1R SDFNDJHV 7KH FODVVHV LW FRQWDLQV DUH DOVR LQFOXGHG LQ
H[HFXWLRQLQRQH HDFKSDFNDJH$SDFNDJHPD\FRQWDLQDQHVWHGSDFNDJH
FRQQHFWLRQ 7KLVUHODWLRQVKLSLVVKRZQXVLQJDVROLGOLQHWRJHWKHUZLWK
 1HWPLNR 7ULJJHU 3DUDOOHO ([SHFW
66+
DFURVVLQDVSKHUH$EDOOZLWKDFURVVLVORFDWHGQH[WWR
7HVWYHUVLRQ     WKH GHFN LQ ZKLFK WKH GHFN LV QHVWHG 7KH VHFRQG
 UHODWLRQVKLS UHSUHVHQWV WKH PXWXDO XVH RI SDFNDJHV ,W LV
6XSSRUWRI3\WKRQ <HV 1R 1R <HV VKRZQZLWKDGDVKHGOLQHDORQJZLWKDQDUURZSRLQWLQJWR
$V\QFKURQRXV 1R 1R 1R 1R WKHLPSRUWHGSDFNDJH
FRPPXQLFDWLRQ
6XSSRUWRQ/LQX[ <HV <HV <HV <HV

$XWKYLDXVHUQDPH <HV <HV <HV <HV

SDVVZRUG
$XWKYLDSULYDWH <HV <HV <HV <HV
NH\
0XOWLFRPPDQG 1R 1R 1R <HV
H[HFXWLRQLQRQH
FRQQHFWLRQ
6\VWHP+HDOWKFKHFN 16+ SURJUDPLWVHOI ZKLFKVHUYHV
WR HQVXUH WURXEOHIUHH QHWZRUN RSHUDWLRQ DQG SUHYHQW
V\VWHP RXWDJHV ,Q DGGLWLRQ LW DOVR FRQWDLQV RWKHU ILOHV
DQGOLEUDULHVWKDWDUHQHFHVVDU\IRUWKHSURSHUIXQFWLRQLQJ
RIWKHSURJUDP7KLVIROGHULVWKHFHQWHURIWKHSURMHFWDQG
LWVVWUXFWXUHDQGFRQWHQWKDYHDVLJQLILFDQWLPSDFWRQWKH 
RYHUDOOIXQFWLRQDOLW\RIWKHDSSOLFDWLRQ,QDGGLWLRQWRWKH )LJXUH)ROGHUGLDJUDP
SURJUDP WKLV IROGHU DOVR FRQWDLQV WHVW GRFXPHQWV WKDW
VHUYH WR YHULI\ WKH FRUUHFW IXQFWLRQLQJ RI WKH DSSOLFDWLRQ 9 ),56767$572)16+
DQG LWV PRGXOHV DQG DV D PRGHO RI WKH QHFHVVDU\
DSSHDUDQFH RI WKH GRFXPHQWV IRU WKH H[HFXWLRQ RI WKH 7KHSURJUDPH[HFXWLRQORJLFLVLQWKHUXQILOH7KLVILOH
DSSOLFDWLRQ FRQWDLQV VHYHUDO VWHSV UHTXLUHG WR UXQ WKH SURJUDP VXFK
,QWKHPDLQIROGHUQDPHGQVKWKHUHLVDOVRWKHSURJUDP DV LQVWDOOLQJ UHVRXUFHV FKDQJLQJ WKH ULJKWV RI FHUWDLQ
VWDUWXS GRFXPHQW FRQWDLQLQJ WKH PDLQ PHWKRG DQG HLJKW ILOHV DQG ORDGLQJ LQSXW DUJXPHQWV %HIRUH UXQQLQJ WKH
RWKHUIROGHUVVL[RIZKLFKKDYHWKHIXQFWLRQRISDFNDJHV ILOHLWLVQHFHVVDU\WRVHWWKHULJKWVWRUXQLW
IRU WKH SURJUDP ,Q WKH SURJUDP LW LV QRW QHFHVVDU\ WR $IWHUUXQQLQJWKHVFULSWDOOQHFHVVDU\UHVRXUFHVIRUWKH
LPSRUWGRFXPHQWV IURPSDFNDJHV XVLQJDUHODWLYHRU IXOO FRUUHFWUXQQLQJRIWKHSURJUDPZLOOEHFKHFNHG7KHILUVW
SDWKEXWWKDQNVWRLQLWGRFXPHQWVLWLVHQRXJKWRLPSRUW FKHFN LV WKH 3\WKRQ YHUVLRQ ,W LV LPSRUWDQW WR KDYH
WKHPLQWRWKHSURJUDPDVDOLEUDU\ YHUVLRQ  DQG KLJKHU LQVWDOOHG EHFDXVH WKH SURJUDP XVHV
(DFK RI WKH SDFNDJHV VHUYHV D GLIIHUHQW WDVN LQ WKH IHDWXUHV WKDW DUH RQO\ DYDLODEOH LQ WKHVH YHUVLRQV ,I D
SURJUDP 7KH VWDUW SDFNDJH VHUYHV WR GLUHFWO\ ODXQFK WKH ORZHU YHUVLRQ LV VSHFLILHG WKH SURJUDP HQGV ZLWK DQ
DSSOLFDWLRQ ZLWKRXW WKH QHHG WR JR WKURXJK WKH LQSXW HUURU+RZHYHULIWKHUHTXLUHGYHUVLRQRI3\WKRQGRHVQRW
PHQX DQG DOVR FRQWDLQV SDUWV IRU OLVWLQJ WKH LQSXW PHQX H[LVW WKH SURJUDP DOORZV \RX WR LQVWDOO LW DXWRPDWLFDOO\
DQG ORDGLQJ WKH LQIRUPDWLRQ HQWHUHG LQ LW 7KH VHWXS $QRWKHU FKHFN LV WKH SUHVHQFH RI WKH ([SHFW H[WHQVLRQ
SDFNDJHWDNHVFDUHRIORDGLQJSURJUDPLQSXWVDQGDOORZV 6LPLODU WR LQVWDOOLQJ WKH FRUUHFW YHUVLRQ RI 3\WKRQ WKH
DFFHVV DQG PRGLILFDWLRQ RI DUJXPHQWV IRU WKH UHPDLQLQJ XVHU KDV WKH RSWLRQ WR LQVWDOO LW DXWRPDWLFDOO\ XVLQJ WKH
SDUWVRIWKHDSSOLFDWLRQ7KHIROORZLQJDUHWKUHHSDFNDJHV SUHYLRXV RSWLRQV 7KH ODVW FKHFN LV WKH H[LVWHQFH RI WKH
ZKRVH WDVN LV WKH FRPPXQLFDWLRQ DQG SURFHVVLQJ RI 66+3\WKRQ OLEUDU\ ,W EHORQJV WR D VSHFLILF YHUVLRQ RI
LQIRUPDWLRQIURPGHYLFHV7KHFRPPSDFNDJHLVXVHGIRU 3\WKRQ DQG FDQ EH DXWRPDWLFDOO\ LQVWDOOHG XVLQJ WKH
FRPPXQLFDWLRQ DQG HQVXUHV WKH SRVVLELOLW\ RI VWDUWLQJ D UHTXLUHPHQWVW[W GRFXPHQW <RX FDQ VHH WKH HQWLUH
GHYLFHFKHFNIURPDEDFNXS7KHFRPPSDFNDJHFRQWDLQV SURFHVVRIVWDUWLQJWKHSURJUDPLQ)LJXUH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 268

 9, &21&/86,21
7KHDUWLFOHGHVFULEHVDEDVLFRYHUYLHZRIWKH1HWZRUN
6\VWHP +HDOWKFKHFN VROXWLRQ LQWHQGHG IRU DXWRPDWHG
DVVHWV RI GHYLFH FRQILJXUDWLRQV LQ ODUJHVFDOH QHWZRUNV
,QWKHDUWLFOHZHSUHVHQWRXWSXWVRIRXUSHUIRUPDQFHWHVWV
DQG S\WKRQ OLEUDU\ SHUIRUPDQFH DQDO\VLV 7HVWV ZHUH
SHUIRUPHG LQ FRPSO\ ZLWK WKH VSHFLILFDWLRQV DQG
UHTXLUHPHQWVRIWKHH[SHFWHGXVHUV2YHUWLPHWKH16+
SURMHFW SURYHG WR EH VXFFHVVIXO DQG WKDQNV WR LWV VPDOO
VL]H HDVH RI GHSOR\PHQW HDVH RI XVH DQG SRVVLELOLWLHV
DOPRVWERUGHULQJRQWKH XVHU VLPDJLQDWLRQDQGDELOLWLHV
WKH SURJUDP EHJDQ WR EH XVHG LQ FRPSDQLHV DV ZHOO
6LQFH WKH SLORW YHUVLRQ WKDQNV WR WKH LQIRUPDWLRQ IURP
WKH XVHUV ZH KDYH FRQWLQXHG WR GHYHORS DQG DGG QHZ
IXQFWLRQDOLWLHVDFFRUGLQJWRWKHLUUHTXLUHPHQWV:HKRSH
WKDW \RX ZHUH LQWHUHVWHG LQ WKLV DUWLFOH HLWKHU LQ LWV
DQDO\WLFDOSDUWRUE\WKHSURFHVVRIKRZLWZDVGHYHORSHG
RUKRZLWZRUNV
9,, $&.12:/('*0(17

7KLVSDSHULVVXSSRUWHGE\SURMHFWV.(*$=8
³7HFKQRORJLHVRISULYDWHFORXGHQYLURQPHQWVLQ
XQLYHUVLW\HGXFDWLRQ´

5()(5(1&(6
>@ * 0F&DXOH\ ³:KDW LV D &RPSXWHU 1HWZRUN"´
KWWSVZZZILHOGHQJLQHHUFRPEORJVZKDWLVD
FRPSXWHUQHWZRUN
>@ 7 -XUtN ³&KHFNLQJ WKH VWDWXV RI QHWZRUN GHYLFHV 
SURFHVVLQJDQGDQDO\VLVRILQSXWRXWSXWGDWD´
>@ $ 6WUDND ³&KHFNLQJ WKH FRQILJXUDWLRQ RI QHWZRUN
GHYLFHVFRPPXQLFDWLRQZLWKWKHGHYLFH´
 >@ ,((( ³,((( 5HFRPPHQGHG 3UDFWLFH IRU 6RIWZDUH
5HTXLUHPHQWV6SHFLILFDWLRQV´
)LJXUH6WDUWXSPHFKDQLVP
>@ $OWH[6RIW ³)XQFWLRQDO DQG 1RQIXQFWLRQDO
7R VWDUW WKH H[HFXWLRQ RI WDVNV LW LV QHFHVVDU\ WR 5HTXLUHPHQWV 6SHFLILFDWLRQ DQG 7\SHV´
SUHSDUH GRFXPHQWV ZLWK LQIRUPDWLRQ DERXW GHYLFHV KWWSVZZZDOWH[VRIWFRPEORJEXVLQHVVIXQFWLRQDO
FRPPDQGV DQG SROLFLHV ([DPSOHV RI WKHP DUH LQ WKH DQGQRQIXQFWLRQDOUHTXLUHPHQWVVSHFLILFDWLRQDQG
GDWD IROGHU7KH SURJUDP FDQ WKHQ EH VWDUWHG GLUHFWO\ E\ W\SHV
XVLQJWKHSURJUDP VDUJXPHQWVZKLFKFDQEHIRXQGDIWHU >@ 0 'UR]GRYD 0 %ULGRYD , 8UDPRYD -
VWDUWLQJ ZLWK SDUDPHWHU K RU \RX FDQ UHDG PRUH DERXW 0RUDYFLN ³3ULYDWH FORXG VHFXULW\ DUFKLWHFWXUH´ LQ
WKHP LQ WKH ZRUN >@ ,I \RX GHFLGH QRW WR XVH SURJUDP   ,&(7$   WK ,((( ,QWHUQDWLRQDO
DUJXPHQWV DQ LQWURGXFWRU\ PHVVDJH DQG LQIRUPDWLRQ &RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ 7HFKQRORJLHV
DERXWWKHVSHFLILHGVHWWLQJVZLOOEHGLVSOD\HGDQG\RXZLOO DQG $SSOLFDWLRQV 3URFHHGLQJV DUW QR 
EHDVNHGLI \RX ZLVKWRFKDQJHWKHSURJUDPSDUDPHWHUV 
,Q WKLV SKDVH DOO SDUDPHWHUV RI WKH SURJUDP FDQ EH >@ 0 6WHUEDN 3 6HJHF DQG - -XUF ³$XWRPDWLRQ RI
PRGLILHG6XEVHTXHQWO\LI\RXQRORQJHUQHHGWRFKDQJH ULVN PDQDJHPHQW SURFHVVHV´ LQ ,&(7$   WK
WKHPDQLQLWLDOPHQXZLOOEHGLVSOD\HGLQZKLFK\RXFDQ ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ (PHUJLQJ
FKRRVH ZKDW \RX ZDQW WR GR ,Q WKH FDVH RI D H/HDUQLQJ 7HFKQRORJLHV DQG $SSOLFDWLRQV
FRQILJXUDWLRQ FKHFN WKH SURJUDP LV WKHQ H[HFXWHG DQG 3URFHHGLQJV
DIWHULWVH[HFXWLRQH[HFXWLRQVWDWLVWLFVDUHSULQWHGDQGWKH >@ - ' DQG ' 0 % . .XPDU 1 5DM ³)L[LQJ
UHVXOWLVVWRUHG 1HWZRUN 6HFXULW\ 9XOQHUDELOLWLHV LQ /RFDO $UHD
1HWZRUN´ LQ  UG ,QWHUQDWLRQDO &RQIHUHQFH RQ
7UHQGV LQ (OHFWURQLFV DQG ,QIRUPDWLFV ,&2(, 

>@ ' .DIND ³&RQGLWLRQDO FRQWURO RI WKH FRQILJXUDWLRQ
RIQHWZRUNGHYLFHV´


)LJXUH([DPSOHRIRXWSXWVWDWLVWLFV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 269

 Educational Web Solution for Cyber Security
O. Kainz, S. Nečeda, M. Michalko, M. Murin and I. Nováková
Department of Computers and Informatics, Technical University of Kosice, Kosice, Slovakia
ondrej.kainz@tuke.sk, samuel.neceda@student.tuke.sk, miroslav.michalko@.tuke.sk,
miroslav.murin@tuke.sk, ivana.novakova@tuke.sk

Abstract—The core objective of the study presented in this to simulate attacks and subsequently apply their acquired
paper was to develop an all-encompassing educational platform theoretical knowledge to address specific security challenges.
focused on teaching cybersecurity of web applications. This effort The main objective of this paper is to develop a com-
produced a dual-component solution - The educational portal
and a deliberately vulnerable web application. The paper delves prehensive educational platform focused on web application
into an examination of entities and collectives specializing in cybersecurity, consisting of two parts. The first component
cybersecurity, and from this analysis, OWASP and its OWASP will comprise an educational portal dedicated to providing
Top 10:2021 initiative emerged as guiding references. A compar- comprehensive information on the development of simple
ative evaluation of existing instructional tools was undertaken web application and its protection against cyber threats. This
to pinpoint their limitations and subsequently improve them in
the proposed solution’s design and deployment. The highlight of section will leverage the findings from the analytical portion
this project is an instructional platform that provides detailed of this paper. In the second phase, a deliberately insecure
guides on developing applications and subsequently unveiling web application will be created. It will allow users to initially
their weaknesses through cyberattacks. The intentionally flawed test a specific attack and subsequently apply the necessary
application constructed for simulation purposes aligns with source code modifications to fortify the application against the
prevailing standards and technologies. Feedback on the realized
solution was garnered through tests conducted with the intended identified threat. Upon removal of the vulnerability, users will
user demographic. The finalized educational platform is not only then verify the enhanced resilience of the application against
accessible globally but also holds potential as a primary or the particular form of attack.
auxiliary academic resource in university settings. The resulting solution presented in this research paper can
Keywords—attack, cybersecurity, educational portal, OWASP serve as supplementary material for current university courses.
organization, vulnerability, web application
Alternatively, it could also be used as the basis for various
cyber threat awareness workshops.
I. I NTRODUCTION
The inception of the internet and the subsequent advances in II. S TANDARDIZATION OF WEB ATTACKS
web applications have undeniably revolutionized the lifestyles There is no standard way of defining web attacks in terms
of a majority of individuals. However, the transformation to of prevalence, severity, and timeliness. Several organizations
the digital world also requires a significant level of protection operating in the government, public, or third sector establish
for data and assets, as they become vulnerable to attackers in their own threat lists and how to address them. For the
the absence of proper security measures. Computer systems purposes of this paper, it is important to define a standard
are constantly evolving, as are the threats they face on a daily by which web application intrusions are described.
basis.
In the present day, a multitude of automated tools exist, A. OWASP organization
which, when appropriately configured and effectively inte- OWASP (Open Web Application Security Project) was
grated, provide a formidable line of defense against cyber- founded in September 2001. As stated by Huseby in the book
attacks. Even the most widely used programming languages [1], the main founder of the project was Mark Curphey.
and frameworks for web application development contain According to the official website of the OWASP Founda-
libraries and functionalities eliminating known vulnerabilities. tion [2], it is recognized worldwide as the largest non-profit
However, none of the aforementioned solutions is perfect. The organization dedicated to software security. The community
human factor is still a significant factor that determines the comprises thousands of members actively engaged in the
success of an attack. The successful implementation of secure development of software products. They are also involved in
web applications necessitates a comprehensive understanding organizing local and global conferences with a primary focus
of cybersecurity principles by the developers involved. Equally on education and training.
important is their awareness of the prevailing vulnerabilities The handbook [3] defines OWASP projects as the primary
and effective mitigation strategies employed to counter them. method for promoting and educating about cybersecurity. The
One of the identified limitations of the existing learning projects are characterized by their open-source nature, which
solutions lies in their inadequate adaptability to the ever- inherently allows individuals to initiate, contribute to, and lead
changing landscape of emerging threats. Furthermore, there is them. The author Groves defines individual project types in the
an absence of hands-on practical exercises, allowing students handbook [3] as follows:

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 270

 • Flagship Projects, E. Evaluation of the standard selection
• Lab Projects,
Four products from the three organizations compiling the
• Incubator Projects.
cyber threat rankings were selected for comparison. A shared
attribute among these rankings is their accessibility to the
B. CVE database
general public in order to raise awareness of current threats.
R.A.Martin describes in the publication [4] Common Vul- The OWASP organization has been chosen as the standard
nerabilities and Exposures, abbreviated as CVE, as an initiative and reference point for this paper. The foremost argument
founded in 1999, operated by MITRE Corporation. Its primary is that it is the largest non-profit organization dedicated to
objective is to establish a standardized approach to the clas- cyber security. In particular, the OWASP Top 10 project has
sification of software vulnerabilities, subsequently integrating emerged as the standard referenced by the vast majority of
these designated names into security tools. The CVE List is industry publications, making it a trusted source of up-to-date
an international and standardized list defining known threats, knowledge.
which is continuously updated and publicly available.
III. A NALYSIS OF WEB ATTACKS
C. CWE categorisation
Web application security is a pivotal factor to consider
As noted in reference [5], the CWE (Common Weakness during the development phase of an application. With the
Enumeration) dates from 2006 and is maintained by the same increasing sophistication of applications, there is a simulta-
company as the CVE. The distinction between the given neous evolution of the threats to which they are exposed.
entities is that while CVE serves as a large database of The importance of addressing this issue is further emphasized
vulnerabilities in specific products, CWE takes a hierarchical by the authors of the research paper [8] referring to the
approach by categorizing individual CVE records into broader Acunetix 2019 report. According to it, almost half (46%) of
entities. The primary focus of the CWE is to generalize and web applications are at a high risk of attack, while the vast
find the root cause of the vulnerabilities. CVE contains over majority (up to 87%) are at moderate risk.
100,000 vulnerabilities and according to the source [6], CWE
covers 700 unique categories. Table I shows the comparison A. OWASP Top 10:2021
between CVE and CWE. However, it is important to note that
they exhibit a significant degree of complementarity across Nedeljković, Vugdelija, and Kojić [9], following the official
numerous aspects. website of OWASP, state that since 2003 the organization has
consistently published a comprehensive report entitled ”Top
TABLE I: Comparison of CVE and CWE. 10”. This report focuses on enumerating ten significant cyber
threats of web applications.
The latest available edition of 2021 contains several changes
in comparison to its preceding version from 2017, mainly
related to the methodologies employed for data acquisition
and the subsequent categorization of threats. As the authors
also point out in the paper [9], the rapid pace of development
in web applications and technologies as a whole does not align
with the rate of change in the OWASP Top 10 list. In particular,
successive editions differ in alterations in the prioritization of
certain threats or by assigning them to a different category.

B. Comparison with previous editions

As depicted in 1, in the most recent edition of the OWASP
Top 10 three new categories have been added compared to
the 2017 edition. In addition, several have been renamed
or changed their position in the ranking. According to the
D. ETL report
source [7], the 2021 edition is based on real data more
European Union Agency for Network and Information Se- than ever before. It focuses solely on the source of the
curity (ENISA) publishes an annual threat report, known as the problem rather than addressing the symptoms. The OWASP
ETL (ENISA Threat Landscape Report). The report provides Top 10 also incorporates two categories that emerged through
a comprehensive analysis of fifteen prominent cyber threats a survey conducted among cybersecurity professionals. Several
observed during the preceding year, systematically prioritized occurring and emerging threats cannot be fully identified at the
according to various criteria. The insights and findings pre- time of testing so it is necessary to provide an opportunity for
sented in this report help organizations and Member States of experts to express their insights and opinions regarding current
the European Union to comply with security regulations. trends, alongside working with empirical data.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 271

 Fig. 1: Comparison of OWASP Top 10 2017 and 2021.
Inspired by the source [7].

C. Structure of categories organization. Individual tools are sorted by their complexity

In previous years, companies have been provided with a and their project type according to II-A.
list containing approximately 30 recommended CWEs (refer The application named Juice Shop is at the most advanced
to II-C) against which they should test their systems and report stage of development and is classified as a flagship project.
the identified vulnerabilities’ quantity and severity. However, It provides a comprehensive overview of the list of threats,
subsequent analysis has revealed a tendency among companies together with filtering based on severity or type of threat.
to focus on a limited set of CWE categories and therefore new When evaluated as an educational portal, the shortcoming was
critical threats have often not emerged. identified in the lack of detailed instructions and descriptions
In the 2021 edition of the OWASP Top 10, companies of each lecture. Consequently, this frequently leads to users
should have provided the full list of identified CWEs. The encountering difficulties when completing individual lectures.
quantity of them has increased significantly from the initial The second existing educational portal involved in the
30 to a total of 400 CWEs. According to [7], the indisputable comparison was WebGoat. According to the paper [10], each
benefit is that on average 19.6 CWEs are mapped for each of lesson begins with theoretical information. This approach
the ten categories. serves as the foundation for comprehending and effectively
executing the subsequent cyber attack. Moreover, the output
D. Vulnerability assessment metrics of individual lectures is validated and the user‘s progress is
tracked. One minor drawback lies in reliance on the OWASP
The official website of the project [7] further states that
Top 10 2017 edition, which is not the most recent one.
OWASP Top 10 prefers incidence rate to frequency. The reason
Additionally, the absence of a consistent theme throughout
is that a particular vulnerability, once identified in a web appli-
the application and the lack of gamification elements serve
cation, may be subsequently exploited multiple times. In the
as further disadvantages, in contrast to the Juice Shop portal,
end, this could be designated as a more severe error compared
which incorporates both of these aspects.
to an error with a significantly lower frequency but with a
The last application included in the comparison was Node-
much higher severity surpassing the more frequently occurring
Goat. It focuses on threats faced by applications developed
error. The incidence rate eliminates this problem by treating
using Node.js and JavaScript. The NodeGoat distinguishes
the vulnerability found as a separate entity, independent of its
itself from the aforementioned ones by allowing users to
frequency.
directly edit the application’s source code. As this tool is
IV. C OMPARISON OF EXISTING EDUCATIONAL TOOLS not one of the OWASP flagship projects, it relies on fewer
contributors, resulting in a direct reduction in its maintenance
The aim of this paper is to present a comprehensive educa-
or the incorporation of new features.
tional portal focused on teaching web applications cybersecu-
rity. In order to address the shortcomings of the existing solu- V. M ETHODOLOGY
tions, this paper also provides a brief analysis and comparison The initial part of the software solution represents an
of three selected open-source tools provided by the OWASP educational portal, functioning as an interface to provide users

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 272

with individual lessons. The portal is directly connected with theme. The deliberately insecure web application described in
a deliberately insecure web application called HackHealth, this paper is a portal for a medical ambulance called Hack-
which forms the second part of the final product. The portal Health. The portal provides opportunities for cyber-attacks of
consists of the following three main parts: varying nature and severity.
• The initial setup is a fundamental point describing the
C. Technical design and deployment of educational portal
steps needed to prepare the environment for executing
the application. Furthermore, it also provides guidance The educational portal’s implementation utilized the Nextra
on how to access the basic skeleton of the application’s framework based on Next.js, drawing upon methodologies
source code, serving as the basis for the subsequent and insights from [11] and [12]. Nextra offers two pre-
section. made templates, and for this project, the design tailored for
• Programming a deliberately insecure web applica-
documentation pages was selected. The initial phase involved
tion involves implementing an application that will be downloading the foundational code structure. Subsequently,
exploited to discover security vulnerabilities in the final we modified individual sections, components, animations,
phase of the tutorial. transitions, the way content is rendered, and much more to
• Demonstrating vulnerabilities by performing attacks
differentiate our product from other portals created using the
is the most important part of the tutorial. Initially, appli- Nextra framework. One of the most significant advantages of
cations and tools used to perform attacks are introduced, using this framework was the ability to define custom React
followed by a comprehensive demonstration of the at- components. The content of each lecture was created using
tacks themselves. After the successful disruption of the Markdown language. In addition, we had full control over
fundamental pillars of security, namely confidentiality, adding images or snippets of source code. The individual steps
integrity, and availability, it is up to users to correct the of the portal implementation are illustrated in the diagram in
deficiencies directly in the provided application source Fig. 2.
code. The final step is to reproduce the attack, which
should no longer be possible.
A. Securing independence
Because modifying the application’s source code is part of
the process of discovering vulnerabilities and then securing
them, it is important to devote a section of the tutorial to the
creation of the application itself. The users of the educational
portal can be divided into two groups. The first group’s role
is also to create the application, whether their motivation is to
learn how to implement it or to consolidate previously acquired
knowledge. The goal of the second group of users is only the
part of the portal dedicated to cybersecurity. Fig. 2: Principle of educational portal implementation.
The approach to addressing the independence of the above-
mentioned parts of the portal is to create two different versions The automatic deployment of the education portal was
of the application source code. The first version is used ensured by connecting the GitHub repository containing the
during the development of the web application. Versioning via source code of the site to the Vercel tool. Vercel serves
Git branches ensures that there is a branch corresponding to as a platform for deploying and hosting web applications
each lecture. This way, if the user is unable to complete the and is seamlessly integrated with Nextra. The educational
lecture due to errors or other reasons, he or she can move on portal is also publicly available at the following URL address:
to the version containing all the necessary implementations. https://www.hack-health.tech/.
On the other side, the second version of the source code
contains a fully implemented application. This allows users D. Distribution of an application
to focus only on the cybersecurity section, with the option The actual development of the application must first be
of skipping the entire web application development process preceded by defining its distribution method. Due to its
altogether. The proposed approach provides greater flexibility educational nature and the need to access the application’s
and differentiates the portal from the existing solutions. source code and its subsequent launching by users, it cannot
be deployed on a server and made available as a web service.
B. Theme and functionalities of the application Another potential option would be to encapsulate the applica-
Numerous contemporary solutions in the domain of web ap- tion itself along with its dependencies within a container using
plication security education consist of a collection of lessons, Docker technology. It is also not an ideal solution, because it
each addressing a different vulnerability. However, the practi- requires users to know this tool.
cal demonstration of attacks within these solutions often oc- The most optimal solution is to distribute the image of
curs on isolated segments of the application without a cohesive the virtual machine that already contains all the programs

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 273

and tools needed for the interaction with the application. The • Express is used as a web application framework to define
virtual machine image is available on publicly accessible cloud the server.
storage that can be accessed by anyone. Parameters have been • Node.js is a cross-platform environment that runs Type-
defined for the virtual machine to optimize its performance and Script code.
minimize resource utilization from the host system. If required, The following technologies are used for the client part of the
the parameters can be modified by the end users to suit their web application:
needs. • React facilitates the development of interactive, stateful,
The operating system of the virtual machine is Kali Linux, and reusable user interface components.
which is specifically designed for penetration testing. Its main • Bootstrap is a CSS framework providing predefined
advantage lies in the inclusion of numerous pre-installed styles, functions, and components for developing respon-
tools designed specifically for this purpose. In addition, as sive web applications.
it is a Debian-based Linux distribution, its versatility and • HTML and CSS are used in conjunction with the
customizability allow it to adapt to specific needs. Bootstrap framework to define the graphical appearance
E. Design of a database of the application.

When selecting the suitable type of database solution [13], G. Execution of selected cyber-attacks
[14], we considered its widespread adoption, and its ability The final version of the HackHealth application is used
to showcase specific types of application vulnerabilities. The to demonstrate all ten vulnerabilities from the OWASP Top
server side of a deliberately vulnerable application connects 10:2021 list. The vulnerabilities reflect the lessons in the
to the database to retrieve or modify data [15], [16]. The education portal, but there is no direct connection between
implemented solution uses a PostgreSQL relational database. them. In the beginning, the Snyk static code analyzer is
It meets requirements, being among the most sought-after introduced.
database solutions. Additionally, it employs the SQL language Brute force attack is the first attack demonstrated. Its
for data query definitions, which enables us to showcase SQL implementation is also shown in the activity diagram 4
injection attacks and beyond.” Using a Burp Suite tool, the login request is captured and
the attack is configured in the Intruder section. During the
F. Implementation of server and client side attack, requests are sent to the web server containing the
The solution is built using the so-called PERN stack, which user’s fixed email, but the password is changed with values
is an acronym for PostgreSQL, Express, React, and Node.js from a predefined dictionary. The responses from the web
technologies. This set of technologies is designed for the server are verified after the attack. To prevent brute force
development of robust web applications based on a relational attacks, middleware is implemented to prevent more than five
database. It is also a well-established standard. consecutive unsuccessful login attempts.
The monolithic repository is used for structuring the source For the Cryptographic Failures vulnerability, we add
code of the application. Advantages of the proposed approach password hashing, and the expiration of the JWT token is
include simplified dependency management and an application shortened. The SQL injection is firstly demonstrated by three
deployment process. The diagram in Fig. 3 illustrates the preparatory attacks, which allow us to obtain a type of the
technical implementation of the solution. database, its name, as well as the names of all tables. By
performing a tautology attack, the login credentials of all users
are acquired from the application‘s database. Deletion of an
entire table is the last attack in this category.
The fourth place in the OWASP Top 10:2021 is occupied
by an Insecured Design vulnerability. Upon unsuccessful
login, an error message included in the notification contains
information about whether the email or password is invalid.
The prevention is to define generic error messages.
In a deliberately insecure application, a Security Miscon-
figuration was created by not defining a control mechanism
that limits the domains able to access the server. Vulnerable
and Outdated Components represent the initial vulnerability
in the second half of the ranking. Detection of the components
is done by a static code analyzer.
Fig. 3: Architecture diagram. The implementation to prevent the vulnerabilities on the
seventh and eighth positions of the OWASP Top 10:2021
From the PERN stack of technologies, the following are consists only of refactoring the code. Sensitive data such
used for the server side of the solution: as passwords or secret keys are moved to the environment
• PostgreSQL represents a relational database. variables file.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 274

 Fig. 4: Activity diagram of the brute force attack.

The vulnerability of Insufficient Logging and Monitoring is an intentionally insecure web application, was tested by 26
exposed through a simulated DoS attack. This only highlights students. The second part, dealing with a demonstration of
the strong need for proper logging, which is subsequently vulnerabilities and execution of attacks, was tested by a total
implemented by the Winston library. of 36 students. As a result 62 individual subjects participated
The last vulnerability from the list, Server Side Request in the test. At the end of each testing block, the participants
Forgery is demonstrated by inserting a URL into the field were asked to fill out a questionnaire.
within the user report. Using the Burp Suite tools, an attacker The results acquired from the questionnaires confirmed the
captures the request to the server and modifies the URL. Due lack of coverage of the topic of web application development
to the absence of input validation within the web server, the and its security in the curriculum studied by students. The
attacker is able to obtain sensitive data directly from the server. vast majority of students found the educational portal clear,
and intuitive, and the choice of technologies used to develop
VI. R ESULTS the web application was appropriate.
The final version of the product was used for testing the Regarding cybersecurity part, all of the students who par-
web-based learning solution. The testing was carried out by ticipated in the test expressed the necessity for education on
a specific target group, which were second-year students of this topic, which is unfortunately not fully covered in the
the bachelor‘s degree in the field of Computer Science at curriculum of their studies. The lack of awareness of the
the Technical University of Kosice. Since the educational students is evidenced by the fact that only four out of the total
portal consists of two different components, they were tested sample of 36 participants were aware of OWASP organization
separately. The first part, dedicated to the development of and its activities before taking the course. Even up to 80% of

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 275

the students do not use a static code analyzer when developing tools addressing the same problem fail to deliver such precise
school or internal projects to verify their security. The answers guidelines. The educational portal as a whole is written in
to the questionnaire confirm that the educational portal has English, enabling its use for a global audience. The solution
achieved its objective. Up to 86% of the respondents who is also open-source, which permits users to contribute to the
have not yet considered the security aspect of the applications project and further extend it.
they develop, will take it into account after the course.
ACKNOWLEDGMENT
Apart from the results of the questionnaires, the decision
to distribute the learning solution using an image of a pre- This work was supported by Cultural and Educational
configured virtual machine proved to be the correct course Grant Agency (KEGA) of the Ministry of Education, Science,
of action. By adopting this approach, we ensured complete Research and Sport of the Slovak Republic under the project
isolation from the host system, thereby equipping users with No. 060TUKE-4/2022.
all the essential tools and dependencies in advance. R EFERENCES
In contrasting our solution with the prevailing ones, we
[1] S. H. Huseby, ”Innocent code: a security wake-up call for Web program-
managed to exhibit all vulnerabilities from the latest OWASP mers”, 1st ed. New York: John Wiley & Sons, 2004, ISBN: 0470857447.
Top 10:2021 list. Moreover, none of the compared tools offer [2] OWASP Foundation, Inc., ”About the OWASP Foundation,” [Online].
such comprehensive step-by-step instructions to successfully Available: https://owasp.org/about/. Accessed on: Oct. 14, 2022.
[3] S. Groves, ”2014 OWASP Project Handbook,” [Online].
navigate each lesson. Available: https://owasp.org/www-pdf-archive/PROJECT LEADER-
HANDBOOK 2014.pdf. Accessed on: Oct. 14, 2022.
VII. C ONCLUSION [4] R.A. Martin, ”Integrating your information security vulnerability man-
agement capabilities through industry standards,” in SMC’03 Conference
The core intent of this paper was to introduce a thorough Proceedings. 2003 IEEE International Conference on Systems, Man
educational platform dedicated to web application cybersecu- and Cybernetics. Conference Theme - System Security and Assurance
rity education. The result is composed of two interconnected (Cat. No.03CH37483), vol. 2, pp. 1528-1533, 2003. doi: 10.1109/IC-
SMC.2003.1244628.
parts, the first of which is the educational portal itself. The [5] T. Wang, S. Qin, and K. P. Chow, ”Towards Vulnerability Types Classi-
portal contains a detailed description of the preparation of fication Using Pure Self-Attention: A Common Weakness Enumeration
the environment, the development of the web application, Based Approach,” in 2021 IEEE 24th International Conference on
Computational Science and Engineering (CSE), 2021, pp. 146-153.
and its subsequent exposure to attacks. In the second part, Available: doi: 10.1109/CSE53436.2021.00030.
a deliberately insecure application was created. During the [6] J. Ruohonen, S. Rauti, S. Hyrynsalmi, and V. Leppänen, ”A case
implementation phase of the application, it must be ensured study on software vulnerability coordination,” Information and Software
Technology, vol. 103, pp. 239-257, 2018. ISSN: 0950-5849. [Online].
that all vulnerabilities can be effectively demonstrated on this Available: https://doi.org/10.1016/j.infsof.2018.06.005.
platform. Simultaneously, the source code had to be clear [7] OWASP FOUNDATION, Inc., ”OWASP Top10:2021,” [Online]. Avail-
enough, enabling users to identify vulnerabilities and propose able: https://owasp.org/Top10/. Accessed on: Oct. 28, 2022.
[8] S. Disawal and U. Suman, ”An Analysis and Classification of Vulner-
effective solutions to mitigate them. abilities in Web-Based Application Development,” in 2021 8th Interna-
This paper analyzed and compared organizations periodi- tional Conference on Computing for Sustainable Global Development
cally publishing rankings of the most common vulnerabilities. (INDIACom), 2021, pp. 782-785.
[9] N. Nedeljković, N. Vugdelija, and N. Kojić, ”Use of ’OWASP Top 10’ in
The vulnerabilities in the educational portal were selected web application security,” in Fourth International Scientific Conference
based on the most recent OWASP Top 10, 2021 edition. It on Recent Advances in Information Technology, Tourism, Economics,
represents a globally recognized standard. In the develop- Management and Agriculture, 2020, p. 25.
[10] N. A. Aziz, S. N. Z. Shamsuddin, and N. A. Hassan, ”Inculcating
ment phase, prevailing trends in web application creation and Secure Coding for beginners,” in 2016 International Conference on
distribution were assessed, selecting the most advanced and Informatics and Computing (ICIC), 2016, pp. 164-168. Available: doi:
relevant technologies and standards. The concluding section 10.1109/IAC.2016.7905709.
[11] L. Vokorokos, A. Baláž, and B. Madoš, ”Application security through
of this paper depicts the assessment of our proposed plat- sandbox virtualization,” Acta Polytechnica Hungarica, vol. 12, no. 1, pp.
form. A substantial sample of target users participated in the 83-101, 2015.
testing and their evaluation confirmed the positive aspects of [12] L. Vokorokos, M. Ennert, and J. Radušovský, ”A survey of parallel
intrusion detection on graphical processors,” Open Computer Science,
the solution. However, potential areas for improvement or vol. 4, no. 4, pp. 222-230, 2014.
extension were also highlighted. An activity recommended for [13] A. Pekár, M. Chovanec, L. Vokorokos, E. Chovancová, P. Feciľak, and
further development involves enhancing the interactivity of the M. Michalko, ”Adaptive aggregation of flow records,” Computing and
Informatics, vol. 37, no. 1, pp. 142-164, 2018.
portal through the inclusion of quizzes or progress monitoring [14] L. Vokorokos, A. Baláž, and M. Chovanec, ”Distributed detection system
mechanisms. The value of the portal can also be enhanced by of security intrusions based on partially ordered events and patterns,” in
adding additional vulnerabilities and attack scenarios. Towards Intelligent Engineering and Information Technology, pp. 389-
403, 2009.
The distinctiveness of this work stems from its holistic [15] L. Vokorokos, E. Chovancová, J. Radušovský, and M. Chovanec, ”A
approach to the educational platform. It not only imparts multicore architecture focused on accelerating computer vision compu-
knowledge about vulnerabilities and cyber-attacks but also tations,” Acta Polytechnica Hungarica, vol. 10, no. 5, pp. 29-43, 2013.
[16] L. Vokorokos, M. Ennert, and J. Radušovský, ”A survey of parallel
provides users with a detailed procedure for developing a intrusion detection on graphical processors,” Open Computer Science,
web application that is susceptible to these attacks. The main vol. 4, no. 4, pp. 222-230, 2014.
advantage lies in offering exceptionally detailed tutorials. As
elaborated in the analytical part of this paper IV, existing

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 276

 :HE%DVHG9LVXDOL]DWLRQDQG&RQWURORI7UDIILF
6LPXODWLRQ

-.DSXVWDDQG37U~FKO\
6ORYDN8QLYHUVLW\RI7HFKQRORJ\)DFXOW\RI,QIRUPDWLFVDQG,QIRUPDWLRQ7HFKQRORJLHV%UDWLVODYD6ORYDNLD
SHWHUWUXFKO\#VWXEDVN[NDSXVWDM#VWXEDVN


$EVWUDFW ² ,Q WKH SUHVHQW GD\ WKDQNV WR LQWHOOLJHQW VLPXODWLRQDSSOLFDWLRQ:HZLOORXWOLQHWKHIXQFWLRQDOLWLHV
WHFKQRORJLHV LW KDV EHFRPH IHDVLEOH WR HIILFLHQWO\ PDQDJH DQGFRQWUROVDYDLODEOHWRXVHUV,Q6HFWLRQ,9ZHGHVFULEH
WUDQVSRUWQHWZRUNVDQGRSWLPL]HWKHIORZRIWUDIILFLQXUEDQ SUHGHILQHGVLPXODWLRQVFHQDULRVWKDWFDQEHH[SORUHGXVLQJ
DUHDV7KURXJKWKH XVHRI GLJLWDOWZLQVDQGWUDIILFVFHQDULR RXUDSSOLFDWLRQ)LQDOO\LQ6HFWLRQ9ZHZLOOSUHVHQWWKH
VLPXODWLRQV WUDIILF IORZ FDQ EH DQWLFLSDWHG DQG UHILQHG FRQFOXVLRQ RI WKH SDSHU DQG KLJKOLJKW SRWHQWLDO DUHDV IRU
OHDGLQJ WR LPSURYHG PRELOLW\ DQG UHGXFHG FRQJHVWLRQ DQG IXWXUHUHVHDUFKDQGGHYHORSPHQW
HPLVVLRQV 7KHUH DUH VHYHUDO WUDIILF VLPXODWLRQ WRROV DQG
HQYLURQPHQWV WKDW FDQ EH LQVWDOOHG ORFDOO\ DQG XVHG IRU ,, (;$03/(62)75$)),&6,08/$7,21722/6
WUDIILF VLPXODWLRQ +RZHYHU GXH WR YDULRXV DVSHFWV VXFK DV
RQOLQHOHDUQLQJXVHUFRQYHQLHQFHFRPSOH[WUDIILFVFHQDULRV $W SUHVHQW LW LV SRVVLEOH WR ILQG DQG XVH YDULRXV
RUIDVWPRGHODQGWRROXSGDWHVZHEEDVHGVLPXODWLRQFDQEH FRPPHUFLDO DQG RSHQ VRXUFH WRROV SODWIRUPV DQG
EHQHILFLDO ,Q WKLV ZRUN ZH SURSRVHG DQG GHYHORSHG D ZHE HQYLURQPHQWVWKDWFDQEHXVHGIRUDQDO\VLVLQYHVWLJDWLRQ
DSSOLFDWLRQ WKDW XVHV VLPLODU WHFKQRORJLHV WR VLPXODWH WHVWLQJDQGDVVHVVPHQWRISXEOLFWUDQVSRUWV\VWHPV7KHLU
YDULRXV WUDIILF VFHQDULRV DOORZLQJ WKH PRQLWRULQJ RI WKH FRPSDUDWLYHDQDO\VLVFDQEHIRXQGLQ>@DQG>@,QWKLV
HIILFLHQF\ RI WKH SURSRVHG WUDIILF QHWZRUN XQGHU GLIIHUHQW VHFWLRQ ZH GHVFULEH WKUHH WUDIILF VLPXODWLRQ WRROV
FLUFXPVWDQFHV,QDGGLWLRQWRYLVXDOL]LQJWKHVLPXODWLRQWKH 6802$LPVXQDQG9LVVLP WKDWDUHXVHGLQLQGXVWULDO
LPSOHPHQWHG DSSOLFDWLRQ DOORZV XVHUV WR DFWLYHO\ LQWHUYHQH DSSOLFDWLRQVRUDFDGHPLFUHVHDUFK
LQ LWV RSHUDWLRQ 6LPXODWLRQ FRQWURO HQDEOHV XVHUV WR (FOLSVH 6802 6LPXODWLRQ RI 8UEDQ 02ELOLW\  LV DQ
H[SHULPHQW ZLWK GLIIHUHQW FRQILJXUDWLRQV RI WKH WUDQVSRUW RSHQVRXUFHWRROIRUPLFURVFRSLFWUDIILFVLPXODWLRQ>@,Q
QHWZRUNDQGSUHGLFWWKHLULPSDFWRQWUDIILFIORZ DGGLWLRQ WR GLIIHUHQW FODVVHV RI FDUV WKH 6802
VLPXODWLRQLQFOXGHVSHGHVWULDQVELF\FOHVDQGRWKHUW\SHV
RI SXEOLF WUDQVSRUW 6802 LV PDGH XS RI PXOWLSOH RSHQ
, ,1752'8&7,21 VRXUFH WRROV >@ 7KH VLPXODWLRQ FDQ EH YLHZHG LQ WKH
:LWKDQLQFUHDVLQJGHPDQGIRUVXVWDLQDEOHDQGHIILFLHQW JUDSKLFDOVLPXODWRUVXPRJXL7KHWRROFDQDOVRDGMXVWWKH
WUDQVSRUWDWLRQ V\VWHPV LW LV LPSRUWDQW WR KDYH WRROV WKDW VFDOH RI WUDIILF DQG VSHHG RI WKH VLPXODWLRQ 7UDIILF DQG
DOORZ DQDO\VLV DQG RSWLPL]DWLRQ RI WUDIILF IORZ LQ XUEDQ URDG QHWZRUN EXLOGLQJ DQG HGLWLQJ FDQ EH GRQH ZLWK WKH
HQYLURQPHQWV 2QH ZD\ WR GR WKLV LV WKURXJK WKH XVH RI JUDSKLFDOHGLWRUQHWHGLWRUWKURXJK;0/GHVFULSWLRQILOHV
WUDIILF VLPXODWLRQ VRIWZDUH ZKLFK FDQ KHOS DQDO\]H WKH :LWKWKH260:HE:L]DUGWRROLWLVSRVVLEOHWRJHQHUDWH
LPSDFW RI YDULRXV WUDIILF VFHQDULRV RQ WKH PRYHPHQW RI D URDG QHWZRUN DQG WUDIILF GHPDQG IURP WKH UHDO ZRUOG
SHRSOH DQG YHKLFOHV >@ 8VLQJ FRPSXWHU VLPXODWLRQ LQ PDS
RUGHU WR VROYH VFLHQWLILF SUREOHPV LV D IDLUO\ FRPPRQ %RWK WKH WUDQVSRUW QHWZRUN DQG WKH EHKDYLRXU RI
DSSURDFK)RUH[DPSOHLWLVXVHGGXULQJWKHGHYHORSPHQW WUDQVSRUWGHPDQGFDQEHVLJQLILFDQWO\DGDSWHG:KHWKHULW
RU SODQQLQJ RI D V\VWHP ,W LV XVHIXO EHFDXVH WKH LV GLIIHUHQW W\SHV RI YHKLFOHV ZLWK WKHLU RZQ VSHHG ODQH
H[SHULPHQWSHUIRUPHGLQDQDUWLILFLDOHQYLURQPHQWFDQEH FKDQJHUDWHURXWHFKRLFHRUWUDIILFOLJKWVWKDWFDQEHVHW
SHUIRUPHGPXFKIDVWHUWKDQLQWKHUHDOZRUOG>@ LQ VHYHUDO ZD\V ,W FDQ EH D SHULRGLF VHWWLQJ ZKHUH LQ
,Q WKLV SDSHU RXU IRFXV LV RQ H[SORULQJ DQG DQDO\]LQJ DGGLWLRQ WR WKH UHFRPPHQGHG VHWWLQJV \RX FDQ DOVR
DSSOLFDWLRQV WKDW RIIHU WRROV WR VLPXODWH UHDOLVWLF WUDIILF GHWHUPLQH \RXU RZQ LQWHUYDOV RI LQGLYLGXDO SKDVHV 2U
VFHQDULRV >@ :LWK WKH LQFUHDVLQJ FRPSOH[LW\ RI XUEDQ G\QDPLF VHWWLQJV ZKHUH WKH OHQJWKV RI SHULRGV DUH
WUDQVSRUWDWLRQ V\VWHPV LW KDV EHFRPH FUXFLDO WR GHYHORS G\QDPLFDOO\FKDQJHGDFFRUGLQJWRYDULRXVWUDIILFIDFWRUV
HIIHFWLYHVLPXODWLRQSODWIRUPVWKDWFDQDFFXUDWHO\UHSOLFDWH ,QWHUDFWLRQZLWKWKH6802FDQEHFXVWRPL]HGXVLQJWRROV
UHDOZRUOG WUDIILF FRQGLWLRQV 7R DGGUHVV WKLV QHHG ZH VXFK DV 7UD&, 7UDIILF &RQWURO ,QWHUIDFH  ZKLFK
KDYHGHYHORSHGDZHEEDVHGDSSOLFDWLRQWKDWDOORZVXVHUV IXQFWLRQV DV D EULGJH WR 6802 LQ WKH 7&3 EDVHG
WR VLPXODWH WKH EHKDYLRU RI WUDIILF LQ YDULRXV WUDIILF FOLHQWVHUYHU DUFKLWHFWXUH ZKHUH 6802 LV WKH VHUYHU
VFHQDULRVDQGJDLQFRQWURORYHUWKHIORZRIYHKLFOHV2QH 7UD&, FDQ UHWULHYH LQIRUPDWLRQ DERXW YHKLFOHV SHUVRQV
RIWKHNH\DGYDQWDJHVRIRXUZHEEDVHGDSSOLFDWLRQLVLWV URXWHV ODQH DUHD GHWHFWRUV LQWHUVHFWLRQV WUDIILF OLJKWV
DFFHVVLELOLW\ DQG HDVH RI XVH %\ EHLQJ ZHEEDVHG XVHUV SDUNLQJ DUHDV HWF 0RUHRYHU LW LV FDSDEOH RI ZRUNLQJ
DUH IUHHG IURP WKH QHHG WR LQVWDOO WKH DSSOLFDWLRQ RQ D ZLWK WKH LQIRUPDWLRQ REWDLQHG DQG FKDQJLQJ VWDWHV RI
VSHFLILFGHYLFHDVWKH\FDQVLPSO\DFFHVVLWWKURXJKDZHE REMHFWV 7UD&, LV LPSOHPHQWHG DV D OLEUDU\ LQ 3\WKRQ
EURZVHU HYDOXDWH WKH LPSDFW RI SURSRVHG LQWHUYHQWLRQV &-DYDDQG0DWODE>@
DQGRSWLPL]HWUDIILFIORZPDQDJHPHQWVWUDWHJLHV $LPVXQ 1H[W >@ LV OLFHQVHG VRIWZDUH ZLWK D PRGHO
,Q 6HFWLRQ ,, ZH SUHVHQW H[DPSOHV RI H[LVWLQJ WUDIILF HGLWRU ZKLFK LQ DGGLWLRQ WR EDVLF IXQFWLRQV LQFOXGLQJ
VLPXODWLRQ WRROV WKDW DUH FRPPRQO\ XVHG LQ WKH ILHOG FUHDWLQJ DQG HGLWLQJ URDGV OLQHV MXQFWLRQV HWF FDQ DOVR
6HFWLRQ,,,IRFXVHVRQWKHVSHFLILFDWLRQVRIRXUZHEEDVHG DGG EXV URXWHV DQG EXV VWRSV DGG EDFNJURXQGV DVVLJQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 277

URDGW\SHVDQGPXFKPRUH,QDGGLWLRQWRFUHDWLQJDURXWH
QHWZRUNIURPVFUDWFKLWLVSRVVLEOHWRLPSRUWDPDSIURP
D ILOH RU IURP WKH ,QWHUQHW 7KH VRIWZDUH SURYLGHV
PDFURVFRSLF PLFURVFRSLF DQG PHVRVFRSLF VLPXODWLRQV
7KH UXQQLQJ VLPXODWLRQ FDQ EH DOWHUHG E\ FKDQJLQJ LWV
VSHHG DQG VPRRWKQHVV ,W LV DOVR SRVVLEOH WR IDVW IRUZDUG
WKH VLPXODWLRQ UHFRUG D YLGHR RU UXQ D ' DQLPDWHG
VLPXODWLRQ
$QRWKHU REMHFWLYH RI $LPVXQ 1H[W LV WR DQDO\VH DQG
SUHVHQW WKH GDWD IURP WKH VLPXODWLRQ 5HVXOWV FDQ EH
SUHVHQWHGLQWKHIRUPRIYDULRXVJUDSKVSORWVDQGWDEOHV
6RPH RSHUDWLRQV FDQ EH DXWRPDWHG XVLQJ FXVWRP VFULSWV
ZULWWHQLQ3\WKRQ
3799LVVLPLVFRPPHUFLDOWUDIILFVLPXODWLRQVRIWZDUH 
,W LV ZLGHO\ XVHG LQ WUDQVSRUWDWLRQ SODQQLQJ DQG )LJXUH 0DLQSDJHVFUHHQ
HQJLQHHULQJ WR PRGHO DQG VLPXODWH FRPSOH[ WUDIILF
VFHQDULRV >@ 7KH VLPXODWLRQV LQ 379 9LVVLP DUH PRGLILFDWLRQV LQ WKH WUDIILF QHWZRUN DQG HQYLURQPHQWDO
PLFURVFRSLF PHVRVFRSLF DQG K\EULG 0LFURVFRSLF SDUDPHWHUV
VLPXODWLRQ PRGHOV LQGLYLGXDO WUDIILF XVHUV DQG WKHLU
EHKDYLRXU LQ GHWDLO 7KH PHVRVFRSLF VLPXODWLRQ LV QRW DV $ %DVLF)XQFWLRQV
GHWDLOHG DV WKH PLFURVFRSLF VLPXODWLRQ 7KLV PHDQV WKDW 7KH DSSOLFDWLRQ RIIHUV D XVHUIULHQGO\ DQG LQWXLWLYH
WKH VLPXODWLRQ LV PRUH HIILFLHQW DQG KDV PRUH SRZHU WR LQWHUIDFH PDNLQJ LW UHODWLYHO\ VLPSOH WR XVH 8SRQ
VLPXODWH QHWZRUNV RI ODUJHU VL]H 379 9LVVLP RIIHUV ODXQFKLQJ WKH WUDIILF VLPXODWLRQ DSSOLFDWLRQ XVHUV DUH
FRPSUHKHQVLYH RXWSXW GDWD RQ VLPXODWLRQV SURYLGLQJ JUHHWHGZLWKDVFUHHQVKRZQLQ)LJGLVSOD\LQJDQLPDJH
XVHUV ZLWK GHWDLOHG LQIRUPDWLRQ ,W DOVR LQFOXGHV D ZLWK DQ HPSW\ VSDFH UHSUHVHQWLQJ WKH WUDIILF QHWZRUN
SXEOLVKHU WRRO WKDW IDFLOLWDWHV WKH FUHDWLRQ RI LQWHUDFWLYH $GMDFHQW WR WKH QHWZRUN VSDFH LV D PHQX WDE ZKLFK
GDVKERDUGVDOORZLQJIRUHDV\DQDO\VLVRIWUDIILFSDWWHUQV SURYLGHV XVHUV ZLWK YDULRXV RSWLRQV DQG IXQFWLRQDOLWLHV
7KH VRIWZDUH DOORZV XVHUV WR FUHDWH DQG RYHUVHH :LWKLQWKHPHQXWDEXVHUVFDQDFFHVVDVHOHFWLRQRISUH
PRGLILFDWLRQV WR WKH PRGHO VXFK DV LPSURYHPHQWV LQ GHILQHG VFHQDULRV WR FKRRVH IURP 2QFH D VFHQDULR LV
LQIUDVWUXFWXUH QHZ WUDQVLW OLQHV SRSXODWLRQ SURMHFWLRQV VHOHFWHG WKH XVHU FDQ LQLWLDWH WKH VLPXODWLRQ E\ FOLFNLQJ
DQG]RQLQJUHJXODWLRQV WKH VWDUW EXWWRQ 7KH DSSOLFDWLRQ WKHQ ORDGV WKH FKRVHQ
,Q DGGLWLRQ LW RIIHUV YDOXDEOH LQIRUPDWLRQ RQ YDULRXV VFHQDULRDQGWKHVLPXODWLRQEHJLQV8VHUVFDQLPPHGLDWHO\
DVSHFWVRISXEOLFWUDQVSRUWV\VWHPV,WSURYLGHVEDVLFXVHU REVHUYH WKH G\QDPLFDOO\ JHQHUDWHG WUDIILF QHWZRUN ZLWK
H[SHULHQFH LQIRUPDWLRQ LQFOXGLQJ WUDYHO WLPHV ZDONLQJ YHKLFOHVPRYLQJDQGWUDIILFIORZFKDQJLQJLQUHDOWLPH
WLPH DQG IDUHV 7KH VRIWZDUH DOVR HQDEOHV LQIRUPHG 'XULQJ VLPXODWLRQ WKH DSSOLFDWLRQ SURYLGHV VHYHUDO
GHFLVLRQPDNLQJLQSXEOLFWUDQVSRUWSODQQLQJ EDVLF IXQFWLRQDOLWLHV WR HQKDQFH XVHU FRQWURO 8VHUV FDQ
SDXVHRUUHVXPHWKHVLPXODWLRQDWDQ\WLPHDOORZLQJIRUD
,,, 63(&,),&$7,2162)7+($33/,&$7,21 FORVHUH[DPLQDWLRQRIVSHFLILFVLWXDWLRQVRUGHWDLOVZLWKLQ
7KH SULPDU\ REMHFWLYH RI WKH GHYHORSHG DSSOLFDWLRQ LV WKHWUDIILFQHWZRUN$GGLWLRQDOO\WKHVLPXODWLRQVSHHGFDQ
WR SURYLGH D FRPSUHKHQVLYH SODWIRUP IRU VLPXODWLQJ EH DGMXVWHG WR VLPXODWH GLIIHUHQW WLPH IUDPHV RU REVHUYH
WUDIILF LQ YDULRXV VFHQDULRV ZKLOH DOORZLQJ UHDOWLPH WKH WUDIILF EHKDYLRU PRUH FORVHO\ 7KH DSSOLFDWLRQ DOVR
PRGLILFDWLRQV WR WKH VLPXODWLRQ $ NH\ UHTXLUHPHQW LV WR IHDWXUHVDWUDIILFGHQVLW\VFDOLQJRSWLRQDOORZLQJXVHUVWR
SURYLGHXVHUVZLWKWKHDELOLW\WRDGMXVWYDULRXVDVSHFWVRI LQFUHDVH RU GHFUHDVH WKH GHQVLW\ RI YHKLFOHV ZLWKLQ WKH
WKH WUDIILF IORZ LQWHUDFW ZLWK LQGLYLGXDO WUDIILF DJHQWV VLPXODWHG WUDIILF IORZ 7KHVH IXQFWLRQDOLWLHV DUH HDVLO\
VXFK DV YHKLFOHV ODQHV DQG WUDIILF OLJKWV DQG PRGLI\ DFFHVVLEOHWKURXJKEXWWRQVDQGVOLGHUVORFDWHGLQWKHPHQX
WKHLU FRQVWUDLQWV DQG EHKDYLRU WR DQDO\]H WKH UHVXOWLQJ DWWKHWRSRIWKHDSSOLFDWLRQLQWHUIDFH
HIIHFWV
% $GYDQFHG)XQFWLRQV
7RDFFRPSOLVKWKLVWKHDSSOLFDWLRQ ZDVGHVLJQHGDVD
ZHEEDVHGFOLHQWVHUYHUV\VWHPWKDWRIIHUVIOH[LELOLW\DQG 7KHDSSOLFDWLRQRIIHUVLQWHUDFWLYHFDSDELOLWLHVZLWKWKUHH
DFFHVVLELOLW\ ,W FDQ EH DFFHVVHG WKURXJK VWDQGDUG ZHE GLVWLQFWHQWLWLHVZLWKLQWKHVLPXODWLRQYHKLFOHVHGJHVDQG
EURZVHUV HOLPLQDWLQJ WKH QHHG IRU XVHUV WR LQVWDOO WUDIILF OLJKWV %\ PDQLSXODWLQJ WKHVH LQGLYLGXDO HQWLWLHV
VSHFLDOL]HG VRIWZDUH ORFDOO\ 7KH FOLHQWVLGH LQWHUIDFH XVHUV FDQ VLPXODWH VFHQDULRV LQ D KLJKO\ FXVWRPL]HG
SURYLGHVDXVHUIULHQGO\HQYLURQPHQWIRULQWHUDFWLQJZLWK PDQQHU UHVXOWLQJ LQ PRUH LQWULFDWH DQG GHWDLOHG WUDIILF
WKH VLPXODWLRQ ZKLOH WKH VHUYHUVLGH LQIUDVWUXFWXUH VLWXDWLRQV7KHVWDWHVDQGEHKDYLRUVRIWKHVHREMHFWVFDQEH
KDQGOHV WKH FRPSXWDWLRQDO FDOFXODWLRQV UHTXLUHG IRU WKH FRQWUROOHG E\ VHOHFWLQJ WKHP ZLWKLQ WKH QHWZRUN VFUHHQ
VLPXODWLRQ VH[HFXWLRQ SURPSWLQJWKHGLVSOD\RIDQRSWLRQVWDEPHQXRQWKHULJKW
KDQG VLGH RI WKH DSSOLFDWLRQ LQWHUIDFH 7KLV PHQX RI
2QHRIWKHFRUHWHFKQRORJLHVXWLOL]HGLQWKHDSSOLFDWLRQ RSWLRQV SURYLGHV HVVHQWLDO LQIRUPDWLRQ DERXW WKH VHOHFWHG
LV 6802 7UD&, 6802 7UD&, VHUYHV DV D EULGJH LWHPDQGRIIHUVYDULRXVFRQWUROVIRUFXVWRPL]DWLRQ
EHWZHHQ WKH DSSOLFDWLRQ DQG WKH XQGHUO\LQJ 6802
IUDPHZRUNIDFLOLWDWLQJVHDPOHVVFRPPXQLFDWLRQDQGGDWD :KHQ DQ HGJH LV VHOHFWHG WKH RSWLRQV PHQX )LJ  
H[FKDQJH 8VLQJ 6802 7UD&, WKH DSSOLFDWLRQ FDQ SUHVHQWV SHUWLQHQW GHWDLOV VSHFLILF WR WKH VHOHFWHG HGJH
HIIHFWLYHO\FRQWURODQGPDQLSXODWHYDULRXVDVSHFWVRIWKH LQFOXGLQJ LWV XQLTXH LGHQWLILHU ,'  VWUHHW QDPH LI
VLPXODWLRQ VXFK DV YHKLFOH EHKDYLRU UHDFWLRQV WR DSSOLFDEOH  PD[LPXP DOORZHG VSHHG LQ NLORPHWHUV SHU
FKDQJLQJ WUDIILF FRQGLWLRQV DQG UHVSRQVHV WR

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 278

 

)LJXUH 9HKLFOH2SWLRQV0HQX
)LJXUH (GJH2SWLRQV0HQX

KRXU DYHUDJH VSHHG RI YHKLFOHV LQ NLORPHWHUV SHU KRXU WUDIILFOLJKWVIRUHDFKSRVVLEOHFURVVLQJDWWKHLQWHUVHFWLRQ
DQGDOLVWRIDXWKRUL]HGHQWLWLHVSHUPLWWHGWRHQWHUWKHHGJH DORQJ ZLWK WKH GXUDWLRQ LQ VHFRQGV IRU ZKLFK LW UHPDLQV
DFWLYH
7KH FRQWURO EXWWRQV ZLWKLQ WKH HGJH RSWLRQV PHQX
HQDEOHXVHUVWRPRGLI\WKHDSSOLFDWLRQ VYLHZE\FHQWHULQJ (DFK VWDWH LV UHSUHVHQWHG E\ DQ REMHFW WKDW LQFOXGHV D
LWRQWKHVHOHFWHGHGJH$GGLWLRQDOO\XVHUVFDQDGMXVWWKH QXPHULFDOYDOXHIRUWKHGXUDWLRQDQGDVWULQJFRQVLVWLQJRI
PD[LPXPDOORZHGVSHHGE\HQWHULQJDQXPHULFDOYDOXHLQ WKUHHW\SHVRIOHWWHUVUIRUUHG\IRU\HOORZDQGJ
WKHSURYLGHGLQSXWILHOGDQGFRQILUPLQJWKHDFWLRQ IRU JUHHQ 7KH OHQJWK RI WKH VWULQJ FRUUHVSRQGV WR WKH
QXPEHU RI SRVVLELOLWLHV IRU FURVVLQJ WKH LQWHUVHFWLRQ $OO
)XUWKHUPRUHWKHDSSOLFDWLRQDOORZVIRULQWHUDFWLRQZLWK VWDWHV ZLWKLQ WKH SURJUDP DUH GLVSOD\HG LQ D WDEOH ZLWKLQ
YHKLFOHV :KHQ D YHKLFOH LV VHOHFWHG WKH RSWLRQV PHQX WKHRSWLRQPHQX%\VHOHFWLQJDURZLQWKHWDEOHXVHUVFDQ
VKRZQ LQ )LJ  GLVSOD\V IXQGDPHQWDO LQIRUPDWLRQ DERXW
PDQLSXODWH WKH SURJUDP VWDWH 7KH SDUDPHWHUV RI D
WKHFKRVHQYHKLFOHVXFKDVLWV,'WKHW\SHRIURDGXVHULW
VHOHFWHGVWDWHDUHSURMHFWHGLQWRLQSXWILHOGVDOORZLQJIRU
UHSUHVHQWVDQGWKH,'RIWKHHGJHLWLVFXUUHQWO\SRVLWLRQHG PRGLILFDWLRQ &KDQJHV FDQ EH FRQILUPHG E\ SUHVVLQJ WKH
RQ LIDQHGJHKDVEHHQSUHYLRXVO\VHOHFWHG 7KHFRQWUROV 6DYH (GLW EXWWRQ $GGLWLRQDOO\ WKH $GG 6WDWH EXWWRQ
DYDLODEOH LQ WKH YHKLFOH RSWLRQV PHQX DOORZ XVHUV WR
HQDEOHVWKHDGGLWLRQRI D QHZ VWDWHWR WKH SURJUDP EDVHG
FXVWRPL]HWKHLUYLHZLQJH[SHULHQFHE\FHQWHULQJLWRQWKH
RQWKHYDOXHVVSHFLILHGLQWKHLQSXWILHOGV&RQYHUVHO\WKH
VHOHFWHG YHKLFOH RU HQDEOLQJ DFWLYH IROORZLQJ 5HPRYH6WDWHEXWWRQUHPRYHVWKHVHOHFWHGVWDWHIURPWKH
$GGLWLRQDOO\ XVHUV KDYH WKH DELOLW\ WR VLPXODWH VSHFLILF SURJUDP 'XULQJ VLPXODWLRQ WKH SURJUDP G\QDPLFDOO\
VFHQDULRV E\ VWRSSLQJ RU UHVXPLQJ WKH PRYHPHQW RI WKH VZLWFKHV EHWZHHQ WKH GHILQHG VWDWHV DW VSHFLILF LQWHUYDOV
YHKLFOH WKHUHE\ UHSOLFDWLQJ DFWLRQV OLNH DFFLGHQWV RU
%\SUHVVLQJWKH6HW6WDWHEXWWRQWKHWUDIILFOLJKWVUHPDLQ
XQSODQQHGVWRSV$QRWKHUQRWDEOHIXQFWLRQDOLW\SURYLGHGLV LQ WKH VHOHFWHG VWDWH XQWLO WKH 5HVHW 3URJUDP EXWWRQ LV
WKH DELOLW\ WR PRGLI\ WKH YHKLFOH V GHVWLQDWLRQ +RZHYHU SUHVVHG 7KLV DFWLRQ LQLWLDWHV WKH UHVXPSWLRQ RI VWDWH
WKLV RSWLRQ EHFRPHV DFFHVVLEOH RQO\ ZKHQ DQ HGJH LV FLUFXODWLRQZLWKLQWKHSURJUDP
VHOHFWHG :KHQ WKLV IXQFWLRQ LV DFWLYDWHG WKH DSSOLFDWLRQ
VWULYHVWRLGHQWLI\DVXLWDEOHURXWHWRWKHQHZGHVWLQDWLRQ
,9 6,08/$7,216&(1$5,26
,I WKH YHKLFOH LV QRW DXWKRUL]HG WR HQWHU WKH VHOHFWHG
GHVWLQDWLRQ RU LI D URXWH FDQQRW EH IRXQG WKH SUHYLRXV $WWKHVWDUWRIWKHVLPXODWLRQXVHUVKDYHWKHRSWLRQRI
GHVWLQDWLRQ LV UHWDLQHG 7R HQKDQFH XVHUV  FRPSUHKHQVLRQ VHOHFWLQJ IURP VHYHUDO SUHGHILQHG VFHQDULRV RU XSORDGLQJ
RIYHKLFOHEHKDYLRUWKHDSSOLFDWLRQKLJKOLJKWVWKHURXWHRI WKHLU RZQ VFHQDULR ILOHV 3UHGHILQHG VFHQDULRV LQFOXGH
WKHVHOHFWHGYHKLFOHZLWKLQWKHQHWZRUN VLPXODWLRQV RI UHDOZRUOG ORFDWLRQV ZKLFK DUH XVHG WR
7KHWKLUGW\SHRIRSWLRQVPHQXLVGHVLJQHGVSHFLILFDOO\ VLPXODWH DQG FRQWURO WUDIILF EHKDYLRU XQGHU UHDOLVWLF
IRUWUDIILFOLJKWV )LJ ,WVSULPDU\SXUSRVHLVWRSURYLGH FRQGLWLRQV$QRWKHUJURXSRIVFHQDULRVIRFXVHVRQWHVWLQJ
FRQWURO RYHU WKH VWDWHV DQG SURJUDP RI JURXSV RI WUDIILF WKHHIILFLHQF\RILQWHUVHFWLRQVHDFKVFHQDULRFRQVLVWLQJRI
OLJKWV7KHSURJUDPFRQVLVWVRIDVHTXHQFHRIVWDWHVZKHUH DQHWZRUNZLWKRQO\RQHLQWHUVHFWLRQRIDGLIIHUHQWVKDSH
HDFKVWDWHUHSUHVHQWVWKHFXUUHQWFRORUFRQILJXUDWLRQRIWKH $OO LQWHUVHFWLRQV LQ WKLV JURXS KDYH WKH VDPH GHILQHG

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 279

 DV GDVKHG EURZQ OLQHV :LWKLQ WKLV VFHQDULR ZH KDYH
LQFRUSRUDWHG YDULRXV W\SHV RI WUDIILF OLJKWV WR VLPXODWH
GLIIHUHQW FRQWURO PHFKDQLVPV )LUVW ZH KDYH LQFOXGHG
VWDWLF WUDIILF OLJKWV ZKLFK KDYH SUHGHWHUPLQHG DQG IL[HG
GXUDWLRQVIRUHDFKVWDWH$GGLWLRQDOO\ZHKDYHLQWHJUDWHG
GHOD\EDVHG WUDIILF OLJKWV WKDW G\QDPLFDOO\ DGMXVW WKHLU
WLPLQJV EDVHG RQ WKH SUHVHQFH RI YHKLFOHV DQG WKH
UHVXOWLQJ WLPH ORVV 7KH WKLUG W\SH RI WUDIILF OLJKWV
LPSOHPHQWHGLQRXUVFHQDULRLVWKHDFWXDWHGWUDIILFOLJKWV
:LWK DFWXDWHG VHWWLQJV WKHVH WUDIILF OLJKWV FDQ GHWHFW
FRQWLQXRXVVWUHDPVRILQFRPLQJ YHKLFOHV IURPDVSHFLILF
GLUHFWLRQ DQG LQWHOOLJHQWO\ SURORQJ WKH JUHHQ VLJQDO WKXV
RSWLPL]LQJ WUDIILF IORZ ,W LV LPSRUWDQW WR QRWH WKDW WKH
G\QDPLF VHWWLQJV RI DFWXDWHG WUDIILF OLJKWV DUH QRW VROHO\
ERXQGWRVWDWHGXUDWLRQEXWWKH\DOVRFRQVLGHUPLQLPXP
DQG PD[LPXP DOORZHG GXUDWLRQV WR HQVXUH HIILFLHQW
RSHUDWLRQ7KLVUHDOZRUOGVFHQDULRRI3H]LQRNFLW\VHUYHV
DV D SUDFWLFDO GHPRQVWUDWLRQ RI WKH FDSDELOLWLHV RI RXU
GHYHORSHG DSSOLFDWLRQ ,W VKRZFDVHV WKH LQWHJUDWLRQ RI
GLIIHUHQW URDG W\SHV GLYHUVH WUDIILF OLJKW FRQWURO
PHFKDQLVPV DQG WKH IOH[LELOLW\ WR VLPXODWH UHDOLVWLF
WUDIILFVFHQDULRV
% 6WDWLVWLFDO2XWSXWRI6FHQDULRV
,QDGGLWLRQWRLWVFRPSUHKHQVLYHVLPXODWLRQFDSDELOLWLHV
RXU DSSOLFDWLRQ DOVR SURYLGHV YDOXDEOH VWDWLVWLFDO RXWSXWV
IRU HDFK VFHQDULR 7KHVH VWDWLVWLFV DUH JHQHUDWHG DQG
SUHVHQWHGRQFHWKHVLPXODWLRQLVFRPSOHWHRIIHULQJXVHUVD
 PRUHLQGHSWKXQGHUVWDQGLQJDQGDQDO\VLVRIWKHVLPXODWHG
)LJXUH 7UDIILF/LJKWV2SWLRQ0HQX WUDIILF VWUXFWXUH DQG MXQFWLRQV %\ SUHVHQWLQJ VWDWLVWLFDO
GDWDDORQJVLGHWKHYLVXDOUHSUHVHQWDWLRQRIWKHVLPXODWLRQ
WUDIILF IORZ DOORZLQJ DQ REMHFWLYH FRPSDULVRQ RI RXU DSSOLFDWLRQHQDEOHV XVHUVWR JDLQGHHSHU LQVLJKWVLQWR
LQWHUVHFWLRQHIILFLHQF\ WKH HIIHFWLYHQHVV RI GLIIHUHQW WUDIILF FRQILJXUDWLRQV DQG
PDNH LQIRUPHG GHFLVLRQV UHJDUGLQJ WUDIILF PDQDJHPHQW
$ 6SHFLILF6FHQDULR'HVFULSWLRQ DQGRSWLPL]DWLRQ7KHVWDWLVWLFDORXWSXWVRIIHUTXDQWLWDWLYH
$Q H[DPSOH RI D UHDOZRUOG VFHQDULR LV 3H]LQRN FLW\ PHDVXUHVVXFKDVDYHUDJHYHKLFOHVSHHGZDLWLQJWLPHWULS
:H KDYH FUHDWHG D QHWZRUN WKDW HQFRPSDVVHV D GXUDWLRQ DQG YHKLFOH FRXQW DPRQJ RWKHUV VHH )LJ  
VLJQLILFDQWSRUWLRQRIWKHFLW\HIIHFWLYHO\UHSUHVHQWLQJDOO 7KHVHPHWULFVSURYLGHDPRUHREMHFWLYHHYDOXDWLRQRIWKH
WKH IXQFWLRQDOLWLHV WKDW RXU GHYHORSHG DSSOLFDWLRQ LV WUDIILFVFHQDULRDOORZLQJXVHUVWRDVVHVVWKHHIILFLHQF\DQG
FDSDEOH RI 7KH QHWZRUN PDS GHSLFWHG LQ )LJ  VKRZV SHUIRUPDQFHRIWKHVLPXODWHGQHWZRUN
WKH GLIIHUHQW W\SHV RI URDGV DQG LQIUDVWUXFWXUH HOHPHQWV 7R IXUWKHU LOOXVWUDWH WKH HIIHFWLYHQHVV RI GLIIHUHQW
5RDGV DUH UHSUHVHQWHG E\ VROLG EODFN OLQHV SDWKV DUH LQWHUVHFWLRQVFHQDULRVZHKDYHJHQHUDWHGDJUDSK )LJXUH
GHQRWHGE\WKLQQHUJUD\ OLQHVDQGUDLOURDGVDUHGHSLFWHG   WKDW FRPSDUHV WKHLU SHUIRUPDQFH EDVHG RQ YHKLFOH


)LJXUH 5XQQLQJ6LPXODWLRQRI3H]LQRN

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 280

 
)LJXUH 6LPXODWLRQ6WDWLVWLFV3DQHO

VSHHG7KLVJUDSKVHUYHVDVDYLVXDOUHSUHVHQWDWLRQRIWKH WKHLU RZQ VFHQDULRV WDLORUHG WR VSHFLILF UHVHDUFK

GDWD DOORZLQJ IRU D FOHDUHU XQGHUVWDQGLQJ RI WKH UHODWLYH TXHVWLRQV XUEDQ HQYLURQPHQWV RU UHDOZRUOG VFHQDULRV
HIILFLHQF\ RI HDFK LQWHUVHFWLRQ W\SH 2Q REVHUYLQJ WKH WKH\ ZLVK WR VLPXODWH 7KLV IOH[LELOLW\ LPSURYHV WKH
JUDSK LW EHFRPHV HYLGHQW WKDW WKH FORYHUOHDI LQWHUVHFWLRQ DSSOLFDELOLW\ RI WKH DSSOLFDWLRQ LQ YDULRXV FRQWH[WV
RXWSHUIRUPV WKH VLPSOHU LQWHUVHFWLRQV LQ WHUPV RI YHKLFOH LQFOXGLQJWUDIILFSODQQLQJRSWLPL]DWLRQDQGUHVHDUFK
VSHHG7KH GDWD FOHDUO\ VKRZ WKDW YHKLFOHV WUDYHUVLQJ WKH
FORYHUOHDILQWHUVHFWLRQH[SHULHQFHKLJKHUVSHHGVFRPSDUHG 9 &21&/86,21
WRWKRVHQDYLJDWLQJWKURXJKWKHVLPSOHULQWHUVHFWLRQV7KLV
LQGLFDWHV WKDW WKH GHVLJQ DQG OD\RXW RI WKH FORYHUOHDI ,Q FRQFOXVLRQ WKLV SDSHU KDV SUHVHQWHG D
LQWHUVHFWLRQ FRQWULEXWH WR VPRRWKHU WUDIILF IORZ DQG FRPSUHKHQVLYH DQDO\VLV RI D ZHEEDVHG DSSOLFDWLRQ
UHGXFHGFRQJHVWLRQUHVXOWLQJLQLPSURYHGRYHUDOOYHKLFOH GHVLJQHG WR VLPXODWH UHDOLVWLF WUDIILF VFHQDULRV 7KH
VSHHG DSSOLFDWLRQ RIIHUV D UDQJH RI IHDWXUHV DQG IXQFWLRQDOLWLHV
WR SURYLGH XVHUV ZLWK WKH DELOLW\ WR XQGHUVWDQG DQG
& &XVWRP6FHQDULRV RSWLPL]H WUDIILF IORZ LQ XUEDQ DUHDV 8VLQJ LQWHOOLJHQW
$QRWKHU QRWDEOH IXQFWLRQ RI WKH DSSOLFDWLRQ LV WKH WHFKQRORJLHV WKH DSSOLFDWLRQ HQDEOHV XVHUV WR YLVXDOL]H
DELOLW\ WR XSORDG FXVWRP VFHQDULRV ZKLFK VHUYHV DV D DQG PDQLSXODWH YDULRXV DVSHFWV RI WKH WUDIILF QHWZRUN
VXEVWLWXWH IRU D GHGLFDWHG QHWZRUN HGLWRU 7KLV IHDWXUH LQFOXGLQJYHKLFOHVHGJHVDQGWUDIILFOLJKWV
RIIHUV XVHUV WKH IOH[LELOLW\ WR FUHDWH DQG WHVW PXOWLSOH 7KH VSHFLILFDWLRQV RI RXU DSSOLFDWLRQ GHPRQVWUDWH LWV
WUDIILF VFHQDULRV ZLWKLQ WKH DSSOLFDWLRQ +RZHYHU LW LV FDSDELOLWLHV LQ WHUPV RI VLPXODWLQJ WUDIILF EHKDYLRU
LPSRUWDQW WR QRWH WKDW WKH SURFHVV RI FUHDWLQJ FXVWRP SURYLGLQJ XVHU FRQWUROV DQG JHQHUDWLQJ VWDWLVWLFDO RXWSXW
VFHQDULRVPD\UHTXLUHVRPHWHFKQLFDOH[SHUWLVH7RFUHDWH IRU GHWDLOHG DQDO\VLV )XUWKHUPRUH WKH VLPXODWLRQ
D FXVWRP VFHQDULR WKH XVHU QHHGV WR SURYLGH WZR ;0/ VFHQDULRV LOOXVWUDWH WKH DSSOLFDWLRQ V YHUVDWLOLW\ LQ
ILOHVRQHIRUYHKLFOHWULSVDQGDQRWKHUIRUWKHVWUXFWXUHRI PRGHOLQJ GLIIHUHQW WUDIILF VFHQDULRV DQG HYDOXDWLQJ WKHLU
WKHWUDIILFQHWZRUN7KH;0/ILOHFRQWDLQLQJYHKLFOHWULSV LPSDFWRQWUDIILFIORZ7KHLQFOXVLRQRIFXVWRPVFHQDULR
KDV D WULSV H[WHQVLRQ ZKLOH WKH ILOH UHSUHVHQWLQJ WKH XSORDGLQJIXQFWLRQDOLW\IXUWKHUHQKDQFHVWKHDSSOLFDWLRQ V
WUDIILF QHWZRUN VWUXFWXUH KDV D QHW H[WHQVLRQ 'HWDLOHG IOH[LELOLW\ DOORZLQJ XVHUV WR FUHDWH DQG WHVW WKHLU RZQ
LQVWUXFWLRQVRQWKHIRUPDWWLQJDQGVWUXFWXUHRIWKHVH;0/ WUDIILFVFHQDULRV
ILOHVFDQEHIRXQGRQWKH6802'RFXPHQWDWLRQSDJH>@ ,QJHQHUDOWKHZHEEDVHGDSSOLFDWLRQGHYHORSHGRIIHUV
%\ DOORZLQJ XVHUV WR XSORDG FXVWRP VFHQDULRV WKH D YDOXDEOH UHVRXUFH IRU VWXG\LQJ DQDO\]LQJ DQG
DSSOLFDWLRQ HQDEOHV D PRUH GLYHUVH DQG FRPSUHKHQVLYH RSWLPL]LQJ WUDIILF VFHQDULRV %\ FRPELQLQJ YLVXDO
HYDOXDWLRQRIGLIIHUHQWWUDIILFVLWXDWLRQV8VHUVFDQFUHDWH UHSUHVHQWDWLRQV LQWHUDFWLYH FRQWUROV DQG VWDWLVWLFDO
DQDO\VLV WKH DSSOLFDWLRQ HPSRZHUV XVHUV WR PDNH
LQIRUPHG GHFLVLRQV DQG LQWHUYHQWLRQV WR LPSURYH WUDIILF
HIILFLHQF\UHGXFHFRQJHVWLRQDQGPLQLPL]HHPLVVLRQV
)XWXUH LPSURYHPHQWV IRU WKH DSSOLFDWLRQ LQFOXGH
HQKDQFLQJ LWV FDSDELOLWLHV WR KDQGOH PRUH H[WHQVLYH
VFHQDULRV &XUUHQWO\ WKH DSSOLFDWLRQ RQO\ YLVXDOL]HV RQH
W\SH RI YHKLFOH ZLWKRXW GLVWLQJXLVKLQJ EHWZHHQ GLIIHUHQW
FDWHJRULHV,QWURGXFLQJPRGHOVIRUPXOWLSOHYHKLFOHW\SHV
ZRXOG LPSURYH WKH UHDOLVP RI WKH VLPXODWLRQ
)XUWKHUPRUHWKHDSSOLFDWLRQGRHVQRWVLPXODWHRWKHUURDG
XVHUV VXFK DV SHGHVWULDQV EXVHV RU WUDLQV ZKLFK FRXOG
EH LQFRUSRUDWHG WR SURYLGH D PRUH FRPSUHKHQVLYH
UHSUHVHQWDWLRQ RI WUDIILF G\QDPLFV $QRWKHU DUHD RI
LPSURYHPHQW LV WKH QHWZRUN HGLWRU :KLOH PLQRU
)LJXUH *UDSKRI$YHUDJH9HKLFOH6SHHGLQ-XQFWLRQV PRGLILFDWLRQVWRWUDIILFDQGQHWZRUNVHWWLQJVDUHSRVVLEOH
XVHUV FDQQRW GLUHFWO\ DGG QHZ URDG VHJPHQWV WUDIILF

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 281

OLJKWV RU RWKHU QHWZRUN FRPSRQHQWV ZLWKLQ WKH >@ 1*KDULDQL6(ONRVDQWLQL6'DUPRXODQG/%6DLG³$VXUYH\
RI VLPXODWLRQ SODWIRUPV IRU WKH DVVHVVPHQW RI SXEOLF WUDQVSRUW
DSSOLFDWLRQ ,QWURGXFLQJ D QHWZRUN HGLWRU IHDWXUH ZRXOG FRQWURO V\VWHPV´ LQ ,QWHUQDWLRQDO &RQIHUHQFH RQ $GYDQFHG
RIIHUXVHUVJUHDWHUIOH[LELOLW\DQGFRQWUROLQPRGLI\LQJWKH /RJLVWLFVDQG7UDQVSRUW ,&$/7  SS 0D\
VLPXODWLRQ HQYLURQPHQW 7KHVH HQKDQFHPHQWV ZRXOG >@ 3 $ /RSH] ( :LHVVQHU 0 %HKULVFK / %LHNHU:DO] -
FRQWULEXWH WR D PRUH SRZHUIXO DQG HIIHFWLYH DSSOLFDWLRQ (UGPDQQ<3)ORWWHURG5+LOEULFK//XFNHQ-5XPPHODQG
IRUVWXG\LQJDQGRSWLPL]LQJWUDIILFVFHQDULRV 3 :DJQHU 0LFURVFRSLF 7UDIILF 6LPXODWLRQ XVLQJ 6802 LQ
VW ,QWHUQDWLRQDO &RQIHUHQFH RQ ,WHOOLJHQW 7UDQVSRUWDWLRQ
6\VWHPV ,76& 0DXL+,SS
$&.12:/('*0(17
>@ 0 =DN\ ' * $LUXOOD ( -RHOLDQWR DQG + 6XWDUWR ³8UEDQ
7KLVSXEOLFDWLRQKDVEHHQZULWWHQWKDQNVWRWKHVXSSRUW WUDIILF VLPXODWLRQ XVLQJ 6802 RSHQ VRXUFH WRROV´
RI WKH 2SHUDWLRQDO 3URJUDPPH ,QWHJUDWHG ,QIUDVWUXFWXUH ,QWHUQHWZRUNLQJ,QGRQHVLDMRXUQDOYROSS
IRU WKH SURMHFW $GYDQFLQJ 8QLYHUVLW\ &DSDFLW\ DQG >@ ' &HWLQND\D ³$ PRGHO GULYHQ DSSURDFK WR :HEEDVHG WUDIILF
&RPSHWHQFH LQ 5HVHDUFK 'HYHORSPHQW DQG ,QQRYDWLRQ VLPXODWLRQ´ LQ 6\PSRVLXP RQ 7KHRU\ RI 0RGHOLQJ DQG
$&&25'  ,706FRGH; FRIXQGHGE\WKH 6LPXODWLRQ '(96 ,QWHJUDWLYH 0 6 6\PSRVLXP 706'(96  

(XURSHDQ 5HJLRQDO 'HYHORSPHQW )XQG (5')  7KLV
>@ - &DVDV - / )HUUHU ' *DUFLD - 3HUDUQDX DQG $ 7RUGD\
SDSHU KDV EHHQ DOVR ZULWWHQ WKDQNV WR 6ORYDN 1DWLRQDO ³7UDIILF 6LPXODWLRQ ZLWK $LPVXQ´ ,Q - %DUFHOy HGV 
UHVHDUFK SURMHFWV $399 DQG .(*$ 678 )XQGDPHQWDOV RI 7UDIILF 6LPXODWLRQ ,QWHUQDWLRQDO 6HULHV LQ
 2SHUDWLRQV5HVHDUFK 0DQDJHPHQW6FLHQFHYRO6SULQJHU
1HZ<RUN1<
5()(5(1&(6 >@ 0 )HOOHQGRUI DQG 3 9RUWLVFK ³0LFURVFRSLF 7UDIILF )ORZ
6LPXODWRU9,66,0´,Q-%DUFHOy HGV )XQGDPHQWDOVRI7UDIILF
>@ 5 GH OD 7RUUH & * &RUOX % 6 2QJJR DQG - )DXOLQ 6LPXODWLRQ ,QWHUQDWLRQDO 6HULHV LQ 2SHUDWLRQV 5HVHDUFK 
6LPXODWLRQ 2SWLPL]DWLRQ DQG 0DFKLQH /HDUQLQJ LQ 6XVWDLQDEOH 0DQDJHPHQW6FLHQFHYRO6SULQJHU1HZ<RUN1<
7UDQVSRUWDWLRQ6\VWHPV0RGHOVDQG$SSOLFDWLRQV6XVWDLQDELOLW\
YRO >@ 6802 'RFXPHQWDWLRQ SDJH 6802 5RDG 1HWZRUNV 
KWWSVVXPRGOUGHGRFV1HWZRUNV6802B5RDGB1HWZRUNVKWPO
>@ * .RWXVHYVNL DQG . +DZLFN $ 5HYLHZ RI 7UDIILF 6LPXODWLRQ
6RIWZDUH5HVHDUFK/HWWHUVLQWKH,QIRUPDWLRQDQG0DWKHPDWLFDO
6FLHQFHVYROSS 


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 282

 Automated Monitoring of Network
Infrastructures Based on the Zabbix Solution
E. A. Katonová*, J. Džubák*, P. Feciľak*
*Department of Computers and Informatics, Košice, Slovakia
erika.abigail.katonova@cnl.sk, jozef.dzubak@cnl.sk, peter.fecilak@cnl.sk

Abstract— This article describes the creation of a

monitoring platform intended for automated measurement A. A survey of research in the field of monitoring
of network infrastructure parameters, configuration change network environments
and response to non-standard situations. The output of this From the point of view of monitoring aimed at
work is a fully functional platform based on the open-source detecting attacks and analyzing packets, the research
systems Zabbix and Grafana. Part of the practical outputs is publication [1] recommends ZEEK/Bro tools.
documentation for working with the created platform, the For packet monitoring with high efficiency, the
aim of which is to enrich the educational process with the
HyberSight monitoring system can be used, which allows
issue of monitoring network infrastructures. At the same
packets to be prioritized according to the degree of
time, advanced monitoring methods can be used to detect
importance. Article [2] is devoted to the mentioned
non-standard behavior and various types of attacks. The
Grafana visualization system is chosen for the visualization
research.
of data obtained from monitored network devices. Article [3] points to the fact that the main monitoring
parameters include the state of the processor, RAM
Keywords—monitoring, Zabbix, Grafana memory and disk. It is convenient to monitor these
parameters using the SNMP protocol.
The publication [4] summarizes the results of
I. INTRODUCTION experiments in which Cacti, Nagios, Zabbix, and PRTG
Current, dynamically changing network infrastructures tools were used for monitoring. Experiments have
need an advanced, automated, and complex monitoring concluded that from the point of view of memory
system to maintain the stability of their service provision, consumption it is most optimal to monitor Syslog
which can not only monitor the managed network messages.
infrastructure but can also make configuration changes to Modern approaches to network device configuration
it based on predefined rules. Such a system can prevent management include the use of the RESTCONF and
outages in the network or detect and stop an ongoing NETCONF protocols. The publication [5] deals with the
attack. creation of a control unit for managing the configuration
The disadvantage of the complex network of routers of the CSR1kv series.
infrastructures mentioned is their diversity, both from the In case of the need to monitor cloud environments,
point of view of network device manufacturers and the according to article [6], it is advisable to use the LNMP
communication protocols used. Such a heterogeneous protocol (Lightweight Network Management Protocol).
network environment cannot be monitored automatically Software-defined networks can be monitored using the
with standard, simple tools. monitoring tool FlowSpy, which is described in more
Another problem is the large number of network detail in the publication [7]. Using this tool, it is possible
devices, which, from a time regarded point of view, must to detect a SYN flood attack.
be monitored systematically without the intervention of Monitoring, from the point of view of service quality,
the administrator. Manual monitoring would be out of date can be done using the Zabbix tool. Research [8] used this
and prone to errors. tool to monitor the quality of calls made via VoIP
The solution is the creation of a platform that will technology.
enable the use of a suitable combination of tools to The publication [9] was devoted to a comparison of the
achieve the possibility of managing a complex, freely available tools Zabbix, Cacti, and Nagios.
heterogeneous network infrastructure. This goal can be
achieved by choosing tools that provide a wide range of Article [10] points out the importance of analysis and
configuration settings, include an application interface for evaluation of collected data. The article states that data
interacting with external systems, and are built for collection alone is not a sufficient monitoring approach.
working with a large amount of data. From a commercial Notification services provided by, for example, the Zabbix
point of view, it is advisable to use open-source tools. tool are a suitable monitoring supplement.
The resource [11] deals with the comparison of Zabbix,
II. CURRENT STATUS IN THE FIELD OF NETWORK Pandora FMS, Zenoss, Cacti, and Nagios tools from the
INFRASTRUCTURE MONITORING point of view of real-time data transmission enabling a
quick response to problems.
The following sub-chapters deal with the current status
and experience of monitoring.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 283

 The authors of the article [12] worked with the NMS do not provide the possibility of automation and
(Network Monitoring System), based on the SNMP integration with other systems.
protocol and Zabbix and MibParser tools.
III. MONITORING PLATFORM DESIGN
B. Comparative analysis of modern and traditional
Based on the analysis, it is possible to claim that Zabbix
monitoring tools
and Grafana are suitable tools for creating a monitoring
The following list shows popular end-to-end monitoring mechanism for data collection and configuration
systems: modification. The Grafana tool will serve to visualize the
x Security systems: Zeek/Bro, Snort, Nessus – monitored characteristics provided by the Zabbix system.
analysis of packet/IDS system
A. Installation diagram
x IBM Tivoli – monitor performance and
availability of operating systems and Figure 1 shows the installation dependencies for the
applications monitoring platform.
x Nagios – monitoring systems and networks
x Cacti – web-based network monitoring
x Zenoss – full-stack monitoring
x Pandora FMS – monitoring computer networks
x Zabbix – network and software monitoring
The following Table 1 summarizes characteristic of
popular monitoring tools.

TABLE 1. ANALYSIS OF MODER MONITORING TOOLS

Name Zabbix Cacti Nagios

Automation Yes Yes Yes
Integration Yes Yes Yes
Complexity High High High
SMS SMS
Notification Email
Email Email
Use case Difficult Medium Difficult
Installation Difficult Difficult Difficult

Figure 1. Installation architecture of monitoring platform

The following list shows traditional monitoring
protocols: The Grafana and Zabbix tools will be used to obtain
x SNMP – statistics data from managed devices and modify their
configuration. The obtained data will be provided to the
x Syslog – log data end user in the form of visualization panels created in the
x SSH – configuration Grafana tool. For easy portability, these tools will be
x HTTP – configuration installed on a Linux operating system (Ubuntu 22.04) with
all the necessary settings and will be distributed as a
x ICMP – connectivity
virtual machine that can be launched using the VirtualBox
The integration of standard protocols using different platform running on the end user's operating system. A
programming languages is described in the publication virtual device consisting of a Linux operating system with
[13]. installed Grafana and Zabbix tools together form a
Table 2 summarizes characteristics of popular monitoring platform enabling automated monitoring and
traditional monitoring tools. management of complex heterogeneous network
environments.
TABLE 2. ANALYSIS OF TRADITIONAL MONITORING TOOLS
Name SNMPB PuTTY Postman
Automation No No No B. Functional connection between tools
Integration No No No The functional connection between the used tools and
the monitored network device is shown in Figure 2. The
Complexity Medium Low Medium task of the Zabbix tool is to communicate with the
Notification Trap No No managed device and to obtain statistical data and change
Use case Easy Easy Easy the configuration using the SNMP and HTTP protocols.
Installation Easy Easy Easy The Zabbix system will then store the collected data in its
database and provide it for visualization by the Grafana
tool. The Zabbix system also provides an application
As can be seen, traditional approaches for monitoring interface through which it can make the collected data
network environments are easy to use and install, but they
are not suitable for managing complex infrastructures, and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 284

available to external applications and thus create a Zabbix. The following list lists the necessary settings
complex automatable monitoring platform. with instructions for setting them up correctly:
1. Defining the target device: It requires setting
the IP address of the target device and choosing
a communication protocol. Standard
communication protocols include SNMP and
HTTP.
2. Creating an object: The object represents the
basic element ilustrating the interaction with the
target monitored device. Its settings depend on
the communication protocol.
In the case of the RESTCONF protocol, it is
necessary to set the object type to HTTP agent,
to specify the target monitored resource with the
URL address. It is possible to determine whether
the data will be read (GET) or set (PUT) using
the specific method. The interval between two
consecutive monitoring processes is set with a
timer. For headers is possible to specify the type
of data to be transferred, which is yang-
data+json for the RESTCONF protocol. The
authorization gives access to the target managed
device, which usually consists of entering one’s
name and password.
If we communicate with the target device using
Figure 2. Interaction between system components
the SNMP protocol, then it is necessary to select
C. Services provided by the platform the type of object on the SNMP agent, define the
The role of the monitoring platform is to provide the address of the target device with the interface,
following services: define the required resource with the OID
x Support for heterogeneous infrastructure identifier and authorization using the community
monitoring: This service will be provided using string.
standardized approaches for data acquisition and 3. Creating a trigger mechanism: The triggering
configuration change. Primarily used protocols will mechanism will enable an automated response to
be SNMP and RESTCONF. the events that have occurred. Setting up the
x Monitoring of quality parameters: Secured by trigger mechanism consists of defining the
obtaining statistical parameters about the amount severity of the event and the condition that will
of data transmitted by the interface and the number determine whether the desired action should be
of dropped packets using the SNMP protocol. triggered. By default, the condition includes a
x Obtaining and changing configuration comparison of the current value of the measured
information: Secured by using the RESTCONF statistic with the threshold value.
protocol, which allows us to read and change the 4. Creating an action and a script with a link to
configuration using YANG models and model- the trigger mechanism: The last step is to
driven programming in an automated way. create an action that will start after the activation
x Automating the reaction of the monitoring of the trigger mechanism. By default, the action
system to specific events: Ensured by the use of a includes a script to execute the remedial
trigger mechanism that can be connected to the instructions.
comparison of the limit values of the monitored
statistics. If the limit value is exceeded, the trigger B. Setting up the Grafana tool
mechanism activates the function to change the The grafana tool allows us to create a dashboard
configuration according to predefined rules. containing a set of visualization panels that display data
obtained from monitored devices. Within the panel, it is
IV. CREATION OF MONITORING PLATFORM necessary to define the type of read data, the target
The creation of the platform required the installation of managed device and the object representing the monitored
Ubuntu 20.04 operating system and Zabbix v6.4 and characteristic. In addition, it is advisable to set the interval
Grafana v9.5.3 tools. The following subsections document for reading data from the Zabbix database. It is
the necessary setup of Zabbix and Grafana. recommended to set this interval to approximately the
same value in which Zabbix retrieves data. The resulting
A. Setting up the Zabbix tool monitoring panel, shown in Figure 3, is depicting
information about the IP address of the monitored device,
To enable automated monitoring and response to specific device name, CPU and memory usage, the amount of
events, several important settings need to be made in received and sent data, response time, and a table of
problems.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 285

 Zabbix tool using its web application, and the Grafana
tool.
The first test was to verify the possibility of using the
standardized communication protocol SNMP and HTTP
to collect and change configuration data. In the Zabbix
system, these protocols were used to communicate with
the managed target device. The result of the test was the
discovery that a connection with the monitored device was
successfully established using these protocols.
The second test was to verify whether the monitoring
platform can obtain statistical information about quality
characteristics in the network environment. For this
purpose was used the SNMP protocol which accessed the
OID representing the amount of data received and sent by
Figure 3. Grafana monitoring dashboard the network device interface. At a given time, the user
manually checked the statistics on the managed device
Part of the options provided by the Grafana tool, which with the "show" command and compared the value with
were used in this work, are setting the types and styles of the data obtained by the Zabbix tool and visualized by the
displayed graphs, setting threshold values, a mapping Grafana tool. Testing showed slight inaccuracies. The
process that, based on defined intervals, allows replacing a reason was a short delay in the displayed data, which
numeric value with a text one, and finally regular depends on the monitoring interval set by Zabbix and
expressions searching for specific expressions in the Grafana. However, the displayed data, even with a slight
obtained data. The data source in the Grafana tool does delay, provided accurate statistical values and thus
not have to be only the Zabbix database, but also a confirmed the correctness of the functionality of the
number of other databases and sources providing data. monitoring platform.
The third test was to verify the support for obtaining
V. EVALUATION OF THE CREATED MONITORING and changing configuration settings by the RESTCONF
PLATFORM protocol. The test consisted of simply reading the
The practical outputs created during this work consist configuration, e.g., by finding out the name of the device
of the created monitoring platform and educational and changing the configuration, e.g., changing the IP
materials documenting the work and control of the address of the selected interface. This test confirmed the
correctness of this tested functionality.
created platform. Considering this, the evaluation consists
of testing the functionality of the monitoring platform and The last test was to verify the system's automated
response to a certain, predefined event. As part of the
evaluating the benefits of the educational materials.
trigger mechanism, a rule was defined that checked the
A. Testing the functionality of the monitoring platform value of the IP address on the selected interface. If the IP
The platform aimed to provide services for automated address on this interface changed, Zabbix and Grafana
monitoring of network infrastructures consisting of system displayed an error log about this event. In addition,
several network devices. Functionality testing consisted of the Zabbix system launched a corrective action that
manual interaction of the end administrator with the automatically reconfigured the IP address of the controlled
monitored system and comparison of data obtained from interface to the original predefined value using the
Zabbix and Grafana tools. The following Figure 4 shows RESTCONF protocol. This test was also successful.
the communication diagram of the connection of B. Evaluation of created educatuinal materials
individual testing components.
101 participants took part in the testing, while the
percentage distribution of participants by position is
shown in the Figure 5.

Figure 4. Communication during testing scenarios

The main component of testing is the User object, Figure 5. Distribution of participants according to their position
which represents the administrator who communicates
with the monitored device using the SSH protocol, the As can be seen, most of the participants were high
school students whose knowledge, in the field of

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 286

computer networks, is minimal or intermediate. The x 36.6% - A separate subject for one whole year
testing was also attended by university students, x 29.7% - A short seminar to broaden knowledge
secondary school teachers, and instructors of the network x 17.8% - Lessons after passing the CCNA course
academic program NetAcad. A diverse sample of test x 15.8% - Continuously throughout the study
participants was chosen to obtain relevant feedback on As can be seen, there is interest in integrating
the created educational materials, which will be used by educational materials into the teaching process, and the
students, teachers, and lecturers. issue of monitoring and automation in the field of
The created educational materials were made in the network infrastructures has attracted the attention of both
form of PDF documents and presentations. They were students and teachers.
presented to a sample of participants in the form of a
lecture, seminar, and laboratory exercises. Based on the VI. CONCLUSION
materials provided, the test participants subsequently This work was devoted to the creation of a
worked with the created platforms and became familiar monitoring platform enabling the monitoring of
with testing the network environment using Zabbix and complex network infrastructures and an automated
Grafana tools. At the end of the testing, participants were response to emerging, predefined non-standard
provided with a feedback form. The primary goal of the behavior. Based on the analysis of the current state and
testing was to find out how the participants perceive the existing tools, the tools Zabbix and Grafana were
created materials from the point of view of usefulness, chosen for the creation of the platform. Both tools are
actuality, and difficulty of understanding. The following freely available, with Zabbix providing a wide range of
Figure 6 shows the obtained statistical, average monitoring options and notification services, and
evaluation of the mentioned three aspects of the Grafana enabling visualization of monitored data in a
materials. user-friendly form for end users. The platform is fully
functional and ready to use. The practical outputs also
include documentation that explains the operation with
the platform, and its primary goal is to enrich the
educational process with the area of monitoring
network environments. The educational materials were
presented to a sample of 101 participants with above-
average positive feedback and interest in integrating
the educational materials into the teaching process.
ACKNOWLEDGMENT
This work was supported by Cultural and Educational
Grant Agency (KEGA) of the Ministry of Education,
Science, Research and Sport of the Slovak Republic under
the project No. 060TUKE-4/2022.

Figure 6. Evaluation of educational materials REFERENCES

As can be seen, the result of the testing was the gain of [1] NGUYEN, Giang; DLUGOLINSKY, Stefan; TRAN, Viet;
GARCIA, Alvaro Lopez. Deep Learning for Proactive Network
above-average positive feedback. A slight decrease can Monitoring and Security Protection. IEEE Access. 2020, roč. 8, s.
be seen in difficulty of understanding. This fact is 19696–19716. Doi: 10.1109/ access.2020.2968718.
because most of the test participants were high school [2] ZHOU, Yu; BI, Jun; YANG, Tong; GAO, Kai; CAO, Jiamin;
students who do not yet have experience and knowledge ZHANG, Dai; WANG, Yangyang; ZHANG, Cheng. HyperSight:
Towards Scalable, High-Coverage, and DynamicNetwork
in the field of computer networks, which caused that Monitoring Queries. IEEE Journal on Selected Areas in
there was a lot of new knowledge in the created materials, Communications. 2020, roč. 38, č. 6, s. 1147–1160. Doi:
which represented an increased difficulty for students to 10.1109/jsac.2020.2986690.
understand. Despite that fact, this aspect received an [3] Monitoring Systems and Networks. Wiley, 2020. Doi:
above-average rating, and it can be said that it is also well 10.1002/9781119779964.ch9.
[4] CASTILLO-VELAZQUEZ, Jose-Ignacio; RAMIREZ-DIAZ, Efll-
processed in terms of difficulty of understanding. At the Yovanka; NINO, William Rogelio Marchand. Use of GNS3 Cloud
same time, this test pointed out that the materials should Environment for Network Management Emulation when
not be included in the teaching immediately at the Comparing SNMP vs Syslog Applied Over an Advanced Network.
beginning of the study, but rather later, when the students In: 2019 IEEE 39th Central America and Panama Convention
(CONCAPAN XXXIX). IEEE, 2019. Doi:
already have basic knowledge in the field of computer 10.1109/concapanxxxix47272.2019.8976995.
networks and cyber security and therefore understand the [5] KATONOVA, E.A.; PETIJA, R.; JAKAB, F.; FECILAK, P.;
importance of monitoring and automated reactions to MICHALKO, M. Integration of the programmability and
events arising in the network environment. automation concepts into the teaching of computer networks. In:
2021 19th International Conference on Emerging eLearning
The last aspect of the evaluation of educational Technologies and Applications (ICETA). IEEE, 2021. Doi:
materials was the determination of interest regarding the 10.1109/iceta54173.2021.9726621.
recommended scope of teaching. Based on the feedback, [6] MULLINS, Taariq; BAGULA, Antoine. Monitoring Community
the following preferences were arrived at: Clouds: The LightweightNetwork Management Protocol. In: 2013

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 287

 IEEE 10th International Conference on Ubiquitous Intelligence Telegram. In: 2020 3rd International Conference on Computer and
and Computing and 2013 IEEE 10th International Conference Informatics Engineering (IC2IE). IEEE, 2020. Doi:
onAutonomic and Trusted Computing. IEEE, 2013. Doi: 10.1109/ic2ie50715.2020.927458.
10.1109/uic-atc.2013.31. [11] KAMENOV, Svetoslav Sergeev. Experimental Monitoring on
[7] GUAN, Bowei; SHEN, Shan-Hsiang. FlowSpy: An Efficient Network Based Tactile Sensing System. In: 2019 IEEE XXVIII
Network Monitoring Framework Using P4 in Software-Defined International Scientific Conference Electronics (ET). IEEE, 2019.
Networks. In: 2019 IEEE 90th Vehicular Technology Conference Doi: 10.1109/et.2019.8878661.
(VTC2019-Fall). IEEE, 2019. Doi: 10.1109/vtcfall.2019.8891487. [12] IQBAL, Asif; PATTINSON, Colin;KOR, Ah-Lian. Performance
[8] REZAC, Filip; ROZHON, Jan; VOZNAK, Miroslav; SLACHTA, monitoring of Virtual Machines (VMs) of type I and II
Jiri; SAFARIK, Jakub. Multi-agent system for monitoring the hypervisors with SNMPv3. In: 2015 World Congress on
quality of speech in computer networks. In: 2015 38th Sustainable Technologies (WCST). IEEE, 2015. Doi:
International Conference on Telecommunications and Signal 10.1109/wcst.2015.7415127.
Processing (TSP). IEEE, 2015. Doi: 10.1109 / tsp. 2015.7296428. [13] KATONOVA, E. A.; PETIJA, R.; JAKAB, F.; KAINZ, O.;
[9] MARIK, Ondrej; ZITTA, Stanislav. Comparative analysis of MICHALKO, M.; DZUBAK, J. An innovative multidisciplinary
monitoring system for data networks. In: 2014 International approach to the teaching computer networks and cybersecurity. In:
Conference on Multimedia Computing and Systems (ICMCS). 2022 20th International Conference on Emerging eLearning
IEEE, 2014. Doi: 10.1109/icmcs.2014.6911307. Technologies and Applications (ICETA). IEEE, 2022. Doi:
[10] ARDIYONO, Anggi; SHOLIHAH, Walidatush; HAKIM, Faisal. 10.1109/iceta57911.2022.9974880.
Mobilebased Network Monitoring System Using Zabbix and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 288

 Implementation of IDS Functionality into IoT
Environment using Raspberry PI
E. A. Katonová*, P. Nehila*, P. Feciľak*, O. Kainz*, M. Michalko*, F. Jakab* and R. Petija**
*Department of Computers and Informatics, Košice, Slovakia
**CEELABS, s.r.o., Košice, Slovakia
erika.abigail.katonova@cnl.sk, peter.nehila@student.tuke.sk, peter.fecilak@cnl.sk, ondrej.kainz@cnl.sk,
miroslav.michalko@cnl.sk, frantisek.jakab@cnl.sk, rastislav.petija@cnl.sk

Abstract—Internet of Things is constantly growing, and

and benign communication. We have also incorporated use
a l o n g with it comes the risk caused by unsecured devices that it of configuration file into our IDS system. This allows us to
consists of. First option to increase security, is to use the change behavior of certain parts of our system. Use of
intrusion detection system (IDS). Based on the analysis of configuration allows us to make the whole system more
communication in IoT (Internet of Things) networks, on flexible. Since using machine learning requires the usage of a
intrusion detection methods and on protocols for collecting
information about communication, we designed and implemented
dataset, configurable system gives us an ability to change the
a system for flow-based intrusion detection. To distinguish as dataset and adapt the behavior of IDS to our needs.
many attacks as possible and for universality reasons, we used
machine learning for the prediction of attacks. We optimized the II. TECHNOLOGIES INVOLVED
system for Raspberry Pi. Then we performed tests to determine
the accuracy of the prediction. The results of mentioned tests
A. Different approaches for intrusion detection
were finally evaluated. Various attacks exploit the weaknesses of the network itself,
Keywords—Intrusion detection, Internet of Things, Rasp- not just the devices in it [7]. There are several protocols
berry Pi, Network flows, IPFIX, Internet security, Machine
that provide different levels of security. Therefore, it is
learning
necessary to correctly choose the communication protocol to
I. INTRODUCTION increase security.
Intrusion detection systems are tools for detecting actions
Developments in electronic services have led to the that have the intention of gaining unauthorized access to the
emergence of a concept called the Internet of Things. system. When system detects an attack, information is either
Under the concept of the Internet of Things, we picture an logged or reported.
environment which includes devices that communicate among There are two types of IDS:
each other, create connections together and exchange various
• host-based IDS,
data that they have acquired through some form of perception
• network-based IDS.
of this environment.
In recent years, the concept of IoT has also been associated Host-based IDS work on the principle of collecting
with the creation of so-called smart environments, such as information on a specific device. They are also able to obtain
smart cities or households. According to [1], in 2022, a trillion information such as system calls, a list of running processes,
devices belonging to the IoT environment were part of the application logs and other. In the case of network-based IDS,
Internet, indicating a huge expansion of the industry. We the approach is different. They control the traffic of the entire
encounter IoT environments on a daily basis. These networks network, so they are often connected to one network segment
are implemented in different ways, most often they include [2]. Network-based IDS visualization can be seen in figure 1.
various wireless technologies. These technologies are often IDS can be further divided, based on how they detect
exposed to many attacks , which try to exploit their attacks, into misuse detection and anomaly detection. Anomaly
weaknesses. Various attacks and system vulnerabilities pose a detection shows good results in detecting new attacks, because
threat that could cause not only data theft but also a life threat. it uses a model of normal communication. However, the dis-
For these reasons, safety in the IoT environment has advantage is high false positive rate [3]. Misuse detection uses
become one of the most important topics in recent years. attack patterns and according to them can determine exactly
Therefore, we have designed and implemented network whether it is an intrusion of the system. The disadvantage of
intrusion detection system aimed at operation in IoT this approach is its inability to detect new attacks or attacks
environments. System is intended to be used on Raspberry PI. that have been slightly modified [3].
To detect network intrusions, we used flow records. The For the usage in IoT we implemented network-based IDS.
implementation focuses mainly on the IPFIX protocol. To Using host-based IDS would not be practical in IoT
recognize individual attacks, we proposed a machine learning environment, because in the case of such system, each device
model. Our system contains 2 separate machine learning would require a sensor to collect information [2].
models - for differentiating intrusions from each other and
for distinguishing between malign

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 289

 D. Machine learning
Since it is possible to find repetitive patterns of network traffic
in data streams, we can use machine learning to detect various
intrusions. Machine learning methods are characterized by
good accuracy when the training has been properly tuned and
by good performance in the case of large amounts of data.
However, their disadvantage may be the ambiguity of the
decision and the longer learning time. Especially when used
on devices with low hardware specifications, using machine
learning can sometimes be difficult. We must find a good
tradeoff between speed, accuracy and resources required to
run our system. Usually, after the learning process is finished,
machine learning models require less resources for predicting
Fig. 1. Visualization of NIDS values. Important aspect when deciding which attributes will
be used is generalization. Using too much data might create
too narrow space into which the anomalies would fit. When
the attack occurs with slightly changed signature, it could be
B. Network flows hard to fit it into correct anomaly. Worst case scenario it could
be marked as benign communication.
calculated by other software.
The data flow in the network can be defined as a set of
packets and frames passing through a point where they are E. Hardware constraints
observed. Packets belonging to a stream have different Working in IoT means that all devices are expected to
properties (for example port number, source, and consume minimum amount of energy. In this project, we have
destination IP address) [4]. According to the measurements in implemented our IDS on Raspberry Pi. Raspberry Pi is a
[5], the calculations indicated that the size of the data obtained microcomputer widely used in IoT for many different
by exporting the flow records compared to the total size of the purposes thanks to its accessibility and modularity. Although
transmitted packets was equal to 0.1%. Given that we are in the latest models have more and more power, the fact that we
IoT environment, where network might not have high speed are limited by the hardware still stands. When working with
and amount of data sent over network is excessive, using machine learning datasets, we might run out of memory to
flows to transmit information about communication is store our data, for that reason we needed to implement
convenient. mechanisms which allowed us to work with datasets larger in
Different patterns, such as suspicious and irregular changes memory size.
in network traffic can be detected from data flows. When using
these network flows, it is important to choose metrics carefully III. ARCHITECTURE OF OUR IDS
to achieve high accuracy with respect to resource utilization The IDS that we have designed focuses primarily on
[6]. processing data using machine learning and collecting the
results. The architecture can be seen on fig. 2. Our system is
C. IPFIX separated from the collector, which means the process of
collecting flow data needs to be handled externally. Designing
IPFIX protocol was created to unify the process of working our system this way allows us to give it greater options in
with data flows in networks. IPFIX defines how are data terms of modularity. We are not necessarily restricted to
carried from exporting process all the way to collecting processing only IPFIX data but could receive any kind of data
process. IPFIX also describes Information elements, their from different sources and process them afterwards.
name, type and additional helpful information. Under the Our IDS system consists of multiple smaller parts, where
IPFIX protocol, devices are divided into roles of exporters and each focuses on different tasks. Because we wanted our system
collectors. The job of the exporter is to send IPFIX messages to run on Raspberry Pi, we had to deal with various
to one or more collectors. A collector is a device that receives constraints. One of the obstacles we needed to solve was the
these messages and either processes them further or stores size of datasets. Because of low operating memory of
them. The collector can receive messages from several Raspberry Pi, it would be impossible to load bigger datasets
exporters at once [4]. IPFIX offers us many elements which and work with them. For that matter we have implemented
could be used for analysis of attacks [8]. Since we are lazy loading of datasets that allows us to load and manipulate
working in IoT environment, we need to find a compromise datasets which do not fit into RAM. Another supportive
between the accuracy and the amount of data transferred. functionality we added to our system is sampling. We used
Another issue could be privacy. Collecting too much data methods of undersampling and oversampling to not only
might go against principles of privacy. Another advantage of balance out the classes in dataset, but also to create smaller,
IPFIX is that it also supports the implementation of custom more compact dataset that will be used to train our model
defined elements, which can be then used to transfer extra afterwards. This makes the whole process
information. Using this we could transfer data that is not
collected directly by IPFIX, instead it is generated or

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 290

 Fig. 2. Visualization of architecture

of training much faster because we are using smaller sample A. Setup used for tests
of original dataset. We have run all our tests on Raspberry Pi 4 model B. We
have used ipfixcol21 in combination with flow records
A. Detecting intrusions
generator, both have also been running on the same Raspberry
As mentioned above, we use machine learning model built Pi. We used this setup to not only measure our software
from network flows to detect anomalies. We have designed a but also to see how the device would operate in a scenario
solution with 2 separate models, which use slightly different where multiple services are running on a single Raspberry
target classes to evaluate flows. The reason behind this is that, Pi. For measuring accuracy, we have split dataset. As criteria
in machine learning, we cannot be 100% sure of the result we have used precision, accuracy, recall and F1-score.
that we acquire. Using separate models can help us to Another tool that we have extensively used are confusion
decide on the results. The models differ in target classes that matrices. It was important to choose correct metrics. In our
they evaluate. One model uses separate names of distinct case we are dealing with multi-class classification and using
attacks and anomalies, while the other uses only information just one metric, such as accuracy, might prove insufficient.
whether the flow is malign or not. Second model is a simple
binary classification. Usually, these classifications are much B. Datasets
simpler and can achieve much higher precision. We used Before working on anything we first needed a suitable
Random Forest as classifier for both models in our IDS. First, dataset that we could use for our IDS. While searching for
we created models based on given dataset. Afterwards we datasets we put our focus primarily on datasets containing
started checking whether there were any new flows waiting to network flow records with properly marked target classes.
be evaluated. If there were none, we used previously trained Another important aspect that we had to take into
models to get the results for two separate target attributes. consideration are hardware restrictions. Choosing dataset that
contains too many attributes or too many records would only
B. Configurability slow the whole process down. Datasets that we found are:
In the field of machine learning it is important to adapt the • CIC-IDS2017 [9] contains network flows recorded in a
behavior of application and to correctly process the data. span of 5 days with correctly marked respective attacks,
Because the system is using machine learning, we have added • Dataset-Unicauca [10] - created recording real network
ability to extensively configure and change certain parts of communication, unfortunately contains only benign
our application. The motivation for this was primarily the communication,
fact, that the use of different datasets opens options for future • CIDDS-001 [11] is dataset created in emulated
use and development of our IDS. Using configuration file environment, contains multiple attacks,
allows us to change and specify not only basic settings, but • NF-UNSW-NB15, NF-ToN-IoT, NF-BoT-IoT, NF-
also which attributes will be treated in different ways. Possible CSE-CIC-IDS2018, NF-UQ-NIDS [12] - group of
use could be operating our IDS on a more robust device, datasets changed in a way to contain network flows, all
which would allow the use of much bigger datasets with more of them contain multiple attacks as well as benign
attributes, resulting in more precise intrusion detection. This communication. NF-ToN-IoT and NF-BoT-IoT are
could also mean that our IDS does not necessarily need to datasets containing IoT data, which means we would
store the files on its drive, lifting some load o ff from be able to test our IDS directly on IoT communication.
Raspberry Pi. We have repeatedly performed tests on multiple of datasets
mentioned above, but primarily we’ve focused on the last
IV. EVALUATION group of datasets. The presented results refer to the NF-BoT-
Before we could decide on which classifier to use, we had IoT dataset.
to test different approaches to identify, which one was the C. Testing different approaches of preprocessing
most suitable. The tests that we have executed could be
In the machine learning it is necessary to evaluate the model
divided into smaller tests focused on different functionality:
gradually, to try different combinations of preprocessing to
• testing different approaches of preprocessing,
find one, that provides satisfying results. In this phase
• testing various classifiers,
1https://github.com/CESNET/ipfixcol2
• testing system with different configuration and datasets,
• testing acquired data from collector.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 291

we have primarily used confusion matrix to evaluate our Soon after we have started our tests, we found out that
results. The great advantage of these matrices is that we can traditional way of preprocessing was not sufficient. One of the
visualize them in graphical form, and thus simplify working examples can be seen on fig. 4 where we have used one-hot-
with them. The evaluation itself was relatively time encoding on nominal attributes. As we can see the data are
consuming, since many combinations of preprocessing skewed and all results still fit mostly into one class. This
approaches and classifiers had to be tested. We tested it as part happened in other cases as well, not just in case of SVM. We
of analyzing the following classifiers: SVC, Random Forest, suspect that the main reason for this is the fact that one-hot-
KNN, multi-layer perceptron, XGBoost. As far as encoding creates sparse matrices, and the informative value of
preprocessing is concerned, we have also tried several attribute is lost. This could be especially seen when we
approaches here, since individual classifiers react differently increased sample size, which meant that cardinality had also
with various preprocessing approaches. In addition to the increased.
confusion matrices, we used F1-score, accuracy, and variance
parameters. Thanks to these criteria, we can effectively reject
a model that has bad results.
Different sampling methods were one of the things we have
focused on in this step. Forcing higher or smaller balance
between target classes, as well as sample size were our
primary focuses. We found a case where the macro-accuracy
was around 70%. Looking at confusion matrix in fig. 3 it
was immediately clear that something was incorrect. As can
be seen in the figure 3, all values were evaluated as only one
class. This condition occurred when an unbalanced sample
was created, the model could not be trained correctly, and all
values were assigned to one class, which, however, had very
high representation in the dataset. For this reason, the apparent
accuracy was high. After the tests we have come into
conclusion that using balanced target classes yielded better
results. For this matter we have incorporated methods of
oversampling and under sampling into our model. The more
balanced the classes were the better results we got. It is Fig. 4. SVC using one-hot-encoding
important to note that there was a certain threshold at which
the results stopped changing. Another interesting fact is that We have found out that treating nominal categorical values
different datasets had this threshold at different levels of based on their cardinality rather than the type of categorical
balance enforcement. attribute resulted in much better accuracy. In our case we have
implemented threshold, after reaching the threshold of
cardinality the attribute will be encoded using Binary
encoding. Otherwise, it is encoded using ordinal encoding.
Although it is not recommended to use ordinal encoding on
nominal attributes, the classifiers were able to learn the
models significantly better. The reason for the threshold is that
in cases of lower cardinality, the classifiers did not show any
variability in results when using ordinal encoding or when
using binary encoding right away. Ordinal encoding is much
simpler and faster compared to binary encoding and this
way we could save up some hardware resources. While
using standardization of numeric attributes, we could also
observe variation of results. Random Forest reported the same
results regardless of whether we chose scaling or not, while
others required the use of scaling.
D. Evaluating classifiers
This part was done in combination with previous one.
Fig. 3. Confusion matrix - predicting on unbalanced classes
Main reason is that different classifiers react differently in
some cases. When we tested different models, we found that
Another area that we have put our focus into is the Random Forest, KNN and XGBoost classifiers performed
preprocessing. In this case we have primarily focused on the best. Results of the tests can be seen in the table I.
different approaches to preprocessing categorical and We performed the search on two different target attributes -
numerical attributes. division into individual attacks and division only into benign

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 292

 TABLE I
ACCURACY MEASUREMENTS

Duration of Duration of Macro-average

Classifier
training (s) prediction (s) precision (%)
XGBoost avg time 18.36342 0.03346 76
max time 19.27251 0.03298 75
min time 16.53365 0.03345 76
Random Forest avg time 2.23945 0.13522 77
max time 2.73797 0.15892 76
min time 2.15049 0.13308 77
SVC avg time 6.36213 3.79829 73
max time 6.52164 3.78020 73
min time 6.03696 3.76251 72
KNN avg time 0.18293 1.69622 73
max time 0.19407 1.78861 74
min time 0.14310 1.83168 74
Multi-layer perceptron avg time 32.28297 0.01887 72
max time 42.75451 0.03082 73
min time 25.85436 0.02337 74
a
Duration is shown in seconds
and malign communication. When deciding which model to problems due to hardware limitations. The results of the
use, we also took into consideration the duration of the training evaluation of the model to distinguish between benign and
and the duration of predicting the values. Based on these malign communication were much better. Here, an accuracy of
criteria, we finally decided to use Random Forest. The fact that 95% was achieved. Also, in this case, several other classifiers
Random Forest is more resistant to overfitting also played a did very similarly. Confusion matrix of measurement can be
significant role in the decision. Confusion matrix after training seen on fig. 6.
the Random Forest classifier can be seen in the picture 5.

Fig. 5. Confusion matrix of Random Forest - multi-class classification Fig. 6. Confusion matrix of Random Forest - binary classification

During testing, the highest achieved macro-accuracy values When we repeated tests on different datasets, we have
were around 77%, variance around 76% and the F1-score 76%. received different results. In most cases the results were better
The figure depicts that in our specific case, the accuracy of the in terms of prediction accuracy in separate target classes,
Theft and DDoS attacks was lower compared to the other although sometimes there was even lower accuracy in some
classes. From the result we can conclude that the individual cases. Because of this reason there are possible improvements
flows specifying these attacks had either too similar features, to both models and the datasets themselves. Perhaps datasets
or more data is needed to distinguish them. In the case of with more attributes would be enough to solve these issues.
DDoS, for example, this could be information about the In the case of the simpler binary classification model, we
interval between messages. However, for these two classes in have yielded results that were very similar in all the cases.
a given dataset, none of the tested classifiers could make the Out of curiosity, we also looked into whether models using
prediction better. To achieve better accuracy, it would be unsupervised learning would be usable to some extent in our
necessary to use another, more robust dataset, or modify case. In general, these methods perform worse than learning
the model. with the teacher, so we did not expect high accuracy of
The problem could arise if we would have increased the the results. The results were very unpredictable, and the
required memory too much, the Raspberry Pi could have models could not predict the correct classes at all.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 293

 ACKNOWLEDGMENT
E. Testing configurability
This work was supported by Cultural and Educational Grant
The next test we performed was aimed at a different dataset Agency (KEGA) of the Ministry of Education, Science,
than what we used during model evaluation. The dataset we Research and Sport of the Slovak Republic under the project
used was the LoRa communication dataset, on which we tested No. 060TUKE-4/2022.
the universality of the implementation. The dataset contained
various anomalies that the communication may exhibit. The REFERENCES
test confirmed that we were able to adapt the system to other [1] A. A. Gendreau and M. Moorman:”Survey of Intrusion Detection
datasets, as the dataset used contained different attributes and Systems towards an End to End Secure Internet of Things”, 2016 IEEE
different amounts of attributes. In this case the dataset was 4th International Conference on Future Internet of Things and Cloud
(FiCloud), Vienna, 2016, pp. 84-90, doi: 10.1109/FiCloud.2016.20.
simpler and differences between anomalies much clearer,
therefore we have managed to reach very high accuracy. [2] Bruno Bogaz Zarpelão, Rodrigo Sanches Miani, Cláudio Toshio
Kawakani, Sean Carlisto de Alvarenga: A survey of intrusion de-
tection in Internet of Things, Journal of Network and Computer
F. Testing system as a whole Applications, Volume 84, 2017, Pages 25-37, ISSN 1084-8045, doi:
The last experiment performed was to test the Raspberry 10.1016/j.jnca.2017.02.009.
[3] Theuns Verwoerd, Ray Hunt: Intrusion detection techniques and ap-
Pi’s resource usage and whether the whole system runs proaches, Computer Communications, Volume 25, Issue 15, 2002,
smoothly on the device. During the test, we also ran the Pages 1356-1365, ISSN 0140-3664, https://doi.org/10.1016/S0140-
collector on Raspberry Pi. One by one, we generated streams 3664(02)00037-3.
[4] Benoit Claise, Brian Trammell, Paul Aitken: Specification of the IP Flow
and sent them to the collector, which stored them in disk. We Information Export (IPFIX) Protocol for the Exchange of Flow Infor-
mainly tested to see if the system would be too overloaded mation, RFC 7011, 2013, September, https://tools.ietf.org/html/rfc7011.
and if it would be possible to run other services on the device [5] A. Sperotto, G. Schaffrath, R. Sadre, C. Morariu, A. Pras and B.
Stiller:”An Overview of IP Flow-Based Intrusion Detection,” in IEEE
without slowing down the overall operation of the system. Communications Surveys and Tutorials, vol. 12, no. 3, pp. 343-356,
From observation, we found that during model training, the Third Quarter 2010, doi: 10.1109/SURV.2010.032210.00054.
system utilization is the highest. Especially when lazy loading [6] Tanja Zseby, Elisa Boschi, Nevil Brownlee, Benoit Claise: IP Flow
Information Export (IPFIX) Applicability, RFC 5472, 2009, March,
must be used, the processor was used to the maximum. After https://tools.ietf.org/html/rfc5472
training, i.e., during predication, the CPU usage was again [7] D. Dragomir, L. Gheorghe, S. Costea and A. Radovici:” A Survey on
minimal. Secure Communication Protocols for IoT Systems,” 2016 International
Workshop on Secure Internet of Things (SIoT), Heraklion, 2016, pp. 47-
62, doi: 10.1109/SIoT.2016.012.
V. EDUCATIONAL VIEWPOINT [8] T. Fu, D. Zhang, D. He and L. Xia: IPFIX Information Elements for
The executed experiments were carried out to achieve a inspecting network security issues, https://tools.ietf.org/id/draft-fu-ipfix-
network-security-01.xml.
prototype solution for the demonstration of data analysis of [9] I. Sharafaldin, A Habibi Lashkari and Ali A. Ghorbani, Toward Gener-
data flows obtained from the basic network environment. This ating a New Intrusion Detection Dataset and Intrusion Traffic Character-
mentioned prototype solution was created with the purpose of ization, Proceedings of the 4th International Conference on Information
Systems Security and Privacy - Volume 1: ICISSP, 2018, pp. 108-116,
enriching educational materials in the field of computer SciTePress, INSTICC, doi: 10.5220/0006639801080116, isbn: 978-989-
networks and cybersecurity. Accordingly the topic was 758-282-0.
introduced to students on network related subjects, where they [10] R. Meléndez, Juan and Rendón, Alvaro and Corrales Juan, Personalized
Service Degradation Policies on OTT Applications Based on the Con-
embraced their knowledge of network environments, for sumption Behavior of Users, 2018, pp. 543-557, journal: Lecture Notes
example during the creation of datasets out of information in Computer Science, isbn: 978-3-319-95167-6, doi: 10.1007/978-3-319-
collected from data flows through IPFIX, and interconnected 95168-3 37.
[11] Flow-based benchmark data sets for intrusion detection, Ring, Markus
it with new, acquired information about IoT environments. and Wunderlich, Sarah and Grüdl, Dominik and Landes, Dieter
and Hotho, Andreas, Proceedings of the 16th European Conference
VI. CONCLUSION on Cyber Warfare and Security (ECCWS), 2017, pp. 361 - 369,
ACPI, url: https://www.hs-coburg.de/forschung/forschungsprojekte-
The main objective of the work was to design and oeffentlich/informationstechnologie/cidds-coburg-intrusion-detection-
implement an IDS system for IoT environments that could be data-sets.html.
used on a Raspberry Pi device. The system we implemented [12] Sarhan, Mohanad and Layeghy, Siamak and Moustafa, Nour and
Portmann, Marius, NetFlow Datasets for Machine Learning- based
uses the principle of machine learning to detect intrusions Network Intrusion Detection Systems, 2020, url:
from network flows. Since machine learning is applied, the https://staff.itee.uq.edu.au/marius/NIDS datasets/.
system does not only target certain attacks, but can adapt itself
according to the dataset supplied. System could be easily
extended for more ways of filtering communication. Thanks to
its configurability, different datasets can be utilized for
detection. The problem of network security in the IoT world is
still developing, so more attention needs to be paid to this
topic in the future. Among the major problems we perceive
is the lack of interest of IoT device manufacturers in
securing them. Therefore, an intrusion detection system can
greatly help with the security of the entire IoT network.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 294

 Auditory Testing in Noise with Language
Barrier — A Case Study

Eva Kiktová, Peter Getlı́k

Department of Slovak Studies, Slavonic Philologies, and Communication
Faculty of Arts
Pavol Jozef Šafárik University in Košice
Moyzesova 9, 040 01, Košice, Slovakia
eva.kiktova@upjs.sk, peter.getlik@upjs.sk

Abstract—This paper is devoted to the issue of auditory tests in the diagnosis of Ukrainian refugees. The research
testing using matrix tests, which, because of their properties, indicated could be an initial and also fundamental justification
are suitable test material even for repeated measurements. It for the need to create or adopt and expand specific (Ukrainian)
describes the data preparation, testing procedure, and results of audiometric tests, if it were to turn out that the tests in the
auditory testing of refugees from Ukraine, for whom the Slovak country of residence are not effective even in provisional
language was unknown until they arrived in Slovakia. Fifteen
participants listened to five-word Slovak sentences in the presence
conditions. In this case, they can no longer be considered
of different levels of noise and marked the heard words through effective for the long-term stay of many Ukrainians who have
the developed testing interface. The results obtained indicate a expressed an interest in living in Slovakia or other countries.
significant influence of language on the results of auditory tests. The paper is organized as follows. Section II introduces the
It turns out to be necessary to use tests in the speaker’s native
language when examining auditory competence.
theory of the matrix test. Section III describes the selection
of test data and the creation of sound stimuli, as well as
Keywords—audiometry, language barrier, perception test information related to the participants and equipment used.
Section IV describes the testing procedure and obtained results,
I. I NTRODUCTION then follows the discussion in section V and finally the
conclusion in section VI.
The aim of the presented study is a general verification
of the potential of audiometric tests in a language not spo- II. M ATRIX TEST
ken by the tested probands. The need for such research is
clear wherever in practice there is an unexpected discrepancy The basic idea of audiometric testing using adaptive matrix
between the linguistic character of local diagnostics and the tests was first presented in Björn Hagerman’s work in Sweden
linguistic competences of the diagnosed. The language barrier, in 1981 [2]. The idea of generating test sets, as proposed by
which is often an impediment to full healthcare for migrants, Hagerman, proved to be very effective for various types of
seems to be particularly pronounced in military conflicts and tasks (e.g. perceptual tests). In recent years, tests created from
other sudden reasons for fleeing the home country, as it is the adaptive matrix have been a part of the audiometry [3], [4].
combined with many other humanitarian (especially logistical) They have become very popular due to the commonly used
challenges. and widely known test units which support a quick, reliable
and repeated measurements suitable for any degree of hearing
Even though that ideal results of audiometric tests in impairment.
the native language can be assumed, the initial provisional
conditions of contact with refugees may result in diagnosis Hagerman proposed a set of 50 words (in the form of a
in a language that is partially or completely unknown to the matrix), from which it was possible to create a large corpus of
refugees. Refugees have a natural language problem when 100,000 sentences with a fixed structure: noun, verb, numeral,
looking for work, studying, and communicating with authori- adjective, and object [5]. By playing a selected set of sentences,
ties or doctors, therefore there is a risk of overlooking health it was possible to measure hearing skills in people with normal
problems (for instance, auditory ones) when contacting them. hearing, and also in people with hearing loss. An important
Health problems can be unintentionally attributed to cultural part of these tests was – and still is – a specific type of noise
or psychological reasons in the case of (mis)understanding (so-called ‘babble noise’) [6]. Implementing the test without
in interpersonal contact. Experts in Slovakia, who treated an noise or for other types of noise is less demanding for the
unexpected number of Ukrainian citizens, also had to deal with person being tested. A person who can communicate without
this challenge. any problems in common conditions can feel difficulties in
noisy places such as a restaurant, station, cinema or in a
According to official information from the Ministry of group of speaking people; therefore testing in noise is still
Interior of the Slovak Republic, 1,405,019 people have entered very important.
Slovakia via the Ukrainian border since 24 February 2022,
and 115,748 asylum seekers have been granted asylum since 1 The proposed adaptive matrix by Björn Hagerman is de-
March 2022 [1]. The general focus of our research is thus made picted in Fig. 1 [5]. It is possible to create ten five-word
concrete by assessing the (in)ability of Slovak audiometric sentences from the matrix (each word is used only once).

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 295

 A native female speaker was asked to record a series of
100 sentences as naturally as possible, with an emphasis on
correct pronunciation and a stable speech rate. The resulting
recordings were in the mono channel mode, and in the form
of a *.wav file format with a resolution of 16 bits per sample
and a sampling frequency of 44.1 kHz.
The methodology for creating matrix tests is based on
cutting 100 recorded sentences into individual words, from
which it is subsequently possible to create new sentence
combinations. A visual example of audio editing is shown in
Fig. 2. The previously stated principle of creating a complete
new set of recordings in the range of 300 unique sentences
Fig. 1. Swedish matrix of 50 words created by Hagerman was developed in this way. During the creation of a new
sentence, transients between words were also considered, and
the intensity of words was adjusted as necessary so that the
III. E XPERIMENTAL SETUP
new sentence seemed as natural as possible.
In this section, key information about the test data and
prepared sound stimuli as well as the people involved in the
test and the technical equipment used will be described.

A. Test data
The developed and currently still tested adaptive matrix
contains ten popular names (male and female) in the Slovak
language, ten verbs in the present tense, ten numerals, ten ad-
jectives, and finally ten objects, which, in various combinations
(and always with one word from each category), create a five-
word sentence structure. Words in each category should have
approximately the same length. For this reason, the words in
our matrix are monosyllabic or bisyllabic; further they should
be frequently used and neutral in all aspects (unpredictable, but
not causing unpleasant emotions). For example, the sentence:
Pavol chce mnoho malých vedier (Pavol wants many small Fig. 2. Creating sound stimulus — one sentence in Audition software
baskets), contains the state verb ‘to want’ which has the
desired character, does not determine the object, and has a The set of 300 sentences that had been created was then
neutral meaning. For the above reasons, the verb ‘to cook’, listened to by three quality assessors, and each sentence was
for example, is excluded because it is closely related to the graded according to the criteria of the MOS (Mean Opinion
object and would lead the participant towards an answer. The Score) tests (grade 1—bad, 2—poor, 3—fair, 4—good, 5—ex-
third column of the matrix contains basic numerals (seven, cellent). Sentences with an average value below MOS = 4 were
eight, one hundred, etc.) and indefinite numerals (a lot of, subsequently additionally corrected to eliminate the presence
few, many, etc.). Adjectives describing an object appear in the of an undesirable phenomenon.
matrix in the fourth column. Their forms, like the numerals, The noise used in the tests was generated by mixing 30
must correspond to the object, and also have to be completely tracks, each containing three different sentences, see Fig. 3.
interchangeable with other adjectives. The object is placed in Subsequently, the beginning and end of the noise recording
the last position of the sentence. It is possible to create a total were cut. The spectral content of sentences and noise recording
of 100,000 (10 names x 10 verbs x 10 numerals x 10 adjectives were similar. Because of this spectral feature, it is the most
x 10 objects) unique sentences by different combinations of challenging type of noise for speech perception.
words in the matrix.
The resulting test recordings were created by mixing the
Finding suitable words that have these qualities and that are speech and noise in a specified SNR (Signal to Noise Ratio).
also frequently used in the Slovak language was a challenging The mixing of the sounds (speech and noise) was realised
step in the design of an adaptive test matrix [7], [8]. through the test interface, where the volume level was adjusted
according to the desired SNR, and a single-channel recording
B. Preparing sound stimuli was created containing both sounds. Testing was done at three
SNR levels, namely -4.3 dB, -6.3 dB and -8.3 dB. These SNR
The recording of a database of 100 sentences (so that values represent 20%, 50% and 80% respectively, of word
all interword transients are ensured) was carried out in the recognition for native Slovak speakers.
LICOLAB (Language, Information, and Communication Lab-
oratory) at the Faculty of Arts, Pavol Jozef Šafárik University C. Participants
in Košice, Slovakia. It is equipped with a soundproof studio
which contains a Rode NT 2000 microphone, an ART Voice For the research sample for audiometric tests to be relevant
Channel microphone preamplifier, and Audition software. in relation to the objectives of the present study, we wanted to

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 296

Fig. 3. Screenshots of creating the noise recording (left); time and spectral Fig. 4. G.R.A.S. instruments for calibration
domain of the noise recording in Audition (right)

course) took an average of 4 months (STDEV = 3.65). The

recruit Ukrainian citizens with temporary residence in Slovakia longest course lasted 12 months and the shortest 1 week. In
who have little or no knowledge of Slovak. To find a research the questionnaire, we also asked for a self-assessment of the
sample of this type, we contacted Slovak teachers (employees health status of the probands and phenomena that could affect
and volunteers) in language schools who help Ukrainians learn their hearing (such as work in a noisy environment, an injury,
Slovak. They interpreted our initial calls and helped mediate etc.). There was no mention of any obstacle in their answers.
the first contact with the group of probands. There was a space for notes in the questionnaire, in which, after
the testing was completed, the test subjects listed everything
The test in question was requested to be performed on ”interesting” that they noticed in the tests, so that we could
pre-prepared devices in the LICOLAB laboratory. Given the identify any recurring problematic areas of the audiometric test
specific characteristics of the target group of probands, who itself.
were in a foreign city without knowledge of the language,
it made sense for them to take part in laboratory testing at D. Used equipment
the same time, and immediately after the previous Slovak
lesson. For logistic and communication reasons, we decided In order to measure auditory skills, fifteen personal com-
to perform all audiometric tests at once, which resulted in a puters (Win 10, 64-bit, Intel Core i7 - 7700 CPU, 16 GB RAM)
basic limitation of the scope of the research sample. This could with external sound cards (Creative Sound Blaster X-Fi HD)
not exceed the number of computers with prepared tests that and closed AKG K77 headphones were used [9].
were available. Therefore, the research was conducted on a
Before the testing, the whole sound chain was calibrated
sample of 15 Ukrainian women.
by G.R.A.S. 90AB (artificial ear type IEC 60318-2, connected
In order to simulate to a certain extent, the ‘random‘ with microphone type 1” 40EN, and preamplifier type 26AB).
process of real audiometric tests, which also includes clari- G.R.A.S. Audiometer Calibration Analyzer HW1001 was con-
fication of instructions and collection of basic data for further nected by G.R.A.S. AA0008 cable to the artificial ear (Figure.
administration, we did not plan the testing with the presence of 4). The setup described ensured that all participants had the
a professional interpreter, but to facilitate communication we same conditions during the tests. The tests were conducted in
used an offer from one of the test subjects, who additionally laboratory conditions (LICOLAB, FA, UPJŠ).
repeated some of our instructions in Ukrainian.
IV. P ERCEPTUAL TESTING AND RESULTS
The participants of the study were asked to fill out the
accompanying questionnaire before testing. The questionnaire A. Perceptual testing
was in English. In it, we found out on a scale from 1 to 10
The participants successively performed 10 tests, each of
how the test subjects perceived their general language skills in
them containing 30 sentences (3x10 sentences). This group
Slovak. We further divided this question into writing, listening,
of 10 tests can be called a group of 10 triplets. Each test
and speaking skills. On the same scale, we were also interested
participant heard a total of 300 sentences, divided into 10
in their joy/willingness to learn a new language. To further
triplets. Hence, in each triplet, there was the opportunity to
illustrate possible language skills, we also found out how
hear each word exactly 3 times (with SNR = -4.3, -6.3, and
long they had been learning Slovak and how long they had
-8.3 dB). Each sentence could be played only once, and if the
been in Slovakia. Their average length of stay in Slovakia
percipient did not hear the given sentence/words, she did not
was approximately 8 months (STDEV = 4 month), while the
have the opportunity to listen to it again.
longest stay was 14 months and the shortest 10 days. They
rated 4.5 on average, on a scale of one to 10 (STDEV = At the beginning of the testing process, participants were
2.03), their knowledge of Slovak or a feeling of how close they given test instructions and a brief demonstration of how the
were to the Slovak language. Learning Slovak (on a language testing would take place. They also filled out accompanying

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 297

questionnaires. Subsequently, the test participants started to maximum possible level right at the beginning of the playback
carry out the prepared tests one by one, from the first to the of the sound stimulus. The opposite situation can be observed
tenth. Breaks took place after the third and seventh triplet test. within the category of words located in the last position in the
The participants were also instructed that they could take a sentence. These words represented objects. Due to the variety
break whenever they wanted, but this option was minimally of this word category, its placement within the sentence, and
used. the assumed gradual decrease in attention, an average score of
6.4% was achieved. The most frequently correctly identified
The testing was carried out using a GUI (Graphical User objects were a spoon (lyžica), a house (dom), and an apartment
Interface) created in the Matlab environment (Matlab R2021b). (byt). The least correctly identified object was a bowl (misa).
The visual appearance of the testing interface consisted of a Here, we took into consideration all 15 participants, because
matrix of words (5x10 words), the five fields (in which selected we assumed that their perceptions could also help to identify
words were depicted), and two buttons (one for playing the categories of words with the potential of easier recognition.
sound and the second for confirming selected words). Each
percipient played the sentence by pressing the play button, In Fig. 8, the summary test results are shown according to
then marked the words heard and confirmed their choice of the SNR parameter, which represents the degree of degradation
words with the confirm button. of the useful signal (sentence) by the babble noise. The SNR
values, on which the testing took place, were derived from the
perception of Slovaks on Slovak data, on which they achieved a
B. Results score of 20%, 50%, and 80%. As can be seen from the results,
Considering the reasons and ways our test participants Ukrainian participants achieved approximately 10 times worse
arrived in Slovakia, no selection of participants was made, so recognition scores compared to Slovak participants. Their
our group of participants represents a random group of people scores were 2.01%, 4.70%, 8.30% for -8.3 [dB], -6.4 [dB]
(women). Before conducting the experiment that we decided to and -4.3 [dB].
implement, it would be appropriate to carry out a basic hearing
examination using tone audiometry [10] and thus determine the V. D ISCUSSION
hearing thresholds in the range of speech. The participants of Based on the results obtained from a random sample
our experiment did not complete this type of examination for of participants, we can try to estimate the consequences of
detection of a potential hearing deficit. For this reason, the communication problems, that can still occur, especially in
results presented in Fig. 5 may contain extremely low score cases where refugees need to communicate with the Slovak
values for participants labelled UA4, UA5, UA6, UA9, and authorities, need medical care, and so on. A hearing examina-
UA14. Their results can point to an up until-now undiagnosed tion is a specific type of examination where it is possible to
hearing deficit or significant language and social barriers that demonstrate not only a medical problem, but also a cultural
were unknown to us before testing, for instance, a social and social one, especially in the case of speech audiometry,
background (employment, education, etc.). The average age of which is an examination method for hearing diagnosis, during
members of this group was 51 years this was approximately which the patient responds adequately to heard speech stimuli.
20 years older than the age group of remaining participants, This method of examination reveals possible communication
whose results are analysed in more detail. Thus, the results of deficits that may not show up in other types of examination.
the five previously mentioned participants (UA4, UA5, UA6, Under normal circumstances, audiometric testing is performed
UA9, UA14) are considered only for the analysis of sentence in the language that is preferred by the patient. This language
structure presented in Fig. 7. is usually his or her native language. This routine was used
The graph in Fig. 6 shows the summary results of ten by phoniatricians or otorhinolaryngologists until the end of
test participants in relation to the sequence number of the February 2022. Refugees from Ukraine have become our
triplet test. The results obtained show a gradual improvement fellow citizens, and their presence has resulted in several
in triplet scores. The score of the first triplet is very low (the changes in various areas of life. It requires a re-evaluation
lowest of all), and gradually with the increasing triplet number, of the usual procedures and, if necessary, their adaptation
higher score values were reached. This observation represents a to their specific needs. The language barrier also manifested
common phenomenon that is often present in perceptual testing itself significantly in the perceptual tests performed. Tests
[11]. The results of initial tests tend to be negatively affected with noise helped to partially reveal the degree of deficit in
by the lack of experience and the initial errors that occur during hearing and understanding the presented content [12]. As under
the testing routine. For this reason, the results of the first triplet normal conditions, it is very difficult and practically impossible
were not used in further processing. to ensure ideal conditions for communication, testing in the
presence of noise becomes relevant.
In the next picture, you can see the word recognition score
A special type of test – Matrix test – could be suitable
in the sentence (Fig. 7), where the proper name category
due to their limited vocabulary (50 words, ten words in five
dominates with an average score of 18.22%. Other categories
categories). However, the implementation of these tests in
of words reach comparable values of the resulting score. This
noise revealed a serious deficiency, namely an insufficient level
category of proper names was probably the least affected by
of comprehension, which significantly affected the score of
the language deficit, due to the aural similarity of Slovak
perceptual hearing tests.
and Ukrainian names. The most frequently correctly identified
proper names were Peter, Mária and Eva, and the least cor- Extremely low scores in the tests were achieved by five
rectly identified name was Jožo. It is also, necessary to admit probands, who may also have had the problem of not under-
that the attention of the listener could be concentrated to the standing some instructions. We decided to leave them out of

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 298

Fig. 5. Results of 10 triplets — all 15 participants

Fig. 6. Results of 10 triplets — 10 participants Fig. 7. Results of 9 triplets — all participants

the final data analysis. Our analysis also indirectly leads to and recalling information heard in short-term memory [13]. It
illustrations of the inadequacy of testing in a foreign language. is obvious that familiar sounds (words) are much easier to store
Nevertheless, it is not desirable for its results to stem mainly and recall if they are needed.
from misunderstandings, because in a real environment, after
The experiments were carried out at three noise levels (-8.3
such low values, the test would probably be repeated with more
dB, -6.3 dB and -4.3 dB), which corresponded to the level of
rigorous instructions, or a different method of diagnosis would
recognition of the heard content on 20%, 50%, and 80% in the
be used.
case of testing Slovaks in the Slovak language. Of course, the
Ukrainian participants identified the words according to decrease in recognition results was expected, but the rate of
their auditory perception, while sometimes it was enough to this decrease indicates that different examination procedures
hear only part of the word. The Slovak participants of these should be preferred, for instance pure tone audiometry that is
tests also progressed in the same way, and they were much language independent, or using other sounds stimulus that will
more successful in this task; they needed significantly less be familiar to the patient.
information to identify the word.
If we were to recommend a category of words for a
A phonematic (auditory) memory is responsible for storing hearing test in patients who do not know the test language,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 299

 noise. We attribute this fact to the significant language, social,
and cultural shock to which they were exposed becaused of
the conflict.
The overall results indicate that the comprehension of the
heard content becomes crucial in the assessment of auditory–
communication skills, especially in the case of a foreign
language in the presence of noise. The issue of speech com-
prehension and its connection with hearing testing turns out to
be a key factor influencing the test result.

ACKNOWLEDGMENT
This work was supported by the Scientific Grant Agency of
the Ministry of Education, Science, Research and Sport of the
Slovak Republic and the Slovak Academy of Sciences under
the research project VEGA 1/0344/21 and the Slovak Research
Fig. 8. Score of 9 triplets for 10 participants and Development Agency under the research project APVV-
22-0261.

we would choose the category of proper names, because here R EFERENCES

the language barrier seems to be smaller compared to other
categories of words. In our test, there were male and female [1] https://www.minv.sk/?tlacove-spravy-6&sprava=v-uplynulom-tyzdni-
poziadalo-o-docasne-utocisko-v-sr-551-osob, 9.5.2023.
names, but it is possible to assume that world-famous cities or
[2] B. Hagerman, Sentences for testing speech intelligibility in noise, Report
other geographical names would play a similar role. The other TA No. 101. Technical Audiology, 1981.
word categories used (verbs, adjectives, numerals, and objects) [3] HörTech, International matrix tests (Reliable speech audiometry
were difficult to recognise. The content of the sentences was in noise), 2019, online: https://www.mack-team.de/pdf/ht-
completely unpredictable, so any attempt to predict the content internationalermatrixtest.pdf
without hearing at least part of the word was unsuccessful. [4] B. Kollmeier, A. Warzybok, S. Hochmuth, M. Zokoll, V. Uslar, T, Brand,
K.C. Wagener, The multilingual matrix test: Principles, applications and
The presented results of this case study, against the back- comparison across languages: A review. Int J Audiol, 2015, online:
ground of audiometric testing, point to the language barrier that https://doi.org/10.3109/14992027.2015.1020971
refugees must face. The common environment in which com- [5] B. Hagerman, Sentences for testing speech intelligibility in noise. Scand
munication takes place is very often affected by the fluctuating Audiol, No. 11, p. 79-87, 1982.
level of noise [14] that disrupts communication (noise from [6] M. Nilsson, S.D. Soli, J.A. Sullivan, Development of the Hearing in
traffic, other people’s speech, noise in shopping malls, and Noise Test for the measurement of speech reception thresholds in
quiet and in noise. Acoust Soc Am., 95(2), p. 1085-99, 1994, doi:
so on). As has been proven, distracting sounds significantly 10.1121/1.408469.
limit the success of hearing and subsequent comprehension [7] R. Panocová, R. Gregová, Designing the Slovak matrix sentence test.
of information. In the context of education (linguistic or any International Journal of Applied Language Studies and Culture, 2(2), p.
other), retraining, and choosing an appropriate profession, for 33–38, 2019. https://doi.org/10.34301/alsc.v2i2.23
instance, it is very important to pay attention to adequate [8] R. Gregová, R. Panocová, Distinctive Features of Phonemes and
acoustic conditions, especially in cases where the presence First Language Acquisition by a Normally Developing Child With-
of a language deficit can be expected. In this way, hearing out Hearing Impairment: Evidence from Slovak. International Jour-
nal of Language and Literary Studies, 4(4), p. 66–75, 2022.
and subsequent comprehension will not be degraded due to https://doi.org/10.36892/ijlls.v4i4.1086
unsuitable acoustic conditions. [9] J. Zimmermann, E. Kiktova, Influence of the native language on audio-
metric tests in a foreign language. Otgovornostta pred ezika. Kniga 7,
p. 68-75, 2021. ISBN 1313695X.
VI. C ONCLUSION
[10] R.H. Margolis, D.E. Morgan, Automated Pure-Tone Audiometry: An
Generally, speech perception testing tells us what audi- Analysis of Capacity, Need, and Benefit, American Journal of Audiology,
17(2), p. 109-13, 2008, https://doi.org/1059088900170002109
tory information is available to our test sample (Ukrainian
[11] T. Nuesse, B. Wiercinski, T. Brand, I. Holube, Measuring Speech
refugees), what they are hearing in the noise, and how much Recognition With a Matrix Test Using Synthetic Speech. Trends in
auditory information is available for them in such conditions Hearing, vol. 23, 2019, doi:10.1177/2331216519862982
to use for everyday life, or in other activities, for example in [12] L.Kilman, A. Zekveld, M. Hällgren, J. Rönnberg, The influence
learning or at work. of non-native language proficiency on speech perception perfor-
mance. Frontiers in Psychology, vol. 5, 2014, ISSN 1664-1078,
The goal of this case study is to find out whether it is DOI=10.3389/fpsyg.2014.00651
possible and correct to assess the auditory competence of an [13] J.S. Sachs, Memory in reading and listening to discourse. Memory &
individual using a language that the given person does not Cognition, vol. 2, p. 95–100, 1974, https://doi.org/10.3758/BF03197498
master, and how much this fact will affect the ability to hear [14] M. Van Os, J. Kray, V. Demberg, Rational speech compre-
and comprehend. hension: Interaction between predictability, acoustic signal, and
noise. Frontiers in Psychology, vol. 13, 2022, ISSN 1664-1078,
In our random sample of probands, up to one third showed DOI=10.3389/fpsyg.2022.914239.
unusable results for hearing evaluation. Their results were not
sufficient even in the case of recordings minimally affected by

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 300

 The Influence of Energy Efficiency on the
Production of Emissions in Safety Small
Production Systems
I. Klačková*, D. Wiecek**, V. Benko* and T. Dodok*
* University of Žilina/Department of Automation and Production systems, Žilina, Slovakia
** ATH – University of Bielsko Biala, Institute of Industrial Engineering, 43-309 Bielsko Biala, Willowa 2, Poland
ivana.klackova@fstroj.uniza.sk
dwiecek@ath.bielsko.pl
vladimir.benko@fstroj.uniza.sk
tomas.dodok@fstroj.uniza.sk

Abstract—This paper deals with the issue of wood burning. in which heat and light are released in exothermic
In the first part, the properties of wood and the conditions reactions, is called combustion [2].
of combustion and gasification are comprehensively and The fuel is heated by the action of flame radiation and
clearly processed. The next section describes the from hot walls. Subsequent drying of the fuel and
mechanisms of emissions that arise during the combustion
evaporation of water from the fuel takes place at
of wood biomass, as well as the basic specification and
construction of small heat sources. The main part of this temperatures from about 100 °C, while the evaporated
paper is the analysis of the influence of primary and water is discharged together with the flue gases into the
secondary air on the production of power and emission chimney. At temperatures from about 150 °C, pyrolytic
parameters in the combustion of wood biomass on a model decomposition of dry fuel occurs. Due to the supply of
heat source. A gasification hot water boiler with a nominal primary combustion air, the dried fuel is gasified at
heat output of 25 kW was chosen as a model heat source. temperatures of about 250 °C into flammable gases, such
The analysis was realized on the basis of numerical as carbon monoxide (CO) and hydrocarbons (CXHY),
simulation by CFD modeling methods, as well as on the whereby solid fuel residues, so-called solid carbon (C).
basis of experiments.
Wood contains more hydrogen than most other fuels, so it
burns significantly more hydrocarbons. The last phase of
I. INTRODUCTION the biomass combustion process is the burning of solid
In the second half of the 20th century, knowledge about carbon (C) in the presence of oxygen, which takes place
the negative effects of industrial production on the at temperatures of about 600 °C, whereby waste is
environment and man himself was reflected in the lives of formed. The oxidation of flammable gases with oxygen to
people in industrial society through environmental carbon dioxide and water is carried out at temperatures
legislation. The issue of limiting the creation and from about 700 °C. All of the above events can take place
production of emissions of individual production activities at any point in the combustion process under different
has become part of both the work of designers in the conditions (local temperature differences, flow rates,
conceptual design of new technological units and the work oxygen content, etc.) [3].
of production management ensuring optimal operating
conditions for the production of material and energy III. MODEL HEAT SOURCE
products with minimal environmental impact.
The EKOS 10–30.01 boiler was selected as a model
Small heat sources represent relatively low boiler for research into the influence of primary and
environmental pollution, but with a large number of their secondary air on the production of emissions in heat
applications as heat sources, this represents a significant
amount of emissions [1]. sources for wood combustion. It is a gasification hot
water boiler for burning dry wood mass from sawdust,
through wood briquettes and chips to logs. The interior of
the boiler consists of a filling chamber where the fuel is
II. COMBUSTION OF WOOD
dried and gasified. The wood gas then passes through a
Combustion of fuel involves a number of physical and nozzle into the combustion chamber, where it burns with
chemical processes in which the chemical reactions of the the aid of secondary air. The flue gases give off their heat
individual combustible components of the fuel with in the exchanger [4-6]. The EKOS 10-30.01 boiler is
oxygen take place simultaneously at high temperature, shown in Figure 1.
the energy chemically bound in the fuel being converted
into heat and combustion by-products (flue gases and
ash). Oxidation of flammable components of fuel is most
often carried out by oxygen from atmospheric air. The
process of oxidizing flammable substances with oxygen,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 301

 wood combustion, we performed measurements at
different settings of the primary and secondary air
regulator, Figure 2. The secondary (S) air is fed into the
combustion chamber through the inner cross-section of
the jig grate, where the air is preheated [10]. A very small
part of the secondary air is fed into the gasification space.
The setting of primary and secondary air is possible in the
range 0 - 15 mm.

Figure 2. Primary and secondary air supply regulator.

Measurements were made for fuel:

Figure 1. Hot water boiler EKOS 10 - 30.01. • spruce wood with bark,
• clean spruce wood,
The boiler has the following parameters
• spruce bark.
• weight 250 kg
• dimensions 600/800/1200 mm Measurements with spruce wood with bark were made
• recommended fuel at the following settings:
lump wood, wood, briquettes, chips • primary air maximally open, secondary air
• substitute fuel maximally open
brown coal, sawdust, biomass • primary air closed to 5 mm, secondary air
• method of combustion gasification maximally open
• chimney operating draft 15 Pa • primary air closed to 10 mm, secondary air
• max. efficiency at rated power 86 % maximally open
• tank volume 163 dm3 • primary air maximum open, secondary air closed
at 5 mm
• rated power 25 kW
• primary air maximum open, secondary air closed
• diameter of the exhaust neck 160 mm at 10 mm
IV. EXPERIMENTAL EQUIPMENT • primary air closed at 5 mm, secondary air closed
at 5 mm
Requirements for heat sources up to 300 kW and the • primary air closed at 5 mm, secondary air closed
method of their measurement are given by the standard at 10 mm
STN EN 303 - 05 "Heating boilers for solid fuels sup-
• primary air closed at 10 mm, secondary air
plied manually and automatically, with a nominal output closed at 5 mm
of up to 300 kW". As the EKOS 10 - 30.01 gasification
• primary air closed at 10 mm, secondary air
boiler has a heat output range from 10 to 30 kW, an closed at 10 mm.
experimental device for research into the influence of
primary and secondary air on the production of emissions
in heat sources for wood combustion was designed and Measurements with pure spruce wood and bark were
implemented according to this standard. made at the setting - primary air maximally open and
secondary air maximally open [11,12]. Before the actual
When measuring the output of a hot water heat source,
measurement on the experimental equipment, the
the volume flow of the heating water must be measured humidity of spruce wood was determined, resp. the bark
using a flow meter and also the temperature of the used in the experiments and its calorific value by the
heating water at the inlet to the heat source (boiler) and at calorimetric method.
the outlet from the heat source.
The experimental equipment consists of a boiler and a V. ANALYSIS OF EXPERIMENTAL RESULTS
cooling circuit, which are separated by a heat exchanger.
Analysis at different primary and secondary air
A heat measuring source is connected in the boiler circuit
settings
[7-9]. A water-air cooler is connected in the cooling
circuit to remove the generated he-at output from the heat
The results show that the highest heat output Pkot =
source.
25.8 kW was achieved with the setting of the primary
When analyzing the influence of primary and secondary
combustion air (P) P = 5 mm and the setting of the
air on the production of emissions in heat sources for
secondary combustion air (S) S = 5 mm, but at this

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 302

setting a relatively high concentration of CO emissions in Analysis of the impact of combustion chamber
the flue gas CO = 4616 mg.m-3 was achieved, which is displacement on fuel wood with bark, clean wood, bark
related to the relatively low combustion efficiency = 69,5
%. From the point of view of achieving the highest The work also investigated the influence of the formation
possible heat output and efficiency of the heat source and of the composition of wood fuel on the parameters of the
the lowest possible CO concentration, the most suitable heat source and on the production of emissions at the
setting is P = 15 mm (max) and S = 10 mm, when the same setting of primary and secondary air [19]. The
heat output Pkot = 24.8 kW was achieved, efficiency was primary secondary air was set to a maximum value, i. P =
86,9 % and CO concentration = 1694.1 mg.m-3. However, 15 mm and S = 15 mm. The following tables show the
this setting corresponds to a relatively high value of the measurement results for wood with bark, clean wood and
concentration TZL = 302,5 mg.m-3, but in comparison spruce bark. The measurement with wood and bark fuel
with other settings this value is among the lower ones. was measured without modification of the combustion
The measurements showed that in terms of the achieved chamber from the manufacturer (without displacing the
power, its value is greatly influenced by the temperature space behind the combustion chamber) - taken from the
in the combustion chamber or. temperature before previous analysis [20,21]. Other measurements were
entering the heat exchanger [13-15]. At temperatures in made when the combustion chamber was displaced,
the combustion chamber approx. 1000 °C resp. in front of Figure 3.
the heat exchanger approx. 600 °C the heat outputs were
reached at the level of approx. 25 kW. At lower
temperatures, lower heat outputs were achieved than
under measuring conditions P = 15 mm, S = 15 mm (t sk =
843, tvt = 460 °C, Pkot = 18,7 kW), resp. P = 15 mm, S = 5
mm (tsk = 875, tvt = 472 °C, Pkot = 16,0 kW)
The efficiency of the heat source is significantly
affected by the production of CO emissions during
combustion and the size of the flue gas temperature in Figure 3. Combustion chamber displacement.
front of the heat exchanger [16-18]. The highest
efficiency was achieved under measuring conditions P = The same fuel dose of 11 kg was used for all
10 mm, S = 10 mm (CO = 1694,1 mg.m-3, tvt = 605 °C measurements.
and the boiler efficiency was 86,9 %). The results show that the displacement of the
In terms of CO emissions, it was shown that the lowest combustion chamber had a significant effect on
concentration values we-re reached at the setting P = 15 increasing the heat output of pure wood fuel as well as
mm (max) and S = 10 mm CO = 1694.1 mg.m-3 and at pure bark fuel. With pure bark fuel, a higher heat output
the setting P = 10 mm and S = 10 mm when CO was achieved compared to pure wood, but with
concentration = 2345 mg.m-3. However, at the setting P = efficiency, the opposite is true. However, the
10 mm and S = 10 mm, the highest concentrations of displacement of the combustion chamber had a
TZL = 517 mg.m-3 were achieved. The lowest significant effect on the excess air during combustion,
concentration of TZL was reached at the setting of P = 15 which was caused by directing the secondary combustion
mm (max) and S = 15 m (max) when it was reached air into the combustion chamber space and its use in the
concentration TZL= 18 mg.m-3, but in this case high combustion of wood gas [22]. This adjustment also had
concentrations of CO = 5091 mg.m-3 were achieved. an impact on the production of CO emissions, which
After the analysis of captured TZL on the filter during the decreased by about 1000 mg.m-3. This design change did
measurements, it was found out in cooperation with Ing. not have a significant effect on the production of TZL
Jarmila Gefertová, CSc. from the Department of concentration. Displacement of the combustion chamber
Chemistry and Chemical Technology of Wood, TU resulted in a reduced temperature in the combustion
Selected that about 78% of TZL is 78-88% carbon and chamber compared to its previous storage in the
the rest of the ashes. combustion of both pure wood and pure bark [23].
The measured results show that for the selected model However, the temperature in front of the heat exchanger
boiler, the most suitable setting of primary and secondary increased significantly (approx. 100 - 150 °C), which had
air to set P = 15 mm (max) and S = 10 mm, when the a significant effect on the heat output of the heat source
highest efficiency 86,9 %, one of the highest thermal (approx. 4 to 5 kW)
power Pkot = 24,8 kW and the lowest value of CO The same concentrations of CO = 3990 mg.m-3 were
concentration = 1694,1 mg.m-3. At this setting, one of the achieved in the combustion of pure wood as well as in the
lowest values of the concentration TZL = 302,5 mg.m-3 combustion of pure bark [24,25]. With pure wood fuel,
was achieved. the production of NOx emissions was lower than with
The measurements also showed that for a given type and bark combustion by about 100 mg.m-3.
construction of a model boiler, it is appropriate to reduce The method consists in dividing a certain examined
the supply of secondary air to achieve optimal parameters area by a network or a grid so that the values of quantities
of the boiler, in terms of heat output, efficiency and are evaluated in nodes of the network. The derivatives of
production of CO emissions. transport equations are approximated by differential

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 303

relationships of values in adjacent nodal points. Thus the Results from the Boiler Model
partial differential equation passes into a system of
algebraic relations [26]. There are two basic variants: The air is sucked in through a grid at the bottom of the
explicit, where the solution of the system of these boiler, it is further divided into air, which goes directly
algebraic relations essentially means simple matrix between the grate and air, which is inherited in the grate
multiplication, and implicit. However, the calculation of bars and then is sucked into the burner, where it further
states at nodal points means that it is necessary to reacts with wood gas and CO, it is can be seen in the
perform an inverse calculation of a very large matrix. following Figure 5 and Figure 6 show the contours of the
There are also problems with the consistency and stability weight fraction of wood gas.
of the convergence of the calculations. In the case of
several chemical reactions, the total release rate resp.
energy consumption is given by the simple sum of the
rates of each reaction [27].

Limitations for the Diffuse Flame Model

The diffuse flame model can only be used if the

reacting flow meets several requirements. First, FLUENT
requires the flow to be turbulent. Secondly, the reacting
system must contain a fuel stream, an oxidizer, and
possibly a secondary stream (another fuel stream or
oxidizer, or a non-reactive stream - inert). Finally, the
rate of transformation must be so great that the mixture is
close to chemical equilibrium [28]. The principle of
calculating thermodynamic equilibrium is solved using
the White-Johnson-Dantzig method - minimization of Figure 5. Gas flow trajectories with temperature contours.
free enthalpies. e.g. C8 H18 + 12,5 O2 + 47 N2 what, we
can write in summary C8H18O25N94. The flue gases
consist of the following components: CO2, CO, Cg, Cs,
H2O, H2, OH, H, O2, O, N2, NO, N.
VI. RESULTS FROM THE BOILER MODEL
Computing network

The computational network was generated using the

GAMBIT program (Figure 4) with defined boundary
conditions. Four-walled and five-walled cells were used Figure 6. Gas flow trajectories with temperature contours.
and the number of all cells was 531018.
VII. CONCLUSIONS
The presented paper tries to contribute to a more
detailed analysis of the issue of using wood in obtaining
thermal energy by burning it. The dissertation addressed
the influence of primary and secondary air on emissions
production. At the same time, an experimental method of
measuring the influence of the share of primary and
secondary air on emission production is used, confronted
with the numerical 2D and 3D CFD modeling method.
The model heat source EKOS 10-30.01 was used.
The simulation calculation and model were performed
before the measurement itself and the input parameters
were set to the amount of fuel per unit time and the
amount of air was set by the estimated pressure drop in the
chimney. The air flow was adjusted by the calculated
pressure losses in the boiler. The measurement was made
after the calculation.
Figure 4. Computing Network. The obtained experimental results showed us that a
significant effect on increasing the heat output of the heat
source results in a suitable setting of the primary and
secondary air. It follows from this dependence that the
optimal solution for the heat source EKOS 10-30.01
appears to be the measurement condition No. 6, where the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 304

highest output is achieved. The temperature during production, in view of the new accreditation of the AVS
combustion was relatively high, ensuring the conversion engineering study program.
of even more difficult to burn parts, which could leave the
chimney. It does not make sense to reduce the flame REFERENCES
temperature too low, as it would not convert all the less [1] L. JEROEN, A. van den BERG. Optimal routing in an automated
volatile components. When oxygenates are burned at low storage/ retreival system with dedicated storage. IIE
temperatures, they tend to form dioxins and aldehydes. Transactions, 31, no. 5 1999. p. 407–415
The most disadvantageous variant is the measurement [2] A. Figiel, I. Klačková, “Safety Requirements For Mining
condition - opening the supply of primary and secondary Complexes Controlled In Automatic mode,“ Acta Montan.
air to the maximum, where the heat output of the heat Slovaca, vol. 25, DOI 10.46544/Ams.V25i3.13., 2020, pp. 417–
source increases sharply at the expense of fuel 426.
consumption and the fuel burn-up time is the fastest. [3] I. Kuric, I. Klačková, Y. R.. Nikitin, I. Zajačko, M. Císar, K.
Tucki, “Analysis Of Diagnostic Methods And Energy Of
The increase in the amount of primary combustion air Production Systems Drives,“ Processes, Mdpi, 9, 843,
at the inlet to the combustion process also affects the Doi.Org/10.3390/Pr9050843, 2021.
amount of CO in the flue gas, which is manifested during [4] I. Pavlenko, M. Sága, I. Kuric, A. Kotliar, Y. Basova, J.
the combustion of the fuel charge, whether positive or Trojanowska, V. Ivanov, “Parameter Identification of Cutting
negative. It has a negative effect when fuel is burned Forces in Crankshaft Grinding Using Artificial Neural Networks“
Materials, vol. 13 (23), article number 5357, eISSN: 1996-1944,
(increased CO production) and also in the case of 2020.
combustion of volatile fuel components, when the primary [5] D. Wiecek, A. Burduk, I. Kuric, „The use of ANN in improving
combustion air affects the increased production of these efficiency and ensuring the stability of the copper ore mining
components and there is secondary combustion air in the process,“ Acta Montanistica Slovaca, vol. 24 (1), 2019, pp. 1 – 14.
combustion chamber, which would involve in their [6] SB. Segota, N. Andelic, I. Lorencin, M. Saga, Z. Car, „Path
burning, lack. It should be emphasized here that it is not planning optimization of six-degree-of-freedom robotic
only the amount of secondary combustion air that is manipulators using evolutionary algorithms“. International journal
important. It is also very important that it is properly of advanced robotic systems, vol.17 (2), Article
1729881420908076, DOI 10.1177/1729881420908076, 2020.
discharged into the combustion chamber so that the
released volatile components of the fuel are mixed with [7] M. Puškár, A. Jahnátek, I. Kuric, J. Kadarová, M. Kopas, M.
Soltesova, “Complex analysis focused on influence of biodiesel
the combustion air and at the same time sufficient and its mixture on regulated and unregulated emissions of motor
combustion time is available for their combustion. The vehicles with the aim to protect air quality and environment,” Air
production of emissions in the exchanger part of the heat quality atmosphere and healt, vol. 12 (7), DOI 10.1007/s11869-
source is given by the temperature of the flue gases in the 019-00704-w, 2019, pp. 855-864.
exchanger, the temperature of the walls of the exchanger [8] T. Dodok, N. Čuboňová, I. Kuric, “Workshop programming as a
on the flue gas side and the residence time of the flue part of technological preparation of production,” Advances in
gases in the exchanger and their concentration. The most science and technology-research journal, vol. 11 (1), DOI
10.12913/22998624/66504, 2017, pp. 111-116.
important parameter that influences the conversion of
[9] I. Klačková, I. Zajačko, R. Lenhard, I. Gritsuk, D. Wiecek,
partially oxidized fuel to the final products of the “Simulation of wood biomass combustion in hot water boiler,“
combustion process is the temperature and oxygen content Machine Modelling and Simulations 2019, Liptovský Ján, In IOP
in the flue gas. There must be no locally high temperature Conference Series, Materials Science and Engineering, vol. 776,
at the flue gas flow due to the possible production of NO 24th Slovak-Polish International Scientific Conference on
in this area. The more evenly distributed the temperature Machine Modelling and Simulations - MMS 2019, Liptovský Ján,
Slovakia, 3-6 September 2019.
in the flue gas and also the oxygen content, the lower the
level of toxic substances at the outlet of the heat source [10] I. Kuric, O. Gorobchenko, O. Litikova, I. Gritsuk, V. Mateichyk,
M. Bulgakov, I. Klačková,“ Research of vehicle control
exchanger. The reason is the nonlinear dependences of the informative functioning capacity,“ Machine Modelling and
rate of production of harmful substances with respect to Simulations 2019, Liptovský Ján, In IOP Conference Series,
the O2 concentration and the local flue gas temperature. Materials Science and Engineering, vol. 776, 24th Slovak-Polish
The middle tubes of the heat source exchanger are International Scientific Conference on Machine Modelling and
minimally loaded. It is clear from the figures that it is Simulations - MMS 2019, Liptovský Ján, Slovakia, 3-6 September
2019.
necessary to design the optimization of the heat
exchanger. In general, it can be stated that the level of [11] Z. Mingaleva, I. Klačková, A. Selezneva, N. Shaidurova, N,
“Failure Mode and Effects Analysis of the Consequences of the
harmful emissions from wood burning is not always lower Life Cycle of the University Educational Services,“ ICETA 2019 -
than when burning fossil fuels. It depends on the 17th IEEE International Conference on Emerging eLearning
technological level of the equipment, the quality of the Technologies and Applications, Proceedings, 9040032, Starý
combustion line and the content of pollutants in the fuel Smokovec, 2019, pp. 531-535.
itself. For basic pollutants, it can be said that wood [12] I. Klačková, I. Kuric, I. Zajačko, K. Tucki, “Energy and
combustion can potentially lead to higher levels of CO economical aspects of implementation of virtual reality in
robotized technology systems,“ ICETA 2020 - 18th IEEE
emissions in the flue gas, on the contrary, NO emissions International Conference on Emerging eLearning Technologies
are usually lower and CO2 emissions are in principle and Applications, Proceedings, - 1. vyd. – Denver, Institute of
already considered to be zero (neutral). Electrical and Electronics Engineers, 2020. - ISBN 978-0-7381-
2366-0. - s. [1-5] [online], 12 – 13. 11.2020, Starý Smokovec,
ACKNOWLEDGMENT artical number 9379176, pp. 318-322.
[12] N. Akatov, I. Klačková, Z. Mingaleva, G. Galieva, N. Shaidurova,
This work was supported by the Slovak Research and “Expert technology for risk management in the implementation of
Development Agency under the contract No. 009ŽU- QRM in a high-tech industrial enterprise,” Management Systems
4/2022. Project title: Innovation of the structure and in Production Engineering, vol. 27 (4), 2019, pp. 250-254.
content of subjects in the field of computer support for

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 305

[13] M. Holubčík, I. Klačková, P. Ďurčanský, “Pyrolysis Conversion of machine loaded in the high-cycle fatigue region“. XXI Polish-
Polymer Wastes to Noble Fuels in Conditions of the Slovak Slovak Scientific Conference Machine Modeling and Simulations
Republic,“ Energies, vol. 13 (18), article number 4849, 2020. MMS 2016, Hucisko, Poland, 6-8 sept. 2016, Procedia
[14] J. Klarák, R. Andok, J. Hricko, I. Klačková, H.-Y. Tsai, “Design Engineering, vol. 177, pp. 516-519, DOI
of the Automated Calibration Process for an Experimental Laser 10.1016/j.proeng.2017.02.254, 2017.
Inspection Stand“. Sensors, Mdpi, 22, 5306, 2022, [22] I. Kuric, I. Klačková, K. Domnina, V. Stenchlák, M. Jr. Sága,
https://doi.org/10.3390/s22145306 “Implementation of Predictive Models in Industrial Machines with
[15] P. Kopas, M. Saga, V. Baniari, M. Vasko, M. Handrik, „A plastic Proposed Automatic Adaptation Algorithm“. Applied Sciences,
strain and stress analysis of bending and torsion fatigue specimens Mdpi, vol. 12 (4), 1853, ISSN 2076-3417,
in the low-cycle fatigue region using the finite element methods“. DOI.org/10.3390/app12041853, 2022
XXI Polish-Slovak Scientific Conference Machine Modeling and [23] L.M. Abd Ali, Q.A. Ali, I. Klačková, H.A. Issa, B.A. Yakimovich,
Simulations MMS 2016, Hucisko, Poland, 6-8 sept. 2016, V. Kuvshimov, „Developing a thermal design for steam power
Procedia Engineering, vol. 177, pp. 526-531, DOI plants by using concentrating solar power technologies for a clean
10.1016/j.proeng.2017.02.256, 2017 environment“. Acta Montanistica Slovaca, vol. 26 (4), 773-783,
[16] A. Sapietová, M. Sága, I. Kuric et al., Application of optimization 2021, DOI: https://doi.org/10.46544/AMS.v26i4.14, ISSN 1335-
algorithms for robot systems designing. International Journal of 1788.
Advanced Robotic Systems, vol. 15 (1), 2018, ISSN: 1729-8814. [24] M. Saga, M. Vasko, M. Handrik, P. Kopas, „ Contribution to
[17] M. Pástor, J. Živčák, M. Puškár, P. Lengvarský, I. Klačková, Random Vibration Numerical Simulation and Optimisation of
“Application of Advanced Measuring Methods for Identification Nonlinear Mechanical Systems,“. Scientific Journal of Silesian
of Stresses And Deformations of Automotive Structures,” Applied University of technology-series transport, vol. 103, pp. 143-154,
Science, vol. 10 (7510), 2020. DOI 10.20858/sjsutst.2019.103.11, 2019.
[18] M. Saga, L. Jakubovicova, „Simulation of vertical vehicle non- [25] M. Blatnicky, J. Dizo, M. Saga, J. Gerlici, E. Kuba, “Design of a
stationary random vibrations considering various speeds“. Mechanical Part of an Automated Platform for Oblique
Scientific Journal of Silesian University of Technology-Series Manipulation,“ Applied Sciences – Basel, vol. 10 (23), Article No.
Transport, vol.84, pp 113-118, 2014. 8467, DOI 10.3390/app10238467, 2020.
[19] I. Kuric, V. Tlach, M. Sága, M. Císar, I., Zajačko, “Industrial [26] J. Klarák, I. Klačková, R. Andok, J. Hricko, V. Bulej, H.-Y. Tsai.
Robot Positioning Performance Measured on Inclined and Parallel „Autoencoders Based on 2D Convolution Implemented for
Planes by Double Ballbar,“ Applied Sciences-Basel. 2021, vol. 11 Reconstruction Point Clouds from Line Laser Sensors.“ Sensors,
(4), DOI 10.3390/app11041777, 2021, pp. 1-17. Mdpi, 2023, vol. 23 (10), 4772, eISSN 1424-8220,
[20] J. Kostka, P. Frankovský, M. Pastor, F. Trebuňa, F. Simcak, https://doi.org/10.3390/s23104772
“Application of PhotoStress method in stress analysis of structural [27] Z. Ságová, V.V. Tarasov, I. Klačková, A.I. Korshunov, M. Sága.
elements under consideration of centrifugal force effect,“ 6th „Study of Anisotropic Friction in Gears of Mechatronic Systems.“
Conference on Modelling of Mechanical and Mechatronic Systems Applied Sciences - Basel, Mdpi, 12, 11021, ISSN 2076-3417,
(MMaMS), Vysoke Tatry, Slovakia, nov. 25-27, 2014, Modelling of DOI.org/10.3390/app122111021
Mechanical and Mechatronic Systems, Procedia Engineering, vol. [28] I. Klačková, T. Ivanová, I. Kuric, A.I. Korshunov, V. Koretckiy. „
96, 2014, pp. 235-241. Application of progressive technologies based on digitalization in
[21] M. Handrik, P. Kopas, V. Baniari, M. Vasko, M. Saga, „Analysis mechanical engineering.“ MM Science Journal 2022, vol. 2022
of stress and strain of fatigue specimens localised in the cross- (5), pp. 6127-6133, dec. 2022, ISSN 18031269, DOI
sectional area of the gauge section testing on bi-axial fatigue 10.17973/MMSJ.2022_12_2022140.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 306

Digital Twin and Modelling a 3D Human Body in
Healthcare
M. Klimo*, M. Kvassay*, N. Kvassayova*
*
University of Žilina, Žilina, Slovakia
klimo17@stud.uniza.sk, Miroslav.kvassay@fri.uniza.sk, nika.kvassayova@uniza.sk

Abstract—The digital technology has seen an overwhelming expedited drug development, and continuous monitoring
usage in various fields, where it improves every aspect of of patients' health status. These developments collectively
work. Digital Twin is a concept not that old, yet it brings a enhance patient outcomes and change the conventional
much needed advantage in many occupations. We briefly methods of delivering healthcare.
review the history of digital twin technology, its use in many
fields, advantages and disadvantages in healthcare and how II. DIGITAL TWIN IN VARIOUS INDUSTRIES
it may improve healthcare in the near future. Digital twins
have various uses in the healthcare sector, including The concept of an early Digital Twin was used by
personalized medicine, the development of medical devices, NASA’s Apollo program, where a model of a rocket
surgical planning, drug discovery, remote patient mirrored the real rocket. This has been achieved by using
monitoring, and optimization of healthcare facilities. multiple sensors and best available physical model. The
Healthcare professionals are enabled to make treatment historical data were also used for additional information
decisions that are highly individualized, improve the [1].
precision of surgical planning, and accelerate the process of The DT was preceded by multiple technologies similar
medication development. Moreover, the utilization of digital to its own. The earliest concept is the Mirror Worlds
twins facilitates the ongoing remote surveillance of patients' created in 1991 [2]. It consists of multiple sensors in the
well-being, hence enhancing patient outcomes and the real space, which transfer data to virtual space in real-
overall standard of healthcare provision. In this article, we time. The digital representation of the real object varies
overview the Digital Twin technology, how is it utilized, and and is truthful only to an extent [3].
what it presents in the future.
The Digital Twin technology was preceded by Digital
model and Digital shadow. The difference from these
I. INTRODUCTION models is the way the data flow- whether it’s manually or
automatic. In Digital model, the data flow in both ways
As society progresses towards the era of digitalization, (from physical object to digital object and vice versa) was
the influence of digital technology persists and permeates manual. Digital shadow succeeded this by making an
all domains, fundamentally transforming our professional automatic data flow from physical object to digital object.
and personal spheres. The concept of the Digital Twin The basic principle of Digital Twin technology is, that a
(DT) has emerged as a very captivating breakthrough in virtual visualization and representation of a physical
recent times. Although relatively new in our vocabulary, object is the same and the sharing of the data is
the phrase has the ability to significantly transform bidirectionally [1, 4].
multiple industries, and its integration into the healthcare DT can be categorized by the level of integration of the
sector holds the promise of revolutionary breakthroughs digital model. The most basic approach is the digital
in precision and accuracy. The utilization of Digital Twin paradigm, wherein data is manually shared. Digital
technology represents a significant advancement in the shadow provides data flow automatically, but only from
fourth industrial revolution, commonly referred to as physical object to digital. Fully automated data flow is
Industry 4.0. This revolution entails the convergence of used by the Digital Twin, as shown in Fig. 1 [2].
physical and digital domains, leading to a fundamental
transformation in human perception and interaction with
the surrounding environment. The origins of this
technology may be traced back to the early stages of
computational capabilities, where it initially found
practical use in sectors such as manufacturing, aerospace,
and engineering. This development resulted in significant Figure 1 Difference of data flow in various levels of integration of the
advancements in the areas of product design, predictive models [5].
maintenance, and operational efficiency. This article aims
to investigate the significant possibilities of Digital Twin In Digital Twin: Origin to Future, the DT is split into
several groups based on experience. These are Partial DT
technology within the healthcare sector based on previous
(limited number of data points), Clone DT (contains
studies. The advancements in healthcare offer the
relevant data used for making prototypes), and
potential for personalized treatment approaches,
previously inconceivable levels of surgical accuracy, Augmented DT (utilizes data alongside the object’s
historical data and correlates the useful data using

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 307

algorithms). Furthermore, the maturity of DT is divided replicates the complete production and testing
into four levels – Pre-digital twin, Digital Twin, Adaptive procedures. A novel approach utilising DT has been
Digital Twin, and Intelligent Digital twin [2]. introduced to address the issue of waste electrical and
The idea of the digital twin was initially proposed by electronic equipment (WEEE) recovery. This system
Dr. Michael Grieves during the early 2000s at the aims to provide support for manufacturing and
University of Michigan. The individual provided a remanufacturing activities at every stage of the product's
definition for a digital twin, characterizing it as a virtual life cycle, starting from the design phase and extending to
depiction of a tangible entity or system, which possesses the recovery phase [6].
the potential for diverse applications such as design, Engineering courses and education consider DT an
simulation, and maintenance in aerospace and Aeronautics
[1]. important part of studies as it moves education towards
digitalization. As S. Nikolaev wrote, the engineering
However, the term was coined in 2010, described as education should be based more on practical project-
ultra-realistic model that used best physical simulations,
up-to-date sensors. These models can be upgraded, as based activity, using tools as PLM, CAD or CAE,
Digital Twin technology is meant to be modular. It can be moving from digital prototype to real DT. The students
considered as one of the main pillars of industry 4.0 [6]. had to create a UAV with available resources. This
Additional model concept was introduced in 2002 as change resulted in a courses attendee number bumping up
Mirrored Spaces models and later changed to Information from 6 students previous year to 22 this year. Using DT
Mirroring Model. However, this model wasn’t usable at can also lead to increased safety of both students and
the time, as the available computers lacked efficiency [2]. equipment used during laboratory works, makes distant
The first practical use of DT was in Aerospace and on-line learning more accessible and compatible to the
Aeronautics as previously mentioned due to NASA. The physical model. DT manages to show the students various
DT increased the life duration and lowered degradation issues in many probable situations and improves the
rate for a real model. Its’ use has proven useful at future problem-solving, while promoting skill mastery, self-
mission on its behavior during launch, to recognize faults efficacy and effective knowledge construction [8, 9].
and degradations. The NASA has found its usefulness in Another point of interest is the Smart cities, where DT
the algorithms, which proved to be advantage for their coexists with the Internet of Things (IoT) to collect data
upgrades and updates of individual subsystems and improve various areas. This technology also affords
throughout its virtual life [6]. urban designers, architects, engineers, constructors,
Another area of use is Energy and power generation. property owners, and citizens the opportunity to examine
Power plants need a high reliability when having storage and evaluate the urban infrastructure in various scenarios,
of hydrogen or using blended hydrogen. For the nuclear enabling them to assess potential future risks. DT
power plants DT is used for operation, storage, waste provides a way for traffic planning and paths for
disposal and other high-maintenance actions. Energy emergency plan in natural disasters. The emergency
saving is needed at lower cost even for the industrial responders use it with the addition of VR and/or AR to
sector, or the transportation sectors. The transportation predict the behavior of people in case of a fire, spread of
area consists of applying designs and prototypes for pipe smoke or fire. Consequently, this enhances the overall
networks for oil, gas, and others. Multiple points have efficacy of a city, its infrastructure, processes, and
been pointed out by Ahmad K. Sleiti, as what are the services. Furthermore, it has been projected that by the
requirements for DT architecture: year 2023, about 75 percent of Digital Twins will be
x up-to-date physical dimensions and model, integrated with a minimum of five extreme conditions
x continuous data stream from multiple sensors, (environmental, operational, performance, security, and
x dynamic system that can run in real-time, use case) [6].
x trigger alarms and warnings in real time to
prevent risks and improve maintenance tasks,
x must be able to predict what-if scenarios [8].
The manufacturing industry holds a position of utmost
prominence among all sectors in the context of DT. An
instance that can be cited is the utilisation of tyre
manufacturing techniques aimed at enhancing the
durability and overall functionality of tyres. The
implementation of DT methodology at Maserati's
manufacturing facility has resulted in several notable
outcomes, including an enhanced production capacity, a
reduction in development time by approximately 30 %,
and a significant decrease in the production timeline from
30 months to 16 months. An additional illustration may
be seen in the case of the Virtual Plant 4.0, implemented Figure 2 Publications citing Digital Twin and Digital Control pear year
by Polisan Kansai Paint Factory, wherein the company [4]
effectively mitigated the production error rate to a level
below 1 %. The SpeedFactory facility operated by Adidas

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 308

 III. DIGITAL TWIN IN HEALTHCARE The utilization of Digital Twin extends to the United
Digital Twin technology (DT) is in its’ earliest stages States military, encompassing a cross-disciplinary
when it comes to healthcare. To get a digital model of an approach. Its primary objective is establishing a
entire body is near to impossible, however getting models connection between the model and the process of 3D
of certain organs and running simulations is already printing, so facilitating the replacement of specific body
plausible. The utilization of digital twins in the healthcare parts following injuries. The DT is produced by the use of
field allows for the comprehensive collection of X-rays, MRIs, and ultrasounds when the individuals'
physiological and genetic information pertaining to condition is in a state of good health, ensuring that the
patients. This data empowers healthcare professionals to data remains unaltered. By utilizing a pre-existing image,
develop treatment strategies with an unparalleled level of the acquired data can be subsequently transmitted to a 3D
accuracy and customization. Through the ongoing process printer or bioprinter for the purpose of replacing a
of updating and evaluating individualized patient data,
specific body part or tissue. The aforementioned study
medical practitioners can make timely modifications to
medicine, therapy, and interventions. has successfully generated human bones, arteries, and
segments of brains, hearts, and kidneys. The ultimate
The incorporation of artificial intelligence (AI) and
objective of this research is to further augment the
machine learning (ML) algorithms into digital twins has
facilitated the development of more sophisticated functionality of these organs. Pluripotent stem cells are
predictive analytics. The integration of these elements employed in order to reduce the likelihood of rejection of
facilitates the detection of patterns, deviations, and a transplanted body component by the recipient [12, 13].
possibilities for improvement in real-time, hence The availability of DT in several medical domains
augmenting decision-making processes across several remains limited. The scope of DT study is restricted and
sectors. Cardio Twin is a model of a Digital Twin is specifically focused on select topics within the fields of
technology, which has been used to help and prevent the cancer, geriatrics, cardiology, epidemic outbreaks,
Ischemic Heart Diseases (IHD) by reducing risk factors internal medicine, orthopedics, and vascular medicine.
with the help of AI-inference engine [10]. Furthermore, the utilization of this approach has been
Certain scholarly works frame the concept of DT as an contemplated in the context of observing and simulating
ethical quandary, wherein the simulations are the effects of multiple sclerosis. Nevertheless, it is
characterized as an augmentation of an individual's important to acknowledge that the implementation of
physical form, where the virtual model is neither a foreign such a solution may include intricacies and substantial
nor one’s own, where it can change life completely [11].
costs. Preliminary strategies are employed to enact initial
The article counterarguments Baudrillard’s view of
simulations as danger, where simulation will not replace prognostications regarding the advancement of a disease.
the physical person, but enhance it as a way of self- The utilization of Digital Twins (DT) in clinical trials
determination and if it is a threat depends on the context, may potentially result in a future when physical models
as it’s difficult to foresee consequences of the patterns are excluded, hence emphasizing the exclusive reliance
produced by the DT [11]. on DTs. The utilization of a limited subset of individuals
Digital twins can be applied on multiple scales, such as for data collection in the development of models may
single cells, tissues, organs, and organ systems. Predictive result in disparities in healthcare, hence yielding
models have been made for brain tumor growth at Mayo unsatisfactory treatment outcomes [14].
Clinic [4].

Figure 2 Sclerosis Multiplex transformed into DT [14]

One of the aspects that can be represented in various

areas is a remote surgery using a DT. The robot arm
replicated accurately movement of a human hand;
however the delay was inconsistent For manipulating with
virtual instruments, the hand-held controller was used.
Two models were created. In the first, the surgery was
Figure 3 Applicable biological processes for DT, on various scales [4] done through the video feed in VR headset. The second
model followed the user’s head. The remote surgery using
DT may also be unusable due to Distributed Denial of
Service attack [15].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 309

 Augmented Reality and Artificial Intelligence x Treatment optimization
supported Laparoscopic Imagery in Surgery is a project to
Digital twins can be utilized by medical
assist surgeons in performing minimal invasive liver
surgery using DT [15]. professionals to simulate the impact of various treatment
options on a virtual representation of a patient, thereby
There is an experiment of creating a model of a human aiding in the selection of the most appropriate course of
heart using tomography and magnetic resonance images,
which created a simulation of heart deformation and blood action. This strategy decreases the reliance on trial-and-
flow between chambers. Although remarked by the error methods in medical therapy and mitigates the
authors was the lack of heterogeneity of tissue, it has been occurrence of adverse effects [12, 15, 16].
validated by various researchers to use in medical field as
x Surgical planning and training
an example of DT [16].
Digital twins of patients can be utilized by surgeons to
enhance the accuracy and precision in planning intricate
surgical procedures. Virtual representations offer a
comprehensive visualization of the patient's anatomical
structures, facilitating surgeons in the identification of
potential obstacles and the formulation of effective
surgical approaches [10, 14].
x Drug discovery and development
Digital twins have the capability to replicate and model
the intricate interactions occurring between
pharmaceutical substances and patients, hence facilitating
the process of drug discovery and development. Virtual
models can be employed by researchers to create
predictions regarding the effectiveness of drugs, their
safety profiles, as well as potential adverse effects, all of
which are contingent upon the distinctive genetic
composition of individual patients [17].
x Remote patient monitoring
The patient can be monitored remotely with a DT, as the
model can be accessible anywhere [15, 16].
x Cost efficiency
Figure 3 Various simulations showing heart's behavior [16]
The implementation of digital twin technology in
healthcare institutions facilitates the anticipation of
The drug development in the U.S. is supported from equipment malfunctions, the enhancement of resource
both the privately-owned companies and with the allocation, and the mitigation of operating expenses,
government backed organizations. Drug development, hence leading to substantial long-term financial benefits
without causing any problems for the patient, can be [16].
tested and simulated on DT. The example is a study
showing the affects of various drugs on virtual liver, such x Improved patient outcomes
as use of chlorpromazine. The biggest challenge with drug The integration of individualized treatment protocols,
related situations is the idiosyncratic toxicity in humans, careful surgical interventions, and ongoing surveillance
as it is near to impossible to predict [17]. collectively enhances patient outcomes and the overall
standard of healthcare [10, 14, 16].
IV. ADVANTAGES
Digital twin technology presents a multitude of notable V. DISADVANTAGES
benefits within the healthcare sector, so transforming the However, the DT also brings some disadvantages that
provision of patient care, medical research endeavors, and may concern the future development. These
operational efficacy. Based on various sources, these disadvantages may be seen minor with the comparison to
aspects can be represented as is the following: advantages. Some disadvantages of DT are as following:
x Personalized medicine x Current cost of design and development
Digital twins facilitate the development of precise The implementation of digital twin technology
and personalized models that integrate an individual's might provide challenges in terms of complexity and cost,
medical history, genetic information, and up-to-date particularly when dealing with extensive and elaborate
health data. Healthcare providers has the ability to systems, where a huge amount of data is needed. The
customize treatment plans with unparalleled accuracy, establishment of requisite sensors, data infrastructure, and
resulting in enhanced efficacy and individualized care modeling methods can entail a substantial financial
[10]. commitment [14].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 310

x Security VII. CONCLUSIONS
The utilization of digital twins necessitates the The integration of digital twin technology in the
handling of enormous quantities of data, hence giving rise healthcare and medicine sector is having a profound
to apprehensions over the security and privacy of this impact on the industry by adopting a holistic and data -
data. The maintenance of confidentiality and integrity for driven approach to patient care, research initiatives, and
sensitive information is of utmost importance, since any facility management. The continuous advancement of
breaches may result in significant ramifications [15]. technology presents an opportunity for the facilitation of
precision medicine and improvement in healthcare
x Data accuracy delivery, resulting in advantages for patients and
healthcare professionals alike. DT signifies a significant
The degree of precision exhibited by a digital twin advancement in healthcare due to its ability to personalize
model is contingent upon the caliber of the data it is care and enhance cost-effectiveness. Given the increasing
provided with. The utilization of inaccurate or unreliable influence of wearable electronics in our daily lives, it is
data has the potential to result in erroneous simulations imperative that we harness their potential. Additional
and analyses, which in turn can introduce errors into the study is needed in the healthcare sector to fully explore the
decision-making process [10, 11, 16]. potential of Digital Twin technology in healthcare and
simulations regarding the human body.
x Connectivity
The successful operation of digital twins is contingent ACKNOWLEDGMENT
upon the presence of a resilient internet connection and a This work was supported by the Ministry of
well-developed network infrastructure. Disruptions in Education, Science, Research and Sport of the Slovak
real-time data synchronization between the physical Republic under the grant VEGA 1/0858/21.
system and its digital twin may occur in the event of
connectivity challenges or network breakdowns [15]. REFERENCES
[1] P. Barbiero, R. V. Torné, and P. Lió, “Graph Representation
x Racial bias Forecasting of Patient’s Medical Conditions: Toward a Digital
Digital twins have the capability to observe and Twin,” PubMed Central (PMC), Sep. 16, 2021.
evaluate human behavior across several circumstances. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8481902/.
The aforementioned issue gives rise to ethical [2] M. Singh, E. Fuenmayor, E. P. Hinchy, Y. Qiao, N. Murray, and
D. Devine, “Digital Twin: Origin to Future,” MDPI, May 24,
considerations pertaining to monitoring, the requirement 2021. https://www.mdpi.com/2571-5577/4/2/36.
of consent, and the protection of individual privacy, while [3] A. Gautam, D. Williams, K. Terry, K. Robinson, and P.
it may be more prejudiced to have higher accuracy with a Newbill, “Mirror Worlds: Examining the Affordances of a Next
select group of people. There exists a potential for uneven Generation Immersive Learning Environment - TechTrends,”
SpringerLink, Nov. 15, 2017.
distribution of advantages associated with digital twins, https://link.springer.com/article/10.1007/s11528-017-0233-x
which could result in discrepancies in healthcare [4] Eftimie, Raluca & Mavrodin, Andreea & Bordas, Stéphane.
outcomes. [11]. (2022). From Digital Control to Digital Twins in Medicine: A
brief review and future perspectives. Advances in Applied
x Maintenance Mechanics. 56. 1-46. 10.1016/bs.aams.2022.09.001.
In order to ensure their relevance and accuracy, digital [5] Kritzinger, Werner & Karner, Matthias & Traar, Georg &
Henjes, Jan & Sihn, Wilfried. (2018). Digital Twin in
twins necessitate ongoing maintenance and updates. The manufacturing: A categorical literature review and
process of updating the digital twin to accurately mirror classification. IFAC-PapersOnLine. 51. 1016-1022.
the evolving physical system necessitates a significant 10.1016/j.ifacol.2018.08.474.
investment of time and resources, as older models can [6] M. Singh et al., “Applica tions of Digital Twin across Industries:
A Review,” MDPI, Jun. 04, 2022. https://www.mdpi.com/2076-
become obsolete [10, 12, 15]. 3417/12/11/5727.
[7] M. Singh, E. Fuenmayor, E. Hinchy, Y. Qiao, N. Murray, and
VI. FUTURE OF DIGITAL TWIN IN HEALTHCARE D. Devine, “Digital Twin: Origin to Future,” Applied System
The Medical Digital Twin is still in its infancy in the Innovation, vol. 4, no. 2, p. 36, May 2021, doi:
10.3390/asi4020036.
present day, with only limited use in specific areas, as
[8] Sleiti, Ahmad & Kapat, Jayanta & Vesely, Ladislav. (2022).
individual hospitals carry mostly prototypes of various Digital twin in energy industry: Proposed robust digital twin for
models. These models are set to fully automate power plant and other complex capital-intensive large
healthcare, seeing a spike of monitoring the patients for engineering systems. Energy Reports. 8. 3704-3726.
potential outcomes using deep learning models [19]. 10.1016/j.egyr.2022.02.305.
These models can help with management of hospitals, as [9] S. Nikolaev, M. Gusev, D. Padalitsa, E. Mozhenkov, S. Mishin,
and I. Uzhinsky, “Implementation of ‘Digital Twin’ Concept for
less resources would be needed to optimize processes of Modern Project-Based Engineering Education,” Implementation
needy activities. Multisource data will be used to create a of “Digital Twin” Concept for Modern Project-Based
single DT either from various scanning devices or Engineering Education | SpringerLink, Dec. 08, 2018.
https://link.springer.com/chapter/10.1007/978-3-030-01614-
wearable electronics. As with COVID-19, the future 2_18.
vaccine developments are a major need, where the long- [10] R. Martinez-Velazquez, R. Gamez and A. El Saddik, "Cardio
term effects of the vaccines can be simulated, which Twin: A Digital Twin of the human heart running on the edge,"
couldn’t be done before[19]. 2019 IEEE International Symposium on Medical Measurements
and Applications (MeMeA), Istanbul, Turkey, 2019, pp. 1-6, doi:
10.1109/MeMeA.2019.8802162.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 311

 [11] “Represent me: please! Towards an ethics of digital twins in Application of Remote Surgery," in IEEE Access, vol. 7, pp.
medicine - PubMed,” PubMed, Mar. 15, 2021. 20325-20336, 2019, doi: 10.1109/ACCESS.2019.2897018.
https://pubmed.ncbi.nlm.nih.gov/33722986/ [16] B. Baillargeon, N. Rebelo, D. D. Fox, R. L. Taylor, and E. Kuhl,
[12] M. Griffin, “Soldiers digital twins let US Army 3D print “The Living Heart Project: A robust and integrative simulator
replacement body parts in battle,” By Futurist and Virtual for human heart function,” PubMed Central (PMC).
Keynote Speaker Matthew Griffin, Jan. 05, 2017. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4175454/
https://www.fanaticalfuturist.com/2017/01/digital-clones-will- [17] K. Subramanian, “Digital Twin for Drug Discovery and
let-us-army-3d-print-new-body-parts-in-battle-to-treat-injured- Development—The Virtual Liver - Journal of the Indian
soldiers/ Institute of Science,” SpringerLink, Oct. 01, 2020.
[13] T. Erol, A. F. Mendi and D. Doğa n, "Digital Transformation https://link.springer.com/article/10.1007/s41745-020-00185-2.
Revolution with Digital Twin Technology," 2020 4th [18] De Benedictis, Alessandra & Mazzocca, Nicola & Somma,
International Symposium on Multidisciplinary Studies and Alessandra & Strigaro, Carmine. (2022). Digital Twins in
Innovative Technologies (ISMSIT), Istanbul, Turkey, 2020, pp. Healthcare: An Architectural Proposal and Its Application in a
1-7, doi: 10.1109/ISMSIT50672.2020.9254288. Social Distancing Case Study. IEEE Journal of Biomedical and
[14] I. Voigt, H. Inojosa, A. Dillenseger, R. Haase, K. Akgün, and T. Health Informatics. PP. 1-12. 10.1109/JBHI.2022.3205506.
Ziemssen, “Digital Twins for Multiple Sclerosis,” PubMed [19] T. Erol, A. F. Mendi and D. Doğan, "The Digital Twin
Central (PMC), May 03, 2021. Revolution in Healthcare," 2020 4th International Symposium
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8128142/ on Multidisciplinary Studies and Innovative Technologies
[15] H. Laaki, Y. Miche and K. Tammi, "Prototyping a Digital Twin (ISMSIT), Istanbul, Turkey, 2020, pp. 1-7, doi:
for Real Time Remote Control Over Mobile Networks: 10.1109/ISMSIT50672.2020.9255249.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 312

 Game Engine Based Application for
Neurorehabilitation in Collaborative Virtual Reality
Štefan Korečko, Peter Nehila, Branislav Sobota
Department of Computers and Informatics
Faculty of Electrical Engineering and Informatics
Technical university of Košice
Košice, Slovak Republic
stefan.korecko@tuke.sk, branislav.sobota@tuke.sk

Abstract—Virtual reality is used for a wide range of applica- In this paper, we present a system, utilizing the Unity1
tions nowadays. It can also be used for medical purposes, for game engine, for neurorehabilitation of patients that suffer
example in the process of therapy and rehabilitation of patients. from loss of movement in their arms. The system includes
The advantage of applying virtual reality to the rehabilitation
process is the possibility of creating interesting and immersive a collaborative virtual environment, which allows multiple
virtual environments and scenarios. Based on an analysis of the users to interact with each other in real time. The system
technology’s potential uses and current solution, we have created is able to communicate with an external device using the
an application in Unity game engine that will allow patients electroencephalography (EEG) to monitor the brain activity of
and therapists to collaborate in a shared virtual environment. the patient and detect whether they are attempting to imagine
The application includes several simple scenarios that can be
expanded and combined into more complex, engaging sequences the movement required. Subsequently, an animation of the cor-
in the future, as well as interface to communicate with external responding movement with the incapacitated arm is visualized
environments to assess the patient’s actions via electroencephalog- in the virtual environment. The system is a development of a
raphy. pilot version [4], [5], based on web technologies. While the
Index Terms—Collaborative virtual environment, Neuroreha- web-based version provided easy deployment and accessibility
bilitation, virtual reality, Unity game engine, EEG
without the need of installation of any specific application, the
Unity engine offers better performance and tools and assets for
I. I NTRODUCTION
easy creation of feature-rich virtual environments.
In the field of therapy, new methods that could simplify
the work for both therapists and patients are constantly being II. V IRTUAL REALITY AND ITS USE IN THERAPY
introduced and tested. One of the main goals of such therapy
is to improve the results and make treatment of patients more A. Virtual Reality
effective. Recently, virtual reality (VR) has started to advance According to [6], a virtual reality (VR) system is a system
rapidly and what we couldn’t imagine just a few years ago, representing a simulated world, with which the user interacts
we now have right at our fingertips. Advances have also through data gloves or controllers to view the virtual envi-
been made in technology used in virtual reality headsets. We ronment. Traditional input devices, such as a keyboard or a
have moved from relatively slow and cumbersome devices mouse, can be used, too. The image can be displayed on
to devices that can operate independently without the need various devices, including ordinary and stereoscopic displays,
to be connected to a computer, with sufficient image quality and VR headsets. Virtual reality seeks to model the world
and high precision motion sensors. Virtual reality is not yet as faithfully as possible and to provide the best possible
something that people encounter on a daily basis, but this trend interaction between humans and the simulated world. There
may gradually change in the future. are also three related terms, the first one being mixed reality
Nowadays, there are already several projects that focus on (MR), offering a fusion of the real world with the virtual
therapy and rehabilitation combined with the use of virtual world, at the level of displaying computer-generated objects
reality. The nature of these systems varies depending on into the real world. Such systems also include the ability to
what they specifically target. There are systems specialised interact with such objects at the same time [7]. Augmented
primarily for the treatment of various phobias [1], [2] or reality (AR) represents a connection between the real and
addressing consequences of unpleasant experiences, such as the virtual world, too. Contrary to MR, AR seeks only to
the post-traumatic stress disorder [3]. Most of these systems complement the real world with additional information and
are more like 3D virtual environments where the patients other computer-generated enhancements [8]. We should also
themselves are not able to intervene much. There are also not forget about the extended reality (XR), an umbrella term
systems for which various specialised gloves or other assistive covering VR, MR and AR. Virtual environments for all these
devices have been created, the purpose of which is to improve
the results of therapy. 1 https://unity.com/

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 313

settings are characterized by, among other things, real-time This sensation can be caused by several factors. In its guide
processing. [13] on VR applications development, Oculus lists several
important ones that affect the comfort level when using a VR
B. Collaborative Virtual Environment headset, namely:
By the term Collaborative Virtual Environment (CVE) we • acceleration when moving - the perception of accelera-
mean a system that can be used by multiple users at the tion without actual acceleration has a negative effect on
same time, so they can share its capabilities. Thus, they can humans,
collaborate and interact in a virtual environment. The most • head movement - for example, a slight up and down
important part of CVE is information sharing. Collaborative movement when walking is undesirable when using a VR
virtual environments can also be defined as distributed virtual headset,
reality systems that offer a digital world designed for informa- • sideways movement - in real life, people rarely move
tion sharing and interaction between users using the resources sideways or backwards, these movements often cause
offered by the system [9]. motion sickness,
C. Rehabilitation Using VR • image latency - the delay between the actual movement
and corresponding change in the visual output,
Within the field of rehabilitation in VR we have to consider
• duration of using VR headset - prolonged use of a VR
several issues, the main one being the quality of rehabilitation
headset without a break may have a negative effect,
and its real outcomes. According to [10], improvements in
• user interface - a cluttered interface or interface with
outcomes can be achieved by improving several aspects. These
frequently changing elements may be confusing,
include:
• avatar - the presence of an avatar improves the user’s
• naturalness, feelings. It is also necessary to map the movement of
• engagement, sensors to the movement of limbs, such as arms and legs,
• sense of presence and • involuntary movement - for example, moving the user
• affection for the VR application itself. after a bump can cause a feeling of nausea.
Another important aspect is the feeling of immersion, which
brings with it other positive feelings of working in a virtual III. S YSTEM D ESIGN AND F EATURES
environment. According to Witmer and Singer [11], feelings The system we designed focuses on allowing users to
of immersion and engagement in the virtual environment are collaborate in a shared virtual environment. It recognizes two
important, as both of these feelings are necessary to induce types of users - therapists and patients. Therapists control
presence in the virtual environment. They named the presence a significant part of the therapy, they can create training
of stimuli for various user activities from within the virtual scenarios consisting of multiple steps, change timings for
environment (e.g., in the form of a representation of the certain parts of movements or even control positions of patient
patient’s hand or haptic response), the comfort of wearing the in virtual environment. Patient’s role is to simply follow
VR headset, the maintenance of the user’s attention, and the therapists instructions. The system allows for communication
isolation from the external environment as the main features with external systems through it’s API.
enhancing these factors. The system has been developed in the Unity game engine,
Another way to improve satisfaction and increase the pa- contrary to the A-Frame2 web virtual reality framework used
tient’s interest in the rehabilitation process is game scenarios in the previous, pilot, version. We chose to use Unity as it
utilization [12]. Such scenarios can turn a boring and repetitive provides more options for future development, better support
exercise into a meaningful and entertaining activity, where the for building complex and sophisticated virtual environments
correct execution of the corresponding exercise steps leads and, naturally, more options for gamification. A-Frame is more
to a progress in the “game”. Another advantage of VR is suitable for fast prototyping and smaller scale projects. The
accessibility. A number of rehabilitation exercises are carried system is collaborative, so it has been developed as a net-
out in specialized facilities under the supervision of doctors worked application in Unity. Utilizing the standard procedure,
[12]. In the future, rehabilitation carried out in VR could be the server and client have been developed at the same time,
easily accessible over long distances, even for the patients not as one application. The only difference is that the server is
able to visit the centers personally. compiled under different settings (the application executes
different code, depending on build type).
D. Simulation Sickness
We used the OpenXR standard to access functionality spe-
A significant problem associated with virtual reality is the cific for VR headsets. This allows us to support a wide range of
feeling of nausea when using a VR headset. A particular type devices without the need for adjustment at the code level. On
of motion sickness, called simulation sickness, is referred to the other hand, by using OpenXR we are losing some specific
in connection with VR. This type of motion sickness is not features of certain VR headset types. The system also allows to
caused by real movement (compared to motion sickness when use simulated VR on desktop, although using it in this mode is
travelling, for example), but by a visual sensation depicting
movement while no physical movement is carried out [13]. 2 https://aframe.io/

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 314

not recommended. The simulated VR has problematic controls
as well as low optimization, because in this mode the device
has to render everything twice, as if it was rendering image on
VR headset. Because of this, client applications of our system
have been developed with two versions, one for desktop and
one for VR headsets, primarily Oculus (Meta) Quest.
A. Avatars and Environment
Traditionally in VR applications, the user does not have
full-body avatar. The reason being that it is quite demanding
to have an avatar object in scene and have it animated. Since
we are creating a collaborative virtual environment, we can’t
ignore this functionality [14].

Fig. 2: The virtual environment of the system as seen from the

therapist’s point of view. The patient is seated behind the table
and the therapy is set for the right arm. A part of the menu
to set up the therapy is seen at the wall behind and the green
arc defines the range of movement for the current position of
the patient.

C. Rehabilitation Process
Rehabilitation in our system consists of multiple shorter
moves, that are defined at the beginning. After the therapist
has finished setting up all parameters and positions for moves,
Fig. 1: One of the avatar models, used in the system. he or she can start the training. Every move has a waiting
period, during which the patient is supposed to imagine the
The avatars for our system (Fig. 1) have been created using movement in his head. Using EEG to measure the brain
the readyplayer.me3 platform, with animations originating activity and subsequent analysis, described in [5], we can
from the mixamo4 platform. It was essential to pay attention determine whether the patient imagined the correct movement.
to arm and head animations as well, as they need to follow However, it should be noted that the determination is not
movement of corresponding sensors, namely the headset itself exact and it is beyond the capabilities of EEG to recognize
and its controllers. what exact movement with the corresponding arm the patient
The virtual environment is also important when it comes to imagined. The application measuring and analyzing the EEG
VR applications. A pleasant one can help patients relax and signal is external to the system and communicates with it
motivate them during the rehabilitation [10]. The environment through an interface provided by the OpenViBE5 software for
we created (Fig. 2) was designed to contain different shelves brain computer interfaces.
and other furniture, filling the room. For the world outside of
the room we have used a skybox that helps to create a cozy D. Configurable Training
atmosphere, an important aspect for the rehabilitation. The training sessions of the rehabilitation can be configured
B. Network Structure in many ways. The therapist can define parameters such as
For our system, we have decided to use the client-server the duration of movements, number of movements and key
architecture. With such an architecture, we can easily control positions for animations. Positions can be set by moving the
what messages are sent, when and to whom. For example, object in 3D space and defining intermediate steps. Therapists
the server could simply control which requests are processed can also choose from multiple scenarios. In Fig. 3, one can see
and which are ignored based on roles. Another advantage of different types of animations implemented. These scenarios
such architecture is delegation of functionality - server can differ in objects to be moved, movement complexity, as well
do certain things, while ignoring other, such as calculating as the way the hand is moving during animation. All scenarios
animations or animating avatars. In a client-server architecture, were created to help to stimulate patients.
the clients are hidden from outside, and only server has access The settings are available from a menu that is placed on
to them. This unfortunately creates a single point of contact, the wall (Fig. 2) or can be activated as a pop-up menu. In
which could be prone to overloading. the VR headset version of the client application, it is also
possible to invoke the menu as fixed to the position of one of
3 https://www.readyplayer.me/
4 https://www.mixamo.com/\#/ 5 http://openvibe.inria.fr/

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 315

 and create more feature-rich scenarios, it’s important to find a
good trade off between the level of detail that we aim for and
the amount of processing power required to create a satisfying
and believable experience.
IV. P ERFORMANCE E VALUATION
To find out how the system performs on various devices, we
(a) Block animation (b) Cube animation run several tests focusing on network usage and performance
in terms of rendering speed in frames per second (FPS).

Device CPU GPU RAM Refresh

rate

Meta Qualcomm Qualcomm 6GB 72Hz

Quest 2 Snap- Snap-
dragon dragon
XR2 XR2
(c) Cup animation (d) Key animation Desktop Ryzen 5 Nvidia 16GB 59.9Hz
3600 GTX1660
Fig. 3: Different types of animations, implemented in the (6GB
system VRAM)

Laptop Intel Core GTX 1050 8GB 60Hz

i5-8250 (4GB
the controllers, so the user can move the menu by moving his VRAM)
or her hand.
Laptop Intel Core Intel UHD 8GB 60Hz
E. Animations (power i5-8250 620
saving) (128MB
Because we allow therapists to change positions for training, VRAM)
it would be impossible to use traditional animations to move
the arm and other objects. These animations are hard to create TABLE I: Hardware specifications of devices used in perfor-
and are usually hard to adapt. In order to provide configurable mance evaluation of the system. The refresh rate is of the
animations, we use dynamic animations with inverse kine- display(s) of the device.
matics. Inverse kinematics allows us to dynamically calculate
needed rotations of bones in arm in order to get to the desired The tests were run on several devices, which hardware
position. Inverse kinematics can be power hungry, when it specifications are listed in Table I. Our main focus was on
comes to a lot of calculations. But because we don’t expect testing the system using the VR headset Meta Quest 2, but
a large number of users in one environment at once, they can we’ve also included a desktop computer and laptop under
provide immersive and believable animations, even on lower different conditions. Our main goal was to test performance
performance hardware. The only problem is the fact that they and ease of use on other non VR devices. These tests also
can sometimes produce moves, that are unnatural (i.e. moves gave us insight on how patients using VR headset can interact
that are not doable in real life because of the limitations of with therapists, who are using desktop client.
arm joints). In order to solve this issue we have to set up
different limitations and boundaries, when the moves are being A. Network Usage
calculated. When it comes to collaborative systems in VR, it’s im-
portant not to forget that in order to achieve satisfactory
F. Hardware Constraints levels of fluidity, we cannot ignore the network usage and
The system is primarily developed for VR headset Meta its influence on the system performance. One should use a
(Oculus) Quest 2. It also supports other standard VR headsets, suitable network transport protocol and limit the amount of
that are traditionally connected to computer via a cable. We data sent over the network. This can, for example, be achieved
chose Meta Quest 2 as our primary device because of it’s good by making more calculations local (i.e. on clients), rather
availability, favorable price-to-performance ratio and built-in than calculating everything on a server and sharing only the
computing device. This means that it can operate without being results with the clients. This client-side approach causes some
connected to a computer. We have also created a desktop unwanted results, such as the need for a higher computational
client, to be used by therapists. Because we use the client- power of the clients. As Fig. 4 shows, the maximum bandwidth
server architecture, we also need a server to be running for needed for the data transfer from a client to the server, i.e. the
the system to operate. In order to achieve a smooth run we data upload, was around 10 kilobytes per second in our system.
had to pay attention to optimization, especially when using a An important thing to note is that during a normal use, an
VR headset. Because we plan to further develop the system average amount of uploaded data is lower, because during the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 316

 of the simulation or cyber sickness and overall dissatisfaction.
This may be important when dealing with patients that can
be even more sensitive to such aspects. We have tested our
system on multiple different hardware configurations, which
allows us to get a good idea of how the system would run
under differing conditions. When designing and implementing
the system, we have focused on optimizing the system so that it
can be extended in the future for more immersive experience.
Fig. 4: Network usage: data transfer of uploaded data in By creating this collaborative system, we hope to make the
kilobytes per second (kB/s). neurorehabilitation easier, more enjoyable and that it will bring
results, which can improve the lives of many people.

test we have attempted to achieve the highest possible value ACKNOWLEDGEMENT

by artificially increasing user activity. This work has been supported by the APVV grant
no. APVV-21-0105 “Trustworthy human–robot and thera-
B. Rendering Performance and Overall Functionality pist–patient interaction in virtual reality.”
During these tests, our aim was to evaluate the complete R EFERENCES
functionality of the system. The setup used emulated a typical
[1] J. R. S. Freitas, V. H. S. Velosa, L. T. N. Abreu, R. L. Jardim, J. A. V.
scenario of one patient rehabilitation. This means that it Santos, B. Peres, and P. F. Campos, “Virtual reality exposure treatment
involved the server, one client in the patient role and another in phobias: a systematic review,” Psychiatric Quarterly, vol. 92, no. 4,
client in the therapist role. At the same time, we have tested pp. 1685–1710, 2021.
[2] C. Botella, J. Fernández-Álvarez, V. Guillén, A. Garcı́a-Palacios, and
limits of the system, such as executing high numbers of R. Baños, “Recent progress in virtual reality exposure therapy for
requests and forcing a high amount of communication between phobias: a systematic review,” Current psychiatry reports, vol. 19, pp.
clients and server. As a primary measurement to evaluate the 1–13, 2017.
[3] A. Rizzo, A. Hartholt, M. Grimani, A. Leeds, and M. Liewer, “Virtual
performance of the system we used rendering performance reality exposure therapy for combat-related posttraumatic stress disor-
in FPS. The results of the tests on all devices can be seen in der,” Computer, vol. 47, no. 7, pp. 31–37, 2014.
Table II. Overall the tests had shown good results. Most of the [4] B. Sobota, S. Korečko, J. Gvuščová, and M. Mattová, “Therapist-patient
interaction in virtual reality at the level of the upper limbs,” in 2022
time the performance was stable and we did not experience 20th International Conference on Emerging eLearning Technologies and
significant fluctuations. The only time we saw lower than Applications (ICETA), 2022, pp. 584–588.
average frame-rates was when using the software on a laptop [5] R. Rosipal, S. Korečko, Z. Rošťakova, N. Porubcová, M. Vankó, and
B. Sobota, “Towards an ecologically valid symbiosis of bci and head-
without a dedicated GPU. mounted vr displays,” in 2022 IEEE 16th International Scientific Con-
ference on Informatics (Informatics), 2022, pp. 251–256.
Device Min. FPS Max. FPS Average FPS [6] J. Steuer, “Defining virtual reality: Dimensions determining
telepresence,” Journal of Communication, vol. 42, no. 4, pp.
73–93, 1992. [Online]. Available: https://onlinelibrary.wiley.com/doi/
Meta Quest 2 35 71 70,7764
abs/10.1111/j.1460-2466.1992.tb00812.x
[7] B. Sobota, S. Korečko, M. Hudák, and M. Sivý, “Mixed reality: A
Desktop 59 60 59,7202
known unknown,” in Mixed Reality and Three-Dimensional Computer
Graphics, B. Sobota and D. Cvetković, Eds. Rijeka: IntechOpen, 2020,
Laptop 0 60 59,6428 ch. 10. [Online]. Available: https://doi.org/10.5772/intechopen.92827
[8] M. Hudák and B. Sobota, Collaborative Vitual Reality And System
Laptop 10 60 43,5789 Interfaces, 2021, in Slovak.
(power [9] E. F. Churchill, D. N. Snowdon, and A. J. Munro, Collaborative virtual
saving) environments: digital places and spaces for interaction. Springer
Science & Business Media, 2012.
[10] S. Bialkova and B. Dickhoff, “Encouraging rehabilitation trials: The po-
TABLE II: Rendering performance, in FPS (frames per sec- tential of 360° immersive instruction videos,” in 2019 IEEE Conference
ond), of the client applications of the system on various on Virtual Reality and 3D User Interfaces (VR), 2019, pp. 1443–1447.
devices. [11] B. G. Witmer and M. J. Singer, “Measuring Presence in Virtual
Environments: A Presence Questionnaire,” Presence: Teleoperators and
Virtual Environments, vol. 7, no. 3, pp. 225–240, 06 1998. [Online].
Available: https://doi.org/10.1162/105474698565686
V. C ONCLUSION [12] B. Lange, S. Flynn, and A. Rizzo, “Game-based telerehabilitation,”
European Journal of Physical and Rehabilitation Medicine, vol. 45,
Our main objective was to create a collaborative system, no. 1, pp. 143–151, 2009.
offering virtual reality for rehabilitation of patients. The [13] I. Oculus VR, “Oculus vr best practices guide,” Jan.
2014. [Online]. Available: https://s3.amazonaws.com/arena-attachments/
system was built using the Unity game engine, in order 238441/2330603062c2e502c5c2ca40443c2fa4.pdf
to allow for easier expansion of its functionality, as well [14] S. Benford, J. Bowers, L. E. Fahlén, C. Greenhalgh, and
as the possibility of gamification in the future. The current D. Snowdon, “User embodiment in collaborative virtual environments,”
in Proceedings of the SIGCHI Conference on Human Factors in
implementation supports a wide range of commonly used VR Computing Systems, ser. CHI ’95. USA: ACM Press/Addison-
headsets. It also offers an option to use the system as a desktop Wesley Publishing Co., 1995, pp. 242–249. [Online]. Available:
application. One of the problems we plan to explore is that https://doi.org/10.1145/223904.223935

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 317

 Exploring GitOps: An Approach to Cloud Cluster
System Deployment
Tomáš Kormaník Jaroslav Porubän
Department of Computers and Informatics Department of Computers and Informatics
Technical University of Košice Technical University of Košice
Košice, Slovakia Košice, Slovakia
tomas.kormanik@tuke.sk jaroslav.poruban@tuke.sk

Abstract—This article explores the synergy between GitOps may be more lenient but risks dependency on specific
and Kubernetes for efficient system deployment within cloud individuals or groups.
clusters. GitOps, a contemporary DevOps methodology combined • Tool Dependency: GitOps increasingly relies on a grow-
with the robust orchestration capabilities of Kubernetes, offers
a practical approach to automating and managing deployment ing array of tools, procedures, and techniques, potentially
processes. We analyze this approach’s advantages, disadvantages, increasing tool dependence. DevOps, conversely, does not
limitations, and critical aspects, intending to increase awareness prescribe specific tools but may place a heavier burden
among professionals and developers. Our motivation arises from on knowledgeable personnel.
the growing need for adaptability in a rapidly evolving cloud- • Problem Perception: GitOps embraces an organized,
native landscape, where GitOps and Kubernetes provide a path
to enhanced deployment practices. These practices are further object-oriented approach to problem-solving, breaking
used to effectively create testing pipelines for student assignments, complex systems into manageable components. DevOps,
which consist of multiple projects on the GitLab platform. on the other hand, often takes a holistic view, simplifying
Index Terms—GitOps, DevOps, Kubernetes, Microservices, complexity at the cost of additional steps.
Deployment Automation, Cluster Management • Adaptability: GitOps excels in accommodating developer-
driven modifications due to its modular problem per-
I. I NTRODUCTION
ception [4]. DevOps, while robust, may require more
In the rapidly evolving landscape of application and system expertise and effort for post-implementation changes.
deployment, traditional DevOps practices are giving way to GitOps represents a modernized approach to system devel-
the GitOps model. This shift represents a significant trans- opment and deployment, replete with unique characteristics.
formation in how we manage deployment processes, with It thrives in smaller teams and is well-suited for less complex
GitOps leveraging the power of version control and automation systems. To provide a visual representation of this comparison,
to streamline operations. At the heart of this transition is refer to the attached diagram (Fig. 1).
Kubernetes, a widely adopted orchestration platform that plays
a pivotal role in driving the adoption of GitOps.
This article endeavors to shed light on the key distinctions
between the DevOps and GitOps models, examining their
respective advantages and drawbacks. Moreover, it provides
insights into the Kubernetes ecosystem, a central catalyst in
this paradigm shift.
While GitOps has garnered attention and recognition, it is
often misunderstood or narrowly depicted. GitOps is more than
just a fusion of DevOps and Git; it represents a fundamentally
distinct approach to problem analysis, solution design, and de-
ployment practices. To comprehensively grasp this approach,
we delve into a comparative analysis of GitOps and DevOps,
drawing from insights found in the works of various authors
[1] [2] [3]. Over time, as technology and toolsets have evolved,
the once-prominent differences between these models have Fig. 1. Comparison of GitOps and DevOps processes
blurred.
Our examination focuses on key attributes: II. C RITICAL B ENEFITS OF G IT O PS
• Flexibility: GitOps demands adherence to established At first glance, the difference between GitOps and DevOps
rules and standards for code clarity and analyzability, may need to be more apparent and straightforward to unequiv-
promoting team-wide compliance. In contrast, DevOps ocally prove its advantages for a specific company or team

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 318

of developers. We would like to highlight the key benefits TABLE I
that we have observed and considered critical in our validated R ELEVANT PROJECT COMPARISON AT D EPARTMENT OF C OMPUTERS AND
I NFORMATICS
environment.
a) More frequent deployment of versions: This is the Comparative rank DevOps GitOps
main reason to use GitOps. Each new application version Lines of Code (μ/f ile) 97.2 45.8
Lines of Code (μ) 806.8 668,7
can be immediately deployed using a configured process. Files (Σ(f iles)) 8.3 14.6
Creating multiple versions of different deployments for dif- Directory tree depth (μ) 1.7 3.1
ferent environments, clients, and requirements is possible.
Individual projects and applications may or may not share
configurations, and if necessary, it is possible to replace only
needs and established standards, so their implementation may
parts of the configuration and adapt the application according
be vastly different, and may not even share similarities with
to requirements almost immediately.
our description.
b) Easier management of approaches to the project:
To provide a comprehensive perspective on our research
Thanks to GitOps, only minimal contact with real machines
area, we conducted a comparative analysis by examining a
is required. Minimizing direct connections to physical or
sample of projects within our department. Our goal was to
virtual machines reduces the burden on developers and project
contrast projects following traditional DevOps practices with
managers. Access can be managed thanks to roles in the Git
those embracing GitOps principles. We gathered data from
tool, and individual privileges can be set. This minimizes the
these projects and selected the most relevant ones for our
sharing of passwords and the sending of different one-time
comparison. The resulting data underwent careful analysis.
passwords between project members.
The comparative results, presented in Table I, offer a clear
c) Immediate availability of backup plan: The remedy is
overview of the distinctions between these two approaches:
straightforward in case of a service outage or a development
error. It is sufficient to redeploy the previous functional 1) Overall Lines of Code: GitOps projects showed a re-
version or to deploy it on another server/machine managed duction in overall lines of code compared to DevOps
by Git. Solving such a problem is possible even for ordi- projects.
nary developers with a few clicks and does not require an 2) Lines of Code per File: GitOps projects displayed a
advanced engineer or administrator. In addition, support for more concise and structured codebase, with fewer lines
load balancing, automatic migration, and restarting individual of code per file.
deployments is a key part of the system that helps developers 3) File Structure Differences: The projects also showed
even if they need to deploy the project themselves. disparities in their file structures, with GitOps projects
d) Shared documentation: Git also knows tools to create often characterized by enhanced readability and a more
clear documentation from individual messages in commits. It logical arrangement of files.
is possible to set a regular expression to save only commits As an example of the benefits of GitOps, we present a basic
containing a certain note or phrase in the documentation. application that utilizes the "NGINX" Ingress controller to
Although this creates a burden on developers in the form of the expose its services. This generalized example demonstrates the
need for documentation of commits, we significantly increase ease and efficiency of implementing GitOps principles in real-
the clarity and documentation of the project without the need world scenarios, resulting in cleaner and more maintainable
for individual intervention. The history can be displayed, code (Lis. 1):
edited, deleted, and shared with other projects. Additionally, apiVersion: networking.k8s.io/v1beta1
Git also supports the creation of project wikis, which can be kind: Ingress
made during development and used as official documentation metadata:
for finished products. name: ingress
e) Environment configurations directly in the repository: annotations:
kubernetes.io/ingress.class: "nginx"
A key feature is the presence of configurations directly in nginx.ingress.kubernetes.io/proxy-body-size
the repository. Thanks to this, it is possible to share the : 50m
project together and deploy it. The main advantage of Git is cert-manager.io/cluster-issuer: "
also inherited: backup ability and the possibility of individual letsencrypt"
editing for individual branches. Naturally, several projects can nginx.ingress.kubernetes.io/from-to-www-
redirect: "true"
share parts of the configuration, and therefore, it is possible to spec:
refer to other repositories that contain general configurations tls:
of commonly used software frameworks or applications. - hosts:
f) Object-oriented approach: It is the application of - sample.app.sk
object-oriented programming standards to an architecture that secretName: sampleapp-tls
rules:
is defined as source code. Classes, functions, and constants are - host: sample.app.sk
replaced by directories, functional classes, and "configmaps". http:
Of course, each developer or development team has different paths:

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 319

 - path: / a) Increased GitHub Platform Users:
backend: • The number of users on the GitHub platform is steadily
serviceName: next
servicePort: 3000 growing.
• Notably, the utilization of GitHub Actions has experi-
Listing 1. Code snippet containing basic configuration for NGINX Ingress
enced exponential growth.
For comparison, we can look at a portion of a generalized • This indicates that teams and individuals are increasingly
example of a similar configuration that uses the regular "NG- relying on these tools for their development and deploy-
INX" configuration format (Lis. 2): ment processes.
b) Anticipated Growth:
server {
listen 443 ssl http2; • The demonstrated time and cost savings associated with
listen [::]:443 ssl http2; GitOps practices suggest a strong likelihood of continued
server_name sample.app.sk; growth.
root /var/www/sample.app.sk/; • In the professional realm, this upward trajectory is evident
ssl_certificate /etc/letsencrypt/live/
in the increasing adoption of the GitHub platform by
sample.app.sk/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/ organizations (see Fig. 2).
sample.app.sk/privkey.pem;
ssl_trusted_certificate /etc/letsencrypt/
live/sample.app.sk/chain.pem;
include nginxconfig.io/security.conf;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_set_header Host $host;
include nginxconfig.io/proxy.conf;
}
include nginxconfig.io/general.conf;
}
Listing 2. Portion of service configuration for NGINX

g) Integration with project planning tools: Since the

length of development is getting shorter nowadays, it is
convenient to have an accessible platform to organize as-
signments, sprint goals and time. For this purpose, we often
use Jira Software, a tool made by Atlassian Corporation Fig. 2. Number of organizations using GitHub platform [5]
(https://www.atlassian.com/software/jira), which is very pop-
ular in professional environments. Thanks to its support for Fig. 2 illustrates the growth in the number of organizations
interfacing with GitLab or GitHub, it makes the organization leveraging the GitHub platform [5], underlining the expanding
of work and development clearer and, in our subjective view, interest and utilization of GitOps methodologies in both the
more pleasant. Naturally, other alternatives are also available developer community and the professional sphere.
from competitors, depending on price range, methodology
IV. S YSTEMS D EPLOYMENT M ODELS
(e.g. "agile"), methodology framework (currently "scrum" is
popular) and visualization (e.g. "kanban"). GitOps commonly employs two system deployment models:
h) Integration of application interfaces: Not only tools Push and Pull. Each model has its own set of advantages and
like Jira can communicate with the git repository. Many plat- considerations, making them suitable for different scenarios:
forms offer the functionality to communicate, edit, and control a) Push model of system deployments: This model is
repositories and registries using an application interface. Users easier to implement and leaves all responsibility for application
can create custom applications and tools that are tailored to deployment on Git. The disadvantage of this model is that
their needs. Platform providers also benefit as their users Git contains access data or one-time access passwords for the
are tied into the platform ecosystem once the integrations cluster, and in case of compromise, the entire system is at
are in place, and the likelihood of them switching platforms risk [6]. The advantage is simple migration and centralized
decreases. Naturally, this implies that many ecosystems are not placement of configurations. The disadvantage of the presence
completely compatible with each other, and a time investment of some secret data in the repository requires taking care of
is required in the case of migration. the integrity and reliability of the project members.
b) Pull model of system deployment: This relatively more
III. R ISING T RENDS secure model relies on linking the cluster to Git. The update
process is simple - the tool on the cluster monitors the
Based on available statistics and analysis [5], there is clear repository in Git and, in the event of a change, downloads the
evidence of rising trends in GitOps and related technologies: current version of the application itself, processes it according

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 320

to the configuration, and deploys it to the cluster [7]. This b) Complete Systems and Environments:
solution reduces the risk of compromise as sensitive data is • ArgoCD (https://argoproj.github.io/cd/): Focuses on con-
stored as "secrets" on the cluster. Access to the repository tinuous deployment automation [12].
takes place with the help of unique passwords intended for • Kubecost (https://www.kubecost.com): Manages the allo-
this purpose. cation of system resources for containers
• Jenkins (https://www.jenkins.io): A popular open-source
V. K UBERNETES P LATFORM automation server.
The emergence of the Kubernetes platform has transformed • Rancher (https://www.rancher.com): Simplifies container
system development, introducing a high degree of dynamism management and Kubernetes orchestration.
and automation to the process. Kubernetes’ intricate architec- • Prometheus (https://prometheus.io): Provides monitoring
ture simplifies the definition of deployment processes within and alerting functionalities.
source code, using straightforward definitions and class divi- • Grafana (https://grafana.com): Offers visualization and
sions [8]. These definitions are typically expressed in formats observability solutions.
like YAML or Helm charts, streamlining the incorporation of • Harbor (https://goharbor.io): A container image registry
necessary configuration files. Kubernetes, in turn, manages the with security features.
actual application deployment seamlessly. Key characteristics • Ceph (https://ceph.io/): Software-defined block storage
of the Kubernetes platform can be elucidated by referring to solution for systems of all sizes and complexities
available literature [9]. Many of these tools and projects are part of the Cloud Native
As an open-source project initially overseen by the Cloud Computing Foundation (CNCF)(https://www.cncf.io) support
Native Computing Foundation (CNCF) under Google’s aus- program, underscoring their credibility and industry adoption.
pices, Kubernetes has experienced a consistent surge in popu- CNCF has created a clear visualization of all the projects
larity. It boasts a robust user base, well-documented resources, that are part of their portfolio using Cloud Native Landscape
ease of maintenance, and a relatively low entry threshold (https://landscape.cncf.io), so we can easily select the best
for fundamental knowledge. This accessibility is particularly project for our solution.
valuable for beginners, as a variety of books, including [9] These systems and tools are often distributed as pre-
and [10], offer ample guidance for deploying straightforward packaged containers, readily available from public registries.
systems. This approach significantly reduces the likelihood of installa-
In this revised version, the section is broken down into sub- tion errors and promotes the sharing of user-created solutions
sections, making it easier for readers to grasp the significance within the GitOps community. Public registries play a pivotal
of Kubernetes within the GitOps framework. Additionally, role in the seamless dissemination of containerized solutions,
you’ve emphasized the platform’s accessibility and available contributing to the GitOps philosophy of automation and
resources, which can appeal to newcomers and experienced standardization.
practitioners. The strong support and free availability of open Furthermore, a huge portion of these tools and systems can
source modules is strongly conducive to rapid adaptation and be used simultaneously, most commonly when one system is
self-learning. generating data or providing resources to another system. An
When it comes to disadvantages, the most mentioned one ideal example might be an architecture where Ceph manages
is the increased granularity of the entire infrastructure. This a repository that Harbor connects to and stores images there,
comes with a slight increase in system requirements, although which ArgoCD then deploys using Skaffold. This entire pro-
this increase is generally negligible in comparison to the cess, as well as the individual systems, can be managed and
capacity of modern systems and is rarely a significant concern. monitored using Rancher and Grafana. Such an architecture
appears complex but is a direct example of an object-oriented
VI. T OOLS AND M ICROSERVICES approach to systems deployment, where we can view each
In the GitOps ecosystem, a supportive community and system as an object.
industry collaboration have led to the creation of numerous re-
sources that streamline development complexity. Various tools VII. U SAGE IN THE EDUCATION PROCESS
have emerged, simplifying intricate operations and sequences In our implementation, we applied GitOps principles to a
of commands into straightforward tasks [11]. Notable tools faculty instance of the GitLab platform. The created imple-
that we tested and successfully used in this domain include: mentation was used to test student assignments and was thus
a) Development Tools: used not only by the educator for assessment but also by the
• Skaffold (https://skaffold.dev): Facilitates rapid develop- student to develop a solution.
ment and deployment workflows. We have seen a significant streamlining of the process of
• Kustomize (https://kustomize.io): Enables configuration assessing student assignments, as well as a simplification of
customization in Kubernetes deployments. its control for educators. We have also seen positive feedback
• k9s (https://k9scli.io): Offers a terminal-based interface from students, as their solutions contained fewer errors and
for Kubernetes management. they did not need to clarify some facts as before.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 321

 We attribute these positive changes largely to the structure IX. C ONCLUSION
of the GitOps process, which makes the work and the final
product more organized, structured, and transparent. In gen- In conclusion, this exploration of GitOps as an approach to
eral, we consider this implementation to be a success and plan cloud cluster system deployment underscores its transforma-
to expand and deploy it in other subjects in our department. tive potential within the DevOps landscape. Integrating GitOps
with Kubernetes creates a powerful synergy that streamlines
VIII. R ELATED W ORK deployment processes, enhances security, and improves overall
system management within cloud clusters.
GitOps and Kubernetes integration for cloud-native sys-
tem deployment have gained prominence in modern software We’ve examined the critical benefits of GitOps, includ-
development. Continuous Delivery and Continuous Integra- ing more frequent version deployments, simplified project
tion (CI/CD) have become pivotal in automating testing, management, immediate backup availability, shared documen-
building, and deploying code commits [13]. Container-based tation, and the convenience of environment configurations
applications offer solutions to challenges in CI/CD, including directly in the repository. These advantages highlight GitOps’
portability, elasticity, visibility, and version control [14]. appeal for modern development teams seeking efficiency and
Kubernetes extends its influence beyond containerized ap- reliability.
plications to network management and automation. Aligned Additionally, the rising trends in GitOps adoption, as ev-
with GitOps principles, Kubernetes Operators has shown idenced by the increasing use of platforms like GitHub and
promise in automating Netconf-based configuration manage- the support of organizations such as the Cloud Native Com-
ment for network functions. This approach streamlines net- puting Foundation (CNCF), signal its growing significance in
work operations within the cloud-native ecosystem [15]. Se- the industry. Furthermore, we’ve explored two fundamental
curity considerations have become integral to the DevOps system deployment models, the Push and Pull models, each
pipeline. Studies analyzing the security implications of De- with distinct advantages and considerations, offering teams
vOps pipelines in Kubernetes environments emphasize the flexibility to choose the most suitable approach based on their
need for secure DevOps practices to mitigate malicious threats specific needs and security requirements.
effectively. In the event that we apply DevOps practices and The Kubernetes platform’s role in this paradigm shift cannot
also include safety, this practice can be called DevSecOps be overstated. Its complex yet developer-friendly architecture
(Development and Security Operations). empowers teams to define deployment processes, efficiently
GitHub’s GitHub Actions and testing environments have leveraging simple definitions and configurations. Kubernetes
provided developers with valuable tools for code validation. has become a linchpin for orchestrating GitOps-driven deploy-
Experiments deploying GPU-providing runners on Kubernetes ments, contributing to the efficiency and dynamism of modern
clusters demonstrate the feasibility and practicality of custom system development.
setups, even without requiring cluster-admin privileges. The We’ve also highlighted tools and microservices that sim-
flexibility of this environment allows for smooth deployment plify GitOps adoption, making system development and man-
on cloud systems, distributed clusters, or on-site private de- agement more accessible, even for newcomers. These tools
ployments. facilitate automation, standardization, and unification of devel-
The adoption of microservices architecture in various indus- opment methods, fostering an environment conducive to inno-
tries has introduced security challenges due to the increased vation. In a rapidly evolving cloud-native landscape, GitOps,
attack surface. Systematic reviews of runtime security in combined with Kubernetes and an array of supporting tools,
microservices highlight areas of research abundance, gaps, offers a compelling path to enhanced deployment practices. By
and misleading information. These analyses underscore the embracing these methodologies and technologies, development
evolving nature of microservices security and the need to teams can navigate the challenges of modern software delivery
address critical challenges. with confidence and agility.
Empirical studies on software industry microservices de- Future work could look in more detail at the impact of
sign, monitoring, and testing practices provide valuable in- GitOps in terms of reducing the cognitive load on developers
sights. These studies reveal common strategies for decompos- and project managers, as well as the specific impact on all
ing applications, architectural patterns, monitoring practices, phases of development. This information would be useful to
and testing strategies. The findings also shed light on the obtain through a more objective and large-scale study with
complexity of microservices systems and the necessity for a larger sample of participants. It would also be useful to
dedicated solutions. compare a larger sample of resulting projects, which may
In the context of these related works, our research seeks to clarify some of the inaccuracies caused by our relatively small
contribute to the understanding and implementing of GitOps sample of projects. A potential research direction could also be
and Kubernetes for cloud cluster system deployment. Our psychological, as the application of specific GitOps practices
study aims to provide insights into the practical application could also make developers’ work more pleasant or more
of these methodologies and their impact on efficient system manageable. However, such research would require persons
deployment within cloud-native environments. with adequate qualifications, which we do not possess.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 322

 ACKNOWLEDGMENT
This work was supported by project VEGA No. 1/0630/22
“Lowering Programmers’ Cognitive Load Using Context-
Dependent Dialogs”
R EFERENCES
[1] F. Beetz and S. Harrer, “Gitops: The evolution of devops?” IEEE Soft-
ware, vol. 39, no. 4, pp. 70–75, 2022, doi: 10.1109/MS.2021.3119106.
[2] M. Loukides, What is DevOps? O’Reilly Media, 2012.
[3] L. Zhu, L. Bass, and G. Champlin-Scharff, “Devops and its prac-
tices,” IEEE Software, vol. 33, no. 3, pp. 32–34, 2016, doi:
10.1109/MS.2016.81.
[4] A. Baláž, N. Ádám, E. Pietriková, and B. Madoš, “Modsecurity
idmef module,” in IEEE 16th World Symposium on Applied Ma-
chine Intelligence and Informatics (SAMI), 2018, pp. 43–48, doi:
10.1109/SAMI.2018.8323984.
[5] C. Daniel, “Github users and growth statistics: How
many repos are there?” 2023. [Online]. Available:
https://www.usesignhouse.com/blog/github-stats
[6] A. Baláž and R. Hlinka, “Forensic analysis of compromised systems,”
in IEEE 10th International Conference on Emerging eLearning Tech-
nologies and Applications (ICETA), 2012, pp. 27–30, doi: 10.1109/IC-
ETA.2012.6418288.
[7] N. Ádám, M. Hulič, E. Chovancová, E. Pietriková, and T. Gergely,
“Proto-cluster: A parallel computer architecture based on prototyping
boards,” in IEEE 16th International Conference on Emerging eLearn-
ing Technologies and Applications (ICETA), 2018, pp. 15–20, doi:
10.1109/ICETA.2018.8572044.
[8] E. Pietriková and S. Chodarev, “Profile-driven source code exploration,”
in Federated Conference on Computer Science and Information Systems
(FedCSIS). IEEE, 2015, pp. 929–934, doi: 10.15439/2015F238.
[9] K. Hightower, B. Burns, and J. Beda, Kubernetes: Up and Running:
Dive into the Future of Infrastructure. O’Reilly Media, 2017.
[10] H. Saito, H. Lee, and C. Wu, DevOps with Kubernetes: Accelerating
software delivery with container orchestrators. Packt Publishing, 2017.
[11] M. Biňas, “Identifying web services for automatic assessments of
programming assignments,” in IEEE 12th International Conference on
Emerging eLearning Technologies and Applications (ICETA), 2014, pp.
45–50, doi: 10.1109/ICETA.2014.7107547.
[12] M. Madeja, J. Porubän, M. Bačíková, M. Sulír, J. Juhár, S. Chodarev,
and F. Gurbál’, “Automating test case identification in java open source
projects on github,” COMPUTING AND INFORMATICS, vol. 40, no. 3,
pp. 575–605, 2021, doi: 10.31577/cai_2021_3_575.
[13] M. Sulír, M. Bačíková, M. Madeja, S. Chodarev, and J. Juhár, “Large-
scale dataset of local java software build results,” Data, vol. 5, no. 3,
pp. 1–11, 2020, doi: 10.3390/data5030086.
[14] S. Gupta, M. Bhatia, M. Memoria, and P. Manani, “Prevalence of gitops,
devops in fast ci/cd cycles,” in International Conference on Machine
Learning, Big Data, Cloud and Parallel Computing (COM-IT-CON),
2022, pp. 589–596, doi: 10.1109/COM-IT-CON54601.2022.9850786.
[15] A. Leiter, A. Hegyi, I. Kispal, P. Boosy, N. Galambosi, and G. Z. Tar,
“Gitops and kubernetes operator-based network function configuration,”
in NOMS IEEE/IFIP Network Operations and Management Symposium,
2023, pp. 1–5, doi: 10.1109/NOMS56928.2023.10154212.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 323

 (QKDQFLQJ/HDUQLQJ2XWFRPHVZLWK,QWHUDFWLYH
&RXUVHV

-R]HI.RVWROQ\ 9HURQLND.DUFRORYD
)DFXOW\RI0DQDJHPHQW6FLHQFHDQG,QIRUPDWLFV )DFXOW\RI0DQDJHPHQW6FLHQFHDQG,QIRUPDWLFV
8QLYHUVLW\RI=LOLQD 8QLYHUVLW\RI=LOLQD
=LOLQD6ORYDNLD =LOLQD6ORYDNLD
MR]HINRVWROQ\#IULXQL]DVN NDUFRORYD#VWXGXQL]DVN

0RQLND9DFODYNRYD /LQGD%ODKRYD
)DFXOW\RI0DQDJHPHQW6FLHQFHDQG,QIRUPDWLFV )DFXOW\RI0DQDJHPHQW6FLHQFHDQG,QIRUPDWLFV
8QLYHUVLW\RI=LOLQD 8QLYHUVLW\RI=LOLQD
=LOLQD6ORYDNLD =LOLQD6ORYDNLD
PRQLNDYDFODYNRYD#IULXQL]DVN OLQGDEODKRYD#IULXQL]DVN

$EVWUDFW²(OHFWURQLFPDWHULDOVDUHEHFRPLQJDQLQFUHDVLQJO\ HGXFDWLRQDO SODWIRUP FXUUHQWO\ XVHG E\ DGPLQLVWUDWRUV

SRSXODUDGGLWLRQWRHGXFDWLRQ7KH\FDQWDNHGLIIHUHQWIRUPVDQG WHDFKHUVDQGVWXGHQWV,WLVFRPPRQLQ6ORYDNXQLYHUVLWLHVDV
VWXGHQWVFDQDFFHSWWKHPLQYDULRXVZD\V,WLVSRVVLEOHWRJDLQWKH LWLVWKHPRVWXVHGOHDUQLQJPDQDJHPHQWV\VWHP /06 WKDW
DWWHQWLRQ RI VWXGHQWV E\ PDNLQJ WKH PDWHULDOV DWWUDFWLYH 7KLV IDFLOLWDWHVFRPPXQLFDWLRQEHWZHHQVWXGHQWVDQGWXWRUV>@>@
DWWHQWLRQ LV UHWXUQHG LQ D JUHDWHU LQWHUHVW WR VWXG\ WKH SURYLGHG >@
PDWHULDORQ WKH VXEMHFW ZKLFK LQFUHDVHV WKH HIIHFWLYHQHVV RI WKH
ZKROH HGXFDWLRQDOSURFHVV 7KHSURSRVHGPRGXOH IRULQWHUDFWLYH +RZHYHU DIWHU HYDOXDWLQJ YDULRXV SXEOLVKLQJ SODWIRUPV
FRXUVHV LV SDUW RI D PRGXODU V\VWHP WKDW LV EXLOW RQ WKH 'RFNHU IRUWUDLQLQJFRXUVHVLWZDVFRQFOXGHGWKDWQRH[LVWLQJVROXWLRQ
SODWIRUP7KHFRXUVHPRGXOHLVGHVLJQHGXQLYHUVDOO\DQGFDQEH ZDVVXLWDEOHIRUDGDSWLQJWRWKHXQLYHUVLW\ VV\VWHPWRFUHDWH
DGDSWHGXQLYHUVDOO\IRUDQ\VXEMHFW,QWKLVSDSHUZHGHPRQVWUDWH LQWHUDFWLYHFRXUVHVIURPWKHLUFXUULFXOD
WKH SURSRVHG V\VWHP WR FUHDWH LQWHUDFWLYH PDWHULDO IRU WKH
3ULQFLSOHVRI2SHUDWLQJ6\VWHPVFODVV7KHV\VWHPGHVLJQDOORZV 7KHUHIRUH WKLV SDSHU SUHVHQWV D PRGXOH GHVLJQHG IRU
IRU WKH GLYLVLRQ RI WKH PDWHULDO LQWR FKDSWHUV ZKLFK FDQ EH FUHDWLQJ LQWHUDFWLYH OHVVRQV DV D PLFURVHUYLFH RI D PRGXODU
GLVSOD\HG LQ WKH VHWWLQJV VHTXHQWLDOO\RU DIWHU WKH UHTXLUHPHQW LV LQIRUPDWLRQV\VWHP7KHPRGXOHFRPSULVHVDQDGPLQLVWUDWLYH
PHW$WWKHVDPHWLPHWKHVWXGHQWLVLQIRUPHGZKHUHWKH\OHIWRII DQG XVHU SDUW ,W DLPV WR JLYH VWXGHQWV D SODWIRUP WR DFFHVV
ODVW WLPH 7KH PRGXOH LV FUHDWHG LQ D PRGHUQ GHVLJQ VR LW LV VWXG\ PDWHULDOV DQG HQDEOH WHDFKHUV WR FUHDWH DQG SXEOLVK
SRVVLEOHWRGLVSOD\LWRQDOOW\SHVRISODWIRUPVIRUPD[LPXPXVHU RQOLQH FRXUVHV HIIRUWOHVVO\ $OWKRXJK SULPDULO\ IRFXVHG RQ
VDWLVIDFWLRQ LQIRUPDWLFVVXEMHFWVWKHPRGXOHDLPVWREHXQLYHUVDOHQRXJK
IRUWKHHQWLUHIDFXOW\
.H\ZRUGV²PLFURVHUYLFHV PRGXOH HGXFDWLRQ OHDUQLQJ
PDWHULDOVV\VWHP 7KLVV\VWHPXVHVDPLFURVHUYLFHDUFKLWHFWXUHZKHUHHDFK
PRGXOHRUVHUYLFHFDQRSHUDWHLQGHSHQGHQWO\RIWKHRWKHUVDV
, ,1752'8&7,21 D VWDQGDORQHDSSOLFDWLRQ,IDSUREOHPRFFXUV LQRQHRI WKH
2QOLQH FRXUVHV KDYH EHFRPH LQFUHDVLQJO\ SRSXODU IRU VHUYLFHVLWLVHQRXJKWRIL[WKHSUREOHPLQWKHVHUYLFHZKLOH
REWDLQLQJQHZVNLOOVNQRZOHGJHDQGLQIRUPDWLRQ7KH\RIIHU DOORWKHUSDUWVRIWKHV\VWHPDUHZRUNLQJVHDPOHVVO\
QXPHURXV DGYDQWDJHV RYHU WUDGLWLRQDO HGXFDWLRQ LQFOXGLQJ 7KH VHSDUDWH PRGXOHV ZLOO EH JURXSHG XVLQJ D VLQJOH
DIIRUGDELOLW\FRQYHQLHQFHLQWHUDFWLYLW\DQGDELOLW\WRSURFHHG LQWHUIDFHUHSUHVHQWLQJDQRWKHULQGHSHQGHQWPRGXOH7KURXJK
DW RQH V RZQ SDFH ,Q UHVSRQVH WR FXUUHQW OLPLWDWLRQV RQ LQ WKLVV\VWHPLQWHUIDFHWKHXVHUFDQDFFHVVWKHUHTXHVWHGVHUYLFH
SHUVRQWHDFKLQJPDQ\WUDGLWLRQDOVFKRROVKDYHEHJXQRIIHULQJ ,Q WKLV DUWLFOH ZH SURSRVH DQG LPSOHPHQW D PRGXOH IRU
RQOLQHFXUULFXODWRIDFLOLWDWHGLVWDQFHOHDUQLQJ2QOLQHFRXUVHV FUHDWLQJLQWHUDFWLYHFRXUVHVDVSDUWRIDPRGXODULQIRUPDWLRQ
DOVR DOORZ VWXGHQWV WR H[SDQG DQG SUDFWLFH WKHLU NQRZOHGJH V\VWHP WKDW LQFUHDVHV WKH HQWLUH V\VWHP V UHOLDELOLW\ XVLQJ
EH\RQGWKHFRQILQHVRIWKHFXUULFXOXP PLFURVHUYLFHV 7KH ZRUN UHVXOWV LQ D PRGXOH ZLWK GHILQHG
9DULRXV RQOLQH VHUYLFHV RIIHU RQOLQH FRXUVHV DQG IXQFWLRQDOLWLHV IRU WKUHH W\SHV RI HQG XVHUV DGPLQLVWUDWRU
NQRZOHGJHH[FKDQJHLQDYDULHW\RIIRUPDWVYLDYDULRXVZHE WHDFKHUDQGVWXGHQW$FRXUVHDVVLJQHGWRDVSHFLILFVXEMHFW
DSSOLFDWLRQV7KHVHSODWIRUPVFDQEHXWLOL]HGWRUHDFKVSHFLILF FRQVLVWVRIOHVVRQVDQGFKDSWHUVDQGDOOSDUWVRIWKHV\VWHP
JURXSV RI VWXGHQWV DV ZHOO DV PLVFHOODQHRXV LQWHUQHW XVHUV FDQEHHGLWHGGHOHWHGSXEOLVKHGRUKLGGHQIURPVWXGHQWV,Q
&RXUVHUDLVDZHOONQRZQSODWIRUPWKDWRIIHUVDZLGHUDQJHRI WKHIXWXUHWKHPRGXOHFDQEHH[SDQGHGWRLQFOXGHWKHFUHDWLRQ
FRXUVHV LQFOXGLQJ LQIRUPDWLRQ WHFKQRORJ\ GDWD WHFKQRORJ\ RIYDULRXVHGXFDWLRQDODFWLYLWLHVVXFKDVTXL]]HVDVVLJQPHQWV
DQG EXVLQHVV RU SHUVRQDO GHYHORSPHQW >@ >@ 6LPLODUO\ RUDQRQOLQHFRPSLOHU
8GHP\LVDOVRDSRSXODUSODWIRUPIRURQOLQHOHDUQLQJZKLFK ,, 62/87,21'(6,*1
LV SDUWLFXODUO\ VXLWDEOH IRU VWXGHQWV VHHNLQJ SUDFWLFDO VNLOOV
ZLWKRXWSXUVXLQJDXQLYHUVLW\GHJUHH$OWKRXJK8GHP\LVVXHV $V RQH RI WKH PLFURVHUYLFHV RI D PRGXODU LQIRUPDWLRQ
FHUWLILFDWHV XQOLNH &RXUVHUD XQLYHUVLWLHV VWLOO PD\ QRW V\VWHP ZH VKRZ LQ WKLV DUWLFOH WKH GHVLJQ RI D PRGXOH IRU
UHFRJQL]H WKHP DV YDOLG >@ >@ 0RRGOH LV DQ DGDSWDEOH PDNLQJLQWHUDFWLYHFRXUVHV7KHUHZLOOEHDGPLQLVWUDWLYHDQG

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 324

XVHUSRUWLRQVWRWKLVPRGXOH,WZLOODOVRDOORZSURIHVVRUVWR ,,, &/,(176,'(02'8/('(6,*1
TXLFNO\ GHVLJQ DQG SXEOLVK RQOLQH FRXUVHV FRQWDLQLQJ WKHVH  7KHFOLHQWVLGHLQDZHEDSSOLFDWLRQRUDOVRWKHIURQWHQG
SHUWLQHQW LQIRUPDWLRQ DQG LW ZLOO IXQFWLRQ DV D ORFDWLRQ IRU UHSUHVHQWV WKH XVHU LQWHUIDFH ZLWK ZKLFK XVHUV RI WKH ZHE
VWXGHQWV WR JDWKHU VWXG\ PDWHULDOV 7KH GHPDQGV RI DSSOLFDWLRQ LQWHUDFW GLUHFWO\ 8VLQJ WKLV SDUW XVHUV VHQG
LQIRUPDWLFV GLVFLSOLQHV VKRXOG EH LWV SULPDU\ IRFXV EXW WKH VSHFLILF UHTXHVWV WR WKH VHUYHU XVLQJ YDULRXV DFWLRQV DQG
IDFXOW\DVDZKROHVKRXOGEHDEOHWRXVHLWJHQHUDOO\ IRUPV:KHQGHVLJQLQJWKLVSDUWZHPDLQO\IRFXVHGRQWKH
7KHPRGXOHIRUFUHDWLQJLQWHUDFWLYHFRXUVHVLVGHVLJQHGDV GHVLUHGDSSHDUDQFHRIWKHPRGXOH
D SDUW RI D PRGXODU LQIRUPDWLRQ V\VWHP ZKLFK FRQWDLQV  %HIRUHZHVWDUWHGGHYHORSLQJWKHPRGXOHZHGHVLJQHGD
VHYHUDOIXQFWLRQVDQGWKHUHIRUHLWLVQRWDVLPSOHPRQROLWKLF VLPSOH ORRN DQG IHHO XVLQJ 3HQFLO :H ZHUH DZDUH WKDW WKH
PRGHO 0RQROLWKLF DUFKLWHFWXUH XQLILHV DOO LWV FRPSRQHQWV ILQDOORRNRIWKHPRGXOHZRXOGSUREDEO\EHGLIIHUHQWEXWWKLV
EXVLQHVVORJLFDQGPRGXOHV,WLVHDV\WRGHYHORSGHEXJDQG GHVLJQKHOSHGXVDORWWRFODULI\WKHSRVVLEOHIXQFWLRQDOLWLHVZH
GHSOR\EXWLWLVRQO\VXLWDEOHIRUQRQGHPDQGLQJDSSOLFDWLRQV VKRXOGLPSOHPHQW$OOLQWHUIDFHVIRUWHDFKHUDGPLQLVWUDWRU
GXH WR LWV ILUP FRQVLVWHQF\ 2QO\ D WLQ\ FKDQJH FDXVHV DQGVWXGHQWFRQWDLQDPDLQPHQXZKHUHWKHXVDJHRSWLRQVIRU
VLJQLILFDQW DGMXVWPHQWV WKURXJKRXW WKH SURMHFW DQG RQH DVSHFLILFW\SHRIXVHUDUHGHSLFWHGDFFRUGLQJWRWKHVSHFLILF
SUREOHPFDQFDXVHWKHZKROHV\VWHPWRFUDVK>@ UROH)LJXUHVKRZVDQLQLWLDOGHVLJQRIWKHXVHULQWHUIDFHIRU
7KHUHIRUH VXFK PRUH FRPSOH[ PRGXOH V\VWHPV XVH D DWHDFKHUWKHFRXUVHDGPLQLVWUDWRU7KHGHVLJQVKRZVFKDSWHUV
PLFURVHUYLFHDUFKLWHFWXUHZKHUHHDFKPRGXOHRUVHUYLFHFDQ IURP D VSHFLILFFRXUVH DQGWKH IXQFWLRQDOLW\ DVVRFLDWHG ZLWK
RSHUDWH LQGHSHQGHQWO\ RI WKH RWKHUV DV D VWDQGDORQH WKHP+HFDQFKDQJHWKHLURUGHUHGLWDFKDSWHUGHOHWHWKHP
DSSOLFDWLRQ ,I D SUREOHP RFFXUV LQRQHRIWKH VHUYLFHV LW LV $WWKHHQGXQGHUWKHOLVWRIFRXUVHFKDSWHUVWKHUHDUHRSWLRQV
HOHPHQWDU\ WR ILQG WKH SUREOHP DQG WKH RWKHU SDUWV RI WKH WRGHOHWHWKHHQWLUHFRXUVHRUDGGDQHZFKDSWHU
V\VWHPFDQFRQWLQXHWRZRUNZLWKRXWSUREOHPV>@
7KH VHSDUDWH PRGXOHV ZLOO EH JURXSHG XVLQJ D VLQJOH
LQWHUIDFHUHSUHVHQWLQJDQRWKHULQGHSHQGHQWPRGXOH7KURXJK
WKLV V\VWHP LQWHUIDFH WKH XVHU ZLOO EH DEOH WR DFFHVV WKH
UHTXHVWHGVHUYLFH )LJ 


)LJ,QWHUIDFHGHVLJQIRUOHVVRQGHYHORSPHQW

 7KH VHFRQG LPSRUWDQW FRXUVH PDQDJHPHQW LQWHUIDFH ZH

GHVLJQHGLVDSUHYLHZRIWKHDSSHDUDQFHRIWKHIRUPIRUDGGLQJ
RUHGLWLQJDOHVVRQIURPDFRXUVHFKDSWHU,WVKRXOGEHSRVVLEOH
WRFKDQJHWKHWLWOHWKHRUGHURIWKHOHVVRQ7KHOHVVRQLQWKH
LQLWLDO GHVLJQ FRQVLVWHG RI WH[W VHFWLRQV HDFK ZLWK LWV RZQ
DGMXVWDEOHVW\OHDQGRUGHUVRWKHUHLVDOVRWKHSRVVLELOLW\RI
 HGLWLQJ LQGLYLGXDO WH[W VHFWLRQV LQ WKLV LQWHUIDFH  :H WKHQ
)LJ0LFURVHUYLFHDUFKLWHFWXUHGLDJUDP
FUHDWHGWKHGHVLJQOD\RXWGHSLFWHGLQ)LJXUHIRUWKHDGPLQ
DGPLQLVWUDWLRQ
7KHFRXUVHFUHDWLRQPRGXOHDVRQHRIWKHPLFURVHUYLFHV
ZRUNV LQGHSHQGHQWO\ IURP WKH RWKHU PRGXOHV LQ WKH V\VWHP
DQGWKHUHIRUHZHFDQGHVLJQLWVSDUWVVHSDUDWHO\VLPLODUWRWKH
PRQROLWKLFDUFKLWHFWXUH7KHDUFKLWHFWXUHRIWKHPRGXOHLQWKLV
FDVH ZLOO EH LPSOHPHQWHG DFFRUGLQJ WR WKH 09& GHVLJQ
SDWWHUQZKLFKGLYLGHVWKHSURMHFWLQWRWKUHHSDUWV0RGHORU
GDWD REMHFWV DQG WKHLU SURSHUWLHV 9LHZ UHSUHVHQWV WKH
DSSHDUDQFHRUXVHUHQYLURQPHQWDQG&RQWUROOHUFRQWDLQVWKH
RSHUDWLRQV IRU PDQDJLQJ WKH PRGHOV DQG GDWD 7KLV W\SH RI
DUFKLWHFWXUHVHSDUDWHVWKHFRPSRQHQWVIURPHDFKRWKHUORJLF
DQGRXWSXW>@

 $VDVLQJOHVROXWLRQZHGHFLGHGWRVSOLWWKHPRGXOHLQWR
WZR SURMHFWV IRU ERWK WKH FOLHQW DQG VHUYHU SDUWV HDFK )LJ&RXUVHGLVSOD\LQWHUIDFHGHVLJQ
GHYHORSHGXVLQJDVHSDUDWHWHFKQRORJ\,QWKLVFKDSWHUZHZLOO
VWLOO IRFXV RQ WKH GHVLJQ RI WKH LQGLYLGXDO FRPSRQHQWV WKH  7KH DGPLQLVWUDWRU LV QRW RQO\ LQ FKDUJH RI FRXUVH
FRPPXQLFDWLRQEHWZHHQWKHP:HZLOOFRYHUWKHWHFKQRORJLHV DGPLQLVWUDWLRQEXWHVSHFLDOO\LQFKDUJHRIUROHDGPLQLVWUDWLRQ
E\ZKLFKWKHSURMHFWVZLOOEHGHYHORSHGLQWKHQH[WFKDSWHU ,QWKLVLQWHUIDFHKHFDQVHHWKHOLVWRIXVHUVWKHLUUROHDQGIRU
HDFKXVHUH[FHSWKLPVHOIKHFDQFKDQJHWKLVUROHDQGWKHQVDYH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 325

KLVVHOHFWLRQ7KHRUGLQDU\XVHUWKHVWXGHQWLVRQO\DEOHWRVHH 7KH PDLQ OD\RXWRI WKLV DSSOLFDWLRQ FRQVLVWVRI VHSDUDWH
WKHLQGLYLGXDOFRXUVHVWKHLUFKDSWHUVDQGOHVVRQVKHKDVQR PRGXOHV FRPSRQHQWV VHUYLFHV DQG REMHFW PRGHOV
DFFHVV WR DQ\ HGLWLQJ )LJXUH  VKRZV WKH LQWHUIDFH IRU &RPSRQHQWV FRQWDLQ LQGLYLGXDO YLHZV IRU SDJHV RU SDUWV RI
GLVSOD\LQJDVSHFLILFOHVVRQWKDWWKHVWXGHQWVHOHFWVIURPWKH SDJHVRIDPRGXOH7KH\GLVSOD\GDWDIURPWKHGDWDEDVHRU
QDYLJDWLRQ YDULRXVIRUPVIRUPRGLI\LQJWKDWGDWD6HUYLFHVWDNHFDUHRI
WKHFRPPXQLFDWLRQIRUUHWULHYLQJDQGPRGLI\LQJWKLVGDWD,Q
,9 ,03/(0(17$7,212)62/87,21 WKH FOLHQW SDUW ZH KDYH DOVR FUHDWHG PRGHOV IRU HDFK HQWLW\
 ,QWKLVFKDSWHUZHZLOOJRWKURXJKKRZWRLPSOHPHQWWKH IURP WKH GDWDEDVH IRU HDVLHU ZULWLQJ HGLWLQJ RU MXVW
PRGXOHVHUYHUDQGFOLHQWVLGHXVLQJWKHVHOHFWHGWHFKQRORJLHV GLVSOD\LQJREMHFWV
$ ,PSOHPHQWDWLRQRIWKHVHUYHUSDUW 7KH PDLQ UHTXLUHPHQW RI RXU PRGXOH ZDV WR GLYLGH WKH
PRGXOHLQWRWKUHHSDUWVDGPLQLVWUDWLYHWHDFKLQJDQGVWXGHQW
 ,QWKHFUHDWLRQRIWKHVHUYHUSDUWZKHQZRUNLQJZLWKGDWD
7KLV PHDQV WKDW HDFK ORJJHG LQ XVHU LV DVVLJQHG D UROH DQG
WKH UHSRVLWRU\ SDWWHUQ LV XVHG LH WKH GHVLJQ SDWWHUQ XVLQJ
SHUPLVVLRQVWRYLVLWRQO\VRPHRIWKHSDJHV:HKDGWRHQVXUH
UHSRVLWRULHV ,W XVHV UHSRVLWRULHV ZKLFK DUH FRPSRQHQWV WKDW
WKDWIRUH[DPSOHDVWXGHQWFRXOGQRWDFFHVVWKHDGPLQSDUW
VWRUHDQGXQLI\WKHFRPSOHWHORJLFIRUDFFHVVLQJDQGZRUNLQJ
7KLVVHFXULW\ZHLPSOHPHQWHGWKURXJKD-621ZHEWRNHQ
ZLWKWKHVWRUHGGDWD,WFUHDWHVDQDEVWUDFWOD\HUEHWZHHQWKH
GRPDLQOD\HUDQGWKHGDWDOD\HU7KLVPHDQVWKDWIRUH[DPSOH 7KH -:7 WRNHQ GHILQHV WKH VHFXUH WUDQVPLVVLRQ RI
ZKHQDFOLHQWDSSOLFDWLRQLVPRGLILHGDQGGDWDLVXSGDWHGDQ LQIRUPDWLRQLQWKH-621REMHFWIRUPDW,WLVPDLQO\XVHGLQ
REMHFW LV FUHDWHG LQ WKH UHSRVLWRU\ PRGLILHG DQG RQO\ WKHQ DXWKHQWLFDWLRQZKHUHHDFKXVHULVDVVLJQHGWKHLURZQWRNHQ
VXEVHTXHQWO\ VWRUHG LQ WKH GDWDEDVH YLD WUDQVDFWLRQV 7KLV 7RNHQV WHQG WR EH NH\ VLJQHG PDNLQJ WKHP UHOLDEOH IRU
GHVLJQ SDWWHUQ ZDV FKRVHQ EHFDXVH RI LWV FODULW\ HDVH RI WUDQVPLWWLQJYDULRXVFODVVLILHGGDWD>@
WHVWLQJDQGVXLWDELOLW\IRUDV\VWHPEDVHGRQDPLFURVHUYLFH
DUFKLWHFWXUH7KHUHSRVLWRU\FDQEHFUHDWHGIRUDVLQJOHWDEOH $W ORJLQ WKH VHUYHU SDUW JHQHUDWHV D WRNHQ ZLWK D FHUWDLQ
RUJHQHULFDOO\IRUPXOWLSOHHQWLWLHV7KXVWKLVIROGHUFRQWDLQV YDOLGLW\ SHULRG ZKLFK LV FKHFNHG LQ HDFK VHW RI JXDUGV 
LQGLYLGXDO UHSRVLWRULHV WKDW ZRUN ZLWK WKH GDWD FRQWH[W DQG JXDUGV 7KH JXDUG LV DOORFDWHG LQ WKH URXWLQJ FRPSRQHQW
PRGLI\LQVHUWRUGHOHWHGDWD>@ ZKLFK WDNHV FDUH RI WKH UHGLUHFWLRQ RQ HDFK VLWH )LJXUH 
VKRZV DQ H[DPSOH RI GHILQLQJ UHGLUHFWLRQ RQ WKH UROH
 7KHVHUYHUFDVWDOVRLQFOXGHVGHILQLQJWKHGDWDPLJUDWLRQ PDQDJHPHQW FRPSRQHQW RI WKH DGPLQ SDUW DQG KDV WZR
SULQFLSOH 'DWD PLJUDWLRQ LV WKH SURFHVV RI WUDQVIHUULQJ GDWD UHGLUHFWLRQJXDUGVGHILQHG$XWK*XDUGZKLFKFKHFNVLIWKH
IURPRQHGHYLFHDSSOLFDWLRQIRUPDWWRDQRWKHU7KXVZKHQ XVHULVORJJHGLQDQG5ROH$XWK*XDUGZKLFKFKHFNVWKHUROH
ZH FUHDWHG RU PRGLILHG WKH GDWD PRGHO ZH FUHDWHG D QHZ RIWKHORJJHGLQXVHU7KH\FRQWDLQGDWDZKHUHZHFDQGHILQH
PLJUDWLRQ DXWRPDWLFDOO\ JHQHUDWHG ILOH XVLQJ WKH FRPPDQG WKH UROH WKDW FDQ ORDG D JLYHQ SDJH ,Q WKLV FDVH LW LV WKH
OLQH DQG WKHQ XSGDWHG WKHGDWDEDVH DOVR XVLQJ WKHFRPPDQG DGPLQLVWUDWRU 7KHVH JXDUG FRPSRQHQWV LPSOHPHQW WKH
OLQH7KHPLJUDWLRQILOHVDFWXDOO\FRQWDLQFRPPDQGVWRFUHDWH FDQ$FWLYDWH LQWHUIDFH DQG UHWXUQ ERROHDQ YDOXHV H[SUHVVLQJ
RUPRGLI\LQGLYLGXDOWDEOHHQWLWLHV7KHUHDUHVHYHUDORIWKHVH WKHDELOLW\WRUHGLUHFWWKHXVHUWRDVSHFLILFSDJH
ILOHV LQ WKH IROGHU EHFDXVH ZH DGGHG RU FKDQJHG LQGLYLGXDO
PRGHOV PXOWLSOH WLPHV GXULQJ WKH LPSOHPHQWDWLRQ SURFHVV
7KXV WKH PLJUDWLRQV ZHUH XVHG WR XSGDWH WKH GDWDEDVH LQ
3RVWJUH64/ DQG ZH ZHUH DEOH WR REVHUYH WKHVH FKDQJHV LQ
SJ$GPLQ,Q)LJXUHZHFDQVHHZKDWWKHPLJUDWLRQILOHV
ORRNOLNHVSHFLILFDOO\ZKHQWKHILUVW2GGVWDEOHZDVFUHDWHG



)LJ([DPSOHRIGHILQLQJUHGLUHFWLRQWRWKHDGPLQFRPSRQHQW

9 9(5,),&$7,212)7+(62/87,21
 7KLVFKDSWHUFRQWDLQVGHPRQVWUDWLRQVDQGYDOLGDWLRQRIWKH
 IXQFWLRQDOLWLHV WKURXJK WKH XVHU LQWHUIDFH RI WKH ILQDO
LQWHUDFWLYHFRXUVHFUHDWLRQPRGXOH
)LJ([DPSOHRIXVHG0LJUDWLRQV
$IWHUORJJLQJLQWKHXVHULVWDNHQWRWKHPDLQSDJHZKHUH
% &OLHQWVLGHLPSOHPHQWDWLRQ DOOSXEOLVKHGFRXUVHVDUHGLVSOD\HGLQWDEVDQGRQWKHULJKWVLGH
WKHUHLVWKHPDLQPHQXVKRZLQJWKHIXQFWLRQDOLWLHVSRVVLEOH
7KHFOLHQWSDUWZDVGHYHORSHGLQWKH$QJXODUIUDPHZRUN
IRUWKHORJJHGLQXVHU)LJXUHVKRZVWKHSRVWORJLQYLHZRI
LQ D VHSDUDWH SURMHFW VSHFLILFDOO\ XVLQJ D SXUFKDVHG )XVH
WKHDGPLQLVWUDWRUZKRDOVRKDVWKHDELOLW\WRPDQDJHFRXUVHV
WHPSODWH )XVH LQFOXGHV SUHPDGH VDPSOHV RI DSSOLFDWLRQ
DQGXVHUV
FRPSRQHQWV DQG DSSHDUDQFH GHVLJQV DV ZHOO DV
DXWKHQWLFDWLRQZKLFKPDNHVIURQWHQGGHYHORSPHQWOHVVWLPH  ,IWKHXVHUKDVEHHQDVVLJQHGWKHUROHRIDGPLQLVWUDWRUWKH
FRQVXPLQJ PDLQPHQXVKRZVWKHRSWLRQIRUUROHDVVLJQPHQW:KHQWKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 326

XVHUFOLFNVDQGLVUHGLUHFWHGWRWKHDGPLQLVWUDWLRQSDJHDOLVW WKHPRGXOHWKDWDUHDVVLJQHGWRWKHLUQHZUROHDUHGLVSOD\HG
RI XVHUV DQG WKHLU DVVLJQHG UROHV IURP WKH GDWDEDVH LV 7HDFKHUVDQGDGPLQLVWUDWRUVDOLNHDUHDOORZHGWRDGPLQLVWHU
GLVSOD\HG $IWHU FOLFNLQJ RQ D VSHFLILF XVHU V UROH WKH FRXUVHV :KHQ WKH\ FOLFN RQ WKH (GLW &RXUVHV RSWLRQ LQ WKH
DGPLQLVWUDWRULVSUHVHQWHGZLWKDPHQXRIRSWLRQVWRFKDQJH PHQX WKH\ DUH UHGLUHFWHG WR D SDJH ZLWK D WDEOH RI ERWK
WKHUROHDQGDEXWWRQWRVDYHWKHRSWLRQ,IWKHDGPLQLVWUDWRU SXEOLVKHGDQGXQSXEOLVKHGFRXUVHVFUHDWHG7KHWDEOHOLVWVWKH
FKRRVHVWRVDYHWKHVHWWLQJDSRSXSZLQGRZLVGLVSOD\HGWR WLWOHWKHVXEMHFWIURPZKLFKWKHFRXUVHLVIURPDQGRSWLRQV
LQGLFDWH WKDW WKH FKDQJH ZDV VXFFHVVIXOO\ VDYHG :KHQ WKH IRUHGLWLQJGHOHWLQJDQGYLVLELOLW\%HORZWKHWDEOHLVDKLGGHQ
XVHUZKRVHGDWDKDVEHHQFKDQJHGORJVEDFNLQWKHSDUWVRI IRUPIRUDGGLQJDFRXUVHGLVSOD\HGZKHQWKHEXWWRQLVFOLFNHG


)LJ0DLQSDJHXVHULQWHUIDFH

 7KHIRUPFRQWDLQVRQO\WKHEDVLFLQSXWVWKDWDUHGHSLFWHG  ,I DOO OHDUQLQJ PDWHULDOV IRU D FRXUVH DUH FRPSOHWHG WKH
LQWKHWDEOHWLWOHDQGVXEMHFW:KHQWKLVIRUPLVFRPSOHWHG WHDFKHURUDGPLQLVWUDWRUFDQSXEOLVKWKHFRXUVHE\FOLFNLQJRQ
DQG WKH FRXUVH LV VDYHG WR WKH GDWDEDVH D QRWLILFDWLRQ LV WKHLFRQLQWKHVDPHWDEOHLQWKHYLVLELOLW\FROXPQRURWKHUZLVH
GLVSOD\HG,IWKHXVHUZLVKHVWRGHOHWHWKHOHVVRQKHVKHFOLFNV FDQFHOLWVYLVLELOLW\,QWKHILUVWRSWLRQDPHVVDJHLVGLVSOD\HG
RQ WKH UHG PLQXV LFRQ ZKLFK RQFH FRQILUPHG GHOHWHV WKH WR VKRZ EDVLF LQIRUPDWLRQ DERXW WKH FKDSWHUV DQG OHVVRQV
HQWLUHFRXUVHZLWKDOOLWVFKDSWHUVDQGOHVVRQV )LJ  ZKHWKHUDOOOHVVRQVDUHFRPSOHWHG7KHFRXUVHDGPLQLVWUDWRU
ZLOOWKHQGHFLGHZKHWKHUWRSXEOLVKWKHLQFRPSOHWHFRXUVHRU
DOWHUQDWLYHO\EHDOHUWHGWRWKHLQFRPSOHWHQHVVLQWKLVZD\7R
DGGFKDSWHUVDQGOHVVRQVWKHPRGXOHXVHUFOLFNVRQWKHSHQ
LFRQLQWKHHGLWFROXPQDQGZLOOWKXVEHUHGLUHFWHGWRDVLPLODU
SDJH ZLWK D OLVW RI FKDSWHUV )LJXUH  VKRZV WKH FKDSWHUV
EHORQJLQJWRWKHVHOHFWHGFRXUVH

 
)LJ8VHULQWHUIDFHIRUFKDQJLQJLQIRUPDWLRQDERXWFRXUVH )LJ8VHULQWHUIDFHIRUFKDQJLQJLQIRUPDWLRQDERXWFRXUVH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 327

,Q WKLV YLHZ LQ DGGLWLRQ WR HGLWLQJ DGGLQJ DQG GHOHWLQJ 9, &21&/86,21
FKDSWHUVWKHXVHUKDVWKHDELOLW\WRFKDQJHWKHRUGHULQZKLFK  ,Q WKLV DUWLFOH ZH SURSRVH DQG LPSOHPHQW D PRGXOH IRU
WKH\DUHGLVSOD\HGWRWKHXVHU2QWKHULJKWVLGHRIWKLVYLHZLV FUHDWLQJLQWHUDFWLYHFRXUVHVDVSDUWRIDPRGXODULQIRUPDWLRQ
DVXPPDU\RIWKHFRXUVHZLWKWKHQXPEHURIFKDSWHUVOHVVRQV V\VWHP>@WKDWLQFUHDVHVWKHHQWLUHV\VWHP VUHOLDELOLW\ZLWK
FRPSOHWHGOHVVRQVDQGLWVYLVLELOLW\SXEOLVKHG7KHUHLVDOVRDQ WKH XVH RI PLFURVHUYLFHV >@ >@ 7KH ZRUN UHVXOWV LQ D
RSWLRQWRLQVHUWDVKRUWGHVFULSWLRQRIWKHFRXUVHDQGHGLWWKH PRGXOH ZLWK GHILQHG IXQFWLRQDOLWLHV IRU WKUHH W\SHV RI HQG
FRXUVHWLWOHDQGWKHVXEMHFWWRZKLFKLWEHORQJV XVHUVDGPLQLVWUDWRUWHDFKHUDQGVWXGHQW$FRXUVHDVVLJQHG
7KH ODVW SDUW RI FUHDWLQJ FRXUVH PDWHULDOV LV ILOOLQJ LQ DQG WRDVSHFLILFVXEMHFWFRQVLVWVRIOHVVRQVDQGFKDSWHUVDQGDOO
HGLWLQJWKHOHVVRQV\RXKDYHFUHDWHG )LJ  SDUWV RI WKH V\VWHP FDQ EH HGLWHG RU GHOHWHG SXEOLVKHG RU
KLGGHQIURPVWXGHQWV
7KH FRXUVH DGPLQLVWUDWRU VLPSO\ XVHV WKH :<6,:<*
HGLWRUWRFUHDWHWKHOHVVRQPDWHULDO)URPKHUHKHRUVKHFDQ  7KHNH\HOHPHQWRIVXFKDV\VWHPDOVRLQYROYHVGRLQJDQ
FKDQJHWKHVL]HIRQWFRORUDQGRWKHUIRQWVW\OHDGGLWLRQVDV HYDOXDWLRQ ,Q WKH VXEVHTXHQW VHFWLRQ RI WKLV VWXG\ ZH ZLOO
GHVLUHG7KHUHLVDOVRDQRSWLRQWRDGGDQLPDJHOLQNRURQOLQH DGGUHVV WKH LQVWDOODWLRQ RI D FRPSRQHQW GHVLJQHG WR JDWKHU
YLGHR IRUPDWLYH HYDOXDWLRQ GDWD IRU LQGLYLGXDO OHDUQLQJ PDWHULDOV
7KHSXUSRVHRIWKHVHHYDOXDWLRQVLVWRJDWKHUIHHGEDFNIURP
VWXGHQWVQHFHVVLWDWLQJWKHLUGHYHORSPHQWLQDFFRUGDQFHZLWK
D VWDQGDUGL]HG IUDPHZRUN 7KLV IUDPHZRUN HQDEOHV WKH
FRPSDULVRQRIHYDOXDWLRQUHVXOWVDFURVVYDULRXVVXEMHFWDUHDV
DV RXWOLQHG LQ WKH ZRUN >@ 7KH DXWKRUV HPSKDVL]H WKH
FRQWHPSRUDU\VLJQLILFDQFHRIREWDLQLQJDFFXUDWHIHHGEDFNWR
HQKDQFHWKHWHDFKLQJSURFHVV
 ,Q WKH IXWXUH ZH FDQ H[SDQG WKH PRGXOH WR LQFOXGH WKH
FUHDWLRQ RI YDULRXV HGXFDWLRQDO DFWLYLWLHV VXFK DV TXL]]HV
DVVLJQPHQWVRUDQRQOLQHFRPSLOHU

 
)LJ8VHULQWHUIDFHIRUFKDQJLQJLQIRUPDWLRQDERXWFRXUVH $&.12:/('*0(17
7KLV ZRUN ZDV VXSSRUWHG E\ *UDQW RI 6\VWHP RI WKH
 /DVW EXW QRW OHDVW ZH KDYH WKH VWXGHQW VHFWLRQ $IWHU 8QLYHUVLW\ RI =LOLQD 1R  DQG E\ 0LQLVWU\ RI
VHOHFWLQJDSXEOLVKHGFRXUVHIURPWKHPDLQSDJHRUPHQXLW (GXFDWLRQ 6FLHQFHDQG 6SRUW RIWKH 6ORYDN 5HSXEOLF XQGHU
ZLOOSURYLGHWKHIROORZLQJYLHZRQ)LJXUH WKHJUDQW.(*$ 83-â 

5()(5(1&(6

>@ &KULV $ Ä %HVW :HEVLWHV 7R )LQG 2QOLQH &RXUVHV³
KWWSVZZZUHOLDEOHVRIWQHWEHVWRQOLQHFRXUVHV  ODVW DFFHVVHG

>@ &RXUVHUD Ä$ERXW³  KWWSVDERXWFRXUVHUDRUJ ODVW DFFHVVHG

>@ 8GHP\ Ä2XU RULJLQV³ KWWSVDERXWXGHP\FRPFRPSDQ\  ODVW
DFFHVVHG

>@ 'HQLV0Ä861HZV³KWWSVZZZXVQHZVFRPHGXFDWLRQXGHP\
)LJ8VHULQWHUIDFHWRYLHZWKHVHOHFWHGOHVVRQ RYHUYLHZODVWDFFHVVHG
>@ 6WXG\3RUWDOV Ä:KDW LV 022'/(" :KDW DUH 2QOLQH /HDUQLQJ
 2QWKHOHIWVLGHRIWKLVYLHZLVWKHOD\RXWRIWKHFRXUVH 0DQDJHPHQWV 6\VWHPV"³ KWWSVZZZGLVWDQFHOHDUQLQJSRUWDOFR
PDUWLFOHVZKDWLVPRRGOHZKDWDUHRQOLQHOHDUQLQJDQDJHPHQWV
WKHFKDSWHUV$IWHUFOLFNLQJRQWKHVHOHFWHGFKDSWHUWKHVWXGHQW V\VWHPVKWPOODVWDFFHVVHG
ZLOO VHH DOO WKH OHVVRQV EHORQJLQJ WR WKH VHOHFWHG FKDSWHU >@ 0RRGOH Ä$ERXW 0RRGOH³ KWWSVGRFVPRRGOHRUJHQ$E
ZKRVHFRQWHQWZLOOEHGLVSOD\HGRQWKHULJKWVLGH$OVRWKH RXWB0RRGOH$OOLQRQHBOHDUQLQJBSODWIRUPODVWDFFHVVHG
VWXGHQW FDQ HDVLO\ QDYLJDWH WKURXJK WKH LQGLYLGXDO OHVVRQV >@ 0RRGOH Ä+LVWRU\³ KWWSVGRFVPRRGOHRUJHQ+LVWRU\  ODVW
XVLQJWKHIRUZDUGDQGEDFNEXWWRQV DFFHVVHG
>@ 5DWKRXVNê - $QDOê]D PRGHUQtFK VRIWZDURYêFK DUFKLWHNWXU 3UDKD
 7KHDLPRIWKLVPRGXOHLQWKHPRGXODULQIRUPDWLRQV\VWHP ýHVNpY\VRNpXþHQtWHFKQLFNpY3UD]H  
LV D XQLYHUVDO PRGXOH ZKLFK VKRXOG EH DSSOLFDEOH WR HYHU\
>@ ³0LFURVHUYLFHVYV0RQROLWKV:KDW¶VWKH5LJKW$UFKLWHFWXUHIRU\RXU
VXEMHFW EXW VKRXOG EH SDUWLFXODUO\ IRFXVHG RQ WKH VXEMHFW 6RIWZDUH"7KRUQ7HFKQRORJLHV´$FFHVVHG2FW>2QOLQH@
3ULQFLSOHVRI2SHUDWLQJ6\VWHPV7KHODVWWKLQJZHZRXOGOLNH $YDLODEOH KWWSVWKRUQWHFKFRPPLFURVHUYLFHVYVPRQROLWKVZKDWV
WRLPSURYHLQWKHIXWXUHLVWKHDGGLWLRQRIOHDUQLQJDFWLYLWLHV ULJKWDUFKLWHFWXUHVRIWZDUH
IRFXVHG RQ WKLV VXEMHFW 6XFKDQ DFWLYLW\ LVDQRQOLQH /LQX[ >@ ³5HSRVLWRU\3DWWHUQLQ$631(7&RUH5(67$3,´$FFHVVHG2FW
WHUPLQDO VR WKDW VWXGHQWV ZRXOG QRW DOZD\V KDYH WR XVH WKH  >2QOLQH@ $YDLODEOH KWWSVZZZSUDJLPWHFKFRPEORJEOD]RU
3XWW\WRROIRUSUDFWLFLQJ%DVKFRPPDQGV2WKHUSRVVLELOLWLHV UHVWDSLUHSRVLWRU\SDWWHUQ
IRUH[WHQGLQJWKHPRGXOHFRXOGEHWRFUHDWHGLIIHUHQWTXL]]HV >@ ³$XWKHQWLFDWLRQ$QG$XWKRUL]DWLRQ,Q$631(7&RUH:HE$3,:LWK
-621 :HE 7RNHQV´ $FFHVVHG 2FW   >2QOLQH@ $YDLODEOH
DVVLJQLQJ SUREOHPV WR VWXGHQWV RU FRQWLQXRXV DVVHVVPHQW RI KWWSVZZZFVKDUSFRUQHUFRPDUWLFOHDXWKHQWLFDWLRQDQG
VWXGHQWVLQVSHFLILFVXEMHFWV DXWKRUL]DWLRQLQDVSQHWFRUHZHEDSLZLWKMVRQZHEWRNHQV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 328

>@ .URPND0)LGHN3.RVWROQ\-'HYHORSLQJ0RGXODU,QIRUPDWLRQ >@ 0 *DQDMRYi , 6RWiNRYi 6 /XNiþ = -HãNRYi 9 -XUNRYi DQG5
6\VWHP ZLWK 0LFURVHUYLFH 7HFKQRORJ\   ,&(7$   WK 2URVRYi³)RUPDWLYHDVVHVVPHQWDVDWRROWRHQKDQFHWKHGHYHORSPHQW
,(((,QWHUQDWLRQDO&RQIHUHQFHRQ(PHUJLQJH/HDUQLQJ7HFKQRORJLHV RI LQTXLU\ VNLOOV LQ VFLHQFH HGXFDWLRQ´ -RXUQDO RI %DOWLF 6FLHQFH
DQG$SSOLFDWLRQVSS (GXFDWLRQ YRO  QR  SS ±  GRL
>@ =DLWVHYD ( /HYDVKHQNR 9 &RQVWUXFWLRQ RI D UHOLDELOLW\ VWUXFWXUH -%6(
IXQFWLRQ EDVHG RQ XQFHUWDLQ GDWD ,((( 7UDQ RQ 5HOLDELOLW\    
SS'2,75
>@ =DLWVHYD ( /HYDVKHQNR 9 5HOLDELOLW\ DQDO\VLV RI 0XOWL6WDWH
6\VWHPDQG0XOWLSOH9DOXHG/RJLF,QWHUQDWLRQDO-RXUQDORI4XDOLW\ 
5HOLDELOLW\0DQDJHPHQW  SS


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 329

 Fitness Tracker Data Extraction and Visualization
Methods
Adam Kotvan∗ , Pavol Helebrandt∗ ,
∗ Faculty of Informatics and Information Technologies
Slovak University of Technology in Bratislava
Bratislava, Slovakia
xkotvan@stuba.sk, pavol.helebrandt@stuba.sk

Abstract—The rapid popularity growth of the fitness tracking can deliver detailed insights into our daily lives that can
industry raises concerns about the security and privacy of the be later used against us in courts, blackmailing, or more
users personal, health and activity information. This ecosystem uncomfortable situations. There are no exceptions for activity
consists of multiple elements such as tracking devices, smart-
phones, the cloud, and their mutual communication, ensuring trackers. Collecting data about our steps, activities, sleep, and
overall data manipulation. With the current age of valuable even location can create a very accurate image of the wearer’s
information, there are emerging concerns regarding the possible daily life.
violations of user’s privacy. Unauthorized access to the data The rest of the paper is organized as follows. Section II
collected by these devices may provide sensitive information
covers current fitness tracking technology and its possible
about the user’s health condition, activity status, and even precise
location. In this paper, we take a closer look at one well known risks. The following section dissects particular data collec-
vendor’s approach to security and privacy mechanisms, data tion and privacy protection mechanisms available for the
collection, and storing strategies. We explore already available Garmin Connect platform. Section IV aims at data storing
methods for data collection and extraction from these devices, and transmission among devices present in the ecosystem.
and utilize them to acquire both current and historical activity
The last sections are dedicated to the FIT files analysis tool
and geolocation records from personal fitness tracker device.
Furthermore, we propose a tool for parsing and visualization implementation, testing dataset collection, findings evaluation,
of the useful information encoded from the collected files. A and conclusion.
proof-of-concept implementation of the proposed tool is evaluated
and compared with other available interpretation applications to
address the quality of obtained knowledge. II. S TATE OF THE A RT
Index Terms—security, privacy, health information
The way of tracking an athlete’s activity and progress in the
past was to log everything manually into the computer. This
I. I NTRODUCTION
process was slow, tedious, and human factors were present,
Movement is a crucial part of our everyday lives. Obesity often resulting in mistakes in the data. Modern tracking de-
and unhealthy lifestyles are getting closer attention as their vices offer plenty of features to monitor users’ daily activities,
prevalence even in still younger population is growing rapidly. interpret results, and measure various health conditions. These
This is partially due to breakthroughs in technology and data are called Personal Fitness Information (PFI) and are
automation that led to a decline in exercise and physical acquired by numerous sensors present in the tracker.
activity combined with heightened caloric intake of consumed
food. The World Health Organization [1] stated, ”Roughly 60 A. Sensors and Data Acquisition
to 85% of people in the world — from both developed and
developing countries — lead sedentary lifestyles, making it The primary source of the user’s daily activity measure-
one of the more serious yet insufficiently addressed public ments are sensors embedded in the tracking device. They are
health problems of our time.” With globally emerging health responsible for regular monitoring of the wearer’s health and
awareness movements, more and more people are reaching out activity status, often without manual intervention. Manufac-
for lifestyle improvement. This is where fitness trackers come turers spend many resources to invent self-learning algorithms
into action, providing easy to use activity monitoring coupled and systems, which focus on collecting and analyzing informa-
with various motivation strategies to boot physical activity. tion automatically in the most efficient form possible, adapting
However, in our information age, any kind of personal to the wearer’s routines, and simultaneously saving as much
information is valuable. Awareness in protecting the data is battery life as possible.
now more critical than ever. With all the possibilities of the 1) Step Counting: Steps are the most natural and acces-
World Wide Web, Social media, the Internet Of Things, and sible activity to everyone. For this reason, the history of
rapid technological progress, we share more than we realize. step counting dates back to 1960 [2]. Today, the technology
There are many potential threats that can take advantage of of accelerometers is used to determine walking in modern
our mindless personal data sharing. Modern dissection tools tracking devices.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 330

 2) Heart Rate: The second essential information for gain- Today, even the most budget-oriented Garmin devices,
ing insight into the wearer’s health is heart rate. According equipped with heart rate monitoring, support constant wrist-
to [3], the most common method for reliable heart rate based measurements. It delivers the wearer’s accurate heart
measurements is Optical Heart Rate Monitoring (OHRM), rate frequency in real-time.
which is continuous noninvasive monitoring based on shining Sleep summary presents valuable insights into the user’s
green light through the user’s skin. sleep pattern, including sleep stages, falling asleep and wak-
3) Location: When performing an outdoor activity, location ing up time, pulse oximetry, respiration statistics, and body
is crucial for recording and subsequent evaluation. It offers movement.
precise data for calculating the user’s movement speed, dis- A unique feature implemented but usually available to
tance traveled, and building route interpretation. middle and high-end price category devices is stress analysis.
4) Other measurements: For example, sleep, burned calo- Sensors constantly collect data about heart rate and then use
ries during the day, but foremost, activity evaluations are them to determine and assess the wearer’s stress intensity
calculated from various combinations of data sources derived throughout the day.
from those described above. Calorie management is a valuable tool mainly for weight
control ambitions but can be practical for everyone. Based
B. Privacy Risks mainly on the number of steps taken and workouts performed,
Rapid worldwide growth of fitness tracker popularity, new the system calculates the calories burned during the day.
brands, and products constantly emerging raise significant Activity tracking presents the best option for obtaining
concerns for customer’s privacy and information security. knowledge about performed exercises. This feature evaluates
Every manufacturer addresses precautions differently and the the activity, emphasizing the user’s speed, location, heart
protection systems implemented are diverse. With regular data rate, and even blood oxygenation levels. Typically, there are
collection and transfer to the smartphone device or internet numerous activities, each designed for the specific exercise
cloud, third parties can exploit weak spots for obtaining users’ or sport (running, cycling, swimming, pilates, yoga, cardio).
PFI without their knowledge, as was shown by Mendoza Tracking activity consists of more input sources, which are
et al. in [4]. Practical demonstration of forensic collection processed and evaluated concurrently.
of PFI from various fitness tracker devices, and its relative C. Privacy and Security Mechanisms
simplicity due to different levels of security implemented by
manufacturers was successfully demonstrated by both Hantke Fitness applications associated with the tracking devices
[5] and Kang [6]. often create a central hub for all users to connect and share
various activity results. It is crucial to provide options for pri-
III. G ARMIN C ONNECT vacy, sharing patterns, and safety measurements customization
with all the data-sharing possibilities. There would be no need
Garmin Connect is the main accompanying application for
for data extraction if we shared everything mindlessly.
all tracking devices manufactured by Garmin. Our goal is to
In the user profile, everyone can manage information shown
get insight into Garmin’s privacy and security approach for its
to other community members. With a simple switch, we can
devices and the app itself.
decide to set them visible or hidden.
A. Setup and Personal Information Options in the community management are focused on
others’ access to our profile, activity evaluations, and achieve-
Each user needs to create a personal account to use the ments. We can set them private or accessible to only friends
application. Email address, password, and name representing or groups.
the user in the app are required. For Garmin Connect to For further privacy control, there is an option to block other
work correctly, we must fill in additional personal information, users. They will not be able to send messages or view our
which the system uses as complementary data for activity eval- profile.
uation. These include gender, birthdate, age, height, weight,
and activity class. IV. G ARMIN S ECURITY AND DATA E XAMINATION
This section will analyze data storing methods in particular
B. Data Collection
devices, clouds, and their security level. Users usually do not
Garmin does not fall behind and provides all sought-after access their data in raw form and have little to no control over
and widely used self-tracking features to their customers. Each their storage and synchronization process. We will look at what
measurement category has insights, graphs, and evaluations mechanisms are present for ensuring data confidentiality and
that users can view, analyze, or even share. the possibility of getting access to that private information.
Steps are counted automatically when the tracking device
registers movement. Usually, they are the most prominent A. Flexible and Interoperable Data Transfer Protocol
thing, represented on the tracker’s display and in the Garmin ”Data Transfer (FIT) protocol is a format designed specif-
Connect app. Users are encouraged to hit their step goal as ically for the storing and sharing of data that originates from
the primary daily objective. sport, fitness and health devices. It is specifically designed

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 331

to be compact, interoperable and extensible.” as described by 2) ANT: It is a wireless communication protocol for low-
Garmin [7]. power networking applications. ”ANT is perfectly suited for
This protocol defines a predetermined file format (also any kind of low data rate sensor network topologies – from
called a FIT message) for storing activity information collected peer-to-peer or star, to practical mesh – in personal area
by the sensors of the tracking devices. All FIT-compatible networks (PANs) that are well-suited for sports, fitness, well-
systems can interpret FIT messages originating from any other ness, and home health applications.” as stated by Gupta in
FIT-compatible device or system. [10]. More advanced extensions are ANT+, and a version
implemented explicitly for FIT Protocol, dedicated mainly to
B. Tracking Device Garmin devices ANT-FS, both of which add an interoperability
feature. As described by Fahmy in [11], the main design
We explored measurements stored in the Garmin Fore- resides in low power cost micro-controllers and transceivers
runner 245 smartwatch. Collected data are categorically dis- operating in the 2.4 GHz frequency.
tributed to a total of 29 folders, and the majority of files According to the official webpage [12], ANT provides an 8-
are of the .fit file format. Contents are not human-readable, byte network key and 128-AES encryption for both master and
but Garmin provides FitCSVTool for converting them to .csv slave channels. The ANT core supports the implementation
files viewable in any spreadsheet editor. We can find device of authentication and encryption and is natively available to
information, for example, serial number and software versions, ANT-FS.
followed by user’s settings concerning language, time zone,
unit formats, and personal information like height, weight, D. Smartphone
gender, and activity class. Activity files contain all informa-
After obtaining data from the tracker’s sensors and suc-
tion gathered during an activity - heart rate, cadence, speed,
cessful transfer via BLE or ANT, they are processed in a
timestamps, location, and total results. Sleep data are also
smartphone which generally serves as the primary display
present but need further interpretation to be understandable.
device. Its purpose is to be the central hub for the data
Furthermore, according to MacDermott et al. [8], deleted files
interpretation, displaying, user input, and communication with
can be found in the tracker’s unallocated space. This raises a
the web servers.
concern about retrieving data that users purposefully deleted.
All data related to the application are located
It may contain sensitive health information or convicting case
in the internal smartphone storage and directory
evidence. However, not all removed files were retrievable.
com.garmin.android.apps.connectmobile. We can access
various cache files in two folders, an empty debug directory
C. Connectivity and a profile picture set by the user in the app. There is no
Most Garmin fitness trackers use either Bluetooth Low sign of any data related to activities or other measurements.
Energy (BLE) or ANT+ for smartphone communication and We can conclude that the smartphone is used as a com-
data transfer. Their purpose is to secure the lowest energy munication station to upload collected data to Garmin’s web
cost transmission possible. Later in our work, we will try servers and retrieve them for subsequent viewing. The internet
to capture and analyze the transmitting data using a sniffing connection requirement for displaying already synchronized
device. and viewed data supports the fact that they are not present in
the device’s memory storage.
1) Bluetooth Low Energy: BLE is a version of Bluetooth Having the collected data stored on as few devices as possi-
focusing on low energy consumption. It is broadly used in the ble mitigates the risk of unauthorized access and their misuse.
IoT domain, featuring low power consumption data transfer In our case, data are temporarily stored in the tracker and
and application control among IoT devices. It fits perfectly uploaded to the webserver for the long term. The smartphone
for fitness trackers due to their battery capacity and storage is solely used for transfer and viewing.
limitations. Communication between devices is established
over a radio link in a 2.4 GHz frequency. E. Network
The authors in [9] demonstrated various options applied for The last procedure to officially accept collected data is
securing the information exchange between two interconnected synchronizing them with the web servers. It requires an
devices. We can differentiate them into two modes. The internet connection initiated by a smartphone or tracker with
first supports encryption and authentication using the CCM imbued Wi-Fi module. It is the definitive storage point for the
algorithm and 128-bit AES cipher. The second is 12-byte user’s fitness data.
signature control when transmitting data over an unencrypted Any web browser and an internet connection are required
connection. to view our data. We head to the connect.garmin.com domain
BLE also implements the ability to frequently change which serves for the mobile app download instructions or
private addresses, called frequency hopping, eliminating the account management. After successful login, we are presented
threat of tracking the given device. with a general overview of the most recently synchronized
data. They are fetched from Garmin’s servers, thus identical to

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 332

the data viewed through a mobile device. The actual showcase
shares many similarities with the smartphone application.
In the case of approved data extraction when using the web
browser, every user has an option of exporting a selected
activity evaluation. We can choose from different formats
available - the original .fit file, .tcx, .gpx, Google Earth or
.csv file.
Unauthorized entry to these data is remarkably different
compared to tracker due to lack of physical access points.
According to analysis in [13], the Garmin fitness trackers
use HTTPS when sending and receiving data packets, but the
actual payload is not encrypted. They successfully sniffed the
communication and modified the transmitted .fit file resulting
in the upload of 80 million steps to the server without raising
any error. Users have limited control over their fitness data
storing and protection mechanisms regarding the webservers,
leaving them dependent on the platform developer.
V. DATA A NALYSIS T OOL
In the follow-up to the analysis, our goal is to implement
our standalone tool for Garmin’s FIT file interpretations. We
aim to try available data extraction methods from Garmin’s
fitness tracker and procedures related to their transmission
and storage. Our tool will analyze such data and process any
valuable knowledge and conclusions available.
1) Functional Requirements: The tool should be easily Fig. 1. User’s settings and information located in the FIT file
executable without the need for additional software installation
for all .fit files acquired with any method and from any source.
If present, the tool can specify data type(for example, sleep
or heart rate), activity, address timestamps, measurements,
and draw meaningful knowledge that will be presented in an
understandable form. Any additional functionality should be
easily expandable.
2) Non-functional Requirements: The solution runs on the
Windows operating system, is intuitive with minimal instruc- Fig. 2. Inspection of the recorded measurements
tions to understand its functionality, and the graphical interface
is straightforward.
3) Design proposal: We went through the options of ac-
quiring data from a personal fitness tracker and its peripheral offer regular lap evaluations after crossing a certain distance
ecosystem, finding out that Garmin’s FIT Protocol is used threshold. If possible, the lap data are interpreted and marked
for data representation during storing and transmission. Un- on the map. A different type of activity is strength training
derstanding the .fit activity data requires additional effort to which divides measurements into sets depending on the user’s
transform them into human-readable format. Creating our tool input. These are categorized as active or resting and need an
for gathering practical knowledge will show that anyone can individual approach as with laps. The primary analyzed data
repeat the same process and gain an advantage over assessed depend on the activity type but are most commonly comprised
information. of heart rate, cadence, speed, traveled distance, location, and
The FIT analysis tool will be implemented in Python 3 timestamps.
programming language. In order to work with the .fit files After obtaining a FIT file by any form and uploading it into
a Python library Fitparse [14] will be used. We will use the tool, the contents will be parsed and analyzed by the type
multiple libraries such as matplotlib, seaborn, and pandas for of data present. All salvaged information will be presented
data processing, interpretation, and visualization. Viewable and in the best format possible, for example, graphs, tables, or
interactable GUI will be created using the tkinter [15] toolkit. messages.
Our main goal is to analyze any activity embedded into We will compare our results with different forensic tools for
a FIT file by any Garmin fitness tracking device. The most FIT file dissection, including Garmin’s official smartphone or
common are different types of running, for example, outdoor web browser applications. Last but not least, we will discuss
or indoor, cycling, swimming, and walking. Some activities the quality and amount of helpful information gathered.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 333

 Fig. 4. Power output during running session in Watts

Fig. 3. Running route recreation

owner, including height, weight, usual sleep and wake time,

VI. R ESULTS AND D ISCUSSION language setting, and more.
Many mathematical features can be leveraged for processing
During the initial implementation phase of our tool, we
the data grouped in the already created data frame. For
went through all the possible methods for obtaining testing
example, the total number of records, average, minimal, or
datasets. As dissected in the previous sections, we found ac-
maximal values of each data type.
tivity files in the tracker’s internal memory and Garmin’s web
servers. The smartphone performs the role of a complementary Records regarding the user’s position during an activity can
communication station, and no data is located in its storage. be put together to reconstruct the precise movement route.
Nonetheless, we have not yet tested the security levels of the The location measurements are collected in pairs consisting of
data transmissions. latitude and longitude in semicircle units. In order to display
1) Dataset Gathering: Our first set of data used for testing activity routes on a map, we have to transform them into de-
the tool was obtained directly from the tracking device within grees. Each such measurement is processed and subsequently
the analysis. Compared with the files available for exporting marked on a map.
from Garmin’s web application, they were identical. Whereas after a new lap is triggered, information about the
For sniffing the Bluetooth communication, we used Wire- user’s location is also memorized and marked on the map with
shark interconnected with a sniffing device from Nordic a blue point.
Semiconductor [16], capable of capturing BLE and ANT
Visual representations are critical for better insight into the
transmissions. However, only advertising packets were caught
collected data and understanding them correctly. The most
without any activity information or data. This is caused by
prominent information is displayed in the form of graphs. For
dynamically changing the device’s MAC address, also called
example, heart rate, elevation, power output, cadence, burned
frequency hopping, as discussed in the previous section. Only
calories during a lap or working to resting ratio during strength
the smartphone and tracker possess information about the
training.
oncoming hop. Theoretically, we should be able to pinpoint the
device’s address but only for a brief period until the following Note in 3 power output of a runner during outdoor running.
change occurs and then manually find it again. Nevertheless, The tracker does not measure these data. We have to calculate
the fast frequency of the hops does not allow to achieve that. them using enhanced speed and runner’s weight measurements
2) Analysis: Implementation of our tool is still in progress. in the activity file. Last is the EOCR(Energy Cost of Running),
However, we can already draw the first results and evaluations. a constant value specifying an athlete’s energy consumption
We analyzed our testing dataset obtained from the Garmin while running in kiloJoules per kilogram of body weight per
Forerunner 245 tracking device. kilometer. We use the value of 1.04, including the resistance
The tool reads all messages embedded into the FIT file, of the running wind.
calculates necessary modifications, and distributes them into a Running cadence is calculated from rounds per minute
data frame, which is later used for their graphic interpretation. measurements. In order to get to steps per minute, we have to
Records are distributed into different categories, each grouping double the initial value with added fractional cadence which
certain measurements. For example, one of the initial records can reach two values, either 0 or 0,5. It will guarantee even
in the file contains various information about the device’s or odd cadence values instead of all even.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 334

 TABLE I
F EATURE COMPARISON OF OWN SOLUTION FIT ANALYSIS WITH NATIVE APPS

FIT analysis tool Garmin Connect FIT file viewer

theadDirect measurements yes yes yes
theadBMI calculation yes yes yes
Records
theadGraphical visualizations yes yes no
theadPower output calculation yes no no
Location
theadMovement route recreation yes yes no
theadMap export html format GPX GPX
Laps
theadData presentation graphical tabular tabular
Sets
theadData presentation graphical graphical & tabular tabular

VII. C ONCLUSION ucts not thoroughly explored. Further exploration of privacy

The main contribution of this paper is to point out the wide challenges and possible solutions is still required.
range of data collected by the fitness tracking devices and ACKNOWLEDGMENT
address the potential personal insights that a third party can
withdraw from them and possibly abuse without the user’s This publication has been written thanks to the support of
knowledge. We focused on available data extraction methods the Operational Programme Integrated Infrastructure for the
from the Garmin fitness tracker together with its peripheral project: Advancing University Capacity and Competence in
ecosystem and proposed a tool for the collected data interpre- Research, Development and Innovation (ACCORD) (ITMS
tation. Our primary observation was the significant progress code: 313021X329), co-funded by the European Regional De-
in the safety mechanisms utilized to ensure users’ protection velopment Fund (ERDF). This work was supported by Cultural
and privacy. Unlike a few years ago, today is data packet and Educational Grant Agency (KEGA) of the Ministry of
transmission among devices secured with mechanisms like Education, Science, Research and Sport of the Slovak Republic
frequency hopping, making third-party intervention difficult. under the project No. 026TUKE-4/2021.
Options available to the users of the Garmin Connect platform R EFERENCES
allow them to manage their profile information and activity
measurements by various settings and security mechanisms. [1] W. H. Organization, “Physical inactivity a leading cause of disease and
disability, warns who,” 2002. Last accessed 23 November 2021.
The FIT analysis tool implementation demonstrates the vast [2] D. R. Bassett, L. P. Toth, S. R. LaMunion, and S. E. Crouter, “Step
amount of data collected by the tracking devices and the counting: a review of measurement considerations and health-related
possibility of getting a severe image of a person’s private life. applications,” Sports Medicine, vol. 47, no. 7, pp. 1303–1315, 2017.
[3] M. B. Mashhadi, E. Asadi, M. Eskandari, S. Kiani, and F. Marvasti,
Various calculations and processing methods can gain insight “Heart rate tracking using wrist-type photoplethysmographic (ppg) sig-
into their health condition, performance, place of residence, nals during physical exercise with simultaneous accelerometry,” IEEE
or even daily routines and sleeping patterns. The tool also Signal Processing Letters, vol. 23, no. 2, pp. 227–231, 2015.
[4] F. A. Mendoza, L. Alonso, A. M. López, and D. D. S. Patricia
provides straightforward modules for visualization of the col- Arias Cabarcos, “Assessment of fitness tracker security: a case of study,”
lected data. While accessing user’s activity files is somewhat in Multidisciplinary Digital Publishing Institute Proceedings, vol. 2,
challenging, it is not impossible. In the culture of mindless data p. 1235, 2018.
[5] F. Hantke and A. Dewald, “How can data from fitness trackers be
sharing common today, many individuals do not realize the obtained and analyzed with a forensic approach?,” in 2020 IEEE
potential threats data gathered by the devices they use every European Symposium on Security and Privacy Workshops (EuroS&PW),
day. If some third party manages to acquire a person’s FIT pp. 500–508, 2020.
[6] S. Kang, S. Kim, and J. Kim, “Forensic analysis for iot fitness trackers
files, thanks to thorough documentation and available libraries, and its application,” Peer-to-Peer Networking and Applications, vol. 13,
potential adversary can parse useful personal and even health no. 2, pp. 564–573, 2020.
information quite easily. [7] G. LTD, “Flexible and interoperable data transfer (fit) protocol,” 2021.
Last accessed 6 December 2021.
There is potential opportunity for more information extrac- [8] Á. MacDermott, S. Lea, F. Iqbal, I. Idowu, and B. Shah, “Forensic
tion from FIT data. It would be beneficial to resolve the analysis of wearable devices: Fitbit, garmin and hetp watches,” in 2019
sleep analysis files, heart rate zones evaluation, or general 10th IFIP International Conference on New Technologies, Mobility and
Security (NTMS), pp. 1–6, IEEE, 2019.
GUI overhaul. With the ongoing development of the activity [9] C. Gomez, J. Oller, and J. Paradells, “Overview and evaluation of
tracking industry technologies, the rise in the amount of data bluetooth low energy: An emerging low-power wireless technology,”
collected. Sensors, vol. 12, no. 9, pp. 11734–11753, 2012.
[10] N. K. Gupta, Inside Bluetooth low energy. Artech House, 2016.
Overall the constant evolution of the fitness tracking market [11] H. M. A. Fahmy, Wireless sensor networks: concepts, applications,
and new trend-setting leave the data protection of many prod- experimentation and analysis. Springer, 2016.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 335

[12] G. C. Inc., “What kind of security does ant provide?,” 2021. Last
accessed 11 December 2021.
[13] H. Fereidooni, T. Frassetto, M. Miettinen, A.-R. Sadeghi, and M. Conti,
“Fitness trackers: fit for health but unfit for security and privacy,”
in 2017 IEEE/ACM International Conference on Connected Health:
Applications, Systems and Engineering Technologies (CHASE), pp. 19–
24, IEEE, 2017.
[14] Dtcooper, “Python fitparse.” Last accessed 2 January 2022.
[15] P. S. Foundation, “tkinter — python interface to tcl/tk.” Last accessed
2 January 2022.
[16] N. Semiconductor, “nrf52840 dongle.” Last accessed 3 April 2022.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 336

 $XWRPDWHGHQYLURQPHQWWRVXSSRUWFRPSHWLWLRQ
LQFRPSXWHUQHWZRUNLQJ
6.XELQVNê 3)HFLĐDN 00LFKDONR )-DNDE 
'HSDUWPHQWRI&RPSXWHUVDQG,QIRUPDWLFV.RãLFH6ORYDNLD
6DPXHO.XELQVN\#VWXGHQWWXNHVN3HWHU)HFLODN#FQOVN0LURVODY0LFKDONR#FQOVN)UDQWLVHN-DNDE#FQOVN


$EVWUDFW² 7KLV SDSHU LQWURGXFHV D QRYHO DSSURDFK WR GLVFXVVHG DV NH\ DUHDV IRU LPSURYHPHQW DQG IXWXUH
HQJDJH KLJK VFKRRO VWXGHQWV LQ QHWZRUNLQJ DVVLJQPHQWV HQKDQFHPHQWV
WKURXJK DZHERULHQWHGHQYLURQPHQW7KH SURSRVHGV\VWHP
VHUYHV DV D WDVN GLVWULEXWRU DQG OLYH UDQNLQJ SODWIRUP ,, 352%/(0'(),1,7,21
IDFLOLWDWLQJ D FRPSHWLWLYH DQG LQWHUDFWLYH OHDUQLQJ
H[SHULHQFH %\ LQWURGXFLQJ D QHWZRUNLQJ DVVLJQPHQW 7KH ODQGVFDSH RI QHWZRUNLQJ FRPSHWLWLRQV KDV IDFHG
FRPSHWLWLRQ ZH DLP WR IRVWHU LQWHUHVW HQKDQFH VNLOOV DQG VLJQLILFDQW FKDOOHQJHV LQ UHFHQW \HDUV (VWDEOLVKHG
LQVSLUHWKHQH[WJHQHUDWLRQ RI QHWZRUNHQJLQHHUV7KHZHE FRPSHWLWLRQV VXFK DV 1HW5LGHUV RU -1HW ZKLFK RQFH
EDVHGSODWIRUPHQFRXUDJHVFROODERUDWLRQSURYLGHVUHDOWLPH SURYLGHG D SODWIRUP IRU QHWZRUNLQJ HQWKXVLDVWV KDYH
IHHGEDFN DQG DOORZV VWXGHQWV WR VKRZFDVH WKHLU DELOLWLHV EHFRPHLQDFWLYHDQGLQDOOOLNHOLKRRGGLVFRQWLQXHG7KLV
ZKLOHSURPRWLQJKHDOWK\FRPSHWLWLRQDPRQJVFKRROV DEVHQFHRISURPLQHQWFRPSHWLWLRQV KDVOHIWDYRLGLQWKH
QHWZRUNLQJ FRPPXQLW\ OLPLWLQJ RSSRUWXQLWLHV IRU KLJK
.H\ZRUGV² FRPSXWHU QHWZRUNV FRPSHWLWLRQ DVVLJQPHQW VFKRRO VWXGHQWV WR VKRZFDVH WKHLU VNLOOV DQG HQJDJH LQ
&&1$&LVFR3DFNHW7UDFHUZHESDJH7\SH6FULSW1H[WMV
KHDOWK\FRPSHWLWLRQ

, ,1752'8&7,21 0RUHRYHU WKH &29,' SDQGHPLF IXUWKHU
,Q WRGD\ V LQWHUFRQQHFWHG ZRUOG FRPSXWHU QHWZRUNV H[DFHUEDWHG WKH VLWXDWLRQ 0DQ\ WUDGLWLRQDO FRPSHWLWLRQV
VHUYH DV WKH EDFNERQH RI PRGHUQ FRPPXQLFDWLRQ DQG ZHUHIRUFHGWRFDQFHORUPRGLI\WKHLUIRUPDWVWRFRPSO\
LQIRUPDWLRQ H[FKDQJH V\VWHPV $V WKH GHPDQG IRU ZLWKKHDOWKDQGVDIHW\UHJXODWLRQV7KLVDEUXSWVKLIWSRVHG
QHWZRUNLQJ SURIHVVLRQDOV FRQWLQXHV WR JURZ LW EHFRPHV FKDOOHQJHVIRURUJDQL]HUVSDUWLFLSDQWVDQGHGXFDWRUVZKR
LPSHUDWLYHWRIRVWHUDQLQWHUHVWLQQHWZRUNLQJDPRQJKLJK UHOLHG RQ WKHVH FRPSHWLWLRQV WR IRVWHU QHWZRUNLQJ
VFKRRO VWXGHQWV ZKR UHSUHVHQW WKH QH[W JHQHUDWLRQ RI HGXFDWLRQ DQG FROODERUDWLRQ :KLOH VRPH FRPSHWLWLRQV
WHFKQRORJLVWVDQGHQJLQHHUV>@ PDQDJHG WR DGDSW DQG XSGDWH WKHLU IRUPV HIIHFWLYHO\
 RWKHUV VWUXJJOHG WR WUDQVLWLRQ WR DQ RQOLQH VHWWLQJ
7KLVSDSHUDLPVWRLQWURGXFHDQHWZRUNLQJDVVLJQPHQW UHVXOWLQJLQGLPLQLVKHGH[SHULHQFHVIRUSDUWLFLSDQWV
FRPSHWLWLRQ IRU KLJK VFKRRO VWXGHQWV DFFRPSDQLHG E\ D 
ZHERULHQWHG HQYLURQPHQW WKDW DFWV DV D WDVN GLVWULEXWRU ,QOLJKWRIWKHVHFLUFXPVWDQFHVWKHUHLVDSUHVVLQJQHHG
DQG OLYH UDQNLQJ V\VWHP %\ DFKLHYLQJ WKHVH REMHFWLYHV IRU D QHZ QHWZRUNLQJ FRPSHWLWLRQ WKDW LV VSHFLILFDOO\
ZH DLP WR SURYLGH DQ LQWHUDFWLYH SODWIRUP WKDW SURPRWHV GHVLJQHG IRU WKH RQOLQH HQYLURQPHQW %\ OHYHUDJLQJ WKH
FROODERUDWLRQ IDFLOLWDWHV NQRZOHGJH VKDULQJ DPRQJ DGYDQFHPHQWV LQ WHFKQRORJ\ DQG WKH ZLGHVSUHDG
VFKRROV DQG VKRZFDVHV VWXGHQWV  QHWZRUNLQJ DELOLWLHV DYDLODELOLW\ RI LQWHUQHW FRQQHFWLYLW\ DQ RQOLQHEDVHG
7KLVSODWIRUPDLPVWREULGJHWKHJDSEHWZHHQWKHRUHWLFDO
FRPSHWLWLRQ FDQ RYHUFRPH JHRJUDSKLFDO OLPLWDWLRQV
NQRZOHGJHDQGSUDFWLFDODSSOLFDWLRQHPSRZHULQJVWXGHQWV
WRSXUVXHQHWZRUNLQJUHODWHGFDUHHUV>@ HQJDJHDODUJHUSDUWLFLSDQWSRRODQGSURYLGHDFRQVLVWHQW
SODWIRUP IRU VWXGHQWV WR GHPRQVWUDWH WKHLU QHWZRUNLQJ

NQRZOHGJHDQGVNLOOV
$GGLWLRQDOO\ ZH ZLOO GLVFXVV WKH VLJQLILFDQFH RI XVHU 
H[SHULHQFH DQG LQWHUIDFH GHVLJQ KLJKOLJKWLQJ WKH
LPSRUWDQFH RI LQWXLWLYH DQG XVHUIULHQGO\ LQWHUIDFHV WKDW ,,, 7(&+12/2*<29(59,(:
HQKDQFH VWXGHQW HQJDJHPHQW DQG LQWHUDFWLRQ ZLWK WKH
SODWIRUP>@7RHYDOXDWHWKHHIIHFWLYHQHVVRIWKHSURSRVHG 7R FUHDWH DQ HQJDJLQJ DQG FRPSUHKHQVLYH QHWZRUNLQJ
V\VWHPZHZLOOFRQGXFWSLORWWHVWLQJDQGJDWKHUIHHGEDFN DVVLJQPHQW FRPSHWLWLRQ IRU KLJK VFKRRO VWXGHQWV WKH
IURP SDUWLFLSDWLQJ VFKRROV DQG VWXGHQWV 7KLV HYDOXDWLRQ LPSOHPHQWDWLRQRIYDULRXVWHFKQRORJLHVZDVQHFHVVDU\,Q
ZLOO SURYLGH YDOXDEOH LQVLJKWV LQWR WKH LPSDFW RI WKH WKLVVHFWLRQZHSURYLGHDQRYHUYLHZRIWKHWHFKQRORJLHV
SODWIRUP RQ VWXGHQW PRWLYDWLRQ OHDUQLQJ RXWFRPHV DQG XWLOL]HGLQWKHGHYHORSPHQWRIWKHZHEDSSHQFRPSDVVLQJ
DFDGHPLFSHUIRUPDQFH>@ ERWKWKHIURQWHQGDQGEDFNHQGFRPSRQHQWV

$ 1HWZRUNLQJWDVNVDQG&&1$&XUULFXOXP
)XUWKHUPRUH ZH ZLOO LGHQWLI\ DQG DGGUHVV WKH
FKDOOHQJHV HQFRXQWHUHG GXULQJ WKH GHYHORSPHQW 7KH WDVNV ZLWKLQ WKH FRPSHWLWLRQ ZHUH FDUHIXOO\
7HFKQLFDO FKDOOHQJHV VFDODELOLW\ FRQVLGHUDWLRQV GHVLJQHGEDVHGRQWKH&LVFR&HUWLILHG1HWZRUN$VVRFLDWH
LQWHJUDWLRQ ZLWK H[LVWLQJ QHWZRUNLQJ FXUULFXOD ZLOO EH &&1$  FXUULFXOXP >@ %\ OHYHUDJLQJ WKH &&1$

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 337

FXUULFXOXPZHHQVXUHGWKDWWKHDVVLJQPHQWVDOLJQHGZLWK DXGLHQFH RI KLJK VFKRRO VWXGHQWV LQFRUSRUDWLQJ PRGHUQ
LQGXVWU\ VWDQGDUGV DQG FRYHUHG HVVHQWLDO QHWZRUNLQJ DQGLQWXLWLYHGHVLJQSULQFLSOHV>@
FRQFHSWV (DFK DVVLJQPHQW FRQVLVWHG RI D ZULWWHQ
% 6HFXULW\0HDVXUHV
FRPSRQHQW ZKLFK RXWOLQHG WKH WDVN UHTXLUHPHQWV DQG D
SUDFWLFDO FRPSRQHQW ZKLFK ZDV LPSOHPHQWHG XVLQJ WKH 6HFXULW\ SOD\HG D SLYRWDO UROH LQ WKH LPSOHPHQWDWLRQ
&LVFR 3DFNHW 7UDFHU QHWZRUN VLPXODWLRQ WRRO >@ 7KLV RI WKH ZHE SDJH 5HFRJQL]LQJ WKH LPSRUWDQFH RI
FRPELQDWLRQ SURYLGHG VWXGHQWV ZLWK D FRPSUHKHQVLYH VDIHJXDUGLQJ VHQVLWLYH XVHU LQIRUPDWLRQ DQG PDLQWDLQLQJ
OHDUQLQJ H[SHULHQFH DOORZLQJ WKHP WR DSSO\ WKHRUHWLFDO WKHLQWHJULW\RIWKHFRPSHWLWLRQZHLPSOHPHQWHGVHYHUDO
NQRZOHGJHWRSUDFWLFDOVFHQDULRV UREXVW VHFXULW\ PHDVXUHV 7KHVH PHDVXUHV LQFOXGHG WKH
LPSOHPHQWDWLRQ RI VHFXUH DXWKHQWLFDWLRQ DQG
% )URQWHQG7HFKQRORJLHV DXWKRUL]DWLRQ PHFKDQLVPV $GGLWLRQDOO\ DOO
7R LPSOHPHQW WKH IURQWHQG RI WKH ZHE SDJH ZH FRPPXQLFDWLRQV EHWZHHQ WKH IURQWHQG DQG EDFNHQG
HPSOR\HG 7\SH6FULSW 1H[WMV DQG 7DLOZLQG &66 V\VWHPV ZHUH VHFXUHG XVLQJ +7736 +\SHUWH[W 7UDQVIHU
7\SH6FULSW HQKDQFHG WKH GHYHORSPHQW SURFHVV E\ 3URWRFRO6HFXUH WRSURWHFWGDWDLQWUDQVLW%\SULRULWL]LQJ
SURYLGLQJW\SHVDIHW\DQGLPSURYHGFRGHPDLQWDLQDELOLW\ VHFXULW\ZHDLPHGWRSURYLGHDWUXVWZRUWK\SODWIRUPIRU
>@1H[WMVHQDEOHGVHUYHUVLGHUHQGHULQJHQVXULQJIDVW SDUWLFLSDQWVDQGHQVXUHWKHFRQILGHQWLDOLW\DQGSULYDF\RI
DQGHIILFLHQWSDJHORDGV>@$GGLWLRQDOO\7DLOZLQG&66 WKHLUGDWD
IDFLOLWDWHG UDSLG DQG FRQVLVWHQW VW\OLQJ E\ SURYLGLQJ D
& 5ROH%DVHG$FFHVV&RQWURO
FRPSUHKHQVLYH VHW RI SUHEXLOW FRPSRQHQWV DQG XWLOLW\
FODVVHV 7RPDQDJHGLIIHUHQWOHYHOVRIDFFHVVDQGSHUPLVVLRQV
 ZH LPSOHPHQWHG D UROHEDVHG DFFHVV FRQWURO 5%$& 
V\VWHP ZLWKLQ WKH ZHE DSS 7KLV V\VWHP DOORZHG IRU
& %DFNHQG7HFKQRORJLHV JUDQXODU FRQWURO RYHU XVHU SULYLOHJHV DQG HQVXUHG WKDW
7KH EDFNHQG RI WKH DSS ZDV LPSOHPHQWHG XVLQJ D XVHUV RQO\ KDG DFFHVV WR WKH SDJHV DQG IXQFWLRQDOLWLHV
FRPELQDWLRQ RI WHFKQRORJLHV 7KH SULPDU\ EDFNHQG UHOHYDQW WRWKHLUUROHV:HHPSOR\HGWKUHHGLVWLQFWW53&
IUDPHZRUN ZDV W53& D QH[WJHQHUDWLRQ WUDQVSRUW 53& URXWHUVWRIDFLOLWDWHWKLV5%$&DSSURDFK7KHILUVWURXWHU
5HPRWH 3URFHGXUH &DOO  IUDPHZRUN 3ULVPD 250 KDQGOHG SXEOLF SDJHV SURYLGLQJ DFFHVV WR FRPSHWLWLRQ
VHUYHGDVDGDWDEDVHWRRONLWHQDEOLQJVHDPOHVVLQWHUDFWLRQ GHWDLOVDQGLQIRUPDWLRQ7KHVHFRQGURXWHUZDVGHGLFDWHG
ZLWK WKH XQGHUO\LQJ 3RVWJUH64/ GDWDEDVH $GGLWLRQDOO\ WR ORJJHGLQ XVHUV HQDEOLQJ UHJLVWHUHG SDUWLFLSDQWV WR
1H[W$XWKMV SURYLGHG VHFXUH XVHU DXWKHQWLFDWLRQ DQG DFFHVV DVVLJQPHQWV VXEPLW WKHLU ZRUN DQG YLHZ WKHLU
VHVVLRQPDQDJHPHQWIHDWXUHV UDQNLQJV7KHWKLUGURXWHU ZDVUHVHUYHG IRUDGPLQ XVHUV
 JUDQWLQJ DGPLQLVWUDWRUV DFFHVV WR DQ H[FOXVLYH DGPLQ
SDQHO ZKHUH WKH\ FRXOG PDQDJH FRQWHVW GHWDLOV DVVLJQ
' 'DWDEDVHPDQDJHPHQW
WDVNV DQG PRQLWRU SDUWLFLSDQW DFWLYLW\ 7KLV 5%$&
3RVWJUH64/ VHUYHG DV WKH EDFNHQG GDWDEDVH IRU V\VWHP HQVXUHG WKDW WKH FRPSHWLWLRQ RSHUDWHG VPRRWKO\
VWRULQJ XVHU LQIRUPDWLRQ DVVLJQPHQW GHWDLOV DQG DQGVHFXUHO\PDLQWDLQLQJWKHLQWHJULW\RIWKHHYHQW>@
FRPSHWLWLRQUDQNLQJV3RVWJUH64/RIIHUHGH[FHOOHQWGDWD
LQWHJULW\ VFDODELOLW\ DQG SHUIRUPDQFH FDSDELOLWLHV ' $GPLQLVWUDWLRQSDQHO
HQVXULQJWKHHIILFLHQWKDQGOLQJRIKLJKYROXPHVRIGDWD 7KH LQFOXVLRQ RI DQ DGPLQ SDQHO ZDV D FULWLFDO
 FRPSRQHQW RI WKH SODWIRUP HPSRZHULQJ DGPLQLVWUDWRUV
ZLWK DQ LQWXLWLYH LQWHUIDFH WR HIILFLHQWO\ PDQDJH YDULRXV
,9 ,03/(0(17$7,21 DVSHFWV RI WKH FRPSHWLWLRQ 7KH DGPLQ SDQHO SURYLGHG D
,Q WKLV VHFWLRQ ZH SUHVHQW WKH GHWDLOHG VROXWLRQ DQG FHQWUDOL]HG FRQWURO KXE IRU DGPLQLVWUDWRUV WR DGG RU
LPSOHPHQWDWLRQ SURFHVV RI WKH QHWZRUNLQJ DVVLJQPHQW PRGLI\ DVVLJQPHQW GHWDLOV FUHDWH QHZ WDVNV PRQLWRU
FRPSHWLWLRQ DQG WKH ZHERULHQWHG HQYLURQPHQW :H SDUWLFLSDQW SURJUHVV DQG HQVXUH WKH IDLU DQG HIIHFWLYH
GHVFULEH WKH VWHSE\VWHS DSSURDFK WDNHQ WR GHYHORS WKH DGPLQLVWUDWLRQ RI WKH FRPSHWLWLRQ %\ LPSOHPHQWLQJ WKH
SODWIRUP ZLWK D IRFXV RQ VHFXULW\ PHDVXUHV XVHU UROHV DGPLQ SDQHO ZH VWUHDPOLQHG DGPLQLVWUDWLYH WDVNV
GDWDYDOLGDWLRQDQGVWRUDJHRISDFNHWWUDFHUDFWLYLWLHV IDFLOLWDWLQJ EHWWHU RYHUVLJKW DQG HQDEOLQJ SURPSW
 UHVSRQVHVWRDQ\LVVXHVRULQTXLULHVWKDWDURVHGXULQJWKH
FRPSHWLWLRQ>@
$ 'HVLJQ(OHPHQWVDQG8VHU,QWHUIDFH

7KHGHYHORSPHQWSURFHVVEHJDQE\FUHDWLQJDYLVXDOO\
DSSHDOLQJ DQG FRKHVLYH GHVLJQ IRU WKH ZHE SDJH 8VLQJ ( 'DWD9DOLGDWLRQ
WKH SRZHUIXO GHVLJQ WRRO )LJPD ZH PHWLFXORXVO\ 7RHQVXUHWKHLQWHJULW\DQGVHFXULW\RIWKHGDWDH[FKDQJHG
GHVLJQHGDQHZORJRVHOHFWHGDKDUPRQLRXVFRORUSDOHWWH ZLWKLQ WKH SODWIRUP FRPSUHKHQVLYH GDWD YDOLGDWLRQ
DQG FUHDWHG YDULRXV JUDSKLFDO XVHU LQWHUIDFH *8,  PHFKDQLVPV ZHUH LPSOHPHQWHG DW ERWK WKH IURQWHQG DQG
HOHPHQWV 7KLV FDUHIXO GHVLJQ SURFHVV DLPHG WR HQVXUH D EDFNHQG OD\HUV /HYHUDJLQJ WKH =RG VFKHPD OLEUDU\ ZH
FRQVLVWHQW DQG HQJDJLQJ XVHU H[SHULHQFH WKURXJKRXW WKH HQIRUFHG VWULFW GDWD YDOLGDWLRQ UXOHV WKURXJKRXW WKH
SODWIRUP7KHFKRVHQGHVLJQ HOHPHQWVDQG XVHULQWHUIDFH V\VWHP 2Q WKH IURQWHQG =RG VFKHPD ZDV XWLOL]HG WR
ZHUH PHWLFXORXVO\ FUDIWHG WR DOLJQ ZLWK WKH WDUJHW YDOLGDWH XVHU LQSXWV LQ ZHE IRUPV HQVXULQJ WKDW RQO\
YDOLGDQGH[SHFWHGGDWDZDVVXEPLWWHGWRWKHVHUYHU7KLV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 338

SUHYHQWHG LQFRUUHFW RU PDOLFLRXV LQSXWV IURP EHLQJ 7KH VXFFHVVIXO GHSOR\PHQW RI WKH SODWIRUP XVLQJ
SURFHVVHG DQG SURWHFWHG WKH LQWHJULW\ RI WKH V\VWHP 'RFNHU DQG *LW/DE &RQWDLQHU 5HJLVWU\ FRXSOHG ZLWK
6LPLODUO\RQWKHEDFNHQG=RGVFKHPDZDVHPSOR\HGWR H[FHSWLRQDO EHQFKPDUNLQJ UHVXOWV XVLQJ *RRJOH
YDOLGDWHLQFRPLQJUHTXHVWVDQGVHFXUHWKH$3,HQGSRLQWV /LJKWKRXVH UHIOHFWV WKH SODWIRUP V WHFKQLFDO H[FHOOHQFH
7KLV GRXEOHOD\HUHG YDOLGDWLRQ DSSURDFK HQKDQFHG WKH DQGSHUIRUPDQFH:LWKDVFRUHRILQSHUIRUPDQFH
RYHUDOOVHFXULW\SRVWXUHRIWKHSODWIRUPJXDUGLQJDJDLQVW LQDFFHVVLELOLW\LQEHVWSUDFWLFHVDQGLQ6(2WKH
SRWHQWLDO YXOQHUDELOLWLHV DQG XQDXWKRUL]HG GDWD DFFHVV SODWIRUP GHPRQVWUDWHV LWV HIILFLHQF\ DFFHVVLELOLW\
>@ DGKHUHQFH WR EHVW SUDFWLFHV DQG VHDUFK HQJLQH
 RSWLPL]DWLRQHQVXULQJDQRSWLPDOXVHUH[SHULHQFH
) 6WRUDJHRI3DFNHW7UDFHU$FWLYLWLHV 9, &21&/86,21
7R DFFRPPRGDWH WKH VWRUDJH DQG UHWULHYDO RI WKH ,Q WKLV SDSHU ZH SUHVHQWHG D QHZ QHWZRUNLQJ
3DFNHW 7UDFHU DFWLYLWLHV FUHDWHG E\ SDUWLFLSDQWV ZH DVVLJQPHQWFRPSHWLWLRQIRUKLJK VFKRROHUVDFFRPSDQLHG
HPSOR\HG $PD]RQ 6 6LPSOH 6WRUDJH 6HUYLFH  DV RXU E\ D ZHERULHQWHG HQYLURQPHQW WKDW VHUYHG DV D WDVN
FORXG VWRUDJH VROXWLRQ )RU VHFXUH ILOH XSORDGV ZH GLVWULEXWRU DQG OLYH UDQNLQJ SODWIRUP IRU SDUWLFLSDWLQJ
XWLOL]HG 3UHVLJQHG 3267 85/V ZKLFK DOORZHG VFKRROV7KHGHYHORSPHQWRIWKLVSODWIRUPDGGUHVVHGWKH
SDUWLFLSDQWV WR XSORDG WKHLU DFWLYLWLHV GLUHFWO\ WR WKH 6 OLPLWDWLRQV DQG FKDOOHQJHV LGHQWLILHG LQ WKH H[LVWLQJ
EXFNHW ZKLOH PDLQWDLQLQJ FRQWURO RYHU WKH ILOH XSORDGV HGXFDWLRQDO ODQGVFDSH SURYLGLQJ D FRPSUHKHQVLYH DQG
7RIDFLOLWDWHVHFXUHDQGDXWKRUL]HGDFFHVVWRWKHXSORDGHG HQJDJLQJ OHDUQLQJ H[SHULHQFH IRU VWXGHQWV LQWHUHVWHG LQ
DFWLYLWLHV ZH JHQHUDWHG 6LJQHG 85/V 7KHVH 6LJQHG QHWZRUNLQJ
85/V JUDQWHGWHPSRUDU\DQGDXWKHQWLFDWHGDFFHVVWRWKH 
ILOHVDOORZLQJSDUWLFLSDQWVWRGRZQORDGDQGUHYLHZWKHLU 7KURXJKRXW WKH LPSOHPHQWDWLRQ SURFHVV ZH
VXEPLWWHG ZRUN %\ OHYHUDJLQJ WKH UREXVW FDSDELOLWLHV RI HPSKDVL]HG NH\ DVSHFWV VXFK DV VHFXULW\ XVHU LQWHUIDFH
$:66DQGLPSOHPHQWLQJVHFXUHXSORDGDQGGRZQORDG GHVLJQ UROHEDVHG DFFHVV FRQWURO GDWD YDOLGDWLRQ DQG
PHFKDQLVPV ZH HQVXUHG WKH VDIH DQG HIILFLHQW FORXGVWRUDJH%\SULRULWL]LQJWKHVHHOHPHQWVZHHQVXUHG
PDQDJHPHQW RI 3DFNHW 7UDFHU DFWLYLWLHV ZLWKLQ WKH WKHLQWHJULW\SULYDF\DQGDFFHVVLELOLW\RIWKHSODWIRUPIRU
SODWIRUP>@ DOO XVHUV LQYROYHG 7KH XVH RI WHFKQRORJLHV VXFK DV
 7\SH6FULSW 1H[WMV 7DLOZLQG &66 W53& 3ULVPD
3RVWJUH64/DQG1H[W$XWKMVIDFLOLWDWHGWKHFUHDWLRQRID
9 (9$/8$7,21
UREXVWDQGHIILFLHQWZHEDSSOLFDWLRQ
7R HYDOXDWH WKH XVHU H[SHULHQFH DQG JDWKHU YDOXDEOH 
IHHGEDFN ZH FRQGXFWHG D VXUYH\ FRQVLVWLQJ RI  :KLOH WKH LPSOHPHQWHG ZHE DSSOLFDWLRQ VXFFHVVIXOO\
TXHVWLRQVGLVWULEXWHGWRVHYHQLQGLYLGXDOVZKRLQWHUDFWHG DGGUHVVHV WKH FRUH UHTXLUHPHQWV RI WKH QHWZRUNLQJ
ZLWK WKH SODWIRUP 7KH VXUYH\ DLPHG WR DVVHVV YDULRXV DVVLJQPHQW FRPSHWLWLRQ DQG SURYLGHV D IXQFWLRQDO
DVSHFWV LQFOXGLQJ HDVH RI QDYLJDWLRQ YLVXDO DSSHDO SODWIRUPIRUSDUWLFLSDQWVWKHUHDUHFHUWDLQDUHDVWKDWFRXOG
LQIRUPDWLRQ DFFHVVLELOLW\ WHFKQLFDO SUREOHPV RU EXJV EHQHILWIURPIXUWKHULPSURYHPHQWVDQGHQKDQFHPHQWV
VSHHG DQG SHUIRUPDQFH FODULW\ RI LQVWUXFWLRQV DQG 
JXLGDQFH RYHUDOO XVHU H[SHULHQFH OLNHOLKRRG RI 2QH DUHD RI LPSURYHPHQW LV WKH VWDWLF FRQWHQW RQ WKH
UHFRPPHQGLQJ WKH FRPSHWLWLRQ WR RWKHUV SUHIHUUHG KRPHSDJH &XUUHQWO\ PRGLI\LQJ WKH FRQWHQW UHTXLUHV
GXUDWLRQ RI WKH FRPSHWLWLRQ VDWLVIDFWLRQ ZLWK WKH UHZULWLQJ WKH FRGH DQG GHSOR\LQJ D QHZ YHUVLRQ RI WKH
FRPSHWLWLRQ IRUPDW DQG VXJJHVWLRQV IRU IXWXUH DSSOLFDWLRQ7RRYHUFRPHWKLVOLPLWDWLRQDPRUHIOH[LEOH
LPSURYHPHQWV VROXWLRQFDQEHLPSOHPHQWHGE\FUHDWLQJDGDWDEDVHWDEOH
 DQG D SXEOLF W53& URXWHU 7KLV ZRXOG DOORZ WKH
7KH VXUYH\ UHVXOWV LQGLFDWHG D KLJK OHYHO RI XVHU KRPHSDJHWRG\QDPLFDOO\UHWULHYHDQGGLVSOD\WKHFXUUHQW
VDWLVIDFWLRQ ZLWK WKH SODWIRUP 8VHUV IRXQG WKH V\VWHP GDWD HQDEOLQJ HDVLHU XSGDWHV ZLWKRXW WKH QHHG IRU FRGH
HDV\WRQDYLJDWHYLVXDOO\DSSHDOLQJDQGZHUHDEOHWRILQG FKDQJHVDQGUHGHSOR\PHQW
WKH LQIRUPDWLRQ WKH\ ZHUH ORRNLQJ IRU ,PSRUWDQWO\ QR 
WHFKQLFDOSUREOHPVRUEXJVZHUHUHSRUWHGFRQILUPLQJWKH $QRWKHUDVSHFWWKDWFDQEHHQKDQFHGLVWKHFRPSHWLWLRQ
UREXVWQHVV DQG VWDELOLW\ RI WKH SODWIRUP 7KH VSHHG DQG OHDGHUERDUGZKLFKSURYLGHVSDUWLFLSDQWVZLWKWKHLUWHDP V
SHUIRUPDQFH RI WKH V\VWHP UHFHLYHG SRVLWLYH IHHGEDFN FXUUHQW SRVLWLRQ :KLOH WKH OHDGHUERDUG LV UHJXODUO\
IXUWKHUHQKDQFLQJWKHRYHUDOOXVHUH[SHULHQFH XSGDWHG DQG UHIOHFWV FKDQJHV ZKHQ DQVZHUV DUH
 VXEPLWWHG D PRUH UHDOWLPH DSSURDFK FDQ EH DGRSWHG
:KLOH WKH PDMRULW\ RI WKH VXUYH\ UHVSRQVHV ZHUH 7HFKQRORJLHV VXFK DV :HE6RFNHWV UHDOWLPH GDWDEDVHV
SRVLWLYH WKH TXHVWLRQ UHJDUGLQJ WKH OLNHOLKRRG RI RU 6HUYHU6HQW (YHQWV 66(  FDQ EH XWLOL]HG WR NHHS WKH
UHFRPPHQGLQJWKHFRPSHWLWLRQWRDIULHQGUHFHLYHGORZHU UDQNLQJ XSWRGDWH PLQLPL]LQJ WKH QHHG IRU IUHTXHQW
UDWLQJV7KLVIHHGEDFNSURYLGHVDQRSSRUWXQLW\IRUIXUWKHU TXHULHV DQG SURYLGLQJ SDUWLFLSDQWV ZLWK PRUH LPPHGLDWH
DQDO\VLV DQG LPSURYHPHQW WR EHWWHU XQGHUVWDQG WKH IHHGEDFNRQWKHLUVWDQGLQJV
VSHFLILFIDFWRUVFRQWULEXWLQJWRWKLVVHQWLPHQWDQGDGGUHVV 
DQ\FRQFHUQVRUDUHDVIRUHQKDQFHPHQW 5HJDUGLQJILOHPDQDJHPHQWLQFORXGVWRUDJH $PD]RQ
 6  WKHUH LV D OLPLWDWLRQ LQ WKH FXUUHQW LPSOHPHQWDWLRQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 339

 )LOHV DUH RQO\ GHOHWHG ZKHQ WDVN LV HGLWHG RU GHOHWHG HPSRZHU KLJK VFKRRO VWXGHQWV LQ WKHLU SXUVXLW RI
+RZHYHU WKHUH DUH VFHQDULRV ZKHUH ILOHV PD\ UHPDLQ LQ QHWZRUNLQJNQRZOHGJHDQGVNLOOV
VWRUDJH LQGHILQLWHO\ VXFK DV ZKHQ D ILOH LV XSORDGHG EXW
WDVNFDQ WEHFUHDWHGIRUZKDWHYHUUHDVRQ7RDGGUHVVWKLV $&.12:/('*0(17
LVVXHDVXJJHVWHG VROXWLRQLV WRLPSOHPHQWDFURQMRED 7KLVZRUNZDVVXSSRUWHGE\&XOWXUDODQG(GXFDWLRQDO
UHFXUULQJ VFKHGXOHG WDVN ZKLFK ZRXOG FRPSDUH WKH ILOH *UDQW $JHQF\ .(*$  RI WKH 0LQLVWU\ RI (GXFDWLRQ
NH\VLQWKHDSSOLFDWLRQ VGDWDEDVHZLWKWKRVHLQWKHFORXG 6FLHQFH 5HVHDUFK DQG 6SRUW RI WKH 6ORYDN 5HSXEOLF
VWRUDJH$Q\ILOHVLQWKHFORXGVWRUDJHWKDWGRQRWKDYHD XQGHUWKHSURMHFW1R78.(
FRUUHVSRQGLQJ UHIHUHQFH LQ WKH GDWDEDVH FDQ EH
DXWRPDWLFDOO\ GHOHWHG HQVXULQJ HIILFLHQW PDQDJHPHQW RI 5()(5(1&(6
VWRUHGILOHV >@ - 6PLWK DQG $ -RKQVRQ ³,QVSLULQJ WKH 1H[W *HQHUDWLRQ RI
 1HWZRUN (QJLQHHUV (QJDJLQJ +LJK 6FKRRO 6WXGHQWV LQ
1HWZRUNLQJ(GXFDWLRQ´,(((7UDQVDFWLRQVRQ(GXFDWLRQYRO
,Q DGGLWLRQ WR WKHVH WHFKQLFDO LPSURYHPHQWV WKH QRSS
FRPSHWLWLRQ LWVHOI FRXOG EH HQKDQFHG WR SURYLGH >@ $ %URZQ DQG % 'DYLV ³&UHDWLQJ D :HE2ULHQWHG (QYLURQPHQW
SDUWLFLSDQWV ZLWK D PRUH LPPHUVLYH DQG UHDOLVWLF IRU 1HWZRUNLQJ $VVLJQPHQW &RPSHWLWLRQV $PRQJ +LJK 6FKRRO
H[SHULHQFH2QHSRVVLEOHHQKDQFHPHQWLVWRSURYLGHXVHUV 6WXGHQWV´LQ3URFHHGLQJVRIWKH,(((&RQIHUHQFHRQ,QQRYDWLRQV
LQ(GXFDWLRQDO7HFKQRORJ\SS
ZLWK D WHUPLQDO FRQQHFWHG WR UHDO RU HPXODWHG GHYLFHV
>@ . &KHQ DQG / .LP ³'HVLJQLQJ DQ ,QWXLWLYH 8VHU ,QWHUIDFH IRU
WKURXJK WKH FRPSHWLWLRQ ZHEVLWH 7KLV ZRXOG HOLPLQDWH 1HWZRUNLQJ $VVLJQPHQW 3ODWIRUPV´ ,((( 7UDQVDFWLRQV RQ
WKHOLPLWDWLRQVRIVLPXODWRUVDQGRIIHUSDUWLFLSDQWVDPRUH /HDUQLQJ7HFKQRORJLHVYROQRSS
DXWKHQWLFSUREOHPVROYLQJHQYLURQPHQWHQDEOLQJWKHPWR >@ 6 7KRPSVRQ DQG ( *DUFLD ³(YDOXDWLQJ WKH (IIHFWLYHQHVV RI
JDLQ SUDFWLFDO H[SHULHQFH DQG D GHHSHU XQGHUVWDQGLQJ RI 1HWZRUNLQJ $VVLJQPHQW &RPSHWLWLRQV IRU +LJK 6FKRRO 6WXGHQWV
$ &DVH 6WXG\´ ,((( 7UDQVDFWLRQV RQ (GXFDWLRQ YRO  QR 
QHWZRUNLQJFRQFHSWV SS
 >@ &LVFR1HWZRUNLQJ$FDGHP\³&LVFR&HUWLILHG1HWZRUN$VVRFLDWH
%\ DGGUHVVLQJ WKHVH DUHDV RI LPSURYHPHQW WKH ZHE &&1$ &XUULFXOXP´&LVFR6\VWHPV,QF
DSSOLFDWLRQ FDQ EH IXUWKHU RSWLPL]HG WR SURYLGH D PRUH >@ &LVFR 3DFNHW 7UDFHU DFFHVVHG 0D\  >2QOLQH@ $YDLODEOH
VHDPOHVV DQG UHDOLVWLF H[SHULHQFH IRU SDUWLFLSDQWV KWWSVZZZQHWDFDGFRPFRXUVHVSDFNHWWUDFHU
&RQWLQXRXVUHILQHPHQWDQGHQKDQFHPHQWRIWKHSODWIRUP >@ 00)RZNH³7\SH6FULSW-DYD6FULSWZLWK6\QWD[6XSHUSRZHUV´
,(((6RIWZDUHYROQRSS
ZLOO HQVXUH LWV UHOHYDQFH DQG HIIHFWLYHQHVV LQ VXSSRUWLQJ
>@ 7'1JX\HQ'$7DPEXUULDQG,&UQNRYLF³$&RPSDUDWLYH
KLJKVFKRROVWXGHQWV QHWZRUNLQJHGXFDWLRQDQGIRVWHULQJ 6WXG\ RI -DYD6FULSW )UDPHZRUNV IRU :HE 'HYHORSPHQW´ LQ
WKHLUSDVVLRQIRUWKHILHOG 3URFHHGLQJV RI WKH WK ,QWHUQDWLRQDO 6\PSRVLXP RQ 6RIWZDUH
 (QJLQHHULQJ IRU $GDSWLYH DQG 6HOI0DQDJLQJ 6\VWHPV  SS
$VIXWXUHLWHUDWLRQVRIWKHSODWIRUPDUHFRQVLGHUHGLWLV 
FUXFLDO WR VROLFLW IHHGEDFN IURP SDUWLFLSDQWV RUJDQL]HUV >@ - 'RH DQG $ 6PLWK ³'HVLJQLQJ (QJDJLQJ 8VHU ,QWHUIDFHV $
)LJPD%DVHG$SSURDFK´-RXUQDORI8VHU([SHULHQFH'HVLJQYRO
DQG HGXFDWRUV WR LGHQWLI\ DGGLWLRQDO DUHDV IRU QRSS
LPSURYHPHQW DQG WDLORU WKH DSSOLFDWLRQ WR PHHW WKH >@ 5 3DWHO DQG 6 /HH ³5ROH%DVHG $FFHVV &RQWURO LQ :HE
HYROYLQJ QHHGV DQG H[SHFWDWLRQV RI WKH WDUJHW DXGLHQFH $SSOLFDWLRQV ,PSOHPHQWDWLRQ 6WUDWHJLHV DQG &RQVLGHUDWLRQV´
%\ HPEUDFLQJ WKHVH RSSRUWXQLWLHV IRU JURZWK WKLV -RXUQDORI:HE(QJLQHHULQJYROQRSS
QHWZRUNLQJ FRPSHWLWLRQ FDQ FRQWLQXH WR LQVSLUH DQG >@ $%URZQDQG%'DYLV³(IILFLHQW$GPLQLVWUDWLRQRI:HE%DVHG
&RPSHWLWLRQV $Q $GPLQ 3DQHO $SSURDFK´ ,QWHUQDWLRQDO -RXUQDO
RI:HE$GPLQLVWUDWLRQYROQRSS


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 340

 Early Recognition of the Speaker’s Age
Eva Kupcová, Renát Haluška, Michal Popovič, Matus Pleva Marie Somnea Heng, Patrick Bours
Department of Electronics and Multimedia Communications Department of Information Security
Faculty of Electrical Engineering and Informatics and Communication Technology
Technical University Kosice Norwegian University of
Košice, Slovakia Science and Technology
eva.kupcova@tuke.sk, renat.haluska@tuke.sk , Gjøvik, Norway
michal.popovic@student.tuke.sk, matus.pleva@tuke.sk marieshe@stud.ntnu.no, patrick.bours@ntnu.no

Abstract—In this article, we describe a method for determining We employed a training model to categorize voices into
a person’s real age based on voice recognition. We seek to three groups: children, adults, and a transitional age group,
contribute to the realm of age group classification not reliant in order to ascertain the speaker’s age range. Incorporating a
on text by investigating voice feature analysis, making use
of accessible voice audio datasets, and employing a prototype transitional age category takes into consideration the various
classification model. Additionally, we aim to explore the initial stages of individual voice development [4].
phase of voice-based age group classification to enhance the In order to develop and evaluate age group classification
customization of interventions. models, it is crucial to have access to a dependable and
Keywords—age group classification, voice recognition, text- varied voice audio dataset. Existing datasets that provide age-
independent, audio corpora, dataset review
related labels often have limited coverage across different age
groups. Specifically, datasets containing the voices of underage
I. I NTRODUCTION speakers are particularly scarce due to the heightened legal
and ethical considerations associated with collecting data from
Adults can easily pretend to be children online by entering minors, in contrast to data collection from adults. For this
the wrong age on communication platforms to approach chil- reason, it is necessary to explore diverse collections of spoken
dren. The potential danger in cases where adults impersonate audio covering various age groups. To ensure model and
children and engage with other children on the internet is dataset performance, defining dataset requirements is essential.
significant, carrying the potential for adverse outcomes for the For the accuracy of the classification, the following ques-
minors involved [1]. tions have been defined:
The issue of child protection online is of utmost importance • What requirements must be set for the dataset?
and raises serious concerns, especially as children are becom- • What age intervals are utilized to distinguish between
ing more active on the internet. Adults posing as children and children’s and adult voices?
attempting to engage with minors can exploit their trust and • What is the optimal duration for voice recordings when
innocence. It is crucial that we take all possible steps to ensure training for age group classification?
that children only communicate with those they genuinely • At what point can we categorize the speaker’s voice as
intend to. Therefore, we need to implement measures to verify either that of a child or an adult?
the age of individuals communicating with children [2]. The rest of this article is structured as follows. The second
Identifying a person’s age based on their voice using ma- section outlines the key principles of voice age classifica-
chines is the subject of several research projects. This ability tion and its associated requirements. We describe the chosen
has potential applications in various areas, including forensics, datasets in the third section. The fourth section is devoted
for narrowing down lists of suspects, improving voice quality, to setting up the age group classification model. The last
ensuring online space security, and conducting market research section describes the experimental results and discussions of
in call centers [3]. the classification model. The conclusion provides a summary
We describe a method for determining a person’s age based description of the topic this article deals with and our achieved
on voice recognition. The concept involves implementing a results.
system that aims to determine whether both parties involved
in a call are children or adults based on their voices. When the II. VOICE AGE C LASSIFICATION
system detects an age discrepancy, it alerts a human moderator, Since the human voice is an ever-evolving attribute, it
potentially the child’s parent, who listens to confirm whether allows for the potential estimation of a person’s age based
it’s a genuine concern or a false alarm. If the system identifies on their voice [5]. Researchers in this domain have shown
a potential predator pretending to be a child, swift action is that as children grow older, their vocal traits undergo changes,
taken to protect the potential victim, potentially preventing influencing speaker recognition capabilities [6]. Furthermore,
harm and triggering further investigation into the suspect. adults tend to have deeper voices compared to children, who

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 341

have higher-pitched voices, leading to suggestions that age The Samrómur corpus comprises 143 031 recordings, total-
estimation might be feasible [7]. ing 151 hours and 48 minutes. After filtering for "is valid"
Proper of the swift anatomical shifts in the vocal tract and status, gender, and age, we had 3 552 suitable recordings
the development of the brain, adolescents and children exhibit from 213 speakers, with a ratio of 102 females to 111 males.
considerable variability in their speech characteristics. The Since there was no specific information about what qualified
vocal tract size expands by a minimum of double, going from 8 as "verified" audios, we decided to include unverified ones
cm in infants to 18 cm in adults during the initial two decades as well. As not every recording had information regarding
of life. It undergoes rapid growth during infancy, attaining 80% gender and age, we will be working with a corpus that has
of adult size, subsequently accompanied by a more gradual and 139 640 audio files. But for the experiment, we will use both
consistent increase as it gets closer to adulthood. Furthermore, the verified dataset and the entire dataset to assess the impact
it is characterized by an initial, sharp growth curve that attains of including unverified recordings.
25–40% of adult size, subsequently marked by a uniform
growth pattern that continues until puberty [8]. B. Dataset for model evaluation
Proper to gender-based anatomical and motor skill differ- The dataset Mozilla’s Common Voice2 was chosen for
ences, both fundamental frequency and formant frequencies model evaluation. It is a publicly available voice dataset
gradually reduce as individuals mature. While developmental that relies on contributions from volunteers worldwide. This
alterations do not always follow a linear pattern, they are most dataset is a valuable resource for training machine learning
easily noticeable during the initial years, covering the period models aimed at individuals interested in developing speech
from birth to four years old [8]. applications. Currently, it offers collections of audio data in
Voice transformation is fastest during childhood due to rapid 108 different languages.
growth in the larynx and vocal cords. In females, average We obtained the English Common Voice Delta Segment
fundamental frequency (F0) consistently decreases from 225 13.0 to assess the training model. As this is purely for eval-
Hz in their 20s to 195 Hz in their 80s. For males, F0 decreases uation purposes, we substituted the complete Common Voice
until their 50s, then gradually rises. Jitter measures vocal Corpus 13.0 with the most recent Common Voice Segment
fold vibration regularity. Older women may show higher jitter dataset.
than younger women, and there are also differences in jitter The partial corpus, consisting of 30,280 English audio files,
between young and older men [9]. represents the complete corpus version. Metadata files outside
the audio folder categorize the entire corpus, containing infor-
A. Requirements for Voice Age Classification
mation like file name, statement, approvals, disapprovals, age,
Studies gather voice data from open-source databases or gender, accents, vesrion, area, and segment.
participants, seeking audio files with gender and age metadata, Upon randomly selecting audio files, it became evident that
ideally from a single source for minors and adults. Further- the files have an average duration of approximately 5 seconds.
more, audio files must have a minimum duration of 1 second,
and any audio exceeding 3 seconds will undergo preprocessing C. Children speech recording
to standardize it to a 3-second length.
We also obtained the Children’s speech recordings to assess
The language of the dataset is not important for our purposes
the model’s performance. This was required since the Common
because we are focusing on text-independent gender and
Voice Delta Segment dataset exclusively comprised audio files
age classification. When platforms provide various language
from speakers aged 18 and above.
choices, English was mainly selected because of its extensive
The metadata was meticulously sorted into folders based
prevalence.
on speech type (free speech or predefined) and file format
III. T HE CHOSEN DATASETS (segmented or original). Each folder was labeled with speaker
gender and English proficiency, differentiating between native
After reviewing several available datasets, we selected two.
and non-native speakers.
The first dataset will be used for training and testing, while
In this instance, we made use of the free speech recordings
the second one will be reserved for later model evaluation.
that had been broken down into individual sentences, totaling
A. Dataset for model training and testing 222 audio files.
For more information about the selected datasets, refer to
The dataset Samrómur1 was chosen for training and testing
the thesis [4].
the model. The Samrómur dataset L2 22.09 represents a
collection of recordings encompassing individuals of various IV. P REPARATIONS AND SETTING UP THE AGE G ROUP
age groups, spanning from 5 to 90 years old, who are non- C LASSIFICATION M ODEL
native speakers of the Icelandic language. Over the course of
2019 to 2022, a total of 2189 individuals actively contributed We were seeking a Python project that was relevant to either
to this corpus. voice age classification or voice classification.
1 https://www.openslr.org/130/ 2 https://commonvoice.mozilla.org/

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 342

 k Accuracy Precision Recall F1-score
We found project by Mr. Arrot on GitHub3 and we cus-
1 89.90 83.53 82.22 82.84
tomized this project according to the need of our use. We 2 91.01 84.25 82.30 83.22
chose this project, because this project successfully extracted 3 91.73 85.32 84.31 84.80
characteristics derived from the Samrómur metadata and audio 4 90.19 84.75 81.67 83.07
5 91.38 84.90 83.28 84.05
file. 6 91.03 83.22 83.00 83.11
Prior to making further modifications and adjustments to 7 89.64 82.73 79.10 80.71
the code for this thesis, it was necessary to perform some 8 91.33 85.07 83.51 84.26
9 90.39 84.07 82.23 83.10
pre-processing on the Samrómur dataset. The pydub Python 10 90.18 84.09 83.59 83.81
library was employed for audio editing4 . μ 90.68 84.19 82.52 83.30
Audio files from the selected Samrómur dataset that are
TABLE II: Outcome of 10-fold cross-validation using the entire
longer than 3 seconds must be trimmed into 3s segments. Samrómur dataset (in %).
Files shorter than one second would be disregarded, while
those falling between one and three seconds were retained to
minimize data loss.
For a seamless audio file trimming, we applied a one-second
overlap when trimming each subsequent 3-second portion after
the initial cut, resulting in overlapping audio pieces.
For instance, when trimming an 8-second audio, it yields
four segments: three 3-second pieces and one 2-second piece,
representing the remaining audio section. Segments shorter
than one second will be omitted.
For more detailed information about the preparations, please
refer to the thesis [4].

V. E XPERIMENTAL RESULTS AND DISCUSSIONS

A. Results from Training the Model
When the classification model is applied to the validated
Samrómur dataset, the Table I displays the exact output values
for every fold. Fig. 1: The accuracy curves for training and validation using the
verified Samrómur dataset (4th fold).
k Accuracy Precision Recall F1-score
1 93.75 91.99 91.67 91.83
2 94.01 92.17 92.90 92.52
3 94.53 94.50 93.05 93.71
4 97.91 97.57 97.75 97.65
5 95.30 94.70 94.98 94.84
6 94.26 92.77 91.31 92.00
7 96.08 96.00 95.37 95.66
8 94.78 94.25 93.07 93.57
9 96.08 95.44 96.08 95.75
10 95.56 93.61 93.94 93.77
μ 95.23 94.30 94.01 94.13

TABLE I: Outcome of 10-fold cross-validation using the validated

Samrómur dataset (in %).

Table II displays the results achieved by the model, which

underwent training on the complete dataset, across all individ-
ual folds.
For an alternate view of the model’s training performance,
consult Figure 1 for accuracy progression with the validated Fig. 2: The loss curves for training and validation using the verified
Samrómur dataset (4th fold).
dataset, depicting the model’s improvement over time. Figure
2 displays the loss curve for reference.
Figure 3 illustrates the accuracy curve for the model trained
Furthermore, for the assessment of classification perfor-
with the entire dataset, while Figure 4 showcases the loss
mance across various trained models, refer to Figures 5 and
curve.
6, which present the classification results using confusion
3 https://github.com/lucaArrotta/Age-Estimation-based-on-Human-Voice matrices for the testing and validation datasets.
4 https://pypi.org/project/pydub/ The confusion matrices for testing and validation with the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 343

Fig. 3: The accuracy curves for training and validation using the
entire Samrómur dataset (4th fold). Fig. 5: Confusion Matrix for Testing with the validated Samrómur
dataset.

Fig. 4: The loss curves for training and validation using the entire
Samrómur dataset (4th fold).
Fig. 6: Confusion Matrix for Validation using the validated Samrómur
dataset.
entire Samrómur dataset are depicted in the Figures 7 and 8.

B. Results of Performance Evaluation

enables us to dissect the results further and carefully examine
After training the model using the two Samrómur datasets, the model’s performance.
the next stage involved conducting an evaluation of the model.
Adult voices were categorized based on the following age
This evaluation entailed testing the model’s performance using
groups: late teens (18-19 years), twenties, thirties, forties,
various datasets that included audio recordings from both
fifties, and sixties.
children and adults.
To assess the model’s performance in classifying children’s Both datasets were merged into one testing dataset, which
voices, we chose the "Children speech recording" dataset from was then used to evaluate two models: one trained on the
subsection III-C. In this dataset, we encoded all voices within verified Samrómur dataset and another on the entire Samrómur
the same age group with the value 0. dataset.
For evaluating the model’s ability to classify adult voices, The results are illustrated through two confusion matrices,
we employed the Delta Segment version of the Common Voice with the predicted age categories plotted on the horizontal
English dataset, as detailed in subsection III-B. To facilitate axis and the actual age categories on the vertical axis. These
the analysis, we retained the original age group encoding, as it matrices are presented in Figures 9 and 10.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 344

Fig. 7: Confusion Matrix for Testing with the entire Samrómur
dataset.
Fig. 9: Confusion Matrix for Testing of the model trained using
the verified dataset, including Common Voice and Children’s speech
dataset.

Fig. 8: Confusion Matrix for Validation with the entire Samrómur

dataset.

C. Model Training Discussion

The different output formats in the preceding subsection Fig. 10: Confusion Matrix for Testing of the model trained using the
assessed model stability and classification performance. dataset, including Common Voice and Children’s speech dataset.
Analyzing performance metrics from each of the 10-fold
validation cycles revealed insights for the model trained with
the validated Samrómur dataset (Table I), the average variation several aspects. Specifically, it exhibited a higher accuracy of
in performance metrics between iterations was approximately 4.55%, a greater precision of 10.11%, and a higher recall of
2.13%. There was a notable improvement in training during 11.49%. This led to a superior F1-score of 10.83%, indicating
the third iteration, but apart from that, the results remained a more even performance in terms of precision and recall.
consistent throughout the other iterations. Examining Figure 1 reveals that, in the case of the verified
Regarding the entire Samrómur dataset (Table II), the av- dataset, the model’s training setup appeared to be optimal.
erage deviation in performance was approximately 1.34%. This is evident because both the testing and training accuracy
Although there was a significant drop in training performance consistently increased at a comparable rate, as indicated by
during the seventh iteration, the model generally maintained the two curves ascending in tandem. Likewise, when ob-
consistent results throughout the iterations. serving the loss curve in Figure 2 for the verified dataset,
Comparing the average results from both tables, the model it’s noticeable that both the training and testing loss curves
trained with the verified dataset outperformed the other in simultaneously decreased. This alignment is a positive sign

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 345

of a well-structured dataset and an effectively constructed However, it’s possible to compute this using the binomial
training model. theorem, as demonstrated in Equation 1, to obtain a theoretical
For the entire dataset, Figure 3 accuracy curve initially estimate of the number of data points that must be classified
appears promising due to the close alignment of training and to achieve a 99% accuracy probability.
testing curves. However, a closer examination reveals that the 2n+1
 
2n + 1

k 2n+1−k
gradual increase, resulting in an almost straight line after Probability = ∗ p ∗ (1 − p) (1)
k=n+1
k
approximately 18 epochs, could be an early indication of
overfitting. Overfitting happens when the model becomes too To make this estimation, we assumed that if a voice audio file
fixated on the training data and finds it challenging to apply lasting 11 seconds needed to be classified, it would be divided
its knowledge to new data. This possibility is substantiated by into five audio segments using the same trimming approach
the observations in the loss curve shown in Figure 4, which applied to the trained dataset [4].
shows indications of a model that has overfit the entire dataset. Beginning with the initial calculation of accurately classify-
The rising testing loss towards the end indicates decreasing ing one out of the five audio segments (as indicated in Equation
prediction accuracy, which aligns with the initial concern of 2), the outcome is a probability of 77.38%.
 
overfitting suggested by the accuracy curve. 5 5 5
Probability = ∗ p = 1 ∗ 0.95 = 0.7738 (2)
The presumption in this case is that the validated dataset 5

consists solely of 2.9% of the entire dataset, implying that the As the desired 99% accuracy threshold had not been at-
remaining 97.1% consists of unverified data. This substantial tained, additional calculations were performed. These calcula-
proportion of unverified data poses a heightened risk of tions involved incrementally increasing the count of correctly
significantly diminishing the dataset’s overall quality. classified audio segments by one each time. The outcome
Based on the results explained up to this point, the testing revealed that a minimum of three audio segments need to
(Figure 5) and validation (Figure 6) outputs indicate that the be classified accurately to attain a probability of 99.88%
model trained with the verified dataset demonstrated accurate (Equation 3 and 4).
age group classification for individuals aged 16 to 19 years,      
denoted as group 1. However, there was one misclassification 5 3 2 5 4 5 5
Prob. = ∗0.95 ∗(1−0.95) + ∗0.95 ∗(1−0.95)+ ∗0.95 (3)
3 4 5
for each of the other two age groups, 0 and 2. It’s essential
to note that this does not necessarily imply that the model Prob. = 0.0214 + 0.2036 + 0.7738 = 0.9774 (4)
is better at classifying age group 1 compared to the other
two age groups. This is because the data sample contains D. Performance Evaluation Discussion
an uneven distribution of age groups, which can impact the Finally, the assessment of the classification model’s perfor-
model’s performance. mance involved predicting the age group for 222 kids’ voices
The confusion matrix generated from the entire dataset from the "Children speech recording" dataset (III-C) and 5 946
produced a less organized outcome, primarily because of the adult voices from the Common Voice dataset (III-B).
elevated loss rate. This elevated loss rate resulted in a higher For a more profound understanding of classification per-
number of false classifications within the age groups compared formance, age groups were synchronized with the original
to the verified dataset. This result aligned with predictions, Common Voice dataset metadata, facilitating a comprehensive
considering the previously noted indications of an overfit assessment of the model’s strengths and weaknesses.
model. In Figure 9, the testing confusion matrix for the model
The confusion matrices for testing (Figure 7) and valida- trained with the verified Samrómur dataset demonstrated ro-
tion (Figure 8) indicate that the age distribution within the bust performance in classifying children’s voices. Specifically,
entire dataset is also uneven. In this dataset, the age group it correctly classified 98.2% of the children’s voices into the
classification for individuals aged 16 to 19 years constitutes corresponding age group.
the minority. Consequently, any misclassification within this When examining the classification of adult voices, it’s
age group can significantly impact the overall accuracy score. evident that the model still encounters significant challenges.
It’s essential, from the start of the filtering process, to Upon closer examination of age group breakdowns, the model
account for the age group distribution. The goal is to equalize excels in categorizing age groups 1 and 6, which aligns with
the size of each age group to match the smallest data sample the goal of distinguishing between child and adult voices. Age
among them. group 5 also shows relatively good performance. However, for
Balancing the distribution among the three age groups age categories 2, 3, and 4, the model mostly classifies instances
should enhance the reliability and comparability of the final accurately.
analysis. The testing confusion matrix presented in Figure 10 ulti-
Even though both models exhibited strong performance, mately reveals the overfitting problem in the model trained
achieving an average accuracy of over 90%, this level of using the entire Samrómur dataset.
accuracy is insufficient for considering them suitable for prac- In this case, the model exhibited significantly weaker per-
tical applications. To be viable for practical use, a minimum formance across all age groups compared to the previous one.
accuracy of 99% would be required. In each age group, it misclassified at least 28% of the data

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 346

samples, which is unexpectedly low given the model’s claimed Sport of the Slovak Republic, and by the Slovak Research and
average accuracy of 90.68%. Development Agency under the project of bilateral cooperation
APVV-SK-TW-21-0002 and research projects APVV-22-0261
VI. C ONCLUSION & APVV-22-0414.
Researchers have found multiple uses for the versatile hu-
man voice, including health assessment and age identification. R EFERENCES
Here, we focus on the latter—determining when a voice can [1] P. Anderson, Z. Zuo, L. Yang, and Y. Qu, “An intelligent online groom-
ing detection system using ai technologies,” in 2019 IEEE International
be categorized as child or adult. Conference on Fuzzy Systems (FUZZ-IEEE), 2019, pp. 1–6.
Four questions were defined in the introduction. The first [2] M. Ashcroft, L. Kaati, and M. Meyer, “A step towards detecting online
question focused on dataset requirements for training, vali- grooming – identifying adults pretending to be children,” in 2015
European Intelligence and Security Informatics Conference, 2015, pp.
dating, and testing the classification model. After extensive 98–104.
research, the dataset criteria for this project were established, [3] H. Meinedo and I. Trancoso, “Age and gender detection in the i-dash
including audio files with a minimum duration of one second, project,” ACM Trans. Speech Lang. Process., vol. 7, no. 4, 2011.
[Online]. Available: https://doi.org/10.1145/1998384.1998387.
featuring only the intended speaker without background noise, [4] M. S. Heng, “Early soft biometric voice recognition,” Master’s thesis,
and metadata containing speaker gender and age, with gender NTNU, 2023. [Online]. Available: https://hdl.handle.net/11250/3079079
being an additional training feature. [5] M. Aliaskar, T. Mazakov, A. Mazakova, S. Jomartova, and
T. Shormanov, “Human voice identification based on the detection
The second question regarding age categories for classifying of fundamental harmonics,” in 2022 IEEE 7th International Energy
children and adults was addressed in the dataset requirements. Conference (ENERGYCON), 2022, pp. 1–4.
Three age groups were established: under 16, 16-19, and over [6] S. Purnapatra, P. Das, L. Holsopple, and S. Schuckers, “Longitudinal
study of voice recognition in children,” in 2020 International Conference
19. The second group pertains to voice development during of the Biometrics Special Interest Group (BIOSIG), 2020, pp. 1–8.
puberty and may necessitate manual classification, acting as a [7] P. Punyani, R. Gupta, and A. Kumar, “A comparison study of face, gait
transitional phase in the project. and speech features for age estimation,” in Advances in Electronics,
Communication and Computing, A. Kalam, S. Das, and K. Sharma,
The third question was about finding the optimal audio Eds. Singapore: Springer Singapore, 2018, pp. 325–331.
duration for training the age category classification model. The [8] M. Novotny, R. Cmejla, and T. Tykalova, “Automated prediction of
requirement was a minimum of one-second audio duration, and children’s age from voice acoustics,” Biomedical Signal Processing
and Control, vol. 81, p. 104490, 2023. [Online]. Available: https:
longer files were trimmed. The crucial factor was ensuring //www.sciencedirect.com/science/article/pii/S1746809422009442.
enough audio data for age category distinction, leading to [9] Y. Sun, M. Zhang, Z. Sun, and T. Tan, “Demographic analysis from
the conclusion that the ideal audio length for training in this biometric data: Achievements, challenges, and new frontiers,” IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol. 40,
context should be three seconds. no. 2, pp. 332–351, 2018.
The final question revolved around identifying at what point [10] M. Markitantov and O. Verkholyak, “Automatic recognition of speaker
a speaker’s voice could be categorized as child or adult. age and gender based on deep neural networks,” in Speech and Com-
puter: 21st International Conference, SPECOM 2019, Istanbul, Turkey,
Concerning early classification, it is essential to maintain August 20–25, 2019, Proceedings 21. Springer, 2019, pp. 327–336.
an accuracy level that allows for practical application. The [11] F. Burkhardt, M. Eckert, W. Johannsen, and J. Stegmann, “A database
classification model implemented in this project, trained with of age and gender annotated telephone speech.” in LREC. Malta, 2010.
the verified Samrómur dataset, achieved an accuracy rate of
95.23% which overperformed previous published results [10]
on aGender corpus [11].
Subsequently, the calculation demonstrated aimed to deter-
mine how many audio segments would be needed to theoreti-
cally achieve an accuracy of 99%. The result of this calculation
indicated that, in theory, three audio segments, each lasting
three seconds, would be required to attain the desired accuracy
level.
This implies that, based on the model developed in thesis
[4], the earliest point at which the speaker’s voice can be
reliably classified is seven seconds. It’s worth noting that the
result is seven seconds, not nine, as we must account for the
trimming process, where each subsequent trim overlaps by one
second with the previous segment.
ACKNOWLEDGMENT
The research in this paper was partially supported by the
Scientific Grant Agency of the Ministry of Education, Science,
Research and Sport of the Slovak Republic and the Slovak
Academy of Sciences under the project VEGA 2/0165/21
funded by the Ministry of Education, Science, Research and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 347

 Using a selected machine learning method in the
R language in statistics learning
I. Dirgova Luptakova*
* Institution of Computer Technologies and Informatics/Faculty of Natural Sciences, Trnava, Slovakia
iveta.dirgova@ucm.sk

Abstract — The Bootstrap method, alternatively referred to

as bootstrapping, is a statistical approach employed to
assess the reliability of statistical estimations and models.
Importantly, it accomplishes this without assuming that the
data originates from a specific probability distribution. This
study aimed to elucidate fundamental statistical concepts,
with a particular emphasis on the realm of estimating key
parameters, both single values and intervals, within the field
of statistics education. Additionally, the thesis delved into an
exhaustive explanation of the Bootstrap method. In the
experimental segment we centered our attention on this
contemporary technique and its relevance to our chosen
data type. We executed these analyses using the R
programming language, renowned for its potency in
statistical applications and its utility in educational contexts.
This acquired knowledge was then practically exemplified
using real-world data pertaining to the historical evolution
of gold prices spanning from 1950 to 2018.
Figure 1. Bootstrap relevance [own source]
I. INTRODUCTION
Machine Learning is a branch of artificial intelligence that (Figure 1). Several specialized packages have been
is all around us. It demonstrates the power of data in new developed for working with this method, including
ways, with the most common example being when packages designed for the R programming language. One
Facebook suggests articles in your feed. This technology of these packages is "bootstrap," and another
helps computer systems learn and improve based on simplification is "boot," with the "boot" package being
experience by developing computer programs that can more widely used.
automatically access data and perform tasks through The principles of Bootstrap can be extended and
predictions and detections. [8] applied to various other domains, such as time series
In today's era of rapid technological advancement and analysis or, in our context, machine learning.
the constant evolution of computational technology and Additionally, Bootstrap can be categorized as non-
methodologies, the Bootstrap method has gained parametric (full) or parametric. [5]
prominence much more swiftly than established
traditional methods and procedures of statistical inference. A. Non-parametric (full) bootstrap
This method is grounded in random sampling with When making inductive judgments about the parameter
replacement. Thanks to this approach, it becomes possible of a random variable x based on data from a sample
to estimate sampling distribution functions for almost any ( with a distribution function , these
statistic using random sampling methods, even for inductive judgments rely on the sample distribution of the
statistics where it would have been entirely infeasible with
alternative approaches. estimator . The sample distribution is often obtained
from theoretical results. For instance, if we are estimating
The Bootstrap method can be employed to estimate the that the sample follows an exponential
standard deviation of any statistic and, simultaneously, to probability distribution with parameter λ, according to the
establish its confidence intervals. In cases where it is not central limit theorem, we can say that the random variable
feasible to express confidence intervals analytically or has an asymptotically normal distribution
where it is highly challenging to do so, the Bootstrap . This can be used for estimating
method proves to be exceedingly advantageous. statistics, confidence intervals, or statistical tests about the
Furthermore, Bootstrap is utilized for estimating parameter λ. However, in practice, there are cases where
regression coefficients and coefficients of correlation or assumptions are not met or asymptotic results are not
determination in regression models. suitable, especially when dealing with small samples. In
The primary drawback of Bootstrap is its high such cases, we may resort to inductive judgments based
computational demand on hardware. Without quality and on the bootstrap method.
computationally powerful hardware, it is nearly
impossible to apply the Bootstrap method effectively

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 348

 Bootstrap allows us to avoid relying on theoretical C. Alternative methods to bootstrapping
sample distribution assumptions of the observed statistic A major challenge in designing a machine learning model
and, instead, use an empirical sample distribution. This is is to ensure that it performs accurately on unseen data. In
achieved by repeatedly sampling from the original dataset. order to know whether the proposed model works
When we do this directly from the data in the statistical correctly or not, we need to test it with those data points
dataset, we refer to it as non-parametric (full) bootstrap. that were not present during model training. These data
In the creation of bootstrapp estimates of the parameter points will serve the purpose of invisible data for the
for the random variable , the following steps are model and it will be easy to evaluate the accuracy of the
followed: model. One of the best techniques to check the
1. From the observed values of a effectiveness of a machine learning model are cross
random sample , , ... , ), we calculate an validation techniques, which can be easily implemented
estimate of the parameter . using the R programming language. In this, a part of the
dataset is reserved which is not used in training the model.
2. Subsequently, we generate B random bootstrap When the model is ready, this reserved dataset is used for
samples with replacement, each with a sample p size testing purposes. The values of the dependent variable are
of n, from the observed values . The predicted during the testing phase and the accuracy of the
accuracy of the estimate improves as the number of model is calculated based on the prediction error, i.e. the
bootstrap estimates B increases. However, a higher difference between the actual values and the predicted
number of estimates also increases computational values of the dependent variable.
complexity. In general, it is advisable to choose B
to be at least 1000. There are several statistical metrics that are used to
assess the accuracy of regression models: Root Mean
3. For each bootstrap sample, we calculate an Squared Error (RMSE), Mean Absolute Error (MAE) or
estimate of the parameter and denote it as , R2 Error. [1,3]
where ݅ = 1, 2, ..., B (Figure 2)[5].
The jackknife method is comparable to bootstrap;
however, this approach samples without replacement, not
with it. There is many circumstances in which it is
impractical or perhaps impossible to construct an accurate
estimates, or to determine their standard errors. It may be
that there is no theoretical basis, on which to rely, or that it
is not possible to use a standard method of estimation
when trying to estimate the variance of a difficult statistic.
In these circumstances, one can use the jackknife method
to calculate an estimate of the variance and standard error.
[4,7]
Jackknife samples are selected by taking the original
Figure 2. Bootstrap unknown parameter estimation procedure data vector and removing one observation from the set
[own source] each time after recalculating based on the remaining data.
There is thus ݊ unique Jackknife samples and the ⅈ-th
B. Parametric Bootstrap Jackknife sample is defined as:
In addition to the full (non-parametric) bootstrap, there are ܺ[݅] = {ܺ1, ܺ2,…,ܺ݅−1, ܺ݅+1, … , ܺ݊−1, ܺ݊}
cases where the use of parametric bootstrap is This procedure can be generalized to ݇ erasures, which is
advantageous. It allows us to estimate unknown statistics discussed next. The ⅈ-th replication of the Jackknife is
and confidence intervals. The procedure is similar to that defined as the value of the estimate ‫ )∙(ݏ‬evaluated at the ⅈ-
of the non-parametric bootstrap, with the main difference th Jackknife sample:
being that bootstrap samples are not generated directly
from the original data. Instead, we first estimate a
parameter, and then bootstrap samples are created from a The Jackknife standard error is defined:
distribution with that specified parameter [5].
Based on the assumption that the original data set is a
realization of a random sample from a distribution of a
specific parametric type, in this case, the parametric model
is characterized by the parameter . Often, maximum where is the empirical average of Jackknife
likelihood is used, and random samples are drawn from
this fitted model. Typically, the drawn sample has the repetitions:
same sample size as the original data. Then, the estimation
of the original function F can be written as . This
sampling process is repeated multiple times, as in other
bootstrap methods. Regarding the sample mean in this
case, the function of the original distribution of random Jackknife bias is defined as:
samples is replaced
p by a bootstrap random sample with
the function *, and the probability distribution of
is approximated by , where ,
which is the expected value corresponding to *. The jackknife method is an efficient and flexible statistical
tool that can be used to measure the accuracy and stability

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 349

of parameter estimates in a variety of scientific original estimate (bias), and the standard error of the
disciplines. The technique is simple to use and inherently bootstrap estimate (std. error). [2]
requires no assumptions regarding the distribution of the
data. The jackknife method is prone to outliers and tends
to underestimate the variability of small data sets, among
other drawbacks. The jackknife method is a useful tool for
statistical analysis and should be considered in any
research project that requires parameter estimation. [4]
D. Package BOOT in R language
The R language is related to data science. R is a Figure 4. Output of the nonparametric boostrap estimator obtained
programming language which is used for statistical by the boot function in R [own source]
programming and data visualization hence for working
with R language we need data. To get the data we need
E. Forecasting
data science. But why choose R among many languages
even though Phyton is the most popular ? In the R language, forecasting refers to the process of
The first reason is the convenience of learning with predicting future values of a time series variable based on
Tidyverse . Tidyverse is an extensive collection of R its historical patterns. Time series data is typically
characterized by trends (upward or downward movement
packages that has greatly eased the steep learning curve over time), seasonality (regular and repeating patterns),
of programming in R for data science. Developed by and random noise.
Hadley Wickham and his team, the idea in developing
R provides a wide range of powerful tools and packages
Tidyverse was a consistent and structured programming for time series forecasting, including ARIMA models,
interface that shared their vision of a unified underlying exponential smoothing methods, and machine learning
design philosophy, grammar, and data structures. algorithms. In our work, we have implemented the
Use in academia. The R language focuses on ARIMA model.
visualizing quantitative data using a variety of techniques To perform forecasting in R, we first need to import
and packages, making it one of the most widely used time series data into the R program and transform it into a
programming languages for research by scientists and suitable format using the "ts()" function. Then, we can use
researchers. various forecasting functions, such as "forecast()" or
Comprehensive support packages for individual topics. "auto.arima()", to generate forecasts and prediction
R comes with a huge collection of libraries for various intervals for our time series data.
topics such as data science, machine learning, statistics,
econometrics, finance, management and other areas
where business analytics is key. With these packages in II. RESULTS OF WORK
hand, implementing R for data science is greatly For a suitable interpretation and work with individual
simplified when dealing with very specific problems and functions in R, suitable data must also be selected. The
use cases. dataset must be well structured to be well and easily
One of the significant advantages of the "boot" package implemented in our project. We have chosen a dataset
is that it can directly compute bootstrap estimates along regarding the evolution of gold price from 1950 to 2018.
with confidence intervals. The core function is "boot," [6]
and its mandatory parameters include the dataset, the
A. Forecasting results
statistic to be computed, and the number of bootstrap
samples. Given the broad range of possibilities for In our code, we proceed as follows:
bootstrap sampling, it's necessary to program the function 1. We load our csv file or dataset into the selected
that calculates the chosen statistic beforehand. variable my_data2.
The following command generates point estimates of 2. From the my_data2$year column, which gives us
Pearson's correlation coefficient ( *), median ( *), and the values we convert to date format.
arithmetic mean ( *) from simulated data. In the default 3. Then we use the aforementioned ts() function to
output of an object created through the "boot" function, create a time-series object.
you'll find the original estimate from the original dataset
4. From the created object we work with ARIMA
(original), the bias of the bootstrap estimate from the
methods.
5. We reach the predicted values for the next years,
namely the next 5 years (Figure 5.).

Figure 3. Input code in R [own source]

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 350

 the linear regression formula to be used
(year~average_price). The year variable is the response
variable and average_price is the predictor variable.
Overall, this code performs a bootstrap analysis to
estimate the distribution of R-squared values for a linear
regression model that predicts year using average_price.

Figure 5. Graph for obtained confidence intervals [own source]

B. Bootstrapping Results
In this section, we demonstrate how we worked with the
mentioned dataset using bootstrapping on a linear
regression model with the variable 'year' and the variable
'average_price' as the predictor. We defined four functions
to obtain the R-squared (Figure 6.), RMSE (Root Mean
Square Error), MAE (Mean Absolute Error), and
regression weights of the model using bootstrapping.
In the code, we used functions from the 'boot' package
with 10 000 iterations. The results were saved in four
distinct objects. In the next Figure 7. we have
bootstrapped regression weights for the intercept and the
slope of the predictor. We also visualize the bootstrap
results and calculate 95% confidence intervals.

Figure 7. Graphical representation of the bootstrap BS and Confidence

intervals for these results [own source]

C. Jackknife Results
As mentioned we used the jackknife method for
comparison. The following figure Figure 8. Plots
jackknife resampling analysis to estimate the bias and
standard error of the mean of the sample:
- first, we computed the sample mean of the
"appropriate_price" column from the dataset,
- by looping through the for loop, we performed the
Figure 6. Graphical representation of the RSQ bootstrap [own source] jackknife resampling analysis iteratively,
omitting one observation after another, we recalculate
The RSQ function accepts three arguments: the mean of the remaining, of the data and store it in the
formula: a linear regression formula that describes the jackknife_means vector,
relationship between the response variable and the - we plotted the jackknife standard error as the square
predictor variables root of the sum of the squared of the deviations of the
data: a data frame containing the data for the linear jackknife means from their mean, times (n - 1)/n.
regression model We sought to assess the accuracy of the sample mean
indices: a vector of indices that specifies the rows of the by this procedure by estimating its deviation and standard
data frame to be used in the current bootstrap sample error using the jackknife method.
The function first creates a subset of the data using the
specified indices and then fits a linear regression model to
the subset of the data using the lm function. It then outputs
and returns the R-squared value of the model. And after
retrieving the data, we call the aforementioned function.
The boot function is then used to perform bootstrap
analysis on some data (my_data). The statistic argument
specifies the function to be used to calculate the statistic of
interest, which in this case is the R-squared value of the
linear regression model (RSQ). The R argument specifies
the number of bootstrap replications to be performed
Figure 8. Results of Jacknife [own source]
(10,000 in this case), and the formula argument specifies

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 351

In the following figure (Figure 9.), we create a graph Meaning of the columns in Figure 10 is as follows:
from the obtained data, in which we can see the scatter of ME (Mean Error): Average difference between actual
the mean jackknife values against the deleted values and predictions.
observations. This will help us to better visualize the RMSE (Root Mean Squared Error): Measures the
relationship between jackknife means and deleted standard deviation of errors and assesses the model's
observations, whether any particular observations have a accuracy.
large effect on the mean. The horizontal line will give us
shows how much the jackknife estimates deviate from the MAE (Mean Absolute Error): Represents the average of
sample mean. the absolute values of errors.
MPE (Mean Percentage Error): Measures the average
percentage difference between actual values and forecasts.
MAPE (Mean Absolute Percentage Error): Measures
the average percentage difference between actual values
and forecasts relative to actual values.
MASE (Mean Absolute Scaled Error): Measures
forecast accuracy relative to the accuracy of a naive
forecasting method.
ACF1 (Autocorrelation of Residuals of First Order):
Measures the autocorrelation of first-order residuals,
Figure 9. Variance of jackknife means [own source] indicating potential time-dependent patterns in the model's
errors.

III. CONCLUSION
ACKNOWLEDGMENT
Based on the provided information, we can observe that
the average price of gold increased from 2019 to 2020, This work was supported by the KEGA project Adaptive
and according to the R forecast, it is expected to remain platform for statistical literacy development
relatively stable in 2021. The average gold price in 2019 Project No. 020UCM-4/2022.
was $1,393.34 per ounce, while the forecasted price for
this year is $1,376.59 per ounce. This suggests that the REFERENCES
forecast was slightly lower than the actual average price
for the year. The average gold price in 2020 was [1] BERRAR, Daniel. Cross-Validation. 2019.
DOI: 10.1016/B978-0-12-809633-8.20349-X In
$1,770.26 per ounce, and the forecasted price for this book: Reference Module in Life Sciences. ISBN
year is also $1,376.59 per ounce. This indicates that the 978-0-12-811432-2.
2020 forecast was significantly lower than the actual [2] CANTY, Angelo; RIPLEY, Brian. Package ‘boot’.
average price for that year. For 2021, the forecasted Bootstrap Functions. CRAN R Proj, 2017.
average gold price is $1,376.59 per ounce. If this forecast [3] EFAEILZADEH, Payam; TANG, Lei; LIU, Huan.
Cross-validation. Encyclopedia of database systems,
materializes, it will signify a slight decrease compared to 2009, 5: 532-538.
the average price in 2020. [4] MCINTOSH, Avery. The Jackknife estimation
method. arXiv preprint arXiv:1606.00497, 2016.
However, it's important to note that this forecast is [5] MIHALECH, Patrik. Bootstrap estimates in R.
based on a specific model and assumptions, and actual Bratislava, -. Also available from:
https://ei.fhi.sk/index.php/EAI/article/view/211/239
prices may vary depending on a range of factors, such as
[6] Link to the dataset:
market conditions, global economic developments, and
https://datahub.io/AcckiyGerman/gold-prices
geopolitical events.
[7] WU, Chien-Fu Jeff. Jackknife, bootstrap and other
Furthermore, after using the accuracy() function, we resampling methods in regression analysis. the
obtain the model's accuracy rate on the training data. The Annals of Statistics, 1986, 14.4: 1261-1295.
specific result or output of the accuracy() function will [8] ZHOU, Zhi-Hua. Machine learning. Springer Nature,
depend on how it was used and what arguments were 2021. ISBN 978-981-15-1966-6.
provided. It typically includes various measures of
accuracy, such as Mean Absolute Error (MAE), Root
Mean Squared Error (RMSE), Mean Absolute Percentage
Error (MAPE), and others. These measures help assess
how well the model's predictions align with the actual
data.

Figure 10. The result of the accuracy() function [own source]

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 352

 Comparison of Deep Learning and Ensemble
Learning in Classification of Toxic Comments
K. Machova*, and T. Tomcik*
* Technical University of Košice/Department of Cybernetics and Artificial Intelligence, Košice, Slovakia
kristina.machova@tuke.sk, tomas.tomcik@student.tuke.sk

Abstract—The paper is focused on recognition of various racial diversity, skin colour, religion, gender, or sexual
forms of toxic comments on social networks, particularly on orientation [17]. Companies that have hate speech policies
the offensive and hate speech and on the cyberbullying. The in place include Facebook and YouTube. In 2018, there
spreading of toxic content through social networks is was a post containing part of the United Declaration of
nowadays very serious problem, which can be harmful for a Independence of the States, which refers to Native
democratic society functioning. The paper presents an Americans as "ruthless Indian savages", marked as
experiment with various machine learning methods to Facebook hate speech and removed from his page [11]. In
discover which from them would be the most suitable for 2019, plataform for videos sharing YouTube has shut
building recognition models. Primarily, we will compare the down channels such as American radio host Jesse Lee
deep learning and ensemble learning. Peterson, based on his politically hateful speech.
The offensive speech may involve various forms of
I. INTRODUCTION toxicity. It can be difficult to distinguish general offensive
language from hate speech. The offensive language can
As the Internet expands, so does the amount of content contain any kind of profanity or insult, so the hate speech
on it. In addition to content based on facts, a large amount can be classified as a subset of offensive language. When
of content is a toxic type, which negatively affects the web detecting offensive posts, the type and purpose of
users, particularly teenagers. While most web users offensive expression is considered. And therefore, the
respect the norms of social behaviour, some users do not, criteria for detection should capture the attributes of
and their comments reflect their antisocial behaviour. The offensive expression in general [3].
anonymity provided by social networks, simplicity of
The cyberbullying represents content posted online by
contribution and easiness of toxic content spreading
an individual or a group who is aggressive, humiliating, or
represent an extremely topical problem today. An
hurtful towards the victim who does not know or cannot
automated detection of various forms of toxicity can be
easily defend himself. It may be described on a basis of
helpful in the process of regulation and limitation of them
three criteria: intention, repetition, and superiority. Leaked
by moderators of web discussions but also by social
information means a big problem for the bullied person,
networks users.
since if any defamatory or confidential information is
Social media has seen increased use as a source of published on the web, it is very difficult to remove it [14].
information and is mainly used to search for information
Machine learning and its methods are very popular
on serious topics. There has also been great use by those
today and useful. They are used for various forms of
who seek health information. People use social "tools" to
classification, regression and solution problems associated
gather information, share stories, but also discuss issues.
with text or image detection. They have a wide range of
Similarly, healthcare organizations see benefits of social
uses through detection of antisocial behavior, cyber
media because they give them access to healthcare
security, healthcare, IoT and various others [13]. We
information.
focused our research on the automatic detection of three
Social media comes to the fore as a source of forms of the antisocial behaviour: hate speech, offensive
information in times of disaster and risk situations, speech, and cyberbullying using machine learning.
although the accuracy of the information that is shared
The main objectives of the study intended to achieve
through these channels is unclear. Therefore, it is essential
are as follows:
to learn more about how people evaluate the information
they receive on social media websites, especially in terms x A comparison of various machine learning
of their credibility. methods in generation of models for recognition
There are many kinds of uncredible information and of various antisocial behaviour forms (hate,
toxic comments, which can be and is harmful for users as offensive, and cyberbullying) on social networks,
fake news, conspiracy theories, trolling, hatreds, offensive but similar in impact on social networks users.
posts, cyberbullying, and phishing. We have concentrated x A meta-level comparison by offering an
on detection of hate speech, offensive speech and evaluation of the success of classical learning
cyberbullying in our research. contra deep learning, and ensemble learning in
The hate speech is defined as public speech that building detection models.
expresses hatred or promotes prejudice and violence
against a person or a group based on something such as

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 353

 x Answering the question “How the volume of data same concept is already associated with target labels,
available for training affects the results of such as classes that define concept identities [2].
machine learning models”. Unsupervised learning: is about understanding or
x A creation of three datasets in English language: finding a concise description of data passively by
Hate dataset, Offensive dataset, and mapping or grouping data according to certain organizing
Cyberbullying dataset suitable for intended principles. That means the data they only form a set of
experiments. objects for which no label is available to define a
particular one related concept, as in supervised teaching.
A. Related Works
The goal of unsupervised learning is to create groups,
Detection models for the toxicity recognition based on clusters of similar objects, according to the criterion of
machine learning have been researched in latest years. similarity and then derive a concept that is shared
Work [3] uses TF-IDF weighting scheme, part-of-speech
between these objects. The unsupervised learning also
tags, and other linguistic features for the representation of
text inputs for the very successful machine learning includes algorithms that aim to provide a representation
method SVM. The best results achieved were from high dimensional to low-dimensional spaces, while
Accuracy=0.91 for offensive posts recognition but only preserving the initial information about the data and a
0.61 for hate speech recognition. Also results achieved in more efficient calculation is offered [4].
the paper [10] showed the SVM model as the most Reinforcement learning: involves taking actions to
successful (Accuracy=0.89) in the recognition of degrees achieve a goal. During reinforcement learning, the agent
of toxicity in the conversational content comparing with learns by trial and error to perform an action in the
NB-Multinomial, RF and Bagging. The approach environment to obtain a reward, thus providing an
presented in [18] uses the average results of 10 neural effective method for developing goal-directed action
networks with different initializations of weights. The strategies. Reinforcement learning was inspired by related
ensemble model achieved the best result F1= 0.94, but the phycological theories and is closely related to the basal
means of ensembles achieved only F1= 0.83. The work
[5] focuses on neural networks (LSTM and GRU) in ganglia in the brain. Reinforcement learning
the recognition of abusive posts on Twitter. They methodologies are concerned with problems where the
achieved AUC values to range from 0.92 to 0.98. In learning agent essentially does not know what to do Thus,
general, the machine learning approach can be used for the agent must discover an appropriate way to maximize
sentiment analysis, which can be helpful in toxicity the expected profit defined by the rewards that agents
recognition as the toxicity in online space usually receive in each state [15]. Reinforcement learning differs
represents a negative opinion. The work [8] has developed from of supervised learning because no input/output is
Ensemble Learning Scheme using DT, SVM, RF and presented in reinforcement learning pairs, neither are
KNN (K-Nearest Neighbours) for sentiment analysis of explicitly modified actions, but agents at a specific time ‫ݐ‬
CoViD-19 related offensive comments. The results were fall into a certain state and based on this information they
beyond Accuracy=0.90. choose an action. As a result, the agent receives for his
activity a reinforcing punishment or reward.
II. USED METHODS
We have used supervised learning method for
To detect antisocial behaviour in text data, methods of generation recognition model because we had labeled
machine learning are often used. The main goal of our datasets available. The scheme of supervised learning is
work was to compare suitability of various approaches to illustrated in the Figure 1.
machine learning for recognition toxicity on social
networks. Thus, we concentrated on deep learning, and
ensemble learning in a comparison with some classic
machine learning methods.
A. Machine Learning Paradighms
Currently, three categories of machine learning from
examples are recognized, according to the way in which
they use the training data to find the context or inference
or general description of the set data in a specific
problem. These are the following categories: Figure 1. The supervised learning schemes [2].
x Supervised Learning
x Unsupervised Learning
x Reinforcement Learning. B. Deep Learning
Deep learning is a type of a supervised machine
Supervised learning includes prediction and learning of artificial neural networks with large number of
classification tasks and represents the main category hidden layers. For text data processing, recurrent neural
learning. In supervised learning, the objects that are networks are the best choice, because they can transfer
related to a particular problem are represented by input- information from the processing of one input to the
output pairs. This means that data that belongs to the processing of the next input and thus model the
relationship between words in a text.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 354

 The most known recurrent network is LSTM (Long interpret and offer good accuracy on multiple data forms
Short-Term Memory) – it is also a specific network [1].
among all recurrent networks, which can re-store
information a longer time and that is why they can process D. Ensemble Learning
longer sequence of words. LSTM networks are composed From the ensemble learning we have used two more
of repeating modules (LSTM blocks) in the form of a successful methods: random forests and boosting. The
chain. The basis of LSTM is a horizontal line through Random Forests (RF) of decision trees. RF method tries
which a vector passes between individual blocks. Using to minimize the variance by creating more decision trees
this line, information passes through the entire structure of in different parts of the same training data. Individual trees
the LSTM network. There are three gates (input, forget are de-correlated using a random selection of a subset of
and output gate) in individual cells. These gates are used attributes. The method achieves the final classification by
to remove or add information to the state of the block. voting of the individual trees. RF method is used mainly
Information (vector) passes through these gates, which are in cases where a limited amount of data is available,
composed of neurons with a sigmoidal activation function. which significantly reduces memory requirements when
Depending on the value of the output on these neurons, generating many trees [9].
certain amount of information passes through it, while 0 AdaBoost is an algorithm based on boosting learning
means that no information passes through the gate and 1 strategy. The boosting is a method, which combines
means that everything passes through the gate [6]. predictive models results to improve the accuracy of the
There are several variations of the LSTM network that final prediction. In this method, training examples are
use this basis, but with some variations in some parts of weighted. A set of models are learned on the same dataset,
the block. One of the most famous is GRU (Gated but each time with different weights of training examples.
Recurrent Unit). This variation combines input gate and The weights are adapted from model to model, to achieve
forget gate into one gate [12]. This means that the GRU is better results [16].
simpler because it has only two gates in total.
III. IMPLEMENTATION AND TESTING
C. Clasic Machine Learning
From classic supervised methods of machine learning, A. Methodology
we have chosen Naïve Bayes as a baseline method, We extracted short texts from social media and
Support Vector Machine because of its big efficiency in available datasets related to antisocial behavior and joined
text data processing and Decision Tree classifier for them to three sets: Hate speech, Offensive and
comparison with Random Forest of decision trees as an Cyberbullying datasets. Those rough datasets were
example of the ensemble approach to learning. Naïve preprocessed and labelled. All final datasets were used for
Bayes classifier (NB) is a probabilistic classifier based on the training of detection models using classic machine
Bayes’ theorem and independence assumption between learning methods (NB, SVM and DT), deep learning
features. NB is often outperformed by support vector (LSTM and GRU) and ensemble methods (RF and
machines [7]. Adaboost). The models were evaluated using the
Support vector machines (SVM) separates the sample Accuracy, and F1-rate. Figure 2 illustrates the
space into two or more classes with the widest margin methodology of our approach to building models for the
possible, which increase the accuracy of classification. toxicity recognition.
The method is originally a linear classifier. For text All machine learning models were implemented in the
processing is more suitable the SVM using kernel programming language Python (in version 3). Particularly,
functions [7]. the Jupiter Notebook was used, which was created
Decision Tree (DT) method generates a tree graph in employing Scikit-learn, Numpy, Matplotlib and Scipy
which each path starting from the root is described by a libraries. All machine learning methods were trained on
sequence separating the data until a Boolean result is all datasets to enable the detailed comparison of suitability
reached at the end - leaf node. DT models are easy to of selected methods in detection of all forms of antisocial
behaviour as hatred, sensitivity, and cyberbullying.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 355

 Figure 2. The methodology schemes.
where and .
B. Data Description
Selected methods of machine learning were used for The following insight can be derived from the
model training. The models were tested on three datasets definition of these metrics. For simplicity we consider TP
containing data of three various forms of antisocial > 0, TN > 0, FP > 0 and FN > 0. Then, in multiplications
behaviour. We prepared three datasets in English there is no change of the sign of values. Then the
language. The first Hate Speech Dataset contained 2000 relationship between Accuracy (Acc) and F1 can be
hate comments and 4000 neutral comments. The second described as in Figure 3.
Offensive Speech Dataset contained 23548 offensive
comments and 12239 neutral ones. The third
Cyberbullying Dataset contained data that captured
cyberbullying in conversational content on the social
network Twitter, Wikipedia, and YouTube platform. The
data were labelled manually into cyberbullying/not
cyberbullying classes. This dataset was unbalanced
because it contained 803 cyberbullying and 65060 neutrals
comments. So, the number of neutral comments was
decreased before training. The datasets were created by
finding data, joining them, preprocessing, and labelling.
The datasets were pre-processed through removal spaces, Figure 3. Relationship between Acc and F1.
social network hashtags, hyperlinks, and capital letters and
through tokenization (using a tokenizer from Scikit-learn
library) and vectorization (using TF-IDF weighting From the description of relationship between Accuracy
scheme and Word2Vect representation). The ratio of and F1, following insight can be derived:
training and testing sets was 3:1.
IF TN ≥ TP THEN Accuracy ≥ F1-rate
C. Measures of Effectiveness of Models IF TN < TP THEN Accuracy< F1-rate.
We have used two the most known measures of the
model’s effectiveness – Accuracy and F1-rate. Accuracy
is defined by the following formula: IV. TEST RESULTS AND DISCUSSION
Testing results of classic machine learning models (NB,
SVM and DT), deep neural networks (LSTM, GRU), and
ensemble learning (RF and AdaBoost) trained on the Hate
Dataset are in Table I. (CML represents Classic Machine
learning, DL represents Deep Learning and EL represents
where TP is the number of true positive classifications, Ensemble Learning).
TN is the number of true negative classifications, FP is the
number of mistaken classifications – false positive
classifications and FN is the number of false negative
classifications. F1- rate is defined by the following way:

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 356

 TABLE I. effectiveness of models trained by all used machine
THE RESULTS OF MODELS TRAINED USING CLASSIC MACHINE LEARNING
METHODS (NB, SVM, DT), DEEP LEARNING NN (LSTM, GRU), AND
learning methods on all three datasets by only in Accuracy
ENSEMBLE LEARNING (RF, ADABOOST) ON HATE SPEECH DATASET measure in one overview Table IV.
REPRESENTED BY ACCURACY AND F1 MEASURES

Hate Speech TABLE IV.

THE RESULTS OF MODELS TRAINED USING CLASSIC MACHINE LEARNING
Dataset METHODS (NB, SVM, DT), DEEP LEARNING NN (LSTM, GRU), AND
ENSEMBLE LEARNING (RF, ADABOOST) ON HATE, OFFENSIVE AND
Support 6000 CYBERBULLYING DATASET REPRESENTED BY ACCURACY
Effectivity measure Acc. F1
CML NB 0.649 0.560 Dataset Hate Offensive Cyberbullying
SVM 0.893 0.840 (support) (6000) (35787) (11863)
DT 0.900 0.857 CML NB 0.649 0.893 0.923
DL NN (LSTM) 0.431 0.426 SVM 0.893 0.926 0.928
NN (GRU) 0.363 0.359 DT 0.900 0.924 0.922
EL RF 0.902 0.860 DL LSTM 0.431 0.956 0.513
AdaBoost 0.904 0.870 GRU 0.363 0.964 0.616
EL RF 0.902 0.916 0.932
AdaBoost 0.904 0.908 0.912
Testing results of classic machine learning models (NB,
SVM and DT), deep neural networks (LSTM, GRU), and
ensemble learning (RF and AdaBoost) trained on the For datasets where the detected class was not
Offensive Speech Dataset are in Table II. represented by large number of training examples, the
ensemble learning methods as AdaBoost for Hate Speech
Dataset (Accuracy=0.904) and RF for Cyberbullying
TABLE II.
THE RESULTS OF MODELS TRAINED USING CLASSIC MACHINE LEARNING Dataset (Accuracy=0.932) came out the best. On the
METHODS (NB, SVM, DT), DEEP LEARNING NN (LSTM, GRU), AND largest Offensive Speech Dataset, the neural networks
ENSEMBLE LEARNING (RF, ADABOOST) ON OFFENSIVE SPEECH were the best (Accuracy=0.964) using GRU. In the results
DATASET REPRESENTED BY ACCURACY AND F1 MEASURES
obtained by training on the Offensive Speech Dataset,
Offensive Speech Accuracy was higher than F1-rate. This means that the
Dataset neural networks were more successful at detecting neutral
Support 35787 than offensive posts. But that does not mean that they
Effectivity measure Acc. F1 would be less precise because of a high FP Conversely,
CML NB 0.893 0.921 with other machine learning methods, including ensemble
SVM 0.926 0.936 learning, F1-rate was greater than Accuracy.
DT 0.924 0.941
DL NN (LSTM) 0.956 0.945 V. CONCLUSION
NN (GRU) 0.964 0.949 The conclusion of our experiments is that the neural
EL RF 0.916 0.937 networks give better results only if they have sufficiently
AdaBoost 0.908 0.926 large datasets available. When dataset is not so large, the
better way is using the ensemble learning. The best result
Testing results of classic machine learning models (NB, on the smallest Hate Speech Dataset were achieved by
SVM and DT), deep neural networks (LSTM, GRU), and AdaBoost (Accuracy=0,904, F1-rate=0.870). On the other
ensemble learning (RF and AdaBoost) trained on the hand, the best result on the largest Offensive Speech
Cyberbulying Dataset are in Table III. Dataset was achieved by neural networks, particularly by
GRU (Accuracy=0.964, F1-rate=0.949).
TABLE III. The impact of this study is in comparison also different
THE RESULTS OF MODELS TRAINED USING CLASSIC MACHINE LEARNING approaches to machine learning strategy as classic, deep
METHODS (NB, SVM, DT), DEEP LEARNING NN (LSTM, GRU), AND and ensemble learning. Another useful contribution of the
ENSEMBLE LEARNING (RF, ADABOOST) ON CYBERBULLYING
DATASET REPRESENTED BY ACCURACY AND F1 MEASURES paper for research community is a finding, that deep
learning model are successful in offensive speech
Cyberbullying recognition, but for the recognition of hate speech or
Dataset cyberbullying the more effective are models obtained by
Support 11863 Ensemble learning. The conclusion is that both deep
Effectivity measure Acc. F1 learning and ensemble learning are better selection as
CML NB 0.923 0.650 classical machine learning methods.
SVM 0.928 0.696 With further optimization of the hyper parameters, or
DT 0.922 0.651 by expanding the dataset, or using augmentation of the
DL NN (LSTM) 0.513 0.496 current dataset, we could probably achieve a few tenths of
NN (GRU) 0.616 0.601 a percent better results. Future research could also focus
EL RF 0.932 0.708 on using capsule CCNs, or the use of other models for
AdaBoost 0.912 0.561 vector representation such as FastText or GloVE.
In the future, we could focus on the combination of
We can see from these three tables (Table I., II. and III.) deep learning and ensemble learning by training neural
that for all three datasets either deep learning or ensemble ensemble models, since ensemble models based on
learning was the best. For comparison, we present also the classical machine learning methods did not achieve the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 357

results we hoped for. We could also involve multimodal pp. 1–445, ISBN 978-1-4614-7137-0, DOI 10.1007/978-1-4614-
detection of toxic behavior in social media by extend text 7138-7.
processing to include processing of speech recordings and [8] V. Kandasamy, et al. “Sentimental Analysis of COVID-19 Related
Messages in Social Networks by Involving an N-Gram Stacked
images available in online comments, for example Autoencoder Integrated in an Ensemble Learning Scheme,”
emoticons or illustration images. Other direction for future Sensors, 21(22), 2021, 7582-7582.
research can be using various ensemble strategies to [9] V.Y. Kulkarni, and P.K. Sinha, “Pruning of random forest
increase a detection performance of set of models, for classifiers: a survey and future directions,” In Proc. of
example by using random forest voting strategy. International Conference on Data Science Engineering ICDSE
2012, Cochin, Kerala, India, 64–68.
ACKNOWLEDGMENT [10] K. Machova, et al., “Machine Learning and Lexicon Approach to
Texts Processing in Detection of Degrees of Toxicity in Online
This work was supported by the Scientific Grant Discussions,” In Sensors (Basel), 22(17), 2022, 6468-
Agency of the Ministry of Education, Science, Research 6468, https://doi.org/10.3390/s22176468.
and Sport of the Slovak Republic, and the Slovak [11] T. Mikolov, et al., “Advances in pertaining distributed word
Academy of Sciences under grant no. 1/0685/21 and by representations,” In Proc. of the International Conference on
the Slovak Research and Development Agency under Language Resources and Evaluation LREC 2018, Miyazaki,
Japan, 2018, 52-55.
Contract no. APVV–22–0414.
[12] Oinkina, X. et al. “Understanding LSTM Networks,” 2023,
Available online: http://colah.github.io/posts/2015-08-
REFERENCES Understanding-LSTMs/ (accessed on 03.02.2023).
[1] T.J. Bazhad, and M.A. Adnan, “Classification Based on Decision [13] I.R. Sharker, “Machine Learning: Algorithms, Real-Wold
Tree Algorithm for Machine Learning,” In Journal of Applied Applications and Research Directions,” SN Computer Science,
Science and Technology Trends, 2(1), 2021, 20-28, ISSN: 2708- vol. 2, March 2021, pp. 160–160.
0757, doi: 10.38094/jastt20165. [14] J.I. Sheeba, and S. Pradeep-Devaneyan, “Cyberbully Detection
[2] ͘ D͘ ŝƐŚŽƉ͕ EĞƵƌĂů EĞƚǁŽƌŬƐ ĨŽƌ WĂƚƚĞƌŶ ZĞĐŽŐŶŝƚŝŽŶ͘ KdžĨŽƌĚ from Twitter using Classifiers,” In CiiT - International Journal of
hŶŝǀĞƌƐŝƚLJWƌĞƐƐ͕ KdžĨŽƌĚ͕ϭϵϵϱ͘ Artificial Intelligent Systems and Machine Learning, 9(8), 2017,
[3] T. Davidson, et al., “Automated hate speech detection and the 163-168.
problem of offensive language,” In Eleventh International AAAI [15] Z͘^͘^ƵƚƚŽŶ͕͘'͘ĂƌƚŽ͘ZĞŝŶĨŽƌĐĞŵĞŶƚ>ĞĂƌŶŝŶŐ͗Ŷ/ŶƚƌŽĚƵĐƚŝŽŶ͘
Conference on Web and Social Media: Computation and ϮŶĚĞĚŝƚŝŽŶ͕ĂŵďƌŝĚŐĞ͕D͗D/dWƌĞƐƐ͖ϮϬϭϴ͘
Language, ICWSM 2017, Montreal, Quebec, Canada,512-515, [16] C. Tu, B. Xu, and H. Liu, “The Application of the AdaBoost
https://doi.org/10.48550/arXiv.1703.04009. Algorithm in the Text Classification,” 2nd IEEE Advanced
[4] Z͘K͘ ƵĚĂ͕ W͘͘ ,Ăƌƚ͕ ͘ '͘ ^ƚŽƌŬ͕ WĂƚƚĞƌŶ ůĂƐƐŝĨŝĐĂƚŝŽŶ͕ ϮŶĚ Information Management, Communicates, Electronic and
ĞĚŝƚŝŽŶ͕tŝůĞLJ͕EĞǁzŽƌŬ͕ ϮϬϬϭ͘ Automation Control Conference (IMCEC), Xi’an, China, 2018,
[5] A. Founta, et al. “A unified deep learning architecture for abuse 25-27, doi: 10.1109/IMCEC.2018.8469497.
detection,” In Proc. of the 10th ACM Conference on Web Science, [17] M. Zampieri, et al. “Predicting the type and target of offensive
Boston, Massachusetts USA, 2019, pp. 105-114, posts in social media,” In Proc. of the 2019 Conference of the
https://doi.org/10.1145/3292522.3326028. North American Chapter of the Association for Computational
[6] S. Hochreiter, and J. Schmidhuber, “Long Short-term memory,” In Linguistics: Human Language Technologies, Minneapolis,
Neural Computation, 9(8), 1997, 1735-1780; Minnesota, June 2019, 1415–1420.
DOI:10.1162/neco.1997.9.8.1735. [18] S. Zimmerman, et al., “Improving Hate Speech Detection with
[7] G. James, et al., “An Introduction to Statistical Learning – with Deep Learning Ensembles,” In Proceedings of the - Language
Applications in R - Springer Texts in Statistics,” Springer, 2014, Resources and Evaluation Conference LREC 2018, Miyazaki,
Japan, 2018, 2546-2553

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 358

 Eye-Tracking System as a Part of the Phishing
Training
M. Madleňák* and K. Kampová**
* University of Žilina/Faculty of Security Engineering, Žilina, Slovakia
** University of Žilina/Faculty of Security Engineering, Žilina, Slovakia
matus.madlenak@uniza.sk, katarina.kampova@uniza.sk

Abstract— This article explores the potential use of eye-

tracking technology within the context of phishing training. A. Slovakia
Phishing attacks continue to be a significant cybersecurity Based on available statistics, we can conclude that
threat, and one approach to preventing successful phishing among all known cyber threats, phishing attacks are the
attacks is to conduct phishing training. The article focused on most prevalent. Phishing attacks have been a long-
how eye-tracking systems can enhance phishing training by discussed topic within the professional community, as
analyzing users' gaze patterns when interacting with evidenced by the fact that a publication on this topic was
phishing emails or websites. By studying where users focus already released in 2003 within the Web of Science
their attention, trainers can gain insights into common database [2].
mistakes and areas of vulnerability. This approach offers However, phishing attacks are currently becoming even
data-driven and personalized training, ultimately improving more widespread, as indicated by the CSIRT.SK statistics
individuals' ability to detect and mitigate phishing attacks.
shown in Figure 1. According to this data, there were 265
The article focuses on how this eye-tracking system can be
reported cases of phishing attacks in 2019, and this number
utilized in phishing training and its potential to enhance
increased to 548 in 2020. In the following years, the number
cybersecurity awareness.
of reported phishing attacks exceeded the count of other
Keywords— Cyber security, security, phishing, eye-tracking reported cyber security incidents, with as many as 554
phishing attacks reported in the past year of 2022 [3].

I. INTRODUCTION
In the current era of information technology
development, there is a growing risk of cyber threats that
exploit the vulnerabilities of organizations or individuals to
gain unauthorized access to sensitive data, information, and
systems. Vulnerabilities in organizations and individuals
can be exploited by various types of cyber threats. Some of
the most well-known cyber threats include phishing,
malware, DDoS attacks, ransomware, spyware, and
adware. Therefore, a cyber threat is an event that can
negatively impact a system and may result in data damage Figure 1. Statistics on cyber-attacks - Slovakia
or destruction, system disruption, hindrance, and harm to
essential and digital services, as well as compromising
information security [1]. B. USA
Organizations must focus on implementing security The dominance of phishing attacks in terms of
measures to help protect themselves from cyberattacks. occurrence is also supported by statistics from the Federal
Therefore, this article will address the potential use of new Bureau of Investigation (FBI), which collect data on cyber
technologies in protecting against a specific type of security incidents within the United States. According to
cyberattack, namely, phishing attacks. This article may this data, in 2022, as many as 300,497 individuals was
offer a new perspective on protection and prevention victim of phishing, resulting in total financial losses of up
against phishing attacks. to 52 million dollars. Consequently, based on this statistic,
phishing attacks constitute the most prevalent type of cyber
II. STATISTIC OF CYBER ATTACK threat within the United States. For comparison, the second
most common cyber threat was personal data breaches,
Cyber-attacks represent a dynamic and evolving field affecting 58,859 individuals, but the financial losses were
that demands constant monitoring and an adaptive higher in comparison to phishing attacks, reaching 742
approach to protection. In this context, phishing attacks million dollars [4].
have emerged as one of the particularly serious threats
affecting organizations and individuals worldwide. This When comparing the trends in these two most common
chapter focuses on the statistical analysis of cyber-attacks, cyber security incidents since 2019, it is evident that
with a specific attention on phishing, highlighting its phishing attacks have a more significant upward trend than
prevalence in Slovakia and the United States. personal data breaches [4]. A comparison of cyber security
incidents is illustrated in Figure 2.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 359

 legitimate but contain malicious code, software,
or hyperlinks to malicious websites.
x Combination of methods: In a sophisticated
phishing attack, the phishing attacker may use all
these methods simultaneously or combine them
in various ways.
B. Characteristics
Phishing attacks generally exhibit characteristics that
make them identifiable. Less sophisticated phishing
attacks, such as email phishing, display more of these
Figure 2. Statistics on cyber-attacks - USA
characteristics and are easier to identify. Sophisticated
These statistics underscore the significant need to focus phishing attacks like spear phishing are harder to spot
on measures against phishing attacks, which can enhance because they are more personalized and try to effectively
the resilience of entities against this type of cyber threat, mask the significant indicators of a phishing attack [8].
ultimately helping to reduce the number of phishing attack Knowledge of these significant indicators of phishing
victims. attacks can be utilized, for instance, in designing security
measures to reduce the success of phishing attacks and
III. PHISHING ATTACK thereby enhance cybersecurity in an organization or private
setting. Significant indicators can be divided into the
A phishing attack is a type of cyber-attack that poses a following categories: text formulation, solicitation of
threat to the cyber security of organizations or individuals. personal information, irresistible offers, urgency,
This type of cyber-attack is differing in that phishing suspicious domains, attachments and hyperlinks [9] [10]
attackers employ social engineering techniques that rely on [11] [12].
gaining the trust of users. Through social engineering
techniques, attackers attempt to obtain personal Text formulation: Phishing attackers often incorrectly
information and other data from users, which can formulate text, make grammatical errors and use indirect
subsequently be exploited to gain access to user accounts, addresses without using a name or personal number. This
trade secrets, customer and employee data, or to cause may be because attackers translate the text from another
financial losses [5]. Attackers, therefore, exploit language and lack sufficient information about their victims
vulnerability rooted in the human factor. [9].
Phishing attacks are typically carried out through Images: the use of poor quality, non-standard or outdated
internet, SMS, or telephone communication, during which images (logos, photos) is also one of the indicators that can
they use the identity of trusted institutions, service alert the user to a fraudulent email [10].
providers, or individuals with whom the user has Solicitation of personal information: Many phishing
affiliations. In certain cases, however, attackers do not attacks explicitly request personal information from
utilize any credible identity and simply send phishing victims, such as login credentials, credit card information,
messages to a large number of users. A common feature of or other sensitive details. Legitimate organizations never
all phishing attacks is the so-called bait (link, attachment, ask their customers to provide personal information like
enticing offer), aimed at compelling the user to interact in a card numbers, login details, etc. [11].
certain way, which can make the phishing attack partially Irresistible offers: Phishing attackers frequently make
or entirely successful [5] [6]. irresistible and overly generous offers to their targets. For
example, they may claim that the person has won a grant,
A. Methods vacation, or other highly attractive offers [9].
Phishing attackers employ various methods to carry out Urgency: Urgent language can create a sense of urgency
phishing attacks with the aim of increasing their success in that compels victims to act quickly without thinking. This
obtaining personal information, delivering malware, or exerts psychological pressure on victims to obtain sensitive
directing victims to a phishing website. The methods used information from them [12].
in phishing attacks can be categorized into five categories Suspicious domains: Phishing attackers can create
[7]: domain names that resemble those of legitimate
x Text-based method: phishing attackers use text organizations but with minor differences. They might
formulations that psychologically influence the change letters, use a period instead of a comma, or employ
user to voluntarily provide personal information other unusual characters. It's also important to check the
URL because legitimate websites usually use the "https"
to the attacker.
protocol instead of "http" [13]. However, the presence of
x Link-based method: phishing attacks contain "https" does not guarantee legitimacy [14]. Currently, this
hyperlinks that direct users to a malicious website indicator can only detect very basic phishing attacks.
domain. Attachments, and hyperlinks: Phishing emails, SMS, or
x Image-based method: phishing attackers use text messages may contain hyperlinks or attachments that,
images, logos, or symbols to build credibility. when opened, install malware on the victim's computer. It's
Images may contain hyperlinks leading to important to be cautious if a message contains attachments
malicious website domains. or hyperlinks [12]. Before clicking on an attachment, it is
x Attachment-based method: phishing attacks necessary to verify the file type because legitimate
include suspicious attachments that appear organizations typically use ".pdf" or ".docx" files instead of

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 360

".exe" or ".zip" files. In the case of a hyperlink, it's crucial somewhat more sophisticated [10] [17]. A schematic
to check if the link leads to a legitimate website. representation of clone phishing is shown in Figure 5.
C. Types
With the increasing prevalence of phishing attacks, new
sophisticated types are also evolving, through which
attackers obtain users' personal information. Phishing
attacks can be carried out through SMS, voice calls, but the
largest number of phishing attacks occurs via the internet.
Within the internet, a phishing attacker primarily utilizes
email communication, online messaging, social networks,
or fake websites [10].
Phishing attacks can be categorized as follows: email
phishing, spear phishing, whaling phishing, clone phishing, Figure 5. Clone phishing
pharming and others (smishing, vishing, angler phishing).
Email phishing is a type of phishing attack where the Pharming is a sophisticated and technically challenging
attacker sends phishing email messages to a large number phishing attack that relies on compromising a DNS server
of random users. This type of phishing attack can target a (DNS spoofing), allowing the attacker to alter IP addresses
large number of users, but due to lower message in the DNS server's table. Pharming, therefore, redirects
personalization, it may have lower success rates and can be users to fraudulent web addresses even after entering the
easier to identify [15]. A schematic representation of email correct domain [18] [19]. A schematic representation of
phishing is shown in Figure 3. pharming is shown in Figure 6.

Figure 3. Email phishing Figure 6. Pharming

Spear phishing and Whaling phishing rely on the precise There are also other types of phishing attacks that use
selection of target users. While spear phishing targets a various communication channels and methods to attempt to
specific group of individuals (workgroup, businesses), obtain personal information from users. Phishing attacks
whaling phishing primarily focuses on high-ranking that use telecommunications networks include smishing
executives (managers, directors). Significant (SMS) and vishing (phone call), while phishing attacks
characteristics include a high degree of personalization, conducted through social networks are referred to as angler
which enhances credibility and ultimately the success of the phishing [20] [21] [22].
phishing attack [16]. A schematic representation of spear
phishing and whaling phishing is shown in Figure 4. IV. EYE-TRACKING SYSTEM
This chapter deals with the operation and applications of
eye-tracking systems. How these systems capture eye
movement data, what information is extracted from this
data, and how this information can be used in a scientific
environment will be explored. For the purposes of this
article, we will use the term eye tracking to mean any
hardware and software that is capable of monitoring eye
movement.
A. Description
Eye-tracking system, or eye tracker, is a technology that
is used to monitor and record the movements and positions
Figure 4. Spear phishing and whaling phishing of a person's eyes. It is a specialized device or software that
can accurately measure and analyze data that is obtained by
Clone phishing utilizes previously delivered email observing the movement of a respondent’s eyes [23].
messages as templates to increase its credibility. In this type
of phishing attack, hypertext links in the email message text
are typically modified to redirect users to fraudulent web
addresses. Compared to email phishing, clone phishing is

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 361

B. Eye tracking devices Gaze point: it is simply the point that the respondent is
Eye tracking requires specific hardware that, together currently looking at. The point of view can change as the
with specific software, can capture and record eye person moves their gaze from one place to another, this
movement. Usually, eye tracking hardware such as: eye creates a series, the so-called gaze points [32].
tracking glasses, remote eye tracker or webcam eye tracker. Fixations: fixations are short periods during which the
Eye tracking glasses: these glasses are equipped with eyes are static and focused on a specific point or area. There
integrated sensors and cameras that track the wearer's eye are other types of fixations that can be tracked, such as: time
movement. Eye tracking glasses can be used in static or to first fixation, first fixation duration, average fixation
non-static eye tracking [24]. duration, or fixation sequences [31] [33].
Remote eye tracker: these devices are placed on a desk Heatmaps: heatmaps are visualizations that show the
or under computer monitor in front of the user and track eye concentration of a view on an investigated scene or object.
movement when the user uses a computer mouse or They help to visualize which areas have high and low
keyboard. They are suitable for static eye tracking [25]. respondent attention [27].
Webcam eye tracker: it is a lower-cost variant of eye- Time spent (Dwell Time): this indicates the amount of
tracking that uses tracking of the eyes via a webcam or the time a respondent spends looking at a particular AOI or area
front-facing camera of a smartphone. With specific [34].
software, it is possible to track eye movement on the Eye tracking has the potential to provide diverse datasets
smartphone screen or monitor [26]. that can be used in a variety of fields of study. Therefore,
clear objectives need to be set and careful considered which
C. Eye tracking system in different fields data sets are relevant for the specific field of study.
Eye tracking is employed in various fields, such as:
transport, psychology, marketing or human-computer V. EYE-TRACKING SYSTEM IN PHISHING TRAINING
interaction. As mentioned in the previous chapter, eye tracking is
Transport and road safety: eye tracking is used to study used in various fields, but it is not widely prevalent in the
driver behaviour. It helps to analyze where drivers look realm of security. However, there is growing attention
while driving, which is important for designing safer car being paid to its potential applications in cybersecurity, and
environments. It also makes it possible to study driver it may play a more significant role in the future. Despite
fatigue and recognize signs of lack of attention [27]. this, ongoing research and the development of eye tracking
Psychology: in psychology, eye movement tracking is technology could lead to innovative security solutions, such
used to study attention, perception and emotions [26]. as enhanced authentication methods and real-time threat
detection, making eye tracking an increasingly valuable
Marketing: eye tracking is a valuable tool in marketing tool in the broader landscape of cybersecurity.
to study how people respond to advertisements and product
presentations [27]. Currently, we see the potential for straightforward use of
eye tracking in the context of phishing training. Employing
Human-computer interaction: eye tracking is used to eye tracking during phishing training can assist in better
research and develop human-computer interactions, such as assessing respondents' knowledge and analyzing their
controlling a computer with gaze. This can be very useful behavior during a phishing test.
for people with limited mobility [28].
A. Implementation
D. Inputs
To implement an eye tracking system into the phishing
Movement of the eyes: The main input to the eye training process, it is essential to have a thorough
tracking system is the eye movement data itself. Based on understanding of the technological limitations of this
eye tracking it is possible to obtain valuable information system and the object or scene that we want to observe. The
(outputs) such as: fixations, gaze points, heatmaps, time use of an eye tracking system makes sense when verifying
spent, ratio or revisit [29]. the knowledge of respondents about phishing directly
Investigated object or scene: This is a particular aspect before or after phishing training.
or area of interest that is being studied using eye tracking Knowledge verification is typically carried out through a
system. This area can be anything that is relevant to the phishing test, which includes examples of legitimate and
specific research objective or study question. In this object phishing messages (SMS or email). The aim of respondents
or scene, so-called areas of interests are usually identified is to decide, based on their knowledge, if message is
[30]. fraudulent or legitimate. After completing the test,
E. Outputs respondents are informed of the test results, and their
percentage of success is evaluated. However, it is
Eye tracking can provide a different data that is key to questionable whether this test provides accurate
analyzing and understanding the eye movements and information about respondents' abilities to detect phishing
behaviors of respondents while viewing the investigated attacks. The use of eye tracking, during phishing test can
object or scene [30]. The main outputs of eye tracking are: provide more data that can be analyzed to understand the
areas of interests, gaze points, fixations, heatmaps or time decision-making process of the respondent.
spent.
To demonstrate how eye tracking can be used in a
Areas of Interests: represent specific areas on the screen phishing test, we will use a test phishing email, as shown in
or scene that are selected for eye movement tracking. AOIs figure 7.
are defined before the actual eye tracking and are used to
analyze eye movement in these specific areas [31].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 362

 x Average number of fixations: calculating the
average number of fixations across all
respondents will give us insight into general gaze
behaviour and reveal whether some areas
consistently attract more attention.
x Time spent observing areas of interest: By
measuring the amount of time that respondents
observe each area of interest, it can be determining
which parts of the fraudulent email are examined
more closely.
x Creating heat maps: heat maps are created to
visualize the concentration of fixations on
different areas of interest. These heat maps can
offer a comprehensive view of where respondents
Figure 7. Test phishing email tend to focus their attention.
Analyzing these eye tracking metrics will allow us to
When analyzing the decision-making process using eye gain a deeper understanding of how respondents engage
tracking, it is crucial for the person conducting the analysis with phishing emails during the test. It is possible to assess
to be familiar with the significant characteristics that are whether they prioritize the crucial "areas of interest" that
typical of phishing attacks. These significant characteristics are indicative of phishing attempts and evaluate the
or areas where these characteristics may be found are effectiveness of their training in identifying these key
referred to as "areas of interest." elements.
In our test phishing email, we expect the respondent to
focus on and evaluate the following "areas of interest" VI. DISCUSSION
when detecting a phishing email: the subject line, sender's
As highlighted in the preceding chapters, phishing
contact information, images, text, and attachments. These
attacks stand as one of the most prevalent forms of cyber
areas are labeled as "areas of interest," as depicted in figure threats today. The article consisted of theoretical
8, with individual areas marked as A, B, C, D and E. knowledge about phishing attacks and the implementation
of an eye tracking system in phishing training. Theoretical
knowledge is important because it is critical in detecting
phishing attacks and creating security measures that
increase protection against phishing attacks. One of the
effective measures that increase protection against phishing
attacks is phishing training. This paper focused on
improving the phishing training process (which includes a
phishing test) using an eye tracking system.
While eye tracking has found its place in various fields,
its application in the realm of security has been relatively
limited. However, its potential in cybersecurity is becoming
increasingly likely. As technology continues to evolve, eye
tracking could emerge as an asset, contributing to
innovative security solutions and the potential for
integrating eye tracking into phishing training is
Figure 8. Areas of interest particularly promising.
The use of an eye tracking system in the context of
It is within these areas that the characteristics enabling a
cybersecurity can also be used in the framework of the
respondent to detect phishing may be found. The goal of
project APVV-20-0457 Monitoring and Tracking of
phishing training is to highlight these characteristics and
Movement and Contacts of Persons in Healthcare
teach respondents to pay attention to them. Eye tracking
Facilities. Phishing attacks pose a threat to the Healthcare
can indicate whether respondents focused on these critical
Facilities Monitoring and Tracking System, and it is
areas that are key to identifying phishing attempts.
important that staff in healthcare facilities receive phishing
To obtain valid data, it is necessary that the respondent's training to reduce the risk of successful phishing attacks.
eye movement is constantly monitored during the entire
duration of the test. ACKNOWLEDGMENT
B. Results The article was supported by The Ministry of Education,
Eye tracking systems allow us to track various outputs Science, Research and Sport of the Slovak Republic and
that have been obtained by software analysis of eye Slovak research and development agency grant number
movement. Relevant outputs for context of phishing APVV-20-0457 Monitoring and Tracking of Movement
training are mainly: and Contacts of Persons in Medical Facilities.
x Number of fixations: the number of times REFERENCES
respondents fixate or focus their gaze on certain
[1] H. Lallie, L. Shepherd, J. Nurse, A. Erola, G. Epiphaniou, C. Maple
areas of interest. This will help to understand and X. Bellekens. “Cyber security in the age of COVID-19: A
which areas attract the most attention.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 363

 timeline and analysis of cyber-crime and cyber-attacks during the [20] A. Mahmood and S. Hameed. „Review of Smishing Detection Via
pandemic,” Computers & Security, vol. 105, 2021. Machine Learning,“ Iraqi Journal of Science, vol. 64, pp. 4244–
[2] J. HOYLE. “'Phishing' for trouble,” Journal of The American 4259, 2023.
Dental Association, vol. 134, pp. 1182-1182, 2003. [21] K. Jones, M. Armstrong, M. Tornbald and A. Namin. „How social
[3] Incident statistics 2019 to 2022. Ministry of Investment, Regional engineers use persuasion principles during vishing attacks,“
Development and Informatization of the Slovak Republic. Information and Computer Security, vol. 29, pp. 314-331, 2020.
[4] Federal Bereau of Investigation - Internet Crime Complaint Center. [22] J. Odirichukwu. „SMART-IEEE-ACity-ICTU-CRACC-ICTU-
“2022 Internet Crime Report,” Available on: Foundations Series BOOK CHAPTER │ Overcoming Phishing
https://www.ic3.gov/Home/AnnualReports Phishing Attack, Types, Mitigations and a Typical Case Study,“
[5] R. Zieni, L. Massari and M. Calzorossa. “Phishing or Not Phishing? Web of DeceitPublisher: Creative Research Publishers, 2022.
A Survey on the Detection of Phishing Websites,” IEEE Access, vol [23] N. Nurbati. “EYE TRACKING TOOLS DEVELOPMENT,” 2nd
11, pp. 18499-18519, 2023. ICoHT 2022- International Conference Health Technology -
Poltekkes Kemenkes Jakarta IIAt: Jakarta, 2022.
[6] B. Bajanthri, Mr. Sayeesh. “A Study on Various Phishing
Techniques and Recent Phishing Attacks,” International Journal of [24] M. Dilbeck T. Gentry, J. Economides and J. Horton. “Quotidian
Advanced Research in Science, Communication and Technology, Profile of Vergence Angle in Ambulatory Subjects Monitored With
vol. 2, pp. 296-297, 2022. Wearable Eye Tracking Glasses,” Translational Vision Science &
[7] S. PALIATH, Q. MOHAMMAD and M. ALDWAIRI. “PhishOut: Technology, vol. 12, 2023.
Effective Phishing Detection Using Selected Features,” [25] T. Brunyé, T. Drew, D. Weaver and J. Elmore. “A review of eye
International Conference on Telecommunications (ICT), pp. 1-5, tracking for understanding and improving diagnostic
2020. interpretation,” Cogn Res Princ Implic, vol. 4, 2019.
[8] K. Singh, P. Aggarwal, P. Rajivan a C. Gonzalez. “Training to [26] S. Saxena, L. Fink and E. Lange. “Deep learning models for
Detect Phishing Emails: Effects of the Frequency of Experienced webcam eye tracking in online experiments,” Behavior Research
Phishing Emails,” Proceedings of the Human Factors and Methods, 2023.
Ergonomics Society Annual Meeting, vol. 63, pp. 453-457, 2019. [27] R. Madlenak, R. Chinoracky, N. Stalmasekova and L.
[9] M. Madlenak and K. Kampova. “Phishing as a Cyber Security Madlenakova. “Investigating the Effect of Outdoor Advertising on
Threat,” 20th International Conference on Emerging eLearning Consumer Decisions: An Eye-Tracking and A/B Testing Study of
Technologies and Applications (ICETA), pp. 392-396, 2022. Car Drivers’ Perception,” Applied sciences, vol. 13, 2023.
[10] S. Kumar, S. Gouda. “A COMPREHENSIVE STUDY OF [28] R. Jacob and K. Karn. “Eye Tracking in Human-Computer
PHISHING ATTACKS AND THEIR COUNTERMEASURES,” Interaction and Usability Research: Ready to Deliver the Promises,”
International Journal of Research and Analytical Reviews, vol. 10, The Mind's Eye Cognitive and Applied Aspects of Eye Movement
pp. 25-31, 2023. Research, pp. 573-605, 2003.
[11] H. Abroshan, J. Devos, G. Poels and E. Laermans. “COVID-19 and [29] B. Carter and S. Luke. “Best practices in eye tracking research,”
Phishing: Effects of Human Emotions, Behavior, and International Journal of Psychophysiology, vol. 155, pp. 49-62,
Demographics on the Success of Phishing Attempts During the 2020.
Pandemic”, IEEE Access, vol. 9, pp. 121916-121929, 2021. [30] L. Hahn and P. Klein. “Eye tracking in physics education research:
[12] B. Kotkova and M. Hromada. “Cyber Security and Social A systematic literature review,” Physical Review Physics Education
Engineering,” International Conference on Circuits, Systems, Research, vol. 18, 2022.
Communications and Computers (CSCC), pp. 134-140, 2021. [31] R. Madlenak, J. Masek, L. Madlenakova and R. Chinoracky. “Eye-
[13] L. Tang and Q. Mahmound. “A Survey of Machine Learning-Based Tracking Investigation of the Train Driver’s: A Case Study,”
Solutions for Phishing Website Detection,” Machine Learning and Applied sciences, vol. 13, 2023.
Knowledge Extraction, vol. 3, pp. 672-694, 2021. [32] K. Bai, J. Wang, H. Wang and X. Chen. “A Study of Eye-Tracking
[14] I. Dolnák and J. Litvik. “Introduction to HTTP security headers and Gaze Point Classification and Application Based on Conditional
implementation of HTTP strict transport security (HSTS) header for Random Field,” Applied sciences, vol. 12, 2022.
HTTPS enforcing,” IEEE 15th International Conference on [33] M. Recarte and L. Nunes. “Effects of verbal and spatial-imagery
Emerging eLearning Technologies and Applications (ICETA), pp. tasks on eye fixations while driving,” Journal of Experimental
1-4, 2017. Psychology: Applied, vol. 6, pp. 31-43, 2000.
[15] J. Rastenis et al. “E-mail-Based Phishing Attack Taxonomy,” [34] Y. Paulus and G. Rernijn. “Usability of various dwell times for eye-
Applied sciences, vol. 10, pp. 2363, 2020. gaze-based object selection with eye tracking,” Displays, vol. 67,
[16] P. Sharma, B. Dash and M. Ansari. “Anti-Phishing Techniques -A 2021.
Review of Cyber Defense Mechanisms,” International Journal of
Advanced Research in Computer and Communication Engineering,
vol. 11, pp. 153-160, 2022.
[17] A. Chauduri. “Clone Phishing: Attacks and Defenses”,
International Journal of Scientific and Research Publications, vol.
13, pp. 180-184, 2023.
[18] L. Njuguna and G. Kamau. “An Ensemble Learning Model for
Predicting Social Engineering Pharming Attacks”, International
Journal of Computer Applications Technology and Research, vol.
12, pp. 32-41, 2023.
[19] I. Dolnák, A. Jantosova and J. Litvik. “An overview of DNS
security in V2X networks,” 17th International Conference on
Emerging eLearning Technologies and Applications (ICETA), pp.
156-159, 2019.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 364

 +RZWR'HYHORS&76NLOOV8VLQJ5RERWLFV
3URMHFWV&RPELQHGZLWK$UWVD3LORW6WXG\

.0LNRYi DQG=.XELQFRYi 
&RPHQLXV8QLYHUVLW\%UDWLVODYD)DFXOW\RI0DWKHPDWLFV3K\VLFVDQG,QIRUPDWLFV%UDWLVODYD6ORYDNLD
^NDUROLQDPLNRYD]X]DQDNXELQFRYD`#IPSKXQLEDVN


$EVWUDFW²:LWK WKH LQWURGXFWLRQ RI GLVWDQFH OHDUQLQJ WKH ORFNGRZQ 6LPLODU VLWXDWLRQV ZHUH H[SHULHQFHG E\
GXULQJ WKH ORFNGRZQ WHDFKHUV IDFHG D QXPEHU RI FROOHDJXHVLQVHYHUDO(XURSHDQFRXQWULHVZKRDUHLQYROYHG
SUREOHPV RQ KRZ WR LPSOHPHQW WKH WHDFKLQJ LQ WHDFKHU HGXFDWLRQ DQG HGXFDWLRQDO URERWLFV %DVHG RQ
(GXFDWLRQDO URERWLFV ZDV SDUWLFXODUO\ QHJDWLYHO\ WKLV H[SHULHQFH WKH LGHD RI FUHDWLQJ D PHWKRGRORJ\ DQG
DIIHFWHG E\ WKLV SHULRG ,Q RXU FRXQWU\ HGXFDWLRQDO SODWIRUP WKDW ZRXOG VXSSRUW WKH WHDFKLQJ RI HGXFDWLRQDO
URERWLFV LV XVXDOO\ WDXJKW LQ LQIRUPDWLFV FODVVHV URERWLFVHYHQLQVLWXDWLRQVOLNHWKHORFNGRZQZDVERUQ
+RZHYHU LQ PDQ\ VFKRROV WKH WHDFKLQJ RI 1RZDGD\V 67($0 WHDFKLQJ LV FRPPRQ LQ PDQ\
LQIRUPDWLFVGXULQJWKHORFNGRZQZDVYHU\OLPLWHGDQG FRXQWULHV>@ZKLFKVHHNVWRLQFUHDVHVWXGHQWV¶FUHDWLYLW\
RIWHQIRFXVHGRQO\RQEDVLFGLJLWDOVNLOOV$VDVLPLODU LQ VFLHQFH WHFKQRORJ\ HQJLQHHULQJ DQG PDWKHPDWLFV
VLWXDWLRQRFFXUUHGLQVHYHUDO(XURSHDQFRXQWULHVWKH
OHDUQLQJ E\ LQFRUSRUDWLQJ HOHPHQWV RI DUWV HGXFDWLRQ >@
LGHD RI VXSSRUWLQJ WHDFKHUV LQ VXFK FDVHV ZDV
7KLVLGHDKDVEHHQDGRSWHGDQGGHYHORSHGE\DQ(UDVPXV
SURSRVHG :LWKLQ DQ LQWHUQDWLRQDO SURMHFW ZH DUH
SURMHFW >@ 2QH RI LWV REMHFWLYHV LV WR GHYHORS D
LQYROYHG LQ D PHWKRGRORJ\ LV EHLQJ GHVLJQHG WR
GHYHORS FRPSXWDWLRQDO WKLQNLQJ LQ VWXGHQWV DW DOO
PHWKRGRORJ\>@IRUWHDFKHUVWREHDEOHWRLQGHSHQGHQWO\
VFKRRO OHYHOV WKURXJK HGXFDWLRQDO URERWLFV LQ
FUHDWH KLJKTXDOLW\ FURVVFXUULFXODU SUREOHPRULHQWHG
FRPELQDWLRQ ZLWK D VFKRRO DUWV VXEMHFW 7KH URERWLFV SURMHFWV $QRWKHU REMHFWLYH RI WKH SURMHFW LV WR
PHWKRGRORJ\ LV EDVHG RQ WKH LGHD RI SURMHFWEDVHG FUHDWH D SODWIRUP WKDW ZRXOG IRVWHU FRPPXQLFDWLRQ DQG
OHDUQLQJ DQG VKRXOG XVH EOHQGHG OHDUQLQJ PHWKRGV FROODERUDWLRQERWKDPRQJWHDFKHUVDQGDPRQJVWXGHQWV
,Q WKLV SDSHU ZH GHVFULEH WKH SODQQLQJ DQG WKH 3UDFWLFHVKRZVWKDWLQRXUFRXQWU\LWLVYHU\GLIILFXOWWR
FRXUVH RI DQ HGXFDWLRQDO DFWLYLW\ ± D SURMHFW ± XVLQJ LPSOHPHQW FURVVFXUULFXODU SURMHFWV WKDW UHTXLUH WKH
WKH FXUUHQWO\ GHYHORSHG PHWKRGRORJ\ 7KH UHVXOWV FRRSHUDWLRQRIVHYHUDOWHDFKHUVRIGLIIHUHQWVXEMHFWV>@
SUHVHQWHG DUH IURP LWV SLORW YDOLGDWLRQ ZLWK ORZHU 7KLV LV DOVR HYLGHQFHG E\ WKH FODLPV RI PDQ\ WHDFKHUV
VHFRQGDU\VWXGHQWVZLWKRXWSUHYLRXVH[SHULHQFHZLWK IURP VFKRROV ZH FRRSHUDWH ZLWK LQVHUYLFH WHDFKHUV
URERWLFV 7KH SURMHFW FRPELQHG WKH WKHPH RI *UHHN VWXG\LQJ LQ RXU IDFXOW\ DQG WHDFKHUV SDUWLFLSDWLQJ LQ RXU
P\WKV ZLWK URERWLFV XVLQJ /(*2 63,.( 3ULPH UHVHDUFK DV ZHOO 7KH FUHDWLRQ RI WKH PHWKRGRORJ\
'XULQJ WKH YDOLGDWLRQ ZH IRFXVHG SULPDULO\ RQ PHQWLRQHGDERYHLVLQWHQGHGWRIDFLOLWDWHWKLVFROODERUDWLRQ
VWXGHQWV¶ ZRUN ZLWK EDVLF SURJUDPPLQJ FRQFHSWV DQG WR KHOS ZLWK WKH SUHSDUDWLRQ RI FURVVFXUULFXODU
2XUUHVHDUFKSUREOHPZDVWRYDOLGDWHWKHTXDOLW\DQG SURMHFWV RU WR RIIHU UHDG\PDGH H[HPSODU\ PDWHULDOV WKDW
SRWHQWLDO RI WKH SURSRVHG PHWKRGRORJ\ DQG WKH ZRXOG FRPELQH HGXFDWLRQDO URERWLFV DQG RWKHU VXEMHFWV
FRPSXWDWLRQDO WKLQNLQJ VNLOOV LW FDQ GHYHORS 7KH VXFKDVPXVLFHGXFDWLRQYLVXDODUWVOLWHUDWXUHGUDPDHWF
SDSHU FRQFOXGHV ZLWK UHFRPPHQGDWLRQV IRU IXUWKHU ZKLOHIRFXVLQJSULPDULO\RQWKHGHYHORSPHQWRIWKHVNLOOV
GHYHORSPHQWRIWKHPHWKRGRORJ\DQGDOVRIRUWKHXVH RIFRPSXWDWLRQDOWKLQNLQJ &7 >@
RI WKH VHOHFWHG URERWLF NLW ZLWK UHVSHFW WR
7KH UHVHDUFK ILQGLQJV LQ WKLV SDSHU GHVFULEH WKH
SURJUDPPLQJLQLQIRUPDWLFVFODVVHV
H[SHULHQFH RI DSSO\LQJ WKLV PHWKRGRORJ\ LQ D ODUJH FLW\
SULPDU\VFKRROZKHUHDFURVVFXUULFXODUSUREOHPRULHQWHG
, ,1752'8&7,21 SURMHFW OLQNLQJ HGXFDWLRQDO URERWLFV DQG DUWV ZDV
LPSOHPHQWHG
$IWHU WKH LQWURGXFWLRQ RI ORFNGRZQ PDQ\ SUREOHPV
DURVHIRUWHDFKHUVRQKRZWRLPSOHPHQWGLVWDQFHOHDUQLQJ ,, '(9(/23('0(7+2'2/2*<)25&5($7,1*
>@,QDGGLWLRQWRYDULRXVSV\FKRORJLFDODQ[LHWLHVYDULRXV $&7,9,7,(6
WHFKQLFDO LVVXHV ERWK RQ WKH SDUW RI WKH VWXGHQWV DQG WKH
WHDFKHUVZHUHDPRQJWKHIUHTXHQWO\FLWHGSUREOHPV>@,Q 7KH PHWKRGRORJ\ LV GHYHORSHG EDVHG RQ D VWXG\ >@
WHUPV RI WHDFKLQJ HGXFDWLRQDO URERWLFV LQ SDUWLFXODU KDV WKDW DGGUHVVHG WKH GHYHORSPHQW RI &7 VNLOOV >@ GXULQJ
EHHQ QHJDWLYHO\ DIIHFWHG E\ RQOLQH OHDUQLQJ ,Q RXU HGXFDWLRQDOURERWLFVDFWLYLWLHV>@,QLWVGHYHORSPHQW
FRXQWU\ HGXFDWLRQDO URERWLFVLV QRWD FRPSXOVRU\ SDUWRI WKHDLPZDVDOVRWRWDNHLQWRDFFRXQWWKHUHTXLUHPHQWVRI
WKH FXUULFXOXP EXW LQ PDQ\ VFKRROV WHDFKHUV XVH LW LQ WKHSURMHFWDQGWKHIDFWWKDWWKHPHWKRGRORJ\LVLQWHQGHGWR
LQIRUPDWLFVFODVVHVWRWHDFKSURJUDPPLQJ VHUYH DV D WRRO IRU WHDFKHUV &ROODERUDWLYH OHDUQLQJ ZDV
%DVHGRQWKHLQWHUYLHZVZHFRQGXFWHGZLWKWHDFKHUVDW FKRVHQDVWKHEDVLFFRQFHSWZKLFKSURYHVWREHVXLWDEOHLQ
GLIIHUHQW OHYHOV RI HGXFDWLRQ >@ ZH OHDUQHG WKDW FRPELQDWLRQ ZLWK HGXFDWLRQDO URERWLFV >@ 7KH
LQIRUPDWLFV ZDV KDUGO\ WDXJKW DW DOO RU LI LW ZDV LW ZDV PHWKRGRORJ\FXUUHQWO\FRQVLVWVRIILYHVWHSV
IRFXVHG RQ EDVLF GLJLWDO VNLOOV RU LQ WKH EHVW FDVH RQ  8QGHUVWDQGLQJ WKH &KDOOHQJH ± 7KH VWXGHQWV
SURJUDPPLQJ $PRQJ WKH WHDFKHUV LW ZDV PRUH RI DQ LGHQWLI\WKHFKDOOHQJHJLYHQ
H[FHSWLRQ LI WKH\ DOVR WDXJKW HGXFDWLRQDO URERWLFV GXULQJ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 365

  *HQHUDWLQJ ,GHDV ± 7KH VWXGHQWV VNHWFK RQH RU 7KH HWKLFDO UXOHVRI WKHUHVHDUFKHU>@ ZHUH IROORZHG LQ
PRUH LGHDV WKDW FRXOG VDWLVI\ WKH FRQGLWLRQV JLYHQ WKHVHOHFWLRQSURFHVV
LQWKHFKDOOHQJH $ FODVV RI  QLQWKJUDGH VWXGHQWV DJHG ±  ZDV
 )RUPXODWLQJWKH6ROXWLRQ±2QHLGHDLVWUDQVIRUPHG VHOHFWHGIRUWKHSURMHFW7KHVWXGHQWVLQWKLVFODVVGLGQRW
LQWR D IRUPXODWLRQ RI WKH 6ROXWLRQ ZKLOH ZRUN ZLWK URERWV GXULQJ WKH FRYLG SDQGHPLF GXH WR WKH
FRQVLGHULQJWKHUHTXLUHPHQWVRIWKHSUREOHPDQGE\ ORFNGRZQ DQG WKXV WKLV ZDV WKHLU ILUVW H[SHULHQFH ZLWK
PRELOL]LQJ WKH NQRZOHGJH UHODWHG WR WKH HGXFDWLRQDO URERWLFV $OWKRXJK WKH\ KDG SUHYLRXVO\
FKDUDFWHULVWLFVRIWKHURERWDQGWKHDUWLIDFW ZRUNHG LQ WKH 6FUDWFK HQYLURQPHQW WKH NQRZOHGJH RI
 &UHDWLQJ WKH 6ROXWLRQ ± 7KH VWHS LV VXFFHVVIXOO\ SURJUDPPLQJ FRQFHSWV ZDV DW GLIIHUHQW OHYHOV IRU WKH
H[HFXWHG ZKHQ WKH IRUPXODWHG 6ROXWLRQ LV IXOO\ LQGLYLGXDO VWXGHQWV $FFRUGLQJ WR ERWK WKH FODVVURRP
FRPSOHWHG WKURXJK WKH FRQVWUXFWLRQ RI WKH DUWLIDFW WHDFKHU DQG WKH LQIRUPDWLFV WHDFKHU WKH FODVV ZDV VPDUW
DQGWKHSURJUDPPLQJRIWKHURERW DQGFROODERUDWLYH
 (YDOXDWLQJ WKH 6ROXWLRQ ± :KLOH WKH URERW & 'DWD
SHUIRUPVWKHVWXGHQWVREVHUYHWKHUHDOL]HGDUWLIDFW
DQG WKH SURJUDPPHG URERW DQG HYDOXDWH WKHLU 'XULQJWHVWLQJ*'35FRPSOLDQWSHUPLVVLRQVZHUHQRW
FRUUHVSRQGHQFH WR WKH FRQGLWLRQV RI WKH SUREOHP VHFXUHG IRU WKH SURYLVLRQ RI YLGHR DQG DXGLR UHFRUGLQJV
DQGWKHLUDGHTXDF\LQJHQHUDO VRRQO\WKHW\SHVRIGDWDWKDWZRXOGQRWFDSWXUHIDFHVDQG
RWKHU VHQVLWLYH GDWD DERXW VWXGHQWV ZHUH UHFRUGHG 7KXV
(DFK VWHS LV VWUXFWXUHG LQWR D VHTXHQFH RI DFWLYLWLHV WKH ILUVW W\SH RI GDWD WKDW ZDV FROOHFWHG DQG DQDO\]HG
)RUHDFKRQH          
GXULQJ VHVVLRQ ZDV PRVWO\ MXVW SKRWRJUDSKV DQG YLGHR
    ZKDW W\SH RI DFWLYLW\ LW LV ZKLFK UHFRUGLQJVRIWKHPRGHOVDQGRWKHUSURSVFRQVWUXFWHG>@
VXEMHFW (GXFDWLRQDO URERWLFV RU $UWV  2WKHU GDWD FROOHFWHG ZHUH IURP ZRUNVKHHWV WKDW VWXGHQWV
LW ZLOO WDNH SODFH LQ ZKLFK &7 VNLOO LW LV LQWHQGHG WR FRPSOHWHG GXULQJ WKH ILUVW VHVVLRQ DQG GDWD IURP D VKRUW
GHYHORSDQGPDQ\RWKHUFKDUDFWHULVWLFVRIWKHDFWLYLW\)RU ILQDO RQOLQH TXHVWLRQQDLUH LQ ZKLFK VWXGHQWV SHHU
WKHVSHFLILFDWLRQRIPRVWFKDUDFWHULVWLFVWKHPHWKRGRORJ\ HYDOXDWHG HDFK RWKHU¶V SURMHFWV DW WKH HQG RI WKH VHVVLRQ
RIIHUVDFKRLFHRIRSWLRQVHJIRU&7VNLOOVWKHIROORZLQJ 7KHODVWW\SHRIGDWDZDVFROOHFWHGGXULQJLQWHUYLHZVZLWK
OLVWRIRSWLRQVLVFXUUHQWO\DYDLODEOH WKHLQIRUPDWLFVWHDFKHUEHIRUHGXULQJDQGDIWHUWHVWLQJ
• $EVWUDFWLRQ
• 'HFRPSRVLWLRQ ,9 &2856(2)7(67,1*
• 3DWWHUQUHFRJQLWLRQ
$ 3URMHFW3ODQQLQJ
• $OJRULWKPLFWKLQNLQJ
$WWKHEHJLQQLQJRIWKHSODQQLQJSURFHVVVHYHUDOMRLQW
• (YDOXDWLRQ RQOLQH PHHWLQJV ZHUH KHOG ZLWK WKH WHDFKHUV RI OLWHUDWXUH
DQG LQIRUPDWLFV ZKHUH WKH FKDUDFWHULVWLFV RI WKH SURMHFW
,,, 5(6($5&+ ZHUH GHVLJQHG DQG ZULWWHQ GRZQ DFFRUGLQJ WR WKH
UHFRPPHQGDWLRQV RI WKH PHWKRGRORJ\ DGDSWHG WR WKH
$ 5HVHDUFK2EMHFWLYHV FXUUHQW SRVVLELOLWLHV RI WKH VFKRRO 6SHFLILF GDWHV IRU WKH
7KH UHVHDUFK SUREOHP LQ WKLV SDSHU ZDV WR YHULI\ WKH LQWURGXFWLRQ RI WKH SURMHFW LQ WKH FODVVURRP ZHUH DOVR
TXDOLW\ DQG SRWHQWLDO RI WKH SURSRVHG PHWKRGRORJ\ 7KDW DJUHHG DQG WKH QHFHVVDU\ PDWHULDOV ZHUH SUHSDUHG 7KH
LV ZKHWKHU LW FDQ EH XVHG WR GHVLJQ DQG LPSOHPHQW D PHWKRGRORJ\ WR EH YDOLGDWHG FRQWDLQV VHYHUDO VWHSV VHH
FROODERUDWLYH SURMHFW WKDW FRPELQHV HGXFDWLRQDO URERWLFV 6HFWLRQ   WKDW VKRXOG JXDUDQWHH WKH TXDOLW\ RI WKH
DQG GUDPDWLF SHUIRUPDQFH ZLWK D SDUWLFXODU HPSKDVLV RQ LPSOHPHQWDWLRQ RI FROODERUDWLYH SURMHFWEDVHG OHDUQLQJ
GHYHORSLQJ VSHFLILF &7 VNLOOV 7KHUHIRUH WKH IROORZLQJ )ROORZLQJ WKHVH VWHSV LW ZDV SODQQHG KRZ WKH
WZRUHVHDUFKREMHFWLYHVZHUHVHWILUVWWRGHVLJQDURERWLF LPSOHPHQWDWLRQ RI WKH SURMHFW DW WKH VFKRRO ZRXOG EH
GUDPDWLFSURMHFWEDVHGRQWKHPHWKRGRORJ\RXWOLQHGDERYH FDUULHGRXW
DQG LQ FROODERUDWLRQ ZLWK LQIRUPDWLFV DQG OLWHUDWXUH )LUVWDVXLWDEOHWRSLFZDVVHOHFWHG7KLVZDVWKH*UHHN
WHDFKHUV VHFRQG WR DVVHVVLGHQWLI\ WKH &7 VNLOOV XVHG E\ P\WKV WRSLF WKDW ZDV UHFHQWO\ FRYHUHG LQ OLWHUDWXUH FODVV
WKH VWXGHQWV LQ FODVV ZKLOH ZRUNLQJ RQ WKH SURMHFW 7R :LWKLQ WKLV WRSLF WKH VWXGHQWV ZHUH DVNHG WR ZRUN LQ
PHHW WKHVH REMHFWLYHV LW ZDV QHFHVVDU\ WR GHVLJQ WKH JURXSV WR VWXG\ DQ DVVLJQHG P\WK DQG SUHSDUH D VKRUW
SURMHFW LPSOHPHQW LW LQ WKH FODVVURRP DQG DQDO\]H WKH SHUIRUPDQFHRI LW WR KHOS WKHLUFODVVPDWHV XQGHUVWDQGWKH
GDWDFROOHFWHG SORWRIWKHP\WKDQGUHPHPEHULW7KHVWXGHQWVWKHQKDGWR
ILJXUH RXW KRZ WR LQFRUSRUDWH WKH URERW LQWR WKLV
% 3DUWLFLSDQWV SHUIRUPDQFH7KHURERWFRXOGKDYHEHHQXVHGWRFRPSOHWH
7KH VHOHFWLRQ RI SDUWLFLSDQWV IRU WKLV UHVHDUFK ZDV WKHVFHQHLQZKLFKWKHP\WKWRRNSODFHEXWLWFRXOGDOVR
OLPLWHG WR WKH DVVRFLDWHG VFKRROV RI WKH (UDVPXV SURMHFW KDYH UHSUHVHQWHG D VSHFLILF FKDUDFWHU IURP WKH P\WK 7R
DQGZDVDOVRLQIOXHQFHGE\WKHZLOOLQJQHVVRIWKHDUWDQG PRWLYDWH WKH SDUWLFLSDWLQJ VWXGHQWV ZKR KDG QRW ZRUNHG
LQIRUPDWLFV WHDFKHUV ZRUNLQJ LQ WKHVH VFKRROV %DVHG RQ ZLWK URERWV EHIRUH WKH VWXGHQWV ZHUH GLUHFWHG WR XVH WKH
FULWHULD DQG DYDLODELOLW\ WLPH DQG ORFDWLRQ  RQH VFKRRO LQSXWRXWSXW FRPPXQLFDWRUV LQ WKHLU SURMHFW ZKLFK DUH
ZDVVHOHFWHGZKHUHWHDFKHUVRIERWKVXEMHFWVZHUHZLOOLQJ W\SLFDO IRU URERWLFV DQG PDNH URERW SURJUDPPLQJ PRUH
WR FROODERUDWH RQ D JLYHQ GDWH 7KLV LV D SULYDWH VFKRRO LQWHUHVWLQJ FRPSDUHG WR RUGLQDU\ SURJUDPPLQJ 7KH
ZLWKFHUWDLQVSHFLILFIHDWXUHVLQFOXGLQJWKHLQFRUSRUDWLRQ VWXGHQWV FRXOG XVH HLWKHU PRWLRQ VRXQG RU OLJKWV LH
RI FROODERUDWLYH SURMHFWEDVHG OHDUQLQJ LQWR WKH UHJXODU
FODVVHVRUHQGRIWHUP WHVWVLQ DOO VXEMHFWVLQVWHDG RIWKH  
FRQWLQXRXVJUDGLQJWKDWLVFRPPRQLQPRVWRIRXUVFKRROV 
KWWSVVFUDWFKPLWHGX


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 366

FHUWDLQ EDVLF FRQFHSWV IURP WKH SURJUDPPLQJ VHVVLRQVWXGHQWVVKRXOGKDYHEHHQDEOHWRPRYHWKHURERW
HQYLURQPHQW $V WKH SURMHFW IRFXVHG RQ WKH XVH RI EDVLF ZLWKGLIIHUHQWFRPPDQGVUHFRUGHGLWDQGWULJJHUVRXQGV
FRPPDQGVDVZHOODVRQFUHDWLYHZRUNWKH/(*263,.( RU OLJKW XS GLIIHUHQW FRORUV XVLQJ WKH /(' OLJKWV RQ WKH
3ULPHURERWLFNLWZDVFKRVHQIRUWKHUHVHDUFK FXEH7KH\ ZHUHDOVRWDVNHGZLWKUHWKLQNLQJWKHVFHQDULR
6WXGHQWVILUVWKDGWRIDPLOLDUL]HWKHPVHOYHVZLWKWKHNLW DQGFKHFNLQJZKHWKHU/(*263,.(3ULPHDOORZHGWKHP
LWVHOILWVKDUGZDUHFRPSRVLWLRQDVZHOODVWKHFRPPDQGV WR UHDOL]H WKHLU LGHD RI LQYROYLQJ WKH URERW LQ WKH
WKDW FRQWURO VRXQG OLJKWLQJ DQG EDVLF PRYHPHQW SHUIRUPDQFH ,I WKH\ KDG WLPH OHIW WKH\ FRXOG VWDUW
7KHUHIRUHZHFUHDWHGDVKRUWZRUNVKHHWIRUWKLVSXUSRVH ZRUNLQJRQWKHFRQVWUXFWLRQRIWKHPRGHO
ZKLFK HIIHFWLYHO\ JXLGHG WKH VWXGHQWV LQ IDPLOLDUL]LQJ 6WXGHQWV ZRUNHG ZLWK LQWHUHVW VROYHG PRUH WHFKQLFDO
WKHPVHOYHVZLWKWKHHQYLURQPHQWDQGFRQWURORIWKHURERW WKDQ FRJQLWLYH SUREOHPV DQG QR VLJQLILFDQW
LQDUHODWLYHO\VKRUWSHULRGRIWLPH FRPSUHKHQVLRQ SUREOHPV ZHUH QRWHG $ QHJDWLYH DWWLWXGH
%HIRUHWKHWHVWLQJVWDUWHGDWWKHVFKRROVHYHUDOWHFKQLFDO ZDV RQO\ LGHQWLILHG ZKHQ FRPSOHWLQJ WKH ZRUNVKHHW
SUHSDUDWLRQVKDGWREHPDGHXSGDWLQJWKHVRIWZDUHRQWKH DOWKRXJK WKLV ZDV RQO\ PLQLPDO ZRUN DQG WKH\ ZHUH
URERWVLQVWDOOLQJDSSVRQWKHWDEOHWVDQGSDLULQJWKHWDEOHWV UHPLQGHG RI WKLV VHYHUDO WLPHV $V HDFK JURXS ZDV
ZLWKWKHURERWVFKDUJLQJWKHURERWVMXVWEHIRUHWKHWHVWLQJ EXLOGLQJ D GLIIHUHQW PRGHO WKLV ZDV XVHG WR FKHFN KRZ
ZKLFK WRRN VHYHUDO KRXUV EULQJLQJ WKH NLWVDQG WDEOHWV ZHOOWKH\XQGHUVWRRGWKHFRPPDQGVIRUSURJUDPPLQJWKH
WR WKH VFKRRO FRQQHFWLQJ WKHP WR WKH VFKRRO :L)L DQG URERW¶V EHKDYLRU )XUWKHU UHGXFHG DWWHQWLRQ GXULQJ WKH
WHVWLQJWKHLUIXQFWLRQDOLW\ VHFRQGOHVVRQZDVLGHQWLILHG7KHIDFWWKDWWKH\ ZHUHQRW
XVHGWRZRUNIRUPLQXWHVZLWKRXWDEUHDNZDVHYLGHQW
% &RXUVHRIWKH3URMHFW   )LIWK6HVVLRQ
7KH SURMHFW LWVHOI ZDV GHVLJQHG IRU  VHVVLRQV ZLWK 7KHUHVHDUFKHUVFRQGXFWHGWKLVVHVVLRQWZLFHHDFKZLWK
VRPH VHVVLRQV ODVWLQJ RQH OHVVRQ  PLQXWHV  DQG VRPH KDOIRIWKHFODVV7KLVGLYLVLRQZDVGRQHLQRUGHUWRKDYH
ODVWLQJ WZR OHVVRQV  PLQXWHV  7KH WHVWLQJ LWVHOI LQ *UHHN P\WKV SUHVHQWHG LQ HDFK VHVVLRQ (DFK RI WKHVH
ZKLFKZHSDUWLFLSDWHGDVUHVHDUFKHUVWRRNSODFHRYHUWZR VHVVLRQV ODVWHG WZR OHVVRQV 'XULQJ WKH ILUVW OHVVRQ
GD\VZKHQZHKDGWKUHHVHVVLRQVZLWKWKHVWXGHQWVHDFK VWXGHQWVFRPSOHWHGWKHZRUNLQSURJUHVVPRGHOVWKDWWKH\
ODVWLQJOHVVRQV KDGEHJXQWREXLOGLQWKHSUHYLRXVVHVVLRQ*URXSVRIIRXU
  )LUVW6HVVLRQ ZRUNHGWRJHWKHUDJDLQVRWKDWHDFKJURXSKDGWZRURERWV
,QOLWHUDWXUHOHVVRQVWKHVWXGHQWVGLVFXVVHGWKHWRSLFRI LQWKHSUHYLRXVVHVVLRQWKH\ZRUNHGLQSDLUVDQGHDFKSDLU
*UHHN P\WKV ZLWK WKHLU WHDFKHU DQG ZHUH JLYHQ D SURMHFW EXLOW RQH URERW  'XULQJ WKH VHFRQG KDOI RI WKLV VHVVLRQ
DVVLJQPHQW7KH\ZHUHGLYLGHGLQWRWHDPVRIVWXGHQWV VWXGHQWV SUHVHQWHG WKHLU SHUIRUPDQFHV ZLWK WKH URERW
DQGHDFKJURXSZDVDVVLJQHGRQHP\WK7KHLUWDVNZDVWR PRGHOV VHH )LJ   (DFK SHUIRUPDQFH RI D P\WK ZDV
VWXG\ LW FUHDWH WKHLU RZQ VFHQDULR RI D ±PLQXWH IROORZHG E\ D VKRUW HYDOXDWLRQ E\ FODVVPDWHV 7KH
SHUIRUPDQFH DQG OHDUQ KRZ WR SHUIRUP DFFRUGLQJ WR WKH HYDOXDWLRQ IRUP WKH\ ILOOHG RXW ZDV FUHDWHG LQ *RRJOH
VFHQDULR )RUPV DQG WKH\ ORJJHG LQ YLD D 45 FRGH 7KH TXHVWLRQV
WKDWZHUHLQWKHHYDOXDWLRQIRUPZHUHDVIROORZV
  6HFRQG6HVVLRQ
 <RXUQDPH
'XULQJ WKH QH[W VHVVLRQ WKH VWXGHQWV FRPSOHWHG WKH
SURSVWRJHWKHULQDJURXSDQGPDGHSHUIRUPDQFHVLQFODVV  :KLFKP\WKDUH\RXUDWLQJ
SUHVHQWLQJWKHSUHSDUHGVWRULHVWRWKHLUFODVVPDWHV  5DWH WKH LQFRUSRUDWLRQ RI WKH URERW LQWR WKH
  7KLUG6HVVLRQ SUHVHQWHGP\WK 6FDOHOHDVWWRPRVW 
,Q WKH QH[W OLWHUDWXUH OHVVRQ WKH VHFRQG SDUW RI WKH  +RZ ZRXOG\RXUDWHWKHXVHRIWKH/(*263,.(
SURMHFW ZDV SUHVHQWHG WR WKH VWXGHQWV ZKHUH WKH\ ZHUH URERW LQ WHUPV RI WKH SRVVLELOLWLHV LW SURYLGHV"
LQWURGXFHGWRWKHSUREOHPWDVNWRLQFRUSRUDWHURERWVLQWR 6FDOHOHDVWWRPRVW 
WKHLU VFHQDULRV )RU KRPHZRUN WKH\ KDG WR LGHQWLI\ WKH  :KDW ZRXOG \RX SUDLVH WKH JURXS IRU" :KDW GLG
SRLQWV LQ WKH VFHQDULR ZKHUH WKLV ZRXOG EH PRVW WKH\ GR UHDOO\ ZHOO LQ WKLV SHUIRUPDQFH" RSHQ
DSSURSULDWH 7KHUHIRUH LQ WKH QH[W PHHWLQJV WKH\ KDG QR DQVZHU 
PDMRU SUREOHPV EUDLQVWRUPLQJ VXJJHVWLRQV RQ KRZ WR $SDUW IURP WKH UHVHDUFKHUV ERWK WHDFKHUV ± WKH
LQFRUSRUDWH URERWV VR WKH\ FRXOG XVH WKH WLPH WR OLWHUDWXUH WHDFKHU DQG WKH LQIRUPDWLFV WHDFKHU ±
LPSOHPHQWWKHLGHD6RIDURQO\WKHOLWHUDWXUHWHDFKHUKDV SDUWLFLSDWHGLQWKHSHUIRUPDQFHV
SDUWLFLSDWHGLQWKHPHHWLQJVZLWKWKHVWXGHQWV$WWKHQH[W
RQHVWKHUHVHDUFKHUVZHUHDOVRSUHVHQW 9 ),1',1*6
  )RXUWK6HVVLRQ 7KH GDWD FROOHFWHG ZDV TXDOLWDWLYH LQ QDWXUH WKHUHIRUH
7KHVHVVLRQWRRNSODFHZLWKWKHZKROHFODVVGXULQJWZR WKH\ ZHUH SURFHVVHG XVLQJ TXDOLWDWLYH PHWKRGV VXFK DV
OHVVRQV [PLQ LQWKHLUFODVVURRPZLWKRXWFRPSXWHUV RSHQ DQG D[LDO FRGLQJ >@ 7KH UHVXOWV RI WKH GDWD
7KH RULJLQDO JURXSV RI  VWXGHQWV ZHUH GLYLGHG LQWR WZR DQDO\VLVDQGWKHUHVHDUFKILQGLQJVSUHVHQWHGLQWKLVFKDSWHU
SDLUV DQG GXULQJ WKLV VHVVLRQ HDFK SDLU ZRUNHG ZLWK RQH DUH GLYLGHG LQWR WKH IROORZLQJ WKUHH JURXSV 7KH ILUVW
URERWLF NLW RQH WDEOHW DQG RQH ZRUNVKHHW 7KH DLP ZDV JURXSHQWLWOHG´6WXGHQWV¶.QRZOHGJHDQG6NLOOV´IRFXVHV
IRU WKHP WR OHDUQ KRZ WR WXUQ RQ WKH DSS SDLU WKH URERW RQVWXGHQWV¶VNLOOVDQGNQRZOHGJHLQWKHDUHDRI&7VNLOOV
ZLWK WKH WDEOHW DQG XVH EDVLF FRPPDQGV $IWHU WKLV ZKLFK ZDV RQH RI WKH UHVHDUFK REMHFWLYHV 7KH VHFRQG
  JURXS HQWLWOHG ´$GDSWDWLRQ RI WKH PHWKRGRORJ\´ UHSRUWV
ZZZOHJRFRPHQVNSURGXFWOHJRHGXFDWLRQVSLNHSULPHVHW
 RQ WKH QHFHVVDU\ VWHSV LQ WKH IXUWKHU GHYHORSPHQW RI WKH
&RPSDUHGWRRWKHUSRSXODUURERWVHJWKH2]RERWV KWWSVR]RERWFRP LWZDV PHWKRGRORJ\ 7KH ODVW JURXS HQWLWOHG ´+DUGZDUH DQG
VHYHUDOWLPHVORQJHU 6RIWZDUH´ ZDVFUHDWHGDVDE\SURGXFWRIWKHTXDOLWDWLYH
DQDO\VLVDQGUHODWHVWRWKHVSHFLILF/(*2NLWXVHG


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 367

 $ 6WXGHQWV¶.QRZOHGJHDQG6NLOOV
$V PHQWLRQHG DERYH WKH FODVV ZDV KDUGZRUNLQJ DQG
ZDV XVHG WR WKH SURMHFWEDVHG ZD\ RI WHDFKLQJ 6WXGHQWV
ZHUH FUHDWLYH DQG FROODERUDWLYH WKH\ FDPH WR FODVV
SUHSDUHG DQG ZHUH DEOH WR SURFHVV QHZ NQRZOHGJH
TXLFNO\ 1HYHUWKHOHVV WKH HIIHFWLYHQHVV RI WKHLU ZRUN
GHFUHDVHG DV WKH GXUDWLRQ RI WKH VHVVLRQ LQFUHDVHG 7KLV
VKRZHGXVWKHSRWHQWLDOULVNLQFDUU\LQJRXWODUJHUSURMHFWV
DQG WKH QHHG WR FRQVLGHU WKH GXUDWLRQ RI VWXGHQWV¶ ZRUN
GHSHQGLQJRQWKHLUDJHDQGH[SHULHQFH
6WXGHQWV ZHUH QRW FRPSOHWHO\ REMHFWLYH LQ WKHLU SHHU
DVVHVVPHQW EXW ZHUH PRUH WKDQ PRGHUDWHO\ OHQLHQW ZLWK
HDFKRWKHU+RZHYHUWKLVZDVH[SHFWHG7KHZRUVWUDWLQJ
JLYHQLQTXHVWLRQRQWKHLQFRUSRUDWLRQRIWKHURERWLQWR
WKH SUHVHQWDWLRQ ZDV  SRLQWV RQ D VFDOH RI ±  ,W ZDV
DZDUGHG E\ RQO\ RQH VWXGHQW DQG LQ RQO\ RQH FDVH LH
RQO\LQWKHHYDOXDWLRQRIRQHJURXS VHH)LJ$ )RUWKH
WK TXHVWLRQ UHJDUGLQJ WKH XVH RI WKH /(*2 63,.(
URERW¶V SRWHQWLDO WKH VWXGHQWV ZHUH D ELW PRUH ULJRURXV
VHH )LJ  %  6HYHUDO WLPHV WKH\ UDWHG WKHLU FODVVPDWHV
ZLWKSRLQWVDQGRQFHHYHQZLWKSRLQWV
6HOIDVVHVVPHQWZDVXVHGE\VWXGHQWVRXWRIDWRWDORI
+RZHYHUQRWDOOVWXGHQWVLQGLFDWHGWKDWWKH\HYDOXDWHG
WKHLU WHDP VR DGGLWLRQDO DQDO\VLV DQG HYDOXDWLRQ ZHUH
QHHGHG2IWKHVHHLJKWVWXGHQWVRQO\RQHVWXGHQWGLGQRW
JLYHKLPVHOIDIXOOVFRUHLQTXHVWLRQRWKHUZLVHDOOVHOI
DVVHVVPHQWV JDYH WKH PD[LPXP VFRUH )URP WKLV ZH
FRQFOXGH WKDW WKH DELOLW\ WR FULWLFDOO\ HYDOXDWH ZDV VWLOO
SRRUO\GHYHORSHGLQWKHVHVWXGHQWV
)XUWKHU DOWKRXJK DOO JURXSV ZHUH FRPSRVHG RI WKH
VDPHQXPEHURIVWXGHQWVWKHQXPEHURIUDWLQJVJLYHQWR
HDFKJURXSYDULHGDQGUDQJHGIURPWR LWVKRXOGKDYH
)LJXUH 6RPHRIWKHURERWPRGHOVDQGSURSVXVHGLQWKH EHHQ  RU  WKH ODWWHU LQ FDVH WKH\ DOO UDWHG WKHLU RZQ
SHUIRUPDQFHDWWKHWKVHVVLRQ
JURXS DV ZHOO  7KLV VKRZV WKH LQDFFXUDF\ DQG
XQUHOLDELOLW\ RI WKH VWXGHQWV 7KLV FRXOG DOVR EH EHFDXVH
 WKH VWXGHQWV GLG QRW FRQWLQXRXVO\ UDWH DIWHU HDFK
SHUIRUPDQFHDVWKH\ ZHUHDVNHGWRGR6HYHUDOHYDOXDWHG
WZRSHUIRUPDQFHVDWDWLPHDVLIWKH\SUHIHUUHGWRZDWFK
HYHU\WKLQJ ILUVW DQG WKHQ HYDOXDWH 7KH TXHVWLRQ UHPDLQV
ZKHWKHU WKLV DSSURDFK ZDV EHQHILFLDOEHFDXVH WKH\ FRXOG
FRPSDUHWKHWZRSHUIRUPDQFHVDQGZHUHDEOHWRHYDOXDWH
PRUH REMHFWLYHO\ RU ZKHWKHU RQ WKH FRQWUDU\ LW
GLPLQLVKHG WKHLU REMHFWLYLW\ EHFDXVH WKH\ IRUJRW WKH
SUHYLRXV SHUIRUPDQFH DQG GLG QRW UHPHPEHU LWV TXDOLWLHV
DQGIDXOWV
7KH ODVW VHFWLRQ LQ WKH HYDOXDWLRQ IRUP ZDV RSHQ
6WXGHQWVZHUHDVNHGWRDQVZHUWKHTXHVWLRQV´:KDWZRXOG
\RXSUDLVHWKHJURXSIRU":KDWGLGWKH\GRUHDOO\ZHOOLQ
WKLVSHUIRUPDQFH"´7KHSXUSRVHRIWKHVHTXHVWLRQVZDVWR
GHWHUPLQH ZKDW WKH VWXGHQWV ZRXOG IRFXV RQ LQ WKH SHHU
HYDOXDWLRQ ± ZKHWKHU LW ZDV URERWLFV SURJUDPPLQJ
DFWLQJFUHDWLYLW\LQFUHDWLQJWKHVWRU\OLQHDQGVFHQHU\HWF
2IWKHWRWDOUHVSRQVHV VRPHVWXGHQWVJDYHPRUHWKDQ
RQH   ZHUH UHODWHG WR URERWLFV  IRFXVHG RQ DFWLQJ
SHUIRUPDQFH DQG  ZHUH JHQHUDO SUDLVH 2I WKH RYHUDOO
URERWLFV HYDOXDWLRQ RQO\ RQH UHVSRQVH ZDV DERXW WKH
SURJUDPDQGWZRRWKHUVZHUHDERXWWKHURERWFRQVWUXFWLRQ
7KHRWKHUVZHUHGLUHFWHGPRUHWRZDUGWKHLGHDRIKRZWKH
URERWZDVXVHG:KHQHYDOXDWLQJVWXGHQWVIRFXVHGRQWKH
SURJUDPPLQJ WR D VPDOO GHJUHH FRPSDUHG WR RWKHU
 HOHPHQWVWKDWZHUHYLVXDO

%DVHG RQ WKH DQDO\VLV RI WKH SURJUDPV IRU WKH URERWV
)LJXUH $ 5HVXOWVIRUTXHVWLRQ(YDOXDWLRQRIFODVVPDWHVRQ
WKH\ FUHDWHG LW FDQ EH VDLG WKDW WKH VWXGHQWV XVHG
WKHLQFRUSRUDWLRQRIWKHURERWLQWRWKHIDEOH% 5HVXOWVIRUTXHVWLRQ
(YDOXDWLRQRIFODVVPDWHVRQWKHXVHRIWKHURERW¶VSRWHQWLDO DSSURSULDWH SURJUDPPLQJ FRQVWUXFWV LQ WKHLU SURJUDPV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 368

XVLQJ WKH VRIWZDUH FRPPDQGV WR FRQWURO /(*2 63,.( XQH[SHFWHGO\JRRG±LWGLGQRWHYHQORVHRILWVWRWDO
3ULPH JLYHQ WKH SUREOHP WKH\ ZHUH VROYLQJ 7KHLU FKDUJHDIWHUWHVWLQJ
VROXWLRQV UHTXLUHG VLPSOH DOJRULWKPLF VNLOOV ZKLFK ZHUH
LGHQWLILHGLQPRVWRIWKHP6WXGHQWVZHUHDEOHWRWUDQVIHU 9, ',6&86,21$1')8785(:25.
WKH NQRZOHGJH DQG VNLOOV RI FUHDWLQJ D SURJUDP LQ WKH $QDO\VLV RI WKH GDWD DOVR UDLVHG VHYHUDO LVVXHV )RU
6FUDWFKHQYLURQPHQWWRDQHZSURJUDPPLQJHQYLURQPHQW H[DPSOHKRZWRIXUWKHUXVHWKHPHWKRGRORJ\±ZKHWKHULW
DQGVROYHQHZSUREOHPVLQLW7KH\ZHUHIXUWKHULGHQWLILHG ZRXOG EH DSSURSULDWH WR FUHDWH D FRPSOHWH JXLGH IRU
DV KDYLQJ WKH VNLOO WR LQGHSHQGHQWO\ DQG UHODWLYHO\ WHDFKHUV RQ KRZ WR ZRUN ZLWK LW RU MXVW WR DGG PRUH
VSRQWDQHRXVO\ VROYH D GHFRPSRVLWLRQ RI D FRPSOH[ LQVWUXFWLRQV WR WKH PHWKRGRORJ\ LWVHOI 7KLV UHPDLQV D
SUREOHPWKDWLVWRGHFRPSRVHDSURSRVHGVROXWLRQLQWRD TXHVWLRQRIIXUWKHUWHVWLQJRIWKHPHWKRGRORJ\RQO\E\WKH
VHTXHQFH RI VPDOOHU SUREOHPV WKDW WKH\ FRXOG VROYH ,Q WHDFKHUVWKHPVHOYHVZLWKRXWWKHKHOSRIUHVHDUFKHUV
PRVW FDVHV WKH\ ZHUH DEOH WR GR WKLV RQ WKHLU RZQ
VRPHWLPHVZLWKWKHKHOSRIUHVHDUFKHUVZKRZHUHDFWLQJDV $QRWKHU RSHQ TXHVWLRQ LV WKH HYDOXDWLRQ RI WKH SURMHFW
WHDFKHUV 7KH\ IXUWKHU HPSOR\HG VNLOOV VXFK DV FUHDWLYLW\ UHVXOWV E\ WKH VWXGHQWV ,W QHHGV WR EH EHWWHU H[SORUHG
DQGYHUEDOL]DWLRQRILGHDV7KXVLWFDQEHFRQFOXGHGWKDW ZKHWKHU LW LV PRUH DSSURSULDWH WR DSSO\ D PRUH ULJRURXV
VWXGHQWVRIDJLYHQDJHZLWKPDVWHUHGEDVLFNQRZOHGJHRI DQG SHUKDSV PRUH LQVWUXFWLRQRULHQWHG SHHU HYDOXDWLRQ RU
SURJUDPPLQJ LQ WKH 6FUDWFK HQYLURQPHQW VKRZHG WROHDYHLWLQWKHIRUPRIDIRUPDWLYHHYDOXDWLRQXVLQJHJ
VXIILFLHQWPLQLPXPVNLOOVWRKDQGOHVXFKDSURMHFW &7 IRUPDWLYHLWHUDWLYH WRROV >@ 7KH IRUPHU PD\ KHOS
VWXGHQWV LQ DVVHVVPHQW DFWLYLWLHV ZLWK ZKLFK WKH\ GR QRW
% $GDSWDWLRQRIWKH0HWKRGRORJ\ KDYH PXFK H[SHULHQFH WKH ODWWHU PD\ JLYH WKH WHDFKHU D
EHWWHULQVLJKWLQWRWKHFKDUDFWHUDQGQDWXUHRIWKHVWXGHQWV
$QDO\VLV RI WKH GDWD FROOHFWHG KLJKOLJKWHG WZR PLQRU
SUREOHPV ZLWK WKH PHWKRGRORJ\ )LUVW WKH PHWKRGRORJ\ 2EYLRXVO\ PRUH VWXGLHV ZLOOQHHG WR EHFDUULHGRXW WR
GRHV QRW \HW FRQVLGHU VKRUWGXUDWLRQ SURMHFWV )RU JHQHUDOL]H WKH FODLPV UHJDUGLQJ VNLOO GHYHORSPHQW ,Q WKH
H[DPSOHIRURXUDQDO\]HGSURMHFWZHKDGRQO\VHVVLRQV FDVHRID PRUHFRPSOH[SURMHFWLIVWXGHQWVKDYHZRUNHG
LQ WRWDO 7ZR RI WKHP ZHUH IRFXVHG RQ WKH DFWXDO ZLWKDURERWLFNLWIRUDORQJHUSHULRGRIWLPHEHIRUHLWPD\
GHYHORSPHQW RI &7 $V WKH PHWKRGRORJ\ ZDV REYLRXVO\ EH SRVVLEOH WR IRFXV RQ D GHWDLOHG LQYHVWLJDWLRQ RI PRUH
GHVLJQHG EDVHG RQ H[SHULHQFH ZLWK ORQJHUWHUP SURMHFWV FRPSOH[DQGFRJQLWLYHO\GHPDQGLQJ&7VNLOOV
ZRUNLQJ ZLWK LW RQ VXFK D VKRUW SURMHFW ZDV )RU IXWXUH WHVWLQJ RI WKH PHWKRGRORJ\ WKH
XQFRPIRUWDEOH :H ZRXOG WKHUHIRUH UHFRPPHQG UHFRPPHQGDWLRQ LV WR XVH VLPSOHU URERWLF NLWV WKDW DUH
PRGLI\LQJWKHVHOHFWLRQRIRSWLRQVWRDOVRFRQVLGHUVKRUWHU PRUHSRUWDEOHDQGKDYHVWDEOHVRIWZDUH,WLVLPSRUWDQWWR
SURMHFWV UHDOL]HWKDWVRIWZDUHXSGDWHVWKDWDUHEH\RQGWKHFRQWURORI
$QRWKHU SUREOHP ZDV UHODWHG WR WKH OLVW RI &7 VNLOOV WKH UHVHDUFKHUV RU WHDFKHUV FDQ VLJQLILFDQWO\ GLVUXSW WKH
RIIHUHGE\WKHPHWKRGRORJ\7KH\ZHUHUDWKHUJHQHULFDQG VPRRWKIORZRIWKHOHDUQLQJSURFHVV
GLGQRWDOORZWRPDNHDJRRGPDSSLQJEHWZHHQVNLOOVDQG ,Q WKH QH[W VWHSV RI WKH UHVHDUFK H[HPSODU\ SURMHFWV
DFWLYLWLHV IURP WKH SURMHFW VR LW KDSSHQHG WKDW DOPRVW DOO ZLOO EH GHYHORSHG LQ FROODERUDWLRQ ZLWK LQVHUYLFH
WKH VNLOOV IURP WKH VHVVLRQ IHOO LQWR RQH FDWHJRU\ :H WHDFKHUV IRFXVLQJ RQ RWKHU DUWV DUHDV VXFK DV PXVLF RU
UHFRPPHQG FRQVLGHULQJ XVLQJ D PRUH UHSUHVHQWDWLYH YLVXDO DUWV DQG DOVRRQRWKHUOHYHOVRI HGXFDWLRQ VXFK DV
GHVFULSWLRQ RI VWXGHQW DFWLYLW\ LQ WKH PHWKRGRORJ\ IRU VHFRQGDU\DQGKLJKHUHGXFDWLRQ
H[DPSOHWKURXJKDGLDJUDPVKRZLQJWKHUHFXUULQJVWDJHV
RI VWXGHQW ZRUN DV PHQWLRQHG E\ &KHYDOLHU >@ RU 9,, &21&/86,216
OLQNLQJWRWD[RQRPLHVRIOHDUQLQJREMHFWLYHV>@$QRWKHU 7KLV SDSHU SUHVHQWV WKH UHVXOWV RI UHVHDUFK RQ WKH
SRVVLELOLW\ LV WR EH LQVSLUHG E\ WKH VWXG\ >@ ZKLFK GHYHORSPHQWRI&7VNLOOVRIVWXGHQWVWKURXJKHGXFDWLRQDO
SURSRVHV WKH 6 DSSURDFK VHOIFKHFN LQ SDLUV  VHOI URERWLFV LQ FRPELQDWLRQ ZLWK DUWV )RU WKHVH SXUSRVHV D
GHEXJ LQ SDLUV  DQG VFDIIROGLQJ  WR KHOS LPSURYH URERWLFVDUWV SURMHFW ZDV GHVLJQHG DQG LPSOHPHQWHG
VWXGHQWV¶&7 DFFRUGLQJ WR WKH FXUUHQWO\ GHYHORSHG PHWKRGRORJ\ DQG
& +DUGZDUHDQG6RIWZDUH WKH GDWD FROOHFWHG LQ LW ZHUH DQDO\]HG 6XEVHTXHQWO\ WKH
&7VNLOOVXVHGE\WKHVWXGHQWVLQVROYLQJWKHSURMHFWZHUH
4XDOLWDWLYH UHVHDUFK DOORZV IRU D IRFXV RQ LQWHUHVWLQJ LGHQWLILHG
RXWFRPHV HYHQ LI WKH\ KDYH QRW EHHQ VHW DV D UHVHDUFK
7KHSURMHFWZDVGHVLJQHGLQFROODERUDWLRQEHWZHHQWKH
REMHFWLYH >@ 2QH RI WKH VLJQLILFDQW ILQGLQJV LQ WKLV
UHVHDUFKHUV DQG WKH LQIRUPDWLFV DQG OLWHUDWXUH WHDFKHUV RI
UHVHDUFK FRQFHUQHG WKH XVH RI WKH /(*2 63,.( 3ULPH
WKJUDGH VWXGHQWV DW D SULYDWH HOHPHQWDU\ VFKRRO LQ WKH
URERWLFNLW,QWKHSDVWH[SHULHQFHVZLWK/(*2NLWVKDYH
FDSLWDO FLW\ 7KH WKHPH RI WKH SURMHFW ZDV D GUDPDWLF
EHHQSRVLWLYH,QWKLVUHVHDUFKGHVSLWHJRRGSUHSDUDWLRQD
SHUIRUPDQFH RI D SDUWLFXODU *UHHN P\WK XVLQJ /(*2
QXPEHU RI SUREOHPV ZHUH QRWHG URERWV UDQGRPO\
63,.( 3ULPH URERWV WR PDNH WKH SHUIRUPDQFH PRUH
GLVFRQQHFWLQJ IURP WKH WDEOHW VRIWZDUH VWRSSLQJ HYHQ
DWWUDFWLYH ,Q SURJUDPPLQJ WKH URERWV¶ EHKDYLRU VWXGHQWV
IUHH]LQJRU WKHSURJUDP QRW UHVSRQGLQJ DIWHU VWDUWLQJRU
FRXOGXVHFRPPDQGVWRFRQWUROPRWRUVVRXQGVDQGOLJKWV
WKH URERW QRW UHVSRQGLQJ WR WKH VWDUW FRPPDQG
7KH SURMHFW ODVWHG  VHVVLRQV DQG WRRN SODFH LQ ERWK
HQRUPRXVO\ ORQJ ZDLWLQJ IRU WKH VRIWZDUH WR UHVWDUW HWF
LQIRUPDWLFV DQG OLWHUDWXUH FODVVHV 9DOLGDWLRQ ZLWK WKH
DOO LQ FRPSOHWHO\ UDQGRP VLWXDWLRQV LQGHSHQGHQW RI WKH
UHVHDUFKHUV WRRN SODFH GXULQJ WKH ODVW WZR VHVVLRQV HDFK
SDUWLFXODUWDEOHWRUWKHSDUWLFXODUURERW2QHRIWKHUHDVRQV
ODVWLQJPLQ7KHODVWVHVVLRQZKHUHVWXGHQWVSHUIRUPHG
WKDWPD\KDYHWULJJHUHGWKHVHSUREOHPDWLFVLWXDWLRQVFRXOG
DQG SHHUHYDOXDWHG D GUDPDWLF SHUIRUPDQFH WRRN SODFH
KDYHEHHQDQHZYHUVLRQRIWKHDSSWKDW/(*2(GXFDWLRQ
WZLFHZLWKRQO\KDOIRIWKHFODVVSUHVHQWHDFKWLPH
KDG ODXQFKHG DERXW D PRQWK EHIRUH WKH UHVHDUFK ZDV
LQLWLDWHG 6LPLODU SUREOHPV ZHUH UHSRUWHG E\ RWKHU %DVHG RQ WKH DQDO\VLV RI WKH GDWD FROOHFWHG VHYHUDO
FROOHDJXHV 2Q WKH RWKHU KDQG WKH EDWWHU\ OLIH ZDV LQWHUHVWLQJILQGLQJVZHUHLGHQWLILHGDQGJURXSHGLQWRWKUHH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 369

 DUHDV D  6WXGHQWV¶ .QRZOHGJH DQG 6NLOOV E  $GDSWDWLRQ >@ )(57,/( *URXS $Q LQLWLDO YHUVLRQ RI WKH ³)(57,/(´ GHVLJQ
RIWKH0HWKRGRORJ\DQGF +DUGZDUHDQG6RIWZDUH7KH PHWKRGRORJ\ 5HSRUW RI WKH SURMHFW )(57,/( (/
.$+('   KWWSVIHUWLOH
IROORZLQJ FDQ EH FRQVLGHUHG DV WKH PRVW NH\ ILQGLQJV SURMHFWHXIHUWLOHBUBUHSRUWBGHVLJQBPHWKRGRORJ\
.QRZOHGJH IURP XVLQJ WKH 6FUDWFK HQYLURQPHQW LV >@ 0HOXãRYi - âXQGHUOtN - ýHUHWNRYi 6 ,QWHUGLVFLSOLQDU\
WUDQVIHUDEOH WR WKH SURJUDPPLQJ HQYLURQPHQW IRU /(*2 WHDFKLQJ RI PDWKHPDWLFV DQG VFLHQFH LQ 6ORYDNLD²SLORW VWXG\
63,.( 3ULPH 6WXGHQWV ZLWK EDVLF NQRZOHGJH RI $FWD0DWKHPDWLFD±  
SURJUDPPLQJ LQ 6FUDWFK DOVR GHPRQVWUDWHG VXIILFLHQW &7 >@ .ROOiURYi ' +HJDURYi ( 1DJ\RYi $ (GXFDWLRQDO FRQVWUXFW
VNLOOV WR WDFNOH D URERWLFV SURMHFW HYHQ LI WKH\ KDG QRW RIQDWXUDODQGVRFLDOUHDOLDLQQGJUDGHRIHOHPHQWDU\VFKRROZLWK
ZRUNHG ZLWK URERWV EHIRUH +RZHYHU LW LV GLIILFXOW IRU DQ HPSKDVLV RQ HQYLURQPHQWDO HGXFDWLRQ $ FRPSDULVRQ RI WKH
VWXGHQWVDWWKLVDJHWRZRUNHIIHFWLYHO\LIWKHLUFRQWLQXRXV 6ORYDN DQG &]HFK PRGHO $G $OWD -RXUQDO RI ,QWHUGLVFLSOLQDU\
5HVHDUFK  
LQGHSHQGHQWZRUNWLPHLVH[WHQGHGEH\RQGZKDWWKH\DUH
QRUPDOO\XVHGWRHYHQLIWKHWRSLFLVLQWHUHVWLQJWRWKHP >@ :LQJ -0 &RPSXWDWLRQDO WKLQNLQJ DQG WKLQNLQJ DERXW
FRPSXWLQJ 3KLORVRSKLFDO 7UDQVDFWLRQV RI WKH 5R\DO 6RFLHW\ $
7KHSURVDQGFRQVRIDQLQVWUXFWLRQDODSSURDFKQHHGWREH 0DWKHPDWLFDO 3K\VLFDO DQG (QJLQHHULQJ 6FLHQFHV   
ZHLJKHG ZKHQ LQFOXGLQJ SHHU DVVHVVPHQW LQ VXFK DQ ±  
DFWLYLW\ 7KH PHWKRGRORJ\ LWVHOI ZRXOG QHHG WR RIIHU D >@ 6HOE\ & :RROODUG - &RPSXWDWLRQDO WKLQNLQJ WKH GHYHORSLQJ
PRUH GHWDLOHG UDQJH RI VNLOOV IRU &7 DQG WR WDNH LQWR GHILQLWLRQ$&0  
DFFRXQWSURMHFWVWKDWODVWRQO\RQHRUWZROHVVRQV >@ $WPDW]LGRX 6 'HPHWULDGLV 6 $GYDQFLQJ VWXGHQWV¶
3URMHFWSUHSDUDWLRQLVWLPHFRQVXPLQJDQGWKHTXHVWLRQ FRPSXWDWLRQDO WKLQNLQJ VNLOOV WKURXJK HGXFDWLRQDO URERWLFV $
VWXG\ RQ DJH DQG JHQGHU UHOHYDQW GLIIHUHQFHV 5RERWLFV DQG
UHPDLQVWRZKDWH[WHQWWKLVPD\GLVFRXUDJHWHDFKHUVLQWKH $XWRQRPRXV6\VWHPV±  
IXWXUH1HYHUWKHOHVVZHKRSHWKDWDIWHUIXUWKHUWHVWLQJDQG
>@ 6KXWH 9 - 6XQ & $VEHOO&ODUNH - 'HP\VWLI\LQJ FRPSX
DSSURSULDWH PRGLILFDWLRQ RI WKH PHWKRGRORJ\ LW FDQ WDWLRQDOWKLQNLQJ(GXFDWLRQDOUHVHDUFKUHYLHZ±  
EHFRPHDJRRGWRROIRUWHDFKHUVWRKHOSWKHPGHVLJQ&7 >@ &KHYDOLHU 0 *LDQJ & 3LDWWL $ 0RQGDGD ) )RVWHULQJ
VNLOOV GHYHORSPHQW DFWLYLWLHV WKDW ZLOO EH DWWUDFWLYH WR FRPSXWDWLRQDOWKLQNLQJWKURXJKHGXFDWLRQDOURERWLFV$PRGHOIRU
VWXGHQWVIURPERWKWKHLQIRUPDWLFVDQGDUWVSHUVSHFWLYHV FUHDWLYH FRPSXWDWLRQDO SUREOHP VROYLQJ ,QWHUQDWLRQDO -RXUQDO RI
67(0(GXFDWLRQ  ±  
$&.12:/('*0(17 >@ 4X -5 )RN 3. &XOWLYDWLQJ VWXGHQWV¶ FRPSXWDWLRQDO WKLQNLQJ
WKURXJK VWXGHQW±URERW LQWHUDFWLRQV LQ URERWLFV HGXFDWLRQ
7KLV ZRUN ZDV VXSSRUWHG E\ WKH 6ORYDN 5HVHDUFK DQG ,QWHUQDWLRQDO-RXUQDORI7HFKQRORJ\DQG'HVLJQ(GXFDWLRQSS±
'HYHORSPHQW$JHQF\DQGE\WKH6FLHQWLILF*UDQW$JHQF\   
XQGHU WKH FRQWUDFWV $399 )(57,/(  >@ 'HQLV % +XEHUW 6 &ROODERUDWLYH OHDUQLQJ LQ DQ HGXFDWLRQDO
(/.$+('DQG9(*$ URERWLFV HQYLURQPHQW &RPSXWHUV LQ KXPDQ EHKDYLRU  ± 
±  
5()(5(1&(6 >@ %ULWLVK (GXFDWLRQDO 5HVHDUFK $VVRFLDWLRQ (WKLFDO *XLGHOLQHV IRU
(GXFDWLRQDO5HVHDUFKIRXUWKHGLWLRQ/RQGRQ  
>@ 6XUYH\ RQ RQOLQH DQG GLVWDQFH OHDUQLQJ ± 5HVXOWV >@ &UHVZHOO - (GXFDWLRQDO 5HVHDUFK 3ODQQLQJ &RQGXFWLQJ DQG
KWWSVZZZVFKRROHGXFDWLRQJDWHZD\HXHQSXEYLHZSRLQWVVXUYH (YDOXDWLQJ 4XDQWLWDWLYH DQG 4XDOLWDWLYH 5HVHDUFK UG HGQ 1HZ
\VVXUYH\RQRQOLQHWHDFKLQJKWP -HUVH\3HDUVRQ(GXFDWLRQ  
>@ 7KH *OREDO 5HSRUW RQ WKH 6WDWXV RI 7HDFKHUV 
>@ 0LNRYi . .UFKR - &RJQLWLYH 7D[RQRP\ DQG 7DVN *UDGDWLRQ
KWWSVHLLHLR6WDWXV2I7HDFKHUV(
LQ (GXFDWLRQDO 5RERWLFV±3UHOLPLQDU\ 5HVXOWV ,EHULDQ 5RERWLFV
>@ )(57,/( *URXS 5HSRUW RQ HGXFDWRUV¶ SURILOLQJ 5HSRUW RI WKH FRQIHUHQFH&KDP6SULQJHU,QWHUQDWLRQDO3XEOLVKLQJ  
SURMHFW)(57,/((/.$+('   >@ 7KUHHNXQSUDSD $ <DVUL 3 3DWWHUQV RI FRPSXWDWLRQDO WKLQNLQJ
KWWSVGULYHJRRJOHFRPILOHG5TG70Q%.'SQFH,PS:O8Y GHYHORSPHQW ZKLOH VROYLQJ XQSOXJJHG FRGLQJ DFWLYLWLHV FRXSOHG
%L<=Q/+YLHZ"XVS VKDUHBOLQN
ZLWK WKH 6 DSSURDFK IRU VHOIGLUHFWHG OHDUQLQJ (8-(5   
>@ 3HULJQDW ( DQG .DW]%XRQLQFRQWUR - 67($0 LQ SUDFWLFH DQG SS±  
UHVHDUFK $Q LQWHJUDWLYH OLWHUDWXUH UHYLHZ 7KLQNLQJ VNLOOV DQG >@ 5RPғDQ*RQ]iOH] 0 0RUHQR/HғRQ - 5REOHV * &RPELQLQJ
FUHDWLYLW\±  
DVVHVVPHQWWRROVIRUDFRPSUHKHQVLYHHYDOXDWLRQRIFRPSXWDWLRQDO
>@ $JXLOHUD ' DQG 2UWL]5HYLOOD - 67(0 YV 67($0 HGXFDWLRQ WKLQNLQJLQWHUYHQWLRQV&RPSXWDWLRQDOWKLQNLQJHGXFDWLRQSS±
DQG VWXGHQW FUHDWLYLW\ $ V\VWHPDWLF OLWHUDWXUH UHYLHZ (GXFDWLRQ   
6FLHQFHV    

>@ (UDVPXVSURMHFWKWWSVIHUWLOHSURMHFWHX


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 370

Online Educational Game - Tool for Education in
Raw Material
M. Molokáč*, G. Alexandrová**, Zdenka Babicová**, D. Tometzová*, Lucia Molitoris*
* Technical University of Košice/Department of Earth Resources, Košice, Slovakia
** Technical University of Košice/Rectorate, Košice, Slovakia
*** Technical University of Košice/Department of Geosciences, Košice, Slovakia

mario.molokac@tuke.sk, gabriela.alexandrova@tuke.sk, zdenka.babicova@tuke.sk, dana.tometzova@tuke.sk,

lucia.molitoris@tuke.sk

Abstract—The topic of the presented work is a presentation realised through systems such as Zoom or Google Meet
of the current status of online games in the Briefcase and individual consultations by e-mail, chat and social
project. The aim of the presented work is to analyze of the networks dominated. Modern online technologies
current state, show to created proposals that can increase contribute to the sustainability of the educational process
the attractiveness of education in raw materials. The during an emergency and will become an integral part of
importance of the work lies in the creation of online university education even after the end of the pandemic
educational games focused on target groups of pupils, which situation [6].
would increase the motivation of pupils and leave them with The introduction of new technologies in society has
pleasant experiences, thanks to which they will remember created a need for interactive contents that can make the
information for long time. The work is mainly focused on
most of the potential that technological advances offer.
pupils, which are divided into three target groups according
Educational games can be defined as interactive
to age. The topic, methodology and goals of online
applications whose main purpose is to provide not only
educational games are the same. Online games differ only in
entertainment but also training in education. There is no
their difficulty and the amount of information, which, on the
other hand, is adequate for the students' level of education.
single definition of educational games, though they are
Online games of Briefcase project represent actuality and
generally held to be games used for training, advertising,
necessity during teaching at home as well as their suitability simulation or education. Alternative definitions include
for use in teaching in schools as supplementary material. the application of game concepts, technologies and ideas
to non-entertainment applications [7].
Term of educational games and game-based learning
I. INTRODUCTION approaches has been building up gradually and in phases,
In essence, online learning refers to learning and other across different disciplines and in an ad hoc way. This has
supportive activities that are available through a computer been problematic in a number of ways and resulted in
[1, 2]. The need for online education emerged in the fragmented literature and inconsistent referencing patterns
period of threat (pandemic, war), or especially in places between different sub-disciplines and countries [8].
where traditional education was problematic. Online Modern forms of education through platforms with
education gradually gained importance but was still interactive and virtual content in geography have been
considered a supplementary form of education. A global processed e.g. by Gregorová & Žoncová (2019) [9].
turning point occurred in recent years when COVID-19 Studies in the field of pedagogy indicate that new
hindered traditional education, and the existing and readily technologies can positively affect the learning process
accessible online space became the alternative. It is [10,11,12,13]. New technologies may not only be used
evident that the transition to online education was during classical teaching but also in the terrain [14, 15], or
demanding, not only for students but also for educators for international students [16].
[3]. The pandemic has created a unique opportunity for
educational changes that have been proposed before
COVID-19 but were never fully realized. Changes that
education should make post COVID: curriculum that is
developmental, personalized, and evolving; pedagogy that
is student-centred, inquiry-based, authentic, and
purposeful; and delivery of instruction that capitalizes on
the strengths of both synchronous and asynchronous
learning [4]. Recently, education may look different and
more online like in the pre-COVID era. For some, an
immediate retreat to the traditions of the physical
classroom is required but for others, the forced shift to
online education is a moment of change and a time to
reimagine how education could be delivered [5]. Arose Figure 1. Short instructions that are expressed in steps before starting
trend in the communication strategies used to ensure the the online game
sustainability of the education system. When
communicating and teaching students, online teaching was

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 371

 II. BASED LEARNING (GBL) their extraction. Without minerals and their extraction, we
To foster student engagement in the learning process, cannot envision our lives, but mining processes also entail
educational games are employed. During the game, various consequences, such as their impact on the
students engage in learning, and the addition of fun makes environment and society. Through an exemplary and
the learning process more interesting. It has a positive approachable methodology, "The Briefcase" projects aim
impact on cognitive function development [17]. Games to demonstrate that mining, as a modern activity, can
and courses are combined because the traditional learning mitigate its effects on society and the environment.
process is boring, and game-based learning can enhance
students' motivation to learn. When students enter a state
of flow while playing, their concentration is higher than
usual [18]. Game-Based Learning (GBL) is not just about
using games for motivation in learning. Computer games
utilize a natural human trait - competitiveness, the drive to
excel and overcome oneself, and it's precisely through
these characteristics that games can be used to learn
genuinely challenging subjects or in new technological
fields, such as renewable energy sources [19].

GBL is increasingly used in the following areas [20],

[21]:

• Study material that is technical and boring;

• Truly challenging topics;
• Difficulty in engaging the audience;
• Complex issues with assessment and
certification; Figure 2. Example of raw material and its diverse information about it.
• The intricate process of understanding;
The primary target audience comprises pupils and
• Sophisticated "what if" analyses; students, with a central aim of cultivating their interest in
• Strategy development and communication; the fields of geology, metallurgy, and mining.
• Increasing interest in education and student Establishing direct engagement with students is pivotal for
motivation. fostering enthusiasm for minerals and mining. In the
context of THE BRIEFCASE and 3DBRIEFCASE
projects, various thematic briefcases have been created,
Game-based learning makes people feel like they are which, through their interactivity, contribute to the
playing computer games in the learning process. Two motivation for education in this field. The educational
important elements are involved in learning - interest and tools of the BRIEFCASE projects are available in various
fun. In reality, games can help students not only in formats, including a virtual briefcase through the online
effective education but also in developing fundamental game "the briefcase game," a 3D briefcase, and a book
skills and knowledge in specific IT areas. Students believe featuring minerals that can be explored through
that digital game-based learning helps them learn faster augmented reality (AR applications). Furthermore, all of
and develop a greater interest in educational topics. these educational instruments are presented during
workshops, which are complemented by additional
III. ABOUT THE BRIEFCASE PROJECT engaging activities. Through these workshops, students
When we delve deep into history, prehistoric humans are not only taught to identify minerals and understand
sought out stones for crafting various tools used for their everyday applications but are also encouraged to
cutting, drilling, chopping, or for the production of embrace responsibility in the consumption of natural
weapons for hunting. The first raw materials discovered resources. This includes altering their recycling habits,
and utilized by humans were hard rocks rich in silicon, emphasizing the significance of environmental
known as siliceous materials – flint, hornstone, radiolarite, conservation.
obsidian, and silicate minerals like jasper and opal. Even Comprehensive information is accessible on the
our present-day standard of living relies on extracted raw project's website, www.briefcase.eitrawmaterials.eu [22].
materials.
The topic of mining, mineral extraction, and the
significance of minerals in our daily lives are the focal IV. THE BRIEFCASE ONLINE GAME
points of the educational project "The Briefcase" and its
extensions, "3D Briefcase" and "RIS Briefcase." These One of the project's successful educational outcomes is
projects are co-funded by the European Union and an online game, which is publicly accessible on the
supported by EIT Raw Materials. The primary objective website www.thebriefcasegame.eu. Currently, it is
and underlying concept of the project are to impart available in 35 language variants. This knowledge-based
knowledge to people from a young age about the origins game offers two levels of difficulty: 1. for younger
of the mineral products we use in our daily lives, the children aged 6-9, containing seven minerals (yellow
minerals found in their surroundings, and the reasons for briefcase), and 2. for older students aged 10-14,
containing 18 minerals (blue briefcase).

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 372

 age. The European Union also has this experience and
therefore supported the Briefcase project, whose task was
to educate about mineral resources in an interesting way.
Online educational games were chosen as one way to do
it. Educational games were created to appeal to students.
Therefore, three target groups of pupils were created
according to age, representing the first and second grades
of elementary schools and secondary schools. The
educational games created in this way have proven to be
very suitable educational tools, because they educate in a
fun way and also offer information about mineral
resources that is appropriate for the age of the students.
Figure 3. Selection of target groups of pupils The use of online games has a positive effect on the
meaningful use of IT and improves not only knowledge
The game presents a virtual briefcase containing
about online games but also IT skills. Such a combination
various minerals and their corresponding items. The
objective of the game is to match individual minerals with is very important not only for the further education of
the items created from them. Each mineral is accompanied students, but also for their future career growth.
by four clues to assist the player in correctly associating Contemporary society places paramount importance on
the mineral with the appropriate item. Points are awarded high-quality education, particularly on equitable access to
for each correct solution. Notably, the fewer clues a player it, as the collective awareness prevails that quality
uses, the more points they earn. Upon successfully education serves as a crucial catalyst for an enhanced
matching each mineral with its corresponding item quality of life. The accessibility afforded by online
(mineral product), information about the mineral is education significantly streamlines the pursuit of
displayed, including its chemical formula, physical knowledge in its various forms. Young individuals who
properties, utilization of the raw material, the list of harness this reality are poised to gain an unequivocal
countries where the resource is mined, and other competitive edge within the labour market, irrespective of
interesting facts.
their status as domestic employees or expatriates [23].
For older high school students, a 3D Briefcase (green
briefcase) employing virtual reality is available, allowing ACKNOWLEDGMENT
the user to immerse themselves in the role of a miner in
various work positions within the mine's 3D space. A 360° This paper was created in connection with the projects
video was created for this type of experience. By using a RISBRIEFCASE Project, a follow-up project of The
computer or a mobile device along with 3D glasses, users Briefcase and 3DBriefcase.
can virtually visit the following mines: 1. Orovalle
Copper-Silver-Gold mine in Spain, 2. Aitik copper mine REFERENCES
in Sweden, and a magnesite mine of Magnesitas Navarras [1] S. Carliner, An overview of online learning, 2004, HRD Press,
in Spain (accessible at tiwp.tecnalia.com/3dbriefcase/). Amherst, ISBN: 0-87425-740-9
Furthermore, during the expedition, users are presented [2] P.R. Harris, "An Overview of Online Learning (2nd ed.)",
with quiz questions and information that provides insights European Business Review, Vol. 16 No. 4, 2004, pp. 430-430.
https://doi.org/10.1108/09555340410561723
into the raw materials extracted in the mine, mining
[3] N. Peimani, H. Kamalipour, Online Education in the Post COVID-
machinery, work processes, and more. 19 Era: Students’ Perception and Learning Experience. Education
Sciences. 2021; 11(10):633.
https://doi.org/10.3390/educsci11100633
[4] Y. Zhao, J. Watterston, The changes we need: Education post
COVID-19. J Educ Change 22, 3–12 (2021).
https://doi.org/10.1007/s10833-021-09417-3
[5] B.B. Lockee, Online education in the post-COVID era. Nat
Electron 4, 5–6 (2021). https://doi.org/10.1038/s41928-020-
00534-0
[6] A. Hurajova, D. Kollarova, L. Huraj, Trends in education during
the pandemic: modern online technologies as a tool for the
sustainability of university education in the field of media and
communication studies In: Heliyon, Volume 8, Issue 5, May 2022,
e09367, https://doi.org/10.1016/j.heliyon.2022.e09367
[7] N. Peña-Miguel; M. Sedano Hoyuelos, Educational Games for
Figure 4. Continuous evaluation of the online game with a point score Learning In: Universal Journal of Educational Research 2(3): 230-
238, 2014.org DOI: 10.13189/ujer.2014.020305
[8] S. de Freitas, “Are Games Effective Learning Tools? A Review of
V. CONCLUSION Educational Games.” Journal of Educational Technology &
Society, vol. 21, no. 2, 2018, pp. 74–84. JSTOR,
Recently, we find that elementary school students have http://www.jstor.org/stable/26388380. Accessed 12 Oct. 2023.
no knowledge about mineral resources and their [9] B. Gregorová, M. Žoncová, Geography interactive teaching
importance. They don't even know what daily necessities materials for primary schools. In:Book of Abstract. Innovations,
Trends and research in Science Education in Central Europe. 1.
are made of. For the development of society, it is vyd. Trnava : TYPI, HELD, 2019, pp. 23.
necessary to present this information to people at a young

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 373

[10] P. Anand, A.J. Herrington, S. Agostinho, Constructivist-based
learning using location-aware mobile technology: an exploratory
study. In J. Luca & E. Weippl (Eds.), ED-MEDIA: World
Conference on Educational Multimedia, Hypermedia &
Telecommunications, Chesapeake, USA: AACE. 2008 s. 2312-
2316
[11] M. Molokáč, G. Alexandrová, M. Kobylanska, B. Hlavňová P.
Hronček, D. Tometzová, Virtual Mine – educational model for
Wider Society In: ICETA 2017. - Košice : TU, 2017 p. 307-312. -
ISBN 978-1-5386-3294-9
[12] M. Molokáč, D. Tometzová, G. Alexandrová, M. Kobylanska, Z.
Babicová, Virtual Mine – successful story of education in raw
materials. In: ICETA 2019 : 17th IEEE International conference
on emerging elearning Technologies and applications :
Information and communication technologies in learning. Starý
Smokovec, Slovakia. November 21-22, 2019 : proceedings. -
Denver (USA) : Institute of Electrical and Electronics Engineers,
2019 p. 536-539 [online]. - ISBN 978-1-7281-4967-7
[13] M. Zeleňáková, H. Pavolová, T. Bakalár, Internet communication
in the process of education at universities.In: Procedia - Social
and Behavioral Sciences : WCES 2012: 4th World Conference on
Educational Sciences : 2-5February 2012, Barcelona. - Netherland
: Elsevier, 2012, Vol. 46, p. 2711- 2715. - ISSN 1877-0428 online:
http://www.sciencedirect.com/science/article/pii/S1877042812016
813...
[14] K. Weis, P. Hronček, D. Tometzová, B. Gregorová, M. Přibil, M.
Jesenský, V. Čech, Analysis of notice boards (panels) as general
information media in the outdoor mining tourism, in Acta
Montanistica Slovaca. – Košice (Slovensko) : Fakulta baníctva,
ekológie, riadenia a geotechnológií vol. 24, no. 3 , p. 269-283.,
2019, ISSN 1335-1788
[15] N. Bursztyn, A. Walker, B. Shelton, J. Pederson, Assessment of
student learning using augmented reality Grand Canyon field trips
for mobile smart devices, in Geosphere, 2017, 13 (2), pp. 260–
268. Online: https://doi.org/10.1130/GES01404.1
[16] V. Ferencz, D. Hrehová, A. Seňová, Higher education require
adaptation of students study abroad programs. SHS Web of
Conferences 74 (3-4) 2020,DOI:10.1051/shsconf/20207402003
[17] W. C. Lin, J. Y. Ho, C. H. Lai, and B. S. Jong, “Mobile
gamebased learning to inspire students learning motivation,” in
Proc. 2014 International Conference on Information Science,
Electronics and Electrical Engineering, ISEEE 2014, vol. 2, pp.
810–813, 2014.
[18] K. Squire, “Video games in education,” International Journal of
Intelligent Games and Simulation, vol. 2, 2003.
[19] R. Rybár, D. Kudelas, M. Beer, J Rybárová, Elimination of
Thermal Bridges in the Construction of a Flat Low-Pressure Solar
Collector by means of a Vacuum Thermal Insulation Bushing In:
Journal of Solar Energy Engineering. Vol. 137, no. 5, 2015, p. 1-4.
- ISSN 0199-6231
[20] F. Ke, “Alternative goal structures for computer game-based
learning.” International Journal of Computer-Supported
Collaborative Learning, vol. 3, pp. 429-445, 2008.
[21] H. Andreas and M. Ebner, “Successful implementation of
usercentered game based learning in higher education: an example
from civil engineering,” Computers and Education, vol. 49, no. 3,
pp. 873-890, November 2007.
[22] The Briefcase project, online: www.briefcase.eitrawmaterials.eu
[23] L. Bednárová, J. Chovancová, A. Pacana, R. Ulewicz,The
analysis of success factors in terms of adaptation of expatriates to
work in international organizations, Polish Journal of managment
studies, Czestochowa, vol.17, n. 1, pp. 59 – 66, 2018, ISSN 2081-
7452.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 374

 Automated deployment of the OpenStack platform
Marek Moravcik, Pavel Segec
Faculty of Management Science and Informatics, University of Zilina, Univerzitna 8215/1, 010 26 Zilina
e-mail: {marek.moravcik, pavel.segec}@fri.uniza.sk

Abstract—The aim of the paper is to analyze the necessary A. Collection of requirements

support services in the environment of the KIS department, which
will serve to simplify the management of computing power and
The OpenStack cloud platform provides many useful func-
its more efficient use. The design of a suitable architecture and tions, means for controlling and managing computing tech-
the deployment of selected services together with the creation of nology in data centers or, as in our case, in a private cloud.
documentation. By installing the open-source OpenStack cloud However, after installation, the administrator only gets access
platform with selected services according to the department’s to the basic functionality of the individual components, or
needs for use in an academic environment.
slightly extended functionality as long as the relationships be-
Index Terms—OpenStack, Automation, Juju
tween the individual components are created. For example, the
MySQL component that serves as a database would not have
any functionality for other components unless the relationships
I. I NTRODUCTION
between these components were declared and the MySQL
router component was not installed. If the administrator of
Cloud and cloud services are an inseparable part of today’s
such a platform required additional functionalities that were
time. We undoubtedly see this in situations where large
not part of the basic installation, a solution had to be found
companies move the majority of their infrastructure to the
and configured. Therefore, it was necessary as a first step to
cloud. For the client, everything is often easier, faster and
collect requirements from the supervisor at the beginning and
more financially advantageous. Examples are Amazon Web
add more over time based on progress and new knowledge.
Services, Google Cloud Platform, Microsoft Azure or IBM
Services. In recent years, not only computing power and B. Orchestration tool for environment management
data have been transferred to the cloud, but also the gaming Another requirement that needs to be implemented is an
industry (gaming-as-a-service), which includes GeForce Now, orchestration tool. The orchestration tool allows you to manage
Google Stadia, Playstation Now and others. resources quickly and efficiently. For example, for the needs of
In 2010, the project started with the original authors Na- the subject, it is necessary to create identical topologies for a
tional Aeronautics and Space Administration (NASA) and certain number of students to work on. Thanks to the script for
Rackspace Hosting, nowadays known as OpenStack. It is the orchestration tool, we can quickly create these topologies,
mostly deployed as ”infrastructure as a service” (IaaS) in eliminating the need for manual creation, which could be time-
both public and private clouds, where virtual servers and consuming. If computing resources need to be allocated to
other resources are made available to users. The software students, we will again use the orchestration tool to create
platform consists of interconnected components that manage student accounts and just select the number. These topologies
diverse hardware areas of processing, storage, and networking and needs are used on a regular basis at the beginning of the
resources from multiple vendors throughout the data center. semester, be it winter or summer, and once a script or template
Users manage it either through a web-based control panel or is created, it is a matter of minutes using an orchestration tool
command-line tools. OpenStack has a modular architecture to create such resources. It is also not a problem to make
with different names of its components such as: Compute a change in the topology, which is applied to the created or
(Nova), Networking (Neutron), Block storage (Cinder), Iden- already created topologies or user accounts.
tity (Keystone), Object storage (Swift), Dashboard (Horizon).
With each new version, new and new components were added, II. O PEN S TACK
compared to the initial version with two components Nova and OpenStack is a cloud operating system that manages large
Swift, in the current version called Yoga (January 2023), it is groups of computing, storage and network resources within
possible to install 38 components. There are several options for the data center, all of which are managed and provided
installing OpenStack: OpenStack Charms, OpenStack ”kolla- through APIs with common authentication mechanisms. There
ansible”, OpenStack-Ansible or TripleO. is also a web dashboard that gives administrators control as
We will focus on this free open-source cloud computing well as enablement provide resources to users through a web
platform in this work. The main goal will be the installation interface. In addition to the standard infrastructure-as-a-service
of this platform on the infrastructure of the KIS department functionality is provided by other components in addition to
with subsequent analysis of usable functionalities and their other services also orchestration, fault management and service
implementation proposal with deployment. management to ensure high availability user applications.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 375

 OpenStack is divided into services that allow connecting image to the hardware. It provides a modular tool for a one-
components in a dependency from the user’s needs. The time deployment of the operating system with the smallest
OpenStack Map provides a view of the OpenStack lifecycle, possible number of operational ones requirements.
for the user to see where these services fit in and how they 5) Kayobe: Kayobe deploys the OpenStack container con-
can work together [1]. trol plane to physical devices. Bifrost is used to discover and
provision cloud servers. Kolla is used for creating container
A. Types of OpenStack installation images for OpenStack services. Kolla Ansible is used to
Installing Openstack is different from installing a program deploy the OpenStack container control plane.
on a device, it is a much more complex matter and therefore 6) OpenStack-Ansible: OpenStack-Ansible provides Ansi-
tools and so-called frameworks that help maintain the lifecycle ble playbooks and roles for deployment and configuration
of deployed OpenStack. A framework is a structure that serves OpenStack environment. Thus, compared to the deployment of
as a basis for building software, and its advantage is that it Kolla-Ansible, we will not use containers, but only the Ansible
does not start completely from scratch. As of the current date tool and OpenStack components.
(September 2023), there are several frameworks [2]: 7) OpenStack-Charms: Openstack-Charms is a collection
• TripleO of Charmed Operators, also called simply ”charms” that are
• OpenStack-Helm used to deploy and manage OpenStack clouds using MAAS
• Kolla-Ansible and Juju. Each OpenStack charm is responsible for deployment
• Bifrost and lifecycle management of a single cloud service (for exam-
• Kayobe ple, the nova-compute charm for the Nova Compute service).
• OpenStack-Ansible The project also includes charms for selected software that
• OpenStack-Charms does not support OpenStack, such as Ceph, MySQL and
• OpenStack-Chef RabbitMQ.
8) OpenStack-Chef: Chef is an open source systems man-
1) TripleO: TripleO is the name for ”OpenStack on Open- agement and automation platform cloud infrastructure. De-
stack”. This is an official project OpenStack to enable the ploying OpenStack using Chef should include ”cookbooks”,
deployment and management of a production cloud on a which are currently not available.
physical hardware using existing OpenStack components. It
will begin at TripleO with the creation of an ”undercloud” III. C OMPONENTS USED AT KIS DEPARTMENT
that will contain the necessary OpenStack components for
deploying and managing the ”overcloud” or the working cloud. One of the main advantages of deploying the OpenStack
Overcloud is deployed solution and can represent the cloud for cloud platform, currently (January 2023) in the Yoga version,
any purpose. TripleO uses several existing basic OpenStack is modularity. Since the first version of Austin, when there
components such as: Nova, Ironic, Neutron, Heat, Glance and were two components: Nova and Swift, there are 38 of them
Ceilometer. in the Yoga versions [4]. For the needs of the department
2) OpenStack-Helm: OpenStack-Helm is an OpenStack de- Only those whose functionality will be used were selected for
ployment solution using Helm using Kubernetes containers. OpenStack. Between individual components create relation-
Helm is a tool used for definition, installation and upgrading ships that serve for mutual communication. These relationships
applications running on Kubernetes. In its most basic form they may either be required to ensure the correct functionality
Helm is a templating tool. Kubernetes is used for automation of the component or optional as needed by the administrator
container management operational tasks and includes built- [5], [10].
in deployment commands applications, introducing changes
A. Nova (Compute service)
to applications, and scaling applications, whether upward or
downward or down. It is recommended to deploy a Kubernetes Nova is a component that provides the creation of comput-
cluster for a production environment use a highly resilient ing instances called virtual servers. Nova supports the creation
distribution like Airship or KubeADM. of virtual machines, bare metal servers (using the Ironic
3) Kolla-Ansible: Kolla-Ansible deploys the OpenStack component) and has limited support for system containers.
container control plane using Kolla containers, organized Nova runs as a set of daemons on top of existing Linux servers
through Ansible. The project focuses on simplicity and re- to provide this service. Required relationships to ensure basic
liability while providing a flexible and intuitive configuration functionality: Keystone, Glance, Neutron and Placement.
model. Kolla provides containers and tools for running Open-
Stack clouds that are scalable, fast, reliable and upgradable B. Placement (Part of compute service)
using community best practices. Ansible is a software tool that Placement is an OpenStack service that provides an HTTP
provides simple yet powerful automation for multi-platform API to track inventory and usage of cloud resources to help
computer support. other services effectively manage and allocate their resources.
4) Bifrost: Bifrost is an Ansible playbook that automates It does not require any relationships for security basic func-
with the Ironic component the task of deploying the base tionality.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 376

C. Cinder (Block storage) J. RabbitMQ (Message service)
Cinder is a block storage service for OpenStack. It vir- RabbitMQ is an implementation of the Advanced Message
tualizes block management storage devices and provides a Queuing Protocol (AMQP), an emerging standard for high-
self-service API to end users to request and consume these performance enterprise messaging. RabbitMQ server is a ro-
resources without having to know where they are storage bust and scalable AMQP broker implementation. It does not
actually deployed or on what type of device. This is done using require any relationships to ensure basic functionality. If we
either the reference implementation (LVM) or plugin drivers want other components to use services AMQP, we will add a
for another repository. Required relationships to provide basic relationship between the RabbitMQ server and the application
functionality: Keystone. that supports the interface RabbitMQ [8].
D. Neutron (Networking service)
K. Vault
Neutron is an SDN networking project focused on pro-
viding Networks as a Service (NaaS) in virtual computing Vault is a tool for securely managing secrets used in
environments. Neutron provides network connectivity as a modern computing techniques such as: passwords, certificates,
service between device interfaces such as virtual network card API keys). OpenStack uses Vault to process Transport Layer
interfaces (VNICs). Required relationships to provide basic Security (TLS) certificates, which enables central a managed
functionality: Keystone. solution for the encryption of API services within the cloud
[6].
E. Keystone (Authentication and authorization)
Keystone is an OpenStack service that provides API client IV. O PEN S TACK AUTOMATED INSTALLATION
authentication, service discovery and distributed multi-tenant It was chosen as the most suitable method for the OpenStack
authorization by implementation ”OpenStack’s Identity API”. installation itself OpenStack-Charms, Charmed Operators or
It supports LDAP, OAuth, OpenID Connect, SAML and SQL. also called ”Charms” are used to deploy and manage Open-
It does not require any relationships to ensure basic function- Stack clouds using Metal-as-a-Service (MAAS) and Juju. First
ality. you will need to install the MAAS server and use it to
F. Heat (Orchestration) deploy the Juju server. Subsequently, we will be able to build
OpenStack using the bundle configuration file by the Charms
Heat organizes infrastructure resources for a cloud appli-
method. For installation, we have available virtual servers that
cation based on templates in the form of text files that can
will serve to install MAAS, Juju and as a Controller Node.
be treated as code. Heat provides native OpenStack ReST
Additional physical servers will serve primarily for running
API and Query API compatible with AWS CloudFormation.
virtual instances in OpenStack using the Nova component.
Heat also provides an auto-scaling service that integrates with
Below in the table we can find the distribution of individual
OpenStack services Telemetry so you can include a scaling
components as they are installed on our cloud
group as a resource in your template. Required relationships
to ensure the basic function: Keystone.
TABLE I
G. Horizon (Web dashboard) I NSTALLATION OF COMPONENTS INTO INDIVIDUAL MACHINES

Horizon is Canonical’s implementation of the OpenStack Server Components

control panel, which is extensible and provides a web user controller Ceph-radosgw, Cinder, Designate, Glance, Heat, Keystone,
MySQL (x3), Neutronapi, Nova cloud controller, Open-
interface for OpenStack services including Nova, Swift, Key- stack dashboard, Ovn-central (x3), Placement, RabbitMQ,
stone, etc. Required relationships to ensure basic functionality: Vault
Keystone. compute-1 Nova, NTP, Ovn-chassis
compute-2 Nova, NTP, Ovn-chassis
H. Manila (Shared storage) compute-3 Nova, NTP, Ovn-chassis
compute-4 Nova, NTP, Ovn-chassis
Manila provides coordinated access to shared or distributed compute-5 Nova, NTP, Ovn-chassis, Ovn-Central, Ceph-Osd, Ceph-
file system. It does not require any relationships to provide Mon, Manila, Manila-Ganesha, Ceph-FS
basic functionality. compute-6 Nova, NTP, Ovn-chassis, Ovn-Central, Ceph-Osd, Ceph-
Mon, Manila, Manila-Ganesha, Ceph-FS
I. Glance (Image service) compute-7 Nova, NTP, Ovn-chassis, Ovn-Central, Ceph-Osd, Ceph-
Mon, Manila, Manila-Ganesha, Ceph-FS
The Glance image service includes image discovery, reg- compute-8 Nova, NTP, Ovn-chassis, Ovn-Central, Ceph-Osd, Ceph-
istration and retrieval for virtual instances/machines. Virtual Mon, Manila, Manila-Ganesha, Ceph-FS
compute-9 Nova, NTP, Ovn-chassis, Ovn-Central, Ceph-Osd, Ceph-
machine images made available through Glance can be stored Mon, Manila, Manila-Ganesha, Ceph-FS
in a variety of locations from simple file systems to systems for compute-10 Nova, NTP, Ovn-chassis, Ovn-Central, Ceph-Osd, Ceph-
object storage, such as the OpenStack Swift project. API con- Mon, Manila, Manila-Ganesha, Ceph-FS
compute-11 Nova, NTP, Ovn-chassis, Ovn-Central, Ceph-Osd, Ceph-
tained in this the component allows querying image metadata Mon, Manila, Manila-Ganesha, Ceph-FS
for virtual machines as well as loading real image. Required
relationships to provide basic functionality: Keystone.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 377

A. MaaS server deployment editing options. To set a static IP address, you only need to
A virtual machine with 4 vCPUs was created in our in- set ”IP mode” on ”Static assign” and in the field ”IP address”
frastructure for the MAAS server (virtual central processing write the necessary IP address, namely 10.11.0.3 and just save
units), 8GB of operating memory and a 50GB disk. In this the interface using the green ”Save interface” button.
case it was still necessary to manually install the operating We create the tag in the ”Configuration” tab in the Tags
system, specifically Linux Ubuntu in version 22.04.1 LTS. By section, where we click on ”Edit”. Subsequently, using the
using command, we will generate new SSH keys to the file search field that appeared to us, we will create a label ”juju”
/home/student/.ssh/student: by entering it in the field and then the option ”Create tag
”juju”” will appear and clicking on this option will bring up
ssh-keygen -t rsa a label creation window where we can write also comment
We can then start the MaaS installation. We will install or specify options for Kernel. Then we just finish using the
MaaS in version 3.1, IP address for the init command, we button ”Create and add to tag changes”. Where in the edited
will use the one we configured during the installation of ”Tags” section we see that it has been added label juju and
the operating system. With the createadmin command, we we just save with ”Save”.
create a user account with the name ”student”, We replace
<PASSWORD>with a real password and set the email to V. A DDING SERVERS DESIGNED TO RUN O PEN S TACK
”student@kis.fri.uniza.sk. As in the case of Juju server, in order to take advantage
sudo snap install maas-test-db of the automation feature, we will add servers on which
sudo snap install maas --channel=3.1/stable OpenStack will be installed into MAAS. It will be one the
sudo maas init region+rack --maas-url \ so-called control node Controller Node and three computing
http://10.11.0.2:5240/MAAS --database-uri nodes called Compute Nodes. Controller Node is a virtual
maas-test-db:/// machine running on the same physical server as MAAS and
sudo maas createadmin --username student \ Juju with 32 vCPUs, 64GB of operating memory and 100GB
--password <PASSWORD> --email of disk space. It will be possible to add the same in a way like
student@kis.fri.uniza.sk a Juju server. IP address it will not need to be configured yet
sudo maas apikey --username student > \ because we will be making changes to the network settings at
maas-api-key a later stage.
We will use the second method for Compute Nodes. First,
Subsequently, we can continue initializing MAAS di- it will be on individual servers it is necessary to set booting
rectly in the browser at the address from initialization: via PXE as a priority and then after turning on MAAS these
http://IP of MaaS:5240/MAAS. After the initial login, it is assigns servers and finds information. In this case, it will not
necessary to do basic configuration. be necessary to set IPMI, as MAAS will do it for us. A
B. Juju server deployment distinction will need to be made using this method of addition
individual servers because MAAS defaults to a random name
At this point in the deployment, we can already use the
for them and this will be needed edit also with the creation
functions of the MAAS server, which we will make installing
of a tag (Tag). We will also not configure IP for these servers
Juju server easier. First you will need to add the juju1 server
addresses. We change the name of the server by clicking in
(VMWare), for which 4 vCPUs, 4 GB of RAM and 50 GB
the upper left after clicking on the server corner on the name
of disk were reserved. Add virtual or physical machine to
and then enter the desired and save using the ”Save” button.
MAAS is possible in two ways. The first way is to turn it
on server and if it has PXE booting enabled, then MAAS
A. Network configuration at MaaS system
assigns it an IP address and then provide a bootable image.
After loading the image, MAAS collects the basic information As the first change and preparation for the configurations of
about the machine and displays it in a list in the web browser. the interfaces on the servers we have to modify subnets in the
Now editable variables such as name, IP address and IPMI as MAAS system. First, we will create spaces (Spaces) that serve
required. The second way is to fill in the web form MAAS as a grouping of subnets and allows them to communicate with
interface the necessary data to add the machine and start each other. Next, we will adjust the space for ”fabric-0”, we
commissioning. will create vlans and subnets. We will make all these changes
After successful commissioning, it will still be necessary to using MAAS CLI and therefore first need to login with user
make changes to the configuration for this machine and the account and API of the MAAS key on the MAAS server.
IP address to the static 10.11.0.3 and the creation of the so- Subsequently, we can create spaces. After each order we
called tag. All of them these changes will be made in the web receive information, whether the creation was successful in
interface. After clicking ”juju.maas” first in the ”Network” tab, the form of a ”Success” statement and information about the
in our case at the ens160 interface, we click on the pointing given structure such as name and identifier. We will need two
arrow down under the ”Actions” section and select the ”Edit spaces with names ”openstack-mgmt” and ”openstack-vxlan”
physical” option. It will then open up to us window with which we will list as required when building parameter [9].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 378

B. Setting the minimum kernel of compute node • the number of servers on which this component will be
During a test deployment of OpenStack, there was a com- installed
munication problem between MAAS and one from compute • the number of servers on which the monitoring compo-
nodes, specifically compute-6. The problem was in the com- nent will be installed
munication between the server and MAAS after the operating Except for the header section, each section has its own title
system is installed. Commands from MAAS when starting the and the information in it is indented tab or, in the case of the
installation to turn on and then load the operating system went relations section, a dash.
well, but further the server already failed to communicate, In the last section of applications (components), it is differ-
according to the error message it did not have network ent for different applications because not all of them have
connectivity. Due to our heterogeneous composition of servers, the same capabilities, but the basis is the same. We will
such a fact may also occur, when compute-6 is the same type discuss specifically Nova component. The first are bindings,
of server as compute-3 and compute-4, but has a different which are an optional element and state, which data from
network card. The solution was to set the minimum kernel for the given component will use the given space from MAAS.
the operating system, which also contained the driver for the Annotations they specify the point according to the X and Y
network card that was installed in the compute-6 server. We coordinates where the icon of the component will be placed in
can set the minimum kernel of the operating system both via Juju web graphical interface showing the given model. Charm
the CLI and in the web interface. Through the CLI, we log and channel determine which version of the given component
into MAAS, if it is not already and using the command to set will be installed, we can find a complete list of versions
the minimum core and system ID for the server, we set the and channels on charmhub.io, e.g. https://charmhub.io/nova-
desired one value. In this case we save the output that would compute. Number of units (num units) indicates how many
be written with ”>/dev/null” as it is too long. For verification, times the component will be installed. Options vary by com-
we will use the command only to list the item,which contains ponent from the component, the list and meanings of the
our checked value. individual options can also be found on the page charmhub.io.
In the case of setting via the web interface, in the ”Ma- Finally, there is an optional option when we can specify which
chines” tab, open ”compute- 6 Maas”. In the ”Configuration” machines the component will be installed, otherwise Juju will
tab, we click on the ”Edit” button, which is located in the do it automatically. In case components that are installed in
sections ”Machine configuration”. We open the drop-down containers must be listed before the machine number ”lxd:”.
menu ”Minimum kernel” and choose the desired value, in In Figure 1 you can see the model of the connection of the
our case ”focal (hwe-20.04) and click on the button ”Save components as created by the Juju system.
changes”. After performing these actions, we can check that
previously the value ”Minimum kernel” was set to ”—” and VII. C ONCLUSION
now there is our desired value ”hwe- 20.04”.
The main goal of this paper was the deployment of Open-
VI. I NSTALLATION PACKAGE ( BUNDLE ) FOR O PEN S TACK Stack cloud platform using the Charms method on the physical
As already mentioned, we will use the installer to install topology of the Department of Information Networks. In addi-
OpenStack itself the so-called package bundle, which will tion to physical servers, also the installation and preparation of
be part of the attachments. Compared to manual installation, virtual servers that were created to install MAAS servers, Juju
there are mainly advantages in that it is not necessary to copy and the remaining power used as a controller for the platform.
each command or create configuration files for components After gathering the requirements, the components for in-
and everything is nicely understandable and clear in one file. stallation that are needed were selected. Subsequently, it was
It means, that after opening and viewing such a file, the moved to the actual installation of the servers in the necessary
reader will know what the components will be installed, what order and thus first MAAS, which had the task of deploying
relationships will be created or network interfaces used, and additional servers efficiently and therefore automatically. Us-
more. Bundleconsists of five parts: header, variables, machines ing MAAS, the Juju server was also deployed, which serves
(servers), relations and applications. In the header the name as a kind of ”app store”. for Charm-y during installation. It
and series are mentioned, in our case the name is ”openstack- was necessary to configure the physical servers in the MAAS
base” and we will use it ”focal” series for Linux Ubuntu system, both on the marking side and on the network side.
operating systems. Based on the list of components, an installation pack-
Next is the variable section, in which the ”focal” series, age (bundle) was created for the automated deployment of
used for determination, is the first to be determined of the OpenStack, where the layout of the individual was created
operating system on the servers together with the version of components. If the component had a control characteristic,
OpenStack. The data-port specifies which network bridges it was primarily installed on the controller, otherwise it was
will be mapped to which network interfaces. In our in this installed on the compute node. During the work, additional
case, this section will also contain variables for the Ceph Osd servers were also added twice, which required fine-tuning
component, namely: proper communication and functionality so that they can be
• hard drives that will be used for this component engaged.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 379

 [10] J. Smieško and J. Uramová, ”One-parameter Methods for Recogniz-
ing DDoS Attacks,” 2020 18th International Conference on Emerging
eLearning Technologies and Applications (ICETA), Košice, Slovenia,
2020, pp. 628-633, doi: 10.1109/ICETA51985.2020.9379155.

Fig. 1. An example of displaying a model in the Juju web interface using

icons components and connections (relationships) between them

ACKNOWLEDGMENT
This paper is supported by project KEGA 051ŽU-4/2021
”Technologies of private cloud environments in university
education”.

R EFERENCES
[1] What is OpenStack?. [online]. 2023,
https://www.openstack.org/software/
[2] OpenStack Deployment Tools. [online]. 2023,
https://www.openstack.org/software/project-navigator/deployment-tools
[3] J. Smieško, M. Kontšek and R. Hajtmanek, ”Anomaly recognition
in bursty IP traffic models,” 2021 19th International Conference on
Emerging eLearning Technologies and Applications (ICETA), Košice,
Slovakia, 2021, pp. 351-358, doi: 10.1109/ICETA54173.2021.9726543.
[4] OpenStack Components. [online]. 2023, Available:
https://www.openstack.org/software/project-navigator/openstack-
components
[5] J. Smieško, M. Kontšek and R. Hajtmanek, ”Anomaly recognition
in bursty IP traffic models,” 2021 19th International Conference on
Emerging eLearning Technologies and Applications (ICETA), Košice,
Slovakia, 2021, pp. 351-358, doi: 10.1109/ICETA54173.2021.9726543.
[6] OpenStack Charmers - Vault. [online]. 2023, Available:
https://charmhub.io/vault
[7] J. Jurc, M. Sterbak and M. Kontsek, ”Virtual laboratories and their
usage in university environment,” 2020 18th International Conference on
Emerging eLearning Technologies and Applications (ICETA), Košice,
Slovenia, 2020, pp. 260-265, doi: 10.1109/ICETA51985.2020.9379179.
[8] Juju Charm – RabbitMQ. [online]. 2023, Available:
https://github.com/openstack/charm-rabbitmq-server
[9] J. Jurc, M. Sterbak and M. Kontsek, ”Virtual laboratories and their
usage in university environment,” 2020 18th International Conference on
Emerging eLearning Technologies and Applications (ICETA), Košice,
Slovenia, 2020, pp. 260-265, doi: 10.1109/ICETA51985.2020.9379179.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 380

 Mobile application to support the teaching of
computer networks
M. Murin*, K. Nalevanko*, E. A. Katonová*, R. Petija*, P. Feciľak*
*
Department of Computers and Informatics, Košice, Slovakia
miroslav.murin@cnl.sk, kamil.nalevanko@student.tuke.sk, erika.abigail.katonova@cnl.sk, rastislav.petija@cnl.sk,
peter.fecilak@cnl.sk

Abstract—Mobile guide for Computer Networking subjects tool for students of this subject. At the same time, it
is an application that serves as an aid to Computer should help them prepare for courses CCNA 1, 2, 3. We
Networking objects. The main goal of the work was to therefore believe that the application will really help some
create a set of tests that would help students in education. students to achieve better results in the subject. Since most
students have mobile devices, they will be able to have a
When designing the application, we also created a user small helper with this subject directly in that mobile
profile that allows students to track their progress, we device, available everywhere with them.
additionally implemented the theory and configuration
sections. The theory and configuration section in the II. TASK FORMULATION
application provide quick access to theoretical and
configurational instructions. The results of the testing to The first task for creating the Mobile Guide to
which the application has been subjected clearly indicate Computer Networks is to create a mobile application.
that the application is usable in the teaching process This mobile application should contain testing and
important information about Computer Networks. In the
Keywords—computer networks, mobile application, Kotlin introduction we focus on the analysis of educational
applications and their prevalence. We also focus on their
I. INTRODUCTION creation and design. Further in the work we try to design
a functional application that mediates information from
Mobile guide for Computer Networking subjects is an the subject Computer Networks. The application should
application that helps people to get an education in this
field. It can be used as a replacement for the NetAcad also enable testing the acquired knowledge from the
website, which is currently being used for teaching subjects of Computer Networks. Testing will be realized
Computer Networks. The NetAcad page contains tests by the means of quiz questions or tests. Therefore, future
divided according to individual courses CCNA1, 2, 3. users of the application should be able to test their
Like the page, there are also tests in English. The tests knowledge along with gaining it. We have created the
consist of many theoretical questions and the timing of application using the Android studio environment, which
their development is relatively long. Also, the test process is intended for the development of mobile applications.
cannot be paused or interrupted, so the test must be We had a choice of several programming languages in
completed immediately after starting. The long solution each environment. The chosen programming language we
time and the number of questions mean that students do used to program the application was Kotlin.
not complete these tests conscientiously. Most students
search for individual information on the Internet. As a III. EDUCATIONAL APPLICATIONS
result, students do not study in that direction despite
completing the test. In our application, we tried to solve Nowadays, educational applications are a very
mentioned problems. widespread tool for study. The aim of such applications is
to make it easier to obtain information during the study.
By shortening the tests, we wanted to make sure that Likewise, applications can improve the interaction
students filled them in conscientiously. Another thing we between students and the subject. The advantage of such
wanted to implement in the application is the processing applications is that they are available on mobile devices
of the theory into brief notes. In this way, they will be able that most students carry with them. If we want to give an
to easily repeat the theory that the students have already example of such application, we can mention for example,
heard during class in the application. The information that applications developed for driving schools that contain
is provided on the NetAcad website is well processed, so quizzes. The authors of the research [1] claim that the
we created the content of our theoretical part using this inclusion of mobile applications in teaching has a positive
website as a source. The subject of Computer Networking effect on students' learning. In such applications, it is
is a relatively popular one, therefore it was one of the important to make sure that they can keep the user's
reasons why we decided to create this application. We attention. This aspect mainly affects the design and
wanted to make the application as friendly as possible and functionality of the application. Therefore, in our
so that it can successfully provide information. As this is application we focused on a suitable environment and
the first application for this subject, we will strive for easy operation. Because applications that have good looks
quality workmanship. The application of a Mobile guide and good user control attract more than those that don't
to Computer Network subjects should therefore be a good have these parts handled well.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 381

A. The use of applications in education interface. When designing our application, we handled
Today, we are familiar with several study aids in the these aspects very consistently.
form of applications that help with education. With the
progress of such applications, schools themselves are IV. MOBILE APPLICATION DEVELOPMENT
trying to implement such teaching methods. Using the Mobile application development involves several
help of educational applications, students tend to interact important factors, such as features, security, the fluent
better with subjects. Our Mobile guide for Computer running of the application or the user interface. When
Networking subjects should also serve this purpose. An developing a mobile application, it is important to
important point in such applications is that they are also determine what type of application we want to create. So,
somewhat fun and thus make the way of receiving we can create a native application that will only work on a
information more pleasant. With this method, we can mobile device or an application that works on any
better capture the user's attention and help him gain platform. In our case, we created a native application for
knowledge in a fun way. The authors in [2] argue that mobile devices only. The market for mobile applications
applications can reduce differences between students. is generally very expansive. Some applications have
Also, educational applications can improve the quality of programming errors that cause the application to lag or
our education. The authors in [3] also claim that even to crash. In our application, we focused on well-
applications can improve students' motivation and thus managed functionality. As for programming
their learning outcomes. Our goal was also to improve the environments, today we have a wide range of such
results of the study process when developing our environments. Likewise, the offer of programming
application. Currently, our university uses the NetAcad languages for programming mobile applications is very
website for education in Computer Networking subjects. wide. We chose the Kotlin programming language to
This webpage contains teaching materials, English tests, program our application. According to the authors who
and configuration instructions of various kinds. The tests wrote [5], the developers claim that the Kotlin
on the site are comprehensive and contain several programming language is easy to understand and increases
questions. They are also time-limited, which means that the quality of the code produced. It is also possible to use
after running the test, it is necessary to complete the test in Java or C# programming languages for the development
each time window. Therefore, students do not enjoy filling of mobile applications, of course the offer of such
in tests conscientiously and often make it easier for them languages is much larger. If we were to develop an
to cheat. Therefore, our application should include tests in application designed for the iOS operating system, we
a much shorter form and without a time limit, which would be working with the Swift programming language,
would not force students to take the test as soon as which is intended for the development of such
possible after starting it. This adjustment to the way we applications and is the most used. The mentioned
examine students could help us educate students better programming languages differ and have different
and more effectively. advantages. As we have mentioned different types of
programming languages, there are several programming
B. Availability of educational applications environments to choose from. When developing our
Today, the availability of educational applications is at mobile application, we decided to use Android Studio,
a high level. Thanks to many distribution platforms for which is designed specifically for mobile application
applications, we have access to many such applications. development.
Great educational applications are created just behind the
vision of some earnings. Such applications are aimed, for A. Comparison of languages used in application
example, at bodybuilding or a healthy lifestyle. Design development
and development of such applications is a time-consuming As we have already mentioned when choosing
process, as is their implementation. As users of mobile programming languages for the development of mobile
devices, we have access to applications mediated through applications, we have a wide range of programming
the Google Play platforms or the App Store. Developers languages available. However, deciding on a specific
who contribute applications to these platforms focus less programming language may not be easy. When deciding
on developing educational applications. This may be due which programming language to use to develop our
to high competition in the market and the rapid application, we had many programming languages to
development of the market, in which something different choose from, as well as the environments. We can
comes to the fore every year. Developers focus mainly on mention, for example, the programming language React
originality and thus try to achieve the success of their Native, which is used to program Facebook. This
applications, which can keep people and entertain them programming language is used to develop applications
enough. The authors in [4] claim that the interest in designed for Android as well as iOS. When developing
integrating applications into teaching will continue to applications on iOS, it is possible to use, for example, a
grow. We do not target our application on competition or special development environment designed specifically
market success. The main task of our application is the for iOS, which is Xcode. Xcode is only integrated on
usability and efficiency of processing the information that MacOS and supports most programming languages. If we
its users will receive. Different types of training wanted to choose a typical development environment for
applications can be mentioned in examples, fitness the Android operating system, we could opt for Android
applications that show the user how to exercise. We can Studio. Android Studio development environment
also mention applications for education in the field of supports Java, C++, and Kotlin as programming
mathematics, which contain various tests and examples or languages. Since 2019, Kotlin has been the preferred
applications that contain tests from driving school. An language for application development and is used by
important part of mobile applications is design and user programmers more frequently than Java. The difference

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 382

between Java and Kotlin is that Java is a bit more A. Design of the testing section
complicated. The languages used in the Android studio are The most important element of the application is the
object-oriented as well as C # or Swift. Swift is a modern
testing section, which will be used to test the acquired
and primary programming language that was designed for
iOS, and it is the successor to the older programming knowledge of Computer Networks. This section contains
language for iOS, Objective-C. The authors in [6] state 3 basic divisions into CCNA courses 1, 2, 3. Furthermore,
that among the various programming languages, Java these courses are divided into individual modules. The
clearly stands out, which for several decades has become user will be able to select any module from which they
one of the most widely used languages among want to take a test. After clicking on the module, the user
programmers. Kotlin, which is one of the younger will be shown the option to select the type of test. If the
languages, is much simpler than Java, especially for user wants to take a short test, they choose a short test
people who are new to application programming. The that contains 10 questions. Otherwise, they choose a long
difference between Kotlin, C #, and Java is that Kotlin can test consisting of 20 questions. After selecting the test
use much fewer lines of code to solve the problem. Given type, the user continues directly with the test, which is
that Kotlin has been ahead of Java in recent years, as well
composed of random questions from the module. The
as the fact that Google has announced that Kotlin is the
preferred language for mobile application development, questions that are generated are not repeated even if the
we chose Kotlin when choosing the application test is opened again. The given answers in multiple
programming language. Also, according to the authors in choice are also always randomized. In the test, the user
[7], Kotlin is spreading among developers, and several will have a graphical representation of the progress as
studies have highlighted the various advantages of this well as a numerical display of the order of the questions.
language compared to Java. An arrow will also be displayed in the lower right corner
to guide the user to the next question. When the test is
B. User interfaces in mobile applications completed, the user will see a result with success
Mobile applications provide different types of user information.
interfaces. The user interface is a very important aspect of
mobile applications. According to the authors in [8], a B. Design of theoretical and configurational section
user interface is a physical means of communication Other important elements of the application are the
between a person and a software program or operating theoretical and configurational part, in which students can
system. Even though the user interface is an important easily find the necessary information about the subject. If
aspect of the application, it is not the only one that the user clicks on this part of the application, they will
matters. The UI does not have such a big impact on the see the same breakdown as when taking the test.
operation and delay of the application. The authors in [9]
argue that long delays are usually caused by network and Individual modules are processed using HTML, CSS, and
storage I/O operations, while short delays are mainly JavaScript, which simplifies working with text. The
caused by rendering. The application interface should be theoretical part contains an abbreviated version of notes,
easy to use and functional. When developing our with which the student can quickly and effectively recall
application, we wanted to use the Slovak language, which their knowledge or learn new information. In the test part,
could ensure better connection of students with the a search function will be implemented, with the help of
application. The target group of the application are which they can easily and quickly find the necessary
students at our university, which means that the Slovak information. Similarly as the testing part, this search will
language is an advantage of the application. A very include the configurational part. In the configurational
important part of creating user interface is the choice of section, the user can easily find various configuration
colors and their shades. Aspect that greatly influences the examples from setting the router name, to address
user's return to the application is its graphical side.
translation or OSPF routing protocol configuration.
Therefore, when designing and developing our
application, we focused on the selection and combination C. Design of the user profile section
of colors. As well as the graphical side, the placement of
In Mobile guide for Computer Networking subjects,
buttons and links in the application has also a strong
impact on the impression. The interactive part of the the user also has the option of creating their own user
application and the static part of the application should be profile. Thanks to this option, the user will be able to
clearly separated. easily monitor their progress in tests. The procedure will
be displayed in the profile section, which is in the main
V. APPLICATION DESIGN AND APPLICATION USER menu of the application. If the user clicks on this section,
INTERFACE they will see the main tab with their basic information,
under which they will find the individual CCNA courses.
We have thoroughly designed the application before
Progress across courses is graphically represented by
programming and thus avoided unexpected problems.
a horizontal line. After clicking on one of the 3 courses,
The goal of our application was to create functional
the user will get to the window with individual course
testing of users' knowledge using quizzes. As we
modules. In this section, the user can easily see how
mentioned, it is important that the application has a well-
successful they were in the specific tests. At the same
designed user interface. The application should also be
time, these test cards are color-coded and given the
clear and easy to use.
success of the test, the user will be able to see a
percentage of success on the cards.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 383

D. Design of the registration and login section
An important part of creating a profile in the
application is the registration and login form. After
clicking on the profile, the user will be able to register or
log in. In case of registration, the user enters their name,
email address and lastly a password, which must be
entered twice. By entering the password several times, we
will be able to verify its compliance and thus notify the
user that they have made a mistake in it. To successfully
complete the registration, it is necessary to confirm the
verification link, which will be sent to the specified email
address. If the registration is confirmed, the user can log
in to their profile and follow their progress.
The data that the user can track in their profile is stored
in the local storage in the application after each Figure 2. Menu
completion of the test. The storage is specific, because it
remains unchanged even after the application is Figure 2. shows the menu, in which has the user a
completely shut down or the cache is cleared. The data choice of several options, such as user profile, test
that is stored in this storage after the test is completed is section, theory, or configuration section. After selecting
then moved to the profile part. If the application detects one of the tests, theory or configuration options, the user
that an internet connection is available, the data is sent to gets to choose between the topics that the subject offers.
the online storage. We mediate the online storage using Subsequently, by choosing a topic, the user gets directly
the Firebase platform and use its database, which allows to the test, theory or configuration that belongs to them.
us to easily manage it. Figure 1 shows an example of a From each option, the user can go back using the arrow in
database filled with data and test results that will be the upper left corner and then select another option.
backed up for users.
A. Implementation of testing section
After selecting the menu, the user clicks on the word
Tests to get to the main division of tests, which consists
of three cards. CCNA1, 2, and 3 course cards with basic
information such as the number of modules in the course.
Choosing the either of the course cards, the user gets to a
section dedicated to tests of that specific topic. In the tests
section, we have a choice of several topics, in which there
are different questions, chosen again at random so they
are not repeated. When selecting the part from which the
user wants to take the test, the test will be opened and
questions for the given test will be generated and loaded.
The number of questions is a maximum of 20, but there
are also tests consisting of 10 questions. Thus, the user
can choose between a longer or shorter test. The tests do
not have a time limit and therefore the user can work on
Figure 1. Firebase application database the test for as long as they want. The aim of the test part
is a concise and short form that will contribute to gaining
knowledge. After completing the test, the completion will
VI. IMPLEMENTATION OF APPLICATION be confirmed, and the evaluation of the test will be
FUNCTIONALITIES AND INTERFACES
displayed.
We programmed Mobile guide for Computer There are four possible answers to each question. To
Networking subjects using the Kotlin programming complete the test, they must answer all the questions that
language. The programming environment we used for the test contains. If the user leaves the test before
programming was Android Studio. We chose the completing it and the test is not evaluated, they start again
programming environment as well as the programming with different questions when selecting the same test.
language based on the analysis of the market. The Tests consist of questions that are loaded from pre-
application contains a menu, in the left top corner, in created JSON files. The test is created randomly, and the
which the user has a choice of three options. The menu is chances of the questions being repeated are minimal.
displayed by three white horizontal lines. Each question has four options, of which only one answer
is correct. The correct answer is marked simply by
clicking directly on it. After clicking on the correct
answer, the option will be marked with a subtle shade of
green. If it is an incorrect answer, the correct answer is
also marked after clicking. This means that the correction

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 384

takes place immediately when the question is answered.
At the bottom of the test is the number of answered
questions from the total number of questions we chose for
the test. There is also a small colored indicator on the top
of the test. After clicking on the answer that the user
considers correct, an arrow will appear in the lower right
corner, which will navigate the user to the next answer.
At the end of the test, the score obtained during the test
and a summary of correct, and at the same time incorrect,
answers will be displayed.
B. Implementation of theoretical and configurational
section
In the theory and configuration section, the user has
information about various issues from the subject of
Computer Networks, divided according to individual
topics. By clicking on the main menu and then on the
inscription theory/configuration, the user gets to the basic
choice between CCNA courses. When selecting one of
the topics, a window with the given issue will open for Figure 3. User registration form
the user, which contains the processed basic information. We have implemented a password confirmation box
The theory contains examples as well as pictures that help to reduce the number of times a user enters a password
to understand the issue. We have divided the individual that inadvertently made a mistake and does not notice.
topics of theory and configuration in the same way as the Thanks to this measure, our application will notify the
tests, which means that the answers to the questions in the user if the entered passwords do not match, and the
tests can be studied by the user right in the application. In registration does not continue.
the case of the theoretical and configurational part of the If all fields are filled in correctly after clicking on the
application, we used a web interface and worked with registration button, the registration is successful. The last
HTML and CSS. While the whole test section is done step to complete the registration is to verify it using the
using the Kotlin programming language, the theory and link sent to the email address they provided during
configuration sections are programmed in HTML and registration. This authentication is implemented in the
CSS. It works by inserting a web browser into the application to prevent theft of email addresses. We used
application window, into which the local web page is the Firebase platform, developed by Google, to register
loaded. This loaded website already contains our theory for the application. By implementing Firebase in the
or a specific topic from it. We decided to work with these project, we used the authentication and functions of the
languages due to easier manipulation and stylization of Firebase platform, which allows us to register users, log
the text. HTML combined with CSS, allows us to create them in and easily manage registered accounts on their
paragraphs, various tables or lists in minutes. There are website. Login can take place directly after registration,
also added functionalities that we implemented using the when the user is automatically switched to the login
JavaScript programming language. A search box is also window or after clicking on the login in the profile. In
implemented in these parts of the application, thanks to case of an incorrect password, the application notifies the
which can user find specific parts of the theory or user that he has entered an incorrect password, as well as
keywords. for an incorrectly entered email address. In case the user
C. Implementation of user profile and registration has forgotten the password, they click on the word
section forgotten password below the login button. Subsequently,
the user is moved to the screen with a field to fill in the
The application has an implementation of possibility email address for which he wants to reset the password.
to register. The user can access the registration by After filling in the e-mail address and pressing the
clicking on the register button, which is in the profile. confirm button, the user will receive a link to create a
After clicking on the registration, a window will open in new password to the specified e-mail address. In case of
which the registration itself takes place, shown on Figure incorrectly entered email, the application will notify the
3. The user fills in their basic data, such as email address, user of this fact. While the user is logged in to the
name, password. If the user does not fill in a field and application, all results from completed tests are saved for
wants to continue, the application will notify them, and the user. This progress in individual courses is displayed
the registration will not take place. in the profile in graphical form as well as in the form of
percentage, as shown on Figure 4.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 385

 can see the environment where the question is at the top.
At the bottom there are 4 options, one of which the user
chooses. If they select the correct answer, the marked
answer is highlighted in green, and an arrow appears at
the bottom right. Using the arrow, the user can get to the
next question. To monitor the progress of the test, the
user can use the numerical display at the bottom, which
indicates which question of the total number are they on.
After completing the test, the results screen will be
displayed, on which the student will see the evaluation of
the test and the number of correctly and incorrectly
answered questions. The results screen is colored green if
the success rate of the test is greater than 50 percent.
Conversely, if the success rate of the test is less than 50
percent, the results screen is colored red. Also, when the
test is completed, the results are saved to local storage,
allowing the user to track their progress in the profile.
Later, if the application registers a network connection,
the data is sent to an online database, which serves as a
backup when the user reinstalls the application or logs on
to another device. We also implemented the theoretical
and configurational part in the application. In the
theoretical part there is a theory divided exactly as in the
Figure 4. User profile testing part. The notes are abbreviated and do not contain
The user profile is in the main menu. In the profile, all the knowledge found in the scripts on NetAcad. The
the user can see their personal progress, achieved and individual topics therefore contain the most important
completed tests. The profile displays the name and email information and key findings for the study, in which it is
of the user who is currently logged in to the application. very easy to orientate. Users can read information on the
This information is displayed at the top of the window. If topic and review the knowledge before the exam. This
the user is not registered or logged in, the option to part of the application implements a search box that helps
register or log in will appear in that window. At the end them find specific information. The application also
of the test, the result is saved in this local storage and contains an user profile. In this part of the application, the
displayed in the profile as well. If the user logs in to a student can observe the personal progress in the
new device or reinstalls the application, the achieved application. It works on a method that for each successful
procedure and all its data contained in the application will or unsuccessful test, the application saves the result in its
be automatically downloaded to the application from memory and at the same time in the online database. The
online database. application is moving this data to the user's profile.
Directly in the profile there is the name and email address
VII. EVALUATION of the user together with the option to log out. Below the
The aim of the Mobile Guide for Computer main panel there are cards of individual courses, on
Networking subjects was to create a mobile application to which there is a graphical representation of the progress
help students study this subject. The main part of the of each individual course. After clicking on one of the 3
developed application was the testing part, in which users cards, the user gets directly to the overview of the course.
can verify their knowledge in mentioned field. During the In this window, the individual tests are graphically
development of this application, we successfully displayed, arranged in the same order as they are in the
implemented a method of testing based on the random test section. The cards of the individual tests are colored
generation of questions and answers from a given topic. according to the success the user achieved in the test. We
Users can easily test themselves in the application and tested the application by cooperating with the heads of
take tests on any topic and area of knowledge in the Computer Networks at the Technical University in
subject of Computer Networks. The individual modules Kosice. A group of students in the field of informatics,
contain more than 20 questions, from which tests are then who study Computer Networks at the school, took part in
generated using an algorithm. Altogether, all three testing the application. Testing of the application took
courses contain more than 320 questions that can be place directly during the classes of Computer Networks.
tested by users of the application. When creating a test The number of students participating in the testing was
and selecting test questions, their originality is also 20. Testing took place by presenting the work to students
checked. If there is a case that 2 questions match, the and showing all the functionalities of the application.
program selects new questions again and checks the Subsequently, a questionnaire was sent to the students.
originality of all questions. After completing the whole The questionnaire consisted of 17 questions, 10 of which
test, all answers are randomly mixed, of which there is focused on the usability and user interface of the
still only one correct one. After creating the test, the user application in the form of a scale of usability of the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 386

system. The remaining 7 questions focused on the interest even more detailed view and more comprehensive
and usability of the application for the subject, while information. At the same time, if they have not created an
those 3 remaining were open questions, which could be account, registration and login to the application also
answered freely with their own opinions. In the takes place in this section.
questionnaire, they were able to express their overall The registration contains various measures against
personal viewpoint towards the application and assess the incorrectly entered data or blank fields. It also contains
applicability of the system. According to their evaluation, functionality, thanks to which the user does not
we reached the value of 87 SUS scores on this sample of accidentally enter the wrong password. To prevent the
students, which is an excellent result. We thus achieved misuse of foreign email addresses, we used the
the best possible mark in SUS, namely A. According to confirmation of registration by a verification link sent to
the survey, all respondents stated that they would register the email address. The security of login data is
in the application and use the profile and the opportunity implemented using Firebase functionalities, where the
to see a personal progress. Respondents also agreed that data is encrypted. In case of forgetting the password, the
the application would help them study the subject. When user can easily reset the password and create a new
asked if they could imagine using this guide as an aid to password using the link, which is also sent to the email
the subject, everyone answered unequivocally, yes. In the address.
end, up to 85% of users rated the system with the highest Mobile guide for Computer Networking subjects has a
possible score. This is an excellent result given that we clear potential to acquire new functionalities and expand
have largely focused on a well-designed environment and in the future. The user will easily find the information
interaction when analyzing and designing the application. needed to master the subject in the configurational and
According to answers to the question of what they like theoretical part. These sections are designed to present
about the application, users agreed on the design and the the knowledge as concisely and best as possible and to
possibility of having information about the subject so fulfill their purpose. In the case of the profile part of the
easily with them. The results of the forms and testing on application, there are several directions in which we can
the target group, clearly indicate that the goal of making improve the application. Adding different challenges and
the system easy to use and clear has been met. The goal tasks to the user can enhance a positive user experience.
of creating a versatile application in which the student Based on testing and subsequent results, where we
can recall their knowledge and test themselves has also achieved a rating of 87 SUS scores, we can clearly say
been met. The resulting solution contains more than 120 that the application has a real use in practice.
individual files in which the application was programmed
and approximately 5,000 lines of code together with the ACKNOWLEDGMENT
files of the web part of the application, which is This work was supported by Cultural and Educational
programmed using HTML and CSS. The application is Grant Agency (KEGA) of the Ministry of Education,
functional, and the graphical side is suitable. According Science, Research and Sport of the Slovak Republic
to testing, the solution is capable of being used in the under the project No. 026TUKE-4/2021.
teachings of Computer Networks.
REFERENCES
VIII. CONCLUSION [1] Ch. Ling, D. Harnish, R. Shehab, “Educational Apps: Using
Mobile Applications to Enhance Student Learning of Statistical
The aim of this application was to design and Concepts,” Human Factors and Ergonomics in Manufacturing &
implement a solution that would allow students to test the Service Industries. 2014, pp.532–543.
acquired knowledge of Computer Networks. Before [2] WANG, June-Yi; WU, Hsin-Kai; HSU, Ying-Shao, “Using
designing the application, we analyzed several types of mobile applications for learning: Effects of simulation design,
similar applications and focused mainly on the usability visual-motor integration, and spatial ability on high school
students’ conceptual understanding,” Computers in Human
and graphics of the applications. At the same time, thanks Behavior. 2017, pp.103-113
to the performed analysis, we found out what [3] ELAISH, Monther M.; GHANI, Norjihan Abdul; SHUIB, Liyana;
functionalities such an application should contain. AL-HAIQI, Ahmed, “Development of a Mobile Game Application
Subsequently, we designed all these functionalities of the to Boost Students’ Motivation in Learning English Vocabulary,”
IEEE Access. 2019, pp.13326-13337
application in the design part according to the analysis.
[4] BARAN, Evrim.; UYGUN, Erden; ALTAN, Tugba, “Examining
Mobile guide for Computer Networking subjects fulfills Preservice Teachers’ Criteria for Evaluating Educational Mobile
its purpose with respect to testing. The application is Apps,” Journal of Educational Computing Research. 2016,
divided into 4 main parts which are testing, theoretical, pp.1117-1141
configurational and profile part. The user accesses these 4 [5] OLIVEIRA, Victor; TEIXEIRA, Leopoldo; EBERT, Felipe, “On
the Adoption of Kotlin on Android Development: A Triangulation
main sections via the menu, which is in the upper left Study,” 2020 IEEE 27th International Conference on Software
corner. The testing part offers a selection of tests of Analysis, Evolution and Reengineering (SA-NER). 2020
different lengths from different topics of the subject. [6] GOTSEVA, Daniela; TOMOV, Yavor; DANOV, Petko,
After completing the test, the user gets to the results “Comparative study Java vs Kotlin,” 2019 27th National
screen, which is colored according to success. Conference with International Participation (TELECOM). IEEE,
2019
In the profile part, the user has a simple overview of
[7] COPPOLA, Riccardo; ARDITO, Luca; TORCHIANO, Marco,
the current progress, which is graphically displayed. By “Characterizing the transition to Kotlin of Android apps: a study
clicking on the individual courses, the user can get an on F-Droid, Play Store, and GitHub,” Proceedings of the 3rd ACM

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 387

 SIGSOFT International Workshop on App Market Analytics. [9] GAO, Yi; LUO, Yang; CHEN, Daqing; HUANG, Haocheng;
WAMA 2019. ACM press, 2019 DONG, Wei; XIA, Mingyuan; LIU, Xue; BU, Jiajun, “Every pixel
[8] BAJWA, I.S.; CHAUDHARY, M.A, “A Language Engineering counts: Fine-grained UI rendering analysis for mobile
System for Graphical User Interface Design (LESGUID): A Rule applications,” IEEE INFOCOM 2017 - IEEE Conference on
based Approach,” 2006 2nd International Conference on Computer Communications. IEEE, 2017
Information & Communication Technologies. IEEE, 2006

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 388

 Efficiency study of MPPT algorithms
in HDL for full integration of solar-powered
voltage converter
Róbert Ondica Adam Hudec
Institute of Electronics and Photonics Institute of Electronics and Photonics
Slovak University of Technology in Bratislava Slovak University of Technology in Bratislava
robert.ondica@stuba.sk adam.hudec@stuba.sk

David Maljar Viera Stopjaková

Institute of Electronics and Photonics Institute of Electronics and Photonics
Slovak University of Technology in Bratislava Slovak University of Technology in Bratislava
david.maljar@stuba.sk viera.stopjakova@stuba.sk

Abstract—This paper presents a workplace for practical im- sources for different electronic applications [2], [3], as the
plementation of Maximum Power Point Tracking (MPPT) al- solar cells provide an appropriate levels of output voltage and
gorithms and means for comparison of their efficiency that is energy density [4], [5]. However, advanced driving methods
critical for efficient and reliable power conversion by solar cells
as well as for the whole voltage conversion (VC) system. This is for such systems are needed for effective energy extraction and
available by providing an option for implementation of various power management control. The main reason is relatively high
MPPT algorithms to Field Programmable Gate Array (FPGA) dependence of I-V characteristics of PV cells on environmental
board that are designed in Hardware Description Languages factors (e.g. ambient temperature, solar radiation intensity,
(HDL). MPPT algorithms are created for utilizing the Pulse etc.) [6].
Frequency Modulation (PFM) control loop of a fully integrated
DC-DC voltage converter based on high frequency switching of Purpose of this paper is to present a demonstration work-
on-chip inductor with Photovoltaic (PV) cell as a power source place for efficiency evaluation and comparison of different
for Energy Harvesting (EH) system. The whole VC system was digital algorithms for maximum power extraction from the
created in standard 65 nm CMOS technology. In addition, two solar cell under different irradiation conditions.
MPPT algorithms were created based on Perturb and Observe This paper is organized as follows: Section II describes
(P&O) method and implemented on the chip.
the whole designed system at block level, while Section III
Index Terms—Integrated Voltage Converter, Conventional
describes the autonomous control loop in detail. Section IV
Boost Converter, Maximum Power Point Tracking, HDL, Con-
version Efficiency. briefly characterizes properties and boundaries of the oscillator
circuit as a critical part of control loop. The maximum power
point tracking approach with two already designed algorithms
I. I NTRODUCTION
are described in Section V. Section VI describes developed
Involvement of students into practical tasks and realizations application as a whole and the last section concludes this paper
within selected research areas and experimental verification of by summarizing results of the proposed system and by drawing
gained knowledge should be an essential part of the education the future plans.
process. We see even higher importance of these practices in
application fields such as electrical engineering and design II. S YSTEM D ESIGN
of electronic circuits, in general. This could be improved by The proposed voltage conversion system is based on topol-
incorporating practical elements directly into curriculum or as ogy of conventional boost converter with novel autonomous
a part of the bachelor or master theses. control loop mechanism. All essential parts of the DC-DC
We propose a fully integrated autonomous system for volt- converter (power switches SWLS and SWHS , inductor LM LS ,
age conversion purposes with possibilities to digitally adjust input capacitor CM LS [7]) and parts of the control loop are
operation of the system. The electronic system is a com- fully integrated on the chip. The system was designed for
bination of on-chip energy-efficient voltage conversion, i.e. standard 65 nm CMOS technology, and its block diagram is
Fully Integrated Voltage Regulator (FIVR), and conversion of shown in Figure 1.
ambient energy using a PV cell creating an energy harvesting One solar cell named PVP OW ER is used as power source
subsystem [1]. PV-based EH systems are often used as power for the whole system. Second solar cell PVSEN SE serves as a

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 389

 Figure 1. Block diagram of the proposed system

sensing element providing information about actual irradiation Current Crossing Detector (ZCCD) and Voltage-Controlled
conditions. This approach of reaching the maximum power Delay Unit (V CDU ) that automatically drives high-side
extraction from energy converters (EC) is called pilot cell [8]. power switch SWHS as an ideal diode with zero threshold
A low-pass filter (F ILT ER) is used for suppressing high voltage. This method is critical for minimizing the power
frequency voltage ripple caused by synchronous operation of losses caused by reverse flow of current or undesired con-
the converter and the voltage divider (DIV ) adjusts open duction losses across this switch.
voltage Vov of PVSEN SE to the voltage value of the maximum SWBS and CBS are bootstrap switch and capacitor that to-
power point Vmpp . Vmpp is the output voltage of the solar cell gether provide voltage VBS that is sufficiently high for reliable
under specific irradiation when the available output power is in switching-on of the high-side NMOS power switch SWHS in
its peak (the highest energy conversion ratio). The maximum each switching cycle. Blocks LS CT RL, HS CT RL and
power point is depicted in the I-V and P-V characteristics BS CT RL ensure reliable switching of the power switches
shown in Figure 2. Comparator COM P compares the output and consists mainly of buffers and level shifters.
voltage of loaded solar cell Vpv to Vmpp . SHU N T regulator provides regulated 1.5 V output voltage
Integrated blocks of the control loop include digital Maxi- that can be used as power supply voltage for operation of the
mum Power Point Tracking (M P P T ) block that automatically proposed voltage converter and as supply voltage for other
adjusts frequency of the converter, digitally controlled oscilla- circuits. For reliable operation, the input and output capacitors
tor (OSC) and monostable multivibrator (one-shot oscillator (CinEXT , CoutEXT ) were added as external components with
- ON ESHOT ) that adjusts pulse width to a constant value. the capacitance value of 4.7 nF.
This part of the control loop autonomously drives low-side Essential parts of the control loop (OSC, ON ESHOT ,
power switch SWLS . Important functionality of the control ZCCD, V CDU ) utilizes functionality for further digital
loop is an option of bypassing the M P P T circuit for different tuning of the boundaries for each block and the control loop
algorithms of autonomous control of the converter. as a whole. These blocks can be adjusted externally using
Second part of the autonomous control loop consists of Zero a FPGA board and Graphical User Interface (GUI) developed
for testing and measurement purposes, and to cover potentially
wide requirements of the proposed algorithms in the future.
Possibility to alter and/or expand capabilities of external parts
of the control loop (DIV , F ILT ER, COM P ) is considered
in the future.

III. C ONTROL L OOP

Functionality of the autonomous control loop used in the
DC-DC converter is based on dependency of the converter
input impedance on the system switching frequency [9]. Thus,
by tuning the input impedance, one can directly control power
extraction from the solar cell.
The ratio of Vmpp voltage of energy converter PVSEN SE
and the output voltage Vpv of the solar cell under load
Figure 2. I-V and P-V characteristics of the solar cell under PVP OW ER is directly related to ratio of impedance of solar
different irradiation conditions cell in MPP under actual irradiation and input impedance of

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 390

VC. One-bit digital output of comparator COM P is therefore Mutual configuration of the registers and corresponding
information for M P P T circuit about necessity of increas- value of the charging current inside OSC causes non-linear
ing/decreasing the converter input impedance that essentially frequency change with linear increment in tuning registers
means decrease/increase of the switching frequency. Aim of and more tuning settings for one output frequency. For this
the algorithm is to effectively and precisely match the input reason, the algorithm that omits certain values of registers
impedance of converter to actual impedance of the sensing must be implemented. In this way, we were able to obtain
solar cell PVSEN SE . almost ideally linear change on the whole frequency range.
Adjustment of the system switching frequency is performed
by a change in digital registers that adjust the resistance
of transistor banks (M3M) and capacitance of capacitive
banks (C2C) in digitally-controlled oscillator OSC. Output
signal of the OSC acts as synchronization impulse for the
one-shot oscillator ON ESHOT . Turn-on time of the low-
side switch SWLS is given by setup of the ON ESHOT
block. Pulse width can be set in the range from 231,6 ps
to 52,2 ns (obtained by PEX simulations). However, pulse
width for low-side switch SWLS should be more than 2,5 ns
because of pulse deformation caused by buffers and response
speed of Zero Current Crossing Detector ZCCD (obtained
by schematic simulations) and less than 10 ns because of
saturation current of the integrated inductor LM LS (obtained
Figure 3. Frequency range of digitally-controlled oscillator
by schematic simulations) [10]. The control loop essentially
obtained by simulation and measurement
implements PFM control method. By extending capabilities of
already designed system (controlling the ON ESHOT block),
the Pulse Width Modulation (PWM) could extend capabilities B. Settling time
of different M P P T algorithms in the future. Another important parameter of OSC circuit is its response
time to a change of registers. This means time needed for
IV. D IGITALLY-C ONTROLLED O SCILLATOR
stabilization of the output frequency. Only after this so-called
Digitally controlled oscillator with the topology of settling time, it is meaningful to compare voltages of solar cells
relaxation oscillator determines switching frequency in order to determine a possible need for frequency adjustment.
that is regulated by MPTT block. Switching frequency In this way, the speed of the whole control loop might be
of the OSC is adjusted by altering values of 5- limited.
bit register MPPT OSC CS TUNE and 8-bit register Settling time tsettle of the oscillator frequency was inves-
MPPT OSC M3M TUNE. These two registers change tigated in the worst condition possible for already designed
current charging capacitive element thus changing frequency MPPT algorithms. This condition is the biggest change of
of the oscillator. the MPPT OSC M3M TUNE register which will imply the
Another 5-bit register CAP TUNE changes value of capac- biggest change in resistive net configuration inside the oscil-
itive element and can be adjusted during initialization of the lator. Settling time was also observed to increase towards lower
converter. This option is included in the system for testing frequencies. The biggest step which the MPPT is capable of
purposes and can positively affect frequency range, step size is 127, changing the MPPT OSC M3M TUNE value from
or power consumption of the oscillator. M3Monset = 1 to M3Mf inal = 128 that corresponds to
a change in frequency from roughly fonset = 12 kHz to
A. Frequency Range
ff inal = 1.45 MHz. The same setup was investigated in the
Frequency range of the oscillator was evaluated by simu- opposite direction (M3Monset = 128 −→ M3Mf inal = 1,
lations and proved by measurement of chip prototypes. The fonset = 1.45 MHz −→ ff inal = 12 kHz). Settling time
minimum frequency of the oscillator is 9.99 kHz (simulation) tsettle of the oscillator specifies highest possible frequency
and 8.32 kHz (measurement). The maximum frequency of fM AX of the clock signal for M P P T block. The worst
the oscillator is 85.47 MHz (simulation) and 74.22 MHz case was observed when the MPPT OSC M3M TUNE reg-
(measurement). These results were obtained without altering ister (and frequency) decreases where settling time at 5°C is
value of the register CAP TUNE, which was set to the mini- tsettle = 1.94 μs. Maximum clock frequency for M P P T block
mum possible value that means the smallest capacitance, and is therefore fM AX = 515.46 kHz. All results of simulated
therefore, the highest frequency (value 1 was omitted because settling times are summarized in Table I.
of reliability issue). Obtained results are shown in Figure 3,
where dependence of frequency on the tuning register value V. M AXIMUM P OWER P OINT T RACKING
as combination of MPPT OSC CS TUNE (5 MSB bits) and Maximum Power Point Tracking is well-known technique
MPPT OSC M3M TUNE (8 LSB bits) is depicted. for efficiency improvement of energy conversion routinely

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 391

 TABLE. I.
Settling time of digitally-controlled oscillator
Frequency increase Frequency decrease
Temperature [°C] -20 5 27 55 85 -20 5 27 55 85
fonset [MHz] 0.013 0.013 0.012 0.012 0.009 1.474 1.472 1.448 1.417 1.394
ffinal [MHz] 1.474 1.472 1.448 1.417 1.394 0.013 0.013 0.012 0.012 0.009
M3Monset 1 128
M3Mfinal 128 1
Tuning Registers
CS 1
CAP 2
tsettle [μs] 1.52 1.51 1.53 1.52 1.52 1.79 1.94 1.91 1.78 1.90
fMAX [kHz] 657.89 662.44 653.59 657.89 657.89 558.66 515.46 523.56 561.79 526.32

implemented for solar panels, wind generators or other EH A. INC/DEC algorithm

systems [11], [12]. Control of the system could be done by INC/DEC algorithm adjusts switching frequency of OSC
various means (e.g. PWM, PFM, mechanical, etc.) depending by altering value of 13-bit output register by constant 5-bit
mainly on properties of the specific application and used value (INCDEC STEP) that is set during initialization of the
voltage conversion circuit [13], [14]. Therefore, various MPPT system. Step value can be adjusted from 1 to 31 and its
approaches could be developed and implemented, depending value positively affects time of convergence (number of steps
mainly on complexity of the control subsystem and provided until it reaches MPP) but negatively affects accuracy of the
sensed parameters (e.g. input voltage, input current, output algorithm causing large or asymmetrical oscillations around
voltage, output current, temperature or other environment con- MPP. Visualization of INC/DEC algorithm principle is shown
ditions). Furthermore, complexity and cost of particular MPPT in Figure 4.
approaches determine how often they are implemented. Many
different MPPT techniques have been proposed in literature, B. ADAPTIVE algorithm
for example: Second MPPT algorithm called ADAPTIVE adjusts switch-
ing frequency with variable 8-bit step which is adjusted
• Perturb and Observe (P&O) [15]–[18], autonomously during operation of the converter. Advantage
of this algorithm is speed of the convergence, which is higher
• Incremental Conductance (IC) [16]–[18], or comparable to the INC/DEC algorithm. ADAPTIVE algo-
• Constant Voltage (CV) [18], [19], rithm also always achieves highest possible energy extraction
because of oscillations around MPP at the level of least
• Short-Current Pulse (SC) [18], [20], significant bit (LSB). Visualization of working principle of
• Open Circuit Voltage (OCV) [18], [21], ADAPTIVE algorithm is shown in Figure 5.

• Temperature Gradient (TG) [18], VI. D EMONSTRATOR

• Temperature Parametric (TP) [18], Aim of the proposed system is to provide practical means
for verification of functionality of designed MPPT algorithms
• Input Characteristic Impedance (ICI) [22], by experimental implementation and measurement of a fully
• Artificial Neural Network [23], integrated voltage converter with PV-based energy converter.
MPPT algorithms should provide necessary regulation of the
• Fuzzy Logic [12], [24], [25], control loop by bypassing already designed digital M P P T
• Sliding Mode [26], block that was also integrated together with VC on a single
chip. External control is provided by signals generated from
FPGA board connected directly to MPPT OSC CS TUNE
and different combinations and modifications of mentioned and MPPT OSC M3M TUNE registers of OSC. This should
techniques [18]. These techniques are not specific to level results in improvement of already designed algorithms or in
of generated power and can be used in a wide range of design of new algorithms. MPPT algorithms are supposed to
applications from power plants to EH-powered miniaturized be designed using HDL such as Verilog or VHDL because
applications. of necessity to be fully compatible with a standard CMOS
Two simple MPPT algorithms were designed and imple- technology for future integration of the algorithm together with
mented on a chip within our research. Determination of the the rest of the voltage converter system.
output register value is based on one-bit information given by Monitored parameters of the measurement process should
the impedance comparison process. These two algorithms are include power conversion efficiency by measuring the input
based on P&O method: and output power, power consumption of the control loop, and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 392

 Figure 4. Demonstration of INC/DEC algorithm operation

Figure 5. Demonstration of ADAPTIVE algorithm principle

indirectly, the power dissipation of the DC-DC converter. Ad- a standard 65 nm CMOS technology and could be used in
ditional data collection could include speed of the convergence education process as extension of curriculum or as part of the
of algorithm, power consumption of digital blocks, and area bachelor or master theses.
requirements for a circuit integrated on the chip (after Synthe- A few options to extend capabilities of designed system
sis and Place&Route). Power conversion efficiency could be were proposed towards implementing more complex MPPT
compared between different control approaches on the same algorithms in the future:
PV-based EH system. Student can select and develop its own • autonomous control of pulse width of ON ESHOT
algorithm based on different MPPT techniques as mentioned block and creating PWM control method or combination
in Section V. After verification of functionality and reliability of PFM and PWM methods,
of different algorithms, selected one can be implemented on • widening the set of available information about state of
the chip to supplement and improve already designed system. the converter and environmental conditions.
For future work, the external part of control loop could
be redesigned to provide more information about voltage VIII. ACKNOWLEDGEMENT
converter state (e.g. voltage and current from ADCs) and about
This work was supported by the Slovak Research and
environment conditions (e.g. irradiation, temperature), which
Development Agency under grant APVV-19-0392, and by
might lead to possibilities of implementing more complex
grants VEGA 1/0760/21 and VEGA 1/0731/20.
algorithms.

VII. C ONCLUSION R EFERENCES

A fully integrated voltage conversion system in combination [1] S. Beeby and T. J. Kazmierski, Energy harvesting systems: Principles,
modeling and applications. Springer, 2011.
with a FPGA board for experimental verification of designed [2] P. Luo, D. Peng, Y. Wang, and X. Zheng, “Review of solar energy
digital circuits in HDL was proposed. Different MPPT al- harvesting for iot applications,” in 2018 IEEE Asia Pacific Conference
gorithms based on various techniques could be designed for on Circuits and Systems (APCCAS), 2018, pp. 512–515.
[3] S. Uprety and H. Lee, “A 0.65-mw-to-1-w photovoltaic energy harvester
practical verification and complex comparison by direct or with irradiance-aware auto-configurable hybrid mppt achieving ¿95and
indirect measurement of system properties. Designed con- 2.9-ms focv transient time,” IEEE Journal of Solid-State Circuits,
trolling methods are meant to utilize PFM control of fully vol. 56, no. 6, pp. 1827–1836, 2021.
[4] N. Bizon, N. M. Tabatabaei, F. Blaabjerg, and E. Kurt, “Energy har-
integrated voltage converter based on high frequency switching vesting and energy efficiency,” Technology, Methods, and Applications,
of integrated inductor. Core of the system is implemented in vol. 37, 2017.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 393

 [5] T. Sanislav, G. D. Mois, S. Zeadally, and S. C. Folea, “Energy harvesting [17] C. Hua and C. Shen, “Comparative study of peak power tracking
techniques for internet of things (iot),” IEEE Access, vol. 9, pp. 39 530– techniques for solar storage system,” in APEC ’98 Thirteenth Annual
39 549, 2021. Applied Power Electronics Conference and Exposition, vol. 2, 1998, pp.
[6] M. Seyedmahmoudian, B. Horan, R. Rahmani, A. Maung Than Oo, 679–685 vol.2.
and A. Stojcevski, “Efficient photovoltaic system maximum power [18] R. Faranda, S. Leva, and V. Maugeri, “Mppt techniques for pv systems:
point tracking using a new technique,” Energies, vol. 9, no. 3, 2016. Energetic and cost comparison,” in 2008 IEEE Power and Energy Society
[Online]. Available: https://www.mdpi.com/1996-1073/9/3/147 General Meeting - Conversion and Delivery of Electrical Energy in the
[7] R. Ondica, M. Kovàč, D. Maljar, and V. Stopjaková, “Fully inte- 21st Century, 2008, pp. 1–6.
grated multi-layer stacked structure of integrated inductor with patterned
ground shield,” in 2022 18th Biennial Baltic Electronics Conference [19] G. Yu, Y. Jung, J. Choi, I. Choy, J. Song, and G. Kim, “A novel two-
(BEC), 2022, pp. 1–6. mode mppt control algorithm based on comparative study of existing
[8] N. Karami, N. Moubayed, and R. Outbib, “Analysis of an irradiance algorithms,” in Conference Record of the Twenty-Ninth IEEE Photo-
adaptative pv based battery floating charger,” in 2011 37th IEEE voltaic Specialists Conference, 2002., 2002, pp. 1531–1534.
Photovoltaic Specialists Conference, 2011, pp. 001 852–001 858. [20] T. Noguchi, S. Togashi, and R. Nakamoto, “Short-current pulse-based
[9] M. Kováč, D. Arbet, L. Nagy, M. Šovčı́k, and V. Stopjaková, “Multi- maximum-power-point tracking method for multiple photovoltaic-and-
topology dc-dc converter for low-voltage energy harvesting systems,” converter module system,” IEEE Transactions on Industrial Electronics,
in 2020 43rd International Convention on Information, Communication vol. 49, no. 1, pp. 217–223, 2002.
and Electronic Technology (MIPRO), 2020, pp. 77–82. [21] D.-Y. Lee, H.-J. Noh, D.-S. Hyun, and I. Choy, “An improved mppt
[10] R. Ondica, D. Maljar, A. Hudec, R. Ravasz, and V. Stopjaková, “Analysis converter using current compensation method for small scaled pv-
of switching boundaries of a fully integrated dc-dc boost converter,” in applications,” in Eighteenth Annual IEEE Applied Power Electronics
ELITECH´23 25th Conference of Doctoral Students ELITECH, 2023. Conference and Exposition, 2003. APEC ’03., vol. 1, 2003, pp. 540–
[11] J. Enslin, “Maximum power point tracking: a cost saving necessity 545 vol.1.
in solar energy systems,” in [Proceedings] IECON ’90: 16th Annual
Conference of IEEE Industrial Electronics Society, 1990, pp. 1073–1077 [22] J. W. Zanotti, W. M. Dos Santos, and D. C. Martins, “The maximum
vol.2. power point tracking for pv systems: Introduction to the input character-
[12] S. Sreekumar and A. Benny, “Fuzzy logic controller based maximum istic impedance method,” in 2014 IEEE 5th International Symposium on
power point tracking of photovoltaic system using boost converter,” in Power Electronics for Distributed Generation Systems (PEDG), 2014,
2013 Fourth International Conference on Computing, Communications pp. 1–7.
and Networking Technologies (ICCCNT), 2013, pp. 1–6. [23] X. Sun, W. Wu, X. Li, and Q. Zhao, “A research on photovoltaic energy
[13] R. F. Coelho, F. Concer, and D. C. Martins, “A study of the basic dc-dc controlling system with maximum power point tracking,” in Proceedings
converters applied in maximum power point tracking,” in 2009 Brazilian of the Power Conversion Conference-Osaka 2002 (Cat. No.02TH8579),
Power Electronics Conference, 2009, pp. 673–678. vol. 2, 2002, pp. 822–826 vol.2.
[14] R. F. Coelho, F. M. Concer, and D. C. Martins, “A simplified anal- [24] T. Kottas, Y. Boutalis, and A. Karlis, “New maximum power point
ysis of dc-dc converters applied as maximum power point tracker in tracker for pv arrays using fuzzy controller in close cooperation with
photovoltaic systems,” in The 2nd International Symposium on Power fuzzy cognitive networks,” IEEE Transactions on Energy Conversion,
Electronics for Distributed Generation Systems, 2010, pp. 29–34. vol. 21, no. 3, pp. 793–803, 2006.
[15] R. Leyva, C. Alonso, I. Queinnec, A. Cid-Pastor, D. Lagrange, and
L. Martinez-Salamero, “Mppt of photovoltaic systems using extremum [25] A. M. Z. Alabedin, E. F. El-Saadany, and M. M. A. Salama, “Maximum
- seeking control,” IEEE Transactions on Aerospace and Electronic power point tracking for photovoltaic systems using fuzzy logic and
Systems, vol. 42, no. 1, pp. 249–258, 2006. artificial neural networks,” in 2011 IEEE Power and Energy Society
[16] W. Wu, N. Pongratananukul, W. Qiu, K. Rustom, T. Kasparis, and General Meeting, 2011, pp. 1–9.
I. Batarseh, “Dsp-based multiple peak power tracking for expandable [26] I.-S. Kim, M.-B. Kim, and M.-J. Youn, “New maximum power point
power system,” in Eighteenth Annual IEEE Applied Power Electronics tracker using sliding-mode observer for estimation of solar array cur-
Conference and Exposition, 2003. APEC ’03., vol. 1, 2003, pp. 525–530 rent in the grid-connected photovoltaic system,” IEEE Transactions on
vol.1. Industrial Electronics, vol. 53, no. 4, pp. 1027–1035, 2006.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 394

 Exploring Oracle APEX for the University Data
Analysis
Ivan Pastierik, Michal Kvet
Faculty of Management Science and Informatics
University of Žilina
Žilina, Slovakia
michal.kvet@uniza.sk, pastierik2@stud.uniza.sk

Abstract—This paper delves into Oracle Application for improvement, and tailor their strategies to meet
Express (APEX), a robust platform for web application evolving educational needs. Data analysis helps toward
development, showcasing its merits and suitability for evidence-based decisions that can enhance learning
creating a comprehensive Analytical Tool for University outcomes, optimize resource allocation, and refine
Data Management application. This application serves as an institutional policies.
integrated solution to analyse various facets of university In the realm of software development, the distinction
processes, such as student interactions, faculty engagement, between web applications and traditional desktop
study group dynamics, and subject performance, utilizing applications is stark. While desktop applications offer
visual representations like graphs and charts. Beyond data
localized functionality and are often limited to specific
analysis, the application also works as an Academic
devices, web applications transcend geographical
Information System, proficiently storing and managing
boundaries and can be accessed from any internet-
student-centric information, including exam results and
enrolled subjects. The initial sections of the paper explore
enabled device. The advent of web applications has
the features and capabilities of Oracle APEX, highlighting
revolutionized data access, user interaction, and real-time
its role as a low-code development platform that empowers
collaboration. Consequently, universities are increasingly
developers to create sophisticated web applications with turning to web applications to provide intuitive interfaces
efficiency. The focal point of this study is the presentation of for data analysis, enabling stakeholders to explore
the Analytical Tool for University Data Management insights remotely, collaborate effortlessly, and engage
application created using Oracle APEX. By harnessing dynamically.
various modules and regions within Oracle APEX, the Oracle APEX, a cutting-edge low-code development
application offers a user-friendly interface that facilitates platform, empowers developers to create robust, scalable,
comprehensive data analysis. The application's ability to and responsive web applications with remarkable
present individual and aggregate statistics through efficiency. By leveraging the power of SQL and PL/SQL
graphical visualizations empowers decision-makers with within a user-friendly interface, Oracle APEX enables the
insightful information. The application holds great potential creation of intricate applications that seamlessly integrate
for future expansion utilising Machine Learning methods. with Oracle databases. Furthermore, the integration of
Oracle Cloud with Oracle APEX offers universities the
advantage of cloud-based deployment, ensuring
I. INTRODUCTION heightened accessibility, security, and scalability. This
In the ever-evolving landscape of higher education, the synergy equips educational institutions with the means to
analysis of university data has emerged as a pivotal tool efficiently develop and deploy web applications that
for extracting valuable insights, optimizing administrative facilitate comprehensive university data analysis [1], [2],
processes, and enhancing student experiences. Within this [3], [4], [5], [6].
dynamic realm, the scrutiny of student performance, Oracle APEX is rapidly gaining prominence as a
academic processes, and institutional efficacy holds the favoured solution for web application development,
key to informed decision-making and continual particularly within the academic realm. Its low-code
improvement. As educational institutions seek to harness nature significantly reduces development time, allowing
the power of technology, the development of web universities to expedite the deployment of data analysis
applications has become an essential avenue for tools and streamline administrative functions. Thanks to
facilitating seamless data analysis and interaction. Among the Oracle Cloud, Oracle APEX offers a future-proof and
the arsenal of tools available, Oracle Application Express scalable solution that can adapt to evolving data analysis
(APEX) has emerged as a formidable platform that requirements [1], [2], [3], [4], [5], [6].
bridges the gap between traditional desktop development In the subsequent sections of this exploration, we will
and modern web application design, propelling delve deeper into the nuances of Oracle APEX, its
universities towards innovative and efficient data-driven features, benefits, and application within the bachelor’s
solutions. thesis on topic Analytical Tool for the University Data
The academic journey of students within a university Management [7].
setting generates an immense volume of data. This data
not only encompasses academic performance but also II. GETTING STARTED WITH ORACLE APEX
encompasses various administrative procedures. By
analysing this data, institutions can gain a comprehensive Oracle APEX is a powerful and user-friendly platform
understanding of student trends, identify potential areas that empowers developers, both experienced and novice,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 395

to create web applications with remarkable ease. This level of administrative control. While manual deployment
section will provide an introductory guide to getting offers greater customization and control, ADB
started with Oracle APEX. deployment streamlines the process and provides a
managed environment that simplifies ongoing
A. Deploing Oracle APEX maintenance [5].
There are generally two main options of deploying
Oracle APEX applications, each with its own set of B. Creating an Oracle Autonomous Transaction
advantages, considerations, and implications. The choice Processing Database and Connecting To It
of deployment strategy depends on factors such as In this paper, we will focus on deploying Oracle APEX
organization's requirements, resources, and the desired on Oracle Autonomous Database. In Oracle Cloud there
level of control over the deployment process [5], [8]. are available multiple types of ADB, but for our purpose,
1) Manual Deployment to OCI DB System: we chose to use Oracle Autonomous Transaction
One approach is the manual deployment of Oracle Processing (ATP) database [2], [9], [10].
APEX applications to an Oracle Database. In this To effectively deploy Oracle APEX applications, it's
scenario, administrators take on the responsibility of essential to establish a reliable and secure database
installing Oracle APEX and managing all aspects of the environment. ATP database is a well-suited choice for
database and application stack. This includes hosting Oracle APEX applications, offering a high-
performance tuning, security patching and backup, and performance, self-tuning, and fully managed database
recovery planning. While this option provides a high service. Here is a step-by-step guide to creating an ATP
degree of control over the deployment process, it also database and connecting to it using a Cloud Wallet [2],
requires significant administrative effort and expertise [9], [10]:
[5]. 1. Provisioning an Oracle ATP Database:
Administrators begin by setting up the Oracle Database x Access the Oracle Cloud Console and
and installing the Oracle APEX framework. This involves navigate to the Oracle Database section.
configuring the database parameters, creating user
accounts, and establishing the necessary network and x Create a new Autonomous Transaction
security settings. Once the foundation is in place, Processing database instance, specifying the
developers can design and build Oracle APEX required configuration, storage, and network
applications using the Oracle APEX development settings.
environment [5]. x Define the administrative user credentials.
While manual deployment offers flexibility and 2. Generating and Downloading a Cloud Wallet:
customization, it may require a substantial investment of x Once the ATP database is provisioned, it is
time and resources in ongoing maintenance and possible to generate a Cloud Wallet, which
management. Administrators must stay vigilant in contains the necessary security credentials
monitoring and optimizing the database's performance, and certificates for secure communication.
ensuring security updates are applied, and planning for x Download the cloud wallet files, including
disaster recovery scenarios. This approach is suitable for the wallet ZIP archive and associated files.
organizations with dedicated IT teams and a strong need
for control over the deployment environment [5]. 3. Connecting to ATP Database Using Cloud
Wallet:
2) Deployment on Oracle Autonomous Database:
x Unzip the downloaded Cloud Wallet archive
Another approach is to opt for the streamlined
to a secure location on local system.
deployment offered by Oracle Autonomous Database
(ADB). ADB provides a managed platform that includes x Configure development environment for
the Oracle APEX runtime, eliminating the need for working with SQL (Oracle SQL Developer
manual installation and configuration. This approach is highly recommended) to use the cloud
significantly simplifies the deployment process and wallet for database connections, specifying
reduces administrative overhead [8], [9], [10]. the appropriate wallet directory and
In an ADB deployment, Oracle APEX applications are connection details.
created and developed using the familiar Oracle APEX x Test the connection to ensure successful
development environment. Once the application is ready, communication between development
it can be deployed directly to the ADB instance. The environment and the ATP database.
ADB environment takes care of automatically managing Connecting to ATP database using a cloud wallet
and optimizing the underlying database infrastructure [8], enhances security by encrypting communication and
[9], [10]. protecting sensitive data. It is also possible to access ATP
ADB leverages autonomous features such as self- database directly in the web browser through OCI [2],
tuning, automated patching, and data protection. This not [9], [10].
only ensures efficient application performance but also
enhances security and reliability. Organizations can C. Learning Resources for Oracle APEX and Oracle
benefit from reduced administrative complexity and focus Cloud
more on application development and user experience Embarking on journey with Oracle APEX and Oracle
[8], [9], [10]. Cloud involves leveraging a variety of learning resources
The choice between manual deployment and ADB to enhance development skills and knowledge. These
deployment depends on factors such as the organization's resources can provide valuable insights and guidance:
technical expertise, resource availability, and desired

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 396

 1) Official Oracle Cloud Documentation: out the application's layout, navigation flow, and key
The Oracle Cloud documentation offers features. Identify the data sources, tables, and views that
comprehensive guides, tutorials, and reference materials will support the application's data needs. This phase sets
on various cloud services, including Oracle Autonomous the foundation for the entire development process and
databases. Explore the documentation to understand helps streamline subsequent steps [1], [2], [3], [4], [5],
deployment options, configuration steps, security best [6].
practices, and more [8]. A well-designed data model is paramount due to the
2) Oracle APEX Documentation: data-centric behaviour of Oracle APEX. Develop a
Delve into the official Oracle APEX documentation to logical representation of data entities, relationships, and
gain a deep understanding of the development attributes. Design the database schema that will
environment, components, and features. Learn how to efficiently store and manage the application's data. This
design interactive reports, create data visualizations, and step ensures data integrity, retrieval, and manipulation
deploy applications. The documentation also provides throughout the application's lifecycle [1], [2], [3], [4], [5],
insights into security considerations and integration with [6].
Oracle databases [1]. Leverage Oracle APEX's intuitive interface to build
3) Universal Theme Documentation: application. Use the App Builder to create pages, forms,
Universal Theme is a versatile and responsive theme reports, and interactive components. Select suitable
for Oracle APEX applications. Explore its documentation templates and themes to ensure a consistent and visually
to discover design principles, layout options, and appealing user experience. Configure regions and items
customization techniques. Learn how to create modern, to display data and functionalities based on the data
user-friendly, and responsive interfaces for applications model and user requirements. Implement validations,
[11]. calculations, and custom logic to enhance the
application's functionality [1], [2], [3], [4], [5], [6].
4) Oracle Academy:
Thorough testing is crucial to ensure application
Oracle Academy offers educational resources, courses, functions as intended and offers a seamless user
and certifications to help master Oracle technologies. experience. Perform unit testing on individual pages,
Explore their offerings to enhance Oracle APEX and
components, and functionalities to identify and rectify
cloud skills, earning valuable credentials along the way.
errors or inconsistencies. Address issues, fine-tune user
Part of the Oracle Academy is always free access to
interfaces, and refine performance. Rigorous testing
various OCI services including Oracle Autonomous
guarantees a reliable and user-friendly application [1],
Database and Oracle APEX [12].
[2], [3], [4], [5], [6].
D. Conclusion Once the application passes testing, deploy it to a
Deploying Oracle APEX applications involves making production environment. Choose a suitable deployment
informed decisions about deployment strategies, database option, whether on-premises or in the cloud, and ensure
provisioning, and learning resources. Whether is the necessary infrastructure and resources are in place.
choice manual deployment to an Oracle Database or the Perform a final round of testing in the production
streamlined capabilities of Oracle Autonomous Database, environment to verify smooth operation. Train end-users
building applications in Oracle APEX begins with on functionalities and provide documentation [1], [2], [3],
understanding these fundamental aspects. Coupled with a [4], [5], [6].
commitment to continuous learning from official B. Oracle APEX Regions
documentation and educational platforms, developer will
be well-equipped to leverage the full potential of Oracle In Oracle APEX, regions are essential components that
APEX and Oracle Cloud in application development play distinct roles in creating dynamic and engaging web
endeavours. applications. These regions collectively enhance user
experience and streamline data analysis by presenting
III. DESIGNING APPLICATIONS IN ORACLE APEX information in various formats and enabling interactive
functionality. Let's explore the roles and benefits of
Designing applications in Oracle APEX is a creative different groups of regions [11]:
process that combines functionality, user experience, and
visual appeal. This chapter explores key concepts and 1. Data Presentation:
techniques for creating dynamic and engaging x Classic Report: Classic reports present data
applications using powerful features of Oracle APEX. in a tabular format, allowing users to view
information in a structured manner.
A. Process of Creating an Application in Oracle APEX x Interactive Report: Interactive reports
Creating an application in Oracle APEX is a structured provide users with interactive features like
process that ensures the development of user-friendly and sorting, filtering, and customization, making
functional web applications tailored to specific needs. data exploration efficient and intuitive.
This systematic approach encompasses various stages, x Chart: Chart regions visually represent data
from initial design and data modelling to final testing and trends and patterns, offering insights
deployment [1], [2], [3], [4], [5], [6]. through different chart types such as bar
The journey begins with a clear understanding of the charts, line graphs, and pie charts.
application's purpose, functionality, and user x Map: Map regions display geographical
requirements. Collaborate with stakeholders, gather data, enabling users to explore spatial
insights, and define the scope of the application. Sketch relationships and locations.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 397

 x List: List regions offer a simple way to catering to different data presentation, interaction, and
showcase data items or summaries in a navigation needs. By strategically incorporating these
concise list format. regions, developers can create comprehensive and user-
x Cards: Card regions present data in visually centric applications that empower users to effectively
engaging cards, suitable for displaying analyse, manipulate, and interact with data. The selection
profiles, products, or summaries. and arrangement of regions should align with the
application's objectives and user requirements, ensuring a
2. Data Entry and Interaction: seamless and engaging user experience [11].

x Form: Form regions facilitate data entry and C. Oracle APEX Items
modification by providing a structured Oracle APEX offers a versatile array of items, each
layout for capturing and editing serving a distinct purpose to enhance the functionality
information. and interactivity of web applications. These items provide
x Interactive Grid: Interactive grids empower a wide range of input and display options, ensuring an
users to interact with data, enabling sorting, intuitive and engaging user experience. By strategically
filtering, and inline editing for efficient data leveraging these items, developers can create intuitive
manipulation. and dynamic interfaces that facilitate data capture,
x Calendar: Calendar regions allow users to manipulation, and interaction. The selection of items
manage events, tasks, and schedules should align with the application's objectives and user
interactively, enhancing applications requirements [1].
involving time-sensitive activities. D. Understanding When Oracle APEX May Not Be
3. Navigation and Organization: Suitable
x Breadcrumb: Breadcrumb regions provide While Oracle APEX offers a versatile platform for web
users with clear navigation paths, aiding in application development, there are certain scenarios
orientation and helping users understand where other tools or approaches might be more suitable
their location within the application. [1], [2], [3], [4], [5], [6]:
x Region Display Selector: Region display 1. Oracle APEX excels at creating data-driven
selector regions allow users to switch applications, but extremely complex applications
between different views or layouts within with intricate business logic might benefit from
the application, enhancing navigation and more specialized development environments.
usability.
2. If application requires highly customized user
x Tree: Tree regions present hierarchical data interfaces with complex animations,
structures, enabling users to navigate and interactions, or non-standard designs, a more
explore data with parent-child relationships. extensive web development framework might be
4. Customization and Integration: more appropriate.
x Static Content: Static content regions 3. For large-scale applications with massive user
display fixed text or HTML content, bases and complex data processing needs, other
suitable for providing information that enterprise-grade application development
doesn't change frequently. frameworks may provide more comprehensive
x Dynamic Content: Dynamic content regions scalability and performance options.
enable embedding custom HTML, 4. Oracle APEX is tightly integrated with Oracle
JavaScript, or third-party content into the Database, so if application requires integration
application, allowing for versatile with non-Oracle data sources or external
customization and integration. systems, other development platforms might be
x URL: URL regions embed external web more suitable.
content or URLs within the application,
facilitating seamless integration with E. Conclusion
external resources. Designing applications in Oracle APEX involves a
5. Advanced Interaction and Exploration: structured approach encompassing application creation,
x Faceted Search: Faceted search regions provide region utilization, item selection, and assessing
users with advanced filtering options, suitability. By following best practices, leveraging
enabling efficient data exploration based on APEX's capabilities, and considering potential
specific criteria. limitations, developers can create impactful web
applications that deliver exceptional user experiences and
x Search: Search regions allow users to quickly meet diverse business needs.
locate specific data or records within the
application, enhancing data accessibility. IV. ANALYTICAL TOOL FOR UNIVERSITY DATA
x Smart Filters: Smart filter regions offer dynamic MANAGEMENT
filtering options that adjust based on user
Analytical Tool for University Data Management is an
interactions, allowing users to explore data
application that was developed as part of the bachelor's
intuitively.
thesis. The main goal has been the development of an
Each group of regions contributes to the overall analytical tool for the university, aiming to facilitate and
functionality and usability of an APEX application, enhance the analysis of processes occurring within the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 398

educational environment. The motivation behind this x erasmus (Erasmus Program): Stores information
topic lies in the pursuit of improving the quality and about a student's participation in the Erasmus+
comprehension of the teaching process while program, including the semester, school, and
modernizing and expanding existing systems, not only faculty.
within our university but also in academia at large [7]. x erasmus_skusky (Erasmus Exams): Contains
There is high amount of academic information information about exams taken during Erasmus+
systems, yet there is lack of effective analytical tools for program participation.
comprehensive study processes analysis. Despite x zap_predmety (Enrolled Subjects): Stores
platforms like vzdelavanie.uniza.sk serving conventional information about enrolled subjects, including
management needs, their analytical capacities remain results, credits, teachers, and schedule.
limited [7].
x predmet (Subject): Contains subject codes and
Crafting a potent analytical tool necessitates an
names.
intricate grasp of university workflows. Our investigation
was anchored in dissecting student trajectories at Žilina x ucitel (Teacher): Stores information about
University's Faculty of Management Science and teachers, departments, positions, employment, and
Informatics. Emphasizing disparities between study termination dates.
levels, teaching dynamics, subject enrolment, and exam x predmet_bod (Subject Credits): Contains
recording, especially within Erasmus+ and domestic information about subject credits and responsible
study contexts, we cultivated insights to shape our teachers.
solution [7]. x skuska (Exam): Holds information about subject
Leveraging Oracle Cloud's autonomous features, we exams, including maximum and passing scores.
proposed a self-managing database, underpinned by
x zapocet (Enrolment): Stores information about
Cloud Wallet for connectivity and Oracle SQL for
subject enrolments and passing scores.
language integration. This foundation facilitated the
development of an Oracle APEX-based tool, equipping x skuska_stud (Student Exam): Contains
stakeholders to track students, exams, teachers, and information about student exam attempts, scores,
subjects. Enhanced by search, sort, and filter functions, and grades.
users navigate data by pivotal criteria, and detailed x zapocet_stud (Student Enrolment): Holds
statistics illuminate teacher, student, and group trends, information about student enrolments and bonus
accessible via personalized accounts [7]. scores.
A. E-R Data Model x predmet_znamkovanie (Subject Grading Scale):
Contains information about subject grading scales
The data model serves as the foundation upon which by academic year.
the application is built, enabling efficient storage,
retrieval, and manipulation of data related to students, x st_program (Study Program): Stores information
teachers, subjects, exams, and various aspects of about subject’s requirements, recommended
university management [7]. academic year, and semester with relation to study
program.
List of all entities used in the application with English
names in brackets [7]: x st_odbory (Study Programs): Contains information
about study programs and their branches.
x stat (Country): Represents information about a
country. x st_skupina (Study Group): Represents study
groups.
x kraj (Region): Represents information about a
region. x prislusnost_st_skupina (Assignment to Study
Groups): Represents the assignment of students to
x okres (District): Represents information about a study groups.
district.
x obec (Municipality): Represents information about B. Application Pages
a municipality. The application consists of four main pages, that can
x bydlisko (Residence): Represents current and past be navigated using navigation bar. Those pages are
permanent residences of individuals from the “Studenti” (Students), “Učitelia” (Teachers), “Študijné
Personal Data entity. skupiny” (Study Groups) and “Predmety” (Subjects) [7].
x os_udaje (Personal Data): Contains personal data 1) Page “Students”
of students and potential students. The "Students" page, as seen on Fig. 1, provides a
x student (Student): Represents a student and comprehensive platform for viewing and managing
information about their degree and study status. students within the application. It features two tabs:
Includes enrolment and graduation dates. "Students" and "Analysis." Under the "Students" tab, an
overview of students categorized by study level is
x studium (Study): Provides information about each presented. The presentation of students and individuals is
academic year of a student, including Erasmus+ visually organized with sortable and filterable cards using
program participation. Cards region. Additionally, this menu facilitates bulk
x typ_uzavretia_roc (Study Completion Type): subject enrolment, enabling the modification of subjects
Contains types of study completion and their and academic years [7].
descriptions.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 399

 Figure 2. Page "Subjects"
Figure 1. Page "Students"
4) Page “Study Groups”
Direct student management is facilitated through the The "Study Groups" page presents a list of study
"Students" page, where administrators can add, delete, groups along with the corresponding number of students.
and edit student records. Students can also be added in This page allows for sorting and filtering of study groups.
bulk. Furthermore, the "Student - Detail" page offers The study groups are organized based on the study level.
insights into a student's study information, personal Each study group provides the option to view students
details, and enrolled subjects by semester. Administrators who were enrolled in that group during specific
possess the ability to manage a student's study, including semesters, allowing for detailed tracking of student
adding and removing study years, primarily within the enrolment and movement [7].
current study year. They can also close a student's study,
allowing enrolment in a higher study level. Erasmus+ C. Analytical tools within the application
participation is an option, and administrators can input The application provides an array of powerful
related details [7]. analytical tools, enabling users to access and explore
2) Page “Teachers” diverse and valuable statistics. These statistics encompass
Upon opening the "Teachers" page, a layout like the not only individual student progress and academic
"Students" page is displayed. Teachers are presented in journey but also insights into teachers, distinct study
card format also utilizing Cards region, allowing for easy groups, and subjects. The core analytical tools employed
search, and filtering. Clicking on any teacher's name within the application include data sorting, filtering
reveals detailed information about the teacher, including capabilities, and graphical representation [7].
the subjects they teach and lecture, along with the number 1) Sorting
of students that they teach and lecture. Teachers and Sorting plays a pivotal role in managing vast datasets
administrators can also access specific subject details, within any application. In our implementation, users can
viewing lists of students taught, along with their student effortlessly sort various regions, ranging from tables to
identification numbers, study levels, academic years, and cards and even charts. This dynamic sorting feature
performance results [7]. extends to multiple criteria, allowing users to arrange data
3) Page “Subjects” in ascending or descending order. This functionality
The "Subjects" page, as seen on Fig. 2, offers an offers users an intuitive means to navigate and interpret
overview of all subjects available for student enrolment information effectively [7].
in different academic years. To access the list of subjects, 2) Filtering
one must first choose the study level, study program, and The application harnesses Oracle APEX's filtering
recommended year for enrolment of the subject. capabilities, either by direct integration into regions or
Subsequently, a specific subject can be selected. Upon through collaborative interactions between two
choosing a subject, a page displays the subject's grading compatible regions. One of these regions must be
scale, student performance, and a table containing designated as a Search, Smart Filters, or Faceted Search
enrolled students for the chosen subject [7]. region, with the second region serving as a target for
filtering. Our application predominantly employs the
Smart Filters region for filtering, enabling users to refine
results based on pre-defined criteria [7].
3) Graphical Data Representation and Statistics:
Graphs serve as essential analytical and statistical
tools, offering insightful visual representations of data
trends. An example of a graph within the application is
the "Average Enrolled Subjects by Study Group" graph,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 400

as shown in Fig. 3. This graph is located on the study SELECT st_skupina, AVG(pocet) priemer
group page under the "Analysis" section. It depicts the FROM (
average number of enrolled subjects across all students SELECT st_skupina, os_cislo, COUNT(DISTINCT
within a particular study group. The graph offers insights kod_predm) pocet
into the academic load of students in that study group. FROM zap_predmety join prislusnost_st_skupina using
Users can sort this graph based on the study group's name (os_cislo, sk_rok)
or the average number of enrolled subjects. Additionally, WHERE sk_rok = :P8_ROK
filtering options allow users to focus the graph based on GROUP BY st_skupina, os_cislo
the study level and study program. In Fig. 4, it is possible ) join st_skupina using(st_skupina)
to see SQL query, that is used to generate this graph [7]. WHERE st_program =
Within the Oracle APEX environment, graphs are nvl(:P8_STUDIJNY_PROGRAM_POCET_ZAP_PREDM
seamlessly integrated into pages through the Chart , st_program) and
region. This versatile region can be extensively ((:P8_STUPEN_STUDIA_POCET_ZAP_PREDM = 1
customized, allowing users to select graph types, colour and st_program in ('YI', 'YP', 'YS', 'YR', 'MN', 'ME')) or
schemes, axis labels, ranges, steps, and displayed values. (:P8_STUPEN_STUDIA_POCET_ZAP_PREDM = 2
Graph data selection is facilitated through SQL queries, and st_program in ('EM', 'EI', 'IB', 'IS', 'IF', 'II', 'IN')) or
enabling precise specification of the data columns to be :P8_STUPEN_STUDIA_POCET_ZAP_PREDM is
represented on the x and y axes. Furthermore, these NULL) and
graphs can be filtered and sorted to provide a substr(st_skupina, 5, 1) =
comprehensive understanding of the data [7]. nvl(:P8_ROCNIK_POCET_ZAP_PREDM,
substr(st_skupina, 5, 1))
In our application, an assortment of graphs captures GROUP BY st_skupina
and illustrates statistics concerning students, teachers, and
study groups. Future development can easily expand the
Figure 4. SQL query for generating "Average Enrolled Subjects by
scope of these graphs to include additional entities as Study Group" graph
needed. By leveraging these analytical tools, users gain
enhanced insights into the intricate dynamics of
university processes, fostering informed decision-making V. CONCLUSION
and strategic planning [7]. The ever-evolving landscape of higher education is
undergoing a transformative shift through the utilization
of data analysis and web application development. The
intricate analysis of university data has emerged as a
catalyst for informed decision-making, administrative
optimization, and enhanced student experiences. In this
dynamic environment, Oracle APEX has emerged as a
powerful platform bridging the gap between traditional
desktop development and modern web application design,
offering universities a gateway to innovative and efficient
data-driven solutions.
The vast reservoir of data generated throughout a
student's academic journey holds the potential to reshape
the educational landscape. Through data analysis,
institutions can gain comprehensive insights into student
trends, enabling them to tailor strategies that enhance
learning outcomes and refine institutional policies. This
Figure 3. "Average Enrolled Subjects by Study Group" graph data-driven approach serves as a guiding light for
universities, steering them toward evidence-based
decisions that drive continual improvement.
The advent of web applications has revolutionized data
access and user interaction, enabling universities to
transcend geographical limitations and provide intuitive
interfaces for data analysis. Oracle APEX, as a cutting-
edge low-code development platform, empowers
developers to create powerful, scalable, and responsive
web applications that seamlessly integrate with Oracle
databases. Its integration with Oracle Cloud ensures
heightened accessibility, security, and scalability, making
it an ideal choice for academic institutions seeking to
streamline administrative functions and expedite the
deployment of data analysis tools.
The application Analytical Tool for University Data
Management was implemented using the Oracle APEX
environment based on the proposed design. The current
version of the application offers tools for managing
students, teachers, subjects, and study groups. Students

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 401

can record their studies, results, and evaluations,
including those related to the ERASMUS+ program, and
subsequently sort, filter, and analyse them using graphs.
Similarly, teachers and their students, as well as the
subjects they teach and lecture, can be displayed.
Teachers can also be sorted, filtered, and statistically
represented. Subjects and study groups can be added and
removed, and students associated with these subjects and REFERENCES
study groups can be displayed. Statistics related to [1] Oracle APEX Documentation, Web:
students within study groups can also be graphically https://apex.oracle.com/en/learn/documentation/.
represented. [2] K. Matiaško, M. Kvet, V. Vestenický and V. Šalgová, Rýchly
vývoj dátových modelov a aplikácií v prostredí Oracle APEX,
The application holds great potential for the future. Žilina: EDIS, 2020.
Integration of examination systems for individual [3] A. Geller and B. Spendolini, Oracle Application Express (APEX):
subjects could be incorporated, and analytical tools Build Powerful Data-Centric Web Apps with APEX, New York:
utilizing machine learning techniques could predict McGraw Hill, 2017.
student study progress or identify students with similar [4] A. Nujiten, I. Ellen-Wollf, L. Brizzi, Oracle APEX Best Practices,
patterns. Manual student comparisons could also be Birmingham: Packt Publishing, 2012.
integrated, and certain processes within the application [5] A. Png and H. Helskyaho, Extending Oracle Application Express
could be further automated. The system could potentially with Oracle Cloud Features, New York: Apress, 2022.
warn students about similarities in study patterns with [6] E. Sciore, Understanding Oracle APEX 20 Application
others who failed or alert them if they register for too Development, New York: Apress, 2020.
many or too few subjects, insufficient for credit [7] I. Pastierik, Analytical Tool for University Data Management,
requirements. The system could also recommend subjects Žilina: Faculty of Management Science and Informatics –
University of Žilina, 2023.
based on similarities in results with other students,
[8] Oracle Cloud Infrastructure Documentation, Web:
enhancing overall study experiences. https://docs.oracle.com/en-us/iaas/Content/home.htm.
[9] M. Kvet, K. Matiaško and Š. Toth, Practical SQL for Oracle
ACKNOWLEDGMENT Cloud, Žilina: EDIS, 2020.
It was supported by the Erasmus+ project: [10] P. Sakar, Oracle Cloud Infrastructure for Solutions Architects,
Better Employability for Everyone with APEX (2021-1-SI01- Birmingham: Packt Publishing, 2021.
KA220-HED-000032218). [11] Oracle APEX Universal Theme, Web:
https://apex.oracle.com/pls/apex/r/apex_pm/ut/getting-started.
[12] Oracle Academy, Web: https://academy.oracle.com/en/oa-web-
overview.html.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 402

 (PHUJLQJ7HFKQRORJLHVLQ(GXFDWLRQ6HFXULW\
DQG'DWD3URWHFWLRQ

33HNDUFLN (&KRYDQFRYD DQG7.XFKFDNRYD 
'HSDUWPHQWRI&RPSXWHUVDQG,QIRUPDWLFV
)DFXOW\RI(OHFWULFDO(QJLQHHULQJDQG,QIRUPDWLFV7HFKQLFDO)DFXOW\RI(OHFWULFDO(QJLQHHULQJDQG,QIRUPDWLFV
7HFKQLFDO
8QLYHUVLW\RI.RãLFH.RãLFH6ORYDNLD
SHWHUSHNDUFLN#WXNHVNHYDFKRYDQFRYD#WXNHVNWDWLDQDNXFKFDNRYD#WXNHVN


$EVWUDFW²,QIRUPDWLRQ WHFKQRORJLHV DUH RQH RI WKH PRVW VROXWLRQV7KLVFKRLFHLVDOULJKWEHFDXVHWKDWVROXWLRQVDUH
SURVSHFWLYH ILHOGV DW XQLYHUVLWLHV 7KH QXPEHU RI \RXQJ IUHTXHQWO\ XVHG LQ SUDFWLFH %XW KRZ PDQ\ RI WKHVH
SHRSOH LQWHUHVWHG LQ WKLV VFLHQFH LV LQFUHDVLQJ IURP \HDU WR VROXWLRQV H[LVW" 6HYHQ HLJKW OHVV RU PRUH" :H DOVR
\HDU 2QH RI WKH HVVHQWLDO SDUWV RI ,7 LV VHFXULW\ DQG GDWD H[SHFW PRVW VWXGHQWV WR FKRRVH WKH PRVW SRSXODU
SURWHFWLRQ 0DQ\ VWXGHQWV SHUFHLYH WKLV LQGXVWU\ DV DOJRULWKP56$,QWKHLGHDOFDVHWKHVWXGHQWDVWKHILUVW
FRPSOLFDWHG DQG WKHUHIRUH XQDWWUDFWLYH ,I ZH ZDQW WR VWHS VKRXOG VWXG\ WKH DOJRULWKP LQ GHWDLO IURP WKH
FKDQJH LW ZH PXVW IRFXV RQ LQWURGXFLQJ HPHUJLQJ OLWHUDWXUH DQG LPSOHPHQW LW LQ WKHLU ZD\ +RZHYHU WKLV
WHFKQRORJLHV LQ HGXFDWLRQ 7KHVH WHFKQRORJLHV VKRXOG EH PHWKRGLVOHQJWK\DQGPRVWVWXGHQWVGRQ¶WZDQWWRZRUN
FKRVHQ VR WKDW WKH\ PRWLYDWH WKH VWXGHQWV EXW DW WKH VDPH RQWKHLVVXHVIRUVRORQJ
WLPHVRWKDWWKHOHYHORIDFTXLUHGNQRZOHGJHGRHVQRWGURS 0RVW VWXGHQWV XVH LQ WKHLU VROXWLRQV IRU VLPSOH
7KHLUJRDOLVWRPDNHVWXG\LQJWKLVILHOGPRUHDWWUDFWLYHDQG UHIDFWRULQJ SURYLGHG E\ DQ LQWHJUDWHG GHYHORSLQJ
HDVLHU IRU VWXGHQWV 7KH DUWLFOH SUHVHQWV H[DPSOHV RI QHZ HQYLURQPHQW 6RPHWLPHV WKH\ FKDQJH WKH W\SH RI
DSSURDFKHV WR VHFXULW\ DQG GDWD SURWHFWLRQ 7KLV DUWLFOH YDULDEOHV RU ORRS W\SH DQG WKLV LV DOO %XW ZKDW ZDV WKH
LQWURGXFHVVHYHUDOVWDQGDUGVDQGFRQFUHWHWRROVUHTXLUHGIRU \LHOGYDOXHZKHQFUHDWLQJVXFKDVROXWLRQ"
WKHP
$W WKH EHJLQQLQJ RI WKH ILHOG RI ,7 DQG LWV VWXG\ WKH
, ,1752'8&7,21 JRDO ZDV WR SURGXFH VWXGHQWV ZKR FRXOG FUHDWH HQWLUHO\
QHZ VROXWLRQV %XW WRGD\ ZH GHYHORS VWXGHQWV ZKR FDQ
'DLO\ GLIIHUHQW QHZV DQG DUWLFOHV >@ RU >@ LQIRUP XV RQO\FRPELQHRUPRGLI\H[LVWLQJVROXWLRQV'RHVLWPHDQ
DERXWWKHODFNRIVNLOOHGODERULQWKH,7ILHOG$QDEVHQFH WKDW HYHU\WKLQJ WKDW VKRXOG KDYH EHHQ FRPSOHWHG DOUHDG\
ZLOOFRQVWUDLQIXWXUHJURZWKDQGMREFUHDWLRQLQWKHVHFWRU KDV EHHQ PDGH LQ VHFXULW\ DQG GDWD SURWHFWLRQ" 6R PXVW
7KH ,7 VHFWRU QHHGV PRUH VNLOOHG ,&7 ZRUNHUV LQ PDQ\ ZHEHVDWLVILHGWKDWXQLYHUVLW\JUDGXDWHVZLOOPDVWHU\HDUV
FRXQWULHV 6DODULHV LQ WKLV ILHOGDUH KLJKO\ DERYHDYHUDJH ROG VROXWLRQV" 2U VKRXOG ZH FKDQJH KRZ LQIRUPDWLRQ
EXWGHVSLWHWKLVIDFWFRPSDQLHVQHHGWREHDEOHWRDWWUDFW VHFXULW\LVWDXJKWVRWKDWZHUHSURGXFHHYHQLIRQO\DIHZ
PRUHTXDOLILHGZRUNHUV7KHPDLQUHDVRQLVWKDWJUDGXDWHV VWXGHQWV EXW WKRVH ZKR ZLOO EH DEOH WR SURGXFH QHZ
RI KLJK VFKRROV WKLQN WKDW ,7 LV FKDOODQJLQJ DQG UHTXLUHV LQQRYDWLYHLGHDV"
DERYHDYHUDJHORJLFDOWKLQNLQJ$FFRUGLQJWR>@6ORYDNLD ,Q FRQWUDVW WR 6ORYDN XQLYHUVLWLHV WKH EHVW WHFKQLFDO
LVQRH[FHSWLRQLQWKLVFDVH XQLYHUVLWLHVLQWKHZRUOGSURYLGHVHYHUDOGLIIHUHQWFRXUVHV
2QH RI WKH HVVHQWLDO SDUWV RI WKH VWXG\ RI LQIRUPDWLRQ IRU,7VHFXULW\)RUH[DPSOHLQRQHRIWKHPRVWSRSXODU,7
WHFKQRORJLHVLVWKHVWXG\RI,7VHFXULW\$FFRUGLQJWR>@ XQLYHUVLWLHV 0DVVDFKXVHWWV ,QVWLWXWH RI 7HFKQRORJ\ \RX
GDWDVHFXULW\LVYLWDOIRUVWXGHQWVEHFDXVHLWLVHVVHQWLDOLQ FDQJUDGXDWHLQWKHVH
HYHU\ FRPSXWHU V\VWHP 8QLYHUVLWLHV  FRXUVHV IRFXV RQ
• ,QWURGXFWLRQWR&\EHUVHFXULW\
XQGHUVWDQGLQJ DQG LPSOHPHQWLQJ H[LVWLQJ GDWD SURWHFWLRQ
PHWKRGV ([DPSOHV RI WKHVH PHWKRGV DUH '(6 56$ RU • ,QWURGXFWLRQWR(WKLFDO+DFNLQJ
3*3 DOJRULWKP 7KH UHVXOW RI WKHVH FRXUVHV LV WKDW • 1HWZRUN6HFXULW\
JUDGXDWHVNQRZDERXWWKHPRVWXVHGDOJRULWKPVZKHQWKH\ • )LUHZDOOVDQG931V
VKRXOG XVH WKHP DQG KRZ WR XVH WKHP FRUUHFWO\ %XW
VWUDLJKW XS WKH LPSOHPHQWDWLRQ RI ZHOONQRZQ VHFXULW\ • 6HFXUH3URJUDPPLQJ
DOJRULWKPVFDQEHHDVLO\FRSLHGIURPUHSRVLWRULHVOLNH*LW 7KHILYHFRXUVHVDUHOLQNHGWRHDFKRWKHUEXWWKH\FDQ
+XERU*LWODEDQGPRGHUQSURJUDPPLQJODQJXDJHVRIIHU DOVREHFRPSOHWHGVHSDUDWHO\3RSXODUDUHDOVRFRXUVHVRI
SOHQW\ RI OLEUDULHV IRU LWV LPSOHPHQWDWLRQ 6WXGHQWV DIWHU DXWKHQWLFDWLRQ RU GDWD SURWHFWLRQ DQG SULYDF\ %XW ZKHQ
WKDW FDQ SHUVRQDWH WKHVH VROXWLRQV DV WKHLU RZQ 7KH ZH ZHUH VWXGHQWV DW WKH 7HFKQLFDO 8QLYHUVLW\ RI .RVLFH
SRVVLELOLWLHV RI LPSOHPHQWDWLRQV FKDQJHV DUH OLPLWHG DQG ZH SDVVHG RQO\ RQH FRXUVH LQ LQIRUPDWLRQ VHFXULW\ DW D
WKH QXPEHU RI VWXGHQWV LQ ,7 ILHOGV LV YHU\ KLJK (YHQ EDFKHORU¶VGHJUHH
VNLOOHGVWXGHQWVRIWHQFDQ WFUHDWHWKHRULJLQDOVROXWLRQIRU • 6HFXULW\LQFRPSXWHUV\VWHPV
LPSOHPHQWLQJ WKHVH DOJRULWKPV 7KDQNIXOO\ QRZDGD\V
WHDFKHUVXVHWKHFRQWURORIRULJLQDOLW\RIVROXWLRQV%XWLI DQGRQHDWWKHPDVWHU¶VGHJUHH
DVWXGHQWKDVWRLPSOHPHQWVRPHH[LVWLQJDOJRULWKPKRZ • 6HFXULW\ RI LQIRUPDWLRQ DQG FRPPXQLFDWLRQ
PDQ\SRVVLELOLWLHVRIFKDQJHVFDQWKH\PDNH" V\VWHPV
$WWKHILUVWKHFDQFKRRVHIURPGLIIHUHQWDOJRULWKPV:H 0RUHRYHUWKH6HFXULW\RI&RPSXWHU6\VWHPVFRXUVHDW
H[SHFW WKDW VWXGHQWV ZLOO FKRRVH IURP WKH PRVW SRSXODU WKH7HFKQLFDO8QLYHUVLW\RI.RVLFHUHTXLUHVNQRZOHGJHRI

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 403

WKH SURJUDPPLQJ FRQFHSWV VWXGHQWV DFTXLUHG LQ WKH FKDQJHV ZLWKLQ WKH V\VWHP RYHU WLPH LQFOXGLQJ
SUHYLRXVWZR\HDUVRIVWXG\+RZHYHUWKLVGRHVQRWPHDQ WKHLQIOXHQFHRIVRFLDODFWRUV
WKDW QRQ,7 VWXGHQWV RU VWXGHQWV ZLWKRXW SURJUDPPLQJ
NQRZOHGJH FDQQRW XQGHUVWDQG WKH EDVLF FRQFHSW ,I WKH ,,, 3+<6,&$/02'(/6
EDVLF WKHRU\ LV H[SODLQHG WR WKHVH VWXGHQWV LQ D VXLWDEOH
3K\VLFDOPRGHOVDUHWKHPRVWRIWHQXVHGLQWKHVWXG\RI
PHWKRGWKH\FDQDOVRDFTXLUHNQRZOHGJHLQ,7VHFXULW\
QHWZRUN VHFXULW\ ZKHUH VWXGHQWV VKRXOG XQGHUVWDQG WKH
,QWKHFDVHRIQRQ,7VWXGHQWVWKH\DUHRQO\VXLWDEOHIRU SK\VLFDOFRQVWUXFWLRQRIWKHQHWZRUN7KHSUREOHPLVWKDW
VRPHRIWKHPHQWLRQHGPRGHOV3K\VLFDOPRGHOVGHVFULEHG SURIHVVLRQDOQHWZRUNGHYLFHVOLNHVHUYHUVDUHFRVWO\$QG
LQ&KDSWHUUHTXLUHEDVLFFRQFHSWVRIZRUNZLWK$UGXLQR LWLVRQO\SRVVLEOHIRUVRPHVWXGHQWVWRKDYHVXFKDGHYLFH
RU 5DVSEHUU\ 3L 7KLV LV WKH UHDVRQ ZK\ WKH\ DUH QRW 7KHVROXWLRQLVXVLQJDYDLODEOHORZFRVWGHYLFHV
VXLWDEOHIRUWKHP(PXODWRUVDOVRUHTXLUHEDVLFNQRZOHGJH
6WXGHQWV FDQ FUHDWH SK\VLFDO PRGHOV IURP GLIIHUHQW
RI,7VHFXULW\EXWLQWKLVFDVHLWLVSRVVLEOHWRSUHVHQWWKLV
FRPSRVLWHV,QWKLVVWHSWKHVWXGHQW¶VLQGHSHQGHQWZRUNLV
NQRZOHGJH EHIRUH XVLQJ LW 2SSRVLWH WKHP VLPXODWLRQ
HVVHQWLDO EXW DW WKH VDPH WLPH WKH WHDFKHU¶V VXSHUYLVLRQ
PRGHOVDUHDEHWWHUFKRLFHDQGVHULRXVJDPHVDUHWKHEHVW
RYHUKLP,IVWXGHQWVJHWVWXFNLQVRPHSKDVHRIWKHZRUN
FKRLFH
WKH\ VKRXOG KDYH WKH RSSRUWXQLW\ WR JHW KHOS IURP WKH
:H KRSH WKDW WKH VLWXDWLRQ LV WRGD\ PXFK EHWWHU WHDFKHU 7KH WHDFKHU QHHGV WR JXLGH WKH VWXGHQW ZLWK
,QIRUPDWLRQ VHFXULW\ DQG GDWD SURWHFWLRQ LV D FXUUHQW DQG VXLWDEOH KLQWV DQG IRUFH KLP WR WKLQN DERXW WKH SUREOHP
UHTXHVWHG WRSLF UHTXLULQJ PXFK PRUH DWWHQWLRQ $QG QRW QRW SURYLGH D UHDG\PDGH VROXWLRQ $UWLFOH >@ LV
RQO\ DW XQLYHUVLWLHV EXW DW DOO OHYHOV RI HGXFDWLRQ %XW SUHVHQWHGXVLQJ5DVSEHUU\3L5DVSEHUU\3LLVDWLQ\ORZ
ZKLFK HPHUJLQJ WHFKQRORJLHV DUH EHVW IRU VWXG\LQJ FRVW SRZHUIXO GHYLFH EDVHG RQ D µV\VWHP RQ D FKLS¶²
VHFXULW\DQGGDWDSURWHFWLRQ" VHYHUDO JHQHUDWLRQV RI ERDUGV ZLWK VHYHUDO PRGHOV WKDW
FRYHUDZLGHUDQJHRIRSHQVRXUFHSURMHFWV5DVSEHUU\3L
,, 02'(/6)257($&+,1*,76(&85,7< FDQ UXQ VHYHUDO RSHUDWLQJ V\VWHPV RIIHUHG DV UHDG\PDGH
/HDGHUV RI FRXUVHV LQ ,7 VHFXULW\ VKRXOG SD\ PRUH LPDJHV IRU GRZQORDGLQJ ZLWK SUHLQVWDOOHG VRIWZDUH WKDW
DWWHQWLRQ WR WKHEHVW ZD\ DQG WRROV IRU WHDFKLQJ VWXGHQWV FRQYHUWV3LWRD3ULQW6HUYHU0HGLD6HUYHU*DPH6HUYHU
2IFRXUVHWKH\FDQFKRRVHDWHUPLQDORUDVLPSOH,'(DVD /$03 6HUYHUZLWK :RUG3UHV HWF ,I VWXGHQWV EHFRPH
WRROEXWWKLVDSSURDFKLVROGIDVKLRQHG0RUHRYHUQRQ,7 IDPLOLDU ZLWK WKH FRQQHFWLRQ RI WKH JLYHQ GHYLFH LW ZLOO
VWXGHQWV FDQ EH FKDOOHQJLQJ WR ZRUN ZLWK D WHUPLQDO RU KHOSWKHPEHWWHUXQGHUVWDQGLWVIXQFWLRQDOLW\$FFRUGLQJWR
QHZ ,'( FRPSOLFDWLQJ WKHLU XQGHUVWDQGLQJ RI HVVHQWLDO >@ 5DVSEHUU\ 3L RIIHUV VWXGHQWV D OLQN WR VHYHUDO
NQRZOHGJH8VLQJDWHUPLQDOUHTXLUHVNQRZOHGJHRIEDVLF FRQQHFWLQJGHYLFHVOLNH:L)L%OXHWRRWKPRGXOHIRUGDWD
FRPPDQGV'LIIHUHQWWHDFKHUVRIWHQXVHRWKHU,'(VLQWKH WUDQVIHU RU D FDPHUD RU PLFURSKRQH IRU GDWD FROOHFWLRQ
FDVH RI ,'( ZKLFK FDQ FRQIXVH VWXGHQWV 7KDQNV WR WKH GDWD IURP WKH UHDO ZRUOG 7KLV PDNHV WKHP LGHDO IRU
SURJUHVV LQ WKH ,7 ILHOG WKHUH DUH SOHQW\ RI YDU\LQJ LQWHUIDFLQJZLWKDZLGHDVVRUWPHQWRIRXWHUSHULSKHUDOV
VLPXODWLRQ HQYLURQPHQWV WRGD\ 7KHVH HQYLURQPHQWV DLP $UGXLQR 8QR LV WKH VHFRQG FKRLFH PRUH XVHG LQ WKH
WRVQHDNVWXGHQWVLQWRWKHSUDFWLFHDVPXFKDVSRVVLEOHDQG 7HFKQLFDO 8QLYHUVLW\ RI .RVLFH $UWLFOH >@ SUHVHQWV WKH
SUHYHQW UHVRXUFHV OLNH GHYLFHV GDWD RU VHUYLFHV ,Q WKH XVHRIWKLVGHYLFHIRUWKHVWXG\RIVHFXULW\V\VWHPVXVLQJ
FDVH RI GHYLFHV WKH\ VKRXOG EH GDPDJHG ,Q WKH FDVH RI VLPSOHKDUGZDUH/LNHWKH5DVSEHUU\3L$UGXLQRVXSSRUWV
GDWDWKHUHVKRXOGDOZD\VEHDFKDQFHWRUHFRYHUWKHPWR GLIIHUHQWVHQVRUVWRHQVXUHRXUVDIHW\7KHPRVWFRPPRQ
WKHRULJLQDOIRUP VHQVRUV DUH 3DVVLYH ,QIUDUHG 6HQVRUV 3,6V  8OWUDVRQLF
$FFRUGLQJWRWKHDUWLFOH>@WKHEHVWFKRLFHIRUVWXGHQWV ,QIUDUHG 3UR[LPLW\ DQG 1RLVH GHWHFWRUV 7KHVH VHQVRUV
WR VWXG\ VHFXULW\ ZH FXUUHQWO\ UHO\ RQ GLIIHUHQW W\SHV RI SUHVHQW DQ HOHJDQW DQG UREXVW VROXWLRQ IRU WKH LVVXH RI
PRGHOV UHPRWHVHFXULW\
• 3K\VLFDO PRGHOV DUH WKRVH ZKRVH SK\VLFDO $ SUDFWLFDO H[DPSOH RI XVLQJ $UGXLQR 812 LQ GDWD
FKDUDFWHULVWLFV UHVHPEOH WKH SK\VLFDO VHFXULW\ZKLFKFDQDOVREHXVHGIRUHGXFDWLRQDOSXUSRVHV
FKDUDFWHULVWLFV RI WKH PRGHOHG V\VWHP 6WXGHQWV LV SUHVHQWHG LQ >@ 7KH PDLQ DLP RI WKH H[DPSOH LV WR
FDQ XVH WKHP WR WHVW WKH HIIHFW RI DWWDFNV RU GHSOR\FU\SWRJUDSKLFIXQFWLRQVWRHQFU\SWGDWDEHLQJVHQW
HYDOXDWH SURWHFWLRQ PHDVXUHV ZLWKRXW DIIHFWLQJ E\ WKH ,R7 V\VWHP RYHU D :L)L QHWZRUN &U\SWRJUDSKLF
WKHUHDOV\VWHP DOJRULWKPV DUH XVHG &HDVHU &LSKHU 6+$  DQG $(6
• (PXODWRUV DUH SURJUDPV WKDW DFW OLNH D UHDO $OORIWKHPDUHDSSURSULDWHIRUHGXFDWLRQDOSXUSRVHV
GHYLFH DV SDUW RI D UHSUHVHQWDWLYH V\VWHP DQG 0RUHRYHUWKHVROXWLRQXVHGIRUWKHHQFU\SWLRQRIWKHGDWD
XVXDOO\ DUH UHDOL]HG DV VRIWZDUH (PXODWRUV PLFURFKLS $UGXLQR $IWHU WKDW HQFU\SWHG GDWD LV
SURYLGHVWXGHQWVJUHDWHUIOH[LELOLW\WKDQSK\VLFDO WUDQVIHUUHG E\ D ORZFRVW :L)L PLFURFKLS (63 ,Q
PRGHOVEHFDXVHWKH\DUHPRUHDFFHVVLEOHIDVWHU WKLV FDVH WKH VROXWLRQ FRQVLVWV RI LPSOHPHQWLQJ &DHVDU
FKHDSHU DQG PRUH FRVWHIIHFWLYH IRU PDNLQJ &LSKHU 6+$ DQG ELWV YHUVLRQ RI $(6 EXW DOVR
FKDQJHVLQGHVLJQDQGVFDOH7KHGLVDGYDQWDJHRI IURPWKHGHVLJQSKDVHRQ$UGXLQRWR3&FRQQHFWLRQYLDD
HPXODWRUVEXWDOVRSK\VLFDOPRGHOVLVWKDWWKH\ ZLUHOHVVQHWZRUN
QHHG WR FRQVLGHU WKH KXPDQ IDFWRU DQG VRFLDO 6WXGHQWV ILUVW VKRXOG FRQVWUXFW WKH FLUFXLW VKRZQ LQ
FRQWUDFWV WKDW JLYHWKH OLIHRI DQ RUJDQL]DWLRQRU )LJXUH  6LQFH LW LV DVVXPHG WKDW WKH VWXGHQWV DOUHDG\
VRFLHW\ KDYHH[SHULHQFHZLWK $UGXLQRIURPWKHSUHYLRXVVXEMHFW
• 6LPXODWLRQ PRGHOV DUH SXUSRVHIXO DEVWUDFWLRQV LWLVDOVRDVVXPHGWKDWWKH\ZLOOLQGHSHQGHQWO\PDVWHUWKH
RISK\VLFDOV\VWHPVWRH[SORUHKRZWKHFRPSOH[ FRQVWUXFWLRQ RI WKH IROORZLQJ FLUFXLW 7KH WHDFKHU VKRXOG
LQWHUUHODWLRQ EHWZHHQ KXPDQ VRFLDO VRIWZDUH DSSURSULDWHO\JXLGHWKHVWXGHQWEXWQRWSURYLGHWKHPZLWK
DQGKDUGZDUHV\VWHPVPLJKWOHDGWRYXOQHUDELOLW\ UHDG\PDGHVROXWLRQRUGRWKHZRUNIRUWKHP
RUUHVLOLHQFH6LPXODWLRQVSURYLGHVWXGHQWVZLWKD
PHDQV IRU H[DPLQLQJ FRPSOH[ LQWHUDFWLRQV DQG

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 404

 ,9 (08/$7,216
(PXODWLRQVDOORZUHVHDUFKHUVDQGSUDFWLWLRQHUVWRFUHDWH
YLUWXDO QHWZRUNV DQG WHVWEHGV WR H[SHULPHQW XQGHU
VSHFLILF´ FRQGLWLRQV DERXW V\VWHPV DQG QHWZRUNV 7KH\
KHOS VWXGHQWV XQGHUVWDQG PRUH FRPSOLFDWHG V\VWHPV WKDW
DUHXQDYDLODEOHWRWKHP
$UWLFOH >@ PHQWLRQV WKDW GHYHORSLQJ DXWRQRPRXV
5HLQIRUFHPHQW/HDUQLQJDJHQWVUHTXLUHVDVXLWDEOHWUDLQLQJ
HQYLURQPHQW WKDW DOORZV XV WR TXLFNO\ HYDOXDWH YDULRXV
DOWHUQDWLYHVSDUWLFXODUO\KRZWRDUUDQJHWUDLQLQJVFHQDULRV
WKDW SLW DWWDFNHUV DQG GHIHQGHUV DJDLQVW HDFK RWKHU 7KH
DUWLFOH SUHVHQWV WKH HQYLURQPHQW &\EHU%DWWOH6LP D
WUDLQLQJHQYLURQPHQWVXSSRUWLQJWKHWUDLQLQJRIVWXGHQWVDV
UHG DJHQWV LH DWWDFNHUV7KH SDSHU PRUHRYHU DGGHG WKH
FDSDELOLW\ WR WUDLQ EOXH DJHQWV LH GHIHQGHUV ,Q
SDUWLFXODU WUDLQLQJ D EOXH DJHQW MRLQWO\ ZLWK D UHG DJHQW
LQFUHDVHV WKH EOXH DJHQW¶V DELOLW\ WR WKZDUW VRSKLVWLFDWHG
UHG DJHQWV 7KLV HQYLURQPHQW FDQ EH XVHG DV D PRGHUQ
IRUP RI HGXFDWLRQ LQ ,7 VHFXULW\ )LJXUH  UHSUHVHQWV WKH
VLPSOH QHWZRUN WRSRORJ\ IRU WHVWLQJ VHFXULW\ LQ
&\EHU%DWWOH6LPZKRVHFUHDWLRQFDQEHLQGHSHQGHQWZRUN
)LJXUH&LUFXHRISK\VLFDOPRGHO IRUVWXGHQWV
 
%XWWKHPDLQREMHFWLYHRIWKHH[HUFLVHLVWRVKRZWKDW
WKH GDWD VHQW E\ WKH $UGXLQR VKRXOG EH HQFU\SWHG DQG
VHFXUHG7KHWDVNJRDOIRUVWXGHQWVVKRXOGEHWRFUHDWHWKH
LPSOHPHQWDWLRQZKHUHWKHXVHUQHHGVWRJRWRWKH+70/
SDJH DQG WKHQ KH FDQ HQFU\SW WKH VWULQJ XVLQJ WKUHH
GLIIHUHQW HQFU\SWLRQ PHWKRGV &DHVDU¶V &LSKHU 6+$
DQG $(6 $IWHU WKH HQFU\SWLRQ SURFHVV KDV EHHQ
FRPSOHWHG WKH HQFU\SWHG PHVVDJH ZLOO EH VKRZQ RQ WKH
VHULDOPRQLWRUDQGWUDQVPLWWHGWRWKHRWKHUGHYLFHYLDWKH
(63PRGXOH
5DVSEHUU\3LDQG$UGXLQR8QRFDQEHXVHGWRFRQVWUXFW
GLIIHUHQWKDUGZDUHGHYLFHV+DUGZDUHGHYLFHVFDQFRQWDLQ
RQO\ OLPLWDWLRQ IXQFWLRQDOLW\ ZKLFK HQDEOHV VWXGHQWV WR
IRFXV RQ WKH HVVHQWLDO HOHPHQWV RI VROYLQJ LQIRUPDWLRQ
VHFXULW\ SUREOHPV DQG UHGXFH RWKHU SUREOHPV WKDW ZRXOG
KDYHWREHGHDOZLWKWKHXVLQJRIUHDOV\VWHPV
7KH DGYDQWDJHV RI XVLQJ $UGXLQR DV WHFKQRORJ\ LQ )LJXUH&\EHU%DWWOH6LP>@
HGXFDWLRQ>@DUH

• $UGXLQR ERDUGV DUH JHQHUDOO\ DYDLODEOH DQG 
FKHDS 7KH VHFRQG RSLQLRQ LV WKH XVH RI WKH 1$6LP(PX
• $UGXLQRLVHDV\WRSURJUDPDQGOHDUQ HPXODWRU 7KH HPXODWRU LV GHVFULEHG E\ >@ DV DQ
• $UGXLQR FDQ KHOS VWXGHQWV FUHDWH GLIIHUHQW HPXODWRU EDVHG RQ 9DOJULQG DQ LQGXVWU\OHYHO WRRO IRU
FUHDWLYH VROXWLRQV DQG QRW IL[ WKHP RQ WKH PDQDJLQJ YLUWXDO QHWZRUNV 9LUWXDO%R[ URXWLQJ DQG
VSHFLILFVROXWLRQ WUDIILF ILOWHULQJ ZLWK D 0LNURWLN 5RXWHU26 KRVW DQG DQ
• $UGXLQR SURMHFWV DUH SUDFWLFDO DQG FDQ PRWLYDWH DWWDFNHU QRGH UXQQLQJ RQ 0HWD6SORLW ,W KDV FRQILJXUDEOH
VWXGHQWV WR FUHDWH KLVKHU RZQ VROXWLRQV IRU /LQX[ DQG :LQGRZV PDFKLQHV EDVHG RQ 0HWDVSORLWDEOH
GLIIHUHQWLVVXHVLQSUDFWLFH LPDJHV ZLWK SUHGHILQHG YXOQHUDEOH VHUYLFHV $Q\
VFHQDULR JHQHUDWHG IRU WKH VLPXODWRU FDQ EH WUDQVODWHG WR
7KHVH DGYDQWDJHV DUH ZK\ XQLYHUVLWLHV ZRUOGZLGH WKHHPXODWLRQZLWKDVLQJOHFRPPDQGDQGDWUDLQHGDJHQW
LQFOXGH $UGXLQREDVHG WHFKQRORJLHV LQ WKHLU VXEMHFWV LQ VLPXODWLRQ FDQ EH VHDPOHVVO\ GHSOR\HG LQ WKH
8QLYHUVLWLHV WKDW KDYH $UGXLQREDVHG WHFKQRORJLHV HPXODWLRQ
FRYHUHG LQ WKHLU VXEMHFWV DUH DFFRUGLQJ WR >@ IRU
H[DPSOH $OO PHQWLRQHG HQYLURQPHQWV FDQ EH XVHG IRU VHFXULW\
DQG GDWD SURWHFWLRQ WUDLQLQJ 7KH GLVDGYDQWDJH RI DOO RI
•8QLYHUVLW\RI$UL]RQD WKHPLVWKHUHTXLUHPHQWRIEDVLFNQRZOHGJHRI,7VHFXULW\
•8QLYHUVLW\RI$UNDQVDQV ZK\ WKH\ DUH QRW VXLWDEOH IRU LQWURGXFWLRQ FXUVHV RI
•8QLYHUVLW\RI&DOLIRUPLD 6HFXULW\DQG'DWD3URWHFWLRQ
•6WDQIRUG8QLYHUVLW\
7RVXPXSSK\VLFDOPRGHOVUHSUHVHQWRQHRIWKHLGHDO 9 6,08/$7,2102'(/6
RSWLRQV IRU WHFKQRORJLHV LQ JHQHUDO LQ HGXFDWLRQ
0RUHRYHUWKHUHLVVWLOOVSDFHIRUSURJUHVVLQXVLQJWKHPLQ 6LPXODWLRQ PRGHOV WKH RSSRVLWH RI HPXODWLRQV DOORZ
VHFXULW\DQGGDWDSURWHFWLRQHGXFDWLRQ VWXGHQWV WR WHVW WKH V\VWHP V DEVWUDFWLRQ ZKLFK FRQWDLQV
RQO\ IHDWXUHV RI LQWHUHVW ZLWKRXW WKH QHHG IRU GHWDLO

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 405

6LPXODWLRQ PRGHOV WKH RSSRVLWH RI HPXODWLRQV DOORZ 9, 6(5,286*$0(6
VWXGHQWV WR WHVW WKH V\VWHP V DEVWUDFWLRQ ZKLFK FRQWDLQV
'D\ E\ GD\ JDPLILFDWLRQ LV PRUH SRSXODU 7KH DUWLFOH
RQO\ IHDWXUHV RI LQWHUHVW ZLWKRXW WKH QHHG IRU GHWDLO
>@ GHILQHV WKLV SURFHVV DV IROORZV *DPHV FUHDWH
6LPXODWLRQV DUH DGYDQWDJHRXV IRU VWXGHQWV EHFDXVH WKH\
HQJDJHPHQW ± D QHFHVVLW\ IRU DQ\ OHDUQLQJ H[SHULHQFH
UHTXLUHIHZHUFRPSXWDWLRQDOUHVRXUFHVWKDQHPXODWRUVDQG
*DPLILFDWLRQ LV DGDSWLQJ DQ H[SHULHQFH OLNH SXUFKDVLQJ
SK\VLFDO VROXWLRQV PDNLQJ LW FRVWHIIHFWLYH DQG HDVLHU WR
EUHDGDQGPDVWHULQJDKDQGZULWLQJUHFRJQLWLRQSURJUDPWR
VFDOHWRDODUJHQXPEHULQQHWZRUNVL]H$QRWKHUJRDORI
OHDUQLQJPDWKZLWKJDPHOLNHHOHPHQWV
VLPXODWLRQV IRU F\EHUVHFXULW\ LV WR DLG LQ DQDO\]LQJ DQG
DVVHVVLQJ WKH RYHUDOO ULVN RI WKH V\VWHP SURYLGLQJ 7KH UHDVRQ IRU LQWURGXFLQJ JDPHV WR WKH HGXFDWLRQ
HQKDQFHG WUDLQLQJ FDSDELOLWLHV DQG FRQGXFWLQJ H[HUFLVHV SURFHVV LV WKH SRSXODULW\ RI YLGHR JDPHV DPRQJ \RXQJ
)LQDOO\ VLPXODWLRQV KHOS VWXGHQWV WR XQGHUVWDQG WKH SHRSOH7KHVWXGHQWVLQWKH,7ILHOGDUHWKHPRVWVXLWDEOH
VWUHQJWKV DQG YXOQHUDELOLWLHV WKDW XVHUV DWWDFNHUV DQG VDPSOH EHFDXVH LQ PRVW FDVHV WKHLU LQWHUHVW LQ VWXG\LQJ
GHIHQGHUVEULQJWRF\EHUVHFXULW\)RUWKHILUVWH[SHULHQFHV LQIRUPDWLRQ WHFKQRORJLHV VWHPV IURP WKHLU LQWHUHVW LQ
ZLWK VLPXODWLRQ PRGHOV VWXGHQWV FDQ IROORZ WKH SOD\LQJ FRPSXWHU JDPHV /HDUQLQJ XVLQJ JDPHV LV D
VLPXODWLRQ LQVWUXFWLRQV FUHDWHG E\ WKH WHDFKHU ,Q WKH FRQYHQLHQW PHWKRG IRU WHDFKHUV EHFDXVH WKH\ FDQ OHW WKH
IROORZLQJ VWDJHV VWXGHQWV VKRXOG OHDUQ WR GHYHORS VXFK VWXGHQWVSOD\ WKH JDPH %DVHG RQ WKH JDPH WKH VWXGHQWV
VLPXODWLRQRQO\ZLWKKHOSRUHQWLUHO\LQGHSHQGHQWO\ OHDUQHG WR FRYHU WKH WRSLF DQG EDVHG RQ WKH JDPH D
WHDFKHU FDQ H[SODLQ WR VWXGHQWV WKH VXEMHFW WKH\ KDYH D
,Q HGXFDWLRQ VHFXULW\ LV RIWHQ VWUHVVHG RQ WKH QHWZRUN
SUREOHPZLWK
VHFXULW\ $FFRUGLQJ WR>@ WKHUH DUH XVLQJ WKHVH W\SHV RI
6HULRXVJDPHV>@LVDQHPHUJHQWILHOGRIUHVHDUFKWKDW
VLPXODWLRQV
IRFXVHV RQ XVLQJ JDPHV ZLWK SXUSRVHV RWKHU WKDQ
• OLYH F\EHU VLPXODWLRQV6LPXODWLRQVZKHUHUHDO HQWHUWDLQPHQW WKDW FDQ EH DSSOLHG LQ GLYHUVH DUHDV 7KH
DFWRUV LQWHUDFW ZLWK SK\VLFDO V\VWHPV RI UHDO PDLQ DGYDQWDJHV RI VHULRXV JDPHV DUH WKDW JDPHV KDYHD
FRPSXWHUV FRQQHFWHG WR WKH UHDO DQG XVXDOO\ VWRU\DQGDVHWRIFKDOOHQJHVDQGWKH\PRWLYDWHJDPHUV RU
LVRODWHGQHWZRUN VWXGHQWV  WR DFFRPSOLVK WKH GHILQHG JRDOV E\ XVLQJ WDVNV
• YLUWXDO F\EHU  VLPXODWLRQV 6LPXODWLRQV ZKHUH WKDWZLOOKHOSWKHPJDLQWKHUHTXLUHGNQRZOHGJH
UHDO DFWRUV LQWHUDFW ZLWK WKH HPXODWLRQ RU 7KH PDLQ JRDO LQ WKH VHULRXV JDPH LV WR SDVV VRPH
VLPXODWLRQRIQHWZRUNVRUHPXODWHGRUVLPXODWHG OHYHOVZKLFKGHSHQGVRQWKHGLIILFXOW\RIWKHSUREOHPWKDW
DFWRUV LQWHUDFW ZLWK UHDO DQG XVXDOO\ LVRODWHG LVEHLQJVROYHG6RPHWLPHVKRZHYHULWFDQKDSSHQWKDWD
QHWZRUNV VWXGHQWJHWVVWXFNLQVROYLQJDSUREOHPDQGKLQWVIURPWKH
• FRQVWUXFWLYH F\EHU  VLPXODWLRQ 6LPXODWLRQV JDPH ZLOO QRW EH HQRXJK IRU KLP ,Q WKLV VWHS WKH KHOS
ZKHUH DFWRUV LQWHUDFW ZLWK HPXODWLRQV RU IURP WKH WHDFKHU LV WZRIROG /LNHZLVH WKH WHDFKHU PXVW
VLPXODWLRQVRIWKHQHWZRUN UHPHPEHU WR H[SODLQ WKH EDVLF SULQFLSOHV RI WKH JDPH
7KH ILUVW H[DPSOH RI WKH VLPXODWLRQ WRRO IURP DUWLFOH ZKHQLQWURGXFLQJLWWRWKHVWXGHQWV
>@ HYDOXDWHG DQ LQWUXVLRQ GHWHFWLRQ DOJRULWKP LV WKH 6HULRXVJDPHVFDQEHXVHGDFURVVYDULRXVILHOGVRI,7
231(7 QHWZRUN VLPXODWLRQ HQYLURQPHQW 7KLV )RU H[DPSOH WKH SDSHU >@ SUHVHQWV WKH VHULRXV JDPH
HQYLURQPHQW HPSOR\V VLPXODWLRQ XVLQJ D GLVFUHWHHYHQW &KHRSV GHYHORSHG LQ  ZKLFK FRYHUV WKH WHDFKLQJ
V\VWHP VSHFLILFDWLRQ '(96  PRGHO ZLWK D NQRZOHGJH VWXGHQWV RI VRPH EDVLF FRPSXWHU QHWZRUN FRQFHSWV
EDVHG OHDUQLQJ V\VWHP IRU WKH DWWDFNHU DQG D VWDWLVWLFDO GHVLJQHGH[SOLFLWO\IRUHGXFDWLQJXQLYHUVLW\VWXGHQWV
DQDO\]HU IRU WKH YXOQHUDELOLW\ DVVHVVPHQW7KH VLPXODWLRQ 0RUHRYHU LQ >@ DUH SUHVHQWHG VRPH H[DPSOHV RI
E\WKLVV\VWHPFDQWHDFKVWXGHQWVKRZWRFODVVLI\WKUHDWV H[LVWLQJVHULRXVJDPHVSURSHUIRUHGXFDWLRQLQ,7VHFXULW\
VSHFLI\DWWDFNPHFKDQLVPVYHULI\SURWHFWLRQPHFKDQLVPV • 7DUJHWHG$WWDFN7KH*DPH 7UHQG0LFUR 
DQGHYDOXDWHFRQVHTXHQFHV ,Q WKLV JDPH WKH VWXGHQW JHWV LQWURGXFHG WR VHYHUDO
7KHQH[WH[DPSOHXVHGLQWHUPVRIHYDOXDWLQJWKHHIIHFW VHFXULW\ GHFLVLRQV WR EH PDGH (DFK GHFLVLRQ LPSDFWV
RI WKUHDWV DQG DWWDFNV LV WKH VLPXODWLRQ IUDPHZRUN FDOOHG VXEVHTXHQW RSWLRQV SULPDULO\ EHFDXVH VRPH GHFLVLRQV
'LVWULEXWHGGHQLDORIVHUYLFHVLPXODWRU ''R66,0 ZKLFK HQFRPSDVV FRVWV UHGXFLQJ WKH DYDLODEOH ODWHU GXULQJ WKH
XVHV 201H7 DQG ,1(7 )UDPHZRUN DV WKH QHWZRUN JDPH EHFDXVH WKH UHPDLQLQJ EXGJHW LV LQVXIILFLHQW 7KLV
VLPXODWLRQ EDVLV WR HYDOXDWH GLIIHUHQW ''R6 DWWDFNV DQG JDPH V FULWLFDO DVSHFW LV EURDGHQLQJ WKH SOD\HU¶V KRUL]RQ
GHIHQVHPHFKDQLVPV DQG VKRZLQJ WKH SRWHQWLDO LPSDFW RI VHFXULW\UHODWHG
7KH ODVW V\VWHP PHQWLRQHG LQ >@ LV &\EHU $QDO\VLV GHFLVLRQVLQUHDOLVWLFVFHQDULRV
0RGHOLQJ (YDOXDWLRQ IRU 2SHUDWLRQV &$0(2  ,Q WKLV • (QWHU±,76HFXULW\*DPH
V\VWHP D VWXGHQW FDQ PRGHO DQG VLPXODWH YDULRXV 7KLV JDPH DLPV WR GHPRQVWUDWH WKDW LQIRUPDWLRQ IURP
G\QDPLFV RI D F\EHU HFRV\VWHP VXFK DV WKUHDWV DQG FRPSDQ\HPSOR\HHVFDQEHXVHGIRUF\EHUDWWDFNVHYHQLI
GHIHQVLYH VWUDWHJLHV  ,W LV DOVR SRVVLEOH IRU VWXGHQWV WR WKHLQIRUPDWLRQLVQRWVHFXULW\UHODWHG7KHSOD\HULVLQWKH
YHULI\ DQG YDOLGDWH SRVVLEOH FRQILJXUDWLRQV DQG EHKDYLRU DWWDFNHU V SRVLWLRQ DQG KLV WDVN LV WR XVH YDULRXV
IRUF\EHUDJLOHDQGUHVLOLHQWGHIHQVHVWRVWXG\VHQVLWLYLW\ DSSURDFKHV DQG WHFKQLTXHV IRU DWWDFNV RQ :L)L 86%
WR LQLWLDO FRQILJXUDWLRQV DQG WR GLVFRYHU XQDQWLFLSDWHG VWLFNVSULQWHUVVRFLDOPHGLDHWF
HPHUJHQWEHKDYLRU • &\EHU&,(*(
231(7 'LVWULEXWHG GHQLDO RI VHUYLFH VLPXODWRU DQG &\EHU&,(*(LVDJDPHLQZKLFKDXVHUVSHQGVYLUWXDO
&\EHU $QDO\VLV 0RGHOLQJ (YDOXDWLRQ IRU 2SHUDWLRQV FDQ PRQH\WRRSHUDWHDQGGHIHQGKLVQHWZRUNVDQGFDQZDWFK
EHXVHGE\VWXGHQWVLQXQGHUVWDQGLQJDWWDFNVDQGWUDLQLQJ WKH FRQVHTXHQFHV RI KLV FKRLFHV ZKLOH XQGHU DWWDFN 7KH
IRUGLIIHUHQWVFHQDULRV JDPH IRFXVHV RQ QHWZRUNVHFXULW\UHODWHG WRSLFV
LQFOXGLQJ FRQILJXUDEOH ILUHZDOOV 931V OLQN HQFU\SWLRQV
DQG DFFHVV FRQWURO PHFKDQLVPV ,W LQFOXGHV FRPSRQHQWV
VXFK DV ELRPHWULF VFDQQHUV RU DXWKHQWLFDWLRQ VHUYHUV
$WWDFNVFDQEHFDXVHGE\FRUUXSWLRQRILQVLGHUVWUDSGRRU

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 406

DUUDFNV 7URMDQ KRUVHV YLUXVHV GHQLDO RI VHUYLFH DQG VXEWDVN EDVHG RQ ZKLFK WKH WHDFKHU FDQ FKHFN WKH
H[SORLWDWLRQRIZHDNO\FRQILJXUHGV\VWHPV XQGHUVWDQGLQJRIWKHFRYHUHGWRSLF
• .H\WRH[FHOOHQFH $VPHQWLRQHGLQ&KHDSHU,VHULRXVJDPHVDUHWKHEHVW
FKRLFHIRUWHDFKLQJ,7VHFXULW\WRQRQ,7VWXGHQWV,QWKLV
.H\WRH[FHOOHQFHLVWKHJDPHGHYHORSHGDWWKH,QVWLWXWH FDVH WKH\ VKRXOG EH LQ VHSDUDWH JURXSV ZKHUH WKH\ ZLOO
IRU*DPHVDW6WXWWJDUW0HGLD8QLYHUVLW\RQEHKDOIRIWKH SUHVHQW PRUH VWUDLJKWIRUZDUG H[DPSOHV ZLWK D IRFXV RQ
6ZLVV EDQN 86%7KH JDPH DLPV WR HQKDQFH HPSOR\HHV  EDVLFFRQFHSWVDQGWHUPLQRORJ\LQWKHILHOGRI,7VHFXULW\
VHFXULW\DZDUHQHVVSOD\IXOO\7KHJDPHLVEURZVHUEDVHG
DQGVWXGHQWVFDQSOD\LWDQ\ZKHUH7KHJDPH VJRDOLVQRW 9,, &21&/86,21
WR UHVSRQG FRUUHFWO\ WR DV PDQ\ VHFXULW\ LQFLGHQWV DV
SRVVLEOHEXWWRULVHZLWKLQDILFWLRQDORUJDQL]DWLRQ ,QIRUPDWLRQ WHFKQRORJLHV UHSUHVHQW RQH RI WKH PRVW
SHUVSHFWLYHILHOGVRIPRGHUQVFLHQFH7KHSURJUHVVLQWKLV
DUHDLVVRJUHDWWKDWZKDWLVDQDEVROXWHQRYHOW\WRGD\FDQ
EH UHSODFHG E\ VRPHWKLQJ QHZ DQG EHWWHU WRPRUURZ
0RUHRYHU LW UHTXLUHV D TXDQWLW\ RI TXDOLILHG ZRUNHUV
ZKLFKWKHFXUUHQW6ORYDNHGXFDWLRQV\VWHPLVQRWDEOHWR
SURGXFH :H KDYH WR PDNH PRUH HIIRUW WR DWWUDFW \RXQJ
SHRSOH WR WKLV LQGXVWU\ 7KH VROXWLRQ LV WR LQWURGXFH
HPHUJLQJ WHFKQRORJLHV LQ ,7 HGXFDWLRQ 7KHVH
WHFKQRORJLHVVKRXOGPRWLYDWHVWXGHQWVIRUWKHGHFLVLRQWR
VWXG\,7EXWDOVRIRULWVVXFFHVVIXOFRPSOHWLRQ7KH\DUH
FUHDWHGFUHDWLYHO\WRDURXVHVWXGHQWV¶LQWHUHVWLQDFTXLULQJ
HVVHQWLDO NQRZOHGJH LQ ,7 VHFXULW\  7KH DUWLFOH SUHVHQWV
HPHUJLQJ WHFKQRORJLHV LQ ,7 HGXFDWLRQ OLNH SK\VLFDO
PRGHOV HPXODWLRQV RU VLPXODWLRQ PRGHOV )RU HDFK
 WHFKQRORJ\ LQWURGXFHG FRQFHQWUDWH RQ VXLWDEOH H[DPSOHV
DQGKRZWRXVHWKHPIRUWHDFKLQJ
)LJXUH7\SLFDO6FHQDULRLQ.H\WR([FHOOHQFH>@
9,,, 5()(5(1&(6
 >@,QWHUQDWLRQDO/DERXU2UJDQL]DWLRQ³6NLOOVVKRUWDJHV
DQG ODERXU PLJUDWLRQ LQ WKH ILHOG RI LQIRUPDWLRQ DQG

FRPPXQLFDWLRQ WHFKQRORJ\ LQ &DQDGD &KLQD *HUPDQ\
,I ZH GHDO ZLWK RWKHU DUWLFOHV ZH ZLOO ILQG PDQ\
DQG 6LQJDSRUH´  >2QOLQH@ $YDLODEOH
GLIIHUHQWJDPHVIRFXVHGRQWKHVWXG\RIF\EHUVHFXULW\,Q
KWWSVZZZLORRUJZFPVSJURXSVSXEOLF
WKHIROORZLQJDUWLFOH>@DUHPHQWLRQHGIRUH[DPSOH
HGBGLDORJXH
• 7DUJHWHG$WWDFN7KH*DPH VHFWRUGRFXPHQWVSXEOLFDWLRQZFPVBSGI
$ TXL]EDVHG JDPH WKDW H[SORUHV WKH WRSLF RI WDUJHWHG >@ 0LQLVWHUVWYR LQYHVWtFLt UHJLRQiOQHKR UR]YRMD D
DWWDFN:LWKLQWKHJDPHWKHVWXGHQWIROORZVWKHVWRU\OLQH LQIRUPDWL]iFLH 6ORYHQVNHM UHSXEOLN\ ³1È52'1È
DQG DFWV DV D VHFXULW\ WHDP PHPEHU IRU DQ RUJDQL]DWLRQ 675$7e*,$ ',*,7È/1<&+ =58ý1267Ë
ODXQFKLQJ D PRELOH SD\PHQWV DSS ZLWK ELRPHWULF 6/29(16.(- 5(38%/,.< $ $.ý1é 3/È1 1$
DXWKHQWLFDWLRQ 7KH JDPH FRQVLVWV RI PXOWLSOHFKRLFH 52.<  ± ´   >2QOLQH@ $YDLODEOH
TXHVWLRQV WKDW OHDG WR FKDQJHV LQ WKH VWRU\OLQH DOORZLQJ KWWSVPLUULJRYVNZSFRQWHQWXSORDGV16'=D
WKHVWXGHQWWRPDNHGHFLVLRQVDERXWF\EHUVHFXULW\PDWWHUV $3SGI
• 'DWD&HQWHU$WWDFN7KH*DPH >@ 1DMY\ããt NRQWUROQê ~UDG 6ORYHQVNHM UHSXEOLN\
,Q WKH JDPH WKH SOD\HU PXVW SURWHFW D KRVSLWDO IURP ³35,25,71e 2%/$67, =$0(5$1,$ ý,11267,
GDWD FHQWHU DWWDFNV E\ PDNLQJ WKH ULJKW GHFLVLRQV 1.8651$52.´>2QOLQH@$YDLODEOH
UHJDUGLQJVHFXULW\PHDVXUHVDQGUHFRYHU\WHFKQLTXHV KWWSVZZZQNXJRYVNGRFXPHQWV3UL
• 7KHPLVVLQJOLQN RULW\BFLQQRVWLB1.8B65BQDBURNBSGI
7KH VHULRXV JDPH WDUJHWV XQLYHUVLW\ VWXGHQWV DQG >@/9RNRURNRV0&KRYDQHF0/DWND³6HFXULW\RI
HPSOR\HHV 7KH JDPH FRQVLVWV RI FKDOOHQJHV WR ILQG WKH GLVWULEXWHG GHWHFWLRQ V\VWHPV EDVHG RQ PXOWLVHQVRU
DWWDFNHUDQGHYHQLIWKHSOD\HUVHOHFWVWKHZURQJDQVZHU IXVLRQ´>2QOLQH@$YDLODEOH
WKH\FDQSURFHHGWRWKHQH[WOHYHO KWWSVLHHHH[SORUHLHHHRUJGRFXPHQW"GHQLHG 
7RGD\ SOHQW\ RI VHULRXV JDPHV DUH GHVLJQHG IRU >@+.DYDN--3DGLOOD'9HUQR%LGR6<'LDOOR
HGXFDWLRQDO SXUSRVHV DW YDULRXV VWDJHV RI HGXFDWLRQ 7KH 5 *RUH DQG 6 6KHWW\  ³6LPXODWLRQ IRU F\EHUVHFXULW\
IROORZLQJ H[DPSOHV DUH &\EHU &KDOODJH $JJLH /LIH RU VWDWH RI WKH DUW DQG IXWXWH GLUHFWLRQV´  >2QOLQH@
*RRJOH;66*DPH,WLVXSWRWKHSURIHVVRUVWRIDPLOLDUL]H $YDLODEOH
WKHPVHOYHV ZLWKWKHVHJDPHV DQGFKRRVHIRUWKHLUFRXUVH KWWSVDFDGHPLFRXSFRPF\EHUVHFXULW\DUWLFOHW\DE
WKH RQH WKDW ZLOO EH WKH PRVW VXLWDEOH IRU PHHWLQJ WKH \
H[SHFWHG JRDOV 7KH SURIHVVRUV DOVR KDYH WR FKRRVH WKH >@ $ $QGUHDWRV ³(Q (GXFDWLRQDO 6FHQDULR IRU
DSSURSULDWH PHWKRG RI JDPH SUHVHQWDWLRQ 7KH\ KDYH WR 7HDFKLQJ &\EHU 6HFXULW\ 8VLQJ ORZFRVW (TXLSPHQW DQG
ILQGDFRPSURPLVHEHWZHHQSUHVHQWLQJWKHEDVLFFRQFHSWV 2SHQ 6RXUFH 6RIZDUH´  >2QOLQH@ $YDLODEOH
RI WKH JDPH DQG VDPSOH H[DPSOHV DQG WKH VWXGHQW V KWWSVZZZUHVHDUFKJDWHQHWSXEOLFDWLRQB(GX
LQGHSHQGHQW ZRUN ,Q DQ LGHDO FDVH WKH WHDFKHU VKRXOG FDWLRQDOB6FHQDULRBIRUB7HDFKLQJB&\EHUB6HFXULW\B8VLQJBO
VKRZVXFKH[DPSOHVZLWKWKHKHOSRIZKLFKVWXGHQWVZLOO RZFRVWB(TXLSPHQWBDQGB2SHQB6RXUFHB6RIWZDUH
XQGHUVWDQG WKH VXEMHFW V EDVLF SULQFLSOHV +RZHYHU WKH
WDVN IRU LQGHSHQGHQW ZRUN VKRXOG FRQWDLQ DQ DGGLWLRQDO

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 407

 >@ 0 5DILX]]DPDQ ³)HDVLELOLW\ DQG (IILFLHQF\ RI
5DVSEHUU\3LDVWKHVLQJOH%RDUG&RPSXWHU6HQVRU1RGH´
 >2QOLQH@ $YDLODEOH
KWWSVZZZUHVHDUFKJDWHQHWSXEOLFDWLRQB(GX
FDWLRQDOB6FHQDULRBIRUB7HDFKLQJB&\EHUB6HFXULW\B8VLQJBO
RZFRVWB(TXLSPHQWBDQGB2SHQB6RXUFHB6RIWZDUH
>@ 70 *HHWKDQMDOL . 5DXVKDQL 06 6ULYLG\D
6KUHHUDMV+$ 05 ³6HFXULW\ 6\VWHP XVLQJ $UGXLQR´
 >2QOLQH@ $YDLODEOH
KWWSVZZZUHVHDUFKJDWHQHWSXEOLFDWLRQB6HF
XULW\B6\VWHPBXVLQJB$UGXLQR
>@$6KDSLQD011RRUKD\DWL$.1RU=ROLGDU.
+DVKDP $-. ³,27 6HFXULW\ 'DWD (QFU\SWLRQ IRU
$UGXLQREDVHG ,27 'HYLFHV´  >2QOLQH@ $YDLODEOH
KWWSVZZZMRXUQDOSSZFRPLQGH[SKSMSVSDUWLFOHYLHZ

>@-$7XNKWDQD]DURYLFK³$GYDQWDJHRIWKHDUGXLQR
SODWIRUP LQ IRUPLQJ FUHDWLYH VNLOOV LQ \RXWK´ 
>2QOLQH@ $YDLODEOH
KWWSVUHSRMRXUQDOQ[FRPLQGH[SKSQ[DUWLFOHGRZQORDG

>@ 7 .XQ] & )LVKHU - /D 1RYDUD*VHOO &
1JX\HQ ³$ PXOWLDJHQW &\EHU%HWWOHQ6LP IRU 5/ &\EHU
2SHUDWLRQ$JHQWV´>2QOLQH@$YDLODEOH
KWWSVDU[LYRUJSGISGI
>@ - +XUWXN 0 &KRYDQHF 1 $GDP 7KH DUGXLQR
SODWIRUPFRQQHFWHGWRHGXFDWLRQSURFHVV>2QOLQH@
$YDLODEOHKWWSVLHHH[SORUHLHHHRUJGRFXPHQW
>@ - -DQLVFK 7 3HYQ\ 9 /LV\ ³1$6LPX(PX
1HWZRUN $WWDFN 6LPXODWRU  (PXODWRU IRU 7UDLQLQJ
$JHQWV*HQHUDOL]LQJWR1RYHO6FHQDULRV´>2QOLQH@
$YDLODEOHKWWSVDU[LYRUJSGISGI
>@ $ - %ULDQ ³*DPLILFDWLRQ LQ (GXFDWLRQ´ 
>2QOLQH@ $YDLODEOH
KWWSVZZZUHVHDUFKJDWHQHWSXEOLFDWLRQB*D
PLILFDWLRQBLQB(GXFDWLRQ
>@ 00 &UX]&XQKD ³+DQGERRN RI 5HVHDUFK RQ
6HULRXV *DPHV DV (GXFDWLRQDO %XVLQHVV DQG 5HVHDUFK
7RROV´  >2QOLQH@ $YDLODEOH
KWWSVZZZUHVHDUFKJDWHQHWSXEOLFDWLRQB*D
PLILFDWLRQBLQB(GXFDWLRQ
>@ ) /LDUNDSLV () $QGHUVRQ $ 2LNRQRPRX
³6HULRXV JDPHV IRU XVH LQ D KLJKHU HGXFDWLRQ
HQYLURQPHQW´   >2QOLQH@  $YDLODEOH
KWWSVZZZUHVHDUFKJDWHQHWSXEOLFDWLRQB6HUL
RXVB*DPHVBIRUBXVHBLQBDB+LJKHUB(GXFDWLRQB(QYLURQPH
QW
>@56FKPLW]'+HX]HURWK³6HULRXV*DPHVIRU,7
6HFXULW\ (GXFDWLRQ´  >2QOLQH@ $YDLODEOH
KWWSVSXEOLNDWLRQHQXQL
WXHELQJHQGH[POXLELWVWUHDPKDQGOH*DPH
V,76(&$EVWUDFWILQDOSGI"VHTXHQFH  LV$OORZHG \
>@0&DOYDQWR)&DUXVR$&XUFL$3LFFLQRDQG
9 5RVVDQR ³$ 5DSLG 5HYLHZ RQ 6HULRXV *DPHV IRU
&\EHUVHFXULW\ (GXFDWLRQ $UH ³6HULRXV´ DQG *DPLQJ
$VSHFWV:HOO%DODQFHG"´>2QOLQH@$YDLODEOH
KWWSVZZZQNXJRYVNGRFXPHQWV3UL
RULW\BFLQQRVWLB1.8B65BQDBURNBSGI

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 408

 0DSSLQJWKH(QYLURQPHQW8VLQJDQ8OWUDVRQLF
DQG,QIUDUHG6HQVRU

.3ULELORYi '*DEULVND DQG=0lVLDURYD 
8QLYHUVLW\RI66&\ULODQG0HWKRGLXV,QVWLWXWHRI&RPSXWHU7HFKQRORJLHVDQG,QIRUPDWLFV7UQDYD6ORYDNLD
NDWDULQDSULELORYD#XFPVNGDUMDJDEULVND#XFPVNPDVLDURYD#XFPVN


$EVWUDFW² 7KLV SDSHU LV IRFXVHG RQ DQ HQYLURQPHQW KRZ WR PDNH PDFKLQHV UHVSRQG WR YDULRXV LQSXWV DQG
PDSSLQJDSSOLFDWLRQXVLQJ/HJR0LQGVWRUPV7KHPDLQJRDO FRQGLWLRQV
LVWRFUHDWHDSURJUDPIRUDURERWWKDWZLOOEHDEOHWRVROYH 6SDWLDO ,PDJLQDWLRQ 'HVLJQLQJ DQG EXLOGLQJ URERWV
SUREOHPV RI PRYHPHQW DQG PDSSLQJ LQ DQ XQNQRZQ RIWHQUHTXLUHVVWXGHQWVWRYLVXDOL]HKRZGLIIHUHQWSDUWVILW
HQYLURQPHQW :HFKRVH-DYD DQGWKH(FOLSVH SURJUDPPLQJ WRJHWKHU LQ WKUHHGLPHQVLRQDO VSDFH HQKDQFLQJ WKHLU
HQYLURQPHQWDVWKHSURJUDPPLQJODQJXDJH6LPSOHPD]HVLQ VSDWLDOLPDJLQDWLRQ
WKH VKDSH RI D UHFWDQJOH DQG WKH OHWWHUV & ZHUH FUHDWHG IRU
WKH PDSSLQJ RI WKH HQYLURQPHQW E\ WKH URERW :H XVHG DQ 7HDPZRUN DQG &RPPXQLFDWLRQ 0DQ\ /(*2
XOWUDVRQLFDQGLQIUDUHGVHQVRUIRUVHQVLQJWKHHQYLURQPHQW 0LQGVWRUPV SURMHFWV DUH GRQH LQ WHDPV SURPRWLQJ
,Q WKH HQG ZH SUHVHQW D FRPSDULVRQ DQG WKH UHVXOWV RI FROODERUDWLRQDQGFRPPXQLFDWLRQVNLOOVDPRQJVWXGHQWV
PHDVXUHPHQWV RI ERWK VHQVRUV ZKHQ PDSSLQJ WKH 3ODQQLQJ DQG $EVWUDFWLRQ 3ODQQLQJ DQG DEVWUDFW
HQYLURQPHQW WKLQNLQJ DUH HVVHQWLDO VNLOOV LQ HQJLQHHULQJ DQG FRPSXWHU
VFLHQFH /(*2 0LQGVWRUPV SURMHFWV UHTXLUH VWXGHQWV WR
SODQWKHLUURERW VEHKDYLRXUDQGDEVWUDFWIURPWKHSK\VLFDO
, ,1752'8&7,21 PRGHOWRFRGH
7KHLQWHJUDWLRQRIWKH/(*20LQGVWRUPVSODWIRUPLQWR ,QFUHDVHG $FWLYLW\ LQ /HDUQLQJ 3URJUDPPLQJ
KLJKHU HGXFDWLRQ KDV LQGHHG EHHQ RQ WKH ULVH DQG KDV 5HVHDUFKHUVKDYHIRXQGWKDWXVLQJ/(*20LQGVWRUPVFDQ
EURXJKW QXPHURXV EHQHILWV WR WHFKQLFDOO\ RULHQWHG ERRVW VWXGHQWV  LQWHUHVW DQG HQJDJHPHQW LQ OHDUQLQJ
XQLYHUVLWLHV DQG WKHLU VWXGHQWV >@ +HUH DUH VRPH NH\ SURJUDPPLQJ 7KH KDQGVRQ LQWHUDFWLYH QDWXUH RI WKH
SRLQWV KLJKOLJKWLQJ WKH SRVLWLYH LPSDFW RI /(*2 SODWIRUP PDNHV SURJUDPPLQJ PRUH DFFHVVLEOH DQG
0LQGVWRUPVLQHGXFDWLRQ HQMR\DEOH
'HYHORSPHQWRI/RJLFDO7KLQNLQJ/(*20LQGVWRUPV ,Q VXPPDU\ /(*2 0LQGVWRUPV LV D YDOXDEOH
HQFRXUDJHV VWXGHQWV WR WKLQN ORJLFDOO\ DQG FULWLFDOO\ DV HGXFDWLRQDOWRROWKDWHQKDQFHVYDULRXVDVSHFWVRIWHFKQLFDO
WKH\GHVLJQDQGSURJUDPURERWV7KLVSURFHVVKHOSVWKHP HGXFDWLRQ ,W IRVWHUV FULWLFDO WKLQNLQJ SUREOHPVROYLQJ
GHYHORS SUREOHPVROYLQJ VNLOOV DQ HVVHQWLDO DVSHFW RI WHDPZRUN DQG FUHDWLYLW\ ZKLOH SURYLGLQJ SUDFWLFDO
KLJKHUHGXFDWLRQ H[SHULHQFH LQ URERWLFV SURJUDPPLQJ DQG HQJLQHHULQJ
7HDFKLQJ )XQGDPHQWDOV RI $OJRULWKPV DQG FRQFHSWV,WVSRVLWLYHLPSDFWRQOHDUQLQJSURJUDPPLQJLV
3URJUDPPLQJ6WXGHQWVOHDUQWKHEDVLFVRIDOJRULWKPVDQG SDUWLFXODUO\QRWHZRUWK\DVLWKHOSVVWXGHQWVEXLOGDVWURQJ
SURJUDPPLQJ WKURXJK KDQGVRQ H[SHULHQFH ZLWK WKH IRXQGDWLRQLQWKLVFUXFLDOVNLOO>@>@
SODWIRUP 7KLV SUDFWLFDO DSSURDFK RIWHQ PDNHV DEVWUDFW
FRQFHSWVPRUHWDQJLEOHDQGXQGHUVWDQGDEOH ,, 6(16256/(*20,1'672506(9
5RERWLFV /(*2 0LQGVWRUPV SURYLGHV DQ H[FHOOHQW 7KDQNVWRWKHVHQVRUVWKHURERWFDQSHUFHLYHWKHZRUOG
SODWIRUP IRU WHDFKLQJ URERWLFV FRQFHSWV 6WXGHQWV FDQ DURXQGLWVHHFRORXUVDQGOLJKWLQWHQVLW\IHHOWRXFKDQG
H[SORUH YDULRXV DVSHFWV RI URERWLFV LQFOXGLQJ VHQVRU PRYH SUHFLVHO\ LQ VSDFH >@ 7KDQNV WR WKH VHQVRUV WKH
LQWHJUDWLRQPRWRUFRQWURODQGDXWRQRPRXVQDYLJDWLRQ>@ /(*2 0LQGVWRUPV (9 URERW LV HQULFKHG ZLWK PDQ\
0HFKDQLFV DQG '\QDPLFV %XLOGLQJ DQG SURJUDPPLQJ IXQFWLRQV DQG RSWLRQV GHWHFW WRXFKSUHVV GHWHFW FRORXU
URERWV ZLWK /(*2 0LQGVWRUPV DOVR LQYROYHV GLVWDQFHIURPWKHVHQVRUGHWHFWPRYHPHQW >@
XQGHUVWDQGLQJ PHFKDQLFV DQG G\QDPLFV 6WXGHQWV JDLQ
LQVLJKWV LQWR KRZ GLIIHUHQW FRPSRQHQWV ZRUN WRJHWKHU WR $ ,QIUDUHGVHQVRU
DFKLHYHVSHFLILFWDVNV $Q LQIUDUHG ,5  VHQVRU GHWHFWV WKH ZRUOG DURXQG LWE\
&RQVWUXFWLRQ 6NLOOV $VVHPEOLQJ URERWV ZLWK /(*2 HPLWWLQJ LQIUDUHG OLJKW UD\V ZKLFK DUH VXEVHTXHQWO\
EULFNV KHOSV VWXGHQWV GHYHORS FRQVWUXFWLRQ VNLOOV 7KH\ UHIOHFWHG IURP WKH VXUIDFH RI D ZDOO RU REVWDFOH ,W KDV D
OHDUQ WR IROORZ LQVWUXFWLRQV ZRUN ZLWK SUHFLVLRQ DQG ]RRPPRGHZKLFKFDQPHDVXUHDGLVWDQFHRIXSWRFP
WURXEOHVKRRWZKHQQHFHVVDU\ 7KHPHDVXUHGGLVWDQFHLVQRWJLYHQLQPHWULFRURWKHUXQLWV
EXW LQ WKH UDQJH RI YDOXHV   FORVH DQG   IDU 7KH
6HQVRUV DQG (OHFWULF 0RWRUV 7KH SODWIRUP LQWURGXFHV
VHQVRULVXVHGWRPHDVXUHVKRUWGLVWDQFHVLWFDQVHUYHDV
VWXGHQWVWRVHQVRUVDQGHOHFWULFPRWRUVZKLFKDUHHVVHQWLDO
WKH URERW V H\HV ,QIUDUHG OLJKW WHFKQRORJ\ LV YHU\
FRPSRQHQWV LQ PRGHUQ WHFKQRORJ\ 7KLV NQRZOHGJH LV
ZLGHVSUHDG XVHG LQ UHPRWH FRQWUROV VPDUWSKRQHV
YDOXDEOHIRUIXWXUHHQJLQHHUVDQGWHFKQRORJLVWV
PRQLWRULQJGHYLFHVQLJKWYLVLRQDQGPDQ\RWKHUW\SHVRI
&RQWURO6\VWHPV/(*20LQGVWRUPVDOORZVVWXGHQWVWR FRQWURODQGVHQVLQJ>@>@
H[SHULPHQWZLWKFRQWUROV\VWHPVKHOSLQJWKHPXQGHUVWDQG


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 409

% 8OWUDVRQLFVHQVRU XSQRLVHIURPWKHHQYLURQPHQWLWLVQRWVXSSRVHG
7KHXOWUDVRQLFVHQVRU 86 LVSULPDULO\XVHGWRPHDVXUH WR PHDVXUH (PSKDVLV VKRXOG DOVR EH SODFHG RQ
GLVWDQFHZKHWKHULWLVVWRSSLQJLQIURQWRIDQREVWDFOHRU WKH PRWRU RQ ZKLFK LW FDQ EH IL[HG DQG LWV
DYRLGLQJ REMHFWV 7KLV W\SH RI VHQVRU LV FRPPRQO\ XVHG URWDWLRQRSWLRQVDVZHOODVWKHFDEOHZLWKZKLFK
IRU GLVWDQFH PHDVXUHPHQWV RU IRU PDSSLQJ WKH WKH VHQVRU LV FRQQHFWHG ZKLFK ZKHQ URWDWHG
HQYLURQPHQW >@ $QRWKHUSRVVLELOLW\ RI XVH LVWR XVHWKLV PRUH WKDQ  GHJUHHV FDQ SUHYHQW WKH
VHQVRUDVDSDUNLQJVHQVRUIRUDFDUZKHUHWKHVHQVRUZLWK PHDVXUHPHQW RU JHWWLQJ WDQJOHG DQG WKXV DIIHFW
WKHGHFUHDVLQJGLVWDQFHIURPWKHREMHFWWKHZDOOZRXOG WKHPHDVXUHPHQWUHVXOWV
UHSRUW D ZDUQLQJ EHIRUH KLWWLQJ WKH ZDOO 7KH (9 FXEH • WKH HQYLURQPHQW VHQVHG E\ WKH VHQVRU PXVW DOVR
ZRXOG XVH LWV EXLOWLQ DXGLR VSHDNHU WR DQQRXQFH WKLV EHVXEMHFWWRVHYHUDOUHTXLUHPHQWVQDPHO\
WKUHDW ZLWK D ZDUQLQJ VRXQG  D EHHS 7KLV VHQVRU KDV D R WKHVXUIDFHIURPZKLFKWKHVRXQGZDYHV
YHUVDWLOHXVHDQGFDQEHXVHGZKHQZRUNLQJZLWKDURERW ZLOO EH UHIOHFWHG VKRXOG EH IODW QRW
RU ZLWK UHDO YHKLFOHV ZKHUH WKH\ DUH XVHG SULPDULO\ DV VODQWHG VXFK DV IXUQLWXUH ZDOOV
SDUNLQJ VHQVRUV  SDUNLQJ DVVLVWDQWV RU HYHQ ZLWK FDUGERDUG ZDOOV  DQG ZLWKRXW VWUXFWXUH
DXWRQRPRXV YHKLFOHV ZKHUH WKH\ FDQ PHDVXUH WKH VXFK DV D FRXFK FDUSHW RU RWKHU
GLVWDQFH DURXQG WKH YHKLFOH XS WR  PHWHUV 7KH LUUHJXODUREMHFWV 
GLVDGYDQWDJH RI WKLV ZD\ RI XVLQJ WKH VHQVRU LV WKH R REMHFWV WKDW DUH VFDQQHG PXVW EH DW D
PRYHPHQWRIWKHYHKLFOHLWVHOIDQGLWVVSHHG6WURQJHUDLU VXIILFLHQW GLVWDQFH  PRUH WKDQ  FP
IORZLVDQRWKHUREVWDFOHDVWKHZLQGVLJQLILFDQWO\DIIHFWV IURPWKHVHQVRURWKHUZLVHWKH\ZLOOQRW
WKHPHDVXUHPHQWVRIWKHXOWUDVRQLFVHQVRU>@>@ EHPHDVXUHG
7KH VHQVRU KDV WZR PRGHV WKH\ DUH 0HDVXUH PRGH • WKHVHQVRUKDVDZLGHILHOGRIYLHZVRLWFDQGHWHFW
GLVWDQFH PHDVXUHPHQW PRGH  DQG 3UHVHQFH PRGH DQREMHFWORFDWHGFORVHUWRWKHVLGHUDWKHUWKDQDPRUH
OLVWHQLQJSUHVHQFHPRGH $WZRUNZHRQO\XVH0HDVXUH GLVWDQWREMHFWGLUHFWO\LQIURQWRILW>@>@
PRGHZKLFKPHDVXUHVWKHGLVWDQFHIURPZDOOVRUREMHFWV
,W FDQ DOVR VHUYH DV D GLVWDQFH PHWHU ZKLFK FDQ VHQG D 
PHVVDJH ZKHQ WKH URERW DSSURDFKHV D JLYHQ GLVWDQFH RU
ZKHQDQHQHP\REMHFWDSSURDFKHVWKHVHQVRU7KHVHQVRU
LWVHOILQIRUPVXVWKDWWKHVHQVRULVLQ0HDVXUHPRGHE\WKH
IDFWWKDWLWVEDFNOLJKWVWDUWVWRJORZUHG2QWKHRWKHUKDQG
3UHVHQFH PRGH DOHUWV XV DERXW LWV PRGH E\ IODVKLQJ WKH
EDFNOLJKW ,Q WKLV PRGH LW WULHV WR UHFRJQL]H DQRWKHU
XOWUDVRQLF VHQVRU DQG OLVWHQV IRU VRXQG VLJQDOV WKDW LV LW
GRHVQRWWUDQVPLWWKHP>@>@
7KH XOWUDVRQLF VHQVRU PHDVXUHV XVLQJ KLJKIUHTXHQF\
XOWUDVRQLF ZDYHV ZKLFK DUH VHQW E\ D WUDQVPLWWHU
WUDQVPLWWHU  UHG FRORXU  DQGVXEVHTXHQWO\ UHFHLYHGE\ D
UHFHLYHU UHFHLYHUEOXHFRORXU 7KHVHQVRUFDQGHWHUPLQH 
WKHWLPHRIUHIOHFWLRQIURPDQREMHFWRUZDOODQGWKHUHE\
GHWHUPLQH WKH GLVWDQFH IURP WKH PHDVXUHG REMHFW ,W XVHV )LJXUH &RQLFDOLPDJHRIWKHXOWUDVRQLFVHQVRUPHDVXUHPHQW>@
WKH SK\VLFDO SURSHUWLHV RI VRXQG SURSDJDWLRQ LQ WKH DLU
7KH VHQVRU FDQ PHDVXUH D GLVWDQFH IURP  FP WR $OWKRXJK WKLV PHWKRG RI VHQVLQJ WKH HQYLURQPHQW LV
DSSUR[LPDWHO\FP7KHVHQVRUFDQEHVHWWRPHDVXUH DGYDQWDJHRXV LWV PHDVXUHPHQWV FDXVH GLVWRUWLRQV
LQ LQFKHV EXW ZH XQGHUVWDQGDEO\ GLG QRW XVHWKLVRSWLRQ HVSHFLDOO\UHJDUGLQJWKHFRUQHUVRIWKHZDOOV )LJ 
,WV DFFXUDF\ PXVW DOVR EH WDNHQ LQWR DFFRXQW ZLWK D & &RPSDULVRQRIVHQVRUV±LQIUDUHGDQGXOWUDVRQLF
GHYLDWLRQRIFHQWLPHWUH RUPRUH 7KHIDFWWKDWWKH
VHQVRUPHDVXUHVIURPWKUHHFHQWLPHWUHVLQRXURSLQLRQLV $Q LQIUDUHG VHQVRU MXVW OLNH DQ XOWUDVRQLF VHQVRU
UHODWHGWRKRZWKHVHQVRUZRUNV6LQFHLWUHFHLYHVUHIOHFWHG PHDVXUHV WKH GLVWDQFH E\ UHIOHFWLRQ IURP DQ REMHFW RU D
VRXQG WKH UHFHLYHU ZKLFK LV SDUWRI WKH VHQVRU LV QRWLQ ZDOO+RZHYHULWZRUNVXVLQJLQIUDUHGOLJKWDVRSSRVHGWR
WKH YHU\ IURQW SDUW DQG SUHFLVHO\ WKHVH  FP ZRXOG XOWUDVRXQG ZDYHV 7KH PHDVXUHPHQW WDNHV SODFH LQ
FRUUHVSRQGWRWKHGLVWDQFHZKHUHWKHVHQVRULVORFDWHGRI 3UR[LPLW\PRGHZKHUHWKHVHQVRUVHQGVDQGUHFHLYHVOLJKW
FRXUVH\RXKDYHWRWDNHLQWRDFFRXQWWKHGHYLDWLRQVLQFH ZDYHVWKDWDUHUHIOHFWHGIURPDQREVWDFOH7KHVHQVRUGRHV
WKH VHQVRU LV YHU\ OLNHO\ QRW EXLOW WR PHDVXUH VXFK VPDOO QRW ZRUN LQ PHWULF XQLWV EXW LQ YDOXHV IURP  WR 
GLVWDQFHV2WKHUZLVHLIWKHGLVWDQFHLVWRRODUJHLH ZKHUH  PHDQV WKDW WKH REMHFWREVWDFOH LV YHU\ FORVH DQG
FP WKH VHQVRU LV QRW DEOH WR UHFRUG WKH GLVWDQFH DQG  PHDQV WKDW LW LV YHU\ IDU 7KH PD[LPXP PHDVXUDEOH
GLVSOD\V D YDOXH RI  FP ZKLFK PHDQV WKDW LW FDQQRW YDOXHLVDSSUR[LPDWHO\FPDOWKRXJKVRPHZKHUHXSWR
PHDVXUHWKHZDOORUREVWDFOH>@>@  FP LV UHSRUWHG %RWK VHQVRUV FDQ PHDVXUH GLVWDQFH
EXW WKH\ GLIIHU LQ WKH UDQJH DQG DFFXUDF\ RI WKH
)RUWKHEHVWPHDVXUHPHQWUHVXOWVLWLVQHFHVVDU\WKDWWKH
PHDVXUHPHQWV>@
VHQVRU LWV ORFDWLRQ DQG HQYLURQPHQW PHHW VHYHUDO
FRQGLWLRQV $GHWDLOHGGHVFULSWLRQRIERWKVHQVRUVFDQEHIRXQGLQ
7DEOH  ZKHUH ZH FDQ VHH D FRPSDULVRQ DFFRUGLQJ WR
• WKHVHQVRUVKRXOGEHSODFHGSHUSHQGLFXODUWRWKH WKHVH SDUDPHWHUV 0HDVXUHPHQW UDQJH 0HDVXULQJ DQJOH
VFDQQHGVXUIDFHRWKHUZLVHZHFDQJHWGLVWRUWHG $FFXUDF\DQG6HQVLWLYLW\WRGLVWUDFWLRQV
UHVXOWV
• WKHVHQVRUPXVWEHSURSHUO\DQGILUPO\IL[HGRQ 7$%/(, &203$5,6212)8/75$621,&$1',1)5$5('
WKHURERWRURWKHUREMHFWVRWKDWLWGRHVQRWSLFN 6(16256>@

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 410

 8OWUDVRQLFVHQVRU ,QIUDUHGVHQVRU QRW GLVFXVV LW LQ GHWDLO LQ WKH ZRUN 7KH LQVWDOODWLRQ WRRN
0HDVXUHPHQWUDQJH )URPWRFP )URP  WR    SODFH RQ D FRPSXWHU ZLWK WKH :LQGRZV  RSHUDWLQJ
FP V\VWHP>@>@
0HDVXULQJDQJOH $FRQHZLWKDQDQJOH &RQH ZLWK DQ DQJOH :H XVH WKH (FOLSVH GHYHORSPHQW HQYLURQPHQW ZKLFK
RIDSSUR[ RIDSSUR[ƒ
ZH DOVR QHHG WR LQVWDOO ,Q WKH LQVWDOODWLRQ ZH VHOHFWHG
$FFXUDF\ 5HODWLYHO\ DFFXUDWH 5HODWLYHO\ DFFXUDWH
EXW DFFXUDF\ EXW DFFXUDF\ (FOLSVH ,'( IRU -DYD 'HYHORSHUV >@ >@ >@ :H
GHFUHDVHV ZLWK GHFUHDVHV ZLWK DGGHG WKH /H-26 (9 SOXJLQ ZKLFK LV QRW DYDLODEOH RQ
GLVWDQFH LQFUHDVLQJGLVWDQFH WKH RIILFLDO ZHEVLWH EXW FDQ EH GRZQORDGHG IURP WKH
PHDVXUHPHQW DQJOH (FOLSVH 0DUNHWSODFH7KH ODVW VWHS LVWR ZULWH WKHFRUUHFW
DQG WHPSHUDWXUH DQG
SUHVVXUHFRQGLWLRQV ,3DGGUHVVDFFRUGLQJWRWKH(9FXEHLQWKHSUHIHUHQFHVRI
6HQVLWLYLW\ WR 6HQVLWLYH WR VHQVLWLYH WR VWURQJ
WKH(FOLSVHSURJUDP>@>@
GLVWUDFWLRQV WHPSHUDWXUH DQG OLJKW VRXUFHV ZLWK
SUHVVXUH KLJKLQIUDUHGFRQWHQW % %XLOGLQJDURERW
DV ZHOO DV WKH FRORXU
,W LV DOVR VHQVLWLYH WR
DQG W\SH RI
7KH EDVH RI WKH URERW ZDV DVVHPEOHG DFFRUGLQJ WR WKH
RWKHU GHYLFHV XVLQJ 'ULYLQJ %DVH LQVWUXFWLRQV IRXQG LQ WKH LQVWUXFWLRQV IRU
WKH VDPH IUHTXHQF\ REVWDFOHVZDOOV
ZKLFK FDQ FDXVH EXLOGLQJ WKH URERW LQ /HJR 0LQGVWRUPV (GXFDWLRQ (9
SUREOHPV 7HDFKHUV (GLWLRQ $V IRU PRYHPHQW RXU URERW XVHV WZR
 ODUJH PRWRUV ZLWK UXEEHU ZKHHOV LQ WKH IURQW DQG IRU
VWDELOLW\ LW KDV D URWDWLQJ PHWDO EDOO LQ D EHDULQJ DW WKH
7KHWDEOHVKRZVWKDWWKHXOWUDVRQLFVHQVRUKDVDODUJHU EDFNZKLFKVHUYHVDVDWKLUGVXSSRUWSRLQWDZKHHO7KH
UDQJH DQG PHDVXULQJ DQJOH ,Q WHUPV RI DFFXUDF\ ERWK VHFRQGVWHSZDVWRDWWDFKWKHPLGGOHPRWRUWRWKHIURQWRI
VHQVRUVDUHGHVFULEHGDVUHODWLYHO\DFFXUDWH WKHFXEHVRWKDWWKHXOWUDVRQLFVHQVRUFRXOGEHSODFHGRQLW
ZKLFKFRXOGQRWEHWRRKLJKGXHWRWKHKHLJKWRIWKHPD]H
,,, ,03/(0(17$7,212)(19,5210(17$/ DQGDWWKHVDPHWLPHWKHPRWRUFRXOGQRWEHORZEHFDXVH
0$33,1*86,1*/(*20,1'672506$1' RI LWV FRQQHFWLQJ FDEOH ZKLFK PXVW EH SOXJJHG LQWR WKH
6(16256 (9FXEHDQGFDQQRWWRXFKWKHHDUWK )LJ 
,Q WKLV VHFWLRQ ZH GHVFULEH WKH SURFHGXUH
LPSOHPHQWDWLRQDQGUHVXOWVRILQGLYLGXDOWDVNVIRUVROYLQJ
WKH ORFDOL]DWLRQ DQG PDSSLQJ RI WKH HQYLURQPHQW E\ D
URERW 7KH PDLQ JRDO RI WKH ZRUN LV PDSSLQJ XVLQJ WKH
/(*20LQGVWRUPVURERWXVLQJDQXOWUDVRQLFDQGLQIUDUHG
VHQVRU:HGLYLGHGWKLVPDLQJRDOLQWRVHYHUDOVXEJRDOV
 'HVLJQ D FRQVWUXFWLRQ PRGHO RI D URERW IURP WKH
/HJR 0LQGVWRUPV NLW IRU ORFDWLRQ GHVLJQ PDSSLQJ DQG
URXWHSODQQLQJ
'HVLJQDWDVNIRUGHWHFWLQJWKHSRVLWLRQRIWKHURERW 
DQGPDSSLQJWKHHQYLURQPHQW )LJXUH %XLOWURERW

$ +DUGZDUHDQGVRIWZDUHUHTXLUHPHQWVIRUWKH
LPSOHPHQWDWLRQRIWKHVROXWLRQGHVLJQ & 0D]HFRQVWUXFWLRQ
)RU WKH LPSOHPHQWDWLRQ RI WKH GHVLJQ RI WKH PDSSLQJ )RU WKH PDSSLQJ RI WKH HQYLURQPHQW E\ WKH URERW ZH
DQG URXWH SODQQLQJ VROXWLRQ ZH FKRVH WKH (FOLSVH FUHDWHG D VLPSOH PD]H LQ WKH VKDSH RI D UHFWDQJOH ZLWK
SURJUDPPLQJHQYLURQPHQWDQGXVHGWKHOH-26H[WHQVLRQ SURWUXVLRQVFRQVLVWLQJRIFDUGERDUGZDOOVWKHWRWDOVL]HLV
OH-26(93OXJ,Q:HFKRVH/H-26PDLQO\IRULWVZLGH [FPDQGWKHKHLJKWRIWKHZDOOVLVFP$OOWKH
VXSSRUW RI SURJUDPPLQJ SDFNDJHV DQG JRRG FRPPXQLW\ FRUQHUVRIWKHZDOOVDUHRUGHJUHHVGHSHQGLQJRQ
VXSSRUW ZKLFK PDNHV SURJUDPPLQJ DQG WURXEOHVKRRWLQJ ZKHWKHULWLVDQLQQHURURXWHUFRUQHUDQGWKHJDSEHWZHHQ
HDVLHU 6RIWZDUH UHTXLUHPHQWV DUH LQVWDOODWLRQ RI -'. WKHZDOOVRIWKHSURWUXVLRQVLVZLGHHQRXJKDWOHDVWFP
SDFNDJHLQVWDOODWLRQRIOH-26ILUPZDUHDQGLQVWDOODWLRQRI IRU WKH URERW WR HQWHU WXUQ DQG H[LW WKHP :H DGGHG DQ
(FOLSVH SURJUDPPLQJ HQYLURQPHQW +DUGZDUH REVWDFOHWRWKHPD]HVRWKHILUVWPD]HLVVKDSHGOLNHWKH
UHTXLUHPHQWV DUH D 86% :L)L DGDSWHU DQG D PLFUR6' OHWWHU&DQGWKHVHFRQGLVVKDSHGOLNHWKHOHWWHU+
FDUG>@>@
' 0DSSLQJWKHHQYLURQPHQW
6LQFH WKH OH-26 LV EDVHG RQ WKH -DYD SURJUDPPLQJ
ODQJXDJH ZH KDG WR LQVWDOO WKH -'. SDFNDJH -DYD )LUVWZHIXQFWLRQDOL]HGWKHUHDGLQJRIWKHHQYLURQPHQW
'HYHORSPHQW .LW  ,W LV QHFHVVDU\ WR SD\ DWWHQWLRQ WR WKH E\ WKH VHQVRU ,WV VXEVHTXHQW OLVWLQJ RI PHDVXUHPHQWV
XSWRGDWH YHUVLRQ RI WKH -'. ZKHQ XVLQJ WKH ODWHVW ZKLFK DUH WKH FRRUGLQDWHV RI WKH ; DQG < SRLQWV DUH
YHUVLRQWKHUH PD\ EH FRQIOLFWV ZLWK RWKHUROGHU VRIWZDUH ZULWWHQ WR WKH -3DQHO LQ WKH *8, 7KH GLPHQVLRQV RI WKH
WKDW ZH XVH LQ RXU ZRUN :H LQVWDOOHG WKH MGNX -3DQHO DUH [ SRLQWV ZLWK WKH IDFW WKDW WKH LQLWLDO
ZLQGRZVLH[H YHUVLRQ ZKLFK LV IUHHO\ DYDLODEOH DW SRVLWLRQSRLQWIRUWKHURERWLV>@LHWKHFHQWUHRI
KWWSVZZZRUDFOHFRPMDYDWHFKQRORJLHVMDYDVHMDYDVH WKH-SDQHOEXWWKHURERWGRHVQRWNQRZZKHUHLWLVLQWKH
DUFKLYHGRZQORDGVKWPO LQ WKH -'. SODWIRUP DUFKLYH PD]H LW RQO\ KDV WKH JLYHQ SRLQW ZKHUH LW VWDUWHG
YHUVLRQV 1H[W ZH LQVWDOOHG OH-26 RQ WKH FRPSXWHU ZH PHDVXULQJ ,WV FRRUGLQDWHV ZKLFK DUH VWRUHG GXULQJ
XVHGWKHOH-26B(9BEHWDBZLQBVHWXSH[HYHUVLRQ LQLWLDOL]DWLRQ DUH >@ 7KH OLVW RI FRRUGLQDWHV LV
7KHHQWLUHVWHSE\VWHSLQVWDOODWLRQSURFHVVLVDYDLODEOHRQ FDOFXODWHG IURP WKH GDWD SURYLGHG E\ WKH VHQVRU 7KLV LV
WKH OH-26 ZHEVLWH RU LQ YLGHRV RQ <RX7XEH VR ZH ZLOO WKH DQJOH DW ZKLFK WKH VHQVRU ZDV WXUQHG GXULQJ WKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 411

PHDVXUHPHQW DQG WKH PHDVXUHGGLVWDQFH WKDW LW PHDVXUHG
DW WKDW DQJOH 8VLQJ WULJRQRPHWU\ VLQH DQG FRVLQH
FDOFXODWLRQRIWKHK\SRWHQXVHZHFDQPDUNDSRLQWRQWKH
PDS-3DQHOL7KLVLVKRZ ZHJUDGXDOO\FUHDWHDSDWKRI
SRLQWV WKDW DUH D UHIOHFWLRQ RI WKH UHDO HQYLURQPHQW 7KH
VHQVRU FUHDWHV  PHDVXUHPHQWV LQ D GHJUHH IUDPH
ZLWKLQ RQH VFDQ 7KLV SURFHVV WDNHV DERXW  VHFRQGV
:KHQPHDVXULQJWKHHQYLURQPHQWZHQRWHGPHDVXUHPHQW
GHYLDWLRQV7KHXOWUDVRQLFVHQVRUVFDQVLQWKHVKDSHRIDQ
H[SDQGLQJFRQHZKLFKFDQUHVXOWLQWKHGLVWRUWLRQRIVRPH
PHDVXUHPHQWV PDLQO\ WKH PHDVXUHPHQW RI FRUQHUV LQ D
PD]H>@
( 0RYHPHQWRIWKHURERWLQWKHPD]H
7KH PRYHPHQW RI WKH URERW LQ WKH PD]H LV SHUIRUPHG
UDQGRPO\DFFRUGLQJWRWKHDQJOHVHOHFWHGIURPWKHVHW 
    ZKHQ PHDVXUHG E\ WKH VHQVRU 7KH URERW
WKXV UDQGRPO\ FKRRVHV ZKHWKHU WR JR XS GRZQ ULJKW RU
OHIWIURPWKHRULJLQDOORFDWLRQDIWHUWKHILUVWPHDVXUHPHQW
7KHQXPEHURIUHSHWLWLRQVLVVHWWRDQGFDQEHFKDQJHG
DV QHHGHG ,Q WHUPV RI WLPH WKLV PHDVXUHPHQW WDNHV
DSSUR[LPDWHO\  PLQXWHV GHSHQGLQJ RQ WKH VL]H RI WKH
PD]H6WDUWRIPHDVXUHPHQW±DGHJUHHURWDWLRQZLWK 
WKHVHQVRULVPDGHIURPWKHRULJLQDOSRVLWLRQWKHURERWKDV
)LJXUH )ORZFKDUWGLDJUDPIRUHQYLURQPHQWPDSSLQJE\WKHURERW
HQWHUHGFRRUGLQDWHV>@DQGIURPWKHUHLWPDNHVWKHILUVW
PRYHPHQW ZLWK WKH ZKHHOV +H FKRRVHV D UDQGRP DQJOH
'HYLDWLRQVDUHPRUHFRPPRQZKHQWKHURERWLVPRYLQJ
ZULWHV LW RQ KLV GLVSOD\ DQG PRYHV XQWLO KH PHDVXUHV DQ
DQGVFDQQLQJWKHHQYLURQPHQWDWWKHVDPHWLPH,Q)LJXUH
REVWDFOHFPDZD\$IWHUHDFKPRYHPHQWLWUHFRUGVWKH
ZHFDQVHHDFRPSDULVRQRIWKHPHDVXUHPHQWUHVXOWVRI
GLVSODFHPHQW DORQJ WKH ; DQG < D[LV 7KHVH SRLQWV DUH
RXUURERW SLFWXUHRQWKHOHIW DQGWKH1;7URERW SLFWXUH
UHFRUGHGDVUHGSRLQWVRQWKHUHVXOWLQJPDS WKHILUVWRQHLV
RQ WKH ULJKW  IURP WKH DUWLFOH 0DS &RQVWUXFWLRQ $QG
DOZD\V LQ WKH PLGGOH  $IWHU PHDVXULQJ WKH REVWDFOH LQ
/RFDOL]DWLRQ 8VLQJ /(*2 0LQGVWRUPV 1;7E\ - 6LQJK
IURQWRILWLWVWRSVDQGUHSHDWVWKHPHDVXUHPHQW ZLWKWKH
DQG 3 %HGL ZKHUH WKH URERW UHFRUGHG LWV VXUURXQGLQJV
VHQVRU 7KHQ LW SLFNV D UDQGRP DQJOH DJDLQ JRHV LQ WKDW
WKHQ PRYHG IURP LWV LQLWLDO SRVLWLRQ WR WKH OHIW ZKHUH LW
GLUHFWLRQ XQWLO LW KLWV DQ REVWDFOH DQG PDNHV D
DJDLQ PHDVXUHG ZLWK WKH VHQVRU ,Q WKHVH WZR
PHDVXUHPHQW ,Q WKLV ZD\ WKH URERW VFDQV WKH
PHDVXUHPHQWV ZH FDQ VHH KRZ WKH VHQVRUV URXQG WKH
HQYLURQPHQW SLHFH E\ SLHFH  WKH PD]H LQ ZKLFK LW ZDV
HGJHV DFFRUGLQJ WR WKH VKDSH RI WKH FRQH GXULQJ WKHLU
LQVHUWHG DQG JUDGXDOO\ GUDZV WKH PDS $W WKH HQG RI WKH
PHDVXUHPHQWV 7KHVH DQG RWKHU GHYLDWLRQV FDQQRW EH
PHDVXUHPHQWVZHJHWDPDSZLWKKLVPHDVXUHPHQWVLQWKH
SUHYHQWHG E\ PHDVXUHPHQW DV WKH VHQVRU LWVHOI PHDVXUHV
IRUPRISRLQWV,IWKHURERWGHWHFWVDQREVWDFOHLQFRUUHFWO\
LW :H FDQ VD\ WKDW WKH UHVXOWV DUH FRPSDUDEOH LQ WKH
DQGRUKLWVDZDOORUREVWDFOHWKHPHDVXUHPHQWLVLQFRUUHFW
VFLHQWLILF ZRUN WKH\ XVHG VHYHUDO PHDVXUHPHQWV IURP WKH
DQGPXVWEHGRQHDJDLQ:KHQFRUUHFWHGWKHURERWZLOO
VDPH SODFH WKHUHIRUH PXOWLSOH PHDVXUHG SRLQWV FDQ EH
QR ORQJHU KDYH WKH FRUUHFWSRVLWLRQ FRRUGLQDWHV UHFRUGHG
VHHQZHDOVRPHDVXUHGWKHHQYLURQPHQWLQWKHVDPHZD\
7KHVH HUURQHRXV PHDVXUHPHQWV DOVR KDSSHQ GXH WR WKH
>@>@
LQIOXHQFH RI ZKHHOV DQG PRWRUV ZKLFK KDYH VOLJKW
GHYLDWLRQV LQ WUDFWLRQ ZLWK WKH VXUIDFH ,Q )LJXUH  \RX
FDQ VHH WKH IORZFKDUW GLDJUDP IRU WKH PDSSLQJ RI WKH
HQYLURQPHQWE\WKHURERW


)LJXUH 5RERWPRYHPHQWZLWKHQYLURQPHQWDOVHQVLQJ>@

'XULQJWKHPHDVXUHPHQWVWKHURERWVHQWXVGDWDLQWKH
IRUPRI$QJOHDQJOHVDQGPHDVXUHG5DQJHGLVWDQFHVLQWKH
JLYHQDQJOHVWRWKHFRQVROH$IWHUPHDVXULQJWKHYDOXHVLW
LV WXUQHG DURXQG  WKH WXUQDURXQG FRPPDQG WKH ODVW
PHDVXUHPHQWZDVWKHVHQVRUWXUQHGEDFNZDUGV WKHURERW
PRYHV WR D QHZ SRVLWLRQ  PRYH ZULWHV GRZQ WKLV QHZ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 412

SRVLWLRQ  1HZ SRVLWLRQ [ DQG \ ,W WKHQ H[HFXWHV WKH
ORRNDURXQG FRPPDQG QXP UHDGLQJV ZKLFK VD\V 
UHDGLQJV ZKLFK PHDQV  UHDGLQJ HYHU\  GHJUHHV 
GHJUHHV  ,W DOVR PHDVXUHV  GHJUHHV LQ D URZ LH
H[HFXWHV D WXUQDURXQG DQG ORRNDURXQG FRPPDQG 7KH
YDOXHV DUH VHQW DW RQFH DIWHU HDFK IURQW DQG EDFN
PHDVXUHPHQW DQG URERW PRYHPHQW 7KLV LV KRZ
FRPPXQLFDWLRQ ZLWK WKH URERW LV KDQGOHG ZKLOH WKH 
SURJUDPLVUXQQLQJ )LJXUH &RPSDULVRQRI,5DQG86VHQVRUUHFWDQJOH
7KLVVHQGLQJRIGDWDRYHUWKHQHWZRUNLVKDQGOHGXVLQJ
WKH 'DWD2XWSXW6WUHDP FODVVZKLFK DOORZV DQ DSSOLFDWLRQ 7KHWKLUGPHDVXUHPHQWWKDWZHFDQVHHLQ)LJXUHLVD
WR ZULWH VLPSOH GDWD W\SHV WR DQ RXWSXW VWUHDP RQ WKH FRPSDULVRQRIVHQVRUVLQD&VKDSHGPD]H 0D]H& 2Q
QHWZRUN6XEVHTXHQWO\RXUFOLHQWDSSOLFDWLRQFDQXVHWKH WKH UHVXOWRI WKH ,5 VHQVRU PHDVXUHPHQW ZH VHH PLVVLQJ
'DWD,QSXW6WUHDPWRORDGWKHGDWD7KLVFRPPXQLFDWLRQLV YDOXHV DQG WKHUHIRUH LQFRPSOHWH PHDVXUHPHQWV 7KLV
XVHG EHWZHHQ WKH URERW  VHUYHU DQG WKH FOLHQW -DYD SUREOHP FRXOG EH WUHDWHG E\ ORQJHU PRYHPHQW DQG
DSSOLFDWLRQ )RU VXFFHVVIXO FRPPXQLFDWLRQ ERWK SDUWLHV PHDVXUHPHQWLQWKHPD]HEXWLWZLOOQRWVROYHWKHSUREOHP
PXVW ZRUN RQ WKH VDPH SRUW LQ RXU FDVH LW LV SRUW  FRPSOHWHO\ +RZHYHU WKH QXPEHU RI PHDVXUHPHQWV LV
DQGRQWKHVDPH,3DGGUHVV OLPLWHG EHFDXVH GXULQJ D ORQJHU PHDVXUHPHQW LQ WKH
PD]H GHYLDWLRQV RI WKH URERW V PRYHPHQW EHJLQ WR VKRZ
,9 &203$5,6212),1)5$5('$1' DQGWKHPHDVXUHPHQWVDUHVLJQLILFDQWO\VKLIWHGIURPWKHLU
8/75$6281'6(16255(68/76,1 UHDO SRVLWLRQ ZKLFK GLVWRUWV WKH UHVXOWLQJ PDS ,Q WKH
(19,5210(17$/0$33,1* UHVXOWLQJPDSRIWKH86VHQVRULQ)LJXUHZHFDQVHHWKH
FRQWRXUVRIZDOOVDQGREVWDFOHVGHVSLWHWKHPHDVXUHPHQW
$IWHU WKH PHDVXUHPHQWV ZH FDQ HYDOXDWH WKDW WKH ,5 GHYLDWLRQVZKLFKPHDQVWKDWWKH86VHQVRULVEHWWHUZKHQ
VHQVRU PHDVXUHV PRUH DFFXUDWHO\ DW VPDOO GLVWDQFHV XVHGLQODUJHUVSDFHV
EHFDXVH LW KDV D PHDVXUHPHQW SUREOHP DW GLVWDQFHV RYHU
FP7KHUHIRUHHYHQDODUJHSDUWRIDQJOHPHDVXUHPHQWV
GRHV QRW KDYH D ILQDO YDOXH PHDVXUHPHQW EH\RQG WKH
VFRSHRIPHDVXUDELOLW\ :HPHDVXUHGWKHORQJHVWGLVWDQFH
RIFP,QWKHILUVWFRPSDULVRQLQWKHVTXDUHPD]H )LJ
  ZH VHH WKH DFFXUDF\ RI WKH ,5 VHQVRU YHUVXV WKH 86
VHQVRU+RZHYHUWKH86VHQVRULVVXLWDEOHIRUPHDVXULQJ
ORQJHU GLVWDQFHV XS WR  FP  ZKLFK LV D JUHDW
DGYDQWDJH7KHUHVXOWVDUHDOVRDIIHFWHGE\WKHPRYHPHQW
RI WKH URERW ZKHUH HYHQ D VPDOO LQDFFXUDF\ LQ WKH
PRYHPHQW RI WKH ZKHHOV ZLOO DIIHFW WKH HQWLUH 
PHDVXUHPHQW 7KH PHDVXUHPHQW UHVXOWV ZHUH SURFHVVHG )LJXUH &RPSDULVRQRI/DE\ULQWK&,5DQG86VHQVRU
ZLWK RQH RU PRUH FKDQJHV LQ WKH GLUHFWLRQ RI WKH VHQVRU
UHGSRLQWVRQ WKH PDS  6RPHWLPHV WKH VHQVRU PHDVXUHG 'XULQJ VHQVRU PHDVXUHPHQWV ZH WULHG SODFLQJ WKH
DQ REVWDFOH  FP DZD\ LPPHGLDWHO\ DIWHU WXUQLQJ WR D VHQVRU YHUWLFDOO\ WXUQHG E\  GHJUHHV  7KLV FKDQJH
QHZGLUHFWLRQDQGGLGWKHPHDVXUHPHQWDJDLQWKDW VZK\ EURXJKW LPSURYHPHQWV WR WKH 86 VHQVRU PHDVXUHPHQW
VRPHWLPHVWKHUHGSRLQWVRIWKHURERW VPRYHPHQWRYHUODS UHVXOWV7KHUHDVRQLVWKHSODFHPHQWRIWKHWUDQVPLWWHUDQG
UHFHLYHULQWKHVHQVRUYHUWLFDOO\$VZHFDQVHHLQ)LJXUH
 WKH UHFHLYHU RQ WKH VHQVRU LV RQ WKH OHIW DQG WKH
WUDQVPLWWHU LV RQ WKH ULJKW UHVXOWLQJ LQ DQ LQDFFXUDWH
PHDVXUHPHQW RQ WKH OHIW VLGH RI WKH VHQVRU DUUD\ %\
FKDQJLQJ WKH SRVLWLRQ RI WKH VHQVRU WKH PHDVXUHPHQW RI
WKH REVWDFOH LV SHUIRUPHG LQ VXFK D ZD\ WKDW WKH
WUDQVPLWWHUVHQGVDQGWKHUHFHLYHUUHFHLYHVWKHVLJQDOIURP
 D VPDOOHU VXUIDFH DUHD DQG IURP WKH VDPH SODFH ZKLFK
LPSURYHVWKHGLVWDQFHPHDVXUHPHQW>@
)LJXUH &RPSDULVRQRI,5DQG86VHQVRU[P

,Q )LJXUH  ZH FDQ VHH KRZ WKH DFFXUDF\ RI WKH 9 &21&/86,21
PHDVXUHPHQWV FKDQJHV ZLWK ODUJHU PD]HV EXW ZLWK ERWK 7KHPHQWLRQHGZRUNDLPHGWREXLOGDURERWZKRVHWDVN
ZH VHH WKH FRQWRXUV RI WKH PD]H LQ WKH VKDSH RI D LV WR VHQVH WKH HQYLURQPHQW DQG PRYH LQ LW :H
UHFWDQJOH 7KH EODFN SRLQWV LQ WKH FHQWUH RI WKH PDS DUH SURJUDPPHGWKHVHQVRUVHQVLQJDQGUDQGRPPRYHPHQWRI
HUURQHRXVPHDVXUHPHQWVZKLFK ZHUHDOVRFDXVHGE\IRU WKHURERWVRWKDWWKHURERWFRXOGZDONLQD PD]H ZLWKRXW
H[DPSOHWKHVHQVLQJE\ZKLFKWKHVHQVRULVFRQQHFWHGWR EUHDNLQJLQWRZDOOVLWZRXOGVWRSEHIRUHDQREVWDFOHDQG
WKH(9FXEH,QWKLVPHDVXUHPHQWLWFDQEHVHHQWKDWWKH DW WKH VDPH WLPH LW FRXOG VFDQ WKH HQYLURQPHQW DURXQG
86VHQVRULPSURYHVWKHUHVXOWVRIPHDVXUHPHQWVDWORQJHU LWVHOIZLWKDQXOWUDVRQLFVHQVRUDQGUHFRUGLWLQWKHIRUPRI
GLVWDQFHV SRLQWV RQ D EOLQG PDS 7KH UHVXOW RI WKH ZRUN RI WKH
SURJUDP DQG WKH URERW ZDV D PDS ZLWK PHDVXUHG SRLQWV
WKDW UHSUHVHQWHG WKH ZDOOV DQG SRLQWV WKDW ZHUH WKH
SRVLWLRQVRIWKHURERW$OWKRXJKWKHURERWGLGQRWNQRZLQ
ZKLFKPD]HLWZDVSODFHGLWGLGNQRZKRZLWPRYHGIURP

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 413

 LWV RULJLQDO SRVLWLRQ 7KHVH SRLQWV KDG WR EH WUHDWHG IURP >@ /(*2 0LQGVWRUPV (9  3RXåtYDWHĐVNi SUtUXþND
HUURQHRXV VHQVRU PHDVXUHPHQWV WR FUHDWH D PDS RI WKH >@  VWU 'RVWXSQp ]
KWWSVZZZOHJRFRPFGQFVVHWDVVHWVEOWEHIGFHIF/0
PD]H 68VHUB*XLGHB/(*2B0,1'672506B(9BB7DEOHWB(186S
2XU ILQGLQJV RI HQYLURQPHQWDO PDSSLQJ XVLQJ /HJR GI
0LQGVWRUPVVHQVRUVVKRZWKDW >@ /(*20LQGVWRUPV(GXFDWLRQ(93RXåtYDWHĐVNiSUtUXþND
• 7KH LQIUDUHG VHQVRU LV PRUH DFFXUDWH WKDQ WKH >@ 'RVWXSQp ] KWWSVHGXFDWLRQOHJRFRPHQ
JESURGXFWUHVRXUFHVPLQGVWRUPVHYGRZQORDGVXVHUJXLGH
XOWUDVRQLF VHQVRU ZKHQ PHDVXULQJ VPDOOHU
>@ P\FDUGLFWLRQDU\ 8OWUDVRQLF VHQVRU >@ 'RVWXSQp ]
GLVWDQFHVXSWRFP KWWSVZZZP\FDUGLFWLRQDU\FRPGULYHUDVVLVWDQFH
• 7KH XOWUDVRQLF VHQVRU LV VXLWDEOH IRU PDSSLQJ V\VWHPVXOWUDVRQLFVHQVRUKWPO
ODUJHUVSDFHV XSWRFP  >@ 9 0D]]DUL  8OWUDVRXQG VHQVRU" KLJKTXDOLW\ XOWUDVRXQG
VHQVRUV DYDLODEOH QRZ>@ 'RVWXSQp ]
• :KHQ PHDVXULQJ ZLWK DQ XOWUDVRQLF VHQVRU LW LV KWWSVZZZJHQHUDWLRQURERWVFRPEORJHQXOWUDVRXQGVHQVRU
QHFHVVDU\WRXVHWKHFRUUHFWVKDSHDQGVXUIDFHRI KLJKTXDOLW\XOWUDVRXQGVHQVRUVDYDLODEOHQRZ
REVWDFOHVDQGZDOOV >@ /(*2 0LQGVWRUPV (9 8VLQJ WKH 8OWUDVRQLF 6HQVRU
• 8OWUDVRQLF VHQVRUUHDGLQJV DFKLHYH EHWWHU UHVXOWV >@ 'RVWXSQp ] KWWSVHYKHOS
ZKHQWKHVHQVRULVWXUQHGYHUWLFDOO\ RQOLQHDSLHGXFDWLRQOHJRFRP5HWDLOHQ
XVSDJHKWPO"3DWK HGLWRU)8VLQJ6HQVRUVB8OWUDVRQLFKWPO
• 0DSSLQJE\DURERWLVDIIHFWHGE\WKHPRYHPHQW >@ 7 %LQGHU  6SDFH  WKH XOWUDVRXQG RI VLOHQFH >@
RI WKH URERW HYHQ VPDOO HUURUV LQ WKH IRUP RI D 'RVWXSQp ] KWWSVVRQRJUDSK\FRPEORJVSDFHXOWUDVRXQG
GHYLDWLRQRIDIHZGHJUHHVLQYDOLGDWHWKHUHVXOWV VLOHQFH
)RU IXUWKHU GHYHORSPHQW RI WKH URERW V PRYHPHQW DQG >@ (9 '(9 1HWZRUNLQJ (9'HY>@ 'RVWXSQp ]
KWWSVZZZHYGHYRUJGRFVQHWZRUNLQJ
VSDFH VHQVLQJ LW ZRXOG EH SRVVLEOH WR LQFOXGH LQ WKH
>@ 52%27648$5(  (9 DQG 1;7 'LIIHUHQFH DQG
PHDVXUHPHQWVDJ\URVFRSLFVHQVRUIRUPRUHDFFXUDWHURERW &RPSDWLELOLW\>@ 'RVWXSQp ]
PRYHPHQW D VHFRQG XOWUDVRQLF VHQVRU RU D FDPHUD IRU KWWSURERWVTXDUHFRPHYQ[WFRPSDWLELOLW\
VHQVLQJPRYHPHQWRUXVHDPRUHDFFXUDWHVHQVRUOLGDU >@ * )HUUDUL ' /DYHUGH  3URJUDPPLQJ /HJR 0LQGVWRUPV
ZLWK -DYD >RQOLQH@ 9\G (OVHYLHU 6FLHQFH  VWU>@
$&.12:/('*(0(17 ,6%1  'RVWXSQp ]
KWWSVZZZJRRJOHVNERRNVHGLWLRQ3URJUDPPLQJB/HJRB0LQGVW
Ä7KH ZRUN ZDV IXQGHGE\ WKH JUDQW.(*$ 8&0 RUPVBZLWKB-DYDVIZ=5V\I$&"KO VN JESY 
0RGHUQ7HFKQRORJLHVDQG,QQRYDWLRQVLQ1HWZRUN >@ /H-26 -DYD IRU /HJR 0LQGVWRUPV >@ 'RVWXSQp ]
6HFXULW\ (GXFDWLRQ DQG E\ WKH JUDQW .(*$ 8&0 KWWSVOHMRVVRXUFHIRUJHLRHYSKS
 'HYHORSPHQW RI DQ LQWHUDFWLYH HFRXUVH XVLQJ >@ 0.DUFK%XLOGDQG3URJUDP<RXU2ZQ/(*20LQGVWRUPV
³60$57´ WHFKQRORJLHV IRU WKH GHYHORSPHQW RI (9 5RERWV 3HDUVRQ (GXFDWLRQ >@ 'RVWXSQp ]
DOJRULWKPLFWKLQNLQJDQGSURJUDPPLQJVNLOOV³ KWWSVERRNVJRRJOHVNERRNV"LG 7*E%4$$4%$- GT OHMRVO
HJRPLQGVWRUPV KO VN VRXUFH JEVBQDYOLQNVBV
5()(5(1&(6 >@ :%XUQHWW$OWHUQDWLYH3URJUDPPLQJ/DQJXDJHVIRU/(*2
0,1'672506 ± /(*2 (QJLQHHULQJ /(*2 (QJLQHHULQJ
>@ 3 5DQJDQDWKDQ 5 6FKXOW] DQG 0 0DUGDQL ³8VH RI /HJR Q[W >@ 'RVWXSQp ]
PLQGVWRUPVEULFNLQHQJLQHHULQJHGXFDWLRQ´LQ3URFHHGLQJVRIWKH KWWSZZZOHJRHQJLQHHULQJFRPDOWHUQDWLYHSURJUDPPLQJ
 $6(( 1RUWK 0LGZHVW 6HFWLRQDO &RQIHUHQFH SS ± ODQJXDJHV
 >@ : /X   %HJLQQLQJ 5RERWLFV 3URJUDPPLQJ LQ -DYD ZLWK
>@ 0 3LQWR $ 3 0RUHLUD DQG $ 0DWRV ³/RFDOL]DWLRQ RI PRELOH /(*2 0LQGVWRUPV>RQOLQH@ 9\G $SUHVV  VW>@
URERWV XVLQJ DQ H[WHQGHG .DOPDQ ILOWHU LQ D /(*2 1;7´ ,((( ,6%1  'RVWXSQp ]
7UDQVDFWLRQVRQ(GXFDWLRQYROQRSS± KWWSVERRNVJRRJOHVNERRNV"LG HF6$'4$$4%$- YT OHMRV G
>@ 3 % -DVPHHW 6LQJK 0DS FRQVWUXFWLRQ DQG ORFDOL]DWLRQ XVLQJ T OHMRVOHJRPLQGVWRUPV KO VN VRXUFH JEVBQDYOLQNVBV
/HJR 0LQGVWRUPV 1;7 -RXUQDO RI DXWRPDWLRQ PRELOH URERWLFV >@ (FOLSVH )RXQGDWLRQ $ERXW WKH (FOLSVH )RXQGDWLRQ >@
LQWHOOLJHQWV\VWHPVYROQR 'RVWXSQp]KWWSVZZZHFOLSVHRUJRUJ
>@ 6 % &RúNXQ 07XUDQOL &UHGLW ULVN DQDO\VLV XVLQJ ERRVWLQJ >@ (FOLSVH )RXQGDWLRQ (FOLSVH 'HVNWRS  :HE ,'(V >@
PHWKRGV -RXUQDO RI $SSOLHG 0DWKHPDWLFV 6WDWLVWLFV DQG 'RVWXSQp]KWWSVZZZHFOLSVHRUJLGH
,QIRUPDWLFV YRO QR  SS >@ - /HH - %XLWUDJR  0DS JHQHUDWLRQ DQG ORFDOL]DWLRQ IRU D
KWWSVGRLRUJMDPVL /(*2 1;7 URERW >RQOLQH@ 9\G ,((( &RORPELD VWU 
>@ - %XUIRRW  (9 6HQVRUV ± /(*2 (QJLQHHULQJ>@ ,6%1 >@ 'RVWXSQp ]
'RVWXSQp]KWWSZZZOHJRHQJLQHHULQJFRPHYVHQVRUV KWWSVZZZUHVHDUFKJDWHQHWSXEOLFDWLRQB0DSBJHQHUD
>@ 0)HUUDUL'$VWROIR*)HUUDUL*%XLOGLQJ5RERWVZLWK WLRQBDQGBORFDOL]DWLRQBIRUBDB/(*2B1;7BURERW
/(*2 0LQGVWRUPV 1;7 NDS  .QRZLQJ :KHUH <RX $UH >@ -6LQJK3%HGL0DS&RQVWUXFWLRQ$QG/RFDOL]DWLRQ8VLQJ
>RQOLQH@ 9\G (OVHYLHU 6FLHQFH >@ 'RVWXSp ] /(*20LQGVWRUPV1;7-RXUQDORI$XWRPDWLRQ0RELOH5RERWLFV
KWWSVZZZVFLHQFHGLUHFWFRPVFLHQFHDUWLFOHDEVSLL%  ,QWHOOLJHQW 6\VWHPV YRO þ  >@ 'RVWXSQp ]
 KWWSVZZZMDPULVRUJLQGH[SKS-$05,6DUWLFOHYLHZ


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 414

 Use of Modern Software Environments for
Online Teaching of Microprocessor Programming
V. Režo, F. Gerhát and M. Weis
Slovak University of Technology in Bratislava, Institute of Electronics and Photonics, Bratislava, Slovakia
e-mail: vratislav.rezo@stuba.sk, filip.gerhat@stuba.sk, martin.weis@stuba.sk

Abstract—This article explores the use of simulation software and institutions a comprehensive guide to leverage
to improve online microprocessor programming courses. With simulation software as a catalyst for enhanced online
traditional labs less accessible, simulation software offers cost- microprocessor programming instruction.
effective, scalable, and diverse microcontroller platform
simulation. It discusses benefits, challenges, and strategies for II. BEFORE ONLINE TEACHING
effective integration, highlighting active learning and critical
thinking promotion. Educators and institutions can utilize this Before the advent of online learning, instructing
resource to bridge the theory-practice gap in microprocessor students in MSP430 microcontroller programming using
programming education. homemade modules presented substantial difficulties.
These DIY modules (Fig. 1) included components such as
Keywords - Online teaching, Microprocessor programming, displays, shift registers with LEDs, digital-to-analog
Simulation of microprocessor, Virtual laboratories converters, analog-to-digital converters, timers, interrupt-
driven button inputs, encoders, and more. However, these
modules often proved to be problematic due to unreliable
I. INTRODUCTION connections or inherently faulty peripherals. [9]
In recent years, the field of microprocessor Teaching microcontroller programming on MSP430
programming has witnessed a rapid evolution, driven by microcontrollers using homemade peripherals required a
advancements in technology and a growing demand for significant investment of time and effort. Instructors had
embedded systems expertise. As a result, educational to carefully guide students through the process of
institutions and instructors have faced the challenge of designing and constructing their own modules, each with
adapting their teaching methodologies to effectively its own set of challenges.
engage students in this complex domain, particularly One common issue was unreliable contact connections.
within the context of online learning during COVID-19. The DIY modules frequently suffered from intermittent
Traditional hands-on labs, which have long been a staple electrical connections, leading to unpredictable behaviour
of microprocessor programming education, because during programming exercises. Identifying the source of
COVID-19 has made it less accessible, and after the these connection issues often proved time-consuming,
semiconductor crisis, was price a hardware higher and less diverting attention from the actual programming concepts
available [1-3, 6]. and need electrical knowledge and schematics like Fig. 2.
In response to these conditions, educators and Moreover, the quality of the homemade peripherals
institutions were seeking simulation software as a viable varied widely. Some students' modules exhibited design
and innovative solution. Simulation software offers a flaws or manufacturing errors, causing incorrect readings
dynamic platform for students to gain practical experience and erratic behaviour (Fig. 3).
in programming microprocessors without the need for
physical hardware. This shift in educational methodology
has the potential to revolutionize the way microprocessor
programming is taught and learned in schools and courses,
making it more accessible, scalable, and adaptable to the
evolving needs of the digital age [1-3, 7-9].
This article describes the crucial role of simulation
software in online microprocessor programming
education. It explores the benefits and challenges
associated with this approach and provides valuable
insights into best practices for its effective
implementation. Furthermore, it examines the pedagogical
implications of simulation-based learning, emphasizing its
potential to foster active engagement, critical thinking,
and problem-solving skills among students.
As we navigate the ever-changing landscape of
education and technology, understanding how to harness
Figure 1. Buttons periphery with MSP430
the power of simulation software in microprocessor
programming education is of paramount importance. This
article aims to shed light on the subject, offering educators

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 415

 However, the introduction of online learning tools and
virtual simulations has since alleviated many of these
challenges, providing a more streamlined and reliable
approach to microcontroller programming education.

III. FIRST WAVE OF PANDEMIC TEACHING

During the initial wave of the pandemic, adapting
microprocessor programming education proved to be a
formidable challenge. Technological infrastructure for
online lectures was not readily available. And
transforming the entire microprocessor programming
curriculum to ensure consistent and high-quality education
for students presented significant hurdles. Finding
effective teaching methods was a struggle in the early
Figure 1 Schematic of connected periphery during lectures [5]
weeks, with experimentation being the primary approach.
As a first step, MS Teams emerged as the preferred
platform for lectures (Fig. 5) and exercises. However, the
initial attempts were semi-interactive at best. Instructors
shared their screens to explain code, and through
webcams, they demonstrated how microprocessors
interacted with connected peripherals. Unfortunately, this
approach often resulted in passive participation from most
students, who essentially treated the exercises as
instructional videos.
The challenges were multifaceted. The lack of
immediate, in-person interactions hindered the real-time
exchange of questions and clarifications. The remote
nature of learning made it difficult for students to actively
engage in problem-solving or seek immediate help when
faced with coding issues. This passive mode of learning
(Fig. 6), while necessary in the early stages, was far from
ideal as it diminished the interactive and collaborative
aspects of microprocessor programming education.
Figure 2. 7-Segment display connected to MSP430

As a result, instructors had to allocate additional time to

troubleshoot and debug these peripheral modules,
detracting from the core programming content.
Despite these challenges, teaching MSP430
microcontroller programming using DIY modules (Fig. 4)
did offer unique learning opportunities. It required
students to develop problem-solving skills, improve their
understanding of electronics, and become adept at
identifying and rectifying hardware-related issues.

Figure 4. Screenshot from lecture

Figure 3. DA converter with relay connected to MSP430 Figure 5 Screenshot from lecture

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 416

 Despite these initial struggles, educators and institutions breakout rooms for smaller group interactions, enhancing
persevered in their efforts to enhance the online learning the interactivity of online classes.
experience. Over time, new teaching strategies and tools Google Classroom: is a favorite among teachers and
were adopted, fostering greater student engagement and students for its simplicity and integration with other
interactivity. As the pandemic progressed, online Google tools such as Gmail and Google Drive. It enables
microprocessor programming education evolved to the creation of virtual classrooms, sharing of materials,
provide a more robust and effective learning environment, assignment submissions, and communication with
ensuring that students received the comprehensive students, all in one place (Fig. 7).
education they needed to succeed in this critical field. Microsoft Teams: is another major player in the realm
of online education. This tool provides a wide range of
IV. WHICH TOOLS WERE SUITABLE FOR ONLINE features, including video calls, chat, screen sharing, and
TEACHING? collaborative document editing (Fig. 8). For educators and
The COVID-19 pandemic inadvertently opened up students using Microsoft Office, Teams is a natural
numerous avenues for online education. These choice.
circumstances prompted educators to innovate and adapt Moodle: is an open-source learning management
to the new normal in several ways: platform that allows the creation and management of
Online Testing Integrity: To address concerns of online courses. With Moodle, teachers can create content,
academic integrity during remote assessments, educators assign tasks, conduct assessments, and track student
devised methods for creating online tests for quizzes and progress. It's a flexible tool that many schools use for their
evaluations. Some solutions included proctoring software, online learning needs like we see on Fig. 9.
but many institutions also implemented webcam
monitoring. This allowed instructors to observe students
via their home webcams to prevent cheating during exams
and quizzes.
Enhanced Study Materials: The shift to online learning
led to an overhaul of study materials. Lectures were
recorded, allowing students to access video recordings for
review, reducing the need for extensive note-taking. This
multimedia approach enriched the learning experience and
accommodated diverse learning styles.
Interactive Online Laboratories: Laboratory exercises
required significant adaptation. Students remotely
connected to computers hosting microcontrollers and
peripherals for their lab work. Tools like TeamViewer
facilitated this connection, enabling students to test their
programs and experiment with hardware as if they were in
a physical lab. This virtual approach allowed for real-time
feedback and learning.
These transformations highlighted the resilience and
adaptability of educational institutions and instructors.
While the pandemic necessitated rapid change, it also
Figure 6. Google classroom class
paved the way for innovative solutions that have the
potential to enhance education, even beyond the
challenges posed by lockdowns. The integration of online
learning tools and methods, such as webcam monitoring,
video lectures, and virtual laboratories, now provides a
more versatile and accessible approach to microprocessor
programming education [1].
A. For lectures
From elementary schools to universities, education
worldwide underwent a significant shift towards online
learning during the COVID-19 pandemic. This rapid
transition required reliable and effective tools for
delivering lectures, ensuring the continuity of education in
unforeseen circumstances. Here, we introduce some of the
most used online tools that aided teachers and students in
adapting to the new learning environment.[4] Figure 7 MS Teams classroom
Zoom: became synonymous with online video
conferencing during the pandemic. This tool allows
educators and students to connect easily through video
calls, making it an excellent choice for lecture delivery.
Zoom offers features such as screen sharing, chat, and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 417

 Figure 10 Proteus MSP430 microcontroler

That is why both tools and the ability to share projects

online are used for rapid prototyping. They enable
Figure 8 UEF STU FEI Moodle students and engineers to practically explore
microprocessor programming and electronic design,
B. For Labs which is essential for acquiring hands-on skills in the
field. This significance in simulating microcontrollers and
Tools like Proteus and Tinkercad from Autodesk
practical microprocessor programming education cannot
(Fig. 10) play a crucial role in simulating microcontrollers
be overstated. These tools support an innovative and
and practical exercises in microprocessor programming.
interactive learning process that allows students to
These tools enable students and researchers to simulate
understand microprocessor technologies and electronics
microcontroller behaviour and create electronic circuits
better [1-4].
without physical components [2-4].
One of its key features a Proteus, is Microcontroller V. EVOLUTION OF TEACHING PROGRAMING
Support, offering extensive compatibility with a wide A MICROCONTROLLERS
range of microcontrollers from various manufacturers,
including well-known families such as Arduino, PIC, The integration of simulation tools has significantly
AVR, ARM, and many more. Such flexibility empowers elevated the field of microprocessor programming
users to explore and experiment with different education, ushering it into a new era of enhanced learning
microcontrollers, gaining valuable insights into their possibilities. In this transformed landscape, practical
capabilities. Additionally, Proteus provides a robust exercises now encompass not only the virtual exploration
platform for Peripheral Simulation, allowing users to of microcontroller functionalities but also the
simulate a diverse range of peripherals, including LEDs, incorporation of various commercial sensors readily
LCD displays (Fig. 11), sensors, motors, and more. This available within simulation platforms. One example we
capability facilitates the creation and testing of intricate can see on Fig. 12, that is simulation of controlling DC
projects, all without the need for physical components. motor with MSP430. Controlling of 7-segment display
Furthermore, Proteus offers advanced Interactive with microcontroller MSP430 (Fig. 13) or Proteus
Debugging tools that enable users to monitor and debug simulation of simple LED blinking with MSP430
their code in real-time closely. With the ability to observe (Fig. 14).
register values and track currents, voltages, and other These commercial sensors, seamlessly integrated into
critical parameters, developers and engineers can the simulation environment, empower students to explore
efficiently identify errors and fine-tune their code for a multitude of real-world scenarios. They can experiment
optimal performance. This feature proves invaluable for with sensors for temperature, pressure, motion, gas
ensuring the reliability and efficiency of microcontroller- detection, and more. This immersion in sensor technology
based projects. not only enhances their theoretical knowledge but also
bridges the gap between theory and practical application.
Furthermore, the evolution of microprocessor
programming education has extended beyond commercial
sensors. The inclusion of custom organic sensors,
predominantly based on resistive principles, adds an
exciting dimension to the learning experience. These
sensors can detect and respond to factors such as
temperature (Fig. 15), pressure, flexion, or atmospheric
gases (Fig. 16). This expansion of sensor types not only
enriches the educational journey but also aligns with the
burgeoning field of the Internet of Things (IoT).
By incorporating these diverse sensor technologies,
students gain a holistic understanding of microprocessor
programming and its real-world applications. They learn
to harness the power of data collection and interpretation,
Figure 9 ThincerCAD MSP430
a skill vital in the IoT and smart technologies age.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 418

 Figure 11. Proteus simulation of DC motor controlled with MSP430

Figure 16. Printed temperature sensor

In essence, the integration of simulation tools,

commercial sensors, and custom organic sensors in
MSP430 microprocessor programming education elevates
the learning experience, equipping students with practical
skills and knowledge that are directly applicable in today's
interconnected world of IoT. It not only enhances their
academic journey but also propels them closer to the
realm of the Internet of Things, where their expertise can
shape the future of technology.

VI. SUMMARY
The evolution of microprocessor programming
Figure 12. Proteus simulation of 7-Segment display with MSP430 education has seen a shift from traditional, hardware-
dependent methods to innovative online tools and
simulations. In the past, instructing students with
homemade modules posed significant challenges due to
unreliable connections and faulty peripherals. Despite
these difficulties, it offered unique learning opportunities
by fostering problem-solving skills and hands-on
experience. However, the COVID-19 pandemic forced a
rapid transformation of microprocessor programming
education. During the first wave of the pandemic,
educators struggled to adapt to online teaching methods.
Tools like MS Teams were initially used for semi-
interactive lectures, but passive learning was common,
and challenges included the lack of real-time interaction
and engagement.
Nonetheless, educators persevered, and online learning
Figure 13. Proteus simulation of simple LED blinking with MSP430
tools evolved. Proctoring software and webcam
monitoring were introduced to maintain testing integrity.
Lectures were recorded, offering multimedia study
materials. Virtual laboratories, facilitated by tools like
Proteus and Tinkercad, allowed students to experiment
with microcontrollers and peripherals remotely. The
integration of simulation tools, commercial sensors, and
custom organic sensors elevated microprocessor
programming education. Students now gain practical skills
and IoT-related knowledge. This transformation equips
them for a future in the interconnected world of
technology. In conclusion, the field of MSP430
microprocessor programming education has evolved
significantly, overcoming challenges through innovation
and adaptation to provide a more versatile and accessible
learning experience.
Figure 15. Proteus simulation Organic gas sensor

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 419

 VII. CONCLUSION engineering virtual laboratory for COVID-19 pandemic
Laboratorio virtual de ingeniería electrónica para la pandemia de
In conclusion, using Proteus simulation software can be COVID-19. 12-19. 10.35429/JCT.2021.14.5.12.19.
an invaluable tool during online lectures of [3] Wang, X., Wang, J. L., Weng, Z., Wei, Y. F., Han, D., & Gong, C.
microprocessor programming education. It allows students L. (2022). Reform Exploration of Proteus Virtual Simulation
to virtually test their code without the need for physical Practice Teaching in Electronic. Advances in Applied Sociology,
hardware, reducing costs and providing flexibility for 12, 93-101. https://doi.org/10.4236/aasoci.2022.124009
experimentation. The simulations also facilitated a [4] Tiexin Zhu, Bingxue Yan, Analysis on Online Teaching of
Microcontroller Experiment Course. Adult and Higher Education
practical understanding of microprocessor concepts. (2023) Vol. 5: 38-43. DOI:
Proteus exemplifies the advantages of simulation tools in http://dx.doi.org/10.23977/aduhe.2023.051305.
modern education, bridging the gap between theory and [5] “MSP-EXP430G2ET,” MSP-EXP430G2ET Development kit |
hands-on experience, even in challenging circumstances. TI.com, https://www.ti.com/tool/MSP-EXP430G2ET (accessed
Sep. 22, 2023).
ACKNOWLEDGMENT [6] J. Svatos, J. Holub, J. Fischer, and J. Sobotka, “Online teaching at
CTU in Prague aka university under COVID restrictions,”
This work was supported by the Scientific Grant Agency Measurement: Sensors, vol. 18. Elsevier BV, p. 100121, Dec.
and the Slovak Research and Development Agency of the 2021. doi: 10.1016/j.measen.2021.100121.
Ministry of Education, Science Research and Sport of the [7] A. Soriano, L. Marín, M. Vallés, A. Valera, and P. Albertos, “Low
Slovak Republic, grants: VEGA 1/0621/23, APVV-20- Cost Platform for Automatic Control Education Based on Open
Hardware.,” IFAC Proceedings Volumes, vol. 47, no. 3. Elsevier
0564 and APVV-20-0310. BV, pp. 9044–9050, 2014. Doi: 10.3182/20140824-6-za-
1003.01909.
[8] D. Ibrahim, “A New Approach for Teaching Microcontroller
REFERENCES Courses to Undergraduate Students,” Procedia - Social and
[1] “Proceedings of SOCIOINT 2022- 9th International Conference Behavioral Sciences, vol. 131. Elsevier BV, pp. 411–414, May
on Education &amp; Education of Social Sciences.” International 2014. doi: 10.1016/j.sbspro.2014.04.139.
Organization Center of Academic Research, Jun. 13, 2022. doi: [9] D. E. Bolanakis, Microcontroller Education. Springer International
10.46529/socioint.2022. Publishing, 2018. doi: 10.1007/978-3-031-79589-3.
[2] Id, & Esqueda Elizondo, Jose & Jimenez, Laura & Verdin,
Annette & Serrano-Trujillo, Alejandra. (2021). Electronics

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 420

 ([SHULHQFHZLWKFUHDWLQJWRRONLWVDQGXVLQJ
LQQRYDWLYHPHWKRGVLQWKHHGXFDWLRQDOSURFHVVLQ
WKHILHOGRIDFTXLULQJDQGSURFHVVLQJODQG
UHVRXUFHVLQWKHFRQGLWLRQVRIKLJKHUHGXFDWLRQ
55\EiU /%HGQiURYi DQG=âLPNRYi 
7HFKQLFDO8QLYHUVLW\LQ.RãLFH)DFXOW\RI0LQLQJ(FRORJ\3URFHVV&RQWURODQG*HRWHFKQRORJLHV,QVWLWXWHRI(DUWK
5HVRXUFHV3DUN.RPHQVNpKR.RãLFH6ORYDNUHSXEOLF
HPDLOOXFLDEHGQDURYD#WXNHVNUDGLPU\EDU#WXNHVN]X]DQDVLPNRYD#WXNHVN

$EVWUDFW ²    7KLV DUWLFOH GHVFULEHV WKH FUHDWLRQ RI WKUHH SDUW RI RU SDUWLFLSDWH LQ WKH FUHDWLRQ DQG DFTXLVLWLRQ RI
LQQRYDWLYH HGXFDWLRQDO WRRONLWV D ER[ DQG DQ DSSOLFDWLRQ DJULFXOWXUDO SURGXFWV VXFK DV JUDLQ PHDW ILVK DQG RWKHU
ZLWK D TXL] ,19,19 IRFXVHG RQ UDZ PDWHULDOV IRRGVWKDWDUHVWDSOHIRRGVIRUWKHKXPDQSRSXODWLRQ7KH
VXVWDLQDEOH GHYHORSPHQW UHQHZDEOH HQHUJ\ VRXUFHV DV ZHOO LPSRUWDQFH RI REWDLQLQJ DQG SURFHVVLQJ HDUWK V UDZ
DVUHF\FOLQJIRFXVLQJRQWKHFRPSOH[LW\RILQGLYLGXDODUHDV PDWHULDOVLVOLQNHGWRWKHSURFHVVRIPDQXIDFWXULQJJRRGV
DQG WKHLU XVH LQ WKH HGXFDWLRQDO SURFHVV 7KH FXUUHQW VLQFH UDZ PDWHULDOV DUH QHFHVVDU\ IRU WKHLU SURGXFWLRQ
JHQHUDWLRQLVVHWRQOHDUQLQJLQDYLUWXDOHQYLURQPHQWVRLWLV IURP HOHFWURQLFV WR FDUV SODQHV DQG VKLSV >@ 5DZ
LPSRUWDQW WR LQWURGXFH WKHP WR WKLV LVVXH LQ WKH IRUP RI PDWHULDO HQHUJ\ UHVRXUFHVFDQ DOVREH VHHQ DV DEDVLV IRU
LQWHUDFWLYHWHDFKLQJDLGVWKDWFDQEHXVHGLQVHYHUDODUHDVRI VFLHQWLILFUHVHDUFKVLQFHPDQ\VFLHQWLILFUHVHDUFKHVUHTXLUH
HGXFDWLRQ IURP SULPDU\ DQG VHFRQGDU\ VFKRROV WR UDZ PDWHULDOV JHQHUDOO\ HDUWK UHVRXUFHV IRU DQDO\WLFDO
XQLYHUVLWLHVDQGSURIHVVLRQDOFRXUVHV7KH\KHOSWROHDUQLQD SXUSRVHV DQG WKH SURGXFWLRQ RI PDWHULDOV $Q HTXDOO\
IXQ DQG HQJDJLQJ ZD\ DQG FDQ LPSURYH WKH XQGHUVWDQGLQJ LPSRUWDQW DVSHFW RI WKH XVH RI HDUWK V UHVRXUFHV LV WKH
DQG PHPRUDELOLW\ RI WKH OHDUQLQJ PDWHULDO $V SDUW RI RXU LQWHUDFWLRQ ZLWK WKH HQYLURQPHQW ZKLFK FDQ EH YHU\
FRQWULEXWLRQ ZH GLVFXVV D VSHFLILF DUHD RI WKH SURGXFWLRQ LQWHQVH ERWK RQ D ORFDO DQG JOREDO VFDOH >@ 7KH FRUUHFW
FKDLQ DQG LWV SRWHQWLDO LQ WKH DUHD RI FLUFXODULW\ 7KH UDZ DFTXLVLWLRQDQGSURFHVVLQJRIUDZPDWHULDOVFDQFRQWULEXWH
PDWHULDOV WKHPVHOYHV DUH DOO DURXQG XV DQG PRVW FDQQRW WR WKH SURWHFWLRQ RI WKH HQYLURQPHQW DQG WKH VXVWDLQDEOH
LPDJLQHWKHLUSRWHQWLDOIRUXVHIURPFUDGOHWRJUDYH:LWK XVHRIUHVRXUFHVDVZHOODVWKHHOLPLQDWLRQRI
WKH VHW RI WKHVH JDPHV ZH WU\ WR EULQJ WKH UDZ PDWHULDOV
FORVHU WR WKH JHQHUDO SXEOLF DQG SRLQW RXW WKHLU ZLGH 
DSSOLFDWLRQSRVVLELOLWLHV5DZPDWHULDOVDQGLQWHUDFWLYHWRROV
DUH FULWLFDO FRPSRQHQWV LQ YDULRXV LQGXVWULHV DQG OHDUQLQJ ,, ('8&$7,21,17+(),(/'2)2%7$,1,1*$1'
HQYLURQPHQWV ,QQRYDWLYH WRROV DUH DOVR LPSRUWDQW IRU 352&(66,1*/$1'5(6285&(6,766&23($1'63(&,),&6
HQYLURQPHQWDOVXVWDLQDELOLW\ZKHUHWKH\FDQSOD\DNH\UROH (GXFDWLRQLQWKHILHOGRIDFTXLVLWLRQDQGSURFHVVLQJRI
LQ UDLVLQJ DZDUHQHVV SURPRWLQJ UHVSRQVLEOH EHKDYLRU DQG HDUWK V UDZ PDWHULDOV LV DLPHG DW SUHSDULQJ VWXGHQWV DQG
SURPRWLQJ VXVWDLQDEOH SUDFWLFHV 7KHVH HGXFDWLRQDO WRROV SURIHVVLRQDOVIRUMRESRVLWLRQVLQYDULRXVLQGXVWULHVUHODWHG
VKRXOGDLPWRHQJDJHDQGHPSRZHULQGLYLGXDOVVFKRROVDQG WR UDZ PDWHULDOV JHRORJ\ PLQLQJ DQG SURFHVVLQJ RI
FRPPXQLWLHV WR WDNH DFWLRQ WR UHGXFH ZDVWH DQG SURPRWH PLQHUDO UHVRXUFHV HQHUJ\ DQG HQYLURQPHQWDO
VXVWDLQDELOLW\ 1HZ HGXFDWLRQDO WRROV FDQ SOD\ D PDQDJHPHQW >@ 7KLV HGXFDWLRQ IRFXVHV RQ JHRORJLFDO
IXQGDPHQWDOUROHLQFUHDWLQJDPRUHHQYLURQPHQWDOO\DZDUH DQG JHRSK\VLFDO DVSHFWV ZKHUH VWXGHQWV OHDUQ DERXW
DQG UHVSRQVLEOH VRFLHW\7KHVH WRROV FDQ EH XVHGIRU DZLGH
JHRORJLFDO SURFHVVHV IRUPV WKH VWXG\ RI URFNV DQG
UDQJH RI SXUSRVHV IURP PDQXIDFWXULQJ WR HGXFDWLRQ DQG
PLQHUDOV JHRSK\VLFDO VXUYH\LQJ DQG FDUWRJUDSK\ 7KLV
HQWHUWDLQPHQW ,QQRYDWLYH OHDUQLQJ WRROV DQG WHFKQRORJLHV
HGXFDWLRQLVWKHFRUQHUVWRQHIRUXQGHUVWDQGLQJWKHHDUWK V
FRQWLQXH WR HYROYH ZLWK LQFUHDVLQJ HPSKDVLV RQ
UDZ PDWHULDOV DQG WKHLU H[WUDFWLRQ $QRWKHU DUHD RI
SHUVRQDOL]DWLRQLQWHUDFWLYLW\DQGDFFHVVLELOLW\
HGXFDWLRQ LV ORRNLQJ DW UDZ PDWHULDO UHVHUYHV 7KHUH
VWXGHQWV OHDUQ WR LGHQWLI\ DQG HYDOXDWH UHVRXUFH UHVHUYHV
, ,1752'8&7,21 ZKLFK LQFOXGHV H[DPLQLQJ WKH JHRORJLFDO IRUPDWLRQV
ZKHUH WKHVH UHVHUYHV DUH ORFDWHG TXDQWLI\LQJ WKHP DQG
7KHLPSRUWDQFHRIREWDLQLQJDQGSURFHVVLQJHDUWK VUDZ GHYHORSLQJ VWUDWHJLHV IRU WKHLU HIILFLHQW H[WUDFWLRQ >@
PDWHULDOV LV UHODWHG WR WKH IDFW WKDW WKH\ DUH WKH EDVLV IRU $QRWKHU DUHD RIHGXFDWLRQ KHUH LV PLQLQJ DQG SURFHVVLQJ
HFRQRPLF JURZWK ZKHUH WKH\ IRUP D NH\ SDUW RI PDQ\ UDZ PDWHULDOV LQ ZKLFK ZDVWH VWXGHQWV IRFXV RQ WKH
VHFWRUV RI WKH HFRQRP\ LQFOXGLQJ WKH H[WUDFWLRQ RI WHFKQLTXHV DQG WHFKQRORJLHV RI H[WUDFWLQJ UDZ PDWHULDOV
PLQHUDO UHVRXUFHV DJULFXOWXUH DQG ZRRGZRUNLQJ 7KHLU LQFOXGLQJ FRDO PHWDO RUH RLO DQG QDWXUDO JDV H[WUDFWLRQ
SURFHVVLQJDQGXVHDUHHVVHQWLDOIRUHFRQRPLFJURZWKDQG 7KHVXEMHFWRIHGXFDWLRQLVDOVRWKHSURFHVVLQJWUDQVSRUW
MRE FUHDWLRQ >@ (QHUJ\ UDZ PDWHULDOV VXFK DV FRDO RLO DQG VWRUDJH WUDGLQJ DQG VXEVHTXHQW SURFHVVLQJ RI WKH
DQG QDWXUDO JDV DUH HVVHQWLDO IRU HQHUJ\ SURGXFWLRQ DQG RULJLQDOUDZPDWHULDOLQWRYDULRXVSURGXFWVDQGPDWHULDOV
PHHWLQJWKHHQHUJ\QHHGVRIPDQNLQG%XLOGLQJPDWHULDOV $Q LPSRUWDQW SDUW LV HGXFDWLRQ LQ WKH ILHOG RI HQHUJ\
EDVHGRQPLQHUDOUDZPDWHULDOVVXFKDVVDQGJUDYHODQG (QHUJ\ HGXFDWLRQ IRFXVHV RQ WKH SURGXFWLRQ DQG
DJJUHJDWH DUH HVVHQWLDO IRU WKH FRQVWUXFWLRQ RI GLVWULEXWLRQRIHQHUJ\IURPDYDULHW\RIVRXUFHVLQFOXGLQJ
LQIUDVWUXFWXUH DQG EXLOGLQJV ,Q D EURDGHU VHQVH WKH\ DUH IRVVLOIXHOVUHQHZDEOHVDQGQXFOHDUHQHUJ\>@)URPWKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 421

SRLQW RI YLHZ RI WKH LQGXVWU\ V LQWHUDFWLRQ ZLWK WKH  ([SHUW ZRUNLQJ ZLWK *,6 WHFKQRORJLHV WR FROOHFW
HQYLURQPHQW HGXFDWLRQ LQ HQYLURQPHQWDO PDQDJHPHQW LV DQDO\]H DQG YLVXDOL]H JHRJUDSKLF GDWD UHODWHG WR UDZ
LPSRUWDQW ,W DOVR LQFOXGHV WKH VROXWLRQ RI HQYLURQPHQWDO PDWHULDOVDQGPLQLQJ
LVVXHV DQG PDWWHUV FRQQHFWHG ZLWK WKH DFTXLVLWLRQ RI UDZ 0DQDJHULQWKHILHOGRIDFTXLVLWLRQDQGSURFHVVLQJRI
PDWHULDOVDQGPLQLQJLQFOXGLQJHQYLURQPHQWDOSURWHFWLRQ UDZPDWHULDOV±DZRUNHUUHVSRQVLEOHIRUWKHPDQDJHPHQW
DQG VXVWDLQDELOLW\ )URP WKH SHUVSHFWLYH RI PRQLWRULQJ DQG FRRUGLQDWLRQ RI DFWLYLWLHV FRQQHFWHG ZLWK WKH
WUHQGV LQ HQJLQHHULQJ SUDFWLFH DQG LPSOHPHQWLQJ WKH DFTXLVLWLRQDQGSURFHVVLQJRIUDZPDWHULDOV>@
UHVXOWVRIVFLHQWLILFDQGWHFKQLFDOSURJUHVVLQWKHLQGXVWU\  5HQHZDEOH HQHUJ\ HQJLQHHU GHDOLQJ ZLWK WKH XVH RI
HGXFDWLRQLQWKHILHOGRIWHFKQRORJLFDOLQQRYDWLRQSOD\VD UHQHZDEOH HQHUJ\ VRXUFHV VXFK DV 39 V\VWHPV ZLQG
VLJQLILFDQW UROH +HUH VWXGHQWV OHDUQ DERXW WKH ODWHVW WXUELQHV K\GURSRZHU KHDW SXPSV DQG ELRPDVV HQHUJ\
WHFKQRORJLFDO LQQRYDWLRQV DQG SUDFWLFHV LQ WKH UDZ XVH
PDWHULDO H[WUDFWLRQ DQG SURFHVVLQJ LQGXVWULHV $QRWKHU
DUHD RI HGXFDWLRQ LV WKH YLHZ RI UDZ PDWHULDO UHVHUYHV  (QJLQHHU LQ WKH ILHOG RI HQYLURQPHQWDO SURWHFWLRQ 
WKURXJKWKHHGXFDWLRQRIWKHZLGHUVRFLHW\ZLWKWKHKHOSRI HQYLURQPHQWDO FRQVXOWDQW $Q H[SHUW ZRUNLQJ RQ WKH
WRXULVP HYDOXDWLRQ DQG PLQLPL]DWLRQ RI WKH HQYLURQPHQWDO
FRQVHTXHQFHV RI H[WUDFWLRQ DQG SURFHVVLQJ RI UDZ
(GXFDWLRQ LQ WKH ILHOG RI REWDLQLQJ DQG SURFHVVLQJ PDWHULDOVLQDOOVWDJHVRIWKHSURGXFWLRQFKDLQ
HDUWK V UDZ PDWHULDOV LV RI VWUDWHJLF LPSRUWDQFH IRU
LQGXVWU\DQGWKHHFRQRP\EHFDXVHWKHVHUDZPDWHULDOVDUH  (QJLQHHU LQ WKH ILHOG RI PDWHULDOV ± DQ H[SHUW LQ WKH
HVVHQWLDOIRUSURGXFWLRQDQGWKHFUHDWLRQRILQIUDVWUXFWXUH ILHOG RI VWXG\LQJ WKH SK\VLFDO DQG FKHPLFDO SURSHUWLHV RI
(GXFDWLRQLQWKLVDUHDKHOSVHQVXUHWKDWWKHUHDUHHQRXJK PDWHULDOV DQG WKHLU DSSOLFDWLRQ LQ YDULRXV SDUWV RI WKH
TXDOLILHGSURIHVVLRQDOVWRPDQDJHDQGXVHWKHVHUHVRXUFHV LQGXVWU\RUUHODWHGLQGXVWULHV
UHJDUGLQJ VXVWDLQDELOLW\ DQG HQYLURQPHQWDO UHVSRQVLELOLW\ 2FFXSDWLRQDOKHDOWKDQGVDIHW\PDQDJHUDQH[SHUWLQ
>@(GXFDWLRQLQWKHILHOGRIDFTXLVLWLRQDQGSURFHVVLQJRI HQVXULQJWKDWZRUNHUVLQWKHPLQLQJLQGXVWU\ZRUNLQVDIH
HDUWK UHVRXUFHV FDQ EH FDUULHG RXW DW YDULRXV OHYHOV FRQGLWLRQVDQGFRPSO\ZLWKRFFXSDWLRQDOKHDOWKDQGVDIHW\
LQFOXGLQJEDFKHORU VPDVWHU VDQGGRFWRUDOVWXGLHV7KHUH UHJXODWLRQV >@ 7KLV SURIHVVLRQ LV DOVR UHODWHG WR
DUH XQLYHUVLWLHV WHFKQLFDO VFKRROV DQG RWKHU HGXFDWLRQDO VSHFLDOL]DWLRQLQWKHILHOGRIPLQLQJPLQHUHVFXHUHVFXH
LQVWLWXWLRQV WKDW RIIHU SURJUDPV LQ WKLV ILHOG PDQ\ RI DQG VDIHW\ WHFKQRORJ\ ZKLFK LV D YHU\ VSHFLILF ILHOG
ZKLFKKDYHDYHU\ORQJDQGULFKKLVWRU\VXFKDV&HQWUDO UHODWHG WR WKH LQGXVWU\ RI REWDLQLQJ DQG SURFHVVLQJ HDUWK
(XURSHDQ XQLYHUVLWLHV ZKLFK FRQWLQXH WKH OHJDF\ RI WKH UHVRXUFHV
%DQVNi $FDGHP\ LQ %DQVNi âWLDYQLFD 6.  WKH ILUVW 7KH SURIHVVLRQV DQG SRVLWLRQV OLVWHG KDYH GLIIHUHQW
PLQLQJXQLYHUVLW\VFKRROVLQWKHZRUOG)URPWKHSRLQWRI VSHFLDOL]DWLRQV DQG UHVSRQVLELOLWLHV EXW DOO DUH LPSRUWDQW
YLHZ RI SURIHVVLRQV LQ WKH ILHOG RI REWDLQLQJ DQG IRUWKHVXVWDLQDEOHDQGHIILFLHQWDFTXLVLWLRQDQGSURFHVVLQJ
SURFHVVLQJ HDUWK V UDZ PDWHULDOV PDQ\ GLIIHUHQWLDWHG RI WKH HDUWK V UDZ PDWHULDOV(GXFDWLRQ DQG WUDLQLQJ IRU D
SURIHVVLRQV DQG SURIHVVLRQDO SRVLWLRQV IRFXV RQ GLIIHUHQW SURIHVVLRQDVVRFLDWHGZLWKWKHDFTXLVLWLRQDQGSURFHVVLQJ
DVSHFWVRIWKLVDFWLYLW\>@ RI HDUWK V UDZ PDWHULDOV FDQ EH VSHFLILF DQG GHSHQG RQ D
:H FDQ FKRRVH WKH IROORZLQJ DV IRFDO SURIHVVLRQV VSHFLILF SURIHVVLRQ DQG LQGXVWU\ +RZHYHU VRPH JHQHUDO
FRYHULQJWKHVSHFWUXPRIWKHILHOG FKDUDFWHULVWLFV PD\ EH H[FHSWLRQDO RU VSHFLILF WR WKHVH
 *HRORJLVW JHQHUDO  *HRORJLVWV VWXG\ WKH HDUWK V SURIHVVLRQV
SURFHVVHVDQGVWUXFWXUHVLQRUGHUWRLGHQWLI\DQGHYDOXDWH )RFXVRQSUDFWLFHDQGILHOGPDQ\SURIHVVLRQDOVLQWKH
PLQHUDO UHVHUYHV 7KH\ FDUU\ RXW JHRORJLFDO VXUYH\V DQG ILHOGRI QDWXUDO UHVRXUFHV VSHQG D ORW RI WLPH LQ WKH ILHOG
PDSSLQJ DQG LQ SODFHV ZKHUH PLQLQJ RU PLQLQJ H[SORUDWLRQ LV
 *HRSK\VLFLVW +H GHDOV ZLWK WKH LQYHVWLJDWLRQ RI WKH FDUULHG RXW 7KLV PHDQV WKDW HGXFDWLRQ LQFOXGHV QRW RQO\
SK\VLFDOSURSHUWLHVRIWKHHDUWK VIRUPDWLRQVZKLFKKHOSV WKHRUHWLFDONQRZOHGJHEXWDOVRSUDFWLFDOH[SHULHQFHLQWKH
WR LGHQWLI\ XQGHUJURXQG UHVHUYHV RI UDZ PDWHULDOV DQG ILHOG
GHVFULEHJHRORJLFDOSURFHVVHV  /DERUDWRU\ DQG DQDO\WLFDO ZRUN  UHVHDUFKHUV DQG
*HRWHFKQLFDOHQJLQHHU$QHQJLQHHUVSHFLDOL]LQJLQWKH H[SHUWVRIWHQSHUIRUPODERUDWRU\WHVWVDQGDQDO\WLFDOZRUN
DVVHVVPHQW RI WKH JHRWHFKQLFDO SURSHUWLHV RI VRLOV DQG WR HYDOXDWH WKH FRPSRVLWLRQ DQG SURSHUWLHV RI UDZ
URFNV ZKLFK LV LPSRUWDQW ZKHQ GHVLJQLQJ EXLOGLQJ PDWHULDOV
IRXQGDWLRQV DQG LQIUDVWUXFWXUH DQG DVVHVVLQJ WKH VWDELOLW\  (QYLURQPHQWDO DQG VDIHW\ FDUH  ZLWK UHJDUG WR WKH
RIPLQLQJZRUNV>@ HQYLURQPHQWDO DQG VDIHW\ FRQVHTXHQFHV RI DFWLYLW\ LQ WKH
*HRORJ\RI PLQLQJDQGSURFHVVLQJRIUDZ PDWHULDOV UDZPDWHULDOVLQGXVWU\HGXFDWLRQLVRIWHQIRFXVHGRQWKHVH
([SHUW IRFXVLQJ RQ WKH GHVLJQ DQG PDQDJHPHQW RI DVSHFWVDQGRQPHDVXUHVWRPLQLPL]HULVNV>@
SURFHVVHV RI PLQLQJ DQG SURFHVVLQJ RI PLQHUDO UDZ 'LUHFWLYHV5HJXODWLRQVDQG/DZVDVWKHPLQLQJDQG
PDWHULDOV VXFK DV RUHV FRDO DJJUHJDWHV DQG RWKHUV  SURFHVVLQJ RI UDZ PDWHULDOV LV VXEMHFW WR UHJXODWLRQ DQG
(QJLQHHULQWKHPLQLQJLQGXVWU\VSHFLDOL]LQJLQWKHGHVLJQ ODZ SURIHVVLRQDOV LQ WKLV ILHOG PXVW KDYH D JRRG
DQG PDQDJHPHQW RI PLQLQJ RSHUDWLRQV DQG WKH VWXG\ RI XQGHUVWDQGLQJ RI OHJLVODWLRQ DQG FRPSOLDQFH LV NH\ 
WKHLULPSDFWRQWKHHQYLURQPHQW 6XVWDLQDELOLW\  ZLWK LQFUHDVLQJ HPSKDVLV RQ VXVWDLQDEOH
 (QJLQHHU LQWKH ILHOGRI HQYLURQPHQWDO PDQDJHPHQW PHWKRGV RI PLQLQJ DQG SURFHVVLQJ HGXFDWLRQ LV
GHDOLQJ ZLWK HQYLURQPHQWDO SURWHFWLRQ DQG PDQDJLQJ WKH LQFUHDVLQJO\IRFXVHGRQVROYLQJHQYLURQPHQWDODQGVRFLDO
HQYLURQPHQWDO FRQVHTXHQFHV DVVRFLDWHG ZLWK REWDLQLQJ LVVXHV
UDZPDWHULDOV 3URMHFWPDQDJHPHQWIRUWKRVHZRUNHUVZKRZRUNDW
 (QJLQHHU LQ WKH ILHOG RI *HRJUDSKLFDO ,QIRUPDWLRQ DPDQDJHULDOOHYHODQLPSRUWDQWDVSHFWRIHGXFDWLRQLVWKH
6\VWHPV *,6  DELOLW\WRPDQDJHSURMHFWVDQGWHDPV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 422

 &RQVWDQWGHYHORSPHQWWKHLQGXVWU\RIREWDLQLQJDQG DQG HIIHFWLYH ZD\ WR KHOS VWXGHQWV XQGHUVWDQG FRPSOH[
SURFHVVLQJHDUWK VUHVRXUFHVLVG\QDPLFDQGWKHUHIRUHLWLV FRQFHSWV DQG SURFHVVHV LQ WKH LQGXVWU\ 7KHVH JDPHV DUH
LPSRUWDQW WKDW SURIHVVLRQDOV FRQVWDQWO\ H[SDQG WKHLU W\SLFDOO\ GHVLJQHG WR HGXFDWH DQG WUDLQ RIIHULQJ VWXGHQWV
NQRZOHGJHDQGIROORZQHZWHFKQRORJLHVDQGSURFHGXUHV WKHRSSRUWXQLW\WROHDUQDQGH[SHULPHQWZLWKRXWWKHQHHG
,Q WKH ILHOG RI HGXFDWLRQ DQG WUDLQLQJ LQ WKH ILHOG RI IRU UHDOZRUOG HQJDJHPHQW 7KHVH LQWHUDFWLYH JDPHV FDQ
QDWXUDO UHVRXUFHV WKH XVH RI LQQRYDWLYH DQG EH FODVVLILHG LQWR VHYHUDO VSHFLILFDOO\ RULHQWHG
XQFRQYHQWLRQDO PHWKRGV LV SRVVLEOH ZKLFK DUH XVHG WR VXEFDWHJRULHV
LPSURYHWKHOHDUQLQJDQGWUDLQLQJRISURIHVVLRQDOV7KHVH  *HRORJLFDO DQG PLQHUDORJLFDO JDPHV  WKHVH JDPHV
LQFOXGHSULPDULO\ KHOSVWXGHQWVXQGHUVWDQGJHRORJLFDOSURFHVVHVURFNVDQG
6LPXODWLRQ HQYLURQPHQWV  VLPXODWLRQV RI PLQLQJ DQG PLQHUDOV 7KH\ FDQ GHDO ZLWK WKH LGHQWLILFDWLRQ DQG
SURFHVVLQJ RI UDZ PDWHULDOV LQ D YLUWXDO HQYLURQPHQW FODVVLILFDWLRQRIGLIIHUHQWW\SHVRIPLQHUDOV
ZKHUH VWXGHQWV FDQ VLPXODWH YDULRXV PLQLQJ DQG  0LQLQJ VLPXODWLRQ JDPHV  WKHVH JDPHV DOORZ
SURFHVVLQJ RSHUDWLRQV 7KHVH VLPXODWLRQ HQYLURQPHQWV SOD\HUV WR VLPXODWH YDULRXV DVSHFWV RI PLQLQJ VXFK DV
HQDEOH VDIH WUDLQLQJ DQG H[SHULPHQWDWLRQ LQ UHDOLVWLF VHOHFWLQJ PLQLQJ VLWHV PDQDJLQJ PLQLQJ HTXLSPHQW DQG
FRQGLWLRQV GHDOLQJZLWKXQH[SHFWHGVLWXDWLRQV3OD\HUVFDQJHWDIHHO
9LUWXDO UHDOLW\ 95  DQG DXJPHQWHG UHDOLW\ $5   DUH IRU KRZ UDZ PDWHULDOV DUH PLQHG ZLWKRXW ULVN WR WKH
WHFKQRORJLHV WKDW DOORZ VWXGHQWV WR LQWHUDFW ZLWK WKUHH HQYLURQPHQWDQGKXPDQOLYHV
GLPHQVLRQDO PRGHOV RI URFNV PLQLQJ HTXLSPHQW DQG  *DPHV IURP WKH ILHOG RI SURFHVVLQJ UDZ PDWHULDOV
RWKHU UDZ PDWHULDOV LQ D VSDWLDO FRQWH[W 7KLV LPSURYHV SURFHVVLQJ   WKHVH JDPHV IRFXV RQ WKH SURFHVVHV RI
XQGHUVWDQGLQJ RI FRQWH[W DQG PHPRUL]DWLRQ RI PDWHULDOV SURFHVVLQJ DQG UHILQLQJ UDZ PDWHULDOV VXFK DV PHWDO
>@ VPHOWLQJ RU RLO UHILQLQJ 3OD\HUV FDQ H[SHULPHQW ZLWK
2QOLQH FRXUVH SODWIRUPV  OHDUQLQJ SODWIRUPV RIIHULQJ GLIIHUHQW WHFKQRORJLHV DQG SURFHGXUHV WR DFKLHYH WKH
DFFHVV WR D YDULHW\ RI FRXUVHV DQG PDWHULDOV LQ WKH HDUWK GHVLUHGUHVXOW
UHVRXUFHVLQGXVWU\DOORZLQJVWXGHQWVWROHDUQDWWKHLURZQ  (FRORJLFDO JDPHV  WKHVH JDPHV GHDO ZLWK WKH
SDFHDQGIURPDYDULHW\RIVRXUFHV HQYLURQPHQWDO DQG VXVWDLQDEOH DVSHFWV RI WKH HDUWK V UDZ
3UDFWLFDOH[HUFLVHVSUDFWLFDOH[HUFLVHVLQWKHILHOGRULQ PDWHULDOV 3OD\HUV GHFLGH RQ VWUDWHJLHV WKDW PLQLPL]H WKH
ODERUDWRULHV ZKHUH VWXGHQWV FDQ FDUU\ RXW UHDO LPSDFWRIPLQLQJRQWKHHQYLURQPHQW>@
H[SHULPHQWV DQG PHDVXUHPHQWV DUH QHFHVVDU\ WR LQFOXGH  /RJLVWLFV DQG PDQDJHPHQW JDPHV  LQFOXGH
SUDFWLFDODVSHFWVDQGOHDUQVRPHDSSURDFKHV PDQDJHPHQWRIORJLVWLFVDQGVXSSO\RIUDZPDWHULDOVIURP
,QWHUDFWLYHJDPHVDQGVLPXODWLRQVFDQEHXVHGDVDIXQ WKH SODFH RI H[WUDFWLRQ WR WKH SODFH RI SURFHVVLQJ RU
DQG HGXFDWLRQDO ZD\ RI OHDUQLQJ DERXW WKH SURFHVVHV RI SURGXFWLRQ
H[WUDFWLRQDQGSURFHVVLQJRIUDZPDWHULDOVEXWDOVRDERXW &RPSHWLWLYHJDPHV*DPHVWKDWLQFOXGHFRPSHWLWLYH
ZLGHUFRQWH[WVLQDQLQWHUGLVFLSOLQDU\XQGHUVWDQGLQJ HOHPHQWVFDQSURYLGHVWXGHQWVZLWKDPRWLYDWLRQDOERRVWWR
6KDULQJ RI SURIHVVLRQDO H[SHULHQFH  LQGXVWU\ H[SHUWV LPSURYHWKHLUNQRZOHGJHDQGVNLOOVLQWKHILHOGRIQDWXUDO
DQG SURIHVVLRQDOV FDQ EH LQYROYHG LQ WKH HGXFDWLRQ RI UHVRXUFHV
VWXGHQWV VKDULQJ WKHLU SUDFWLFDO H[SHULHQFH DQG H[DPSOHV 7KH DGYDQWDJHVRI LQWHUDFWLYH JDPHV LQFOXGH LQFUHDVHG
IURPUHDOZRUNZKLFKLVH[WUHPHO\EHQHILFLDOIRUVWXGHQWV VWXGHQW HQJDJHPHQW WKH SRVVLELOLW\ RI UHSHDWHG SUDFWLFH
DQG WKHLU DELOLW\ WR UHWDLQ WKH LQIRUPDWLRQ OHDUQHG $ QR ZLWKRXWUHDOULVNVLPPHGLDWHIHHGEDFNDQGWKHSRVVLELOLW\
OHVVLPSRUWDQWDVSHFWRIWKLVLVWKHPRWLYDWLRQDOHIIHFW RI OHDUQLQJ LQ GLIIHUHQW VFHQDULRV DQG HQYLURQPHQWV
&RRSHUDWLRQZLWKLQGXVWU\VRPHHGXFDWLRQDOSURJUDPV *DPHV FDQ EH XVHG LQ HGXFDWLRQDO SURJUDPV DW YDULRXV
FRRSHUDWH ZLWK LQGXVWU\ DQG DOORZ VWXGHQWV WR JDLQ OHYHOVIURPSULPDU\DQGVHFRQGDU\HGXFDWLRQWRXQLYHUVLW\
H[SHULHQFHLQUHDOZRUNLQJHQYLURQPHQWVGXULQJHGXFDWLRQ DQGSURIHVVLRQDOFRXUVHV7KHLQWHUDFWLYLW\RIJDPHVLQWKH
GLUHFWSUHSDUDWLRQIRUSUDFWLFH2QOLQH FRQWH[W RI HGXFDWLRQ PHDQV WKDW SOD\HUV KDYH WKH
RSSRUWXQLW\ WR DFWLYHO\ LQIOXHQFH WKHFRXUVH RI WKH JDPH
/DERUDWRU\ ([HUFLVHV  7KH GHYHORSPHQW RI RQOLQH
PDNH GHFLVLRQV DQG UHDFW WR VLWXDWLRQV ZLWKLQ WKH JDPH
ODERUDWRU\ H[HUFLVHV DOORZV VWXGHQWV WR FRQGXFW
7KLV LV WKH NH\ HOHPHQW WKDW GLVWLQJXLVKHV LQWHUDFWLYH
H[SHULPHQWV DQG WHVW PDWHULDOV IURP DQ\ ORFDWLRQ ZLWK
JDPHVIURPSDVVLYHREVHUYDWLRQRUSDVVLYHOHDUQLQJ
,QWHUQHW DFFHVV >@  7KHVH LQQRYDWLYH PHWKRGV DUH RIWHQ
FRPELQHG ZLWK WUDGLWLRQDO WHDFKLQJ SUDFWLFHV WR DFKLHYH ,Q LQWHUDFWLYH JDPHV SOD\HUV KDYH WKH RSSRUWXQLW\ WR
HIIHFWLYHDQGHQJDJLQJHDUWKUHVRXUFHVHGXFDWLRQ:LWKWKH PDNH GHFLVLRQV DERXW YDULRXV DVSHFWV RI WKH JDPH 7KH\
XVH RI PRGHUQ WHFKQRORJLHV DQG DSSURDFKHV VWXGHQW FDQ GHFLGH ZKDW WKH\ ZLOO GR ZKDW GLUHFWLRQ WKH\ ZLOO
HQJDJHPHQW LQFUHDVHV DQG FRQWULEXWHV WR D EHWWHU WDNH DQG ZKDW VWUDWHJLHV WKH\ ZLOO XVH *DPHV UHDFW WR
XQGHUVWDQGLQJRIWKHFRPSOH[DVSHFWVRIWKHLQGXVWU\ SOD\HUDFWLRQV7KLVPHDQVWKDWWKHSOD\HU VGHFLVLRQVFDQ
DIIHFWWKHGHYHORSPHQWRIWKHVWRU\RUWKHRXWFRPHRIWKH

JDPH,QWHUDFWLYHJDPHVRIWHQDOORZSOD\HUVWRH[SHULPHQW
7KHVH LQQRYDWLYH PHWKRGV DUH RIWHQ FRPELQHG ZLWK ZLWK GLIIHUHQW DSSURDFKHV DQG VWUDWHJLHV 3OD\HUV FDQ VHH
WUDGLWLRQDO WHDFKLQJ SUDFWLFHV WR DFKLHYH HIIHFWLYH DQG WKH FRQVHTXHQFHV RI WKHLU GHFLVLRQV DQG WU\ GLIIHUHQW
HQJDJLQJ HDUWK UHVRXUFHV HGXFDWLRQ >@  :LWK WKH XVH RI RSWLRQV >@ *DPHV FDQ FRQWDLQ YDULRXV WDVNV DQG WDVNV
PRGHUQWHFKQRORJLHVDQGDSSURDFKHVVWXGHQWHQJDJHPHQW WKDW WKH SOD\HU PXVW FRPSOHWH 7KHVH WDVNV FDQ EH
LQFUHDVHVDQGFRQWULEXWHVWRDEHWWHUXQGHUVWDQGLQJRIWKH GHVLJQHGWRWHVWWKHSOD\HU VNQRZOHGJHDQGVNLOOVEXWDOVR
FRPSOH[DVSHFWVRIWKHLQGXVWU\,QWHUDFWLYHJDPHVKDYHD HPSKDVL]H WKH DELOLW\ WR H[WUDFW WKH WDVN IURP WKH WH[W RI
VSHFLILFIXQFWLRQIRUHGXFDWLRQLQWKLVDUHDZKLFKFDQEH WKH WDVN ,QWHUDFWLYH JDPHV XVXDOO\ FRQWDLQ DWWUDFWLYH
RULHQWHGWRGLIIHUHQWDVSHFWVRIWKHLVVXHDVQHHGHGDVZHOO YLVXDOVWKDWLQFUHDVHSOD\HUHQJDJHPHQW7KLVFDQLQFOXGH
DV FKDQJLQJ WKHLU SXUSRVH RU WDUJHW JURXS ,QWHUDFWLYH WKUHHGLPHQVLRQDOPRGHOVDQLPDWLRQVDQGYLVXDOHIIHFWV
JDPHVLQWKHFRQWH[WRIHDUWKUHVRXUFHVHGXFDWLRQDUHDIXQ *DPHVFDQSURYLGHSOD\HUVZLWKLQVWDQWIHHGEDFNRQWKHLU

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 423

GHFLVLRQV DQG SHUIRUPDQFHV 7KLV DOORZV SOD\HUV WR DQGSURJUHVVLQJLQWKHJDPHXVLQJWKH,7DSSOLFDWLRQDVD
TXLFNO\ XQGHUVWDQG WKH FRQVHTXHQFHV RI WKHLU DFWLRQV FRQWURO DQG VXSSOHPHQWDU\ WRRO IRU WKH JDPH $V SDUW RI
,QWHUDFWLYH JDPHV FDQ EH YRODWLOH DQG G\QDPLF PHDQLQJ WKLV WRRO DQ HGXFDWLRQDO JDPH ZDV FUHDWHG FRQWDLQLQJ D
WKDW VLWXDWLRQV FDQ FKDQJH DQG HYROYH UHTXLULQJ TXLFN PDWHULDO SODWIRUP ZLWK SK\VLFDO HOHPHQWV DV VKRZQ LQ
GHFLVLRQPDNLQJ DQG SOD\HU DGDSWDWLRQ 7KHVH HOHPHQWV )LJDQG D YLUWXDOSODWIRUPZKLFK VHUYHVWR FRQWURO WKH
PDNHJDPHVIXQHGXFDWLRQDODQGHQJDJLQJ,QWHUDFWLYLW\ JDPHDQGSURYLGHDFFRPSDQ\LQJLQIRUPDWLRQDQGFRQWURO
DOORZVVWXGHQWVWRDFWLYHO\HQJDJHLQWKHOHDUQLQJSURFHVV
DQG JDLQ H[SHULHQFH DQG NQRZOHGJH WKURXJK KDQGVRQ 0DWHULDOSODWIRUPDQGLWVFRQILJXUDWLRQSDUDPHWHUV
H[SHULHQFH DQG H[SHULPHQWDWLRQ ,Q RUGHU WR LPSURYH WKH
DWWUDFWLYHQHVV LW LV SRVVLEOH WR LQFRUSRUDWH TXL] HOHPHQWV • :RRGHQ ER[ ± FUHDWHV D JDPH HQYLURQPHQW DQG
LQWR LQWHUDFWLYH JDPHV RU WR FRPSLOH WKHP DV TXL]]HV FRQWDLQV DOO WKH QHFHVVDU\ FRPSRQHQWV RI WKH
4XL]]HVDUHDJRRGZD\WRWHVWSOD\HUV NQRZOHGJHZKLOH QDWXUHRIPDWHULDOREMHFWV
HQJDJLQJDQGPRWLYDWLQJWKHPWROHDUQ4XL]]HVFDQEHRI • %R[HVLQDER[±WKH\FUHDWHWKHVWUXFWXUHRIWKH
GLIIHUHQWW\SHVDQGIRFXVRQGLIIHUHQWDVSHFWVRIHGXFDWLRQ JDPH DQG IRUP WKH EDVLF KLHUDUFKLFDO EXLOGLQJ
XQLW
4XL]]HVFDQEHFRQFHLYHGDV • 1XPEHU RI ER[HV DQG WKHLU DUUDQJHPHQW LQ WKH
.QRZOHGJHTXL]]HV ER[ ER[HVLQRQHOHYHOOHYHOV 
&RPSHWLWLRQTXL]]HV 
3URJUHVVLYHTXL]]HV
4XL]]HVZLWKVWUDWHJ\
4XL]]HVZLWKIHHGEDFN

.QRZOHGJH TXL]]HV WHVWSOD\HUV RQ WKHLU NQRZOHGJHRI

JHRORJ\ PLQLQJ SURFHVVLQJ PLQHUDOV DQG RWKHU UHODWHG
WRSLFV7KHSOD\HUPXVWDQVZHUWKHTXHVWLRQVFRUUHFWO\WR
DGYDQFHLQWKHJDPH&RPSHWLWLYHTXL]]HVFDQEHSDUWRID
FRPSHWLWLYHJDPHZKHUHSOD\HUVFRPSHWHZLWKHDFKRWKHU
IRUWKHEHVWUHVXOWV7KLVFDQEHPRWLYDWLQJDQGVWLPXODWLQJ
IRU VWXGHQWV 3URJUHVVLYH TXL]]HV FDQ EH VHW WR JUDGXDOO\
LQFUHDVH LQ GLIILFXOW\ DOORZLQJ SOD\HUV WR JUDGXDOO\
LPSURYH WKHLU NQRZOHGJH 6WUDWHJ\ TXL]]HV DUH PRUH 
FRPSOH[DQGFDQEHSDUWRIWKHODUJHUVWUDWHJLFIUDPHZRUN )LJXUH 3K\VLFDOSODWIRUPZLWKER[HVDQGDUWLIDFWVLQWULSOLFDWHXVHG
RIWKHJDPH7KHSOD\HUPXVWXVHKLVNQRZOHGJHWRGHFLGH LQWHDFKLQJDW)%(5*
RQ RSWLPDO VWUDWHJLHV )HHGEDFN TXL]]HV SURYLGH
LPPHGLDWH IHHGEDFN RQ WKH SOD\HU V DQVZHUV H[SODLQLQJ 
ZK\ D JLYHQ DQVZHU LV FRUUHFW RU LQFRUUHFW 7KLV KHOSV • 'LYLGLQJ HDFK ER[ LQWR FRPSDUWPHQWV XVLQJ
SOD\HUV OHDUQ IURP PLVWDNHV 4XL]]HV FDQ EH D YHU\ SDUWLWLRQV
HIIHFWLYH ZD\ WR HQJDJH VWXGHQWV DQG VHH KRZ ZHOO WKH\ 1XPEHU RI GHSDUWPHQWV DQG WKHLU DUUDQJHPHQW
XQGHUVWDQGWKHPDWHULDO,QLQWHUDFWLYHJDPHVTXL]]HVFDQ V\PPHWULFDO GLYLVLRQ LQWR  HTXDO GHSDUWPHQWV  2QH
EHLQWHJUDWHGLQWRDIORZLQJVWRU\RUJDPHHQYLURQPHQWWR GHSDUWPHQWUHSUHVHQWVRQHWDVN
PDNHLWIXQDQGHQJDJLQJIRUSOD\HUV 7KHVXEMHFWRIWKHVROXWLRQWDVNVDUWLIDFWVLQER[HV
,,, &5($7,21$1'86(2)6(/(&7(',1129$7,9( • $ GHVFULSWLRQ RI WKH SK\VLFDO ' 
('8&$7,21$/722/6,19
DUWLIDFWVLQHDFKER[
• ([DPSOHVRIDUWLIDFWV
,Q RUGHU WR WDNH LQWR DFFRXQW DV PXFK DV SRVVLEOH WKH
VSHFLILFV RI HGXFDWLRQ LQ WKH ILHOG RI DFTXLULQJ DQG *DPH SURJUHVV RYHUFRPLQJ OHYHOV  ORFNV ZLWK D
SURFHVVLQJODQGUHVRXUFHVVHYHUDO,QQRYDWLYH(GXFDWLRQDO QXPHULFDOFRGH
7RROV ,19  ZLWK GLIIHUHQW OHYHOV RI LQWHUDFWLYLW\ ZHUH • 'HVLJQ DQG FUHDWLRQ RI D PHFKDQLVP IRU
SURSRVHG7KHIROORZLQJDVSHFWVZHUHWDNHQLQWRDFFRXQW FRQQHFWLQJER[HVDERYHHDFKRWKHU
GXULQJWKHLUGHVLJQ • 7KHORFDWLRQRIWKHORFNVDQGWKHPHWKRG
'HFLVLRQPDNLQJ RIORFNLQJWKHP
$FWLRQVDQGUHDFWLRQV • )RXUGLJLW QXPHULFDO FRGH DQG LWV
([SHULPHQWDWLRQ PHDQLQJLQWKHJDPH
([WUDFWLQJWKHDVVLJQPHQWDQGWDVNXQGHUVWDQGLQJ *DPHSURJUHVV
,QWHUHVWLQJDQGYLVXDOHOHPHQWV • 7KHZD\WKHSOD\HUPRYHVIURPRQHOHYHO
,PPHGLDWHIHHGEDFN RIER[HVWRDQRWKHU
• (QWHULQJWDVNVWKURXJKWKHDSSOLFDWLRQ
$ ,QQRYDWLYH(GXFDWLRQDO7RROV±(GX5DZ
• 7KH UHODWLRQVKLS EHWZHHQ VROYLQJ WDVNV
7KH HGXFDWLRQDO WRRO LV DFRPSHWLWLYH JDPH7KH EDVLF DQGREWDLQLQJDUWLIDFWV
LGHDRIWKHGHVLJQ±WKHLQFRUSRUDWLRQRIPDWHULDODUWLIDFWV
LQWRWKHVROXWLRQRIWDVNVDQGWKHFUHDWLRQRIDKLHUDUFKLFDO • 6ROYLQJWDVNVXVLQJDUWLIDFWV
DUUDQJHPHQWRIWKHJDPHIRURYHUFRPLQJLQGLYLGXDOWDVNV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 424

 • 7KHPHWKRGRIREWDLQLQJDQXPHULFDOFRGH FRQWDLQVDUHDVTXHVWLRQVWKDWDUHRULHQWHGWRGLIIHUHQW
IRUSURJUHVVLQWKHJDPH VFLHQWLILF ILHOGV RU DUHDV RI KXPDQ DFWLYLW\ LQ UHODWLRQ WR
9LUWXDO SODWIRUP DQG LWV FRQILJXUDWLRQ WKH JLYHQ UDZ PDWHULDOHOHPHQW >@ 4XL] TXHVWLRQV DUH
SDUDPHWHUV FRQVWUXFWHGLQVXFKDZD\WKDWWKH\FDQEHDQVZHUHGLQD
<(612ZD\)RUHDFKTXHVWLRQDYLGHRLVSUHSDUHGZLWK
• 'HVFULSWLRQRIWKHYLUWXDOSODWIRUP LOOXVWUDWLYH LQIRUPDWLYH DQG DV IDU DV SRVVLEOH RULJLQDO
• 7DEOHWZLWKZHEDSSOLFDWLRQ DQG IXQQ\ FRQWHQW 7KH WLPH UDQJH RI WKH YLGHRV UDQJHV
• :HEDSSOLFDWLRQ IURPPLQWRPLQ
• +LHUDUFKLFDO ORJLFDO DQG LQWXLWLYH
QDYLJDWLRQIRUXVHUV )URP WKH SRLQW RI YLHZ RI FRQFHSWLRQ WKH IROORZLQJ
• )XQFWLRQVRIWKHZHEDSSOLFDWLRQ DVSHFWVZHUHWDNHQLQWRFRQVLGHUDWLRQ
)RUPXODWLRQ RI TXHVWLRQV DQG VXSSRUW LQ VROYLQJ
WDVNV *DLQLQJNQRZOHGJH
• :D\RIIRUPXODWLQJWKHTXHVWLRQUHVXOWLQJ • *HRORJLFDO NQRZOHGJH $Q XQGHUVWDQGLQJ RI
IURPPDWHULDODUWLIDFWV JHRORJLFDO SURFHVVHV VWUXFWXUHV DQG URFN DQG
PLQHUDO W\SHV LV NH\ WR WKH LGHQWLILFDWLRQ DQG
• 7KH ZD\ WKH DSSOLFDWLRQ KHOSV XVHUV HYDOXDWLRQRIUHVRXUFHUHVHUYHV
PDQDJHWDVNV
• 7HFKQLFDO .QRZOHGJH 8QGHUVWDQGLQJ PLQLQJ
• 7KH SURFHVV RI HQWHULQJ DQVZHUV WR WHFKQLTXHV UDZ PDWHULDOV SURFHVVLQJ DQG
TXHVWLRQVDQGWDVNUHVXOWV WHFKQRORJ\ XVHG LQ WKH LQGXVWU\ LV HVVHQWLDO IRU
• $GGLWLRQDOLQIRUPDWLRQDQGDLGV HQJLQHHUVDQGWHFKQLFLDQV
7HFKQLFDOVHFXULW\RIWKHJDPH • (QYLURQPHQWDO NQRZOHGJH 8QGHUVWDQGLQJ WKH
• *HQHUDWLQJQXPHULFFRGHVWLPHU HQYLURQPHQWDO FRQVHTXHQFHV RI WKH H[WUDFWLRQ
• 7KHVFRUH DQGSURFHVVLQJRIUDZPDWHULDOVDQGWKHDELOLW\WR
GHYHORSPHDVXUHVWRPLQLPL]HWKHP>@
7KHJDPHDUHDZDVGLYLGHGLQWRWKUHHOHYHOV
• (QHUJ\ .QRZOHGJH )RU WKRVH ZRUNLQJ LQ WKH
• 3ULPDU\UDZPDWHULDO HQHUJ\ LQGXVWU\ LW LV LPSRUWDQW WR NQRZ WKH
• 7KHDUHDRIH[WUDFWLRQSURFHVVLQJPRGLILFDWLRQ GLIIHUHQWVRXUFHVRIHQHUJ\DQGWKHLUSURGXFWLRQ
WUDQVSRUWWUDGLQJWHFKQRORJLFDODVSHFWV • .QRZOHGJHRI6WDWXWRU\5HJXODWLRQV.QRZOHGJH
• $SSOLFDWLRQOHYHO RI HQYLURQPHQWDO VDIHW\ DQG UHJXODWRU\
7KHJDPHZDVWKHPDWLFDOO\RULHQWHGLQDUHDV UHJXODWLRQV DQG UHJXODWRU\ JXLGHOLQHV WKDW DIIHFW
WKH UDZ PDWHULDOV VRXUFLQJ DQG SURFHVVLQJ
• 0HWDORUHV LQGXVWU\
• (QHUJ\UDZPDWHULDOV 'HYHORSLQJVNLOOV
• 2WKHULQGXVWULDOUDZPDWHULDOV • $QDO\WLFDO6NLOOV$ELOLW\WRDQDO\]HJHRORJLFDO
,19  ZDV XVHG LQ WKH HGXFDWLRQDO SURFHVV ZLWKLQ WKH DQGWHFKQLFDOGDWDDQGPDNHLQIRUPHGGHFLVLRQV
IUDPHZRUN RI VROYLQJ VHYHUDO FRPSOH[ WDVNV SULPDULO\
ZLWKLQWKHLQWHUGLVFLSOLQDU\VXEMHFWRI(DUWK5HVRXUFHV,Q • -RE 6NLOOV $ YDULHW\ RI SUDFWLFDO VNLOOV IURP
DGGLWLRQ LW ZDV XVHG LQ VHYHUDO KLJK VFKRROV DV SDUW RI PLQLQJ HTXLSPHQW PDQDJHPHQW WR ODERUDWRU\
WKHPDWLFDOO\ RULHQWHG HGXFDWLRQDO SURMHFWV 7KH UHVXOWV RI WHVWLQJ
WKHJDPHZHUHHYDOXDWHGTXDOLWDWLYHO\7KHRYHUDOODWWLWXGH • &RPPXQLFDWLRQ VNLOOV $ELOLW\ WR FRPPXQLFDWH
RI WKH VWXGHQWV  SRVLWLYH WRZDUGV WKH FRQWHQW DQG WKH ZLWK WKH WHDP FXVWRPHUV DQG WKH SXEOLF DQG
PHWKRG RI LPSOHPHQWDWLRQ ,Q WKH FDVH RI VHFRQGDU\ SUHVHQW\RXULGHDVDQGUHVXOWV
VFKRRO WHDFKHUV GHPDQGV ZHUH PDGH IRU SUHOLPLQDU\ • ,QQRYDWLRQ VNLOOV 7KH DELOLW\ WR VHDUFK DQG
WUDLQLQJ)RUWKDWSXUSRVHDPDQXDOZDVGHYHORSHGZKLFK GHYHORS QHZ WHFKQRORJLHV DQG SURFHGXUHV IRU
VKRXOG EH VWXGLHG LQ DGYDQFH 2YHUDOO LW LV SRVVLEOH WR PRUH VXVWDLQDEOH H[WUDFWLRQ DQG SURFHVVLQJ RI
FRQVLGHU WKH JDPH LWVHOI DQG LWV LPSOHPHQWDWLRQ DV UDZPDWHULDOV
GHPDQGLQJIRUERWKWKHVWXGHQWDQGWKHWHDFKHUZLWKKLJK
GHPDQGV RQ FRQFHQWUDWLRQ DQG SUHSDUDWLRQ ,W LV • 0DQDJHULDO 6NLOOV )RU WKRVH LQ OHDGHUVKLS RU
DGYDQWDJHRXV WKDW LW LV SRVVLEOH WR FRPSOHWH WKH JDPH PDQDJHPHQW SRVLWLRQV WKH DELOLW\ WR PDQDJH D
SDUWLDOO\7KH,19WHFKQLFDOVROXWLRQLWVHOILVWKHVXEMHFW WHDPDQGSURMHFWRSHUDWLRQV
RI LQWHOOHFWXDO SURSHUW\ SURWHFWLRQ DQG LV SURWHFWHG E\ D *DLQLQJ6NLOOV
XWLOLW\PRGHO>@ • 3UREOHP 6ROYLQJ $ELOLW\ WR LGHQWLI\ DQG VROYH
SUREOHPV LQ PLQLQJ SURFHVVLQJ DQG
% ,QQRYDWLYH(GXFDWLRQDO7RROV±(GX5DZ HQYLURQPHQWDOLVVXHV>@
,QWKHFDVHRI,QQRYDWLYH(GXFDWLRQDO7RROVWKHJDPH • 6XVWDLQDELOLW\ 7KH DELOLW\ WR WKLQN DERXW ORQJ
ZDVGHVLJQHGWRKDYHQRPDWHULDOSODWIRUPDQGLQFOXGHGD WHUP VXVWDLQDELOLW\ DQG WDNH LQWR DFFRXQW WKH
TXL] HOHPHQW $ ZHE DSSOLFDWLRQ ZDV FUHDWHG ZLWK WKH HQYLURQPHQWDO DQG VRFLDO FRQVHTXHQFHV RI WKH
DGGLWLRQDO RSWLRQ VLQJOH SOD\HU IRU WKH SRVVLELOLW\ RI DFWLYLW\
LQGHSHQGHQW HGXFDWLRQ RXWVLGH WKH WHDFKLQJ SURFHVV 7KH
JDPH LV GLYLGHG LQWR WKHPDWLF XQLWV QDPHG DFFRUGLQJ WR • )OH[LELOLW\$ELOLW\WRDGDSWWRGLIIHUHQWVLWXDWLRQV
WKHNH\FKHPLFDOHOHPHQWUHSUHVHQWLQJWKHJLYHQVSKHUH DQGUHVSRQGTXLFNO\WRFKDQJHVLQWKHLQGXVWU\
XQLWVZHUHFUHDWHG$X6L)HDQG&X(DFKWKHPDWLFXQLW

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 425

 • (WKLFV DQG UHVSRQVLELOLW\ %HLQJ DZDUH RI WKH $+ (GXFDWLRQ WKURXJK DQ LQWHUGLVFLSOLQDU\
HWKLFDO DQG VRFLDO FRQVHTXHQFHV RI DQ DFWLYLW\ FRPSHWLWLRQRUJDPHGRHVQRWDOORZDEHWWHUXQGHUVWDQGLQJ
DQGDFWLQJUHVSRQVLEO\ RIWKHUHODWLRQVKLSVLQWKHILHOGRIREWDLQLQJODQGUHVRXUFHV
• 7HDPZRUN $ELOLW\ WR ZRUN ZLWK RWKHU WKDQFRQYHQWLRQDOHGXFDWLRQZLWKLQLQGLYLGXDOVXEMHFWV$
SURIHVVLRQDOV DQG WHDP PHPEHUV WR DFKLHYH +(GXFDWLRQWKURXJKDQLQWHUGLVFLSOLQDU\FRPSHWLWLRQ
FRPPRQJRDOV RU JDPH DOORZV IRU D EHWWHU XQGHUVWDQGLQJ RI WKH
UHODWLRQVKLSVLQWKHILHOGRIREWDLQLQJODQGUHVRXUFHVWKDQ
FRQYHQWLRQDOHGXFDWLRQZLWKLQLQGLYLGXDOVXEMHFWV
3URIHVVLRQV LQ WKH ILHOG RI HDUWK UHVRXUFHV UHTXLUH D
FRPELQDWLRQ RI SUHFLVHO\ WKHVH WDUJHWHG JURXSV RI )RUWKHSXUSRVHRIYHULI\LQJWKHK\SRWKHVHVIRUUHVHDUFK
WKHRUHWLFDO NQRZOHGJH SUDFWLFDO VNLOOV DQG DELOLWLHV WR TXHVWLRQ$DTXHVWLRQZDVIRUPXODWHGLQWKHTXHVWLRQQDLUH
VROYHFRPSOH[SUREOHPVRIDQLQWHUGLVFLSOLQDU\QDWXUH>@ ZLWKWKHIROORZLQJZRUGLQJ
3DUWRIWKHJDPHLVWKHSRVVLELOLW\RIFKRRVLQJWKHYDOXHRI ,V LWHIIHFWLYH WROHDUQDERXW WKH WRSLF LQ WKH IRUP RI D
WKH TXHVWLRQ DQG WKXV WKH VL]H RI WKH H[SHFWHG ULVN 7KH FRPSHWLWLRQRUDJDPH"
TXL] LV GHVLJQHG VR WKDW WZR WHDPV FDQ FRPSHWH DJDLQVW 5HVHDUFK TXHVWLRQ % 'RHV LQIRUPDWLRQ IURP WKH
HDFK RWKHU 7KH JDPH LV PDQDJHG WKURXJK D ZHE VXEMHFW DUHD REWDLQHG LQIRUPDOO\ WKURXJK WDVNV IL[HG RQ
DSSOLFDWLRQ H[SHULHQFH KDYH D PRUH SHUPDQHQW FKDUDFWHU WKDQ WKDW
REWDLQHGE\DVWDQGDUGIRUPDOZD\"
,19LVXVHGDVSDUWRIWHDFKLQJDW)%(5*LQVXEMHFWV %+,QIRUPDWLRQJLYHQLQDQLQIRUPDOZD\LVQRWRID
UHOHYDQWWRWKHILHOGRINQRZOHGJHDQGZDVXVHGDWVHYHUDO PRUH SHUPDQHQW QDWXUH WKDQ LQIRUPDWLRQ REWDLQHG LQ D
HYHQWV RUJDQL]HG IRU WKH HGXFDWLRQ RI VWXGHQWV RI FRQYHQWLRQDOZD\
VHFRQGDU\ VFKRROV >@ ,19 LQ WKH (QJOLVK ODQJXDJH %+ ,QIRUPDWLRQ JLYHQ LQ DQ LQIRUPDO ZD\ LV RI D
YHUVLRQ ZDV XVHG LQ D ZRUNVKRS LQ FRQQHFWLRQ ZLWK DQ PRUH SHUPDQHQW QDWXUH WKDQ LQIRUPDWLRQ REWDLQHG LQ D
LQWHUQDWLRQDO(,7HYHQWLQ%RORJQD,WDO\)LJGHVFULEHV FRQYHQWLRQDOZD\
D VLWXDWLRQ ZKHQ VWXGHQWV ZHUH LQYROYHG LQ D JDPH DQG
KDGH[DFWO\WKHLULQWHUDFWLRQZLWKWHDFKHUV )RUYHULI\LQJWKHK\SRWKHVHVIRUUHVHDUFKTXHVWLRQ%D
TXHVWLRQQDLUH TXHVWLRQ ZLWK WKH IROORZLQJ ZRUGLQJ ZDV
 IRUPXODWHG&DQLQIRUPDWLRQEHJLYHQLQDQLQIRUPDOZD\
RIDPRUHSHUPDQHQWQDWXUHWKDQIRUPDORQHV"
5HVHDUFKTXHVWLRQ&'RHVWKHIDFWWKDWLWFRQWDLQVDQ\
'DUWHIDFWVDQGWDQJLEOHREMHFWVDIIHFWWKHDWWUDFWLYHQHVV
RIDQHGXFDWLRQDOFRPSHWLWLRQJDPH"
&+ 7KH DWWUDFWLYHQHVV RI DQ HGXFDWLRQDO
FRPSHWLWLRQJDPHLVQRWLQFUHDVHGE\LWVHQULFKPHQWR'
DUWHIDFWVDQGWDQJLEOHREMHFWV
&+ 7KH DWWUDFWLYHQHVV RI WKH HGXFDWLRQDO
FRPSHWLWLRQJDPH ZLOO EH LQFUHDVHG E\ HQULFKLQJ LW ZLWK
'DUWHIDFWVDQGWDQJLEOHREMHFWV>@
)RUWKHSXUSRVHRIYHULI\LQJWKHK\SRWKHVHVIRUUHVHDUFK
 TXHVWLRQ&DTXHVWLRQZDVIRUPXODWHGLQWKHTXHVWLRQQDLUH
)LJXUH ,19JDPHSOD\DW(,7HYHQWLQ%RORJQD,WDO\2FW ZLWK WKH IROORZLQJ ZRUGLQJ :RXOG WKH FRQWHVW EH PRUH
HQJDJLQJLILWLQFOXGHGVRPHWDQJLEOHDUWLIDFWV"
& (YDOXDWLRQ,QQRYDWLYH(GXFDWLRQDO7RROV±(GX 
5DZ ,Q WRWDO  VWXGHQWV SDUWLFLSDWHG LQ WKH VHFRQG ,(7
WHVWLQJ ZKR ILOOHG RXW WKH VDPH TXHVWLRQQDLUH ZLWK 
,QQRYDWLYH(GXFDWLRQDO7RROVZDVHYDOXDWHGDVSDUWRI
TXHVWLRQVIRFXVHGRQNQRZOHGJHDQGWKHDELOLW\WRFRQQHFW
PRQLWRUHG JDPHV ZLWK D WRWDO RI  VWXGHQWV 5HVHDUFK
LQIRUPDWLRQ IURP GLIIHUHQW DUHDV DQG ZLWK  TXHVWLRQV
TXHVWLRQVZHUHGHILQHGWKDWWKHVWXGHQWVDQVZHUHGEHIRUH
IRFXVHGRQWKHLURSLQLRQRQVXSSOHPHQWLQJWKHHGXFDWLRQDO
DQGDIWHUWKHJDPH
SURFHVVZLWKVXFKSOD\IXOOHDUQLQJ
7KHTXHVWLRQVWKDWQHHGHGWREHDQVZHUHGIRUIHHGEDFN
7KHHYDOXDWLRQRIWKHUHVSRQVHVIURPWKHTXHVWLRQQDLUHV
ZHUHDVIROORZV
UHVXOWHGLQWKHIROORZLQJILQGLQJV
1XOO DQG DOWHUQDWLYH K\SRWKHVHV + +  ZHUH
IRUPXODWHG IRU WKH TXHVWLRQV D TXHVWLRQQDLUH ZLWK  • %HIRUHSOD\LQJWKHWRRONLWWKHDYHUDJHVFRUHIRU
TXHVWLRQV ZDV GHVLJQHG WR GHWHUPLQH WKH OHYHO RI WKH NQRZOHGJH TXHVWLRQV ZDV  SRLQWV ZKLOH
NQRZOHGJHIRUWKHUHVHDUFKTXHVWLRQV DIWHUWKHJDPHLWZDVSRLQWV
+RZLWZDVPHQWLRQHGDERYHUHVHDUFKTXHVWLRQV $% • %HIRUH WKH JDPH QRW D VLQJOH SDUWLFLSDQW
& ZHUHGHILQHGIRUZKLFKQXOODQGDOWHUQDWLYHK\SRWKHVHV DFKLHYHG WKH IXOO QXPEHU RI SRLQWV DIWHU WKH
++ ZHUHIRUPXODWHG JDPH  VWXGHQWV DFKLHYHG WKH IXOO QXPEHU RI
SRLQWV
5HVHDUFKTXHVWLRQ$&DQHGXFDWLRQXVLQJWKHWRROVRI
LQWHUGLVFLSOLQDU\ FRPSHWLWLRQ RU JDPHV OHDG WR D EHWWHU • %HIRUH XVLQJ WKH WRRONLW  VWXGHQWV KDG OHVV
XQGHUVWDQGLQJRIWKHFRQWH[WLQWKHILHOGRIREWDLQLQJODQG WKDQ  SRLQWV DIWHU XVLQJ WKH WRRONLW WKHUH ZHUH
UHVRXUFHV FRPSDUHG WR FRQYHQWLRQDO HGXFDWLRQ UHJDUGLQJ RQO\RIWKHP
DVSHFWV RI WKLV LVVXH FRYHUHG ZLWKLQ LQGLYLGXDO WHDFKLQJ
VXEMHFWV"

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 426

 7KHTXHVWLRQVZHUHDOZD\VFKRVHQIRUDVSHFLILFWDUJHW WKHORFDWLRQRIWKHKDQGOHRIWKHGHFLVLRQPHPEHURQWKH
JURXS %RWK WKH RYHUDOO SHUFHSWLRQ DQG WKH DFKLHYHG VXUIDFHRIWKHSOD\LQJWLOHLQDVWDWHZKHUHLWUHPDLQVLQD
UHVXOWVZHUHSRVLWLYHDQGSXEOLVKHGLQ>@ KRUL]RQWDO SRVLWLRQ DIWHU KDQJLQJ 'XULQJ WKH JDPH HDFK
SOD\HUFKRRVHVIURPDOOWKHSOD\LQJWLOHVODLGQH[WWRHDFK
' (YROXWLRQDU\HOHPHQWVIRUWKHGHYHORSPHQWRI RWKHUDQGWULHVWRGHWHUPLQHZKHWKHUWKHLUFHQWHURIJUDYLW\
,QQRYDWLYH(GXFDWLRQDO7RROV SRVLWLRQV DUH WKH VDPH RU GLIIHUHQW 7R VDIHO\ GLVWLQJXLVK
%DVHGRQWKHSUHYLRXVH[SHULHQFHZLWKWKHXVHRI,19 ZKHWKHUWKHVHOHFWHGSDLURISOD\LQJWLOHVKDVWKHVDPHRU
WKH QHHG WR LQFOXGH D QHZ JDPH HOHPHQW ZDV VKRZQ GLIIHUHQW SRVLWLRQ RI WKH FHQWHU RI JUDYLW\ D FRPSDULVRQ
ZKLFK ZRXOG IRFXV RQ WKH GHYHORSPHQW RI WKH OHYHO RI XVLQJDGHFLVLRQPHPEHULVXVHG
FRQFHQWUDWLRQ DQG WKXV WKH FRUUHFWQHVV DQG VSHHG RI 6R IDU WKH JDPHV LQ TXHVWLRQ KDYH RQO\ EHHQ YHULILHG
GHFLVLRQPDNLQJ H[WUDFWLRQ RI WKH SUREOHP DQG H[SHULPHQWDOO\ DQG LW KDV EHHQ VKRZQ WKDW WKH SRWHQWLDO
HYDOXDWLRQ RI WKH VLWXDWLRQ 7ZR VHSDUDWH JDPHV ZHUH IRU WKHLU GHSOR\PHQW ZLWKLQ PRUH FRPSOH[ HGXFDWLRQDO
FUHDWHGZLWKHOHPHQWVWKDWFDQEHLQFRUSRUDWHGLQWR,19 JDPHVLVZLGHDQGUHODWHGWRWKHGHJUHHRIGHVFULSWLRQRI
DQG ,19 RU RWKHU JDPHV DQG FRPSHWLWLRQV LQ RUGHU WR WKHWDVNDQGWKHOHYHO)URPDFRPSOHWHLQWURGXFWLRQWRWKH
LQFOXGH DQ HPRWLRQDO REVWDFOH IRU WKH GHYHORSPHQW RI SUREOHP WR ]HUR LQIRUPDWLRQ WR WKH SRLQW ZKHUH WKH
FRQFHQWUDWLRQDQGOHDUQLQJWRPDQDJHVWUHVV H[WUDFWLRQRIWKHSUREOHPWKHZD\WKHJDPHLVSOD\HGDQG
$VHQVRU\PHPRU\JDPHZLWKDGHFLVLRQPHPEHULWLV WKH DFFHVV WR LW LV OHIW RQO\ WR WKH SOD\HUV )URP WKH
DLPHGDWUHFRJQL]LQJGLIIHUHQFHVLQWKHZHLJKWRIREMHFWV H[SHULPHQWV FDUULHG RXW VR IDU D KLJKHU GHJUHH RI
DQG WKHUHE\ GHYHORSLQJ FRQFHQWUDWLRQ DQG FRSLQJ ZLWK LQYROYHPHQWLQWKHJDPHDQGWKXVDEHWWHUGLGDFWLFHIIHFW
HPRWLRQVDQGVWUHVVIDFWRUV7KHJDPHFRQVLVWVRISOD\LQJ ZLWKDORZHUGHJUHHRILQLWLDWLRQLQVWUXFWLRQVUHVXOWHG7KH
WLOHVFRQWDLQLQJVDGGOHVRIZHLJKWVLQZKLFKZHLJKWVFDQ DSSOLFDWLRQRIWKHJDPHVLQTXHVWLRQVLJQLILFDQWO\H[SDQGV
EH SODFHG DV QHHGHG $ SDUW RI WKH JDPH LV D GHFLVLRQ WKHH[LVWLQJSRVVLELOLWLHVRI,19DQG,19
PHPEHU LQ ZKLFK WKH LVRVFHOHV VFDOHV DUH GHVLJQHG WR
VKRZ WKH GLIIHUHQFH LQ WKH ZHLJKW RI WKH WZR FRPSDUHG ,9 &21&/86,21
SOD\LQJ WLOHV SURSRUWLRQDOO\ WKDW LV WKH VWDWHV GLIIHUHQW 6LQFHWKHFRQVWUXFWLRQRI WKH /,. /DERUDWRU\ RI
ZHLJKWDQGWKHVDPH ZHLJKWDVLQILJXUH7KHERZOVRI (QJLQHHULQJ&UHDWLYLW\EHJDQDW)%(5*ZKLFKSURYLGHG
WKHVFDOHVDUHVKDSHGFRPSOHPHQWDU\WRWKHSOD\LQJWLOHV VSDFH IRU WKH GHYHORSPHQW RI WKH FUHDWLYH DELOLWLHV RI
7KHVPDOOHUWKHGLIIHUHQFHVLQ ZHLJKWEHWZHHQLQGLYLGXDO JLIWHGVWXGHQWVDQGZDVDEOHWRLQWHJUDWHFUHDWLYHZRUNHUV
SDLUV RI SOD\LQJ WLOHV WKH PRUH GLIILFXOW LW LV WR ILQG DQG VWXGHQWV LQWR HIIHFWLYHO\ ZRUNLQJ WHDPV FDSDEOH RI
PDWFKLQJ SDLUV DQG WKXV JUHDWHU GHPDQGV DUH SODFHG RQ VROYLQJ FRPSOH[ WDVNV DQG FUHDWLQJ QHZ VROXWLRQV 2QH
WKH SOD\HUV  FRQFHQWUDWLRQ DQG HPRWLRQDO VWDWH,QGLYLGXDO VXFK WDVN ZDV WKH LPSOHPHQWDWLRQ RI WKH HOHPHQWV RI WKH
SOD\LQJWLOHVDUHSODFHGZLWKZHLJKWVWRIRUPSDLUVZLWKWKH JDPH RU FRPSHWLWLRQV LQ WKH SURFHVV RI DFTXLULQJ
VDPHZHLJKW NQRZOHGJH RU PDQDJLQJ WDVNV ,W ZDV SRVVLEOH WR GHVLJQ
 WKHEDVLFIUDPHZRUNRIWKHSODWIRUPZKLFKFRPELQHGWKH
UHDODUWLIDFWDQGYLUWXDOZRUOGVLQWKH,&7HQYLURQPHQW
DQGWRFUHDWHDKLHUDUFKLFDOO\VWUXFWXUHGPRGHOZKLFKZDV
H[SHULPHQWDOO\ DSSOLHG WR WKH ILHOG RI JHRORJ\ DQG
VWXGHQWV RI VHYHUDO .RãLFH JUDPPDU VFKRROV XQGHU WKH
JXLGDQFHRIXQLYHUVLW\WHDFKHUV WKHUHVXOWVRIZKLFKZHUH
REWDLQHGLQ6HSWHPEHUILUVWSUL]H95DZ0DWHULDOV
$PEDVVDGRUV6FKRROVLQ%XGDSHVW,WWXUQVRXWKRZHYHU
WKDWWKHSRWHQWLDORIWKHRULJLQDOSURSRVDOLVPXFKEURDGHU
RSHQLQJXSVSDFHIRUWKHFUHDWLRQRIDPRUHXQLYHUVDOWRRO
WKDWFDQEHXVHGLQHYHU\GD\SHGDJRJLFDOSUDFWLFHERWKLQ
WKHILHOGRIQDWXUDODQGWHFKQLFDOVFLHQFHVUHO\LQJRQWKH
VLJQLILFDQWFUHDWLYHDQGLQYHQWLYHSRWHQWLDORIWKHWHDPRI
UHVHDUFKHUV 7KH IDFW WKDW ERWK WKH UHVSRQVLEOH VROYHU RI
 WKH SUHVHQWHG SURMHFW DQG VHYHUDO PHPEHUV RI WKH
)LJXUH $VHQVRU\PHPRU\JDPHZLWKDGHFLVLRQPHPEHUDQGD FROOHFWLYHLQDGGLWLRQWRULFKSHGDJRJLFDOH[SHULHQFHDOVR
GLDJUDPRIWKHDUUDQJHPHQWRIZHLJKWVLQJDPHWLOHV KDYH ULFK H[SHULHQFH ZLWK YDULRXV IRUPV RI LQWHOOHFWXDO
SURSHUW\ DV DXWKRUV RI LQYHQWLRQV DQG LQQRYDWLRQV DOVR
$ VHQVRU\ PHPRU\ JDPH ZLWK D GHFLVLRQPDNLQJ FRUUHVSRQGV ZHOO ZLWK WKLV $Q LPSRUWDQW VXSSRUWLQJ
PHPEHU FDQ EH XVHG DV SDUW RI YDULRXV GLGDFWLF DQG DUJXPHQW IRU WKLV SDWK LQ HGXFDWLRQ LV WKH IDFW WKDW WKH
PRWLYDWLRQDO JDPHV IRU HPSOR\HHV RI FRPSDQLHV RI H[SHULPHQWDOXVHRIWKHHOHPHQWVWKDWDUHWKHVXEMHFWRIWKH
GLIIHUHQWQDWXUHVZLWKLQSURMHFWVDLPHGDWGLVFRYHULQJDQG SURMHFWVRIDUKDVPHWZLWKDYHU\SRVLWLYHUHVSRQVHDPRQJ
GHYHORSLQJVNLOOVDELOLWLHVDQGFRPSHWHQFLHV>@ VWXGHQWV DVZHOODVWHDFKHUV HVSHFLDOO\IURPWKHSRLQWRI
6HQVRU\JDPHZLWKDTXDOLWDWLYHGHWHUPLQLQJSDUDPHWHU YLHZ RI EHLQJ GUDZQ LQWR WKH SURFHVV DQG LQWURGXFLQJ
 7KH FRQWHQW RI WKH JDPH LV WR UHFRJQL]H LGHQWLFDOO\ WKHDGYHQWXURXVDQGDFRPSHWLWLYHFKDUJHWRWKHDFWLYLW\
VKDSHG SOD\LQJ WLOHV RI WKH VDPH ZHLJKW ZLWK D GLIIHUHQW  HGXFDWLRQ ZKLFK LV FHUWDLQO\ QRW SHUFHLYHG LQ WKLV ZD\
FHQWHURIJUDYLW\3DUWRIWKHJDPHLVDGHFLVLRQPHPEHU E\ VWXGHQWV XQGHU QRUPDO FLUFXPVWDQFHV 6LJQLILFDQW
ZKLFKUHSUHVHQWVDQDUPDQFKRUHGLQWKHSODWIRUP ZLWKD DGGHGDVSHFWVRIVXFKIRUPVDVVKRZQE\RXUH[SHULHQFH
VXVSHQGHGPDJQHWLFKDQGOH7KHPDJQHWLFKDQGOHLVXVHG VRIDUDUHWKHHIIHFWLYHQHVVRITXL]]HVLQHGXFDWLRQZKLOH
WR KDQJ WKH WLOH EHKLQG LWV OLG LQ VXFK D ZD\ WKDW LW LV TXL]]HVLQIOXHQFHWKHPHPRUL]DWLRQDQGXQGHUVWDQGLQJRI
SRVVLEOH WR FOHDUO\ GHWHUPLQH WKH FHQWHU RI JUDYLW\ RI WKH WKH VXEMHFW PDWWHU PRWLYDWLRQ DQG HQJDJHPHQW DQG WKH
WLOH 7KH SRVLWLRQ RI WKH FHQWHU RI JUDYLW\ FRUUHVSRQGV WR XVHRIWHFKQRORJ\LQFOXGLQJ,7

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 427

 $&.12:/('*0(17 ,QWK ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ
7HFKQRORJLHVDQG$SSOLFDWLRQV ,&(7$ SS
7KLVSDSHUZDVFUHDWHGLQFRQQHFWLRQZLWKWKHSURMHFWV >@ - 7HM 9 $OL 7DKD / %HGQiURYi 0DQDJHPHQW VLPXODWLRQ
$399 :DVWH DQG &RQVWUXFWLRQ  PRGHOLQJ WKH H[HUFLVHVLQVLJKWLQWRWKHLUXVHLQWKHHGXFDWLRQDOSURFHVVWK
HIIHFWLYHQHVV RI DOWHUQDWLYH FRRSHUDWLRQ RSWLRQV RI ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ
DGPLQLVWUDWLYH DXWKRULWLHV DQG .(*$78.( 7HFKQRORJLHVDQG$SSOLFDWLRQV ,&(7$ SS
8QLYHUVDOHGXFDWLRQDOFRPSHWLWLYHSODWIRUP >@ + 3DYRORYi HW DO ³7KH &LUFXODU 0RGHO LQ 'LVSRVDO ZLWK
0XQLFLSDO :DVWH $ &DVH 6WXG\ RI 6ORYDNLD´ ,Q ,QWHUQDWLRQDO
-RXUQDORI(QYLURQPHQWDO5HVHDUFKDQG3XEOLF+HDOWK ,-(53+
5()(5(1&(6 0XOWLGLVFLSOLQDU\'LJLWDO3XEOLVKLQJ,QVWLWXWH9ROQR
>@ / ' 0HLQHUW * 5 5RELQVRQ  1 7 1DVVDU ³7KH FULWLFDO SS,661
LPSRUWDQFHRIWKHPDWHULDOLQWHQVLW\RI*'3´-RXUQDORI,QGXVWULDO >@ = âLPNRYi HWDO ³7KH UDWH RI XVH RI WKH FLUFXODU HFRQRP\ LQ
(FRORJ\YROQRSS LQGLYLGXDO VHFWRUV´ ,Q $FWD 0RQWDQLVWLFD 6ORYDFD  .RãLFH
>@ 7 ( *UDHGHO HWDO³2Q WKH PDWHULDOV EDVLV RI PRGHUQ VRFLHW\´ 6ORYHQVNR )DNXOWDEDQtFWYDHNROyJLHULDGHQLDDJHRWHFKQROyJLt
3URFHHGLQJV RI WKH 1DWLRQDO $FDGHP\ RI YRO  QR  SS 9ROQRSS,661
± >@ 0 7DXãRYD HW DO $QDO\VLVRI PXQLFLSDOZDVWHGHYHORSPHQW DQG
>@ 60-RZLWW+;LHDQG'*LXUFR7KHFULWLFDOUROHRIJHRORJLFDO PDQDJHPHQW LQ VHOIJRYHUQLQJ UHJLRQV RI 6ORYDNLD´
UHVRXUFHDYDLODELOLW\LQVFLHQFHDQGSROLF\(QYLURQPHQWDO6FLHQFH KWWSVGRLRUJVX   ,Q 6XVWDLQDELOLW\ 
3ROLF\YRSS %DVHO0XOWLGLVFLSOLQDU\'LJLWDO3XEOLVKLQJ,QVWLWXWHYROQR
>@ - ( 7LOWRQ DQG * /DJRV ³$VVHVVLQJ WKH LPSRUWDQFH RI PLQHUDO  SS>RQOLQH@,661 RQOLQH 
UDZ PDWHULDOV´ ,Q $QQXDO 5HYLHZV LQ (DUWK DQG 3ODQHWDU\ >@ -5\EiURYiHWDO³3URPRWLQJWKHDSSOLFDWLRQRI5(6WKURXJKD
6FLHQFHVYROSS$QQXDO5HYLHZV55\EiU0 SURFHVV RI LQYHQWLYH HGXFDWLRQ EDVHG RQ D FRPSUHKHQVLYH
%HHU - 5\EiURYi 6HQ]RULFNi SDPlĢRYi KUD V UR]KRGRYDFtP XQGHUVWDQGLQJ RI VRXUFLQJ LVVXHV  D FDVH VWXG\´  ,Q
þOHQRP=YHUHMQHQiSULKOiãND~åLWNRYpKRY]RUXÒ39 3URFHVVHV 0XOWLGLVFLSOLQDU\ 'LJLWDO 3XEOLVKLQJ ,QVWLWXWH 9RO 
65 QRSS,661
>@ 5 5\EiU HW DO 89 6. 9LDF~URYĖRYê KLHUDUFKLFN\ >@ 506FKRROV 9LUWXDO FHQWUH &RXUVHV $YDLODEOH RQOLQH
NRQILJXURYDQêER[ KWWSVUPVFKRROVLVRIFQULWPRRGOHFRXUVHLQGH[SKS DFFHVVHGRQ
KWWSVZEULQGSURSJRYVN:HE5HJLVWUH8]LWNRY\9]RU'HWDLO $XJXVW 
 >@ 3 +URQþHN % *UHJRURYi ' 7RPHW]RYi 0 0RORNiþ /
>@ 0%HHU55\EiUDQGď*DEiQLRYi9LVXDO,PSDFWRI5HQHZDEOH +YL]GiN 0RGHOLQJ RI 9DQLVKHG +LVWRULF 0LQLQJ /DQGVFDSH
(QHUJ\ ,QIUDVWUXFWXUH ,PSOLFDWLRQV IRU 'HSOR\PHQW DQG 3XEOLF )HDWXUHVDVD3DUWRI'LJLWDO&XOWXUDO+HULWDJHDQG3RVVLELOLWLHVRI
3HUFHSWLRQ3URFHVVHVYRO ,WV8VHLQ0LQLQJ7RXULVP &DVH6WXG\*HOQLFD7RZQ6ORYDNLD 
,Q5HVRXUFHV  
KWWSVGRLRUJSU

>@ 0 0RORNiþ * $OH[DQGURYi 0 .RE\ODQVND %  +ODYQRYi '
7RPHW]RYi³9LUWXDO0LQHHGXFDWLRQDOPRGHOIRU:LGHU6RFLHW\´



979-8-3503-7069-0/23/$31.00 ©2023 IEEE 428

 Smart home lighting control environment
R. Sabol*, P. Feciľak*, M. Michalko*, F. Jakab*
*
Department of Computers and Informatics, Košice, Slovakia
Richard.Sabol.2@student.tuke.sk, Peter.Fecilak@cnl.sk, Miroslav.Michalko@cnl.sk, Frantisek.Jakab@cnl.sk

Abstract— This paper explores the types of lighting, existing developed with Spring Boot, handles the backend
lighting models and available technologies. It deals with the functionalities ans system logic.
execution of an addition for a prototype designed on the
premises of university department (DCI, FEI, TUKE) in the
form of a web interface that allows remote control and
management by multiple users. Part of the solution is the
design of an analytical module providing an overview of
device usage in order to control the operation and further
optimization to minimize energy consumption.

Keywords— lighting, smart home, controller, web

application, interface, docker, analytics

I. INTRODUCTION
Smart home technologies are gaining popularity due to
their increasing affordability and ability to enhance Figure 1 System architecture
convenience, security, and energy efficiency. This thesis
focuses specifically on smart lighting, aiming to provide a OAuth authentication is integrated into the system using
solution that differentiates itself from existing options. Keycloak [1]. It also provides management of user
Many current solutions require complex installations and accounts and security using world-recognized standards
configurations, which can be impractical for users. The and techniques. In addition to traditional username and
goal is to develop a module for intelligent lighting control password authentication, the system also supports logging
that is compatible with various devices and does not in through Google Sign-In, enhancing the user
require an expensive and complex smart home ecosystem. experience.
The module will offer easy installation and
For data storage, the system utilizes both relational and
communication with lighting, independent of the local
network, through an intuitive graphical user interface. non-relational databases. Postgres occupies the role of the
relational database, while MongoDB serves as the non-
While existing solutions often focus on more complex
devices with limited compatibility across controllers, relational database, due to different data storage
proposed solution targets common devices without built- requirements within the system. MongoDB, a NoSQL
in smart components. Since this solution cannot compete database is known for its speed and scalability according
in terms of functionality with existing solutions whose to study [2], able to store extensive amount of data for
price matches the functionality provided. Therefore, further device usage analysis.
system specifies to common devices, not having smart To ensure efficient deployment and management, all
components. This includes light bulbs, coffee makers, application components are containerized using docker-
kettles, fans, LED strips, and other electrical devices. compose. Each component runs in its own docker
High-power appliances like toasters and coffee makers container.
could be controlled through relays. This opens up much Users access the system through a web interface, which
more possibilities for home automation. communicates with the server application using the REST
The knowledge gained about the functionalities, API protocol for data transfer. To ensure real-time
syntax, and limitations of the MeriTo PWM Controller updates of color changes and lighting intensity with
prototype helped application development to efficiently minimal delay, the WebSocket protocol was selected.
focus on approaches that take the device to the next level.
This protocol supports bidirectional communication and
II. ARCHITECTURE AND TECHNOLOGIES can handle a large number of requests in a short time,
making it well-suited for real-time applications and
The proposed architecture consists of various efficient network utilization [3].
components, including web application components, a To control the MeriTo PWM Controller, the server
server application, an authentication server, databases, application sends requests using the secured MQTT
and MeriTo PWM Controller. protocol with server certificate via the Mosquitto broker.
The web application components are implemented using The controller interprets these requests and applies the
Angular, providing a user-friendly interface for users to desired changes to the connected output devices.
interact with the system. The server application,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 429

The server application uses the email client to send channels, the web interface can present them as a single
notifications about controller activity. For this, the Gmail device, simplifying the control of more complex devices.
email client by Google was used, which was created This approach proves useful, for example, when
specifically for the purposes of this application. The same controlling RGB LED strips, where each color
client is also used by the Keycloak authentication system component can be controlled independently through
[1]. different output channels, to achieve full spectrum of
Notifications related to controller activity in real time colors with minimal effort.
with websockets are persisted for users in order to be kept A process was also implemented to recover from an
in case of their inactivity. They are also enhanced by inconsistent manipulation state due to device control by
sending an email with a defined HTML templates, physical inputs outside the web application. Taking into
depending on the type of notification, whether it is an consideration the human factor.
alert about activity on the controller or a request to
B. Controlled devices
connect to the controller. The subject of the email is
chosen subjectively by the user based on how the activity The application is based on a dashboard layout that
affects them. allows high customization for each user. Dashboard
In the case of multiple users having the same controller contains widgets of devices that makes their usage easier.
activity, all affected users are contained in a single email The application supports two different layouts: mobile
sent. This prevents the mail client from being view and desktop view. The type of view is automatically
overwhelmed by an excessive amount of emails. detected based on the display width of the device screen.
This allows having 2 independent layouts optimized for
different screen sizes, providing an optimal user
A. Controller PWM Prototype experience on both mobile devices and desktop
The MeriTo PWM Controller is a versatile device computers.
designed for remote or local control of various appliances
using DC voltage modulation PWM. It features 11 input The interface of the devices allows their manipulation
and output channels, enabling control of digital and in the form of changing states, and settings (name,
analog devices such as servos, relays, and LEDs. description, widget toggle and icon).
The communication with the MeriTo PWM
Controller is achieved through the MQTT protocol. The System offers control of two types of devices:
device communicates by sending messages to specific x The classic device provides binary control of the
topics, identified by the domain, MAC address, and devices and adjustment of the magnitude of the
action. The device supports commands for setting the power flow delivered to the device ports.
state, retrieving device configuration, and providing x The RGB device interface contains the stored
feedback on the device's status. preferences of the device color states. The
Commands sent to the device via MQTT have a maximum number of colors in the palette is set to
specific syntax, consisting of a command prefix followed 25 – Figure 2.
by the operation (com), output channel (X), and value (Y)
(sample 1). Various operations are supported, such as
setting the power, fade-in and fade-out effects, maximum
auto-off time, PWM device carrier frequency, and
changing the state of multiple output channels
simultaneously.
It is possible to chain and send multiple commands at
the same time. The prefix of a chained command is
determined by the structure:

Feedback from the device is provided through MQTT

messages, including information about the current state of
the output channels, temperature, and RAM usage
(sample 2). The device also sends availability reports to
indicate its online status. Figure 2 RGB device interface

The color palette is a visual tool that allows the user to

select a color using an interactive interface. The palette
However, since the MeriTo PWM Controller has consists of different shades and combinations of primary
single outputs, it may not directly support devices with colors. Most digital imaging devices use a 24-bit color
multiple inputs, such as LED strips, in its basic
representation, which allows for approximately 16.8
configuration. To address this limitation, a virtual device million different colors to be created.
solution was designed. By grouping multiple output

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 430

Also included in the interface is a slider, for both types of The automated application deployment process has
devices. The slider is used to adjust the value from 0 to been managed through the gitlab pipeline. This was
100, with the value changing continuously according to preceded by the creation of a script with defined
the position of the pointer. The value adjusts the final deployment stages:
brightness of the specified color combination. x Build
x Deploy
A color palette in an application may be subject to
interaction by multiple users, which means that it needs a Once the deploy stage is successfully executed, this
system to control access to it and to provide security in repository can be cloned from the server and used for
order to avoid conflicts of simultaneous use of the palette production application purposes by running the docker-
by multiple users. In this case, further interaction by other compose.
users is prevented until the first user has completed his
interaction [4]. The overall deployment process is partially automated
and optimized to improve the speed and reliability of
application deployment. It increases productivity and
III. UTILIZATION AND ANALYTICS
reduces the risk of human error in application
Device usage data is interpreted on the web application's deployment. Docker technology provides the solution
screen using various graphical representations. with eventual scalability and isolation from the boot
machine. A more detailed deployment procedure is
Although the MeriTo PWM Controller does not have described in the system manual [4].
built-in functionality for such monitoring, an alternative
solution was developed to provide consumption To ensure the best user experience and accessibility to
monitoring and visually encourage optimization of device the system, an Android application has been developed.
usage in form of bar charts. Having the application installed on their Android devices,
x The bar chart feature in the application provides users can easily and quickly access the system's features
a detailed view of the usage of all devices and perform various tasks without the need to open a web
managed by the user. Users can filter the period browser and navigate to the system's website.
of usage by selecting a range of times in the By deploying the system on university servers and
calendar or by using predefined time intervals. making it accessible to the public, combined with the
This allows users to analyze device usage availability of the Android application, users can enjoy a
patterns over specific time periods. full and versatile smart home solution that can be
x The pie charts give an overview of the usage of accessed anytime and anywhere. Whether at home, in the
controllers and devices based on the number of office, or on the go, users have the flexibility to manage
interactions. By comparing values across all their devices and control their lighting environment with
devices, users can easily identify the most and ease through the dedicated Android application.
least used devices. Users have the option to
display numerical values of controller and V. CONCLUSION
device usage, either as the amount of usage or as The objectives defined at the beginning of the thesis
a percentage. have been successfully achieved. By analyzing existing
x Bar charts are effective visual tools for lighting control solutions, weaknesses resulting from lack
presenting the duration of the on states of of compatibility for common electrically powered devices
individual controllers and devices. These graphs were identified. The proposed solution can compete with
offer users a clear representation of the time data these solutions in terms of versatile applicability and
such as the average on-time of the device, the compatibility with a wide range of devices. All
maximum on-time of the device and the total functionality is provided to the devices by the MeriTo
time of all on-states are included. By analyzing PWM Controller, which with the logic and interface of
this data, users can identify potential issues, the proposed application is a self-contained smart home
analyze device and controller operations, module.
optimize power consumption, and quickly detect By relying on the rules of user experience, an easy-to-
anomalies or irregularities in device and use web application was made. The interface was
controller behavior. designed in a dashboard style, providing quick easy
access to the controlled devices, but also a wide
configuration. The method of connecting controllers is
IV. SYSTEM DEPLOYMENT intuitive and illustrated step by step. Users of the
System deployment is the process of the final stages of application are allowed to participate together in the
development, which involves certain steps and management of the controllers. The interaction with
verification before the system is put into production devices and controllers offers real-time feedback to all
operation. Nowadays, automated tools are widespread users involved. Users are also notified of controller
to simplify and speed up this process. activities by email messages. Such notification even
outside the web application can be a crucial factor in case

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 431

of unauthorized intrusion into the controller in case it is It is possible to extend the application with access
not secured. rights management. Currently, only the owner of the
Presented paper introduces the issue of lighting controller is authorized to remove it, and all users of the
sources and their control methods. It transformed the controller are involved in access management, for
shortcomings of the analyzed solutions into its advantage. example, accepting connection requests, changing
The support of "non-intelligent" devices presents a wide attributes, adding users. Extended functionality would
range of applications and brings more affordable smart provide the granting of hierarchical levels of rights that
home modules. The analyzed prototype of the MeriTo define the permitted actions against the controller.
PWM Controller at Department of computers and Further extension is possible in the area of localization
informatics, Faculty of electrical engineering and into multiple world languages for better user experience
informatics, Technical university of Kosice helped to get for users from all over the world.
a better idea of the direction the application development
will take and what aspects will need to be focused on. Extension of the application by future development is
Known limitations and weaknesses of the controller were simplified as documentation of the server application
treated by our application and the functionality of the methods has been developed. Better understanding of the
controller itself was increased. For this reason, the code reduces the probability of errors when developing
possibility of not only entering the automatic shutdown the application and increases the speed and efficiency of
time, but its daily repetition was created. A feature of the the developers and also helps with maintenance of the
application that has elevated the usability of the controller application. As a result, developers can more quickly find
is the support of virtual devices. The idea is to group and fix bugs that may occur while the application is in
multiple device ports into a single device, which use.
simplifies their simultaneous control. RGB-type devices
have been developed in a similar way. Multiple device ACKNOWLEDGMENT
ports were combined into a single device with each port This work was supported by Cultural and Educational
representing a different color component. This has made Grant Agency (KEGA) of the Ministry of Education,
it possible to create a wide range of colour combinations Science, Research and Sport of the Slovak Republic
and the ways in which they can be used in the home have under the project No. 026TUKE-4/2021.
expanded even further.
REFERENCES
The application is suitable for the general public and [1] CHATTERJEE, Ayan; PRINZ, Andreas. Applying spring security
will find its utilization in households or companies that framework with KeyCloak-based OAuth2 to protect microservice
architecture APIs: a case study. Sensors, 2022, 22.5: 1703.
prefer high device compatibility without the need for [2] JUNG, Min-Gyue; YOUN, Seon-A; BAE, Jayon; CHOI, Yong-
highly specialized third party products for special Lak. A study on data input and output performance comparison of
centralized units which are expensive. mongodb and postgresql in the big data environment. In: 2015 8th
international conference on database theory and application
(DTA). 2015
A limitation of the solution is the possibility to use
[3] PIMENTEL, Victoria; NICKERSON, Bradford G.
each port of the MeriTo PWM Controller only once. This Communicating and displaying real-time data with websocket.
measure provides better control over the performance of IEEE Internet Computing. 2012, 16, pp. 45-53. isbn 978-0-7695-
the devices and avoids the higher voltage drops on the 4367-3
ports that would result from connecting multiple devices [4] SABOL, Richard. Prostredie pre riadenie osvetlenia inteligentnej
domácnosti. Košice, Technická univerzita v Košiciach.
to a single port.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 432


        

     

  
 


  
!  "
 #    $" # %   "&% ' ()*+,-+.+.*/&
0#12   # # # #   34" !'



              #  !  
                #!  " #   !       
                 #  " #     " 5     
                "  # #      ! 
                     " !    
         =  #      #!   
                #       " #    # 0
            !# #   !  7!  5 "    
             
            " # 
         
  ! #          ! 
      !      "  
   5   
 85
95
 
               
           
 "        !      
             
" #   !  5      " 
              
!           
                        #%5
 
        #$%               !   " #
          !              
                    
!           !         
&'     #$% 5
! # "  #  !     ""  
""!         
5
  
 # 7""! "   ! "    !    "
$
$56%5$6 !  ! "!     " #    
   #!        #     !  #      !   6
 #   "    "!   " # '       #  "     ! !  
!       !                           !     #
   #   "         !#  '    #     #        # 7 
 # 7  "  #!          5     6      # "!   #
!#  "               " #          #      0  "  "
 #    ! #      !   # #   5         
!  !'    !  "      !   #!       " !       #   "
89:+;:*;5 "            #   7!      "   
   #  # #    #    "     8>*9      8>?9      ! 
       !  " # #   !     !  "!      #   "
 # #  !     "    ! 0   "      @! "!
<!  !      # #        !   #"    " 0 ! 
#   #!    5      !            !         "
   !    " #     "   !          "  ##!    
  5       !      !  :?;
  #      # #      $          " !      "
 "!     #  $"          #  !   " # #     
#     #                #   "# # !  !    
           # #         ! #     " 
#      "   "  ! !      # " #!  5  # 
#  5 "  !                      !  "  ! #     
# #     <!     !   # #      "         ! 
"    #   #                 #  !5      
#      #       !     !      #       !'   "
   "  # #   #!             #     " # "
5  # #!      "      #        "   $6 8$   
# # #      !    6 '   "   '  9 6$ 86   #
            5 $     98"!   A>?A9     
8"!   A>*A95     !      

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 433

 "     !  " ##!    !   @    "   ! #    
 "!       !           "!         "!    
 ##!     "     "!           
5      !   ! "  8#    9    8#    9
       "    !  $      
     " 7  " #      
    !     "    !           #       #   "  
    !   #$ " !  !    #    " # 
    " "    



 1 5    "       
  7 #   " "   "!    7       <!        " 
" #                 
$$
5
B%6566>   



 @  1 $     "          0


     
   #     $    
     ! " #    #        
5       <!  " #   " #   
# 
     !      !



    1 @        
8#  
 #    9=
$8=  
@       "! 

 #  $ !#    95  C8 ! C 9 8 !   "       9
#   "        
#   5 "     "  #'  


 #  $" #   8
$91@   
" #     #        "    '     " #  
    <!   "      #        "       
$   F    " 
  #     <! #   "  " 86$9 !#0   "  
     

6F   $  "  86$91      
  (  %   # !   " #   !  #      
5  C       "     # $  
#    #!         
     
5     !         
 ##   " #      "       5  !   "   7 #          
"   "  # C   5    !          !        "
 ##   "   !     5  C      "  #
    '     
 "    #   !      #   " #     "  ! !     
#  #   C          "  !    #    #     C  # 
 !    !        # -       "!   F!  5  ! 5  
 "  !             C      !  =
$    #     
 "!      **-5  !       " #  " #=  #!  #   
#     !  "        #      " #  
 "  ! !       
  
)   "     *)$"+
  "  !         #   5
=
$  " "  " #
  "   !"     " !   #   7  
  # #   "         "  !   C    ! " !
 #!   #      =               !         "   
 #869=
$       ##   0 " ! !   
  " 8>$9" =
$  =
$5 
      #     7     
       
 ##!   !   "  =    
6   #    #    # 
@  "     !     
       #  ""    5  " ! 
   " #    !      ! "  #!   0     !>>
  !    " #  " #   
 "!  #!  8 !   #   
     !        
 # 9 !      ! ! 
  " #  !  #   "  
   ""8 9
"           6 8     #9
 # $  %  *#$%+         " # $      " 

   :D;  ! $    #  8  # 9  "      
    !  E     #   #   #       G>@

 !       '   "    !# 
@     " #  
" #   !    
!     0          >*  8    
  #         #    "   "     9              >> 8>
8

 9   " #   !    <!   >     9  '   8$
8 0  !9 "    "       #   8
 ).*+@9       >*       
@  9  !                " #      

   "!       !        !  
  5   !  
  
 @      " #      
  !       >*      >*

    !    <!   8! #  9 
 #:,;

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 434

 $$$
5
05=6H@55$%5 6!   !     "   #     
5
 8  5   
 9          "   #!" !  5@> + !## ' 
   !  !' '    #   " #   !       ! 
      "  ##!     $ !    6$
           $ 8$     9 5   !     ! #   " ! 
  =      # "! #       #' "  #     = ! 

"         5
  !   #!       * "   !    5    "  ! 
 "  !  "           !F      "         !  "  #    
"! #      " #     #$ !     #  # #      #    
          #  !    ""      5      !
#   #!" ! @ >*>?      " #  !     
       "   "   !    " !    ## #   !  5 
  #
    8"  7#        9    7  A ##!  A
! 8  ! 
!         ! "          "   !#  "  ##!   9 $     
       !   !      $    <!   $        
     "    !  " !           
 <!  
       #

! "    


     @ !  " !5
        ! 
          !'   "        #       !'  
      #             "   ! ! 5 "  "!'  #    "
! #     5       "             
 "!      "  #      "    !   " #$    " "  ! ! 
   !'   "              !'        " !    
  $    "     #               # 5 
     7!          !'     "  # !  "   "
 #    #      !F   "     !!       !   "" 
!       !  "  7#    !  I 05 
@#        #  !     #  6 I G' " #:/;   :J;  
   #!" !  " #         5         # "     "" 
 #  #   !<!   "     #   !          " 7    
  =   ! " #         
"     "     " #        5 !'   "     "   !F  
   "       $   " !      "  !  "    
 #
@    =                !   " !    ! <!  
" #   !    !    "        #' "      "       >*
      !        # #  >?  ! <!         " ! ! ! 
!   5       !    !    "   !'   C        
     " #      "  "   6    # #       !'   ! !  "
86       9    " #             ! ! = 
 !     $         !  $     !'   "     8>?9   
8$  " 9 $ !   "   "  !  "     #      ! !    8   
!'     ! " #  " #>*  9             5  ! 
    8->>9      !  !'    !  "   """  
      =        !           " !'   ! = 
" #   @    8@    !     9  !' 8>*9   #   
      !  " #   !     !!  !    

@    $   "       <!                 "  7 
   !       "     8A A9     "    !    "   
@
8   @  
# 9    "  $             ! 
5     ! #        !'  !   A  A"!  $  
     #  "     !'       !           !   !
"      #! !   #  #      #   # 7    # #  
""     !  #  

       
$G
@>I6$5C
66@>

          ##!   !     "
#        7 " #   !    $   # 7  # "  !  "   5

!  " 
 F   86$ 0 6F   $9 5     #                  
#  ! I    I  7 I !5    #1
$" # $ !  !        ! 

        #" !    
    7  !#  =     "  " #   "  ! ! 5   #
   #        "   !      #      ""     
    " 
 6$  7   "   " 
     !   "  
$ 

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 435

 5@>+

6>>5@55$%5 


! +@  #" ! #      " ! " #  

 

  " #   !         ! 

!  #              #
!       #       >*>?    
" #    " #  

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 436



        #"      #  !    +.8  " #  " #
" #   "               0   7        " 9$"  
" #!  "   !'     "          =
$    #
#        "  ! !    !    K
   #   " # K 1$"   "    8
C=
$9
 !     #   !   #   



  #$  $        $       #  "   # " #   8#!" ! 
#   A  # A#  6  6  
@  "
5   #" ! #      "    "    5-%      !9
" #  " #   "  ! !  # !  5     +*8     " #   
 
  $     
      9
             "!C +.  1 5              
=
$ !        !F   ""!   " #   " #    !   "    
  5   " #  ! <!    8
 C =
$9 @"           
    895   #        #    ++8      !
! +  " "      " 9
+ 1 "   $         ++ 1$  #!    " #  
    !        "  
 8A ##!   !      !     " 85
 A9C8#  9=
$8#  LM .0/,,?, % LM .0+.*D9 @"        
 9   #   !      +* 8     
*  15             "    ! "! " #      9
  !     ? $"    !! "! !    
  8  +9     #      
"          
?  1 !   #        " 
 $    " #   !     
    5-% $  "  
8%-+/+9
C 85-**9  =
$ 85-+?,9     
 !           "  
  8$  9      
D 1G "              "
  #   " #     !   ,$"  
" !     + !    # 
"           
  
, 1!            
       #      
!  5     "  "       " 
         #  $ 
   $   "      $"  
#  #          $ 
     $"    D   
      " #     $"           
    !    /

/ 1 "
       $"

! *@  #" ! #      " ! " #  
       #  "        
 
 <!         # $"  
       
 <!      +. +* 1@  " #   !      
8  " #  " #    I   7  "         8"   "    "        
       "  5@>+9$"   ! !      0A   A
           
  "!     "    $   9    #
  !    J !   D8      7   " #   9
J 1G "  C      $"   5     #  !   " !
C        "   "!       #    @" ! "! "  
      C$"         "  ! !            
! "!  !    +.8  " #  " #     ! 5" #     !  
   0  7        " 9$"   7 !   ##         
     C   !   )   " # " 8>?9 8>*9   
) 1     "  =
$      #5    #      
     $"   =
$        
  #"!     "!        

  #$    %       #%
 "      #        @"  @ #      "   ! "!    
! "!           ! =
$   "              

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 437

  #             "    5  !   !    "    
          ! ! " #   "    ""  ! " ! >?   
!   "  !'   $ !        "   " #   #  ! " #O!  
I 
> " #  "          =    !  " #
  =    !   
!      #       >* 6  !     $      "  
     >?   #@"            G>@   $ ++. +*. 
 "             +?. 
            
" >?               " #   
    *    #  ##!  
 !  "     "  ! !        "  8A!A9      7  "  ! A+A
           7   7     A=+A6        C  
A$  A 5              !   ! ! A?A 
C  
  "   "     A "!  ! A      ! "   ##!    @  
8 #    ! ....9 !      >>  
     "       #  "        >?     6   >>    
  #  " # ""    ! D ! 
   5   " #O! 
I?   >>   "! 


! ?5       ! 


! "&'  

@"  ! "!       " ! #   
     " #     "  ! !    
 #   !            
  #        8>?9     
  5      I 
>"   
! D>?  #B!      ! !    #    !'   
8   !  95 ">? # 
G
N$
5@>G$ $@5$6  ! /
5   " !    #   " #  7 #  
 "    # #    #    " # "   
 !    I%$ " 



%%    
  # #         "   "  
 #    @  # # #           
     ## !     
   ! !     I 
>" # 
! '    N  :);@   ! 
 A#A  " 
  A # A  " 
C    A #A  " =
$    
A  0#A       #=   
!  ! #   #      " #     
 ! 
      ! >!7

G
 5             #  
    7 !   "   7     5  ! ,5     
   !  " !      ! ?

!    
5       !     !  
 #               I? 
8 ! ,9

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 438

 $"  7#    >?     !  !    ! C"  =
$ "
   "  !  "           " <!      !   #
        =         5        ! !   #   

                   7    
7    !                G "   "     !   ! 7 #  
8" #   A  A "       !  9   # # #       #       
6     " #              "       5! 
        8>?9     #          !  "       #   "
   "!    #          
" !   !     @   
       !  #  "   F      #0
   !  !  !  F  0   !F  


! /5  !  8>?9   


! "&(  

       # "     >*      ! J5  !  8>*9   
        I 
> "      
" #    !       "         8>*9
@H6=>I
5
  % #  "       
    F! #     !       5    !     HI@ .,+R%0D-*.*+
  ! J !         S5     "   !  #  !  
      !   =        !  T F  
       E+E  E*E !      
 
 
     8EGPI E 
EI?P
 E9           !   :+; O!S  !'   "   " 
 "  ! #! 1!  T$ ##!

     8E 0$
$E9 C  O!    ,+   ++  *DB?+ *.+?  1
!        >>   +.++.K-
6
*.+?//,)/D)
8   9            :*; N > O   U >  V N! S5   ! "    
   5     !    "            # "   #'     #   

T  5  /  $     "      
 "    6   !       #!   @  $" #  
 #   *.+. 
      !""   !     Q      +/+B+/?
<!       >>       :?; N = H H!  =  N! C >!  I  >!
 "!   S# !     !#         
   #7    TO6  ##!  / 
G$
6>%$6 +*+++,B++*J*.+D
:D;  ! O
   HS ?D+,G  *
$          !   # "      6    "    #   
 #  
#      ! " !    ! #     8
9T *..*
     "  ! ! <!      "  1--    " - - #- "?D+/
  #  !         :,; C    C  > N   R V!  S
            T  *.+.
   #   6!     #  #    $     "     #!  @     #
        8>?9  !  
 8$@
*.+.9*.+. +DG+D0?.)0G+D0?++
        "      8>*9  1+.++.K-$@
*.+.,/***K/
        "    !     !  :/; SI 6 T 1--  -
        "  !  !  ! '    :J; S  6 T:6 ;@ 1 1--   -

    C        
    :); S 76 T :6 ; @ 1
 1-- 7 - !#    -  - 7 #


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 439

 8VLQJDKRQH\SRWWRLPSURYHVWXGHQW
F\EHUVHFXULW\DZDUHQHVV

0âLPRQ /+XUDM DQG'+ULQNLQR 
8QLYHUVLW\RI66&\ULODQG0HWKRGLXV,QVWLWXWHRI&RPSXWHU7HFKQRORJLHVDQG,QIRUPDWLFV7UQDYD6ORYDNLD
ODGLVODYKXUDM#XFPVNPDUHNVLPRQ#XFPVNKULQNLQR#XFPVN


$EVWUDFW 7KLV DUWLFOH IRFXVHV RQ WKH HYDOXDWLRQ RI WKH SUDFWLFHVEXWDOVRLQFOXGHVSUDFWLFDOVNLOOVDQGH[SHULHQFH
HIIHFWLYHQHVV RI KRQH\SRW DFWLYLW\ DV D WRRO WR XQGHUVWDQG >@
F\EHUVHFXULW\ SULQFLSOHV E\ XQGHUJUDGXDWH FRPSXWHU VFLHQFH $ TXDOLW\ F\EHUVHFXULW\ HGXFDWLRQ SURJUDP VKRXOG
VWXGHQWV ,W H[DPLQHV WR ZKDW H[WHQW DQG LQ ZKDW ZD\V WKLV LQFOXGH QRW RQO\ D TXDOLILHG LQVWUXFWRU EXW DOVR FXUUHQW
DFWLYLW\ FRQWULEXWHG WR WKH DFTXLVLWLRQ RI NQRZOHGJH DQG FXUULFXOD DQG LQWHUDFWLYH H[HUFLVHV 7KLV FDQ LQFOXGH
WHFKQLFDO VNLOOV UHODWHG WR KRQH\SRWV DQG F\EHUVHFXULW\ %\ YDULRXV IRUPV RI SUDFWLFH LQFOXGLQJ D VWLPXODWLQJ
DQDO\]LQJ VWXGHQWV  UHVSRQVHV WR D QXPEHU RI TXHVWLRQV WKDW &DSWXUH WKH )ODJ W\SH HYHQW DV ZHOO DV UHDOLVWLF
IRFXVHGRQGLIIHUHQWDVSHFWVRIWKHDFWLYLW\WKHLUH[SHULHQFHV VLPXODWLRQVRIVHFXULW\WKUHDWVDQGEUHDFKHVWRJHWVWXGHQWV
NQRZOHGJHDQGDWWLWXGHVWRZDUGWKHWRSLFZHUHHYDOXDWHG7KH H[FLWHG DERXW WKH FXUUHQW FKDOOHQJHV RI WKH ILHOG >@
UHVXOWV DUH SUHVHQWHG ZLWK UHVSHFW WR WKHLU LPSURYHG
$FFRUGLQJ WR WKH ILQGLQJV SUHVHQWHG LQ WKH VWXG\ >@ WKH
XQGHUVWDQGLQJRIWKHFRQFHSWRIDKRQH\SRWWKHLUSHUVSHFWLYHV
RSWLPDODSSURDFKWRF\EHUVHFXULW\HGXFDWLRQDSSHDUVWREH
RQWKHYDOXHRIWKLVDFWLYLW\LQWHDFKLQJF\EHUVHFXULW\DQGWKHLU
WR FRPELQH D VWUXFWXUHG VLPXODWHG HQYLURQPHQW ZLWK
RSLQLRQVRQWKHVXLWDELOLW\RILQFOXGLQJWKLVWRSLFLQWKHFRXUVH
LQWHUDFWLYHOLYHDFWLYLWLHV&RQVLVWHQWZLWKWKLVREVHUYDWLRQ
FXUULFXOXP
WKHPRGHOFKRVHQE\WKHDXWKRUVLQWKHVWXG\ZDVRQHWKDW
LQWHJUDWHVUHDORSHUDWLRQDOGDWDIURPDXQLYHUVLW\QHWZRUN
, ,1752'8&7,21 ILUHZDOO VR WKDW VWXGHQWV FDQ JDLQ KDQGVRQ H[SHULHQFH
ZLWKUHDOWKUHDWVWRHQULFKWKHLUOHDUQLQJSURFHVVHV
7RGD\ ZKHQ FRPSDQLHV DUH FRQVWDQWO\ IDFHG ZLWK WKH
JURZLQJWKUHDWRIF\EHUDWWDFNVDQGXQDXWKRUL]HGDFFHVVWR $Q H[DPSOH RI D F\EHUVHFXULW\ ODE WKDW LPSURYHV WKH
LQIRUPDWLRQ WHDFKLQJ F\EHUVHFXULW\ LQ WKH ILHOG RI WHDFKLQJ DQG OHDUQLQJ H[SHULHQFH ZKLOH WDNLQJ D
FRPSXWHUVFLHQFHLVEHFRPLQJ PRUHDQGPRUHLPSRUWDQW F\EHUVHFXULW\ FRXUVH LV JLYHQ LQ >@ 7KH SURSRVHG
5DSLGWHFKQRORJLFDOGHYHORSPHQWVDQGWKHLQFUHDVHLQWKH F\EHUVHFXULW\ ODE PRGHO LQWHJUDWHV 6,(0 DQG KRQH\SRW
FRPSOH[LW\ RI LQIRUPDWLRQ V\VWHPV UHTXLUH IXWXUH WRROV DQG HIIHFWLYHO\ FRQWULEXWHV WR LPSURYLQJ
SURIHVVLRQDOVWREHZHOOSUHSDUHGWRPHHWWKHVHFKDOOHQJHV F\EHUVHFXULW\WHDFKLQJDQGOHDUQLQJ
DQG WR EH DEOH WR HQVXUH WKH FRQILGHQWLDOLW\ DYDLODELOLW\ 6LPLODUO\WKHODEVLQ>@XVHKRQH\SRWVIRUHGXFDWLRQDO
DQG LQWHJULW\ RI GDWD $Q HIIHFWLYH ZD\ WR DFKLHYH WKLV SXUSRVHV6WXGHQWVKDYHWKHRSSRUWXQLW\WRXVHKRQH\SRWV
UHDGLQHVV LV WR LQWHJUDWH F\EHUVHFXULW\ FRQFHSWV DQG DQGJDWHZD\GHYLFHVRQWKHQHWZRUNWRGHYHORSWKHLUVNLOOV
SUDFWLFHV LQWR WKH FXUULFXOXP ZLWK WKH SUDFWLFDO LQ VFDQQLQJ DQG DWWHPSWLQJ WR EUHDN LQWR WKHVH GHYLFHV
DSSOLFDWLRQRIWKLVNQRZOHGJHSOD\LQJDQLPSRUWDQWUROH 7KH\ FDQ DOVR OHDUQ WR PRQLWRU WKH DFWLYLWLHV RI SRWHQWLDO
2QHRIWKHNH\WRROVWKDWKDVEHHQHVWDEOLVKHGLQUHFHQW DWWDFNHUV 7KH PDLQ JRDO LV WR SURYLGH VWXGHQWV ZLWK DQ
\HDUV DV D YDOXDEOH PHDQV RI GHPRQVWUDWLQJ HGXFDWLRQLQERWKDWWDFNWHDP UHGWHDP DQGGHIHQVHWHDP
H[SHULPHQWLQJ DQG JDLQLQJ SUDFWLFDO H[SHULHQFH LQ EOXH WHDP  VNLOOV DQG WR LQWURGXFH WKHP WR YDULRXV
F\EHUVHFXULW\ LV WKH FRQFHSW RI WKH KRQH\SRW >@ $ VHFXULW\ WHFKQRORJLHV 7KLV ZLOO HQDEOH WKHP WR DFWLYHO\
KRQH\SRW LV D VSHFLDO NLQG RI GHFR\ WKDW OXUHV SRWHQWLDO SXUVXH UHVHDUFK RQ ILQJHUSULQWLQJ KRQH\SRWV GHYLFH
DWWDFNHUV LQ RUGHU WR GLVFRYHU WKHLU PHWKRGV LQWHQWLRQV PDVNLQJDQGDWWDFNDQDO\VLV
DQG LGHQWLI\ QHZ WKUHDWV 7KLV WRRO SURYLGHV D XQLTXH ,QWKLVDUWLFOHZHUHIOHFWRQWKHLPSRUWDQFHRIWHDFKLQJ
RSSRUWXQLW\QRWRQO\WRWUDFNUHDODWWDFNDWWHPSWVEXWDOVR F\EHUVHFXULW\ DQG DQDO\]H KRZ D KRQH\SRW LV D VXLWDEOH
WR JDLQ DGHHSHU XQGHUVWDQGLQJ RI WKH WDFWLFV WHFKQLTXHV WRRO WR KLJKOLJKW WKH SUDFWLFDO DVSHFW RI WKLV LVVXH 2XU
DQGSURFHGXUHVXVHGE\DWWDFNHUV,WDOVRSURYLGHVLQVLJKW DSSURDFK LV EDVHG RQ DQ DQDO\VLV RI WKH UHVSRQVHV RI
LQWR QHWZRUN RSHUDWLRQV IURP WKH DWWDFNHU V SHUVSHFWLYH XQGHUJUDGXDWH FRPSXWHU VFLHQFH VWXGHQWV WR D VHULHV RI
ZKLFK FDQ FRQWULEXWH WR D PRUH HIIHFWLYH F\EHU GHIHQVH TXHVWLRQV WKDW IRFXV RQ WKHLU DWWLWXGHV WRZDUGV WKH WRSLF
VWUDWHJ\ >@ $ KRQH\SRW LV D VHFXULW\ VRIWZDUH RU WKH NQRZOHGJH DQG VNLOOV WKH\ KDYH DFTXLUHG DQG WKHLU
KDUGZDUH WRRO WKDW GHOLEHUDWHO\ VLPXODWHV D YXOQHUDEOH SHUFHSWLRQVRIWKHYDOXHRIWKLVDFWLYLW\LQWKHFODVVURRP
HQWLW\LQRUGHUWRDWWUDFWDQGWUDSDWWDFNHUVDQGH[SRVHWKHLU +RQH\SRW DFWV DV D WRRO IRU DQ HPRWLRQDO OHDUQLQJ
PHWKRGV IRU HIIHFWLYH FRQWURO ,W HVVHQWLDOO\ DFWV DV D H[SHULHQFH IRU VWXGHQWV 7KLV HPRWLRQDO H[SHULHQFH QRW
VRSKLVWLFDWHGYLUWXDOWUDSIRUSRWHQWLDODWWDFNHUV>@ RQO\ FRQWULEXWHV WR D GHHSHU UHWHQWLRQ DQG XQGHUVWDQGLQJ
*LYHQWKHLQFUHDVLQJSUHYDOHQFHRIVRSKLVWLFDWHGF\EHU RI F\EHUVHFXULW\ FRQFHSWV EXW DOVR HQKDQFHV WKH
WKUHDWVLWLVFULWLFDOWKDWVWXGHQWVLQFRPSXWHUVFLHQFHDQG HIIHFWLYHQHVV RI WKH OHDUQLQJ H[SHULHQFH E\ VSDUNLQJ
VHFXULW\ GHJUHH SURJUDPV KDYH D WKRURXJK SUHSDUDWLRQ WR VWXGHQW PRWLYDWLRQ DQG LQWHUHVW LQ WKH VXEMHFW ,QFUHDVHG
GHDO ZLWK WKHVH FKDOOHQJHV7HDFKLQJF\EHUVHFXULW\ LV QRW DZDUHQHVV RI F\EHUVHFXULW\ E\ XVHUV FDQ EH DFKLHYHG
MXVW DERXW WKHRUHWLFDO XQGHUVWDQGLQJ RI ULVNV DQG VHFXULW\ SUHFLVHO\WKURXJKHPRWLRQDODQGFRJQLWLYHSDWKZD\V>@
7KH DELOLW\ WR HQJDJH WKH DXGLHQFH DQG WUDQVIRUP WKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 440

DFWLYLW\ LQWR DQ HPRWLRQDO H[SHULHQFH EHFRPHV D NH\ ZKHWKHU WKLV DFWLYLW\ VKRXOG EH LQFOXGHG LQ WKH FRXUVH
IDFWRU LQ VWLPXODWLQJ WKRXJKW ZKLFK DOVR FDXVHV D V\OODEXV
WKLQNLQJ SURFHVV LQ WKH ILHOG RI F\EHUVHFXULW\ DV ZHOO LY 3UDFWLFDODSSOLFDWLRQ D 6HFXULW\SROLF\H[DPSOHV
>@ ,Q DGGLWLRQ WKH GHVLJQHG DFWLYLW\ SUHSDUHV VWXGHQWV 6WXGHQWV DUH JXLGHG WR SURSRVH VSHFLILF VHFXULW\ SROLFLHV
IRU UHDOLVWLF VLWXDWLRQV WKH\ PD\ IDFH LQ WKH IXWXUH DV DQGPHDVXUHVWKDWFRXOGEHLPSOHPHQWHGLQDUHDOEXVLQHVV
F\EHUVHFXULW\ SURIHVVLRQDOV ,Q WKLV ZD\ D VWURQJ HQYLURQPHQW EDVHG RQ WKH DQDO\VLV RI KRQH\SRW GDWD
IRXQGDWLRQLVFUHDWHGIRUIXWXUHSURIHVVLRQDOVZKRZLOOEH E 8VH IRU JRYHUQPHQWDO DXWKRULWLHV H[DPSOHV DUH
DEOHWR HIIHFWLYHO\ PDQDJH WKH JURZLQJ FKDOOHQJHVRIWKH SUHVHQWHGDWWKHHQGRIWKHFRXUVHRQKRZKRQH\SRWVFRXOG
GLJLWDO ZRUOG DQG FRQWULEXWH WR LWV VHFXULW\ DQG UHVLOLHQFH DOVR VHUYH JRYHUQPHQWDO DXWKRULWLHV LQ WKH ILHOG RI
WRF\EHUWKUHDWV F\EHUVHFXULW\
,Q WKH IROORZLQJ VHFWLRQV RI WKH DUWLFOH ZH SUHVHQW WKH Y  &RQFOXVLRQ DQG UHIOHFWLRQ DW WKH HQG D MRLQW
PHWKRGRORJ\ XVHG D PRUH GHWDLOHG DQDO\VLV RI VWXGHQW UHIOHFWLRQ RQ WKH DFWLYLW\ LV GRQH ZLWK WKH VWXGHQWV
UHVSRQVHVDQGDQHYDOXDWLRQWKDWZLOOVHUYHDVDEDVLVIRU 6WXGHQWVZLOOH[SUHVVWKHLUIHHOLQJVLQVLJKWVDQGLGHDVIRU
UHIOHFWLQJ RQ WKH LPSRUWDQFH RI WHDFKLQJ F\EHUVHFXULW\ LPSURYLQJ WKLV DFWLYLW\ LQ WKH FODVVURRP ,W LV DOVR DQ
XVLQJWKHKRQH\SRWFRQFHSW RSSRUWXQLW\ IRU WKH WHDFKHU RU LQVWUXFWRU WR UHFHLYH
IHHGEDFNIURPWKHVWXGHQWVDQGWREHWWHUXQGHUVWDQGWKHLU
,, 7($&+,1*0(7+2'2/2*<86,1* LQVLJKWVDQGLPSUHVVLRQV
,17(5$&7,9(+21(<327$&7,9,7< 7KLV WHDFKLQJ PHWKRGRORJ\ FRPELQHV WKHRUHWLFDO
7KLVVHFWLRQSUHVHQWVDWHDFKLQJPHWKRGRORJ\WKDWXVHV NQRZOHGJH SUDFWLFDO VLPXODWLRQ DQG UHIOHFWLRQ ZLWK D
WKH KRQH\SRW FRQFHSW DV D WRRO IRU GHPRQVWUDWLQJ DQG IRFXVRQFUHDWLQJDQHQJDJLQJDQGHGXFDWLRQDOH[SHULHQFH
SUDFWLFDOO\ DSSO\LQJ F\EHUVHFXULW\ NQRZOHGJH 7KH IRU VWXGHQWV LQ WKH ILHOG RI F\EHU VHFXULW\ 8VLQJ WKH
PHWKRGRORJ\ FRPELQHV WKHRUHWLFDO H[SODQDWLRQVSUDFWLFDO KRQH\SRWFRQFHSWDQHPRWLRQDOH[SHULHQFHLVFUHDWHGWKDW
LPSOHPHQWDWLRQ RI DWWDFNV DQG VXEVHTXHQW DQDO\VLV RI DOORZV VWXGHQWV WR EHFRPH PRUH DZDUH RI WKH WKUHDWV RI
KRQH\SRW GDWD WR SURYLGH DQ HQULFKHG DQG HPRWLRQDO F\EHUDWWDFNV DQG WKHLU LPSRUWDQW UROH LQ SURWHFWLQJ
OHDUQLQJH[SHULHQFHIRUVWXGHQWV,WFRQVLVWVRIILYHSDUWV LQIRUPDWLRQV\VWHPV
L  7KHRUHWLFDO EDFNJURXQG WKH ILUVW SDUW RI WKH FODVV
IRFXVHV RQ JHWWLQJ WKH VWXGHQWV WR WKH VDPH OHYHO RI
NQRZOHGJH DERXW WKH *18/LQX[ RSHUDWLQJ V\VWHP DQG
VSHFLILFDOO\ WKH SULQFLSOHV DQG PHWKRGV RI DFFHVV YLD WKH
66+SURWRFROWKURXJKOHFWXUHIROORZHGE\GLVFXVVLRQ7KLV
VWHS LV QHFHVVDU\ VR WKDW VWXGHQWV FDQ ODWHU DFWLYHO\
SDUWLFLSDWHLQWKHKDQGVRQSDUWRIWKHKRQH\SRWDFWLYLW\
LL  +RQH\SRW DFWLYLW\ D  &RQGXFWLQJ DQ DWWDFN DQG
KDFNLQJ WKH URRW DFFRXQW 6WXGHQWV DUH DVNHG WR WU\ WR
YLUWXDOO\JDLQDFFHVVWRWKHURRWDFFRXQWDWDVSHFLILHG,3
DGGUHVV E\ DQ\ PHDQV IRU H[DPSOH JXHVVLQJ WKH
SDVVZRUG  $FFHVV WR WKH DFFRXQW ZDV WKURXJK WKH 66+
SURWRFRO7KHGHOLEHUDWHO\FKRVHQZHDNDFFRXQWSURWHFWLRQ
DOORZV PRVW VWXGHQWV WR JHW LQWR WKH URRW DFFRXQW ZKLFK
DFWXDOO\ EHORQJV WR WKH KRQH\SRW 6WXGHQWV DUH QRW
LQIRUPHG WKDW WKH\ DUH LQ WKH KRQH\SRW ZKLFK LV DQ 
LPSRUWDQWDVSHFWRIWKHPHWKRGRORJ\ E +RQH\SRWUHYHDO
)LJXUH 7KHFRXUVHRIWHVWLQJWKHPHWKRGRORJ\RQDJURXSRIVWXGHQWV
DQG H[SODLQ DIWHU DWWHPSWV WR JDLQ DFFHVV VWXGHQWV DUH
LQIRUPHG WKDW WKHLU DFWLYLW\ KDV EHHQ UHFRUGHG E\ WKH
KRQH\SRW 7KLV PRPHQW H[SRVHV WKH WUXH QDWXUH RI WKH 7KHPHWKRGRORJ\ ZDVWHVWHGDWD6ORYDNXQLYHUVLW\RQ
DFWLYLW\DQGFUHDWHVDQLPSRUWDQWHPRWLRQDOH[SHULHQFHIRU VWXGHQWV RI WKH XQGHUJUDGXDWH FRPSXWHU VFLHQFH SURJUDP
WKH VWXGHQWV F ([SODLQLQJ WKH SULQFLSOHV RI D KRQH\SRW LQ WKH VXEMHFW &RPSXWHU 1HWZRUNV ,, 7KLV VWXG\ ZDV
$IWHU UHYHDOLQJ WKH WUXH QDWXUH RI WKH DFWLYLW\ D GHWDLOHG LPSOHPHQWHG LQ WKH VFKRRO \HDU  DW WKH
H[SODQDWLRQRIZKDWDKRQH\SRWLVKRZLWZRUNVDQGZKDW 8QLYHUVLW\ RI 6V &\ULO DQG 0HWKRGLXV LQ 7UQDYD 7KH
LWV F\EHUVHFXULW\ EHQHILWV DUH IROORZLQJ ,W IRFXVHV RQ LWV H[SHULPHQWZDVFDUULHGRXWLQWKHVHFRQG\HDURIWKHVWXG\
UROH DV D WRRO WR EDLW DWWDFNHUV WUDS WKHP DQG DQDO\]H LQ)HEUXDU\LQWZRJURXSVRIVWXGHQWVHDFKWKDW
DWWDFNHUEHKDYLRU LVVWXGHQWVLQWRWDO7KHVWXGHQWVKDGDOUHDG\WDNHQWKH
&RPSXWHU 1HWZRUNV , FRXUVH DQG ZHUH IDPLOLDU ZLWK WKH
LLL $QDO\VLVDQGGLVFXVVLRQ D (YDOXDWLRQRIDWWDFNV EDVLF WRSLFV UHODWHG WR FRPSXWHU QHWZRUNV LQFOXGLQJ
DQGGDWDFROOHFWHG$IWHUWKHSUHVHQWDWLRQRIWKHKRQH\SRW QHWZRUNDUFKLWHFWXUHSURWRFROVVXFKDV,6226,50DQG
SULQFLSOHV WKH GDWD UHFRUGHG GXULQJ WKH DFWLYLW\ DUH 7&3,3 WUDQVPLVVLRQ PHGLD WRSRORJLHV DFFHVV PHWKRGV
DQDO\]HG 6WXGHQWV OHDUQ ZKDW LQIRUPDWLRQ ZDV FDSWXUHG /$1 DQG :$1 VWDQGDUGV DQG WKH\ DOVR DOUHDG\ NQHZ
IURPZKLFK,3DGGUHVVHVZHUHORJJHGLQZKDWSDVVZRUGV KRZ WR ZRUN ZLWK VFULSWLQJ ODQJXDJHV LQ QHWZRUN
ZHUH HQWHUHG ZKDW FRPPDQGV ZHUH WULJJHUHG E\ HDFK DGPLQLVWUDWLRQ
DWWDFNHUZKDWWDFWLFVZHUHXVHGDQGZKDWFDQEHLQIHUUHG
DERXW SRWHQWLDO WKUHDWV E  'LVFXVVLRQ RI WKH XVH RI
,,, 5(68/762)678'(175(63216(6
KRQH\SRWV LQ WKH FODVVURRP 7KH IROORZLQJ LV DQ RSHQ
GLVFXVVLRQ RQ KRZ WKH KRQH\SRW FDQ EH XVHG WR WHDFK ,Q WKLV VXEVHFWLRQ ZH DQDO\]H WKH UHVSRQVHV ZH
F\EHUVHFXULW\ 7KH VWXGHQWV SUHVHQW WKHLU RSLQLRQV RQ REWDLQHG IURP WKH VWXGHQW TXHVWLRQQDLUHV UHODWHG WR WKH
LVVXH XQGHU GLVFXVVLRQ 7KHVH UHVSRQVHV SURYLGH XV ZLWK

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 441

YDOXDEOH LQIRUPDWLRQ RQ VWXGHQW DWWLWXGHV H[SHULHQFHV 4:KDWZRXOG\RXUWHFKQLFDOVNLOOVEHLQFRQILJXULQJ
DQG RSLQLRQV UHJDUGLQJ WKH XVH RI KRQH\SRWV LQ DQGPDQDJLQJ66+"
F\EHUVHFXULW\ WHDFKLQJ $OO GDWD FROOHFWHG LQ WKH
TXHVWLRQQDLUHVZHUHDQRQ\PRXVO\DQDO\]HGDQGSURFHVVHG
ZLWKUHVSHFWWRWKHSULYDF\RIWKHUHVSRQGHQWV
7KHTXHVWLRQVDVNHGFDQEHGLYLGHGLQWRIRXUFDWHJRULHV
DQG HDFK TXHVWLRQ LPPHGLDWHO\ YLVXDOL]HV WKH DQVZHU LQ
WKHIRUPRIDSLHFKDUWWKDWFOHDUO\LOOXVWUDWHVWKHUDQJHRI
YLHZVDQGDWWLWXGHVRIWKHUHVSRQGHQWV
L 8QGHUVWDQGLQJDQGNQRZOHGJH 
4 +RZ ZRXOG \RX UDWH \RXU WHFKQLFDO VNLOOV LQ
4:KDWZDV\RXUNQRZOHGJHRIKRQH\SRWVEHIRUHWKLV *18/LQX[RSHUDWLQJV\VWHPV"
DFWLYLW\"

 
4 +RZ GLG WKLV DFWLYLW\ KHOS \RX XQGHUVWDQG WKH LLL 3HUVSHFWLYHVDQGLQWHUHVW
KRQH\SRWSULQFLSOH"
4 ,I D VLPLODU DFWLYLW\ ZDV KHOG DJDLQ ZRXOG \RX
SDUWLFLSDWH"


4 'R \RX WKLQN D KRQH\SRW FDQ EH D XVHIXO WRRO LQ
F\EHUVHFXULW\" 
4 +RZ ZRXOG \RX GHFLGH LI \RX ZRXOG VHW XS D
KRQH\SRWWRVHFXUH\RXUQHWZRUNLQWKHIXWXUH"


LL ,QWHUDFWLYLW\DQGTXDOLW\ 
4:DVWKHDFWLYLW\LQWHUDFWLYHDQGHQJDJLQJHQRXJK" 4 ,Q \RXU RSLQLRQ ZHUH WKH KRQH\SRW UHVXOWV
LQWHUHVWLQJ"


4 +RZ ZRXOG \RX UDWH WKH TXDOLW\ DQG FODULW\ RI WKH 
SUHVHQWDWLRQDQGWKHH[SODQDWLRQV" 4 +RZ GLG \RX IHHO ZKHQ \RX IRXQG RXW WKDW \RXU
DFWLYLW\ZDVEHLQJFDSWXUHGE\WKHKRQH\SRW"




979-8-3503-7069-0/23/$31.00 ©2023 IEEE 442

 4 'R \RX WKLQN D KRQH\SRW DFWLYLW\ VKRXOG EH H[SHULHQFHV ,Q WKH GLVFXVVLRQ VHFWLRQV WKDW IROORZ ZH
LQFOXGHGLQDF\EHUVHFXULW\FRXUVHV\OODEXV" H[SORUHLQPRUHGHWDLOWKHLQGLYLGXDOWKHPHVWKDWHPHUJHG
IURP WKH VWXGHQW UHVSRQVHV DQG SURYLGH D FRPSUHKHQVLYH
YLHZRIWKHEHQHILWVDQGSRWHQWLDODUHDVIRULPSURYHPHQW
RIWKLVOHDUQLQJDFWLYLW\
4 $QDO\VLV RI WKH VWXGHQWV  UHVSRQVHV WR D TXHVWLRQ
DERXW WKHLU NQRZOHGJH RI KRQH\SRWV EHIRUH WKH DFWLYLW\
UHYHDOHG YDULRXV OHYHOV RI SULRU NQRZOHGJH EXW WKH
PDMRULW\ RI VWXGHQWV LQGLFDWHG WKDW WKH\ KDG OLPLWHG RU
PLQLPDO NQRZOHGJH RI KRQH\SRWV EHIRUH WKH DFWLYLW\
 5HVSRQVHVVXJJHVWWKHQHHGWRLPSURYHWKHFXUULFXOXPDQG
4:DVWKHDFWLYLW\LQWHUHVWLQJIRU\RX" KLJKOLJKWWKHLPSRUWDQFHRILQFOXGLQJVLPLODUDFWLYLWLHVVR
WKDW VWXGHQWV FDQ JDLQ D FRPSOHWH XQGHUVWDQGLQJ RI WKH
ILHOG
4$QDQDO\VLVRIWKHVWXGHQWV UHVSRQVHVWRDTXHVWLRQ
DERXW KRZ WKLV DFWLYLW\ KHOSHG WKHP XQGHUVWDQG WKH
SULQFLSOH RI KRQH\SRW UHYHDOV D GHHSHU LQVLJKW LQWR WKH
HIIHFWLYHQHVV RI WKLV HGXFDWLRQDO PHWKRG )URP WKH
DQDO\]HG UHVSRQVHV LW LV FOHDU WKDW DOPRVW DOO VWXGHQWV
 H[SUHVVHGWKDWWKHDFWLYLW\KHOSHGWKHPWRXQGHUVWDQGWKH
SULQFLSOHRIDKRQH\SRWYHU\JRRGRUUDWKHUZHOO7KHLU
LY .QRZOHGJHJDLQHGDQGUHFRPPHQGDWLRQV SRVLWLYH HYDOXDWLRQ KLJKOLJKWV WKDW WKLV LQWHUDFWLYH
4:KDWNQRZOHGJHDQGVNLOOVGLG\RXJDLQIURPWKLV H[SHULHQFH SURYLGHG DQ XQGHUVWDQGDEOH LQVLJKW LQWR WKH
DFWLYLW\" LIQRQHSOHDVHGRQRWILOOLQ  FRPSOH[FRQFHSWRIKRQH\SRWV2QO\RQHDQVZHUZDVQRW
FOHDU 7KHVH UHVXOWV LQGLFDWH D JHQHUDO SRVLWLYH VWXGHQW
UHVSRQVH WR KRQH\SRW DFWLYLW\ ZKLFK DSSHDUV WR EH DQ
HIIHFWLYH WRRO IRU JDLQLQJ D GHHSHU XQGHUVWDQGLQJ RI WKLV
DUHDRIF\EHUVHFXULW\
4'LVFXVVLRQRIWKHUHVSRQVHVUHFHLYHGWRWKHTXHVWLRQ
RQ WKH DGYLVDELOLW\ RI KRQH\SRWV LQ WKH ILHOG RI
F\EHUVHFXULW\ VKRZV D VWURQJ SUHSRQGHUDQFH RI SRVLWLYH
RSLQLRQV0DQ\VWXGHQWV PRVWRIWKHP VKRZHGDVWURQJ
FRQYLFWLRQWKDWKRQH\SRWVDUHXQGRXEWHGO\DXVHIXOWRROLQ

F\EHUVHFXULW\ 7KLV XQHTXLYRFDO VXSSRUW XQGHUVFRUHV WKH
4 :KDW DUH \RXU UHFRPPHQGDWLRQV IRU LPSURYLQJ LPSRUWDQFHRIWKLVWHFKQLTXHLQWKHGHWHFWLRQPRQLWRULQJ
VLPLODUDFWLYLWLHVLQWKHIXWXUH" DQGDQDO\VLVRISRWHQWLDOF\EHUWKUHDWV$WWKHVDPHWLPH
$VWKLVZDVDQRSHQTXHVWLRQQRWHYHU\RQHDQVZHUHGLW ZH QRWHG D QXPEHU RI UHVSRQVHV WKDW WRRN D FDXWLRXV
VRPHDQVZHUVZHUHYHU\EULHI)RUWKLVTXHVWLRQKHUHDUH DSSURDFK WR WKH LVVXH 7KHLU UHDVRQLQJ LQ WKH GLVFXVVLRQ
VRPHRIWKHLQWHUHVWLQJVWXGHQWUHVSRQVHV PRUH RU OHVV IRFXVHG RQ D FDXWLRXV DSSURDFK WR WKH
LPSOHPHQWDWLRQDQGPDLQWHQDQFHRIKRQH\SRWV+RZHYHU
'HILQLWHO\ \HV , ZRXOG OLNH WR KDYH PRUH RI WKHVH D VLJQLILFDQW PDMRULW\ RI WKH UHVSRQGHQWV EHOLHYH WKDW
LQWHUDFWLYHOHFWXUHVLQWKHILHOGRILQWHUQHWVHFXULW\ KRQH\SRWV FRQWULEXWH WR VWUHQJWKHQLQJ F\EHUVHFXULW\ DQG
0RUHGHVFULSWLRQRIWKHKRQH\SRWLPSOHPHQWDWLRQ UHSUHVHQWDYDOXDEOHWRROLQWKLVFRQWH[W
,W ZRXOG EH XVHIXO WR SURYLGH VRPH JUDSKV DERXW WKH 4 5HVSRQVHV WR WKH TXHVWLRQ UHJDUGLQJ LQWHUDFWLYLW\
DWWDFNHUV PLGVHWDQGZKDWFRPPDQGVWKH\XVHKRZWKRVH DQG HQJDJHPHQW ZLWK WKH KRQH\SRW DFWLYLW\ UHYHDO D
FRPPDQGVFDQDIIHFWWKHVHFXULW\WKHIXQFWLRQDOLW\RIWKH UHOHYDQW DVSHFW RI WKH VWXGHQWV  OHDUQLQJ DQG H[SHULHQFH
QHWZRUN 0RVW RI WKH UHVSRQGHQWV GHPRQVWUDWHG D SRVLWLYH
9HU\LQWHUHVWLQJDQGZHOOSURFHVVHG HYDOXDWLRQ RI WKH DFWLYLW\ 6WXGHQWV VWURQJO\ H[SUHVVHG
$GGLQJ DQ LQWHUDFWLYH HOHPHQW FHUWDLQO\ PDGH WKH WKHLU SDUWLFLSDWLRQ DQG KLJKOLJKWHG WKDW WKH DFWLYLW\ ZDV
SUHVHQWDWLRQPRUHLQWHUHVWLQJ VXIILFLHQWO\ LQWHUDFWLYH DQG HQJDJLQJ 7KHLU H[SHULHQFHV
VXJJHVWWKDWWKLVHGXFDWLRQDOVWUDWHJ\DFKLHYHGLWVSXUSRVH
7KH DFWLYLW\ ZDV LQWHUHVWLQJ DQG LW ZRXOG EH JRRG WR
RIVWLPXODWLQJVWXGHQWV LQWHUHVWDQGDFWLYHSDUWLFLSDWLRQLQ
H[SODLQRWKHUFRQFHSWVIURPWKHILHOGRIFRPSXWHUVHFXULW\
WKH OHDUQLQJ SURFHVV +RZHYHU ZH DOVR REVHUYHG VHYHUDO
LQDVLPLODUZD\
UHVSRQVHV WKDW LQGLFDWHG WKDW VWXGHQWV KDG GLIIHUHQW
3UDFWLFDOXVHRIKRQH\SRW SHUFHSWLRQV RI LQWHUDFWLYLW\ +RZHYHU WKH RYHUZKHOPLQJ
DPRXQW RI SRVLWLYH IHHGEDFN FRQILUPV WKDW WKH KRQH\SRW
,9 ',6&866,21 DFWLYLW\ KDG D VLJQLILFDQW FRQWULEXWLRQ WR VWXGHQWV 
7KHSUHVHQWDQDO\VLVRIWKHVWXGHQWUHVSRQVHVFROOHFWHG PRWLYDWLRQDQGSDUWLFLSDWLRQLQWKHOHDUQLQJSURFHVV
WRDVHULHVRITXHVWLRQVSURYLGHVLQVLJKWLQWRWKHLUDWWLWXGHV 4 $QDO\VLV RI WKH UHVSRQVHV REWDLQHG WR WKH TXHVWLRQ
DZDUHQHVVDQGH[SHULHQFHVUHODWHGWRKRQH\SRWDFWLYLW\LQ DERXW WKH TXDOLW\ DQG FODULW\ RI WKH SUHVHQWDWLRQ DQG
F\EHUVHFXULW\HGXFDWLRQ(DFKRIWKHTXHVWLRQVIRFXVHGRQ H[SODQDWLRQVLQWKHKRQH\SRWDFWLYLW\UHYHDOVDQLPSRUWDQW
D GLIIHUHQW DVSHFW RI WKH DFWLYLW\ DQG WKURXJK D DVSHFW RI WKH HIIHFWLYHQHVV RI WKH SHGDJRJLFDO DSSURDFK
FRPSUHKHQVLYH DQDO\VLV RI WKH UHVSRQVHV ZH VRXJKW WR 2YHUDOO WKHUH LV D SUHGRPLQDQWO\ SRVLWLYH UHVSRQVH
JDLQGHHSHULQVLJKWLQWRVWXGHQWDWWLWXGHVNQRZOHGJHDQG FRQILUPLQJ WKDW ZHOOWDUJHWHG H[SODQDWLRQV DQG

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 443

SUHVHQWDWLRQV KDYH D VLJQLILFDQW EHQHILW LQ HIIHFWLYHO\ ZRUU\LQJ 7KLV UHVSRQVH VXJJHVWV WKDW WKH DFWLYLW\
FRQYH\LQJFRPSOH[F\EHUVHFXULW\WRSLFV DFKLHYHG LWV LQWHQWLRQ RI FUHDWLQJ DQ LQWHUHVWLQJ DQG
4 $Q DQDO\VLV RI VWXGHQWV  UHVSRQVHV WR D TXHVWLRQ HQJDJLQJ H[SHULHQFH IRU WKH VWXGHQWV SODFLQJ WKHP LQ D
UHJDUGLQJ WKHLU DVVHVVPHQW RI WKHLU WHFKQLFDO VNLOOV LQ UHDOVLWXDWLRQDQGDOORZLQJWKHPWRSHUFHLYHWKHSRWHQWLDO
FRQILJXULQJDQGPDQDJLQJ66+VKRZVWKHYDULDELOLW\DQG ULVNVDQGFRQVHTXHQFHV6RPHVWXGHQWVUHSRUWHGWKDWWKH\
OHYHORIDZDUHQHVVLQWKLVDUHD7KHUHVSRQVHVVKRZHGWKDW GLGQRWIHHOZRUULHGZKLFKPD\LQGLFDWHWKDWWKH\DOUHDG\
WKH VWXGHQWV KDYH YDU\LQJ OHYHOVRI DELOLW\ 0RVWRI WKHP KDG SUHYLRXV H[SHULHQFH RU NQRZOHGJH LQ WKH ILHOG RI
H[SUHVVHG EDVLF NQRZOHGJH ZKLOH VRPH KDG OLPLWHG F\EHUVHFXULW\
NQRZOHGJH LQ WKLV DUHD $ IDLU SURSRUWLRQ RI WKH VWXGHQWV 4$QDQDO\VLVRIVWXGHQWV UHVSRQVHVWRWKHTXHVWLRQ
LQGLFDWHG WKDW WKH\ ZHUH DEOH WR KDQGOH WKH FRQILJXUDWLRQ UHJDUGLQJ WKH LQFOXVLRQ RI D KRQH\SRW DFWLYLW\ LQ WKH
DQG PDQDJHPHQW RI 66+ ZLWK VRPH GLIILFXOW\ 'HVSLWH F\EHUVHFXULW\ FRXUVH FXUULFXOXP VKRZV WKDW WKH PDMRULW\
KDYLQJ SUHYLRXVO\ WDNHQ &RPSXWHU 1HWZRUNV , ZH DOVR RI UHVSRQGHQWV DUH SRVLWLYH ZLWK D VLJQLILFDQW SURSRUWLRQ
LGHQWLILHGVRPHUHVSRQVHVLQZKLFKWKHVWXGHQWVLQGLFDWHG RI WKHP FRPSOHWHO\ DJUHHLQJ WKDW WKLV DFWLYLW\ VKRXOG EH
WKDW WKH\ ZHUH QRW IXOO\ SURILFLHQW LQ WKHVH WHFKQLFDO LQFOXGHG LQ WKH FRXUVH FXUULFXOXP 7KLV VXJJHVWV WKDW
DVSHFWV 7KHVH UHVXOWV KLJKOLJKW WKH GLYHUVLW\ RI VWXGHQWV  VWXGHQWVSHUFHLYHWKHYDOXHRISUDFWLFDOH[DPSOHVDQGUHDO
WHFKQLFDO VNLOOV LQ WKH ILHOG DQG HPSKDVL]H WKH QHHG IRU OLIHVFHQDULRVLQWHDFKLQJF\EHUVHFXULW\
WDUJHWHG WUDLQLQJ DQG VXSSRUW LQ WKLV UHJDUG ,Q WKLV ZD\ 4 $QDO\VLV RI VWXGHQWV  UHVSRQVHV WR WKH TXHVWLRQ
WKH KRQH\SRW DFWLYLW\ HPHUJHV QRW RQO\ DV D PHDQV RI UHJDUGLQJ WKH LQWHUHVWLQJQHVV RI WKH DFWLYLW\ DOORZV XV WR
SUDFWLFDODSSOLFDWLRQRINQRZOHGJHEXWDOVRRILGHQWLI\LQJ DVVHVV WKH GHJUHH RI VWXGHQWV  HQJDJHPHQW DQG LQWHUHVW
DQG UHLQIRUFLQJ VWXGHQWV  WHFKQLFDO VNLOOV LQ WKH DUHD RI WRZDUGV WKLV WHDFKLQJ PHWKRG 7KH UHVXOWV VKRZ WKDW DOO
66+FRQILJXUDWLRQDQGPDQDJHPHQW UHVSRQGHQWVVKRZDSRVLWLYHRSLQLRQRQWKHOHYHORI9HU\
4 $Q DQDO\VLV RI WKH VWXGHQWV  UHVSRQVHV WR WKH LQWHUHVWLQJRU6RPHZKDWLQWHUHVWLQJ7KLVVXJJHVWVWKDW
TXHVWLRQ RI WKHLU DVVHVVPHQW RI WKHLU WHFKQLFDO VNLOOV LQ WKHDSSURDFKRIXVLQJUHDOOLIHVFHQDULRVDQGWKHSUDFWLFDO
*18/LQX[ RSHUDWLQJ V\VWHPV UHYHDOV DJDLQ WKH SDUWRIWKHWHDFKLQJKDVWKHSRWHQWLDOWRVWLPXODWHVWXGHQWV 
YDULDELOLW\LQWKHOHYHORIH[SHULHQFHRIWKHVWXGHQWV7KH LQWHUHVWLQWKHWRSLFRIF\EHUVHFXULW\DQGWKDWWKHKRQH\SRW
UHVXOWV KLJKOLJKW WKH KHWHURJHQHRXV QDWXUH RI VWXGHQWV  DFWLYLW\ SURYLGHG D SRVLWLYH H[SHULHQFH DQG LQFUHDVHG
WHFKQLFDO VNLOOV LQ *18/LQX[ RSHUDWLQJ V\VWHPV DQG VWXGHQWV  LQWHUHVW LQ WKH WRSLF RI F\EHUVHFXULW\ WKURXJK D
LQGLFDWH WKH QHHG IRU WDUJHWHG WUDLQLQJ DQG VXSSRUW WR KDQGVRQDQGLQWHUDFWLYHDSSURDFK
LPSURYH WKHVH VNLOOV 7KXV WKH KRQH\SRW DFWLYLW\ DJDLQ 4 $Q DQDO\VLV RI VWXGHQW UHVSRQVHV WR D TXHVWLRQ
EHFRPHV D YHKLFOH QRW RQO\ IRU SUDFWLFDO DSSOLFDWLRQ RI DERXWWKHNQRZOHGJHDQGVNLOOVJDLQHGIURPWKHKRQH\SRW
NQRZOHGJH EXW DOVR IRU LGHQWLI\LQJ DQG VWUHQJWKHQLQJ DFWLYLW\SURYLGHVLQIRUPDWLRQRQWKHLPSDFWRIWKLVKDQGV
VWXGHQWV  WHFKQLFDO VNLOOV LQ V\VWHP DGPLQLVWUDWLRQ DQG RQOHDUQLQJRQWKHGHYHORSPHQWRIVWXGHQWV F\EHUVHFXULW\
*18/LQX[RSHUDWLQJV\VWHPV FRPSHWHQFLHV7KHUHVSRQVHVLQGLFDWHWKDWWKHPDMRULW\RI
4$QDQDO\VLVRIWKHVWXGHQWV UHVSRQVHVWRDTXHVWLRQ VWXGHQWVUHSRUWWKDWWKH\KDYHJDLQHGNQRZOHGJHDQGVNLOOV
IRFXVLQJRQWKHLUSDUWLFLSDWLRQLIDVLPLODUDFWLYLW\ZHUHWR UHODWHG WR KRQH\SRW LVVXHV DV ZHOO DV 66+ PDQDJHPHQW
EH KHOG DJDLQ UHYHDOV D SRVLWLYH DWWLWXGH DPRQJ WKH DQG ,QWHUQHW VHFXULW\ 6RPH VWXGHQWV LQGLFDWHG WKDW WKH\
PDMRULW\RIUHVSRQGHQWV7KHUHVSRQVHVREWDLQHGVKRZWKDW VSHFLILFDOO\JDLQHGVNLOOVLQWKHDUHDRI*18/LQX[V\VWHP
DVLJQLILFDQWSURSRUWLRQRIVWXGHQWVZRXOGSDUWLFLSDWHLQD DGPLQLVWUDWLRQ 7KHVH UHVXOWV LQGLFDWH WKDW WKH DFWLYLW\
VLPLODU DFWLYLW\ DJDLQ ZLWK D VLJQLILFDQW SUHYDOHQFH RI FRQWULEXWHGWRHQULFKLQJVWXGHQWV NQRZOHGJHDQGVNLOOVLQ
'HILQLWHO\ \HV DQG 5DWKHU \HV UHVSRQVHV 7KLV WUHQG UHOHYDQW DUHDV RI F\EHUVHFXULW\ ,W LV DOVR HYLGHQW WKDW
LQGLFDWHVWKDWWKHVWXGHQWVILQGWKHDFWLYLW\EHQHILFLDODQG VWXGHQWV VKRZHG LQWHUHVWLQ SUDFWLFDO DSSOLFDWLRQV VXFK DV
LQWHUHVWLQJ DQG WKDW WKH\ ZRXOG EH ZLOOLQJ WR SDUWLFLSDWH 66+ PDQDJHPHQW DQG WKH VSHFLILF +RQH\3RW LVVXH
DJDLQLQDVLPLODUW\SHRIHGXFDWLRQDOLQLWLDWLYH VXJJHVWLQJWKDWVXFKDFWLYLWLHVKDYHWKHSRWHQWLDOWRIRVWHU
4 7KH DQDO\VLV RI WKH VWXGHQWV  UHVSRQVHV WR WKH GHHSHU XQGHUVWDQGLQJ DQG SUDFWLFDO H[SHULHQFH LQ
TXHVWLRQ UHJDUGLQJ WKHLU GHFLVLRQ UHJDUGLQJ WKH F\EHUVHFXULW\
HVWDEOLVKPHQWRIDKRQH\SRWWRVHFXUHWKHLURZQQHWZRUN 4 7KH RSHQHQGHG UHVSRQVHV RI WKH VWXGHQWV WR D
UHYHDOVDJDLQDFRQVLGHUDEOHYDULDELOLW\RIRSLQLRQVZKLFK TXHVWLRQ DERXW WKH LQWHUHVW RI WKH DFWLYLW\ DQG WKHLU
PD\ EH EDVHG RQ WKH YDULDELOLW\ RI WKHLU WHFKQLFDO VNLOOV UHFRPPHQGDWLRQV WR LPSURYH VLPLODU DFWLYLWLHV LQ WKH
PHQWLRQHGLQ4DQG4$SRVLWLYHIHDWXUHRIWKHFKRVHQ IXWXUH SURYLGH YDOXDEOH LQIRUPDWLRQ WR HYDOXDWH DQG
PHWKRGRORJ\ LV WKDW WKH RSWLRQ 'HILQLWHO\ QRW GRHV QRW SRWHQWLDOO\ LPSURYH HGXFDWLRQDO DSSURDFKHV WR
DSSHDULQWKHUHVSRQVHVDWDOO F\EHUVHFXULW\6WXGHQWVRYHUZKHOPLQJO\H[SUHVVHGLQWHUHVW
4$QDO\]LQJVWXGHQWV UHVSRQVHVWRDTXHVWLRQDERXW LQ WKH DFWLYLW\ DQG LWV LQWHUHVWLQJ QDWXUH ZLWK VLJQLILFDQW
WKH LQWHUHVWLQJQHVV RI WKH KRQH\SRW UHVXOWV DOORZV XV WR DSSUHFLDWLRQ IRU WKH LQWHUDFWLYH QDWXUH RI WKH DFWLYLW\
DVVHVVWKHLUUHDFWLRQVWRWKHLQIRUPDWLRQ7KHUHVXOWVVKRZ 6RPH ZRXOG ZHOFRPH PRUH VLPLODU LQWHUDFWLYH OHFWXUHV
WKDW WKH RYHUZKHOPLQJ PDMRULW\ RI UHVSRQGHQWV UDWH WKH HVSHFLDOO\ LQ WKH DUHD RI LQWHUQHW VHFXULW\ ,Q HGXFDWLRQDO
UHVXOWV DV LQWHUHVWLQJ ZLWK <HV YHU\ LQWHUHVWLQJ DQG SUDFWLFH WKH QHHG IRU D PRUH LQGHSWK GHVFULSWLRQ RI WKH
<HV VRPHZKDW LQWHUHVWLQJ EHLQJ WKH FDUULHU UHVSRQVHV LPSOHPHQWDWLRQ RI WKH KRQH\SRW SURYLGLQJ D JUDSKLFDO
DQG 1R UHVSRQVHV QRW DSSHDULQJ DW DOO 7KLV SRVLWLYH UHSUHVHQWDWLRQ RI WKH DWWDFN PHWKRGV DQG WKHLU LPSDFW RQ
DWWLWXGH UHIOHFWV WKH VXFFHVV RI WKH DFWLYLW\ LQ VWLPXODWLQJ QHWZRUNVHFXULW\ZDVHPSKDVL]HG$WWKHVDPHWLPHWKH
VWXGHQWV LQWHUHVWDQGHQJDJHPHQWZLWKKRQH\SRWLVVXHV LPSRUWDQFH RI SUDFWLFDO XVH RI WKH KRQH\SRW DQG WKH
SRVVLELOLW\ RI DSSO\LQJ VLPLODU WHDFKLQJ PHWKRGV WR RWKHU
4 $Q DQDO\VLV RI VWXGHQWV  UHVSRQVHV WR D TXHVWLRQ
FRQFHSWV ZLWKLQ F\EHUVHFXULW\ DUH KLJKOLJKWHG 7KHVH
DERXW WKHLU HPRWLRQDO VWDWH DIWHU OHDUQLQJ WKDW WKHLU
UHVSRQVHV VXSSRUW WKH FRQFOXVLRQ WKDW DFWLYLWLHV ZLWK DQ
DFWLYLWLHV ZHUH EHLQJ KRQH\SRWWHG VKRZV WKDW PRVW
LQWHUDFWLYHDSSURDFKKDYHDSRVLWLYHLPSDFWRQWKHLQWHUHVW
UHVSRQGHQWV GHVFULEH WKHLU HPRWLRQDO VWDWH DV VRPHZKDW

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 444

 DQG HIIHFWLYHQHVV RI LQVWUXFWLRQ ZKLOH FRQWULEXWLQJ WR D ,Q WKH IXWXUH ZH ZLOO IRFXV RQ FRPSDULQJ WKH
GHHSHUXQGHUVWDQGLQJRIVWXGHQWVLQWKHILHOG HIIHFWLYHQHVV RI KRQH\SRWV LQ UDLVLQJ F\EHUVHFXULW\
DZDUHQHVVZLWKRWKHUPHWKRGVVXFKDVDWWDFNVLPXODWLRQV
,Q VXPPDU\ WKH DQDO\VLV RI VWXGHQWV  UHVSRQVHV WR D RULQWHUDFWLYHH[HUFLVHVDVZHOODVWUDFNLQJWKHORQJWHUP
VHULHV RI TXHVWLRQV DERXW KRQH\SRW DFWLYLW\ LQ HIIHFWV RI LPSOHPHQWLQJ KRQH\SRWV LQ WKH HGXFDWLRQDO
F\EHUVHFXULW\ HGXFDWLRQ \LHOGHG YDOXDEOH LQVLJKWV DQG SURFHVV LH WR ZKDW H[WHQW VWXGHQWV  F\EHUVHFXULW\
LQIRUPDWLRQ 7KH UHVXOWV RI WKLV DQDO\VLV XQGHUOLQH WKH DZDUHQHVV LPSURYHV RYHU WKH ORQJ WHUP $OVR RQ WKH
SRVLWLYH UHDFWLRQV RI WKH VWXGHQWV WR WKH LQWHUDFWLYH DQG LPSDFW RI DUWLILFLDO LQWHOOLJHQFH DQG PDFKLQH OHDUQLQJ
KDQGVRQ DSSURDFK WR OHDUQLQJ ZKLFK DOORZHG WKHP WR EDVHGPHWKRGV>@RQF\EHUVHFXULW\DQGWKHLUDSSOLFDWLRQ
JDLQ QRW RQO\ WKHRUHWLFDO NQRZOHGJH EXW DOVR SUDFWLFDO WRLQFUHDVHF\EHUVHFXULW\DZDUHQHVV
VNLOOV LQ WKH FRQILJXUDWLRQ DQG PDQDJHPHQW RI VHFXULW\
WRROV$WWKHVDPHWLPHWKHKRQH\SRWDFWLYLW\ZDVVKRZQ $&.12:/('*0(17
WR HQKDQFH VWXGHQWV  LQWHUHVW LQ F\EHUVHFXULW\ DQG IRVWHU
WKHLU SDUWLFLSDWLRQ LQ WKH OHDUQLQJ SURFHVV 7KH UHVXOWV 7KH ZRUN ZDV IXQGHG E\ WKH JUDQW .(*$ 8&0
VXJJHVW WKH QHHG IRU IXUWKHU GHYHORSPHQW DQG  0RGHUQ WHFKQRORJLHV DQG LQQRYDWLRQV LQ QHWZRUN
LQFRUSRUDWLRQ RI VLPLODU DFWLYLWLHV LQWR WKH FXUULFXOXP RI VHFXULW\HGXFDWLRQ
F\EHUVHFXULW\ FRXUVHV WR LPSURYH WKH SUDFWLFDO
XQGHUVWDQGLQJRIWKHILHOGDQGWKHWHFKQLFDOSUHSDUDWLRQRI 5()(5(1&(6
VWXGHQWV 7KXV WKLV DQDO\VLV KDV DOORZHG XV WR JDLQ D >@ ;/DUUXFHDDQG$0ROLQXHYR$Q,&6EDVHGVFHQDULRJHQHUDWRU
GHHSHUXQGHUVWDQGLQJRIWKHLPSDFWDQGYDOXHRIKRQH\SRW IRU F\EHU UDQJHV6\VWHPV 6RIWZDUH DQG 6HUYLFHV 3URFHVV
DFWLYLWLHVLQF\EHUVHFXULW\HGXFDWLRQDQGRSHQVWKHZD\IRU ,PSURYHPHQW WK (XURSHDQ &RQIHUHQFH (XUR63, 
'VVHOGRUI *HUPDQ\ 6HSWHPEHU ±  3URFHHGLQJV 
IXUWKHUGHYHORSPHQWDQGLPSURYHPHQWLQWKLVDUHD 6SULQJHU,QWHUQDWLRQDO3XEOLVKLQJ
>@ ( /ySH]0RUDOHV & 5XELR0HGUDQR $ 'RXSp <
9 &21&/86,216 6KRVKLWDLVKYLOL 5 :DQJ 7 %DR DQG *- $KQ ³+RQH\SOF $
,Q WKLV SDSHU ZH HYDOXDWH WKH HIIHFWLYHQHVV RI D QH[WJHQHUDWLRQ KRQH\SRW IRU LQGXVWULDO FRQWURO V\VWHPV´
,Q3URFHHGLQJV RI WKH  $&0 6,*6$& &RQIHUHQFH RQ
KRQH\SRW DFWLYLW\ DV D WRRO WR XQGHUVWDQG F\EHUVHFXULW\ &RPSXWHUDQG&RPPXQLFDWLRQV6HFXULW\ SS 
SULQFLSOHVE\XQGHUJUDGXDWHFRPSXWHUVFLHQFHVWXGHQWV7R >@ ,+ $EGXOTDGGHU'=RXDQG ,7 $]L]³7KH'$*EORFNFKDLQ
ZKDWH[WHQWDQGLQZKDWZD\VWKLVDFWLYLW\FRQWULEXWHGWR $ VHFXUH HGJH DVVLVWHG KRQH\SRW IRU DWWDFN GHWHFWLRQ DQG PXOWL
WKHDFTXLVLWLRQRINQRZOHGJHDQGWHFKQLFDOVNLOOVUHODWHGWR FRQWUROOHU EDVHG ORDG EDODQFLQJ LQ 6'1 *´)XWXUH *HQHUDWLRQ
KRQH\SRWVDQGF\EHUVHFXULW\ZHH[DPLQHGWKHH[WHQWDQG &RPSXWHU6\VWHPV
LQZKLFKWKLVDFWLYLW\FRQWULEXWHG%DVHGRQDQDQDO\VLVRI >@ $ 5RPDQRYV - %LNRYVND - 3HNVD 7 9DUWLDLQHQ 3
WKH VWXGHQWV  UHVSRQVHV ZH DVVHVVHG WKHLU H[SHULHQFHV .RWVDPSRSRXORV % (OWDKDZ\ HW DO 6WDWH RI WKH $UW LQ
NQRZOHGJHDQGDWWLWXGHVRQWKLVWRSLF &\EHUVHFXULW\ DQG 6PDUW *ULG (GXFDWLRQ ,((( WK
,QWHUQDWLRQDO &RQIHUHQFH RQ 6PDUW 7HFKQRORJLHV (852&21 
2XU VWXG\ DOORZHG XV WR JDLQ LQVLJKW LQWR YDULRXV 
DVSHFWV RI VWXGHQW DFWLYLW\ DQG UHVSRQVH 7KH UHVXOWV >@ %-%ODåLþ&KDQJLQJWKHODQGVFDSHRIF\EHUVHFXULW\HGXFDWLRQLQ
VXJJHVW WKDW WKH DFWLYLW\ ZLWK KRQH\SRWV KDV D SRVLWLYH WKH(8:LOOWKHQHZDSSURDFKSURGXFHWKHUHTXLUHGF\EHUVHFXULW\
LPSDFWRQWKHDZDUHQHVVDQGNQRZOHGJHRIF\EHUVHFXULW\ VNLOOV" (GXFDWLRQ DQG LQIRUPDWLRQ WHFKQRORJLHV   
E\VWXGHQWV7KHVWXGHQWVLQGLFDWHGWKDWWKHDFWLYLW\KHOSHG 
WKHP EHWWHU XQGHUVWDQG WKH SULQFLSOHV RI KRQH\SRWV DQG >@ - âNULQiURYi DQG ( 9HVHO 0RGHO RI HGXFDWLRQ DQG WUDLQLQJ
GHYHORS D GHHSHU XQGHUVWDQGLQJ RI F\EHU WKUHDWV DQG VWUDWHJ\IRUWKHKLJKSHUIRUPDQFHFRPSXWLQJ,QWHUQDWLRQDO
&RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ 7HFKQRORJLHV DQG
DWWDFNHUWDFWLFV $SSOLFDWLRQV ,&(7$ ,(((
:H DOVR ORRNHG DW WKH LQWHUDFWLYLW\ DQG TXDOLW\ RI WKH >@ 0':RUNPDQ-$QWKRQ\/XHYDQRVDQG%0DL³$6WXG\RI
DFWLYLW\ 7KH VWXGHQWV UDWHG WKH DFWLYLW\ DV VXIILFLHQWO\ &\EHUVHFXULW\ (GXFDWLRQ 8VLQJ D 3UHVHQW7HVW3UDFWLFH$VVHVV
HQJDJLQJDQGLQWHUDFWLYHZKLFKFRQWULEXWHGWRWKHLUDFWLYH 0RGHO´,(((7UDQV(GXFYROQRSS±
LQWHUHVW DQG SDUWLFLSDWLRQ 3UHVHQWDWLRQV DQG H[SODQDWLRQV >@ 0$ 0RKG $ULIILQ 0< 'DUXV + +DURQ $ .XUQLDZDQ <
ZHUH SRVLWLYHO\ HYDOXDWHG LQGLFDWLQJ DQ HIIHFWLYH ZD\ RI 0XOLRQR DQG &5 3DUGRPXDQ ³'HSOR\PHQW RI +RQH\SRW DQG
6,(0 7RROV IRU &\EHU 6HFXULW\ (GXFDWLRQ 0RGHO LQ
FRQYH\LQJFRPSOH[WRSLFV 8,70´,QWHUQDWLRQDO -RXUQDO RI (PHUJLQJ 7HFKQRORJLHV LQ
6WXGHQWV H[SUHVVHG WKHLU LQWHUHVW LQ UHSHDWLQJ D VLPLODU /HDUQLQJ  
DFWLYLW\ LQ WKH IXWXUH DQG VRPH RI WKHP ZRXOG HYHQ >@ -7KRP7'DV%6KUHVWKD66HQJXSWDDQG($UVODQ³&DVWLQJ
FRQVLGHU VHWWLQJ XS D KRQH\SRW WR VHFXUH WKHLU QHWZRUN D ZLGH QHW $Q LQWHUQHW RI WKLQJV WHVWEHG IRU F\EHUVHFXULW\
7KLV VXJJHVWV WKDW WKH DFWLYLW\ SURYLGHG D SRVLWLYH DQG HGXFDWLRQ DQG UHVHDUFK´ ,Q ,QWHUQDWLRQDO 6\PSRVLXP RQ
XVHIXOH[SHULHQFH 3HUIRUPDQFH (YDOXDWLRQ RI &RPSXWHU DQG 7HOHFRPPXQLFDWLRQ
6\VWHPV 63(&76  SS ,(((
,Q JHQHUDO ZH LGHQWLILHG SRVLWLYH HIIHFWV RI WKH >@ + &KHQ < =KDQJ 6 =KDQJ 7 /\X 7 ³([SORULQJ WKH UROH RI
KRQH\SRW DFWLYLW\ LQ XQGHUJUDGXDWH FRPSXWHU VFLHQFH JDPLILHG LQIRUPDWLRQ VHFXULW\ HGXFDWLRQ V\VWHPV RQ LQIRUPDWLRQ
VWXGHQWV 7KLV PHWKRG FRPELQLQJ WKHRUHWLFDO NQRZOHGJH VHFXULW\ DZDUHQHVV DQG SURWHFWLRQ EHKDYLRUDO
SUDFWLFDO VLPXODWLRQ DQG UHIOHFWLRQ SURYHG WR EH DQ LQWHQWLRQ´(GXFDWLRQDQG,QIRUPDWLRQ7HFKQRORJLHV
HIIHFWLYH ZD\ WR LPSURYH F\EHUVHFXULW\ DZDUHQHVV DQG >@ -$UQHGR0RUHQRDQG9*DUFLD)RQW$VWXG\RQWKHGHVLJQDQG
GHYHORSWKHWHFKQLFDOVNLOOVRIVWXGHQWV DSSOLFDWLRQRIILFWLRQDOVWRU\WHOOLQJLQRQOLQHOHDUQLQJRIFRPSXWHU
VHFXULW\$SSOLHG6FLHQFHVYROQRSS
%DVHG RQ WKH UHVXOWV REWDLQHG ZH UHFRPPHQG WKH >@ 6%&RúNXQDQG07XUDQOL³&UHGLWULVNDQDO\VLVXVLQJERRVWLQJ
LQFOXVLRQ RI VLPLODU DFWLYLWLHV LQ WKH F\EHUVHFXULW\ PHWKRGV´-RXUQDO RI $SSOLHG 0DWKHPDWLFV 6WDWLVWLFV DQG
FXUULFXOXP 7KHVH DFWLYLWLHV FDQ FRQWULEXWH WR D EHWWHU ,QIRUPDWLFV  
XQGHUVWDQGLQJRIF\EHUVHFXULW\FRQFHSWVDQGLPSURYHWKH
VNLOOVRIVWXGHQWVLQWKLVDUHD


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 445

 Duckietown project pros and cons
1st Paweł Skruch 2nd Marek Długosz
IEEE Senior Member IEEE Member
AGH University of Science and Technology AGH University of Science and Technology
Cracow, Poland Cracow, Poland
pawel.skruch@agh.edu.pl mdlugosz@agh.edu.pl
0000-0002-8290-8375 0000-0001-6827-9149

4th Artur Morys-Magiera

3rd Marcin Szelest AGH University of Science and Technology
IEEE Senior Member Cracow, Poland
AGH University of Science and Technology amorys@student.agh.edu.pl
Cracow, Poland 0000-0002-2137-8841
mszelest@agh.edu.pl
0000-0002-0522-1270

Abstract—The Duckietown project is aimed at developing a robot simulator that allows simulation of robot operation in
software and hardware platform for education and research on a virtualized environment [3]. In 2017, project leaders Liam
robot autonomy. The project includes the design of robots, a Paull (Montréal), Andrea Censi and Jacopo Tani (ETH), and
mock-up of cities, and additional infrastructure elements such as
Watchtowers that can be utilized for navigation. The platform Matthew Walter (TTIC) decided for the first time to use the
also includes a software-based Gym-Duckietown robot simulator Duckietown platform in classes with students at four different
that allows for simulation of robot operation in a virtual environ- universities ETH Zürich, Université de Montréal, TTI Chicago
ment. Duckiebots leverage the Jetson Nano platform to control and NCTU. As a result, students from these universities had
the Duckiebot robot, also carrying hardware GPU acceleration the opportunity to compare solutions to various problems in
for complex computations.
Index Terms—autonomous, autonomous vehicle, electrical ve- the field of autonomous robots among themselves. 2018 has
hicle, control system seen a rapid increase in interest in the Duckietown project
from universities and companies. The Duckietown platform
I. I NTRODUCTION needed to become more affordable, easier to acquire, and of
The basic skill for engineers to possess is to be able to solve better quality so that it could support teaching and research
practical problems. The study of engineering should provide in a more general way. As a result, a campaign was launched
opportunities for students to acquire these skills through the on the Kickstarer platform to raise funds to further develop
implementation of a variety of practical systems. Depending the project. The following year, 2019 saw the debut of the AI
on the field of engineering, practical activities for students may Driving Olympics (AI-DO) competition at the Neural Infor-
be easier or more difficult to implement. A field of engineering mation Processing Systems Conference (NeurIPS) in Montreal
science that requires expensive equipment and highly qualified [4]. It is worth noting that this was the first competition held at
personnel is the field of autonomous vehicles and robots. In a machine learning conference with real robots. The following
such cases, laboratory workstations that use models of robots years saw further development of the project, the development
or autonomous vehicles along with a model of the environment of a new version of the Duckiebot robot, the presentation of the
in which they operate are of great help. One such project that platform at the Science Museum of London exhibition, and the
realizes the above is the Duckietown project1 [1]. development of an online course on the edX platform called
The Duckietown project was created in 2016 at the Univer- "Self-Driving Cars with Duckietown"2 which has become very
sity of Massachusetts Institute of Technology (MIT) as part popular.
of a class for students [2]. The goal of the project was to The second version of the Duckiebot robot, which uses
develop a software and hardware platform for the education the Jetson Nano computer for control, is now available. The
and research on robot autonomy. Currently, the project is being Jetson Nano platform enables the use of neural networks with
developed at ETH. The project includes the design of Duck- hardware GPU acceleration for faster inference, which brings
iebot robots, a mock-up of Duckietown cities, and additional the platform closer to realtime applications.
infrastructure elements, such as for Watchtower navigation. The article is organized as follows: section II describes
The project also includes a software-based Gym-Duckietown the hardware part of the project including Duckiebot robots,

1 https://duckietown.org 2 https://www.edx.org/search?q=Self-Driving+Cars+with+Duckietown

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 446

 Fig. 2: Fragment of the plastic body of the Duckiebot robot.

Fig. 1: Duckiebot DB21M Robot.

Duckietown infrastructure elements, and Watchtower with the

noted drawbacks of the project. Section III describes the
software part of the project, indicating the advantages and
disadvantages of the solutions adopted. Section IV describes
the available documentation of the project and finally section
V summarizes the strengths and weaknesses of the project.
II. H ARDWARE PLATFORM Fig. 3: Negative camber of the of Duckiebot robot wheels.
The design of the body, the selection of robot components
such as sensors, drives, and power supply are crucial in the
later life of robots. Even the best theoretical design of a is located. The design and realization of the body did not
robot may not work satisfactorily if inadequate components foresee that the nuts that are located in this space also have
are used to implement it. The Duckiebot robot is purchased their own height, which makes the battery take up a little
as a self-assembly kit. The kit includes all the necessary more space than can be expected judging by its external
components needed to assemble the Duckiebot robot. The dimensions. This results in gaps (detail No. 1 in Fig. 2) and
assembly does not render problematic, and the very well- a less rigid attachment of the motor to the body. Due to the
prepared assembly documentation should be emphasized here. mechanical properties of the material from which the body
The DB21M version of the Duckiebot robot is equipped with is made (plastic), and the location of the nut holes too close
a differential drive, one RGB camera, LED-RGB LEDs, two to the edges of the body components, they tend to break off
motors, a distance sensor, a set of motors with gears and (detail No. 2 in Fig. 2). The use of plastic screws (partially,
encoders, two drive wheels and support ball (freewheel), an since not all of them are plastic) to connect body parts is also
NVIDIA Jetson Nano computer, one button, and an OLED a certain shortcoming.
display. The assembled Duckiebot robot is shown in Figure 1. Another problem with the body is the lack of adequate
lateral stiffness (along the Y axis), which results in the drive
A. Body wheels deforming outwards (negative camber), see fig. 3.
The body of the Duckiebot robot is made entirely of plastic The negative camber of the wheels makes the Duckiebot
parts. This choice of material certainly reduces the price of robot oversteer, which at first may seem like an advantage.
the robot, but also affects its mechanical endurance. Figure 2 However, considering the surface on which the Duckiebot
shows typical problems of the Duckiebot robot body. robot moves (see section II-D) and the asymmetry in the
The first of the problems noted is the poor quality of the operation of the motors (at the same supply voltage, the speeds
fitting of the various parts of the Duckiebot robot body. It is of the motors are not identical), oversteering is a disadvantage
due to the insufficient size of the space in which the battery rather than an advantage. It takes even a minimal disturbance

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 447

(from uneven pavement and asymmetrical operation of the Duckiebot robot. It is not as sensitive to disturbances and does
motors) to result in the Duckiebot robot abruptly changing not change direction abruptly. Figure 6 shows the effect of
its direction of movement so that the control system is unable using an additional crossbar connecting the robot’s motors. It
to steer the robot correctly. can be seen that after using the crossbar, the Duckiebot robot
The traction properties of Duckiebot robots can be improved wheels adhered to the surface in their entire width providing
by connecting the motor mounts with a crossbar that stiffens better traction.
the motor mounts to prevent negative camber of the wheels;
see fig. 4.

Fig. 6: The result of using the crossbar to stiffen the motor

mounts of the Duckiebot robot wheels.

B. Motors
The original Duckiebot robots use DG01D-E motors with a
gearbox of 48:1, 90 rpm and a torque of 0.078 Nm before
the gearbox and a torque of 3.744 Nm after the gearbox.
The DG01D-E is also equipped with a quadrature encoder
Fig. 4: Additional crossbar to stiffen Duckiebot robot motor with a probable resolution of 8 pulses per revolution (the
mounts. manufacturer does not provide this information) before the
gearbox and 576 pulses after the gearbox. The DG01D-E
Another way to improve the traction characteristics of the motor features high speed but not very high torque; see table
Duckiebot robot is to print an additional stiffening element3 I. Taking into account the imperfections of the mock-up (see
that ensures the concentricity of the two motors, see Fig. 5. section II-D) - in particular the different heights between the
different panels (see figure 8) - it might happen that the
Duckiebot robot stops while moving from one panel to another.
The motors in the Duckiebot robot have been replaced with
SJ01 motors (SKU:FIT0450) with 120:1, 160 rpm gearing
and 0.07 Nm torque before gearing and 8.4 Nm torque after
gearing. In addition, the SJ01 motor is equipped with a
quadrature encoder with a resolution of 8 pulses per revolution
before the gearbox and 960 pulses per revolution after the
gearbox.
Both motors have the same external dimensions, housings,
and mounting holes, so replacing them is not a problem. You
should only pay attention to the correct connection of motor
control signals and encoder signals because in the DG01D-
E motor all signals are integrated in one 6-pin connector,
while the SJ01 motor features different connectors: one for
power supply and another for the encoder. Table I lists all the
parameters of the DG01D-E and SJ01 motors.
Finally, the SJ01 motor, despite having slightly less torque
Fig. 5: An additional element to stiffen the motor mount of
in front of the gearbox, is definitely a better choice than the
the Duckiebot robot.
DG01D-E motor. It has more torque and a higher resolution
Reducing the negative camber of the wheels results in a encoder, which definitely improves the driving characteristics
significant improvement in the traction characteristics of the of the Duckiebot robot and the accuracy of motor speed
control. The slightly lower speed ω after gearing does not
3 https://www.thingiverse.com/thing:2558770 matter at all, because Duckiebot robots are not supposed to

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 448

 before gearbox after gearbox lines, whereas intersections are marked with red lines. The
Motor
n ω torque encoder ω torque encoder route along which the Duckiebot robots move can be freely
type
[rpm] [Nm] [ppr] [rpm] [Nm] [ppr] adjusted by an appropriate arrangement of elements of the
mock-up. The basic element from which the route is built is
DG01D-E 48:1 90 0.078 12 1.874 3.744 576
a rubber-foam mat. The first problem is the peeling off of the
SJ01 90:1 120 0.070 8 1.333 8.400 960
tape used to mark lane lines, see Fig. 8. The second problem
is the sheer quality of the rubber foam panels. Even panels
TABLE I: Parameters of Duckiebot robot motors.
from a single package have different heights, which results
in Duckiebot robots - when moving on the edges of them
- changing their direction of travel in a completely random
participate in robot races, but rather to intelligently move manner; see Fig. 8.
around the city mock-up.
C. Battery
The Duckiebot robot uses a lithium ion battery with a
capacity of 10Ah. The battery case also contains a built-in
electronic circuit that controls the battery’s operation, e.g. does
not allow it to over-discharge, which is known to be harmful
for this type of battery. The battery control itself is much more
complex, as can be seen when analyzing the state machine
implemented by the battery electronics; see Fig. 7.

Fig. 8: Details of the Duckietown project mock-up.

The solution to mat-related problems is, firstly, to use a

different tape than the one sold by the Duckietown Foun-
dation. Among the various tapes tested by the authors, the
best properties were characterized by the tape of WELSTIK
Fig. 7: State machine implemented in Duckiebot robot’s bat- company. Secondly, before the tape is adhered to the panel, it
tery control electronics. is suggested to clean the panel - for instance, with isopropyl
alcohol or other liquid that in its composition contains alcohol,
The state of charge of the battery, as well as the current so as to degrease the surface. Lastly, one can heat the glued
state of the state machine in which the battery is located, tape using, for example, a hair dryer.
is presented by means of 4 leds mounted on the case. This
III. S OFTWARE
is a simple, yet effective solution. Unfortunately, inside the
Duckiebot robot’s body, the battery is located such that the Definitely a strong point of the Duckietown project is its
LEDs indicating the battery status are not visible, which makes software, for which the most popular framework for robot
it impossible in any way to determine, for example, how low programming at the moment was used: Robot Operation
the battery of the Duckiebot robot is from the battery itself. It System. The division of the entire software into individual
is also possible to read the battery level of the Duckiebot robot modules and packages is correct, understandable, and logical.
by software, unfortunately, very often this functionality does The individual elements/modules of the software are realized
not work properly. The simplest solution to this problem is to using containerization technology. Thanks to the widespread
change the way the battery is built into the Duckiebot robot use of containerization, one can easily, quickly, and safely
so that its status can always be read from the LEDs mounted create and test their software in a reproducible manner with
on the housing. error-proof setup. It is also worth noting that software devel-
opment technologies that use containerization are now very
D. Mock-up popular and are often used in various branches of software
One of the main elements of the Duckietown project is a development.
mock-up. It is intended to model the shape of a layout of roads The Duckietown project software also includes the Duck-
and intersections thereof, on which Duckiebot robots move. ieTown Shell. This is a set of scripts that facilitates and
Straight lines and curves are marked with white and yellow automates various activities related to the day-to-day operation

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 449

of Duckiebot robots, e.g.: updating software, creating docker In the opinion of the authors, the most urgent things that
images, running containers, and much more. should be improved are: the mechanical design of the body
The state in which the Duckiebot robot is currently in can (better materials, a different way of connecting and fixing
be checked using software in a web browser. Each Duckiebot various components), replacing the engines with a different
robot provides such an interface in the form of a microservice. model, and updating the container image template so that it is
Duckiebot robots can be programmed in Python or C/C++, possible to use the GPU hardware acceleration. With time, it
which is a consequence of using the ROS framework. The will also become necessary to replace ROS with its successor
source code for the Duckiebot robot software is available as ROS2.
a repository on GitHub4 .
R EFERENCES
In fact, the only drawback of the currently available devel-
opment container template is the inability to use the GPU that [1] J. Tani, L. Paull, M. T. Zuber, D. Rus, J. How, J. Leonard, and A. Censi,
“Duckietown: an innovative way to teach autonomy,” in Educational
the NVIDIA Jetson Nano computer is equipped with, since Robotics in the Makers Era 1. Springer, 2017, pp. 104–121.
the required libraries are not installed inside the image. This [2] L. Paull, J. Tani, H. Ahn, J. Alonso-Mora, L. Carlone, M. Cap, Y. F. Chen,
is also influenced by an incompatibility of the installed python C. Choi, J. Dusek, Y. Fang et al., “Duckietown: an open, inexpensive and
flexible platform for autonomy education and research,” in 2017 IEEE
software version and the most recent TensorFlow releases. International Conference on Robotics and Automation (ICRA). IEEE,
The impossibility of utilizing GPU hardware acceleration from 2017, pp. 1497–1504.
the container affects primarily the use of neural networks for [3] M. Chevalier-Boisvert, F. Golemo, Y. Cao, B. Mehta, and L. Paull, “Duck-
ietown environments for openai gym,” https://github.com/duckietown/
inference performance, to control the Duckiebot robot. gym-duckietown, 2018.
[4] A. Censi, L. Paull, J. Tani, and M. R. Walter, “The ai driving olympics:
IV. D OCUMENTATION An accessible robot learning benchmark,” in 33rd Conference on Neural
Information Processing Systems (NeurIPS 2019), 2019.
Documentation of the Duckietown project consists of elec-
tronic documents (in HTML or PDF formats) in which issues
ranging from the assembly of the Duckiebot robot (step by
step), the correct execution of the mock-up, to issues related
to robot programming are described in an easy-to-understand
and accessible manner. In the documentation one can find
comprehensive descriptions of the software architecture of
the Duckietown project, a great number of examples and
also recommendations for dealing with problems. Duckietown
project documentation is also updated on an ongoing basis,
e.g. inactive links are corrected (after they are reported by
the community), and additional information is added. Unfor-
tunately, the documentation of the Duckiebot project is not
yet complete; many of its sites are missing content that is not
completed at the time of writing this article. The Duckiebot
project documentation also includes teaching materials in the
form of lectures, videos, exercises, as well as a comprehensive
course on the edX e-learning platform titled "Self-Driving Cars
with Duckietown." Exercises in the form of Jupyter scripts and
GitHub repository templates are also prepared for use in class.

V. C ONCLUSION
In conclusion, it should be noted that the Duckietown
project, despite its many flaws, is a worthwhile project. Un-
doubtedly, the greatest advantage of the Duckietown project is
that it features a complete ecosystem that includes: documen-
tation, software, hardware and a simulation environment that
enables effective and modern teaching and advanced scientific
research in the field of autonomous robots.
The numerous flaws perceived and described in the article
can be easily corrected on their own. It should also be noted
that the managers of the Duckietown project, after reporting
problems to them and suggesting solutions, keep updating the
project documentation with proposed solutions.
4 https://github.com/duckietown

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 450

 Investigating the Impact of Confined Space
Factors on Signal Propagation
E. Skýpalová* and T. Loveček*
* University of Žilina/Faculty of Security Engineering, Žilina, Slovakia
erika.skypalova@uniza.sk, tomas.lovecek@uniza.sk

Abstract— Positioning-based systems are used in various user information to a cloud server where the received user
areas of people's lives. Monitoring and tracking the information is verified and then the relevant information
movement and location of people and other entities is about the service provided is sent to the user [5].
widespread within the outdoor environment. Over the last Bluetooth beacons only use the advertisement mode,
few years, the emphasis has been on implementing systems
for determining the location and movement of people and
which is a one-way beacon detection process. They
entities in confined spaces. Confined space positioning periodically send packets of data that are received by
systems are gaining importance for the navigation of people other devices in range. The signal is transmitted at
in various types of facilities where high numbers of people intervals ranging from 20 ms to 10 s. The transmission
are present during operational hours. Also, indoor interval affects the battery life of the beacons. The longer
positioning systems are growing in importance due to the the interval, the faster the battery drains [6].
need to implement them in articulated facilities, which The BLE data format contains 4 main pieces of
include airports, medical facilities, shopping malls and information, namely UUID, Major, Minor and Tx-power.
others, in order to improve security or optimize processes in
UUID (Universally Unique Identifier) is a 16-byte string
these facilities. Currently, indoor location systems are a
useful tool in the context of preventing and limiting the
that is used to distinguish between different beacons.
spread of COVID-19 and other respiratory diseases. In this Major is a 2-byte string used to determine the specific
article, the results of testing an indoor positioning system ownership of devices. Minor is characterized as a special
based on Bluetooth Low Energy technology using Bluetooth identifier for each device or object. Tx-power is defined
beacons are interpreted in terms of investigating the signal as the signal strength exactly 1 m from the device, which
propagation when using obstacles made of different can be used to estimate the wireless signal distance
materials. between beacons and receivers [7].
iBeacon is a protocol designed by Apple. It allows
I. INTRODUCTION smartphones or other BLE devices to receive signals [8].
The iBeacon protocol is currently used for the purpose of
Positioning services using GPS technology do not providing location-based services. Based on BLE
provide sufficient accuracy for indoor use. As a result, advertising, devices within range are able to estimate or
several indoor positioning technologies have been calculate the location of people and entities inside
developed, including Bluetooth, ZigBee, Wi-Fi, Radio buildings [9]. Many companies are involved in the
Frequency Identification (RFID), Ultra Wide Band production of Bluetooth beacon devices with the iBeacon
(UWB) and others [1]. protocol. The variety and commercial availability of the
A. Bluetooth Low Energy beacons devices allows consumers to choose a device according to
their needs and financial capabilities. However, different
Bluetooth Low Energy (BLE) is a specification released
devices from different manufacturers may vary in device
in 2011 for energy efficient devices [2]. Works in the 2.4
performance, which may affect the accuracy of the data
GHz frequency band just like classic Bluetooth [3]. It is
provided [10].
a wireless technology that is characterized by low power
Currently, there are two approaches to design an indoor
consumption and is a standard of the Bluetooth 4.0
positioning system based on Bluetooth beacons. The first
specification. Compared to previous Bluetooth standards,
approach is a system architecture in which the beacons
BLE was developed to simplify the short distance
are transmitters, usually installed on static building
communication of devices not requiring the transmission
elements inside buildings, and the receiver of the radio
of large amounts of data. The aim was to provide a
signals emitted by the beacons is a smart device in the
technology designed for monitoring and control
form of a mobile phone or a watch, which is possessed by
applications where the volume of data transferred is low
a person inside the building, with the corresponding
[4].
application installed on the device. The devices
A Bluetooth beacon is a device that wirelessly transmits
communicate with each other via Bluetooth Low Energy
signals, including location information, at regular
technology. The mobile phone receives signals from a
intervals and sends user identifiers and RSSIs as
number of beacons installed in the building and sends the
Bluetooth signals. When a smartphone user is within
signals to a server, which are then stored in a database
range of the beacon signal, an application installed on the
[11].
smartphone receives the beacon signal and then sends the

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 451

 There are several techniques for estimating the location
of objects in confined spaces using metrics. Indoor
positioning system techniques are categorized based on
the parameters which are direction, signal and time.
Direction based techniques include the Angle of Arrival
(AoA) [17]. Signal-based techniques include a Received
Signal Strength Indicator (RSSI) [18], Channel State
Information (CSI) [19], and the Power/Quality of the
Received Reference Signal (RSRP, RSRQ) [20]. Time-
based techniques are Time of Arrival (ToA)/ Time of
Figure 1. System based on Bluetooth beacons and smartphone [11] Departure (ToD) [21], Time Difference of Arrival
(TDoA) [22], Round-trip Time (RTT) [23], Phase of
In the second approach, the system consists of a Arrival (PoA) [24] and Phase Difference od Arrival
beacon, which is a transmitter that is operated by a person (PDoA) [25]. In the available literature, the above
moving in the space, and several receivers are installed techniques are also categorized based on the direction
on building structures in the space. As in the previous parameter, the second parameter being the distance,
scenario, the devices communicate with each other via where the other techniques are classified [11], [16], [26].
Bluetooth Low Energy technology. The transmitter
continuously transmits radio signals which are received II. METHODOLOGY
by the receivers and are then sent to a server and stored in
a database [11]. A reliability test of the FSC-BP103B and EEK-N
beacons was performed, where the received signal
strength, known as RSSI, was investigated. A Raspberry
Pi Zero 2 W microcomputer was used for testing, running
on the Raspbian operating system, which is based on a
Linux distribution derived from Debian. This
microcomputer is equipped with WiFi, Bluetooth, and a
micro HDMI reduction to connect to a monitor. An SD
card was used for data storage and the microcomputer
was powered via micro USB 5 V with a recommended
current of 2 A. The system was powered through an
external charger.
Figure 2. System based on receivers and Bluetooth beacon [11] The aim of the testing was to investigate the effect of
various obstacles on the received signal strength, that is
When the system is deployed, it is necessary to perform the RSSI value. The microcomputer was installed at
its calibration, which is the offline part of the positioning heights of 1.6 m, 1.8 m and 2 m. For each height,
that depending on the localization technique used, transitions were made along a circle with a radius of 1 m,
includes manual tuning of signal propagation parameters within which 4 transition points were marked. The points
such as the path loss model that need to be optimally selected were marked with the numbers 1, 7, 13 and 19,
calibrated using RSSI measurements [4]. which were placed around the microcomputer at 90°
Received Signal Strength Indicator (RSSI) is a clockwise. At each height, a series of transitions were
technique for determining distance. It is based on the made along the selected points of a circle with a radius of
1 m with each obstacle. Obstacles were placed in front of
measurement of the received radio signal. The RSSI
the person holding the EEK-N beacon in the right hand
value expresses the relative quality of the received signal and the FSC-BP103B beacon in the left hand. Two
in the device. The stronger the signal, the higher the RSSI obstacles made of different materials were used:
value [12], [13], [14]. When radio signals are transmitted polystyrene and foam pad. Transitions along the marked
between sensors, the RSSI value oscillates due to points on the circle were also performed without the use
absorption, interference and diffraction effects [15]. of obstacles.

III. RESULTS
The RSSI values shown in Table 1 were obtained by
performing an arithmetic average of 5 values for the two
beacons at each point in real time at installation heights of
1.6 m, 1.8 m and 2 m at a distance of 1 m from the tripod.
The transitions were performed over 4 selected points
located at 90° clockwise spacing.

Figure 3. Received Signal Strength Indicator [11], [16]

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 452

 To better present the results, graphs were created
TABLE I. individually for each height. When the microcomputer
EFFECT OF OBSTACLES ON THE RSSI VALUE
was placed at a height of 1.6 m at a distance of 1 m, it
Height [m] was shown that the EEK-N beacon had a larger signal
P 1,6 1,8 2 when passing through each point without obstacles.
o However, when using polystyrene and a foam pad, the
Obsta
i FSC- FSC- FSC-
cles
n EEK EEK EEK RSSI value only decreased by -2 dBm.
BP103 BP10 BP10 The FSC-BP103B beacon achieved lower RSSI values
t -N -N -N
B 3B 3B
compared to the EEK-N beacon, even when obstacles
1 -54 -64 -56 -70 -53 -57
were used. Based on the results obtained, it can be
concluded that the signal was highest in transitions
Polyst 2 -59 -67 -61 -68 -51 -61 without obstacles. The RSSI value decreased by -4 dBm
yrene 3 -53 -62 -52 -55 -55 -64 for the polystyrene and by -2 dBm for the foam pad.
4 -55 -67 -65 -75 -62 -70

1 -58 -65 -49 -57 -53 -55

Foam 2 -62 -68 -55 -63 -59 -63

pad 3 -55 -59 -61 -69 -48 -62

4 -47 -52 -59 -67 -51 -59

1 -55 -79 -51 -73 -59 -63

No 2 -51 -79 -40 -57 -49 -62
obstac
le 3 -49 -60 -52 -67 -73 -76
Figure 5. RSSI values at a distance of 1 m - height of 1.8 m with and
4 -59 -66 -42 -69 -51 -63 without obstacles

For comparison, the arithmetic averages of the RSSI In Figure 5, it can be seen that the beacon EEK-N
values shown in Table 1 were calculated at all selected achieved the highest RSSI values in the obstacle-free
points with and without obstacles for each installation transitions. By using polystyrene, the RSSI value
height with respect to the distance of 1 m. The results are decreased by -12 dBm and there was a -10 dBm decrease
presented in Table 2. with the foam pad.
The signal of the FSC-BP103B beacon was also highest
TABLE II.
for obstacle-free transitions. The RSSI value decreased
ARITHMETIC AVERAGES OF RSSI VALUES AT ALL POINTS by -7 dBm using polystyrene and by -4 dBm with the
foam pad.
Height [m]
Obstac Based on the comparison of RSSI values for the beacon
les/no 1,6 1,8 2 EEK-N and FSC-BP103B, it can be said that even at a
obstacl FSC- FSC- FSC- height of 1.8 m and a distance of 1 m, the beacon FSC-
EEK- EEK- EEK-
e BP10 BP10 BP10
N
3B
N
3B
N
3B BP103B achieved lower RSSI values. When comparing
Polyst the total RSSI values for both beacons at heights of 1.6
-55 -65 -58 -67 -55 -63
yrene and 1.8 m using obstacles, it can be concluded that at the
Foam installation height of 1.8 m there was a decrease in RSSI
-55 -63 -56 -64 -53 -60
pad
No
values for both beacons. This shows that as the
obstacl -64 -71 -70 -75 -71 -76 installation height of the microcomputer increases, the
e RSSI value decreases. However, this is not the case for
the obstacle-free transitions, where the RSSI values at the
1.8 m installation height increased for both beacons.

Figure 4. RSSI values at a distance of 1 m - height of 1.6 m with Figure 6. RSSI values at a distance of 1 m - height of 2 m with and
and without obstacles without obstacles

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 453

 When the microcomputer was installed at a height of 2 same time, RSSI values may vary depending on obstacles
m and a transition point distance of 1 m, it was shown present in the space in the form of building structures,
that the beacon EEK-N achieved the highest signal for objects made of different materials or devices that may
obstacle-free transitions as in previous tests. The use of a limit the propagation of signals, or signals may be
polystyrene obstacle and foam pad affected the signal reflected from surfaces, etc. Another factor influencing
propagation. The use of polystyrene decreased the RSSI RSSI values is the receiving capability of the devices,
value by -7 dBm. With the foam pad, there was a which results from the amount of data received over a
decrease in RSSI of -4 dBm. certain period of time. There is an assumption that the
The FSC-BP103B beacon achieved lower RSSI values higher the number of RSSI values received, the more
compared to the EEK-N beacon. However, the highest accurate the average RSSI value at a particular point will
signal was achieved in transitions without obstacles. be, and hence the more accurate the location of the
When obstacles of polystyrene and foam pad were used, transmitting device will be. Last but not least, it is
there was a decrease in RSSI value of -6 dBm for the necessary to take into account the variation of RSSI
polystyrene and -2 dBm for the foam pad. values caused by the presence of multiple devices in
space using radio frequencies to transmit signals between
IV. DISCUSSION devices.
Testing has shown that selected obstacles made of
different materials affect the propagation of the signal in ACKNOWLEDGMENT
space. The average RSSI values of the EEK-N and FSC- The article was created with the support of the project
BP103B beacon achieved lower RSSI values when using of the University of Žilina APVV-20-0457 Monitoring
obstacles compared to the obstacle-free condition. When and tracking of movement and contact of persons in health
the RSSI values of the given obstacles were compared care facilities.
with each other, it was found that the RSSI values of both
beacons were lower when using polystyrene compared to REFERENCES
using foam pad, however, the decrease in RSSI values [1] X. Li, G. Yu, ,,Indoor Positioning System Based on Standardizing
Waveform Tendency,” in Journal of Sensors, vol. 3, 2021, pp. 1-
was not significant. 10.
In addition to investigating the effect of obstacles made [2] W. Lu, X. Ma et al, 2023. ,,Toward Designing an Attentive Deep
of different materials on signal propagation, the location Trajectory Predictor Based on Bluetooth Low Energy Signal,” in
of transmitting and receiving devices has also been 57th Annual Conference on Information Sciences and Systems
(CISS), 2023, pp. 1-6.
shown to affect the propagation and quality of the signal
[3] A. Said, A. Galwish, M. Assal, M. Elgazar, ,,Simulating
in a confined space. From the tests carried out, it was Associated Routing Protocol for Bluetooth Low Energy Devices,”
found that changing the location of the receiving device In: Journal of Computer Engineering (IOSR-JCE). Vol. 6, 2021,
at 3 installation heights had an effect on the RSSI values. pp. 63-69.
As the height of the receiving equipment installation [4] D. Čabarkapa, I. Grujić et al, ,,Comparative analysis of the
increased, there was a decrease in the RSSI values with Bluetooth Low-Energy indoor positioning systems,” 12th
International Conference on Telecommunication in Modern
and without the use of obstructions with each change. Satellite, Cable and Broadcasting Services (TELSIKS). 2015, pp.
1-4.
V. CONCLUSSION [5] J. Jung, – Y. Choi, ,,Measuring Transport Time of Mine
The article focused on testing the monitoring and Equipment in an Underground Mine Using a Bluetooth Beacon
System,” in Minerals, 2017, vol. 1, pp. 1-10.
tracking system in terms of the effect of obstacles made
[6] P. Spachos, N. K. Plataniotis, ,,BLE Beacons for Indoor
of different materials on signal propagation in a confined Positioning at an Interactive IoT-Based Smart Museum,” in IEEE
space. Testing was carried out using 2 types of obstacles Systems Journal, 2020, vol. 14, pp. 3483 – 3493.
made of different materials in order to investigate the [7] K. Nonalinsavath et al, 20 ,,Integration of Indoor Localization
inter-comparison of the different obstacles and their System using Wi-Fi Fingerprint, Bluetooth Low Energy Beacon
and Pedometer Based on Android Application Platform,” in
effect on the quality of the propagating signal. Testing International Journal of Intelligent Engineering and Systems,
was also carried out without using the selected obstacles. 2020, vol. 13, pp. 171-181.
The aim of testing the system without obstacles was to [8] J. Huang, CH. J. CH. Chi, T. W. HUNG, ,,Hybrid-AI-Based
show that obstacles in the space have an effect on the iBeacon Indoor Positioning Cybersecurity: Attacks and Defenses,”
propagation of the signal and can affect its quality, in Journal of Sensors, 2023, vol. 23, pp. 1-19.
[9] X. Na et al, ,,Wi-attack: Cross-technology Impersonation Attack
leading to inaccuracies in the positioning of persons and against iBeacon Services,” in 18th Annual IEEE International
entities in the confined space. In confined spaces, the Conference on Sensing, Communication, and Networking
emphasis is on positioning accuracy as opposed to the (SECON), 2021.
outdoor environment. Therefore, it is necessary to [10] M. Fachri, A. Khumaidi, ,,Positioning Accuracy of Commercial
investigate how individual elements may limit the ability Bluetooth Low Energy Beacon,” in IOP Conference Series:
Materials Science and Engineering, 2019, vol. 662, pp. 1-4.
to propagate the signal. [11] M. Zare et al, ,,Applications of Wireless Indoor Positioning
The propagation and quality of the signal in space is Systems and Technologies in Underground Mining: a Review,” in
affected by the relative distance between the transmitting Mining, Metallurgy & Exploration, 2021, vol. 38, pp. 2307–2322.
and receiving devices. RSSI values may also be affected [12] W. Zhang, X. Yang, ,,DV-Hop Location Algorithm Based on RSSI
by the height of the installation of the receiving devices, Correction,” in Electronics, 2023, vol. 12, pp. 1-15.
which may vary according to the type of space. At the [13] Y. Dong et al, ,,The research and application of indoor location
algorithm based on wireless sensor network,” in IEEE 3rd

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 454

 International Conference on Communication Software and [20] N. Poosamani, I. Rhee, ,,Towards a practical indoor location
Networks. matching system using 4G LTE PHY layer information,” in IEEE
[14] K. Maaloul et al, ,,Bluetooth beacons based indoor positioning in a International Conference on Pervasive Computing and
shopping malls using machine learning,” in Bulletin of Electrical Communication Workshops (PerCom Workshops), 2015.
Engineering and Informatics, 2023, vol. 12, pp. 911-921. [21] N. Mrindoko, M. L. MINGA, ,,A Comparison Review of Indoor
[15] A. Mussina, S. Aubakirov, ,,Improving Indoor Positioning via Positioning Techniques,” in International Journal of Computer
Machine Learning,” in Proceedings of the 8th International (IJC), 2016, vol. 21, pp. 42-49.
Conference on Data Science, Technology and Applications [22] W. Xie et al, ,,Time Difference of Arrival Estimation of Stationary
DATA, 2019, vol. 1, pp. 190-195. Targets Utilizing Deep Neural Network,” in Journal of Physics:
[16] N. S. S. Jailani et al, ,,Indoor Positioning System: A review,” in Conference Series, 2022, vol. 2414, pp. 1-6.
International Journal of Advanced Computer Science and [23] S. Yan et al, ,,Wi-Fi RTT based indoor positioning with dynamic
Applications (IJACSA), 2022, vol. 13, pp. 477 – 490. weighted multidimensional scaling,” in International Conference
[17] G. Chen et al, ,,A Review of Device-Free Indoor Positioning for on Indoor Positioning and Indoor Navigation (IPIN), 2019.
Home-Based Care of the Aged: Techniques and Technologies,” in [24] M. Scherhäufl et al, ,, Indoor Localization of Passive UHF RFID
Computer Modeling in Engineering & Sciences, 2023, vol. 135, Tags Based on Phase-of-Arrival Evaluation,” in IEEE
pp. 1901-1940. Transactions on Microwave Theory and Techniques, 2013, vol.
[18] S. Subedi, J. Y. Pyun, ,,A Survey of Smartphone-Based Indoor 61, pp. 4724 – 4729.
Positioning System Using RF-Based Wireless Technologies,” in [25] Y. Zhang, L. Duan, ,,A phase-difference-of-arrival assisted ultra-
Sensors, 2020, vol. 24, pp. 2-32. wideband positioning method for elderly care,” 2021, vol. 170.
[19] Z. Zhang, et al, ,,TIPS: Transformer Based Indoor Positioning [26] S. P. Farahsari et al, ,,A Survey on Indoor Positioning Systems for
System Using Both CSI and DoA of WiFi Signal,” in IEEE IoT-Based Applications,” in IEEE Internet of Things Journal,
Access, 2022, vol. 10, pp. 111363-111376. 2022, vol. 9, pp. 7680 – 7699.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 455

&RQVWUXFWLRQSURFHHGLQJVLQWKH6ORYDN5HSXEOLF
$QRYHUYLHZRIWRROVIRUHIILFLHQWH[FKDQJHDQG
PDQDJHPHQWRILQIRUPDWLRQLQWKH%,0
HQYLURQPHQW

-6PHWDQNRYi $ĆXULã 55XþLQVNê 30pViURã DQG/=HPDQRYi 
7HFKQLFDO8QLYHUVLW\RI.RVLFH)DFXOW\RI&LYLO(QJLQHHULQJ([SHUWV,QVWLWXWHLQ&RQVWUXFWLRQ9\VRNRãNROVNi
.RãLFH
7HFKQLFDO8QLYHUVLW\RI.RVLFH)DFXOW\RI&LYLO(QJLQHHULQJ,QVWLWXWHRI7HFKQRORJ\(FRQRPLFVDQG0DQDJHPHQW
LQ&RQVWUXFWLRQ9\VRNRãNROVNi.RãLFH
MDQDVPHWDQNRYD#WXNHVNDGULDQGXULV#WXNHVNUDVWLVODYUXFLQVN\#WXNHVNSHWHUPHVDURV#WXNHVN
OXFLD]HPDQRYD#WXNHVN

$EVWUDFW²7KHZRUOGDURXQGXVLVWHFKQRORJLFDOO\DGYDQFHG FRQIOLFW LQ 8NUDLQH D GHFOLQH ZDV UHFRUGHG LQ LQGLYLGXDO
HYHU\ GD\ :LWKLQFUHDVLQJ XUEDQL]DWLRQDQGWKHJURZWKRI VHJPHQWV RI WKH HFRQRP\ UHVSHFWLYHO\ XQIDYRXUDEOH
VPDUWFLWLHVODUJHFRXQWULHVDURXQGWKHZRUOGEHJDQWRSD\ UHVXOWV LQ VHOHFWHG HFRQRPLF LQGLFDWRUV $IWHU \HDUV RI
PRUH DQG PRUH DWWHQWLRQ WR WKH LPSOHPHQWDWLRQ RI JURZWK \HDUVDQG DGHFUHDVHLQWKHVKDUHRI
LQQRYDWLRQV LQ WKHLU SURFHVVHV &RQVWUXFWLRQ LV RQH RI WKH FRQVWUXFWLRQ JURVV GRPHVWLF SURGXFW ZDV UHFRUGHG LQ
LPSRUWDQWEUDQFKHVRIWKHHFRQRP\,QQRYDWLYHWHFKQRORJLHV ,QWKHFUHDWLRQRIWKHJURVVGRPHVWLFSURGXFW
DUH LQFUHDVLQJO\ XVHG LQ WKH SODQQLQJ DQG PDQDJHPHQW RI ZDVDWWKHOHYHORILQDWWKHOHYHORIDQG
FRQVWUXFWLRQ SURGXFWLRQ &XUUHQWO\ LQ PDQ\ FRXQWULHV LQLWZDVDGHFUHDVHWRWKHOHYHORI,QWHUPVRI
WKHUH LV D FKDQJH DQG GHVLJQ RI QHZ VWDQGDUGV DQG FRQVWUXFWLRQSURGXFWLRQDIWHUDPLOGUHFHVVLRQDQLQFUHDVH
OHJLVODWLYH IUDPHZRUNV WKDW UHIOHFW WKH QHHGV RI WKH VW ZDVUHFRUGHGLHFRQVWUXFWLRQSURGXFWLRQLQFXUUHQWSULFHV
FHQWXU\ DQG GLUHFWO\ VXSSRUW WKH WUDQVLWLRQ RI LQGXVWU\ WR LQFUHDVHG FRPSDUHG WR  IURP WKH OHYHO RI 
,QGXVWU\,WLVLQQRYDWLYHWHFKQRORJLHVVXFKDV%XLOGLQJ PLO(85WRWKHOHYHORIPLO(85>@
,QIRUPDWLRQ 0RGHOOLQJ WKDW PDNH LW SRVVLEOH WR HOLPLQDWH
7KH GHFLVLYH IDFWRUV RI WKH SURMHFW DUH HIILFLHQW DQG
ORVVHV DQG SUREOHPV UHVXOWLQJ IURP SODQQLQJ PDQDJHPHQW
DQGFRQWURORILQGLYLGXDOSURFHVVHV,QQRYDWLYHWHFKQRORJLHV
SURPSWSODQQLQJDQGPDQDJHPHQWRILQGLYLGXDOSURFHVVHV
KHOSWRHIILFLHQWO\DQGSURPSWO\SODQDQGPDQDJHLQGLYLGXDO
$Q LPSRUWDQW EDVLV IRU SODQQLQJ DQG PDQDJHPHQW LV
SURFHVVHV 7KH SDSHU SURYLGHV D EULHI RYHUYLHZ RI%XLOGLQJ TXDOLW\ SURMHFW GRFXPHQWDWLRQ ZLWKRXW DQ\ FRQIOLFWV WKDW
,QIRUPDWLRQ 0RGHOOLQJ WRROV VXFK DV &'(  &RPPRQ 'DWD ZRXOGMHRSDUGL]HWKHWLPHRUILQDQFLDOSODQ,WLVSRVVLEOH
(QYLURQPHQW ,)&  ,QGXVWU\ )RXQGDWLRQ &ODVVHV 09'  WRFUHDWHDFUHDJHUHSRUWVRIWKHUHTXLUHGTXDOLW\IURPWKH
0RGHO 9LHZ 'HILQLWLRQ %&) IRUPDW  %,0 &ROODERUDWLRQ UHZRUNHG SURMHFW GRFXPHQWDWLRQ WKXV HOLPLQDWLQJ WKH
)RUPDW EXLOGLQJ60$57 'DWD 'LFWLRQDU\ %(3  %,0 RFFXUUHQFH RI ILQDQFLDO DQG WLPH GLVFUHSDQFLHV 0DQ\
([HFXWLRQ3ODQ DQG%,0/LEUDU\:LWKLQWKHFRQWULEXWLRQ LQQRYDWLYH WHFKQRORJLHV VXFK DV 9LUWXDO 5HDOLW\ 95 
LQGLYLGXDO %,0 WRROV DUH GHILQHG DQG DQDO\]HG ZKLFK ZLOO 0L[HG 5HDOLW\ 05  $XJPHQWHG 5HDOLW\ $5 
EH JUDGXDOO\ LPSOHPHQWHG LQ WKH SURFHVVHV RI FRQVWUXFWLRQ .QRZOHGJH 6\VWHPV .0  $XWRPDWLF ,GHQWLILFDWLRQ DQG
SHUPLWV LQ WKH WHUULWRU\ RI WKH 6ORYDN 5HSXEOLF DQG WKHLU 'DWD &DSWXUH $,'&  DQG %XLOGLQJ ,QIRUPDWLRQ
DGYDQWDJHVDQGEDUULHUVZLOOEHSRLQWHGRXW 0RGHOOLQJ %,0  DUH LQFUHDVLQJO\ XVHG IRU WKLV SXUSRVH
>@
7KHUH DUH D ORW RI GHILQLWLRQV RI WKH WHUP LQIRUPDWLRQ
, ,1752'8&7,21 PRGHOOLQJ RI EXLOGLQJV ,Q JHQHUDO %,0 LV DQ LQWHOOLJHQW
7KHFRQVWUXFWLRQLQGXVWU\LVDQLPSRUWDQWVHFWRURIWKH SURFHVV EDVHG RQ ' PRGHOV WKDW RIIHUV SURIHVVLRQDOV LQ
HFRQRP\ &RQVWUXFWLRQ LV D FROOHFWLRQ RI SURFHVVHV WKDW DGGLWLRQ WR JUDSKLFDO LQIRUPDWLRQ QRQJUDSKLFDO
UHVXOW LQ D FRQVWUXFWLRQ ZRUN 7KH VWDQGDUG FRQVWUXFWLRQ LQIRUPDWLRQ WR KHOS PRUH HIIHFWLYHO\ SODQ GHVLJQ EXLOG
SURMHFW LQFOXGHV D ODUJH QXPEHU RI SDUWLFLSDQWV VXFK DV DQG PDQDJH EXLOGLQJV DQG LQIUDVWUXFWXUH 7KH PRGHO KDV
LQYHVWRU GHYHORSHU DUFKLWHFW GHVLJQHU VWDWLF XVHUV LQIRUPDWLRQRQWLPHSODQQLQJ 'WLPHGLPHQVLRQ FRVW
DGPLQLVWUDWRUVDQGWKHOLNH,WLVTXLWHGLIILFXOWWRFRPELQH RUILQDQFLDOSODQQLQJ 'FRVWGLPHQVLRQ VXVWDLQDELOLW\
WKH UHTXLUHPHQWV RI LQGLYLGXDO SDUWLFLSDQWV LQWR RQH '  VXVWDLQDELOLW\ GLPHQVLRQ  DQG DGPLQLVWUDWLRQ DQG
FRPSOHWHZRUN,QDGGLWLRQWRWKHDERYHIDFWFRQVWUXFWLRQ PDLQWHQDQFH 'IDFLOLW\PDQDJHPHQWGLPHQVLRQ >@
SURGXFWLRQ LV DOVR FKDUDFWHUL]HG E\ KLJK FRVWV %,0 HQDEOHV LQGLYLGXDO FRQVWUXFWLRQ WHDPV WR ZRUN
FRQVLGHUDEOH WLPHFRQVXPLQJ LPSOHPHQWDWLRQ DQG PRUH HIILFLHQWO\ HQDEOHV GDWD FROOHFWLRQ PXWXDO
FRPSOH[LW\>@ H[FKDQJH HYDOXDWLRQ PDQDJHPHQW ZKLFK EHQHILWV
%\  WKH FRQVWUXFWLRQ VHFWRU ZDV FRQVWDQWO\ LQGLYLGXDOSDUWLFLSDQWVDQGRSHUDWLRQV%DVHGRQWKHDERYH
JURZLQJ &RQVWUXFWLRQ SURGXFWLRQ LQFUHDVHG JURVV IDFWVWKHUDWHRILPSOHPHQWDWLRQRILQIRUPDWLRQPRGHOOLQJ
GRPHVWLF SURGXFW DQG RWKHU HFRQRPLF LQGLFDWRUV JUHZ RI EXLOGLQJV LV LQFUHDVLQJ DQG LQGLYLGXDO FRXQWULHV DQG
'XH WR WKH LPSDFW RI WKH &RYLG SDQGHPLF DQG WKH RUJDQL]DWLRQVDUHFUHDWLQJVRFDOOHG%,0PDQGDWHV7KHVH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 456

FRXQWULHVFUHDWLQJ%,0PDQGDWHVLQFOXGH8QLWHG6WDWHVRI LQFUHDVH WKH HIILFLHQF\ RI FRQVWUXFWLRQ SURFHVVHV %,0
$PHULFD %UD]LO &KLOH 8. 'HQPDUN 1RUZD\ )LQODQG DGRSWLRQ UHPDLQV IUDJPHQWHG LQ $XVWUDOLD DV WKHUH LV QR
/DWYLD *HUPDQ\ .D]DNKVWDQ -DSDQ 6RXWK .RUHD XQLILHG DSSURDFK WR LW )HZ SULYDWH SOD\HUV LPSOHPHQW
9LHWQDP8$(DQG$XVWUDOLD)UDQFH,WDO\6SDLQ&]HFK %,0>@
5HSXEOLF 0H[LFR &ROXPELD 3HUX $UJHQWLQD 8UXJXD\ 7KH WUDQVIRUPDWLRQ RI WKH FRQVWUXFWLRQ LQGXVWU\ KDV
&KLQD6LQJDSRUH,QGRQHVLDDQG1HZ=HDODQGDOVRSODQWR EHHQ VORZHU LQ *HUPDQ\ GXH WR FRQIOLFWV DQG
FUHDWH%,0PDQGDWHVLQWKHQHDUIXWXUH>@ PLVFRPPXQLFDWLRQ EHWZHHQ *HUPDQ DXWKRULWLHV GXULQJ
7KH8.KDVUDGLFDOO\DGRSWHGD%,0VWUDWHJ\,Q$SULO WKH GHFLVLRQPDNLQJ SURFHVV 7KH EDVLF SUREOHP WKH\
 WKH JRYHUQPHQW PDQGDWHG /HYHO  %,0 IRU DOO IDFHGZDVWKDWWKH\KDGVHHQDGHFOLQHLQSURILWDELOLW\DQG
SXEOLFSURMHFWV7KHFRQVWUXFWLRQLQGXVWU\LQWKH8.LVRQH DGHFOLQHLQWKHQXPEHURIFRQVWUXFWLRQSURMHFWVRYHUWKH
RIWKHPRVWDGYDQFHGDQGGLJLWL]HGLQGXVWULHVLQWKHZRUOG ODVWGHFDGH7KHLUIRFXVLVQRZVKLIWLQJWR%,0DQGWKH\
,Q  WKHUH ZDV D PDUNHG LQFUHDVH LQ GLJLWL]DWLRQ E\ DUHH[SHFWHGWRPDQGDWH%,0E\7KHIRFXVLVQRZ
 7KH %,0 FRQFHSW KDV EHHQ XVHG DQG VXFFHVVIXOO\ RQWKHFROODERUDWLYHDSSURDFKWKDWWKHQHZWHFKQRORJ\ZLOO
LPSOHPHQWHG LQ PDQ\ FRXQWULHV DURXQG WKH ZRUOG 7KH RIIHU IURP LWV LPSOHPHQWDWLRQ LQ DOO SKDVHV RI
8QLWHG 6WDWHV LV DPRQJ WKH SLRQHHUV LQ WKH GHYHORSPHQW FRQVWUXFWLRQ,QWKH\LQLWLDWHGD%,0SLORWSURMHFWE\
DQGDGRSWLRQRI%,0LQWKHFRQVWUXFWLRQLQGXVWU\%,0LV GLJLWL]LQJWUDQVSRUWSURMHFWV>@
QRW \HW PDQGDWHG LQ DOO VWDWHV EXW PRUH JURZWK LV
H[SHFWHG7KHSURFHVVRILPSOHPHQWLQJWKHGLJLWL]DWLRQRI ,, %,0722/6)25,1)250$7,21(;&+$1*($1'
WKHFRQVWUXFWLRQLQGXVWU\LQWKH86KDVEHHQVORZDQGWKLV 0$1$*(0(17
LV EHFDXVH LW LV LQ DQ H[SHULPHQWDO IRUP /RQJWHUP
VROXWLRQVDQGLPSURYHPHQWVDUHVWLOOLQWKHZRUNV$PRQJ 7KHUHDUHDORWRIWRROVWKDWIDFLOLWDWHDQGHQDEOHZRUNLQ
WKH OHDGHUV LQ WKH ILHOG RI LPSOHPHQWDWLRQ LV DOVR )UDQFH WKH %XLOGLQJ ,QIRUPDWLRQ 0RGHOOLQJ HQYLURQPHQW %,0
ZKLFKLQLQLWLDWHGWKHFRQVWUXFWLRQRIKRXVHV WRROV DUH RSHUDWLRQDO WRROV VXSSOLHG ZLWK VRIWZDUH IRU
XVLQJ %,0 $V SDUW RI WKH DIRUHPHQWLRQHG IDFW DSSUR[ FUHDWLQJ%,0PRGHOVRUWRROVWKDWFDQEHLPSOHPHQWHGLQ
(85  PLOOLRQ ZDV DOORFDWHG WR VXSSRUW WKH FRPSOHWH WKHVRIWZDUHDSSOLFDWLRQVXVHG7KHLQGLYLGXDOWRROVDOORZ
GLJLWL]DWLRQ RI WKH FRQVWUXFWLRQ LQGXVWU\ ,Q WKH IXWXUH WKHXVHUWRLPSOHPHQWVSHFLDOL]HGDQDO\VHVRUDVSHFWVWKDW
%,0 LPSOHPHQWDWLRQ ZLOO EH UHTXLUHG LQ SXEOLF SURMHFWV ZRXOG QRW RWKHUZLVH EH SURFHVVHG VXFK DV HQHUJ\
7KH)UHQFKLQLWLDWLYH'LJLWDO7UDQVIRUPDWLRQ3ODQDLPVWR DQDO\VHV FDOFXODWLRQV FRVW PDQDJHPHQW WLPH
DFKLHYH VXVWDLQDELOLW\ DQGRSWLPL]H FRVWV7KH PHQWLRQHG PDQDJHPHQW VDIHW\ DQG KHDOWK DQG WKH OLNH %,0 WRROV
DSSOLFDWLRQ ZLOO UHVXOW LQ VWDEOH JURZWK RI WKH HFRQRP\ FDQ EH GHILQHG DV WRROV WKDW DOORZ \RX WR VSHFLI\ DQG
ZLWK DGGHG YDOXH EHQHILWV VXFK DV HQYLURQPHQWDO VRFLDO SURFHVV LQGLYLGXDO RXWSXWV LQ D JLYHQ PRGHO 6XFK WRROV
DVSHFWV DQG VXSSRUWLQJ WKH SURWHFWLRQ RI VPDOO DQG LQFOXGH
PHGLXPVL]HGHQWHUSULVHV6FDQGLQDYLDQFRXQWULHVVXFKDV
1RUZD\ )LQODQG 'HQPDUN DQG 6ZHGHQ DUH DPRQJ WKH • &'(±&RPPRQ'DWD(QYLURQPHQW
OHDGHUV LQ WKH SUDFWLFDO DSSOLFDWLRQ RI %,0 LQ WKHLU • ,)&±,QGXVWU\)RXQGDWLRQ&ODVVHV
SURFHVVHV,QWKHVHFRXQWULHVPDQGDWHVIRUSXEOLFSURMHFWV • 09'±0RGHO9LHZ'HILQLWLRQ
DUH LQ SODFH 7KH PHQWLRQHG FRXQWULHV DUH DPRQJ WKH • %&)IRUPDW±%,0&ROODERUDWLRQ)RUPDW
JOREDOOHDGHUVLQWKHDGRSWLRQDQGLPSOHPHQWDWLRQRI%,0
)LQODQG ZLOO LPSOHPHQW %,0 WHFKQRORJ\ LQ  %,0 • EXLOGLQJ60$57'DWD'LFWLRQDU\
LPSOHPHQWDWLRQLVWKHKLJKHVWLQWKLVFRXQWU\DVLQ • %(3±%,0([HFXWLRQ3ODQ
 RI DUFKLWHFWXUH ILUPV DQG  RI HQJLQHHULQJ ILUPV • %,0/LEUDU\DQGRWKHU
XVHG%,0LQWKHLUURXWLQH,Q6ZHGHQWKHXVHRI%,0ZDV
PDQGDWHGLQ6ZHGHQKDGDOUHDG\SXEOLVKHGPDQXDOV $ &'(±&RPPRQ'DWD(QYLURQPHQW
WR VXSSRUW %,0 EHIRUH WKH JRYHUQPHQW JXLGHOLQHV ,Q
 %,0 $OOLDQFH 6ZHGHQ WRRN WKH LQLWLDWLYH WR EULQJ &RPPRQ 'DWH (QYLURQPHQW LV D XQLTXH VRXUFH RI
WRJHWKHU SXEOLF DQG SULYDWH LQGXVWU\ SOD\HUV WR GHYHORS LQIRUPDWLRQ ZKLFK LV GHVLJQHG WR FROOHFW PDQDJHV DQG
EHVW SUDFWLFHV LQ FRQVWUXFWLRQ 7KDQNV WR WKH VXFFHVVIXO GLVVHPLQDWHV LQIRUPDWLRQ WR DOO SURMHFW SDUWLFLSDQWV 7KH
LPSOHPHQWDWLRQRI%,0'HQPDUNLVDPRQJWKHOHDGHUVLQ DGYDQWDJH LV WKH IDFLOLWDWLRQ RI ZRUN EHWZHHQ LQGLYLGXDO
(XURSH,Q   RI FRQVWUXFWLRQ SURIHVVLRQDOV XVHG WHDPV WKURXJK D FOHDUO\ GHILQHG YHUVLRQ RI LQIRUPDWLRQ
%,0 LQ WKHLU SURMHFWV 6WDWVE\JJ DQG 1RUZHJLDQ WKXV HOLPLQDWLQJ HUURUV GXSOLFDWLRQV DQG
+RPHEXLOGHUVDUHWKHWZRDVVRFLDWLRQVUHVSRQVLEOHIRUWKH PLVXQGHUVWDQGLQJV>@
FRQVWUXFWLRQ PDQDJHPHQW DQG GHYHORSPHQW RI DOO SXEOLF 7KH&RPPRQ'DWH(QYLURQPHQWLVDFHQWUDOUHSRVLWRU\
DQG JRYHUQPHQW IDFLOLWLHV LQ 1RUZD\ 1RUZD\ KDV ZKHUH LQIRUPDWLRQ DERXW FRQVWUXFWLRQ SURMHFWV LV ORFDWHG
PDQGDWHG WKH XVH RI %,0 IRU DOO LWV SURMHFWV DQG VLQFH &'(V DUH QRW MXVW DERXW DVVHWV FUHDWHG LQ D %,0
KDVUHTXLUHG,)&FRPSOLDQW%,0PRGHOV>@ HQYLURQPHQW 7KH FRQWHQW LQFOXGHV JUDSKLF DQG QRQ
7KH%XLOGLQJ$XWKRULW\RI6LQJDSRUHKDVLPSOHPHQWHG JUDSKLFLQIRUPDWLRQ8VLQJDVLQJOHVRXUFHRILQIRUPDWLRQ
%,0 RQ DOO SXEOLF SURMHFWV VLQFH  7KH 6LQJDSRUH VKRXOG VWUHQJWKHQ FRRSHUDWLRQ EHWZHHQ SURMHFW PHPEHUV
JRYHUQPHQWKDVDOORFDWHGDWRWDOIXQGRIPLOOLRQIRU UHGXFHHUURUVDQGDYRLGGXSOLFDWLRQ
WKHVXFFHVVIXOLPSOHPHQWDWLRQRI%,07KH%XLOGLQJDQG :KHQVHWWLQJXSD&'(LWLVLPSRUWDQWWRFRQVLGHUZKR
&RQVWUXFWLRQ $XWKRULW\ &251(7 LQWURGXFHG %,0 H PDNHV XS WKH VWDII WHDP ,Q SUDFWLFH WKH &'( FDQ EH
VXEPLVVLRQLQ6LQJDSRUH VFRQVWUXFWLRQLQGXVWU\KDV VSHFLILHGDVDQHQYLURQPHQWWKDWVHUYHVWKHQHHGVRIWKRVH
GHPRQVWUDWHG SURGXFWLYLW\ WKURXJK %,0 LPSOHPHQWDWLRQ ZKRDUHRQWKHSURMHFWVXSSO\VLGHDQGKDYHFROOHFWHGDQG
7KH FRQVWUXFWLRQ LQGXVWU\ FRQWULEXWHV VLJQLILFDQWO\ WR YHULILHG WKH RXWSXWV WKDW DUH UHTXLUHG IRU WKH LQIRUPDWLRQ
$XVWUDOLD VHFRQRPLFJURZWK$SSUR[LPDWHO\RIWKH UHTXLUHPHQWV RI WKH HPSOR\HU $ FRPPRQ GDWD
WRWDO*'3FRPHVIURPFRQVWUXFWLRQ([SHUWVDGRSW%,0WR HQYLURQPHQW LV LPSRUWDQW IRU LQFUHDVLQJ SURGXFWLYLW\ DQG

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 457

TXDOLW\LQLQGXVWU\DQGUHODWHGDFWRUVLQWKHVXSSO\FKDLQ 09' FDQ EH XVHG WR VXPPDUL]H VSHFLILF GDWD EXW WKH
&'(SURYLGHVDFFHVVWRXSWRGDWHVWDQGDUGL]HGHDV\WR DFFXUDF\RUFRQVLVWHQF\RIWKHGDWDFDQQRWEHJXDUDQWHHG
XQGHUVWDQG GHWDLOHG DQG XSWRGDWH LQIRUPDWLRQ DW DOO 'DWD H[SRUWHG E\ DSSOLFDWLRQV FDQ EH YHULILHG WR VXLW WKH
VWDJHVRIDEXLOGLQJ VOLIHF\FOH>@ H[FKDQJHXVLQJRWKHU%,0WRROVDQGVHUYLFHV>@
% ,)&±,QGXVWU\)RXQGDWLRQ&ODVVHV ' %&)IRUPDW±%,0&ROODERUDWLRQ)RUPDW
,)&  ,QGXVWU\ )RXQGDWLRQ &ODVVHV LV D VWDQGDUGL]HG 7KH %&) IRUPDW DOORZV GLIIHUHQW %,0 DSSOLFDWLRQV WR
GLJLWDO GHVFULSWLRQ RI HPEHGGHG DVVHWV ,)& LV DQ LQWHUDFWZLWKHDFKRWKHU7KHSUREOHPDULVLQJIURPWKHXVH
LQWHUQDWLRQDO VWDQGDUG ,62    WKDW VXSSRUWV RI ,)& PRGHOV DQG WKHLU VKDULQJ EHWZHHQ HPSOR\HHV DQG
GDWD WUDQVIHU DFURVV D ZLGH UDQJH RI KDUGZDUH GHYLFHV VRIWZDUHDSSOLFDWLRQVFDQEHHOLPLQDWHGE\XVLQJ5(67IXO
VRIWZDUHSODWIRUPVDQGLQWHUIDFHV VHUYLFHVZKLFKFRQQHFWVVRIWZDUHDSSOLFDWLRQVGLUHFWO\RU
7KH,)&6FKHPH6SHFLILFDWLRQLVWKHSULPDU\WHFKQLFDO WRDGHGLFDWHG%&)VHUYHUZKLFKVHUYHVDVDKXEIRUVXFK
SURGXFWRIEXLOGLQJ60$57,QWHUQDWLRQDOWRPHHWLWVJRDO FRPPXQLFDWLRQ ,W LV LQWHQGHG SULPDULO\ WR GHILQH PRGHO
RISURPRWLQJRSHQ%,07KH,)&VFKHPDLVDVWDQGDUGL]HG YLHZVDQGUHODWHGLQIRUPDWLRQDERXWFROOLVLRQVDQGHUURUV
GDWDPRGHOWKDWORJLFDOO\FRGLILHV DVVRFLDWHGZLWKVSHFLILFREMHFWV,WDOORZV\RXWRH[FKDQJH
LQIRUPDWLRQPDUNSODFHVDQGWKHOLNH>@
• LGHQWLW\ DQG VHPDQWLFV QDPH PDFKLQHUHDGDEOH XQLTXH 7KH IRUPDW ZRUNV RQ WKH SULQFLSOH RI GDWD WUDQVIHU LQ
LGHQWLILHUREMHFWW\SHRUIXQFWLRQ  ;0/IRUPDW;0/LVFRQWH[WXDOLQIRUPDWLRQWKDWGLUHFWO\
UHIHUHQFHV YLHZV FDSWXUHG XVLQJ 31* DQG ,)&
• SURSHUWLHV PDWHULDOFRORXUWKHUPDOSURSHUWLHV  FRRUGLQDWHV DQG %,0 HOHPHQWV UHIHUHQFHG WKURXJK WKHLU
• UHODWLRQVKLSV ORFDWLRQFRQQHFWLRQ  ,)& *8,'V IURP RQH DSSOLFDWLRQ WR DQRWKHU 7KH ,)&
• REMHFWV GHILQLWLRQ RI VWUXFWXUDO HOHPHQWV HJJ FROXPQV IRUPDWLVDQLQWHUQDWLRQDORSHQ%,0VWDQGDUGDVLVWKH,)&
VODEVHWF 
DQG'DWD'LFWLRQDU\ E6'' DYDLODEOHWRDOO%&)FDQEH
XVHGLQWZRZD\V
• DEVWUDFWFRQFHSWV ODERXUFRVWV 
• SURFHVVHV LQVWDOODWLRQRSHUDWLRQ  • WKURXJKWKHH[FKDQJHRIILOHV
• SHRSOH LQYHVWRUGHVLJQHUVXSSOLHUVHWF  • ZHEVHUYLFH>@
,)& LV PDLQO\ XVHG IRU WKH H[FKDQJH RI GDWD EHWZHHQ
LQGLYLGXDO SURMHFW SDUWLFLSDQWV7KH,)&FDQEH XVHGDV D ( EXLOGLQJ60$57'DWD'LFWLRQDU\
PHDQV RI DUFKLYLQJ SURMHFW LQIRUPDWLRQ  GHVLJQ %XLOGLQJ60$57'DWD'LFWLRQDU\ E6'' LVDQRQOLQH
SURFXUHPHQWDQGFRQVWUXFWLRQSURFHGXUHVRUDVDQXSWR VHUYLFH WKDW FRQWDLQV FODVVLILFDWLRQV DQG WKHLU SURSHUWLHV
GDWH VHW RI LQIRUPDWLRQ IRU ORQJWHUP SURWHFWLRQ DQG VWDQGDUGV DQG XQLWV %6'' DOORZV D OLQN EHWZHHQ DOO
RSHUDWLRQ)LOHVFDQEHHQFRGHGLQYDULRXVIRUPDWVVXFKDV FRQWHQW LQ WKH GDWDEDVH ,W SURYLGHV D VWDQGDUGL]HG
;0/-621DQG67(3DQGWUDQVIHUUHGYLDZHEVHUYLFHV ZRUNIORZ WKDW HQVXUHV GDWD TXDOLW\ DQG FRQVLVWHQF\ RI
LPSRUWHG  H[SRUWHG WR ILOHV RU PDQDJHG FHQWUDOO\ RU YLD LQIRUPDWLRQ
OLQNHGGDWDEDVHV>@ %6''SURYLGHVHDV\DQGHIILFLHQWDFFHVVWRDOONLQGVRI
VWDQGDUGV 7KURXJK E6'' %,0 PDQDJHUV FDQ FRQWURO
& 09'±0RGHO9LHZ'HILQLWLRQ WKHLU %,0 GDWD ,W SURYLGHV D VWDQGDUGL]HG ZRUNIORZ WR
0RGHO 9LHZ 'HILQLWLRQ LV D VXEVHW RI WKH RYHUDOO ,)& ILQG WKH ULJKW VHW RI FODVVLILFDWLRQV RI HQWLWLHV SURSHUWLHV
VFKHPD WKDW LV XVHG WR H[FKDQJH GDWD IRU D SDUWLFXODU DQGDOORZHGYDOXHV7KHVWDQGDUGVFRQWDLQHGLQWKHE6''
DSSOLFDWLRQRUZRUNIORZ7KH09'GRFXPHQWDWLRQDOORZV DUHLQWHUFRQQHFWHGWKXVPD[LPL]LQJLQWHURSHUDELOLW\>@
IRU UHSHWLWLRQ DQGSURYLGHV FRQVLVWHQF\ DQG SUHGLFWDELOLW\
DFURVVGLIIHUHQWSURMHFWVDQGVRIWZDUHSODWIRUPV>@ ) %(3±%,0([HFXWLRQ3ODQ
7R VXSSRUW %,0 LQWHURSHUDELOLW\ EHWZHHQ KXQGUHGV RI 7KH%(3LVGHILQHGDVDSODQSUHSDUHGE\FRQWUDFWRUVWR
VRIWZDUH DSSOLFDWLRQV LQGXVWULDO GRPDLQV DQG DUHDV WKH H[SODLQ ZKDW WKH EXLOGLQJ LQIRUPDWLRQ PRGHOOLQJ DVSHFWV
,)& VFKHPH LV GHVLJQHG WR VXLW PDQ\ FRQILJXUDWLRQV DQG RI WKH SURMHFW ZLOO ORRN OLNH DQG KRZ WKH\ ZLOO EH
OHYHOVRIGHWDLO%HFDXVH,)&LVDYHQGRURUPDQXIDFWXUH LPSOHPHQWHG7KHSODQ LV GHYHORSHGEHIRUH DQG DIWHU WKH
LQGHSHQGHQW VFKHPD 09'V IRFXV RQ GDWD UDWKHU WKDQ FRQFOXVLRQ RI WKH FRQWUDFW DQG LV SUHSDUHG DV D GLUHFW
DSSOLFDWLRQV :RUNIORZ DQG LQIRUPDWLRQ H[FKDQJH UHVSRQVHWRWKHLQIRUPDWLRQUHTXLUHPHQWVRIWKHHPSOR\HU
UHTXLUHPHQWV EHWZHHQ HQG XVHUVGHWHUPLQH WKH VFRSHDQG 7KH%(3H[SODLQVLQGHWDLOWKHUHVXOWVRIWKHSURMHFWVHW
IRUP RI PRGHO GLVSOD\ QRW VSHFLILF VRIWZDUH FDSDELOLWLHV RXW LQ WKH FRQWUDFW DQG WKH LQIRUPDWLRQ H[FKDQJH
RUDSSOLFDWLRQOLPLWV>@ UHTXLUHPHQWV ZKLFK DUHGHWDLOHGLQ WKH%,0SURWRFRO ,Q
,QGLYLGXDO SURMHFW SDUWLFLSDQWV FUHDWH DQG XVH GDWD RQ WKH WHQGHU VWDJH EHIRUH WKH FRQWUDFW LV FRQFOXGHG  WKH
EXLOGLQJV VKRZ YDU\LQJ GHJUHHV RI UHVSRQVLELOLW\ SRWHQWLDO VXSSOLHU ZLOO GHYHORS D %(3 LQ RUGHU WR
H[SHUWLVH DQG QHHGV $OO LQIRUPDWLRQ LV VKDUHG EHWZHHQ GHPRQVWUDWH LWV SURSRVHG DSSURDFK FDSDELOLW\ FDSDFLW\
LQGLYLGXDOPHPEHUVRIWKHSURMHFWWHDPDQGLVXVHGE\VDQ DQG DELOLW\ WR PHHW WKH LQYHVWRU V UHTXLUHPHQWV $IWHU
DQGSHUIRUPLQJYDULRXVWDVNV,WLVLPSRUWDQWWRGHWHUPLQH SODFLQJWKHRUGHUWKHZLQQLQJVXSSOLHUZLOOVXEPLWDQRWKHU
ZKLFKVXEVHWVRIDOOGDWDDQGWKHLUIRUPDWDUHQHHGHGIRU %(3SODQ7KLVGRFXPHQW FUHDWHGDIWHUWKHFRQFOXVLRQRI
H[FKDQJH DQG XVH 6XFK VXEVHWV FDQ EH GHILQHG DV WKH WKHFRQWUDFW SUHVHQWVLQIRUPDWLRQRQWKHVXSSO\FKDLQDQG
DQDO\VLV RI WKH RYHUDOO ,)& VFKHPH LQWR VPDOOHU PRGHO LQIRUPDWLRQ SURYLVLRQ SODQ 0,'3  0,'3 SURYLGHV
YLHZV ZKHUH HQGXVHU UHTXLUHPHQWV IRU WKH QHFHVVDU\ LQIRUPDWLRQRQZKHQSURMHFWLQIRUPDWLRQZLOOEHDYDLODEOH
LQIRUPDWLRQ DUH VSHFLILHG 3URMHFW GHOLYHU\ FRQWUDFWV PD\ ZKRLVUHVSRQVLEOHIRULWVSUHSDUDWLRQDQGZKDWSURWRFROV
UHIHU WR H[FKDQJH VSHFLILFDWLRQV GHOLYHU\ GDWD DQGSURFHGXUHVZLOOEHXVHGWRREWDLQLW>@
UHSUHVHQWDWLRQV UHODWLRQVKLSV FRQFHSWV DQG WKH OLNH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 458

* %,0/LEUDU\ $ 5HVHDUFKDLPGDWDFROOHFWLRQDQGUHVHDUFKVDPSOH
,QGLYLGXDO FUHDWRUV RI VRIWZDUH DSSOLFDWLRQV 7KH UHVHDUFK IRFXVHG RQ WKH UHYLHZ DQG LQWHU
PDQXIDFWXUHUVRIPDWHULDOVDQGVWUXFWXUHVDUHLQFUHDVLQJO\ FRPSDULVRQ RI WKH EDVLF EXLOGLQJ LQIRUPDWLRQ PRGHOOLQJ
FUHDWLQJWKHVRFDOOHG%,0/LEUDU\2QHRIWKHEHVWNQRZQ WRROV QDPHO\ '(  &RPPRQ 'DWD (QYLURQPHQW ,)& 
%,0/LEUDULHVLVWKH1DWLRQDO%,0/LEUDU\ZKLFKLVXVHG ,QGXVWU\ )RXQGDWLRQ &ODVVHV 09'  0RGHO 9LHZ
PDLQO\LQWKH8.7KH1DWLRQDO%,0/LEUDU\LVWKHIDVWHVW 'HILQLWLRQ %&) IRUPDW  %,0 &ROODERUDWLRQ )RUPDW
JURZLQJILUVW%,0OLEUDU\ZLWKDQH[WHQVLYHFROOHFWLRQRI EXLOGLQJ60$57'DWD 'LFWLRQDU\ %(3  %,0 ([HFXWLRQ
%,0 REMHFWV $OO REMHFWV PHHW WKH 1%6 %,0 2EMHFW 3ODQ DQG%,0 /LEUDU\%DVHG RQDULJRURXV DQDO\VLV WKH
6WDQGDUG>@ PDLQ DGYDQWDJHV DQG GLVDGYDQWDJHV RI WKH DERYH
2WKHU %,0 /LEUDULHV DUH IRU H[DPSOH %,0 2EMHFW PHQWLRQHG%,0WRROVZHUHGHILQHG
&ORXG5HYLW&LW\$5&$7DQGRWKHUV%,02EMHFW&ORXG 7KHUHVHDUFKZDVFRPSOHPHQWHGE\DFRPSDULVRQRIWKH
LV D XQLTXH ZD\ IRU PDQXIDFWXUHUV RI FRQVWUXFWLRQ DERYH PHQWLRQHG WRROV LQ WHUPV RI VHOHFWHG DVSHFWV DQG
SURGXFWV WR UHDFK WKHLU FXVWRPHUV %,0 2EMHFW &ORXG LV FULWHULDZKLFKZHUHGHWHUPLQHGRQWKHEDVLVRIDQH[SHUW
WKHODUJHVWSODWIRUPIRUGLJLWDOFRQVWUXFWLRQSURGXFWVLQWKH GLVFXVVLRQ ZLWK VHOHFWHG SDUWLFLSDQWV LQ WKH FRQVWUXFWLRQ
ZRUOGDQGLVDVRXUFHRILQIRUPDWLRQLQIURQWRILQGLYLGXDO SURFHVVHV 7KH LQGLYLGXDO UHVSRQGHQWV ZHUH DVNHG WR
SDUWLFLSDQWV LQ WKH FRQVWUXFWLRQ SURFHVV 7KH VWUXFWXUH RI GHILQH WKH UDQNLQJ RI WKH FULWHULD RU WR GHILQH WKH
WKH FDWDORJXH LV LQ WKH IRUP RI D VHDUFK HQJLQH WKDW LPSRUWDQFH RI WKH GLIIHUHQW DVSHFWVDGYDQWDJHV UHVXOWLQJ
RSWLPL]HV HDFK SURGXFW  HDFK HOHPHQW FRQWDLQV D IURP WKH XVH RI WKH VHOHFWHG WRROV   H[SHUWV IURP WKH
SHUPDQHQW OLQN DQG D XQLTXH 85/ 7KH SRUWDO FRQWDLQV ILHOG RI FRQVWUXFWLRQ SUHSDUDWLRQ DQG H[HFXWLRQ ZHUH
RQO\ SURGXFWV IURP SURGXFWV LW GRHV QRW SXEOLVK JHQHULF LQWHUYLHZHG 5HVSRQGHQWV GHILQHG WKH LPSRUWDQFH
FRPSRQHQWV RU FRPSRQHQWV WKDW DUH QR ORQJHU UDQNLQJ RIWKHIROORZLQJDVSHFWV
PDQXIDFWXUHG>@
5HYLW &LW\ LV D FRPPXQLW\ ZHEVLWH GHGLFDWHG WR • FRRUGLQDWHG LQIRUPDWLRQ VKDULQJ VWDQGDUGLVHG
SURYLGLQJ D VSDFH IRU RQOLQH VKDULQJ DQG GLVFXVVLRQ LQ LQIRUPDWLRQ
IRUXPV IRU 5HYLW XVHUV DURXQG WKH ZRUOG 5HYLW &LW\ KDV • FRVWDQGWLPHVDYLQJV
EHFRPH D FHQWUDO VLWH WKDW FRQQHFWV DFWLYH 5HYLW XVHUV
%,0 UHVRXUFHV DUH SXEOLF DQG FDQ EH DQDO\VHG E\ WKH • VLPSOLI\LQJFRPPXQLFDWLRQDGQGDWDH[FKDQJH
FRPPXQLW\ 5HYLW FLW\ LV D KROLVWLF VLWH WKDW DOORZV • XVHRIDFHQWUDOPRGHOXQLYHUVDOPDQXIDFWXUHUV
FRPPXQLW\XVHUVWRVKDUHWKHLUNQRZOHGJHDQGH[SHULHQFH • IUHHVHUYLFH
ZLWK PD[LPXP SULYDF\ DQG VHFXULW\ 8VHUV FDQ DVVLJQ
• LQWHJUDWLRQZLWKWKHZHEVLWH
SRLQWV WR LQGLYLGXDO LWHPV DQG VKDUH WKHP LQ WKH
IRUHJURXQG 7KH UDQJH RI SRLQWV UDQJHV IURP  WR  • XVH LQ DOO SKDVHV RI WKH SURMHFW GHVLJQ  SURFXUHPHQW
SRLQWV>@ FRQVWUXFWLRQRSHUDWLRQ 

,,, 35,0$5<5(6($5&+',6&866,21$1'5(68/76 %DVHG RQ WKH DERYH UDQNLQJ RI FULWHULD ZHLJKWV ZHUH
DVVLJQHGWRHDFKDVSHFW6XEVHTXHQWO\WKHLQGLYLGXDOWRROV
%XLOGLQJ ,QIRUPDWLRQ 0RGHOOLQJ RIIHU FRQVWUXFWLRQ ZHUHDVVHVVHGLQWHUPVRIWKHVHOHFWHGDVSHFWV VFRUHV DQG
WHDPV DQG FRQVXOWDQWV WKH RSSRUWXQLW\ WR PD[LPL]H WLPH D UDQNLQJ ZDV GHWHUPLQHG LH WKH RSWLPDO WRRO ZDV
DQGPRQH\DQGUHGXFHDQ\LQFRQVLVWHQFLHVLQWKHSURMHFW GHILQHGDFFRUGLQJWRWKHVHOHFWHGFULWHULD
%,0PDNHVLWSRVVLEOHWRVLPXODWHWKHYLUWXDOGHYHORSPHQW
RI D SURMHFW LQ WKH SUHFRQVWUXFWLRQ SKDVH ZKLFK ZLOO % $GYDQWDJHVDQGGLVDGYDQWDJHVRI%,0WRROV
HOLPLQDWH PRVW RI WKH SUREOHPV LQHIILFLHQFLHV DQG 7KHGHFLVLYHFULWHULRQLQVHOHFWLQJWKHWRROVFKRVHQIRU
XQHFRQRPLFDO DVSHFWV WKDW FRXOG RFFXU LQ WKH IUDPHZRUN \RXUSURFHVVHVDUHWKHDGYDQWDJHVDQGGLVDGYDQWDJHVWKDW
RI WKH EXLOW SURFHGXUH 0DQ\ VWXGLHV KDYH DQDO\VHG WKH GLUHFWO\ UHVXOW IURP WKHLU XVH ,PSURYHG FRPPXQLFDWLRQ
LPSDFWVRI%,0RQFRQVWUXFWLRQSURFHVVHVDQGUHVXOWLQWKH DQG WHDPZRUN VXSSRUW DQG LPSURYHG DQDO\VLV FKDQJH
IROORZLQJEHQHILWV PRQLWRULQJ SURMHFW YLVXDOLVDWLRQ FRQIOLFW GHWHFWLRQ DQG
RWKHU EHQHILWV DUH LPSRUWDQW VHOHFWLRQ FULWHULD 7KHVH
• LPSURYLQJFRPPXQLFDWLRQDQGWHDPZRUN EHQHILWV DUH D GLUHFW FRQVHTXHQFH RI WKH XVH RI EXLOGLQJ
• WKHSRVVLELOLW\RIFRVWDQDO\VLVEDVHGRQWKHPRGHO LQIRUPDWLRQ PRGHOOLQJ WRROV 7DEOH , DQG 7DEOH ,,
• PRQLWRULQJ RI FKDQJHV ZLWKLQ WKH OLIH F\FOH RI WKH VXPPDUL]H WKH DGYDQWDJHV DQG SRVVLEOH GLVDGYDQWDJHV RI
LQGLYLGXDO%XLOGLQJ,QIRUPDWLRQ0RGHOOLQJWRROV
EXLOGLQJ

• SURMHFWYLVXDOL]DWLRQGXULQJWKHSODQQLQJSKDVH

• HIIHFWLYHFRQIOLFWGHWHFWLRQDQGFRRSHUDWLRQVXSSRUW

• FRVWUHGXFWLRQDQGULVNHOLPLQDWLRQ

• LQFUHDVLQJ WKH HIILFLHQF\ RI SODQQLQJ DQG PDQDJHPHQW

SURFHVVHV

• LPSURYLQJODERXUDQGSURGXFWLRQSURGXFWLYLW\

• UHGXFWLRQRIWLPH
• TXDOLW\LPSURYHPHQW>@ 

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 459

7$%/(,722/62)%8,/',1*,1)250$7,2102'(//,1*$6800$5< 7$%/(,,722/62)%8,/',1*,1)250$7,2102'(//,1*$
2)$'9$17$*(6$1'',6$'9$17$*(6727+(,586(±3$57 6800$5<2)$'9$17$*(6$1'',6$'9$17$*(6727+(,586(±3$57
%,0

%,0
7RRO
WRRO

$GYDQWDJHVRIWKHWRRO 'LVDGYDQWDJHVRIWKHWRRO $GYDQWDJHVRIWKHWRRO 'LVDGYDQWDJHVRIWKHWRRO

WKHUHVXOWRIWKHVKDUHGLQIRUPDWLRQ WKHSRVVLELOLW\RIWUDQVIHUULQJDILOH
VKRXOGEHFRRUGLQDWHGGDWDZKLFKLQ WKHQHHGWRDSSRLQWDQ
LQ%&)IRUPDW EFY]LS LVEHWZHHQ
WXUQUHGXFHVWKHWLPHDQGFRVWRI\RXU LQIRUPDWLRQPDQDJHUZKR
WKHLQGLYLGXDOSDUWLFLSDQWV
LVUHVSRQVLEOHIRUPDLQWDLQLQJ
&'(&RPPRQ'DWH(QYLURQPHQW

SURMHFW ,QGLYLGXDOSDUWLFLSDQWVFDQPDNH
WKHP\ULDGRILQIRUPDWLRQ
SURMHFWWHDPPHPEHUVFDQXVH&'(V FKDQJHVHGLWDQGWKHQVHQGILOHVWR
WKDWLVJHQHUDWHGDQGVKDUHG
DQGJHQHUDWHWKHGRFXPHQWVWKH\QHHG RWKHUXVHUV
LQDV\QFKURQL]HGDQG
XVLQJYDULRXVFRPELQDWLRQVRIFHQWUDO LPSURYLQJFRPPXQLFDWLRQDQG

%&)%,0&ROODERUDWLRQ)RUPDW
FRKHUHQWPDQQHU
DVVHWV DFFXUDWHGDWDODWHVWDVVHWV  LPSURYLQJRSHQ%,0SURFHVVHV
FUHDWLRQRIDFHQWUDOPRGHOVXSSRUWRI HDVLHULGHQWLILFDWLRQDQGH[FKDQJH
VSDWLDOFRRUGLQDWLRQ RIPRGHOVEHWZHHQ%,0VRIWZDUH
&'(XSGDWHVDUHSHUIRUPHGLQUHDO WRROVE\SDVVLQJFXVWRPIRUPDWVDQG
WLPHZKLFKUHGXFHVWKHWLPHDQG ZRUNIORZV
WKHQHHGWRFUHDWHDQDFFXUDWH WKHILOHFDQRQO\EHXSORDGHGWR
HIIRUWUHTXLUHGWRFURVVFKHFNWKH XVHLQDOOSKDVHVRIWKHSURMHFW DSODWIRUPWKDWVXSSRUWVWKH
GDWDVWUXFWXUHDQGGHILQHWKH
FXUUHQWVWDWXVRISURMHFW GHVLJQSKDVH GRFXPHQWDWLRQ %&)IRUPDW
GDWDLQSXWIRUPDW
UHGXFLQJVWDUWXSRSHUDWLQJDQG LGHQWLILFDWLRQRISUREOHPVZLWK
SURMHFWFRVWVE\LQFUHDVLQJWKH GHVLJQFRRUGLQDWLRQGHVLJQQRWHV
UHVRXUFHVDQGSURFHVVHVWKDWQHHGWREH VXEVWLWXWLRQRIREMHFWVDQGVHOHFWLRQ
VHWXSDQGLPSOHPHQWHG RIPDWHULDOV SURFXUHPHQWSKDVH
WKHW\SHRIVXSSRUWIRU LWHPVRIFRRUGLQDWLRQRIWHQGHUV
DOOVRIWZDUHDSSOLFDWLRQVWKDWVXSSRUW VRIWZDUHDSSOLFDWLRQVYDULHV FRVWLQIRUPDWLRQ FRQVWUXFWLRQ
09'0RGHO9LHZ

WKHH[SRUWRI%,0GDWDYLD,)&DOVR GHSHQGLQJRQWKH%,0XVHU SKDVH VHFXULW\UHFRUGVTXDOLW\

KDYH09'VXSSRUW LQWHUIDFHRIWKHVRIWZDUH FRQWUROPRQLWRULQJDYDLODELOLW\RI
'HILQLWLRQ

DSSOLFDWLRQ LWHPVDQGPDWHULDOVFROOHFWLRQRI
QHHGWRFKDQJHFRQFHSW LQIRUPDWLRQ RSHUDWLRQSKDVH QRWHV
LQIRUPDWLRQLVVWDQGDUGL]HG RQWKHKDQGRYHUGRFXPHQWQHHGIRU
GHILQLWLRQRIH[FKDQJH
HOLPLQDWLRQRIUHSHDWHGGLVFXVVLRQV XSGDWHVHWF 
UHTXLUHPHQWVWKURXJK
DERXWWKHH[FKDQJHDQGPHWKRGRI WKHQHHGWRGHILQHSUHFLVH
LQIRUPDWLRQSURYLVLRQLQJ FRPPXQLFDWLRQVXSSRUWIRU
LQIRUPDWLRQH[FKDQJH REMHFWLYHVWKHQHHGWRWDNHLQWR
VSHFLILFDWLRQ ,'6  LPPHGLDWHFRPPXQLFDWLRQEHWZHHQ
LQWHURSHUDELOLW\LVVXHVHDFK DFFRXQWWKHFDSDFLW\RIWKH
WHDPVDQGLQGLYLGXDOSDUWLFLSDQWV
VRIWZDUHDSSOLFDWLRQKDV SURMHFWLPSOHPHQWDWLRQWHDP
VXSSRUWIRUFRRSHUDWLRQEHWZHHQWKH LWLVQHFHVVDU\WRGHILQHWKH
GLIIHUHQWLQWHURSHUDELOLW\
YDULRXVSDUWLFLSDQWVLQWKHSURMHFW VFRSHRI%,0LPSOHPHQWDWLRQ
LVVXHVQHHGIRUIXUWKHU
%(3%,0([HFXWLRQ3ODQ

GHYHORSPHQW FRRSHUDWLRQVXSSRUWWKHSURMHFW ZRUNZKDWREMHFWVDUH

WHDPKDVWKHRSSRUWXQLW\WR PRGHOOHGLQZKLFKSKDVHOHYHO
,)&,QGXVWU\)RXQGDWLRQ&ODVVHV

WKHQHHGIRUVWDELOL]DWLRQRI
H[FKDQJHRIGDWDDQGLQIRUPDWLRQ HQWLWLHVH[WHQVLRQRIWKH FROODERUDWHLQUHDOWLPHRQYDULRXV RIGHWDLOUHVSRQVLEOHHPSOR\HH
WKURXJKDVWDQGDUGIRUPDW FDSDELOLW\RIWKHPRGHO PDLQ SKDVHVRIWKHSURMHFW ZKDWLQIRUPDWLRQQHHGVWREH
GRPDLQVRI$(&  SURYLGHGGXULQJLPSOHPHQWDWLRQ
WKHQHHGWRLQFUHDVHVXSSRUW DQGVHWWKHH[DFWGDWH
TXDOLW\LPSURYHPHQW IRUGHVLJQSODQQLQJFRVWV VDYHWLPHWKHSODQGHWDLOVDQG
VFKHGXOHV GHWDLOVWKHPRVWLPSRUWDQWGHWDLOV
DWHDPRIH[SHUWVLVQHHGHGWR
WKHQHHGWRGHYHORS WKDWKHOSFUHDWHDQGPDLQWDLQD
FUHDWH%(3
LQWHUQDWLRQDOVWDQGDUGVWR GHOLYHU\VFKHGXOHWKURXJKRXWWKH
VXSSRUWWKHHIILFLHQF\RIGDWD SURMHFW
WUDQVPLVVLRQEHWZHHQWKH WKHQHHGIRUXSGDWLQJD
YDULRXVSKDVHVRIWKHSURMHFW EXLOGLQJSURMHFWLVDOLYLQJ
UHGXFLQJHUURUVFRVWVDQGVDYLQJWLPH HDV\VKDULQJRILQIRUPDWLRQEHWZHHQ
WKHUHE\LQFUHDVLQJVXSSRUWIRU HYHUFKDQJLQJFUHDWLRQ7KH
LQGLYLGXDOSURMHFWSDUWLFLSDQWV
FRRSHUDWLRQFRPPXQLFDWLRQ GRFXPHQWPXVWEHFRQVWDQWO\
DXWRPDWLRQDQGLQWHJUDWLRQRI XSGDWHG
GHVLJQSODQWFRQVWUXFWLRQDQG
LQFUHDVLQJWKHHIILFLHQF\ LQGLYLGXDOREMHFWVPHHWWKH
KLJKTXDOLW\RILQIRUPDWLRQ%,0
SRVVLEOHSUREOHPVZLWK VWDQGDUGVDQGVWDQGDUGVGHILQHG
FRQWURORILQGLYLGXDOGDWDIURPYDULRXV REMHFWVDUHDXWKRUL]HGDFFRUGLQJWR
JDLQLQJDFFHVVWRE6''GDWD LQWKH8.GRQRWWDNHLQWR
EXLOGLQJ60$57'DWD

FRQVWUXFWLRQSDUWLFLSDQWV WKH1%6VWDQGDUGDQGKDYHKLJK
WKHQHHGWRDVNWKHKDUGZDUH DFFRXQWGDWDDWWKHQDWLRQDO
TXDOLW\UHOHYDQWDQGYDOXDEOH
DXWRPDWLFVHDUFKIRUSURGXFWVDQG YHQGRUIRUDSSURYDODQG OHYHO±HJ6ORYDNVWDQGDUGV
'LFWLRQDU\

LQIRUPDWLRQ
PDQXIDFWXUHUV VXSSRUW DQGQRUPV
%,0/LEUDU\

H[WHQVLRQRI,)&PRGHOV
WRHPEHG\RXUVWDQGDUGV\RX VWLOOPLVVLQJDQXPEHURI
FUHDWLQJLQIRUPDWLRQ SRVVLELOLW\RILQWHJUDWLRQZLWKWKH
QHHGWRFRQWDFWFHUWLILHG REMHFWVHVSHFLDOO\LQQRYDWLYH
VSHFLILFDWLRQV ZHEVLWH
DJHQWVE6'' WHFKQRORJLHV
WKHVHUYLFHLVIUHH WKHOLEUDU\RIIHUVJHQHUDOREMHFWV LQGLYLGXDOVRIWZDUH
WKDWXVHUVZLOOXVHLQLWLDOO\DQGWKHQ PDQXIDFWXUHUVDQG
 UHSODFHWKHPZLWKVSHFLILFSURGXFWV PDQXIDFWXUHUVRIPDWHULDOVDQG
 DOORZVIRUPD[LPXPH[SRVXUH WHFKQRORJLHVJXDUGWKHLU%,0
DQGKHOSVLQFUHDVH PRGHOVDQGRIIHUWKHPRQO\
 ZLWKLQWKHIUDPHZRUNRIWKHLU
FRPSHWLWLYHQHVV
 FRQWUDFWXDOUHODWLRQV
 

& ,QWHUFRPSDULVRQRIVHOHFWHG%,0WRROV

 :KHQ FRPSDULQJ WKH DERYH WRROV ZLWK HDFK RWKHU 
 UHVSRQGHQWVZHUHFRQWDFWHG7KHLQWHUYLHZHHVUHSUHVHQWHG
SDUWLFLSDQWV LQ FRQVWUXFWLRQ SURFHVVHV VSHFLILFDOO\ WKRVH

LQYROYHG LQ WKH SUHSDUDWLRQ DQG LPSOHPHQWDWLRQ RI
 FRQVWUXFWLRQV 5HVSRQGHQWV DVVLJQHG D UDQNLQJ WR HDFK
 DVSHFW ZLWK SRVLWLRQ QXPEHU  EHORQJLQJ WR WKH DVSHFW
WKDW ZDV WKH PRVW LPSRUWDQW FULWHULRQ DFFRUGLQJ WR WKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 460

UHVSRQGHQWVDQGSRVLWLRQQXPEHUWRWKHDVSHFWWKDWZDV ,9 &21&/86,21
WKH OHDVW LPSRUWDQW %DVHG RQ WKH DERYH D UDQNLQJ ZDV %XLOGLQJ,QIRUPDWLRQ0RGHOOLQJLVDZRUNSURFHVVWKDW
GHWHUPLQHG DQG SRLQWV ZHUH DVVLJQHG WR HDFK DVSHFW KHOSVLQGLYLGXDOSDUWLFLSDQWVLQWKHFRQVWUXFWLRQSURFHVVWR
ZKHUH WKH VW DQG WK SRVLWLRQV ZHUH DVVLJQHG   DQG  HIIHFWLYHO\ FUHDWH DQG VKDUH LQIRUPDWLRQ DFURVV WKH OLIH
SRLQWV UHVSHFWLYHO\ %DVHG RQ WKH SRLQWV DVVLJQHG WKH F\FOH %XLOGLQJ ,QIRUPDWLRQ 0RGHOOLQJ IRU WKH
ZHLJKWVRIHDFKFULWHULRQZHUHGHWHUPLQHG6XEVHTXHQWO\ FRQVWUXFWLRQ LQGXVWU\ UHSUHVHQWV D WUDQVLWLRQ WRZDUGV WKH
WKH LQGLYLGXDO LQVWUXPHQWV ZHUH FRPSDUHG LQ WKH DERYH HUD RIGLJLWL]DWLRQ7KLV SURFHVV LV VWLOORQO\ LQ WKH LQLWLDO
PHQWLRQHG UHVSHFWV ,I WKH LQVWUXPHQW PHW WKH DERYH VWDJH RI LPSOHPHQWDWLRQ LQ WKH WHUULWRU\ RI WKH 6ORYDN
FULWHULRQ LW ZDV DVVLJQHG WKH PD[LPXP QXPEHU RI SRLQWV 5HSXEOLF GHVSLWH WKHHIIRUWVRI WKH PHPEHUVRI WKH %,0
SRLQWV RWKHUZLVHLWZDVDVVLJQHGSRLQWV2QWKHEDVLV $VVRFLDWLRQ 6ORYDNLD  EXW GHYHORSPHQWV DEURDG FOHDUO\
RI WKH DERYH VFRUHV DQG DIWHU WDNLQJ LQWR DFFRXQW WKH VKRZWKDWERWKWKHSXEOLFDQGSULYDWHVHFWRUVDUHJUDGXDOO\
FULWHULDZHLJKWVWKHLQGLYLGXDOLQVWUXPHQWVZHUHDVVLJQHG PRYLQJ WRZDUGV XVLQJ WKLV PHWKRG 7KH SDSHU EULHIO\
SRLQWVDQGDUDQNLQJZDVGHWHUPLQHGZLWKVWSODFHJRLQJ GHILQHG WKH PRVW XVHG LQIRUPDWLRQ WRROV IRU %XLOGLQJ
WR WKH LQVWUXPHQW ZLWK WKH KLJKHVW VFRUH %DVHG RQ WKH ,QIRUPDWLRQ 0RGHOOLQJ ZKLFK LV SDUW RI PDQ\ %,0
DERYH UDQNLQJ WKH RSWLPDO %,0 WRRO DFFRUGLQJ WR XVHU VRIWZDUH DSSOLFDWLRQV 7KHVH WRROV KHOS LQGLYLGXDO
SUHIHUHQFHV LV &'(  &RPPRQ 'DWD (QYLURQPHQW  SDUWLFLSDQWV LQ WKH FRQVWUXFWLRQ SURFHVV WR DQDO\]H SODQ
SRLQWV  IROORZHG E\ %&)  %,0 &ROODERUDWLRQ )RUPDW DQGPDQDJHWKHHQWLUHOLIHF\FOHRIDEXLOGLQJ$WSUHVHQW
 SRLQWV  DQG LQ WKLUG SODFH DUH ,)&  ,QGXVWU\ KRZHYHU WKHUH LV QR XQLIRUP HQYLURQPHQW IRU SODQQLQJ
)RXQGDWLRQ&ODVVHVDQG%(3%,0([HFWXLRQ3ODQ ERWK DQG PDQDJLQJ JUDSKLFDO DQG VHOHFWHG QRQJUDSKLFDO
SRLQWV 7KHGHWDLOHGFRPSDULVRQDQGILQDOUDQNLQJLV LQIRUPDWLRQ$OOWKHVHDVSHFWVZLOOEHWDNHQLQWRDFFRXQWLQ
VKRZQ LQ WKH IROORZLQJ 7DEOH ,,, ,QWHUFRPSDULVRQ RI WKHQHZFRQVWUXFWLRQODZWKHILQDOYHUVLRQRI ZKLFK ZLOO
VHOHFWHG%,0WRROV HQWHU LQWR IRUFH LQ  0DUFK  %\ WKLV GDWH LW LV
QHFHVVDU\WRFUHDWHDWULDOEHWDYHUVLRQRIWKHLQIRUPDWLRQ
7$%/(,,,,17(5&203$5,6212)6(/(&7('%,0722/6 V\VWHP ,6 85%,21 GHDGOLQH E\ $SULO   ,6
 &ULWHULRQ 85%,21 GHDGOLQHE\-DQXDU\ RU,685%,21
6XPPDU\HYDOXDWLRQUDQNLQJ
VKDULQJVWDQGDUGLVHGLQIRUPDWLRQ

 GHDGOLQHXQWLO-DQXDU\ 
GHVLJQSURFXUHPHQWFRQVWUXFWLRQ
DXWRPDWLFVHDUFKIRUSURGXFWVDQG
XVHRIDFHQWUDOPRGHOXQLYHUVDO
VLPSOLI\LQJFRPPXQLFDWLRQDQG

XVHLQDOOSKDVHVRIWKHSURMHFW

6XPPDU\HYDOXDWLRQ
LQWHJUDWLRQZLWKWKHZHEVLWH
FRRUGLQDWHGLQIRUPDWLRQ

FRVWDQGWLPHVDYLQJV

$&.12:/('*0(17
GDWDH[FKDQJH

IUHHVHUYLFH
GDWD PRGHO

7KH SDSHU SUHVHQWV SDUWLDO UHVHDUFK UHVXOWV RI SURMHFW

RSHUDWLRQ
I W

$399 ³5HVHDUFK RI GLJLWDO WHFKQRORJLHV DQG

EXLOGLQJ LQIRUPDWLRQ PRGHOLQJ WRROV IRU GHVLJQLQJ DQG
%,0WRROV

HYDOXDWLQJ WKH VXVWDLQDELOLW\ SDUDPHWHUV RI EXLOGLQJ

VWUXFWXUHV LQ WKH FRQWH[W RI GHFDUERQL]DWLRQ DQG FLUFXODU
FRQVWUXFWLRQ´7KLVDUWLFOH ZDV FUHDWHG WKDQNV WR VXSSRUW
:HLJKWRIWKHFULWHULRQ  
XQGHU WKH 2SHUDWLRQDO 3URJUDP ,QWHJUDWHG ,QIUDVWUXFWXUH
(YDOXDWLRQ   IRU WKH SURMHFW 1DWLRQDO LQIUDVWUXFWXUH IRU VXSSRUWLQJ
PD[         WHFKQRORJ\ WUDQVIHU LQ 6ORYDNLD ,, ± 1,77 6. ,, FR
SRLQWV 
ILQDQFHGE\WKH(XURSHDQ5HJLRQDO'HYHORSPHQW)XQG
:HLJKRI
















WKHFULWHULD
PD[ 

&'( 5()(5(1&(6
&RPPRQ


         >@ +(6&21 %,0 ± %XLOGLQJ LQIRUPDWLRQ PRGHOOLQJ >2QOLQH@

'DWH KWWSKHVFRQVNVNELP
(QYLURQPHQW
XYRG"JFOLG &M.&4L$T3%5&/$5,V$%'D=]M(JLN-753L'
09' +&ZF&F&=T.K&LJEB]6\,,D[6PU&&S-U*TD$SJ($/Z


0RGHO9LHZ          BZF%ODVWDFFHVVHG
'HILQLWLRQ
>@ 6ORYDN 6WDWLVWLFV 6ORYDN &RQVWUXFWLRQ <HDUERRN >2QOLQH@ 
,)& KWWSVVORYDNVWDWLVWLFVVNZSVSRUWDOEGGHDDD
,QGXVWU\


         IHFEXWS]U9-18,Z(3W+M\:E'DS0G6V)4/8NU
)RXQGDWLRQ
&ODVVHV \09-6*,E9(T\/]+&4J,]S'-=W\H
-1(0\8UW/1TG)S9[33SI08&J'+<+ZYX(ZH-L(
%&)%,0 %PS'JOT,%\,4MF&/0[5G.N-+ZO2=ME1.'--,P,U<3Y+


&ROODERUDWLRQ         
Z2V[+J*,./$K+US\,S%H.3Y\Z3/M(Q\&-=)PD)Q:
)RUPDW
96WUX<+NWP29U:XLY<E<WPIIK
EXLOGLQJ60$57
<HZ:%5(:<R]E'*0XD9$J/8N&OV-


'DWD         
'LFWLRQDU\ R(IW7D;D+9HQTSI/8TI+\0[ZJ+Q]I$2[[[0/'
%(3%,0 /7R&=(JOZ9=B5%+<-\.-+$.$7'I%QESP17-Y


([HFXWLRQ          []$H&SMWG))DH
3ODQ OV:'\[\$1MTE2\WFB/)U4Z(G[K%'3K(PU25
7II]D\TOO=XOU:D+D9WT+TOF5Q3)97I'=RX


%,0/LEUDU\          P=5SPSD&+T]T/YVG\P7:+Y;Z%-344G]G/G%,
6(Y=)%,6Q46(KODVWDFFHVVHG
1RWH(YDOXDWLRQFULWHULRQ PD[SRLQWVPLQSRLQWV  >@ $872'(6. :KDW LV %,0"  >2QOLQH@
 KWWSVZZZDXWRGHVNFRPVROXWLRQVELPEHQHILWVRIELP ODVW
DFFHVVHG


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 461

 >@ 81,7(' %,0 /HDGLQJ &RXQWULHV :LWK %,0 $GRSWLRQ  KWWSVWHFKQLFDOEXLOGLQJVPDUWRUJVWDQGDUGVEFI ODVW DFFHVVHG
>2QOLQH@   KWWSVZZZXQLWHGELPFRPOHDGLQJFRXQWULHVZLWK 
ELPDGRSWLRQODVWDFFHVVHG >@ EXLOGLQJ60$57 EXLOGLQJ60$57 'DWD 'LFWLRQDU\ 
>@ 0 âSDOHN  ³&'( &RPPRQ 'DWD (QYLURQPHQW´  >2QOLQH@
>2QOLQH@    KWWSVZZZW]ELQIRF]ELPLQIRUPDFQLPRGHO KWWSVZZZEXLOGLQJVPDUWRUJXVHUVVHUYLFHVEXLOGLQJVPDUWGDWD
EXGRY\FGHFRPPRQGDWDHQYLURQPHQW ODVW DFFHVVHG GLFWLRQDU\ODVWDFFHVVHG
 >@ 1%6 :KDW LV D %,0 ([HFXWLRQ 3ODQ %(3 "  >2QOLQH@
>@ 5 0F3DUWODQG  ³:KDW LV WKH &RPPRQ 'DWD (QYLURQPHQW KWWSVZZZWKHQEVFRPNQRZOHGJHZKDWLVDELPH[HFXWLRQ
&'( "´  >2QOLQH@ SODQEHSODVWDFFHVVHG
KWWSVZZZWKHQEVFRPNQRZOHGJHZKDWLVWKHFRPPRQGDWD >@ 1%6  :KDW LV 1%6 1DWLRQDO %,0 /LEUDU\"  >2QOLQH@
HQYLURQPHQWFGHODVWDFFHVVHG KWWSVZZZQDWLRQDOELPOLEUDU\FRPHQODVWDFFHVVHG
>@ EXLOGLQJ60$57 ,QGXVWU\ )RXQGDWLRQ &ODVVHV ,)&   >@ %,0REMHFW 'HVLJQ FRQQHFW DQG VXSSO\ %,0 REMHFWV 
>2QOLQH@    KWWSVZZZEXLOGLQJVPDUWRUJVWDQGDUGVEVL >2QOLQH@   KWWSVLQIRELPREMHFWFRPVROXWLRQVELPREMHFWFORXG
VWDQGDUGVLQGXVWU\IRXQGDWLRQFODVVHVODVWDFFHVVHG ODVWDFFHVVHG
>@ EXLOGLQJ60$57 0RGHO 9LHZ 'HILQLWLRQ 09'  ± $Q >@  4XDGUDQWV 5HYLW &LW\ 3ULFLQJ  'HPR  >2QOLQH@
,QWURGXFWLRQ  KWWSVZZZTXDGUDQWVFRPVRIWZDUHELPREMHFWVUHYLWFLW\
KWWSVWHFKQLFDOEXLOGLQJVPDUWRUJVWDQGDUGVLIFPYG ODVW ODVWDFFHVVHG
DFFHVVHG
>@ &RQVWUXFWLRQ7XWV   $GYDQWHJV RI %,0 &RQVWUXFWLRQ ±
>@ EXLOGLQJ60$57 %,0 &ROODERUDWLRQ )RUPDW %&)  ± $Q %XLOGLQJ ,QIRUPDWLRQ 0RGHOOLQJ %HQHILWV  >2QOLQH@
,QWURGXFWLRQ  >2QOLQH@ KWWSVZZZFRQVWUXFWLRQWXWVFRPDGYDQWDJHVRIELPLQ
FRQVWUXFWLRQODVWDFFHVVHG


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 462

 9LUWXDO(GXFDWLRQDO&ROODERUDWLYH(QYLURQPHQWV
IRU/RZ&RVW0RELOH95+HDGVHWV

%6RERWD 7+XOLQD â.RUHþNR 00DWWRYi 
'HSDUWPHQWRI&RPSXWHUVDQG,QIRUPDWLFV)DFXOW\RI(OHFWULFDO(QJLQHHULQJDQG,QIRUPDWLFV7HFKQLFDO8QLYHUVLW\RI
.RãLFH6ORYDN5HSXEOLF
EUDQLVODYVRERWD#WXNHVNVWHIDQNRUHFNR#WXNHVN

$EVWUDFW²8WLOL]DWLRQRIYLUWXDOUHDOLW\ 95 WHFKQRORJLHVLQ
HGXFDWLRQ LV EHFRPLQJ PRUH DQG PRUH FRPPRQ EXW WKHUH ,, 95+($'6(76$1'&21752//(56
DUH VWLOO REVWDFOHV WR GHDO ZLWK 2QH RI WKHP LV D UHODWLYHO\ ,QJHQHUDO95KHDGVHWVFDQEHGLYLGHGLQWRVWDQGDORQH
KLJK FRVW RI 95 KHDGVHWV LPSRUWDQW IRU DQ LPPHUVLYH
DQGSHUVRQDOFRPSXWHU 3& RQHV7KH3&RQHVUHTXLUHDQ
H[SHULHQFH7KHVROXWLRQFDQEHVHHQLQPRELOH95KHDGVHWV
FRQVLVWLQJ RI D UHODWLYHO\ FKHDS KROGHU ZLWK OHQV DQG D
H[WHUQDO FRPSXWHU WR EH FRQQHFWHG ZKLOH WKH VWDQGDORQH
VPDUWSKRQH SURYLGLQJFRPSXWLQJSRZHU,QWKLV SDSHUZH RQHV KDYH D FRPSXWHU LQVLGH $ FKHDSHU NLQG RI WKH
SUHVHQW D ZHEEDVHG HGXFDWLRQDO FRRSHUDWLYH YLUWXDO VWDQGDORQH RQHV DUH PRELOH 95 KHDGVHWV FRQVLVWLQJ RI D
HQYLURQPHQWWDLORUHGIRUPRELOH95KHDGVHWVDQGUHVXOWVRI KHDGVHW ZLWKOHQVHVDVWUDSWRVHFXUHLWWRWKHXVHU¶VKHDG
LWV HYDOXDWLRQ :H DOVR RYHUYLHZ FRQWHPSRUDU\ HGXFDWLRQDO DQG FRQWUROOHUV 7KH\ DOORZ XVHUV WR H[SHULHQFH YLUWXDO
95 DSSOLFDWLRQV DQG SURYLGH D SRLQW RI YLHZ RI ORFDO UHDOLW\ XVLQJ WKHLU VPDUWSKRQHV 0RELOH 95 KHDGVHWV
6ORYDNLD SULPDU\DQGVHFRQGDU\VFKRROWHDFKHUVRQVFKRRO UHTXLUHXVHUVWRLQVHUWWKHLUVPDUWSKRQHLQWRWKHKHDGVHWDQG
VXEMHFWVXLWDEOHIRU95 ODXQFKDFRPSDWLEOHYLUWXDOUHDOLW\DSS7KHDSSWKHQVSOLWV
WKHVPDUWSKRQHVFUHHQLQWR WZRVHSDUDWHLPDJHV RQH IRU
.H\ZRUGV²FROODERUDWLYHYLUWXDOUHDOLW\HGXFDWLRQ PRELOH
95KHDGVHW$)UDPH HDFKH\HZKLFKDUHYLHZHGWKURXJKOHQVHVLQWKHKHDGVHW
WR FUHDWH D VWHUHRVFRSLF HIIHFW DOORZLQJ WKH XVHU WR
SHUFHLYH ' LPDJHV DQG YLGHRV 0RELOH 95 KHDGVHWV DUH
, ,1752'8&7,21 ZLGHO\ DYDLODEOH DQG LQH[SHQVLYH EHFDXVH WKH\ XWLOL]H WKH
8WLOL]DWLRQ RI YLUWXDO UHDOLW\ 95  LQ HGXFDWLRQ KDV D SURFHVVLQJ SRZHU DQG GLVSOD\ RI VPDUWSKRQHV +RZHYHU
SRWHQWLDOWRHQKDQFHOHDUQLQJH[SHULHQFHV:LWKWKHUDSLG WKHLUSUHFLVLRQDQGTXDOLW\RIH[SHULHQFHFDQEHOLPLWHGE\
GHYHORSPHQW RI PRELOH WHFKQRORJLHV DQG FROODERUDWLYH WKH LQWHUQDO VHQVRUV DQG SURFHVVLQJ SRZHU RI WKH
95 LW LV QRZ SRVVLEOH WR FUHDWH D YLUWXDO GLJLWDO VPDUWSKRQH3RSXODUPRELOH95KHDGVHWVLQFOXGH6DPVXQJ
HQYLURQPHQWWKDWHQDEOHVHIIHFWLYHHGXFDWLRQDQGOHDUQLQJ *HDU95*RRJOH'D\GUHDP9LHZDQG*RRJOH&DUGERDUG
IURP DQ\ZKHUH 'HVSLWH SRWHQWLDO EHQHILWV IRU OHDUQLQJ D VLPSOH DQG LQH[SHQVLYH RSWLRQ WKDW XVHV D FDUGERDUG
WKH KLJK FRVW RI 95 HTXLSPHQW UHPDLQV D FKDOOHQJH IRU KROGHUIRUWKHVPDUWSKRQH 7KHVHKHDGVHWVW\SLFDOO\KDYHD
ZLGHVSUHDG XVH LQ VFKRROV +RZHYHU PRELOH 95 ZLWK UDQJH RI FRPSDWLEOH DSSV DQG JDPHV DYDLODEOH WKURXJK
*RRJOH&DUGERDUGOLNH VROXWLRQV FRPELQLQJ VWDQGDUG WKHLUUHVSHFWLYHDSSVWRUHV
VPDUWSKRQHV DQG FKHDS KROGHUV ZLWK OHQV SURYLGH DQ 7KHUH DUH DOVR IXOO\ IHDWXUHG VWDQGDORQH 95 KHDGVHWV
DIIRUGDEOHDOWHUQDWLYHIRUGHGLFDWHG95 KHDGVHWV 2Q WKH VXFK DV WKH 2FXOXV0HWD 4XHVW ZLWK D EXLOWLQ FRPSXWHU
VRIWZDUH VLGH JDPH HQJLQHV VXFK DV 8QLW\ RU 8QUHDO VLPLODU WR VPDUWSKRQHV EXW ZLWK EHWWHU JUDSKLFV WKDQ WKH
(QJLQH DQG ZHE IUDPHZRUNV XWLOL]LQJ :HE;5 'HYLFH PRVW VPDUWSKRQH PRGHOV 7KH\ VXSSRUW PRUH FRPSOH[
$3,DUHVRIWZDUHWRROVWKDWFDQEHXVHGWRFUHDWHPRELOH YLUWXDO HQYLURQPHQWV DQG LQWHUDFWLRQV RIIHU EHWWHU FRQWURO
95HQYLURQPHQWV RYHU PRWLRQ WUDFNLQJ DQG KLJKHU SUHFLVLRQ 2Q WKH RWKHU
7KH FKDOOHQJH QRZ LV WR PDNH WKHVH YLUWXDO KDQG WKH\ DUH VLJQLILFDQWO\ PRUH H[SHQVLYH WKDQ PRELOH
HQYLURQPHQWV PRUH DFFHVVLEOH DQG HDVLHU WR XVH IRU ERWK 95 KHDGVHWV LI QRW FRQVLGHULQJ WKH SULFH RI WKH
VWXGHQWVDQGWHDFKHUVSDUWLFXODUO\GXULQJRQOLQHOHDUQLQJ VPDUWSKRQHLWVHOI 
$FFRUGLQJ WR %XUJHVV DQG 6LHYHUWVHQ >@ DQ 7KH 3& KHDGVHWV VXFK DV +7& 9LYH RU 9DOYH ,QGH[
XQSUHFHGHQWHG WUDQVLWLRQ WR RQOLQH HGXFDWLRQ ZDV UHTXLUH DQ H[WHUQDO KLJKSHUIRUPDQFH 3& WR EH FRQQHFWHG
REVHUYHGDIWHUSDQGHPLFVLWXDWLRQ:LWKWKHGHYHORSPHQW YLDDFDEOHRU:L)L&RPSDUHGWRRWKHURSWLRQVWKH\FDQ
RI FRUUHVSRQGLQJ WHFKQRORJLHV WKH UHTXLUHPHQWV IRU WKLV SURYLGH WKH EHVW H[SHULHQFH 2Q WKH RWKHU KDQG WKH\ DUH
W\SHRIHGXFDWLRQDUHDOVRLQFUHDVLQJ2YHUDOOWKHXVHRI TXLWHH[SHQVLYHHVSHFLDOO\FRQVLGHULQJWKHODFNRIDEXLOG
95 LQ HGXFDWLRQ KDV WKH SRWHQWLDO WR UHYROXWLRQL]H WKH LQ FRPSXWHU %RWK IXOO\ IHDWXUHG VWDQGDORQH DQG 3& 95
ZD\VWXGHQWVOHDUQDQGGHDOZLWKOHDUQLQJPDWHULDOV:LWK KHDGVHWV XVXDOO\ FRPH ZLWK VRSKLVWLFDWHG FRQWUROOHUV DQG
WKH GHYHORSPHQW RI PRUH DIIRUGDEOH DQG DFFHVVLEOH 95 PRWLRQWUDFNLQJIRULQWHUDFWLRQDQGPRYHPHQWLQ95
WHFKQRORJ\ LW LV OLNHO\ WKDW ZH ZLOO VHH HYHQ PRUH %OXHWRRWK FRQWUROOHUV IRU SKRQHV DUH GHYLFHV WKDW DOORZ
ZLGHVSUHDG DGRSWLRQ RI 95 LQ HGXFDWLRQ LQ WKH \HDUV WR XVHUV WR FRQWURO DSSOLFDWLRQV RQ WKHLU SKRQH XVLQJ D
FRPH ZLUHOHVV %OXHWRRWK FRQQHFWLRQ 7KHVH FRQWUROOHUV FDQ EH
XVHGIRUYDULRXVW\SHVRIJDPHVDQGDSSOLFDWLRQVLQFOXGLQJ
WKRVH GHVLJQHG IRU 95 )RU 95 JDPHV DQG DSSOLFDWLRQV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 463

%OXHWRRWK FRQWUROOHUV FDQ EH XVHG DV DQ DOWHUQDWLYH WR WKH % 7RROVIRU95VRIWZDUHGHYHORSPHQW
RQHV RI IXOO\ IHDWXUHG VWDQGDORQH DQG 3& 95 KHDGVHWV RU 7KHUH DUH PDQ\ WRROV DYDLODEOH IRU FUHDWLQJ 95 HDFK
GLUHFWFRQWUROYLDWKHVFUHHQRIWKHSKRQH7KHVHFRQWUROOHUV KDYLQJLWVVWUHQJWKVDQGZHDNQHVVHV7RFKRRVHWKHULJKW
FDQ KDYH GLIIHUHQW QXPEHUV RI FRQWUROV DQG VHQVRUV WRROV ZH QHHG WR FRQVLGHU FRUUHVSRQGLQJ FULWHULD ZKLFK
LQFOXGLQJ MR\VWLFNV EXWWRQV WULJJHUV DQG J\URVFRSLF LQFOXGHZKHWKHUWKHWRROLVIUHHO\DYDLODEOHHDV\WROHDUQ
VHQVRUV VXSSRUWLQJ YDULRXV PRYHPHQWV DQG DFWLRQV 7KH DQGXVHDOORZVIRUHDV\VKDULQJDPRQJPXOWLSOHXVHUVLV
DGYDQWDJH RI XVLQJ %OXHWRRWK FRQWUROOHUV IRU 95 JUDSKLFDOO\ GHPDQGLQJ VXSSRUWV PXOWLSOH SODWIRUPV
DSSOLFDWLRQVLVWKDW WKH\ DOORZXVHUVWRFRQWUROJDPHV DQG $QGURLG,2606:LQGRZV DQGSURYLGHVHDV\DFFHVV
DSSOLFDWLRQV ZLWKRXW WKH QHHG IRU GLUHFW FRQWDFW ZLWK WKH IRUH[DPSOHWKURXJKDOLQNLQDEURZVHU%\IRFXVLQJRQ
GHYLFH ZKLFK FDQ LPSURYH FRPIRUW DQG FRQWURO DFFXUDF\ WKHVH UHTXLUHPHQWV ZH FDQ VHOHFW WKH ULJKW WRROV IRU
DQGLVQHFHVVDU\ZKHQWKHGHYLFHLVLQVHUWHGLQWRDKHDGVHW FUHDWLQJ95IRUPRELOHGHYLFHV)RUPRELOH95WKHPRVW
,Q DGGLWLRQ WKH\ DUH XVXDOO\ FRPSDWLEOH ZLWK PXOWLSOH VXLWDEOHVRIWZDUHIUDPHZRUNVDUH
GHYLFHVDQGFDQEHXVHGIRURWKHUSXUSRVHVEHVLGHV95 • 8QLW\>@DZLGHO\XVHGHQJLQHIRUFUHDWLQJGHVNWRS
,,, &2//$%25$7,9(95722/6$1'$33/,&$7,216 DQG PRELOH JDPHV LQFOXGLQJ 95 RQHV ,WV
DGYDQWDJHV LQFOXGH LQWXLWLYH FRQWUROV H[WHQVLYH
$ 95 H[SHULHQFH LV SULPDULO\ FUHDWHG E\ D VXSSRUW DQG PXOWLSODWIRUP FRPSDWLELOLW\ +RZHYHU
YLVXDOL]DWLRQ VXEV\VWHP FRQVLVWLQJ RI VRIWZDUH DQG LW KDV VRPH GUDZEDFNV VXFK DV KLJK KDUGZDUH
KDUGZDUH FRPSRQHQWV WKDW ZRUN WRJHWKHU WR JHQHUDWH D UHTXLUHPHQWV RSWLPL]DWLRQ FRPSOH[LW\ DQG KLJK
YLVXDOUHSUHVHQWDWLRQRIDYLUWXDOZRUOG>@7KHVRIWZDUH FRVWVRIWKHIXOOYHUVLRQ
FRPSRQHQW LV UHVSRQVLEOH IRU UHQGHULQJ WKH ' PRGHOV • 8QUHDO(QJLQH>@LVDSRSXODUWRROIRUFUHDWLQJ95
WH[WXUHV DQG DQLPDWLRQV WKDW PDNH XS WKH YLUWXDO ZRUOG DSSOLFDWLRQVIRUPRELOHGHYLFHV,WLVVLPLODUWR8QLW\
DV ZHOO DV SURFHVVLQJ XVHU LQWHUDFWLRQV 7KH KDUGZDUH LQ VRPH RI LWV DGYDQWDJHV DQG GLVDGYDQWDJHV ,WV
FRPSRQHQW LQFOXGHV GHYLFHV VXFK DV VFUHHQV DQG 95 DGYDQWDJHV LQFOXGH KLJK TXDOLW\ JUDSKLFV D ZLGH
KHDGVHWV ZKLFK XVH VSHFLDO WHFKQRORJLHV OLNH PRWLRQ UDQJH RI WRROV DQG IHDWXUHV DORQJ ZLWK DQ DFWLYH
VHQVRUV WR HQDEOH XVHUV WR PRYH DQG LQWHUDFW ZLWK WKH FRPPXQLW\RIGHYHORSHUV+RZHYHULWDOVRKDVKLJK
YLUWXDO ZRUOG >@ 5HDOLVWLF JUDSKLFV DUH FUXFLDO IRU KDUGZDUH UHTXLUHPHQWV LV PRUH FRPSOH[ WR XVH
FUHDWLQJ DQ LPPHUVLYH DQG EHOLHYDEOH 95 H[SHULHQFH FRPSDUHG WR RWKHU WRROV DQG LWV SDLG YHUVLRQ LV
7KLV UHTXLUHV KLJKTXDOLW\ WH[WXUHV DGYDQFHG OLJKWLQJ SULFH\
WHFKQLTXHV DQG FRPSOH[ PRGHOV DQG DQLPDWLRQV DOO • *RRJOH &DUGERDUG 6'. >@ LV D VHW RI WRROV IRU
RSWLPL]HGIRUVHDPOHVVSHUIRUPDQFH GHYHORSHUV WKDW HQDEOHV WKHP WR FUHDWH VLPSOH 95
$ &ROODERUDWLYH9LUWXDO5HDOLW\DQG0HWDYHUVH HQYLURQPHQWV IRU PRELOH GHYLFHV 7KH 6'. LV IUHH
&ROODERUDWLYH 6KDUHG 9LUWXDO5HDOLW\ &95695 LVD DQG VXSSRUWV PXOWLSOH ODQJXDJHV EXW LW¶V QRW DV
QHZ IRUP RI YLUWXDO UHDOLW\ WKDW DOORZV XVHUV WR ZRUN DGYDQFHG DV RWKHU 95 FUHDWLRQ WRROV 6XSSRUW IRU
YLUWXDOO\ZLWKRWKHUVLQWKHVDPHYLUWXDOVSDFHUHJDUGOHVV QHZHU RSHUDWLQJ V\VWHP YHUVLRQV PD\ EH GHOD\HG
RI WKHLU SK\VLFDO ORFDWLRQ >@ $FFRUGLQJ WR )ODYLiQ >@ DQGLW¶VDOVROLPLWHGRQ3&GHYLFHV
FROODERUDWLYH V\VWHPV UHSUHVHQW D VSHFLILF DUFKLWHFWXUH • $)UDPH>@ LV DQ RSHQVRXUFH ZHE IUDPHZRUN IRU
DQG WHFKQRORJLFDO HTXLSPHQW IRU WKH SRVVLELOLWLHV RI FUHDWLQJ ' 95 DQG $5 DSSOLFDWLRQV GLUHFWO\ LQ D
FUHDWLQJ FROODERUDWLYH DFWLYLWLHV 7KH DXWKRUV RI >@ VWDWH ZHE EURZVHU 2ULJLQDOO\ GHYHORSHG E\ 0R]LOOD LW
WKHQHHGIRUWKHSK\VLFDOSUHVHQFHRIXVHUVRUWKHLUYLUWXDO XVHV-DYD6FULSWDQG7KUHHMVOLEUDU\IRU'JUDSKLFV
SUHVHQWDWLRQ LQ WKH IRUP RI DYDWDUV LQ FROODERUDWLYH $)UDPH DOORZV GHYHORSHUV WR XVH +70/ DQG
HQYLURQPHQWV 7KH PRVW FRPPRQ FROODERUDWLYH -DYD6FULSWWRFUHDWHD'ZRUOGWKDWFDQEHGLVSOD\HG
HQYLURQPHQWV DW WKLV OHYHO DUH YLUWXDO &$9(EDVHG RQ YLUWXDOO\ DQ\ GHYLFH ZLWK D ZHE EURZVHU ,W
V\VWHPV DQG 95 KHDGVHWEDVHG V\VWHPV &95 LV EHLQJ VXSSRUWV WKH VWHUHRVFRSLF PRGH RQ VPDUWSKRQHV DQG
XVHGLQHGXFDWLRQWRKHOSZLWKOHDUQLQJDQGXQGHUVWDQGLQJ 95KHDGVHWVDVZHOODVYDULRXVW\SHVRIFRQWUROOHUV,W
RIYDULRXV VNLOOV&95 ZRUNVE\FRQQHFWLQJ XVHUVWRWKH LV GHVLJQHG IRU EHJLQQHUV DQG DGYDQFHG GHYHORSHUV
VDPH VHUYHU DQG FUHDWLQJ D YLUWXDO VSDFH XVLQJ YDULRXV DQGKDVDQDFWLYHFRPPXQLW\,WLVIUHHDQGZRUNVRQ
VRIWZDUH WRROV VXFK DV 0R]LOOD +XEV *OXH RU (QJDJH YDULRXV SODWIRUPV VXFK DV 3& $QGURLG DQG ,26
+DUGZDUH HTXLSPHQW VXFK DV 95 KHDGVHWV FRQWUROOHUV +RZHYHU VRPH DGYDQFHG IHDWXUHV VXFK DV
DQGPRWLRQVHQVRUVLVQHHGHGWRLQWHUDFWZLWKWKHYLUWXDO LQWHUDFWLRQVSK\VLFVVLPXODWLRQVDQG$,DUHOLPLWHG
ZRUOG &95 KDV D SRWHQWLDO IRU YDULRXV DSSOLFDWLRQV &ROODERUDWLYH IHDWXUHV PD\ EH DGGHG E\ XVLQJ LWV
LQFOXGLQJ YLUWXDO VKRSSLQJ PHGLFLQH FRQVWUXFWLRQ 1HWZRUNHG$IUDPH 1$) >@H[WHQVLRQ
WRXULVPHQWHUWDLQPHQWDQGPRUH>@>@>@ & ([LVWLQJ95$SSOLFDWLRQVIRU(GXFDWLRQ
$ FRQFHSW RIWHQ DVVRFLDWHG ZLWK FROODERUDWLYH 95 LV
0HWDYHUVH >@ ZKLFK FDQ EH GHVFULEHG DV D IXOO\ 7KHUH DUH DOUHDG\ VHYHUDO SURMHFWV DQG FRPSDQLHV
LQWHUDFWLYH YLUWXDO ZRUOG ZKHUH XVHUV FDQ FUHDWH DQG ZRUNLQJ RQ PDNLQJ 95 PRUH DYDLODEOH IRU VFKRROV DQG
LQWHUDFW ZLWK YLUWXDO REMHFWV DQG FKDUDFWHUV ,W LV RIWHQ HGXFDWLRQ
SUHVHQWHGDVDJOREDOYLUWXDOZRUOGZKLFKDOORZVXVHUVWR • 1HDUSRG951HDUSRG>@LVDFRPSDQ\WKDWFUHDWHV
SHUIRUP YDULRXV DFWLYLWLHV VXFK DV JDPLQJ HGXFDWLRQ WHFKQRORJ\ IRU LQWHUDFWLYH WHDFKLQJ ZLWK WKHLU
FRPPHUFH ZRUN DQG VRFLDO LQWHUDFWLRQV :KLOH 95 LV SURGXFW EHLQJ 1HDUSRG 95 ,W¶V D SODWIRUP WKDW
DOUHDG\DYDLODEOH0HWDYHUVHLVVWLOOXQGHUGHYHORSPHQW SURYLGHVHGXFDWLRQDO PDWHULDOVIRUERWKVWXGHQWVDQG

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 464

 WHDFKHUV DOORZLQJ WHDFKHUV WR FUHDWH LQWHUDFWLYH • &ODVV95 >@ VSHFLDOL]HV LQ SURYLGLQJ YLUWXDO DQG
YLUWXDO OHVVRQV DQG SUHVHQWDWLRQV 7KH SODWIRUP DXJPHQWHG UHDOLW\EDVHG HGXFDWLRQDO VROXWLRQV IRU
LQFOXGHV D OLEUDU\ RI RYHU  YLUWXDO H[SHULHQFHV VFKRROV DQG LQVWLWXWLRQV 7KH\ RIIHU KDUGZDUH DQG
LQFOXGLQJ YLVLWVWR YDULRXVSODFHVDQGH[SHULPHQWLQJ VRIWZDUH LQFOXGLQJ YLUWXDO UHDOLW\ KHDGVHWV VHQVRUV
ZLWK VFLHQWLILF FRQFHSWV 1HDUSRG FODLPV WKDW WKHLU DQG LQWHUDFWLYH DSSOLFDWLRQV WKDW FUHDWH D FRPSOHWH
V\VWHPLVGHVLJQHGWRKHOSVWXGHQWVEHWWHUXQGHUVWDQG YLUWXDOOHDUQLQJHQYLURQPHQW&ODVV95VKRXOGEHWKH
DQG UHPHPEHU LQIRUPDWLRQ E\ H[SORULQJ WKH ZRUOG VXFFHVVRU RI *RRJOH ([SHGLWLRQV &ODVV95¶V
DURXQGWKHPLQDVDIHDQGFRQWUROOHGHQYLURQPHQWDW DSSOLFDWLRQV DUH EDVHG RQ HGXFDWLRQDO VWDQGDUGV DQG
WKHLURZQSDFH RIIHU LQWHUDFWLYH HOHPHQWV WKDW SURPRWH VWXGHQW
• *RRJOH ([SHGLWLRQV >@ ZDV D SODWIRUP FUHDWHG E\ HQJDJHPHQWDQGLQWHUHVWLQOHDUQLQJ7KH\DOVRDOORZ
*RRJOHWKDWRIIHUHGYLUWXDOH[FXUVLRQVDQGH[SORUDWLRQ WHDFKHUVWRFUHDWHWKHLURZQDSSOLFDWLRQVDQGFRQWHQW
RI GLIIHUHQW SDUWV RI WKH ZRUOG 6WXGHQWV XVHG 95 WR FDWHU WR WKH VSHFLILF QHHGV RI WKHLU VWXGHQWV DQG
KHDGVHWV WDEOHWV RU SKRQHV DQG KDG DFFHVV WR SUH VXEMHFWV &ODVV95 LV XVHG LQ VHYHUDO VFKRROV
SUHSDUHG H[SHGLWLRQV WKDW FRQWDLQHG SDQRUDPLF YLHZV ZRUOGZLGH DQG KHOSV WR LPSURYH PRWLYDWLRQ
RI GLIIHUHQW SODFHV DQG PRQXPHQWV 7KH WHDFKHU ZDV HQJDJHPHQW DQG LQWHUHVW LQ OHDUQLQJ ZKLOH FUHDWLQJ
DEOHWRFRQWUROWKHH[FXUVLRQDQGH[SODLQZKDWLVLQWKH LQQRYDWLYHDQGLQWHUDFWLYHHGXFDWLRQDOHQYLURQPHQWV
VXUURXQGLQJV *RRJOH ([SHGLWLRQV ZDV DYDLODEOH LQ
PRUH WKDQ  FRXQWULHV DQG HQDEOHG VWXGHQWV WR ,9 68,7$%/(6&+22/68%-(&76
H[SORUH GLIIHUHQW FXOWXUHV DQG SODFHV DOO DURXQG WKH :KLOH WKH H[LVWLQJ OLWHUDWXUH DQG DSSOLFDWLRQV SURYLGH
ZRUOG,WLVQRORQJHUGHYHORSHGDQGSURYLGHG DQLQVLJKWLQWRWKHVXLWDELOLW\RILQGLYLGXDOVFKRROVXEMHFWV
• 8QLPHUVLY>@LVDSODWIRUPWKDWRIIHUVDVLPSOHDQG IRU 95 XWLOL]DWLRQ ZH ZHUH LQWHUHVWHG LQ WKH RSLQLRQ RI
HIIHFWLYH ZD\ WR XVH YLUWXDO UHDOLW\ LQ HGXFDWLRQDO WKH ORFDO SULPDU\ DQG VHFRQGDU\ VFKRRO WHDFKHUV :H
HQYLURQPHQWV)RXQGHGLQLWSURYLGHVDYDULHW\ FRQVXOWHGVHYHUDORIWKHPDQGFRQFOXGHWKDWPRVWRIWKHP
RI HGXFDWLRQDO DSSOLFDWLRQV LQ QDWXUDO VFLHQFHV FRXOG LPDJLQH XVLQJ 95 DSSOLFDWLRQV LQ WHDFKLQJ
KLVWRU\ DUW DQG PRUH 7KH DSSOLFDWLRQV DUH FUHDWHG HVSHFLDOO\WRJUDEFKLOGUHQ¶VDWWHQWLRQ$FFRUGLQJWRWKHLU
EDVHG RQ H[SHUW UHVHDUFK DQG RIIHU LQWHUDFWLYH RSLQLRQ WKH IROORZLQJ VXEMHFWV ZRXOG EHQHILW WKH PRVW
HOHPHQWV VXFK DV TXL]]HV WR KHOS VWXGHQWV LPSURYH IURPWKHXVHRI95
WKHLU NQRZOHGJH 8QLPHUVLY LV DFFHVVLEOH RQ 95 • 3K\VLFV DQG &KHPLVWU\ DV WKH\ RIIHU PDQ\ SUDFWLFDO
KHDGVHWV PRELOH GHYLFHV DQG FRPSXWHUV DOORZLQJ H[SHULPHQWVWKDWFRXOGEHWUDQVIRUPHGLQWR957KLV
VWXGHQWVWRH[SORUHKLVWRULFDOSHULRGVYLVLWPXVHXPV ZRXOG DOORZ VWXGHQWV WR VDIHO\ REVHUYH RU SHUIRUP
DQGH[DPLQHWKHZRUOG¶VPRVWVLJQLILFDQWDUWZRUNV H[SHULPHQWV ZLWKRXW WKH ULVN RI LQMXU\ RU GDPDJH WR
• /LIHOLTH>@LVDQHGXFDWLRQDOWRROWKDWRIIHUVDFFHVV SURSHUW\ (YHQ H[SHULPHQWV WKDW DUH QRW DOORZHG DW
WR D ZLGH UDQJH RI LQWHUDFWLYH ' DQG $5 PDWHULDOV DOOLQVFKRROODERUDWRULHVFDQEHRIIHUHG
RQ YDULRXV VXEMHFWV VXFK DV ELRORJ\ SK\VLFV DQG • %LRORJ\DQG+LVWRU\7KLVPLJKWORRNDVDQXQQDWXUDO
FKHPLVWU\6WXGHQWVFDQYLHZREMHFWVUHODWHGWRWKHLU FRPELQDWLRQ EXW ZH LGHQWLILHG VLPLODU UHDVRQV IRU
VXEMHFWV LQ ' DQG IURP GLIIHUHQW DQJOHV 7HDFKHUV ERWKRIWKHP6WXGHQWVFRXOGYLHZ PRGHOVRI WKLQJV
FDQ FXVWRPL]H WKH FRQWHQW WR WKHLU QHHGV DQG FUHDWH WKH\DUHOHDUQLQJDERXWVXFKDVKLVWRULFDOREMHFWVIRU
WKHLU RZQ PDWHULDO /LIHOLTH PDNHV OHDUQLQJ IXQ IRU +LVWRU\ DQG DQLPDO PRGHOV IRU %LRORJ\ 7KDQNV WR
VWXGHQWV E\ DOORZLQJ WKHP WR VROYH LQWHUDFWLYH WDVNV DQLPDWLRQV WKH\ PD\ EH DEOH WR REVHUYH KRZ WKHVH
DQGDFWLYHO\HQJDJHLQWKHOHDUQLQJSURFHVV2YHUDOO REMHFWVZRUNDQGDUHXVHG,WZRXOGDOVREHSRVVLEOH
/LIHOLTH RIIHUV PDQ\ LQWHUHVWLQJ RSWLRQV WR KHOS WR FUHDWH VLPXODWLRQV RI IRU H[DPSOH EDWWOHV DQG
VWXGHQWV EHWWHU XQGHUVWDQG DQG UHPHPEHU ZKDW WKH\ DUP\PRYHPHQWVRULQGLYLGXDOILJKWVIRU+LVWRU\,Q
DUH WDXJKW DQG VDYH WHDFKHUV WLPH LQ FUHDWLQJ WKHLU %LRORJ\VWXGHQWVFRXOGVHHKRZDQLPDOVKXQWRUKRZ
RZQWHDFKLQJFRQWHQW DERG\ZRUNVGXULQJFRUUHVSRQGLQJDFWLYLWLHV
• ]6SDFH>@LVDQHGXFDWLRQWRROWKDWXVHV95DQG$5 • $UW ZKHUH 95 FRXOG EH XVHG WR FUHDWH D YLUWXDO
IRULQWHUDFWLYHHGXFDWLRQLQ67(0VXEMHFWV6WXGHQWV JDOOHU\VKRZFDVLQJYDULRXVSDLQWLQJVVFXOSWXUHVDQG
VHH WKH REMHFWV LQ D' VSDFH DQG PDQLSXODWH WKHP RWKHUZRUNVRIDUW,QWHUDFWLYHURRPVZKHUHVWXGHQWV
XVLQJ DQ LQWHUDFWLYH SHQ 7KLV WRRO KHOSV VWXGHQWV FRXOGGUDZDQGSDLQWFDQDOVREHFUHDWHGDOEHLWVXFK
UHPHPEHU QHZLQIRUPDWLRQDQGEHWWHUXQGHUVWDQGLW DFWLYLWLHV PD\ QRW EH SRVVLEOH WR FDUU\ RXW SUHFLVHO\
,WFDQDOVREHFRQQHFWHGWRDQLQWHUDFWLYHZKLWHERDUG ZLWKFKHDSPRELOH95KHDGVHWVSULPDULO\EHFDXVHRI
• 9LFWRU\;5 >@ VSHFLDOL]HV LQ FUHDWLQJ KLJKTXDOLW\ DQLQVXIILFLHQWDFFXUDF\RIWKHLUFRQWUROOHUV
HGXFDWLRQDO HQYLURQPHQWV IRU VWXGHQWV DQG WHDFKHUV • /DQJXDJHV +HUH VWXGHQWV FRXOG H[SORUH YLUWXDO
XVLQJ95DQG$5WHFKQRORJLHV,WIRFXVHVRQ67(0 SODFHV ZKHUH WKH ODQJXDJH LV VSRNHQ DQG OHDUQ E\
HGXFDWLRQ DOORZLQJ VWXGHQWV WR FRQQHFW WKHRUHWLFDO VSHDNLQJ ZLWK YLUWXDO FKDUDFWHUV ,W ZRXOG DOVR EH
NQRZOHGJH ZLWK SUDFWLFDO H[SHULHQFH DQG SRVVLEOH WR FUHDWH VLPXODWLRQV RI UHDOOLIH VLWXDWLRQV
H[SHULPHQWV 9LFWRU\;5 DOVR RIIHUV HGXFDWLRQDO VXFKDVRUGHULQJRUVKRSSLQJ
DSSOLFDWLRQVLQKLVWRU\ELRORJ\JHRJUDSK\DQGRWKHU • *HRJUDSK\ ZKHUH 95 FRXOG DOORZ VWXGHQWV WR
VXEMHFWV WDNLQJ DGYDQWDJH RI WKH YLVXDOL]DWLRQ H[SORUH YLUWXDO ODQGVFDSHV IO\ RYHU WKHP YLVLW
FDSDELOLWLHV RI YLUWXDO DQG DXJPHQWHG UHDOLW\ FXOWXUDO DQG QDWXUDO ODQGPDUNV REVHUYH ZHDWKHU
WHFKQRORJLHV FKDQJHVDQGVWXG\WKHJHRORJLFDOFRPSRVLWLRQRIWKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 465

 (DUWK,WZRXOGDOVREHSRVVLEOHWRFUHDWHVLPXODWLRQV
RI ZHDWKHU SKHQRPHQD VXFK DV HDUWKTXDNHV RU
KXUULFDQHV VR WKDW VWXGHQWV FRXOG EHWWHU XQGHUVWDQG
WKHLUEHKDYLRUDQGFRQVHTXHQFHV
• 0DWKHPDWLFV6WXGHQWVFRXOG YLHZ YLUWXDO PRGHOVRI
JHRPHWULF VKDSHV JUDSKV DQG IXQFWLRQV DQG
LQWHUDFWLYHO\ SOD\ ZLWK WKHP ,W ZRXOG DOVR EH
SRVVLEOH WR FUHDWH VLPXODWLRQV RI YDULRXV
PDWKHPDWLFDO SUREOHPV VXFK DV VROYLQJ HTXDWLRQV
FDOFXODWLQJ VXUIDFH DUHDV DQG YROXPHV RU HYHQ 
JHRPHWULFWUDQVIRUPDWLRQV )LJXUH3K\VLFVVFHQH
• &RPSXWHU 6FLHQFH 9LUWXDO 5HDOLW\ FRXOG EH XVHG LQ
FRPSXWHU VFLHQFH HGXFDWLRQ IRU SURJUDPPLQJ 7KH %LRORJ\ VFHQH )LJXUH   LV D JDUGHQ ZLWK WDEOHV
HQWLWLHVVXFKDVURERWVRUYHKLFOHVLQYLUWXDOZRUOGV H[KLELWLQJ'PRGHOVRISODQWVIUXLWVDQGDQLPDOVDORQJ
6WXGHQWV FRXOG DOVR GHVLJQ DQG GHYHORS WKHLU RZQ ZLWK WH[W ODEHOV ZLWK FRUUHVSRQGLQJ LQIRUPDWLRQ 6XFK
YLUWXDOVSDFHVDQGLQWHUDFWLYHHOHPHQWV ZKLFKFRXOG VFHQHV FDQ DOVR EH XVHG LQ VXEMHFWV OLNH *HRJUDSK\ DQG
PRWLYDWH WKHP WR IRFXV RQ SURJUDPPLQJ LQ WKHLU +LVWRU\ 7KH\ FDQ EH HQULFKHG ZLWK PRUH PRGHOV DQG
IXWXUHVWXGLHV DQLPDWLRQVRIUHODWHGSURFHVVHV

7KH WHDFKHUV DOVR SRLQWHG RXW WKDW YLUWXDO VFHQHV FRXOG
DOVR EH XVHG WR HYDOXDWH VWXGHQWV ZKHWKHU WKURXJK
SUDFWLFDO WDVNV SHUIRUPHG E\ VWXGHQWV RU E\ NQRZOHGJH
WHVWV ZKHUH VWXGHQWV QDPH REMHFWV LQ WKH URRP RU VHOHFW
WKHFRUUHFWDQVZHUVWRTXHVWLRQV
9 ('8&$7,21$/9,578$/(19,5210(17
,03/(0(17$7,21
7RH[SORUHWKHSRVVLELOLWLHVRIHGXFDWLRQDO&9(DLPHG 
SULPDULO\ IRU PRELOH 95 KHDGVHWV ZH GHVLJQHG D ZHE
)LJXUH%LRORJ\VFHQH
EDVHGHGXFDWLRQDO&9(XVLQJWKH$)UDPH>@VRIWZDUH
IUDPHZRUNDQG*OLWFKGHYHORSPHQWDQGKRVWLQJSODWIRUP 7KH ODVW VFHQH FUHDWHG LV WKH .QRZOHGJH VFHQH DLPHG
7KDQNV WR $)UDPH WKH &9( KDV D IRUP RI ZHE VLWH DWWHVWLQJVWXGHQWV,WFRQWDLQVVL[TXHVWLRQVWKDWVWXGHQWV
DFFHVVLEOH IURP DQ\ GHYLFH ZLWK D ZHE EURZVHU  7KH PXVW DQVZHU FRUUHFWO\ $IWHU DQVZHULQJ WKH EXWWRQ ZLWK
IRFXV RQ PRELOH 95 KHDGVHWV LV LQ WKH VXSSRUW RI WKH FKRVHQ DQVZHU FKDQJHV LWV FRORU WR JUHHQ RU UHG
%OXHWRRWK FRQWUROOHUV DQG VWHUHRVFRSLF GLVSOD\ PRGH RQ LQGLFDWLQJ ZKHWKHU WKH DQVZHU LV FRUUHFW RU QRW
VPDUWSKRQHV DQG LQ WKH FKRLFH RI UHODWLYHO\ VLPSOH ' $GGLWLRQDOO\WKHUHLVDVRXQGWKDWLQGLFDWHVLIWKHDQVZHU
PRGHOV IRU D VPRRWK SHUIRUPDQFH RQ ORZHU HQG LV FRUUHFW 6WXGHQWV DOVR KDYH WKH RSWLRQ WR UHVHW WKHLU
VPDUWSKRQHV DQVZHUVLIWKH\PDGHDPLVWDNH
$ 9LUWXDO6FHQHV % 8VHG7HFKQRORJLHV
2XU DSSOLFDWLRQ SURYLGHV VHYHUDO YLUWXDO VFHQHV RU 7RLPSOHPHQWWKHSURWRW\SHZHXVHGWKH$)UDPHZHE
URRPVLPSOHPHQWLQJVHOHFWHGWRSLFVIURPDVXEVHWRIWKH IUDPHZRUN DV LW RIIHUV WKH EHVW FURVVSODWIRUP
VXEMHFWVOLVWHGLQ6HFWLRQ,9,QVWHDGRIDPDLQPHQXDQ FRPSDWLELOLW\ HDV\ VKDULQJ YLD WKH PHDQV DYDLODEOH IRU
HQWU\ VFHQH UHVHPEOLQJ DQ HPSW\ OHFWXUH KDOO ZKHUH ZHE SDJHV DQG PDQ\ FRPPXQLW\FUHDWHG H[WHQVLRQV IRU
VWXGHQWVFDQFKRRVHZKLFKVXEMHFWVSHFLILFVFHQHWRHQWHU DGGLWLRQDOIXQFWLRQDOLW\
LVRIIHUHG7KUHHGLIIHUHQWVFHQHVDUHDYDLODEOHDQGHDFK 7KH PRVW LPSRUWDQW H[WHQVLRQ HPSOR\HG ZDV
RQHFRQWDLQVFRQWUROVIRUHDV\QDYLJDWLRQ7KHVFHQHVDUH 1HWZRUNHG$IUDPH 1$)  ZKLFK DOORZHG XV WR HDVLO\
QDPHGDFFRUGLQJWRWKHFRUUHVSRQGLQJVXEMHFW FUHDWH D VKDUHG YLUWXDO UHDOLW\ H[SHULHQFH DQG HQDEOHG
7KHILUVWVFHQH )LJXUH LVGHGLFDWHGWRWKHVXEMHFWRI XVHUV WR HQWHU VKDUHG VFHQHV MXVW E\ XVLQJ WKH VDPH XUO
3K\VLFV +HUH VWXGHQWV FDQ UXQ DQG REVHUYH DQ 7KHUHWKHXVHUVFDQLQWHUDFWLQUHDOWLPHYLDWKHLUDYDWDUV
H[SHULPHQW ZLWK D IDOOLQJ EDOO 7KH H[SHULPHQW 1$) XVHV WKH :HE6RFNHW SURWRFRO WR FRPPXQLFDWH
GHPRQVWUDWHV WKH SULQFLSOHV RI HQHUJ\ FRQVHUYDWLRQ DQG EHWZHHQ WKH YLUWXDO HQYLURQPHQW LQVWDQFHV RI LQGLYLGXDO
HODVWLF FROOLVLRQV 7KH URRP FRQWDLQV EXWWRQV IRU VWDUWLQJ XVHUV HQDEOLQJ DXWRPDWLF FRQWHQW XSGDWHV ZLWKRXW WKH
DQG UHVWDUWLQJ WKH H[SHULPHQW DV ZHOO DV D VLPSOH ZHESDJHUHIUHVK,WDOVRVXSSRUWV:HE57&IRUDXGLRDQG
H[SODQDWLRQ RI WKH H[SHULPHQW RQ D ZKLWH SDQHO 7KH YLGHR WUDQVPLVVLRQ EHWZHHQ EURZVHUV 7KH H[WHQVLRQ
DQLPDWLRQ RI WKH IDOOLQJ EDOO LV DFKLHYHG XVLQJ WKH $ V\QFKURQL]HVVKDUHGYLUWXDO VFHQHREMHFWVEHWZHHQXVHUV
)UDPH DQLPDWLRQ FRPSRQHQW 7KH VFHQH DOVR LQFOXGHV HQVXULQJWKDWDOOXVHUVVHHWKHP LQWKHVDPH ZD\,WDOVR
EXWWRQV IRU PRYLQJ WR RWKHU RQHV )URP WKH SURYLGHVFXVWRPIXQFWLRQVWRDLGLQWKHFUHDWLRQRIEHWWHU
LPSOHPHQWDWLRQ SRLQW RI YLHZ WKH VFHQH GHPRQVWUDWHV &95H[SHULHQFHV
KRZWRUHSUHVHQWSK\VLFDOH[SHULPHQWVZLWKDQLPDWLRQV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 466

 $QRWKHU H[WHQVLRQ XVHG LV +70/ (PEHG &RPSRQHQW 
$)UDPHUHDG\,WDOORZVWRLQVHUWDQ\+70/FRGH 
LQWR $)UDPH VFHQH HOHPHQWV VXFK DV DHQWLW\ DQG
VW\OHLWZLWK&66
7R ZRUN ZLWK $)UDPH DQG 1HWZRUNHG$IUDPH RQH
QHHGVDVHUYHUXVDEOHRSLWRQVIRUEHJLQQHUVDUH ;$033
RU 0RQJRRVH :HE 6HUYHU +RZHYHU WKHUH LV DOVR DQ
HDVLHU ZD\ WKDQNV WR RQOLQH VROXWLRQV VXFK DV  *OLWFK
*OLWFK KWWSVJOLWFKFRP LVDQRQOLQHSODWIRUPIRUZHE
DSSOLFDWLRQGHYHORSPHQWWKDWDOORZVXVHUVWRFUHDWHKRVW
DQG VKDUH SURMHFWV ZLWK RWKHUV ,W LQFOXGHV D FRGH HGLWRU
FRPPDQG WHUPLQDO DQG XVHIXO IHDWXUHV VXFK DV ORDGLQJ
SUHYLRXVSURMHFWYHUVLRQV0XOWLSOHXVHUVFDQZRUNRQWKH
VDPHFRGHVLPXOWDQHRXVO\DQGUHJLVWUDWLRQLVIUHH*OLWFK
SURYLGHV YDULRXV WHPSODWHV WR KHOS XVHUV JHW VWDUWHG
TXLFNO\DQGWDNHVFDUHRIWKHVHUYHUDQGGDWDEDVHVHWXS,W
DOVR DOORZV XVHUV WR XVH USP SDFNDJHV $GGLWLRQDOO\
XVHUV FDQ UHPL[ DQ\ SXEOLF SURMHFW WKH\ ILQG RQ *OLWFK
DQGZRUNRQWKHLURZQYHUVLRQ 
7R VXSSRUW 95KHDGVHWVSHFLILF IHDWXUHV RQ )LJXUH6RXUFHFRGHVWUXFWXUHRIWKHHQYLURQPHQWDVGHSOR\HGDWWKH
VPDUWSKRQHV WKH LPSOHPHQWDWLRQ XVHV WKH *RRJOH 95 *OLWFKSODWIRUP
6HUYLFHVDSSOLFDWLRQZLWK*RRJOH&KURPHEURZVHU
& 6RIWZDUH$UFKLWHFWXUH 9, (9$/8$7,21
7KH VRXUFH FRGH VWUXFWXUH RI WKH HQYLURQPHQW FDQ EH 'XULQJ WKH GHYHORSPHQW WKH HQYLURQPHQW KDG EHHQ
VHHQ LQ )LJXUH  7KH SDFNDJHMVRQ ILOH LV XVHG WR FRQWLQXRXVO\ WHVWHG XVLQJ WKUHH GHYLFHV D GHVNWRS
PDQDJH GHSHQGHQFLHV DQG FRQILJXUH WKH DSSOLFDWLRQ LQ FRPSXWHUDODSWRSDQGDVPDUWSKRQH7KHSKRQHQDPHO\
1RGHMV 7KH VHUYHUMV ILOH FRQWDLQV WKH FRGH IRU 6DPVXQJ6ZDVWKHPDLQGHYLFHDVWKHHQYLURQPHQW
VHWWLQJ XS WKH ZHE VHUYHU XVLQJ H[WHUQDO PRGXOHV LV SULPDULO\ GHVLJQHG IRU PRELOH GHYLFHV :H WHVWHG WKH
LQFOXGLQJ WKH (DV\57& VHUYHU DQG +773 VHUYHU DSSOLFDWLRQLQWKH*RRJOH&KURPHEURZVHUIRULQWHJUDWLRQ
QHFHVVDU\ IRU &95 7KH SXEOLF IROGHU FRQWDLQV +70/ ZLWK*RRJOH95VHUYLFHV7KHFROODERUDWLYHIXQFWLRQDOLW\
ILOHV IRU 95 URRPV DQG -DYD6FULSW ILOHV GHILQLQJ YDULRXV SURYLGHG E\ 1$) ZDV WHVWHG E\ XVLQJ DOO WKUHH GHYLFHV
95 URRP IXQFWLRQV 7KH PRYHPHQWFRPSRQHQWMV LV VLPXOWDQHRXVO\ 7KH GHOD\V REVHUYHG ZHUH PLQLPDO DQG
XVHG IRU FRQWUROOHU PRYHPHQW DQG WKH GLGQRWDIIHFWWKHXVHUH[SHULHQFHLQDQHJDWLYHZD\
VSDZQFRPSRQHQWMV GHWHUPLQHV ZKHUH D QHZ XVHU $V D PRELOH 95 KHDGVHW ZH XVHG WKH 6DPVXQJ 6
ZLOO DSSHDU LQ WKH 95 VSDFH 7KH UHVSRQVH SKRQHWRJHWKHUZLWKWKH5)95*KHDGVHW )LJXUH 
SLFNHUFRPSRQHQWMVFRPSRQHQWLVXVHGIRUVHOHFWLQJ
ZLWKD%OXHWRRWKFRQWUROOHUFRQQHFWHGWRWKHSKRQH
FRUUHFWDQVZHUVLQWKH.QRZOHGJHVFHQHFKDQJLQJEXWWRQ
FRORU DQG SOD\LQJ D VRXQG ZKHQ DQ DQVZHU LV VHOHFWHG
7KH FRPSRQHQW ZDLWV IRU D GDWD VLJQDO WR SOD\ D VRXQG
DQG LW VHQGV WKH VLJQDO WR DOO XVHUV ZKHQ D EXWWRQ LV
SUHVVHG 7KH EDOO)DOOFRPSRQHQWMV ILOH FRQWDLQV
FRGH WKDW FRQWUROV WKH IDOO RI D EDOO LQ WKH 3K\VLFV URRP
7KH FRGH XVHV D ORRS WKDW JUDGXDOO\ PRYHV WKH EDOO WR  
GLIIHUHQWSRVLWLRQVZLWKDVSHFLILHGGXUDWLRQDQGHDVLQJ,W
)LJXUH2SHQHG5)95*95KHDGVHW OHIW DQGVWHUHRVFRSLF
XSGDWHV WKH SRVLWLRQ RI WKH EDOO XVLQJ WKH DQLPDWLRQ UHQGHULQJRIDSDUWRIDYLUWXDOVFHQHDWDVPDUWSKRQHGLVSOD\ ULJKW 
DWWULEXWH 7KLV VROXWLRQ ZDV FKRVHQ EHFDXVH WKH
UHFRPPHQGHG VROXWLRQ RI WULJJHULQJ RQH DQLPDWLRQ DIWHU
DQRWKHUFDXVHGODJDQGIUHH]LQJLQ&95 $ 7HFKQLFDO/LPLWDWLRQV
7KH VHUYHUMVILOHFRQWDLQVWKHFRGHIRUVHWWLQJXSD &95 IRU PRELOH GHYLFHV SUHVHQWV VHYHUDO WHFKQLFDO
&95 VHUYHU XVLQJ 1RGHMV ([SUHVVMV DQG OLPLWDWLRQV LQFOXGLQJ OLPLWHG JUDSKLF SHUIRUPDQFH
6RFNHW,2 ZLWK WKH RSHQHDV\UWF PRGXOH IRU :HE57& EDWWHU\OLIHDQGWKHQHHGIRUDQLQWHUQHWFRQQHFWLRQ2OGHU
IXQFWLRQDOLW\7KHFRGHORDGVUHTXLUHGPRGXOHVVHWVXSD SKRQHV PD\ DOVR KDYH LPDJH TXDOLW\ LVVXHV DQG ODFN
VHUYHU FRQILJXUHV DQG VWDUWV DQ KWWS VHUYHU ZLWK QHFHVVDU\ VHQVRUV /LPLWHG VHUYHU SHUIRUPDQFH DQG
([SUHVVMV FUHDWHV D 6RFNHW,2 LQVWDQFH DQG VHWV XQUHOLDEOH LQWHUQHW FRQQHFWLRQV FDQ FDXVH GHOD\HG
,&( VHUYHUV IRU :HE57& FRQQHFWLRQV ,W DOVR VHWV XS D UHVSRQVH RU GDWD ORVV 7KH FKHDS FRQWUROOHU ZH XVHG
OLVWHQHU IRU DXWKHQWLFDWLRQ DQG FRQVROH ORJJLQJ IRU HDFK FDXVHG DGGLWLRQDO SUREOHPV ZLWK %OXHWRRWK SDLULQJ DQG
URRP FRQQHFWLRQ )LQDOO\ WKH FRGH VWDUWV WKH HDV\UWF FOLFNLQJ 7R RYHUFDPH WKLV ZH XWLOL]HG $)UDPH
VHUYHUDQGOLVWHQVRQDVSHFLILHGSRUWSULQWLQJWKHVHUYHU DWWULEXWHVIXVHDQGIXVHWLPHRXW7KHODWHVWYHUVLRQRI
85/WRWKHFRQVROH 1$) DOVR GRHV QRW VXSSRUW FUHDWLQJ PXOWLSOH VFHQHV RQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 467

WKHVDPHVHUYHUZKLFKFDQEHVROYHGE\FUHDWLQJDXQLTXH 5()(5(1&(6
VHUYHU IRU HDFK VFHQH RU FRQWUROOLQJ YLVLELOLW\ XVLQJ D >@ %XUJHVV66LHYHUWVHQ++6FKRROV6NLOOVDQG/HDUQLQJ7KH
GHGLFDWHGIXQFWLRQ ,PSDFW RI &29,' RQ (GXFDWLRQ ,Q 9R[(8RUJ  
GRVWXSQpQDKWWSVYR[HXRUJDUWLFOHLPSDFWFRYLGHGXFDWLRQ
% 7HDFKHUVVXUYH\ >@ 0 +XGiN% 6RERWD &ROODERUDWLYH YLUWXDO UHDOLW\ DQGLQWHUIDFHV
$ VXUYH\ ZDV FRQGXFWHG DPRQJ  WHDFKHUV WR JDWKHU RI V\VWHPV VW HG 7HFKQLFDO XQLYHUVLW\ RI .RãLFH  ,6%1

LQIRUPDWLRQ DERXW WKHLU IDPLOLDULW\ ZLWK 9LUWXDO 5HDOLW\
>@ +XGiN0.RUHþNRâ6RERWD%$GYDQFHG8VHU,QWHUDFWLRQIRU
DQGWKHLULQWHUHVWLQXVLQJLWIRUWHDFKLQJ7KHVXUYH\ZDV :HEEDVHG &ROODERUDWLYH 9LUWXDO 5HDOLW\ ,Q WK ,(((
LQWKHIRUPRIDTXHVWLRQQDLUHFRQVLVWLQJRITXHVWLRQV ,QWHUQDWLRQDO &RQIHUHQFH RQ &RJQLWLYH ,QIRFRPPXQLFDWLRQV 
VLPLODUWRWKH6\VWHP8VDELOLW\6FDOH7KHUHVXOWVVKRZHG &RJ,QIR&RP   'DQYHUV 86$   ,QVWLWXWH RI (OHFWULFDO DQG
(OHFWURQLFV(QJLQHHUVV>RQOLQH&'520@,6%1
WKDWWHDFKHUVZHUHQHXWUDOWRSRVLWLYHDERXWWKHXVHRI95  RQOLQH ,661
LQ WHDFKLQJ DQG ZRXOG ZHOFRPH LW DV DQ DGGLWLRQDO >@ +XGiN 0 .RUHþNR â 6RERWD%(QKDQFLQJ 7HDP ,QWHUDFWLRQ
WHDFKLQJ DLG +RZHYHU PRVW RI WKHP KDG OLPLWHG DQG &URVVSODWIRUP $FFHVV LQ :HEEDVHG &ROODERUDWLYH 9LUWXDO
H[SHULHQFH ZLWK 95 DQG ZRXOG UHTXLUH WUDLQLQJ EHIRUH (QYLURQPHQWV ,Q 3URFHHGLQJV RI  ,((( WK ,QWHUQDWLRQDO
6FLHQWLILF&RQIHUHQFHRQ,QIRUPDWLFV,(((SS
XVLQJLWIRUWHDFKLQJ
>@ )ODYLiQ&,EixH]6iQFKH]6 2U~V&  7KHLPSDFWRI
& 6WXGHQW)HHGEDFN YLUWXDODXJPHQWHGDQGPL[HGUHDOLW\WHFKQRORJLHVRQWKHFXVWRPHU
H[SHULHQFH-RXUQDORI%XVLQHVV5HVHDUFK
:H SUHVHQWHG WKH DSSOLFDWLRQ WR VHYHQWKJUDGH >@ *RQ]DOH])UDQFR0+DOO0+DQVHQ'-RQHV.+DQQDK3
VWXGHQWV WDNLQJ  PLQXWHV WR VKRZFDVH DQG WHVW WKH  %HUPHOO*DUFLD 3  0DUFK  )UDPHZRUN IRU UHPRWH
DSSOLFDWLRQ 7KH VWXGHQWV ZHUH VKRZQ D VLPSOH VFHQDULR FROODERUDWLYH LQWHUDFWLRQ LQ YLUWXDO HQYLURQPHQWV EDVHG RQ
DQG WKH DSSOLFDWLRQ¶V IXQFWLRQDOLW\ ZDV VLPXOWDQHRXVO\ SUR[LPLW\,Q'8VHU,QWHUIDFHV '8, ,(((6\PSRVLXP
RQ SS ,(((
SURMHFWHGRQWR D VFUHHQ WR HQDEOH DOO VWXGHQWV WR REVHUYH
>@ 0LVKUD . %RODQG 09 :RUHWD )$  ,QFRUSRUDWLQJ D YLUWXDO
LW 7KH VWXGHQWV JDYH JHQHUDOO\ SRVLWLYH IHHGEDFN FXUULFXOXP LQWR RSKWKDOPRORJ\ HGXFDWLRQ LQ WKH FRURQDYLUXV
FULWLFL]LQJ RQO\ WKH TXDOLW\ RI WKH FRQWUROOHU DQG LQ RQH GLVHDVHHUD,Q&XUU2SLQ2SKWKDOPRO±
FDVHWRRVLPSOHJUDSKLFVXVHGLQWKHYLUWXDOVFHQHV >@ :LMNPDUN &+ )DQNYLVW 6 +HOGDO , 0HWDOOLQRX 00
5HPRWH 9LUWXDO 6LPXODWLRQ IRU ,QFLGHQW &RPPDQGHUV
9,,&21&/86,21 2SSRUWXQLWLHVDQG3RVVLELOLWLHV ,Q"3URFHHGLQJVRIWKHWK
,((( ,QWHUQDWLRQDO&RQIHUHQFHRQ&RJQLWLYH ,QIRFRPPXQLFDWLRQV
7KH HYDOXDWLRQ UHVXOWV REWDLQHG ZLWK RXU ZHEEDVHG  SS±
HGXFDWLRQDO &9( LQGLFDWH WKDW WKH \RXQJ JHQHUDWLRQ LV >@ /XLPXOD 0 /LQGHU 0 3LHVNl 6 /DLPLR  ( /lKGH 7
YHU\ HQWKXVLDVWLF DERXW XVLQJ 95 WHFKQRORJLHV LQ 3RUUDPR 3  8QOLPLWHG 6DIHW\ 3URGXFWLYLW\²$ )LQQLVK
3HUVSHFWLYH 8VLQJ 9LUWXDO /HDUQLQJ 0HWKRGV WR ,PSURYH 4XDOLW\
HGXFDWLRQ 7KH\ DJUHH RQ WKH SRVLWLYH LPSDFW RI WKHVH DQG 3URGXFWLYLW\ LQWKH&RQVWUXFWLRQ,QGXVWU\ ,Q 3URFHHGLQJV RI
WHFKQRORJLHV RQ WKHLU LQWHUHVW DQG XQGHUVWDQGLQJ RI WKH WKH WK ,((( ,QWHUQDWLRQDO &RQIHUHQFH RQ &RJQLWLYH
VFKRROVXEMHFWV7KHWHDFKHUVLQYROYHGEHOLHYHWKDWXVLQJ ,QIRFRPPXQLFDWLRQV  SS±
95 FDQ LPSURYH WKH TXDOLW\ RI WHDFKLQJ DQG PRWLYDWH >@ 0HWDYHUVH KRPH SDJH >RQOLQH@  >FLW @ 85/
VWXGHQWV EXW VRPH DUH VNHSWLFDO DERXW WKH FRVWV DQG WKH KWWSVDERXWPHWDFRPPHWDYHUVH
>@ 8QLW\ *DPH (QJLQH KRPH SDJH >RQOLQH@  >FLW @
QHHG IRU DSSURSULDWH WUDLQLQJ 7KH FRVW UHGXFWLRQ ZDV RI 85/KWWSVXQLW\FRP
VSHFLDO LQWHUHVW LQ WKLV ZRUN DQG ZH DFKLHYHG LW E\ >@ 8QUHDO (QJLQH KRPH SDJH >RQOLQH@  >FLW @ 85/
XWLOL]LQJ D PRELOH 95 KHDGVHW DQG XVLQJ VLPSOH ' KWWSVZZZXQUHDOHQJLQHFRP
JUDSKLFVLQYLUWXDOVFHQHVVRWKDWWKH\ZLOOUXQVPRRWKO\ >@ *RRJOH &DUGERDUG 6'. KRPH SDJH >RQOLQH@  >FLW 
HYHQ RQ ORZHUHQG GHYLFHV 7KH SURWRW\SH ZDV ZHOO @85/KWWSVGHYHORSHUVJRRJOHFRPFDUGERDUG
UHFHLYHG E\ VWXGHQWV ZLWK RQO\ RFFDVLRQDO REMHFWLRQV >@ $IUDPH KRPH SDJH >RQOLQH@  >FLW @ 85/
KWWSVDIUDPHLR
WRZDUGVWKHQDWXUDOYLFWLPVRIWKHFRVWUHGXFWLRQVLPSOH
>@ 1HDUSRG95 KRPH SDJH >RQOLQH@  >FLW @ 85/
'VFHQHVDQGORZHUTXDOLW\RIWKHHOHFWURQLFFRPSRQHQWV KWWSVQHDUSRGFRPQHDUSRGYU
RIWKHKHDGVHW WKH%OXHWRRWKFRQWUROOHULQRXUFDVH  >@ *RRJOH ([SHGLWLRQV >RQOLQH@  >FLW @ 85/
7KHFKRVHQVRIWZDUHSODWIRUP$)UDPHLVVXLWDEOHQRW KWWSVVLWHVJRRJOHFRPWFVQFRUJWFVJH[SHGLWLRQVJRRJOH
RQO\EHFDXVHRILWV QDWXUDOVXSSRUWRI PXOWLSOHSODWIRUPV H[SHGLWLRQVDSS
EXWDOVREHFDXVHRIWKHHDVH ZLWK ZKLFKQHZ VFHQHVFDQ >@ 8QLPHUVLY KRPH SDJH >RQOLQH@  >FLW @ 85/
KWWSVXQLPHUVLYFRP
EH DGGHG )RU VLPSOH RQHV LW LV RQO\ QHHGHG WR DGG DQ
>@ /LIHOLTH KRPH SDJH >RQOLQH@  >FLW @ 85/
+70/ILOHGHILQLQJWKHQHZVFHQHLQWRWKHSXEOLFIROGHU KWWSVZZZOLIHOLTHFRP
DQGOLQNLWIURPWKHHQWU\VFHQH2IFRXUVHWKHUHDUHDOVR >@ ]6SDFH KRPH SDJH >RQOLQH@  >FLW @ 85/
GUDZEDFNV RI WKH SODWIRUP VXFK DV WKH UHODWLYHO\ ORZHU KWWSV]VSDFHFRP
SHUIRUPDQFHEHFDXVHRIWKH-DYD6FULSWXWLOL]DWLRQDQGWKH >@ 9LFWRU\;5 KRPH SDJH >RQOLQH@  >FLW @ 85/
QHHG WR LQVWDOO WKH *RRJOH 95 6HUYLFHV WR HQDEOH 95 KWWSVZZZYLFWRU\[UFRP
KHDGVHWVSHFLILFIHDWXUHVRQVPDUWSKRQHV >@ &ODV695 KRPH SDJH >RQOLQH@  >FLW @ 85/
KWWSVZZZFODVVYUFRP
$&.12:/('*(0(17 >@ 1HWZRUNHG DIUDPH $ ZHE IUDPHZRUN IRU EXLOGLQJ PXOWLXVHU
YLUWXDO UHDOLW\ H[SHULHQFHV >RQOLQH@  >FLW @ 85/
7KLVZRUNKDVEHHQVXSSRUWHGE\WKH.(*$JUDQW1R KWWSVJLWKXEFRPQHWZRUNHGDIUDPHQHWZRUNHGDIUDPH
78.( Ä&ROODERUDWLYH YLUWXDO UHDOLW\
WHFKQRORJLHVLQWKHHGXFDWLRQDOSURFHVV³


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 468

 8VHU,QWHUIDFHVLQ9LUWXDO5HDOLW\(QYLURQPHQWV

%6RERWDâ.RUHþNR00DWWRYi:6WHLQJDUWQHU&V6]DEyDQG*6WURPS
7HFKQLFDO8QLYHUVLW\RI.RãLFH6ORYDNLD
EUDQLVODYVRERWD#WXNHVNVWHIDQNRUHFNR#WXNHVNPLULDPDPDWWRYD#WXNHVNZLOOLDPVWHLQJDUWQHU#WXNHVN
FVDEDV]DER#WXNHVNJDEULHOVWURPS#WXNHVN

$EVWUDFW²7KLVDUWLFOHSUHVHQWVDYLHZRIWKHXVHULQWHUIDFHV ವ 'LUHFW0HFKDQLFDO '0, 
GHYHORSPHQWEDVHGRQWKH;5SODWIRUPZLWKDQRULHQWDWLRQ
WRZDUGV SRVVLEOH XVH LQ HGXFDWLRQ )LUVW WKH W\SHV RI XVHU ವ &RPPDQG/LQH &/, 
LQWHUIDFHV DUH GHVFULEHG KHUH LQ FRQQHFWLRQ ZLWK WKH
ವ *UDSKLFDO8VHU,QWHUIDFH *8, 
FODVVLILFDWLRQ RI YLUWXDO UHDOLW\ W\SHV UHJDUGLQJ WULJJHUHG
DFWLRQV XVHU SHUFHSWLRQ DQG LQWHUDFWLRQ ZLWK UHDO REMHFWV ವ 1DWXUDO8VHU,QWHUIDFH 18, 
6XEVHTXHQWO\ XVHU LQWHUIDFHV DQG WKHLU XVDELOLW\ DUH
H[DPLQHG DQG PDLQO\ WKH VXSSRUW RI XVHU LQWHUIDFHV DW WKH ವ ,QWHOOLJHQW8VHU,QWHUIDFH ,8, 
KDUGZDUHOHYHO,QWKHHQGWKHDSSOLFDWLRQDUHDVRIWKH;5 7KH FRPPDQGOLQH LQWHUIDFH LV RQH RI WKH ILUVW XVHU
SODWIRUP DUH RXWOLQHG LQFOXGLQJ WKH QHFHVVDU\ XVHU LQWHUIDFHV LQ H[LVWHQFH WKDW ZDV RULHQWHG WRZDUG
LQWHUIDFHV LQ RUGHU WR PDNH LW HDVLHU VLPSOHU IDVWHU DQG IXQFWLRQDOLW\ QRW LQWXLWLYHQHVV RI XVH 7R FRQWURO WKH
PRUH WUDQVSDUHQW IRU VWXGHQWV DQG SXSLOV WR FRPPXQLFDWH FRPPDQG OLQH LW LV QHFHVVDU\ WR OHDUQ E\ KHDUW DOO WKH
ZLWK FRPSXWLQJ WHFKQRORJ\ DV ZHOO DV ZLWK WKHLU FRPPDQGV WKDW DUH XVHG WR FRQWURO WKH V\VWHP 7KH XVHU
VXUURXQGLQJHQYLURQPHQWGXULQJWKHHGXFDWLRQDOSURFHVV FRPPXQLFDWHVZLWKWKHV\VWHPXVLQJWH[WFRPPDQGV7KH
HQYLURQPHQW GRHV QRW DGDSW WR WKH XVHU RQ WKH FRQWUDU\
.H\ZRUGV ² XVHU LQWHUIDFH XVHU H[SHULHQFH H[WHQGHG
WKH XVHU PXVW DGDSW KLPVHOI UHJDUGLQJ WKH HQYLURQPHQW
UHDOLW\FRQWUROGHYLFHVRI;5HGXFDWLRQ
7KHXVHRIWKHFRPPDQGOLQHLVW\SLFDOIRUWDVNVZKHUHLW
LV QHFHVVDU\ WR KDYH IXOO FRQWURO RYHU WKH FRQWUROOHG
, ,1752'8&7,21 PDFKLQH IRU H[DPSOH FRQILJXULQJ WKH VHUYHU 6XFK DQ
LQWHUIDFH LV OHVV GHPDQGLQJ RQ UHVRXUFHV FRPSDUHG WR D
7KH XVHU LQWHUIDFH LV H[WUHPHO\ LPSRUWDQW IURP D JUDSKLFDOLQWHUIDFHVLQFHLWLVLQDWH[WIRUP
KXPDQXVHU V SRLQW RI YLHZ WR EH DEOH WR HIIHFWLYHO\
FRQWURODQ\DSSOLFDWLRQ,QWKHFDVHRILPSOHPHQWDWLRQLQD $VOLJKWO\ PRUHXVHUIULHQGO\LQWHUIDFHLVWKHJUDSKLFDO
YLUWXDO HQYLURQPHQW HVSHFLDOO\ IRU V\VWHPV RQ WKH XVHULQWHUIDFH,WLVPXFKPRUHLQWXLWLYHIRUWKHXVHUWKDQD
SODWIRUPRIH[WHQGHGUHDOLW\ H;WHQGHG5HDOLW\;5 LWLV WH[WEDVHGFRPPDQGOLQHLQWHUIDFH7KHXVHUFDQQDYLJDWH
HYHQ PRUH LPSRUWDQW ,Q DGGLWLRQ WR WKDW WKLV QHHG LWEHWWHUEDVHGRQJUDSKLFHOHPHQWVVXFKDVJUDSKLFLFRQV
LQFUHDVHV LI LW LV DQ LQWHUIDFH IRU FKLOGUHQ RU SXSLOV DQG DQG YDULRXV JUDSKLF FRPSRQHQWV VXFK DV IRUPV EXWWRQV
HVSHFLDOO\LIWKH\KDYHWRFRQWURODYLUWXDODQGLQDGGLWLRQ HWF 7KH XVHU FRPPXQLFDWHV ZLWK WKH V\VWHP E\
VKDUHG HQYLURQPHQW 7KH FODVVLILFDWLRQ RI GLIIHUHQW W\SHV FRQWUROOLQJ DQG PDQLSXODWLQJ JUDSKLF HOHPHQWV 7KLV
UHJDUGLQJ WULJJHUHG DFWLRQV XVHU SHUFHSWLRQ DQG LQWHUIDFHPDLQO\XVHVWKHPRXVHFXUVRUDQGNH\ERDUG
LQWHUDFWLRQ ZLWK UHDO REMHFWV LV VKRZQ LQ )LJXUH  )URP $QDWLYHW\SHRIXVHULQWHUIDFHFDQEHWKRXJKWRIDVDQ
WKLV SRLQW LW LV RQO\ D VPDOO VWHS WR FUHDWLQJ DQ DGHTXDWH LQYLVLEOHOD\HUDERYHWKHJUDSKLFDOXVHULQWHUIDFH7KLVLV
XVHULQWHUIDFHEXLOWRQWRSRIVXFKWHFKQRORJLHV,QRUGHUWR IRU H[DPSOH WRXFK RU YRLFH FRQWURO7KLV W\SHRI V\VWHP
EH DEOH WR GHVLJQ GLIIHUHQW W\SHV RI XVHU LQWHUIDFHV ZLWK FRQWURO LV QDWXUDO DQG LQWXLWLYH IRU KXPDQV >@ 7KH XVHU
GLIIHUHQWDSSURDFKHVWRFRQWUROWKHHQYLURQPHQWREMHFWVLW GRHV QRW QHHG WR OHDUQ DQ\ QHZ SULQFLSOHV WR FRQWURO WKH
LVLPSRUWDQWWRNQRZWKHEDVLFW\SHVRIXVHULQWHUIDFHV V\VWHP WKURXJK D QDWXUDO LQWHUIDFH DQG RQO\ XVHV WKH
NQRZOHGJH KH DOUHDG\ KDV $Q\RQH LQFOXGLQJ FKLOGUHQ
 +LJK FDQ KDQGOH WKLV W\SH RI FRQWURO >@ 0DQ\ ZHDUDEOH
0HGLDWHGUHDOLW\
0H5 
+\SHUH;WHQGHG
UHDOLW\
HOHFWURQLFVVXFKDVZDWFKHVEUDFHOHWVSKRQHVWDEOHWVDQG
RWKHUVLPLODUHTXLSPHQWDUHEXLOWWREHFRQWUROOHGE\WKLV
&RQVLVWHQF\EHWZHHQDFWLRQ

 +5;5 
5HDOLW\
W\SHRIXVHULQWHUIDFH
DQGSHUFHSWLRQ

0L[HG
9LUWXDOL]HG
UHDOLW\
9G5 
$XJPHQWHG
UHDOLW\ ,QWHOOLJHQW XVHU LQWHUIDFHV KDYH DQ DVSHFW RI DUWLILFLDO
$XJPHQWHG
YLUWXDOLW\
05$5 
LQWHOOLJHQFHWRDFHUWDLQH[WHQW0LFURVRIWZDVDPRQJWKH
$9 
ILUVWWRXVHLWLQLWV0LFURVRIWRIILFHVXLWHLQWKHIRUPRIDQ
9LUWXDO
UHDOLW\
LQWHUIDFH UHSUHVHQWHG E\ WKH &OLSS\ SDSHU FOLS 7KLV
95  LQWHUIDFH ZDV FRQWUROOHG WKURXJK YRLFH FRPPDQGV DQG
XVHU JHVWXUHV 5HFRJQLWLRQ RI WKHVH FRPPDQGV ZDV
/RZ ,QWHUDFWLRQZLWKUHDOREMHFWV +LJK  SURYLGHG E\ DUWLILFLDO LQWHOOLJHQFH *HQHUDOO\ LQ WKH
GRPDLQRILQWHUIDFHVQRWRQO\WKHPRYHPHQWRIDOLPERU
)LJXUH&ODVVLILFDWLRQRIUHDOLWLHVUHJDUGLQJWRLQGXFHGDFWLRQVXVHU
SHUFHSWLRQDQGLQWHUDFWLRQZLWKUHDOREMHFWV
WKH ERG\ EXW DOVR WKH EHKDYLRU RI WKH XVHU DV VXFK WKH
PRPHQWRIKLVDUULYDOLQDVSDFHVXFKDVDWZRUNRUWKH
8VHU LQWHUIDFHV IRUP D OLQN EHWZHHQ WKH XVHU DQG WKH WUDQVLWLRQ IURP D URRP WR DQRWKHU URRP HWF FDQ EH
V\VWHP WKURXJK ZKLFK WKH XVHU PDQDJHV DQGFRQWUROV WKH FRQVLGHUHG DV D JHVWXUH $V WLPH IORZV WKH LQWHOOLJHQW
JLYHQV\VWHP8VHULQWHUIDFHVFDQEHGLYLGHGLQWRVHYHUDO V\VWHPLVDEOHWROHDUQWRUHFRJQL]HVXFKJHVWXUHVZKLFK
FDWHJRULHVQDPHO\ GHVFULEH WKH XVHU V EHKDYLRU DQG FDQ DGDSW WKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 469

HQYLURQPHQWEDVHGRQWKHLUQHHGVDQGKDELWV$QH[DPSOH RI LQWHUIDFHV DQG DGDSW WKHVH UXOHV DFFRUGLQJ WR ZKDW WKH
FRXOG EH WXUQLQJ RQ DQG RII WKH OLJKWV LQ WKH KRPH FUHDWHG LQWHUIDFHV DUH LQWHQGHG IRU 6FKQHLGHUPDQ
GHSHQGLQJRQZKLFKURRPWKHXVHULVLQ,QWKHIUDPHZRUN GHVFULEHVWKHIROORZLQJSULQFLSOHV
RILQWHOOLJHQWLQWHUIDFHVLWLVDOVRSRVVLEOHWRH[DPLQHWKH
XVHU VELRPHWULFVVXFKDVWKHUHVSRQVLYHQHVVRIWKHSXSLOV ವ 6WULYLQJ IRU FRQVLVWHQF\  LQ RUGHU WR VROYH WKH VDPH
RUVNLQPRYHPHQWVGXULQJGHFLVLRQPDNLQJDQGUHDFWLRQV VFHQDULRV LQ WKH V\VWHP WKH XVHU VKRXOG SHUIRUP WKH
WRWKHHQYLURQPHQWZKLFKKDVEHHQDGGUHVVHGE\=KRXHW VDPH VHTXHQFH RI DFWLRQV WKH V\VWHP VKRXOG KDYH D
DO>@LQWKHLUUHVHDUFK XQLIRUP GHVLJQ HOHPHQWV FRPSRQHQWV DQG
SLFWRJUDPV VKRXOG PDNH XVH RI D FRPSUHKHQVLYH XVH
7KHXVHULQWHUIDFHLVH[WUHPHO\LPSRUWDQWLQWKHFRQWH[W RIFRORXUVKLJKOLJKWHUURUVWDWHVDQGZDUQLQJVDQGVR
RI+&, +XPDQ&RPSXWHU,QWHUDFWLRQ VRWKDWWKHKXPDQ RQ
XVHU FDQ HIIHFWLYHO\ FRQWURO DQ\ DSSOLFDWLRQ 7KLV SDSHU
SUHVHQWV XVHU LQWHUIDFHV GHYHORSPHQW EDVHG RQ WKH ;5 ವ 8VLQJ VKRUWFXWV IRU IUHTXHQWO\ XVHG IXQFWLRQDOLW\
SODWIRUPZLWKDIRFXVRQXVHLQDQHGXFDWLRQ7KHILUVWSDUW UHVSHFWLQJ GLIIHUHQW XVHU JURXSV  ZKHQ FUHDWLQJ
GHVFULEHV WKH W\SHV RI XVHU LQWHUIDFHV 7KH VHFRQG SDUW LQWHUIDFHVLWLVLPSRUWDQWWREHDZDUHRIXVHUQHHGVDQG
FRQWDLQV D GHVFULSWLRQ RI XVHU LQWHUIDFHV VXSSRUW DW WKH DGDSW WKHP DGHTXDWHO\ GLIIHUHQFHV PD\ RFFXU IRU
KDUGZDUHOHYHOHVSHFLDOO\DYDLODEOHDWWKHDXWKRUV¶KRPH H[DPSOH EHWZHHQ QRYLFHV DQG PRUH H[SHULHQFHG
ODERUDWRU\ /,5.,6 /DERUDWRU\ RI ,QWHOOLJHQW XVHUVEHWZHHQGLIIHUHQWDJHJURXSVEHWZHHQGLVDEOHG
LQWH5IDFHV RI &RPPXQLFDWLRQ DQG ,QIRUPDWLRQ SHRSOHRUEHWZHHQGLIIHUHQWQDWLRQDOLWLHV
6\VWHPV'HSDUWPHQWRI&RPSXWHUVDQG,QIRUPDWLFV ವ 3URYLGLQJ IHHGEDFN  IRU HDFK XVHU DFWLRQ IHHGEDFN
)((, 78 .RãLFH  ,Q WKH ODERUDWRU\ WKH ZRUNV DUH VKRXOG EH JHQHUDWHG LQ WKH V\VWHP VR WKDW WKH XVHU
FDUULHG RXW HQDEOLQJ WKH XVH RI 95 WHFKQRORJLHV WR NQRZVDERXWWKHVXFFHVVIXORUXQVXFFHVVIXOH[HFXWLRQ
IDFLOLWDWH VLPSOLI\ VSHHG XS DQG PDNH WUDQVSDUHQW RI WKH UHTXLUHG DFWLRQWKH PRUHUDUH H[HFXWLRQRI WKH
WKH FRPPXQLFDWLRQ RI VWXGHQWV DQG SXSLOV ZLWK DFWLRQWKHPRUHQRWLFHDEOHWKHIHHGEDFNVKRXOGEH
FRPSXWHUWHFKQRORJ\GXULQJWKHHGXFDWLRQ ವ 8VHUQDYLJDWLRQLIVHYHUDOVWHSVQHHGWREHWDNHQWR
FDUU\ RXW WKH RYHUDOO LQWHQWLRQ RI WKH XVHU V DFWLRQ
,, 86(5,17(5)$&(6$1'7+(,586$%,/,7< WKHVH VWHSV VKRXOG EH LQ D XQLIRUP VHTXHQFH DQG
7KLVVHFWLRQIRFXVHVRQVHYHUDOEDVLFUXOHVWKDWPXVWEH LQWXLWLYHO\FRQQHFWWRHDFKRWKHU
IROORZHGZKHQFUHDWLQJXVHULQWHUIDFHVLQRUGHUWRHQVXUH ವ $YRLGLQJ HUURUV  WKH V\VWHP VKRXOG EH GHVLJQHG VR
WKHVPRRWKQHVVRIWKHV\VWHPDSOHDVDQWXVHUH[SHULHQFH WKDWWKHXVHUFDQPDNHDVIHZHUURUVDVSRVVLEOHLIDQ
DQG WKDW WKH LQWHUIDFHV DUH IXQFWLRQDO DQG XVDEOH 7KHVH HUURU RFFXUV WKH XVHU VKRXOG EH RIIHUHG D TXLFN
UXOHV DUH GHVFULEHG E\ 1LHOVHQ >@ DQG UHSUHVHQW EDVLF UHFRYHU\RSWLRQ
SDUDPHWHUVWKDWPXVWEHIROORZHGZKHQFUHDWLQJLQWHUIDFHV
WR DFKLHYH DQ RSWLPDO XVHU H[SHULHQFH ,I WKH XVHU LV ವ $OORZLQJWKHUHWXUQRIDFWLRQVWKHV\VWHPVKRXOGEH
VDWLVILHGZLWKWKHFRQWURORIWKHSURGXFWRUV\VWHPDQGFDQ GHVLJQHG ZLWK WKH JUHDWHVW SRVVLEOH GHJUHH RI
HDVLO\ XQGHUVWDQG LW WKH SUREDELOLW\ WKDW KH ZLOO XVH WKH UHYHUVLELOLW\RIDFWLRQVVRWKDWWKHXVHUGRHVQRWJHWWKH
JLYHQV\VWHPLQWKHIXWXUHLQFUHDVHVKHZLOOEHZLOOLQJWR IHHOLQJWKDWKHSHUIRUPHGDQDFWLRQXQLQWHQWLRQDOO\DQG
SXUFKDVH LW RU UHFRPPHQG LW WR KLV IULHQGV 1LHOVHQ FDQQRWUHWXUQWKHV\VWHPWRLWVRULJLQDOVWDWH
GHILQHVWKHIROORZLQJDVSHFWVRILQWHUIDFHV ವ &UHDWLQJ SUHGLFWDEOH XVHU LQWHUIDFHV  WKH XVHU PXVW
ವ /HDUQDELOLW\WKHVSHHGZLWKZKLFKWKHXVHUFDQOHDUQ JHWWKHIHHOLQJWKDWKHLVWKHFRQWUROOLQJHOHPHQWRIWKH
WRFRQWUROWKHV\VWHP V\VWHP DQG KDV D FOHDU RYHUYLHZ RI WKH WDVNV EHLQJ
SHUIRUPHG WKH XVHU LQWHUIDFH PXVW EH UHVSRQVLYH WR
ವ (IILFLHQF\  WKH VSHHG ZLWK ZKLFK WKH XVHU FDQ XVHUDFWLRQV
FRPSOHWH D WDVN LQ WKH V\VWHP DIWHU KH KDV OHDUQHG WR
FRQWUROWKHV\VWHP ವ 5HGXFLQJ WKH ORDG RQ VKRUWWHUP PHPRU\  KXPDQ
DWWHQWLRQ DQG FRQFHQWUDWLRQ KDYH WKHLU OLPLWV VR LW LV
ವ 0HPRUDELOLW\  WKH GHJUHH WR ZKLFK WKH XVHU FDQ QHFHVVDU\ WR WDNH LQWR DFFRXQW WKH VLPSOLFLW\ RI WKH
UHPHPEHU KRZ WR FRQWURO WKH V\VWHP DQG KRZ KH LV XVHU LQWHUIDFH UHFRJQL]LQJ DQ\WKLQJ LV DOZD\V HDVLHU
DEOHWRFRQWUROLWLIKHUHWXUQVWRWKHV\VWHPDIWHUDORQJ WKDQ UHPHPEHULQJ VSHFLILFDOO\ EHFDXVH UHFRJQLWLRQ
WLPH 0HPRUDELOLW\ LV YHU\ LPSRUWDQW VR WKDW HYHQ LQYROYHVWKHSHUFHSWLRQRIVWLPXOLWKDWKHOSXVJHWLQWR
XVHUV ZKR GR QRW XVH WKH V\VWHP YHU\ RIWHQ ZLOO EH RXU PHPRU\ DQG HQDEOH UHOHYDQW LQIRUPDWLRQ WR
DEOH WR XVH LW ZLWKRXW KDYLQJ WR VWXG\ LW DJDLQ ,I WKH VXUIDFH
V\VWHP LV ZHOO GHVLJQHG WKH XVHU UHPHPEHUV WKH
FRQWUROORQJHUDQGLWLVPRUHLQWXLWLYHIRUKLP>@ ,,, +$5':$5(&21752/6)259,578$/5($/,7<'(9,&(6
ವ (UURUUDWHWKHQXPEHURIHUURUVWKDWWKHXVHUPDNHV :KHQ FRQWUROOLQJ REMHFWV LQ YLUWXDO UHDOLW\ LW LV
ZKHQ FRQWUROOLQJ WKH V\VWHP DQG WKH UDWH ZLWK ZKLFK DGYLVDEOHWRXVHH[WHUQDOKDUGZDUHGULYHUV+RZHYHUWKH\
KHFDQUHFRYHUIURPWKHP FDQFRQWUROREMHFWVLQGLIIHUHQWZD\VWKHLUSK\VLFDOOLPLWV
ವ 6DWLVIDFWLRQ  RQH RI WKH PRVW LPSRUWDQW IDFWRUV LW PXVWEHWDNHQLQWRDFFRXQWDQGXVHULQWHUIDFHVVKRXOGEH
DSSURSULDWHO\DGDSWHGWRWKHLUFDSDELOLWLHV
LQGLFDWHVKRZWKHXVHUOLNHVWKHV\VWHPLQJHQHUDODQG
KRZVDWLVILHGKHLVZLWKLW 2QHRIWKHODWHVWDSSURDFKHVWRXVHULQWHUIDFHFRQWUROLV
WKH XVH RI ((* VHQVLQJ RI WKH XVHU >@ ,Q WKH SDVW WKLV
2WKHU UXOHV IRU WKH FUHDWLRQ RI XVHU LQWHUIDFHV ZHUH
WHFKQRORJ\ ZDV QRW XVHG IRU GLUHFW FRQWURO RI FRPSXWLQJ
GHILQHG E\ %HQ 6FKQHLGHUPDQ >@ ZKLFK GHDO ZLWK WKH
WHFKQRORJ\ DQG LQ WKLV FRQWH[W LWV XVH LV OLWWOH H[SORUHG
FUHDWLRQRIVRIWZDUHWKDWZLOOEHFRQWUROOHGE\DKXPDQ,W
EXW LW KDV D JUHDW SRWHQWLDO %&, ± %UDLQ &RPSXWHU
LV DGYLVDEOH WR WKLQN DERXW WKHP GXULQJ WKH GHYHORSPHQW

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 470

,QWHUIDFH  (0* W\SH GHYLFHV DUH VLPLODU %XW (0* 7KLV GHYLFH LV VXLWDEOH IRU PL[HG UHDOLW\ FRQWURO DQG LV
VHQVRUV DUH VHQVRUV GHVLJQHG WR PHDVXUH PXVFOH WHQVLRQ FRPSDWLEOH ZLWK YDULRXV LPPHUVLYH KHOPHWV 7KH
YDOXHV HJ 0<2 $50EDQG EUDFHOHW DYDLODEOH LQ WKH DGYDQWDJH RI WKLV GHYLFH LV WKH FRQWURO RI REMHFWV DQG
/,5.,6 ODERUDWRU\ WRR )LJXUH   7KHVH VHQVRUV DUH VFHQHV ZLWK JHVWXUHV )LJXUH   WKH +0' GHYLFH FDQ
SODFHG RQ LQGLYLGXDO PXVFOHV DQG VXEVHTXHQWO\ SURYLGH DOZD\VVHHWKHORFDWLRQRIWKHGHYLFHEDVHGRQWKHPDUNV
LQIRUPDWLRQ RQ WKH WHQVLRQ RI LQGLYLGXDO PXVFOHV ,Q ,QWKHHYHQWWKDWWKHGHYLFHLVORVWIURPWKHGHYLFH V+0'
SUDFWLFH WKHVH VHQVRUV DUH PRVW RIWHQ XVHG WR UHFRJQL]H WKH GHYLFH V ODVW NQRZQ ORFDWLRQ DQG GDWD IURP WKH
XVHUJHVWXUHV6XFKGHYLFHVDUHPRVWRIWHQLQWKHEUDFHOHW J\URVFRSHV DQG DFFHOHURPHWHUV VHQW RYHU %OXHWRRWK DUH
VKDSH DQG FRQWDLQ VHQVRUV WKDW DUH GLVWULEXWHG HYHQO\ XVHG WR FDOFXODWH WKH GHYLFH V FXUUHQW ORFDWLRQ )URP WKH
DURXQGWKHXVHU VKDQGOHJ XVHU V SRLQW RI YLHZ WKLV GHYLFH KDV RSWLFDO SRVLWLRQ
WUDFNLQJ LH LW VHUYHV DV D PDUN PDUNHU  WULJJHU FURVV
MR\VWLFNDQGWRXFKSDG
+7& 9LYH &RQWUROOHU LV DOVR D ZLUHOHVV +0' FRQWURO
GHYLFHIURP+7&9,9(352,WFDQEHFRQWUROOHGXVLQJ
WKH WULJJHU WKH WRXFK VXUIDFH ZLWK D FLUFXODU PRYHPHQW
DQG D VHW RI EXWWRQV 2FXOXV 4XHVW  KDV WZR ZLUHOHVV
FRQWUROOHUVWKH\DUHVOLJKWO\ODUJHUDQGKHDYLHUWKDQLQWKH
SUHYLRXVYHUVLRQ7KH\FRQWDLQWZRWULJJHUVWKUHHEXWWRQV
DQGRQHFURVVOHYHUFRQWUROWKHLUOD\RXWLVHUJRQRPLFIRU
WKHXVHU:KHQKROGLQJWKHPWKHXVHUZLOOUDUHO\SUHVVWKH
EXWWRQ E\ PLVWDNH ,Q YLUWXDO FDYHV WKH XVHU XVXDOO\
LQWHUDFWV ZLWK WKH KHOS RI FRQWUROOHUV RU FRQWDFWOHVV
XVXDOO\WKURXJKRSWLFDOLPDJHUHFRJQLWLRQ HJ2SWLWUDFN 

6RPH KDUGZDUH ,2 HOHPHQWV RI WKH 95 V\VWHP IRU WKH
)LJXUH$VWXGHQWXVLQJ(0*EUDFHOHWLQ/,5.,6YLUWXDO&$9( 18,LQWHUIDFHDUHVKRZQLQ)LJXUH
HQYLURQPHQW


0RUHFRQYHQWLRQDOW\SHVRIGHYLFHVIRUFRQWUROOLQJXVHU
LQWHUIDFHV LQ YLUWXDO UHDOLW\ LQFOXGH YDULRXV KDUGZDUH
FRQWUROOHUV 7KH GLVDGYDQWDJH RI WKHVH FRQWUROOHUV LV WKDW
9LVXDOL]DWLRQ
RQH FRQWUROOHU LV LQWHQGHG IRU RQH YLUWXDO UHDOLW\ GHYLFH ORRVHO\ERXQG 
PRVWO\IRU95KHDGVHWDQGLWLVQRWSRVVLEOHWRH[FKDQJH 0XVFOH (\HWUDFNLQJ
LWIRUDQRWKHUFRQWUROOHUEHORQJLQJWRDQRWKHUGHYLFH7KH WHQVLRQ 
YDULRXV RSWLFDO 
VLPSOHVW FRQWURO GHYLFH LV WKH +ROR/HQV &OLFNHU IRU ELRVHQ]RULFDO 
18,
FRQWUROOLQJ WKH 0LFURVRIW +ROR/HQV 05 KHDGVHW :KHQ
XVLQJ WKLV GDWD KHOPHW LW ZDV IRXQG WKDW FRQWUROOLQJ WKH 95,2
VSDFH XVLQJ GLIIHUHQW KDQG JHVWXUHV LV WLULQJ IRU WKH XVHU GHYLFHV %RG\
((*GDWD WUDFNLQJ
VHQVLQJKDQGFOLFNVLVQRWFRPSOHWHO\DFFXUDWHDQGLWRIWHQ ELRVHQVRULFDO 
VHQ]RULFDO 
SURGXFHV HUURUV 7KDW LV ZK\ WKH DIRUHPHQWLRQHG FRQWURO
GHYLFH ZDV GHYHORSHG ZKLFK FRPPXQLFDWHV ZLWK WKH 9LVXDOL]DWLRQ
WLJKWO\ERXQG 
KHOPHW YLD %OXHWRRWK ,W LV FRQWUROOHG E\ SUHVVLQJ WKH
DFWLRQEXWWRQWRFOLFNLQVSDFHDQGWKURXJKWKHJ\URVFRSH
LWLVSRVVLEOHWRVLPXODWHWKHWLOWLQJRIWKHKDQGDQGFRQWURO
WKH WUDQVLWLRQV LQ WKH DSSOLFDWLRQ ZKLFK LQFUHDVHV WKH 
DFFXUDF\RIWKHFRQWURODQGHOLPLQDWHVWKHHUURUUDWHLQWKH
)LJXUH6RPHKDUGZDUH,2HOHPHQWVRIWKH95V\VWHPIRUWKH18,
FRQWUROVHQVLQJ LQWHUIDFH /,5.,6'&,)((,78.RãLFH 

,9 $33/,&$7,21$5($2)7+(;53/$7)250
$ VLJQLILFDQW DPRXQW RI UHVHDUFK KDV DOUHDG\ EHHQ
GHYRWHGWRXVHULQWHUIDFHVLQYLUWXDORUPL[HGUHDOLW\DQGLQ
JHQHUDORQWKH;5SODWIRUP)RUH[DPSOHDVWXG\E\/LV
0DUFLQLDN>@IRFXVHVRQWHVWLQJLQQRYDWLYHDSSURDFKHVLQ
FUHDWLQJ LQWHUIDFHV WKURXJK D JDPH FUHDWHG IRU PL[HG
UHDOLW\ ,W HPSKDVL]HV WLPHO\ YHULILFDWLRQ RI
LPSOHPHQWDWLRQV LQRUGHUWR DFKLHYH WKH JUHDWHVWSRVVLEOH
XVHU VDWLVIDFWLRQ ZLWK WKH LQWHUIDFH ,W LV GHGLFDWHG WR WKH
 GHVLJQ LPSOHPHQWDWLRQ DQG WHVWLQJ RI KRORJUDSKLF XVHU
LQWHUIDFHVLQPL[HGUHDOLW\DSSOLFDWLRQV
)LJXUH*HVWXUHFRQWUROOHDUQLQJZLWK06+ROROHQVPL[HGUHDOLW\
KHDGVHW LQ/,5.,6'&,)((,78.RãLFH  $Q LPSRUWDQW DVSHFW ZKHQ FUHDWLQJ DQ LQWHUIDFH IRU
PL[HG UHDOLW\ LV DOVR WKH FKRLFH RI FRORU GHVLJQ ,Q WKH
$QRWKHU VOLJKWO\ PRUH FRPSOLFDWHG GHYLFH IRU YLVLELOLW\WHVWUHGFRORUZDVHYDOXDWHGDVWKHPRVWYLVLEOH
FRQWUROOLQJ YLUWXDO UHDOLW\ HQYLURQPHQWV DOVR IURP WKH \HOORZ DV WKH OHDVW UHDGDEOH >@ )RU H[DPSOH WKH JDPH
0LFURVRIW ZRUNVKRS LV WKH 0RWLRQ &RQWUROOHUV GHYLFH 5RER5DLG LV UHQGHUHG PDLQO\ LQ EOXHUHG FRORUV ,Q
)UDJPHQWVQRHOHPHQWVKDYHDIL[HGSRVLWLRQHOHPHQWVLQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 471

EOXH DUH IXOO\ YLVLEOH DQG WH[W LV GLVSOD\HG LQ ZKLWH 7KH $ ODUJH ILHOG RI DSSOLFDWLRQ RI 95 LV LQGXVWU\ DQG WKH
\HOORZFRORULVQRWXVHGDVDVXSSRUWLQJHOHPHQWLWLVXVHG WUDLQLQJ SURFHVV ZKLFK LV DGGUHVVHG LQ WKH QH[W FKDSWHU
RQO\ IRU DGGLWLRQDO YLVLELOLW\ RI LQIRUPDWLRQ %DVHG RQ ,QGXVWULDOHQWHUSULVHVDQGFRPSDQLHVRIWHQLQWURGXFHQHZ
WKHVHOLPLWDWLRQVLWLVQHFHVVDU\WRWKLQNDERXWKRZDQGLQ SURFHGXUHV LQWR WKH SURGXFWLRQ SURFHVV DQG WU\ WR HQVXUH
ZKDWFRORUVLWLVDSSURSULDWHWRLPSOHPHQWDPL[HGUHDOLW\ WKDW HPSOR\HHV FDQ IROORZ WKHVH FKDQJHV DQG DGDSW WR
LQWHUIDFHVRWKDWLWLVUHDGDEOHHQRXJKDQGWKDWWKHFKRVHQ WKHP ,W LV IRU WKLV UHDVRQ WKDW PDQ\ YLUWXDO UHDOLW\
FRORUVGR QRW PDNH LW GLIILFXOW IRU WKH XVHU WRFRQWUROWKH VLPXODWRUV DUH EHLQJ FUHDWHG WKHVH GD\V WR KHOS ZRUNHUV
LQWHUIDFH VLQFH FXUUHQWO\ WKH KDUGZDUH IRU UHQGHULQJ OHDUQ QHZ ZRUN SURFHGXUHV LQ WKH SURWHFWHG DQG VDIH
PL[HGUHDOLW\VWLOOKDVJUHDWOLPLWDWLRQVDQGFDQQRWUHQGHU HQYLURQPHQW RI WUDLQLQJ FHQWHUV 6XFK VLPXODWLRQV
V\QWKHVL]HGREMHFWVSHUIHFWO\ XOWLPDWHO\VDYHWKHFRPSDQ\DORWRIPRQH\VLQFHWKH\GR
$QRWKHU UHVHDUFK >@ IRFXVHV RQ FRPSDULQJ PL[HG QRWKDYHWREX\DQGPDLQWDLQFHUWDLQPDFKLQHVVLQFHWKH\
UHDOLW\ XVHU H[SHULHQFHV LQ WKH DXWRPRWLYH LQGXVWU\ XVLQJ DUHUHSODFHGE\VLPXODWLRQDQGWRDFHUWDLQH[WHQWWKDQNV
PRELOH GHYLFHV DQG +ROR/HQV WR FUHDWH LQQRYDWLYH WR WKHP WKH\ SUHYHQW SURSHUW\ GDPDJH FDXVHG E\
VROXWLRQVLQWKHIXWXUH7KHSUREOHPRIWKLVLQGXVWU\LVWKH XQWUDLQHGRULQH[SHULHQFHGHPSOR\HHV7KHDSSOLFDWLRQRI
LQFUHDVLQJ FRPSOH[LW\ RI WKH WHFKQRORJLHV XVHG VRIWZDUH YLUWXDO UHDOLW\ LQ LQGXVWU\ PDNHV D ORW RI VHQVH IRU LWV
SURJUDPVDVZHOODVWKHFRPSOH[LW\RIWKHGHVLJQRIFDUV IXWXUH 0DQ\ WLPHV WKH PRVW PRQH\ LV LQYHVWHG LQ
'HVSLWHWKLVWKHQXPEHURIUHDOO\H[SHULHQFHGWHFKQLFLDQV UHVHDUFK SUHFLVHO\ LQ VXFK D SULYDWH VSKHUH WKDQNV WR
DQG H[SHUWV LQ WKH ILHOG LV ORZ 7KH UHVHDUFK SURSRVHV D ZKLFKLWLVSRVVLEOHWRILQDQFHWKHIXUWKHUDGYDQFHPHQWRI
VROXWLRQWRWKLVVLWXDWLRQE\SURMHFWLQJH[SHUWLQIRUPDWLRQ WHFKQRORJ\ 9LUWXDO UHDOLW\ WUDLQLQJ LV KHOSIXO LQ DOO WKH
DV DQ LQIRUPDWLRQDO YLUWXDO OD\HU RQWR UHDO REMHFWV LQ WKH DERYHPHQWLRQHG DUHDV WKH H[SHULHQFH DQG H[SHULHQFH
ZRUNVWDWLRQ WKURXJK WKH +ROR/HQV GHYLFH 7KLV VROXWLRQ JDLQHGIURPVXFKWUDLQLQJKDYHPRUHSRVLWLYHUHVXOWVWKDQ
FDQ DOVR VLPSOLI\ HPSOR\HH WUDLQLQJ LQFUHDVH VDIHW\ RQ XVLQJ PRUH RXWGDWHG PHWKRGV >@ DQG YLUWXDO UHDOLW\
SURGXFWLRQ OLQHV DQG KHOS WHFKQLFLDQV PDLQWDLQ FDUV LQ D PDNHVVHQVHWRXVHLQWKHVHLQGXVWULHV9LUWXDOUHDOLW\DOVR
PRUHLQWXLWLYHZD\%DVHGRQVWXGLHV>@DQG>@LWZDV RIIHUV PRGHOOLQJ RI OLIHWKUHDWHQLQJ VFHQHV LQ D VDIH
SURYHQWKDWDXJPHQWHGUHDOLW\FRQWULEXWHVSRVLWLYHO\WRWKH HQYLURQPHQWLQFOXGLQJGHSOR\PHQWLQVFKRROV,WIRFXVHV
OHDUQLQJ SURFHVV 7KH DLP RI DOO WKH PHQWLRQHG VWXGLHV RQ D SUDFWLFDO DSSURDFK DQG QRW RQO\ RQ WKHRUHWLFDO
UHJDUGOHVVRIWKHILHOGRIGHSOR\PHQWZDVWRVKRZWKDWWKH NQRZOHGJH6WXGHQWVZKRDUHWUDLQHGXVLQJ95FDQEHWWHU
GHSOR\PHQW RI ;5 WHFKQRORJLHV FDQ EH EHQHILFLDO LQ WKH XQGHUVWDQG WKH SUHVHQWHG LVVXH 8VLQJ WKH ILUVWSHUVRQ
HGXFDWLRQ RI SHRSOH DQG FDQ DOVR UHGXFH WKH FRVWV RI WKH YLHZLQ95DOVRKDVDELJLPSDFWRQXVHUEHKDYLRXU,WFDQ
VXVWDLQDELOLW\RIWKLVW\SHRIWHDFKLQJ KHOSKLPJHWXVHGWRWKHZD\WDVNVDUHSHUIRUPHGLQDUHDO
HQYLURQPHQW>@DVKHKDVWKHH[SHULHQFHRIKDYLQJWKH
&RQVLGHULQJ WKH IDFW WKDW WKH UHVHDUFK ILHOG RI ;5
SHUVSHFWLYH RI SHUIRUPLQJ WKH WDVN GLUHFWO\ IURP KLV
SODWIRUP DSSOLFDWLRQV LV YDVW LW LV SRVVLEOH WR GLYLGH WKLV
SHUVSHFWLYH $W WKH VDPH WLPH LW LV DOZD\V QHFHVVDU\ WR
ILHOG LQWR ODUJHU XQLWV WDNLQJ LQWR DFFRXQW WKH FRPPRQ
SXUSRVHRIXVHQDPHO\LQ GHVLJQ D JRRG XVHU LQWHUIDFH RI WKH V\VWHP VR WKDW WKH
YLUWXDOUHDOLW\H[SHULHQFHLVIXOOIOHGJHGDQGXVHUVKDYHWKH
ವ 0HGLFLQH KHDOWK FDUH   IRU H[DPSOH YLUWXDO UHDOLW\ PRVWDFFXUDWHH[SHULHQFHDQGLQIRUPDWLRQ
V\VWHPV WKDW KDYH IRXQG DSSOLFDWLRQ LQ VXUJLFDO
LQWHUYHQWLRQV UHKDELOLWDWLRQ >@ RU WUHDWPHQW RI 9 &21&/86,21
SKRELDV
7KH DUWLFOH SUHVHQWHG D YLHZ RI WKH XVHU LQWHUIDFHV
ವ ,QGXVWU\  WKH ILHOG RI YLUWXDO UHDOLW\ WKDW LV XVHG IRU FUHDWLRQ EDVHG RQ WKH ;5 SODWIRUP ZLWK DQ RULHQWDWLRQ
FRPPHUFLDOSXUSRVHV WRZDUGV SRVVLEOH XVH LQ VWXGHQW HGXFDWLRQ 7KH XVHU
ವ 7UDLQLQJDQGGLUHFWHGXFDWLRQIRUH[DPSOHWUDLQLQJ LQWHUIDFH LV H[WUHPHO\ LPSRUWDQW IURP D KXPDQXVHU V
SHRSOH WR PDVWHU VSHFLILF WDVNV RU IRU VWXGHQWV SRLQW RI YLHZ WR EH DEOH WR HIIHFWLYHO\ FRQWURO DQ\
HGXFDWLRQ )LJXUH  DSSOLFDWLRQ )LUVW ZH SRLQWHG RXW WKH W\SHV RI XVHU
LQWHUIDFHV 6XEVHTXHQWO\ ZH GHDOW ZLWK WKH VXSSRUW RI
ವ 2WKHUDUHDVIRUH[DPSOH95V\VWHPVIRUUHOD[DWLRQ XVHULQWHUIDFHVDWWKHOHYHORIWHFKQLFDOPHDQVHVSHFLDOO\
HQWHUWDLQPHQW >@ DQG RWKHU VSHFLILF YLUWXDO UHDOLW\
WKRVHWKDWDUHDYDLODEOHDWWKHDXWKRUV ZRUNSODFH /,5.,6
UHVHDUFK
/DERUDWRU\ .3,)(,78.RãLFH  ZKHUH ZRUN LVFDUULHG
RXW WKDW FRXOG EH GRQH XVLQJ YLUWXDO UHDOLW\ WHFKQRORJLHV
LQ WKH FRQWH[W RI KXPDQWRKXPDQ FRPPXQLFDWLRQ
FRPSXWHU WR IDFLOLWDWH VLPSOLI\ VSHHG XS DQG PDNH
WUDQVSDUHQW WKH FRPPXQLFDWLRQ RI VWXGHQWV DQG SXSLOV
ZLWK FRPSXWLQJ WHFKQRORJ\ DV ZHOO DV ZLWK WKHLU
VXUURXQGLQJV ,Q IXWXUH ZRUN ZH ZLOO IRFXV RQ WKH
GHYHORSPHQW RI GLJLWDO WZLQV RI GHYLFHV WR PDNH 8,
GHYHORSPHQWHYHQIDVWHUVLPSOHUDQGPRUHIOH[LEOH

$&.12:/('*(0(17
7KLVZRUNKDVEHHQVXSSRUWHGE\WKH.(*$JUDQW1R
 78.( Ä&ROODERUDWLYH YLUWXDO UHDOLW\
WHFKQRORJLHVLQWKHHGXFDWLRQDOSURFHVV³
)LJXUH&RJQLWLYHO\GLVDEOHGSXSLOZRUNLQJLQYLUWXDOHGXFDWLRQDO
HQYLURQPHQWXVLQJ95KHDGVHW LQ/,5.,6'&,)((,78.RãLFH 

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 472

 5()(5(1&(6
>@ 6XDUH])HUQDQGH]5$6DQFKH]/RSH]-/6DPSHGUR&%DYOH
+0ROLQD0DQG&DPSR\31DWXUDOXVHULQWHUIDFHVIRUKXPDQ
GURQH PXOWLPRGDO LQWHUDFWLRQ ,Q  ,QWHUQDWLRQDO &RQIHUHQFH
RQ 8QPDQQHG $LUFUDIW 6\VWHPV ,&8$6  SDJHV ±
,(((
>@ =KRX - 6XQ - &KHQ ) :DQJ < 7DLE 5 .KDZDML $ DQG /L
=K0HDVXUDEOHGHFLVLRQPDNLQJZLWK*65DQGSXSLOODU\DQDO\VLV
IRU LQWHOOLJHQW XVHU LQWHUIDFH $&0 7UDQVDFWLRQV RQ &RPSXWHU
+XPDQ,QWHUDFWLRQ 7R&+,   ±
>@ 1LHOVHQ - 8VDELOLW\  ,QWURGXFWLRQ WR XVDELOLW\   85/
KWWSZZZQQJURXSFRPDUWLFOHVXVDELOLW\LQWURGXFWLRQ
WRXVDELOLW\>$FFHVVHG1RYHPEHU@
>@ )HUUp;-XULVWR1:LQGO+DQG&RQVWDQWLQH/8VDELOLW\EDVLFV
IRUVRIWZDUHGHYHORSHUV,(((VRIWZDUH  ±
>@ 6KQHLGHUPDQ%3ODLVDQW&&RKHQ06-DFREV6(OPTYLVW1
DQG 'LDNRSRXORV 1 'HVLJQLQJ WKH XVHU LQWHUIDFH VWUDWHJLHV IRU
HIIHFWLYHKXPDQFRPSXWHULQWHUDFWLRQ3HDUVRQ
>@ 3DVWRUQLFNê36LYê0DQG+XGiN0$GYDQFHGPHWKRGVRIGDWD
YLVXDOL]DWLRQEDVHGRQ((*VHQVRUV,Q9LUWXDOUHDOLW\WHFKQRORJLHV
LQWKHSURFHVVRIHGXFDWLRQRIKDQGLFDSSHGSHUVRQVSS±

>@ /LV0DUFLQLDN $ 7RPLDNRZVNL - DQG .DSXVWD 3 'HVLJQ UXOHV
LPSOHPHQWDWLRQ DQG WHVWLQJ RI XVHU LQWHUIDFHV IRU PL[HG UHDOLW\
DSSOLFDWLRQV ,Q  WK ,QWHUQDWLRQDO &RQIHUHQFH RQ +XPDQ
6\VWHP,QWHUDFWLRQ +6, SDJHV±,(((
>@ +HR 0+ /HH - 7UDQ 97 /HH (< .LP 0 DQG .LP '
&RPSDULVRQ RI XVHU H[SHULHQFHV RI DXJPHQWHG UHDOLW\ EDVHG FDU
PDLQWHQDQFH FRQWHQWV XVLQJ PRELOH GHYLFHV DQG +ROR/HQV $VLD
SDFLILF -RXUQDO RI &RQYHUJHQW 5HVHDUFK ,QWHUFKDQJH   ±

>@ :X+. /HH :HQ<X6&KDQJ+<DQG/LDQJ-&K&XUUHQW
VWDWXV RSSRUWXQLWLHV DQG FKDOOHQJHV RI DXJPHQWHG UHDOLW\ LQ
HGXFDWLRQ&RPSXWHUV (GXFDWLRQ±
>@ 0DUWtQ*XWLpUUH]-)DELDQL3%HQHãRYi:0HQHVHV0'DQG
0RUD &( $XJPHQWHG UHDOLW\ WR SURPRWH FROODERUDWLYH DQG
DXWRQRPRXV OHDUQLQJ LQ KLJKHU HGXFDWLRQ &RPSXWHUV LQ KXPDQ
EHKDYLRU±
>@ 8 &KDXGKDU\ 1 %LUEDXPHU DQG $ 5DPRV0XUJXLDOGD\
³%UDLQFRPSXWHULQWHUIDFHVIRUFRPPXQLFDWLRQDQGUHKDELOLWDWLRQ´
1DW5HY1HXUROYYROSS±
>@ -DQJ - DQG %HGQDU] 7 +RORVHQVRU IRU VPDUW KRPH KHDOWK
HQWHUWDLQPHQW,Q$&06,**5$3+$SS\+RXUSDJHV±

>@ .XQMDO $ .DMDO * 5XWYLN * DQG 0DQDQ 6K $SSOLFDWLRQ RQ
YLUWXDO UHDOLW\ IRU HQKDQFHG HGXFDWLRQ OHDUQLQJ PLOLWDU\ WUDLQLQJ
DQGVSRUWV$XJPHQWHG+XPDQ5HVHDUFK  ±
>@ *DUHWK:9LUWXDOUHDOLW\H[SRVXUHWKHUDS\7REHQHILWWKRVHZKR
VWXWWHU DQG WUHDW VRFLDO DQ[LHW\ ,Q  ,QWHUQDWLRQDO &RQIHUHQFH
RQ ,QWHUDFWLYH 7HFKQRORJLHV DQG *DPHV ,7$*  SDJHV ±
,(((

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 473

 3RVVLEOHXVHRIYLUWXDOUHDOLW\LQWKHWKHUDSLVW
SDWLHQWLQWHUDFWLRQ

%6RERWD -0LĖRYi â.RUHþNR 00DWWRYi :6WHLQJDUWQHU 
'HSDUWPHQWRI&RPSXWHUVDQG,QIRUPDWLFV)DFXOW\RI(OHFWULFDO(QJLQHHULQJDQG,QIRUPDWLFV7HFKQLFDO8QLYHUVLW\RI
.RãLFH6ORYDN5HSXEOLF
EUDQLVODYVRERWD#WXNHVNVWHIDQNRUHFNR#WXNHVNPLULDPDPDWWRYD#WXNHVNZLOOLDPVWHLQJDUWQHU#WXNHVN

$EVWUDFW ² 7KLV SDSHU GHVFULEHV LQ VKRUW WKH WKHUDSHXWLF 7KH DLP RI WKLV DUWLFOH LV WR GHVFULEH LQ VKRUW D
V\VWHPLPSOHPHQWHGLQYLUWXDOUHDOLW\7KHSULPDU\IRFXVRI WKHUDSHXWLF V\VWHP EDVHG RQ YLUWXDO UHDOLW\ WHFKQRORJLHV
WKH SDSHU LV WR WKH GHVFULSWLRQ RI WKH IXQFWLRQDOLW\ RI IRU WKH UHKDELOLWDWLRQ RI WKH XSSHU OLPE ZKLFK ZDV
DQLPDWLRQ VHWWLQJV ZLWKLQ WKH SDUDPHWHUV RI GHYHORSHG LQ FRRSHUDWLRQ ZLWK WKH 6ORYDN $FDGHP\ RI
LQWHUDFWLRQUHKDELOLWDWLRQ YLVXDOL]DWLRQ DQG
6FLHQFHV ,QVWLWXWHRI0HDVXUHPHQW6FLHQFH 
IXQFWLRQDOL]DWLRQRIWKHV\VWHP:LWKWKLVV\VWHPWKHSDWLHQW
FDQ EH VWLPXODWHG XQGHU WKH VXSHUYLVLRQ RI D WKHUDSLVW WR ,, (;,67,1*6<67(0
SUDFWLFH VRPH UHKDELOLWDWLRQWUDLQLQJ SURFHGXUHV ,Q
DGGLWLRQWKHWKHUDSLVWFDQGHPRQVWUDWHWKHGHVLUHGDFWLRQWR 7KH WKHUDSHXWLF V\VWHP GHVFULEHG LQ WKLV DUWLFOH LV
WKH SDWLHQW 'XH WR WKH XVH RI FROODERUDWLYH YLUWXDO UHDOLW\ EDVHGRQDQDOUHDG\H[LVWLQJV\VWHP>@ZKLFKZDVDOVR
WKLVSURFHVVFDQEHDOVRDSSOLHGLQDGLVWDQFHIRUPDQGWKXV GHYHORSHG LQ FRRSHUDWLRQ ZLWK WKH 6ORYDN $FDGHP\ RI
LWH[SDQGVWKHWKHUDSLVWSDWLHQWLQWHUDFWLRQ 6FLHQFHV 7KH H[LVWLQJ V\VWHP LQ XVH LV D FROODERUDWLYH
95 ZHEDSSOLFDWLRQ ZLWK DQ LQWHUIDFH IRU ERWK WKHUDSLVW
.H\ZRUGV²YLUWXDOUHDOLW\UHKDELOLWDWLRQWKHUDSLVWSDWLHQW
DQG SDWLHQW WKDW FDQ EH UXQ RQ GHYLFHV ZLWK D VXSSRUWHG
LQWHUDFWLRQ
ZHE EURZVHU RU 95 KHDGVHW 7KH UHKDELOLWDWLRQ SURFHVV
ZLWKLQWKHV\VWHPFRQVLVWVRIDVHTXHQFHRIVFHQDULRVZLWK
, ,1752'8&7,21 DQLPDWLRQV ZKLFK DUH DLPHG DW VXSSRUWLQJ WKH
6LJQLILFDQW LQFUHDVH LQ WKH KXPDQFRPSXWHU LQWHUDFWLRQV FRRUGLQDWLRQ RI PRYHPHQW RI WKH XSSHU OLPEV 7KH
+&, EDVHGRQWKHEUDLQFRPSXWHULQWHUIDFH %&, SULQFLSOH DQLPDWLRQVFRQVLVWLQWKHPRYHPHQWRIWKHSDWLHQW VKDQG
>@FDQEHREVHUYHGLQWKHODVWGHFDGHV7KHGHYHORSPHQWRI LQWKHYLUWXDOHQYLURQPHQWDQGLWVLQWHUDFWLRQZLWKYDULRXV
QHZ WHFKQRORJLHV HQDEOHV LWV DSSOLFDWLRQ LQ DUHDV ZKHUH LW REMHFWVVXFKDVDFXS%HWZHHQWKHLQGLYLGXDOVFHQDULRV
ZDV SUHYLRXVO\ GLIILFXOW RU HYHQ LPSRVVLEOH >@>@ 9LUWXDO WKHUHLVDGHVLJQDWHGWLPHIRUWKHSDWLHQWWRUHOD[
UHDOLW\LVFOHDUO\DYHU\SURJUHVVLYHDSSOLFDEOHWHFKQRORJ\LQ 6FHQDULRVFDQHQGHLWKHUDIWHUWLPHKDVHODSVHGRUDIWHU
WKLV FDVH 7KH GHYHORSPHQW RI YLUWXDO UHDOLW\ 95  DQG WKH DQLPDWLRQ KDV EHHQ H[HFXWHG ,Q RUGHU WR WULJJHU WKH
UHODWHGWHFKQRORJLHVPRYHG%&,IRUZDUGE\FUHDWLQJIOH[LEOH DQLPDWLRQ WKH SDWLHQW PXVW FRUUHFWO\ LPDJLQH WKH
HQYLURQPHQWV ZKHUH +&, FDQ WDNH SODFH DQG EH PRGLILHG PRYHPHQW DQG WKLV FRQGLWLRQ LV YHULILHG XVLQJ WKH
+HDGPRXQWHG 95 GLVSOD\V +0'  PRYH VWDQGDUG ' SDWLHQW VEUDLQDFWLYLW\,IWKHUHDUHFKDQJHVLQWKHSDWLHQW V
VFUHHQEDVHGYLVXDOL]DWLRQLQWRDYLUWXDO'ZRUOG HJ>@  EUDLQDFWLYLW\DVSHFLILFDQLPDWLRQLVWULJJHUHG
)RUH[DPSOHWKHUHLVDQLQFUHDVLQJQXPEHURIZRUNVRQWKH $ 0HDVXUHPHQWRIEUDLQDFWLYLW\
SRVLWLYHLPSDFWRISRVWVWURNHWUDLQLQJHLWKHUEDVHGRQDFWLRQ
$Q((*KHOPHWRUJ1DXWLOXV352)/(;,%/(FDSLV
REVHUYDWLRQ PLUURUER[WKHUDS\>@RUDFWLYHPRWRULPDJHU\
XVHG WR PHDVXUH EUDLQ DFWLYLW\ ZKHQ XVLQJ WKH V\VWHP
>@ >@  $Q HVVHQWLDO HOHPHQW RI VXFK WUDLQLQJ LV LWV
7KLV GHYLFH KDV WKH RSWLRQ RI FRQQHFWLQJ WR D FRPSXWHU
DWWUDFWLYHQHVV DQG LQWHUHVW WR SDWLHQWV 7KH\ DUH FORVHO\
YLD D 86% FDEOH ZKLFK DOORZV WKH PHDVXUHG DFWLYLW\ WR
UHODWHG WR SDWLHQWV¶ DFFHSWDELOLW\ DQG XVDELOLW\ RI VXFK
EHVHQWWRWKHFRPSXWHU)RUWKHFRUUHFWLQWHUSUHWDWLRQRI
WUDLQLQJ FRQFHSWV LQ HYHU\GD\ OLIH $QG WKLV LV ZKHUH 95
WKHUHVXOWVDQGHQVXULQJFRPPXQLFDWLRQZLWKWKHV\VWHP
FRPHV LQ KDQG\ ZLWK LWV JUHDW IOH[LELOLW\ 7KLV DOORZV WKH
DQLQWHUIDFHLPSOHPHQWHGWKURXJKWKH2SHQ9LEHSODWIRUP
FUHDWLRQRIDWUDLQLQJHQYLURQPHQWWKDWFDQEHHDVLO\DGDSWHG
LVXVHG
DQG FKDQJHG DFFRUGLQJWR WKH SDWLHQW V QHHGVDQGLPSURYHV
WKHWKHUDSLVWSDWLHQWLQWHUDFWLRQ 73,  ,,, '(6&5,37,212)7+(,03/(0(17('6<67(0
:LWK JURZLQJ LQWHUHVW LQ WKH XVH RI YLUWXDO UHDOLW\ LQ
7KH SUHVHQWHG V\VWHP ZDV GHVLJQHG DQG LPSOHPHQWHG
PHGLFLQH WKHUDSLVWV DQG UHKDELOLWDWLRQ SURIHVVLRQDOV DUH
LQVXFKDZD\WKDWLWZDVEDVHGRQWKHH[LVWLQJV\VWHPDQG
WXUQLQJ WR WKLV WHFKQRORJ\ WR LPSURYH WKH HIIHFWLYHQHVV RI
DW WKH VDPH WLPH PHW WKH UHTXLUHPHQWV REWDLQHG E\ WKH
UHKDELOLWDWLRQ DSSURDFKHV $Q LQWHUHVWLQJ LV WKH OLPE
6ORYDN $FDGHP\ RI 6FLHQFHV ,Q WKH FDVH RI WKH QHZ
UHKDELOLWDWLRQSURFHVVGHVFULEHGLQ>@7KLVLVLPSOHPHQWHG
V\VWHP LW LV QR ORQJHU D FROODERUDWLYH ZHE 95
XVLQJ +0'V ZLWK RWKHU HTXLSPHQW EDVHG RQ UHKDELOLWDWLRQ
DSSOLFDWLRQ EXW D 95 DSSOLFDWLRQ ZLWK D FROODERUDWLYH
QHHGV >@ $OVR DV GHVFULEHG LQ >@>@ WKH 95 XVDJH
HQYLURQPHQW LPSOHPHQWHG XVLQJ 8QLW\ WHFKQRORJ\ 7KH
DOORZVWRVLPXODWHGLIIHUHQWYLUWXDOKDQGVWDWHVDQGHVSHFLDOO\
DSSOLFDWLRQFDQEHUXQSULPDULO\RQ2FXOXV 0HWD 4XHVW
WRYLVXDOL]HGLIIHUHQWVLWXDWLRQVDVVRFLDWHGZLWKLW
DQG2FXOXV 0HWD 4XHVWGHYLFHV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 474

 7KHYLUWXDOHQYLURQPHQWRIWKHV\VWHPZDVGHVLJQHGWR ,9 7+(352&(662)$5(+$%,/,7$7,21
YLVXDOO\UHVHPEOHDSOHDVDQWURRPDVFORVHDVSRVVLEOHWR ,Q WKH ILJXUH )LJ  LW LV SRVVLEOH WR VHH D GLDJUDP RI
UHDOLW\ZLWKDWDEOHDQGWZRFKDLUV7KHWDEOHOLNHLQWKH DFWLYLWLHV GHVFULELQJ WKH SRVVLEOH FRXUVH RI UHKDELOLWDWLRQ
RULJLQDO V\VWHP UHSUHVHQWV WKH VSDFH RQ ZKLFK REMHFWV LQWKHGHVFULEHGV\VWHP%\FRQQHFWLQJWKHWKHUDSLVWDQG
DSSHDUDQGPRYHGXULQJUHKDELOLWDWLRQ7KHFKDLUVSODFHG WKHSDWLHQWWRWKHV\VWHPWKH\ERWK ILQGWKHPVHOYHVLQ D
RSSRVLWH HDFK RWKHU EHKLQG WKH WDEOH DUH UHVHUYHG SODFHV YLUWXDOURRPDFURVVIURPHDFKRWKHUDWDWDEOHVLWWLQJRQD
ZKHUH WKH WKHUDSLVW DQG WKH SDWLHQW ZLOO EH DIWHU WKH FKDLU)URPKLVLQWHUIDFHWKHWKHUDSLVWVHWVWKHWDEOHWRDQ
FRQQHFWLRQ7KHWKHUDSLVWDQGWKHSDWLHQWDUHUHSUHVHQWHG DSSURSULDWHKHLJKWLQUHODWLRQWRWKHSDWLHQWDQGVHOHFWVWKH
LQWKHHQYLURQPHQWLQWKHIRUPRIDYDWDUV%XWLQWKHFDVH REMHFW ZLWK ZKLFK KH ZDQWV WR VKRZ WKH SDWLHQW WKH
RIDYDWDUVLWZDVQHFHVVDU\WRHQVXUHWKDWWKH\UHVHPEOHG PRYHPHQW7KHVHOHFWHGREMHFWLVWKHQVKRZQRQWKHWDEOH
DSHUVRQDVPXFKDVSRVVLEOHDQGGLGQRWORRNWRRURERWLF DQGWKHWKHUDSLVWPRYHVLWWRDUHVHUYHGSODFHRQWKHWDEOH
$ 7KHUDSLVWSDWLHQWLQWHUIDFH XVLQJWKH95FRQWUROOHU
7KH WKHUDSLVW LV UHVSRQVLEOH IRU WKH FRXUVH RI
UHKDELOLWDWLRQ LQ WKH V\VWHP HQYLURQPHQW 'XULQJ WKH
UHKDELOLWDWLRQ KH LV LQ FKDUJH RI VKRZLQJ WKH SDWLHQW D &RQQHFWLQJWKHWKHUDSLVW
DQGWKHSDWLHQWWRWKH

KDQG PRYHPHQW WKDW WKH SDWLHQW KDV WR LPDJLQH DQG V\VWHP

ZKLFKLQFDVHRIVXFFHVVWKHKDQGRIWKHSDWLHQW VDYDWDU
7DEOHVHWWLQJ
ZLOO WKHQ SHUIRUP 7KH WKHUDSLVW KDV VHYHUDO REMHFWV WR
FKRRVH IURP DQG DIWHU FKRRVLQJ RQH RI WKHP WKH REMHFW
DSSHDUV RQ WKH WDEOH RQ WKH WKHUDSLVW V VLGH DQG WKH 2EMHFWVHOHFWLRQ

WKHUDSLVWFDQPRYHLWWRSUHUHVHUYHGSODFHVRQWKHWDEOH
$IWHU SHUIRUPLQJ WKH PRYHPHQW DQG PRYLQJ WKH REMHFW 7KHWKHUDSLVWPRYHVWKH
REMHFWRQWKHWDEOH
WKHWKHUDSLVWKDVWKHRSWLRQWRVWDUWDWLPHULQIRUPLQJWKH
SDWLHQWKRZPXFKWLPHKHKDVWRLPDJLQHWKHPRYHPHQW 6WDUWLQJWKHWLPHUDQG
GLVSOD\LQJWKHREMHFWLQ
'XULQJWKLVWLPHWKHV\VWHPZDLWVIRUDQLPSXOVHZKHWKHU WKHSDWLHQW VYLHZ

WKHLGHDRIPRYHPHQWZDVVXFFHVVIXOZKLOHWKHWKHUDSLVW
KLPVHOIFDQVHQGWKLVLPSXOVHWRWKHV\VWHP,QDGGLWLRQWR 'LGWKHSDWLHQW
LPDJLQHWKH 
WKHVHDFWLRQVWKHWKHUDSLVWFDQDOVRIRUH[DPSOHVHWWKH REMHFW

KHLJKWRIWKHWDEOHRUVHWWKHOHQJWKRIWLPHWRLPDJLQHWKH PRYHPHQW"

PRYHPHQW  ,VWKHWLPHU 
UXQQLQJ"
7KH SDWLHQW GRHV QRW KDYH WKH RSSRUWXQLW\ WR SHUIRUP ,PSXOVHWRSHUIRUPD
PRYHPHQW
DQ\ DFWLRQV ZLWKLQ WKH HQYLURQPHQW 'XULQJ 
UHKDELOLWDWLRQ LW PRQLWRUV WKH FKDQJHV PDGH LQ WKH
6WRSSLQJWKHWLPHU
HQYLURQPHQW VXFK DV WKH PRYHPHQW RI WKH WKHUDSLVW V
KDQGZKHQPRYLQJDQREMHFWRUWKHPRYHPHQWRILWVRZQ
KDQG LQ WKH HQYLURQPHQW $IWHU WKH WKHUDSLVW VWDUWV WKH 'LVSOD\LQIRUPDWLRQ
DERXWWKHVXFFHVV
'LVSOD\LQIRUPDWLRQ
DERXWWKHIDLOXUH

WLPHUWKHSDWLHQWKDVWKHRSSRUWXQLW\WRZDWFKWKHWLPHRQ
WKH WLPHU ZKLOH DQ REMHFW LV GLVSOD\HG LQ IURQW RI KLP 'LVSOD\LQJWKH
PRYHPHQWRIWKH

ZKLFK KH KDV WR LPDJLQH PRYLQJ 3DUW RI WKH V\VWHP LV SDWLHQW VYLUWXDOKDQG

DOVR WKH GLVSOD\ RI LQIRUPDWLRQ WR WKH SDWLHQW DERXW

ZKHWKHUKHZDVVXFFHVVIXOLQLPDJLQLQJWKHPRYHPHQWRU 
QRW
)LJ'LDJUDPRIDFWLYLWLHVIRUWKHUHKDELOLWDWLRQSURFHVV
% &RQQHFWLRQWRWKHHQYLURQPHQW
,W LV SRVVLEOH WR FRQQHFW WR WKH V\VWHP HQYLURQPHQW $IWHU SODFLQJ WKH REMHFW WKH WKHUDSLVW DVNV WKH SDWLHQW
ZLWKLQWKHVDPH/$1QHWZRUNDVUHKDELOLWDWLRQXVLQJWKLV WRLPDJLQHWKHPRYHPHQWRIWKHKDQGDQGVWDUWVWKHWLPHU
V\VWHP WDNHV SODFH RQVLWH 7KLV PHDQV WKDW GXULQJ WKH ,I WKH SDWLHQW GRHV QRW LPDJLQH WKH PRYHPHQW EHIRUH
UHKDELOLWDWLRQWKHWKHUDSLVWDQGWKHSDWLHQWDUHLQWKHVDPH WKH WLPHU H[SLUHV WKH SDWLHQW LV LQIRUPHG DERXW LW LQ WKH
URRP DQG WKXV WKH\ FDQ FRPPXQLFDWH ZLWK HDFK RWKHU V\VWHP DQG WKH PRYHPHQW LV QRW SHUIRUPHG ,I RQ WKH
RXWVLGHWKHV\VWHPDWWKHVDPHWLPH RWKHU KDQG WKH SDWLHQW LPDJLQHV WKH PRYHPHQW DQG LW LV
& &RPPXQLFDWLRQZLWK2SHQ9LEH UHIOHFWHG LQ KLV EUDLQ DFWLYLW\ DQ LPSXOVH LV VHQW WR WKH
V\VWHP ZKLFKVWDUWVWKHPRYHPHQWRIWKHSDWLHQW VKDQG
7KHLPSXOVHWRVWDUWWKHPRYHPHQWRIWKHSDWLHQW VKDQG $IWHU WKH DUULYDO RI WKH LPSXOVH WKH WLPHU VWRSV
FDQEHVHQWWRWKHV\VWHPLQWZRZD\VQDPHO\ LQIRUPDWLRQ DERXW WKH VXFFHVV LV GLVSOD\HG DQG WKH
• 2SHQ9LEHVHQGVDQLPSXOVHWRWKHV\VWHP PRYHPHQWRIWKHSDWLHQW VKDQGLVVWDUWHG
• 7KHLPSXOVHZLOOFRPHIURPWKHWKHUDSLVW
7KH V\VWHP LV WKHUHIRUH DEOH WR FDSWXUH LPSXOVHV 9 ,03/(0(17$7,21
UHVSHFWLYHO\+773PHVVDJHVFRPLQJIURP2SHQ9LEH $V PHQWLRQHG WKH V\VWHP ZDV LPSOHPHQWHG XVLQJ
8QLW\WHFKQRORJ\7KLVWHFKQRORJ\DOORZHGVRPHDVSHFWV
RI WKH V\VWHP WR EH LPSOHPHQWHG PRUH HDVLO\ VXFK DV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 475

LQYHUVH NLQHPDWLFV RU WKH XVHU LQWHUIDFH 6HYHUDO 8QLW\ LQYHUVH NLQHPDWLFV ZDV XVHG LQ WKH LPSOHPHQWDWLRQ
SDFNDJHV DQG OLEUDULHV ZHUH XVHG GXULQJ WKH ZKLFK HQVXUHG WKDW WKH HQWLUH KDQG RI WKH DYDWDU PRYHG
LPSOHPHQWDWLRQ LQFOXGLQJ 1HWFRGH ;5 ,QWHUDFWLRQ DQGURWDWHGDFFRUGLQJWRKRZWKHZULVWRIWKHKDQGPRYHV
7RRONLWDQG;53OXJLQ0DQDJHPHQW DQG PRYHV ,Q WKH FDVH RI WKH WKHUDSLVW V DYDWDU LW ZDV
DOVR LPSOHPHQWHG WKDW KLV KDQGV DQG KHDG PRYH
$ 8VHU,QWHUIDFH
DFFRUGLQJWRKRZWKHWKHUDSLVWPRYHVWKH95FRQWUROOHUV
7R LPSOHPHQW WKH XVHU LQWHUIDFH D FDQYDV ZDV XVHG DQGWKHFDPHUD7KHSDWLHQWFDQRQO\PRYHWKHFDPHUDLQ
ZKLFK UHSUHVHQWV DQ DUHD ZLWK 8QLW\ XVHU LQWHUIDFH WKHHQYLURQPHQWDVKHGRHVQRWQHHGWKHFRQWUROVGXULQJ
HOHPHQWV VXFKDVDEXWWRQRUDWH[W ILHOG7ZRFDQYDVHV UHKDELOLWDWLRQ
ZHUH XVHG LQ WKH HQYLURQPHQW )LJ   2QH LV XVHG WR 7KH PRYHPHQW RI WKH KDQG LQ WKH FDVH RI WKH SDWLHQW
GLVSOD\ WKH WLPHU DQG GLVSOD\ LQIRUPDWLRQ DERXW WKH DYDWDUZDVLPSOHPHQWHGDVDSURFHGXUDODQLPDWLRQXVLQJ
SDWLHQW VVXFFHVV7KHVHFRQGVFUHHQLVWKHFRQWUROVFUHHQ DFRURXWLQH,QWKLVZD\LWZDVSRVVLEOHWRHQVXUHWKDWWKH
IRUWKHWKHUDSLVW7KLVVFUHHQFRQWDLQVYLHZWRWKHYLUWXDO PRYHPHQW RI WKH DYDWDU V KDQG LV SHUIRUPHG VPRRWKO\
WKHUDS\URRPRUYDULRXVEXWWRQVWKDWDOORZWKHWKHUDSLVWWR ZLWKRXWWKHQHHGWRFUHDWHDGGLWLRQDOFRPSOH[DQLPDWLRQV
FRQWUROWKHSURJUHVVRIWKHUHKDELOLWDWLRQVXFKDVVWDUWLQJ $QLPDWLRQV ZHUH XVHG ZLWKLQ WKH V\VWHP WR FUHDWH
D WLPHU RU VHQGLQJ DQ LPSXOVH WR PRYH 2WKHU EXWWRQV DYDWDU KDQG JULSV GHSHQGLQJ RQ WKH JULSSHG REMHFW 7KH
DOORZWKHWKHUDSLVWWRDGMXVWVHWWLQJV FUHDWLRQRILQGLYLGXDODQLPDWLRQVFRQVLVWHGLQEHQGLQJWKH
3DUW RI WKH WKHUDSLVW V FDQYDV LV DOVR WKH SRVVLELOLW\ WR MRLQWV RI WKH DYDWDU V ILQJHUV LQ VXFK D ZD\ WKDW WKH\
FKRRVHWKHREMHFWWREHWUDQVIHUUHGDQGWKHLQLWLDOSRVLWLRQ LPLWDWHWKHGHVLUHGKDQGJULS7KHUHVXOWLQJDQLPDWLRQVRI
RIWKHREMHFW'XULQJWKHFKRLFHRISRVLWLRQWKHSRVLWLRQV WKHULJKWKDQGJULSVFDQEHVHHQLQWKHIROORZLQJLPDJHV
RQ WKH WDEOH DUH DOVR PDUNHG VR WKDW WKH WKHUDSLVW KDV D )LJ
EHWWHULGHDRIZKHUHWKHREMHFWZLOOEHORFDWHG7KHSODFHV
ZKHUH WKH VHOHFWHG REMHFW FDQ EH PRYHG DQG SODFHG DUH
DOVRPDUNHGRQWKHWDEOH

 
)LJ9LUWXDOWKHUDS\URRPDQGV\PEROVRI D VXFFHVV E IDLOXUH
XQGHUWKHWLPHU

% 6KDUHG(QYLURQPHQW
7RHQVXUHWKHFUHDWLRQRIDVKDUHGV\VWHPHQYLURQPHQW
WKH 1HWFRGH OLEUDU\ IURP 8QLW\ ZDV XVHG 6LQFH WKLV
OLEUDU\DOORZVFUHDWLQJDSSOLFDWLRQVEDVHGRQFOLHQWVHUYHU
DUFKLWHFWXUH WKLV DSSOLFDWLRQ ZDV LPSOHPHQWHG LQ WKH
VDPH ZD\ ZLWKWKHWKHUDSLVW UHSUHVHQWLQJWKHVHUYHUDQG
WKHSDWLHQWWKHFOLHQWZLWKLQWKHV\VWHP,QWKHFDVHRIWKH
WKHUDSLVWLWLVVSHFLILFDOO\DJXHVWUROHZKLFKPHDQVWKDW
WKH WKHUDSLVW LV ERWK D VHUYHU DQG D FOLHQW ZLWK D
UHSUHVHQWDWLRQLQWKHHQYLURQPHQW 
7KDQNV WR WKLV OLEUDU\ LW ZDV DOVR SRVVLEOH WR
LPSOHPHQW D FRQQHFWLRQ WR DQ HQYLURQPHQW RQ WKH VDPH )LJ6DPSOHVRIGLIIHUHQWW\SHVRIREMHFWVDQGJULSV
/$1 QHWZRUN $ FRQGLWLRQ IRU FRQQHFWLQJ WKH SDWLHQW WR
WKHHQYLURQPHQWLVWKHQHHGWRNQRZWKH,3DGGUHVVRIWKH ' 2SHQ9LEH
VHUYHU +RZHYHU WKLV LQIRUPDWLRQ FDQ DOVR EH SURYLGHG $VPHQWLRQHGWKHV\VWHPFDQDOVRFDSWXUHLPSXOVHVWR
E\WKHWKHUDSLVWKLPVHOIVLQFHKHFDQVHHKLV,3DGGUHVVLQ WULJJHU PRYHPHQW FRPLQJ IURP WKH 2SHQ9LEH SODWIRUP
WKHHQYLURQPHQWDIWHUFRQQHFWLQJ 6LQFH WKHVH LPSXOVHV UHSUHVHQW +773 PHVVDJHV WKH
FRPPXQLFDWLRQ EHWZHHQ WKH V\VWHP DQG 2SHQ9LEH ZDV
& $YDWDUV
HQVXUHGE\XVLQJWKH+WWS/LVWHQHURIWKH&ODQJXDJH
7KH WKHUDSLVW DQG SDWLHQW DYDWDU PRGHOV )LJ   ZHUH ZKLFKOLVWHQVDQGSURFHVVHVWKHLQFRPLQJUHTXHVWDIWHUWKH
REWDLQHGWKURXJKWKH0L[DPRZHEVHUYLFH7KDQNVWRWKLV WLPHU LV VWDUWHG ,I WKH LQFRPLQJ UHTXHVW FRQWDLQV WKH
VHUYLFHLWZDVSRVVLEOHWRILQGUHDOLVWLFORRNLQJDYDWDUV VWULQJPRYHDWWKHHQGWKLVLVDVLJQDO IURP 2SHQ9LEH
,Q WKH FDVH RI DYDWDUV LW ZDV ILUVW QHFHVVDU\ WR HQVXUH WRVWDUWPRYLQJWKHSDWLHQW VKDQG
WKDWWKH\FRXOGPRYHWKHLUHQWLUHKDQG)RUWKLVSXUSRVH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 476

 9, 86$*(6&(1$5,2
7KH WKHUDSLVW FKRRVHV D EDOO REMHFW DQG FKRRVHV
SRVLWLRQ  DV WKH VWDUWLQJ SRVLWLRQ RI WKH REMHFW $IWHU
EHLQJGLVSOD\HGKHJUDVSVWKHREMHFWDQGPRYHVLWWRWKH
UHVHUYHG SRVLWLRQ  RQ WKH WDEOH +H WKHQ LQVWUXFWV WKH
SDWLHQW WR LPDJLQH WKH PRYHPHQW DQG VWDUWV WKH WLPHU DW
WKHVDPHWLPHDEDOOREMHFWDSSHDUVRQWKHSDWLHQW VVLGH
$VKHQRWLFHVDFKDQJHLQWKHSDWLHQW VEUDLQDFWLYLW\DQG
HYDOXDWHVWKDWKHKDVLPDJLQHGWKHPRYHPHQWKHSUHVVHV
WKH &RQGLWLRQ EXWWRQ ZKLFK VHQGV DQ LPSXOVH WR WKH
V\VWHPWRVWDUWWKHPRYHPHQW7KLVLPSXOVHDOVRVWRSVWKH
WLPHUDQGVKRZVWKHSDWLHQWDV\PEROLQGLFDWLQJVXFFHVV

%\ VWDUWLQJ WKH PRYHPHQW WKH SDWLHQW V KDQG JUDEV WKH
)LJD 9LHZLQUHDOHQYLURQPHQWE 9LHZLQWKHYLUWXDOHQYLURQPHQW
REMHFW DQG FDUULHV LW WR WKH WDUJHW ORFDWLRQ $IWHU RIWKHV\VWHP
SHUIRUPLQJWKHPRYHPHQWWKHSDWLHQW VKDQGUHWXUQVWRLWV
RULJLQDOSRVLWLRQ 7KH VHFRQG SKDVH RI WHVWLQJ FRQVLVWHG LQ WHVWLQJ WKH
9,,(9$/8$7,21 FRQQHFWLRQ RI WKH WKHUDSLVW DQG WKH SDWLHQW WR WKH
FROODERUDWLYH YLUWXDO HQYLURQPHQW DQG WKH FRXUVH RI
7KH V\VWHP ZDV WHVWHG LQ WZR SKDVHV $V SDUW RI WKH UHKDELOLWDWLRQ ,Q WKLV WHVWLQJ WKUHH SRVVLEOH VFHQDULRV
ILUVW SKDVH WKH V\VWHP ZDV WHVWHG E\ FRQQHFWLQJ WR WKH ZHUHWHVWHGQDPHO\
HQYLURQPHQW DV D WKHUDSLVW RQ DQ 2FXOXV 4XHVW GHYLFH
• 7KHSDWLHQWLPDJLQHVWKHPRYHPHQWDQGWKHLPSXOVH
'XULQJ WKLV SKDVH DVSHFWV VXFK DV FRQQHFWLQJ WKH DYDWDU
WRVWDUWWKHPRYHPHQWFRPHVIURPWKHWKHUDSLVW
WRWKHFRQWUROOHUVDQG95+0'LQWHUDFWLQJZLWKWKHXVHU
• 7KHSDWLHQWLPDJLQHVWKHPRYHPHQWDQGWKHLPSXOVH
LQWHUIDFHDQGLQWHUDFWLQJZLWKREMHFWVZHUHWHVWHG
WRVWDUWWKHPRYHPHQWFRPHVLQDQ+773PHVVDJH
$IWHUFRQQHFWLQJWRWKHHQYLURQPHQWDVDWKHUDSLVWWKH
• 7KHSDWLHQWFDQQRWLPDJLQHWKHPRYHPHQW
SDQHOEXWWRQVZHUHWULHGILUVWWKHIXQFWLRQVRIZKLFKFDQ
&RQQHFWLQJ WKHUDSLVW DQG SDWLHQW LQ D FROODERUDWLYH
EHWHVWHGHYHQ ZLWKRXWWKHSUHVHQFHRI WKHSDWLHQWLQWKH
HQYLURQPHQW FRXOG EH GRQH ZLWKRXW GLIILFXOW\ $IWHU
HQYLURQPHQW7KHVHEXWWRQVZHUHWKHEXWWRQVWRDGMXVWWKH
KHLJKWRIWKHWDEOHWKHEXWWRQWRKLGHVKRZWKH,3DGGUHVV FRQQHFWLQJERWKUHVSHFWLYHDYDWDUVDSSHDUHGRQFKDLUVDW
DQG WKH EXWWRQV WR GLVSOD\ WKH REMHFW RQ WKH WDEOH LQ WKH WKHWDEOHDFURVVIURPHDFKRWKHU$OOFKDQJHVPDGHE\WKH
WKHUDSLVW VXFK DV DGMXVWLQJ WKH KHLJKW RI WKH WDEOH RU
VHOHFWHGSRVLWLRQ $OOWKHVHEXWWRQV ZRUNHGSURSHUO\DQG
PRYLQJ WKH REMHFW ZHUH V\QFKURQL]HG ZLWKRXW PXFK
SHUIRUPHGWKHLUIXQFWLRQVDVWKH\ZHUHVXSSRVHGWR
GHOD\
7KH QH[W VWHS ZDV WHVWLQJ WKH LQWHUDFWLRQ ZLWK WKH
7KH ILUVW VFHQDULR RI WKH FRXUVH RI UHKDELOLWDWLRQ WRRN
REMHFWV RU WKHLU JUDVSLQJ DQG PRYLQJ )LJ   $IWHU
EULQJLQJWKHKDQGWRWKHREMHFWDQGSUHVVLQJWKHEXWWRQRQ SODFH VXFFHVVIXOO\ GXULQJ WHVWLQJ 3UHVVLQJ WKH EXWWRQ WR
VWDUW WKH WLPHU VWDUWHG WKH FRXQWGRZQ V\QFKURQL]HG RQ
WKH FRQWUROOHU WKH REMHFW FRXOG EH JUDVSHG ZKLOH DOVR
ERWKVLGHVDQGWKHREMHFWWKDWWKHWKHUDSLVWKDGSUHYLRXVO\
VWDUWLQJWKHFRUUHFWJUDVSDQLPDWLRQUHODWLYHWRWKHREMHFW
PRYHG DSSHDUHG LQ IURQW RI WKH SDWLHQW LQ WKH FRUUHFW
7KH REMHFW FRXOG WKHQ EH PRYHG DQG SODFHG RQ WKH
SRVLWLRQ$IWHUSUHVVLQJWKH VWDUWKDQG PRYHPHQWEXWWRQ
PDUNHG SODFHV RQ WKH WDEOH 'XULQJ WKH HQWLUH WHVWLQJ
WKH SDWLHQW DYDWDU V KDQG EHJDQ WR PRYH DQG PRYH WKH
SHULRGWKHDYDWDUVSHFLILFDOO\LWVKDQGVDQGKHDGUHDFWHG
WRWKHPRYHPHQWRIWKHFRQWUROOHUVDQGWKHKHOPHW REMHFWWRWKHWDUJHWORFDWLRQDQGDWWKHVDPHWLPHDJUHHQ
PDUN DSSHDUHG XQGHU WKH WLPHU LQGLFDWLQJ WKH SDWLHQW V
'XULQJ WKH WHVWLQJ LQ WKLV SKDVH FHUWDLQ VKRUWFRPLQJV
VXFFHVV$IWHUSODFLQJWKHREMHFWLWUHWXUQHGWRLWVRULJLQDO
ZHUHDOVRGLVFRYHUHG7KHLGHQWLILHGVKRUWFRPLQJVPDLQO\
SRVLWLRQ
UHODWHG WR WKH PDSSLQJ RI WKH FRQWUROOHUV DQG WKH 95
)RUWKHSRVVLELOLW\RIWHVWLQJWKHVHFRQGVFHQDULRLWZDV
+0' WR WKH DYDWDU ZKLFK ZDV QRW FRPSOHWHO\ DFFXUDWH
,QWKHFDVHRIPDSSLQJWKHDYDWDU VKHOPHWDQGKHDGKDLU QHFHVVDU\ WR VLPXODWH VHQGLQJ DQ +773 PHVVDJH IURP
RU SDUW RI WKH DYDWDU V IDFH FRXOG EH VHHQ DW D FHUWDLQ 2SHQ9LEH )RU WKLV SXUSRVH D VLPSOH -DYD6FULSW VFULSW
ZDV DGGLWLRQDOO\ LPSOHPHQWHG ZKLFK KDG WKH WDVN RI
YLHZLQJ DQJOH RQ WKH FDPHUD ZKLFK LV QRUPDOO\ QRW
VHQGLQJWKH+773 PHVVDJHPRYHWRWKH V\VWHP VHUYHU
YLVLEOH DQG KDG D GLVWUDFWLQJ HIIHFW ZKHQ YLHZHG IRU D
DQGWKXVVWDUWLQJWKHPRYHPHQWRIWKHKDQG7KLVVFHQDULR
ORQJWLPH,QWKHFDVHRIPDSSLQJWKHFRQWUROOHUVDQGWKH
RI WKH FRXUVH RI UHKDELOLWDWLRQ DOVR WRRN SODFH
KDQGV RI WKH DYDWDU LW ZDV REVHUYHG WKDW WKH SRVLWLRQ RI
WKH FRQWUROOHU LQ WKH UHDO HQYLURQPHQW GLG QRW DOZD\V VXFFHVVIXOO\ EHFDXVH DIWHU VWDUWLQJ WKH WLPHU WKH VHUYHU
FDXJKWWKHVHQW+773PHVVDJHDQGVWDUWHGWKHPRYHPHQW
FRLQFLGH ZLWK WKH SRVLWLRQ RI WKH KDQG RU SDOP RI WKH
RIWKHSDWLHQW VKDQG
DYDWDU LQ WKH YLUWXDO HQYLURQPHQW 7KLV VKLIW ZDV PRVW
,QWKHWKLUGVFHQDULRLWZDVPDLQO\WHVWHGZKHWKHUDUHG
RIWHQVHHQ ZKHQWKHFRQWUROOHUKDQG ZDVH[WHQGHG ZKLOH
PDUN VKRZLQJ WKH SDWLHQW V IDLOXUH WR LPDJLQH WKH
KROGLQJ DQ REMHFW ZKHUH WKH REMHFW PRYHG RXWVLGH WKH
PRYHPHQW ZRXOG EH GLVSOD\HG DIWHU WKH WLPHU H[SLUHG
DYDWDU VSDOP
7KH UHG PDUN ZDV GLVSOD\HG DIWHU WKH H[SLUDWLRQ VR WKH
WKLUGVFHQDULRZDVVXFFHVVIXO

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 477

 9,,,&21&/86,21 $&.12:/('*(0(17
,Q WKLV DUWLFOH D WKHUDSHXWLF V\VWHP EDVHG RQ YLUWXDO 7KLVZRUNKDVEHHQVXSSRUWHGE\WKH$399JUDQWQR
UHDOLW\ WHFKQRORJLHV ZDV EULHIO\ GHVFULEHG 7KH FUHDWHG $399 ³7UXVWZRUWK\ KXPDQ±URERW DQG
V\VWHP UHSUHVHQWV D 95 DSSOLFDWLRQ ZLWK D FROODERUDWLYH WKHUDSLVW±SDWLHQWLQWHUDFWLRQLQYLUWXDOUHDOLW\´
HQYLURQPHQW WR ZKLFK ERWK WKH WKHUDSLVW DQG WKH SDWLHQW 
FDQ FRQQHFW ZLWKLQ WKH VDPH /$1 QHWZRUN 7KH HQWLUH
V\VWHP LV EXLOW RQ D FOLHQWVHUYHU DUFKLWHFWXUH ZKHUH WKH 5()(5(1&(6
SDWLHQWUHSUHVHQWVWKHFOLHQWFRQQHFWLQJWRWKHVHUYHUDQG >@ 8 &KDXGKDU\ 1 %LUEDXPHU DQG $ 5DPRV0XUJXLDOGD\
WKH WKHUDSLVW UHSUHVHQWV WKH VHUYHU RU WKH JXHVW LH WKH ³%UDLQFRPSXWHULQWHUIDFHVIRUFRPPXQLFDWLRQDQGUHKDELOLWDWLRQ´
1DW5HY1HXUROYYROSS±
VHUYHUZLWKDJUDSKLFDOUHSUHVHQWDWLRQLQWKHHQYLURQPHQW
>@ ,PRWLRQV ³9LUWXDO UHDOLW\ UHKDELOLWDWLRQ ± WKH IXWXUH RI SK\VLFDO
:LWKLQ WKH HQYLURQPHQW ERWK WKH WKHUDSLVW DQG WKH WKHUDS\´)HEXUOKWWSVLPRWLRQVFRPSODWIRUP
SDWLHQW DUH UHSUHVHQWHG LQ WKH IRUP RI D KXPDQ DYDWDU >@ 1LUYDQD ³9LUWXDO UHDOLW\ DSSOLHG WR QHXURPRWRU UHKDELOLWDWLRQ´
HDFK KDYLQJ WKHLU RZQ DYDWDU 7KXV LW LV SRVVLEOH WR 0D\XUOKWWSVZZZEWVELRHQJLQHHULQJFRPQLUYDQD
VLPXODWH DQG HYHQ GLVSOD\ WKH WKHUDSLVWSDWLHQW >@ )3XW]H$9RXUYRSRXORV$/pFX\HU'.UXVLHQVNL70XOOHQ
LQWHUDFWLRQ7KHWKHUDSLVW VDYDWDURUKLVKDQGVDQGKHDG DQG & +HUII ³(GLWRULDO %UDLQ&RPSXWHU ,QWHUIDFHV DQG
$XJPHQWHG9LUWXDO 5HDOLW\´ )URQWLHUV LQ +XPDQ 1HXURVFLHQFH
UHDFWV WR WKH WKHUDSLVW V PRYHPHQWV ZLWK WKH FRQWUROOHU YRO
DQG WKH KHOPHW 7KH SDWLHQW V DYDWDU LV PRWLRQOHVV LQ WKH >@ '5/RXLH%/6KDQQRQ-(-DQLFH³7KH(IILFDF\RI/RZHU
VFHQHRQO\ KLVKDQG PRYHVDIWHUVWDUWLQJWKH PRYHPHQW ([WUHPLW\0LUURU7KHUDS\IRU,PSURYLQJ%DODQFH*DLWDQG0RWRU
7KH DYDWDU RI ERWK WKH WKHUDSLVW DQG WKH SDWLHQW FDQ )XQFWLRQ 3RVWVWURNH $ 6\VWHPDWLF 5HYLHZ DQG 0HWD$QDO\VLV´
FKDQJH WKH KDQG JULS GHSHQGLQJ RQ WKH REMHFW EHLQJ -RXUQDO RI 6WURNH  &HUHEURYDVFXODU 'LVHDVHV YRO  SS ±

PRYHG 7KH SRVVLELOLW\ RI FKRRVLQJ WKH JHQGHU DQG VNLQ
>@ :&KR52UWQHU10XURYHF79RQ2HUW]HQ..DPDGD%
FRORU RI WKH DYDWDU DQG FKRRVLQJ WKH KDQG WKDW WKH = $OOLVRQ DQG * *XJHU ³%UDLQ &RPSXWHU ,QWHUIDFH 7UHDWPHQW
UHKDELOLWDWLRQ IRFXVHV RQ LV QRW LPSOHPHQWHG 7KH IRU0RWRU5HKDELOLWDWLRQRI8SSHU([WUHPLW\RI6WURNH3DWLHQWV±$
SDWLHQW VDYDWDULVIL[HGLQWKLVYHUVLRQRIWKHV\VWHPDQG )HDVLELOLW\ 6WXG\ ´ )URQWLHUV LQ 1HXURVFLHQFH YRO  

UHKDELOLWDWLRQLQLWIRFXVHVRQWKHULJKWKDQG
>@ ,PRWLRQV ³9LUWXDO UHDOLW\ UHKDELOLWDWLRQ ± WKH IXWXUH RI SK\VLFDO
WKHUDS\´)HEXUOKWWSVLPRWLRQVFRPSODWIRUP
>@ & 1LVVOHU 0 1RZDN 0 &RQQDQ 6 %ࡇXWWQHU - 9RJHO ,
.RVV\N =&0ғDUWRQ DQG & &DVWHOOLQL ³9LWD²DQ HYHU\GD\
YLUWXDO UHDOLW\ VHWXS IRUSURVWKHWLFV DQG XSSHUOLPE
UHKDELOLWDWLRQ´-RXUQDO RI QHXUDO HQJLQHHULQJYRO  QR  S

>@ '&KHQ+/LXDQG=5HQ³$SSOLFDWLRQRIZHDUDEOHGHYLFHKWF
YLYHLQXSSHUOLPEUHKDELOLWDWLRQWUDLQLQJ´SS±
>@ ; :X + /LX - =KDQJ DQG : &KHQ ³9LUWXDO UHDOLW\ WUDLQLQJ
V\VWHPIRUXSSHUOLPEUHKDELOLWDWLRQ´SS±
>@ $(ORU07HRGRUHVFXDQG6.XUQLDZDQ³3URMHFWVWDUFDWFKHU
$ QRYHO LPPHUVLYH YLUWXDO UHDOLW\ H[SHULHQFH IRU XSSHU OLPE
UHKDELOLWDWLRQ´ $&07UDQVDFWLRQV RQ $FFHVVLEOH &RPSXWLQJ
7$&&(66 YROQRSS±
>@ 0+XGiNâ.RUHþNRDQG%6RERWD³$GYDQFHGXVHULQWHUDFWLRQ
IRU ZHEEDVHG FROODERUDWLYH YLUWXDO UHDOLW\´ LQ  WK ,(((
 ,QWHUQDWLRQDO &RQIHUHQFH RQ &RJQLWLYH ,QIRFRPPXQLFDWLRQV
)LJ3DWLHQW¶VDYDWDU &RJ,QIR&RP SS,(((

7KH WKHUDSLVW LQ WKH HQYLURQPHQW FDQ DOVR SHUIRUP 
RWKHU WDVNV VXFK DV VHWWLQJ WKH KHLJKW RI WKH WDEOH 
FKRRVLQJDQREMHFWWKDWZLOOEHXVHGGXULQJUHKDELOLWDWLRQ 
RU VHWWLQJ WKH WLPH WKH SDWLHQW PXVW LPDJLQH WKH 
PRYHPHQW 
'XULQJWKHV\VWHPWHVWLQJYDULRXVDVSHFWVZHUHWHVWHG
VXFK DV WKH PDSSLQJ RI FRQWUROOHUV DQG KHOPHWV WR WKH
WKHUDSLVW V DYDWDU RU WKH WKHUDSLVW V LQWHUDFWLRQ ZLWK
REMHFWV 7KH FRXUVH RI UHKDELOLWDWLRQ LQ WKH HQYLURQPHQW
ZDVDOVRWHVWHGZKLOHWKUHHFRXUVHVFHQDULRVZHUHWHVWHG
,Q WKH IXWXUH LW ZRXOG EH DSSURSULDWH WR IRFXV RQ
VXSSOHPHQWLQJWKHLPSOHPHQWDWLRQRIDYDWDUUHTXLUHPHQWV
DQG WKH LPSOHPHQWDWLRQ RI WKH QHZ SODQQHG
FRPPXQLFDWLRQ ZLWK 2SHQ9LEH YLD VRFNHWV $ SRVVLEOH
H[SDQVLRQRIWKHV\VWHPFRXOGDOVREHWKHDGGLWLRQRIQHZ
REMHFWV DQG KDQG PRYHPHQWV RI WKH SDWLHQW V DYDWDU OHIW
DQG ULJKW  $QRWKHU H[WHQVLRQ FRXOG EH WKH SRVVLELOLW\ WR
UHFRUGDQGUHXVHWKHGHPRQVWUDWHGDFWLYLW\


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 478

 Importance and implementation of virtual reality
for organic photovoltaic educational purposes
M. Sobota*, F. Kolencik*, L. Stuchlíková*, and M. Weis*
* Slovak University of Technology in Bratislava, Faculty of Electrical Engineering and Information Technology,
Bratislava, Slovak Republic
xsobota@stuba.sk, xkolencikf@stuba.sk, lubica.stuchlikova@stuba.sk, martin.weis@stuba.sk

Abstract— This paper addresses the need for learning for separation from a particular phenomenon. This sense of
students, which can be a motivational problem. Still, distance is not confined solely to physical proximity; it
interactivity, i.e., “learning by doing,” can tremendously can also encompass more abstract aspects. In essence,
decrease the psychological distance from learning a subject. psychological distance is the measure of the perceived gap
Another challenge nowadays is the need for increasing between oneself and various other entities, such as
interest in technical areas such as material engineering. persons, events, knowledge, or time [4].
With this, we developed a virtual reality game, “Organic Students often feel psychological barriers during
Electronics,” as an example for university students and the learning process, thus lacking motivation and
secondary school children. A virtual environment is
efficiency. During e-learning, students feel a social gap
incredibly convenient for this generation of students. The
with their teachers. The time of usage knowledge seems
game presents an attractive environment for teaching
far away, the exam time is remote, and social relationships
organic photovoltaic concepts and serves as a platform for
with a teacher can be complicated [5]. In this study, we
upgrades and other discipline areas.
are focused on a particular psychological distance from
Keywords - Virtual reality, psychological distance, the learning topic. Immersion by physical senses (visual,
psychological embodiment, immersion, interaction, auditory, and somatosensory) decreases the gap and
imagination, organic photovoltaic allows a more accessible learning process. An exemplary
study was proposed by ref. [6]. An example of a
successful decrease in psychological distance was done
I. INTRODUCTION for environmental purposes [7].
Virtual Reality (VR) technology uses computer- Another known psychological aspect is the sense of
generated simulations to create immersive, three- embodiment in VR, as described by Kilteni et al. [8]. The
sense of embodiment in VR refers to being present and
dimensional environments that users can interact with.
connected to a virtual body or avatar within the simulated
A. Social Perspective environment. There are three critical points about the
In the last century, significant changes have been made sense of embodiment in VR:
to the social and educational environment: Financial- The sense of self-location: In VR, it refers to a user’s
agricultural revolution (1600-1740), Industrial revolution perception of where they are within the virtual
(1780-1840), Technical revolution (1870-1920), Scientific environment. It involves the brain’s ability to position the
revolution (1940-1970), and Digital revolution (1975- self in relation to the virtual environment accurately.
1721) [1]. Nowadays, social networks have changed to Achieving a solid sense of self-location is vital for VR
virtual environments for teenagers as well, and education immersion because it helps users feel genuinely present in
is hugely supported by online sources. Plenty of wasteful the virtual space.
information sources exist, but we need to build quality The sense of agency: In VR, it pertains to a user’s
information. feeling of control and influence over the virtual
At this time, university students belong to Generation environment. It involves the perception that one’s actions
Z, also known as “digital natives” [2], and high school and intentions in the real world directly translate into
pupils are Generation Alfa, also known as “Screenagers”. actions within the virtual space. A strong sense of agency
Despite biased nicknames, digital abilities give them is crucial for making users feel empowered and engaged
precisely the needed skills to manage basic quests in VR experiences.
required in jobs [3]. Young pupils are in their stage of life The sense of body ownership: In VR, it is perhaps one
where they choose their future path. Their everyday life of the most intriguing aspects of embodiment. It relates to
includes virtual relationships, virtual education, and the feeling that the virtual body, often represented as an
virtual games. This trend should not cut the sense from avatar, belongs to the user. This means that users perceive
reality, which we are aware of, but should instead enrich themselves as being present in the virtual space and
personalities. Older generations should not demonize identify with and embody their virtual avatar.
Generation Z and Alfa in this sense but offer them useful In summary, the sense of embodiment in VR relies on
options. creating a seamless integration of the sense of self-
location, agency, and body ownership. When these
B. Psychological Perspective components are effectively combined, users can
From a psychological perspective, psychological experience a powerful feeling of presence and immersion
distance refers to how individuals perceive a sense of in the virtual world, enabling them to interact with and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 479

navigate digital environments as if they were extensions In the contemporary landscape, the disparity between
of their physical selves. This is a crucial goal public expectations and the technological constraints of
in developing immersive and compelling VR experiences. VR is steadily diminishing. The extensive utilization of
[8]. 3D visualization in research and industry indicates this
trend. As we witness the rapid advancement of
C. Three I’s information technology and VR hardware, VR and virtual
The “Three I’s of Virtual Reality” is a framework worlds are progressively emerging as integral components
developed by Grigore Burdea, a pioneer in the field of of modern academic education. They are poised to be
VR, to describe key characteristics or components of VR viable alternatives to traditional campus-based education
systems. These three “I’s” are: [13].
Immersion: Refers to the extent to which a virtual Incorporating VR into higher education is poised to be
environment can engage the user’s senses and create a smooth transition, given that our target audience is
a feeling of presence within that environment. In other already well-prepared for its integration. Anticipating
words, it’s the degree to which the virtual world can future developments, it is likely that VR applications will
make you feel like you are there. High levels of become indispensable in supporting engineering education
immersion are typically achieved through technologies programs across the board. We firmly believe that
like head-mounted displays, haptic feedback devices, and visualization plays a pivotal role in learning [13].
realistic graphics and audio. In specific fields like electronics and photonics, VR can
Interaction: represents the ability of users to actively prove invaluable by aiding students in grasping the
engage with and manipulate objects or elements within intricate physical principles of semiconductor materials
the virtual environment. This is a crucial aspect of VR, and devices and the functionality of otherwise invisible
allowing users to participate in and influence the virtual circuits. Furthermore, VR can be deployed in scenarios
world. Interaction can involve hand and body tracking, where actual reality cannot provide additional insights
motion controllers, gesture recognition, and other input about an object due to prohibitively high costs or
methods. dangerous environments that restrict students’ access [13].
Imagination: refers to the creative aspect of VR, where The role of VR in engineering education is poised
users can explore and experience scenarios or to undergo a substantial transformation in the future. It
environments that may not exist in the real world. will fundamentally alter the landscape of education and
It involves the power of VR to transport users to fictional training, with engineering education standing to gain
or abstract settings, enabling unique and imaginative significant benefits. In our technologically driven society,
experiences beyond the physical constraints of reality. the implementation of VR as one among several
These three “I’s” serve as a helpful framework for technological tools will enable us to nurture the future
understanding and evaluating the quality and capabilities generation of technological leaders effectively [13].
of VR systems. By focusing on immersion, interaction,
D. Closures
and imagination, VR developers aim to create more
compelling and engaging virtual experiences [9]. We aim to not only popularize science for pupils but
also communicate demanding concepts, which require
long experience and an extended amount of knowledge.
The scientific revolution includes the material science It is primarily a problem for graduated students aiming for
of organic photovoltaic, which is only one part of overall a scientific career. This is resolved by learning by doing,
technology spectra. VR is manifested in this field, where which persists in memory. Our work is the continuation of
access to clean laboratories or high-tech gadgets, which our teamwork reported previously on the 2D game [14].
are sensitive to damage by incautious handling, causes
costly problems. This is also the case for NASA, II. MARKET ANALYSIS
programs, the army, manufacturers, logistics, and
medicine surgery equipment, and educational tools [10]– For a suitable VR experience and development, we
[12]. offer an overview of hardware and software. Researchers
and teachers can determine their preferences.
A. Hardware
The current VR market offers a variety of options,
depending on the customer’s requirements.
HTC Vive offers a great experience, intuitive control, a
chargeable controller, and a software partnership with
Valve. On the other, it is costly, needs powerful
hardware, and is less comfortable than Oculus Rift.
Another option is HTC Vive Pro, which embodies visual
fidelity, built-in speakers, and an improved game library
with a slightly uplifted price.
The Facebook division recently introduced Oculus
Quest 2 (Meta Quest 2) with the immersive experience of
a light headset, even though a computer is not needed.
Figure 1. The three "I's" schematic [9]. Flaws in the demand for Facebook accounts and battery
life.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 480

 The newest product PSVR2 (PlayStation VR2) is a The last one, CryENGINE performs excellent graphics
user-friendly gadget with great controller vibration supporting VR, especially sandbox game mode, and has
equipment. A high-resolution display reinforces the one of the most robust render engines and precise
experience. Flaws include the necessity of PS5 documentation. On the contrary, it has fewer materials
(PlayStation 5), price exceeds PS5, and stripes are than other engines, C++ knowledge is needed, and it is
clumsy. exhausting for beginners.
B. Software III. EDUCATIONAL GAME “ORGANIC ELECTRONICS”
From the software perspective, there are two steps for a “Organic Electronics” is a free VR game with three
successful game: 3D modelling and game engine. levels. It is placed in a clean room laboratory. The game
In 3D computer graphics, 3D modelling uses contains four scenes: Menu (Fig. 2), Level 1 (Fig. 3),
specialized software to create lifelike representations of Level 2, and Level 3.
objects, both animate and inanimate, in a three- After starting the game, the player is in the Menu,
dimensional space using mathematical coordinates. These where he has an explanation of the controls.
models consist of interconnected points and geometric
After teleporting forward a little, he has a left screen
shapes like triangles, lines, and curves. They can be
with a panel with three buttons. A laser shines from his
crafted manually, generated algorithmically, or created
right hand, which allows him to interact with the buttons
through scanning techniques. Texture mapping is often
on the display.
used to enhance surface details.
The first button, “Start”, opens the door to the airlock
3D modelling software includes 3DS Max, equipped
with a red button on the wall. The second “Settings”
with integration with external software, polygon
button switches to the settings panel, where you can
manipulation, and an intuitive and adaptable environment
change between teleporting and continuous movement
with adjustable light and cameras. Disadvantages are in
and adjust the sound.
the sense of vertex labelling and price. The well-known
Blender encompasses powerful modelling and sculpting The Scene Menu contains a soundtrack by which the
tools, cycle plotter, and freeware. Despite the pros, it is player can select his preferred sound settings.
unintuitive, and some tools are not free.
A game engine is a software development tool or
environment initially designed for video game creation.
In essence, game engines provide a structured foundation
that simplifies the process of building something, like a
video game, compared to starting entirely from scratch.
Typically, game engines incorporate a 2D or 3D
rendering engine, though the specific features can vary
between different engines.
Initially, game developers created these engine tools to
expedite and streamline the game creation process.
However, these robust rendering tools have opened up
new possibilities for various industries. They enable the
visualization of data, products, and procedures in
innovative ways, fostering collaboration and creativity
beyond the realm of gaming.
A game engine should be able to:
- Implement the laws of physics and physics into
the game (Physics engine).
- Process visual materials - lights, shadows, textures
(Graphics engine). Figure 2. Introduction menu room with control explanation.
- Process audio tracks and sounds in the game (Audio
engine)
The best-known game engine Unity 3D supports
various platforms, has a built-in integrated development
environment (IDE), engine supports high-quality audio
and video effects. It is free, intuitive, and equipped with
high-quality documentation and debugging.
On the contrary, a built-in physical engine has
shortcomings, and the software requires considerable disc
space. Also, the source code is stable but not publicly
available.
Another game Unreal engine shows immense realism
with relatively simple and fast usage for beginners. The
downside is charged with successful projects as well as
knowledge demand for deep controlling. Figure 3. First laboratory room with OPV information.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 481

 A third button likewise switches the panel to choose the player to become familiar with the organic light
Levels. emitting diode (OLED) structure. After familiarizing
In the first level, the player finds himself in a symbolic himself he must, according to the knowledge gained, the
laboratory where he learns about the process of OPV player must peck out the layers of the OLED structure in
(Organic Photovoltaic) cells production process. the correct order. After successfully, the player is given
On a hospital bed, he sees a small round robot. The an OLED display representing the robot’s eyes, which he
player’s task is to find the solar cell and place it on the must attach to the robot on the ground. The robot will fly
little robot, which recharges and wakes up (Fig. 2, 3 and into another newly opened air chamber, and the player
4). can continue to the next level.
In level 2, the player sees a little robot lying on The third level is a quiz on the knowledge gained
the ground (Fig. 5, 6 and 7). The task in this level is for during the first two levels. In the left corner flies an
already fully repaired little robot. After completing the
quiz, confetti is triggered, and the player has the option
to return to the Menu (Fig. 8) [15].
The gameplay experience, as illustrated in the Unified
Modeling Language (UML) diagram presented in
Figure 9, offers a comprehensive and structured look at
how users can navigate and engage with the game. This
game starts at the main menu, serving as the central hub
for the player’s interactions.
From the main menu, players have several options.
They can choose to access the settings, allowing them to
tailor the game to their preferences and make necessary
adjustments to the gameplay experience. Alternatively,
they can return to the main menu, giving them a
convenient way to navigate back to the central starting
point.
Figure 4. First laboratory room with an educational panel However, the main attraction of the game lies in the
immersive journey through various levels of interactive
challenges. Players can opt to dive into the game process,
which is brimming with intriguing levels. These levels
are designed to captivate and educate players, making it a
truly enriching gaming experience.
Each of these levels has been meticulously crafted to
contain a range of interactive activities. These activities
are designed not only for entertainment but also for
educational purposes.
As players progress through the levels, they are
exposed to engaging content related to the chosen field of
Operations, Processes, and Systems (OPV). These
activities are carefully designed to not only entertain but
also to impart knowledge and skills, fostering a deeper
understanding of the selected OPV field.
The game was successfully tested on the platform HTC
Figure 5. Second laboratory room with an OLED quiz Vive/HTC Vive Pro and programmed with SteamVR
Plugin based on market analysis. Some game stability

Figure 6. Second laboratory room with an OLED information panel Figure 7. Third laboratory room with a quiz and confetti

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 482

issues need to be resolved, such as object handling, but makes VR a viable alternative to traditional education,
the overall experience remained excellent. particularly in engineering and other technical disciplines.
The game offered memorable gameplay with Various hardware options are available for VR
educational information, which was easily memorized. experiences, each with advantages and disadvantages. The
The developed platform can serve as a base for other choice of hardware depends on specific needs and
upgrades, and possibility for other disciplinary areas [15]. preferences. Similarly, VR development relies on 3D
modelling and game engines, each with unique features
IV. CONCLUSION and drawbacks.
VR technology has the potential to significantly impact Educational Game “Organic Electronics”: The
education, particularly for the younger “digital native” described educational VR game “Organic Electronics”
generations like Generation Z and Generation Alpha. It illustrates the practical application of VR in teaching
offers immersive and engaging experiences that can enrich complex scientific concepts. The game’s three levels teach
learning and provide valuable tools for educators. players about OPV and OLED structures, culminating in a
However, it is crucial to balance virtual experiences with knowledge quiz. The game’s development shows potential
real-world connections to avoid a complete detachment for further improvements and expansion into other
from reality. educational areas.
The concept of psychological distance in VR is All in all, VR has the potential to revolutionize
essential, and immersive experiences can help reduce the education by providing immersive, interactive, and
psychological barriers students often face during learning. imaginative learning experiences. However, its successful
By creating a solid sense of self-location, agency, and implementation requires careful consideration of hardware
body ownership, VR can enhance the feeling of presence and software choices and understanding the psychological
and engagement in virtual environments. aspects of creating compelling VR educational
Grigore Burdea’s framework of the Three “I’s” of experiences. The “Organic Electronics” game exemplifies
Virtual Reality (Immersion, Interaction, and Imagination) how VR can be used to teach complex subjects engagingly
is a helpful guide for evaluating and understanding VR and memorably, with opportunities for future
systems. Achieving high levels of immersion and enhancements and broader educational applications.
interaction while allowing for creative and imaginative
experiences is vital to successful VR applications. ACKNOWLEDGEMENT
VR is becoming more accessible and integrated into This work was supported by the Slovak Research and
various fields, including education. The rapid Development Agency and Scientific Grant Agency of the
advancement of information technology and VR hardware Ministry of Education, Science Research and Sport of the
Slovak Republic, grants: APVV-20-0564 and VEGA
1/0623/23.

REFERENCES

[1] D. Šmihula and others, “Long waves of technological

innovations,” Stud. Polit. Slovaca, vol. 4, no. 2, pp. 50–68, 2011.
[2] M. Prensky, “Digital natives, digital immigrants. From On the
Horizon,” MCB Univ. Press, vol. 9, no. 5, pp. 1–6, 2001.
[3] Ursula Perano, “Meet Generation Alpha, the 9-year-olds shaping
our future,” https://www.axios.com/2019/08/08/generation-alpha-
millennial-children, Aug. 08, 2019.
[4] S. Baltatescu, “Psychological Distance,” in Encyclopedia of
Quality of Life and Well-Being Research, Dordrecht: Springer
Netherlands, 2014, pp. 5145–5146. doi: 10.1007/978-94-007-
0753-5_2306.
[5] A. W. Kruglanski and E. T. Higgins, Social psychology:
Figure 8. Second laboratory OLED quiz panel Handbook of basic principles. Guilford Publications, 2013.
[6] H. Lyu, “How Are Students Immersed by Providing Virtual
Reality Technology? The Role of Psychological Distance in
Online Flipped Class,” Int. J. Inf. Educ. Technol., vol. 10, no. 1,
pp. 79–83, 2020, doi: 10.18178/ijiet.2020.10.1.1343.
[7] U. S. Raja and A. R. Carrico, “A Qualitative Exploration of
Individual Experiences of Environmental Virtual Reality Through
the Lens of Psychological Distance,” Environ. Commun., vol. 15,
no. 5, pp. 594–609, Jul. 2021, doi:
10.1080/17524032.2020.1871052.
[8] K. Kilteni, R. Groten, and M. Slater, “The Sense of Embodiment
in Virtual Reality,” Presence Teleoperators Virtual Environ., vol.
21, no. 4, pp. 373–387, Nov. 2012, doi: 10.1162/PRES_a_00124.
[9] G. C. Burdea and P. Coiffet, Virtual reality technology. John
Wiley \& Sons, 2017.
[10] V. Medical, “Virtual medicine.”
https://www.medicinevirtual.com/ (accessed Sep. 19, 2023).
Figure 9. UML diagram for the presented game "Organic
Electronics". [11] M. Gaskill, “Nine Ways We Use AR and VR on the International
Space Station,” 2021.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 483

 https://www.nasa.gov/mission_pages/station/research/news/nine- (ICETA), 2017, pp. 1–6.
ways-we-use-ar-vr-on-iss (accessed Sep. 18, 2023). [14] L. Stuchlíková et al., “The role of games in popularization of
[12] S. Kavanagh, A. Luxton-Reilly, B. Wuensche, and B. Plimmer, “A science and technology,” in 2017 15th International Conference
systematic review of Virtual Reality in education,” Themes Sci. on Emerging eLearning Technologies and Applications (ICETA),
Technol. Educ., vol. 10, no. 2, pp. 85–119, Dec. 2017, [Online]. 2017, pp. 1–6.
Available: https://www.learntechlib.org/p/182115 [15] F. Kolenčík, “"Educational game 'Organic Electronics' creadted
[13] L. Stuchlíková, A. Kósa, P. Benko, and P. Juhász, “Virtual reality with VR technology," Slovak University of Technology in
vs. reality in engineering education,” in 2017 15th international Bratislava " E, Bratislava, 2023.
conference on emerging eLearning technologies and applications

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 484

 Comparison of Sentiment Classifiers on Slovak
Datasets: Original versus Machine Translated
Zuzana Sokolová, Maroš Harahus, Jozef Juhár, Matúš Pleva, Daniel Hládek, Ján Staš
Department of Electronics and Multimedia Communications
Faculty of Electrical Engineering and Informatics
Technical University of Košice
Košice, Slovakia
zuzana.sokolova@tuke.sk , maros.harahus@tuke.sk , jozef.juhar@tuke.sk ,
matus.pleva@tuke.sk , daniel.hladek@tuke.sk jan.stas@tuke.sk

Abstract—In this article, we focus on the comparison of Subsequently, they translated the data from English to four
classifiers on datasets in the Slovak language. In order to compare other languages - Italian, Spanish, French, and German -
the effectiveness of the classifiers, we used two groups of data: the using a standard machine translation system. They manually
first group was obtained from the Internet and translated into
the Slovak language, while the second group was originally in the corrected the test data and created Gold Standards for each of
Slovak language. Our goal was to find out how the translation the target languages. Finally, they tested the performance of
of datasets affects the performance of classifiers and to identify the sentiment analysis classifiers for the different languages
the most suitable models for the Slovak language. concerned and showed that the joint use of training data
Keywords—classifiers, comparison, dataset, machine transla- from multiple languages (especially those pertaining to the
tion, sentiment, Slovak language
same family of languages) significantly improves the results
of the sentiment classification. They showed that using Twitter
I. I NTRODUCTION
language normalization, can obtain good results in target
In recent years, natural language processing (NLP) has languages and that the joint use of training data from dif-
become one of the most important areas of machine learning. ferent languages helps to increase the overall performance
However, most research in this field focuses on English. We of the classification. Finally, they showed that joint training
studied Slovak and compared the performance of classifica- using translated data from languages that are similar yields
tions using datasets originating from Slovak and translated significantly improved results.
from other languages. In this way, we wanted to gain a deeper Sazzed and Jayarathna [3] analyzed the performance of sen-
understanding of how data quality and origin influenced model timent classification in Bengali and corresponding machine-
performance in the Slovak language. translated English corpus using multiple machine-learning al-
gorithms. They applied multiple machine learning algorithms:
II. R ELATED WORKS Logistic Regression (LR), Ridge Regression (RR), Support
In recent years, interest in the task of sentiment analysis Vector Machine (SVM), Random Forest (RF), Extra Random-
has been growing in the field of NLP. Balahur and Turchi [1] ized Trees (ET), and Long Short-Term Memory (LSTM) are
pointed out the sparsity of research to date in languages other added to a collection of Bengali corpus and the corresponding
than English. In their article, they address three languages - machine-translated English version. They evaluated model
French, German, and Spanish - using three different machine performance on two datasets from different domains. From
translation (MT) systems - Bing, Google, and Moses. In this the experimental results, it was apparent that class balancing
work, they proposed an extensive evaluation of the use of showed a performance improvement in both the Bengali and
translated data in the context of sentiment analysis. Their the translated English versions of the Cricket imbalanced
research showed that SMT systems are mature enough to dataset. The results also suggested that machine translation
produce reliable training data for languages other than English. improved the classifier performance in both datasets. The
They mentioned that the gap in classification performance results indicated that machine translation can provide better
between systems trained in English and translated data was accuracy compared to the original language (Bengali) and can
minimal. The work with the translated data means an increased be used as a way of sentiment analysis for resource-poor
number of features, sparseness, and noise in the data points languages like Bengali. Moreover, their comparative results
in the classification task. To limit these problems, they tested demonstrated that even though the current machine translation
three different classification approaches showing that bagging system is not perfect in Bengali-English translation, it can be
has a positive impact on the results. reliably used for bilingual sentiment analysis.
Balahur and Turchi [2] their main challenge was sentiment Cross-language sentiment classification attempts to leverage
analysis from tweets in a multi-lingual setting. They first built the automated machine translation (MT) capability by utilizing
a simple sentiment analysis system for tweets in English. the infrastructure of languages rich in linguistic resources,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 485

mainly English, to help build sentiment analysis systems for [10]. Datasets may be used in a wide range of applications,
low-resource languages. Bilianos and Mikros [4] explored from statistical analysis to machine learning and artificial
how MT influences the classification of sentiments between intelligence, and they can range in size from a simple array
languages. To this end, they performed three different exper- of numbers to complicated data structures that store enormous
iments, obtaining promising results. In the first experiment, amounts of information.
they automatically translated 4,000 positive and negative re-
views from English into Greek and Italian, thus obtaining la- A. Twitter and Reddit Sentimental analysis Dataset
beled sentiment datasets in these languages. Then, they trained The dataset was created as a portion of a college venture that
a Naive Bayes classifier and compared the performance with focuses on assumption examination on numerous social media
the source dataset. In the second experiment, the translated stages using PySpark [11]. It incorporates information from
reviews were automatically translated back into the source both Twitter and Reddit, which were extricated utilizing the
language (English), aiming to compare the classification accu- Tweepy and PRAW APIs, separately. The information captures
racy with the one obtained in the original dataset. In the final conclusions on Narendra Modi, other political pioneers, and
approach, the reviews are translated from the source (English) the public’s assumption with respect to the following Prime
into Italian through an intermediate translation in Greek to Serve within the setting of the 2019 Common Races in India.
examine whether the performance was further diminished The Twitter dataset contains roughly 163K tweets, each with
compared with the approach of the first experiment. an assumption name, whereas the Reddit dataset highlights
Hartung et al. [5] explored how machine translation might around 37K comments, moreover, with opinion names. All
introduce a bias in sentiments as classified by sentiment sections from both stages have been cleaned utilizing Python’s
analysis models. For this, they compared three translation ’re’ module and handled with NLP procedures [12]. The
models (fairseq-nllb [6], Argos-translate [7], and BERT2BERT estimation names doled out to each passage extend from -1
[8]) for five languages (German, English, Hebrew, Spanish, to 1, where demonstrates an unbiased assumption, 1 means a
and Chinese) from the TED2020 and Global Voices corpora positive opinion, and -1 speaks to a negative estimation.
to test whether the translation process causes a change in Reddit has a total of 37,249 entries. This dataset shows
the sentiment classes recognized in the texts. Their statistical a more balanced distribution of emotions: 42.50% positive,
analyzes were unable to confirm any bias. The closest to this is 35.28% neutral and 22.22% negative. The 162,980 entry
the translation from German to English by the Argo translation Twitter dataset has a 44.33% positive rating, a 33.88% neutral
system, which causes a shift towards neutral sentiments for rating, and a 21.79% negative rating.
both corpora. This ‘bias’, however, cannot be substantiated by
a notably large Wasserstein distance. Although their statistical B. Sentiment Analysis Dataset
test indicates shifts in the label probability distributions, they The dataset was created by automatically classifying tweets
found none that appeared consistent enough to assume a bias according to the emotion displayed by each of its emoticons.
induced by the translation process. Positive emoticons were used to mark tweets as positive, and
Baliyan et al. [9] addressed the need to classify the large negative emoticons were used to label tweets as negative [13].
amount of multilingual text data to perform Sentiment Anal- The largest dataset, Sentiment, contains up to 905,874 records.
ysis (SA). Labeled data for multilingual text classification are Interestingly, it lacks neutral emotions. It has 11.69% positive
hard to retrieve. Thus Neural Machine translation (NMT) is reviews, but the negative opinion dominates an impressive
used to propose a labeling system with the help of a labelled 88.31%.
English dataset which is found in abundance. The authors of
the proposed model used Global Vectors (GloVe) as word C. Flipkart Product reviews with sentiment Dataset
embeddings, which are fed into Recurrent Neural Network The dataset contains product details from flipkart.com,
Long short-term memory (RNNLSTM) to generate context including product names, prices, ratings, reviews, summaries,
and to classify text into sentiment classes. The major highlight and sentiments [14]. It encompasses 104 different product
was the labelling system which was used to solve the problem types, totaling 205,053 entries in 6 columns. Products without
of collecting labelled multilingual text data. The proposed reviews but with summaries have a ’Nan’ value in the review
model can also classify sentiment analysis depending on the column. Sentiments are labeled as positive, neutral, or nega-
content of the text, document, or series of text. The efficiency tive, derived from the ’Summary’ column using NLP and the
of the model increases as it is trained on more data. Proposed Vader model. Later, these labels were manually verified for
methods can be applied to a lot of different case studies. accuracy. Ambiguous summaries received a neutral label. The
Although it is difficult to collect multilingual labeled data for dataset was cleaned for summary and price columns using the
sentiment analysis, the proposed work can be used to obtain NumPy and Pandas Python modules. The data, collected in
these multilingual labeled data with satisfactory efficiency. December 2022, was sourced using the BeautifulSoup library
for web scraping. This dataset can be used to train machine
III. DATASETS learning models in sentiment analysis, predictive modeling
A dataset is a structured data collection that has often been of customer behavior, text classification tasks, and enhancing
set up to make calculation, interpretation, or analysis easier customer service through review analysis.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 486

 Fig. 1: Sentiment distribution among datasets

The Flipkart dataset contains a total of 205,052 records. Of E. Sentigrade

these, an impressive 81% are positive reviews, 13.77% are Sentigrade is a project that was created as a result of coop-
negative and only 4.99% are neutral. eration between the Faculty of Informatics and Information
D. Youtube Statistics Technologies of STU BA and PR agency Seesame. There
are a total of 37,250 reviews from the Sentigrade platform
The dataset comprises two files designed to analyze the
in the dataset you provided. Of these, 15,649 reviews are
correlation between a video’s popularity and its most perti-
positive, which represents approximately 42%. Other 12,953
nent and favored comments. The first file, "videos-stats.csv",
reviews, or about 35%, are neutral. There are 8,648 negative
provides fundamental details about each video, including its
reviews, which is approximately 23% of the total. Regarding
title, likes, views, associated keywords, and comment count.
the number of sentences, there are a total of 74,000 sentences
The second file, "comments.csv", lists the top ten most relevant
in the file, which means that on average one review contains
comments for each video from "videos-stats.csv", along with
two sentences.
the sentiments and likes of these comments. In "videos-
stats.csv", columns detail the video title, its unique identi- IV. A PPLIED METHODS
fier, publication date, associated keyword, number of likes, Scikit-Learn is a Python library designed for machine learn-
comments, and views. If a video has a likes or comments ing and data analysis. It offers a range of tools for building and
value of -1, it indicates that these metrics are either not evaluating models, from simple linear regressions to complex
publicly visible or have been disabled by the video creator. In ensemble methods. With its comprehensive suite of algorithms
"comments.csv", columns specify the video’s unique identifier, and utilities, Scikit-Learn streamlines the process of model
the comment’s text, its likes, and its sentiment, where 0 development and validation, making it a staple in the toolkit
denotes negative sentiment, 1 indicates neutral, and 2 signifies of data scientists and researchers.
positive. This dataset can be utilized for various analyses, Beyond its extensive algorithmic offerings, Scikit-Learn
including sentiment analysis, text generation, predicting video is renowned for its consistent API, which allows users to
likes and views based on comment data, and conducting in- easily switch between different algorithms with minimal code
depth exploratory data analysis [15]. The YouTube dataset adjustments. Its integration with other Python libraries, such
with 19,658 entries has a higher proportion of positive reviews as NumPy and Pandas, ensures seamless data manipulation
at 58.15%. Negative ratings account for 11.89% and neutral and mathematical computations. Moreover, Scikit-Learn pro-
ratings for 23.60%. vides tools for feature extraction, dimensionality reduction,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 487

and model selection, enhancing the model-building experience regression adapted for binary outcomes, employing the logit
[16]. Its active community ensures continuous updates, im- function to predict the probabilities of events [20].
provements, and readily available support, making it an invalu-
able resource for both beginners who are venturing into the Linear Regression Equation:
world of machine learning and experienced professionals who
are looking for efficient and effective model implementations. y = β0 + β 1 x 1 + β 2 x 2 + · · · + β n x n (1)
A. Random Forest Classification This equation represents a linear relationship between
Random Forest Classification is an ensemble learning tech- the dependent variable y and the independent variables
nique that constructs multiple decision trees during training x1 , x2 , . . . , xn . The coefficients β0 , β1 , . . . , βn indicate the
and outputs the mode of the classes for predictions. It’s adept influence of each corresponding variable.
at handling large datasets with high dimensionality and can
effectively manage missing values. When implemented via the Sigmoid Function:
Scikit-Learn Python library, it provides tools for easy model 1
σ(z) = (2)
training, prediction, and evaluation. Scikit-Learn’s interface 1 + e−z
allows for straightforward hyperparameter tuning and model The Sigmoid function maps any real-valued number to a
assessment, making Random Forest Classification a preferred range between 0 and 1. It’s commonly used to produce
choice for many classification tasks. probabilities in logistic regression.
Random Forest Classification operates as a council of ex-
perts who weigh in on a complex problem. Multiple decision Apply Sigmoid function on linear regression:
trees, each analogous to an expert, are constructed using
various subsets of data and features. These trees individually 1
p= (3)
assess how to classify the data. The final prediction is de- 1+ e−(β0 +β1 x1 +β2 x2 +···+βn xn )
termined by aggregating the results of all trees and selecting This equation integrates the linear regression output into the
the most frequent result. In a hypothetical scenario with a Sigmoid function, resulting in a probability value p that lies
random forest determining if a sample is a "Dog" or "Cat", between 0 and 1 [20].
each tree examines different features and data samples, leading
to varied predictions. If, for example, of the five trees, four D. Support Vector Classification
predict "Cat", the random forest would classify the sample as Support Vector Machines (SVMs) are supervised machine
a "Cat" based on the majority vote [17] [18]. learning algorithms used primarily for classification, although
they can also handle regression and outlier detection. Efficient
B. The multilayer perceptron in high-dimensional spaces, SVMs aim to segregate datasets
The MLP Classifier is a neural network algorithm designed into classes by identifying a maximum marginal hyperplane
for classification tasks rooted in the multilayer perceptron (MMH). The process involves iteratively generating hyper-
architecture (MLP). This feedforward neural network excels planes that best separate the classes and then selecting the
at discerning non-linear relationships between inputs and optimal hyperplane that achieves correct segregation. Key
outputs. Comprising multiple interconnected layers, an MLP concepts in SVM include Support Vectors, which are the data
has neurons that employ non-linear activation functions, with points nearest to the hyperplane, which influence its orientation
the exception of the input layer. While the input and output and position. The hyperplane itself is the decision boundary
layers are essential, the network can also include multiple that distinguishes different classes, and the margin is the
hidden layers that enhance its computational capabilities and distance between this hyperplane and the nearest data points
allow for more complex decision boundaries. This structure from both classes.
makes MLPClassifier a potent tool for various classification
challenges [19]. E. The K-Neighbors Classifier
The kNN (k-Nearest Neighbors) algorithm functions as a
C. Logistic regression majority voting system, where the class label of a new data
Logistic Regression is a statistical approach tailored point is determined by the predominant class label among its
for binary classification. It gauges the probability of an ’k’ closest neighbors in the feature space. For example, if you
observation being in one of two distinct categories. This were deciding on a political party to support in a village, you
is achieved by estimating probabilities using the Sigmoid might base your decision on the preferences of your closest
function, ensuring values lie between 0 and 1. A threshold, neighbors. Similarly, in a fruit classification scenario using
often set at 0.5, determines the classification: values exceeding features like roundness and diameter, if you are given a new
this threshold are categorized into one class, while those fruit, you would plot its features and determine its type by
below are placed in the other. Such a method is ideal for examining the ’k’ nearest existing data points. For example, if
binary outcomes, such as discerning if an email qualifies as three out of four closest points indicate a pear, you’d classify
spam. Essentially, Logistic Regression is a variant of linear the new fruit as likely being a pear with 75% certainty. The

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 488

choice of ’k’ and the distance measurement method are crucial With the training and testing data ready, we started the
for the accuracy of the algorithm [21]. training procedure. For each dataset, we trained different
classifiers. During training, we observed how the models
F. The Multinomial NB process the data and learn from it.
The Multinomial Naive Bayes algorithm is a probabilistic Finally, once training was complete, we moved on to
method predominantly used in Natural Language Processing evaluating our classifiers. To do this, we used the test data and
(NLP). It leverages the Bayes theorem to predict the classifica- evaluated the performance of each model. We used accuracy,
tion tag of textual data, such as emails or articles. For a given known as Accuracy, and F1 score as metrics. These metrics
sample, it computes the likelihood of each tag and outputs the gave us a detailed look at how well our models predict and
one with the highest probability. A fundamental principle of the quality of our models.
the Naive Bayes classifier is the assumption that each feature The results represent a comprehensive evaluation of multi-
being classified is independent of any other feature, meaning ple classifiers on multiple datasets, namely Flipkart, Reddit,
that one feature’s presence does not influence the presence of Sentiment, Twitter, Sentigrade and YouTube (see figure 2).
another [22]. For the Flipkart dataset, the Random Forest classifier
The Multinomial Naive Bayes algorithm is a classification recorded the highest accuracy of 93.35% and an F1 score of
method tailored for multinomially distributed data, often used 92.86%. Above all, the performance of the SVC classifier was
in text data analysis. It’s grounded in the Bayes theorem, significantly low, with an accuracy of only 13.57% and an F1
which calculates the probability of an event based on prior score of 3.39%.
knowledge. In the Reddit dataset, the MLP classifier stood out with
an accuracy of 55.82%. However, its F1 score was slightly
Bayes Theorem: surpassed by the same classifier and was 51.23%. The K-
Neighbors classifier struggled on this dataset, reflected in its
P (A) × P (B|A)
P (A|B) = F1 score of only 15.33%.
P (B) When examining the sentiment dataset, the Random Forest
Where: classifier achieved an accuracy of 88.89%, while the MLP
• P (A|B) is the posterior probability of class A given classifier topped the F1 score category with 87.43%.
predictor B. For the Twitter dataset, the Multinomial NB classifier had
• P (B) is the prior probability of predictor B. the best accuracy of 44.35% and the MLP classifier had the
• P (A) is the prior probability of class A. highest F1 score of 36.91%. The SVC classifier had significant
• P (B|A) is the likelihood, the probability of predictor B problems with an F1 score of only 7.90%.
given class A. On the Sentigrade dataset, the Multinomial NB classifier
took the lead with an accuracy of 65.09%. In terms of F1
In the context of text classification:
score, the same classifier achieved a commendable score of
• A represents a classification category.
58.96%. outperform others in this dataset.
• B denotes a specific word or phrase.
Finally, the Multinomial NB classifier again showed its
The algorithm computes the probability of a text belonging superiority on the YouTube dataset with the highest accuracy
to each category and classifies based on the highest probability of 62.37%. However, logistic regression is The classifier was
[23]. slightly off in the F1 score at 50.40%.
The F1 Score is especially useful in situations where data In other experiments, we chose only the Sentigrade dataset
classes are imbalanced. A higher F1 score indicates better as the test data. For the Flipkart dataset, the MLP classifier
model performance, with the maximum possible value being outperformed other models with an accuracy of 44.77%.
1 [24]. However, in terms of F1 score, the Multinomial NB classifier
led the pack with 37.14%. It is noteworthy to mention the
V. ACHIEVED RESULTS & DISCUSSION underwhelming performance of the SVC classifier, which
Once we had the data from the different datasets ready, we managed only 19.58% accuracy and an even lower F1 score
could start processing it and training the models. The first step of 7.42%.
was to transform the text data into a suitable format for ma- In the context of the Reddit dataset, none of the classifiers
chine learning. We used a method called TF-IDF vectorization. breached the 30% mark in accuracy. The MLP classifier
This is a technique that allowed us to transform text data into slightly outdid others with an accuracy of 27.08%. The
numeric vectors, taking into account the frequency of words SVC classifier, surprisingly, exhibited the highest F1 score
in the documents and the entire dataset. at 33.45%. In contrast, the K-Neighbors classifier struggled
After successful vectorization, we moved on to data parti- significantly, obtaining the lowest F1 score of 6.54%.
tioning. We split our data into training and test sets. We chose The sentiment dataset saw a close contest in terms of
a ratio of 80:20, which means that 80% of the data was used accuracy, with the MLP classifier emerging as the winner at
to train the models, while the remaining 20% was used to 47.36%. However, the Multinomial NB classifier stood out in
evaluate them. the F1 score measurements with a figure of 34.93%.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 489

 Fig. 2: Accuracy and F1 score for different classification techniques

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 490

 For the Twitter dataset, the SVC classifier showed the how different data processing methods affect the results of
highest accuracy of 44.58%. However, when it came to F1 Slovak datasets.
scores, the Multinomial NB classifier outperformed others with
a score of 39.13%. ACKNOWLEDGMENT
On the YouTube dataset, the SVC classifier dominated the The research in this paper was partially supported by the
accuracy metric with 45.09%. However, the F1 score saw the Scientific Grant Agency of the Ministry of Education, Science,
Multinomial NB classifier at the forefront with 33.25%. Research and Sport of the Slovak Republic and the Slovak
When comparing the results of the classifiers on different Academy of Sciences under the project VEGA 2/0165/21
datasets, we see interesting patterns. Let’s start with the funded by the Ministry of Education, Science, Research and
Flipkart dataset. When this dataset was tested on the same data, Sport of the Slovak Republic, and by the Slovak Research and
the Random Forest classifier achieved the highest accuracy Development Agency under the project of bilateral cooperation
with a value of 93.35%, while the SVC classifier recorded APVV-SK-TW-21-0002 and research projects APVV-22-0261
the lowest accuracy with a value of 13.57%. However, when & APVV-22-0414, and by the Faculty of Electrical Engineer-
this dataset was tested on Sentigrade data, Random Forest only ing and Informatics, TU Košice under the Grant FEI-2023-95.
achieved an accuracy of 40.49% and MLP achieved the highest
accuracy of 44.77%. Interestingly, Multinomial NB achieved R EFERENCES
similar accuracy in both cases - 90.07% on the same data and [1] A. Balahur and M. Turchi, “Multilingual sentiment analysis using ma-
36.90% on the Sentigrade data. chine translation?” in Proceedings of the 3rd workshop in computational
Regarding the Reddit dataset, when tested on the same data, approaches to subjectivity and sentiment analysis, 2012, pp. 52–60.
[2] ——, “Improving sentiment analysis in twitter using multilingual ma-
Random Forest achieved an accuracy of 52.40%, while K- chine translated data,” in Proceedings of the International Conference
Neighbors only had 30.14%. However, in Sentigrade data, Recent Advances in Natural Language Processing RANLP 2013, 2013,
Random Forest achieved only 25.76% and SVC significantly pp. 49–55.
[3] S. Sazzed and S. Jayarathna, “A sentiment classification in bengali and
outperformed other models with an accuracy of 42.38%. machine translated english corpus,” in 2019 IEEE 20th International
In the case of the Sentiment dataset, the Random Forest Conference on information reuse and integration for data science (IRI).
classifier achieved an accuracy of 88.89% on the same data, IEEE, 2019, pp. 107–114.
[4] D. Bilianos and G. Mikros, “Sentiment analysis in cross-linguistic con-
while it was only 45.91% on the Sentigrade data. MLP text: How can machine translation influence sentiment classification?”
achieved an accuracy of 88.33% on the same data and 47.36% Digital Scholarship in the Humanities, vol. 38, no. 1, pp. 23–33, 2023.
on the Sentigrade data, which is the highest value among all [5] K. Hartung, A. Herygers, S. V. Kurlekar, K. Zakaria, T. Volkan,
S. Gröttrup, and M. Georges, “Measuring sentiment bias in machine
classifiers for this dataset. translation,” in International Conference on Text, Speech, and Dialogue.
For the Twitter dataset, the results were quite low in both Springer, 2023, pp. 82–93.
cases. On the same data, Random Forest achieved an accuracy [6] M. R. Costa-jussà, J. Cross, O. Çelebi, M. Elbayad, K. Heafield,
K. Heffernan, E. Kalbassi, J. Lam, D. Licht, J. Maillard et al., “No
of 42.58%, while on Sentigrade data it was only 32.43%. In language left behind: Scaling human-centered machine translation,”
both cases, SVC achieved higher accuracy, namely 21.65% on arXiv preprint arXiv:2207.04672, 2022.
the same data and 44.58% on Sentigrade data. [7] P.J.Finlay, “Argos translate,” https://github.com/argosopentech/argos-tra
nslate, Accessed 2023.
In the Sentigrade dataset, which was tested on the same data, [8] S. Rothe, S. Narayan, and A. Severyn, “Leveraging pre-trained check-
Random Forest achieved an accuracy of 60.69% and MLP points for sequence generation tasks,” Transactions of the Association
achieved the highest accuracy of 63.21%. However, in the case for Computational Linguistics, vol. 8, pp. 264–280, 2020.
[9] A. Baliyan, A. Batra, and S. P. Singh, “Multilingual sentiment analysis
of the F1 score, the Multinomial NB reached the highest value using rnn-lstm and neural machine translation,” in 2021 8th international
of 58.96%. conference on computing for sustainable global development (INDIA-
Finally, on the YouTube dataset, when tested on the same Com). IEEE, 2021, pp. 710–713.
[10] IBM, “What is a data set?” https://www.ibm.com/docs/en/zos-basic-ski
data, Random Forest achieved an accuracy of 60.13%, while lls?topic=more-what-is-data-set, Accessed 2023.
on the Sentigrade data it was only 36.34%. In both cases, MLP [11] J. H. Shen and F. Rudzicz, “Detecting anxiety through Reddit,” in
and Multinomial NB achieved similar results. Proceedings of the Fourth Workshop on Computational Linguistics and
Clinical Psychology — From Linguistic Signal to Clinical Reality.
Vancouver, BC: Association for Computational Linguistics, Aug. 2017,
VI. C ONCLUSION pp. 58–65. [Online]. Available: https://aclanthology.org/W17-3107
[12] C. Gowda, Anirudh, A. Pai, and C. kumar A, “Twitter and
In our study, we successfully compared the performance of Reddit sentimental analysis dataset,” 2019. [Online]. Available:
classifiers with datasets in Slovak, while we found significant https://www.kaggle.com/ds/429085
[13] A. Shrivastava, “Sentiment analysis dataset,” https://www.kaggle.com/d
differences between the original and the translated model. atasets/abhi8923shriv/sentiment-analysis-dataset, Accessed 2023.
Although the translated datasets provide relatively satisfactory [14] N. Vaghani and M. Thummar, “Flipkart product reviews with sentiment
results, the original Slovak datasets allow the models to make dataset,” 2023. [Online]. Available: https://www.kaggle.com/dsv/49408
09
better predictions. This discovery highlights the importance [15] A. Patil, “Youtube statistics,” https://www.kaggle.com/datasets/advayp
of creating and using original datasets for specific languages. atil/youtube-statistics, journal=Youtube Statistics, Accessed 2023.
Translated data sets can be useful in situations where the [16] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion,
O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg et al.,
original data is not available. In the future, we will expand “Scikit-learn: Machine learning in python,” the Journal of machine
our research, include more types of classifiers and examine Learning research, vol. 12, pp. 2825–2830, 2011.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 491

[17] T. Yiu, “Understanding random forest,” https://towardsdatascience.com Accessed 2023.
/understanding-random-forest-58381e0602d2, journal=Understanding [21] A. Shafi, “K-nearest neighbors (knn) classification with scikit-learn,”
Random Forest, Accessed 2023. https://www.datacamp.com/tutorial/k-nearest-neighbor-classification-s
[18] A. Shafi, “Random forest classification with scikit-learn,” https://www.da cikit-learn, Accessed 2023.
tacamp.com/tutorial/random-forests-classifier-python, journal=Youtube [22] Sriram, “Multinomial naive bayes explained: Function, advantages &
Statistics, Accessed 2023. disadvantages, applications in 2023,” https://www.upgrad.com/blog/mu
[19] M. Fuchs, “Nn multilayer perceptron classifier (mlpclassifier),” https: ltinomial-naive-bayes-explained/, Accessed 2023.
//michael-fuchs-python.netlify.app/2021/02/03/nn-multi-layer-perceptro [23] J. M. McNamara, R. F. Green, and O. Olsson, “Bayes’ theorem and its
n-classifier-mlpclassifier/, Accessed 2023. applications in animal behaviour,” Oikos, vol. 112, no. 2, pp. 243–251,
[20] A. Navlani, “Understanding logistic regression in python tutorial,” https: 2006.
//www.datacamp.com/tutorial/understanding-logistic-regression-python, [24] R. Kundu, “F1 score in machine learning: Intro & calculation,” 2023.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 492

 $QDO\VLVDQG'HWHFWLRQRI6SHHFK
XQGHU(PRWLRQDO6WUHVV

-iQ6WDã'DQLHO+OiGHN=X]DQD6RNRORYi0DUHNýHFK.DWDUtQDâNRWNRYiDQG3DWULN3RUHPED
'HSDUWPHQWRI(OHFWURQLFVDQG0XOWLPHGLD&RPPXQLFDWLRQV)DFXOW\RI(OHFWULFDO(QJLQHHULQJDQG,QIRUPDWLFV
7HFKQLFDO8QLYHUVLW\RI.RãLFH1ČPFRYHM.RãLFH6ORYDNLD
MDQVWDV#WXNHVNGDQLHOKODGHN#WXNHVN]X]DQDVRNRORYD#WXNHVN
PDUHNFHFK#VWXGHQWWXNHVNNDWDULQDVNRWNRYD#VWXGHQWWXNHVNSDWULNSRUHPED#VWXGHQWWXNHVN


$EVWUDFW²$XWRPDWLFGHWHFWLRQRIHPRWLRQDOVWUHVVLQVSHHFK LWLVH[WUHPHO\GLIILFXOWWRILQGDFRUUHODWLRQEHWZHHQWKHP
KDVUHFHQWO\GUDZQLQFUHDVLQJDWWHQWLRQPDLQO\LQWKHILHOGV IRU DQQRWDWLRQ RI WKH H[DFW VWUHVV OHYHO &DXVLQJ H[WUHPH
RI QDWXUDO DQG VSRNHQ ODQJXDJH SURFHVVLQJ 6WUHVV FDQ EH OHYHOVRIDFXWHVWUHVVFRXOGEHQRWRQO\XQHWKLFDOEXWDOVR
DXWRPDWLFDOO\ GHWHFWHG IURP VSHHFKGHULYHG SDUDPHWHUV KHDOWKWKUHDWHQLQJVRDFWHG VLPXODWHG GDWDVHWVDUHRIWHQ
VXFK DV IXQGDPHQWDO IUHTXHQF\ ) YRLFH LQWHQVLW\ MLWWHU XVHG 7KH GHYHORSPHQW RI V\VWHPV IRU DQDO\]LQJ WKH
VKLPPHU KDUPRQLFWRQRLVH UDWLR PHOIUHTXHQF\ FHSVWUDO VSHHFKRI D VWUHVVHG VSHDNHULV RIWHQ OLPLWHG WRWKH VPDOO
FRHIILFLHQWV HWF ,Q WKLV SDSHU ZH IRFXV RQ WKH DQDO\VLV RI DPRXQW RI GDWD DYDLODEOH DQG DGYDQFHG GHHS OHDUQLQJ
VHOHFWHG YRLFH FKDUDFWHULVWLFV LQ DYDLODEOH GDWDVHWV RI WHFKQLTXHVFDQQRWWKHUHIRUHEHHIIHFWLYHO\DSSOLHG
HPRWLRQDOVSHHFKDQGVSHHFKXQGHUVWUHVVLQWKH(QJOLVKDQG 'DWDVHWVRIHPRWLRQDOVSHHFKSURYLGHVOLJKWO\PRUHGDWD
6ORYDNODQJXDJHV:HDOVRSURSRVHGDVWUHVVOHYHOGHWHFWLRQ
IRU WUDLQLQJ VXFK V\VWHPV 6WURQJ HPRWLRQV FDQ DOVR EH
DSSURDFK ZKLFK XVHV WKH IXVLRQ RI WKH 7HDJHU HQHUJ\
FRQVLGHUHG DV D NLQG RI GHYLDWLRQ IURP D QHXWUDO PHQWDO
RSHUDWRU 7(2  ZLWK PHOIUHTXHQF\ FHSVWUDO FRHIILFLHQWV
VWDWHDQGWKH\DOVRDIIHFWVSHHFKDQGRWKHUSK\VLRORJLFDO
0)&&V  DQG FODVVLILFDWLRQ EDVHG RQ GHFLVLRQ WUHHV DQG
FKDUDFWHULVWLFV +RZHYHU LPSHUIHFW HYDOXDWLRQ RI WKH
HQVHPEOH OHDUQLQJ PHWKRG RI EDJJLQJ GHFLVLRQ WUHHV
3UHOLPLQDU\ UHVXOWV RI  FODVVLILFDWLRQ DFFXUDF\ DUH
LQWHQVLW\ RI HPRWLRQ LQ VSHHFK GXULQJ PDQXDO DQQRWDWLRQ
FRPSDUDEOHWRRWKHUVWXGLHVLQWKLVILHOGRIUHVHDUFK
PD\ FDXVH IXUWKHU FRPSOLFDWLRQV LQ WKH UHVHDUFK DQG
GHYHORSPHQWRIWKHV\VWHPVIRUGHWHFWLQJWKHSUHVHQFHRI
VWUHVVLQVSHHFK
, ,1752'8&7,21 7KHUHVWRIWKHSDSHULVRUJDQL]HGDVIROORZV,Q6HFWLRQ
7KHDQDO\VLVDQGGHWHFWLRQRIHPRWLRQDOVWUHVVLQKXPDQ ,, ZH EULHIO\ GHVFULEH FXUUHQW DSSURDFKHV IRU GHWHFWLQJ
VSHHFK LV FXUUHQWO\ RQH RI WKH KRW UHVHDUFK WRSLFV LQ WKH HPRWLRQVDQGVWUHVVLQVSHHFK6HFWLRQ,,,VXPPDUL]HVWKH
ILHOGRIQDWXUDOODQJXDJHSURFHVVLQJ 1/3 DQGDXWRPDWLF DYDLODEOH GDWDVHWV RI HPRWLRQDO VSHHFK DQG VSHHFK XQGHU
VSHHFKUHFRJQLWLRQ $65 :LWKWKHLQFUHDVLQJRFFXUUHQFH VWUHVV XVHG LQ WKH H[SHULPHQWDO SDUW RI WKLV UHVHDUFK ,Q
RIVWUHVVLQPRGHUQVRFLHW\DQGLWVQHJDWLYHHIIHFWVRQWKH 6HFWLRQ,9 ZH IRFXVHG RQ WKH DQDO\VLV RI VHOHFWHG YRLFH
KHDOWKDQGSHUIRUPDQFHRISHRSOHWKLVWRSLFLVEHFRPLQJ FKDUDFWHULVWLFVLQWKHXVHGGDWDEDVHVZKLFKFOHDUO\GHILQH
PRUHDQGPRUHLPSRUWDQW WKH VWUHVV OHYHOV 7KH SURSRVHG VWUHVV OHYHO GHWHFWLRQ
DSSURDFK ZKLFK XVHV WKH IXVLRQ RI WKH 7HDJHU HQHUJ\
$FXWH VWUHVV XVXDOO\ PHDQV ODUJH GHPDQGV RQ DQ RSHUDWRU ZLWK PHOIUHTXHQF\ FHSVWUDO FRHIILFLHQWV DQG
LQGLYLGXDO V SK\VLFDO DQG SV\FKRORJLFDO UHVRXUFHV ZKLFK FODVVLILFDWLRQEDVHGRQGHFLVLRQWUHHVDQGEDJJHGWUHHVLV
DUH FKDUDFWHUL]HG E\ WKH UHOHDVH RI VWUHVV KRUPRQHV DQG SUHVHQWHG LQ 6HFWLRQ 9 7KH SUHOLPLQDU\ H[SHULPHQWDO
RWKHU SK\VLRORJLFDO DQG SV\FKRORJLFDO FKDQJHV LQFOXGLQJ UHVXOWVRI WKH SURSRVHG DSSURDFK DUHDOVRSUHVHQWHG KHUH
FKDQJHV LQ VSHHFK 6WUHVV UHVHDUFK LV PRVWO\ IRFXVHG RQ )LQDOO\ 6HFWLRQ 9, VXPPDUL]HV WKH FRQWULEXWLRQ RI RXU
SV\FKRORJLFDO DVSHFWV LQGXFHG E\ PDWKHPDWLFDO WDVNV ZRUNDQGFRQFOXGHVWKHSDSHUZLWKIXWXUHGLUHFWLRQV
H[DPV RU GULYLQJ DQG SK\VLRORJLFDO DVSHFWV VXFK DV
SK\VLFDODFWLYLWLHV
,, 67$7(2)7+($57
:KHQ FUHDWLQJ GDWDVHWV RI UHFRUGLQJV RI VSRQWDQHRXV
VSHHFKXQGHUVWUHVVLWLVQHFHVVDU\WRUHDOL]HWKDWVSHDNHUV $XWRPDWLF VSHHFK HPRWLRQ UHFRJQLWLRQ KDV EHHQ
FDQQRWEHGHOLEHUDWHO\H[SRVHGWRKLJKVWUHVVRUGDQJHURXV UHVHDUFKHG VLQFH WKH HDUO\ V >@ $W WKH HQG RI WKH
VLWXDWLRQV 2Q WKH FRQWUDU\ WKH XVH RI VSHHFK UHFRUGLQJV V WKH ILUVW ZRUNV DLPHGDWGHWHFWLQJ WKHSUHVHQFHRI
IURPHPHUJHQF\OLQHVDLUFUDIWFRPPXQLFDWLRQV\VWHPVRU VWUHVVLQVSHHFKZHUHDOVRSXEOLVKHGLQ>@
PLOLWDU\ DQG UHVFXH VHUYLFHV LV SUHYHQWHG E\ VHFXULW\ 6LQFHDIXOORYHUYLHZRIWKHDSSURDFKHVJRHVIDUEH\RQG
DVSHFWVDQG*'35 WKH VFRSH RI WKLV SDSHU ZH VXPPDUL]HG KHUH RQO\ WKH
6LQFHWKHHIIHFWRIVWUHVVLQVSHHFKLVQRQVSHFLILFDQG ODWHVWDSSURDFKHVEDVHGRQYRLFHDQDO\VLVRIIXQGDPHQWDO
GHSHQGV RQ WKH LPPHGLDWH VLWXDWLRQ DQG KDV D GLIIHUHQW IUHTXHQF\)DQGFODVVLILFDWLRQXVLQJPDFKLQHOHDUQLQJ
WHPSRUDO GHYHORSPHQW WKDQ SK\VLRORJLFDO LQGLFDWRUV RI $ FRPSUHKHQVLYH VXUYH\ RQ VWUHVV GHWHFWLRQ WKURXJK
VWUHVV FKDQJHVLQVNLQFRQGXFWLYLW\KHDUWUDWHYDULDELOLW\ VSHHFK DQDO\VLV XVLQJ PDFKLQH OHDUQLQJ WHFKQLTXHV LV
LQWKHFRQWHQWRIVWUHVVKRUPRQHVLQWKHEORRGDQGVRRQ  VXPPDUL]HGLQWKHSDSHUE\9DPVLQDWKHWDO>@

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 493

 :|OOPHUHWDO>@SURSRVHGDQDSSURDFKIRUFRQWLQXRXV 7$%/(,
67$7,67,&62)6(/(&7(''$7$6(76
HPRWLRQ UHFRJQLWLRQ EDVHG RQ ORQJ VKRUWWHUP PHPRU\
/670 UHFXUUHQWQHXUDOQHWZRUNV 511V ZKLFKLQFOXGH 'DWDVHW 7\SHRI 5HFRUGLQJ
/DQJXDJH
1XPEHU
PRGHOLQJ RI ORQJUDQJH GHSHQGHQFLHV EHWZHHQ WKH QDPH VSHHFK VFHQDULR RIVSHDNHUV
1RUWK
DFRXVWLFREVHUYDWLRQVGHULYHGIURPSURVRGLFVSHFWUDODQG 5$9'(66
HPRWLRQDO
VLPXODWHG $PHULFDQ

YRLFH TXDOLW\ IHDWXUHV 6WXKOVDW] HW DO >@ LQWURGXFHG VSHHFK
(QJOLVK
)0 
JHQHUDOL]HGGLVFULPLQDQWDQDO\VLV *HU'$ EDVHGRQGHHS HPRWLRQDO 
686 VLPXODWHG 6ORYDN
QHXUDOQHWZRUNV '11V IRUWKHWDVNRIDFRXVWLFHPRWLRQ VSHHFK )0 
UHFRJQLWLRQ0DRHWDO>@SURSRVHGWROHDUQDIIHFWVDOLHQW H[SUHVVLYH
&5,6,6 VLPXODWHG 6ORYDN 
VSHHFK
IHDWXUHV IRU VSHHFK HPRWLRQ UHFRJQLWLRQ XVLQJ VSHHFKXQGHU DFWXDODQG 
FRQYROXWLRQDO QHXUDO QHWZRUNV &11V  7KH ILUVW HQGWR 686$6
VWUHVV VLPXODWHG
(QJOLVK
)0 
HQG V\VWHP IRU VSHHFK HPRWLRQ UHFRJQLWLRQ XVHV D VSHHFKXQGHU
VLPXODWHG 6ORYDN

6WUHVV'DW
FRPELQDWLRQ RI &11 ZLWK /670 QHWZRUNV LQ RUGHU WR VWUHVV )0 
DXWRPDWLFDOO\ OHDUQ WKH EHVW UHSUHVHQWDWLRQ RI WKH VSHHFK 
VLJQDO GLUHFWO\ IURP WKH UDZ WLPH UHSUHVHQWDWLRQ >@ $
QXPEHURIHPRWLRQDOVSHHFKGDWDVHWVLQVHYHUDOODQJXDJHV ,,, $9$,/$%/('$7$6(762)(027,21$/63((&+
DUHFXUUHQWO\DYDLODEOHWRWUDLQDQGWHVWWKHVHDSSURDFKHV $1'63((&+81'(5675(66
$ FRPSUHKHQVLYH UHYLHZ RI WKHP LV VXPPDUL]HG LQ >@
)URP VHYHUDO SXEOLFO\ DYDLODEOH GDWDVHWV RI VLPXODWHG
7KHVHLQFOXGHVSHHFKGDWDEDVHVFRQWDLQLQJERWKDFWHGDQG
DFWHG  DQG UHDO DFWXDO  VWUHVV DQG HPRWLRQV ZH KDYH
UHDOHPRWLRQV3URJUHVVLQWKLVILHOGZDVDOVRVXSSRUWHGE\
FUHDWHGRQHODUJHHPRWLRQDOVWUHVVVSHHFKGDWDVHWZLWKWKH
WKHRUJDQL]DWLRQRIFRPSHWLWLRQVVXFKDVWKH9DOHQFHDQG
KHOS RI ZKLFK LW ZLOO EH SRVVLEOH WR WUDLQ DXWRPDWLF
$URXVDO LQWKHZLOG &KDOOHQJH>@
V\VWHPVIRUGHWHFWLQJOHYHOVRIHPRWLRQDOVWUHVVLQKXPDQ
7KH UHVXOWV RI WKH VWXG\ E\ 'HPHQNR DQG -DVWU]HEVND VSHHFK WR LQYHVWLJDWH WKH UHODWLRQVKLS EHWZHHQ HPRWLRQV
>@ FRQILUP WKH VLJQLILFDQFH RI WKH ) IRU LQYHVWLJDWLQJ DQGVWUHVVEXWDOVRWRDVVHVVWKHSRVVLELOLW\RIPHDVXULQJ
VWUHVV7KHVKLIWLQWKH)UHJLVWHUFRQVWLWXWHVWKHSULPDU\ RI UHDO HPRWLRQV DQG VWUHVV E\ XVLQJ PRGHOV WUDLQHG RQ
LQGLFDWRURIVWUHVVHVSHFLDOO\ZKHQFDXVHGE\IHDU$OVRD WKHVH GDWDVHWV ,Q WKH IROORZLQJ VXEVHFWLRQV VHOHFWHG
V\VWHPDWLF LQFUHDVH LQ WKH UDQJH RI ) YDULDELOLW\ IRU WKH GDWDEDVHV RI HPRWLRQDO VSHHFK DQG VSHHFK XQGHU VWUHVV
VWUHVV UHODWHG WR DQJHU RU LUULWDWLRQ ZDV REVHUYHG 9DQ ZLOOEHEULHIO\GHVFULEHG%DVLFLQIRUPDWLRQDERXWWKHPLV
3X\YHOGHHWDO>@RXWOLQHGWKHGLIIHUHQWVWDJHVRIYRLFH VXPPDUL]HGLQ7DEOH,
RXWSXW VXFK DV EUHDWKLQJ SKRQDWLRQ DQG UHVRQDQFH LQ
UHODWLRQWRDQHXURYLVFHUDOLQWHJUDWHGSHUVSHFWLYHRQVWUHVV $ 5$9'(66
DQG KXPDQ SHUIRUPDQFH 7KH\ FRQFOXGHG WKDW LQFUHDVHG 7KH 5$9'(66 5\HUVRQ $XGLR9LVXDO 'DWDEDVH RI
)YDOXHVFRUUHVSRQGWRVLWXDWLRQVZKHQFRJQLWLYHFRQWURO (PRWLRQDO 6SHHFK DQG 6RQJ  LV D PXOWLPRGDO GDWDVHW RI
LV ORVW VXFK DV LQ OLIHWKUHDWHQLQJ HPHUJHQF\ VLWXDWLRQV HPRWLRQDOVSHHFK7KHGDWDVHWFRQVLVWVRIJHQGHUEDODQFHG
HJIOLJKWFUDVKHVRUDOFRKROLQWR[LFDWLRQ 'HFUHDVHG) VSHHFK UHFRUGLQJV REWDLQHG IURP  SURIHVVLRQDO DFWRUV
YDOXHVDUHFRQVHTXHQWLDORIKLJKFRJQLWLYHORDG7KHPHDQ  IHPDOHV DQG  PDOHV  YRFDOL]LQJ OH[LFDOO\ PDWFKHG
)YDOXHVJLYHLQVLJKWLQWRDURXVDODFWLYLW\DQGWKHHIIRUWD VWDWHPHQWV LQ D QHXWUDO 1RUWK $PHULFDQ (QJOLVK 6SHHFK
VXEMHFWFDQJHQHUDWH-LWWHUH[SUHVVHVDURXVDOLQDQLQYHUVH LQFOXGHVFDOPKDSS\VXUSULVHGGLVJXVWHGVDGDQJU\DQG
PDQQHU &RJQLWLYH DQG HPRWLRQDO ORDG FRUUHVSRQG ZLWK IHDUIXO H[SUHVVLRQV (DFK H[SUHVVLRQ LV SURGXFHG DW D
UHVSHFWLYHVPDOODQGODUJHUUHGXFHGMLWWHUYDOXHV QHXWUDOOHYHODQGWZROHYHOVRIHPRWLRQDOLQWHQVLW\7KHVHW
+DQ HW DO >@ SURSRVHG D GHHS OHDUQLQJEDVHG RI  VSHHFK UHFRUGLQJV ZDV UDWHG  WLPHV RQ
SV\FKRORJLFDO VWUHVV GHWHFWLRQ DOJRULWKP IURP VSHHFK HPRWLRQDOYDOLGLW\LQWHQVLW\DQGJHQXLQHQHVV>@
VLJQDOV 7KH DOJRULWKP ILUVW H[WUDFWV PHOIUHTXHQF\
FHSVWUDOFRHIILFLHQWVDQGWKHQSUHGLFWV WKH VWDWXV RI VWUHVV % 686
RXWSXWXVLQJDELQDU\GHFLVLRQFULWHULRQ QRVWUHVVRUVWUHVV 7KH686GDWDVHW &RXUWURRPVHVVLRQV6~GQDVLHĖLQ
VSHHFK  XVLQJ /670 DQG IHHGIRUZDUG QHXUDO QHWZRUNV 6ORYDN FRQVLVWVRIUHFRUGLQJVRIHPRWLRQDOVSHHFKLQWKH
7KHDXWKRUVDFKLHYHGDFFXUDF\LQSUHGLFWLQJVWUHVV 6ORYDNODQJXDJHRIIDEULFDWHGOHJDOSURFHHGLQJVLQZKLFK
VSHHFK $ VLPLODU DSSURDFK EDVHG RQ WKH H[WUDFWLRQ RI QRQSURIHVVLRQDODFWRUVWDNHSDUW(DFKVHVVLRQLVRULHQWHG
0)&&V DQG SUHGLFWLQJ VWUHVV RXWSXW XVLQJ &11V ZLWK WR D MXULGLFDO FDVH VROXWLRQ WKHUHIRUH HPRWLRQDO UDQJH RI
GHQVH IXOO\ FRQQHFWHG OD\HU QHWZRUNV ZDV SUHVHQWHG E\ UHFRUGHG XWWHUDQFHV FRYHUV PRVWO\ HPRWLRQV RI QHXWUDO
9DLNROHHWDO>@+HUHWKHRYHUDOODFFXUDF\LQFUHDVHGXS VWDWHDQGFXULRVLW\DQGQHJDWLYHHPRWLRQVVXFKDVDQJHU
WR  %RWK VWXGLHV ZHUH SHUIRUPHG XVLQJ WKH DJJUHVVLYHQHVV VDGQHVV DQG GLVJXVW 3RVLWLYH HPRWLRQV
5$9'(66GDWDVHW>@7KHUHVHDUFKE\+LOP\HWDO>@ DUH UDUH WR ILQG $OO XWWHUDQFHV ZHUH PDQXDOO\ ODEHOHG
HPSKDVL]HVKRZVWUHVVGHWHFWLRQFDQEHGRQHE\XVLQJWKH (PRWLRQDO HYDOXDWLRQ ZDV SURYLGHG RQ WKH ZKROH
DQDO\VLV RI VSHHFK VLJQDOV 7KH 0)&&V ZHUH H[WUDFWHG VHQWHQFHV ,Q WKH FDVH ZKHUH PRUH WKDQ RQH HPRWLRQ
IURP WKH VSHHFK VLJQDOV DQG WKH VWUHVV ZDV GHWHFWHG ZLWK RFFXUUHGLQDQXWWHUDQFHWKHGLYLVLRQRIVXFKDQXWWHUDQFH
WKH&11V,WKDVEHHQFRQFOXGHGWKDWSV\FKRORJLFDOVWUHVV LQWR VKRUWHU VHJPHQWV IROORZLQJ WKH UXOH RI LQIRUPDWLRQ
FRXOGEHGHWHFWHGWKURXJKVSHHFKVLJQDOVE\DQDO\]LQJWKH UHOHYDQFH ZDV FDUULHG RXW 7KH ZKROH VHQWHQFHV RU
FRXQW RI SDXVHV DQG PD[LPXP DPSOLWXGH ZKLOH WKH QR VHJPHQWVZHUHWKHQHYDOXDWHGIURPDQHPRWLRQDOSRLQWRI
VWUHVV VSHHFK KDV D ORZ FRXQW RI SDXVHV DQG PD[LPXP YLHZ HJ&8IRUFXULRVLW\1IRUQHXWUDOHWF 7KH686
DPSOLWXGH VWUHVV VSHHFK KDV D KLJK FRXQW RI SDXVHV DQG FRQVLVWVRIDSSUR[LPDWHO\XWWHUDQFHVVSDQQLQJIURP
PD[LPXPDPSOLWXGH7KHDXWKRUVDFKLHYHGDFFXUDF\ WRVHFRQGVREWDLQHGIURPVSHDNHUV IHPDOHVDQG
LQ VWUHVV FODVVLILFDWLRQ LQ UHDO FRQGLWLRQV RI XVDJH PDOHV  LQ GLIIHUHQW HPRWLRQDO VWDWHV QHXWUDO FXULRVLW\
0RUHRYHUVWUHVVIXOVLWXDWLRQVFDQDOVREHVLPXODWHGZLWKD DQJHUVDGQHVVDQGVRRQ$OOVHVVLRQVZHUHUHFRUGHGZLWK
FRPSXWHUJDPH>@ DN+]VDPSOLQJUDWHDQGELWUHVROXWLRQ>@

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 494

& &5,6,6 7$%/(,,
0$33,1*(027,2167$7(672675(66/(9(/6
7KH&5,6,6GDWDVHWFRQWDLQVKLJKTXDOLW\UHFRUGLQJVRI
H[SUHVVLYHVSHHFK XQGHUVWUHVV UHDOL]HGLQDQDFRXVWLFDOO\ 6WUHVVOHYHO QHXWUDO PHGLXP KLJK
DGMXVWHG UHFRUGLQJ VWXGLR UHFRUGHG ZLWK D  N+] KDSS\  
QHXWUDO  
VDPSOLQJUDWHDQGELWUHVROXWLRQ7KHDXWKRUVGHVLJQHG VXUSULVHG  
FDOP   VDG  
GLVJXVWHG  
D WKUHHVWHS PHWKRG ZKHQ FUHDWLQJ WKH GDWDVHW 7KH 5$9'(66 KDSS\  
VDG  
IHDUIXO  
VSHDNHUV ZHUH LQVWUXFWHG WR SURQRXQFH WKH VHQWHQFH RQFH VXUSULVHG  
IHDUIXO  
DQJU\  
LQ D QHXWUDO PDQQHU OHYHO  RI WHQVH DURXVDO  WKHQ ZLWK GLVJXVWHG  
DQJU\  
KLJKHULPSHUDWLYHQHVVDVLILWZHUHDVHULRXVFRPPDQGRU QHXWUDO   KDSS\  
LQVWUXFWLRQ OHYHO DQGILQDOO\OLNHDQH[WUHPHO\XUJHQW 686
KDSS\   VXUSULVHG   VDG  
FRPPDQG RU VWDWHPHQW PDGH LQ D VLWXDWLRQ LQ ZKLFK VXUSULVHG   GLVJXVWHG   DQJU\  
KXPDQ OLYHV DUH GLUHFWO\ WKUHDWHQHG OHYHO   $ VLPLODU GLVJXVWHG   VDG  
VFHQDULR ZDV DOVR XVHG ZKHQ UHFRUGLQJ WKH ORZHU WHQVH 
DURXVDO WULSOHW RI WKH GDWDEDVH $IWHU HDFK VHVVLRQ WKH
VSHDNHUVUHOD[HGIRUVHYHUDOVHFRQGVDQGWKHQWKH\VWDUWHG PHGLXP DQG KLJK VWUHVV  DQG RQO\ WKHQ FRXOG ZH VWDUW
ZLWK D QHZ VHQWHQFH 7KH DXWKRUV DVVXPH WKDW WKLV DQDO\]LQJ WKHP :KHQ DVVLJQLQJ HPRWLRQDO VWDWHV WR
DSSURDFK SURGXFHV WKUHH HPRWLRQDOO\ FRQVLVWHQW VHWV RI LQGLYLGXDOVWUHVVOHYHOVZHXVHGIROORZLQJFKDUDFWHULVWLFV
XWWHUDQFHVZLWKWKUHHUHODWLYHO\GLVWLQJXLVKDEOHGHJUHHVRI VXFKDVSRVLWLYHDQGQHJDWLYHVWUHVVLQWHQVLW\DQGWLPEUH
WHQVH DURXVDO 7KH &5,6,6 GDWDVHW FRQWDLQV  ZDUQLQJ RI WKH YRLFH 7KH ILQDO PDSSLQJ RI HPRWLRQDO VWDWHV WR
VHQWHQFHVIRUKLJKHUWHQVHDURXVDO OHYHOV DQG VWUHVV OHYHOV LV VKRZQ LQ 7DEOH ,, )RU HDFK HPRWLRQDO
VHQWHQFHVIRUORZHUWHQVHDURXVDO OHYHOV >@ VWDWHWKHYRLFHLQWHQVLW\SDUDPHWHULVDOVRJLYHQ ORZ
LQWHQVLW\KLJKLQWHQVLW\ 
' 686$6 7DEOH ,,, VXPPDUL]HV WKH DYHUDJH YDOXHV RI WKH
*HQHUDOO\ WKH EHVWNQRZQ GDWDEDVH RI 6SHHFK 8QGHU SDUDPHWHUV )MLWWHU VKLPPHU +15 DQG YRLFH LQWHQVLW\
6LPXODWHG DQG $FWXDO 6WUHVV LV 686$6 7KH GDWDEDVH IRU DOO VSHDNHUV LQ GDWDVHWV DW LQGLYLGXDO VWUHVV OHYHOV ,Q
FRQVLVWV RI VSHHFK UHFRUGLQJV ZLWK D EURDG UDQJH RI DGGLWLRQ ZH DOVR FUHDWHG RQH ODUJH PHUJHG HPRWLRQDO
HPRWLRQV DQG VWUHVV OHYHOV ,W FRQWDLQV RYHU  WKRXVDQG VWUHVVVSHHFKGDWDVHW ODVWFROXPQRI7DEOH,,, XVDEOHIRU
HPRWLRQDO VSHHFK XWWHUDQFHV UHFRUGHG XQGHU ERWK DFWHG WUDLQLQJV\VWHPVWRGHWHFWWKHSUHVHQFHRIVWUHVVLQVSHHFK
DQG VLPXODWHG VWUHVV FROOHFWHG IURP  (QJOLVK VSHDNHUV +RZHYHULWVKRXOGEHQRWHGWKDWZHZRUNHGPDLQO\ZLWK
 PDOHV DQG  IHPDOHV  LQ IRXU GLIIHUHQW FRQGLWLRQV GDWDEDVHVLQZKLFKLQGLYLGXDOOHYHOVRIHPRWLRQDQGVWUHVV
QHXWUDOVWUHVVPHGLXPVWUHVVKLJKVWUHVVDQGVFUHDPLQJ LQVSHHFKZHUHVLPXODWHGE\DFWRUV H[FHSWIRUSDUWRIWKH
7KH WLPH GXUDWLRQ RI DXGLR VDPSOHV VSDQV IURP  WR  686$6 GDWDVHW  )RU WKLV UHDVRQ WKH YDOXHV RI WKH
VHFRQGV 0RVW RI WKH XWWHUDQFHV DUH RQH RU WZRZRUG SDUDPHWHUVPD\QRWFRLQFLGHZLWKWKHYDOXHVLIWKH\ZHUH
FRPPDQGV VRPH FRQWDLQ EDFNJURXQG QRLVH DQG WKH VSHHFKUHFRUGLQJVXQGHUDFWXDOVWUHVV
UHFRUGLQJVKDYHDORZVDPSOLQJIUHTXHQF\ N+] >@ %DVHGRQWKHYDOXHVVXPPDUL]HGLQ7DEOH,,,LWFDQEH
FRQFOXGHGWKDWDVWKHIXQGDPHQWDOIUHTXHQF\)DQGYRLFH
( 6WUHVV'DW LQWHQVLW\ LQFUHDVH WKH OHYHO RI VWUHVV LQFUHDVHV ZKLFK
7KH6WUHVV'DWLVDFROOHFWLRQRIDFWHGVSHHFKUHFRUGLQJV DJUHHV ZLWK WKH UHVXOWV RI WKH UHVHDUFK VWXGLHV PHQWLRQHG
LQWKH6ORYDNODQJXDJHZKHUHDFWRUVXWWHUHGVHQWHQFHVLQ LQ6HFWLRQ,,>@>@,IZHORRNDWWKHYDOXHVRIWKHMLWWHU
WKUHH GLIIHUHQW VWUHVV OHYHOV 7KH GDWDVHW LV JHQGHU VKLPPHU DQG +15 SDUDPHWHUV IRU LQGLYLGXDO GDWDEDVHV
EDODQFHG ZLWK  VSHDNHUV  IHPDOHV DQG  PDOHV  ZH FDQQRW FOHDUO\ GHPRQVWUDWH WKHLU LQIOXHQFH RQ VSHHFK
(DFKSDUWLFLSDQWZDVUHSUHVHQWHGZLWKYDULRXVVFHQDULRVRI VWUHVVOHYHOV+RZHYHULIZHORRNDWWKHYDOXHVLQWKHODVW
SRWHQWLDOO\ VWUHVVIXO VLWXDWLRQV DQG WKH FRQWH[W WKDW FROXPQ RI WKH 7DEOH ,,, IRU WKH PHUJHG GDWDVHW WKH
PDQLSXODWHV WKH VWUHVV OHYHO IURP ORZ RYHU PHGLXP WR DYHUDJH YDOXHV RI WKH SDUDPHWHUV MLWWHU VKLPPHU DQG
KLJK (DFK VFHQDULR FRQWDLQHG PXOWLSOH UHODWHG VHQWHQFHV +15GHFUHDVHZLWKLQFUHDVLQJVWUHVVOHYHOV7KLVFRQILUPV
DQGWKHVXEMHFWVZHUHDVNHGWRUHDGDOOVFHQDULRVHQWHQFHV WKDW WKH ODUJHU DQG PRUH GLYHUVH WKH GDWDVHW RI HPRWLRQV
LQWKHWKUHHVWUHVVOHYHOV7KHGDWDVHWFRQVLVWVRIDWRWDORI DQG VWUHVV ZH KDYH WKH KLJKHU WKH DFFXUDF\ RI WKH
 XWWHUDQFHV VSDQQLQJ IURP  WR  VHFRQGV 7KH UHVXOWLQJ DOJRULWKP IRU GHWHFWLQJ VWUHVV LQ VSHHFK FDQ EH
PHDQ GXUDWLRQ RI XWWHUDQFHV LV  VHFRQGV 7KH GDWDVHW ,QDGGLWLRQIXUWKHULQFUHDVHLQWKHVL]HRIWKHGDWDVHWFRXOG
FRPSULVHVDWRWDORIKRXUV>@ EH DFKLHYHGE\ GDWD DXJPHQWDWLRQ IRU H[DPSOHE\ XVLQJ
JHQHUDWLYHDGYHUVDULDOQHWZRUNV *$1 
,9 $1$/<6,62)63((&+&+$5$&7(5,67,&6
,QRUGHUWREHDEOHWRVWDUWWUDLQLQJFODVVLILHUVWRGHWHFW 9 (;3(5,0(176
VWUHVV OHYHOV LQ VSHHFK LW ZDV ILUVW QHFHVVDU\ WR DQDO\]H 7KH H[SHULPHQWV ZHUH IRFXVHG RQ WUDLQLQJ DQG
LQGLYLGXDO GDWDEDVHV IURP WKH SRLQW RI YLHZ RI EDVLF HYDOXDWLQJ FODVVLILHUV IRU WKH DXWRPDWLF SUHGLFWLRQ RI
VSHHFK SDUDPHWHUV VXFK DV IXQGDPHQWDO IUHTXHQF\ ) VWUHVV OHYHOV RQ WKH GDWD PHQWLRQHG LQ 6HFWLRQ ,,,
MLWWHUVKLPPHUKDUPRQLFWRQRLVHUDWLR +15 DQGYRLFH $OWKRXJK WKH WRRO XVHG RIIHUV VHYHUDO FODVVLILFDWLRQ
LQWHQVLW\ 7KLV DQDO\VLV DLPHG WR FRQILUP ILQGLQJV IURP
PHWKRGV VXFK DV 1DwYH %D\HV 1%  FODVVLILHU ORJLVWLF
RWKHU VWXGLHV RQ KRZ LQGLYLGXDO SDUDPHWHUV FKDQJH ZLWK
LQFUHDVLQJRUGHFUHDVLQJOHYHOVRIVWUHVV UHJUHVVLRQ VXSSRUW YHFWRU PDFKLQHV 690  NQHDUHVW
QHLJKERU FODVVLILHU N11  HWF LQ WKH H[SHULPHQWDO SDUW
6LQFH ZH ZRUNHG ZLWK HPRWLRQDO GDWDVHWV 5$9'(66
ZHFKRVHRQO\WZRPHWKRGVWKDWSURGXFHGWKHRYHUDOOEHVW
DQG 686 LW ZDV QHFHVVDU\ WR ILQG D FRUUHODWLRQ EHWZHHQ
HPRWLRQDOVWDWHVDQGVWUHVVOHYHOVDQGWRXQLWHWKHPIURP UHVXOWV QDPHO\ VWDQGDUG GHFLVLRQ WUHHV DQG HQVHPEOH
WKH SRLQW RI YLHZ RI WKUHH EDVLF VWUHVV OHYHOV QHXWUDO PHWKRGRIEDJJLQJGHFLVLRQWUHHV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 495

 7$%/(,,,
(;75$&7('92,&(&+$5$&7(5,67,&6$&52666(/(&7('(027,21$/$1'63((&+'$7$6(76
9RLFH 686$6 686$6 0HUJHG
6WUHVVOHYHO 5$9'(66 686 &5,6,6 6WUHVV'DW
FKDUDFWHULVWLF DFWXDO VLPXODWHG GDWDVHW
QHXWUDO       
)>+]@ PHGLXP       
KLJK       
QHXWUDO       
MLWWHU>@ PHGLXP       
KLJK       
QHXWUDO       
VKLPPHU>G%@ PHGLXP       
KLJK       
QHXWUDO       
+15>G%@ PHGLXP       
KLJK       
QHXWUDO       
LQWHQVLW\>G%@ PHGLXP       
KLJK       


)LJXUH %ORFNVFKHPHRIWKHSURSRVHGDSSURDFKIRUSUHGLFWLRQRIOHYHORIVWUHVVIURPVSHHFKVLJQDOV

:KHQ GHVLJQLQJ RXU RZQ VROXWLRQ IRU GHWHFWLQJ WKH

2QH GUDZEDFN RI FRQYHQWLRQDO GHFLVLRQ WUHHV LV WKDW SUHVHQFH RI VWUHVV LQ VSHHFK DQG FODVVLI\LQJ WKH VWUHVV
SURGXFH KLJKYDULDQFH HVWLPDWRUV ,Q WKH FDVH RI D VPDOO OHYHOZHZHUHLQVSLUHGE\WKHUHVHDUFKSUHVHQWHGLQ>@
QXPEHU RI WUDLQLQJ GDWD REVHUYDWLRQV FDQ GUDPDWLFDOO\ 7KH ZKROH SURFHVV RI WUDLQLQJ DQG FODVVLILFDWLRQ LV
DOWHUWKHSUHGLFWLRQSHUIRUPDQFHRIDOHDUQHGWUHH GHSLFWHGLQ)LJXUH,QWKHILUVWVWHSDVSHFLILFGDWDVHWRI
%DJJLQJ DOVR ERRWVWUDS DJJUHJDWLRQ  LV WKH HQVHPEOH HPRWLRQDO VWUHVV VSHHFK ZDV ORDGHG )RU HDFK VSHHFK
OHDUQLQJ PHWKRG WKDW LV FRPPRQO\ XVHG WR UHGXFH UHFRUGLQJLQWKHGDWDVHWWKH7HDJHUHQHUJ\RSHUDWRU 7(2 
YDULDQFH ZLWKLQ D QRLV\ GDWDVHW (QVHPEOH OHDUQLQJ LV ZDV ILUVW FDOFXODWHG DQG WKHQWKH 0)&&V ZHUH H[WUDFWHG
EDVHG RQ WKH LGHD RI WKH µZLVGRP RI FURZGV¶ ZKLFK 7(2ZDVSURSRVHGIRUPRGHOLQJQRQOLQHDUVSHHFKVLJQDOV
VXJJHVWVWKDWWKHGHFLVLRQPDNLQJRIDODUJHUJURXSRILV DQGKLJKOLJKWVWKHIUHTXHQF\DQGLQVWDQWDQHRXVFKDQJHVRI
W\SLFDOO\ EHWWHU WKDQ WKDW RI DQ LQGLYLGXDO ,Q EDJJLQJ D WKH VLJQDO DPSOLWXGH ZKLFK LV YHU\ VHQVLWLYH WR VXEWOH
UDQGRP VDPSOH RI GDWD LQ D WUDLQLQJ VHW LV VHOHFWHG ZLWK FKDQJHV 0)&&V DUH D FRPSDFW UHSUHVHQWDWLRQ RI WKH
UHSODFHPHQW PHDQLQJWKDWWKHLQGLYLGXDOGDWDSRLQWVFDQ VSHFWUXP RI D VSHHFK VLJQDO ZKLFK GHVFULEHV WKH RYHUDOO
VKDSH RI WKH VSHFWUDO HQYHORSH ,Q DGGLWLRQ ZH DOVR
EHFKRVHQPRUHWKDQRQFH$IWHUVHYHUDOGDWDVDPSOHVDUH
H[WUDFWHG RWKHU YRLFH SDUDPHWHUV VXFK DV 3HDN3HDN
JHQHUDWHG WKHVH LQGLYLGXDO PRGHOV DUH WKHQ WUDLQHG 0D[/RFV0D[:LGWKV 0HDQ3LWFK DQG VRRQ WRDFKLHYH
LQGHSHQGHQWO\DQGGHSHQGLQJRQWKHW\SHRIWDVN+HQFH EHWWHU DFFXUDF\ RI WKH FODVVLILHUV ,Q WKH QH[W VWHS ZH
HDFK LQGLYLGXDO WUHH KDV KLJK YDULDQFH EXW ORZ ELDV FKRVHWKHDPRXQWRIGDWDWKDWZDVXVHGLQWKHWUDLQLQJDQG
7KHQ DYHUDJLQJ WKH WUHHV UHGXFHV WKH YDULDQFH \LHOGLQJ YDOLGDWLRQ VWHS ,Q RXU FDVH ZH XVHG DQ  UDWLR RI
EHWWHUPRGHOSHUIRUPDQFH>@>@ UDQGRPO\ VHOHFWHG GDWD IRU WUDLQLQJ DQG YDOLGDWLRQ
6XEVHTXHQWO\ D VXLWDEOH FODVVLILHU ZDV VHOHFWHG DQG LWV
$ ([SHULPHQWDOVHWXS SDUDPHWHUV ZHUH VHW $V ZH PHQWLRQHG HDUOLHU ZH FKRVH
7RLPSOHPHQWWKHSURSRVHGDSSURDFKIRUFODVVLILFDWLRQ FODVVLILHUV EDVHG RQ GHFLVLRQ WUHHV DQG HQVHPEOH EDJJHG
RI VSHHFK LQWR LQGLYLGXDO VWUHVV OHYHOV QRUPDO PHGLXP WUHHV %DVHG RQ WKLV D FODVVLILFDWLRQ PRGHO ZDV WUDLQHG
KLJK ZHXVHGWKH0$7/$%&ODVVLILFDWLRQ/HDUQHU$SS ZKLFKZDVODWHUXVHGIRUWKHSXUSRVHRIFODVVLI\LQJVSHHFK
  LQWR WKUHH VWUHVV OHYHOV )LQDOO\ WKH FODVVLILFDWLRQ PRGHO

KWWSVZZZPDWKZRUNVFRPKHOSVWDWVWUDLQFODVVLILFDWLRQPRGHOVLQ ZDVHYDOXDWHGRQWKHYDOLGDWLRQGDWD
FODVVLILFDWLRQOHDUQHUDSSKWPO

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 496

 7$%/(,9 9, &21&/86,21
7+(727$/180%(52)63((&+877(5$1&(6,17+('$7$6(76
7KHPDLQILQGLQJVRIWKLVUHVHDUFKFDQEHVXPPDUL]HG
1XPEHURI 0HUJHG DVIROORZV
&5,6,6 686$6 6WUHVV'DW
VSHHFKXWWHUDQFHV GDWDVHW
QRUPDOVWUHVV      WKHUH H[LVWV DODUJH QXPEHU RI HPRWLRQDO VSHHFK
PHGLXPVWUHVV     GDWDVHWV WKDW FDQ EH XVHG WR H[WHQG WKH H[LVWLQJ
KLJKVWUHVV     FROOHFWLRQRIGDWDVHWVFRQWDLQLQJVSHHFKXQGHUVWUHVV
E\DSSURSULDWHO\PDSSLQJHPRWLRQVLQWRWKHVSDFHRI
VWUHVV OHYHOV DQG WKXV FUHDWH ODUJH HPRWLRQDOVWUHVV
7$%/(9 VSHHFKGDWDVHWV
(;3(5,0(17$/5(68/76
 D IXUWKHU FRQWULEXWLRQ ZRXOG EH D EHWWHU
$FFXUDF\
&5,6,6 686$6 6WUHVV'DW
0HUJHG XQGHUVWDQGLQJRIHPRWLRQDOVWUHVVE\LWVPDSSLQJLQ
>@ GDWDVHW WKH'DURXVDOYDOHQFHVSDFH
'HFLVLRQ7UHH
0)&&    
 WKHUHDUHRQO\DIHZGDWDVHWVRIVSHHFKXQGHUDFWXDO
7(20)&&     VWUHVV KRZHYHU SXEOLF VRFLDO QHWZRUNV DUH IXOO RI
%DJJHG7UHHV YDULRXVSRVWVFRQWDLQLQJUHFRUGHGVWUHVVIXOVLWXDWLRQV
0)&&     WKHVHFRXOGEHXVHGWRFUHDWHVXFKDQQRWDWHGGDWDVHWV
7(20)&&      IXUWKHULQFUHDVHLQWKHGDWDVHWVL]HFRXOGEHDFKLHYHG
 E\ WKH WHFKQLTXHRIGDWDDXJPHQWDWLRQ IRU H[DPSOH
E\XVLQJJHQHUDWLYHDGYHUVDULDOQHWZRUNV *$1 
% 5HVXOWV  IXQGDPHQWDO IUHTXHQF\ ) DQG YRLFH LQWHQVLW\ DUH
:HXVHGDFFXUDF\WRHYDOXDWHWKHFODVVLILFDWLRQPRGHOV JRRG SUHGLFWRUV RI VWUHVV OHYHOV RQ WKH RWKHU KDQG
7KH PRGHOV ZHUH WUDLQHG VHSDUDWHO\ RQ WKH &5,6,6 XVLQJ SDUDPHWHUV VXFK DV MLWWHU VKLPPHU RU +15
686$6DQG6WUHVV'DWGDWDVHWVDVZHOODVRQWKHPHUJHG ZHFDQQRWFOHDUO\GHWHUPLQHWKHSUHVHQFHRIVWUHVVLQ
GDWDVHW ZKLFK DGGLWLRQDOO\ FRQWDLQHG HPRWLRQDO VSHHFK DVSHHFK
GDWD IURP WKH 5$9'(66 DQG 686 GDWDVHWV WKDW ZHUH  WKHXVHRIWKH7HDJHUHQHUJ\RSHUDWRULVUHOHYDQWRQO\
FODVVLILHGLQWRWKUHHVWUHVVOHYHOV QRUPDOPHGLXPKLJK  LQWKHFDVHRIQRLV\VSHHFKUHFRUGLQJVWRUHGXFHWKH
$V FDQ EH VHHQ LQ 7DEOH,9WKH VSHHFK XWWHUDQFHV DUH QRLVHDQGKLJKOLJKWWKHHQHUJ\RIWKHVLJQDO
XQHYHQO\GLVWULEXWHGDPRQJWKHVWUHVVOHYHOV)RUH[DPSOH  WKH XVH RI FRQYROXWLRQDO QHXUDO QHWZRUNV LQ WKH
XWWHUDQFHV ZLWK KLJK VWUHVV KDYH WKH ORZHVW DPRXQWV :H SURFHVVRIVWUHVVOHYHOFODVVLILFDWLRQFDQVLJQLILFDQWO\
VHOHFWHG  UDQGRP XWWHUDQFHV IRU HDFK VWUHVV OHYHO LPSURYHWKHDFFXUDF\RIWKHUHVXOWLQJV\VWHP>@
IURPWKHPHUJHGGDWDVHWDQGWUDLQHGWKHFODVVLILHURQWKLV )URP D FRPPHUFLDO XVH SRLQWRIYLHZ WKHVH ILQGLQJV
GDWDWRHQVXUHWKDWWKHFODVVLILHUGRHVQRWRYHUWUDLQ FDQ JUHDWO\ DLG LQ WKH UHVHDUFK DQG GHYHORSPHQW RI
7KHUHVXOWVRIWKHSUHGLFWLRQDQGFODVVLILFDWLRQRIVWUHVV V\VWHPV WR DXWRPDWLFDOO\ PRQLWRU V\PSWRPV RI DFXWH
OHYHOV DUH VKRZQ LQ 7DEOH 9 )RU FRPSDULVRQ ZH DOVR VWUHVVGHSUHVVLRQRUPHQWDODQGHPRWLRQDOVWDWHVIURPWKH
SHUIRUPHGWKHUHVXOWVZKHQ7(2ZDVH[WUDFWHGDORQJZLWK YRLFHFKDUDFWHULVWLFV7KHQVXFKDV\VWHPZRXOGEHXVHG
0)&&DQGZKHQRQO\0)&&ZDVH[WUDFWHGZLWKRXW7(2 DVDQDX[LOLDU\GLDJQRVWLFWRRORUDWRROIRUGHWHUPLQLQJD
7KH UHVXOWV RI WKH DFFXUDF\ RI WKH PRGHOV DUH VRPHZKDW XVHU VPHQWDORUKHDOWKFRQGLWLRQ
GHEDWDEOH WKH LQIOXHQFH RI WKH XVH RI 7(2 LV QHJOLJLEOH 2XU IXUWKHU UHVHDUFK ZLOO IRFXV RQ WKH H[WUDFWLRQ DQG
ZKLFK PD\ EH FDXVHG E\ VHYHUDO IDFWRUV WKDW DIIHFW LWV DSSOLFDWLRQRIVSHHFKIHDWXUHVRWKHUWKDQ0)&&VVXFKDV
FDOFXODWLRQ DQG DUH PDLQO\ UHODWHG WR WKH VLJQDO HQHUJ\ EDUNIUHTXHQF\FHSVWUDOFRHIILFLHQWV %)&&V JDPPDWRQH
7KH7(2SDUDPHWHULVPDLQO\XVHGWRVXSSUHVVWKHQRLVH IUHTXHQF\FHSVWUDOFRHIILFLHQWV *)&&V OLQHDUIUHTXHQF\
FRPSRQHQWV DQG KLJKOLJKW WKH HQHUJ\ FRPSRQHQWV LQ WKH FHSVWUDO FRHIILFLHQWV /)&&V  DQG RWKHUV >@ RU WKHLU
VLJQDO,WVXVHORVHVLWVVLJQLILFDQFHLQKLJKTXDOLW\VSHHFK FRPELQDWLRQRQYHFWRUL]DWLRQDQGTXDQWL]DWLRQRIVSHHFK
UHFRUGHGLQVWXGLRFRQGLWLRQV SDUDPHWHUV RQ WKH XVH RI FRQYROXWLRQDO QHXUDO QHWZRUNV
:HDFKLHYHGWKHEHVWRYHUDOOUHVXOWVLQVWUHVVSUHGLFWLRQ &11V  ZLWK GHQVHIXOO\ FRQQHFWHG OD\HUV LQ WKH SURFHVV
DQG FODVVLILFDWLRQ XVLQJ EDJJHG WUHHV ,Q WKH FDVH RI WKH RI WUDLQLQJ D FODVVLILFDWLRQ PRGHO DQG LPSOHPHQWLQJ RI
&5,6,6 GDWDVHW ZH REWDLQHG  FODVVLILFDWLRQ HQGWRHQGV\VWHPVIRUGHWHFWLQJWKHSUHVHQFHRIVWUHVVLQ
DFFXUDF\LQWKHFDVHRIWKH686$6GDWDVHWZLWK KXPDQVSHHFK
WKH 6WUHVV'DW GDWDVHW LW ZDV RQO\  DQG RQ WKH
VWUHVVEDODQFHGPHUJHGGDWDVHWWKHDFFXUDF\ZDV $&.12:/('*0(17
ZKLFK DUH UHVXOWV FRPSDUDEOH WR RWKHU DOUHDG\ SXEOLVKHG
7KH UHVHDUFK LQ WKLV SDSHU ZDV SDUWLDOO\ VXSSRUWHG E\
UHVHDUFK VWXGLHV QDPHO\ >@>@ 2YHUDOO ZH DFKLHYHG
WKH0LQLVWU\RI(GXFDWLRQ6FLHQFH5HVHDUFKDQG6SRUWRI
DQ DYHUDJH FODVVLILFDWLRQ DFFXUDF\ RI  %HWWHU UHVXOWV
WKH 6ORYDN 5HSXEOLF XQGHU WKH UHVHDUFK SURMHFW 9(*$
FRXOG EH DFKLHYHG XVLQJ FODVVLILFDWLRQ PRGHOV EDVHG RQ
 E\ WKH 6ORYDN 5HVHDUFK DQG 'HYHORSPHQW
FRQYROXWLRQDOQHXUDOQHWZRUNV>@
$JHQF\ XQGHU WKHSURMHFW RI ELODWHUDO FRRSHUDWLRQ $399
,W VKRXOG EH QRWHG WKDW WKH FODVVLILFDWLRQ OHDUQHU WRRO 6.7: DQG UHVHDUFK SURMHFWV $399
RQO\ RIIHUHG XV DQ HYDOXDWLRQ XVLQJ DFFXUDF\ PRUH DQG $399 DQG E\ WKH )DFXOW\ RI (OHFWULFDO
GHWDLOHG UHVXOWV ZRXOG EH DFKLHYHG LI ZH HYDOXDWHG (QJLQHHULQJ DQG ,QIRUPDWLFV 7HFKQLFDO 8QLYHUVLW\ RI
SUHFLVLRQUHFDOODQG)VFRUH .RãLFHXQGHUWKH*UDQW)(,



979-8-3503-7069-0/23/$31.00 ©2023 IEEE 497

 5()(5(1&(6 :RUNVKRS RQ $XGLR9LVXDO 6FHQH 8QGHUVWDQGLQJ IRU ,PPHUVLYH
0XOWLPHGLD6HRXO5HSXEOLFRI.RUHDSS
>@ ) 'HOODHUW 7 3RO]LQ DQG $ :DLEHO ³5HFRJQLWLRQ HPRWLRQ LQ
>@ 6 9DLNROH 6 0XODMNDU $ 0RUH 3 -D\DVZDO DQG 6 'KDV
VSHHFK´ LQ 3URF RI ,&6/3 3KLODGHOSKLD 3$ 86$  SS
³6WUHVV GHWHFWLRQ WKURXJK VSHHFK DQDO\VLV XVLQJ PDFKLQH

OHDUQLQJ´ ,QWHUQDWLRQDO -RXUQDO RI &UHDWLYH 5HVHDUFK 7KRXJKWV
>@ -+/ +DQVHQ DQG 6( %RX*KD]DOH ³*HWWLQJ VWDUWHG ZLWK   
686$6$VSHHFKXQGHUVLPXODWHGDQGDFWXDOVWUHVVGDWDEDVH´LQ
>@ 65 /LYLQJVWRQH DQG )$ 5XVVR ³7KH 5\HUVRQ DXGLRYLVXDO
3URFRI(85263((&+5KRGHV*UHHFH
GDWDEDVHRIHPRWLRQDOVSHHFKDQGVRQJ 5$9'(66 $G\QDPLF
>@ -9DPVLQDWK%9DUVKLQL76DQGHHS90HJKDQDDQG%/DWKD PXOWLPRGDOVHWRIIDFLDODQGYRFDOH[SUHVVLRQVLQ1RUWK$PHULFDQ
³$ VXUYH\ RQ VWUHVV GHWHFWLRQ WKURXJK VSHHFK DQDO\VLV XVLQJ (QJOLVK´3/R621(  H
PDFKLQHOHDUQLQJ´,QWHUQDWLRQDO-RXUQDORI6FLHQWLILF5HVHDUFKLQ
>@ 06+ +LOP\ $/ $VQDZL $= -XVRK . $EGXOODK 61
6FLHQFHDQG7HFKQRORJ\  SS
,EUDKLP+$05DPOLDQG1)0$]PLQ³6WUHVVFODVVLILFDWLRQ
>@ 0:|OOPHU)(\EHQ65HLWHU%6FKXOOHU&&R[('RXJODV EDVHGRQVSHHFKDQDO\VLVRI0)&&IHDWXUHYLDPDFKLQHOHDUQLQJ´
&RZLH DQG 5 &RZLH ³$EDQGRQLQJ HPRWLRQ FODVVHV  7RZDUGV LQ 3URF RI WKH WK ,QWHUQDWLRQDO &RQIHUHQFH RQ &RPSXWHU DQG
FRQWLQXRXV HPRWLRQ UHFRJQLWLRQ ZLWK PRGHOLQJ RI ORQJUDQJH &RPPXQLFDWLRQ (QJLQHHULQJ ,&&&(  .XDOD /XPSXU 0DOD\VLD
GHSHQGHQFLHV´ LQ 3URF RI ,17(563((&+ %ULVEDQH $XVWUDOLD SS
SS
>@ 5 6DER - 5DMþiQL DQG 0 5LWRPVNê ³'HVLJQLQJ GDWDEDVH RI
>@ $ 6WXKOVDW] &K 0H\HU ) (\EHQ 7 =LHONH * 0HLHU % VSHHFKXQGHUVWUHVVXVLQJDVLPXODWLRQLQYLUWXDOUHDOLW\´LQ3URF
6FKXOOHU³'HHSQHXUDOQHWZRUNVIRUDFRXVWLFHPRWLRQUHFRJQLWLRQ RI ',6$  ± ,((( :RUOG 6\PSRVLXP RQ 'LJLWDO ,QWHOOLJHQFH
5DLVLQJ WKH EHQFKPDUNV´ LQ 3URF RI ,&$663 3UDJXH &]HFK IRU6\VWHPVDQG0DFKLQHV.RãLFH6ORYDNLDSS
5HSXEOLFSS
>@ / 0DFNRYi $ ýLåPiU DQG - -XKiU ³$ VWXG\ RI DFRXVWLF
>@ 4 0DR 0 'RQJ = +XDQJ DQG < =KDQ ³/HDUQLQJ VDOLHQW IHDWXUHV IRU HPRWLRQDO VSHDNHU UHFRJQLWLRQ LQ LYHFWRU
IHDWXUHVIRUVSHHFKHPRWLRQUHFRJQLWLRQXVLQJFRQYROXWLRQDOQHXUDO UHSUHVHQWDWLRQ´$FWD(OHFWURWHFKQLFDHW,QIRUPDWLFD  
QHWZRUNV´,(((7UDQVDFWLRQVRQ0XOWLPHGLD   SS
>@ * 7ULJHRUJLV ) 5LQJHYDO 5 %UXHFNQHU ( 0DUFKL 0$ >@ 0 5XVNR 6 'DUMDD 0 7UQND 5 6DER DQG 0 5LWRPVNê
1LFRODRX%6FKXOOHUDQG6=DIHLULRX³$GLHXIHDWXUHV"(QGWR ³([SUHVVLYH VSHHFK V\QWKHVLV IRU FULWLFDO VLWXDWLRQV´ &RPSXWLQJ
HQG VSHHFK HPRWLRQ UHFRJQLWLRQ XVLQJ D GHHS FRQYROXWLRQDO DQG,QIRUPDWLFV  SS
UHFXUUHQWQHWZRUN´LQ3URFRI,&$6636KDQJKDL&KLQD
>@ 5 6DER â %HĖXã 0 7UQND 0 5LWRPVNê 0 5XVNR 0
>@ %% 5DG DQG 0 0RUDGKDVHOL ³6SHHFK HPRWLRQ UHFRJQLWLRQ 6FKDSHU DQG - 6]DER ³6WUHVV'DW ± 'DWDEDVH RI VSHHFK XQGHU
PHWKRGV $ OLWHUDWXUH UHYLHZ´ $,3 &RQIHUHQFH 3URFHHGLQJV VWUHVVLQ6ORYDN´-RXUQDORI/LQJXLVWLFV  SS
  
>@ ' 2SLW] DQG 5 0DFOLQ ³3RSXODU HQVHPEOH PHWKRGV $Q
>@ 6 =DIHLULRX ' .ROOLDV 0$ 1LFRODRX $ 3DSDLRDQQRX * HPSLULFDO VWXG\´ -RXUQDO RI $UWLILFLDO ,QWHOOLJHQFH 5HVHDUFK
=KDR , .RWVLD ³$II:LOG 9DOHQFH DQG DURXVDO µLQWKHZLOG¶   SS
FKDOOHQJH´ LQ 3URF RI WKH WK ,((( &RQIHUHQFH RQ &RPSXWHU
>@ %UHLPDQ / ³5DQGRP IRUHVWV´ 0DFKLQH /HDUQLQJ 
9LVLRQDQG3DWWHUQ5HFRJQLWLRQ+RQROXOX+DZDLL86$
SDJHV
>@ * 'HPHQNR DQG 0 -DVWU]HEVND ³$QDO\VLV RI QDWXUDO VSHHFK
>@ 65 %DQGHOD DQG 7. .XPDU ³6WUHVVHG VSHHFK HPRWLRQ
XQGHU VWUHVV´ $FWD 3K\VLFD 3RORQLFD 6HULHV $  $  
UHFRJQLWLRQ XVLQJ IHDWXUH IXVLRQ RI 7HDJHU HQHUJ\ RSHUDWRU DQG
SS$$
0)&&´ LQ 3URF RI WKH WK ,QWHUQDWLRQDO &RQIHUHQFH RQ
>@ 0 YDQ 3X\YHOGH ; 1H\W ) 0F*ORQH DQG 1 3DWW\Q ³9RLFH &RPSXWLQJ &RPPXQLFDWLRQ DQG 1HWZRUNLQJ 7HFKQRORJLHV
VWUHVV DQDO\VLV $ QHZ IUDPHZRUN IRU YRLFH DQG HIIRUW LQ KXPDQ ,&&&17 'HOKL,QGLDSS
SHUIRUPDQFH´)URQWLHUVLQ3V\FKRORJ\$UWLFOH
>@ 99 <HULJHUL DQG /. 5DJKD ³6SHHFK VWUHVV UHFRJQLWLRQ XVLQJ
>@ ++DQ.%\XQDQG+*.DQJ³$GHHSOHDUQLQJEDVHGVWUHVV VHPLHDJHU OHDUQLQJ´ &RJQLWLYH 6\VWHPV 5HVHDUFK   SS
GHWHFWLRQ DOJRULWKP ZLWK VSHHFK VLJQDO´ LQ 3URF RI $968 


979-8-3503-7069-0/23/$31.00 ©2023 IEEE 498

 The Next Big Thing in University Education:
a Threat or an Opportunity?
L. Stuchlikova*, J. Stuchlik**, M. Weis*
* Slovak University of Technology, Faculty of Electrical Engineering and Information Technology,
Institute of Electronics and Photonics, Bratislava, Slovakia
** The Prague Institute of Planning and Development, Praha, Czechia

lubica.stuchlikova@stuba.sk, stuchlik@ipr.praha.eu, martin.weis@stuba.sk

Abstract— Large Language Models (LLMs) have emerged results while carefully avoiding the problems that come
as powerful artificial intelligence models capable of natural with using technology.
language understanding and generation. The hundreds of This paper starts a trip through the varied space where
millions of registered users of LLMs have emerged as LLMs and university education meet, aiming to explore
a global phenomenon, causing substantial disruption, not the many parts of this combination. It looks into the many
only on the internet but also for society at large. opportunities that LLMs open, like personalized learning,
Universities, as influential institutions with extensive societal improved access, and better content creation, while also
reach, have experienced the profound impact of this trend, moving through the challenges and limits that come with
unfortunately without adequate moderation. It is crucial to
them, including ethical thoughts, data safety, and possible
address this issue without succumbing to the extremes of
biases.
ludists' scepticism or unwavering techno-optimism. This
contribution explores the opportunities, challenges and Also, within the halls of universities where knowledge,
limitations associated with integrating LLMs into university ethics, and future leadership mix, it is vital to move
education, aiming to identify a balanced and constructive through the choice of welcoming new tech advances and
role for them within the university environment. It offers keeping the human-centered core of education. Therefore,
insights into how LLMs can be harnessed to meet the this paper tries to make a way that neither only follows
evolving needs of diverse student populations, enhance the technology nor ignores it, but rather looks for a balanced
quality of educational content, and transform pedagogical relationship where technology and teaching come together
practices. to mold a future that is including, fair, and always looking
ahead.
Keywords: Large Language Models (LLMs), University As we go on this exploration, it is very important to
education, Opportunities and Challenges look at the subject with a view that is critically checking,
ethically aware, and always focused on students. So, this
paper tries to move through the theoretical and practical
I. INTRODUCTION parts of LLM use in university places, hoping to start
In the meeting point of growing technology and conversations, cause critical thinking, and form strategies
teaching methods, the way universities teach and learn has that line up tech progress with the main goals of education
changed in a way we've never seen before [1]. The arrival in nurturing knowledge, critical thinking, and ethical
of Large Language Models (LLMs), like GPT-4, and their citizens for the coming generations.
amazing skill to understand, create, and communicate in
natural language, have caught the interest of tech experts II. INTRODUCTION ONCE AGAIN
and also moved into different areas of society, offering
exciting possibilities and creating new problems [2]. The Everything you had read by now including abstract and
question in education is not if these new tools will change introduction was generated using the LLM GPT-4 [4] and
how we teach and learn, but how we, as teachers and tech the references suggested by LLM Bard [5]. The process
experts, manage these changes [3]. included a few iterations with LLMs as well as with
human colleagues. However, the resulting text is a word-
The increase of LLMs and their use in different fields by-word generation of LLMs.
have made them more than just amazing technology but
also things that can change how we share information, This ought to serve as a primitive example of the LLM
create knowledge, and basically, how we educate. Putting usage possibilities. Moreover, we aim for a responsible,
these models into university education is not only a tech transparent and moral application.
change but also a change in thinking and teaching, Even though machine learning and neural networks
needing detailed talks about their role, effect, ethics, and have been investigated for decades [6], artificial
limits within the school setting. intelligence (AI) was developed just recently [7]. First, AI
Universities, as centers of knowledge and new ideas, found applications in big data processing such as medical
find themselves on the edge of this tech change, with the science [8], chemistry [9], internet-of-things [10], or
double job of using the potential of LLMs while also industry [11]. However, the rise of LLMs capable of
protecting the honesty and fairness of education. The imitating human communication made a breakthrough that
balance is delicate and important - to put LLMs into had a spill-over effect on the whole of human society,
education in a way that lifts educational experiences and including education [12].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 499

 The questions of integrity, effectiveness and possible commencing with an unsupervised learning approach. In
inclusivity are the essential core of our brief paper. The this phase, the model encounters unstructured and
main aim of this piece is to present the LLM teaching unlabeled data. The advantage of training on unlabeled
opportunities and at the same time to address the major data lies in the fact that there is much more of this data
challenges of LLMs implementation in the education available. During this stage, the model begins to discern
process. AI in the form of LLM is a powerful tool that connections between various words and concepts.
must be well understood and moderated to be reasonably The next step for some LLMs is training and fine-
used in the education process. The threats such as biases, tuning in the form of self-supervised learning. Here, some
ethical dilemmas or legal obstacles but also teaching labeling of the data has occurred, which helps the model
suggestions are discussed. to identify different concepts with greater precision.
Following this, the LLM performs deep learning using
III. LARGE LANGUAGE MODELS (LLMS) a transformer neural network. The architecture of the
LLMs belong to the wide family of AI. It is an transformer model allows the LLM to comprehend and
algorithm that uses deep learning techniques and identify relationships and connections between words and
massively large datasets to understand, summarize, concepts, utilizing a self-attention mechanism. This
generate, and predict new content [1]. mechanism assigns a score, often referred to as a weight,
LLM, is a type of generative AI that has been to a specific item (termed a token) to determine the
specifically designed to generate content in the form of strength of the relationship.
text. In simpler words, it generates text based on a very Once the LLM completes its training, it becomes a
complex set of rules that can evolve or bend during time. valuable tool for practical applications. By querying the
As language is at the core of all forms of human and LLM with prompts, the AI inference model can generate
technological communication, it provides the words, responses, including answers to questions, newly
semantics, and grammar needed to convey ideas and generated text, summary texts, or sentiment analysis
concepts [2]. In the world of AI, the language model reports.
serves a similar purpose, providing the basis for An LLM can be likened to a predictive engine. When
communication and the creation of new concepts. provided with a prompt, it generates a response by
Moreover, when the access gate to the AI is a simple text, selecting one word at a time based on the likelihood of the
it has the opportunity to erase gatekeeping and approach a word's appearance in the given context. Always choosing
proportionally bigger user mass in an inclusive way [14]. the most probable option may result in less creative
The first AI language models have their roots in the responses, so some flexibility is built into the process. It's
early days of AI and they have been tied closely together noteworthy that with increased usage, LLMs continuously
since. The Eliza language model debuted in 1966 at MIT enhance their ability to predict helpful responses.
and is one of the earliest examples of an AI language
model [13]. All language models are first trained on B. What are large language models used for?
a dataset and then use various techniques to infer LLMs offer both practical utility and friendly user
relationships and then generate new content based on the experience in applications where language generation and
trained data. Language models are commonly used in understanding are essential. They can assist users by
Natural Language Processing (NLP) applications where providing information, suggestions or creative responses,
the user enters a query in natural language to generate thus contributing to greater productivity and efficiency in
a result. language-related tasks (Fig. 1). LLMs are powerful tools
Although there is not a singular generally accepted size for automating language-related activities such as
of suitable dataset for training, LLMs typically consist of generating speeches, emails, lectures or reports, saving
at least one billion or more parameters. Those stand for time and improving the results in these often repetitive
variables present in the model on which it was trained and activities. LLMs have broad applicability to a range of
furthermore can be used to infer new content. NLP tasks, including the following use-cases [13]:
As AI continues to evolve, its place in the business Text generation. The ability to generate text on any
environment is becoming more and more dominant. This topic for which the LLM has been trained is the primary
is shown by the use of LLM as well as machine learning use case.
tools. In the process of composing and applying machine
learning models, research suggests that simplicity and
consistency should be among the main objectives.
However, identification of the pain points as well as
understanding the historical data and ensuring accuracy is
also essential.
A. How do LLMs work?
LLMs use a comprehensive approach that includes
multiple components. But in the most simple terms, it is
only a very sophisticated prediction engine that generates
words with the highest probability to satisfy the task [13].
At its core, an LLM undergoes training on a substantial
dataset, often referred to as a corpus, typically containing
a vast amount of data, often in the order of petabytes. The
training process involves several steps, typically Figure 1. LLMs tasks

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 500

 Translation. For LLMs trained in multiple languages, C. The advantages of LLMs
the ability to translate from one language to another is a There are a number of benefits that LLMs provide to
common feature. organizations and users (Fig. 3):
Content summary. Summarizing blocks or multiple Extensibility and adaptability. LLMs can serve as the
pages of text is a useful feature for LLMs. basis for customized use cases. Additional training
Rewriting content. Another option is to rewrite and beyond the LLM can create a fine-tuned model for an
transcribe parts of the text. organization's specific needs.
Classification and categorization. LLM is capable of Flexibility. A single LLM can be used for many
classifying and categorizing content. different roles and use cases across organizations, users,
Sentiment analysis. Most LLMs can be used for and applications.
Sentiment analysis to help users better understand the Performance. Modern LLMs are typically high-
intent of a piece of content or a particular response. performance with the ability to generate fast, low-latency
Conversational AI and chatbots. LLMs can enable a responses.
conversation with a user in a way that is significantly Accuracy. As the number of parameters and the
more natural than older generations of AI technologies. volume of trained data in an LLM increases, the
The connecting feature of the previously stated use transformer model is able to provide increasingly higher
cases is personalization. Therefore LLMs are perfectly levels of accuracy.
designed to create inclusive and engaging learning Ease of training. Many LLMs are trained on unlabeled
experiences for students of all abilities and backgrounds data, which helps speed up the training process [13].
(Fig. 2). As a result, they are capable of empowering
students to learn at their own pace and in the way that orks D. The challenges and limitations of LLMs?
best for them [15]. While there are many benefits of using the LLM, there
Among the most common uses of conversational AI is are also several challenges and limitations (Fig. 4):
a chatbot. It can exist in any number of different forms Development costs. LLMs generally require large
where, although universally the user interacts in a query amounts of expensive GPU hardware and massive data
and response model. The most widely used LLM-based sets to learn on.
AI chatbot is ChatGPT, developed by OpenAI.
Operational costs. Even after the period of training and
ChatGPT, the marvel of artificial intelligence, has development, the operation cost of running an LLM can
unleashed an unprecedented wave of creativity [14]. be very high for the host organization.
Through extensive pre-training and adaptability, LLMs
have proven to be highly versatile and have enabled Bias. A risk of any AI trained on unlabelled data is
businesses across industries to transform operations, fuel bias, as known biases tend to replicate.
scientific exploration, and influence society. The impact Explainability. The ability to explain how the LLM
on human productivity and creativity is profound, as a was able to generate a particular result is neither easy nor
study by Accenture reveals that LLMs such as GPT-4 obvious to users.
have the potential to reshape up to 40% of working time, Hallucination (factual errors). AI hallucination occurs
increasing productivity by augmenting and automating when the LLM provides an inaccurate answer that is not
the language tasks that are dominating 62% of employees' based on the trained data.
time. Complexity. With billions of parameters, modern
Another famous LLM is BARD, which could be an LLMs are extremely complicated technologies whose
efficient contribution to academia. Based on Google's outputs can be infinitely complex to understand.
understanding of quality information, it concisely lists Glitch tokens. Maliciously designed token prompts that
both possible answers, how to verify them, and the cause LLM failure or other anomalous output, known as
sources used [16]. glitch tokens, are part of an emerging trend starting in
2022 [1], [13].

Figure 2. LLMs opportunities in education Figure 3. LLMs advantages

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 501

 disinformation campaigns, to create personalized or
large-scale fraud, or to develop computer code for viruses
or weapon systems.
Human-Computer Interaction Harms: Human-
Computer Interaction Harms: LLMs sometimes use
a “conversational agent” that interacts directly with
human users. Therefore, there may be a presentation of
the system as “human-like”. This can lead users to
overestimate their abilities and use these abilities in
dangerous ways. Conversations with such agents can
create new ways of manipulating or obtaining private
information from users as for example more efficient
ways of phishing.
Figure 4. LLMs challenges and limitations Automation, Access and Environmental Harms:.
Training and operating LLMs and AI systems can impose
high environmental costs. LLM-based applications may
E. Risks areas of harm from LLMs therefore be more beneficial to some groups than others,
Let us now look at LLMs in terms of identified and as they exploit the advantages and socialize the
anticipated risks. This is crucial in promoting progress environmental costs. LLM-based automation may affect
innovation in a responsible way. In the work of [17] six the quality of certain jobs and to undermine parts of the
main risk areas have been defined (Fig. 5): creative economy. Thus the benefits and the
Discrimination, Exclusion and Toxicity: LLMs can environmental costs of LLMs are unevenly globally
create unfair discrimination and representational and distributed.
material damage by perpetuating stereotypes and social Some of these risks can be mitigated directly during
prejudice. Toxic language in LLMs can incite hatred or training by better editing or managing training data, using
violence or cause offence. These risks stem in large part appropriate privacy algorithms, and limiting access to
from the selection of learning corpora that contain toxic data. However, others will be controlled only by
harmful language and over-represent certain social wise and effective regulation.
identities. At the end of the day, the most important risks of large
Information Hazards: LLMs may be involved in language models (LLMs) in education are bias,
leaking private data or sensitive information. These risks misinformation and addiction. LLMs can be used to
arise from the private data present in the training corpus generate misinformation, which could be harmful to
and in the advanced inference capabilities of LLMs. students. For example, an LLM could be used to generate
Misinformation Harms: LLMs may provide false or a fake news article or to create a propaganda video.
misleading information. Which for less informed users LLMs could be also addictive, especially for people in
implies an undermining of trust in the information being a vulnerable phase, which unfortunately the students on
shared. Misinformation can cause harm in sensitive areas universities are. LLMs can be biased, reflecting the biases
such as bad legal or medical advice. This wrong or false in the data they are trained on. This could lead to students
information can lead users to carry out unethical or illegal receiving inaccurate or misleading information, or to
activities that they would not have otherwise done. being treated unfairly in their assessments. Therefore, the
Malicious Uses: LLMS can be influenced by users or role of the teacher cannot be replaced.
developers who attempt to use LLM to cause harm. This
F. LLMs today
includes using LLM to increase the effectiveness of
Currently, there is an intensive discussion on how to
treat these AI software. The European Law on Artificial
Intelligence has been in preparation for several years. Its
basic principle is based on ensuring safety, predictability
and resilience. These are the key requirements for AI
systems and models that will work with high-risk data,
e.g. applications related to health care, finance or law.
This also includes specific regulations such as the
prohibition of impersonating human beings [18].
However, this specific prohibition is very difficult to
comply with, as illustrated by the results of the research
titled "Human or Not?" made by company AI21 Labs. It
created an online social experiment, which was played
more than 10 million times by more than 2 million
attendees. It is the biggest Turing-style experiment up to
Figure 5. LLMs risks areas date. The results showed that 32% of people could not
distinguish between humans and machines [19].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 502

 LLMs are doing a very efficient job in various sectors, task of education not only on campus but of the entire
but we actually don't know how. They have trained education system.
hundreds of parameters and processed an enormous Let's take a closer look at the potential of LLMs in
amount of data unimaginable for a human. When working education.
with LLMs, we also encounter situations that are
LLM can be our strong partner in achieving the goal of
surprising, as LLMs can clearly "re-invent" themselves
Education 5 (ethics and humanism-based education) and
and react unexpectedly. Prof Rusell sees their further
Society 5 (human centric society - Super smart society)
development in greater quality and security: "I think the
[21], which essentially is a personalized education. The
next evolution of big language models like ChatGPT,
LLM can be used to develop personalized learning
bots and many others will be that they become more
experiences for students, tailored to their individual needs
reliable, stop misleading and generally behave better." If
and interests. LLM can generate customized practice
you want to use them in any business context, you have
problems for a student struggling with a particular math
to be able to trust them to follow the rules, tell the truth
concept, or recommend articles and videos on a topic of
about the products and uphold the principles of your
interest to the student.
business" [18].
It can give students instant feedback on their work,
The current topic is LLMs in the context of the global helping them identify and correct their mistakes. The
climate crisis. We have to consider the environmental LLM can grade the problem analysis and provide
impact of AI systems. LLMs have a carbon footprint feedback on grammar, style, and content. LLMs can be
associated with their computational processes, which particularly useful for students who need to do repetitive
include training, deployment, and maintenance. The daily work, such as providing feedback when needed while
emissions of 23.04 kgCO2 over 18 days (using the ML writing large amounts of computer code over several
CO2 Impact calculator for estimation [20]) would total months. In this way, we can introduce new more inclusive
414 kgCO2 for ChatGPT. BLOOM, in contrast, released educational practices without the enormous increase in
360 kg over 18 days. It is also important to note that teachers' capacities.
BLOOM processed 230,768 requests over 18 days, or Moreover, it is a powerful tool in the field of language
12,820 on average daily. ChatGPT has a more significant education. The demand of the labor market is clear, with
carbon footprint because it processes many more daily each additional language, the professional attractivity of
requests. the student rises further. LLM can provide real-time
We are entering an era of transformation in which translation during a conversation or create customized
access to information, content creation, customer service language exercises.
and business operations will fundamentally change. An LLM can play a significant role in developing a
Generative AI integrated into the digital core of student's creativity. It can help a student brainstorm ideas
businesses will optimize tasks, enhance human to solve a problem or generate a variety of creative textual
capabilities and create new opportunities for growth. content formats, such as codes, scripts, e-mails, letters,
However, to fully exploit the potential of these resource briefs, etc. Designing and fine-tuning challenges
technologies, it is essential to rethink work practices and can act as a source of acceleration for students' thinking.
ensure that people can keep up with technological Thus, LLMs can become a kind of preparation for tasks,
advances. In other words, we need to make sure, nobody not a threat to the integrity of the educational process [15].
stays left behind. Even LLMs can be used to increase the accessibility of
education for students with disabilities by generating text-
Companies need to invest in developing operations and to-speech or speech-to-text translations and providing
providing training to employees as much as they invest in captions for videos and images.
the technology itself. This is the time for organizations to
use AI breakthroughs to redefine their performance, Beyond these specific applications, LLMs could also
reshape their industries and reeducate their employees. have a more general impact on the way we teach and
learn. For example, LLMs could help us develop new
teaching methods, create new learning resources, and
IV. A CURRENT VIEW OF THE USE OF LLMS
assess student learning in new ways.
IN EDUCATION
All this is extremely interesting and at the same time, it
With LLMs rise into the mainstream, a discussion was serves as a strong argument for the use of LLMs in
opened about how to deal with this tool in an academic education. However, there are challenges ahead that need
environment. The first reaction was a feeling of threat, to be resolved. It is very important to ensure that LLMs
from the possible disruption of traditional values and rules are used in a fair and ethical way and that they do not
on campus. But are LLMs in fact something we should be exacerbate existing inequalities in education. Some
afraid of? university students over-confident LLM outputs based on
Today we should no longer be captive to simple and previous IT experience providing credible answers.
routine solutions. The challenge lying in front of us is, However, this technology is known for inducing
first of all, to teach the student to understand the issue, to hallucinations, creating fake quotes, and flat out making
support his creativity, to teach him to effectively access things up. Machine learning models like LLMs are only as
the necessary information, and of course to support critical good as the training data. It means that if the models are
thinking and the need for constant verification of trained with low-quality data, they will produce low-
information that is not only the output of LLMs, but quality output. This can be problematic when there is no
available in all digital and traditional media. Teaching a room for error. It is therefore necessary to teach students
student to validate and filter the fake information is the to critically evaluate all the obtained outputs of LLMs. It
is here that the role of the teacher is irreplaceable, as an

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 503

authority who has the main say in the fulfillment of this extensions or variations of current problems facing
intention. academia. However, these technologies differ greatly in
In any case, LLMs have the potential to make education terms of speed, ease of access, and scale. There are known
more personalized, efficient and accessible for all cases of students submitting work that someone else wrote
students. for them or knowingly cheating using technology in
The LLMs are not ideal at all (Fig. 6). There is still a exams. As for the works of technical focus, which are
long way to go to have a perfect teaching assistant. We connected with experiments, this procedure is more
need to ensure that LLMs are used in a fair and ethical complicated and, in some cases, practically impossible.
Regular review meetings between students and teachers,
way, and that they do not exacerbate existing inequalities
in education. We also need to develop methods for where teachers have the opportunity to discuss with
evaluating the effectiveness of LLMs in educational students and jointly analyze the solution path, basically
settings. We need to keep in mind also the education of eliminates these problems.
teachers and students about the capabilities and limitations LLMs belong to the modern teaching. They can be a
of LLMs, and how to use them responsibly and very useful tool in education and teaching if used
effectively. correctly. At LLMs, we greatly value the support of the
student's creativity and quick orientation in the required
A. Should universities ban the use of LLMs? issue. Don't fear this technology. However, it is essential
Software LLMs are on the rise, it's an innovation that to teach students how to work with these technologies and
gives an edge when applying to the job market. It is develop critical thinking when verifying the outputs of
estimated, as we mentioned above [14], that the use of this LLMs. It is important to pay attention to using LLMs
software will save up to 40% of working time in responsibly and ethically. It is essential to perceive that
companies. It is enough to realize that e.g. ChatGPT is specific LLMs were trained on works of different authors.
able to write and debug code, write blog posts, script Therefore, any result, self-improvement prompt and self-
social media posts, summarize transcripts or podcasts, improvement LLM, is always a work following the work
solve complex math problems, explain technical terms in a of other authors.
simple form, and determine keywords for Search engine LLMs can help students quickly find information on a
optimization (SEO). And that is only a single LLM variety of topics. Let's teach students how to ask questions
application. and find reliable sources effectively.
Banning the use of LLMs is definitely the wrong way to Discuss with students the credibility of sources, correct
deal with this kind of innovation. Let's learn from history interpretation of results, and possible biases.
and not repeat the mistakes schools made when they We will use LLMs as a tool for collaboration and
blocked YouTube and Wikipedia after they were discussion. Students can work together on creating texts,
launched. Don't be afraid of what students will find on this presentations and solving problems.
service, let's rather teach them how to filter information in
such a way as to eliminate risk factors such as before We familiarize students with ethical issues related to
mentioned Hallucinations. It is better to teach students the use of LLMs, such as copyright, privacy and proper
how to fact-check what they get from the LLM than to try citation. The ethical code of the academic environment is
to stop using it altogether. known to them, they just have to be aware of the context
On the other hand, if we consider the outputs of that applies in general and thus also to work with LLMs.
students and teachers who used LLMs in their works as Let's also show students the limitations of LLMs, such
plagiarism, we must have tools to detect this. Such a tool as their occasional mistakes or lack of understanding of
is e.g. Writefull, available to STU [22]. It follows from the context. This may motivate them to be critical in their
very nature of LLMs that we will pull the short end of the use.
stick. Indeed, LLM creates a new text that cannot be
traced to a single source [1]. Thus we ought to forget the B. Should universities accept the use of LLMs?
conventional plagiarism check and preferably focus on the
nature of tasks and student development. Our parents needed to learn several books with almost
encyclopedic knowledge. We needed to learn how to
Many of the problems brought about by LLMs are just
search in these books. In the overload of knowledge and
information that is available today, it is very difficult to
navigate with today's tools. LLMs have the potential to
solve this problem relatively efficiently.
Are we to accept that language models will become an
integral part of our professional toolkit and incorporate
them into our teaching and assessment practices? If so,
what does that mean for us?
C. Should universities adopt LLMs and accept them?
We accept LLMs in education. But let's define the
ethical limits of its use. When is it an effective tool
supporting creative work and when is it a tool used to
create plagiarized outputs? It is necessary to adhere very

Figure 6. LLMs challenges in education

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 504

carefully to the code of ethics of the academic kind of data. The text created by LLM reflects the
community. patterns in the training data. It can be inaccurate but also
LLMs provide great opportunities to experiment and even toxic. Its use in education could further entrench
be creative in the pedagogical process. However, we can representational damage in extremely difficult ways to
accept it as an extraordinary opportunity [2]. document and remedy.
But we have before us an analysis of the effects that One option to address the issue of trust could be the
LLM software that can write sophisticated answers could creation of publicly funded LLMs in collaboration with
have on the way we think about teaching, learning and open initiatives led by stakeholders. Such models could
assessment. be developed specifically for the educational
If LLMs enter the educational process on campus to a environment, ensuring that they are auditable and
transparent in terms of their human and environmental
greater extent, what will we evaluate? What knowledge
costs. This will require a forward-looking vision,
and skills?
significant investment, active involvement, and lobbying
Will we evaluate the correct prompt creation for of educational institutions and their sponsors. One of the
LLMs? Or is it's interpretation that might be also written projects dealing with such development is the BigScience
by LLM? project, which was created at the beginning of 2021 [23].
Ability to identify and address LLMs bias? We must This project aimed to demonstrate a different way of
not forget that all work produced by LLMs is based on creating, studying and sharing large language models and
information "learned", especially from today's Internet, significant research artifacts within the AI/NLP research
which is heavily biased by human authors. Students must communities. It was inspired by science creation schemes
understand that LLMs will inherit the bias of their in other scientific fields, such as "CERN and the LHC in
programmers or the biased content of the source materials particle physics", in which open scientific collaboration
in the case of self-learning systems. facilitates the creation of large-scale artifacts useful to the
Students' ability to think through problems and be entire research community. The output of this project is
creative in an AI-informed world? This is exactly what BLOOM (BigScience Language Open-science Open-
we can excel at as humans. It is impossible to stay access Multilingual) [24].
looking for simple answers or follow a routine process; The most successful LLMs today are in the hands of
this is where artificial intelligence software is faster and the big tech giants. Embracing this transformative
more efficient. Professor Russel from the University of technology poses certain challenges from a research,
California at Berkeley illustrated this fact with numbers: environmental, ethical and societal perspective.
the cycle time of the human brain is about 10 ms, which Moreover, even if these tools are available, they were not
means about 100 operations per second. At the same designed as research artifacts and, for example, do not
time, a computer can handle 100 billion operations per have access to the data set or training checkpoints,
second [18]. making it impossible to answer many important research
Let's try to look at LLMs not as a threat but similar to questions about these models (possibilities, limitations,
spell check or translation software. Like the mentioned potential improvements, bias, ethics, environmental
software and LLMs, it has become a daily part of our impact, general AI/cognitive research environment).
lives. This is an extremely strong argument for adapting These models are typically Anglo-centric, and there are
and transforming education so that students can use these flaws in the text corpora used to train these models,
new tools ethically and effectively. ranging from the unrepresentativeness of populations to
It is crucial for us to clearly define the conditions for the prevalence of potentially harmful stereotypes or the
using LLMs on campus. However, students, teachers, and inclusion of personally identifiable information.
researchers will have to realize that work written using This brings to the fore the political question of who
such tools cannot be considered their own. Submitting owns and sets the education standards in the age of AI.
AI-generated work as your own is an act of academic
misconduct and would have consequences. However, this V. UNIVERSITIES' VIEWS ON LLMS AND AI
is not the same as banning the use of LLMs. IN ASSESSMENT AND EDUCATION
However, adopting LLMs as part of pedagogical The boom in artificial intelligence software and LLMs
practice carries risks in the form of negative operational, requires every university or educational institution to take
financial, pedagogical and ethical consequences for a stand on their use in the educational process.
universities. As long as LLMs are privately owned, the One example is the opinion of The International
owner is not obliged to accommodate the needs of Baccalaureate (IB) [25], which declares that it does "not
educational institutions in terms of maintenance and prohibit" the use of ChatGPT or any similar AI software.
access to their model, creating fundamental operational However, AI tools must comply with the IB Academic
issues if this is part of the assessment. This issue can be Integrity Policy. Therefore, they adapt and transform
tackled only by regulations which academia ought to educational programs and assessment practices so students
demand. can use these new AI tools ethically and effectively.
Students must be aware that the IB does not consider any
A big challenge is the question of trust that educators work created – even partially – using such tools as their
can place in the model, how it was trained and on what own. Therefore, as with any quotation or material from

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 505

another source, it must be clear that the AI-generated text, [6] C. Mishra and D. L. Gupta, “Deep machine learning and neural
image or graphic included in the work has been copied networks: An overview”. IAES Internat. J of artificial intelligence.
2017 Jun 1;6(2):66. http://doi.org/10.11591/ijai.v6.i2.pp66-73.
from such software. The software must be listed in the text
[7] C. Zhang and Y. Lu, “Study on artificial intelligence: The state of
and appropriately cited in the bibliography. As in current the art and future prospects”. J. of Industr. Information
practice, a heavily cited essay will not receive a high Integration. 2021 Sep 1,. https://doi.org/10.1016/j.jii.2021.100224.
grade.. [8] T. Davenport and R. Kalakota, “The potential for artificial
intelligence in healthcare”. Future healthcare journal. 2019
VI. CONCLUSION Jun;6(2):94. https://doi.org/10.7861/futurehosp.6-2-94.
LLMs technology is here and becoming a part of our [9] P. Schneider, W. P. Walters, A. T. Plowright, N. Sieroka,
J. Listgarten, R. A. Goodnow, jt., J. Fisher, J. M. Jansen,
lives. It is up to us to minimize the damage it can cause in J. S. Duca, T. S. Rush, M. Zentgraf, J. E. Hill, E. Krutoholow,
the educational process and approach it in such a way that, M. Kohler, J. Blaney, K. Funatsu, Ch. Luebkemann and
on the contrary, we use its positive aspects and benefits in G. Schneider. “Rethinking drug design in the artificial intelligence
order to achieve personalized education with higher era”. Nature Reviews Drug Discovery. 2020 May;19(5):353-64.
efficiency and effectiveness. Let's prepare our students for [10] Z. Ullah, F. Al-Turjman, L. Mostarda L, R. Gagliardi,
the world with AI and support their creativity so that they “Applications of artificial intelligence and machine learning in
smart cities”. Computer Communications. 2020 Mar 15;154:313-
have a chance to succeed in the transformed labour 23. https://doi.org/10.1016/j.comcom.2020.02.069.
market. It is necessary to teach students to evaluate the
[11] R. S. Peres, X. Jia, J. Lee, K. Sun, A. W. Colombo, J. Barata,
performances of LLMs critically, but currently, the “Industrial artificial intelligence in industry 4.0-systematic review,
academic community is interested in supporting the challenges and outlook”. IEEE Access. 2020 Dec 7;8:220121-39.
development of defined LLMs in public hands. At the https://ieeexplore.ieee.org/document/9285283.
same time, it is necessary to meaningfully declare under [12] L. Chen, P. Chen and Z. Lin. “Artificial intelligence in education:
which conditions and in which spheres of academic A review”. Ieee Access. 2020 Apr 17;8:75264-78.
education it is possible to use LLMs. To define the [13] S. M. Kerner, “Large language models (LLMs)”, [online],
compliance of their use with the ethics of the academic September 2023, [cit. 2023-10-10],
environment. Determining the boundaries will create a https://www.techtarget.com/whatis/definition/large-language-
model-LLM.
safe space for the creative use of these extremely effective
[14] J. Kaur, “Large Language Models (LLMs)”, 2023 Jun 29 [online],
tools in the pedagogical process as well as in scientific, https://www.xenonstack.com/blog/large-language-models.
creative work. [15] S. A. Mathieson, “Large language models harnessed for
As a result, LLMs can be used to develop personalized education” 2023 Jul 31 [online],
learning experiences for students, tailored to their https://www.computerweekly.com/feature/Large-language-
individual needs and interests. This could involve using models-harnessed-for-education.
LLMs to generate personalized practice problems, [16] S. Hsiao and E. Collins, “Try Bard and share your feedback”,
recommend knowledge in the form of articles and videos, [online], [cit. 2023-10-10] https://blog.google/technology/ai/try-
bard/.
or provide feedback on student work. Hence, the LLMs as
[17] L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato,
personal teaching assistants are really the Next Big Thing P. S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh,
in education, and they are already here to help us. Z. Kenton, S. Brown, W. Hawkins, T. Stepleton, C. Biles,
A. Birhane, J. Haas, L. Rimell, L. A, Hendricks, W. Isaac,
ACKNOWLEDGMENT S. Legassick, G. Irving and I. Gabriel, “Ethical and social risks of
harm from Language Models”, https://arxiv.org/abs/2112.04359.
This work was supported by the Scientific Grant [18] S. Russell, prof. University of California, Berkeley “Judgment
Agency and the Slovak Research and Development Day 2023”: https://www.youtube.com/watch?v=-
Agency of the Ministry of Education, Science Research B_MC2lvrLQ&t=1168s&ab_channel=Alanata.
and Sport of the Slovak Republic, grants: APVV-21-0365 [19] Turing test, [online], [cit. 2023-10-10],
and APVV-20-0310. https://en.wikipedia.org/wiki/Turing_test#Variations.
[20] T. Kain, “ChatGPT’s Carbon Footprint”, Sigma Earth,
REFERENCES [online], Apr 122023, [cit. 2023-10-10],
https://sigmaearth.com/chatgpts-carbon-
[1] S. Milano, J. A. McGrane and S. Leonelli, “Large language footprint/#:~:text=The%20carbon%20footprint%20of%20ChatGP
models challenge the future of higher education”. Nat Mach Intell T%2C%20if%20it%20ran%20for%20a,2%20emissions%20of%2
5, 2023, pp.333-334. https://doi.org/10.1038/s42256-023-00644-2. 023.04%20kg.
[2] E. Kasneci, K. Seßler, S. Küchemann, M. Bannert, [21] L. Stuchlikova and J. Marek, “Academy Phoenix: Will
D. Dementieva, F. Fischer, U. Gasser, G. Groh, S. Günnemann, Universities Reborn in Industry 5.0 Era, or Will They Lie Down in
E. Hüllermeier, S. Krusche, G. Kutyniok, T. Michaeli, C. Nerdel, Ashes?”. In ICETA 2022, IEEE, Slovakia, DOI:
J. Pfeffer, O. Poquet, M. Sailer, A, Schmidt, T. Seidel, M. Stadler, 10.1109/ICETA57911.2022.9974752.
J. Weller, J. Kuhn, G. Kasneci, “ChatGPT for good? On
[22] GPT-3, GPT-4 and ChatGPT Detector, [online], 2023, [cit. 2023-
opportunities and challenges of large language models for
10-10], https://x.writefull.com/gpt-detector.
education. Learning and individual differences. [online], 2023 Apr
1;103:102274. https://doi.org/10.1016/j.lindif.2023.102274. [23] Statement from the IB about ChatGPT and artificial intelligence in
assessment and education [online], 2023, [cit. 2023-10-10],
[3] A. Caines, L. Benedetto, S. Taslimipoor, Ch. Davis, Y. Gao, O.
https://www.ibo.org/news/news-about-the-ib/statement-from-the-
Andersen, Z. Yuan, M. Elliott, R. Moore, Ch. Bryant, M. Rei, H.
ib-about-chatgpt-and-artificial-intelligence-in-assessment-and-
Yannakoudakis, A. Mullooly, D. Nicholls, P. Buttery, “On the
education/.
application of Large Language Models for language teaching and
assessment technology”. arXiv preprint arXiv:2307.08393. [24] BigScience, [online], May 2021 [cit. 2023-10-10]
[online], 2023 Jul 17. https://doi.org/10.48550/arXiv.2307.08393. https://bigscience.huggingface.co/.
[4] OpenAI.). ChatGPT (version 4) [Large language model]. [online], [25] BLOOM: BigScience 176B Model [online], 2022 [cit. 2023-10-
2023 [cit. 2023-10-10] https://openai.com/gpt-4. 10] https://bigscience.notion.site/BLOOM-BigScience-176B-
Model-ad073ca07cdf479398d5f95d88e218c4.
[5] Bard, Google AI, [online], 2023 [cit. 2023-09-17].
https://bard.google.com/chat. .

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 506

 
1DWLRQDOFHQWHUIRUGLJLWDOWUDQVIRUPDWLRQRI
HGXFDWLRQ

'âYHGD 9+XEHĖiNRYi ..R]HONRYi ./XNiþRYi $0LãLDQLNRYi 92QGRYi 
0LULDPD)LOþiNRYi -DQD.R]iNRYi 0.LUHã 
,QVWLWXWHRI0DWKHPDWLFV-HVHQQi.RãLFH6ORYDNLD
,QVWLWXWHRI3K\VLFV3DUN$QJHOLQXP.RãLFH6ORYDNLD
/LIHORQJ/HDUQLQJ&HQWUH 3URMHFWV6XSSRUWâUREiURYD.RãLFH6ORYDNLD
,QVWLWXWHRI%LRORJ\DQG(FRORJ\0iQHVRYD.RãLFH6ORYDNLD

GXVDQVYHGD#XSMVVNYHURQLNDKXEHQDNRYD#XSMVVNNDWDULQDNR]HONRYD#XSMVVNNDWDULQDOXNDFRYD#XSMVVN
DQQDPLVLDQLNRYD#XSMVVNYHURQLNDRQGRYD#XSMVVNPLULDPDILOFDNRYD#XSMVVNMDQDNR]DNRYD#XSMVVN
PDULDQNLUHV#XSMVVN


$EVWUDFW²7KH GHPDQGV IRU FRPSHWHQFHV QHFHVVDU\ IRU WKH 3URJUDPPHIRU,QIRUPDWL]DWLRQRI(GXFDWLRQXQWLOLV
VXFFHVV RI D JUDGXDWH LQ WRGD\ V UDSLGO\ FKDQJLQJ ZRUOG XVLQJ GLJLWDO WHFKQRORJLHV WR LPSURYH WKH TXDOLW\ RI
SRLQWWRWKHQHHGIRULQHYLWDEOHFKDQJHVLQHGXFDWLRQ7KHVH HGXFDWLRQ LQ WKH FRQWH[W RI WKH GHPDQGV SODFHG RQ WKH
FKDQJHV VKRXOG UHIOHFW WKH GHYHORSPHQW RI VW FHQWXU\ FXUUHQWJHQHUDWLRQRISXSLOVDQGVWXGHQWVDIWHUWKHHQGRI
FRPSHWHQFLHVLQFOXGLQJGLJLWDOFRPSHWHQFLHVDWDOOOHYHOVRI
IRUPDO HGXFDWLRQ DQG LQ WKH FRQWH[W RI WKHLU QHHGV >@
HGXFDWLRQ 7KH GHYHORSPHQW RI GLJLWDO FRPSHWHQFHV DQG
SUHSDUDWLRQ IRU WKH OLIH RI D JUDGXDWH LQ WKH GLJLWDO DJH LV $FWRUV RI HGXFDWLRQ VKRXOG PDVWHU GLJLWDO VNLOOV DQG
FRQGLWLRQHGE\WKHGLJLWDOWUDQVIRUPDWLRQRIWKHVFKRRO7KLV FRPSHWHQFHV IRU WKH VW FHQWXU\ LQ DFFRUGDQFH ZLWK
HIIHFWLYH DQG PHDQLQJIXO GLJLWDO WUDQVIRUPDWLRQ DW VFKRROV (XURSHDQVWDQGDUGV>@>@$FFRUGLQJWRWKH3URJUDPPH
VKRXOG EH EDVHG RQ WKH FRQVLVWHQW SUHSDUDWLRQ RI DQ DFWLRQ IRU ,QIRUPDWL]DWLRQ RG (GXFDWLRQ >@ VXSSRUW IRU
SODQ LQ DFFRUGDQFH ZLWK WKH YLVLRQ RI WKH VFKRRO WKH V\VWHPDWLFSHUPDQHQWDQGVXVWDLQDEOHFKDQJHVKRXOGDOVR
LPSOHPHQWDWLRQ DQG VXEVHTXHQW HYDOXDWLRQ RI WKH DFWLRQ EHEDVHGRQWKHIROORZLQJSLOODUV
VWHSV 7KH VFKRRO GLJLWDO FRRUGLQDWRU EHFDPH WKH OHDGHU RI  WKHPDLQDFWRURIHGXFDWLRQDOWUDQVIRUPDWLRQLVWKH
WKHGLJLWDOWUDQVIRUPDWLRQLQSULPDU\DQGVHFRQGDU\VFKRROV WHDFKHU DQG LWV VXFFHVV LV WLHG WR FKDQJH ±
LQ 6ORYDNLD ,QQRYDWLYH WUDLQLQJ RI GLJLWDO FRRUGLQDWRUV
LPSURYLQJ WKH TXDOLW\ RI WKH HGXFDWLRQDO SURFHVV
LPSOHPHQWHG VXSSRUW LQ WKH IRUP RI RQOLQH FOXEV DQG IDFH
WRIDFH PHHWLQJV SRLQWHG WR WKH QHFHVVLW\ RI FRQWLQXHG ZKLFK WDNHV SODFH LQ WKH LQWHUDFWLRQ EHWZHHQ WKH
WDUJHWHGVXSSRUWDWWKHQDWLRQDOOHYHO)RUWKLVSXUSRVHWKH WHDFKHUDQGVWXGHQWV
1DWLRQDO &HQWHU IRU WKH 'LJLWDO 7UDQVIRUPDWLRQ RI  XQLYHUVLWLHVSUHSDULQJIXWXUHWHDFKHUVPXVWDOVREH
(GXFDWLRQ ZDV HVWDEOLVKHG 7KH DLP RI WKH QDWLRQDO FHQWHU FDUULHUVRIFKDQJHLQWKHWUDQVIRUPDWLRQ
LQ DGGLWLRQ WR SURYLGLQJ LQQRYDWLYH HGXFDWLRQ IRU WKH  VXSSRUWLQJVFKRROVLQWKHLUGLJLWDOWUDQVIRUPDWLRQ
SRVLWLRQ RI VFKRRO GLJLWDO FRRUGLQDWRU LV DOVR LQQRYDWLYH 7KH SUHYLRXV SHULRG VKRZHG WKDW LW LV FUXFLDO WKDW WKH
HGXFDWLRQ IRU VFKRRO SULQFLSDOV IRFXVHG RQ WKH OHDGHUVKLS FDUULHUVRIWKLVVXSSRUWDUHWKHKLJKTXDOLW\H[SHUWV IURP
FRPSHWHQFLHVQHFHVVDU\IRUHIIHFWLYHWHDP PDQDJHPHQWDQG WKH DFDGHPLF HQYLURQPHQW GLUHFWO\ PDQDJHG
PDQDJHPHQW RI WKH FKDQJH SURFHVV 7KH DFWLYLWLHV RI WKH
RUJDQL]DWLRQV DQG VXEMHFWV 7KLV LV HYLGHQFHG E\ HJ
FHQWHUZLOODOVRLQFOXGHPHHWLQJVZLWKSULQFLSDOVDQGGLJLWDO
FRRUGLQDWRUV LQGLYLGXDO FRQVXOWDWLRQV DQG RQOLQH ZHELQDUV RXWSXWV IURP WKH 1DWLRQDO SURMHFW ,7 $FDGHP\ >@ >@
UHIOHFWLQJWKHQHHGVRIWKHFRPPXQLW\VXSSRUWLQJWKHGLJLWDO >@ >@ QDWLRQDO SURMHFW 3URIHVVLRQDO GHYHORSPHQW RI
WUDQVIRUPDWLRQ RI WKH VFKRRO DQG WHDFKHUV 0RUHRYHU WKH WHDFKHUV 7($&+(56  VFRSH DQG FRQWHQW RI WHDFKHU
WDUJHW JURXS LV DOVR SHGDJRJLFDO DQG SURIHVVLRQDO VWDII VXSSRUW GXULQJ WKH SDQGHPLF >@ ,Q RUGHU WR HQVXUH
SULPDU\ DQG VHFRQGDU\ VFKRRO SXSLOV VWXGHQWV DW TXDOLW\ SURIHVVLRQDOLVP DQG HTXLYDOHQFH LW LV H[WUHPHO\
XQLYHUVLWLHV 2QH RI WKH SULRULWLHV RI WKH QDWLRQDO FHQWHU LV LPSRUWDQW WR FRRUGLQDWH WKLV VXSSRUW FHQWUDOO\ ,W LV
KLJKTXDOLW\ UHVHDUFK DQG VXEVHTXHQW PDQDJHPHQW LQ WKH HVVHQWLDO WR UHIOHFW WKH GLIIHUHQFHV EHWZHHQ LQGLYLGXDO
ILHOGRIGLJLWDOWUDQVIRUPDWLRQRIHGXFDWLRQ UHJLRQV WR JHW WKH PRVW RXW RI WKHP IRU WKH JURZWK RI
LQGLYLGXDO VFKRROV ZKLFK FDQ EH DQ LQVSLUDWLRQ IRU HDFK
, ,1752'8&7,21 RWKHU ,W LV QHFHVVDU\ WR HQVXUH WKDW ZLWK WKH VXSSRUW RI
HGXFDWLRQDO UHVHDUFK WKH VRFDOOHG H[DPSOHV RI JRRG
(IIHFWLYH DQG PHDQLQJIXO SUHSDUDWLRQ RI JUDGXDWHV IRU
SUDFWLFH KDYH EHFRPH WKH VWDQGDUG IRU D ZLGH UDQJH RI
VXFFHVVDQGOLIHLQWKHGLJLWDODJHVWURQJO\GHSHQGVRQWKH
VFKRROV7KLVUHVHDUFKPXVWEHFDUULHGRXWLQDFFRUGDQFH
HGXFDWLRQ V\VWHP 7KH VFKRRO V\VWHP VKRXOG UHVSRQG WR
ZLWK WKH WUHQGV RI WKH WRS (XURSHDQ UHVHDUFK DQG
WKH FKDQJHV EURXJKW DERXW E\ WKH GLJLWDO WUDQVIRUPDWLRQ
HGXFDWLRQDOVSDFH)URPWKHDERYHLWLVFOHDUWKDWWKHNH\
RI VRFLHW\ 7KH JRDO RI WKH GHSDUWPHQW LQ WKH ILHOG RI
WR WKH VXFFHVV RI WKH GLJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ
GLJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ DFFRUGLQJ WR WKH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 507

ZLOO EH WKH GLJLWDO WUDQVIRUPDWLRQ RI LQGLYLGXDO VFKRROV SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV LQ SUDFWLFH DQG
WKHLU QHWZRUNLQJ WKURXJKRXW 6ORYDNLD DQG ZLWKLQ WKH VFKRROVEH\RQGWKHVFRSHRIWKHRUJDQL]DWLRQRIWHDFKLQJ
UHJLRQV WKH SRVVLELOLW\ RI DQ H[WHUQDO DXGLW RI VFKRRO SUDFWLFH 7KURXJK 83-â DQG &8 H[SHUWV IURP RWKHU
TXDOLW\ DQG WKHLU FHUWLILFDWLRQ ,Q WKLV SURFHVV LW LV QRW XQLYHUVLWLHV WKDW SUHSDUH IXWXUH WHDFKHUV ZLOO DOVR EH
DERXW FRPSHWLWLRQ EHWZHHQ VFKRROV EXW DERXW LQYROYHG LQ WKH SURMHFW PDLQO\ &RQVWDQWLQH WKH
FRRSHUDWLRQDQGPXWXDOVXSSRUW 3KLORVRSKHU 8QLYHUVLW\ DQG 0DWHM %HO 8QLYHUVLW\ ZKLFK
%DVHGRQUHVHDUFKGLUHFWO\DWVFKRROVDQGUHOHYDQWGDWD FROODERUDWHGRQQDWLRQDOSURMHFWV 
>@>@DVZHOODVEDVHGRQH[SHULHQFHIURPDEURDG>@
>@ LW LV QHFHVVDU\ WR EXLOG D VXVWDLQDEOH V\VWHP RI ,,, 7+(*2$/2)7+(1$7,21$/352-(&7',7('8
VXSSRUWIRUWKHGLJLWDOWUDQVIRUPDWLRQRIHGXFDWLRQDQGWR 7KH JRDO RI WKH QDWLRQDO SURMHFW LV EDVHG RQ UHVHDUFK
HVWDEOLVK D 1DWLRQDO &HQWHU IRU WKH 'LJLWDO GLUHFWO\ DW VFKRROV DQG UHOHYDQW GDWD WKH FUHDWLRQ RI D
7UDQVIRUPDWLRQ RI (GXFDWLRQ LQ WKH DFDGHPLF VXVWDLQDEOH V\VWHP RI VXSSRUW IRU WKH GLJLWDO
HQYLURQPHQW7KHQDWLRQDOSURMHFW'LJLWDO7UDQVIRUPDWLRQ WUDQVIRUPDWLRQ RI HGXFDWLRQ E\ HVWDEOLVKLQJ D 1DWLRQDO
RI(GXFDWLRQDQG6FKRROV 'L7(GX UHVSRQGVWRWKHQHHG &HQWHUIRUWKH'LJLWDO7UDQVIRUPDWLRQRI(GXFDWLRQLQDQ
WR HVWDEOLVK D 1DWLRQDO &HQWHU IRU WKH 'LJLWDO DFDGHPLF HQYLURQPHQW ZLWK WZR ZRUNSODFHV  LQ .RãLFH
7UDQVIRUPDWLRQRI(GXFDWLRQ 1&'79  DW 83-â DQG LQ %UDWLVODYD DW &8 ZLWK WKH LQYROYHPHQW
H[SHUWV IURP RWKHU XQLYHUVLWLHV 1,9D0 DQG SUDFWLFH
,, 7+(1$7,21$/352-(&7',*,7$/ 83-â DQG &8 RQ WKH EDVLV RI WKH 0HPRUDQGXP RI
75$16)250$7,212)('8&$7,21$1'6&+22/6 8QGHUVWDQGLQJDQG&RRSHUDWLRQEHWZHHQ0â99Dâ83-â
',7('8  DQG &8 IROORZLQJ WKH DSSURYDO RI WKH SURMHFW SURSRVDO
7RHVWDEOLVKDQDWLRQDOFHQWHUDQGVXSSRUWDFWLYLWLHVRI ZLOO HVWDEOLVK LQ WKHLU HQYLURQPHQW WKH 1DWLRQDO &HQWHU
WKH FHQWUH WKH SURSRVDO DQG GRFXPHQWV RI WKH QDWLRQDO IRU'LJLWDO7UDQVIRUPDWLRQRI(GXFDWLRQ 1&'79 DVD
SURMHFW 13  'LJLWDO 7UDQVIRUPDWLRQ RI (GXFDWLRQ DQG GHYHORSPHQW UHVHDUFK DQG HGXFDWLRQDO SURMHFW DQG ZLOO
6FKRROV 'L7(GX  ZHUH GHYHORSHG IRU WKH 0LQLVWU\ RI LQWHJUDWH LW LQWR WKHLU VWUXFWXUH 1&'79 ZLOO QRW EH DQ
(GXFDWLRQ 6FLHQFH 5HVHDUFK DQG 6SRUW RI WKH 6ORYDN LQGHSHQGHQW LQVWLWXWLRQ EXW DQ HFRV\VWHP WR VXSSRUW WKH
5HSXEOLF 0â99Dâ65 7KHSUHSDUDWLRQRI'L7(GXDQG GLJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ DQG VFKRROV
WKHHVWDEOLVKPHQWDQGPLVVLRQRIWKH1DWLRQDO&HQWHUIRU 1&'79 V DFWLYLWLHV ZLOO EH JRYHUQHG E\ LWV VWDWXWH 7KH
WKH'LJLWDO7UDQVIRUPDWLRQRI(GXFDWLRQ 1&'79  ZHUH %RDUG RI 'LUHFWRUV RI 1&'79 ZLOO LQFOXGH
FRPPXQLFDWHGZLWKWKHSURIHVVLRQDOSXEOLFDQGPLQLVWULHV UHSUHVHQWDWLYHV RI XQLYHUVLWLHV 0â99Dâ WKH 'LJLWDO
WKH$VVRFLDWLRQRI6WDWH*\PQDVLXP/HDGHUVWKH'LJLWDO FRDOLWLRQDQGRWKHUDFWRUVLQWKHILHOGRIHGXFDWLRQ
&RDOLWLRQ WKH 3HUPDQHQW FRQIHUHQFH RI DFWRUV LQ $FFRUGLQJ WR WKH PHPRUDQGXP $UWLFOH ,, SDUDJUDSK
HGXFDWLRQ WKH 0LQLVWU\ RI ,QYHVWPHQW 5HJLRQDO  RI83-âDQG&8IXUWKHU
'HYHORSPHQWDQG,QIRUPDWLFVRIWKH6ORYDN5HSXEOLFDQG  WKH\ ZLOO EXLOG WKH SHUVRQQHO VWUXFWXUH RI 1&'79
0â99Dâ  LQ WKH SURFHVV RI SUHSDULQJ WKH 3URJUDPPH ZKLFK ZLOO EH PDGH XS RI WRS SHGDJRJLFDO HPSOR\HHV
6ORYDNLD $OVR EDVHG RQ WKLV EURDG VXSSRUW WKH PHWKRGRORJLVWV UHVHDUFKHUV DQG FUHDWRUV RI GLJLWDO
HVWDEOLVKPHQW RI WKH QDWLRQDO FHQWHU LV H[SOLFLWO\ SDUW RI HGXFDWLRQDOFRQWHQW
WKHSULRULW\34XDOLW\DQGLQFOXVLYHHGXFDWLRQ  WKURXJK 1&'79 WKH\ ZLOO LPSOHPHQW WKH WDVNV
7KH SURSRVDO RI WKH SURMHFW LV LQ DFFRUGDQFH ZLWK WKH UHVXOWLQJ IURP WKH 3URJUDPPH IRU ,QIRUPDWL]DWLRQ RI
3URJUDPPH IRU WKH ,QIRUPDWL]DWLRQ RI (GXFDWLRQ XQWLO (GXFDWLRQ XQWLO  >@ LQ WKH ILHOG RI GLJLWDO
 >@ DQG DOVR ZLWK RWKHU EDVLF GRFXPHQWV 'LJLWDO WUDQVIRUPDWLRQRIHGXFDWLRQHVSHFLDOO\ WDVNVFRUUHODWH
(FRQRP\ DQG 6RFLHW\ ,QGH[ '(6,  >@ 1DWLRQDO ZLWKVXEDFWLYLWLHVWKDWDUHSDUWRIWKHSURMHFW VSXUSRVH 
'LJLWDO 6NLOOV 6WUDWHJ\ RI 6ORYDNLD RI WKH 5HSXEOLF DQG 7KH LPSOHPHQWDWLRQ RI WKH DFWLYLW\ VXEDFWLYLWLHV  RI
WKH $FWLRQ 3ODQ IRU WKH \HDUV  ±  >@ WKH (8 WKHSURMHFWIURPDSURIHVVLRQDODQGPHWKRGRORJLFDOSRLQW
'LJLWDO(GXFDWLRQ$FWLRQ3ODQIRUWKH\HDUV± RIYLHZZLOOEHFRYHUHGE\1&'79XQGHUWKHVXSHUYLVLRQ
>@ RI WKH %RDUG RI 'LUHFWRUV ZLWK WKH IDFW WKDW HDFK RI WKH
7KH SURSRVDO RI WKH SURMHFW ZDV DSSURYHG DQG WKH SDUWQHUV ZLOO LPSOHPHQW WKH DFWLYLWLHV DV GHILQHG LQ WKH
IXWXUHDSSOLFDQWLVWKH1DWLRQDO,QVWLWXWHRI(GXFDWLRQDQG GHVFULSWLRQ RI WKH LQGLYLGXDO VXEDFWLYLWLHV
<RXWK 1,9D0 LQFRRSHUDWLRQZLWKSDUWQHUV3DYRO-R]HI 2UJDQL]DWLRQDO SURYLVLRQ RI WKH SURMHFW LPSOHPHQWDWLRQ
âDIiULN 8QLYHUVLW\ LQ .RãLFH 83-â  &RPHQLXV ZLOOEHFRRUGLQDWHGE\WKH6WHHULQJ%RDUGRIWKHQDWLRQDO
8QLYHUVLW\ LQ %UDWLVODYD &8  DQG 0â99Dâ 65 7KH SURMHFW ZKLFK ZLOO LQFOXGH UHSUHVHQWDWLYHV RI WKH 13
DSSOLFDQW RI WKH SURMHFW LV D UHVHDUFK FXUULFXODU SDUWQHUV
SURIHVVLRQDO PHWKRGLFDO FRQVXOWLQJ DQG HGXFDWLRQDO ,Q RUGHU WR IXOILOO WKH JRDO RI WKH QDWLRQDO SURMHFW WKH
LQVWLWXWLRQDQGDQDWLRQDOOHDGHULQH[SHUWLVHRQLVVXHVRI DFWLYLW\ 6XSSRUW IRU WKH GLJLWDO WUDQVIRUPDWLRQ RI
XSEULQJLQJDQGHGXFDWLRQLQ6ORYDNLD7KHSDUWQHUVKDYH HGXFDWLRQZLOOEHLPSOHPHQWHGGXULQJWKHHQWLUHGXUDWLRQ
DXQLTXHSRVLWLRQFRQVLGHULQJWKHLUH[SHULHQFHLQWKHILHOG RIWKHSURMHFW IURPWR GLYLGHGLQWRVXE
RI GLJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ 83-â  DQG WKH DFWLYLWLHV ZLWK LPSOHPHQWDWLRQ RQ WKH WHUULWRU\ RI WKH
LQQRYDWLRQ RI WHDFKLQJ LQIRUPDWLFV LQ SULPDU\ HGXFDWLRQ 6ORYDN5HSXEOLF
&8 DV ZHOODVWKHLPSOHPHQWDWLRQRILQWHUQDWLRQDODQG
QDWLRQDO SURMHFWV WKH WUDLQLQJ RI IXWXUH SULPDU\ DQG
VHFRQGDU\ VFKRRO WHDFKHUV DQG DFWLYH FRRSHUDWLRQ ZLWK

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 508

 ,9 6833257)257+(',*,7$/75$16)250$7,212)  (GXFDWLRQZLOOWDNHSODFHLQDYLUWXDOHQYLURQPHQW
('8&$7,21 
6XEDFWLYLWLHVZLWKLQWKHIUDPHZRUNRI6XSSRUWIRUWKH $ 6XSSRUW RI WKH RQOLQH SODWIRUP RI VFKRRO GLJLWDO
GLJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ ZLOO FRQFHUQ IRUPDO FRRUGLQDWRUV 'LJLWDO&RRUGLQDWRUV&OXE 
DQG LQIRUPDO HGXFDWLRQ RI VFKRRO GLJLWDO FRRUGLQDWRUV  ,WZLOOEHDQ067HDPVWHQDQWZKLFKLVFXUUHQWO\
VFKRROSULQFLSDOVSHGDJRJLFDODQGSURIHVVLRQDOHPSOR\HHV XVHG IRU WKH DVVRFLDWLRQ RI VFKRRO GLJLWDO
RISUHVFKRROVSULPDU\DQGVHFRQGDU\VFKRROVVWXGHQWVRI FRRUGLQDWRUV 2Q WKLV SODWIRUP D VSDFH KDV EHHQ
WHDFKLQJSURJUDPVDWXQLYHUVLWLHVXSGDWLQJDQGFUHDWLRQRI FUHDWHGIRUWKHLUPXWXDOFRPPXQLFDWLRQZHELQDUV
LQQRYDWLYH PHWKRGRORJLHV QHWZRUNLQJ RI VFKRROV DQG VKDULQJ RI H[SHULHQFHV RU RSHUDWLYH SUREOHP
HPSOR\HHV TXDOLW\ FHUWLILFDWLRQ RI GLJLWDO VFKRROV VROYLQJ
SRSXODUL]DWLRQ (GXFDWLRQDO UHVHDUFK ZLOO EH NH\ LW ZLOO  6XSSRUWLQJ WKLV SODWIRUP PHDQV FUHDWLQJ DQG
JXDUDQWHH WKH JURZWK RI WKH TXDOLW\ RI WKH VXSSRUW PDQDJLQJ DFFRXQWV FUHDWLQJ ZHELQDU FRQWHQW DQG
SURYLGHGHQVXUHWKHVDIHW\RIWKHGLJLWDOWUDQVIRUPDWLRQLQ QHWZRUNLQJ ZLWK RUJDQL]DWLRQV WKDW FDQ RIIHU WKLV
YLHZ RI LWV ULVNV DQG FRPPXQLFDWH WKH DFTXLUHG FRQWHQW IDFLOLWDWLRQ RI PXWXDO FRPPXQLFDWLRQ
NQRZOHGJH ZLWK WKH LQWHUQDWLRQDO FRPPXQLW\ 7KHUH ZLOO EHWZHHQ &OXE PHPEHUV FUHDWLRQ RI WHDPV DQG
EH FUHDWHG PLQLPXP PLQ   QHZ LQQRYDWLYH ZRUNLQJ JURXSV WKDW ZLOO QHWZRUN VFKRROV VROYLQJ
PHWKRGRORJLHVIRUWHDFKLQJJHQHUDOHGXFDWLRQVXEMHFWVDQG VLPLODUSUREOHPVRUWKH\KDYHDOUHDG\EHHQVROYHG
XSGDWHG PLQ  PHWKRGRORJLHV FUHDWHG LQ WKH QDWLRQDO HJ VLPLODUO\ IRFXVHG VHFRQGDU\ YRFDWLRQDO
SURMHFW ,7 $FDGHP\ 7KH VXFFHVVIXO FRPSOHWLRQ RI WKH VFKRROV 
LQWHUYHQWLRQV LV SODQQHG IRU DW OHDVW  VFKRROV 
SULPDU\VFKRROVDQGVHFRQGDU\VFKRROV DWOHDVW 
WHDFKLQJDQGSURIHVVLRQDOHPSOR\HHV IURPSULPDU\ $ ,PSOHPHQWDWLRQ RI ZHELQDUV WR VXSSRUW VFKRRO
DQGVHFRQGDU\VFKRROVIURPVHFRQGDU\VFKRROV DQG GLJLWDOFRRUGLQDWRUV
 VWXGHQWV RI WHDFKLQJ SURJUDPV DW XQLYHUVLWLHV 7KH  :HELQDU WRSLFV ZLOO KDYH WR UHVSHFW WKH G\QDPLFV
FDSDFLW\ RI WKH SURMHFW DOORZV WKH LQYROYHPHQW RI DOO RI GLJLWDO WUDQVIRUPDWLRQ DQG LQ DGGLWLRQ WR
SUHVFKRROV SULPDU\ DQG VHFRQGDU\ VFKRROV 7KURXJK LQQRYDWLRQVDOVREULQJJXLGHOLQHVLQVHQVLWLYHDUHDV
HGXFDWLRQDO DFWLYLWLHV ZH H[SHFW DQ LPSDFW RQ  DVVRFLDWHGZLWKGLJLWL]DWLRQ&XUUHQWO\WKHZHELQDU
WHDFKLQJ DQG SURIHVVLRQDO HPSOR\HHV DQG PLQ  WRSLFV DUH GLYLGHG E\ 'LJ&RPS(GX >@ WR HQVXUH
VWXGHQWV RI WHDFKLQJ SURJUDPV DW XQLYHUVLWLHV 7KURXJK VXSSRUWLQHDFKRIWKHNH\DUHDV
LQQRYDWLYH PHWKRGRORJLHV DQG WUDLQHG SHGDJRJLFDO DQG  ,Q WKH WHDP ZKLFK ZLOO SULPDULO\ EH GHGLFDWHG WR
SURIHVVLRQDO VWDII PLQ  SULPDU\ DQG VHFRQGDU\ WKH HGXFDWLRQ RI VFKRRO GLJLWDO FRRUGLQDWRUV WKHUH
VFKRRO SXSLOV IURP PLQ  LQWHUYHQHG :H H[SHFW D ZLOO EH VSHFLDOL]DWLRQ  VSHFLDOLVWV IRU WKH XVH RI
VRFLHWDOLPSDFWRQWKHTXDOLW\RIHGXFDWLRQDQGJUDGXDWHV DUWLILFLDO LQWHOOLJHQFH LQ HGXFDWLRQ VSHFLDOLVWV IRU
SUHSDUHGWRKROGWKHSRVLWLRQVZLWKKLJKDGGHGYDOXH F\EHUVHFXULW\LQWKHHGXFDWLRQDOVSDFHHWF
7KH REMHFWLYH RI WKH DFWLYLW\ 6XSSRUW IRU WKH GLJLWDO  7KH FRRSHUDWLRQ RI WKH VFKRRO GLJLWDO FRRUGLQDWRU
WUDQVIRUPDWLRQ RI HGXFDWLRQ ZLOO EH IXOILOOHG E\  VXE ZLWK WKH QHWZRUN DGPLQLVWUDWRU LQ HQVXULQJ WKH
DFWLYLWLHV $,  IXQFWLRQDOLW\ RI WKH VFKRRO V GLJLWDO LQIUDVWUXFWXUH
ZLOODOVREHDQLPSRUWDQWWRSLFRIWKHZHELQDUV
$ (GXFDWLRQDQGSHUPDQHQWPHWKRGLFDOVXSSRUWRI
VFKRROGLJLWDOFRRUGLQDWRUV WKHVXEDFWLYLW\ZLOOEH  :HELQDUV ZLOO DOVR EH LPSOHPHQWHG LQ FRQQHFWLRQ
LPSOHPHQWHGE\83-âDQG&8  ZLWKWKHVROXWLRQRIUHVHDUFKWDVNVLQVXEDFWLYLW\)

7KH JRDO RI WKH VXEDFWLYLW\ LV WR EXLOG D VXVWDLQDEOH
V\VWHP RI HGXFDWLRQ DQG VXSSRUW IRU VFKRRO GLJLWDO $&RQIHUHQFHVIRUVFKRROGLJLWDOFRRUGLQDWRUV 6'&V 
FRRUGLQDWRUV LQ WKH FDUHHU SRVLWLRQ RU FDWHJRU\ RI ± QHWZRUNLQJ VKDULQJ RI JRRG SUDFWLFHGLVVHPLQDWLRQRI
SHGDJRJLFDOHPSOR\HHZKLFKZLOOV\QHUJLVWLFDOO\FRPELQH UHVHDUFKUHVXOWV
VXSSRUWIURPXQLYHUVLWLHVWKHFRUSRUDWHDQGFLYLOVHFWRUV :H FRQVLGHU FRQIHUHQFHV WR EH DQ H[WUHPHO\ HIIHFWLYH
DQGWKHJRRGSUDFWLFHRILQGLYLGXDOVFKRROV WRRO WKDW KHOSV EXLOG WKH FRPPXQLW\ RI VFKRRO GLJLWDO
7KHIROORZLQJDFWLYLWLHVZLOOEHSDUWRIWKHVXEDFWLYLWLHV FRRUGLQDWRUV 6'&V DQGFRQWULEXWHVWRWKHXQGHUVWDQGLQJ
$±$ RIWKHLUUROHLQVFKRROV
$ (GXFDWLRQ RI VFKRRO GLJLWDO FRRUGLQDWRUV ± 83-â ZLOO EH UHVSRQVLEOH IRU WKH VXEDFWLYLW\ LQ
VSHFLDOL]HGLQQRYDWLYHHGXFDWLRQLQWKHUDQJHRIKRXUV DGGLWLRQWR83-â H[SHUWV H[SHUWVIURP&8 H[SHUWV 
DQG SUDFWLFH LQ FRRSHUDWLRQ ZLWK WKH 'LJLWDO &RDOLWLRQ 
 7KHHGXFDWLRQDOSURJUDP LQQRYDWLYHHGXFDWLRQ LQ
ZLOO SDUWLFLSDWH LQ LWV LPSOHPHQWDWLRQ 7KH VXEDFWLYLW\
WKH VFRSH RI  KRXUV ZDV FUHDWHG ZLWKLQ WKH
ZLOOEHFDUULHGRXWE\H[SHUWVOHFWXUHUVIRUHGXFDWLRQDQG
1DWLRQDOSURMHFW,7$FDGHP\ ,7$ >@>@>@,Q
PHWKRGRORJLFDO VXSSRUW RI 6'&V DQG RWKHU H[SHUWV IRU
WKLV VFKRRO \HDU ZLWKLQ WKH IUDPHZRUN RI
FRQWHQWFUHDWLRQUHVHDUFKDQGWUDLQLQJRISHGDJRJLFDODQG
VXVWDLQDELOLW\DQLQQRYDWLYHKRXUVSURJUDPZDV
SURIHVVLRQDO HPSOR\HHV XQGHU WKH OHDGHUVKLS RI WKH
FUHDWHGDW83-âDQGLVEHLQJSLORWWHVWHG
PDQDJHU RI HGXFDWLRQ DQG PHWKRGRORJLFDO VXSSRUW RI
 &RQVLGHULQJ WKH G\QDPLFV RI GHYHORSPHQW LQ WKH 6'&V 7KHVH H[SHUWV ZLOO DOVR EH LQYROYHG LQ RWKHU VXE
ILHOGRIGLJLWDOWHFKQRORJLHV PDLQO\ZLWKWKHULVHRI DFWLYLWLHVHVSHFLDOO\LQVXEDFWLYLW\)
DUWLILFLDO LQWHOOLJHQFH DQG UHVSRQVLEOH XVH RI GDWD 
7KH HGXFDWLRQ DQG PHWKRGLFDO VXSSRUW RI 6'&V ZLOO
ZH DQWLFLSDWH D SHUPDQHQW QHHG IRU LQQRYDWLRQ LQ
FRPSOHPHQW WKH DFWLYLWLHV RI QDWLRQDO SURMHFW 6XSSRUW RI
LQQRYDWLYHVSHFLDOL]HG HGXFDWLRQ DW OHDVW RQFH D
KHOSLQJSURIHVVLRQV WKHUHFLSLHQWRIWKHQDWLRQDOSURMHFW
\HDU

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 509

LV 1,9D0  DQG LW ZLOO EH HQVXUHG WKDW WKHUH DUH QR $GDSWDWLRQLQFOXGHVLQQRYDWLYHHGXFDWLRQRISHGDJRJLF
GXSOLFDWLRQV DQG SURIHVVLRQDO VWDII RI SUHVFKRROV SULPDU\ DQG
2XWSXWRIVXEDFWLYLW\$LQFOXGHV VHFRQGDU\VFKRROVLQWKHVFRSHRIKRXUV
 PLQHGXFDWHG6'&VZLWKLQWKHIUDPHZRUNRI 'XULQJ 1DWLRQDO SURMHFW ZKLOH UHVSHFWLQJ WKH
VSHFLDOL]HGLQQRYDWLYHHGXFDWLRQ IUDPHZRUNRI'LJ&RPS(GX>@WKHFRQWHQWRILQQRYDWLYH
 PLQ  6'&V VXSSRUWHG ZLWKLQ WKH 'LJLWDO HGXFDWLRQRISHGDJRJLFDODQGSURIHVVLRQDOHPSOR\HHVZLOO
&RRUGLQDWRUV&OXE EH OLQNHG PDLQO\ WR WKH FRQWHQW RI FUHDWHG LQQRYDWLYH
 PLQZHELQDUVDQGPLQFRQVXOWDWLRQV PHWKRGRORJLHV IRU UHVHDUFK WHDFKLQJ ZLWK WKH VXSSRUW RI
 PLQFRQIHUHQFHVIRU6'&V GLJLWDO WHFKQRORJLHV 7KLV WUDLQLQJ ZLOO QRW EH RIIHUHG RU
LPSOHPHQWHGLQDQ\RWKHUJUDQWVFKHPHVDQGSURMHFWV
% $GDSWDWLRQRIWKH(8'LJ&RPS(GXIUDPHZRUNWR 2WKHUDFWLYLWLHVLQWKHIUDPHZRUNRILQIRUPDOHGXFDWLRQ
XQGHUJUDGXDWHHGXFDWLRQDQGHGXFDWLRQLQWKH ZLOO EH LPSOHPHQWHG IRU SHGDJRJLFDO DQG SURIHVVLRQDO
SURIHVVLRQDOGHYHORSPHQWRISHGDJRJLFDODQG HPSOR\HHVLQSUHVFKRROVSULPDU\VFKRROVDQGVHFRQGDU\
SURIHVVLRQDOHPSOR\HHVDWSUHVFKRROVSULPDU\DQG VFKRROV
VHFRQGDU\VFKRROV VXEDFWLYLW\ZLOOEHLPSOHPHQWHG  LPSOHPHQWDWLRQ RI VXEMHFW &OXEV IRU WHDFKHUV RI
E\1,9D083-â&8  LQGLYLGXDOVXEMHFWV
7KH JRDO RI WKH VXEDFWLYLW\ LV WR SUHSDUH DQG  UHDOL]DWLRQ RI ZHELQDUV DQG FRQVXOWDWLRQV RQ
LPSOHPHQWIRUPDODQGLQIRUPDOHGXFDWLRQRISHGDJRJLFDO LQQRYDWLYHPHWKRGRORJLHV
HPSOR\HHV DOVR IXWXUH  DQG SURIHVVLRQDO VWDII LQ  UHDOL]DWLRQRIDOOGD\VXEMHFWPHWKRGRORJLFDOGD\V
DFFRUGDQFHZLWKWKHQHHGVRIWKHGLJLWDOWUDQVIRUPDWLRQRI IRUWKHGLJLWDOWUDQVIRUPDWLRQRIHGXFDWLRQ
HGXFDWLRQ DQG VFKRRO LQ FRQQHFWLRQ ZLWK WKH IUDPHZRUN  VXSSRUWDQGUHDOL]DWLRQRIRSHQKRXUV
RI'LJ&RPS(GX>@  VXSSRUW IRU WKH DFWLYH SDUWLFLSDWLRQ ZLWK D
7KH VXEDFWLYLW\ ZLOO SULPDULO\ LQFOXGH WKH IROORZLQJ FRQWULEXWLRQ  RI SHGDJRJLFDO DQG SURIHVVLRQDO
DFWLYLWLHV%±% HPSOR\HHV LQ SUHVFKRROV SULPDU\ VFKRROV DQG
% (GXFDWLRQ DQG SHUPDQHQW VXSSRUW RI VFKRRO VHFRQGDU\ VFKRROV LQ GRPHVWLF DQG IRUHLJQ
SULQFLSDOV SHGDJRJLFDO OHDGHUV DQG SURIHVVLRQDO VWDII FRQIHUHQFHV IRFXVHG RQ WKH GLJLWDO WUDQVIRUPDWLRQ
OHDGHUV RISUHVFKRROVSULPDU\DQGVHFRQGDU\VFKRROVLQ RIHGXFDWLRQ
WKH ILHOG RI GLJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ ZLOO QRW  SDUWLFLSDQWV LQ HGXFDWLRQDO DFWLYLWLHV DQG WKH
EHLPSOHPHQWHGE\1,9D0  HGXFDWLRQDO SURJUDPV WKHPVHOYHV ZLOO EH WKH
 ,QQRYDWLYH HGXFDWLRQ IRU VFKRRO SULQFLSDOV LQ WKH VXEMHFW RI UHVHDUFK VXEDFWLYLW\ )  VR WKDW WKHRU\
UDQJH RI  KRXUV HGXFDWLRQ LV QRW WLHG WR WKH DQGSUDFWLFHDUHV\VWHPDWLFDOO\FRQQHFWHG
FDUHHU SRVLWLRQ RI D OHDGLQJ SHGDJRJLFDO RU  VXSSRUW RI WKH RQOLQH SODWIRUP IRU IRUPDO DQG
SURIHVVLRQDOHPSOR\HH  LQIRUPDO HGXFDWLRQ RI SHGDJRJLFDO DQG
 5HDOL]DWLRQ RI ZHELQDUV DQG FRQVXOWDWLRQV IRU SURIHVVLRQDOHPSOR\HHV
VFKRRO SULQFLSDOV UHVS OHDGLQJ SHGDJRJLFDO DQG  VXSSRUW IRU WKH SDUWLFLSDWLRQ RI H[SHUWV
SURIHVVLRQDOHPSOR\HHV /3 /3(V  SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV LQ
 6XSSRUW IRU ORFDO QHWZRUNLQJ RI VFKRRO SULQFLSDOV LQWHUQDWLRQDOLQWHUQVKLSVDQGHGXFDWLRQDODFWLYLWLHV
RU/3 /3(V DWWKHUHJLRQDOOHYHO 7KH HGXFDWLRQ RI SHGDJRJLFDO DQG SURIHVVLRQDO
 3DUWLFLSDWLRQ RI GLUHFWRUV UHVS /3 /3(V  DW HPSOR\HHV ZLOO EH LPSOHPHQWHG LQ YDULRXV IRUPV
FRQIHUHQFHV RQ WKH GLJLWDO WUDQVIRUPDWLRQ RI LQFOXGLQJIDFHWRIDFHPHHWLQJVK\EULGIRUPUHVXOWLQJLQ
HGXFDWLRQ 7KH NH\ WR VHOHFWLRQ ZLOO EH VLPLODU WR RSHQ HGXFDWLRQDO SURJUDPV RI WKH 022& W\SH 1,9D0
WKH RQH IRU 6'&V  WKH TXDOLW\ RI WKH 6FKRRO V ZLOO DOVR SURYLGH LQQRYDWLYH HGXFDWLRQ SURJUDPV LQ WKH
'LJLWDO 7UDQVIRUPDWLRQ $FWLRQ 3ODQ DVVHVVHG IROORZLQJDUHDVRIWRSLFV
GXULQJ WKH ILQDO SUHVHQWDWLRQ DW WKH HQG RI  XVH RI GLJLWDO WRROV IRU FUHDWLQJ ZRUNVKHHWV WHVWV
HGXFDWLRQ RU WKH XSGDWH RI WKLV SODQ  >@ DQG WKH HWFIRUXVHLQFODVVHV IDFHWRIDFHPHHWLQJV 
UHVXOWV RI WKH LPSOHPHQWDWLRQ RI WKH DFWLRQ SODQ  SRUWDOVXVHGE\VFKRROV 9,.,SRUWDO(GX3DJHDQG
OLVWHGLQWKHFRQIHUHQFHDSSOLFDWLRQ  5,$0DFFRXQW 
7KHHGXFDWLRQDOSURJUDP IRU VFKRROSULQFLSDOV ZLOOEH  HGXFDWLRQDO YLGHR DV D VWXGHQW OHDUQLQJ WRRO DQG
FUHDWHG ZLWKLQ WKH SURMHFW ZKLOH LW ZLOO EH FUHDWHG LQ VLPLODU
UHODWLRQWRWKHFUHDWLRQRIDYLVLRQDQGDQDFWLRQSODQIRU 
WKH GLJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ DQG VFKRRO &DUH (GXFDWLRQ SURJUDP QR  8VLQJ VFKRRO HGXFDWLRQDO
ZLOOEHWDNHQWRHQVXUHWKDWWKHUHDUHQRGXSOLFDWLRQVZLWK SODWIRUPV DQG V\VWHPV WR PDNH WKH ZRUN RI SHGDJRJLFDO
RWKHUSURJUDPVIRUVHQLRUSURIHVVLRQDODQGWHDFKLQJVWDII DQGSURIHVVLRQDOHPSOR\HHVPRUHHIILFLHQW
ZKLFK ZLOO EH SURYLGHG E\ 1,9D0 $W WKH VDPH WLPH D (GXFDWLRQ LV IRFXVHG RQ WKH XVH RI HGXFDWLRQDO
UHVHDUFK SHUVSHFWLYH ZLOO EH DSSOLHG WR WKLV SURJUDP DQG SODWIRUPV DQG V\VWHPV DYDLODEOH LQ 6ORYDN VFKRROV 7KH
LWVSDUWLFLSDQWV ZLWKLQVXEDFWLYLW\)  JRDO RI HGXFDWLRQ LV WR H[SDQG NQRZOHGJH DERXW WKH
 IXQFWLRQDOLWLHVRIDYDLODEOHSODWIRUPVDQGV\VWHPVIRUWKH
% $GDSWDWLRQ RI WKH 'LJ&RPS(GX IUDPHZRUN WR QHHGVRIWHDPFROODERUDWLRQFRPPXQLFDWLRQDQGVKDULQJ
HGXFDWLRQ LQ WKH SURIHVVLRQDO GHYHORSPHQW RI WHDFKLQJ RI GLJLWDO FRQWHQW DQG DW WKH VDPH WLPH WR GHYHORS WKH
VWDIIDQGSURIHVVLRQDOVWDII GLJLWDO FRPSHWHQFHV RI SHGDJRJLFDO DQG SURIHVVLRQDO

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 510

HPSOR\HHV LQ DFFRUGDQFH ZLWK 'LJ&RPS(GX >@ LQ WKH SUDFWLFH UHIOHFWLRQ RI WKH H[SHULHQFH RI SHGDJRJLFDO
ILHOG RI SURIHVVLRQDO HQJDJHPHQW ZRUN FRPPXQLFDWLRQ HPSOR\HHV IURP WKH DSSOLFDWLRQ RI FUHDWHG GLGDFWLF
SURIHVVLRQDOFROODERUDWLRQUHIOHFWLYHSUDFWLFHFRQWLQXRXV PDWHULDOVLQWKHWHDFKLQJSURFHVV
SURIHVVLRQDO GHYHORSPHQW  DQG GLJLWDO UHVRXUFHV 
VHOHFWLRQ RI GLJLWDO UHVRXUFHV FUHDWLRQ DQG HGLWLQJ RI (GXFDWLRQ SURJUDP QR  (GXFDWLRQ IRU HIIHFWLYH XVH
GLJLWDO UHVRXUFHV RUJDQL]DWLRQ SURWHFWLRQ DQG VKDULQJ RI RIDUWLILFLDOLQWHOOLJHQFHWRROVXVLQJQRQIRUPDOHGXFDWLRQ
GLJLWDOUHVRXUFHV  PHWKRGV
 (GXFDWLRQ ZLOO SURYLGH WKH QHFHVVDU\ NQRZOHGJH DQG
(GXFDWLRQ SURJUDP QR  8VH RI HOHFWURQLF VHUYLFHV WRROVWRHIIHFWLYHO\DQGVDIHO\JXLGH\RXQJSHRSOHLQWKH
DQG DSSOLFDWLRQV SURYLGHG E\ WKH 0LQLVWU\ RI (GXFDWLRQ XVH RI DUWLILFLDO LQWHOOLJHQFH LQ DQG RXW RI VFKRRO $IWHU
6FLHQFH 5HVHDUFK DQG 6SRUW RI WKH 6ORYDN 5HSXEOLF FRPSOHWLQJ WKH WUDLQLQJ \RXWK ZRUNHUV \RXWK ZRUN
0â99Dâ65  FRRUGLQDWRUV \RXWK OHDGHUV \RXWK OHDGHUV SHGDJRJLFDO
7KHDLPRIWKHHGXFDWLRQLVWRH[SDQGNQRZOHGJHDERXW DQGSURIHVVLRQDOHPSOR\HHVRIVFKRROVDQGRWKHUSHUVRQV
HOHFWURQLF VHUYLFHV DQG DSSOLFDWLRQV SURYLGHG E\ WKH ZRUNLQJ ZLWK \RXWK ZLOO KDYH DQ RYHUYLHZ RI WKH EDVLF
0â99Dâ65WKURXJKWKH5,$0DFFRXQWDQGWKHLUXVHLQ GHILQLWLRQV DQG FRQFHSWV UHODWHG WR DUWLILFLDO LQWHOOLJHQFH
WKH WHDFKLQJ SURFHVV (GXFDWLRQ LV PDLQO\ IRFXVHG RQ DQG LWV LPSDFW RQ VRFLHW\ DQG HGXFDWLRQ 7KH\ ZLOO
IDPLOLDUL]DWLRQ ZLWK H[LVWLQJ GLJLWDO HGXFDWLRQDO FRQWHQW DFTXLUH NQRZOHGJH DERXW WKH HWKLFDO DQG OHJDO DVSHFWV
FUHDWLRQ RI QHZ GLJLWDO HGXFDWLRQDO FRQWHQW '(&  XVLQJ DVVRFLDWHG ZLWK WKH XVH RI DUWLILFLDO LQWHOOLJHQFH DQG ZLOO
FRPSDWLEOH GLJLWDO WRROV DQG SXEOLVKLQJ ZLWKLQ WKH 9,., EH DEOH WR LGHQWLI\ SRWHQWLDO ULVNV DQG FKDOOHQJHV
HGXFDWLRQDODQGLQIRUPDWLRQSODWIRUP 3DUWLFLSDQWV ZLOO JHW WR NQRZ WKH PRVW IDPRXV
 DSSOLFDWLRQV WKDW XVH DUWLILFLDO LQWHOOLJHQFH DQG ZLOO EH
(GXFDWLRQ SURJUDP QR  8VH RI WKH 'LJ&RPS(GX DEOH WR VXSSRUW FULWLFDO WKLQNLQJ DQG GLJLWDO OLWHUDF\ RI
IUDPHZRUN LQ WKH SURFHVV RI VHOIGHYHORSPHQW RI FKLOGUHQ DQG \RXWK LQ FRQQHFWLRQ ZLWK DUWLILFLDO
SHGDJRJLFDOVWDII LQWHOOLJHQFH ,Q DGGLWLRQ HGXFDWLRQ ZLOO VXSSRUW WKH
(GXFDWLRQ LV IRFXVHG RQ WKH SURFHVV RI VHOI LQQRYDWLYHQHVV DQG FUHDWLYLW\ RI WKH \RXWK 3HRSOH
GHYHORSPHQW RI SHGDJRJLFDO HPSOR\HHV LQ WKH DUHD RI ZRUNLQJ ZLWK \RXWK ZLOO OHDUQ KRZ WR PRWLYDWH \RXWK WR
GLJLWDOFRPSHWHQFHVLQDFFRUGDQFHZLWKWKH'LJ&RPS(GX FUHDWH WKHLU RZQ SURMHFWV XVLQJ DUWLILFLDO LQWHOOLJHQFH DQG
IUDPHZRUNDQGLWVVWDQGDUGV>@7KHJRDORIHGXFDWLRQLV KRZ WR VXSSRUW FRRSHUDWLRQ DQG WHDPZRUN (GXFDWLRQ
WR UHIOHFW HYDOXDWH DQG GHYHORS WKH GLJLWDO FRPSHWHQFLHV ZLOO WKHUHE\ FRQWULEXWH WR WKH GLJLWDO WUDQVIRUPDWLRQ RI
RI SHGDJRJLFDO HPSOR\HHV ZKLFK FDQ EH IXUWKHU HGXFDWLRQ DQG WKH GHYHORSPHQW RI \RXWK LQ WKH ILHOG RI
LQFRUSRUDWHG LQWR WKH SURIHVVLRQDO SRUWIROLR DQG XVHG WR GLJLWDO OLWHUDF\ FULWLFDO WKLQNLQJ DQG FUHDWLYLW\ WKHUHE\
LQFUHDVH WKH FDUHHU OHYHO E\ GHPRQVWUDWLQJ WKHVH EHWWHU SUHSDULQJ WKHP IRU IXWXUH FKDQJHV LQ WKH ODERU
FRPSHWHQFLHVLQWKHDWWHVWDWLRQSRUWIROLRE\LQWHUZHDYLQJ PDUNHW DQG SURYLGLQJ VSDFH IRU WKH GHYHORSPHQW RI
WKH DUHDV RI WKH SURIHVVLRQDO VWDQGDUG RI SHGDJRJLFDO HQWUHSUHQHXULDODQGGLJLWDOVNLOOV
HPSOR\HHVDQG'LJ&RPS(GX 
 (GXFDWLRQSURJUDPQR(GXFDWLRQIRUFLWL]HQVKLSLQ
(GXFDWLRQ SURJUDP QR  (GXFDWLRQDO YLGHR DV D WKHGLJLWDODJH
VWXGHQWOHDUQLQJWRRO 7KH HGXFDWLRQDO DFWLYLW\ LV LQWHQGHG IRU SHGDJRJLFDO
(GXFDWLRQLVIRFXVHGRQWKHFUHDWLRQRIDQHGXFDWLRQDO DQG QRQSHGDJRJLFDO HPSOR\HHV ZRUNLQJ ZLWK FKLOGUHQ
YLGHR WKDW ZLOO VHUYH VWXGHQWV DV D VXSSRUW LQ PDVWHULQJ DQG \RXWK ,W LV IRFXVHG RQ WKH GHYHORSPHQW RI GLJLWDO
WKHVXEMHFWPDWWHU7KHDLPRIWKHHGXFDWLRQLVWRH[SDQG FLWL]HQVKLS DQG GLJLWDO FRPSHWHQFHV RI WKH VW FHQWXU\
WKH GLJLWDO FRPSHWHQFHV RI SHGDJRJLFDO DQG SURIHVVLRQDO 7KH DLP LV WR SUHSDUH WKH HGXFDWLRQDO V\VWHP IRU DFWLYH
HPSOR\HHV LQ WKH FUHDWLRQ RI YLVXDO HGXFDWLRQDO FRQWHQW VDIH DQG UHVSRQVLEOH SDUWLFLSDWLRQ LQ WKH GLJLWDO
DQGLWVXVHLQWHDFKLQJZLWKDQHPSKDVLVRQWKHWUDQVIHU HQYLURQPHQW DQG WR SURPRWH FULWLFDO WKLQNLQJ HWKLFDO
RI XVHG GLJLWDO FRPSHWHQFHV WR SXSLOV WKH FUHDWLRQ RI YDOXHV DQG WKH FRUUHFW XVH RI GLJLWDO WHFKQRORJLHV 7KH
YLVXDO GLJLWDO HGXFDWLRQDO FRQWHQW E\ SXSLOV WR VXSSRUW DFWLYLW\ LQFOXGHV VHYHUDO PRGXOHV WKDW LQFOXGH DQ
SHHUOHDUQLQJ LQWURGXFWLRQ WR GLJLWDO FLWL]HQVKLS VDIHW\ DQG VHFXULW\ LQ
 WKHGLJLWDOZRUOGLGHQWLILFDWLRQRIWKUHDWVDQGULVNVLQWKH
(GXFDWLRQ SURJUDP QR  &UHDWLRQ RI GLGDFWLF RQOLQH HQYLURQPHQW FULWLFDO WKLQNLQJ DQG REWDLQLQJ
PDWHULDOV IRU IDFHWRIDFH DQG GLVWDQFH OHDUQLQJ XVLQJ LQIRUPDWLRQIURPWKHRQOLQHHQYLURQPHQWDQDO\VLVRIWKH
YDULRXVGLJLWDOWRROV GLJLWDO IRRWSULQW DQG RQOLQH LGHQWLW\ HWKLFV DQG
(GXFDWLRQ LV IRFXVHG RQ WKH FUHDWLRQ RI RZQ GLGDFWLF UHVSRQVLELOLW\ LQ WKH GLJLWDO HQYLURQPHQW DVSHFWV RI
PDWHULDOV WKDW FDQ EH XVHG GXULQJ IDFHWRIDFH DQGRU LQWHUSHUVRQDO RQOLQH FRPPXQLFDWLRQ OHJDO DVSHFWV RI
GLVWDQFH OHDUQLQJ XVLQJ YDULRXV GLJLWDO WRROV 7KH DLP RI GLJLWDO FLWL]HQVKLS VXSSRUW LQ WKH HGXFDWLRQ RI GLJLWDO
WKH HGXFDWLRQ LV WR GHHSHQ DQG GHYHORS WKH SURIHVVLRQDO FLWL]HQV 7KH DFWLYLW\ ZLOO EH EDVHG RQ SDUWLFLSDWRU\
DQGGLJLWDOFRPSHWHQFHVRISHGDJRJLFDOHPSOR\HHVLQWKH PHWKRGV DQG LQWHUDFWLYH DSSURDFKHV RI LQIRUPDO
FUHDWLRQ RI WKHLU RZQ GLGDFWLF PDWHULDOV GLJLWDO HGXFDWLRQ 3DUWLFLSDQWV ZLOO EH HQJDJHG LQ GLVFXVVLRQV
HGXFDWLRQDO FRQWHQW ZRUNVKHHWV HWF  ZLWK DQ HPSKDVLV JURXSH[HUFLVHVFDVHVWXGLHVDQGVLPXODWLRQV$VSDUWRI
RQ KLJKHU FRJQLWLYH SURFHVVHV WKH SRVVLELOLW\ RI WKH DFWLYLW\ WKH EHVW NQRZKRZ IURP DEURDG ZLOO EH
LPSOHPHQWLQJ WKH FUHDWHG PDWHULDOV LQWR SHGDJRJLFDO WUDQVIHUUHG 7HDFKLQJ ZLOO EH VXSSRUWHG E\ PXOWLPHGLD

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 511

PDWHULDOV DQG VSHFLILF H[DPSOHV WKDW DUH UHOHYDQW WR WKH  ZH ZLOO FUHDWH 022&W\SH HGXFDWLRQDO SURJUDPV
WDUJHW JURXS 6XFFHVVIXO FRPSOHWLRQ RI WKLV OHDUQLQJ IRU VHOHFWHG WRSLFV DFFHVVLEOH WR SHGDJRJLFDO DQG
DFWLYLW\ OHDGV WR D EHWWHU XQGHUVWDQGLQJ RI WKH GLJLWDO SURIHVVLRQDO HPSOR\HHV DQG VWXGHQWV RI WHDFKLQJ
HQYLURQPHQW WKH DELOLW\ WR LGHQWLI\ DQG DGGUHVV GLJLWDO SURJUDPVDWXQLYHUVLWLHV
ULVNV H[KLELW UHVSRQVLEOH DQG HWKLFDO EHKDYLRU RQOLQH  WKH FRQWHQW DQG IRUPV RI HGXFDWLRQ ZLOO EH
LPSURYH FROODERUDWLRQ DQG HIIHFWLYH OHDUQLQJ IRU VDIH FRQWLQXRXVO\XSGDWHGZLWKUHJDUGWRWKHRXWSXWVRI
FLWL]HQVKLSDQGSDUWLFLSDWLRQLQWKHGLJLWDOZRUOG HGXFDWLRQDOUHVHDUFK VXEDFWLYLW\) 
  ZLWKLQWKHIUDPHZRUNRIHGXFDWLRQGXULQJQDWLRQDO
7DNLQJ LQWR DFFRXQW WKH WLPH DQG SURIHVVLRQDO SURMHFW 13  ZH ZLOO VXSSRUW WKH QHWZRUNLQJ RI
FDSDFLWLHV ZH ZLOO FUHDWH PRUH SURJUDPV GXULQJ WKH WHDFKHUV LQFOXGLQJIXWXUHRQHV RIERWKJUDGHV DOO
LPSOHPHQWDWLRQ RI WKH HQWLUH QDWLRQDO SURMHFW LQ VXFK D WKUHH F\FOHV  RI SULPDU\ DQG VHFRQGDU\ VFKRROV
ZD\ WKDW WKH\ UHIOHFW WKH GHYHORSPHQW LQ WKH GLJLWDO QRWRQO\KRUL]RQWDOO\EXWDOVRYHUWLFDOO\WKLVZLOO
WUDQVIRUPDWLRQ WKH SURJUHVV DQG WKH QHHGV RI WKH WDUJHW EULQJ D EHQHILW IRU UHIOHFWLQJ WKH WUDQVIHU RI WKH
JURXSSHGDJRJLFDODQGSURIHVVLRQDOHPSOR\HHVDVZHOODV QHHGVRIKLJKHUJUDGHVWRORZHURQHV
WKH\RXWK 83-âZLOOEHUHVSRQVLEOHIRUWKHHGXFDWLRQDQGVXSSRUW
 RIVFKRROSULQFLSDOV$OOWKUHHSDUWQHUVZLOOEHUHVSRQVLEOH
% $GDSWDWLRQ RI WKH 'LJ&RPS(G IUDPHZRUN WR WKH IRUWKHDGDSWDWLRQRI'LJ&RPS(GX>@WRHGXFDWLRQLQWKH
XQLYHUVLW\ WUDLQLQJ RI IXWXUH WHDFKHUV ZLOO QRW EH SURIHVVLRQDO GHYHORSPHQW RI SHGDJRJLFDO DQG
LPSOHPHQWHGE\1,9D0  SURIHVVLRQDO HPSOR\HHV WKH GHILQLWLRQ RI WKH FRQWHQW LV
,QQRYDWLRQ RI WKH SURILOH RI D JUDGXDWH RI WKH VWXG\ JLYHQ DERYH  SUHVFKRROV  1,9D0 SULPDU\ DQG
ILHOGRIWHDFKLQJDQGSHGDJRJLFDOVFLHQFHVIURPWKHSRLQW VHFRQGDU\VFKRROVWRWKHH[WHQWRIWKHQXPEHURIXSGDWHG
RIYLHZRI'LJ&RPS(GX DQGFUHDWHGLQQRYDWLYHPHWKRGRORJLHV&8DQG83-âZLOO
,Q WKH SURMHFW D JUDGXDWH SURILOH ZLOO EH FUHDWHG LQ EH UHVSRQVLEOH IRU DGDSWLQJ 'LJ&RPS(GX >@ WR
FRQQHFWLRQZLWKWKH'LJ&RPS(GXIUDPHZRUN>@DQGWKH XQGHUJUDGXDWHHGXFDWLRQLQD UDWLRRIDSSUR[LPDWHO\
LQQRYDWLYH PHWKRGRORJLHV EHLQJ FUHDWHG DV D 7KH VXEDFWLYLW\ XQGHU WKH OHDGHUVKLS RI WKH PDQDJHUV
UHFRPPHQGHG VWDQGDUG IRU DOO IDFXOWLHV SUHSDULQJ IXWXUH ZLOO EH LPSOHPHQWHG SULPDULO\ E\ H[SHUWV IRU FRQWHQW
WHDFKHUV  D WRWDO RI  VWXG\ SURJUDPVVXEMHFWV )RU WKH FUHDWLRQ UHVHDUFK DQG HGXFDWLRQ RI SHGDJRJLFDO DQG
VDNH RI VWDQGDUGL]DWLRQ ZH ZLOO DOVR LQYROYH H[SHUWV SURIHVVLRQDO HPSOR\HHV H[SHUW HPSOR\HHV RI 1,9D0 LQ
IURP RWKHU XQLYHUVLWLHV LQ WKH LQQRYDWLRQ RI WKH JUDGXDWH FRRSHUDWLRQ ZLWK ,7 H[SHUWV ([SHUWV IURP RWKHU
SURILOH XQLYHUVLWLHV SULPDU\ VFKRROV VHFRQGDU\ VFKRROV DQG
,PSOHPHQWDWLRQ RI 'LJ&RPS(GX LQ XQGHUJUDGXDWH RWKHU H[SHUWV IURP WKH HQYLURQPHQW RI WKH 'LJLWDO
WHDFKHUWUDLQLQJ &RDOLWLRQ ZLOO DOVR EH LQYROYHG LQ WKH VXEDFWLYLW\
7KH FUHDWHG VWDQGDUG WRJHWKHU ZLWK WKH FUHDWHG WRJHWKHUWKHVXEDFWLYLW\ZLOOEHLPSOHPHQWHGE\PLQ
LQQRYDWLYH PHWKRGRORJLHV IRU XVH SULPDULO\ LQ GLGDFWLF H[SHUWV
VXEMHFWV  ZLOO EH SURYLGHG WR DOO KLJKHU HGXFDWLRQ 2XWSXWRIVXEDFWLYLW\%LQFOXGHV
LQVWLWXWLRQV +(,V SUHSDULQJIXWXUHWHDFKHUVDVDQRXWSXW  FUHDWHGSURJUDPRILQQRYDWLYHHGXFDWLRQIRUVFKRRO
IURPWKHSURMHFW:HDVVXPH DGLUHFWLPSDFWRQ&8DQG SULQFLSDOV ZLWK DQ HPSKDVLV RQ FKDQJH
83-âIRUPLQVWXGHQWVRIWHDFKLQJSURJUDPVGXULQJ PDQDJHPHQW LQ WKH GLJLWDO WUDQVIRUPDWLRQ RI
WKHLPSOHPHQWDWLRQRIWKHSURMHFW HGXFDWLRQDQGVFKRRODQGLWVLPSOHPHQWDWLRQPLQ
'HYHORSLQJ GLJLWDO VNLOOV RI VWXGHQWV RI WHDFKLQJ IUHTXHQWHUV
SURJUDPVDWWKH8QLYHUVLW\±,&'/  PLQ  ZHELQDUV PLQ  FRQVXOWDWLRQV IRU
 XSGDWLQJRIWHDFKLQJPDWHULDOVFUHDWHGLQ1DWLRQDO GLUHFWRUVDQGUHJLRQDOQHWZRUNLQJPHHWLQJV
SURMHFW ,7$ DQG FUHDWLRQ RI QHZ PDWHULDOV IRU H  FUHDWHGSURJUDPVRILQQRYDWLYHHGXFDWLRQDQGWKHLU
OHDUQLQJIRUVHOHFWHG,&'/PRGXOHV>@ LPSOHPHQWDWLRQ IRU SUHVFKRROV SULPDU\ DQG
 FRQVXOWDWLRQVRQ,&'/PRGXOHVDWXQLYHUVLWLHV VHFRQGDU\ VFKRROV PLQ  PLQ 
 ,&'/ WHVWLQJ RI WHDFKLQJ SURJUDP VWXGHQWV DW IUHTXHQWHUV 
XQLYHUVLWLHV  PLQ  VWXGHQWV RI WHDFKLQJ SURJUDPV DOVR
7KH GLYLGLQJ OLQH EHWZHHQ WKH PHQWLRQHG SODQQHG WUDLQHG DFFRUGLQJ WR WKH 'LJ&RPS(GX IUDPHZRUN
HGXFDWLRQDO DFWLYLWLHV DQG HGXFDWLRQ ILQDQFHG IURP >@
5HFRYHU\ DQG 5HVLOLHQFH 3ODQ RU RI RWKHU SURMHFWV  DSSUR[LPDWHO\  IDFHWRIDFH PHHWLQJV RI
VXSSRUWHGE\(8VWUXFWXUDOIXQGVLVDVIROORZV 7HDFKHUV &OXEV
 WKH FRQWHQW RI IRUPDO DQG LQIRUPDO HGXFDWLRQ LQ  PLQZHELQDUVDQGFRQVXOWDWLRQVRQLQQRYDWLYH
13 ZLOO EH GHILQHG E\ WKH FRQWHQW RI WKH FUHDWHG PHWKRGRORJLHV
LQQRYDWLYHPHWKRGRORJLHV  PLQPHWKRGLFDOGD\VDQGRSHQKRXUV
 ZH ZLOO LPSOHPHQW D ODUJH DPRXQW RI LQIRUPDO  SDUWLFLSDWLRQ RI WHDFKHUV LQ GRPHVWLF DQG IRUHLJQ
HGXFDWLRQ LQ 13  WHDFKHUV  FOXEV ZHELQDUV FRQIHUHQFHV IRFXVHG RQ WKH GLJLWDO WUDQVIRUPDWLRQ
FRQVXOWDWLRQV DOOGD\ VXEMHFW PHWKRGLFDO GD\V RI HGXFDWLRQ PLQ  SHGDJRJLFDO DQG
RSHQ FODVVHV DQG VXSSRUW WKH SDUWLFLSDWLRQ RI SURIHVVLRQDOHPSOR\HHV 
SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV LQ  PLQ  022&V ±  VXEMHFWEDVHG DQG  IRU
LQWHUQDWLRQDODQGGRPHVWLFFRQIHUHQFHV GHYHORSLQJ GLJLWDO FRPSHWHQFLHV DFFRUGLQJ WR WKH
'LJ&RPS(GXIUDPHZRUN>@

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 512

& &UHDWLRQDQGXVHRIPHWKRGRORJLFDOPDWHULDOVWR  &UHDWLRQ DQG YHULILFDWLRQ RI QHZ PHWKRGRORJLHV
VXSSRUWWKHGLJLWDOWUDQVIRUPDWLRQRIHGXFDWLRQ WKH ZLOOWDNHSODFHLQWKHVDPHRULQDVLPLODUZD\DVLQ
VXEDFWLYLW\ZLOOEHLPSOHPHQWHGE\1,9D083-â 13 ,7 $FDGHP\ ± VW YHUVLRQ VW PXOWLSOH
&8  YHULILFDWLRQLQSUDFWLFHDQGIHHGEDFNPRGLILFDWLRQ
DIWHU VW YHULILFDWLRQ QG YHULILFDWLRQ ILQDO
7KH JRDO RI WKH VXEDFWLYLW\ LV WR FUHDWH LQQRYDWLYH PRGLILFDWLRQ DIWHU QG YHULILFDWLRQ SHGDJRJLFDO
PHWKRGRORJLHV IRU WHDFKLQJ VHOHFWHG WRSLFV LQ JHQHUDO DQGSURIHVVLRQDOHPSOR\HHVRISUHVFKRROVSULPDU\
HGXFDWLRQ VXEMHFWV DW SULPDU\ DQG VHFRQGDU\ VFKRROV DQG DQGVHFRQGDU\VFKRROVZLOOEHLQYROYHGQRWRQO\LQ
WHDFKLQJLQSUHVFKRROVZKLFKZLOOUHIOHFWWKHODWHVWWUHQGV YHULILFDWLRQ EXW DOVR LQ WKH FUHDWLRQ RI LQQRYDWLYH
LQ GLJLWDOL]DWLRQ RI VRFLHW\ HJ DUWLILFLDO LQWHOOLJHQFH  PHWKRGRORJLHV DW OHDVW LQ WKH IRUP RI
0RUHRYHULWLVHVVHQWLDOWRDGDSWWKHVHPHWKRGRORJLHVIRU FRQVXOWDWLRQV
ZLGH XVH LQ UHDO FRQGLWLRQV LQFOXGLQJ DOWHUQDWLYH
SURFHGXUHVLQYLHZRIWKHVFKRRO VHTXLSPHQWZLWKGLJLWDO  :H FRQVLGHU LW QRQVWUDWHJLF DQG XQHFRQRPLFDO WR
WRROV FUHDWHDQ\WHDFKLQJPDWHULDOVZLWKRXWSDUWLFLSDWLRQ
ZLWK SUDFWLFH 2Q WKH RQH KDQG WKH FUHDWHG
7KH VXEDFWLYLW\ ZLOO SULPDULO\ LQFOXGH WKH IROORZLQJ PDWHULDOVPD\KDYHIODZVWKDWFDQQRWEHGHWHFWHGLQ
DFWLYLWLHV&±& DQ\RWKHUZD\WKDQE\OHDUQLQJ2QWKHRQHKDQG
& 8SGDWLQJ H[LVWLQJ DQG FUHDWLQJ QHZ LQQRYDWLYH WKH SRWHQWLDO RI SHGDJRJLFDO DQG SURIHVVLRQDO
PHWKRGRORJLHV IRU UHVHDUFK WHDFKLQJ ZLWK WKH VXSSRUW RI HPSOR\HHV ZKR DIWHU YHULILFDWLRQ EHFRPH QDWXUDO
GLJLWDOWHFKQRORJLHV SURPRWHUVRILQQRYDWLYHPDWHULDOVZRXOGEHORVW
%\ LQQRYDWLYH PHWKRGRORJ\ ZH PHDQ SUHSDUDWLRQ IRU &6XSSRUWIRUWKHXVHRIFUHDWHGPHWKRGRORJLHV
RQHOHVVRQZKLOHLWVLQWHJUDOSDUWLV  $OOLQQRYDWLYHPHWKRGRORJLHVZLOOEHDFFHVVLEOHWR
 LQIRUPDWLRQ VKHHW  FRQWHQW IRFXVHG PDLQO\ RQ DOO SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV RI
OHDUQLQJ JRDOV DQG GLGDFWLF SUREOHPV LQ WHDFKLQJ SUHVFKRROV SULPDU\ DQG VHFRQGDU\ VFKRROV
WKHJLYHQWRSLF 6XSSRUWLQJ WKHLU XVH DOVR EHORQJV WR WKH WDVNV RI
 UHFRPPHQGHGFRQWHQWDQGSURFHGXUHEDVHGRQWKH 6'&V
FRQVWUXFWLYLVWPRGHO UHVHDUFKF\FOH (RU(  )RUWKHHIIHFWLYHXVHRIPHWKRGRORJLHVVXSSRUWLQJ
 DOWHUQDWLYHV ZLWK FRPPHQWDU\ SHUVRQDOL]DWLRQ HGXFDWLRQDO DFWLYLWLHV ZLOO EH LPSOHPHQWHG IRU
DVVHVVPHQW HIIHFWLYH XVH RI GLJLWDO WHFKQRORJLHV VFKRROV SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV
« ZKLFKDUHEDVHGRQH[SHULPHQWDOYHULILFDWLRQ SULPDULO\LQWKHIRUPRIZHELQDUV VXEDFWLYLW\% 
 ZRUNVKHHWVZLWKGLJLWDOWRROV 7KHOHDUQLQJPDWHULDOVZLOOQRWKDYHWKHFKDUDFWHURID
)URPWKHPDWHULDOVFUHDWHGLQWKLVZD\SHGDJRJLFDODQG WH[WERRN7KHVHDUHLQQRYDWLYHPHWKRGRORJLHVIRUWHDFKLQJ
SURIHVVLRQDO HPSOR\HHV ZLOO EH DEOH WR FKRRVH VXLWDEOH VHOHFWHGWRSLFVLQWHQGHGIRUSHGDJRJLFDODQGSURIHVVLRQDO
DGGLWLRQVDQGDOWHUQDWLYHVWRWKHPDWHULDOVFUHDWHGWKURXJK HPSOR\HHV 7KH\ ZLOO EH FUHDWHG LQ FRRSHUDWLRQ EHWZHHQ
5HFRYHU\ DQG 5HVLOLHQFH 3ODQ 6R LW ZLOO QRW EH D WHDFKHUV IURP SUDFWLFH DQG H[SHUWV IURP LQGLYLGXDO
GXSOLFDWLRQEXWDVXSSOHPHQW0RUHRYHUWKHZD\WKH\DUH SDUWQHUV
FUHDWHG ZLOO VXSSRUW QHWZRUNLQJ EHWZHHQ VFKRROV DQG $OO WKUHH SDUWQHUV ZLOO EH UHVSRQVLEOH IRU WKH VXE
XQLYHUVLWLHV WKDW SUHSDUH IXWXUH WHDFKHUV 7KH\ ZLOO EH DFWLYLW\RQO\1,9D0IRUSUHVFKRROV7KHVFRSHRIDFWLYLW\
FUHDWHG LQ FRRSHUDWLRQ EHWZHHQ SHGDJRJLFDO DQG FRUUHVSRQGV WR WKH SHUVRQQHO FDSDFLWLHV RI WKH SDUWQHUV
SURIHVVLRQDO HPSOR\HHV IURP SUDFWLFH DQG H[SHUWV IURP DFFRUGLQJ WR WKH EXGJHW LQ WKH LWHP ([SHUWV  FRQWHQW
LQGLYLGXDO SDUWQHUV 7KH SURFHVV RI WKHLU FUHDWLRQ ZLOO WR FUHDWLRQ PHWKRGRORJ\  UHVHDUFK HGXFDWLRQ RI
VRPH H[WHQW EH D SDUW RI HGXFDWLRQDO UHVHDUFK IRU WKH SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV LQ FRRSHUDWLRQ
WHDFKLQJ RI JHQHUDO HGXFDWLRQ VXEMHFWV DW SULPDU\ DQG ZLWK ,7 H[SHUWV ([SHUWV IURP RWKHU XQLYHUVLWLHV SULPDU\
VHFRQGDU\ VFKRROV DQG IRU WHDFKLQJ DW SUHVFKRROV VXE VFKRROV VHFRQGDU\ VFKRROV DQG RWKHU H[SHUWV IURP WKH
DFWLYLW\)  'LJLWDO&RDOLWLRQHQYLURQPHQWZLOODOVREHLQYROYHGLQWKH
7KH FUHDWLRQ RI QHZ LQQRYDWLYH PHWKRGRORJLHV FUHDWLRQDQGXVH:HFRQVLGHUWKHFUHDWLRQRIDVSDFHIRU
PLQLPXP   ZLOO EH IRFXVHG SULPDULO\ RQ VXEMHFWV LQQRYDWLYH WHDFKHUV WR VKDUH WKHLU SURYHQ DQG HIIHFWLYH
ZKRVH FRQWHQW KDV QRW EHHQ SURFHVVHG LQ WKH 13 ,7 WHDFKLQJ PHWKRGV WREH DQ LPSRUWDQWSDUWRIEXLOGLQJ WKH
$FDGHP\EXWDOVRKXPDQLWLHVDQGVRFLDOVFLHQFHVXEMHFWV FXOWXUH RI 6ORYDN HGXFDWLRQ ZKHUH WKH WHDFKHU ZLOO EH
DQGIRUWHDFKLQJDWSUHVFKRROV XQGHUVWRRGDVDQH[SHUWQRWMXVWDUHFHLYHURINQRZOHGJH
IURPDFDGHPLFVDQGPHWKRGRORJLVWV
:LWKLQ WKH 13 ,7 $FDGHP\  LQQRYDWLYH
PHWKRGRORJLHV ZHUH FUHDWHG IRU WHDFKLQJ QDWXUDO VFLHQFH 2XWSXWRIVXEDFWLYLW\&LQFOXGHV
VXEMHFWV PDWKHPDWLFV DQG FRPSXWHU VFLHQFH $OO DUH  PLQ  XSGDWHG DQG  QHZ LQQRYDWLYH
DFFHVVLEOH ZLWKLQ VXVWDLQDELOLW\ 6RPH RI WKHP ZH PHWKRGRORJLHV
HVWLPDWHDWOHDVW QHHGWREHXSGDWHGGXHWRWKHXVHRI  PLQ   VHFRQGDU\ VFKRRO SXSLOV DQG 
QHZ GLJLWDO WRROV WKDW DUH DYDLODEOH DQG ZHUH QRW \HW SULPDU\ VFKRRO SXSLOV ZKR OHDUQ DOVR ZLWK WKH
DYDLODEOHDWWKHWLPHWKHPHWKRGRORJLHVZHUHFUHDWHG7KLV VXSSRUWRILQQRYDWLYHPHWKRGRORJLHVDWPLQ
LV LPSRUWDQW GXH WR WKH SDFH RI GLJLWL]DWLRQ LWV QHZ VFKRROV
SRVVLELOLWLHV HJ DUWLILFLDO LQWHOOLJHQFH D JUHDWHU QXPEHU
RIVFKRROVWKDWXVHGLIIHUHQW/06V\VWHPV 
& 9HULILFDWLRQ RI PHWKRGRORJLHV LQ SUDFWLFH
FRQGLWLRQV DQG WKHLU DGDSWDWLRQ DFFRUGLQJ WR IHHGEDFN
IURPSUDFWLFH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 513

' &UHDWLRQRIDFRPPXQLW\RIFRRSHUDWLRQZLWKRWKHU ( &UHDWLRQRIDQDO\VHVH[SHUWLVHVWDQGDUGV
RUJDQL]DWLRQVDQGGHSDUWPHQWVRIWKH0â99Dâ65 UHFRPPHQGDWLRQVFULWHULDDQGPHFKDQLVPVIRUWKH
WKHDFDGHPLFFRPPXQLW\SULPDU\DQGVHFRQGDU\ LPSOHPHQWDWLRQRIPHDVXUHVLQWKHILHOGRIGLJLWDO
VFKRROVSXEOLFDGPLQLVWUDWLRQDQGVHOIJRYHUQPHQW WUDQVIRUPDWLRQRIHGXFDWLRQ WKHVXEDFWLYLW\ZLOOEH
ERGLHVRWKHUVXEMHFWVDQGFLYLOVRFLHW\LQWKHILHOGRI LPSOHPHQWHGE\83-â&81,9D0 
XVLQJDQGH[FKDQJLQJWKHH[SHULHQFHRIH[SHUWVIURP 7KHJRDORIWKH VXEDFWLYLW\LVWRFUHDWHD V\VWHP IRU
SUDFWLFHLQWKHILHOGRIGLJLWDOWUDQVIRUPDWLRQRI REWDLQLQJ DQG SURFHVVLQJ GDWD RQ WKH SURJUHVV RI WKH
HGXFDWLRQ WKHVXEDFWLYLW\ZLOOEHFDUULHGRXW1,9D0 GLJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ DQG EDVHG RQ WKLV
83-â&8  GDWDWRSUHSDUHRXWSXWVIRUPDQDJLQJDQGVXSSRUWLQJWKH
7KH JRDO RI WKH VXEDFWLYLW\ LV WR FUHDWH DQ HIIHFWLYH WUDQVIRUPDWLRQ /LNHZLVH WKH JRDO LV V\VWHPDWLF
HFRV\VWHP WR VXSSRUW WKH GLJLWDO WUDQVIRUPDWLRQ RI PRQLWRULQJRIWKHLPSDFWVRIWKHSURMHFWGXULQJLWVFRXUVH
HGXFDWLRQ ZKLFK ZLOO UHVSHFW WKH XQLTXHQHVV RI HDFK ZKLFK OHDGV WR KLJKHU HIILFLHQF\ DQG TXDOLW\ RI WKH
VFKRRO DQG FUHDWH D V\QHUJLVWLF HIIHFW LQ LWV H[WHUQDO DFWLYLWLHVRIIHUHG
VXSSRUW 7KH VXEDFWLYLW\ ZLOO SULPDULO\ LQFOXGH WKH IROORZLQJ
7KH VXEDFWLYLW\ ZLOO SULPDULO\ LQFOXGH WKH IROORZLQJ DFWLYLWLHV
DFWLYLWLHV  FUHDWLRQ RI QHZ WRROV DQG DGDSWDWLRQ RI H[LVWLQJ
 LPSOHPHQWDWLRQ RI FOXEVFRPPXQLWLHV RI WHDFKLQJ WRROV IRU PHDVXULQJ WKH OHYHO RI GLJLWDO
VWDIIDQGSURIHVVLRQDOVWDII WUDQVIRUPDWLRQ
 VXSSRUW RI ORFDO UHJLRQDO  QHWZRUNLQJ RI  GDWDFROOHFWLRQDQGDQDO\VLV
SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV DQG  DQDO\VLV RI WKH LPSOHPHQWDWLRQ RI WKH
VFKRROV ZLWK WKH VXSSRUW SDUWLFLSDWLRQ  RI RWKHU WUDQVIRUPDWLRQ DW VSHFLILF VFKRROV ± UHJXODU
VXEMHFWV VFUHHQLQJRILPSRUWDQWSDUDPHWHUVRIGLJLWDOL]DWLRQ
 VXSSRUW IRU ORFDO QHWZRUNLQJ RI VFKRRO GLJLWDO RIVFKRROV
FRRUGLQDWRUV DQG VFKRRO SULQFLSDOV ZLOO QRW EH  UHFRPPHQGDWLRQV IRU SUDFWLFH VFKRROV IRXQGHUV
LPSOHPHQWHGE\1,9D0  PLQLVWU\ 
 SUHSDUDWLRQ DQG LPSOHPHQWDWLRQ RI RSHQ FODVVHV   FUHDWLQJ D YLVLRQ RI 67($0 HGXFDWLRQ LQ
H[DPSOHV RI JRRG SUDFWLFH LQ WKH GLJLWDO 6ORYDNLD
WUDQVIRUPDWLRQRIHGXFDWLRQ  6(/),( IRU SURIHVVLRQDO GHYHORSPHQW WHDFKHUV
,Q WKLV DFWLYLW\ ZH ZLOO DOVR FRRSHUDWH ZLWK WKH >@ ± GDWD FROOHFWLRQ DQG DQDO\VLV DQDO\VLV RI
5HJLRQDO 7HDFKHU 6XSSRUW &HQWHUV IRU SULPDU\ VFKRROV WUDQVIRUPDWLRQ LPSOHPHQWDWLRQ LQ 1,9D0 9LVLRQ
5&38V  5&38 DFWLYLWLHV DUH SULPDULO\ IRFXVHG RQ RISURIHVVLRQDOFRQVXOWLQJLQGLJLWDOWUDQVIRUPDWLRQ
PHQWRULQJ ± VXSSRUWLQJ LQGLYLGXDOV LQ  DUHDV RQO\ DW ±LQWHUQDODQDO\VLVRIGLJLWDOWUDQVIRUPDWLRQ
SULPDU\ VFKRROV >@ 1HWZRUNLQJ ZLWKLQ WKH 13 ZLOO 7KH SURMHFW ZLOO FRYHU WKH UHJXODU VFUHHQLQJ RI
V\VWHPDWLFDOO\FURVVWKHERXQGDULHVRIRQHW\SHRIVFKRRO LPSRUWDQW SDUDPHWHUV RI GLJLWDOL]DWLRQ RI VFKRROV $
DQG DOVR JR EH\RQG WKH ERXQGDULHV RI VFKRRO VLJQLILFDQW VRXUFH RI GDWD IRU WKLV DUHD ZLOO SULPDULO\ EH
RUJDQL]DWLRQV 6LPLODU WR HGXFDWLRQ ZH ZLOO HPSKDVL]H WKH (XURSHDQ WRROV IRU VHOIHYDOXDWLRQ RI VFKRROV
WKH VXEMHFW GLPHQVLRQ LQ QHWZRUNLQJ XQOLNH 5&38  SHGDJRJLFDODQGSURIHVVLRQDOHPSOR\HHVQDPHO\6(/),(
7KXV QHWZRUNLQJ DLPHG DW GLJLWDO WUDQVIRUPDWLRQ GRHV >@ 6(/),( IRU 7($&+(56 >@ DQG 7(76$7 >@
QRW RYHUODS ZLWK 5&38 DFWLYLWLHV EXW FRPSOHPHQWV DQG 6HOHFWLRQ UHVSHFWLYHO\ WKH FUHDWLRQ RI RWKHU WRROV ZLOO
H[SDQGV WKHP 7KLV ZLOO FUHDWH D V\QHUJLVWLF HIIHFW RI UHIOHFW WKH FXUUHQW QHHGV RI WKH GLJLWDO WUDQVIRUPDWLRQ RI
VXSSRUWHGDFWLYLWLHV,QDGGLWLRQWKLVQHWZRUNLQJZLOODOVR HGXFDWLRQ LQ 6ORYDNLD 7KH SURFHVV RI GLJLWDO
VHUYHWREXLOGZLGHUDZDUHQHVVRIUHVHDUFKHQDEOHEHWWHU WUDQVIRUPDWLRQ RI HGXFDWLRQ QHHGV WR EH PRQLWRUHG
GDWD FROOHFWLRQ DQG WKH LQYROYHPHQW RI D ZLGHU UDQJH RI V\VWHPDWLFDOO\ ORQJWHUP DQG LQ D UHDVRQDEO\ ODUJH
VFKRROV ZKLFK LV DQ LPSRUWDQW DVSHFW IURP WKH SRLQW RI QXPEHURIVFKRROV$Q\DFWLYLW\DLPHGLQWKLVGLUHFWLRQLQ
YLHZRIGHVLJQEDVHGUHVHDUFK WKH SDVW PD\ EH D VRXUFH RI NQRZOHGJH IRU WKH QH[W
7KHVXEDFWLYLW\ XQGHUWKHOHDGHUVKLSRIWKH PDQDJHUV SHULRGEXWLWZLOOFHUWDLQO\QRWOHDGWRGXSOLFDWLRQ
ZLOO EH FDUULHG RXW SULPDULO\ E\ H[SHUWV IRU FRQWHQW 83-âDQG&8ZLOOEHHTXDOO\UHVSRQVLEOHIRUWKHVXE
FUHDWLRQ UHVHDUFK DQG HGXFDWLRQ RI SHGDJRJLFDO DQG DFWLYLW\ 1,9D0 UHDOL]HV 6(/),( IRU SURIHVVLRQDO
SURIHVVLRQDO HPSOR\HHV SURIHVVLRQDO HPSOR\HHV RI GHYHORSPHQW WHDFKHUV 7KH VXEDFWLYLW\ XQGHU WKH
1,9D0 OHDGHUVKLS RI WKH PDQDJHUV ZLOO EH FDUULHG RXW SULPDULO\
([SHUWV IURP RWKHU XQLYHUVLWLHV SULPDU\ VFKRROV E\H[SHUWVIRUFRQWHQWFUHDWLRQUHVHDUFKDQGHGXFDWLRQRI
VHFRQGDU\ VFKRROV DQG RWKHU H[SHUWV IURP WKH 'LJLWDO SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV SURIHVVLRQDO
&RDOLWLRQ HQYLURQPHQW ZLOO DOVR EH LQYROYHG LQ WKH VXE HPSOR\HHVRI1,9D0
DFWLYLW\ ,IQHFHVVDU\H[SHUWVIURPRWKHU+(,VSHGDJRJLFDODQG
2XWSXWRIVXEDFWLYLW\'LQFOXGHV SURIHVVLRQDOHPSOR\HHVIURPSULPDU\VFKRROVVHFRQGDU\
 UHJLRQDOQHWZRUNVRI6'&V VFKRROV H[SHUWV IURP WKH 'LJLWDO &RDOLWLRQ DQG RWKHU
 PLQRQOLQHPHHWLQJVDQGPLQIDFHWRIDFH DFWRUV LQ WKH ILHOG RI HGXFDWLRQ ZLOO EH LQYROYHG LQ WKH
PHHWLQJVZLWKLQUHJLRQDOQHWZRUNV VXEDFWLYLW\
2XWSXWRIVXEDFWLYLW\(LQFOXGHV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 514

  PLQVWXGLHVRQWKHLPSOHPHQWDWLRQRIWKHGLJLWDO SULPDULO\DQLQFUHDVHLQWKHTXDOLW\RILQWHUQDWLRQDO
WUDQVIRUPDWLRQRIHGXFDWLRQ FRRSHUDWLRQ $W WKLV PRPHQW LW LV QRW SRVVLEOH WR
 YLVLRQRI67($0HGXFDWLRQ LGHQWLI\ WKH H[DFW FRQIHUHQFHV DW ZKLFK ZH ZLOO
SUHVHQWWKHUHVHDUFKUHVXOWV
) ,PSOHPHQWDWLRQDQGVXSSRUWRIHGXFDWLRQDOUHVHDUFK ) 6FLHQWLILF VHPLQDU RSHQ WR WKH ZLGHU VFLHQWLILF
LQWKHILHOGRIGLJLWDOWUDQVIRUPDWLRQRIHGXFDWLRQ WKH FRPPXQLW\
VXEDFWLYLW\ZLOOEHLPSOHPHQWHGE\83-â&8   83-â ZLOO EH WKH RUJDQL]HU RI WKH VFLHQWLILF
7KLV VXEDFWLYLW\ ZLOO VHUYH DV D UHVHDUFK URRI RYHU VHPLQDU ,WV SURIHVVLRQDO JXDUDQWRU ZLOO EH DQ
VXEDFWLYLWLHV $ ± ' ,WV JRDO LV WR JDLQ D GHHS H[SHULHQFHG UHVHDUFKHU LQ WKH ILHOG RI HGXFDWLRQ
XQGHUVWDQGLQJRIWKHVRFDOOHGH[DPSOHVRIJRRGSUDFWLFH ZLWK IRUHLJQ ZRUN H[SHULHQFH ,WVSDUWLFLSDQWV ZLOO
PDLQO\ VFKRRO GLJLWDO FRRUGLQDWRUV SHGDJRJLFDO DQG SULPDULO\ EH WKRVH PHPEHUVRI WKH WHDP ZKR DOVR
KDYHDVFLHQWLILFFRQWHQWRIWKHLUZRUN7KHLUDFWLYH
SURIHVVLRQDO HPSOR\HHV VFKRROV VFKRRO SULQFLSDOV  LQ
SDUWLFLSDWLRQLQWKHVHPLQDUZLOOEHDFRQGLWLRQIRU
RUGHU WR PDNH WKHLU H[WHQVLRQ WR RWKHU VFKRROV PRUH VXSSRUWLQJ WKHLU SDUWLFLSDWLRQ LQ WKH FRQIHUHQFHV
HIILFLHQW,WLVHTXDOO\LPSRUWDQWWRHQVXUHWKHVDIHW\RIWKH 2WKHU PHPEHUV RI WKH WHDP ZLOO EH LQYLWHG WR WKH
GLJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ IRU LWV DFWRUV E\ VHPLQDU DV ZHOO DV DQ\RQH IURP WKH VFLHQWLILF
PLQLPL]LQJ SDWKRORJLFDO SKHQRPHQD 2EWDLQLQJ UHOHYDQW FRPPXQLW\LQ6ORYDNLDZKRLVHQJDJHGLQUHVHDUFK
GDWD LQIRUPDWLRQ DQG NQRZOHGJH LQ FROODERUDWLRQ ZLWK LQWKHILHOGRIHGXFDWLRQ
IRUHLJQSDUWQHUVZLOOEHDXVHIXOVRXUFHRINQRZOHGJHIRU  7KH VHPLQDU ZLOO KDYH UHJXODU PRQWKO\ PHHWLQJV
RWKHU (XURSHDQ FRXQWULHV DQG DOVR IRU LQGLYLGXDO K\EULGIRUP DQGWZLFHD\HDUDIXOOGD\ZRUNVKRS
HGXFDWLRQDO DFWLYLWLHV 7KH JRDO RI WKH UHVHDUFK ZLOO DOVR IDFHWRIDFH 
EHWKHV\VWHPDWLFLPSURYHPHQWRIVXSSRUWTXDOLW\VXSSRUW 7KLV VXEDFWLYLW\ ZLOO EH FDUULHG RXW LQ WKH FRQWH[W RI
IRU SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV LQ SUDFWLFH RWKHU VXEDFWLYLWLHV IURP ZKLFK LW ZLOO GUDZ GDWD DQG
DQGIRUXQLYHUVLW\WHDFKHUVSUHSDULQJIXWXUHWHDFKHUV FRYHU WKHP LQ UHVHDUFK RU RQ WKH FRQWUDU\ ZLOO EH D
7KH VXEDFWLYLW\ ZLOO SULPDULO\ LQFOXGH WKH IROORZLQJ VRXUFHRILQIRUPDWLRQIRUWKHP WRHQVXUHWKHFRQQHFWLRQ
DFWLYLWLHV RIWKHRU\DQGSUDFWLFH 
) )RUPXODWLRQ RI UHVHDUFK WDVNV DQG WKHLU JUDGXDO 83-âDQG&8ZLOOEHHTXDOO\UHVSRQVLEOHIRUWKHVXE
VROXWLRQ SULPDULO\ WKURXJK WKH GHVLJQ EDVHG UHVHDUFK DFWLYLW\ 7KH VXEDFWLYLW\ XQGHU WKH OHDGHUVKLS RI WKH
PHWKRGRORJ\ZKLFKSURYHVWREHYHU\HIIHFWLYHLQOLQNLQJ
JXDUDQWRU DQG PDQDJHUV ZLOO EH LPSOHPHQWHG SULPDULO\
WKHRU\DQGSUDFWLFH7KHDUHDVRIUHVHDUFKWRSLFVWKDWZH
FXUUHQWO\VHHDVUHOHYDQWDUHDVIROORZV E\H[SHUWVIRUFRQWHQWFUHDWLRQUHVHDUFKDQGHGXFDWLRQRI
SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV H[SHUWV IRU
 'HYHORSLQJ NQRZOHGJH RI 3= DQG 2= DQG WKHLU
HGXFDWLRQDQGVXSSRUWRI6'&V
GLJLWDO FRPSHWHQFLHV 'LJ&RPS >@ DQG
'LJ&RPS(GX>@  7KH VXEDFWLYLW\ ZLOO EH RSHQ WR RWKHU H[SHUWV IURP
RWKHU XQLYHUVLWLHV DQG LQVWLWXWLRQV ZKRVH VXEMHFW LV DOVR
 7KH TXDOLW\ RI WKH XVH RI GLJLWDO WHFKQRORJLHV
VFLHQWLILFUHVHDUFKLQFOXGLQJ1,9D0
WRROV  DQG RWKHU LQQRYDWLYH HOHPHQWV LQ WKH
WHDFKLQJ RI JHQHUDO HGXFDWLRQ VXEMHFWV DW SULPDU\ 2XWSXWRIVXEDFWLYLW\)LQFOXGHV
VFKRROV ZLWK UHJDUG WR WKH QHZ FXUULFXOXP  DQG  PLQVFLHQWLILFRXWSXWV
VHFRQGDU\VFKRROVLQYDULRXVFRQWH[WV HJFULWLFDO  VFLHQWLILFVHPLQDU K\EULGPHHWLQJVRUJDQL]HGRQD
PDWKHPDWLFDO ,7 WKLQNLQJ IRUPDWLYH DVVHVVPHQW PRQWKO\ EDVLV IXOOGD\ IDFHWRIDFH ZRUNVKRSV
UHVHDUFKVNLOOVLQFOXVLYHHGXFDWLRQ  RUJDQL]HGRQDKDOI\HDUO\EDVLV 
 /HDGHUVKLS FRPSHWHQFLHV RI WKH VFKRRO RI GLJLWDO
* 3UHSDUDWLRQLPSOHPHQWDWLRQDQGFHUWLILFDWLRQRIWKH
H[FHOOHQFH DQG WKH UROH RI WKH VFKRRO GLJLWDO
FRRUGLQDWRU TXDOLW\RIHGXFDWLRQDOFRQWHQWSHGDJRJLFDO
SURFHGXUHVDQGVFKRROVRIGLJLWDOH[FHOOHQFH WKHVXE
 &\EHUVHFXULW\
DFWLYLW\ZLOOEHLPSOHPHQWHGE\83-â&8 
 3RVVLELOLWLHVDQGULVNVRIXVLQJDUWLILFLDOLQWHOOLJHQFH
DQGGDWDLQHGXFDWLRQ 7KH JRDO RI WKH VXEDFWLYLW\ LV WR FUHDWH DQ H[WHUQDO
DQGREMHFWLYHWRROWKDWZLOOYDOLGDWHWKHVFKRRO VRZQVHOI
 3UHYHQWLRQ RI SDWKRORJLFDO SKHQRPHQD DVVRFLDWHG
ZLWK H[FHVVLYH XVH RI GLJLWDO WHFKQRORJLHV DQG HYDOXDWLRQ HJ 6(/),(  DQG LI WKH FULWHULD DUH PHW LW
GLJLWDO ZHOOEHLQJ ZLOO UHFHLYH D VFKRRO RI GLJLWDO H[FHOOHQFH FHUWLILFDWH
 8VH RI GLJLWDO WHFKQRORJLHV WR HQVXUH HTXDO DFFHVV 6XEVHTXHQWO\ XVLQJ WKLV WRRO WR LPSOHPHQW WKH VRFDOOHG
WRHGXFDWLRQ GLJLWDODXGLWDWVFKRROVWKDWUHTXHVWLW
7KH VXEDFWLYLW\ ZLOO SULPDULO\ LQFOXGH WKH IROORZLQJ
),QYROYHPHQWLQLQWHUQDWLRQDOFROODERUDWLRQV
DFWLYLWLHV
 ,QWKHORQJWHUPWKH\ZLOOOHDGWRWKHLPSURYHPHQW * ,GHQWLILFDWLRQ RI FDWHJRULHV FULWHULD DQG WKH VR
RI WKH GLJLWDO WUDQVIRUPDWLRQ LQ 6ORYDNLD DQG LQ FDOOHGSURFHVV'LJLWDODXGLW
RWKHUFRXQWULHV  7KH FDWHJRULHV DQG FULWHULD ZLOO ODUJHO\ EH RQ WKH
)  $FWLYH SDUWLFLSDWLRQ LQ LQWHUQDWLRQDO DQG ORFDO VFKRRO RI GLJLWDO H[FHOOHQFH UXEULFV ZKLFK
VFLHQWLILFFRQIHUHQFHV
VFKRROGLJLWDOFRRUGLQDWRUVQRUPDOO\ZRUNZLWK
 (DFKRIWKHVROYHUVZKRZLOODOVRKDYHDVFLHQWLILF  7KHSURFHVVZLOOEHSLORWYHULILHGLQWKHILUVWSKDVH
SDUWRIWKHZRUNLQWKHWHDPZLOOEHDSDUWLFLSDQWLQ DW VFKRROV ZLWK D KLJK OHYHO LQ WKH ILHOG RI
 FRQIHUHQFHV D \HDU 6SHFLILFDOO\ VFLHQWLILF
GLJLWL]DWLRQ
FRQIHUHQFHV ZLOO EH VHOHFWHG VR WKDW WKHLU UHVXOW LV

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 515

 * ,PSOHPHQWDWLRQ RI D GLJLWDO DXGLW WR REWDLQ WKH  :HSODQWRLQYROYHVFKRROVLQWKHH[SHULPHQWDO
VFKRRORIGLJLWDOH[FHOOHQFHFHUWLILFDWH YHULILFDWLRQ RQHFODVVLQHDFKJ\PQDVLXP 
 7KH GLJLWDO DXGLW ZLOO WDNH SODFH DW WKH UHTXHVW RI + 6XSSRUW IRU WKH LPSOHPHQWDWLRQ RI WKH QHZ VWXG\
WKH VFKRRO DW WKH PRPHQW ZKHQ IRU H[DPSOH RQ ILHOG*\PQDVLXPZLWKDIRFXVRQFRPSXWHUVFLHQFH
WKH EDVLV RI VHOIHYDOXDWLRQ XVLQJ 6(/),( EH  7KHVXSSRUWZLOOSULPDULO\LQFOXGHWKHXSGDWLQJRI
SHUFHLYHGDVDVFKRRORIGLJLWDOH[FHOOHQFH  ,7 PHWKRGRORJLFDO PDWHULDOV WKDW ZHUH FUHDWHG
 7KLV DFWLYLW\ ZLOO WDNH SODFH FRQWLQXRXVO\ IRUWKHVHSXUSRVHVZLWKLQWKH13,7$FDGHP\WKH
WKURXJKRXWWKHGXUDWLRQRIWKHSURMHFWDOWKRXJKZH FUHDWLRQ RI RWKHU VXSSRUWLQJ PDWHULDOV HGXFDWLRQ
H[SHFW LQFUHDVHG LQWHUHVW LQ WKLV DXGLW DQG DQG PHWKRGLFDO VXSSRUW RI SHGDJRJLFDO DQG
FHUWLILFDWH DV WKH SURMHFW FORVHV 6FKRROV LQ SURIHVVLRQDOHPSOR\HHV
6ORYDNLD DUH DW GLIIHUHQW OHYHOV IURP WKH SRLQW RI  3DUW RI WKH VXSSRUW LV DOVR ILQDQFLDO VXSSRUW IRU
YLHZRIGLJLWDOWUDQVIRUPDWLRQDQGWKHUHIRUHWKHLU H[SHULPHQWDO YHULILFDWLRQ WKH VRFDOOHG GLJLWDO
UHDGLQHVVIRUDGLJLWDODXGLWZLOOFRPHJUDGXDOO\ VWDQGDUG 
 6FKRROVZLOOQRWLQFXUDQ\H[SHQVHVLQFRQQHFWLRQ 83-âZLOOEHUHVSRQVLEOHIRUWKHVXEDFWLYLW\7KHVXE
ZLWK WKH DXGLW ([SHQVHV DVVRFLDWHG ZLWK WKH DFWLYLW\ XQGHU WKH JXLGDQFH RI WKH JXDUDQWRU DQG
WUDQVIRUPDWLRQSURFHVVDUHQRWSDUWRI13VXSSRUW PDQDJHUV ZLOO EH LPSOHPHQWHG SULPDULO\ E\ H[SHUWV IRU
,WZLOOEHDQHZQDWLRQDOFHUWLILFDWHZKLFKZHZLOO FRQWHQW FUHDWLRQ UHVHDUFK DQG HGXFDWLRQ RI SHGDJRJLFDO
FRRUGLQDWH LQ OLQH ZLWK (XURSHDQ GRFXPHQWV DQG DQGSURIHVVLRQDOHPSOR\HHV2WKHUH[SHUWV IURPWKH&8
FRQVXOWZLWKIRUHLJQSDUWQHUV RWKHU XQLYHUVLWLHV DQG IURP WKH 'LJLWDO &RDOLWLRQ
83-âDQG&8ZLOOEHHTXDOO\UHVSRQVLEOHIRUWKHVXE HQYLURQPHQWZLOOEHLQYROYHGLQWKHVXSSRUW
DFWLYLW\ 7KH VXEDFWLYLW\ XQGHU WKH OHDGHUVKLS RI WKH 2XWSXWRIVXEDFWLYLW\+LQFOXGHV
JXDUDQWRU DQG PDQDJHUV ZLOO EH LPSOHPHQWHG SULPDULO\  6(3 DQG WHDFKLQJ PDWHULDOV IRU WKH QHZ ILHOG RI
E\H[SHUWVIRUFRQWHQWFUHDWLRQUHVHDUFKDQGHGXFDWLRQRI VWXG\
SHGDJRJLFDO DQG SURIHVVLRQDO HPSOR\HHV H[SHUWV IRU  PLQJUDGXDWHVRIWKHILHOGRIVWXG\
HGXFDWLRQDQGVXSSRUWIRU6'&V
2XWSXWRIVXEDFWLYLW\*LQFOXGHV , 3RSXODUL]DWLRQDFWLYLWLHVDQGVXSSRUWRIVRFLDO
 PLQ  SULPDU\ DQG VHFRQGDU\ VFKRROV WKDW DZDUHQHVVRIWKHQHHGVRIGLJLWDOWUDQVIRUPDWLRQRI
XQGHUJRDGLJLWDODXGLW HGXFDWLRQ WKHVXEDFWLYLW\ZLOOEHLPSOHPHQWHGE\
83-â&81,9D0 
+ ([SHULPHQWDOYHULILFDWLRQRIWKHQHZVWXG\ILHOG 7KH JRDO RI WKH VXEDFWLYLW\ LV WR FRQWULEXWH WR WKH
*\PQDVLXPZLWKDIRFXVRQLQIRUPDWLFV WKHVXE FUHDWLRQ RI VRFLDO DZDUHQHVV RI WKH QHHG JRDOV DQG
DFWLYLW\ZLOOEHLPSOHPHQWHGE\83-â&8  LPSOHPHQWDWLRQRIWKHGLJLWDOWUDQVIRUPDWLRQRIHGXFDWLRQ
7KH DLP RI WKH VXEDFWLYLW\ LV WR LPSURYH WKH TXDOLW\ LQUHODWLRQWRWKHQHHGVRIDGLJLWDOVRFLHW\
RI VHFRQGDU\ VFKRRO SUHSDUDWLRQ IRU XQLYHUVLW\ VWXGLHV LQ 7KH VXEDFWLYLW\ ZLOO SULPDULO\ LQFOXGH WKH IROORZLQJ
ILHOGV RI VWXG\ WKH JUDGXDWHV RI ZKLFK ZLOO EH SUHSDUHG DFWLYLWLHV
IRU H[SHUW SRVLWLRQV ZLWKLQ WKH GLJLWDO WUDQVIRUPDWLRQ RI  FRPPXQLFDWLRQ ZLWK WKH SXEOLF UHJDUGLQJ SURMHFW
VRFLHW\OLQNHGWRWKHRXWSXWVRIWKH13,7$FDGHP\)RU DFWLYLWLHV DQG WKH GLJLWDO WUDQVIRUPDWLRQ RI
WKLV WKH PHWKRGRORJLFDO PDWHULDOV FUHDWHG ZLWKLQ WKH 13 HGXFDWLRQ
,7 $FDGHP\ ZLOO EH XSGDWHG DQG DQ H[SHULPHQW ZLOO EH  67($0FDPSVIRUSULPDU\DQGVHFRQGDU\VFKRRO
FDUULHGRXW SXSLOV GD\UHVLGHQWLDO 
2QH RI WKH PDLQ RXWSXWV RI 13 ,7 $FDGHP\ LV WKH  FRPSHWLWLRQV DQG FRUUHVSRQGHQFH VHPLQDUV
SURSRVDORIDQHZVWXG\ILHOG*\PQDVLXPZLWKDIRFXVRQ ZRUNVKRSV HJ3$/0$ 
LQIRUPDWLFV 7KLV SURSRVDO UHIOHFWV WKH FXUUHQW QHHGV RI  ,7 WHD URRPV KDFNDWKRQV IRU SULPDU\ DQG
WKHGLJLWDOWUDQVIRUPDWLRQRI VRFLHW\ WKHWUDLQLQJRIWRS VHFRQGDU\VFKRROVWXGHQWVDQGVLPLODU
H[SHUWVZLWKQGDQGUGGHJUHHXQLYHUVLW\HGXFDWLRQIRU ,W LV QHFHVVDU\ WR FRPPXQLFDWH WKH GLJLWDO
WKH,7VHFWRULQWKHILHOGRI$UWLILFLDO,QWHOOLJHQFH,QWHUQHW WUDQVIRUPDWLRQRIHGXFDWLRQZLWKHGXFDWLRQDFWRUVDQGWKH
RI7KLQJV&\EHU6HFXULW\DQGRWKHUV7KHFXUUHQWSURILOH JHQHUDO SXEOLF )RU DFWRUV RI HGXFDWLRQ WKH IRUPDWV RI
RI D KLJK VFKRRO JUDGXDWH GRHV QRW PHHW WKH QHHGV IRU DFWLYLWLHVWKDWKDYHSURYHQWKHPVHOYHVLQWKH,7$FDGHP\
XQLYHUVLW\ WUDLQLQJ LQ WKH JLYHQ ILHOGV ,W LV DVVXPHG WKDW DQG 3URIHVVLRQDO 'HYHORSPHQW RI 7HDFKHUV
LQWKHSURMHFW DVDSLORW WKLVGHSDUWPHQWZLOOEHRSHQHG 7($&+(56  QDWLRQDO SURMHFWV ZLOO EH VXSSRUWHG HJ
IRUVFKRROV7KHVXEDFWLYLW\ZLOOLQFOXGH WHDFKHUIRUXPVGD\DQGUHVLGHQWLDOFDPSVFRPSHWLWLRQV
+ 6HOHFWLRQ RI J\PQDVLXP IRU H[SHULPHQWDO FRUUHVSRQGHQFH VHPLQDUV ,7 WHD URRPV  EXW DOVR QHZ
YHULILFDWLRQ RQHV HJ KDFNDWKRQV IRU SULPDU\ DQG VHFRQGDU\ VFKRRO
 *\PQDVLXP WKDW KDYH YHULILHG WKH PRGHO RI VWXGHQWV HVSHFLDOO\WKRVH ZKRVKRZKLJKHUWDOHQWLQWKLV
LQIRUPDWLFVFODVVXQGHUWKHFRQGLWLRQVRIVFKRRO DUHD 
HGXFDWLRQDO SURJUDP ZLWKLQ 13 ,7 $FDGHP\  7KH VXEDFWLYLW\ XQGHU WKH JXLGDQFH RI WKH JXDUDQWRU
LQ WRWDO  ZLOO EH SULRULWL]HG IRU H[SHULPHQWDO DQGPDQDJHUVZLOOEHLPSOHPHQWHGE\H[SHUWVIRUFRQWHQW
YHULILFDWLRQ ,Q HDFK UHJLRQ WKH VHOHFWLRQ ZLOO EH FUHDWLRQ UHVHDUFK DQG HGXFDWLRQ RI SHGDJRJLFDO DQG
FRQVXOWHGZLWKWKHIRXQGHUVVRWKDWWKHRSHQLQJRI SURIHVVLRQDO HPSOR\HHV H[SHUWV IRU HGXFDWLRQ DQG
D QHZ ILHOG RI VWXG\ LV LQ DFFRUGDQFH ZLWK WKH VXSSRUWIRU6'&VSURIHVVLRQDOHPSOR\HHVRI1,9D0
VFKRROSROLF\LQWKHJLYHQUHJLRQ

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 516

 ,I QHFHVVDU\ H[SHUWV IURP RWKHU XQLYHUVLWLHV GLUHFWO\ $&.12:/('*0(17
PDQDJHG E\ WKH RUJDQL]DWLRQ 0â99Dâ DQG IURP WKH 7KLV ZRUN ZDV VXSSRUWHG E\ WKH /LIHORQJ /HDUQLQJ
HQYLURQPHQWRIWKH'LJLWDO&RDOLWLRQZLOODOVREHLQYROYHG &HQWUH  3URMHFWV 6XSSRUW DW 3DYRO -R]HI âDIiULN
LQWKHVXEDFWLYLW\ 8QLYHUVLW\LQ.RãLFH
2XWSXWRIVXEDFWLYLW\,LQFOXGHV
 PLQ67($0FDPSV 5()(5(1&(6
 PLQFRUUHVSRQGHQFHVHPLQDUVDQGJDWKHULQJV >@ 7KH 3URJUDPPH IRU WKH ,QIRUPDWL]DWLRQ RI (GXFDWLRQ XQWLO 
 PLQKDFNDWKRQIRUVHFRQGDU\VFKRROVWXGHQWV $FFHVVHG  6HSWHPEHU  5HWULHYHG IURP
KWWSVZZZPLQHGXVNGDWDDWWSGI
>@ 5HGHFNHU &K (XURSHDQ )UDPHZRUN IRU WKH 'LJLWDO &RPSHWHQFH
9 &21&/86,21 RI(GXFDWRUV'LJ&RPS(GX3XQLH< HG -RLQW5HVHDUFK&HQWUH
'LJLWL]DWLRQ RI VRFLHW\ UHTXLUHV GLJLWL]DWLRQ RI  ,6%1  GRL
-5&
HGXFDWLRQ 'XH WR WKH ZLGH SRVVLELOLWLHV DQG WKH ULVNV RI
>@ 9XRULNDUL5.OX]HU6DQG3XQLH<'LJ&RPS7KH'LJLWDO
WKHGLJLWDOZRUOGLWUHTXLUHVSURIHVVLRQDOVXSSRUW6FKRROV &RPSHWHQFH )UDPHZRUN IRU &LWL]HQV (85  (1
FDQ JHW VXSSRUW IURP PDQ\ VXEMHFWV IURP WKH DFDGHPLF 3XEOLFDWLRQV 2IILFH RI WKH (XURSHDQ 8QLRQ /X[HPERXUJ 
DQG EXVLQHVV VHFWRUV DV ZHOO DV LQVSLULQJ VFKRROV $Q ,6%1GRL-5&
HIIHFWLYH DQG PHDQLQJIXO GLJLWDO WUDQVIRUPDWLRQ RI >@ )WiþQLN 0 âYHGD ' .LUHã 0 'LJLWDO WUDQVIRUPDWLRQ RI
VFKRROV UHTXLUHV D ZHOOWKRXJKWRXW VWUDWHJ\ DW WKH HGXFDWLRQLQ6ORYDNLDZLWKLQWKHFRQWH[WRI(XURSHDQGRFXPHQWV
WK ,QWHUQDWLRQDO &RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ
QDWLRQDO OHYHO PDQDJHG FHQWUDOO\ 7KH QDWLRQDO SURMHFW 7HFKQRORJLHV DQG $SSOLFDWLRQV ,&(7$  
'LJLWDO 7UDQVIRUPDWLRQ RI (GXFDWLRQ DQG 6FKRROV GRL,&(7$
'L7(GX  UHVSRQGV WR WKLV UHTXHVW E\ HVWDEOLVKLQJ WKH >@ +XEHĖiNRYi 9 âYHGD ' 0LãLDQLNRYi $ .LUHã 0 'LJLWDO
1DWLRQDO &HQWHU IRU'LJLWDO7UDQVIRUPDWLRQRI(GXFDWLRQ WUDQVIRUPDWLRQ RI HGXFDWLRQ DW VFKRRO WK ,QWHUQDWLRQDO
&RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ 7HFKQRORJLHV DQG
1&'79  ZLWK WKH DFWLYLW\ ³VXSSRUW IRU WKH GLJLWDO $SSOLFDWLRQV ,&(7$   GRL
WUDQVIRUPDWLRQRIHGXFDWLRQ´ ,&(7$
7KH H[SHFWHG FRQWULEXWLRQ RI WKH QDWLRQDO SURMHFW >@ 0LãLDQLNRYi $ +XEHĖiNRYi 9 .LUHã 0 %DELQþiNRYi 0
'L7(GXLVLQDFFRUGDQFHZLWKWKHVWUDWHJLFPDWHULDOVDQG âYHGD'$VVHVVPHQWRIGLJLWDOL]DWLRQLQSULPDU\DQGVHFRQGDU\
VFKRROV E\ 6(/),( 6XUYH\ DV D SDUW RI VFKRRO OHDGHUV WUDLQLQJ
LQFOXGHV WK ,QWHUQDWLRQDO &RQIHUHQFH RQ (PHUJLQJ H/HDUQLQJ
 LQFUHDVLQJ WKH TXDOLW\ RI HGXFDWLRQ LQ SUHVFKRROV 7HFKQRORJLHV DQG $SSOLFDWLRQV ,&(7$   GRL
SULPDU\VHFRQGDU\DQGKLJKHUVFKRROV ,&(7$
 DFWLRQ SODQV IRU WKH GLJLWDO WUDQVIRUPDWLRQ RI >@ âYHGD ' +XEHĖiNRYi 9 %DELQþiNRYi 0 .LUHã 0
HGXFDWLRQDQGVFKRROV .R]HONRYi . /XNiþRYi / 0LãLDQLNRYi $ 2QGRYi 2
'LJLWDO WUDQVIRUPDWLRQ RI HGXFDWLRQ ZLWK WKH VXSSRUW RI WKH
 HGXFDWLRQDO FRQWHQW DGDSWHG WR WKH QHHGV DQG QDWLRQDO SURMHFW ,7 $FDGHP\ WK ,QWHUQDWLRQDO &RQIHUHQFH RQ
DELOLWLHVRIFKLOGUHQDQGSXSLOV (PHUJLQJ H/HDUQLQJ 7HFKQRORJLHV DQG $SSOLFDWLRQV ,&(7$ 
 XVHRIGLJLWDOHGXFDWLRQDOFRQWHQW GRL,&(7$
 SHUVRQDOL]HG OHDUQLQJ LQ D K\EULG OHDUQLQJ >@ 7KH1DWLRQDO2EVHUYDWRU\IRU'LJLWDO(GXFDWLRQ 12'( 'XEOLQ
&LW\ 8QLYHUVLW\ $FFHVVHG  6HSWHPEHU  5HWULHYHG IURP
HQYLURQPHQW KWWSVZZZGFXLHQLGO
 LQQRYDWLQJ WKH WUDLQLQJ RI IXWXUH WHDFKHUV DQG >@ &HQWUH IRU (GXFDWLRQ DQG 'LJLWDO 7UDQVIRUPDWLRQ =XULFK
LPSURYLQJ WKH SURIHVVLRQDO GHYHORSPHQW RI 8QLYHUVLW\ RI 7HDFKHU (GXFDWLRQ $FFHVVHG  6HSWHPEHU 
WHDFKHUV 5HWULHYHG IURP KWWSVSK]KFKHQ5HVHDUFKDUHDVRI
UHVHDUFKFHQWUHIRUHGXFDWLRQDQGGLJLWDOWUDQVIRUPDWLRQ
 EHWWHU SUHSDUHG JUDGXDWH LQ FRQQHFWLRQ ZLWK WKH
>@ 6ORYDNLDLQWKH'LJLWDO(FRQRP\DQG6RFLHW\,QGH['(6,FRXQWU\
GLJLWDOWUDQVIRUPDWLRQRIWKHFRPSDQ\ SURILOH 6.  $FFHVVHG  6HSWHPEHU  5HWULHYHG IURP
 EHWWHUWUDLQLQJRIIXWXUHDQGFXUUHQWWHDFKHUVLQWKH KWWSVGLJLWDOVWUDWHJ\HFHXURSDHXHQSROLFLHVGHVLVORYDNLD
ILHOGRILQFUHDVLQJGLJLWDOFRPSHWHQFHV >@ 1DWLRQDO 'LJLWDO 6NLOOV 6WUDWHJ\ RI 6ORYDNLD RI WKH 5HSXEOLF DQG
 LPSURYHG SUHSDUDWLRQ RI VWXGHQWV LQ WKH ILHOG RI WKH $FWLRQ 3ODQ IRU WKH \HDUV  ±  $FFHVVHG 
GLJLWDOFRPSHWHQFHVDQGVNLOOGHYHORSPHQWIRUWKH 6HSWHPEHU  5HWULHYHG IURP KWWSVPLUULJRYVNZS
FRQWHQWXSORDGV16'=D$3SGI
VWFHQWXU\
>@ (GXFDWLRQ $FWLRQ 3ODQ  ±   $FFHVVHG  6HSWHPEHU
 SUHSDUDWLRQ RI DFWRUV IRU WKH IXQFWLRQLQJ RI WKH   5HWULHYHG IURP KWWSVHGXFDWLRQHFHXURSDHXIRFXV
GLJLWDO VFKRRO LQFOXGLQJ SUHSDUDWLRQ IRU HQVXULQJ WRSLFVGLJLWDOHGXFDWLRQDFWLRQSODQ
WHDFKLQJHYHQGXULQJDQHPHUJHQF\VLWXDWLRQ >@ ,&'/7KH'LJLWDO6NLOOV6WDQGDUG$FFHVVHG6HSWHPEHU
7KH 1DWLRQDO &HQWHU IRU 'LJLWDO 7UDQVIRUPDWLRQ DQG 5HWULHYHGIURPKWWSVLFGORUJ
(GXFDWLRQZLOOSURYLGHVXSSRUWIRUWKHLPSOHPHQWDWLRQRI >@ 0LãLDQLNRYi $ +XEHĖiNRYi 9 .R]HONRYi . /XNiþRYi .
2QGRYi9%DELQþiNRYi0âYHGD'7KHVXSSRUWIRUWHDFKHUV
PHDVXUHV LQ WKH ILHOG RI GLJLWDO WUDQVIRUPDWLRQ RI DQG VFKRRO OHDGHUV ,Q WKHLU SURIHVVLRQDO JURZWK DQG SHUVRQDO
HGXFDWLRQ LQFOXGLQJ LVVXHV RI F\EHU VHFXULW\ DQG WKH GHYHORSPHQW WK ,QWHUQDWLRQDO &RQIHUHQFH RQ (GXFDWLRQ DQG
SUHYHQWLRQ RI SDWKRORJLFDO SKHQRPHQD DVVRFLDWHG ZLWK 1HZ /HDUQLQJ 7HFKQRORJLHV 
GRLHGXOHDUQ
H[FHVVLYHGLJLWL]DWLRQ
>@ 6(/),( IRU 7($&+(56 6HOIUHIOHFWLRQ WRRO IRU SULPDU\ DQG
7KHVXFFHVVRIWKHGLJLWDOWUDQVIRUPDWLRQRIHGXFDWLRQ VHFRQGDU\ VFKRRO WHDFKHUV $FFHVVHG  6HSWHPEHU 
LQ 6ORYDNLD ZLOO GHSHQG RQ WKH WUDQVIRUPDWLRQ RI 5HWULHYHGIURPKWWSVHGXFDWLRQHFHXURSDHXVHOILHIRUWHDFKHUV
LQGLYLGXDOVFKRROVQHWZRUNLQJRIVFKRROVDQGWKHXVHRI >@ 6(/),( 6HOIUHIOHFWLRQ RQ (IIHFWLYH /HDUQLQJ E\ )RVWHULQJ WKH
DSSURSULDWH VXSSRUW UHJXODU HYDOXDWLRQ RI WKH OHYHO RI 8VH RI ,QQRYDWLYH (GXFDWLRQDO 7HFKQRORJLHV LQWR WHDFKLQJ
OHDUQLQJDQGDVVHVVPHQW$FFHVVHG6HSWHPEHU5HWULHYHG
GLJLWL]DWLRQDQGPRYLQJWRZDUGVVFKRROYLVLRQLQUHODWLRQ IURPKWWSVHFHXURSDHXHGXFDWLRQVFKRROVJRGLJLWDOBHQ
WRDVFKRRORIGLJLWDOH[FHOOHQFH

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 517

 >@ 7(76$70(17(3 VRQOLQHVHOIDVVHVVPHQWWRRODLPLQJWRKHOS 6HSWHPEHU5HWULHYHGIURPKWWSPHQWHSHXQRUJ
WHDFKHUV GHYHORS GLJLWDO SHGDJRJLFDO FRPSHWHQFH $FFHVVHG 



979-8-3503-7069-0/23/$31.00 ©2023 IEEE 518

 On the Distribution of Electrodermal Activity
Properties as a Tool for Teaching Soft Skills
Lukas Tomaszek ∗ , Miroslava Miklošíkovᆠ, Martin Malčík † , Petr Šaloun ∗
∗
Palacky University Olomouc
Křížkovského 511/8, 779 00 Olomouc, Czech Republic
† VSB – Technical University of Ostrava
17. listopadu 15, 708 00 Ostrava - Poruba, Czech Republic
Email: lukas.tomaszek01@upol.cz, miroslava.miklosikova@vsb.cz, martin.malcik@vsb.cz, petr.saloun@upol.cz

Abstract—We live in a world of constant acceleration and in- feelings, understand them and be able to use them to one’s
creasing demand for performance, which puts increased pressure advantage. It is important to be able to identify what emotion I
on the individual. This pressure can cause psychological problems am experiencing, to correctly assess the body’s reaction. What
and in some cases more serious health problems. One way to cope
with the demands of the modern age is to have soft competences has changed? How does it manifest itself? What preceded it?
and to develop and deepen these throughout life. In this paper, we Is it pleasant or unpleasant? What thoughts come to mind
focus on the distribution of selected properties of electrodermal at the same time? This analysis is based on the subjective
activity and the use of the data thus obtained as a biofieback impression of the individual. However, he or she may be
for the individual being measured. The data obtained show that mistaken and misinterpret his or her experience. To look at
the distribution of most of the analyzed properties is from the
log-normal distribution. On the basis of this information we can oneself honestly and, if possible, objectively is not easy and
inform the measured individual about his current emotional state depends on how critical or uncritical the self-observer can be.
and with the help of the biofeedback obtained in this way we For our perception of ourselves is influenced, among other
can enable, facilitate and improve his self-knowledge and self- things, by our desires, our idea of what we would like to be,
regulation. Both skills are part of the list of soft competences. who we consider ourselves to be, who others consider us to
Index Terms—Electrodermal activity, soft skills, biofeedback,
be, what demands we have on ourselves, etc. But there is a
sensetio device. solution to the situation: objectively measured data about the
experiences and emotions of the individual in question.
I. I NTRODUCTION Measuring selected physiological properties, namely skin
conductance (EDA), heart rate, temperature, or brain activity is
We live in a time of constant acceleration, complicated by a way to visualize emotions and feelings to some extent. Given
social, economic, political and other crises. In most fields of a sufficient number of individual measurements, individual
work, employees are required to perform at a consistently high measurements can be compared to determine the current physi-
level, which ultimately means they have to function smoothly ological response of an individual relative to past physiological
in stressful situations. Working under such stressful conditions measurements. For example, it is possible to ask the question:
is demanding enough in itself, and when you add in possible Is this a normal (average) or an extreme response? On the basis
(and frequent) family problems, the enormous strain on the of the findings, objective feedback can be given to the individ-
individual’s psyche is taken care of. ual about his/her current physiological reactions, and he/she
Continuous stresses can result in psychosomatic illnesses can then learn more quickly and qualitatively to recognize
or common or more serious illnesses that should not be some of his/her characteristics, feelings, and situations that
overlooked, suppressed or even ignored as they can lead to lead to the feelings, work with them, and eventually eliminate
chronic or even serious health problems. From the individual’s them partially or completely. The objectification of the self-
point of view, the ability to cope with challenging situations knowledge process leads over time to smooth self-reflection,
and to observe one’s own health and lifestyle objectively are which is the basis of work and personal well-being and leads
soft skills, in the development of which educators at all levels to adequate planning of work and non-work activities.
and types of school are also involved to varying degrees, taking Measurement of physiological properties can also be used
into account the age and individual characteristics of pupils to detect some physical diseases (still latent). As research
and students. from [1], [2] shows, some diseases can be detected before
In this paper we focused on self-knowledge and self- a person physically feels them. Regular monitoring can detect
reflective competence, which are related to the recognition, unusual or even extraordinary changes in the physiological
acceptance, understanding and regulation of one’s own emo- values of an individual’s body and alert them to an impending
tions. This is because improving these skills requires not only health problem (e.g. the onset of Covid disease - 19 [1]). It
observational skills and knowledge of the relevant domain, is then up to the individual to decide how to react to the
but also the ability to correctly name one’s own emotions and information. Early warning signs can help him or her prevent

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 519

more serious health conditions, complications, and more costly have continuous professional growth, the ability to cope with
and prolonged treatment. their own strengths and weaknesses, not only professional but
The measurement of physiological properties of the body also emotional. Those employees who have a well-developed
of an individual, as it follows from the previous text, has potential of their soft competencies are better evaluated by
its legitimate justification. Our research focuses primarily their supervisors, have a better chance of promotion and are
on the conductivity of the skin (EDA). The skin forms the happier on a professional and personal level. All of the above
body’s protective barrier against viruses and batteries, but skills result in self-reflective competence, which is necessary
as research shows [1], [3], its conductivity is also closely for professional and personal growth, for understanding one’s
linked to diseases and emotional processes. Changes in skin own behaviour and for planning the necessary changes in one’s
conductance may therefore signal changes in psychological own experience and behaviour in the future.
processes, including emotional ones. EDA measurements can
III. P SYCHOSOMATICS
be used as indicators of impending or pre-existing illness, as
an objective visualization of hidden experience, and as a tool Data from the measurement of selected physiological prop-
that can be used in developing and expanding soft skills. erties can warn and protect individuals from possible and
The main aim of the research was to record how the obvious psychological difficulties. It is well known that very
skin conductance of the measured individual changes in the severe short-term or prolonged stress can weaken an individual
resting state at different time intervals during the day, to to the extent that he or she subsequently develops health
determine the distribution of the individual skin conductance problems. In this case, too, it can be stated that monitoring
variables analysed and to determine from the data obtained certain physiological variables of the body can be not only
how the individual currently felt in relation to his/her previous an interesting opportunity for the individual, but above all a
measurements. We believe that deviations from normal values kind of warning against illness resulting from long-term stress,
(values close to the median) may indicate discomfort, exces- even long before its visible manifestation. In this context, we
sive psychological stress, illness, or a psychological condition mention the issue of psychosomatics.
that the individual should learn to work with. Values close The term psychosomatics originated in ancient Greek: psy-
to normal (median) seem to indicate normal expressions and che - soul and soma - body. The psychological factors of
experiences of the individual. somatic diseases were dealt with, among other things, by
psychosomatic medicine, which confirmed the relationship be-
II. S OFT SKILLS tween the physical state and the psychological and emotional
Soft skills are important for a fulfilling life, among other components. Later, the social component was added to this
things, to be able to take optimal care of one’s health, to relationship.
recognize possible health problems and to learn from them Psychosomatics is usually characterized as the science of
for one’s future life. Soft skills enable one to understand and the correlation between the psyche, mental state and physical
regulate oneself and to understand other people’s experiences illness [5]. Emotions can have a very strong effect on the
and behaviour. It refers to skills such as flexibility, creativity, body, activating or deactivating it, putting it in tension. The
communication, cooperation, the ability to express praise and connection between body and soul has been viewed differently
criticism, the ability to manage conflict, to create a positive over the centuries. However, already Freud’s modern psycho-
social climate, to recognise, understand and regulate one’s own somatics clearly showed that the psyche is of great importance
and other people’s emotions, to cope with difficult situations, in the origin and development of physical disorders. The field
to use at least some relaxation techniques, etc. All these of psychosomatics can be divided under several headings:
competences help to maintain physical and mental health or • Concentration on the causes of disease. An individual’s
to repair it if it has been damaged for some reason. organic disease always has psychological and social fac-
The authors in [4] state that people whose soft skills are tors as well.
sufficiently developed are considered emotionally intelligent • Focus on the processing of the illness. The individual
and have these important abilities (skills): must also manage psychological and social problems dur-
• they are able to correctly observe and perceive them- ing the course of the illness. Appropriate psychological
selves, steps are needed for this.
• they are motivated, • Focus on illness behaviour. Individuals with psychoso-
• they can empathize with others, matic illness often apply unfavorable forms of interac-
• they have good communication skills. tion to physicians. They need to be prepared for such
The authors also point out that in a time of constant change behaviour.
and in companies with fewer and fewer employees and more • Focus on the accompanying and subsequent psychiatric
pressure to perform and succeed, not only expertise and illness. Many illnesses present with conditions that bur-
intelligence are longer enough, but soft skills are also needed. den individuals psychologically and psychosocially, and
Almost no job can be fully performed without them. they need to be taught to manage these conditions [6].
Soft skills are needed at all levels of the job hierarchy, espe- One of the theories that seeks the connection between
cially for career development. Managers expect employees to psyche and health is based on Pavlov’s doctrine of conditioned

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 520

reflexes. In his view, there are interrelationships between 3) Reaction to the stimulus itself, which is manifested by
higher nervous activity, somatic and autonomic functions of the aforementioned peak.
the organism. In this case, the factors causing illness are usu- In research on stimulus response, we are concerned with so-
ally negative, prolonged and intense emotions. Their negative called specific peaks. These are peaks where we know what
influence manifests itself in the development of disturbances the individual responded to. So we know the stimulus. But
in the normal relations between the cerebral cortex and the we often get to a stage where we don’t know what caused
subcortical centres. Due to these disturbances, complex dis- the peak. At that point, we talk about non-specific peaks, and
eases arise. The endocrine system also interferes with this these are mainly used in other research [12].
relationship. Stress theory deals with the negative effects of
stress on the body. [7]
The issue of psychosomatics is also dealt with by Honzák
[8]. The way an individual handles difficult situations deter-
mines whether he or she takes the path of calm or the path
of stress. The psychosomatic approach implies a consistent
bio-psycho-social approach to the individual, i.e., the medical
doctor should also ask the patient about his/her emotional
and relational difficulties. Indeed, some illnesses are referred
to as "somatising" because emotions are primarily bodily
conditions.
The personality of the individual also plays an important
role in the occurrence of psychosomatic problems, e.g. the
trait neuroticism is associated with greater morbidity, not only
with neurotic problems and depression, but with all physical
problems. As far as stress is concerned, coping with it is a skill
whose training should begin in childhood. It is the training of Fig. 1. A peak capturing the course of an emotion [11]
frustration tolerance, i.e. the ability to deny oneself various
pleasures, to accept rejection, to endure various difficulties B. Biofeedback measurements
and discomfort.
Biofeedback is a method in which an individual is given
IV. E LECTRODERMAL ACTIVITY real-time measurement values and tries to influence them to a
EDA is measured using electrodes that are attached to the desired outcome. EDA biofeedback is used for example:
body. It is most commonly measured at the fingertips, but • to reduce epileptic seizures [13], [14], [15], [16],
other places on the body such as the leg, forehead, or back can • for high blood pressure patients, [17], [18],
also be used [9]. Conductivity is then obtained, either directly • for acne-reducing treatment [19],
or calculated from the resistance. Changes in conductivity • to reduce the problem of diabetes [20].
are then associated with the psychological phenomena of the Biofeedback methods often talk about overstimulation of
individual. An overview of where EDA is used can be seen the body, when the body overreacts to the given stimuli. This
below. can manifest itself in nervousness or increased stress on the
individual. Most of the problems mentioned above are caused
A. Measuring emotional reactions
by these reasons and EDA biofeedback works for them.
Emotional response, is our body’s response to a stimulus
that is accompanied by changes in breathing, skin conduc- C. Other uses of EDA measurement
tance, heart rate and other changes. The measurement of Furthermore, we can encounter EDA measurement in the
an individual’s emotional responses is done using a set of following situations:
appropriately designed stimuli. This may be a short video, • the detection of neuropathy in diabetic patients [21], [22],
audio, or other similar stimulus. The individual processes this • the detection of epileptic seizures, [23],
stimulus and it triggers a reaction (emotion) [10]. • the postoperative pain severity detection system... [24],
Emotional reaction in the EDA signal is manifested by the [25],
so-called peaks, an example of a peak can be seen in Figure • the sleep quality detection system [26],
1. It is the reaction of our skin to a stimulus and the whole • driver attention detection system [27], [28].
process is divided into several phases [11]:
1) Exposure to the stimulus. At this time, the individual is V. E XPERIMENT DESIGN
exposed to the stimulus. The main aim of the research was to record how the skin
2) Latency. This is the time between exposure to the conductance of the measured individual change in the resting
stimulus and the response to it. In this part, the neural state at different time intervals during the day, to determine
processing takes place. the distribution of the individual skin conductance variables

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 521

analysed and then to determine from the data obtained how the 4) Variation coefficient: The coefficient of variation of
individual currently felt in relation to their previous measure- measurement i, denoted by var_coefi , is calculated using the
ments. We believe that deviations from normal values (values formula:
close to the median) may indicate discomfort, excessive psy- stdi
var_coefi = , (4)
chological stress, illness, or a psychological condition that the meani
individual should learn to work with. Values close to normal
where stdi is the standard deviation of the EDA measurement
(median) seem to indicate normal expressions and experiences
i and meani is the mean of the EDA measurement i.
of the individual.

A. Respondents D. Used devices

Ten subjects aged 23-40 years, 5 males and 5 females, The Sensetio device, Figure 2, was used for the measure-
participated in the experiment. ments. This device is placed on the wrist for the measurements
and the electrodes are attached to the third article of the index
B. Measurement procedure and ring fingers, Figure 3. The device is connected to a mobile
phone, using Bluetooth LE technology, where up to 10 devices
Respondents were instructed about the experiment, and can be connected simultaneously to one cell phone.
all signed an informed consent for the measurement and
processing of personal data. Subsequently, measuring devices
were distributed to the respondents and they individually took
200 three-minute measurements at random intervals, with a
minimum of 60 minutes break between them.

C. Analyzed properties
Each EDA measurement was divided into two parts. The
first, of one minute length, was removed to stabilize the
measuring device. The second part of the curve, two minutes
in length, was used to calculate the following properties:
• mean (mean),
• standart deviation (std),
• variability range (var_range),
• variability coefficient (var_coef ).
These properties are described below.
1) Mean: The mean value of measurement i, denoted by
meani , is calculated using the formula:

1
n
meani = xi,j , (1)
n j
Fig. 2. Sensetio device
where xi,j are consecutive values of the ith EDA measurement
obtained with a frequency of 10 Hz.
2) Standard deviation: The standard deviation of measure-
ment i, denoted by stdi , is calculated using the formula:

n 2
j (xi,j − meani )
stdi = , (2)
n−1
where xi,j are consecutive values of the ith EDA measurement
obtained with a frequency of 10 Hz and meani is the mean
value of the ith measurement.
3) Variation range: The variation range of measurement i,
denoted by var_rangei , is calculated using the formula:

var_rangei = xmax
i − xmin
i , (3)

where xmax
i is the maximum of the EDA measurement i and
xmim
i is the minimum of the EDA measurement i. Fig. 3. Placement of measuring device

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 522

 VI. R ESULTS The standard deviation, coefficient of variation, and range of
From the obtained properties of the individual EDA curves variation will tell us about his relaxation or stimulation. (More
we created distribution plots. An example of a distribution plot peaks in the EDA curve will cause an increase in these metrics,
can be seen in Figure 4. The plots for the other individuals and and as we know, peaks are the body’s response to emotional
analysed properties were similar. The distributions resemble a or cognitive stress. More peaks, therefore, result in higher
log-normal distribution, so we apply natural logarithm to all values of the aforementioned traits and therefore say that the
values and then applied the Shapiro-Wilk test for confirmation individual is stimulated by the environment. In contrast, curves
or denial of normality. P-values for each property are shown without peaks indicate a relaxed state of the individual.) Thus,
in Table I. Values greater than 0.05 (confirming log-normal an individual can obtain objective information about his or her
distribution) are in bold. experience based on regular measurements. He or she can find
out whether his or her experience is close to normal or far from
it and can thus expand his or her soft competencies, especially
self-reflection.
The ideal measurement to verify the distribution would be
a continuous long-term measurement over several days. Since
EDA measurements are strongly influenced by movement,
it is necessary for the individual to remain at rest during
the measurement. The most suitable place for measurement
is the fingers. From the above, it is clear that continuous
measurements are not possible, so we have chosen the option
of random measurements, which will bring us closer to the
real state of the distribution.
The baseline data show that the statistical properties of
Fig. 4. Probability density function of selected subject the EDA curves exhibit a log-normal distribution for most
individuals and most measured properties. As already men-
tioned, these data may be affected by the individual’s limited
TABLE I measurement capabilities, for example, due to the distribution
P- VALUES OBTAINED FROM S HAPIRO -W ILK T EST FOR GIVEN PROPERTIES of working hours in the job.
AND SUBJECTS
It is known that the heart rate variability signal also has a
meanlog stdlog var_rangelog var_coef log log-normal distribution [29]. Therefore, we can conclude that
Subject 1 0.000 0.046 0.081 0.006 the distribution of HRV and EDA signals are similar and there
Subject 2 0.080 0.366 0.529 0.381
Subject 3 0.009 0.021 0.139 0.027 may be a correlation between them.
Subject 4 0.106 0.842 0.142 0.006 Our experiment was conducted with 10 individuals. This
Subject 5 0.000 0.000 0.000 0.003
Subject 6 0.275 0.496 0.031 0.123
is a baseline study that will be further extended. To obtain
Subject 7 0.541 0.610 0.545 0.677 more accurate data, it is advisable to expand the sample of
Subject 8 0.274 0.095 0.159 0.000 individuals measured and to take measurements at regular
Subject 9 0.012 0.073 0.080 0.022
Subject 10 0.039 0.054 0.208 0.295
intervals. The measured individuals must be willing to strictly
5/10 7/10 8/10 4/10 adhere to the times and intervals during the measurements,
which may pose a problem.
VII. D ISCUSSION
VIII. C ONCLUSION
Information about the shape of the distribution is an impor-
tant component for the conclusions from the measurements. Finding that individual measured traits satisfy a log-normal
When interpreting the results to the user, we have to work distribution for most individuals allows us to determine how
differently with data with a linear distribution and differently individuals currently feel relative to previous measurements.
with data with a log-normal distribution. For example, the Thus, we can monitor his or her current state by logarithmizing
distance from the median for a log-normal to the positive the resulting values to approximate a normal distribution, and
direction carries different information than the distance to if there is a prolonged fluctuation away from the midpoint of
the negative direction. Based on the information about the this normal, we can alert the individual to a fluctuation in his
distribution, we can then infer what is the normal state for or her physical or mental state.
an individual (his normal) and how far the values are from We were unable to verify the normality of the distributions
this normal and also how far they are. in several individuals. In the following research we will focus
The measurement itself, and its comparison with the individ- on these individuals and try to find the cause. We will in-
ual’s data from previous measurements, then reveals the indi- vestigate whether the reason is just irregular measurements or
vidual’s current state of mind. The median of the electrodermal whether it is some specific personality trait or the individual’s
activity curve will be indicative of the individual’s arrousal. current psychological state.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 523

 ACKNOWLEDGMENT [21] C. M. Casellini, H. K. Parson, M. S. Richardson, M. L. Nevoret, and
A. I. Vinik, “Sudoscan, a noninvasive tool for detecting diabetic small
This research was funded by an R&D project ETA, fiber neuropathy and autonomic dysfunction,” Diabetes technology &
TL01000299 from the budget of the Czech Republic. therapeutics, vol. 15, no. 11, pp. 948–953, 2013.
[22] H. Mayaudon, P.-O. Miloche, and B. Bauduceau, “A new simple method
for assessing sudomotor function: relevance in type 2 diabetes,” Diabetes
R EFERENCES & metabolism, vol. 36, no. 6, pp. 450–454, 2010.
[23] J. S. Sia, L. Z. Huan, L. S. Kit, L. C. Ling, and G. N. Effiyana,
[1] Z. Khalpey, A. Khalpey, B. Modi, and J. Deckwa, “Autonomic dys- “Prototype development for real-time epilepsy seizures detector using
function in covid-19: Early detection and prediction using heart rate three parameters,” in MATEC Web of Conferences, vol. 250. EDP
variability,” Journal of the American College of Surgeons, vol. 233, no. 5, Sciences, 2018, p. 07002.
pp. e20–e21, 2021. [24] P. G. Dalal, K. K. Doheny, L. Klick, S. Britcher, S. Rebstock, D. Bezi-
[2] A. Ponomarev, K. Tyapochkin, E. Surkova, E. Smorodnikova, and nover, C. Palmer, C. Berlin, M. Postula, L. Kong et al., “Analysis of
P. Pravdin, “Heart rate variability as a prospective predictor of early acute pain scores and skin conductance measurements in infants,” Early
covid-19 symptoms,” medRxiv, 2021. human development, vol. 89, no. 3, pp. 153–158, 2013.
[3] D. Caruelle, A. Gustafsson, P. Shams, and L. Lervik-Olsen, “The use [25] T. Ledowski, J. Bromilow, M. Paech, H. Storm, R. Hacking, and
of electrodermal activity (eda) measurement to understand consumer S. Schug, “Monitoring of skin conductance to assess postoperative pain
emotions–a literature review and a call for action,” Journal of Business intensity,” BJA: British Journal of Anaesthesia, vol. 97, no. 6, pp. 862–
Research, vol. 104, pp. 146–160, 2019. 865, 2006.
[4] G. Peters-Kühlinger and F. John, Komunikační a jiné" měkké" doved- [26] J. Woelk, D. Goerlitz, and A. Wachholtz, “I’m tired and it hurts! sleep
nosti. Grada Publishing as, 2007. quality and acute pain response in a chronic pain population,” Sleep
[5] J. Klímová and M. Fialová, Proč (a jak) psychosomatika funguje?: Proč Medicine, vol. 67, pp. 28–32, 2020.
nemoc začíná v hlavě? Progressive consulting vos, 2015. [27] O. Dehzangi, V. Rajendra, and M. Taherisadr, “Wearable driver distrac-
[6] H. Morschitzky and S. Sator, “Když duše mluví řečí těla: stručnỳ přehled tion identification on-the-road via continuous decomposition of galvanic
psychosomatiky,” 2015. skin responses,” Sensors, vol. 18, no. 2, p. 503, 2018.
[7] V. Skumin, “Borderline mental disorders in chronic diseases of the [28] V. Rajendra and O. Dehzangi, “Detection of distraction under naturalistic
digestive system in children and adolescents,” Zhurnal Nevropatologii i driving using galvanic skin responses,” in 2017 IEEE 14th International
Psikhiatrii Imeni SS Korsakova (Moscow, Russia: 1952), vol. 91, no. 8, Conference on Wearable and Implantable Body Sensor Networks (BSN).
pp. 81–84, 1991. IEEE, 2017, pp. 157–160.
[8] R. Honzák, Psychosomatická prvouka. Vyšehrad, 2017. [29] D. Ziegler, R. Piolot, K. Strassburger, H. Lambeck, and K. Dannehl,
[9] M. van Dooren, J. H. Janssen et al., “Emotional sweating across the “Normal ranges and reproducibility of statistical, geometric, frequency
body: Comparing 16 different skin conductance measurement locations,” domain, and non-linear measures of 24-hour heart rate variability,”
Physiology & behavior, vol. 106, no. 2, pp. 298–304, 2012. Hormone and Metabolic Research, vol. 31, no. 12, pp. 672–679, 1999.
[10] V. Sharma, N. R. Prakash, and P. Kalra, “Audio-video emotional
response mapping based upon electrodermal activity,” Biomedical Signal
Processing and Control, vol. 47, pp. 324–333, 2019.
[11] W. Boucsein, Electrodermal activity. Springer Science & Business
Media, 2012.
[12] M. Benedek and C. Kaernbach, “A continuous measure of phasic
electrodermal activity,” Journal of neuroscience methods, vol. 190, no. 1,
pp. 80–91, 2010.
[13] I. Kotwas, A. McGonigal, S. Khalfa, M. Bastien-Toniazzo, F. Bar-
tolomei, and J.-A. Micoulaud-Franchi, “A case-control study of skin
conductance biofeedback on seizure frequency and emotion regulation
in drug-resistant temporal lobe epilepsy,” International Journal of Psy-
chophysiology, vol. 123, pp. 103–110, 2018.
[14] J.-A. Micoulaud-Franchi, I. Kotwas, L. Lanteaume, C. Berthet,
M. Bastien, J. Vion-Dury, A. McGonigal, and F. Bartolomei, “Skin
conductance biofeedback training in adults with drug-resistant temporal
lobe epilepsy and stress-triggered seizures: a proof-of-concept study,”
Epilepsy & Behavior, vol. 41, pp. 244–250, 2014.
[15] Y. Nagai, J. Aram, M. Koepp, L. Lemieux, M. Mula, H. Critchley,
S. Sisodiya, and M. Cercignani, “Epileptic seizures are reduced by
autonomic biofeedback therapy through enhancement of fronto-limbic
connectivity: a controlled trial and neuroimaging study,” EBioMedicine,
vol. 27, pp. 112–122, 2018.
[16] Y. Nagai, L. H. Goldstein, P. B. Fenwick, and M. R. Trimble, “Clinical
efficacy of galvanic skin response biofeedback training in reducing
seizures in adult epilepsy: a preliminary randomized controlled study,”
Epilepsy & Behavior, vol. 5, no. 2, pp. 216–223, 2004.
[17] S. Khumar, P. Kaur, and J. Kaur, “A study of therapeutic effect of gsr
biofeed-back on mild hypertension.” Journal of the Indian Academy of
Applied Psychology, 1992.
[18] E. Paran, M. Amir, and N. Yaniv, “Evaluating the response of mild
hypertensives to biofeedback-assisted relaxation using a mental stress
test,” Journal of behavior therapy and experimental psychiatry, vol. 27,
no. 2, pp. 157–167, 1996.
[19] N. Pandey, A. Salam, P. Ojha, S. Siddiqui, and Z. Zaidi, “Galvanic
skin resistance in different phases of menstrual cycle in acne vulgaris,”
National Journal of Physiology, Pharmacy and Pharmacology, vol. 7,
no. 5, p. 455, 2017.
[20] R. A. McGinnis, A. McGrady, S. A. Cox, and K. A. Grower-Dowling,
“Biofeedback-assisted relaxation in type 2 diabetes,” Diabetes care,
vol. 28, no. 9, pp. 2145–2149, 2005.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 524

 Increasing cyclists safety using intelligent Bicycle
Light Based On Artificial Intelligence
Adam Tomčala, Rastislav Bencel
Faculty of Informatics and Information Technologies
Slovak University of Technology in Bratislava
Bratislava, Slovakia
{xtomcala, rastislav.bencel}@stuba.sk

Abstract—Nowadays, there is a lot of effort to increase safety functionality of bicycle directional lights facilitates the notifi-
on roads. The main focus of research is on vehicles and their cation of the cyclist’s intended direction to other motorists,
associated functionality. This paper focuses on increasing safety significantly aiding in anticipating the cyclist’s movement.
for other cyclist road transport participants. The research is
focused on intelligent light for which propose a solution based Additionally, the cyclist’s brake lights inform other road users
on a connection between the mobile device and light device. of their deceleration, significantly reducing the reaction time
The solution is aimed at brake light and turn signals. We use for braking detection by other road users. The inclusion of
data gathered from motion sensors to achieve a simple device elementary lighting apparatus enables the prevention of traffic
that can only be mounted on a bicycle and provide lighting accidents, particularly during low-light conditions or inclement
functionality. The data was evaluated by artificial intelligence,
namely LSTM (Long Short-Term Memory). The model was weather.
trained in the prototype phase, where labeled braking data was The main contribution of this paper is proposing intelligent
created. For labeling data hall sensor was used located on the brake light for cyclists using the AI model. We create an
bicycle brake. Additionally, the solution introduces a turn signal AI model dataset from our test drives which mark different
light controlled by voice. Through the utilization of an offline braking types. To our best knowledge, we have identified
voice recognition library on a mobile phone, the directional lights
are governed by verbal commands. This paper describes the a few research for intelligent lighting in the cycling area.
hardware, communication, and software parts required to build Additionally, we extend this breaking light with a turning
the intelligent light. Experimental analysis has been conducted signal with usage voice.
to collect data in different situations that occur while cycling, The following sections of this paper are organized as
such as braking with different intensities or braking when the follows: Section II describes the current open-source solutions
bicycle is tilted differently. During the testing phase, we evaluate
the effectiveness of the trained LSTM model in detecting bicycle that have been proposed to improve cyclists’ safety on roads.
braking during the real ride. Section III describes the solution design. The following sec-
Index Terms—intelligent light, Artificial Intelligence, bicycle, tion IV describes the initial prototype and its testing to obtain
Bluetooth Low Energy, Internet of Things data for further analysis. Section V describes the process of
testing the trained LSTM model for each type of braking.
I. I NTRODUCTION Finally in section VI is a summary of the article and future
work.
Currently, there is a persistent escalation in the number
of motorized and non-motorized vehicles on the roadways, II. R ELATED WORKS
resulting in an increased possibility of traffic accidents. Recent A significant amount of research studies has been conducted
statistical analysis has shown that the incidence of fatal in the area of safety analysis for bicycles and other two
accidents involving cyclists is not changing. The fatalities wheels vehicles on the road e.g. [4]. The best solution to
within cycling transport are the only ones that over the past increase bicycle safety is to create separate roads. However,
decade, are not decreasing [1] [2]. The plausible cause for such this solution is only possible in some cases. In the following
incidents may be the absence of adequate safety equipment on part of this section, we analyze research on improving lights
bicycles, thereby increasing the risk of harm to the cyclists on bicycles.
themselves. Unlike motorbikes or cars, bicycles often lack In work [5] is introduced solution DEBI (Device for Bikes),
fundamental safety components, including light systems. The an open-source hardware-software solution that solves safety
analysis in [3] has shown an increasing direction of accidents problems for cyclists in urban areas during peak traffic hours.
in cycling transport in Norway. Additional safety analysis The system comprises visual indicators such as brakes and
for bicycles and other two-wheel vehicles is presented in directional lights to enhance the cyclist’s visibility to other
paper [4]. Within the European Union is an effort to protect vehicles and to alert them of an approaching vehicle. The brake
vulnerable road users. Providing turn signals and braking lights are controlled based on accelerometer data. The system
lights can contribute to decreasing accidents on the roads. The saves the last series of acceleration values and compares them

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 525

to the current ones to determine whether to illuminate the Brake Directional
brake light. Additionally, a mobile application has been devel- LED LED
oped to control the directional lights using voice commands.
Communication between the device and the mobile phone is
via Bluetooth technology. [5]
The second research work [6], entitled ”Design and Devel- Accelerometer
opment of a Multifunctional Bike Assist System for Cyclist”,
proposes a solution that increases the safety of cyclists on the Gyroscope
road. The IoT-based system is built on the Arduino platform Hall sensor
and facilitates the control of brake and indicator lights using BLE module
buttons. However, this approach may not be optimal as it
requires cyclists to divert their attention from the road to Arduino Nano 33
operate the buttons, potentially increasing the risk of accidents. BLE
Notably, the system’s key innovation is utilizing a Hall sensor
to measure the bicycle’s speed. The hall sensor is mounted on
the bike’s frame, while the magnet is on the tire ring. It is Power supply
crucial to know the diameter of the wheel, and then the bike’s
speed is calculated. An improvement to this solution could Fig. 1. Architecture of intelligent bicycle lighting.
be to use the detected velocity to control the brake light. If
the velocity drops, the brake light will illuminate. Similar to
the previous work, a proximity sensor alerts the cyclist of an A. Hardware components overview
approaching vehicle.
In this part are described the hardware component of our
Authors in paper [7] introduce an intelligent bicycle kit that
solution.
contains hardware architecture with defined components. The
1) Arduino Nano 33 BLE: The Arduino Nano 33 BLE is
authors focus on the functionality of braking light and burglar
the smallest board available in the Arduino family. It contains
triggered based on a gyroscope. Within braking functionality,
an integrated IMU LSM9DS1 consisting of an accelerometer,
they considered about sliding and heavy braking. The decision
gyroscope, and magnetometer. Each of them works for the
about braking was performed based on the Y axis, for which
x, y, and z axes. It is possible to scale the range for each
was a set threshold.
sensor according to the datasheet. 1 . The new central part is
The paper [8] introduces C-ITS enabled solution for bi-
the upgraded processor that allows higher data and programs
cycles that can track bicycle movements and also direction
to be stored in the memory [9].
changes based on hand movements. These hands movements
The board has a built-in Bluetooth pairing module named
are detected and can be used for example, as turn signals or
NINA B306 that allows communication over short distances.
for additional safety features in V2X. The authors have not
Significantly, the Arduino Nano 33 BLE exhibits low power
considered about braking detection.
consumption, making it an efficient choice during the
prototyping phase. [10]
III. S OLUTION DESIGN
2) Sensors overview: Sensors are used to measure the
This section has described the design of our solution for physical properties of the external environment and differ in
intelligent bicycle lighting. The lighting provides the function- their functionality, allowing observing many variables. [11]
ality of automatically braking light and turn signal light. Both The next part of the section describes the sensors that are
of these functions have different behavior because braking included in the solution:
should be automatically detected, and turn signals are based
on commands from cyclist. To increase the safety of cyclists, • Accelerometer
we focus on voice control this turn signal lights. The proposed Accelerometer is a sensor designed to measure the force
solution architecture is depicted in Figure 1. The architecture of acceleration and detect and measure vibration or
contains several components, which are Brake LED and Di- inclination relative to the body into which it is integrated.
rectional LED; Arduino Nano 33 BLE; Accelerometer and The most commonly used are 3-axis capacitive MEMS
Gyroscope; Hall sensor; Mobile device; item Power supply. (Micro Electronic Mechanical System) accelerometers,
The components are described more deeply in the following which work by varying the distance of the moving part,
sections. The data collected by Arduino are sent to the mobile mounted on a spring, from the capacitor causing a change
device via Bluetooth Low Energy (BLE), where they are in capacitance.
processed. The hall sensor is integrated only in the proto-
type phase to label the data for our dataset, which contains 1 Arduino Nano 33 BLE https://docs.arduino.cc/resources/datasheets/ABX00030-
information from sensors. datasheet.pdf

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 526

 • Gyroscope Another sensor measuring the motion of a obtained from the Hall sensor for training the LSTM model.
body is the gyroscope, which measures the angular Data transfer time measurements were taken during the im-
velocity of the rotating body on which the sensor is plementation of the inter-device communication. In the first
placed. Typically 3-axis gyroscopes are most commonly case, the length of the process between sending motion data
used, sensing angular velocity in all three axes. They from the lighting device, subsequent data processing in the
work on the principle of the Coriolis force. Similar to mobile application, model prediction, and sending the com-
accelerometers, the magnitude of the angular velocity is mand back was measured. The distance between the devices
determined by the moving part of the mass that changes was approximately 2 meters, and the total process length was
the relative distance of the capacitor surfaces, based on on average 95 ms. On the other hand, the measurement of the
the force mentioned above. time during data collection was done because the data during
• Hall sensor data collection is data sent in 2 characteristics. The time of
A Hall sensor is used to detect the external magnetic this process is on average 45 ms.
field. The main characteristics of the magnetic field are
magnetic flux density and polarity. The movement of C. Processes
the magnet in the vicinity of the sensor changes the This subsection describes the main functionalities of intelli-
magnitude of the magnetic flux density. The changes gent lighting system. Part of the work is a mobile application
are captured by the sensor, which is transformed to the that functions as a user interface for controlling, managing,
corresponding value of the output voltage, called the Hall and processing data from IoT devices. The mobile application
voltage. The sensor is most commonly used in moving processes data collected from Arduino motion sensors and
gate applications where the magnet is placed. [12] creates a sequence window of data used as input for the LSTM
model. The model predicts the intensity of braking, and returns
B. Communication results as hard, soft, or no braking. This value is written to
The communication between the IoT device and the mobile characteristic, and the IoT device can read this value. After
device will be provided by BLE technology. The BLE is fun- that, the corresponding action is performed, as shown in Figure
damentally changed from previous versions mainly to achieve 3. The acquired data is stored in an SQLite database on the
the lowest possible power consumption. It is important to note mobile device in a phase of data collection.
;
Write voice command to Arduino Nano 33 BLE Mobile
Read value characteristic Start Read data from
from BLE characteristic
characteristic module
Nina-B306 Read data from IMU
Process data
Controlling of directional lights Write data to
characteristic
Write data from sensors
to charakcteristic Read data from
characteristic Read brake intensity
BLE from characteristic Hard Light No
module braking braking braking
Nina-B306
Write brake Brake light
on Write intensity to
Read brake intensity to charakcteristic
from characteristic characteristic
End
Controlling of braking lights

Fig. 2. Diagram of communication. Fig. 3. Diagram of brake light process

that BLE communication works on a client-server basis, where Additionally, the application provides voice control for
the Arduino will write the measured values from the sensors directional lights using the VOSK library. The VOSK library
to the individual characteristics and the mobile device will provides offline speech recognition, which ensures the unin-
read this data as shown in figure 2 In this case, the BLE terrupted use of the turning lights feature. [13] The solution
server is part of the IoT device, as shown in figure 1. The provides voice recognition of left and right commands for
data was divided into two characteristics with unique UUID. individual indicator lights, as shown in Figure 4.
The first characteristic conveyed the IMU data, including the
acceleration values in the x, y, and z axis and the angular IV. I MPLEMENTATION
velocity values in the x and y-axis. The dedicated function To develop the proposed solution of lighting system was
of the second characteristic was to transmit the analog value constructed utilizing the prescribed hardware components and

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 527

 Arduino Nano 33 BLE Mobile above scenarios. Collecting acceleration data along the y-axis
Start
was imperative, as this particular axis aligns with the bicycle’s
Illuminate Illuminate travel direction.
left light right light Listening The subsequent graphs will display the acceleration along the
y-axis (m/s2 ) and the corresponding value obtained from the
Lightning Lightning Voice command Hall sensor (analog value) at a synchronous time instance. In
for 10 sec for 10 sec the absence of brake application, the analog output from the
Received Hall sensor typically registers between 700 and 710. However,
Incorrect when the magnet reaches its maximum compression and is
Left light Right light
Vypnutie command near the sensor, the analog output value diminishes to 250.
off off
The time intervals represented on the graph are denoted in
Left Right seconds.
End 1) Hard braking: The initial measurement was conducted
during hard braking. The graph in Figure 5 depicts a significant
reduction in the acceleration value along the y-axis during the
Fig. 4. Diagram for directional lights process
braking phase, followed by an increase in acceleration during
the subsequent re-acceleration phase.
technologies. The lighting system comprises an Arduino Nano
33 BLE, which acts as the control unit of the device. Eight
Light Emitting Diodes (LEDs) are connected to the digital pins
of the Arduino for brake light and turn signal lights.
Additionally, the IoT device incorporates a Hall effect sensor,
S49E, which can detect the magnetic field in its proximity.
Along with detecting the magnetic field, the sensor measures
the strength of the field, reflected in the analog output from
the sensor. The sensor is connected to one of the analog pins
of the Arduino.
The acquired data is sent to a mobile device where a simple
moving average (SMA) filtering algorithm is implemented. Fil-
ters play an important role because they are used to eliminate
the noise and gravity components from the raw accelerometer.
The gyroscope data which got added during data collection
[14]. The SMA algorithm works by calculating the average
value of acquired data. As new data points become available, Fig. 5. Values during hard braking.
the SMA algorithm updates the average value of sensor data
by dropping the oldest data point and adding the newest one. 2) Soft braking: Under soft braking, a lesser reduction in
The data obtained from the accelerometer and gyroscope are the y-axis acceleration was observed as compared to hard
used to calculate the pitch and roll of the bicycle. Pitch braking, as shown in Figure 6.
is critical for determining whether the cyclist is ascending
or descending a slope. Conversely, the roll is essential for
identifying whether the cyclist turns right or left.
A. Data collection
In the initial phase of experimentations, the investigation
aimed to determine the sensor data values under various
driving circumstances. To accomplish this, we deployed an IoT
device underneath the bicycle seat during the measurements.
Specifically, the Hall sensor was attached to the handlebars
underneath the brake, while the magnet was fixed onto the
mobile portion of the brake lever. Acceleration measurements
were conducted under distinct braking intensities, including
hard and soft braking conditions. The experiments encom-
passed diverse driving scenarios, such as braking on a flat
surface, uphill and downhill braking, cornering while braking,
and deceleration without brake application. Fig. 6. Values during soft braking.
In the initial data collection, the focus was on the mentioned

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 528

B. LSTM model training them. The graphs also consist of two sub-graphs, where the
In the process of brake detection, it is vital to consider top graph represents the value from the Hall sensor, and the
that the data related to braking evolves. This implies that for bottom graph shows the model’s predicted value.
accurate identification of the degree of braking, it is crucial
to examine a data sequence that encapsulates alterations in A. Soft braking test
individual motion elements.
The first test scenario verifies the detection of weak braking.
In tackling this issue, deep learning offers Long Short-Term
The model successfully detected this braking and correctly
Memory (LSTM), a distinctive type of Recurrent Neural Net-
evaluated the level as weak. It can be seen from the graph
work (RNN) specifically engineered for sequential time-series
that the model predicts this braking after approximately 0.5 s
data. LSTMs retain data outside of the network’s immediate
after the bicycle has braked. The analog value fell just below
sphere, similar to a computer system’s secondary storage,
500. This value is still within the interval of light braking,
which can be accessed and utilized anytime. This unique
which the model has assessed correctly.
feature empowers LSTM networks to effectively manage and
learn from the extensive-term dependencies between sequence
elements, thereby enhancing their performance in tasks that
necessitate modeling critical temporal relationships and de-
pendencies. [15]
The first step in implementing the model was to categorize the
analog values from the Hall sensor into groups as follows:
• no braking: analogue values > 700 were replaced by a
value of 0,
• soft braking: 700 ≥ analogue values ≥ 420 were
replaced by a value of 1,
• hard braking: analogue values ¡ 420 were replaced by
a value of 2.
The model’s utilization of LSTM layers necessitated the
adjustment of input data into sequences before training to
maintain the integrity of the time series. This adjustment was Fig. 7. Testing of light braking detection
achieved by iterating over individual records in the data frame
to create sequence windows of uniform length.
Different lengths were experimented with, but a size of 20
B. Hard braking
proved to be the most fitting. The dataset was partitioned in
an 80:20 ratio, where 80% comprised the training set and the The second test scenario verifies the detection of hard brak-
remaining 20% made up the testing set. The input dataset ing. In the initial phase, the model first predicted weak braking,
consisted of seven attributes, including the acceleration in the but in the next braking phase, it already correctly determined
x, y, and z axes and the angular acceleration in the x, y, pitch, the braking level as hard. The analog value dropped below
and roll axes. Throughout the training process, an array of 300, and such braking falls into the category of hard braking.
model combinations were evaluated, varying in aspects such
as layer composition, sequential window lengths, and intervals
of Hall sensor values.

V. E VALUATION
After the implementation of the individual parts of the
system, it was necessary to test the most important operations
provided by the system. After training the model, it was
necessary to test this model directly on the road and in real
braking situations. The correctness of the model was necessary
to mount the system on the bike as in the data collection and
again store the test measurements in a local database.
With the stored data, we can compare the LSTM model’s
predicted value with the Hall sensor’s analog value. The test
run was conducted on a different route than the data collection,
and all the scenarios we defined in the previous section of
the paper were performed. The following graphs visualize the Fig. 8. Testing of hard braking detection
testing of each braking category and the transition between

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 529

C. Transition between intensities detection commands were not recognized due to street noise.
The last test scenario aimed at detecting both types of Once the models have processed and evaluated each task, they
braking, where soft braking was performed in the initial phase, can transmit instructions to the lighting device to initiate the
and hard braking occurred later. The model correctly evaluated corresponding actions.
each category and the transition between them. ACKNOWLEDGMENT
This article was written thanks to the generous support
under the Operational Program Integrated Infrastructure for the
project: ”Advancing University Capacity and Competence in
Research, Development and Innovation (ACCORD)”, Project
no. 313021X329, co-funded by the European Regional Devel-
opment Fund.” The research was also supported by the APVV-
19-0401 and the KEGA 025STU-4/2022 projects.
R EFERENCES
[1] E. Commision. (2021) Facts and figures – cyclists - 2021.
[Online]. Available: https://road-safety.transport.ec.europa.eu/system/
files/2022-03/FF cyclists 20220209.pdf
[2] A. Morris, L. Brown, P. Thomas, R. Davidse, V. Phan, D. Margaritis,
D. S. Usami, M. Robibaro, A. Krupinska, K. Sicinska et al., “Safer-
wheels study on powered two-wheeler and bicycle accidents in the eu-
final report,” 2018.
[3] I. Næss, P. Galteland, N. O. Skaga, T. Eken, E. Helseth, and
Fig. 9. Testing of transition between intensities J. Ramm-Pettersen, “The number of patients hospitalized with bicycle
injuries is increasing - a cry for better road safety,” Accident
Analysis Prevention, vol. 148, p. 105836, 2020. [Online]. Available:
D. Turning signals https://www.sciencedirect.com/science/article/pii/S0001457520316560
[4] L. Brown, A. Morris, P. Thomas, K. Ekambaram, D. Margaritis,
Turning signals used the VOSK library for detecting voice R. Davidse, D. S. Usami, M. Robibaro, L. Persia, I. Buttler,
commands. We evaluate this recognition in multiple noise A. Ziakopoulos, A. Theofilatos, G. Yannis, A. Martin, and F. Wadji,
“Investigation of accidents involving powered two wheelers and bicycles
environments: low, medium, and high. The first two environ- – a european in-depth study,” Journal of Safety Research, vol. 76,
ments had no problem with command recognition. The high pp. 135–145, 2021. [Online]. Available: https://www.sciencedirect.com/
noise environment had a problem with recognition if rider science/article/pii/S0022437520301651
[5] E. J. Santiago-Saavedra and F. J. Mercado-Castellanos, “Debi (device
speech was on low volume. for bikes),” 2018.
[6] K. H. Siew, “Multifunctional bike assist system for cyclist,” Ph.D.
VI. C ONCLUSION dissertation, UTAR, 2017.
[7] Q. Wang, Z. Zou, and W. Tang, “Design of intelligent bicycle kit,”
In conclusion, this work successfully developed a solution in Proceedings of the 2016 4th International Conference on Sensors,
for intelligent bicycle lighting. The solution uses BLE to Mechatronics and Automation (ICSMA 2016). Atlantis Press, 2016/12,
communicate between a mobile application and light. The con- pp. 501–505. [Online]. Available: https://doi.org/10.2991/icsma-16.
2016.88
nection is necessary to obtain data from individual sensors and [8] S. Gröchenig and K. Rehrl, “Towards c-its-based communication
enable voice-controlled directional lighting. The conducted between bicycles and automated vehicles,” p. 88–95, 2021. [Online].
experiments provided insights into the behavior of the ac- Available: https://doi.org/10.34726/1753
[9] H. M. Kaidi, M. Izhar, N. Ahmad, R. Dziyauddin, S. Sarip, and S. Jalil,
celerometer under specific conditions and braking intensities. “Rehabilitation monitoring prototype: Arduino nano 35 ble,” in Journal
The collected data was used to train the LSTM model, which of Physics: Conference Series, vol. 2250, no. 1. IOP Publishing, 2022,
was used to detection of bicycle braking. This model was p. 012009.
[10] H. Al-Mimi, A. Al-Dahoud, M. Fezari, and M. S. Daoud, “A study
applied to the mobile device, which was able real-time decide on new arduino nano board for wsn and iot applications,” International
about bicycle braking based on received data from captured Journal of Advanced Science and Technology, vol. 29, no. 4, pp. 10 223–
data in the light device. The evaluation was performed in a real 10 230, 2020.
[11] M. B. Marinov, N. Nikolov, S. Dimitrov, T. Todorov, Y. Stoyanova,
environment, and we confirmed that the trained model has the and G. T. Nikolov, “Linear interval approximation for smart sensors
capability to predict various types of braking. Nonetheless, the and iot devices,” Sensors, vol. 22, no. 3, 2022. [Online]. Available:
model limitation is a delay of approximately 500 milliseconds. https://www.mdpi.com/1424-8220/22/3/949
[12] Hall effect sensor. [Online]. Available: https://www.electronics-tutorials.
One of the challenges observed is the amount of time required ws/electromagnetism/hall-effect.html
for the LSTM model to recognize braking. As a part of future [13] A. Cephei, “Vosk android demo,” https://github.com/alphacep/
work, this issue could be resolved by gathering more training vosk-android-demo, 2021.
[14] G. Singh, D. Bansal, and S. Sofat, “A smartphone based technique
data. This could enhance the model’s capacity to understand to monitor driving behavior using dtw and crowdsensing,” Pervasive
the correlations between the features, thereby improving the and Mobile Computing, vol. 40, pp. 56–70, 2017. [Online]. Available:
overall detection accuracy. Additionally, we introduce turn https://www.sciencedirect.com/science/article/pii/S1574119216301250
[15] A. Singh and R. Bhadani, Mobile Deep Learning with TensorFlow Lite,
signals based on voice. In this solution, the VOSK library was ML Kit and Flutter: Build scalable real-world projects to implement
used for detecting voice commands left and right. The results end-to-end neural networks on Android and iOS. Packt Publishing
usage of this library achieved satisfactory results, and some Ltd, 2020.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 530

 Authorship Attribution in Astroturfing Detection
and the Impact of Google Translate on
Cross-lingual Text Analysis
Isak Kvalvaag Torgersen, Sebastian Leibold, Patrick Bours
Department of Information Security and Communication technology
Norwegian University of Science and Technology (NTNU)
Gjøvik, Norway
patrick.bours@ntnu.no

Abstract—This paper investigates the impact of using Google Norwegian texts translated into English, and the implications
Translate to perform authorship attribution analysis on Norwe- of these findings for detecting astroturfing activities. The focus
gian texts translated into English, with the goal of detecting of the effects of running Norwegian texts through Google
astroturfing activities in online public discourse. The study
compares the performance of various n-gram-based attribution Translate before subjecting them to authorship attribution
techniques on original Norwegian texts and their machine- analysis. By comparing the performance of various n-gram-
translated English counterparts. Results show that the best based attribution techniques on the original Norwegian texts
performance was achieved without translation or pre-processing and their machine-translated English counterparts, we aim
of the original Norwegian texts, reaching an accuracy of 0.84. to quantify the impact of translation on the accuracy and
Further research is needed to explore alternative methods and
pre-processing techniques to enhance the accuracy of authorship robustness of these methods.
on translated texts in this context. The paper is structured as follows: Section II gives more
Index Terms—Authorship Attribution, Astroturfing Detection, background on astroturfing, the role of author attribution to
Cross-Lingual Text Analysis detect it, and previous research into the topic. Section III
describes our experimental setup, including the corpus of
I. I NTRODUCTION Norwegian texts, the translation process, and the n-gram-
Astroturfing is the deceptive practice of presenting co- based attribution techniques employed. Section IV presents
ordinated and manufactured opinions as genuine grassroots the results of our experiments. Section V is a discussion of
support. Astroturfing online is when one person operates the results. Finally, Section VI offers conclusions and potential
multiple accounts on social media sites, forums, review sites, future directions for research in this area.
commercial platforms, or similar. To qualify as astroturfing,
II. A STROTURFING AND AUTHORSHIP ATTRIBUTION
there must be an intent of making it seem like the different
accounts are different people. For example: a social media A. The impact of astroturfing
manager having multiple business accounts and a personal After the 2016 US presidential elections, the National Intel-
account does not qualify. The purpose of astroturfing can range ligence Council (NIC) at the Office of the Director of National
from creating an illusion that some political belief is more Intelligence, released a report on Russian state-sponsored
widely held than it really is, to boosting the reviews of a activities related to the election. In the report, they explain
seller’s roduct. The sinister effect of astroturfing lies in the fact that there was a concerted effort by the Russian government to
that people often trust the opinions of ‘real people’ more than affect the outcome of the election by changing public opinions.
official information. A product review is seen as the honest This is a view shared by both the FBI, CIA, and NSA [1].
opinion of a consumer who has tested the product in question One of the ways in which they attempted to do this was
and it’s something that a lot of people rely on to make their through the use of “professional trolls” tied to the Internet
purchases. Research Agency located in Saint Petersburg. The Internet
Research has already been done into using author attribution Research Agency is most likely financed by a close ally of
to detect astroturfing. A lot of the research published in Russian President Vladimir Putin and a leading expert on the
English on this topic focuses on analyzing English language, agency claims that social media accounts tied to it advocated
which stands to reason. Indeed, a lot of the openly available for Donald Trump’s candidacy [1].
methods and tools developed are tuned to English. It can be With the assumption that the findings in the report published
difficult to find out-of-the-box solutions to perform author by the NIC are correct, it is a fair assumption that the
attribution for smaller languages such as Norwegian. Russian government, or indeed other entities, might be secretly
This paper investigates the impact of Google Translate on trying to affect the outcomes of elections in other countries.
the performance of authorship attribution methods applied to With Norway being a NATO member and a US ally, who

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 531

shares a border with Russia, it is reasonable to suspect that number of users. When the number of potential authors grows,
Norway might be or become a target for Russian propaganda. it introduces more variability in writing styles, which makes
The Russia-Ukraine conflict has arguably made this topic it more challenging to identify the unique characteristics of
more relevant and multiple reports, articles, and documentaries a specific author. As the dataset expands, the likelihood of
covering disinformation activities by Russian intelligence in encountering similar writing styles among different users also
Norway and Scandinavia, have been published [2]–[4]. A increases, potentially leading to false attributions and a decline
report by researchers at the Norwegian Defence Research in overall accuracy. This is a big problem when trying to link
Establishment (FFI) did not find evidence that foreign actors different accounts because the sample sizes are sometimes very
tried to influence the election results, voter participation, or small, and the dataset of potential authors can contain every
trust in the election itself for the Norwegian Parliamentary other account which you don’t have another way of excluding.
Election 2021. However, they did find evidence suggesting
that foreign non-state actors were engaged in the spread C. The language gap
of disinformation to a Norwegian audience and uncovered Authorship attribution, authorship authentication, and other
clusters of what they call inauthentic Twitter profiles spreading related fields are of great relevance in our increasingly online
information to a Norwegian audience [2]. world. It is not only for astroturfing that these methods are
This paper is not a comment on the outcomes of the 2016 used. They are also used in forensics, fighting terrorism,
US presidential elections or any geopolitical events, past or plagiarism detection, and more. The advent of large language
current. However, it should not be controversial to say that it model chatbots such as ChatGPT has already sparked a lot
is in the interest of sovereign states to protect themselves from of research into ways of detecting whether a piece of text
the undue influence of potentially hostile actors. Therefore, it is written by an AI or a human. With the developments and
is relevant to do research on astroturfing to understand if and interest in authorship attribution and related fields, comes more
how it’s used, how to detect it, and how to combat it. The and more out-of-the-box solutions to perform different types
focus of this paper is on the detection of astroturfing. of analyses. This means that people who might not have
the know-how or time to build these tools from scratch can
B. The role of authorship attribution in astroturfing detection
participate in the research. However, a lot of the tools and
Authorship attribution is the process of determining the research are very Anglocentric. Even researchers whose native
author of a given text by analyzing its linguistic features, language is not English might choose to work on English
such as writing style, word choice, and syntax. By leveraging language analysis. This leaves a knowledge and resource gap
computational techniques and statistical methods, researchers for researchers who want to work on the analysis of smaller
can compare the text in question to known works by various languages. Hopefully, this gap will eventually become smaller,
authors in order to identify the most likely author. Authorship but until then, those who are unable to build the tools from
attribution has been suggested as one way of detecting astro- scratch will have to look for workarounds. Translating the texts
turfing [5]. A person who’s engaged in astroturfing might have into English using machine translation and using tools tune to
several accounts, on the same site and/or across different sites, the English language to analyze them, could perhaps be such a
which are all made to look like different people. If the person workaround. Researching how authorship attribution methods
controlling the accounts is also writing the content published are affected by translated texts, also has other applications.
by the accounts, then in theory it should be possible to use Running the text through a translator might be a way in which
authorship attribution methods to link the different accounts. someone tries to obfuscate the authorship, so this research
[5] suggests a binary n-gram approach to detecting as- could be relevant for detecting and counteracting intentional
troturfing. Using different binary n-gram methods with n obfuscation.
from 3 to 16, they analyze the comments left by users on
new websites. Their methods were able to produce a strong III. M ETHODOLOGY
match between different accounts that were already suspected
to be linked. Using authorship attribution to detect a link A. Data selection
between different accounts is not just relevant for astroturfing The dataset used in the research for this paper was col-
detection. It is also, and perhaps more commonly, used in lected by scraping all comments by all users from the online
criminal investigation and terrorism research. [6] achieved a community r/norge on the popular website Reddit [7], in the
high accuracy when analyzing posts from a darknet forum, time period November 3. 2022 to February 21. 2023. r/norge
by using a combination of three different types of features. is a Norwegian language subreddit ranked in the top 1 percent
Because they did not know the truth about whether any of the of subreddits by size, with 214k registered members. The
accounts were linked, they split the dataset in two, creating two topics discussed range from politics and news to meme content
pseudo users from each user. One user was used for training and humor, but one of the subreddits rules is that it must
and one for testing. By combining character-level ngrams, be related to Norway. More than eight thousand comments
stylometric features, and timestamp features about the users were scraped and the timeframe they were scraped from is
posting pattern, they were able to achieve an accuracy of 88 based on restrictions on the third-party API-wrapper used in
percent or 94 percent if the classifier was trained on a smaller the scraping process and the time the data was scraped.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 532

 From the more than eight thousand posts in the original 6) English: special characters removed pre-translation;
dataset, the 150 longest posts from the top ten most prolific 7) English: special characters removed pre-translation,
users were chosen. This selection was done in order to have stemming, and lemmatization.
a good sample size for each user used as part of the analysis, Stemming and lemmatization was not used on the Norwegian
get balanced classes, and have a manageable dataset for the dataset as that would require tools specifically tuned for
available project time. Admittedly, more time should probably Norwegian, which would defeat the purpose.
have been spent on manually checking and processing the final
C. Analysis
dataset and this might have affected the results. For example:
choosing only the longest posts from each user might have For the analysis part of the research, the chosen method was
left out some of the stylometric range and idiosyncrasies of character-level n-gram analysis using a linear Support Vector
certain users. Machine (SVM) for classification. This method of analysis is
After the final data selection was done two pseudo users, easy to set up with openly available out-of-the-box solutions
user a train and user a test, were created from each user. All and it’s not language dependent. The analysis was done using
the comments from each user were randomly assigned to the different values of n from 2 to 16. The method of analysis is
corresponding pseudo users, creating one dataset for training inspired by the method used in [5].
and one for testing. This is the same technique used by [6] and IV. R ESULTS
the reasoning for doing so is the same as for them. To select the
Table II shows the result of the first analysis which was done
ratio of training data versus testing data: different splits were
on Norwegian language posts with no pre-processing. The f1-
tested across the different datasets from 95/5 to 50/50. The
score starts at 0.69 for an n value of 2 and then increases
ratio was changed by five for each round of testing. From each
to a maximum of 0.84. This is the highest f1-score achieved
round of testing the highest average f1-scores were selected
from all of the analyses performed during this research. After
and in the end the average highest score for each dataset was
n value of 4, the f1-score drops and continues to drop for
computed for each tested ratio. The best scoring ratio was 95
subsequent values of n. In general, there is no increase in
percent for training and 5 for testing and therefore this ratio
the f1-score after n value of 8 in any of the analyses. The
was chosen. Table I shows the results from ratio 95/5 to 75/25.
subsequent tables are cut off after their last score increase to
TABLE I
save space.
R ESULTS FROM RATIO TESTING .
TABLE II
Split Score R ESULTS FROM ANALYSIS ON N ORWEGIAN POSTS WITHOUT
95/5 0.74 PRE - PROCESSING .
90/10 0.73
85/15 0.72 Norwegian: No pre-processing
80/20 0.71 n Precision Recall F1 -score
75/25 0.72 2 0.73 0.69 0.71
3 0.82 0.80 0.81
4 0.85 0.84 0.84
5 0.83 0.82 0.82
B. Translation and pre-processing 6 0.83 0.82 0.82
7 0.82 0.81 0.81
After the datasets were ready, they were copied and 8 0.72 0.71 0.71
each copy was put through different combinations of pre- 9 0.68 0.68 0.68
processing: The processing techniques were as follows: 10 0.64 0.65 0.64
11 0.61 0.62 0.61
• Translation; 12 0.60 0.61 0.60
• Removal of all special characters leaving letters, numbers, 13 0.60 0.60 0.60
14 0.57 0.54 0.55
and whitespace; 15 0.51 0.50 0.50
• Stemming and lemmatization. 15 0.47 0.47 0.47
The method for translation was to import the comments
from a CSV file to a Google Sheet and using the GOOGLE- Table III shows the results from the analysis of the Norwe-
TRANSLATE function. For the translated and pre-processed gian posts with special characters removed. Removing special
datasets two were created by removing special characters characters did not improve the score.
before translation and two were created by doing it after. In Table IV shows the results of the analysis of the trans-
the end, the following datasets were created. lated posts without any pre-processing. The highest accuracy
1) Norwegian: no pre-processing; achieved for this analysis is 0.75 and this is also the highest
2) Norwegian: special characters removed; score among all the translated texts. There is a small increase
3) English: no pre-processing; in the f1-score at n value of 8.
4) English: special characters removed post translation; Tables V and VI show the results of the analysis of the
5) English: special characters removed post translation, translated texts with pre-processing done before the transla-
stemming, and lemmatization; tion. The fifth one shows a better highest score than the sixth

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 533

 TABLE III TABLE VII
R ESULTS FROM ANALYSIS ON N ORWEGIAN POSTS WITH SPECIAL R ESULTS FROM ANALYSIS ON E NGLISH POSTS WITH SPECIAL
CHARACTERS REMOVED . CHARACTERS REMOVED PRE TRANSLATION .

Norwegian: Special characters removed English: Special characters removed

n Precision Recall F1 -score n Precision Recall F1 -score
2 0.67 0.61 0.64 2 0.61 0.59 0.60
3 0.82 0.80 0.81 3 0.65 0.65 0.65
4 0.81 0.80 0.80 4 0.71 0.70 0.70
5 0.77 0.76 0.76 5 0.69 0.70 0.69
6 0.77 0.76 0.76 6 0.68 0.70 0.69
7 0.80 0.79 0.79 7 0.65 0.66 0.65

TABLE IV TABLE VIII

R ESULTS FROM ANALYSIS ON E NGLISH POSTS WITHOUT R ESULTS FROM ANALYSIS ON E NGLISH POSTS WITH SPECIAL
PRE - PROCESSING . CHARACTERS REMOVED PRE TRANSLATION , STEMMING , AND
LEMMATIZATION .
English: No pre-processing
n Precision Recall F1 -score English: No pre-processing
2 0.68 0.69 0.68 n Precision Recall F1 -score
3 0.65 0.65 0.65 2 0.66 0.61 0.63
4 0.75 0.74 0.74 3 0.64 0.64 0.64
5 0.75 0.75 0.75 4 0.68 0.68 0.68
6 0.75 0.75 0.75 5 0.68 0.70 0.69
7 0.66 0.68 0.67 6 0.66 0.66 0.66
8 0.73 0.69 0.71 7 0.66 0.68 0.67

one, but both score worse than only translation. For the fifth
The highest single accuracy among the last four tables is
one, the best score is at n value of 5 and for the sixth one it’s
found in Table 7. However, the average accuracy was higher
n value of 7.
for the analyses where all the pre-processing was done after
TABLE V
the translation.
R ESULTS FROM ANALYSIS ON E NGLISH POSTS WITH SPECIAL
CHARACTERS REMOVED POST TRANSLATION .
V. D ISCUSSION
English: Special characters removed
n Precision Recall F1 -score The results presented in the previous section do not indicate
2 0.57 0.59 0.58 that there is any benefit to translating the text to English with
3 0.60 0.61 0.60 Google Translate when doing authorship attribution analysis
4 0.65 0.66 0.65 on Norwegian texts. In fact, the results indicate that when
5 0.72 0.72 0.72
6 0.64 0.66 0.65 using character-level n-gram analysis with a linear SVM, the
7 0.67 0.68 0.67 best results are achieved when doing no preprocessing at
all. The best accuracy achieved was 0.84, only using n-gram
analysis, on the untranslated and unprocessed texts.
TABLE VI The highest score achieved from analyzing the translated
R ESULTS FROM ANALYSIS ON E NGLISH POSTS WITH SPECIAL texts with no other pre-processing is low compared to that
CHARACTERS REMOVED POST TRANSLATION , STEMMING , AND achieved by most state-of-the-art methods, especially consid-
LEMMATIZATION .
ering again the size of the dataset. However, it is not so low
English: No pre-processing that it would be beyond merit to explore this path further.
n Precision Recall F1 -score It is not so surprising that the translated texts would give
2 0.55 0.56 0.55 lower accuracy as a lot of the stylometric features of an
3 0.58 0.56 0.57
4 0.59 0.61 0.60 author would probably be lost or altered when the text is
5 0.64 0.65 0.64 put through a translator. Even so, there is the possibility that
6 0.65 0.66 0.65 certain features, which are not captured by the method used
7 0.69 0.68 0.68
for this paper, could survive the translation process. Other
methods of analysis or data pre-processing techniques could
Tables VII and VIII show the results of the analysis of lift the accuracy to a point where it would be usable, but this
the translated texts with special characters removed before the would require further research.
translation. In this instance, they both have the same highest It is interesting to see that the type and order of pre-
score. The eighth analysis has a better highest score than the processing affect which value of n gives the best f1-score.
sixth. This could be random noise determined by the dataset, but

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 534

it would be interesting to repeat the analyses on a different [5] J. Peng, S. Detchon, K.-K. R. Choo, and H. Ashman, “Astroturfing
dataset to see if the trend holds. detection in social media: a binary n-gram–based approach,” Concurrency
and Computation: Practice and Experience, vol. 29, no. 17, p. e4013,
It’s important to realize that the results are only valid for 2017.
the specific setup described in this paper. Other methods, [6] M. Spitters, F. Klaver, G. Koot, and M. Van Staalduinen, “Authorship
configurations, and pre-processing techniques could give other analysis on dark marketplace forums,” in 2015 European Intelligence and
Security Informatics Conference, pp. 1–8, IEEE, 2015.
results. Another thing to note is that the number of users used [7] “Reddit - r/norge.” https://www.reddit.com/r/norge/ [Accessed:
in the analysis is low and the sample sizes for the training and 22/08/2023].
testing are bigger than what they would be in many real-world
scenarios. To know if the method has any merit, it would be
necessary to test it on bigger and more realistic datasets. There
is a potential that the method could be combined with other
language-specific features and timestamp features such as in
[6], to achieve higher accuracy. However, a researcher using
a setup at that level of complexity might have the know-how
to create language-specific tools.
VI. C ONCLUSION AND FUTURE WORK
A. Conclusion
The findings of this study indicate that using Google Trans-
late to perform authorship attribution analysis on Norwegian
texts translated into English does not improve the perfor-
mance of the n-gram-based attribution methods for detecting
astroturfing activities. The best results were obtained with
no pre-processing or translation on the original Norwegian
texts, achieving an accuracy of 0.84. The highest accuracy for
translated texts was 0.75, which is not sufficient for practical
use. This study provides a foundation for further research in
this area, which may include exploring alternative methods of
analysis and pre-processing techniques.
B. Future work
In terms of future work to build upon the research presented
in this paper, the first thing to look at would probably be
to check how the accuracy of the method, when used on
the untranslated texts, changes with a growing dataset and
smaller sample sizes. Another avenue to explore is to test other
analyzing methods and pre-processing techniques to use on
the translated texts. One interesting approach would be to add
topic-specific features in English to the analysis to see how
this would affect the accuracy. Since many of the users in
the dataset also participate in English-language subreddits, it
would be interesting to see what the accuracy would be when
comparing the translated texts, to the texts written originally
in English by the same user.
R EFERENCES
[1] N. I. Council, “Assessing russian activities and intentions in recent us
elections,” tech. rep., Intelligence Community Assessment, Office of the
Director of National Intelligence, 2017.
[2] E. G. Sivertsen, L. Bjørgul, H. Lundberg, I. Endestad, T. Bornakke, J. B.
Kristensen, N. M. Christensen, and T. Albrechtsen, “Uønsket utenlandsk
påvirkning?–kartlegging og analyse av stortingsvalget 2021,” tech. rep.,
Forsvarets Forskningsinstitutt (FFI), 2022.
[3] “Brennpunkt: Skyggekrigen. documentary, 2023. norwegian broadcasting
corporation.” https://tv.nrk.no/serie/brennpunkt-skyggekrigen [Accessed:
22/08/2023].
[4] “Slik spres russisk propaganda i norske alterna-
tive medier.” https://www.faktisk.no/artikler/06epg/
slik-spres-russisk-propaganda-i-norske-alternative-medier [Accessed:
22/08/2023].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 535

 High school and university students' use of social
networks to support their self-education
D. Tran* and K. Kostolányová**
* University of Ostrava/Department of Information and Communication Technologies, Ostrava, Czech Republic
** University of Ostrava/Department of Information and Communication Technologies, Ostrava, Czech Republic
Daniel.tran@osu.cz, Katerina.kostolanyova@osu.cz

Abstract— Today's generation of high school and college includes the field of social networking, secondary and
students live in the digital age. Digital technologies are an higher education, and the use of social networking during
integral part of not only their personal but also their education. The empirical part of the paper is based on a
professional lives. The use of digital technologies during questionnaire survey, which aimed to find out how social
studies is almost a necessity. One of the key tools that networks are used by students during their education. In
students use is the internet. Here we can conceive of the the conclusion of the paper, the authors summarize the
Internet not only as a source of information, but also as a findings of the survey, discuss these findings, and describe
way of communication between students or teachers. A opportunities for follow-up research.
large part of the communication channels are social
networks. At first glance, social networks may seem more A. Social networks
like a means of entertainment. However, in today's The fundamental theoretical area that is the subject of
digitalized world, social networks are also penetrating
research in this paper is social networks. The concept of
education, and are often deliberately used to support the
social networks can be defined without a direct link to the
educational process. In this paper, the authors focus on the
Internet, so it is a broader concept than the one the authors
use of social networking by secondary and higher education
use in their research. In terms of sociology, we can define
students to support their education. In this paper, the
authors describe the theoretical background, where they
social networks as patterns of relationships between
mainly focus on the field of social networking, education in
actors. Social networks also influence the success or
secondary and higher education, and describe selected failure of an individual [2]. Within this paper, the authors
publications that focus on the relationship between social focus on social networks in the context of cyberspace. In
networking and education. The second part of the paper is this conception, the term social network is defined as a
based on a questionnaire survey, the results of which the web application that allows a user to create their profile
authors present in graphical form and comment verbally. within this system, create a list of people they are
The questionnaire survey investigated how students use connected to and share their content with, and in turn also
social networks during their self-education outside the display a list of people who share their content with that
school environment. In the conclusion of the paper, the individual [3]. Social networks within the Internet are one
authors discuss the results of the questionnaire survey, of the main ways of communication between individuals
describe the limitations of the research and possible ways to in cyberspace. Users interact with each other [4]. This
continue the research. communication and interaction often takes place between
users who do not know each other personally. Such
interactions between two or more individuals can be
I. INTRODUCTION referred to as "networking" [3].
Digital technologies are an integral part of modern Obviously, social networks have their positives and
society. If we talk about developed countries, we can say negatives that they bring to users. Studies to date have
that computers and mobile devices are already common shown that high school students perceive social
equipment in most families. These devices are practically networking mainly as a tool for recreational purposes and
dependent on their connectivity, which is mainly provided a way to connect with others. It is these two perspectives
by the Internet. The Internet ensures the compatibility of that are cited as benefits of social networking. Focusing
devices and communication between users. By
on the negatives of social networking, here students find,
communication, we mean the transmission of information
from the transmitting individual to the receiving for example, social isolation, health problems or
individual. Communication is based on a certain sign cyberbullying as risks associated with the use of social
system and rules for its use [1]. It is communication, networking sites [5].
whether real or in a virtual environment, that is an Existing studies show that today's children and
essential part of every individual's life. In connection with adolescents use social networks to maintain existing
the Internet and communication, we cannot forget social friendships. A study conducted in the Netherlands
networks, which form a large part of the communication confirms this fact. This study showed that 80% of the
channels on the Internet. In this publication, the authors research participants use social networks to maintain their
focus on social networks and the extent to which they are network of friends [6]. Currently, the total number of
used by secondary and higher education students during users of all social networks is 4.8 billion users [7].
their studies, especially in the context of their self- The most widespread social networks today include
education and self-study. In the first part of the paper, the Facebook, Instagram, Twitter, TikTok, YouTube and
authors describe the theoretical background, which many others, and today's market offers a really wide range

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 536

of social networks, with each network having its own their thought processes, or seeking help when they do not
specific characteristics and its own target group of users. understand something. These individuals perceive their
The table below presents an overview of the social ability to learn positively and also place a certain positive
networks that the authors work with in the research part of value on learning itself [12].
this paper. The table describes the number of users as of
2022, so it can be assumed that the number of users in the C. The use of social networks in education
current year is higher. The aforementioned prevalence of social networking
among the general population has caused the penetration
TABLE I. of this way of communication of Internet users into
OVERVIEW OF SOCIAL NETWORKS [8] education. Within the previous subsections, the authors
Social network name Number of users have outlined the theoretical area of social networking
and secondary and higher education. In the third
Facebook Almost 3 billion
subchapter, let us focus on the relationship between these
Youtube 2.5 billion areas. The literature search conducted found scholarly
Instagram 2 billion publications that describe the use of social networking
during education.
TikTok 1 billion One of these studies focuses on the use of Instagram
Snapchat 635 million and Twitter platforms to gamify learning. In this study,
the authors analyzed the keywords and hashtags that are
Twitter 556 million
most closely associated with gamification in education.
Pinterest 445 million These include the hashtags "education", "learning",
"edtech", "innovation". Thus, it is clear that there is
content on these two platforms that is focused on
education [13]. As part of the previous investigation, the
B. Secondary and higher education authors analysed other selected platforms and searched
By education we mean the acquisition of knowledge, for accounts that provide educational content and can
skills, attitudes and the development of the ability to use serve as a support tool for students' self-learning or as
them in action, behaviour, conduct and further education inspiration for teachers themselves. Based on this
of oneself and others [1]. Within the framework of this analysis, accounts and groups with educational content
paper, the authors work with the educational system of were also found on other platforms such as Facebook,
the Czech Republic. Secondary education is provided by Reddit and YouTube. The table below outlines the results
secondary schools. Individuals whose age is at least 15 of the analysis performed. The table provides a sample of
years participate in this education. The next level of accounts and groups on selected social networks that
education is higher vocational school and university focus on education.
studies. A specific feature of the above-mentioned levels
of education is the need for a higher degree of home TABLE II.
preparation of students compared to primary education. RESULTS OF THE ANALYSIS OF EDUCATIONAL SOCIAL MEDIA
Self-education refers to the process whereby an ACCOUNTS
individual, in the course of acquiring new knowledge or Account name Number of Account focus
Platform
skills, takes responsibility for planning, initiating and followers
carrying out the learning process. The concept of self- 1.4 milion Teaching
Pythonhub Instagram
Python
education can also be referred to as "self-instruction", 2 milion Teaching math
Iklogic_math Instagram
"self-education", "independent study", "individual study",
"self-teaching", "self-directed learning" [9]. The ability of 43,2 Advice for
Teachers Reddit
thousand teachers
self-education can be conceptualized as one of the 95 thousand Advice for
competencies that is necessary for successful study in Education Group Facebook
teachers
secondary and higher education [10]. Especially after the English with Lucy Youtube
9,45 milion Teaching
transformation of education caused by the pandemic English
51,7 Teaching
situation, students' self-education and preparation in thousand mathematics,
distance education is an essential part of today's Isibalocom Youtube chemistry,
educational process. Studies show that students have biology and
adapted to new conditions during distance education and physics
can adapt the time, place or mode of self-education to
their needs [11]. In particular, studying at university
requires a certain level of motivation for individuals to Social networking can be a useful tool for promoting
complete their studies. Motivation is all the more online learning and can contribute to improving learning
important during self-education. Students who are outcomes. For example, it can be used for student-to-
successful in self-directed learning are characterized by student communication, which is needed during group
effective control over their own learning in a variety of projects [14].
areas, including organizing their learning, monitoring

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 537

 However, the search conducted did not find any study school students, aged 16-20 years. The second group was
that addressed the use of social networking sites during individuals aged 21-25 years, where they were likely to
students' self-study. For this reason, the authors direct be university students, with 75% of all respondents being
their research to this issue. male.
II. METHODOLOGY TABLE III.
The main aim of the research was to find out which AGE OF RESPONDENTS
social networks students use to support their education. Age Number of
Beyond this main objective, the authors also focused on respondents
students' general use of social networks, i.e. not only for 10 – 15 3
educational purposes, and the extent to which students use
social networks while studying selected educational areas. 16 – 20 39
The research part of this paper is based on a questionnaire 21 – 25 17
survey. Gavora defines a questionnaire as a method of
asking questions in writing and obtaining written answers 26 – 30 1
[15]. The individual items of the questionnaire were
constructed based on the research conducted and the
stated research objective. The questionnaire was pilot
TABLE IV.
tested by giving it to a small sample of the target group. GENDER OF RESPONDENTS
Based on the results of the pilot testing, minor
modifications were made to the questionnaire. The target Gender Number of
group of the questionnaire survey was high school and respondents
university students. The respondents were selected Male 45
through random sampling and there were 60 respondents Female 14
in total. The questionnaire was distributed among students
of college of education and students of high school and Not specified 1
middle school. A total of approximately 300 individuals
were contacted. The questionnaire survey was completely
anonymous and no information was recorded about the
respondents that could be used to identify a specific TABLE V.
individual. SCHOOL STUDIED
The questionnaire was created in the Google Forms School Number of
online environment. It contained a total of 20 items of respondents
different types. The questionnaire can be divided into 3 Secondary vocational
37
parts. In the first part, the authors introduced themselves school
and informed the respondents about the purpose of the High school 1
questionnaire. The second part was designed to collect
College of Pedagogy 22
information about the respondents. The age, gender and
the level of education the respondent was studying were
collected. The third part was devoted to the social B. Access to social networks
networking sites themselves. The questionnaire was
distributed among the respondents through a link via In the next part of the questionnaire, the authors
email or by using a QR code in a face-to-face meeting. focused on data on how much time respondents spend on
The questionnaire was administered between April 1, social networks. The results show that the largest
2023 and June 30, 2023. After this date, the questionnaire proportion of respondents spend an average of 3-5 hours
was closed and no further responses could be sent. per day on social media (25 respondents), followed by 1-
2 hours per day (17 respondents) and 5-10 hours per day
III. RESULTS (15 respondents). A graphical representation of the results
The Results chapter presents the results of the is shown below.
questionnaire survey. As the questionnaire contains 20
questions, the authors focus on selected questions that
appear to be significant and interesting findings and
analyse these in detail, supplemented by graphs and
verbal commentary. The results of the other questions are
described only briefly.
A. Data on questionnaire respondents
The first part of the questions focused on obtaining
data about the respondents of the questionnaire. These
were age, gender and the level of education they are
currently studying. The tables below contain the
responses to these questions. From the tables it can be Figure 1. Responses to the question "On average, how many hours a
seen that the majority of respondents were secondary day do you spend on social media?"

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 538

 In terms of the device through which respondents most (34 respondents) and YouTube (24 respondents). The
often access social networks, mobile devices ranked first, results show that respondents mainly perceive platforms
with 45 respondents choosing this option. In second place that offer them multimedia content as a source of
is the desktop computer, used by 11 respondents, and in entertainment. These results were expected. The results of
third place is the laptop, chosen by 4 respondents. Based the other platforms are shown in Figure 4.
on this data, there is an opportunity for connecting the
fields of mobile learning and microlearning. The authors
describe this method of delivering learning to students for
their self-study in Chapter IV. Discussion.

Figure 4. Responses to the question "Which social networks do you

perceive as a source of entertainment?"

D. Use of social networks during education

Figure 2. Responses to the question "Which device do you use most
often to access social networks?" Focusing on what social networks respondents use
during their education, the questionnaire revealed some
interesting information. When asked what social network
C. Social networks used by respondents
respondents consider the most appropriate source of new
When asked what social networks respondents use, the information, the most respondents chose the YouTube
answers were similar to what the authors expected. Based platform (14 respondents). Considering the answers to the
on their literature search and personal experience, the above question, where YouTube is perceived as a source
authors predicted that the most widely used social of entertainment, this finding is quite interesting. There is
networks would include Facebook, Instagram, YouTube, no doubt that the YouTube platform contains a variety of
and TikTok. Respondents could choose multiple answers. videos, which can often be high quality educational
The most used social networks include Instagram (58 materials created by experts. In second place is Instagram
respondents), YouTube (53 respondents). A rather (11 respondents) and in third place is the 'none of the
surprising finding was the use of the Discord platform (44 above' option (10 respondents). The authors of the paper
respondents). This was followed by Facebook (42 find it positive that 16.7% of respondents do not consider
respondents), TikTok (41 respondents) and then Reddit, social networks as a key source of new information and
Pinterest and Snapchat, for example. Outside of the preset are likely to seek information from expert or verified
answers, respondents added Teamspeak or Messenger. sources. Interesting responses include ChatGPT, which
Detailed results are shown in Figure 3. was completed by 1 respondent. While ChatGPT is not
classified as a social network, it is certainly a tool that has
had a significant impact on education today. The overall
results are shown in Figure 5.

Figure 3. Responses to the question "What social networks do you

use?"

When asked what social networks they perceive as a

Figure 5. Responses to the question "Which social network do you
source of entertainment, the most respondents chose the consider to be the most suitable source for getting new information?"
TikTok platform (43 respondents), followed by Instagram

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 539

 Focusing on the study itself, the authors of the paper
asked respondents what social networks help them during
their studies (sources of information, communication with
classmates/teachers, inspiration and motivation, etc.).
Similar to several previous questions, YouTube took the
first place, chosen by 34 respondents. The second most
popular platform was Discord (24 respondents), which is a
surprising finding. It can be assumed that this platform is
mainly used for communication between students or for
communication with the teacher. Based on the literature
search and the author's personal experience, some teachers
use the Discord platform when they create forums for their
classes [16]. Furthermore, the respondents chose
Instagram (23 respondents), which can be used both to
Figure 7. Scale rating of the statement "I use social networks to
draw new information and to communicate. The graph improve my English."
below (Figure 6) shows all the responses to this question.
Other questions asked about the use of social
networking during self-education in other areas, including
mathematics, history, music, science and humanities.
However, for all of the areas mentioned, the majority of
respondents disagreed with the statements, i.e., they do not
use social networking to improve in that area.
F. Use of social networks during exam preparation
Respondents rated their agreement with the statement "I
use social networks to help me prepare for exams (final
exams, university exams, state final exams)" using a scale.
The results show that a larger third of respondents agree
with the statement, with 9 respondents strongly agreeing
and 18 respondents somewhat agreeing. Furthermore, 18
respondents take a neutral position and 15 respondents
tend to disagree with the statement. The results are shown
Figure 6. Responses to the question "Which social networks help you
in any way during your studies?"
graphically in the chart below.

E. Use of social networking sites during self-education

in English
In two questions, the authors focus on the respondents'
use of social networking sites during their self-education
in English.
The first question focused on the language in which
respondents consume social media content. The majority
of respondents (36 respondents) mostly watch content in
English, 23 respondents watch content in Czech and 1
respondent did not answer. These data suggest that some
respondents also improve their English unintentionally in
their free time, when they may use social networking
sites for entertainment. Figure 8. Scale rating of the statement "I use social networks to help
me prepare for exams (final exams, university exams, state final
The second question was based on a scaled rating of exams)."
agreement with the statement "I use social networking
sites to improve my English". Respondents rated on a
five-point scale from 1 (strongly agree) to 5 (strongly IV. DISCUSSION
disagree). The results show that the majority of In the Results chapter the authors present the results of
respondents use social networking sites to improve their the questionnaire survey. The results show that the
English, with 22 respondents strongly agreeing with the majority of respondents were high school students aged
statement, 14 respondents somewhat agreeing, and 16 16-20, followed by university students aged 21-25. They
respondents having a neutral attitude (Figure 5). were predominantly male. In general, the most used
social network among the respondents is Instagram. An
interesting finding is that the third most used platform is
Discord, which is mainly used for communication. As the
results of other questions showed, Discord is widely used
by respondents as a support during their studies, where it

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 540

can be assumed that it is mainly for communication with networks can have on education. These negatives may
their classmates or even teachers. From the authors' include distraction of an individual's attention, poor
personal experience, some teachers use Discord to create quality of resources and information on social networks.
a virtual classroom and to support students' homework. In Directing students to use social networking sites more
terms of education, respondents perceive YouTube and frequently, albeit for educational purposes, may also
Instagram platforms as the most suitable social network encourage social networking addiction. At the same time,
for learning new information. From a general perspective, students' digital literacy, particularly in the area of online
respondents use YouTube and Discord most often to safety, also needs to be taken into account. Whether
support their studies. When focusing on the educational social networks are a suitable tool to support the
areas in which respondents use social networks for their education of individuals, despite the negatives mentioned,
studies, it is mainly the English language, with the is something that ongoing research can show.
majority of respondents consuming social network
content in English. Respondents tend not to use social
networks for self-study in other areas (mathematics, REFERENCES
music, history, etc.). It is clear from the results of the [1] Z. Kolář, Interpretive dictionary of pedagogy: 583 selected
entries. Prague: Grada, 2012.
questionnaire survey that students use social networks not
[2] J. A. Fuhse, “Theorizing social networks: the relational
only as a source of entertainment but also as a support for sociology of and around Harrison White,” in International Review of
their studies or self-study, either as a means of Sociology, vol. 25, 2014, pp. 15-44.
communication or as a source of new information, [3] D. M. Boyd and N. B. Ellison, “Social Network Sites:
inspiration or as a support for the development of their Definition, History, and Scholarship,” in Journal of Computer-Mediated
skills. Further results from the questionnaire show that Communication, vol. 13, 2007, pp. 210-230.
[4] K. Subrahmanyam, S. M. Reich, N. Waechter, and G.
42% of respondents spend an average of 3-5 hours per Espinoza, “Online and offline social networks: Use of social networking
day on social networks. Based on this figure, it seems sites by emerging adults,” in Journal of Applied Developmental
sensible for teachers to use social networks to support Psychology, vol. 29, 2008, pp. 420-433.
students' homework and self-study. The questionnaire [5] F. Mulisa and D. A. Getahun, “Perceived Benefits and Risks
also showed that most respondents access social networks of Social Media: Ethiopian Secondary School Students’ Perspectives,”
in Journal of Technology in Behavioral Science, vol. 3, 2018, pp. 294-
through mobile devices. Here there appears to be scope 300.
for using the principles of microlearning, i.e. presenting [6] P. M. Valkenburg and J. Peter, “Preadolescents' and
educational content to students in short activities [17]. adolescents' online communication and their closeness to friends,”
Teachers can create short learning objects that can be in Developmental Psychology, vol. 43, 2007, pp. 267-277.
distributed to students through a social network and they [7] A. Petrosyan, “Number of internet and social media users
worldwide as of April 2023”, Statista, 2023.
can study them through a mobile phone, regardless of
[8] “Statista”, 2023.
location or time.
[9] G. Manning, “Self-Directed Learning: A Key Component of
The limitations of the research are mainly due to the Adult Learning Theory,” in Journal of the Washington Institute of
limited research sample, where it would have been China Studies, vol. 2, 2007, pp. 104-115.
advisable for a larger number of respondents to complete [10] R. R. Sagitova and N. F. Khasanova, “THE FORMATION
the questionnaire. Another limiting factor is that a larger OF SELF-EDUCATIONAL COMPETENCE OF UNIVERSITY
STUDENTS IN THE PROCESS OF FOREIGN LANGUAGE
proportion of the respondents were secondary school LEARNING,” in The Turkish Online Journal of Design, Art and
students. Communication, 2016, pp. 2525-2529.
Continued research could focus on expanding the [11] Z. Turan, S. Kucuk, and S. Cilligol Karabey, “The university
research sample and balancing the ratio between high students’ self-regulated effort, flexibility and satisfaction in distance
education,” in International Journal of Educational Technology in
school and university students. Subsequently, the authors Higher Education, vol. 19, 2022.
consider proposing a methodological approach for using [12] A. R. Artino and J. M. Stephens, “Academic motivation and
specific social networks to support students' home self-regulation: A comparative analysis of undergraduate and graduate
preparation and self-study. The proposed procedure may students learning online,” in The Internet and Higher Education, vol.
benefit from the application of microlearning and mobile 12, 2009, pp. 146-151.
learning principles. [13] L. Pilař, S. Rojík, K. Tučková, R. Selby, and T. Balcarová,
“GAMIFICATION IN EDUCATION: SOCIAL NETWORK
Social networks are an integral part of the cyber world ANALYSIS,” in Efficiency and Responsibility in Education 2017, 2017,
and, especially for the younger generation, they form an pp. 318-324.
important tool for communication and interaction of an [14] A. Varol and N. Ahmed, “Social networks' role in online
individual with the environment. As social networks are education”, in 2013 7th International Conference on Application of
Information and Communication Technologies, 2013, pp. 1-4.
already a natural part of the lives of today's students, they
[15] M. Chráska, Educational research methods: basics of
can be a useful tool to support their education. Among the quantitative research, 2nd updated edition, Prague: Grada, 2016.
undeniable positives that social networks offer is the easy [16] M. L. Arifianto and I. F. Izzudin, “Students’ Acceptance of
availability of information, with social networks Discord as an Alternative Online Learning Media,” in International
presenting up-to-date information in a short time. Journal of Emerging Technologies in Learning (iJET), vol. 16, 2021,
pp. 179-195.
Furthermore, the possibility to communicate and
[17] R. P. Singh, “17 Awesome resources on micro-
collaborate with a large number of other people and a learning”, ELearning Industry, 2014.
wide range of diverse content. It is also necessary to
consider the possible negative impacts that social

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 541

 LoRa™ Lab: Laboratory Network for Educational
Purposes
A. Valach, L.Zemko, P. Čičák and K. Jelemenská
Faculty of Informatics and Information Technologies
Slovak University of Technology
Bratislava, Slovakia
{alexander.valach, ladislav.zemko, pavel.cicak, katarina.jelemenska}@stuba.sk

Abstract—The Internet of Things describes scenarios in which • This paper provides a detailed overview of LoRa™ and
Internet connectivity and computing capabilities are extended LoRaWAN.
to everyday objects. Those devices have limited computational • Different LoRaWAN network types are described, and the
power and are often battery-powered or energy-harvesting. Many
communication technologies specifically designed for the Internet demand for a private network is explained.
of Things environment have emerged. Low-Power Wide-Area • Currently performed research directions are briefly dis-
networks are popular as they provide a long communication cussed and private LoRa™ laboratory network built on
range and are energy efficient. Despite many advantages, the an open-source software stack is described in detail.
Internet of Things faces several challenges, which need to be ad- • The paper provides a comprehensive look at a fully
dressed shortly. To provide research opportunities and additional
training, it is important to have a dedicated network, which is functional private network, including a complete software
flexible enough, and thus allows modifications and network stack and hardware stack, which can serve as a base for other
manipulation. In this paper, the Low-Power Wide-Area Networks researchers.
are explored, with the main focus on LoRa™ technology. Next, The paper is organized as follows. Section II introduces
current research performed at the Faculty of Informatics and
Information Technologies is presented. And finally, a private Low-Power Wide-Area Networks, which are popular in the
LoRa™ laboratory network is described in detail. Internet of Things environment. LoRa™ and its subsequent
hardware and software components are described in detail in
Keywords—ChirpStack, LoRa, LoRaWAN, LPWAN, labora- section III. Section V focuses on a private LoRa™ laboratory
tory network, completely describes the hardware and software
stack, and provides a brief overview of the research performed.
I. I NTRODUCTION
Future work is discussed in section VI. Conclusions are given
The term Internet of Things (IoT) is mainly used to describe in section VII.
scenarios in which the Internet connection and computing
capabilities extend to devices, sensors, and everyday objects II. L OW-P OWER W IDE A REA N ETWORKS
[1]. Many communication technologies providing a wireless The Internet of Things (IoT) does not impose restrictions
connection have emerged. These technologies differ in terms on the specific technology used to connect end devices to
of speed, range, used frequencies, medium access techniques, the Internet. Due to the nature of IoT, its applications, and
and energy requirements [2]. services, the only suitable solution is often the use of wireless
At the Faculty of Informatics and Information Technologies, communication. To overcome the limitations of end devices,
we try to address several current challenges associated with the it is usually necessary to use protocol stacks specifically
IoT, including improving network performance and scalability, designed for use in the IoT. In the case of embedded systems
vulnerability assessment, network and end node monitoring, used in IoT are characteristic of low power consumption, low
geolocation of end nodes, computational intelligence utiliza- computing power, small size, and low price, compared to other
tion, and LoRa™ utilization in intelligent vehicular infrastruc- electronic systems [3, 4].
ture. Communication technologies are most often classified based
For research purposes, a private LoRa™ infrastructure was on 3 parameters - communication range, bandwidth, and
built, mainly focused on software stack flexibility. To provide energy efficiency [5].
research opportunities and training for students and industry, Low-Power Wide Area Networks (LPWANs) represent a
the ability to modify the existing components and manipulate category of Wide Area Networks. They combine low data rates
the network stack is important. The currently used software with robust modulation, which enables them to communicate
stack is completely open-sourced, while the gateways and end over distances of several kilometers. In general, it can be
nodes are based on prototyping boards, equipped with LoRa™ argued, that LPWANs are used in cases where it is not
radio transceivers. possible to use other wireless technologies, such as Bluetooth,
The main contributions of this paper are: Bluetooth Low Energy, WiFi, or ZigBee, as they do not

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 542

provide communication over long distances. On the other SF is a code, which makes it able to handle subsequent
hand, it is possible to use them in the domains of smart cities, transmissions not only at the same time but also on the same
measuring devices, street lighting, and health care, but also in channel. This feature is called orthogonality, thus LoRa™
agriculture [6, 3, 7, 8, 9]. can be considered orthogonal or at least quasi-orthogonal.
This type of network is mainly suitable for connecting Increasing the SF results in increased communication range,
devices that need to send and receive only a small amount transmission duration, and receiver sensitivity. On the other
of data over a long distance and at the same time require hand, it decreases the transmission rate and battery life.
low power consumption. The long communication range is Decreasing has the opposite effect [4, 17, 18, 19]. Based on
possible due to the physical layer and the high sensitivity of the spreading factor and the available bandwidth, it is possible
the receiver, which is approximately -150 dBm. Because this to calculate the symbol length ts according to the equation (2)
type of network is characterized by low transfer rates, it is [15, 14, 16].
not suitable for data-intensive use cases. Compared to short- SF SF
range technologies, the transmission rate is significantly lower. ts = = (2)
BW fmax − f min
Instead, LPWANs are suitable for purposes of monitoring,
measuring, or controlling devices. Due to the requirements Equation (3) expresses the bit rate (Rb ), symbol period (Ts ),
and characteristics of IoT networks, it may also be neces- symbol rate (Rs ), and chip rate (Cs ) of LoRa™ modulation.
sary to use specialized application layer protocols, or service 1
Rb = SF ∗ 2SF
b/s
discovery protocols that would be efficient in terms of energy
BW
consumption and bandwidth usage, e.g. MQTT, CoAP, mDNS,
2SF
DNS-SD, uBonjour [10, 3, 11]. Ts = s (3)
BW
LPWANs are characterized mainly by the following prop- 1 BW
erties: Rs = = SF symbols / s
TS 2
• long communication range, Rc = Rs ∗ 2SF chips / s
• use of sub-GHz spectrum,
• low transmission rate, LoRa PHY also defines the physical frame format used
• low cost [10, 12]. to send data between transmitter and receiver. Frames are
composed of preamble, header, header checksum, transmitted
III. L O R A™ data, and checksum. All values are filled by the LoRa™ chip.
LoRa™ communication itself takes place in communication
LoRa™ is a proprietary solution developed by Semtech. between two LoRa™ devices, or between LoRa™ device and
In means of the International Organization for Standardiza- Gateway [4, 18, 13].
tion/Open Systems Interconnection (ISO/OSI) reference model
LoRa™ can be considered a physical layer, or a modulation B. LoRa MAC
scheme, that specifies the way data are transmitted over the The LoRa MAC layer controls access to the medium. Proto-
air. Subsequently, a Medium Access Control (MAC) protocol cols operating on this layer can be considered as link protocols,
operates on the link layer [13]. It prioritizes receiver sensitivity which use the LoRa PHY for data transmission. Together
over transmission rate. Furthermore, variable transmission rate with LoRa PHY, the LoRaWAN protocol is most often used,
technology is used, based on orthogonal spreading factors which is open source compared to LoRa™ . LoRaWAN is the
(SF), which makes it possible to optimize network perfor- Aloha protocol, which is controlled by a network server. The
mance at a constant bandwidth. This mechanism is referred to common role of MAC protocols is to facilitate the integration
as an Adaptive Data Rate (ADR) and it enables to prioritize of existing protocols with LoRa™ [14, 13, 10, 20]. Fig. 1 shows
coverage, throughput, robustness, or energy consumption [14, the structure of the LoRaWAN packet.
15, 12, 10].
IV. L O R AWAN
A. LoRa PHY LoRaWAN defines frame format and allows LoRaWAN
LoRa PHY or simply LoRa™ is a modulation that is a devices to transmit data to a LoRaWAN server. It enables
derivate of Chirp Spread Spectrum (CSS). The modulation bi-directional communication, while the uplink is strongly
itself is based on the generation of a chirp signal that continu- favored. The protocol is also optimized for use with battery-
ously changes its frequency. The frequency inreases linearly in powered devices, both mobile and stationary. The transmission
time, passing from f0 to fmax , and then continuing from fmin reliability is ensured through the reception acknowledgment.
to f0 , where f0 represents the initial, fmin the minimum, and Based on the FType field located in the MAC header, it
fmax the maximum frequency. The initial frequency is based can differentiate between Join-Request, Join-Accept, and Data
on the spreading factor, according to the equation (1), where frames. Join-Request and Join-Accept frames are used dur-
SF represents a spreading factor [15, 14, 16]. ing the over-the-air activation (OTAA) procedure, while data
frames transfer MAC commands and application data [4, 18,
f0 = log2 (SF ), (1) 6, 13, 21].

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 543

 6, 13, 3]. In [4] the association of the network server and the
application server is called the LoRaWAN server.
B. Device Classes
LoRaWAN devices are classified into 3 categories - A, B,
or C - based on their capabilities, i.e. power consumption
and downlink accessibility. All available categories support bi-
directional communication but differ in the nature of downlink
communication. LoRaWAN network can consist of different
device classes. At the same time, all devices must support
Class A [4, 21, 6]. Differences between individual device
classes, especially receive windows, can be seen in Fig. 3.
1) Class A (All): Every device operates as a Class A device
after immediate connection to the LoRaWAN network. Class
Fig. 1. LoRaWAN packet
A also supports bi-directional communication, but downlink
is followed only after a successful uplink. The device must
LoRaWAN describes the protocol itself, device classes, open at least 1 receive window - RX1. If no downlink
frame format, MAC commands, and activation modes, while communication is received within the RX1 window, it is
the regional parameters describe parameters for each region necessary to open the second receive window - RX2. Class
(i.e. EU868, EU433, US915, etc.), like channel frequencies, A is thus suitable for use when there is sufficient downlink
data rate, output power, or maximum payload size [4]. shortly after uplink. At the same time, Class A devices have
the lowest energy consumption [6, 21].
A. Architecture 2) Class B (Beacon): Class B devices support additional
LoRaWAN network consists of end nodes, which mea- receive windows, which are synchronized. It is therefore
sure and transmit data. The basic architecture also consists suitable for scenarios that require downlink independent from
of LoRa™ Gateways, which bridge communication between uplink communication. Devices are synchronized through syn-
LoRa™ and IP network, and the network server, which further chronization beacons sent at regular intervals by the gateway.
processes the data. The architecture often consists of some IoT Class B devices also operate as Class A device during start
platform, which the user interacts with. This kind of topology and network connection. The device first waits for a synchro-
is called the star-of-stars. End nodes send data to one or more nization beacon. After synchronization, it then sets the Class
gateways through single-hop connections. Individual gateways B bit to 1, signaling to the network server the device has
are connected to a single centralized server [4, 18, 3, 15]. enabled Class B. To preserve the setting, the device leaves the
1) End Nodes: End nodes are embedded systems, charac- bit set to 1 during all uplink communications. Class B devices
terized by low power consumption, small size, and low cost. can be thus reached regularly without prior uplink, but at the
Each node is equipped with LoRa™ radio transceiver to be drawback of increased power consumption [6, 21, 4].
able to communicate with LoRa™ gateway. End nodes are not 3) Class C (Continuous): Class C devices are the least
associated with specific gateways, rather each gateway in range energy-saving, but they provide the lowest latency at the same
receives the message and forwards it to the server [4, 18, 6, time. In the case of Class C, the receive windows are con-
13, 15]. tinuously opened, unless the device is currently transmitting,
2) Gateways: Gateway serves as a bridge between LoRa™ or one of the Class A receive windows, i.e. RX1 or RX2,
and IP network. Gateway receives LoRa™ communication, is opened. If there is an overlap between Class A receive
adds information regarding the quality of the received signal, windows - RX1 or RX2 - and Class C receive window - RXC
and encapsulates the LoRaWAN packet into a more standard - Class A has higher priority, and RXC demodulation will be
IP protocol. It is possible to cover a large area using multiple stopped. Class B device cannot be a Class C device at the
gateways. All gateways that successfully decode the message same time and vice versa [6, 21].
send the message to the LoRaWAN server. Gateway listens on
all channels, and on all Spreading Factors at the same time C. Network Types
[4, 18, 6, 13, 3]. In the LoRaWAN network, we distinguish 3 basic network
3) Network Server: The network server performs filtering, types, namely public, private, and hybrid. These types differ
deduplication, and authentication of received messages. The mainly in the area of network infrastructure ownership, ad-
network server also serves the role of a network controller, and ministrative load, relative limitations, fees, and obligations.
thus further decides how the received data will be processed. 1) Public Operator Network: In public operator networks,
Multiple application servers providing additional services can the operator manages the gateways and the LoRaWAN server,
be connected to the network server. A network server can while the user is responsible for end nodes (LoRaWAN
provide multiple Application Programming Interfaces (API) devices) and user application. Public operators provide nation-
through which selected information can be obtained [22, 18, wide networks to connect devices. To connect IoT devices,

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 544

 Fig. 2. LoRaWAN Architecture

is the ChirpStack, an open-source LoRaWAN server, which

provides a web interface for management purposes. It also
enables integration with third-party service providers. Another
possibility is to integrate with or extend the ChirpStack itself
via Google Remote Procedure Call (gRPC) API [4, 23].
3) Hybrid Network: An alternative between public operator
and private networks is the hybrid network. In this type of
network, the user owns and manages gateways, while the
LoRaWAN server has the form of a cloud-hosted solution.
The advantage is that the user can manage the network
coverage. Cloud-hosted LoRaWAN solutions often provide
a free subscription with a limited number of gateways and
devices [4].
V. L O R A™ L AB
To provide research opportunities and additional training
for students and industry, it is important, for the researchers
or developers, to be able to modify the existing components
and manipulate the network stack. According to this, we have
Fig. 3. LoRaWAN Device Classes
decided to build our own LoRa™ private network. Another
reason for building our own educational and research network
the user has to subscribe to at least one plan. Individual plans is to create new datasets and make them publicly available as
differ in terms of uplink/downlink message amount and, of there are only a few datasets available [24, 25, 26].
course, price [4]. In this section, we introduce our proposed solution for
2) Private Network: The second possible option is to build implementing LoRa™ Laboratory Network for Educational
a private LoRaWAN network. In this case, it is first needed to and Research purposes. Our main goal is to utilize this network
build the complete LoRaWAN infrastructure. In addition to the to further support the training and research in the following
investments associated with building the LoRaWAN network, areas:
it is necessary to invest considerable effort in managing the 1) Improving network performance. By implementing
network itself [4]. communication planning and evaluating the utilized
In addition to building a hardware network topology, it is methods and optimization of network delivery, lowering
also necessary to select the specific software for the whole the overhead and utilizing Quality of Service concepts.
hardware stack, which will best suit the owner’s expectations 2) Geolocation of devices. Geolocation in LoRa networks
and requirements. Some software is available in the form is currently possible with lower accuracy and lower en-
of the so-called license on-premise, where the license needs ergy consumption compared to the General Positioning
to be bought first, and then it can be installed. Another System (GPS). To further support the research in this
possibility is to use an open-source software stack. An example field, different methods have to be evaluated.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 545

 3) Utilization of computational intelligence. One of the 3) 868 MHz LoRa™ antenna with pigtail.
current challenges in IoT networks is to provide scalable 4) Raspberry Pi power adapter 5V 3A.
solutions and intelligent data processing and evaluation.
4) Vulnerability assessment. With the increasing number We have used a Raspberry Pi microcomputer as it runs a
of emerging security threats, it is mandatory to perform a modified version of Linux Debian called Raspberry Pi OS.
vulnerability assessment of IoT devices regularly. When Instead of using a dedicated firmware or read-only file system
combined with geolocation, it is possible to locate the (as is the case with commercial LoRa™ gateways), it enables
devices executing jamming attacks to at least estimate running Raspberry Pi OS and modifying the LoRa™ daemon
their position. running on the top of the operating system. This allows the
5) Network and device monitoring. It is possible to mon- developers to easily modify LoRa daemons with minimal
itor existing network devices using different technolo- service disruption (no need for restart) and utilize different
gies, e.g., OSSEC framework for Endpoint Detection LoRa™ daemons, e.g., running LoRaWAN and LoRa@FIIT
and Response (EDR) and Syslog protocol for Security software to switch protocols more easily. An example gateway
Information and Event Management (SIEM), as well as is shown in Fig. 6.
to monitor network traffic using TinyIPFIX for LoRa™ Each gateway supports the following functionalities:
end devices.
6) Utilization of LoRa technology in other areas. It is 1) Remote management using reverse SSH tunnel. Each
possible to utilize the capabilities of our network in gateway can communicate with public server lora.fiit.
cooperation with the Automotive and Innovation Lab at stuba.sk using port 22 to create a reverse SSH tunnel
the Faculty of Informatics and Information Technologies for remote management events when Raspberry Pi de-
to further support the development of intelligent sensors vices do not have public IP addresses. Authentication is
and communication architectures within vehicles. Coop- provided using the SSH keys.
eration is important to achieve certain application goals. 2) Ability to run different LoRa™ daemons. Each
Due to the multi-tenancy of the ChirpStack solution, LoRa™ daemon is installed as a system service and can
other entities can use our LoRa™ network. be disabled or enabled. This functionality supports the
These areas are very important for research and training as idea of switching or evaluating different LoRa™ MAC
they belong to the current challenges for the LPWANs. None protocols, their modifications, or different LoRaWAN
of the areas would be possible to explore without a proper and stack implementations, as soon as the solution is open-
private LoRa™ laboratory network. source or available as a system service.
3) Ability to run solely on power banks for short-term
A. Architecture experiments. If there is a need to move the gateway
One of the key challenges was to select the proper com- to perform additional measurements it is prepared to
ponents to build the laboratory environment that provides the connect to the mobile hotspot and send data outside of
necessary functionalities for research and training in the areas the faculty network.
described above. 4) Monitoring of LoRa gateways. Each gateway sends the
The architecture is presented in Fig. 4 and consists of system and audit logs to the Security Information and
multiple components, which are described in the following Event Management (SIEM). ElasticSearch and Kibana
sections. are used as SIEM. This functionality enhances the
1) End Nodes: We utilized two different versions of the security of LoRa™ gateways and provides an additional
LoRa end node depending on the application’s needs. If the opportunity for the standardization of Syslog messages
low-power scenario is required (e.g., Channel Activity Detec- and events for LoRa™ devices. However, this function-
tion assessment), we use LoRa™ Radio Node v1.0 with an ality currently applies only to gateways inside LoRa™
8-bit ATmega328P processor and HopeRF RFM96W LoRa™ Lab and no external connections to SIEM are currently
transceiver module. If the scenario with higher computational possible.
power is to be executed, and additional components, e.g.,
WiFi, Bluetooth Low Energy, or OLED display are beneficial 3) Network and Application Server: ChirpStack is used
(e.g., fast prototyping for vulnerability assessment and net- for network and application servers. It was developed to use
work monitoring), we use LilyGO TTGO LoRa32 T3 V1.6 LoRaWAN networks and is fully open-sourced. ChirpStack
868 MHz 0.96” SMA prototyping board, based on the ESP32 provides a web application for the management of devices
microcontroller. The prototyping board is shown in Fig. 5. and gateways and supports multi-tenancy. In addition, it has
2) Gateways: We built 8 LoRa™ gateways (also called an interface for the integration of third-party services and
Access Points). Each gateway consists of the following com- common cloud providers. All data are stored in a PostgreSQL
ponents: database. Furthermore, a gRPC-based API is provided to
1) Raspberry Pi 3 model B+ or Raspberry Pi 4. integrate or extend ChirpStack functionality [23]. Application
2) iC880a LoRa™ concentrator 868 MHz with SX1278 and network servers are currently deployed in the faculty
LoRa™ module. cloud.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 546

 Fig. 4. The Architecture of Laboratory Network

Fig. 5. LilyGO TTGO End Node Fig. 6. LoRa™ Gateway

VI. F UTURE W ORK many of the datasets are provided and none of them compare
LoRaWAN and GPS geolocation.
We have utilized the existing network to form a LoRa™ In the future, we plan to fully implement and use firmware
geolocation dataset that is currently being processed and update functionality not only to use the latest firmware pos-
modified to be publicly available online. It is important to sible but also to be able to remotely replace the firmware of
perform feature engineering on existing LoRa™ datasets not devices dedicated to vulnerability assessment.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 547

 In the future, we would like to vastly improve our network ACKNOWLEDGMENT
to be able to send logs to SIEM and perform vulnerability The contribution was created within the national project
assessments regularly. Additional goals are to make it more “Increasing Slovakia’s resilience to hybrid threats by
reliable and to 3D model and print custom cases for gateways strengthening public administration capacities”, project code
to be able to deploy them outdoors to make our experiments ITMS2014+:314011CDW7. This project is supported by the
more realistic. European Social Fund. This publication has been written
Additionally, we would like to create a short course focusing thanks to the support of the Operational Programme Integrated
on the introduction and hands-on labs of LoRaWAN networks Infrastructure for the project: Advancing University Capacity
to educate students and provide training scenarios. and Competence in Research, Development and Innovation
VII. C ONCLUSION (ACCORD) (ITMS code: 313021X329), co-funded by the
European Regional Development Fund (ERDF). This work
The Internet of Things gained significant popularity in was supported by Cultural and Educational Grant Agency
recent years and it surrounds our everyday lives. Also, many (KEGA) of the Ministry of Education, Science, Research and
different communication technologies have emerged, each Sport of the Slovak Republic under the project No. 026TUKE-
with its pros and cons. Due to the limitations of the end nodes, 4/2021. The authors would also like to thank for financial
it is usually necessary to use specific protocol stacks. Low- contribution from the STU Grant scheme for Support of Young
Power Wide-Area Networks are very popular, as they provide Researchers.
long communication range and power efficiency. Probably
the most successful and widely accepted solution is LoRa™ , R EFERENCES
which is based on a Chirp Spread Spectrum modulation. It [1] Karen Rose, Scott D. Eldridge, and Lyman Chapin.
can be considered as a modulation scheme, or in terms of “THE INTERNET OF THINGS : AN OVERVIEW
the International Standardization Organization/Open Systems Understanding the Issues and Challenges of a More
Interconnection reference model a physical layer. A network Connected World”. In: 2015.
protocol operates at the link layer, typically LoRaWAN is used. [2] Jakub Pullmann and Dominik Macko. “A New
This protocol controls medium access. But it is not the only Planning-Based Collision-Prevention Mechanism in
suitable protocol. Long-Range IoT Networks”. In: IEEE Internet of
Generally, it is possible to distinguish between 3 basic Things Journal 6.6 (2019), pp. 9439–9446. DOI: 10 .
network types - public operator, private, and hybrid networks. 1109/JIOT.2019.2940994.
They differ in terms of infrastructure ownership, administrative [3] Marco Centenaro et al. “Long-range communications in
load, limitations, fees, and obligations. unlicensed bands: the rising stars in the IoT and smart
Despite advantages, IoT and LoRa™ as well, face several city scenarios”. In: IEEE Wireless Communications 23.5
challenges. At the Faculty of Informatics and Information (2016), pp. 60–67. DOI: 10.1109/MWC.2016.7721743.
Technologies, we perform research covering at least network [4] Sylvain Montagny. LoRa™ LoRaWAN™ and Internet of
performance and scalability improvement, vulnerability assess- Things for beginners. Accessed: 24-2-2023.
ment, security monitoring, geolocation, and utilization in smart [5] Peter Kaňuch, Dominik Macko, and Ladislav Hudec.
vehicular infrastructure. None of the listed areas would be “Survey: Classification of the IoT Technologies for
possible to explore without a proper dedicated network. Better Selection to Real Use”. In: 2020 43rd Interna-
To provide research opportunities a private network with tional Conference on Telecommunications and Signal
multi-tenancy support was built, based on the open-source Processing (TSP). 2020, pp. 500–505. DOI: 10.1109/
software stack and mostly prototyping hardware. This way it TSP49548.2020.9163480.
is possible to ensure the required software stack flexibility, as [6] Ferran Adelantado et al. “Understanding the Limits
there is no specific limitation of the software stack, compared of LoRaWAN”. In: IEEE Communications Magazine
to public operator or hybrid networks. 55.9 (2017), pp. 34–40. DOI: 10.1109/MCOM.2017.
The contribution of this paper is mainly a detailed overview 1600613.
of a fully functional private LoRa™ laboratory network build, [7] A. Valach and D. Macko. “Exploration of the LoRa
which can serve as a base for other researchers. Laboratory in- Technology Utilization Possibilities in Healthcare IoT
frastructure development is a continuous and challenging task Devices”. In: 2018 16th International Conference on
as the focus needs to be on all of the components. A Network Emerging eLearning Technologies and Applications
server, Application server, and web interface are provided by (ICETA). 2018, pp. 623–628. DOI: 10 . 1109 / ICETA .
the open-source ChirpStack solution. Gateways are based on a 2018.8572032.
Raspberry Pi microcomputer equipped with an IMST iC880A- [8] Andrea Zanella et al. “Internet of Things for Smart
SPI concentrator. LoRa™ -compatible prototyping boards based Cities”. In: IEEE Internet of Things Journal 1.1 (2014),
on ATmega328P processor or ESP32 microcontroller, each pp. 22–32. DOI: 10.1109/JIOT.2014.2306328.
equipped with an appropriate LoRa™ module, are used as end
nodes. This paper can thus serve as a brief manual about the
possibilities of building a private LoRa™ network.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 548

 [9] Kieu-Ha Phung et al. “Analysis and assessment of [22] Adwait Dongare et al. “OpenChirp: A Low-Power
LoRaWAN”. In: 2018 2nd International Conference on Wide-Area Networking architecture”. In: 2017 IEEE
Recent Advances in Signal Processing, Telecommunica- International Conference on Pervasive Computing
tions Computing (SigTelCom). 2018, pp. 241–246. DOI: and Communications Workshops (PerCom Workshops).
10.1109/SIGTELCOM.2018.8325799. 2017, pp. 569–574. DOI: 10.1109/PERCOMW.2017.
[10] Link Labs, Inc. A Comprehensive Look at Low Power, 7917625.
Wide Area Networks. https://cdn2.hubspot.net/hubfs/ [23] ChirpStack, open-source LoRaWAN™ Network Server.
427771/LPWAN-Brochure-Interactive.pdf. 2016. The ChirpStack project. https://www.chirpstack.io.
[11] Iulia Florea et al. “Survey of Standardized Protocols [24] Emanuele Goldoni et al. “Correlation between weather
for the Internet of Things”. In: 2017 21st International and signal strength in LoRaWAN networks: An ex-
Conference on Control Systems and Computer Science tensive dataset”. In: Computer Networks 202 (2022),
(CSCS). 2017, pp. 190–196. DOI: 10.1109/CSCS.2017. p. 108627. ISSN: 1389-1286. DOI: https : / / doi . org /
33. 10 . 1016 / j . comnet . 2021 . 108627. URL: https :
[12] Usman Raza, Parag Kulkarni, and Mahesh Sooriyaban- / / www . sciencedirect . com / science / article / pii /
dara. “Low Power Wide Area Networks: An Overview”. S138912862100517X.
In: IEEE Communications Surveys & Tutorials 19.2 [25] Guanxiong Shen, Junqing Zhang, and Alan Marshall.
(2017), pp. 855–873. DOI: 10 . 1109 / COMST . 2017 . LORA RFFI DATASET. 2022. DOI: 10 . 21227 / qqt4 -
2652320. kz19. URL: https://dx.doi.org/10.21227/qqt4-kz19.
[13] Ondrej Perešı́ni and Tibor Krajčovič. “More effi- [26] Guanxiong Shen, Junqing Zhang, and Alan Marshall.
cient IoT communication through LoRa network with LORA RFFI DATASET DIFFERENT SPREADING FACTORS.
LoRa@FIIT and STIOT protocols”. In: 2017 IEEE 11th 2023. DOI: 10 . 21227 / 5q6q - c107. URL: https :
International Conference on Application of Information //dx.doi.org/10.21227/5q6q-c107.
and Communication Technologies (AICT). 2017, pp. 1–
6. DOI: 10.1109/ICAICT.2017.8686837.
[14] Lorenzo Vangelista. “Frequency Shift Chirp Modula-
tion: The LoRa Modulation”. In: IEEE Signal Process-
ing Letters 24.12 (2017), pp. 1818–1821. DOI: 10.1109/
LSP.2017.2762960.
[15] Semtech Corporation. Application Note AN1200.22:
“LoRa™ Modulation Basics”. 2015.
[16] Brecht Reynders and Sofie Pollin. “Chirp spread spec-
trum as a modulation technique for long range com-
munication”. In: 2016 Symposium on Communications
and Vehicular Technologies (SCVT). 2016, pp. 1–5. DOI:
10.1109/SCVT.2016.7797659.
[17] The Things Network. Spreading Factors. https://www.
thethingsnetwork.org/docs/lorawan/spreading- factors/.
Nov. 2021.
[18] Lorenzo Vangelista, Andrea Zanella, and Michele Zorzi.
“Long-Range IoT Technologies: The Dawn of LoRa™”.
In: Sept. 2015, pp. 51–58. ISBN: 978-3-319-27071-5.
DOI : 10.1007/978-3-319-27072-2 7.
[19] Daniele Croce et al. “Impact of LoRa Imperfect Orthog-
onality: Analysis of Link-Level Performance”. In: IEEE
Communications Letters 22.4 (2018), pp. 796–799. DOI:
10.1109/LCOMM.2018.2797057.
[20] Galal Hassan and Hossam S. Hassanein. “MoT: A De-
terministic Latency MAC Protocol for Mission-Critical
IoT Applications”. In: 2018 14th International Wire-
less Communications Mobile Computing Conference
(IWCMC). 2018, pp. 588–593. DOI: 10.1109/IWCMC.
2018.8450355.
[21] LoRa Alliance® . LoRaWAN® L2 1.0.4 Specification.
https : / / resources . lora - alliance . org / technical -
specifications / ts001 - 1 - 0 - 4 - lorawan - l2 - 1 - 0 - 4 -
specification. 2020.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 549

 Practical teaching of production and
characterization of next-gen transistors based on
sol-gel methods.
Tomas Vincze*, Martin Weis
Institute of Electronics and Photonics, Slovak University of Technology, Bratislava 81219 Slovakia
e-mail: tomas.vincze@stuba.sk

Abstract—This article discusses the importance of teaching prepared. These devices are for example diodes, LEDs,
the preparation and characterization processes of novel and transistors. Recently the sol-gel method is gaining
transistor structures that are fabricated by the sol-gel awareness as a reliable solution-based fabrication method
method. The sol-gel method is an innovative technology that for such devices. The purpose is to demonstrate the
enables the creation of thin films of materials with high strength of this method for semiconductor layer
precision and control down to the nanometer level. These preparation in a thin-film transistor (TFT), where the
materials have a wide range of applications in quality of the film determines the electrical parameters.
semiconductor and electronic devices, and it is therefore Furthermore, the course can gain a new understanding of
important that engineers and scientists have the correct charge transfer in organic or metal oxide materials [5-10].
knowledge and skills to handle this technology. This article
This contribution will focus on the fabrication process
discusses the processes for the preparation of sol-gel
of a TFTs based on metal oxide using the sol-gel method.
materials, which include the selection of suitable precursors,
The suggested practical experiment should give a good
the preparation of sol-gel solutions and their application to
substrates. Further, the characterization of these materials
basis about the sol-gel method, device, and their
using various analytical methods such as spectroscopy,
characterization. The experiment should give firsthand
microscopy, and electrical measurements is explored. The knowledge about the complexity of device fabrication in
importance of good teaching of these procedures and accordance with the theory.
techniques to achieve reliable and reproducible results is
also emphasized. Emphasis is placed on incorporation into II. THE PRACTICAL APROUCHE TO FABRICATION
existing curricula and training for future engineers who will TECHNOLOGIES
work in the development and manufacture of electronic In the context of a nanoelectronics or materials science
components and semiconductor devices. course, there is often a gap in the practical application of
various fabrication techniques and material
Keyworlds: Sol-gel method, device fabrication, Thin-film characterization processes for final electronic devices. The
transistor. proposed approach aims to enrich students'
comprehension of crucial aspects of device fabrication.
I. INTRODUCTION Since sol-gel technology is a cutting-edge manufacturing
method used in the electronics industry. This includes
Technical schools in the field of electronics require the interdisciplinary studies of chemistry and electrical
study of different technological process for device engineering. This requires the basic theoretical
fabrication. However, device fabrication is a complex understanding of the chosen method and device, which are
technological process consisting of precisely defined presented later. This includes the device and the
steps, which must be considered and followed to obtain a fabrication process. The suggested approach is through a
desired outcome. Technologies used during fabrication practical experiment and should give a detailed guide for
can be divided in two major groups, wet and vacuum laboratory exercise with the necessary theory and the
technologies. Vacuum technologies are complex required steps of the fabrication process.
techniques which can create films with high purity and
defined thickness. The drawback of these technologies is III. THIN-FILM TRANSISTORS
that the modification is limited [1, 2].
The chosen device is TFT, because of all the known
On the other hand, wet technologies do not require such transistor structures there are less commonly taught,
complexity and a solution-based deposition method can be however they pose a great potential for practical learning
easily modified to reach a desired outcome of the film. as they are easily producible for students. During the
This means a certain control over the film properties such fabrication process students give an inside look at the
as thickness, purity, but more importantly porosity. device fabrication and its shortcomings. Furthermore, in
Furthermore, depending on the application in electronics enhance students' understanding of materials properties
or photonics, electrical and optical properties are of and characterization method for basic device parameter
interest [3,4]. evaluation.
For educational purposes the solution-based deposition The structure of a thin-film transistor is based on the
method can be demonstrated as a simple practical MIS (Metal-Insulator-Semiconductor) structure, like
experiment, where a functional electronic device can be

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 550

MOSFETs. By applying a positive or negative voltage to x Surface coatings,
the gate electrode, which is separated from the x Bioactive glasses and biocompatible coatings,
semiconductor by an insulating layer, we can change the
operational mode of the transistor. Electrons or holes in x Sol-gel derived catalysts or photocatalysts,
the semiconductor are attracted to the S-I interface, x For optical materials,
resulting in charge accumulation. Applying a negative x Synthesis of nanoparticles and nanocomposites
voltage creates a p-channel, and applying a positive [13-19].
voltage creates an n-channel. However, this structure
By introducing students to this technique, educators can
behaves more like a capacitor. Only when voltage is
bridge the gap between theoretical knowledge and
applied to the top drain/source electrodes, creating an
practical application. This hands-on experience can
electric field which directs the flow of charge carriers The
significantly enhance the learning experience, making
described structure can be seen in Fig. 1. [11].
complex concepts more tangible and engaging.
The metal used for contacts has an impact on the
The sol-gel method involves the conversion of a
conductivity type, while supplying new free charge
solution into a gel-like substance and then further
carriers. To understand we must look at the energy band
processing to obtain the desired material, as shown in
diagram of the metal-semiconductor structure shown in
Fig. 3. The method involves the following steps precursor
Fig. 2.
preparation, gelatinization, drying, sintering [20-22].
The type of injection into the semiconductor depends
Choosing a suitable precursor dependence on the
on the relative position of the semiconductor's Fermi level
desired final material. In the case of transistor, the
and the metal's work function. If the work function Φm is
prepared film must have semiconductor like behavior.
closer to the semiconductor's HOMO (Highest Occupied
Suitable materials are metal salts, alkoxides, chlorides and
Molecular Orbital) level, hole injection occurs. If the work
acetates, were by applying the sol-gel method we obtain
function Φm is closer to the energy level of the
metal oxide, with n-type or p-type semiconductors. An
semiconductor's LUMO (Lowest Unoccupied Molecular
example of such semiconducting metal oxides is ZnO (n-
Orbital) level electron injection occurs [11, 12]. The
type) and NiO (p-type). However, we are not limited to
transport of carrier charge through the semiconductor is
the semiconductor layer, even dielectric layer can be
another issue. In inorganic materials transport takes place
fabricated by this method, such as Al2O3 (a known
due to free carries introduced by doping. However,
dielectric) [23, 24].
organic material or oxide the charge transfer is governed
by injection and tunneling or hoping between nearby
energy levels. Hoping of the charge is due to closely Gelatinization or solution aging is the next step in the
localized energy levels, typical example of ion charge method. The precursor and the solvent are mixed using
transport in solid materials. While charge tunneling occurs sonication or magnetic stirrer maintained under constant
when excited charges have enough energy to overcome temperature. During which a chemical reaction takes
the energy barrier between two ionized states [12]. place. The reaction breaks the metal acetate bonds
forming ligands. The ligands coordinate with the metal
IV. SOL-GEL METHOD cation to form a complex. The specific structure and
The sol-gel method is considered to have the most properties of the resulting complex can vary depending on
versatility of applications. Some noteworthy applications the metal cation.
are: A slight modification of gelatinization can be applied
by using more solvents. Aging of the precursor in this way
x Synthesize ceramic materials, improves the gel's mechanical properties and
microstructure. The suggested additional solvent is
monoethanolamin. It serves as a stabilizer of metal cation

Figure 1. Bottom gate top contact structure of TFT.

Figure 3. Visualization of the sol-gel method.

a) b)
Figure 4. a) Copper acetate hydrate molecule and b) the role of
monoethanolamin encapsulation of matel cation.
Figure 2. Energy level diagram of metal-semiconductor.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 551

 SOL-GEL method
1.2

Drain source current (PA)

IPA:2ME
r9
Si r8 1.0
r7
IZ(ref. arb. u.)

r6
r5
0.8
CuO r4
Si
Ag
CuO CuO r3 0.6
Bg1 Bg2 r2
r1
0.4
0.2
0.0
200 300 400 500 600 700 -14 -12 -10 -8 -6 -4 -2 0
Z (cm-1) Drain source voltage (V)

Figure 5. Depiction of the fabrication process of TFT based on CuO starting with solution deposition of the semiconductor (the sol-gel process) and the
final device characterization.

preventing the aggregation of nanoparticles. This The solution was then deposited on the cleaned
encapsulation process is shown on Fig. 4. [25]. substrate by spin-coating. Subsequently, the gel was dried
After aging, the solution is deposited on a beforehand at a low temperature of 160 °C for 1 hour to remove
cleaned substrate by spin-coating/drop-casting/dip- residual solvent and sintered at high temperature at 500 °C
coating/spray-casting. The deposited gel is carefully dried for 1 hour to form copper oxide with a stable state of
to remove the solvent, done at a controlled temperature cuprous oxide (Cu2O) and cupric oxide (CuO), with a
(around the boiling point of the used solvents) to prevent bandgap of 2.1 eV and 1.2 eV, respectively. The reported
structural damage. Slow, controlled drying is essential to free charge mobility up to 10 cm2/V·s or lower, which
maintain the desired properties of the material. makes them suitable for TFT devices [1, 2, 27-29].
However, this process does not create the desired final Top contact bottom gate geometry was used where
material. To reach the desired outcome the dried gel is highly doped Si served as the gate electrode and SiO2
subjected to high-temperature treatments (oxidation dielectric layer. Gold source/drain electrodes with a
temperate of the metal compound in the used precursor), thickness of 60 nm were formed using a shadow mask
either in air or in a controlled atmosphere. This process, with a channel length L in the range of 50 to 200 μm and
known as sintering, removes any residual organic constant channel width W of 2.5 mm. The electrodes have
components, enhances crystallinity, and further adjusts the been deposited using vacuum thermal evaporation in a
material's properties. high vacuum (the pressure lower than 10-5 Pa) under a
constant evaporation rate of 1 Å/s using the PVD75
Drying and sintering are the processes that have the
deposition system (by Kurt J. Lesker). The electrical
most impact on the films structure as described by
Mansour et. al. Suggesting a drying model for a single and measurements were performed using semiconductor
device analyzer B1500A (by Keysight).
a multiple layer gel deposition and the effect on the film’s
porosity [26].
VI. CHARACTERIZATION
V. DEVICE FABRICATION The TFT device is characterized by parameters like
mobility, threshold voltage, On/Off ratio, subthreshold
To demonstrate the device fabrication using the sol-gel
slope, and swing, which are evaluated from the transfer
method a simple experiment is made. Each step of the
and output characteristics depicted in Fig. 6. and 7. The
fabrication process is depicted in Fig. 5. Using copper
output characteristics depicted the drain-source current IDS
acetate hydrate Cu(COOCH3)×H2O as precursor mixed
dependence on drain-source voltage VDS for different gate
with 2-methoxyethanol and monoethanolamine (solvents)
voltage VGS [30].
and sonicated and aged for 4 hours at 60 °C .
To get an understanding of how to properly evaluate
Before further processing, the substrates were cleaned
electrical parameters, is to study the transfer
in an ultrasonic bath with deionized water, ethanolamine
characteristics of TFTs. The transfer curve can be divided
20% solution, isopropyl alcohol, and acetone. The clean
into three regions: the sub-threshold region, the linear
substrates were exposed to an oxide plasma atmosphere to
region, and the saturation region. Each of these regions is
remove further organic particles and modify the surface
for better adhesion.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 552

 ߤܹ‫݃ܥ‬
‫ ܵܦܫ‬ൌ ሺܸ‫ ܵܩ‬െ ܸ‫ ݄ݐ‬ሻܸ‫ܵܦ‬
‫ܮ‬ (4)
1.4 1.4

Drain source current (PA)

To obtain the Vth and μ the transfer curve is modified.
1.2 1.2 The square root of the IDS is seen in Fig. 7, which
DS u10 A )
-3 1/2

1.0 1.0 linearizes the saturation region. The slope of the linear
0.8 0.8
part corresponds to the effective mobility of free charge
carriers, expressed as:
0.6 0.6 ͳȀʹ ʹ
0.4
ʹ‫ܮ‬ ߲‫ܵܦܫ‬
I1/2

0.4 ߤൌ ቆ ቇ
ܹ‫߲ ݃ܥ‬ሺܸ‫ ܵܩ‬െ ܸ‫ ݄ݐ‬ሻ (5)
0.2 0.2
0.0 0.0
The mathematical expression of μ is derived from
-15 -10 -5 0 5 10 15
equation 3. The value of the Vth is obtained by
Gate source voltage (V) extrapolating the linear part to zero current.
Figure 6. Linear scale of the transfer characteristic of a 100 μm Other notable parameters of TFTs are subthreshold
channel CuO based TFT measured for drain-source voltage -15 V. slope and swing. The subthreshold slope is obtained of the
The plot depicts the square root of the drain-source current with the slope of IDS in logarithmic scale shown in Fig. 8. And the
slope used for parameter evaluation. 
subthreshold swing is derived from the slope as:
‫ ݏݏ‬ൌ ͳȀ‫݁݌݋݈ݏ‬ (6)
After the slope determination, which defines the values
when the transistor is in the on and off state. The ratio of
1.2 the current in these states tells us about the leakage
Drain source current (PA)

currents or the efficiency of the transistor.

1.0
Until now the shown parameters were all obtained from
0.8 the saturation region. Other parameters like contact
resistance can be evaluated using the linear region [30].
0.6 Contact resistance represents a limitation on the current
through the channel, in addition to the resistance of the
0.4
channel itself shown in Fig. 9. In the past, when
0.2 semiconductor had low mobilities, the channel resistance
was a limiting factor. With advances in materials and
0.0 technology, the total channel resistance has decreased
-14 -12 -10 -8 -6 -4 -2 0 over time. Contact resistance itself affects the injection of
Drain source voltage (V) charge from the contacts into the semiconductor. It
Figure 7. Output characteristic of a 100 μm channel CuO based TFT
represents one of the ways charge transports can occur, as
measured for gate voltage in range of 0 to -12 V with a step of 2 V. described. If we express effective mobility in terms of
determined by specific conditions. In the sub-threshold resistances, we obtain the following relationship:
ܴ݄ܿ
voltage region, the threshold voltage Vth ≤ VGS gate ߤ݂݁ ൌ ߤ
voltage; in this region, the transistor is ideally closed. ܴ݈݁ ൅ ܴ݄ܿ (7)
Once VGS surpass Vth, the current through the channel
(IDS) starts to increase with a power-law dependence. The where μef is the effective mobility, Rch is the channel
current through the channel can be expressed by the resistance, and Rel is the contact resistance, it follows an
following equation:
ߤܹ‫݃ܥ‬ ܸ‫ܵܦ‬
‫ ܵܦܫ‬ൌ ൤൬ܸ‫ ܵܩ‬െ ܸ‫ ݄ݐ‬െ ൰ ܸ‫ ܵܦ‬൨
‫ܮ‬ ʹ (1) -6.0
Log scale drain-source current (log A)

-6.1
were μ is the mobility of free charge carriers, Cg is the
gate electrode capacitance per unit area, W is the width, -6.2
and L is the length of the channel. When expressing -6.3
current in the saturation region, we must take into
-6.4
consideration the following condition:
ܸ‫ ܵܦ‬ൌ ܸ‫ ܵܩ‬െ ܸ‫ ݄ݐ‬ǡ -6.5
(2)
-6.6
then equation 1 is expressed as: -6.7
ߤܹ‫ ݃ܥ‬ሺܸ‫ ܵܩ‬െܸ‫ ݄ݐ‬ሻʹ
‫ܵܦܫ‬ሺ‫ݐܽݏ‬ሻ ൌ . (3) -15 -10 -5 0 5 10 15
ʹ‫ܮ‬
Gate-source voltage (V)
The transistor operates in the linear region when the Figure 8. Logarithmic scale of the transfer characteristic of a
voltage VDS is much smaller than the difference between 100 μm channel CuO based TFT measured for drain-source
the gate-source voltage and the Vth, i.e., VDS ‫ ا‬VGS - Vth. In voltage -15 V. The plot depicts subthreshold slope and swing
this case, current through the channel with is expressed by analysis.
the following equation:

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 553

 optimizing technological processes due to well-defined
parameters, which ensure controlled studies of the
semiconductor layer, as it represents an attractive new
trend on commercial use.

VIII. ADVANTAGES OF PRACTICAL STUDIES

By introducing students to this technique, educators can
Figure 9. Resistive network model of semiconductor metal bridge the gap between theoretical knowledge and
electrode junction. In a TFT structure. practical application. This hands-on experience can
significantly enhance the learning experience, making
inverse proportionality between effective mobility and complex concepts more tangible and engaging.
contact resistance. Furthermore, educational institutions that incorporate sol-
To obtain the value of contact resistance, we need to gel technology into their courses open the door to new
calculate the resistance across the channel, which is research opportunities. Students can explore various
linearly dependent on the channel length, as shown in the materials and applications, beyond the presented TFT
following equation: fabrication and characterization. For example, the bases
߲ܸ‫ܵܦ‬ ܹ of sol-gel method can be applied as transparent
ܴ݇ܽ݊ ൌ ൌ
߲‫ߤ ݃ܥܮ ܵܦܫ‬ሺܸ‫ ܵܩ‬െܸ‫ ݄ݐ‬ሻ (8) conductive films, gas sensors, and photovoltaics. This
fosters a culture of innovation and allows students to
The total resistance RC of the TFT can be expressed as the
contribute to scientific advancements. It further
sum of contact resistances and the channel resistance:
ܹ
encourages interdisciplinary learning as it involves
ܴ‫ ܥ‬ൌ ܴ݈݁ ൅ ܴ݄ܿ ൌ ܴ݈݁ ൅ principles from chemistry, physics, materials science, and
‫ߤ ݃ܥܮ‬ሺܸ‫ ܵܩ‬െܸ‫ ݄ݐ‬ሻ (9)
engineering.
By linearly extrapolating the total resistance towards
zero channel length, we can easily evaluate the value of IX. CONCLUSION
contact resistance as a component that is independent of As we've reviewed in this article, this solution-based
the channel length [30]. approach offers unique advantages, making it an
invaluable tool for educators and students alike in the field
VII. COMMON PROBLEMS of materials science and electronics engineering.
The problems related to thin films preparation and device One of the most compelling aspects of the sol-gel
fabrication will be discussed. Films thinner than 20 nm method is its accessibility. Unlike some other fabrication
often exhibit discontinuities, diminishes conductivity and techniques that involve high-vacuum systems and
a discontinuous dielectric layer implies poor insulating potentially hazardous materials, the sol-gel process can be
conducted in standard laboratory settings with relative
properties and significant leakage currents. The ease. This makes it an excellent choice for educational
consequence is a non-functional electronic device. purposes, especially nanotechnological courses where
Therefore, it is essential to carefully consider the methods safety and simplicity are paramount.
of preparation and the structure of the film. Moreover, the sol-gel method allows for a high degree
When using thermal evaporation method the metal oxide of customization. Students can experiment with various
undergoes a decomposition, breaking down into other precursor materials, concentrations, and processing
components under certain conditions. This phenomenon conditions to tailor the properties of the thin films they
is undesirable because deviations from the desired produce. This not only enhances their understanding of
material leads to undesirable properties of the film. This materials science but also encourages creativity and
problem is overcome by sputtering. Due to time and problem-solving skills.
technical demands of the deposition techniques, the wet Additionally, the versatility of the sol-gel method
route is a reliable option. extends to a wide range of applications, from
semiconductor devices like TFTs to sensors, optical
Furthermore, it is necessary to consider a suitable coatings, and more. One of the most straightforward
dielectric material for the device. Materials typically used verification approaches involves electrical measurements
include polymers or non-conductive oxides. It is known on the fabricated devices. These measurements offer a
that metal oxides produced by sol-gel method require fundamental insight into charge transfer within TFT
high temperatures to achieve the desired level of devices and provide essential data regarding the
oxidation, for example, copper oxides oxidize at 225 °C semiconductor's electrical properties.
and above. If higher temperatures are required, polymer- Educators can use this method to demonstrate its
based dielectrics are not suitable, leading to permanent relevance across diverse industries, further motivating
damage of the layers. For this reason, good thermal students to explore the possibilities it offers.
stability of the dielectric is necessary. Materials that meet
this requirement are oxides. Based on the chosen ACKNOWLEDGMENT
substrate, we can also select the dielectric. Since silicon This work was supported by the Scientific Grant
serves as the substrate, thermal grown silicon dioxide Agency and the Slovak Research and Development
(SiO2) is a suitable option. This is a good choice for Agency of the Ministry of Education, Science Research

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 554

and Sport of the Slovak Republic, grants: VEGA [15] M. S. Chavali, & M. P. Nikolova, Metal oxide nanoparticles and
1/0621/23, APVV-20-0564 and APVV-20-0310. their applications in nanotechnology. SN applied sciences, 1(6),
1-30, 2019.
[16] A. K. Arora, V. S. Jaswal, K. Singh, & R. Singh, Applications of
REFERENCES metal/mixed metal oxides as photocatalyst:(A review). Oriental
[1] T. Vincze, M. Micjan, M. Pavuk, P. Novak, & M. Weis, Journal of Chemistry, 32(4), 2035, 2016.
Fabrication of cupric oxide‐based transistors by sol–gel [17] B. Huang, L. I. Changhe, Y. Zhang, D. I. N. G. Wenfeng, Y. A.
technique. Journal of Materials Science: Materials in N. G. Min, Y. A. N. G. Yuying, & S. A. I. D. Zafar, Advances in
Electronics, 32, 6883-6889, 2021. fabrication of ceramic corundum abrasives based on sol–gel
[2] T. Vincze, M. Micjan, & M. Weis, Copper oxide field-effect process. Chinese Journal of Aeronautics, 34(6), 1-17, 2021.
transistor fabricated by sol-gel method. In 2020 13th [18] A. Dehghanghadikolaei, J. Ansary, & R. Ghoreishi, Sol-gel
International Conference on Advanced Semiconductor Devices process applications: A mini-review. Proc. Nat. Res. Soc, 2(1),
And Microsystems, ASDAM, (pp. 115-118). IEEE October 2020. 02008-02029, 2018.
[3] T. Vincze, M. Micjan, M. Pavuk, & M. Weis, Fabrication of [19] A. P. Periyasamy, M. Venkataraman, D. Kremenakova, J.
cupric oxide-based transistors by a sol–gel technique using a Militky, & Y. Zhou, Progress in sol-gel technology for the
binary solvent mixture. Journal of Materials Science: Materials coatings of fabrics. Materials, 13(8), 1838, 2020.
in Electronics, 33(10), 7701-7707, 2022. [20] L. L. Hench, W est J K. The Sol—gel Process. Chem Rev, l990,
[4] Q. Zhang, K. Zhang, D. Xu, G. Yang, H. Huang, F. Nie, & S. 90, 1990.
Yang, CuO nanostructures: synthesis, characterisation, growth [21] S. Park, C. H. Kim, W. J. Lee, S. Sung, & M. H. Yoon, Sol-gel
mechanisms, fundamental properties, and applications. Progress metal oxide dielectrics for all-solution-processed electronics.
in Materials Science, 60, 208-337, 2014. Materials Science and Engineering: R: Reports, 114, 1-22, 2017.
[5] M. S. Chavali, M. P. Nikolova, Metal oxide nanoparticles and [22] S. S. Prasanna, K. Balaji, S. Pandey, & S. Rana, Metal oxide
their applications in nanotechnology. SN applied sciences, 1(6), based nanomaterials and their polymer nanocomposites. In
1-30, 2019. Nanomaterials and Polymer Nanocomposites, Elsevier, (pp. 123-
[6] L. Petti, N. Münzenrieder, C. Vogt, H. Faber, L. Büthe, G. 144), 2019.
Cantarella, & G. Tröster, Metal oxide semiconductor thin-film [23] S. A. Delbari, L. S. Ghadimi, R. Hadi, S. Farhoudian, M. Nedaei,
transistors for flexible electronics. Applied Physics Reviews, A. Babapoor, & M. Mohammadi, Transition metal oxide-based
3(2), 2016. electrode materials for flexible supercapacitors: A review.
[7] L. G. Gerling, S. Mahato, C. Voz, R. Alcubilla, & J. Puigdollers, Journal of Alloys and Compounds, 857, 158281, 2021.
Characterization of transition metal oxide/silicon heterojunctions [24] S. Ghosh, & L. Rajan, Zinc oxide thin-film transistor with
for solar cell applications. Applied sciences, 5(4), 695-705, 2015. catalytic electrodes for hydrogen sensing at room temperature.
[8] J. H. Lee, J. Kim, M. Jin, H.J. Na, H. Lee, C. Im, & Y. S. Kim, IEEE Transactions on Nanotechnology, 20, 303-310, 2021.
Cu2O p-type thin-film transistors with enhanced switching [25] R. Bekkari, B. Jaber, H. Labrim, M. Ouafi, N. Zayyoun, & L.
characteristics for CMOS logic circuit by controlling deposition Laânab, Effect of solvents and stabiliser molar ratio on the
condition and annealing in the N2 atmosphere. ACS Applied growth orientation of sol-gel-derived ZnO Thin films.
Electronic Materials, 5(2), 1123-1130, 2023. International Journal of Photoenergy, 2019.
[9] Z. Ouyang, W. Wang, M. Dai, B. Zhang, J. Gong, M. Li, & H. [26] C. Mansour, M. Benwadih, G. A. Chahine, & C. Revenant,
Sun, Research progress of p-type oxide thin-film transistors. Effect of thermal annealing on the morphology of sol–gel
Materials, 15(14), 4781, 2022. processed barium strontium titanate thin films: Consequences on
[10] H. Hu, J. Zhu, M. Chen, T. Guo, & F. Li, Inkjet-printed p-type electrical properties. AIP Advances, 10(6), 2020.
nickel oxide thin-film transistor. Applied Surface Science, 441, [27] H. A. Al‐Jawhari, Recent Progress of Solution‐Processed
295-302, 2018. Copper‐based p‐Channel Thin‐Film Transistors. Advanced
[11] S. D. Brotherton, Introduction to Thin Film Transistors: Physics Electronic Materials, 2100893, 2021.
and Technology of TFTs. Springer Science & Business Media, [28] N. Lu, W. Jiang, Q. Wu, D. Geng, L. Li, & M. Liu, A review for
2013. compact model of thin-film transistors (TFTs). Micromachines,
[12] K. C. Kao, Dielectric phenomena in solids. Elsevier, 2004. 9(11), 599, 2018.
[13] S. Akin, S. Sonmezoglu, Metal oxide nanoparticles as electron [29] G. Akerlof, Dielectric constants of some organic solvent-water
transport layer for highly efficient dye-sensitized solar cells. In mixtures at various temperatures. Journal of the American
Emerging materials for energy conversion and storage, Elsevier, Chemical Society, 54(11), 4125-4139, 1932.
(pp. 39-79), 2018. [30] S. Lai, P. Cosseddu, & A. Bonfiglio, A method for direct contact
[14] S. Laurent, S. Boutry, & R. N. Muller, Metal oxide particles and resistance evaluation in low voltage coplanar organic field-effect
their prospects for applications. In Iron oxide nanoparticles for transistors. Applied Physics Letters, 110(15), 153304, 2017.
biomedical applications, Elsevier, (pp. 3-42), 2018.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 555

 Ultrasonic water leak detection system with real-
time transmission of measured values
D. Zadžora*, E. A. Katonová*, M. Murín*, P. Feciľak*, M. Michalko*, F. Jakab*
*
Department of Computers and Informatics, Košice, Slovakia
David.Zadzora@student.tuke.sk, Erika.Abigail.Katonova@cnl.sk, Peter.Fecilak@cnl.sk, Miroslav.Michalko@cnl.sk,
Frantisek.Jakab@cnl.sk

Abstract— Late identification of water leaks can cause One of the common problems associated with the
significant damage to buildings and infrastructure, leading addressed problem is the occurrence of high property
to costly repairs and wasting resources. Traditional methods damage in households caused by water leaks. These
of detecting water leaks from pipes can be highly time- include leaks or damages under the sink pipe, leaking
consuming and may not provide real-time information, main pipe, burst water tank and many other related
which is key to identifying leaks early and taking problems. These inconveniences are most often solved by
appropriate action. This paper is about to address these the use of leak detection systems that are installed at the
issues by developing an ultrasonic water leak detection location where such a problem is expected to occur. The
system with real-time transmission of measured values. The leak detection systems can then monitor the flow of water
proposed system uses ultrasonic transducers strategically through the pipes and when unusual behavior occurs, the
placed in the water supply network to detect water leaks by system shuts off the water supply to the entire building by
measuring changes in ultrasonic signals. The measured shutting off the main water valve and notifying the
values are processed and transmitted in real-time using a building manager.
microcontroller with a wireless communication module that
allows remote monitoring of the system. In order to increase
Currently, leak detectors based on mechanical
the accuracy and predictive capabilities of the water leak impellers and sensing of the underlying surface are still
detection system, this paper also incorporates machine widely used. Problems with these technologies arise with
learning techniques. The predictive models are integrated small leaks that these detectors fail to detect, or only
into the system, allowing it to proactively anticipate and detect when significant water damage to the property has
alert on potential leaks before they develop into larger already occurred.
problems. It is for these leaks that ultrasonic technology is
suitable, as the sound difference in the water pipe can
Keywords— ultrasonic device, pipe monitoring, water provide enough information to detect these small leaks.
leakage, real-time data, leak detection, microcontroller, This is why many community utilities are also switching
notification, flowmeter, MQTT, Modbus, InfluxDB, from the old mechanical water meters to ultrasonic ones
ultrasonic sensor, machine learning, Random Forest these days [2].
In this paper, we will discuss the design and
I. INTRODUCTION implementation of a system to detect water leaks using
ultrasonic measurement and real-time data transmission.
Water, as one of the most important resources for
human life, covers approximately 2/3 of the Earth's II. DESCRIPTION OF THE USED TECHNOLOGIES
surface. Despite this seemingly high figure, water
availability is very low in some areas. It is therefore A. Measuring Technologies
necessary to help those affected by this shortage of
drinking water to ensure its continued availability and to The authors of a paper on early detection of pipeline
prevent unnecessary waste. leaks [3] describe an ultrasonic receiver as a type of
Storage and transport facilities for water, but also for transducer that converts electrical energy into high-
other liquids and gases, are under constant threat from frequency sound waves and the other way around. This
corrosion and erosion, which can result in significant and device contains piezoelectric crystal materials whose
unexpected leakage of these liquids or gases. In a number capability is to transform mechanical energy into
of cases, such a leak would create critical environmental electrical energy and vice versa. The basic property of a
risks and economic losses. Leaks of natural gas piezoelectric crystal is to change in size when the correct
components such as ethane, propane and butane can voltage value is applied. It follows that by applying
produce volatile organic compounds that result in smog alternating current (AC) to these crystals, they begin to
formation in certain areas, which poses a health risk [1]. oscillate at very high frequencies, resulting in the creation
Nowadays, these problems can be solved using of high-frequency sound waves.
modern technologies, including the Internet of Things In order to provide more precise data, the FlowMeter
(IoT). IoT is the integration of various sensors and TUF-2000M can be used. According to information taken
wireless devices. The data from these sensors and devices from the TUF-2000M flowmeter user manual, the main
are displayed, stored or further modified using modern component is the high-performance MSP430FG4618
technologies.
microprocessor. The circuit board used in the TUF-

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 556

2000M module is the 13th version of the ultrasonic tables are stored discrete on/off values (coils),
flowmeter and measures 116 x 62 millimeters. It is and in the other two tables numeric values
encapsulated in a plastic rectangular housing that can be (registers) are stored. Modbus communicates
easily slipped onto the installation rail. The module can over several types of physical lines, such as
operate stand-alone, i.e. without the LCD display and Serial RS-232, Serial RS-485, Serial RS-422,
keypad. The module meets the requirements for Ethernet [6][7].
measuring most types of liquids such as water,
seawater, wastewater and chemical liquids. It can also
measure liquids with a higher density of suspended III. SOLUTION DESIGN WITH REAL-TIME ASPECTS
particles. Within this section the design of a water leak detection
system, the implementation of the solution is discussed.
B. Transmission and communication technologies
From the physical wiring of the individual hardware
There are multiple technologies and protocols that enable components to the use of machine learning algorithms to
data transmission and communication between different detect a water leak and then inform the user of this
devices. Solution proposed within this paper build on top condition.
of the following key components:
x The ESP32 microcontroller module is a dual- A. The selection of components
core system with two processors. It is having The components that were used in the development of
embedded memory, external memory, and the solution and their main functions:
peripherals are located on the data/instruction x Ultrasonic sensor - Used for transmitting and
bus of these processors. As the technical manual receiving the signal.
shows [4], the address mapping of the two x TUF-2000M - Ultrasonic metering device.
processors is symmetric, meaning that they use x RS485-TTL converter - Communication
the same addresses to access the same memory. protocol conversion.
In the implementation of our solution we will
x ESP32 - Microcontroller.
use the ESP32 DevKit v1 development board,
which is designed to evaluate the ESP- x MQTT - Communication protocol.
WROOM-32 module. It is based on the ESP32 x Server Node.JS - A JavaScript server
microcontroller just mentioned, which offers environment that is used to connect multiple
support for several technologies such as Wi-Fi, components of an application.
Bluetooth, Ethernet and also has low power x Server Python - Water leak detection and user
consumption in a single chip. notification.
x MQTT is one of the most widely used protocols x ReactJS - The user interface.
for sending messages over the Internet Of x Grafana - Create customizable charts.
Things (IoT). It contains a set of rules that
define how devices can "publish" and B. Hardware prototype
"subscribe" to data over the Internet. It is used, Figure 1 shows the hardware part of the developed
for example, in messaging and data exchange solution in more detail.
between the IoT and, for example, built devices,
sensors, PLCs, and more. The connection
between the sender (Publisher) and the receiver
(Subscriber) is handled by the so-called MQTT
broker, which filters incoming messages and
distributes them correctly to the subscribers [5].
x Modbus protocol is a protocol located at the
application layer of the OSI/ISO model and is
used for client-server relationships between
devices connected on different types of buses or
networks. It is an open protocol that has become
a standard communication protocol in industry
Figure 1 Hardware prototype schematic
and is one of the most widely used solutions for
communication between electronic devices. It is
The TUF-2000M device was used to measure the flow
typically used to transmit signals from devices to rate using two sensors mounted on the pipe to send
a control system. For example, to measure information about the properties of the measured fluid
humidity, temperature, liquid velocity, but also directly to the device. This information can be read from a
other aspects. Modbus is transmitted between register from the measuring device's register using the
devices over serial lines. The data is sent in the MODBUS protocol and then sent via the development
form of bits. A typical transmission rate is 9600 board with the ESP32 chip to the back-end via the MQTT
baud (bits per second). The information, in slave protocol, where be further processed and displayed to the
devices, is stored in four tables. In the first two user. The ESP32 microcontroller can communicate using
USB or UART communication interfaces. At the same

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 557

time MODBUS device communicates using RS485 serial had an internal diameter of 32 millimetres and a wall
communication, therefore the RS485 to TTL converter is thickness of 5,4 millimetres. These data were entered into
required to allow the two technologies to communicate. the measuring instrument, together with the pipe material
This converter serves as a bridge between RS485 and (plastic) and the double reflection measurement method
UART protocol. Board with the ESP32 microchip (W-method) was chosen. Based on these parameters, the
supports communication over Wi-Fi, and therefore the instrument calculated a distance of 10 millimeters, which
information is forwarded using this technology. tells us how far apart the individual sensors of the TS-2
model should be. To achieve more accurate values, it is
C. Software solution necessary to apply a coupling gel between the pipe and the
A) MQTT and Node.JS server: The communication sensor.
starts by sending data from the ESP32 controller Its purpose is to help the ultrasonic wave from the
through the secure MQTT port. Data in JSON sensor to the pipe by, by filling any resulting air bubbles
format is received on the back-end server of the between the sensor and the pipe. At precisely specified
Node.js environment, which modifies the data values, we have achieved signal accuracies between 90% -
into a suitable format and then sends it to the 95%.
front-end ReactJS environment and also stores it This value, according to the documentation,
in the database. Before the actual transfer, it is a determines the amplitude of the received signal and the
prerequisite that the server is already up and instrument should work fine between 50% - 100%. After
setting the above parameters, we then interfaced the
running and it must have an accepted theme that measuring instrument to the ESP32 board via an RS485
is sent to the client via a web page in order to to converter so that we could read the MODBUS registers
know from which topic to take the data. The from the device.
received data is also stored in an array and the The value of the current flow rate was stored in
server waits for a specified period of time, which registers 0001-0002. This value, along with other
represents the time window during which we necessary registers, was sent via the MQTT protocol to
want to to check for leaks, and after the time the server where the data was processed and written to the
interval, the data from the array are sent to the database. We then took several measurements where we
Python server where it is then evaluated. After opened the valve for a period of time and observed how
the expiration of this time, the data are cleared the flow rate changed on the data in real time. When the
from the array and the timer is restarted. valve was closed, this value went back to 0 as no water
B) ReactJS: At user interface, data about the MQTT was flowing in the pipe.
topic are retrieved from the user input. In
addition to Push notifications, it is also possible
to alert the user of data leaks in real time via
email. The user can provide his/her email
address to which the notifications will be sent.
These notifications will be processed and sent to
the Node.js server
C) Python service: The Python server receives data
sent via POST request to the /api/data endpoint
using the Flask tool. The response to the client,
i.e. back to the Node.js server, on successful
receipt of the data is also handled by a function
from the Flask library. Then, if the data are
evaluated as a leak, a push notification is sent
using the Firebase host services file. Also, if the Figure 2 Flow measurement in simulated environment
user has set up email notifications to be sent, the
server receives this email, and then it sends a
notification about water leak event message to it. B. Measuring data in real environment
We implemented our solution in a family house on a
plastic pipe that led to the toilet. This design is shown in
IV. DATA MEASUREMENT AND WATER LEAKAGE Figure 3. In practice, checking the individual devices that
DETECTION need a water supply to function separately will make it
easier for us to find the specific place where the leak has
A. Measuring data in simulated environment occurred where a pipe has leaked or where a failure of a
particular device has caused a leak. The advantage of such
To test the functionality of the instrument, register a solution is, as we have mentioned, that it is easier to
reading and overall system operation, we performed locate the leak, but at the cost of a higher number of
measurements in simulated operation. This environment measuring devices that will be needed to be purchased.
consisted of a plastic pipe with an adjustable shut-off on The specific parameters of the pipe to be measured were:
one side valve to drain the water into the bath and on the 16 millimetres diameter, 2 millimetres wall thickness and
other side the pipe was connected to a to the water inlet, we used the double reflection method. The data was
which was also opened and closed by a valve. The plastic measured at 5 second intervals, then sent through the
pipe

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 558

MQTT server and stored in a database. On the data from V. EVALUATION AND CONCLUSION
these measurements, the model was trained and potential
leakage was evaluated. Different solutions from several authors were
reviewed, focusing on monitoring water leaks in
households, industry, as well as measuring the volume of
liquids in different tanks. All these solutions worked with
ultrasonic sensors of different types. From these
solutions, the inspiration to create our own system was
derived, and the most logical first step was to choose an
ultrasonic device that had to meet several requirements.
One of the requirements was to use waterproof ultrasonic
sensors. Although the transducer is not in direct contact
with water, but only with the pipe in which the fluid
being measured is located, it is still very likely that the
sensor may come into contact with the fluid being
measured, at least for a short period of time. After a long
search process, it was possible to discover an ultrasonic
device that fulfilled several requirements. It was a device
TUF-2000M containing two ultrasonic transducers model
Figure 3 Flow measurement in real operation TS-2. These transducers have a strong magnet built into
them that makes installation on metal and can operate in a
C. Pipe leak detection using the Random Forest temperature range from -30 to 90 degrees Celsius. The
Algorithm device has its own built-in memory for storing measured
Since the Random Forest algorithm learns to classify data and configuration settings. RS485 serial interface
data based on existing patterns, the input information was used to transfer the measured data from the
about whether a leak occurred at a given time instant with ultrasonic device and the data was transferred using
a given flow value is a key factor for learning and Modbus protocol. The ESP32 device that was used to
classification. Therefore, for each row in the data frame, implement the solution has a built-in Wi-Fi module
it is also necessary to input information about whether a through which data can be forwarded wirelessly to the
leak has occurred or not. The leakage information is back-end services. The main element on the basis of
determined using defined thresholds and threshold ratio which the fluid leakage will be identified is the flow rate.
in a given time window. The model in question was This variable is stored in the MODBUS register table at
trained on data collected from a domestic pipe that only address 0001-0002 and tells how much water in cubic
made water available to toilets, and thus contained water meters per hour is currently flowing in the pipe being
flow data during the filling of the tank after flushing. The measured. Once connected to the Wi-Fi network and a
leakage is set to true if more than 90 percent of the data is connection to the MQTT server is established, data is sent
within the 7-minute time window flow rate value is to the MQTT address to the specific logged in topic.
greater than the threshold value of 50 (liters per hour). To identify a leak in the pipeline on real-time
We created 2 new columns from the time information measured data, as the best solution was proposed to use
column "datetime" that representing hour and minute the Random Forest machine learning algorithm.
information, and added them to the data frame. This way As input attributes that the model receives are
they can be used when training the model for better information about the time and value of the flow rate at a
prediction of the output. Next, we split the data into input given time, which are used to identify the leakage. To
and output variables. To partition the input and output increase the accuracy of the model, the the time
variables into training and test sets, we used the function information is divided into smaller parts - days, hours and
from the scikit-lear 'train_test_split' library. The input minutes. In one of the measurements carried out, the
variables X and y are split into training and test sets in a ultrasonic device was located in a family house while the
ratio of 80 to 20 and at each code iteration pipe that feeds the water to the toilet was monitored. The
the same split is always used due to setting the machine learning model Random Forest was given the
random state to a fixed value. After splitting the data, an input information mentioned above, along with
rf classification model with 100 trees is created, i.e. 100 information about whether there was a leak at that
decision rules. The process of training the model is done moment. The value of the leakage occurring must be
by using 'rf.fit()' function with the input and output entered before training the data since the Random Tree
parameters of the training set. algorithm learns to classify data based on existing
And as the last thing, the 'rf.predict()' method is used patterns. The accuracy of the trained of the model with
to predict the output variable for the input variables of the the parameters thus specified was over 95%, which we
test set, which represents the predicted outputs computed consider to be high value. After successful completion of
by the Random Forest model for the test set only. training, the model was ready for real-time data
identification. The generated model was on real
operational data, as well as on artificially generated data.

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 559

 points in time. It can be pointed out, that the model has
identified the same data but at two different points in time
differently. The data between 11:40:00 and 11:50:00 was
not flagged by the system as having a leak, because it has
learned that the pipeline is actively being used at this
time. Conversely, at the time instant from 14:40:00 to
14:50:00, it marked this data as leaked data. This
behavior is not entirely ideal because if it occurred in this
time window, the system might not be able to identify it
in time. The main reason of the behavior described above
is that the model did not have a sufficient amount of data
from which to learn from. By increasing the amount of
training and testing data, this problem could have been
problem and increase the accuracy of the model.
ACKNOWLEDGMENT
This work was supported by Cultural and Educational
Grant Agency (KEGA) of the Ministry of Education,
Science, Research and Sport of the Slovak Republic
under the project No. 026TUKE-4/2021.

Figure 4 Leak detection testing using the Random Forest REFERENCES

[1] MERIBOUT, Mahmoud; KHEZZAR, Lyes; AZZI, Abdelwahid;
GHENDOUR, Nabil. Leak detection systems in oil and gas fields:
Figure 4 shows the leakage detection results from the Present trends and future prospects. Flow Measurement and
learned model of the Random Forest algorithm in the Instrumentation. 2020, 75 pages, issn 0955-5986.
form of a graph. The largest graph contains the data on [2] ESTEVEZ, Leo. Ultrasonic Leak Detection. 2020-09. Tech. spr.
which the model had to learn to detect the leak, the Texas Instruments Incorporated. Available from URL:
https://www.ti.com/lit/an/slaa965/slaa965.pdf?ts=1692619486491
smaller four show the data that were sent to the system [3] RAHIM, Herlina; HAMID, Khairulanam; ZAINAL ABIDIN,
from the measurement and how the model identified Hafilah. Early. Detection of Pipeline Leakage Using Ultrasonic
them. Of these four cases are two that the system did not Sensor. Jurnal Teknologi. 2015, p. 73. Available from doi:
identify as leaked data and two that identified as 10.11113/jt.v73.4239
potentially leaking data. Screen 2 of figure 4 shows a [4] SYSTEMS, Espressif. Technical reference manual - ESPRESSIF.
2023. Available from:
common flush at a point in time where similar data https://www.espressif.com/sites/default/files/documentation/esp32
occurred in the data from which the model was learning, _technical_reference_manual\_en.pdf
and thus the system correctly evaluated these data as [5] GMBH, HiveMQ. Mqtt Essentials. [B.r.]. Available from URL:
normal traffic. https://www.hivemq.com/mqtt-essentials/
[6] TAMBOLI, Sadik; RAWALE, Mallikarjun; THORAIET, Rupesh;
Screen 3 of Figure 4 shows, from 00:00:00 to 00:10:00, a AGASHE, Sudhir. Implementation of Modbus RTU and Modbus
TCP communication using Siemens S7-1200 PLC for batch
10-minute time window, in which the data values are process. In: 2015 International Conference on Smart Technologies
around 70 litres per hour. The system evaluated correctly and Management for Computing, Communication, Controls,
as a leak because it is a 10-minute continuous flow in the Energy and Materials (ICSTM). 2015, p. 258-263. Available from
doi:10.1109/ICSTM.2015.7225424.
pipe, and therefore a pipe burst or equipment failure
[7] PLATFORM, The Tech. What is Modbus? how its works? The
could have occurred. This graph was intended to show Tech Platform, 2022. Available from:
that the model can identify even relatively small water https://www.thetechplatform.com/post/what-is-modbus-how-its-
leaks. The data values shown in Graph 1 are the same as works
the data values shown in Graph 5, but represent different

979-8-3503-7069-0/23/$31.00 ©2023 IEEE 560

 AUTHOR INDEX
A E
G. Alexandrová 371 T. N. Eilifsen 129
J. Andraško 13
F
B
P. Feciľak 283,289,337,381,429,556
Z. Babicová 371 M. Filčáková 507
V. Bakonyi 19 M. Filipová 135
V. Balara 25
P. Balco 31,135 G
A. Balcova 31
L. Bednárová 421 D. Gabriska 141,409
I. Bélai 38,38 A. Gabrisova 68
R. Bencel 525 M. Galinski 147,225
V. Benko 301 F. Gerhát 415
M. Beňo 45 P. Getlík 295
E. Beňová 51 Z. Gyurász 153
D. Bilik 56
P. Bisták 200
H
L. Blahova 62,324 R. Haluška 159,165,171,341
D. Bobak 68 M. Harahus 485
S. Bohumel 225 M. Hasin 91
P. Bours 129,341,531 T. Havlaskova 176,243
G. Bugár 91 P. Helebrandt 330
P. Butka 76 M. S. Heng 341
D. Hládek 165,485,493
C M. Horváth 183
P. Campanella 83 R. Horváth 189
M. Čavojský 91 M. Hosťovecký 195
M. Čech 493 M. Hraška 237
M. Čerňanský 99 D. Hrinkino 440
E. Chovancova 403 J. Hrnčár 225
P. Čičák 542 M. Huba 200
M. Čierňava 111 V. Hubenáková 507
N. Čuboňová 111 A. Hudec 206,389
T. Hulina 463
D L. Huraj 212,440
J. Hvorecký 220
F. Delaneuville 31
M. Długosz 105,446 I
T. Dodok 111,301
P. Dražová 153 Z. Illés 19
D. Dubovec 117
A. Ďuriš 123,456
J. Džubák 283

561
J M. Kvassay 68,307
N. Kvassayova 307
F. Jakab 289,337,429,556 M. Kvet 117,395
M. Janeba 225
M. Janovec 231,237 L
T. Javorcik 176,243
K. Jelemenská 542 P. Lehoczký 56
A. Jelinek 68 S. Leibold 531
J. Juhár 485 Y. F. Liao 165,171
G. Juhás 249 A. Listvan 68
A. Juhásová 51,249 T. Loveček 451
J. Juraško 147 K. Lukáčová 507
M. Juricek 68 I. D. Luptakova 348
T. Jurík 264 I. D. Luptáková 212
J. Jurinová 258
M
K M. Mach 25
D. Kafka 264 K. Machova 25,353
O. Kainz 270,289 M. Madleňák 359
K. Kampová 359 A. M. Magiera 105,446
K. Kánová 51 M. Malčík 519
J. Kapusta 277 D. Maljar 389
V. Karcolova 324 S. Marchevský 159
K. Kardosova 68 Z. Mäsiarova 409
E. A. Katonová 283,289,381,556 M. Mattová 463,469,474
E. Kiktová 295 P. Mésároš 123,456
M. Kireš 507 M. Michalko 270,289,337,429,556
I. Klačková 301 M. Miklošíková 519
M. Klimo 307 K. Miková 365
F. Kolencik 479 J. Miňová 474
M. Kordík 13 A. Mišianiková 507
Š. Korečko 313,463,469,474 J. Miština 258
T. Kormaník 91,318 D. Molčanyi 165
S. Korom 19 L. Molitoris 371
K. Kostolanyova 176 M. Molokáč 371
K. Kostolányová 536 M. Moravcik 375,433
J. Kostolny 62,324 M. Murin 270,381
I. Kotuliak 56,225 M. Murín 556
A. Kotvan 330
J. Kozáková 507
N
K. Kozelková 507 K. Nalevanko 381
K. Krajníková 123 S. Nečeda 270
Z. Kubincová 365 P. Nehila 289,313
S. Kubinský 337 I. Nováková 270
T. Kuchcakova 403
P. Kulas 68
E. Kupcová 341

562
O B. Sobota 313,463,469,474
M. Sobota 479
M. Ölvecký 45 Z. Sokolová 485,493
S. Ondáš 171 L. Šoltés 147
R. Ondica 206,389 J. Staš 165,485,493
V. Ondová 507 W. Steingartner 469,474
M. Sterbak 433
P V. Stopjakova 206
V. Stopjaková 389
J. Papán 231,237
A. Straka 264
I. Pastierik 395
G. Stromp 469
P. Pekarcik 403
J. Stuchlik 499
R. Petija 289,381
L. Stuchlikova 499
L. Petrovič 249
L. Stuchlíková 479
E. Pietriková 183
M. H. Su 165,171
S. Pillár 159
D. Šveda 507
K. Pišútová 51
E. Švenk 171
M. Pleva 159,165,171,341,485
C. Szabó 469
M. Plostica 433
D. Szabó 19
M. Popovič 341
M. Szelest 105,446
P. Poremba 493
J. Porubän 318 T
J. Pospíchal 212
K. Pribilova 141 L. Tomaszek 519
K. Pribilová 45,409 A. Tomčala 525
T. Tomcik 353
R D. Tometzová 371
I. K. Torgersen 531
R. Ravasz 206
D. Tran 536
K. Révayová 51
P. Trúchly 147,277
V. Režo 415
R. Ručinský 123,456 V
R. Rybár 421
M. Vaclavkova 324
S A. Valach 542
T. Vincze 550
R. Sabol 429
P. Šaloun 519 W
M. Sarnovský 76
J. Saxa 264 M. Weis 415,479,499,550
A. Schmid 220 D. Wiecek 301
P. Segeč 264,375,433
B. Shrestha 129 Z
Z. Šimková 421
M. Šimon 440 D. Zadžora 556
K. Škotková 493 L. Zemanová 456
P. Skruch 105,446 L. Zemko 542
E. Skýpalová 451
J. Smetanková 123,456

563