TinyML for Edge Networks: Challenges and

Future Directions
1 Veera Manikantha Rayudu Tummala* , 2 Sonish Korada*, 3 Sai Pavan Lingamallu*,
4 Bandaru Sai Hari*, 5 Abhishek Hazra
1,2,3,4,5
Computer Science and Engineering, Indian Institute of Information Technology, Sri City, India
1
manikantharayudu.t21@iiits.in, 2 sonish.k21@iiits.in, 3 saipavanlingamallu@gmail.com
4
saiharibandaru2002@gmail.com, 5 abhishek.hazra1@gmail.com

Abstract—TinyML, deploying low-complexity Machine II. L IMITATION OF T RADITIONAL ML: W HY T INY ML?
Learning (ML) models on microcontroller units, which have
constrained the availability of resources, is revolutionizing
TinyML enables ML algorithms on low-resource devices
the technologies for extremely low-end devices. This study such as microcontrollers. We use compact models for energy-
investigates the importance, categorization, and uses of TinyML efficient inference to increase the decision-making rate on ad-
in edge networks, providing an overview of workflow and vanced IoT devices. TinyML can enable faster inference and
the importance of different compression techniques. TinyML reduce cloud or fog reliance, even with restricted resources.
can be applied in various industries such as healthcare,
Industrial Internet of Things (IIoT), and agriculture where it
It can also analyze sensor data on battery-powered devices
demonstrates the capability of doing local data processing and for continuous applications.
fast decision making which means need to provide the results • Resource Requirement : Traditional ML models require
there on spot. Nevertheless, TinyML encounters limitations considerable computing resources, especially memory
and challenges such as interoperability, memory usage, energy
efficiency, and standardization, which are also discussed in this
and processing capacity. Inadequate resources of an edge
article. device might limit the usage of traditional ML.
Index Terms—TinyML, Traditional ML, Internet of Things, • Energy Efficiency : Traditional model inference on edge
Model Compression Techniques devices can waste energy, especially when transferring
data and its updates to the cloud. TinyML saves energy
costs by doing local computations for energy efficient
I. I NTRODUCTION inference rather than sending data to cloud servers [3].
• Impact on Hardware : Deploying large ML models
The proliferation of Internet of Things (IoT) devices has on edge devices may produce physical side effects.
led to an explosion in data generation, and it is time for smart Complex ML algorithms may increase processor tem-
processing. Dispatching this vast amount of data to the cloud peratures and affect the performance of the device.
could be problematic because it brings more challenges and These cases need to be handled by the device in order
vulnerabilities, like data breaches and single points of failure for it to be reliable and sustainable in the long term at
[1]. These issues can be overcome by edge computing, a the edge level.
possible technology that can process data near its source.
Edge devices gather and transmit sensor data as quickly
and privately as possible. Nevertheless, providing high-end
Energy Efficient Reduced Latency
network services is difficult because of memory and power
boundaries at the edge level. This restriction requires the most Better battery lifespan Rapid data processing
optimal solutions to tackle complicated edge functionality. Low power intake Faster responses
Localized computations Low bandwidth requirement
Edge devices embedded with Machine Learning (ML) could
enhance data processing and decision-making speed at the
edge level. Traditional ML models need more computing
1 2 3 4
resources, which makes them infeasible to deploy at the edge
[2]. TinyML, based on Micro Controller Units (MCUs) in
edge devices, promises an intrinsic shift in which small ML Cost Effective Offline Working Mode
algorithms for resource-constrained components guide into a
On-device Analysis In remote environment
new embedded intelligence age. Less reliance on cloud Un-interrupted inference
No ample data transmission Assured reliability

978-1-6654-6883-1/22/$31.00 ©2022 IEEE.

*has equal contributions. Fig. 1: Major benefits of TinyML at the edge.
A. Features of TinyML The model is optimized by trimming out the weights that are
By deploying compact versions of ML models at the edge below this threshold level.
level, TinyML ensures scalability and simplifies IoT applica- Quantization reduces the connection’s weight precision
tions by lowering the need for continuous online connectiv- from converting floating-point values (32-bit or 64-bit) to
ity. Using fewer cloud services fosters privacy and respon- lower fixed-point numbers (8-bit), as fixed-point arithmetic
siveness. It allows for real-time on-device decision-making, is faster and more efficient than floating-point. Quantization
which is essential for sensitive applications like surveillance can be done either during or after training. Low precision re-
and time-critical applications like predictive maintenance. duces accuracy substantially.These methods can decrease the
TinyML also goes over distributed ML, which promotes storage memory required for the network parameters. Other
cooperative learning amongst nodes [1]. This enables decen- compression techniques exist, such as Low-Rank Factoriza-
tralized model training, such as federated learning, with low tion, which approximates a high-dimensional matrix with a
latency and the capability to deal with non-uniform online lower-dimensional one while maintaining information [5].
connectivity. It also preserves privacy and supports learning Huffman Coding employs binary codes to encode data values
from different data with privacy risks, potentially changing for lossless compression. This method is preferable when
the IoT landscape. transmitting machine parameters like weight biases from
edge to cloud because frequent data symbol codes are shorter,
B. Benefits of TinyML in Edge Networks thus taking up less space.
While localized processing helps to cut down on the costs Besides model compression, knowledge distillation is one
associated with plugging into the cloud, tinyML presents this of the approaches that can be used in TinyML. In this case, a
approach for edge networks that really makes a dent in terms student model, with fewer parameters, is created to emulate
of expense. It is therefore cost-effective to run it inference the performance of a more parameterized teacher model.
on the edge vs. doing so in the cloud. Ability to go energy The procedure starts with the training of the teacher model
efficient by managing processes locally, executing operations on a large dataset, which achieves high accuracy due to
on low power edge device saves power, and hence extend the its capacity.The student model is then trained in order to
battery life. equal the output of the teacher model. Such method lowers
Reducing the duration of data transmission for remote model computational and memory requirements without loss
analysis accelerates decision-making [4]. So, it improves the of performance.
productivity of edge networks, especially in tasks such as
IV. T INY ML C LASSIFICATION AT E DGE
autonomous vehicles and industrial automation. These help
to function in areas with restricted or unreliable network con- This section covers Static and Reformable TinyML cate-
nectivity by minimizing the reliance on external connections. gorization methods for edge devices in various deployment
As mentioned in Fig. 1, TinyML removes the need for a scenarios.
continuous internet connection. This feature guarantees con-
A. Static TinyML
tinuous operation and processing, even without connectivity
for remote or disconnected operations [5]. Keeping inferences Static TinyML is a fundamental technique used to deploy
at the local level and focusing on only transmitting a few ML models on edge devices [1]. It mainly follows the
relevant insights reduces network congestion and optimizes “first-train-then-test” principle. Once the model is trained
bandwidth utilization. TinyML improves the robustness of and deployed on the device, updateability is unlikely. The
edge networks by prioritizing inference tasks over other deployed model does not support upgrades depending on
operations. new data or feedback. During the offline training phase on
powerful hardware or in the cloud, the model’s parameters are
III. W ORKING OF T INY ML AT E DGE set and consistent during deployment. While not appropriate
Fig. 2 illustrates the comprehensive workflow of TinyML for dynamic surroundings or adaptive applications, Static
in multiple steps, from data collection and preprocessing TinyML offers a simple and fast solution for picture catego-
to model deployment and inference, describing key pro- rization, keyword detection, and basic sensor data processing
cesses and important aspects involved in each step. Model on resource-constrained edge devices.
optimisation forms a critical element in the development
of a TinyML model. Once the training of the model is B. Reformable TinyML
carried out, optimization techniques such as Quantization, The Reformable TinyML approach to deliver ML models
Pruning, Huffman Coding, and Knowledge Distillation are on restricted hardware is revolutionary and supports ongoing
handled with care so that the model could perform within improvement through dynamic updates [1]. Unlike static im-
the desired accuracy limits, size, and power consumption. We plementations, reformable models can improve performance
apply these techniques to make the model more suitable for locally or over the air. IoT environments with fluctuating data
running on targeted edge devices. Pruning involves training distributions benefit from their dynamic nature. Reformable
a neural network and selecting pivotal synapses by pointing TinyML updates keep edge computation efficient and reduce
out weights above a specific fixed threshold or loss function. model drift. The decision between Reformable and Static
 Data
Manipulation

Support Vector K-Nearest Model

Decision Trees Neural Networks
Machines Neighbours Development

Using Along with

Model Training Using Pytorch
TensorFlow Sci-kit Learn

Weight Huffman Knowledge Pruning Quantization Model

Clustering Coding Distillation Technique Technique Optimization

Model Power Prediction Model

Evaluation Model Size
Consumption Latency Accuracy

Using
Hardware Using Model
Compatability TensorFlow Lite
Integration TensorFlow Lite Deployment
Macro
Text
Sensors

Real-time Edge Continuous More Text

Inference Predictions Computing Monitoring Secure & Private

Edge Level
Text

Fig. 2: General working process of TinyML.

inferences extremely fast. Excited learners freeze the

TinyML
TinyML model weights and adaptively alter the biases to adjust
over new data. The authors in [6] have introduced
Tiny Transfer Learning(TinyTL) that freezes weights
and gradually learns biases, minimising memory con-
Static Reformable
Reformable
TinyML sumption during training.
TinyML TinyML
b) Lazy Learning Techniques: Certain algorithms save
training data until testing data becomes accessible
and provide predictions based on stored samples.
Networking Reliant Online
OnlineLearning
Learning In [7], the authors implemented Online Learning on
Networking Reliant On-device Offline
Approaches Approaches
Approaches
Approaches Learning Approaches
Raspberry Pi 3B+ and STM32F7, employing a non-
trainable deep learning (DL)-based Feature Extractor
Eager
and kNN classifier for binary tasks. kNN yielded
Distributed
DistributedMachine
Machine Eager
Learning
Learning slightly lower accuracies than SVM on Pi, with the
Learning
LearningApproaches
Approaches
feature extractor showing longer processing times on
Over-the-air
Lazy STM32F7.
Learning
Approaches
2) Network Reliant Approaches:
a) Distributed TinyML: These methods enable collabo-
Fig. 3: Classification of TinyML.
rative model training on decentralised data sources,
using federated or gossip learning mechanisms for
TinyML depends upon the application’s requirements, avail-
privacy and edge computing. A distributed TinyML
ability of resources, and the intended trade-off between model
model can be trained locally on edge devices and
intricacy and adaptability. Based on different techniques used
aggregated on a central server for global updates.
to update the model, it can be further classified as follows:
Globe2Train framework by Sudharsan et al. [8] lets
1) Online Learning Approaches: geographically distributed IoT devices train a single
a) Eager Learning Techniques: Requires greater compu- model. Globe2Train is driven by GPU’s scarcity com-
tational resources and longer training time, they make pared to IoT devices. G2T-Cloud and G2T-Device
 operate together to promote collaborative training in it faster, less latency concentration. TinyML is helping to
the proposed system. achieve better industrial processes in the era of smart manu-
b) Over-the-air (OTA) Approaches: Allow wire-free facturing by optimising condition monitoring and predictive
model updates to be distributed remotely and format- maintenance.
ted without requiring physical access to the device.
C. TinyML & Agriculture
Over time, OTA updates are essential to the model’s
security and performance. Badawy et al. introduced TinyML is significantly changing smart agriculture, rev-
FOTA to wirelessly flash ATMEL AVR MCUs in [9]. olutionizing crop monitoring, disease detection, and yield
This is done with Wifi and LoRa, where FOTA allows optimisation [5]. TinyML can greatly enhance regions like
dynamic TinyML updates. Africa, where embedded systems and AI are underutilized.
The Nuru programme by PlantVillage1 utilises algorithms
V. U SECASES & A PPLICATIONS based on TinyML and can be accessed through mobile
The combination of TinyML and IoT opens a vast amount phones. This empowers farmers in rural Africa to analyse
of possibilities across various industries, such as healthcare, sensory data in real time and effectively manage crop threats
agriculture, industrial IoT, UAVs or drones and autonomous such as cassava. Furthermore, TinyML efficiently monitors
cars etc. This versatility and efficiency propagate innovation coffee beans during roasting, improving the beans’ over-
by other sectors that depend on IoT or embedded technol- all quality. TinyML enables advanced prediction models to
ogy [10]. TinyML has the potential to spark a revolution provide farmers with real-time meteorological data, allow-
across multiple disciplines, and this is due to its real-time data ing them to make well-informed decisions. The agriculture
handling capacity and decentralised design, which means it industry adopts sustainable methods and optimises resource
can be carried out at any time. Below are several use case utilisation by incorporating TinyML into IoT devices, thereby
scenarios illustrating the application of TinyML in various addressing the needs of a growing population.
domains. VI. O PEN R ESEARCH C HALLENGES
A. TinyML & Healthcare Research into the limitations of TinyML and the future
TinyML has the potential to transform healthcare [5] by directions in an edge-wide scope are needed for this technol-
improving monitoring and personal health products. Wear- ogy to realize its full potential. It is important to recognize
able devices have various biological sensors that constantly this myriad of opportunities and how they connect with other
monitor important bodily processes such as heart rate, blood areas, a few key ones are outlined here.
oxygen levels, and activity. These gadgets provide imme- A. Interoperability
diate and confidential information about the user’s health.
As the heterogeneity of the devices at the edge is in-
TinyML allows for extensive real-time data analysis using
creasing daily, things like outdated standards and integration
compact, pre-trained inference models, thus avoiding contin-
compatibility issues hinder interoperability. Even though pre-
uous streaming. In addition, smart camera sensors powered
vailing partially, difficulties with categorizing, scaling, and
by TinyML allow for real-time diagnosis, nurse alerts, and
emerging innovations still exist. Advanced methods and stan-
personalised treatment. Intelligent microprocessors that lever-
dards are needed for interoperability testing, and integration.
age TinyML effectively predict the likelihood of accidents,
improving patient monitoring, diagnosis, and treatment. This B. Resource Constraints
integration improves healthcare quality through increased The limiting tools are devoid of memory, performance
monitoring and faster and cost-effective patient care. requirements, and processing capability; therefore, certain
B. TinyML & Industrial IoT sacrifices regarding precision or functionality should be made
to maximize these resources. Future research will focus on
This transformation, driven by TinyML and Industrial IoT, optimization methods that tend to save resources, increase
is revolutionizing the Industrial Internet of Things (IIoT) throughput, and improve energy efficiency, as well as the
by pushing the envelope of automation, connectivity, and use of hardware accelerators and domain-specific designs.
intelligent sensing. In the context of Industry 4.0 digital Resource-aware, adaptive, and optimal algorithms and meth-
transformation, TinyML benefits from these methods by ods for task offloading to cloud and fog environments
enabling predictive data analysis on embedded gadgets [11]. will be the deciding factors. Improving Neural Architecture
This enhances live analysis, which is not possible given Search (NAS) techniques in particular for the deployment
certain time constraints. Integrating ML into microcontrollers and inference of TinyML can be deployed towards optimal
eases making smart decisions and hence, attains enhanced neural network architecture design automatically, satisfying
production efficiency and monitoring assets near real-time the constraints of tiny devices. MCUs utilize a minimal
with the desired quality assurance. Anomaly detection falls memory unit and are, therefore, incapable of doing much
under the purview of tinyML, which is concerned with more than inference; the consequence is slowed response
identifying irregular patterns in industrial machinery items
during production. Deploying TinyML to edge devices makes 1 https://plantvillage.psu.edu/
times and adversely affects scalability. Model compression [9] W. Badawy, A. Ahmed, S. Sharf, R. A. Elhamied, M. Mekky, and
also proves useful but requires balancing of compressed- M. A. Elhamied, “On Flashing Over The Air “FOTA” for IoT Ap-
pliances – An ATMEL Prototype,” in 2020 IEEE 10th International
to-original parameters. The research, therefore, should be Conference on Consumer Electronics (ICCE-Berlin), 2020, pp. 1–5.
carried out on integrated hardware-software in memory sys- [10] C.-H. Wang, K.-Y. Huang, Y. Yao, J.-C. Chen, H.-H. Shuai, and W.-
tems, new storage designs, and importantly, efficient data H. Cheng, “Lightweight deep learning: An overview,” IEEE Consumer
Electronics Magazine, vol. 13, no. 4, pp. 51–64, 2024.
structuring. [11] M. Sharma, A. Tomar, and A. Hazra, “From Connectivity to Intelli-
gence: The Game-Changing Role of AI and IoT in Industry 5.0,” IEEE
C. Optimal Trade-off For Energy Usage Consumer Electronics Magazine, pp. 1–7, 2024.
Given that environmental impacts are an issue, it is im-
portant to have the right balance in energy limits and form
and functionality trade-off. And the important future updates
in TinyML should be low-power circuits, efficient design
and smart task load managing to achieve high-performance
with energy savings. TinyML applications at the edge level
could be more trusted using techniques like energy harvesting
(by scheduling and forecasting methods), thermal control and
approximation computation.
VII. C ONCLUSION
Nowadays, TinyML can assist us in resolving the issues
we’re experiencing with low-end devices at the edge level.
This technology has created an impact and is advancing
swiftly, but it is still in its initial implementation phases and
needs more development. There should be a standard trade-
off between what we can expect and what it produces at the
edge. To exploit TinyML’s benefits even more, procedures
and frameworks must be standardized. This article discussed
how a traditional ML model can be turned into a TinyML
model and the various types of TinyML suited to different
circumstances. It also presented views about how TinyML is
being used in innovative solutions and what complications or
research needs we have alongside potential solutions.
R EFERENCES
[1] V. Rajapakse, I. Karunanayake, and N. Ahmed, “Intelligence at the
Extreme Edge: A Survey on Reformable TinyML,” ACM Computing
Surveys, vol. 55, no. 13s, pp. 1–30, 2023.
[2] A. Hazra, A. Kalita, S. Deb, M. Gurusamy, and V. G. Menon, “Cogni-
tive Computing and Machine Intelligence in Fog-Cloud Infrastructure
for Industry 5.0,” IEEE Consumer Electronics Magazine, pp. 1–6,
2024.
[3] M. A. Togou, T. Bi, K. Dev, K. McDonnell, A. Milenovic, H. Tewari,
and G.-M. Muntean, “DBNS: A Distributed Blockchain-Enabled Net-
work Slicing Framework for 5G Networks,” IEEE Communications
Magazine, vol. 58, no. 11, pp. 90–96, 2020.
[4] N. Schizas, A. Karras, C. Karras, and S. Sioutas, “TinyML for ultra-
low power AI and large scale IoT deployments: A systematic review,”
Future Internet, vol. 14, no. 12, p. 363, 2022.
[5] Y. Abadade, A. Temouden, H. Bamoumen, N. Benamar, Y. Chtouki,
and A. S. Hafid, “A comprehensive survey on tinyml,” IEEE Access,
vol. 11, pp. 96 892–96 922, 2023.
[6] H. Cai, C. Gan, L. Zhu, and S. Han, “Tinytl: Reduce memory, not
parameters for efficient on-device learning,” in Advances in Neural In-
formation Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell,
M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020,
pp. 11 285–11 297.
[7] S. Disabato and M. Roveri, “Incremental On-Device Tiny Machine
Learning,” 11 2020, pp. 7–13.
[8] B. Sudharsan, J. G. Breslin, and M. Intizar Ali, “Globe2Train: A
Framework for Distributed ML Model Training using IoT Devices
Across the Globe,” in 2021 IEEE SmartWorld, Ubiquitous Intelligence
& Computing, Advanced & Trusted Computing, Scalable Computing
& Communications, Internet of People and Smart City Innovation
(SmartWorld/SCALCOM/UIC/ATC/IOP/SCI), 2021, pp. 107–114.