UNIT4 DATA ANALYTICS

Introduction to TensorFlow & PyTorch and Background Knowledge of ML Data

Analytics for IoT Basics Data Analytics for IoT Regression & ANN-based
Classification Data Analytics for IoT Regression & ANN-based Classification Data
Analytics for IoT Modern DNNs and their Programming in TF.
 DATA ANALYTICS

What is data analytics?

• Data analytics is the science of analysing raw data to make conclusions about that
information.

• Many of the techniques and processes of data analytics have been automated into
mechanical processes and algorithms that work over raw data for human
consumption.
Why is data analytics important?

•Gain more visibility and a deeper understanding of their processes and services.

•It gives them detailed insights into the customer experience and customer problems.

•personalized customer experiences, build related digital products, optimize

operations, and increase employee productivity.

•helps businesses optimize their performances.

•help reduce costs by identifying more efficient ways of doing business.

•to make better business decisions and help analyze customer trends and satisfaction,
which can lead to new—and better—products and services.
 Types of data Analytics

1. Predictive data analytics- Eg: Tshirt with target audience’s in facebook

2. Prescriptive data analytics – Eg: Trying possible solutions and Choosing the best option

3. Diagnostic data analytics – Eg: Analysis the past and predicting future

4. Descriptive data analytics -Eg: Analyse a query and improving

 Introduction to TensorFlow
• symbolic math library used for neural networks

• building and training models

• can train and run the deep neural networks for image recognition,
handwritten digit classification, recurrent neural network, word
embedding, natural language processing, video detection, and many more.

• TensorFlow is run on multiple CPUs or GPUs and also mobile operating

systems.
 History of TensorFlow
• Many years ago, deep learning started to exceed all other machine learning algorithms when
giving extensive data. Google has seen it could use these deep neural networks to upgrade its
services:
• Google search engine
• Gmail
• Photo
• They build a framework called TensorFlow to permit researcher’s and developers to work
together in an AI model. Once it approved and scaled, it allows lots of people to use it.
• It was first released in 2015, while the first stable version was coming in 2017. It is an open-
source platform under Apache Open Source License.
• We can use it, modify it, and reorganize the revised version for free without paying anything to
Google.
• The word TensorFlow is made by two
words, i.e., Tensor and Flow
• Tensor is a multidimensional array
• Flow is used to define the flow of data in
operation.
• TensorFlow is used to define the flow of
data in operation on a multidimensional
array or Tensor.
 Components of TensorFlow
• Tensor - The name TensorFlow is derived from its core framework, "Tensor." A tensor is
a vector or a matrix of n-dimensional that represents all type of data. All values in a tensor
hold similar data type with a known shape. The shape of the data is the dimension of the
matrix or an array.

• Graphs - TensorFlow makes use of a graph framework. The chart gathers and describes
all the computations done during the training.

• Session- A session can execute the operation from the graph. To feed the graph with the
value of a tensor, we need to open a session. Inside a session, we must run an operator to
create an output.
 Applications
Voice/Sound Recognition
• Voice recognition is used in the Internet of Things, automotive, and security.

• Sentiment Analysis is mostly used in customer relationship management (CRM).

• Flaw Detection (engine noise) is mostly used in automotive and Aviation.

• Voice search is mostly used in customer relationship management (CRM)

Image recognition
• For example, deep learning uses TensorFlow for analyzing thousands of photos of cats.
So a deep learning algorithm can learn to identify a cat because this algorithm is used to
find general features of objects, animals, or people
Time Series
• For example, it can be used to recommend us TV shows or movies that people like based on
TV shows or movies we already watched.

Video Detection
• For example, NASA is developing a deep learning network for object clustering of asteroids
and orbit classification. So, it can classify and predict NEOs (Near Earth Objects).

Text-Based Applications
• For example, Google Translate supports over 100 languages.

• Some companies who are currently using TensorFlow are Google, Intel, DropBox, Deep
Mind, Snapchat, SAP, Uber, Twitter, and IBM.
 Introduction to PyTorch
History of PyTorch

• PyTorch is a relatively new deep learning framework based on Torch.

• Developed by Facebook’s AI research group and open-sourced on GitHub in 2017, it’s used for
natural language processing applications.

• Natural Language Processing - enables machines to understand and respond to text or voice data.

• PyTorch has a reputation for simplicity, ease of use, flexibility, efficient memory usage, and
dynamic computational graphs.

• It also feels native, making coding more manageable and increasing processing speed.
 Deep Learning Frameworks
• PyTorch was developed to provide high flexibility and speed
during implementing and building the Deep Learning Neural
Network.

• It is a machine learning library for Python programming

language

• PyTorch is completely pythonic (using widely adopted python

idioms rather than writing Java and C++ code) so that it can
quickly build a Neural Network Model successfully.
 Basics of PyTorch
Common PyTorch Modules are used to represent neural networks.
Autograd
• The autograd module is PyTorch’s automatic differentiation engine that helps to compute the
gradients in the forward pass in quick time.
Optim
• The Optim module is a package with pre-written algorithms for optimizers that can be used to build
neural networks.
nn
• The nn module includes various classes that help to build neural network models.
• All modules in PyTorch subclass the nn module.
 Background Knowledge of ML
Data Analytics for IoT Basics:

• The Internet of Things (IoT) is a network of interconnected devices & gadgets that can
collect & share data by themself.

• IoT data analytics refers to the procedure of gathering, examining, and deciphering data
produced by these devices to gain knowledge and make wise decisions.

• Data analytics uses bunches of hardware, software, and data science techniques to collect
accurate information from massive data created by IoT devices.
 Concept and components
1. IoT:
• network of physical objects or "things" embedded with sensors, software, and other
technologies that enable them to collect and exchange data over the internet.
• Eg: smart devices and wearable fitness trackers to industrial machinery and autonomous
vehicles.
2. Data Collection and Sensors:
• IoT devices collect data from the physical world through various types of sensors such as
temperature sensors, humidity sensors, accelerometers, cameras, GPS modules, etc.
• These sensors continuously gather data from the environment and transmit it to central servers
or cloud platforms.
3. Data Storage:
• The data collected from IoT devices is often stored in databases or cloud storage systems.
• This data can be structured or unstructured, and it may be generated in real-time or in batches.
4. Data Preprocessing:
• Raw IoT data can be noisy, incomplete, or contain outliers.
• Data preprocessing involves cleaning, transforming, and aggregating the data to make it suitable
for analysis.
• This step is crucial for ensuring the accuracy and reliability of subsequent analysis.
5. Feature Engineering:
• Selecting, transforming, or creating relevant features (variables) from the raw data that will be
used as input for machine learning algorithms.
• Effective feature engineering can significantly impact the performance of ML models.
6. Machine Learning Algorithms:
• Used to analyze IoT data and uncover patterns, trends, and relationships.
• Common ML algorithms used in IoT analytics include regression, clustering, classification, time
after series analysis, and deep learning techniques like neural networks.
7. Model Training and Evaluation:
• the model is trained on historical IoT data to learn patterns and relationships.
• The model's performance is evaluated using various metrics to ensure it can make accurate
predictions or classifications on new, unseen data
8. Real-Time Analytics:
• ML models can be deployed on edge devices or cloud platforms to provide real-time insights and
predictions, enabling rapid decision-making and timely responses to events.
9. Anomaly Detection:
• ML techniques can be used to detect anomalies or unusual patterns in IoT data.
• This is particularly important for identifying potential issues or security breaches in real time.
10. Predictive Maintenance:
• ML models can predict when IoT devices or equipment are likely to fail, allowing for proactive
maintenance and minimizing downtime.
11. Energy Efficiency:
• ML algorithms can optimize energy consumption in IoT devices by analyzing usage patterns and
making intelligent decisions to conserve energy.
12. Data Security and Privacy:
• IoT data often contains sensitive information.
•Ensuring data security and privacy is crucial, and ML techniques can be employed to detect and
mitigate potential security threats.
13. Visualization and Reporting:
• Data visualization tools can create intuitive graphs, charts, and dashboards to convey insights
from IoT data to non-technical stakeholders effectively.
14. Scalability:
• IoT generates vast amounts of data, and ML algorithms need to be scalable to handle this
volume of information efficiently.
15. Cloud Services:
• Cloud platforms provide the infrastructure for storing, processing, and analyzing IoT data.
• Services like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud offer
specialized tools for IoT analytics.
Data Analytics for IoT Regression:

• It involves using regression techniques to analyze and model relationships between

variables in Internet of Things (IoT) data.

• Regression is a supervised machine learning technique that aims to predict a continuous

numeric output based on input variables.

• Regression can help uncover patterns, trends, and dependencies within the data, enabling
better decision-making, prediction, and optimization.
key concepts and actions involved in utilizing regression techniques for analyzing Internet of
Things (IoT) data:

1. Problem Definition and Data Collection:

• Define the specific problem you want to address using IoT data and regression analysis. For
instance, you might aim to predict energy consumption based on temperature and time of day data.
• Collect data from IoT devices equipped with relevant sensors. This could include temperature
sensors, humidity sensors, time-stamped data, and any other pertinent measurements.

2. Data Preprocessing:
• Clean the raw IoT data by handling missing values, outliers, and noise. Impute missing values
using appropriate methods, and remove or correct outliers that could adversely affect the analysis.
• Normalize or standardize the data if necessary. This ensures that different variables with different
scales do not unduly influence the regression model.
3. Feature Selection and Engineering:
• Identify the variables (features) that you believe could influence the target variable (e.g., energy
consumption). These could be sensor readings, time-related attributes, or any other relevant data.
• For example, you might extract day of the week, hour of the day, or seasonal indicators from
timestamp data to capture temporal patterns.
4. Splitting the Data:
• Divide the preprocessed data into two sets: a training set and a testing set. The training set is
used to build and train the regression model, while the testing set evaluates the model's
performance.
5. Choosing a Regression Algorithm:
• Select an appropriate regression algorithm based on the nature of your data and the problem at
hand. Depending on your data's complexity and the relationship you're trying to model, you
might choose linear regression, polynomial regression, or another method.
6. Model Training and Hyperparameter Tuning:
• Train the chosen regression model on the training data. During training, the model learns the
relationships between the input features and the target variable.
• Tune hyperparameters, such as regularization strength in Ridge or Lasso regression, to
optimize the model's performance. Cross-validation techniques can help determine the best
hyperparameters.
7. Model Evaluation:
• Use the testing set to evaluate the trained regression model's performance. Calculate metrics
like Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error
(MAE), and R-squared to assess how well the model predicts the target variable.
• Analyze any discrepancies between predicted and actual values to identify potential areas for
improvement.
8. Model Interpretation:
• Interpret the coefficients of the regression model to understand the impact of each feature on
the target variable. Positive or negative coefficients indicate the direction and magnitude of
influence.
9. Prediction and Insights:
• Deploy the trained regression model to make predictions on new, unseen IoT data. For
example, you can predict future energy consumption based on real-time sensor readings.
• Extract insights from the model about how different factors contribute to the predicted
outcome. This can aid in decision-making and resource optimization.
10. Continuous Monitoring and Refinement:
• Continuously monitor the performance of the regression model as new IoT data becomes
available. Retrain the model periodically to incorporate new information and ensure its
accuracy over time.
• Refine the model by updating features, incorporating new variables, or exploring more
sophisticated regression techniques as your understanding of the problem evolves.
 ANN-based Classification Data Analytics for IoT Regression
What is Artificial Neural Network?
• Biological neural networks that develop the structure of a human brain.
• Similar to the human brain, artificial neural networks also have neurons that are interconnected to one
another in various layers of the networks.
• These neurons are known as nodes.
The architecture of an artificial neural
network
Types of Artificial Neural Network
Steps involved in ANN-based Classification Data Analytics
for IoT Regression
• Step 1: Data Collection
• Step 2: Data Pre-processing
• Step 3: Regression Model Training
• Step 4: Classification Model Training
• Step 5: Combining Regression and Classification
• Step 6: Anomaly Detection and Insights
• Step 7: Visualization and Reporting
• Step 8: Continuous Learning
• Step 10: Scalability and Efficiency
• Step 11: Privacy and Security
 ANN-based Classification Data Analytics for IoT Modern DNNs
• Step 1: Data Collection and Preprocessing:
• Step 2: Deep Neural Network Architecture:
• Step 3: Model Training:
• Step 4: Feature Sharing:
• Step 5: Combining Regression and Classification:
• Step 6: Anomaly Detection and Insights:
• Step 7: Visualization and Reporting:
• Step 8: Continuous Learning and Adaptation:
• Step 9: Evaluation and Optimization:
• Step 10: Scalability and Efficiency:
• Step 11: Privacy and Security:
• Step 12: Incorporating Modern Techniques:
• Step 13: Edge Computing Integration:
• Edge computing reduces latency and conserves network bandwidth.