MTP p2 Aashish
MTP p2 Aashish
disaggregation
Master of Technology
by
Electrical Engineering
INDIAN INSTITUTE OF TECHNOLOGY BOMBAY
July 2023
Dissertation Approval
Low Power Programmable CMOS Buck-Boost Regulator for Energy Harvesting Systems
by
Aashish Pandia
Roll No. 203196001
..................................................... .....................................................
Prof. Anupama Kowli Prof. Mukul Chandorkar
(Supervisor) (Examiner)
..................................................... .....................................................
Prof. . Prof. .
(Chairman) (Examiner)
i
Declaration
I declare that this written submission represents my ideas in my own words and where others’
ideas or words have been included, I have adequately cited and referenced the original sources.
I also declare that I have adhered to all principles of academic honesty and integrity and have
not misrepresented or fabricated or falsified any idea/data/fact/source in my submission. I un-
derstand that any violation of the above will be cause for disciplinary action by the Institute and
can also evoke penal action from the sources which have thus not been properly cited or from
whom proper permission has not been taken when needed.
..................................
Aashish Pandia
Roll No. 203196001
ii
Acknowledgements
I would like to express my sincere gratitude to Prof. Anupama Kowli for her valuable guidance,
constant support and kindness, & motivation at each and every step of this work. I would like to
thank her for her insightful suggestions which helped me in smooth and successful completion
of the project.
I would like to thank Sust Labs. for the field data and technical insights they provided
during the course of the thesis.
I would also like to thank Technocraft center for Applied Artificial Intelligence (TCAAI)
for funding this research.
Lastly, I thank all the wonderful people who kept making me learn new things everyday.
iii
Abstract
Domestic electricity usage accounts for around a quarter of the total electricity consumption
in the country. A detailed breakdown of energy use by different household appliances has
many applications – it can be used to design better tariffs, and/or identify opportunities for
demand response. It can also help consumers better understand their own consumption so as
to better manage the end-use loads and leverage available incentives such as time-of-use tariffs.
Appliance level monitoring can provide such breakdowns but it is expensive and increases the
sensor footprint. Prior work has established that the energy, current, and voltage signatures of
typical home use appliances exhibit distinct patterns. This thesis investigates the use of AI and
ML to leverage these distinctions and disaggregate appliance power signals at a single point-
of-connection. The majority of industry based solutions work on the cloud and dis-aggregates
energy demand on monthly timescales. This thesis will try to extend the product capabilities
to facilitate online disaggregation for better demand-side management, a first proof-of-concept
algorithm is provided here which is lightweight and allows the possiblity of AI/ML applications
on the Edge, this architecture tries to detect On-off and Intermittent loads (FSM type) so that
clean data can be made available for cloud inference as well as provide better system-wide
decision-making.
iv
Contents
Abstract iv
1 Introduction ix
1.1 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
1.2 Proposed improvements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
3 Load Disaggregation xx
3.1 Motivation for a house wise model . . . . . . . . . . . . . . . . . . . . . . . . xx
3.2 NILM: a state based approach . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii
4 Methodology xxvii
4.1 Data cleaning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii
4.2 Unsupervised Learning for FSM type Loads . . . . . . . . . . . . . . . . . . . xxix
v
6 Future work xl
6.1 Limitations of current work . . . . . . . . . . . . . . . . . . . . . . . . . . . . xl
6.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xli
References xlii
vi
List of Figures
vii
5.2 FSM loads in different scenarios example . . . . . . . . . . . . . . . . . . . . xxxvi
5.3 FSM extracted by NILM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxvii
5.4 Discovered FSM faulty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxvii
5.5 Discovered FSM correct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxxviii
viii
Chapter 1
Introduction
In India, domestic power use makes up about 24% of the nation’s overall electricity consump-
tion. Studies that break down the energy use of various home appliances offer significant in-
sights. For instance, while lighting accounts for around 20% of the demand, air conditioners,
water heaters, fans, and air coolers account for more than half of the overall electricity use. [1]
The inhabitants of an energy-aware house can benefit from these insights if they have proper
feedback such as the consumption of each appliance in the sum of the total billing.
The operating condition of appliances used in a home cannot be truly determined without
the proper monitoring system. In general terms, load monitoring is the process of identifying
and acquiring the load measurement in a power system. This load monitoring will determine
the consumption and appliances’ status, in order to comprehend the behavior of individual loads
in the whole system. [2]
Depending on the approach used for monitoring appliances, the load monitoring can be
classified into
ix
Non - Intrusive Load Monitoring
• Cheaper and easier installation, since typically only one metering device is used.
Energy disaggregation, a quantitative method for calculating the power demand of individ-
ual appliances from a single meter that measures the total demand across numerous appliances,
is another name for NILM.
Introduced by Hart et al. [3] the basic problem statement is, given the aggregate power
consumption can we find the power of the individual appliance.
N
Pt = ∑ Pti + Et
i=1
where,
Pt = Total power consumed at tth time instant
Pt i = Power consumed by ith appliance at tth time instant
Et = measurement noise at tth time instant
x
1.1 Literature review
Each appliance has steady-state features as well as transient features, depending on sampling
frequency these features can be used to distinguish them, transient features need high sampling
rate of the meter while steady-state features can be observed after one-tenth to one-third of a
second depending on the appliance, a low sampling rate of about 1Hz is sufficient to capture
these. Some non traditional features(like time of day, ambient temperature etc.) are also used
in current state-of-art techniques. [4] [5]
In current literature various techniques based on high frequency sampling are used which
are able to identify unique appliance loads using their transient waveforms[6]. Use of high
frequency sampling gives better resolution when similar types of loads are present because of
involvement of higher harmonics and use of Fourier analysis like techniques but both acquir-
ing the data and the space and computation time needed for analysis of data sampled at such
a high rate are difficult task in themselves. Low frequency sampling is enough to disaggregate
appliances in ∆I-∆φ domain the data acquisition is much simpler and cheaper compared to high
frequency sampling and for these reasons low frequency sampling is preferred.
Figure 1.2: Relation between appliance disaggregation possibilty and data frequency
Many Machine learning based techniques are in practice to get the task of load disaggrega-
tion done, the NILM can be proposed as a classification problem where we have to detect which
xi
Sampling
Ref Fre- Features
ML Method Appliances of interest Remarks
No. quency Used
Used
LSTM, Denois-
kettle fridge washing
1
[7] 6 Hz P, Q ing Autoencoder, only major loads
machine dishwasher
Specific deep NN
training on PC
supervised dryer washing ma-
implementation
1
[8] 60 Hz P regression, mo- chine dishwasher
on Edge mobile
bileNet microwave cooker
device
1 Hz microwave coffee ma- classification of
[9] and 15 P, Q, φ KNN, SVM chines toaster incan- light load at high
KHz descent lamp LCD TV freq
use of real world
fan, lamp, hair dryer,
[10] 10 Hz VI k-NN Classifier voltage varia-
LED light
tions
hair dryer fridge
K-NN, Gaussian washing machine variety of loads
30 I, P, Q, V-I
[11] Naive Bayes, microwave AC CFL at very high fre-
KHz trajectory
SVM fan heater bulb laptop quency sampling
vacuum
appliances are active at a time t, classify ON and OFF, or it can be represented as a regression
problem in which one estimates how much each appliance contributes to the total percentage of
consumption. [12] Techniques based on supervised methods (like pattern recognition), unsu-
pervised clustering, [13] HMM, [14] CNNs, LSTMs and Deep learning are already in practice
to solve these problems of regression and classification. [15]
There are a few edge based solutions in the literature also, these edge devices use similar
ML techniques but detects a reduced number of appliances or their overall disaggregation ac-
curacy suffers either because of computational power limitation or lack of storage space. Event
based detection is commonly seen in such methods where inferencing only happens if a large
xii
change is detected in the features. Edge based NILM techniques use pruning methods to re-
duce the overall storage needed for rule set[16]. Moreover, powerful devices like RaspberryPi
or android mobile device are used or in-house developed complete system are used for load
inferencing[16][8][17].
xiii
Chapter 2
xiv
2.2 The Simulator
Motivated by the need, a simulator is designed to simulate the behaviour of a house as seen
from a single PoC, like a real-world house’s energy meter. In the model of this simulator,
just like a real house, the loads are connected parallel to each other and between the Mains
and the neutral terminal. When the switch corresponding to a load is connected the load is
considered in operating state (ON) and it will draw power according to its rating, else the load
is in disconnected state(OFF) no power is drawn and that branch is open circuited.
For simulations, different appliances need to be modeled, each appliance is modeled as a series
combination of resistance R and inductance X, further, the appliances having multiple operating
states in themselves (like a Fridge) are modeled as having separate R-X combinations for each
state which can be switched internally by a selection switch.
xv
The R and X values of different loads are obtained from ACS data set via the use of data-
driven methods. The ACS Data set contains 1-hour monitored electrical recording of appliances.
The data set contains recordings of the values of voltage, current, phase, active power, reactive
power, and frequency for a type of appliance. The R and X values are extracted from this data
set using formulas
V×V
S=
Z̄
Z̄ = R − ιX
The simulator takes the number of branches in the house connected in parallel between
Mains and the neutral lines, the type of load i.e. the R and X values of each branch and the time
series of the state of operation of each branch for a total time duration of ’T’ as inputs.
The simulator then simulates the electrical appliances in the house at a duration of 1 sec
interval, only the steady state behaviour is of importance, the transient behavior is ignored as
this behaviour of appliances settles well within one sec. and provides the total AC current
magnitude, phase AC current, total active power, and total reactive power consumed at t th time
instance, as observed at the PoC. The mathematical equations being simulated are as follows
S = V I∗
xvi
N
I = ∑ Ii
i=1
where Ii is the complex current in branch i of the system.
The simulator also gives the value of branch currents,and power for all branches at all times for
further analysis.
Intending to verify the correctness of the simulator, some nominal values for R and X are chosen
(for some type of household appliance) and the simulation are run, the resulting magnitude
of current, phase, active power, and reactive power values are checked against the available
datasets. The different combinations of appliances are tested and verified using the rigor based
analytical methods.
The simulator works very close to ideality, but in real-world scenarios, there are inherent uncer-
tainties and variability in the data and measurements, incorporating a noise model can replicate
these uncertainties, making it more realistic and representative of the actual house being sim-
ulated. A noise model allows to compare the output of the simulator with real-world data or
measurements. The ruleset developed on such simulator are expected to work well on real
houses. As the algorithm thus generated have enhanced generalizability and robustness.
xvii
Figure 2.3: Appliance data recorded in a dataset
xviii
Figure 2.4: Appliance data simulated for validation
xix
Chapter 3
Load Disaggregation
xx
Figure 3.1: PoC collected readings of real house
xxi
Figure 3.2: collected data of 3 appliances(Fridges) from different manufacturer
xxii
3.2 NILM: a state based approach
The operational electrical condition of a house containing different appliances can be repre-
sented as a system which takes a finite state depending on the combination of appliances which
are ON in one of their own valid operating states. In essence, we transformed the ON combi-
nation of set of appliances to a unique number, the total number of states in this system will be
equal to 2N (considering only on-off conditions and ignoring the rest of the states in multi-state
appliances)
where N is the total number of appliances in the house.
A case where nothing is switched on is mapped to a state 0 while each appliance being on is
mapped to a state 2N -1. The I, φ , P, Q recorded at PoC of such a house on any given time instant
is representative of the combination of appliances which are operating at that particular time.
As the total electrical power is equal to the sum of the individual power components by each
appliance.
N
St = ∑ Xi Si + εt
i=1
where Si is the power contributed by ith appliance
Xi is the operating state of appliance i
Going from one state to another we can observe changes in one or more quantities, this means
that appliances operation of a house is now completely described by a system having some
states and the transition in between those states, if at any time we know the state of the system,
i.e., the appliance combination ON at that time, we can deduce the subsequent time states just
by observing the change in the elec. quantity’s values. Such knowledge of states and their
transition with time is equivalent to knowing which load(s) are on at that particular time instant
owing to the mapping of load status to state number.
St = ∆St + St−
We can solve this by using the sub-set sum method, the problem with this method is that
either it gives one correct solution and terminates resulting in same answer each time even for
different combinations or it searches the whole subset space to get all the possible solutions,
which is very time consuming due to the fact that it searches the all possible subsets (powerset).
xxiii
Figure 3.3: Subset sum algorithm
Figure 3.4: dealing in ∆S . Courtesy: G.W. Hart, NALM, proceedings of IEEE 1992
If we match the up and down transactions in power and bind them together, we form the
model of an appliance.
Now the system can have multiple appliances and there are some which have more than
one internal state such appliances are termed as Finite state machine (FSM) type appliances,
xxiv
such appliances have more than one transition attributed to them. If we associate each observed
transition correctly to appliances, we will have a model for each appliance and consequently for
the house.
Since each appliance consumes no power in off state, we can club the positive changes
with their equivalent negative counter parts and in so we will label the on off appliances. If those
xxv
changes are observed for more than a finite time and time between the matching transactions is
in some finite range.
Because of the nature of the problem, if more than one appliance of the same kind (i.e.
having same R and X value) are present in the house, each will behave exactly similarly when
observed from the PoC hence those 2 states are indistinguishable and it is impossible to de-
termine which one of the two appliances are exactly operating, this phenomenon gives rise to
classes of equivalent states.
xxvi
Chapter 4
Methodology
2202
Snormt = St
Vt2
Some errors in readings are introduced by the recording device itself maybe because of poor
quality instrument or maybe the hardware and software techniques employed to collect data
causes the introduction to such noise. In order to mitigate the effect of this noise we use a piece-
wise linearisation of the observed quantities. (On real house data it was found by experiments
that a boundary of decision (∆ P = 20 and ∆ Q = 15) is sufficient to remove further noise and
make data linear.
xxvii
Figure 4.1: decision threshold for piecewise linearisation
xxviii
4.2 Unsupervised Learning for FSM type Loads
In order to do the unsupervised clustering, we need some features and metrics of similarity so
that the computing machine can learn, there are 2 orthogonal features that the smart meters
record namely active power(W) and reactive power (VAr) other than that higher features can be
derived from data set which include (but not limited to) the initial position and the final position
of the house after the completion of transaction, number of time a transaction is observed ,
time spend in a particular state etc. The 2 such metric that are used in this thesis report are the
connected graph of house in a limited time frame and the exact wave shape as a time series.
The connected graph of a house in a limited time window represents the transactions and
the states encountered in that time interval. The FSM transactions can start from any initial state
but will follow the same pattern of the transitions(changes). In order to reduce the search space,
we use the time between transitions. The transition delta and counts are the metric of similarity
in this domain. As can be seen in two cases the operation starts from s3 but in other it starts
from s22, the transactions are similar but an additional change was observed in 1 case due to
another appliance.
The other metric of similarity that we use is the time domain wave form subtraction. As
xxix
Figure 4.4: Connected graph in limited time
stated earlier any FSM load in house can start operating at electrical state ‘S’ of the house but
it will operate through its cycles, these cycles can vary in time a little. The subtraction of 2
such time-series instances give the measure of similarity between them. A zero-resulting wave
after wave subtraction means both time slice instances are equal in time. A constant difference
indicates that the waves are similar in shape but shifted with a constant bias, this constant bias
corresponds to the switching of one of the appliances or switching of one of the states of any
FSM appliance.Thus, the piecewise flatness is the metric of similarity between 2 time series in
this domain.
An FSM based load is going to give transition much more frequently and consistently
than a human can. So, when an FSM load is operating the features like time between events
provide us with the time windows where frequent transitions can be seen. The length of these
windows will vary, now we group the obtained slices of time with similar time duration together.
xxx
Figure 4.5: Time domain Waveform subtraction
We then proceed to make the connected graph and do the time domain wave subtraction of
each of the members of one group and calculate the similarity between them. To aid to this the
xxxi
connected graph will give us the information about duration of state and the transition seen in
that time slice while amount of piecewise flatness of the waveform subtraction will give us the
metric of similarity in time domain between any two given time slices, if the two-time domain
slices are similar to each other, we assign the differential changes observed in those windows to
one appliance regardless of the state position at the start of the time slice window.
In this manner the more data we have the more similarities can be found and hence more
appliances can be discovered with high accuracy. We gradually assign the observed deltas to
appliances, once all FSM are done, we assign remaining delta to on off devices.
Single connected graph is a graph where the edges are only present between those vertex
which transited to and from a state to another, such types of graphs differentiates between FSM
and On-off kind of loads.
xxxii
Figure 4.8: Flowchart of Complete NILM
xxxiii
Chapter 5
For the purpose of the proof of this thesis a house with 3 FSM loads (one slow changing and
other rapidly changing), 1 cyclic always on, and 4 on-off is simulated and the data is clustered
unsupervised to find the FSMs. The proposed method was shown the simulated data of a house
and with training on 3 days and asked to find FSM loads. Once completed the disaggregation
is done on newly generated schedule of the house from simulator having same appliances.
Different kinds of loads are included to that the algorithm can be tested thoroughly.
Moreover, for the first implementation only on off type of loads are used. The value of
impedance for each type of appliance is provided to the simulator from the data-set.
xxxiv
Figure 5.1: simulated data
xxxv
As there are 8 appliances there will be 256 states, not all of them are distinct, moreover,
some transitions are not possible because of the constrain that any load can operate in only one
of its internal states. Once the states and transitions are known, we bind them to n number of
apps (to be decided by user), then we can perform the load disaggregation of appliance on any
running schedule. To test this stratergy a random schedule of operation of appliances of this
house is generated in which each appliance is turned on and off accordingly. Since we need
to know the initial state of the system for convenience and easier debugging it is set to state 0
where all appliances are in OFF state.
The results shows that a highly active FSM load is easily disaggregated because the pres-
ence of more transition of same type makes it easy to recognise similarities in the connected
graph domain. Similarly, an FSM with a few dilation and expansion in time of one of the run-
ning states of an appliance is also recognised easily because of the presence of the long sections
of piece wise linearity in the time waveform subtracted plot domain.
On the other hand, a slowly transiting FSM load whose time to cycle between states is
near to some other cyclic always on type of appliance is difficult to find, typically because the
ontime of always on appliance almost always overlaps with the slow changing FSM, in such
cases more than one FSM type behaviour is registered and attributed to same appliance. The
value of this threshold decides the final operating state of the system, as we can see in Figure
4.2 a low value of threshold results in correct classification of none of the operating states of the
system and the accuracy of the algorithm is very poor(nearly 0), as the threshold is increased the
accuracy improves and at some threshold the algorithm becomes 100 % accurate, as also can be
xxxvi
Figure 5.3: FSM extracted by NILM
seen from the figure the value of threshold for 100 % correct classification is highly dependent
on the presence of load, this is because the ∆values of electrical quantities are dependent on
the initial state of the system, turning a small inductive load (say a fan) on or off will produce
a different set of ∆values when a large resistive only load was already operating than only
inductive load was operating. Thus changing the operating condition of a particular load will
xxxvii
Figure 5.5: Discovered FSM correct
xxxviii
Table 5.1: Golden running schedule
Total 3677160 –
Total 3667805 –
Error 9355 –
xxxix
Chapter 6
Future work
The present work is a state based approach in which the total number of states are a
function of the number of appliances present in the house(ON or OFF), addition of one
appliance double the max number of states, this exponential increase in the state blows
out of proportion very very quickly as appliances increase. Also, a multi-state appliance
having ’M’ distinct states increases the number of states by ’M’ fold instead of doubling
them.
The variable type of loads whose R and X values are a function of external factors (am-
bient temperature etc.) can have a range of R and X values and it may happen that one
of the appliance have the R and X values comparable to the change in R and X of other
appliance in that case the small load will be interpreted as operational while it was never
turned ON.
The method used is of unsupervised learning and it may happen that 2 or more appliances
are mixed into one appliance because they both were operated simultaneously and have
similar duration of operation.
xl
Figure 6.1: Variation in Power of an inverter AC
2. Field trials: field trials on edge devices, monitor how the environmental noise affects
the algorithm, augmentation steps to remedy the effects and complete implementation on
edge device.
xli
References
[1] C. Aditya and A. Sreenivas, “Towards an understanding of residential electricity consump-
tion in india,” Building Research & Information, vol. 47, no. 1, pp. 75–90, 2019.
[3] G. W. Hart, “Nonintrusive appliance load monitoring,” Proceedings of IEEE, vol. 80,
no. 12, pp. 1870—-1891, 1992.
[4] D. V. Christoforos Nalmpantis, “Machine learning approaches for non-intrusive load mon-
itoring: from qualitative to quantitative comparation,” 2018.
[5] M. Zeifman, “Disaggregation of home energy display data using probabilistic approach,”
IEEE Trans Consum Electron, vol. 58, no. 1, pp. 23–31, 2012.
[6] e. a. Laughman, C., “Power signature analysis,” IEEE Power Energy Mag, vol. 1, no. 2,
pp. 56—-63, 2003.
[7] S. M. Jack Kelly and I. V. Bajic´, “Neural nilm: Deep neural networks applied to energy
disaggregation,” Conference on Acoustics, Speech, and Signal Processing, vol. 44, 2019.
[10] Y.-H. L. S.-K. H. M.-S. Tsai, “Study on the influence of voltage variations for non-
intrusive load identifications,” International Power Electronics Conference, vol. 44, 2018.
xlii
[12] D. G.-U. Daniel Precioso, “Nilm as a regression versus classification problem: The im-
portance of thresholding,” 2020.
[15] A. L. M. e. a. Jorge Revuelta Herrero, “Non intrusive load monitoring (nilm): A state of
the art,” Advances in Intelligent Systems and Computing, 2018.
[16] D. B. Enrico Tabanelli and L. Benini, “A feature reduction strategy for enabling
lightweight non-intrusive load monitoring on edge devices,” IEEE, pp. 805 – 810, 2020.
[17] S. M. Alon Harell and I. V. Bajic´, “Wavenilm: A causal neural network for power disag-
gregation from the complex power signal,” Conference on Acoustics, Speech, and Signal
Processing, vol. 44, 2019.
[18] L. W. Z. G. H. Cao, S. Liu and X. Du, “Achieving differential privacy against non-intrusive
load monitoring in smart grid: A fog computing approach,” Concurrency and Computa-
tion: Practice and Experience, vol. 31, no. 22, 2019.
[19] Q. Z. Y. L. W. Shi, J. Cao and L. Xu, “Edge computing: Vision and challenges,” IEEE
Internet of Things, vol. 3, no. 5, pp. 637–646, 2016.
xliii