International Journal of Combined Research & Development (IJCRD)

eISSN:2321-225X;pISSN:2321-2241 Volume: 11; Issue: 4; April -2022

IoT based Smart Refrigerator for Food Management

System
Karthikeyan S, Manojkumar S, Sharvin Vivian Moras , Suhel Pasha M A
Department of Robotics and Automation
JAIN (Deemed-to-be University)
Bangalore , India
Abstract— Grocery Management System based on the Internet of Things (IoT) which includes a smart refrigerator as well as a smart
cabinet are available is presented in this paper. Food that has been left in refrigerators and cabinets is commonly forgotten by people. Food
waste rises as food is discarded after a period of time. Due to physical limitations or hectic schedules, grocery shopping is challenging for
many people. Our proposed solution addresses these issues. Users can use the device to see what's inside the refrigerator and cabinet via the
internet. Food waste is substantially minimized when you know how much food is remaining in the refrigerator and cabinet. Furthermore,
when the food items in the refrigerator and cabinet are depleted, our system places an automatic online purchase with the user's permission
to replenish the depleted food products. As a result, the proposed approach simplifies life for persons who are unable to go grocery
shopping. Unlike earlier models, our smart refrigerator uses cameras rather than sensors such as weight and infrared sensors. As a result,
the system produces more accurate and information-rich outputs without the need for costly wiring that earlier systems necessitated. Many
firms have issues determining how much of their products are sold in specific retailers. As a result, they will have to hire workers in order
to obtain this information. They would quickly know the data about their items if they use our recommended system, and they will be able
to arrange their sales efforts accordingly.

Keywords— IoT, grocery management, smart refrigerators, smart cabinet.

I. INTRODUCTION
The Internet of Things (IoT) is a phrase that describes how data collected by sensors and actuators embedded in machines and
other physical objects is harnessed by intelligent networked devices and systems. The Internet of Things (IoT) creates a network
fabric that can be monitored, modified, and programmed. IoT-enabled items have embedded technology that allows them to connect
directly or indirectly with one another or the Internet [1].
The use of internet-connected devices is becoming more common. The Internet of Things is made up of most of these devices,
which are embedded with actuators and sensors (IoT). It is a global network of smart gadgets that allows information to be shared
through the internet. IoT devices are now present in nearly every facet of life. These devices are being used in a wide range of
situations. The term "smart kitchen" comes to mind when we discuss the Internet of Things, also known as the Cloud of Things. The
reason for this is that the kitchen is the home's top trash generator and second highest energy consumer. As a result, manufacturers
are constantly looking for new methods to develop smart kitchen items that minimize waste and energy
consumption while also increasing convenience.For example, a Samsung Smart Home, which uses the slogan "Enrich Your Life" to
offer a "whole home solution" with the purpose of restoring balance to your life. It allows you to manage domestic tasks from afar
while also complementing your daily activities at home. As a result, the consumer has less concerns and more time to spend with
family and friends [2].
A refrigerator is common household equipment that consists of a thermally insulated compartment and a heat pump that
transports heat from the fridge's interior to the outside environment, allowing it to be cooled to a temperature lower than the room's
ambient temperature. In developed countries, refrigeration is a necessary food preservation strategy. The refrigerator slows the rate
of deterioration because germs reproduce more slowly at colder temperatures.
However, no matter how costly or well-known a refrigerator is, it will not be able to keep food fresh for an extended period of
time, resulting in food rotting. Users, unfortunately, have less time to regularly check the condition of their food in the refrigerator.
They might also forget what kind of food they have in the fridge and how long they've had it. As a result, anytime they want to cook
with the food, they may discover that it has already ruined and end up tossing it away. It's a complete waste.
As a result, a smart refrigerator system has been developed that can provide users with information about the state of their food in
the refrigerator. The technology will send users a notification telling them of how long they have stored certain foods in the
refrigerator. When the user's limit is reached, the system will send a warning message to the user, encouraging them to consume
the food as quickly as possible before it spoils, hence preventing food waste and spoilage.

www.ijcrd.com 1205 | P a g e
 International Journal of Combined Research & Development (IJCRD)
eISSN:2321-225X;pISSN:2321-2241 Volume: 11; Issue: 4; April -2022

II. LITERATURE REVIEW

This section summarizes the state-of-the-art work on the smart cabinet and smart refrigerator. Polycapou et al. proposed and tested
on smart cabinets, an RFID-based library management system. All relevant library features were supported using the GUI [8]. Wu
et al. To deliver Intelligent Care Services, a smart kitchen cabinet was designed in an IoT-based smart home environment. Sensors
for height, proximity, light actuators, and buzzers were used to create the locking/unlocking mechanism [9]. Warnick et al. created a
blood stock management system that included a cabinet model, a transceiver unit, and a computer with a graphical user interface
This method was created to assist blood banks in managing and storing blood for transfusion. The device could tell the difference
between different types of blood.
Sgârciu et al. created RFID sensors and Arduino were used to create a proof-of-concept smart refrigerator. New and withdrawn
products were identified with the use of RFID sensors. Based on this information, a shopping list and inventory were created, which
the user could access via a web application [13]. GSM module, microprocessor, LCD, and sensors were used by Guruler et al. to
develop an intelligent refrigerator the user can get information on product quantities and refrigerator temperature by sending a
message or calling. Because it is not operating system dependent, this system is widely used. Texts and phone calls can be used to
obtain information by users with any operating system [14].
Sgârciu et al. used RFID sensors and Arduino to create a proof-of-concept smart refrigerator. New and withdrawn products were
identified with the use of RFID sensors. Based on this information, a shopping list and inventory were created, which users could
access via a web application [13]. Guruler and colleagues used a GSM module, microprocessor, LCD, and sensors to construct a
smart refrigerator. By sending a message or making a phone call, the user can obtain information regarding product amounts and
refrigerator temperatures. Because it is not depending on the operating system, this system is widely used. Texts and phone calls can
be used by users with any operating system to get information [14]. The contents of the refrigerator are determined by this system,
and the user is notified by SMS or email. The Arduino microcontroller collects all data and sends it to ThingSpeak, where it can be
accessed by the user. When sensor readings surpass the predefined ranges, the user receives notifications. Qiao et al. used RFID
technology to create an intelligent internet refrigerator [16]. The refrigerator could keep track of food and manage it intelligently. A
week before the food's expiration date, a notice was given to the user. When the food was finished, the internet was used to locate
businesses that offered the greatest prices for the cuisine.
RFID sensors were also used in the refrigerator created by Hachani et al. in [17]. However, a separate Android application was
developed so that the user could use the app to view the contents of the refrigerator. A low-cost smart refrigerator with sensors,
cameras, and a Raspberry Pi was presented by Wu et al. [6]. The cameras collected the images, which were then transmitted to the
database via the Raspberry Pi, allowing users to view the contents of the refrigerator using an Android application. Shewta
demonstrated an intelligent refrigerator that counted the amount of different vegetables using image processing [18]. It maintained
track of the contents and count of the vegetables, and when they were past their expiration date, it sent an SMS to the user.
Our proposed solution consists of a Smart Refrigerator and Smart cabinets at a minimal cost. With only three cameras and a few
cables, the smart refrigerator is unnoticeable to users. We also utilised deep learning to check if the amount of eggs available was
adequate.
III. PROPOSED SYSTEM
There are four modules in the proposed system: a data storage module, a refrigerator, a cabinet, and a receiving module. Figure
1 depicts the system's conceptual diagram. In the refrigerator, three cameras are installed: one on the egg shelf, which utilizes
image processing and a neural network to track egg availability, and the other two on the vegetable box and the top corner of the
refrigerator shelf. The photographs are taken and sent to the data storage module, where they will be briefly saved and
processed.As shown in Fig. 2, the cabinet module is made up of load cell sensors. The goal is to figure out how many different
objects weigh when they're placed on the sensor. After then, the weight is sent to the data storage module, where it is briefly stored
and processed. In our Smart Refrigerator system, we employ two USB cameras, one IR camera, and a Raspberry Pi. The placement
and coverage of the cameras are depicted in Figure 3.

Fig. 1: The proposed Grocery Management System's conceptual framework

www.ijcrd.com 1206 | P a g e
 International Journal of Combined Research & Development (IJCRD)
eISSN:2321-225X;pISSN:2321-2241 Volume: 11; Issue: 4; April -2022

Fig. 2: Load cell sensors are installed in the cabinet.

Fig. 3: Different cameras' placement and coverage in the shelves and veggie box.

Before being compiled and transferred to the receiving module, the data from the smart refrigerator and cabinet is momentarily
saved on the cloud. Laptops and mobile phones with a web-based application serve as the receiving module. All information about
the refrigerator and cabinet may be found on a web application that is protected by the user's account and password. This web
application is accessible from anywhere in the world, and it has the option of placing an online order after receiving confirmation
from the user if the weight of any item is less than the desired
w

Fig. 4: The implementation of the suggested smart refrigerator and smart cabinet is shown in a block diagram.
Figure 4 depicts a block diagram of the proposed smart refrigerator and smart cabinet installation. For the purpose of weighing
supermarket items, the proposed Smart Cabinet system employs load cells. The cabinet can be used to store rice, vegetables, flour,
lentils, and spices, among other things. The load cells are coupled to a HX711 module, which converts the analogue signals from the
load cells into digital signals. The NodeMCU controller is then connected to the HX711 module, which reads and scales the digital
data. This scaled data is subsequently transferred to Google Firebase's real-time database. The Raspberry Pi is used to connect all of
the cameras. The use of exactly three cameras was chosen to achieve the ideal balance of accuracy and usability. To cover all of the
shelves, we installed one camera on the refrigerator door.

www.ijcrd.com 1207 | P a g e
 International Journal of Combined Research & Development (IJCRD)
eISSN:2321-225X;pISSN:2321-2241 Volume: 11; Issue: 4; April -2022
The result will be more accurate if we utilise more than one camera, but the wiring will increase along with the expense. Because
our goal is to design a low-cost refrigerator with minimal connections, we opted to employ three cameras to cover the egg shelf,
vegetable box, and refrigerator shelves. All of the camera angles were chosen after a lot of trial and error. The existing camera angles
allow us to take the greatest photographs possible without sacrificing performance. The cameras allow us to take wide-angle images,
allowing us to see items that are closer to us. One camera is mounted on the egg shelf, another on the refrigerator door, and a third is
hidden in the vegetable box.
When the refrigerator door is closed, the camera on the egg shelf captures a photo. Because this is an infrared camera, it can
capture fine photographs of the egg shelf even in the dark. In the veggie box, we've placed a flashlight and a camera. The flashlight is
turned on and the camera snaps photographs whenever the user opens and closes the veggie box. When the user closes the refrigerator
door and it reaches a certain position, the camera on the door takes a photo of the shelves. After a number of trials, we decided on this
exact posture.
When the fridge door reaches such position, the camera takes pictures. The refrigerator door and vegetable box have infrared sensors
that detect movement and location. The Raspberry Pi is also protected from the humidity by being placed inside the refrigerator in a
protective shell. The Raspberry Pi uses deep learning algorithms on an image of an egg shelf to compute the number of eggs. The
images from all of the cameras, as well as image processing data, are delivered to Google Firebase's real-time database. Our web
application retrieves data from Google Firebase for Smart Cabinet and Smart Refrigerator. This online application allows users to view
this data in real-time over the internet.A convolutional network (CNN) influenced by the LeNet-5 design is proposed with significant
results to determine the sufficiency of eggs in the shelf [19]. For a randomly chosen image, Figure 5 shows the network architecture as
well as feature maps from the two convolutional layers. We photographed hundreds of eggs in various locations and numbers and
classified them as little or large. The label signalling egg sufficiency is defined as follows if ne is the number of eggs:

Insufficiency of eggs would be indicated by a small quantity of eggs. It was physically impossible to take all of the potential
combinations of places and egg numbers. As a result, we used data augmentation to improve the convolutional network's accuracy.
Three separate sets of photos were created:

(a)

(b)

Fig. 5 For a randomly chosen image, a feature map from the first and second convolutional layers.

validation, training, and testing for each set, there are 108 photos in the training and validation sets. The trained network was put to
the test with 500 extra photos that were not part of the training or validation sets. The training and validation curves are shown in
Figure 6. The accuracy of the 516 independently collected test photos was 85.85%.
IV. Results

A smart cabinet and smart refrigerator are part of a grocery management system has been presented for enabling and
offering convenience to consumers who are unable to purchase their groceries owing to a hectic schedule, physical
incapacity, or country unavailability. It was created utilizing Internet of Things (IoT) principles. Using Raspberry Pi,
NodeMCU, load cells, cameras, and

www.ijcrd.com 1208 | P a g e
 International Journal of Combined Research & Development (IJCRD)
eISSN:2321-225X;pISSN:2321-2241 Volume: 11; Issue: 4; April -2022
Google Firebase, the refrigerator, cabinet, data storage, and reception modules of our suggested system were implemented on a
household refrigerator. The data was imported from Google Firebase into the web application running on a computer and a mobile
device to present to the user.
The refrigerator and cabinet modules provided real-time images and weights, respectively. Two cameras were placed on
the refrigerator shelf and the vegetable shelf. Another camera was mounted on the egg shelf. The image from the egg shelf was
analysed by Raspberry PI, which employed convolutional networks to determine whether there were enough eggs.
Using their respective item names, individual item weights were acquired from the load cells in the cabinet module. By measuring
the weights according to the scaling factor, the load cells calculate the actual weight of the item placed on it. The user received the
weights after they were submitted to the data storage and receiving module. Through the web application, the user could find out how
much each item weighed. The web programme exhibited images of real-time data from the refrigerator and cabinet, as well as the
availability of eggs and the weights of each cabinet item.
The proposed technology is also suited for use in a manufacturing environment. Using our technology, business managers will be
able to observe product availability and sales status in real time, as well as the places where they are sold. Businesses can benefit from
this data since they will not have to engage workers to obtain it, and they will be able to change their Supply chain and promotion
strategies should be adjusted accordingly.

V. CONCLUSION

Users from all over the world can access a specialised online application to see what food products are remaining in their
refrigerator and cabinets in our suggested system. This will also make it easier for people to remember how much food they have left
in their houses when they go grocery shopping. These individuals may simply pull out their cell phones and see how much food is left.
Furthermore, people who find it difficult to go food shopping due to a physical limitation or a busy schedule would benefit from our
method. The system recognises which food items are out of stock and, with the user's permission, places an automatic online order to
restock the out-of-stock items.

The user is notified when the order has been placed through email. In this way, the user will be aware of how much food is left in
the refrigerator and cabinets. As a result, the user is better able to prevent food from spoiling and so reducing food waste. This
technology is also totally portable. The technology may be taken out of one refrigerator and put into another, and the new refrigerator
will be just as intelligent as the old one. In the future, images of vegetable cartons may be submitted to image processing. This will
allow us to distinguish between different sorts of vegetables and alert the user as to which vegetables he or she has left in the
refrigerator. We can identify the presence of expired food with the use of various sensors such as odour sensors. We can also use the
smart meter to control the amount of electricity used in the refrigerator.
REFERENCES

[1] S. K. Roy, S. Misra, and N. S. Raghuwanshi, “SensPnP: Seamless integration of heterogeneous sensors with IoT devices,” IEEE Trans. Consum. Electron., vol.
65, no. 2, pp. 205–214, 2019.
[2] Bill Montgomery, “FUTURE SHOCK: IoT benefits beyond traffic and lighting energy optimization,” IEEE Consum. Electron. Mag., vol. 5, no. 1, pp. 69–73,
2016.
[3] C. Koverman, “Next-Generation Connected Support in the Age of IoT: It’s time to get proactive about customer support,” IEEE Consum. Electron. Mag., vol.
5, no. 1, pp. 69–73, 2016.
[4] E. Summary, “THE INTERNET OF THINGS : MAPPING THE VALUE BEYOND THE HYPE,” no. June, 2015.
[5] D. Evans, “The Internet of Things How the Next Evolution of the Internet The Internet of Things How the Next Evolution of the Internet Is
Changing Everything,” no. April, 2011.
[6] H. Wu, “Low-Cost Smart Refrigerator,” pp. 228–231, 2017.
[7] H. Nasir, W. B. W. Aziz, F. Ali, K. Kadir, and S. Khan, “The Implementation of IoT Based Smart Refrigerator System,” 2018 2nd Int.
Conf. Smart Sensors Appl. ICSSA 2018, pp. 48–52, 2018
[8] A. C. Polycarpou, T. Samaras, and J. N. Sahalos, “An RFID-based library management system using smart cabinets: A pilot project,” no. EuCAP, pp. 2954–
2955, 2014.
[9] C. L. Wu, C. W. You, C. Y. Chen, C. C. Chuang, and T. C. Chiang, “Exploring the collaborative context reasoning in IoT based intelligent care services,”
Proc. - IEEE 7th Int. Conf. Serv. Comput. Appl. SOCA 2014, pp. 241–245, 2014.
[10] K. F. Warnick et al., “RFID-based Smart Blood Stock System,” IEEE Antennas Propag. Mag., vol. 57, no. 2, pp. 54–65, 2015.
[11] M. Edward, K. Karyono, H. Meidia, and A. O. Design, “Home Server based on Raspberry Pi 3,” pp. 148–151, 2017.
[12] W. Zhongmin and Y. Yanan, “Design of an Interactive Smart Refrigerator Based on Embedded System,” Proc. - 2018 Int. Conf. Sensing, Diagnostics, Progn.
Control. SDPC 2018, pp. 589–592, 2019.
[13] V. Sgârciu, “Smart Refrigerator,” 2016.
[14] H. Guruler, “The design and implementation of a GSM based user- machine interacted refrigerator,” INISTA 2015 - 2015 Int. Symp. Innov. Intell. Syst. Appl.

www.ijcrd.com 1209 | P a g e
 Proc., no. 2209, 2015.
[15] P. Jarupunphol, W. Buathong, T. Chansaeng, and N. Laosen, “A descriptive design for a smart kitchen management application (SKM),” 2018 Int. Conf. Inf.
Comput. Technol. ICICT 2018, pp. 61–66,2018.
[16] S. Qiao, H. Zhu, L. Zheng, and J. Ding, “Intelligent Refrigerator Based on Internet of Things,” Proc. - 2017 IEEE Int. Conf. Comput. Sci. Eng. IEEE/IFIP Int.
Conf. Embed. Ubiquitous Comput. CSE EUC 2017, vol. 2, pp. 406–409, 2017.
[17] A. Hachani, I. Barouni, Z. Ben Said, and L. Amamou, “RFID based smart fridge,” 8th IFIP Int. Conf. New Technol. Mobil. Secur., pp. 0–3, 2016.
[18] A. S. Shewta, “Intelligent refrigerator using ARTIFICIAL INTELLIGENCE,” in 11 th International Conference on Intelligent Systems and Control (ISCO),
2017, pp. 464–468.

www.ijcrd.com 1210 | P a g e