-
Notifications
You must be signed in to change notification settings - Fork 1
Home
Welcome to the edge-ml wiki. Here you can find all the information you need to get started with your first project using edge-ml today.
edge-ml helps developers build models faster and more robustly with an open-source toolchain for embedded machine learning. With a few simple steps edge-ml lets you record data, label samples, train models and deploy validated embedded machine learning directly on the edge. edge-ml requires minimal initilization and supports upload in real-time as well as in bulk from the edge. Models are generated using AutoML therefore requiring minimal user configuration. The models are optimized for resource-constrained embedded chips based on hardware-aware neural network training.
If you are using edge-ml and would like to cite it in your research please use:
@misc{edge-ml,
title={edge-ml.org - End-To-End Embedded Machine Learning},
url={edge-ml.org},
author={Tobias Röddiger and Tobias King and Philipp Lepold and Jennifer Münk
and Shanshan Du and Till Riedel and Michael Beigl}
}
This wiki page gives you a quick intro into edge-ml.
Out of the box, edge-ml currently supports connecting to the following Arduinos:
- Nicla Sense ME (directly from your browser over BLE)
- ESP32 (over WiFi using the dedicated edge-ml Arduino library)
edge-ml also supports Android and node.js.
In this tutorial, we will focus on the Nicla Sense ME which already has sensors pre-installed and requires minimal setup. Please first install the BLE library from Bosch on your Nicla Sense ME board by following the steps listed here.
Then, navigate to the project that you have created earlier on edge-ml. Within in your project navigate to the tab "Devices" which will give you the possibility to connect to a new device using Bluetooth Low Energy by clicking the "Scan" button. Once your device is found, select it from the list of devices and click "Connect". The device then shows up in your list of connected devices with the possibility to enable different sensors and set their respective sampling rates. You can now activate the sensors that you would like to record by selecting them from the list shown on your screen. In this example, we will activate the accelerometer and gyroscope sampling at 50 Hz.
Please note: If edge-ml tells you that WebBLE is not available, follow the instructions to activate the experimental feature in your browser. Currently, WebBLE is only available on Chrome, Chrome Android, and Microsoft Edge. Also, make sure that your computer supports Bluetooth Low Energy. Otherwise, connecting to the Nicla Sense ME will not be possible.
Now that you have connected to your device and configured the sensors according to your requirements, the next step is to record the first dataset. To do so, click the "Record" button next to your connected device. You will start see data coming in which will be automatically pushed to the edge-ml cloud. Once you click "Stop", edge-ml will redirect you to the dataset you have just recorded. If you want to record another dataset, just go back to your device and click "Record" again.
In this example, we collect two datasets. The first dataset containes multiple repetitions of a 'Shake' gesture. The second dataset contains multiple repetitions of a 'Bump' gesture. We leave small breaks between the gesture repetitions to classify 'No Gesture' later.
Now that you have collected your training data, you have to add labels to it so that they can be processed in the model training generation step later. Please navigate to "Labelings" in the top navigation bar in your project. Then click "+ Add Labeling Set" to add a new set of labels. You can now enter the name of the labeling set (e.g., in our case "Gesture Labels"). Then click on "+ Add Label" to add your desired labels. In our example scenario we are adding the labels based on the data we recorded earlier by adding 'Shake', 'Bump', and 'No Gesture'. The colors for the labels are generate automatically for you and you can easily change them by entering a different color HEX code.
Release pending.
Coming soon.
edge-ml is open-source and we love to see others contribute. Our dedicated team of researchers and developers constantly works on making edge-ml the best end-to-end embedded machine learning framework for everyone.
Did you experience a bug? 🐛 Do you have a feature request? Please join the discussion and make yourself heard.
- docker, nodejs (> v10) and npm must be installed. It is recommended to have nodemon installed as well.
- do
git clone --recurse-submodules https://github.com/edge-ml/edge-ml.git - make sure docker (desktop) is running
- cd into the edge-ml folder and compose & start the mongo database with
docker-compose up -d mongo - cd into ./authentication and do npm install
- cd into ./backend and do npm install
- cd into ./frontend and do npm install
(if you don't want automatic server refreshs when you modify the code, you can substitute "nodemon" with "node")
- make sure docker (desktop) is running, do
docker-compose up -d mongoto start the database - cd into ./authentication and do nodemon server.js
- cd into ./backend and do nodemon server.js
- cd into ./frontend and do npm start (or yarn start)
The order is important, the authentication part and the backend need to run before the frontend is started. You should now get a message similar to this one:
Something is already running on port 3000. Probably:
/path/to/nodejs server.js (pid 8753)
in /path/to/edge-ml/backend
Would you like to run the app on another port instead? (Y/n)
- type Y and hit enter. A browser window should open. If not, open one yourself with URL localhost:3001
Every time you run edge-ml, you need to go through these steps again. You can put these commands in a bash script for convenience.
Now you should be good to go. Any change in the source code will automatically refresh the server and frontend when nodemon is used. If you use Visual Studio Code, debugging React code is easy, click here for a guide.
-
if you use Windows Subsystem for Linux or WSL 2 and cannot access localhost:3001, try http://127.0.0.1:3001 instead of localhost, or restart the Subsystem and try again. If that still does not work, you might be better off using edge-ml in Windows natively, instead of WSL.
-
if you use macOS and have trouble installing the requirements for the machine learning component (namely matrixplofile), resulting in this error, try the following:
- download the matrixprofile source from github
- head into the matrixprofile folder and do
python setup.py install, this will result in a crash as well, we're interested in the exact clang command that's the cause of the crash, e.g.:
clang -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -L/usr/local/opt/llvm/lib -I/usr/local/opt/llvm/include build/temp.macosx-12-arm64-3.9/matrixprofile/algorithms/cympx.o -o build/lib.macosx-12-arm64-3.9/matrixprofile/algorithms/cympx.cpython-39-darwin.so -fopenmp
ld: warning: directory not found for option '-L/usr/local/opt/llvm/lib'
ld: library not found for -lomp
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/opt/homebrew/opt/llvm/bin/clang' failed with exit code 1
- (you need to have homebrew installed for this) do
brew install clang-omp - locate the lib folder for the llvm installation from homebrew, probably
/opt/homebrew/Cellar/llvm/<<VERSION_NUMBER>>/lib/ - back in the matrixprofile folder execute the same command that created an error, but with following option:
-L<<PATH TO llvm LIB>>, like so:
clang -L/opt/homebrew/Cellar/llvm/13.0.1_1/lib/ -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -L/usr/local/opt/llvm/lib -I/usr/local/opt/llvm/include build/temp.macosx-12-arm64-3.9/matrixprofile/algorithms/cympx.o -o build/lib.macosx-12-arm64-3.9/matrixprofile/algorithms/cympx.cpython-39-darwin.so -fopenmp
ld: warning: directory not found for option '-L/usr/local/opt/llvm/lib'
- now do
pip install -e ., which hopefully should raise no error anymore
Datasets can be labelled to mark features e.g. for machine learning. We use following terminology:
- "labelings" are basically groups of "labels"; a dataset cannot be labelled with labelings directly, but with individual labels of a labeling
- multiple labelings (label groups), each containing multiple labels, can be created or imported
- labels of a labeling (aka the label group) must each have a unique name
- one specific label must not overlap in a dataset; this is prevented by edge-ml, but custom datasets need to comply
- different labels, even from the same labeling (label group), are allowed to overlap though
Importing and exporting datasets is possibles via CSV (comma separated values) files. The first record/line in the CSV file is the header containing a list of field names. Before reading on, make yourself familiar what labelings/labels are in edge-ml and how they are used.
We use the following (header )format, with which you have to comply with, if you want to import custom datasets:
- 'time', used for UNIX timestamps
- all sensors used, in the following format: "sensor_sensorName[sensorUnit], for each value that a specific sensor captured at a specific point in time; e.g. sensor_accX[m/s²] for acceleration in the x-direction.
- all labelings and labels used in the following format: label_labelingName_labelName; e.g. label_labeling1_label11, label_labeling1_label12,..., label_labeling2_label21, label_labeling2_label22
Some things to keep in mind:
- Labels of a labeling with the same name must not overlap. Edge-ml ensures this when creating/editing label(ing)s, but custom datasets must conform to this. Overlapping is forbidden, because it cannot be recreated when uploading and downloading the same dataset again.
- Different labels of a labeling are allowed to overlap though.
- To label a time stamp, simply add any string between the commas, expect the empty string. We simply use 'x' when downloading a dataset.
Consider following scenario: We took sensor data of acceleration sensors in three dimensions on the 5th August 2021 on four points in time. The unit for acceleration is m/s². We have labeling1 with labels label1, label2 and labeling2 with labels label3, label4. The CSV file looks like this:
time,sensor_accX[m/s²],sensor_accY[m/s²],sensor_accZ[m/s²],label_labeling1_label1,label_labeling1_label2,label_labeling2_label3,label_labeling2_label4
1628168415000,2,2,2,x,,x,
1628168416000,4,2,4,x,,,
1628168417000,8,3,8,,x,,x
1628168418000,16,4,16,,x,,
Datasets can be uploaded as CSV files if they comply with our CSV format. Simply click on "+ Add Project" in the navbar on the left side, and the choose "Upload CSV Files"
Datasets can be downloaded as CSV file. Select your project and dataset in the navbar on the left side, and click "View" to open a dataset. Then simply click "Download as CSV" in the "Management" section on the right. Everything you need to know about the CSV file is explained here.
Please follow the readmes of the repositories to learn how to use our Arduino, Android and node.js libraries: