LSTM-based classifier written in PyTorch for detecting predefined gestures from IMU sensor.

This project is developed for Machine Learning course on AGH University of Science and Technology. Authors:

• Mateusz Woźniak
• Maciej Pawłowski

You can download mobile app with .apk file. Download APK

Mobile app source code: https://github.com/maciekpawlowski1/ImuCollector

docker build -t imutensor .
docker run -p 4199:4199 --name imutensor -d imutensor

or:

pip3 install torch numpy pandas matplotlib flask seaborn scikit-learn
python3 train.py

This machine learning model takes imu data as a series in .csv in format like:

gyro_x;gyro_y;gyro_z;magnetometer_x;magnetometer_y;magnetometer_z;accelerometer_x;accelerometer_y;accelerometer_z
0.48433977;-0.28244883;1.5225816;-23.34;3.36;-42.78;-2.6527755;-0.9696517;9.857227
0.2222786;-0.95989835;1.2843442;-23.519999;3.1799998;-43.32;-0.4141969;-0.12689269;11.772589
0.61995184;-0.38385245;2.3576343;-24.66;3.8999999;-42.899998;-0.7062895;-1.733402;15.330672

and classifies it into one from following classes:

classes = {
    'CIRCLES_RIGHT': 0,
    'CIRCLES_LEFT': 1,
    'TRIANGLE': 2,
    'SQUARE': 3,
    'FORWARD_BACK': 4,
}

We developed mobile app for Android in Kotlin that grabs data from imu sensor and uploads it into Golang microservice which archives files.

We are using LSTM layer with two Linear transformations. Recurrent neural network is used, because gesture-classification task is time-invariant (.csv files has diffrent number of timesteps).

class Net(nn.Module):
    def __init__(self, input_size):
        super(Net, self).__init__()
        self.lstm = nn.LSTM(input_size, 22, batch_first=True)
        self.fc1 = nn.Linear(22, 32)
        self.fc2 = nn.Linear(32, len(classes.keys()))

    def forward(self, x):
        _, (h_n, _) = self.lstm(x)
        x = h_n[-1, :, :]
        x = self.fc1(x)
        x = self.fc2(x)
        return x

We are able to achieve ~95% accuracy on validation dataset using gyroscope-only features.

Model is trained inside docker build and saved into model.pt weights files. serve.py file launches flask HTTP server which exposes POST /inferece endpoint on port 4199.