Playing Snake using Reinforcement Learning.

The goal of this project is to train a Reinforcement Learning Agent capable of playing the classic Snake game. The observation space is an image-like representation of the screen, encoding the snakes head, body and food. The action space consists of four possible actions: UP, RIGHT, DOWN and LEFT.

• Game Simulation as gymnasium Environment
• Training using SB3 Algorithms
• Use Action-Masks for preventing illegal moves (turning the opposite direction)
• Experiment-Tracking using Tensorboard and MLflow
• Log Evaluation Videos to MLflow.
• TODO: Get the Agent to learn a decent strategy

Some ideas:

• Improve the reward function
• Use multiple apples at the beginning, to increase the chance of eating one and gaining reward.
• Use CNNs instead of a simple flattening of the observation space
• Find a good architecture for the policy

1. Install uv
2. Run

uv sync --dev

The experiment is tracked using MLflow and Tensorboard.
MLflow has a better overview over Hyperparameters, but currently doesn't support the evaluation Videos. To show these, you can use Tensorboard.

You need to start a local MLflow Tracking Server, before running an experiment.
You can simply use the Docker Configuration.
Make sure to create a .env file in mlflow before starting the containers.

cd mlflow | docker compose up -d

To start the Tensorboard:

tensorboard --logdir runs

To start training and run an experiment, use the following CLI:

uv run snakeai train --n_envs=1 MY_EXPERIMENT

To load a trained model and let it play, run

uv run snakeai validate MY_EXPERIMENT_1