Currently support:
cloudal /ˈklaʊdəl/, an experiment management tool, is one of the contributions of the RainbowFS project. It is created with the mission to simplify the management of designing and running a full factorial experiment on a cloud system automatically.
cloudal provides 3 main modules: provisioner, configurator and experimenter. You can use provisioner to provision nodes on a specific cloud system by simply describing your requirements in a YAML file. Moreover, by calling our ready-to-use configurators you can easily install and configure some software or services on these provisioned nodes. After configuring the environment, you can use the experimenter module to perform an experiment workflow automatically from creating and controlling each run to collecting the results.
By using these 3 modules as Lego blocks, you can assemble them to write a script that describes all steps to perform a cloud experimental scenario.
Read the doc for more technical detail.
cloudal should be painless to work with. Just pick the appropriate module with the right arguments and move along 😉
from cloudal.provisioner import g5k_provisioner
from cloudal.configurator import docker_configurator, packages_configurator
provisioner = g5k_provisioner(config_file_path="/path/to/configuring/file.yaml")
provisioner.provisioning() # provisioning hosts on Grid5000 base on requirement in a configuring file.
hosts = provisioner.hosts
configurator = packages_configurator()
configurator.install_packages(['sysstat', 'htop'], hosts) # install sysstat and htop on all hosts
configurator = docker_configurator(hosts)
configurator.config_docker() # install and start Docker engine on all hostsTo write your own script to perform your custom actions such as provisioning, configuring or experimenting, you could use the provided templates and follow the detail explanation in templates.
You can also try some examples in the tutorials section.
The following are steps to install cloudal. If you want to test it without affecting your system, you may want to run it in a virtual environment. If you're unfamiliar with Python virtual environments, check out the user guide.
- Clone the repository.
git clone https://github.com/ntlinh16/cloudal.git
- Activate your virtualenv (optional), and then install the requirements.
cd cloudal
pip install -U -r requirements.txt
- In other to run
execo, we need to installtaktuk
apt-get install taktuk
- Set the
PYTHONPATHto the directory ofcloudal.
export PYTHONPATH=$PYTHONPATH:/path/to/your/cloudal
You can add the above line to your .bashrc to have the env variable set on new shell session.
- Set up the SSH configuration for
execo:
If you want to specify the SSH key to use with cloudal, you have to modify the execo configuration file.
In ~/.execo.conf.py, put these lines:
default_connection_params = {
'user': '<username_to_connect_to_nodes_inside_cloud_system>',
'keyfile': '<your_private_ssh_key_path>',
}
for example:
default_connection_params = {
'user': 'root',
'keyfile': '~/.ssh/cloudal_key/id_rsa',
}
Execo reads ~/.execo.conf.py file to set up the connection. If this file is not exist, execo uses the default values that you can find more detail here
To working on specific cloudal systems, you need more installation. Please find the detail instruction in the following links:
- Working on Grid5000 (G5K)
- Working with Kubernetes on G5K
- Working on Google Cloud Platform (GCP)
- Working with Google Kubernetes Engine (GKE)
- Working on MS Azure
- Working on OVHCloud
I provide here some quick tutorials on how to perform an action with cloudal.
- Provisioning on G5K: reserving some hosts
- Provisioning on G5K: creating a Kubernetes cluster
- Provisioning on G5K: creating a Docker Swarm cluster
- Provisioning on GCP: reserving some hosts
- Provisioning on GCP: creating a Docker Swarm cluster
- Provisioning on GKE: reserving Kubernetes clusters
- Provisioning on Azure: reserving some hosts
- Provisioning on OVHCloud: reserving some hosts
- Provisioning on OVHCloud: creating a Kubernetes cluster