Simple script to execute a command on a remote server. Great to deploy experiments on a cluster.
remote <cmd>Searches for a remote.toml file in parent directories. The directory where
this file is found is treated as the project directory. Everything under this
directory will be rsynced to the remote, <cmd> will be executed, and
changed/added files will be synced back to the local machine again.
This is to keep the remote in sync with the local project directory.
Note
This also means that the upload to the remote will delete files that are not on the local machine.
Make sure the remote project directory does not contain files that you don't have locally before you run this command the first time.
Example remote.toml:
# name of the remote server
remote = "submit"
# the corresponding project directory on the remote server
remote_project_dir = "~/projects/fluffy"
# an optional command prefix to prepend to the <cmd> given above
command_prefix = "bsub -I -q gpu_a100 -gpu 'num=1' uv run --with 'jax[cuda]'"
[rsync]
# exclude certain files/directories from syncing
exclude = [".git", "__pycache__", "*.tmp"]
# other options to pass to rsync
# extra_options = ["--dry-run"]