Tool Caches

Tool caches enable incremental caching in tasks. They're the best way to speed up tasks using tools that support incremental behavior, such as package installers, build systems, or language toolchains. Without a tool cache, even a small change to a task's inputs requires a complete re-execution of the task. With a tool cache, a task can restore the output filesystem of a previous execution of that task instead of starting from a clean filesystem.

For example, a task running npm install will cache miss and re-execute whenever any dependency is added, removed, or updated from package.json. With a tool cache, that re-execution starts with the node_modules directory and NPM cache produced by a previous execution of npm install, so NPM only needs to download the dependencies that changed. This can lower the execution time of a multi-minute install down to seconds.

Tool caches don't replace content-based caching. Tasks are still checked for a content-based cache hit first; the tool cache only matters on cache misses.

Configuring a tool cache

Tool caches are scoped to a vault. Add a tool-cache block at the top level of your run definition, then add a tool-cache: <name> key to any task that should use one. Tool cache names must be unique within a vault. By convention, encode the run and task — for example, ci-npm-install for the npm-install task in .rwx/ci.yml:

tool-cache:
  vault: awesome_project_main

tasks:
  - key: npm-install
    run: npm install
    tool-cache: ci-npm-install

Locked vaults and write access

Write access to a cache is gated by the vault it belongs to. A run can always read from a tool cache, but it can only write to the cache when the vault is unlocked for that run. This ensures runs on main only read cache contents produced by other runs on main. You should always configure tool caches against a vault that is unlocked only by your protected branch (e.g., your_repo_main unlocked by main).

How layers work

A tool cache is structured as an initial layer plus up to 3 incremental layers on top of it. The first successful execution of a task with write access to a tool cache creates the initial layer. A subsequent successful execution of a task (which starts with that initial layer) appends a new incremental layer. When a fourth incremental layer would be added, the tool cache resets to just the initial layer, the task is executed, and then a new incremental layer is added on top of the initial layer.

Because the initial layer is not reset even when the incremental layers are, it can drift over time from a clean install. There are three ways to invalidate the entire tool cache entry and force the initial tool cache layer to be repopulated by the next successful execution:

  • You manually evict the tool cache from the RWX UI under Vaults.
  • A cache rebuild trigger executes.
  • The tool cache's 48-hour TTL expires.

The TTL counts from when the cache was last created or reset, and it isn't extended by incremental writes. Configure a cache rebuild trigger to refresh the cache and generate a brand new initial layer with a new 48 hour TTL.