Skip to content

Deprecating run_kubernetes (and maybe run_docker). #92

@nbren12

Description

@nbren12

These functions served us well, but haven't been used in fv3net for a while, mainly because it was difficult to extend run_kubernetes to support I/O patterns involving shared volumes. We have largely replaced use of run_kuberernetes with argo workflows as well as these composable functions for working with containers: https://github.com/VulcanClimateModeling/fv3net/blob/master/external/fv3kube/fv3kube/containers.py. In both cases, we have decomposed the process of running fv3 into a few steps

  1. prepare the full fv3config.yml object from a shorter diff configuration
  2. write the run directory
  3. run the model
  4. post process

Since each of these steps have very different requirements (e.g. 2 and 4 needs auth, but 3 does not) we now run them in separate containers.

IMO, for containerized workflows the role of fv3config is really isolated to step 2, so I think it would make sense to deprecate/remove run_kubernetes. Thoughts?

It is important to support run_native as a legacy option for testing purposes, and I'm not sure about run_docker for local development. I don't really use it, but others might.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions