Apache Spark - A unified analytics engine for large-scale data processing

Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark

Clojure(Script) library for declarative data description and validation

Open source dataflow execution framework