Capture the logical plan from Spark (SQL)

This project is used for tracking lineage when using spark. Our team is aimed at enhancing the ability of column relation during logical plan analysis.

挖坑与填坑

Notes talking about the design and implementation of Apache Spark

English SDK for Apache Spark

Qubole Sparklens tool for performance tuning Apache Spark

Dr. Elephant is a job and flow-level performance monitoring and tuning tool for Apache Hadoop and Apache Spark

Spark Knowledge Base

Spark reference applications

Development in Shark has been ended.

SQL parser written using Scala's parser combinator library

A Macro library for working with Spark SQL in a typesafe way.

Apache Spark docker image

Extensible Rules Engine for custom Dataframe / Dataset validation

Official Dockerfile for Apache Spark

Bitnami container images

Compass is a task diagnosis platform for bigdata

Examples for High Performance Spark

Magic to help Spark pipelines upgrade