Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

Smart Automation Tool for building modern Data Lakes and Data Pipelines

Data Lineage Tracking And Visualization Solution

Flowman is an ETL framework powered by Apache Spark. With its declarative approach, Flowman simplifies the development of complex data pipelines.

基于Spark+SparkMLlib+Debezium+Deequ打造的简单易用、超高性能大数据治理引擎。适用于批流一体的数据集成和数据分析，支持CDC实时数据采集、机器学习算法模型、数据质量校验、数据标注、敏感数据识别、数据建模、算法建模和OLAP数据分析

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.

Projects from TrendyTech

High performance data store solution

WASP is a framework to build complex real time big data applications. It relies on a kind of Kappa/Lambda architecture mainly leveraging Kafka and Spark. If you need to ingest huge amount of heterogeneous data and analyze them through complex pipelines, this is the framework for you.

This project is a data processing application built with Apache Spark and Scala. This is designed to efficiently process, analyze and transform large datasets related to people data. It leverages Spark’s distributed computing capabilities to handle scalable data ingestion, cleaning and reporting. Shell scripts are included for hadoop deployment.

For artificial intelligence with sbt to build - maven build pls step forward to https://github.com/bgfurfeature/AI

Dynamic Conformance Engine

Architecture et développement des systèmes distribuées à large echelle

全国大数据竞赛三等奖解决方案，省赛二等奖解决方案。一键安装大数据环境脚本，自动部署集群环境，包括zookeeper、hadoop、mysql、hive、spark以及一些基础环境。已通过实际服务器测试，效果极佳，仅需要输入密码等少量人为干预。解放安装部署配置所需人力。并添加若干scala案例，结合spark用以进行数据准备。

An application for DNA sequence analysis written in Scala and deployed using Apache Spark&Hadoop.

hadoop, spark...

深圳地铁大数据客流分析系统🚇🚄🌟

Waimak is an open-source framework that makes it easier to create complex data flows in Apache Spark.

Average Temperature - Hadoop - Mapper - Reducer