Library for remote JVM ExecutorService with only dependency being password-less SSH -- Run clustered Hadoop/Spark jobs from IDE -- IDE-pimped Spark shell with full auto-completion!

Scope of this project is to calculate Daily Revenue from retail products

A system to collect, analyze and visualize the tweets

An ETL pipeline that extracts data from HDFS , transforms using spark and writes back to HDFS.

Spark Streaming Big Data Hadoop

This is the implementation of an algorithm that finds traceability links in two graphs such that the other graph is a perturbed version of the original graph.

Data pipeline using nifi, kafka and spark

Hadoop balancer which helps balance disks on a single node

Lab with Scala and Spark Streaming

A skeleton to generate a Spark job project in Scala with local distributed environment for development, example at (https://github.com/s3ni0r/spark-app-example)

Batch ETL data pipeline built on HDP 3.0 to process daily sales and business data to procedure power Bi reports. Automated the pipelines using Airflow.

🌟Spark Ceph Connector: Implementation of Hadoop Filesystem API for Ceph

some codes are created when learning BigData

Semester assignment for ECE NTUA 3189 Advanced Topics in Database Systems

Average Temperature - Hadoop - Mapper - Reducer

PredictionIO Lead Scoring Engine Template (Scala-based parallelized engine)

Large-scale Geospatial Data Analysis using SparkSQL