QryGraph is a platform for managing Pig queries and creating and modifying queries in a graphical user interface.

Library for remote JVM ExecutorService with only dependency being password-less SSH -- Run clustered Hadoop/Spark jobs from IDE -- IDE-pimped Spark shell with full auto-completion!

This Big Data project consists of obtaining data on vehicle theft in the city of São Paulo and consolidating it in a counting and heat map, in order to show areas with a higher index of this type of crime. All applicable in AWS Resources.

Scope of this project is to calculate Daily Revenue from retail products

Spark Benchmark suite to evaluate cluster configuration and compare the performance with other big data frameworks.

Demoing Spark 2.2 and Elasticsearch Hadoop connector

Lab with Scala and Spark Streaming

Hadoop balancer which helps balance disks on a single node

A skeleton to generate a Spark job project in Scala with local distributed environment for development, example at (https://github.com/s3ni0r/spark-app-example)

Batch ETL data pipeline built on HDP 3.0 to process daily sales and business data to procedure power Bi reports. Automated the pipelines using Airflow.

🌟Spark Ceph Connector: Implementation of Hadoop Filesystem API for Ceph

some codes are created when learning BigData

Optimal distributed data deduplication and supervised learning pipeline using Apache Spark

Semester assignment for ECE NTUA 3189 Advanced Topics in Database Systems

A Spark application to process the DBLP dataset to find out the Page Rank of faculty at the UIC CS department based on their co-authorships on publications.

This is the implementation of an algorithm that finds traceability links in two graphs such that the other graph is a perturbed version of the original graph.

Simple and intuitive Hadoop Paths

An ETL pipeline that extracts data from HDFS , transforms using spark and writes back to HDFS.