Spark Benchmark suite to evaluate cluster configuration and compare the performance with other big data frameworks.

Library for remote JVM ExecutorService with only dependency being password-less SSH -- Run clustered Hadoop/Spark jobs from IDE -- IDE-pimped Spark shell with full auto-completion!

QryGraph is a platform for managing Pig queries and creating and modifying queries in a graphical user interface.

This Big Data project consists of obtaining data on vehicle theft in the city of São Paulo and consolidating it in a counting and heat map, in order to show areas with a higher index of this type of crime. All applicable in AWS Resources.

Scope of this project is to calculate Daily Revenue from retail products

Demoing Spark 2.2 and Elasticsearch Hadoop connector

Simple and intuitive Hadoop Paths

An ETL pipeline that extracts data from HDFS , transforms using spark and writes back to HDFS.

Hadoop balancer which helps balance disks on a single node

Lab with Scala and Spark Streaming

A skeleton to generate a Spark job project in Scala with local distributed environment for development, example at (https://github.com/s3ni0r/spark-app-example)

Batch ETL data pipeline built on HDP 3.0 to process daily sales and business data to procedure power Bi reports. Automated the pipelines using Airflow.

Update PubMed articles daily on HDFS by using Spark Cluster

Spark Streaming Big Data Hadoop

A Spark application to process the DBLP dataset to find out the Page Rank of faculty at the UIC CS department based on their co-authorships on publications.

This is the implementation of an algorithm that finds traceability links in two graphs such that the other graph is a perturbed version of the original graph.

Import and process HDF5 files on Spark with Hadoop

Implementation of data processing in the MapReduce model.