Spark Benchmark suite to evaluate cluster configuration and compare the performance with other big data frameworks.

Demoing Spark 2.2 and Elasticsearch Hadoop connector

QryGraph is a platform for managing Pig queries and creating and modifying queries in a graphical user interface.

Library for remote JVM ExecutorService with only dependency being password-less SSH -- Run clustered Hadoop/Spark jobs from IDE -- IDE-pimped Spark shell with full auto-completion!

This Big Data project consists of obtaining data on vehicle theft in the city of São Paulo and consolidating it in a counting and heat map, in order to show areas with a higher index of this type of crime. All applicable in AWS Resources.

Scope of this project is to calculate Daily Revenue from retail products

A Spark application to process the DBLP dataset to find out the Page Rank of faculty at the UIC CS department based on their co-authorships on publications.

This is the implementation of an algorithm that finds traceability links in two graphs such that the other graph is a perturbed version of the original graph.

Update PubMed articles daily on HDFS by using Spark Cluster

Spark Streaming Big Data Hadoop

Lab with Scala and Spark Streaming

Hadoop balancer which helps balance disks on a single node

A skeleton to generate a Spark job project in Scala with local distributed environment for development, example at (https://github.com/s3ni0r/spark-app-example)

Batch ETL data pipeline built on HDP 3.0 to process daily sales and business data to procedure power Bi reports. Automated the pipelines using Airflow.

Simple and intuitive Hadoop Paths

An ETL pipeline that extracts data from HDFS , transforms using spark and writes back to HDFS.

In this project, we are going to build a Bicycle sharing demand prediction service using Apache Spark and Scala. I have created a two spark application one for model generation and another for model demand prediction.

🌟Spark Ceph Connector: Implementation of Hadoop Filesystem API for Ceph

some codes are created when learning BigData