Implementation of data processing in the MapReduce model.

STM data enrichment, Extract, Transform, Load (e.g., ETL)

Import and process HDF5 files on Spark with Hadoop

Distributed computational problem-solving project, which aims to perform large-scale graph matching using cloud computing technologies. The project allows users to import two directed graphs and analyze the differences between them.

Spark Benchmark suite to evaluate cluster configuration and compare the performance with other big data frameworks.

Spark Streaming Big Data Hadoop

Update PubMed articles daily on HDFS by using Spark Cluster

Wealth Management Platform

Simple and intuitive Hadoop Paths

An ETL pipeline that extracts data from HDFS , transforms using spark and writes back to HDFS.

COMP9313 Big Data Management 2019T2

Data Algorithms for Apache Spark RDD and Hadoop Mapreduce in Scala and Java

This project is a data processing application built with Apache Spark and Scala. This is designed to efficiently process, analyze and transform large datasets related to people data. It leverages Spark’s distributed computing capabilities to handle scalable data ingestion, cleaning and reporting. Shell scripts are included for hadoop deployment.

QryGraph is a platform for managing Pig queries and creating and modifying queries in a graphical user interface.

Library for remote JVM ExecutorService with only dependency being password-less SSH -- Run clustered Hadoop/Spark jobs from IDE -- IDE-pimped Spark shell with full auto-completion!

Scope of this project is to calculate Daily Revenue from retail products

This Big Data project consists of obtaining data on vehicle theft in the city of São Paulo and consolidating it in a counting and heat map, in order to show areas with a higher index of this type of crime. All applicable in AWS Resources.

Demoing Spark 2.2 and Elasticsearch Hadoop connector