Spark Streaming Big Data Hadoop

Update PubMed articles daily on HDFS by using Spark Cluster

Wealth Management Platform

Implementation of data processing in the MapReduce model.

STM data enrichment, Extract, Transform, Load (e.g., ETL)

Import and process HDF5 files on Spark with Hadoop

Distributed computational problem-solving project, which aims to perform large-scale graph matching using cloud computing technologies. The project allows users to import two directed graphs and analyze the differences between them.

QryGraph is a platform for managing Pig queries and creating and modifying queries in a graphical user interface.

Library for remote JVM ExecutorService with only dependency being password-less SSH -- Run clustered Hadoop/Spark jobs from IDE -- IDE-pimped Spark shell with full auto-completion!

This Big Data project consists of obtaining data on vehicle theft in the city of São Paulo and consolidating it in a counting and heat map, in order to show areas with a higher index of this type of crime. All applicable in AWS Resources.

Scope of this project is to calculate Daily Revenue from retail products

some codes are created when learning BigData

Optimal distributed data deduplication and supervised learning pipeline using Apache Spark

Semester assignment for ECE NTUA 3189 Advanced Topics in Database Systems

🌟Spark Ceph Connector: Implementation of Hadoop Filesystem API for Ceph

Simple and intuitive Hadoop Paths

An ETL pipeline that extracts data from HDFS , transforms using spark and writes back to HDFS.

Demoing Spark 2.2 and Elasticsearch Hadoop connector