Big data training material

A distributed file system program that works like Hadoop with minor changes. A completely working program that incorporates asynchronous distribution of files and map and reduce components. It has its own command line interfaces with all the required commands.

Trying best case apache spark working environment for robust data pipelines

Repository containing python code for MapReduce jobs to answer questions about Udacity forum data.

This repository contains a python script that can be used to solve multiple use-cases starting from os use cases like creating partitions, LVM, Hadoop tasks, AWS tasks, Docker tasks, etc.

Final Year Project, Big Data Technologies and Related Topics, Hadoop, Kafka, Spark, Hive, Dash, Plotly, Visualizations

Airflow pipeline consisting of spark job

Advanced Topics in Database Systems @ NTUA | 2022- 2023

Usage of different Big Data apps

A Simple SQL engine using Hadoop MapReduce

Crime Analytics is a project aimed at analyzing crime data in Los Angeles from 2020 onwards. The project utilizes statistical methods and data analysis techniques to identify trends, high-risk groups, and the effectiveness of law enforcement agencies.

HDP related tools

A mini-Hadoop clone capable of performing all DFS functionalities through a CLI

Running Map Reduce in Hadoop using Docker

Real Time Streaming: Twitter Data Pipeline Using Big data Tools

Big Data Assignments

Step By Step guide for Hadoop installation on Ubuntu 16.04.3 with MapReduce example using Streaming

Simple python script to convert multiple json files into a parquet or and ORC file to be used on Hadoop.