Working examples for some components on GCP, and instructions on how to run them.

Under construction....

Neo4j, Cassandra, Hadoop, PySpark, RDD, MapReduce, Cluster-Computing, DataProc

Support code for the article "Connecting GCP Dataproc and Elasticsearch: Bridging the Worlds of Big Data and (vector) Search"

Co-Purchase Analysis with Scala and Spark. Project for Scalable and Cloud Programming (81942) university course @unibo

dataprochs 🐘🐝 : Dataproc Cluster # Apache # Hadoop # MapReduce # Spark # YARN # HDFS

数据处理工作流，elt

An e-commerce data lakehouse implemented on Google Cloud Platform (GCP). This project features an end-to-end data pipeline, from raw data generation via Cloud Functions, layered processing with PySpark on Dataproc, to structured data warehousing in BigQuery. It's fully orchestrated by Apache Airflow, enabling analytics and BI with Metabase.

Implementation of data processing in the MapReduce model.

Modular Global HTTP Load Balancer for GCE using forwarding rules.

Learning spark the right way

The ETL pipeline for Stock info.

Coding book counter words with PySpark for Digital Innovation One challenge

Data Engineering Using Google Could Platform and Mage

Repositorio para realizar el curso en Udemy llamado "Airflow2.0 De 0 a Héroe", de la academia "Datapath".

PySpark-based ETL pipeline leveraging Dataproc, Cloud Storage, Cloud Run Functions and BigQuery, to automate Spotify "New Releases" data processing and visualization in Looker Studio.

Running a wordcount job on a Google Dataproc cluster

The Tokyo Olympic Data Analysis on GCP project is a comprehensive solution for analyzing and visualizing Olympic Games data using various GCP services.

Spark-приложение для потоковой обработки датасета олимпийских достижений и рекордов

Extract stock market data, then uploading it to GCP using Airflow, GCS, BigQuery and Spark