Skip to content
View MekWiset's full-sized avatar
🤓
Nerding
🤓
Nerding
5 followers · 6 following

Achievements

Achievement: Quickdraw

Achievements

Achievement: Quickdraw

Block or report MekWiset

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
MekWiset/README.md

Hi there 👋, I'm Mek

Banner

Data Engineer with 2.5 years of experience building and maintaining data pipelines across clouds and large-scale migration projects.


💻 Languages and Tools:

Python Apache Airflow Apache Kafka Pandas Apache Spark Docker AWS Azure Google Cloud Postgres ApacheCassandra MongoDB macOS Visual Studio Code Notion

📝 Recent Medium Articles

Recent Article 0

🚀 Featured Projects

1. MultiCloud Data Migration Pipeline

  • ℹ️ Description: Big Data Migration from GCP to Azure.
  • 🏆 Achievements: Successfully migrated over 19 million rows of data from Google Cloud Storage to Azure Data Lake, ensuring data integrity with zero data loss.
  • 🎯 Technologies used:
    • Processing Tools: PySpark
    • GCP Services: Google Cloud Storage (GCS), BigQuery
    • Azure Services: Azure Data Factory, Azure Data Lake Storage Gen 2, Databricks, Key Vault
    • Others: Docker

2. Medallion Data Lakehouse

  • ℹ️ Description: Building a Data Lakehouse using the Medallion architecture.
  • 🏆 Achievements: Developed a scalable Data Lakehouse architecture using the Medallion framework, facilitating efficient data storage, processing, and analysis with seamless integration across Azure services.
  • 🎯 Technologies used:
    • Processing Tools: DBT (Data Build Tool)
    • Azure Services: Azure SQL Database, Azure Data Lake Storage Gen 2, Databricks, Azure Key Vault

3. Realtime Data Streaming

  • ℹ️ Description: Real-time data ingestion to Cassandra using Airflow, Kafka, and Spark.
  • 🏆 Achievements: Engineered a robust real-time data streaming pipeline, enabling low-latency data ingestion into Cassandra and ensuring consistent data flow and processing across multiple platforms.
  • 🎯 Technologies used:
    • Processing Tools: PySpark
    • Orchestration Tools: Airflow, Kafka, Zookeeper
    • Monitoring: Confluent
    • Storage: Cassandra
    • Others: Docker

Pinned Loading

  1. MultiCloud_Data_Migration_Pipeline MultiCloud_Data_Migration_Pipeline Public

    Big Data Migration Across Cloud Platforms.

    Python 1

  2. Medallion_DataLakehouse Medallion_DataLakehouse Public

    Building a Data Lakehouse using the Medallion architecture.

    Jupyter Notebook 6 7

  3. PostgresToMongoDB_migration_project PostgresToMongoDB_migration_project Public

    Migrate data from Postgres Database to MongoDB.

    Python 2

  4. Realtime_Data_Streaming Realtime_Data_Streaming Public

    Real-time data ingestion to Cassandra using Airflow, Kafka, and Spark.

    Python 3 2