dataengineering

Efficiently diff data in or across relational databases

Open Standard for Metadata. A Single place to Discover, Collaborate and Get your data right.

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

Your CLI for ELT+. It's open source, flexible, and scales to your needs. Confidently move, transform, and test your data using tools you know with a data engineering workflow you’ll love. ---------------------------------------This month, every ⭐ on GitHub removes 2 lb/~1 kg of trash from our waterways 🌊

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)

A Data Platform built for AWS, powered by Kubernetes.

An open source development framework to help you build data workflows and modern data architecture on AWS.

Build, test, deploy, iterate - Dev and prod tool for data science pipelines

Data engineering interviews Q&A for data community by data community

Predict stock price based on financial news feeds

Apply for a job at Olist's Data Team: https://olist.gupy.io/

Instant search for and access to many datasets in Pyspark.

Dockerizing an Apache Spark Standalone Cluster

Found a data engineering challenge or participated in a selection process ? Share with us!

Forecasting Solar Power: Analysis of using a LSTM Neural Network

Build & Learn Data Engineering,Machine Learning over Kubernetes. No Shortcut approach.

kedro cli plugin for generating a static kedro viz site (html, css, js) that can be deployed on many serverless tools.

Challenge Data Engineer

Duke MIDS: Data Engineering and DataOps Course