data-engineering

Apache Superset is a Data Visualization and Data Exploration Platform

Learn how to responsibly develop, deploy and maintain production machine learning applications.

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

The Data Engineering Cookbook

Roadmap to becoming a data engineer in 2021

The easiest way to coordinate your dataflow

Data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes.

Always know what to expect from your data.

Free Data Engineering course!

An orchestration platform for the development, production, and observation of data assets.

Fancy stream processing made operationally mundane

Open Source Feature Flagging and A/B Testing Platform

Feature Store for Machine Learning

pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

lakeFS - Git-like capabilities for your object storage

Kestra is an infinitely scalable orchestration and scheduling platform, creating, running, scheduling, and monitoring millions of complex pipelines.

The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️

A list of useful resources to learn Data Engineering from scratch

🧙 A modern replacement for Airflow.

Memphis is an Open-Source, Real-Time Data Processing Platform