Apache DolphinScheduler is the modern data workflow orchestration platform with powerful user interface, dedicated to solving complex task dependencies in the data pipeline and providing various types of jobs available `out of the box`
An end to end ML project. Using MLflow for experiment tracking and model registry. Prefect for workflow orchestration. S3 for artifacts storage. AWS Lambda/ ECR for serverless model serving. AWS REST API gateway as endpoint to lambda function. GitHub Actions for CI/CD.