sqoop

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

bi kafka spark hive hadoop etl scheduler ide hbase portal mask sqoop data-quality data-map

Updated Jul 2, 2020

WeBankFinTech / Exchangis

Star

Exchangis is a lightweight,highly extensible data exchange platform that supports data transmission between structured and unstructured heterogeneous data sources

plugin lightweight data-source sqoop datax exchangis transmission-engine

Updated Nov 16, 2020
Java

Pushkr / Apache-Spark-Hands-On

Star

Educational notes,Hands on problems w/ solutions for hadoop ecosystem

spark hive hadoop bigdata cloudera handson cheatsheet flume basics sqoop exposure cca175

Updated Jan 22, 2019
Python

aliyun / aliyun-maxcompute-data-collectors

Star

bigdata kettle ogg flume sqoop aliyun-maxcompute

Updated Oct 13, 2020
Java

bluishglc / bdp

Star

A prototype project of big data platform, the source codes of the book Big Data Platform Architecture and Prototype

redis demo kafka spark prototype bigdata spark-streaming quickstart sparksql oozie sqoop spark-sql spark-streaming-examples sqoop-import spark-demo middle-end middle-office spark-examples

Updated Aug 12, 2020
Java

dimajix / spark-training

Star

Repository used for Spark Trainings

python scala spark hive hadoop pyspark spark-streaming sqoop spark-ml hadoop-training spark-training

Updated Jul 2, 2020
Jupyter Notebook

Cigna / ibis

Star

IBIS is a workflow creation-engine that abstracts the Hadoop internals of ingesting RDBMS data.

workflow hadoop ingestion oozie sqoop sqoop2 workflow-automation workflow-scheduler hadoop-ecosystem hadoop-framework ibis cigna

Updated Oct 21, 2018
Python

ameizi / cloud

Star

云计算之hadoop、hive、hue、oozie、sqoop、hbase、zookeeper环境搭建及配置文件

hive hadoop hbase zookeeper pig flume oozie hue sqoop flume-ng

Updated Apr 25, 2017
Shell

vivek2319 / Learn-Hadoop-and-Spark

Star

This repository focuses on gathering and making a curated list resources to learn Hadoop for FREE.

Updated Jun 10, 2018
Python

Powerspace / pg2bq

Star

Export PostgreSQL tables to Google BigQuery

bigquery postgresql gcloud sqoop

Updated Feb 20, 2018
Scala

peiliping / meepo

Star

异构存储数据迁移

mysql parquet roaringbitmap sqoop datax

Updated Nov 20, 2017
Java

gouchaoer / Increment_Backup_To_Hive

Star

一个增量备份关系数据库(MySQL, PostgreSQL, SQL Server, SQLite, Oracle等)到hive的php脚本工具

mysql php hive sqoop

Updated Jun 14, 2018
PHP

yangzheng10 / web-click-flow

Star

网站点击流离线日志分析

hive hadoop etl mapreduce flume sqoop

Updated Sep 13, 2018
Java

san089 / Cloudera_Material

Star

Cloudera_Material: Study Material to help people preparing for Cloudera CCA Spark and Hadoop Developer Exam (CCA175). Feel free to collaborate.

big-data spark hive hadoop bigdata cloudera pyspark cca flume certification sqoop cca175 hive-metastore sqoop-session sqoop-export sqoop-import

Updated Apr 21, 2020

Jayvardhan-Reddy / BigData-Ecosystem-Architecture

Star

Life-cycle: Internal working of HDFS, SQOOP, HIVE, SPARK, HBASE, KAFKA with code.

kafka big-data spark hive hadoop architecture bigdata hbase zookeeper spark-streaming hdfs sqoop hadoop-ecosystem architecture-components yarn-hadoop-cluster bigdata-module hbase-cluster big-data-essentials hadooparchitecture

Updated Sep 10, 2019
Shell

milindjagre / HDPCD

Star

This repository contains all the documents related to HDPCD certification.

data hive hadoop bigdata apache pig flume big certification pig-latin hortonworks sqoop hdpcd hcatalog

Updated Sep 13, 2017
PigLatin

maniram-yadav / Big_DataHadoop_Projects

Star

Big data projects implemented by Maniram yadav

spark hive hadoop pig hdfs mapreduce flume pig-latin sqoop hadoop-mapreduce big-data-analytics hadoop-hdfs big-data-projects

Updated May 5, 2018
PigLatin

rss161030 / ETL-processes-using-Sqoop-Hadoop-Hive-Spark-and-Scala

Star

I implemented various ETL processes like loading the data using sqoop from mysql to hdfs, transform the data using Spark and Scala, perform analytics using Spark and Scala and loading the data back to HDFS.

scala spark hdfs sqoop