Skip to content
#

spark-sql

Here are 322 public repositories matching this topic...

imback82
imback82 commented Mar 19, 2019

We create multiple jars during our builds to accommodate multiple versions of Apache Spark. In the current approach, the implementation is copied from one version to another and then necessary changes are made.

An ideal approach could create a common directory and extract common classes from duplicate code. Note that even if class/code is exactly the same, you cannot pull out to a common clas

carlbrochu
carlbrochu commented Apr 18, 2019

Describe the bug
Some utilities project are duplicated between singular (utility) and plural versions (utilities). Let's align to plural versions

To Reproduce
Steps to reproduce the behavior:

  1. Go to Services\DataX.Utilities folder
  2. Notice duplicate folders like 2 cosmosdb utils

Expected behavior
One dll per area

A complete example of a big data application using : Kubernetes (kops/aws), Apache Spark SQL/Streaming/MLib, Apache Flink, Scala, Python, Apache Kafka, Apache Hbase, Apache Parquet, Apache Avro, Apache Storm, Twitter Api, MongoDB, NodeJS, Angular, GraphQL

  • Updated Feb 1, 2019
  • TypeScript

Apache Spark is a fast, in-memory data processing engine with elegant and expressive development API's to allow data workers to efficiently execute streaming, machine learning or SQL workloads that require fast iterative access to datasets.This project will have sample programs for Spark in Scala language .

  • Updated May 15, 2020
  • Scala
frank-dkvan
frank-dkvan commented Sep 2, 2019

Describe the bug
I just write number 1 to 10 into a Pulsar Topic thru spark-pulsar connector . When I use Pulsar SQL to read the 10-number, the exception info occured "Query 20190902_101219_00035_9r62s failed: Topic persistent://public/default/topic_test1 does not have a valid schema"

Screenshots

  1. A simple Spark-Pulsar code to write 1-10 value into a Pulsar Topic
    ![image](https://u

Improve this page

Add a description, image, and links to the spark-sql topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the spark-sql topic, visit your repo's landing page and select "manage topics."

Learn more

You can’t perform that action at this time.