dotnet / spark

5 Pull requests merged by 3 people

• Return UTF-8 encoded string from JVM => CLR

  #661 merged Sep 9, 2020

• Set up build pipeline for Spark-3.0.0

  #348 merged Sep 8, 2020

• Update azure-pipeline.yml to include comments on why E2E forward compatibility test does not run against Spark 2.4.6.

  #656 merged Sep 6, 2020

• TestSignaturesV2_3_X Tests for Spark-3.0.0

  #655 merged Sep 6, 2020

• Switch on C# pattern matching for matching types

  #646 merged Sep 5, 2020

3 Pull requests opened by 3 people

• Spark 3.0 readiness part 2

  #649 opened Sep 2, 2020

• FeatureHasher

  #652 opened Sep 3, 2020

• Broadcast and GroupedMapUdf Tests for Spark-3.0.0

  #653 opened Sep 4, 2020

12 Issues closed by 3 people

• [FEATURE REQUEST]: Ability to reference Nuget packages in UDFs in notebooks

  #409 closed Sep 9, 2020

• Microsoft.Data.Analysis.DataFrame

  #375 closed Sep 9, 2020

• [Error] [JvmBridge] java.sql.SQLException: No suitable driver

  #271 closed Sep 9, 2020

• Error encountered "An illegal reflective access operation has occurred" when running %SPARK_HOME%\bin\spark-submit --class org.apache.spark.deploy.dotnet.DotnetRunner --master local bin\Debug\netcoreapp3.1\microsoft-spark-2.4.x-0.10.0.jar dotnet bin\Debug\netcoreapp3.1\mySparkApp.dll

  #525 closed Sep 9, 2020

• 'spark-submit' is not recognized as the name of a cmdlet, function

  #532 closed Sep 9, 2020

• [BUG]: "failed to connect to" error using Get Started documentation

  #582 closed Sep 9, 2020

• [FEATURE REQUEST]: Support for Bucketizer

  #313 closed Sep 4, 2020

• spark-submit command not recognised

  #631 closed Sep 4, 2020

• [FEATURE REQUEST]: The ability to process a bunch of messages in Udf

  #377 closed Sep 4, 2020

• How to pass StructType between .NET & JVM?

  #651 closed Sep 3, 2020

• [FEATURE REQUEST]: Implement Broadcast Variable.

  #253 closed Sep 3, 2020

• [FEATURE REQUEST]: Structured DataFrame.Explain

  #471 closed Sep 3, 2020

6 Issues opened by 5 people

• .net Spark submit your application to run on Apache Spark

  #662 opened Sep 9, 2020

• Not able to debug UDF

  #660 opened Sep 9, 2020

• The system cannot find the path specified.

  #659 opened Sep 7, 2020

• [FEATURE REQUEST]: Clean up azure-pipelines.yaml (duplicate Test stages)

  #658 opened Sep 6, 2020

• Fix E2E tests for Spark 3.0.

  #657 opened Sep 6, 2020

• [FEATURE REQUEST]: Support GroupedMapUdf in Spark-3.0.0

  #654 opened Sep 4, 2020

11 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Broadcast encryption/decryption support

  #489 commented on Sep 9, 2020 • 6 new comments

• Add simpleworker test

  #644 commented on Sep 8, 2020 • 6 new comments

• Using with .NET Interactive notebooks locally

  #529 commented on Sep 9, 2020 • 2 new comments

• Support ArrayType

  #536 commented on Sep 5, 2020 • 2 new comments

• [BUG]: Not able to return error from UDF

  #518 commented on Sep 3, 2020 • 1 new comment

• [FEATURE REQUEST]: Consider adding support for common RDD operations

  #284 commented on Sep 3, 2020 • 1 new comment

• [FEATURE REQUEST]: add example showing N Choose K combinations

  #627 commented on Sep 3, 2020 • 1 new comment

• Spark 3.0 readiness part 1

  #647 commented on Sep 8, 2020 • 1 new comment

• [FEATURE REQUEST]: Implement ML Features

  #381 commented on Sep 3, 2020 • 0 new comments

• Launch dotnet backend jvm bridge process ondemand

  #562 commented on Sep 6, 2020 • 0 new comments

• [WIP] Full support for multithreaded applications

  #641 commented on Sep 9, 2020 • 0 new comments