Azure Daily 2022 by Gordon

Generally available: Apache Spark 3.1.2 in Azure HDinsight

The latest release of Apache Spark 3.1 includes:

  • Adaptive query execution
  • Convert sort merge join to broadcast hash join
  • Spark catalyst optimizer
  • Dynamic partition pruning
  • Create new Spark 3.1 clusters (not Spark 3.0 clusters (preview))

Source: Generally available: Apache Spark 3.1.2 in Azure HDinsight

Apache Spark

Comments

Leave a Comment

All fields are required. Your email address will not be published.