Implements an insert with TABLOCK option to improve write performanceĭisables strict data frame and sql table schema check when set to false Set this value to data source name to write a data pool table in Big Data Clusters None implies the value is not set and the connector should write to SQL Server single instance. NO_DUPLICATES implements an reliable insert in executor restart scenarios In addition following options are supported OptionīEST_EFFORT or NO_DUPLICATES. The Apache Spark Connector for SQL Server and Azure SQL supports the options defined here: SQL DataSource JDBC

Reliable connector support for Sql Server Single Instance.

Support for write to SQL Server Single instance and Data Pool in SQL Server Big Data Clusters.Basic authentication and Active Directory (AD) Key Tab support.Support for all Spark bindings (Scala, Python, R).For the latest information about the connector, see SQL Spark connector GitHub repository.

You can also build the connector from source or download the jar from the Release section in GitHub. Both versions can be found here and can be imported using the coordinates below: ConnectorĬom.microsoft.azure:spark-mssql-connector:1.0.2Ĭom.microsoft.azure:spark-mssql-connector_2.12:1.1.0Ĭom.microsoft.azure:spark-mssql-connector_2.12:1.2.0 There are two versions of the connector available through Maven, a 2.4.x compatible version and a 3.0.x compatible version. This library contains the source code for the Apache Spark Connector for SQL Server and Azure SQL.Īpache Spark is a unified analytics engine for large-scale data processing. The connector allows you to use any SQL database, on-premises or in the cloud, as an input data source or output data sink for Spark jobs.

The Apache Spark connector for SQL Server and Azure SQL is a high-performance connector that enables you to use transactional data in big data analytics and persist results for ad-hoc queries or reporting.

Maven microsoft sql server jdbc driver