Prerequisites
Before proceeding, ensure the following prerequisites are met:- Install MindsDB locally via Docker or use MindsDB Cloud.
- To connect Databricks to MindsDB, install the required dependencies following this instruction.
- Install or ensure access to Databricks.
Implementation
This handler is implemented usingdatabricks-sql-connector
, a Python library that allows you to use Python code to run SQL commands on Databricks clusters and Databricks SQL warehouses.
The required arguments to establish a connection are as follows:
server_hostname
is the server hostname for the cluster or SQL warehouse.http_path
is the HTTP path of the cluster or SQL warehouse.access_token
is a Databricks personal access token for the workspace.
session_configuration
is a dictionary of Spark session configuration parameters.http_headers
stores additional (key, value) pairs to set in HTTP headers on every RPC request the client makes.catalog
is the catalog to use for the connection. Typically, defaults tohive_metastore
if not provided.schema
is the schema (database) to use for the connection. Defaults todefault
if not provided.