databricks/databricks-sdk-py

databricks/databricks-sdk-py

Description: Databricks SDK for Python (Beta)

Language: Python

License: Apache-2.0

Stars: 550

Forks: 204

Open issues: 322

Created: 2022-06-21T13:49:09Z

Pushed: 2026-06-10T09:05:03Z

Default branch: main

Fork: no

Archived: no

README:

Databricks SDK for Python (Beta)

  

Beta: This SDK is supported for production use cases, but we do expect future releases to have some interface changes; see [Interface stability](#interface-stability). We are keen to hear feedback from you on these SDKs. Please file issues, and we will address them. | See also the SDK for Java | See also the SDK for Go | See also the Terraform Provider | See also cloud-specific docs (AWS, Azure, GCP) | See also the API reference on readthedocs

The Databricks SDK for Python includes functionality to accelerate development with Python for the Databricks Lakehouse. It covers all public Databricks REST API operations. The SDK's internal HTTP client is robust and handles failures on different levels by performing intelligent retries.

Contents

• [Getting started](#getting-started)
• [Code examples](#code-examples)
• [Authentication](#authentication)
• [Long-running operations](#long-running-operations)
• [Paginated responses](#paginated-responses)
• [Retries](#retries)
• [Single-sign-on with OAuth](#single-sign-on-sso-with-oauth)
• [User Agent Request Attribution](#user-agent-request-attribution)
• [Error handling](#error-handling)
• [Logging](#logging)
• [Integration with dbutils](#interaction-with-dbutils)
• [Interface stability](#interface-stability)

Getting started

1. Please install Databricks SDK for Python via pip install databricks-sdk and instantiate WorkspaceClient:

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
for c in w.clusters.list():
print(c.cluster_name)

Databricks SDK for Python is compatible with Python 3.7 _(until June 2023)_, 3.8, 3.9, 3.10, and 3.11. Note: Databricks Runtime starting from version 13.1 includes a bundled version of the Python SDK. It is highly recommended to upgrade to the latest version which you can do by running the following in a notebook cell:

%pip install --upgrade databricks-sdk

followed by

dbutils.library.restartPython()

Code examples

The Databricks SDK for Python comes with a number of examples demonstrating how to use the library for various common use-cases, including

• Using the SDK with OAuth from a webserver
• Using long-running operations
• Authenticating a client app using OAuth

These examples and more are located in the `examples/` directory of the Github repository.

Some other examples of using the SDK include:

• Unity Catalog Automated Migration heavily relies on Python SDK for working with Databricks APIs.
• ip-access-list-analyzer checks & prunes invalid entries from IP Access Lists.

Authentication

If you use Databricks configuration profiles or Databricks-specific environment variables for Databricks authentication, the only code required to start working with a Databricks workspace is the following code snippet, which instructs the Databricks SDK for Python to use its [default authentication flow](#default-authentication-flow):

from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
w. # press for autocompletion

The conventional name for the variable that holds the workspace-level client of the Databricks SDK for Python is w, which is shorthand for workspace.

In this section

• [Default authentication flow](#default-authentication-flow)
• [Unified host support](#unified-host-support)
• [Databricks native authentication](#databricks-native-authentication)
• [Azure native authentication](#azure-native-authentication)
• [Overriding .databrickscfg](#overriding-databrickscfg)
• [Additional authentication configuration options](#additional-authentication-configuration-options)

Default authentication flow

If you run the Databricks Terraform Provider, the Databricks SDK for Go, the Databricks CLI, or applications that target the Databricks SDKs for other languages, most likely they will all interoperate nicely together. By default, the Databricks SDK for Python tries the following authentication methods, in the following order, until it succeeds:

1. [Databricks native authentication](#databricks-native-authentication) 2. [Azure native authentication](#azure-native-authentication) 3. [GCP…