Setting up Databricks - Kameleoon Documentation

Data warehouse integrations are available as a premium add-on for Kameleoon’s Web Experimentation and Feature Experimentation module. For more information, contact your Customer Success Manager.

Steps 1 to 3 happen in your Databricks account, and step 4 happens in Kameleoon. Kameleoon recommends that your Databricks administrator performs steps 1 to 3. With the Databricks integration, you can simplify data retrieval for targeted campaigns and personalized user experiences. Key benefits:

Lets you target visitors with audiences built from your Databricks tables.
Lets you use Databricks conversions as goal metrics in experiment results.

Sync frequency:

Databricks as a source: Each Data Ingestion Task runs at the Frequency you set when you create it, for example per day. Kameleoon polls goal queries every hour and merges the collected conversions into experiment results once per day. See Data Ingestion Tasks.
Databricks as a destination: Kameleoon sends exposure events to Databricks whenever one of your experiments exposes a visitor. See how the destination works.

Prerequisites

To configure this integration, you need the following information:

Databricks personal access token (PAT)
Proper access to create Databricks schema and grant access.

Setup

1. Create a personal access token (PAT)

Kameleoon will authenticate to your Databricks SQL warehouse with a personal access token. You should create a Databricks service principal and then create a PAT for that service account. Once you create a service principal, you can generate a PAT with the Databricks CLI, using the Service Principal “Application Id” that you can find in the Service Principal management page of the Databricks UI.

databricks token-management create-obo-token {Service Principal Application Id} --lifetime-seconds 7776000 --comment "Token for Kameleoon service principal"

You create the token in your Databricks workspace and then enter it in Kameleoon. For the full procedure, see the Databricks documentation on personal access tokens.

2. Create Kameleoon schemas and grant Unity Catalog access

For both source and destination, give the service principal CAN USE on the Databricks SQL warehouse in its Permissions settings. For Unity Catalog, grant access to each catalog Kameleoon uses:

GRANT USE CATALOG ON CATALOG my_catalog TO `{Service Principal Application Id}`;

Your Databricks administrator creates the schemas below. Kameleoon’s service principal doesn’t need CREATE SCHEMA on the catalog.

When using Databricks as a source

For audience ingestion, create a dedicated schema in the catalog that contains the data Kameleoon polls. Name it kameleoon_configuration. Kameleoon creates, reads, and removes audience snapshot tables in this schema. Grant the service principal access to this schema only. It doesn’t need write access to your source tables.

CREATE SCHEMA my_catalog.kameleoon_configuration;
GRANT USE SCHEMA ON SCHEMA my_catalog.kameleoon_configuration TO `{Service Principal Application Id}`;
GRANT CREATE TABLE ON SCHEMA my_catalog.kameleoon_configuration TO `{Service Principal Application Id}`;
GRANT SELECT ON SCHEMA my_catalog.kameleoon_configuration TO `{Service Principal Application Id}`;

When using Databricks as a destination: Create a separate schema for Kameleoon event tables. Kameleoon creates tables and inserts events there. MODIFY also requires SELECT on the tables.

CREATE SCHEMA my_catalog.kameleoon_events;
GRANT USE SCHEMA ON SCHEMA my_catalog.kameleoon_events TO `{Service Principal Application Id}`;
GRANT CREATE TABLE ON SCHEMA my_catalog.kameleoon_events TO `{Service Principal Application Id}`;

The service principal owns the tables it creates, so you don’t need an extra table grant for them. If event tables already exist under another owner, grant SELECT and MODIFY on those tables. The following schema-level grants also cover current and future event tables in this dedicated schema:

GRANT SELECT ON SCHEMA my_catalog.kameleoon_events TO `{Service Principal Application Id}`;
GRANT MODIFY ON SCHEMA my_catalog.kameleoon_events TO `{Service Principal Application Id}`;

Replace {Service Principal Application Id} in these commands with your service principal’s application ID.

You can omit the my_catalog prefix when running queries directly in the necessary catalog.

3. Grant read access to your data

For Databricks as a source, grant access only to the customer tables or views used by your audience and goal queries. First grant usage on their schema:

GRANT USE SCHEMA ON SCHEMA my_catalog.user_data TO `{Service Principal Application Id}`;

Then grant SELECT on each required table or view. Alternatively, grant it on the schema if Kameleoon should read all current and future tables and views in it:

GRANT SELECT ON TABLE my_catalog.user_data.user_account_table TO `{Service Principal Application Id}`;

For a view, use:

GRANT SELECT ON VIEW my_catalog.user_data.my_view TO `{Service Principal Application Id}`;

GRANT SELECT ON SCHEMA my_catalog.user_data TO `{Service Principal Application Id}`;

The destination doesn’t need SELECT or MODIFY on your customer-data schema. If you use goals without audience ingestion, Kameleoon only reads your query results and doesn’t create audience snapshots.

4. Enter your credentials in Kameleoon

In the Databricks integration settings of your project in Kameleoon, complete the Credentials fields:

Databricks Server Hostname: the server hostname of your Databricks workspace.
HTTP Path: the path to your Databricks SQL warehouse.
Personal Access Token (PAT): the token you created in step 1.
PAT expiry timestamp in milliseconds (optional): the expiry time of the token, as a Unix timestamp in milliseconds. This field is optional, and you enter the value yourself.

Edit form for the Databricks credentials, with fields for the server hostname, HTTP path, personal access token (empty), and the optional PAT expiry timestamp in milliseconds.

To replace a token, enter the new token in the Personal Access Token (PAT) field and click Confirm. After you save, Kameleoon shows the credentials in a summary card. The card displays the hostname, HTTP path and PAT expiry, but not the token.

Saved Databricks credentials card showing the server hostname, HTTP path and PAT expiry timestamp, with edit and delete icons.

After the credentials are saved, you can send experiment exposure events to Databricks. See Use Databricks as a destination.

Network and security

Kameleoon authenticates to Databricks with a personal access token. Kameleoon supports IP allowlisting. If you implement IP access lists, contact your Kameleoon account manager, so they can provide you with the list of Kameleoon IPs you must authorize.

Data region

Kameleoon stores the data from this integration in the Kameleoon data region of your account, which is the US or the EU.

Integrations

​Prerequisites

​Setup

​1. Create a personal access token (PAT)

​2. Create Kameleoon schemas and grant Unity Catalog access

When using Databricks as a source

​3. Grant read access to your data

​4. Enter your credentials in Kameleoon

​Network and security

​Data region

Prerequisites

Setup

1. Create a personal access token (PAT)

2. Create Kameleoon schemas and grant Unity Catalog access

3. Grant read access to your data

4. Enter your credentials in Kameleoon

Network and security

Data region