Skip to main content
Once you’ve activated Databricks for a specific project, you can base a goal in Kameleoon on a query against your Databricks tables. Kameleoon reads the conversions for each goal in two steps: it queries your Databricks tables, then computes lift and renders outcomes in the reporting UI.
To create a goal using Databricks, follow the Data warehouse tracking flow described in Create a goal, and choose Databricks from the Datawarehouse integration dropdown. Step 1 then asks for these Databricks-specific fields:
  • Frequency: Select how often the task should run.
  • Catalog: Enter the name of the Databricks catalog that contains the schemas you wish Kameleoon to read from.
  • Schema: Enter the schema that contains your conversion data.
  • Query: Define the SQL query to retrieve the necessary data from Databricks.
Once Step 1 is complete, configure Step 2, Advanced settings, if needed, then save your goal. Data warehouse retention period: For Kameleoon to poll an event, it must remain accessible by your input query for at least 72 hours after the event occurred.

​
Query format

The query must adhere to a specific format:
Where visitor_id is the column representing your visitors’ unique ID, and conversion_timestamp is a column representing the exact time at which the conversion took place. In Databricks, the conversion_timestamp column must be a Timestamp type column. If you want to associate a revenue to each conversion, the query should adhere to an alternate format:
Where revenue is a column containing the revenue for each conversion. For more complex queries you can adhere to this format by formulating a sub-query as such:

​
How Kameleoon polls conversion data

Kameleoon polls the Databricks conversion data behind your goal every hour, appending a WITH clause to your query that filters by timestamps. Rather than reloading the full event history on every run, Kameleoon polls incrementally: each polling job tracks the last successful polling point and, on the next run, queries only the time window of new records since that point. Querying only that window keeps each poll fast, since Kameleoon never rescans data it has already imported. Kameleoon merges the collected conversions into your experiment results once per day, around noon CET. Until the next merge, the last 24 hours figures for a Databricks goal can show zero. Because polling always advances forward, Kameleoon can miss a record that lands in Databricks with a timestamp older than the last successful polling window. Late records typically arrive because your data pipeline takes time to write the conversion event, so the record reaches Databricks only after Kameleoon has already polled past the time window it belongs to. To prevent missed records, set the ingestion timeframe below to match your pipeline’s typical arrival lag.

​
Ingestion timeframe

By default, Kameleoon allows up to 1 hour between when a conversion event for your goal occurs and when it becomes available in your Databricks warehouse, and its polling window accounts for that delay automatically. If your pipeline takes longer than 1 hour to write events into Databricks, for example due to batch or nightly ETL jobs, those late-arriving records fall outside the polling window and get missed. Contact your Customer Success Manager to extend the delay so it matches your pipeline’s actual arrival lag.

​
Run your query before ingestion

Before saving your ingestion task, you can test your query directly in Kameleoon. Testing allows you to:
  • Verify the connection in real time.
  • Confirm your credentials and access rights are correct, which helps detect issues immediately, without having to wait for the first data import.
  • Validate your data’s structure and accessibility.
You can run a simple query to ensure Kameleoon can access the right databases and tables. You can also download a small sample of the dataset to confirm that the schema, permissions, and contents match your expectations.
Data warehouse goal configuration with Databricks selected, showing the Frequency, Catalog, Schema and Query fields and the Run Query button.

​
Using your Databricks goal

Now that you’ve created a goal, you can incorporate it into your Kameleoon campaigns. When setting up an experiment or personalization, you can select the goal, allowing you to track and analyze specific conversions in your Databricks database. Kameleoon merges these conversions with your experiment results once per day, around noon CET. To learn how to set up a goal in an experiment, see Set up a goal in a campaign.

​
Data retention

Conversion data imported from Databricks becomes visit data in Kameleoon and follows Kameleoon’s standard retention policy. For the retention period, see the Data retention policy.

​
Delete visitor data

You can remove Kameleoon’s copy of visitor data with the Data API endpoint POST /visit/forget, which requires authentication. For the request format, see Remove data for several visitors. The endpoint doesn’t delete tables in your Databricks workspace.