Data warehouse integrations are available as a premium add-on for the Web Experimentation and Feature Experimentation module. Contact your Customer Success Manager for more information.
visitor_id and a conversion_timestamp column, and the data must stay queryable for at least 72 hours after each event.
Before you create the goal, enable the BigQuery integration for your project.
Create a BigQuery goal
To create a goal that reads conversions from BigQuery, follow the Data warehouse tracking flow in Create a goal and choose BigQuery from the Datawarehouse integration dropdown. Step 1 then asks for these BigQuery-specific fields:- Frequency: Define how often you want Kameleoon to update the goal data.
- BigQuery project ID: Enter the project ID of your BigQuery project.
- Region: Select the appropriate region from the provided list.
- Query: Define the SQL query to retrieve the necessary data from BigQuery.
Query format
The query must adhere to a specific format:SELECT visitor_id, conversion_timestamp FROM your_events_table
Where visitor_id is the column representing the unique ID of your visitors, and conversion_timestamp is a column that represents the exact time at which the conversion took place. In BigQuery, the conversion_timestamp column must be a Timestamp type column.
If you want to associate revenue with each conversion, the query must use this alternate format:
SELECT visitor_id, conversion_timestamp, revenue FROM your_events_table
Where revenue is a column containing the revenue for each conversion.
For more complex queries, you can keep this format by wrapping your original query in a sub-query:
How Kameleoon polls conversion data
Kameleoon polls the BigQuery conversion data behind your goal every hour, appending aWITH clause to your query that filters by timestamps. Rather than reloading the full event history on every run, Kameleoon polls incrementally: each polling job tracks the last successful polling point and, on the next run, queries only the time window of new records since that point. Querying only that window keeps each poll fast, since Kameleoon never rescans data it has already imported. Although Kameleoon collects conversions hourly, it only merges them into your experiment results once per day.
Because polling always advances forward, Kameleoon can miss a record that lands in BigQuery with a timestamp older than the last successful polling window. Late records typically arrive because your data pipeline takes time to write the conversion event, so the record reaches BigQuery only after Kameleoon has already polled past the time window it belongs to. To prevent missed records, ask your Customer Success Manager to extend the ingestion timeframe to match your pipeline’s typical arrival lag.
Kameleoon imports a late record only while its timestamp falls in a window it hasn’t polled yet. Once the polling point passes a full day, Kameleoon doesn’t reread that day, so Kameleoon doesn’t import a late or corrected row for that day.
Ingestion timeframe
By default, Kameleoon allows up to 1 hour between when a conversion event for your goal occurs and when it becomes available in your BigQuery warehouse, and its polling window accounts for that delay automatically. If your pipeline takes longer than 1 hour to write events into BigQuery, for example due to batch or nightly ETL jobs, those late-arriving records fall outside the polling window and get missed. Contact your Customer Success Manager to extend the delay so it matches your pipeline’s actual arrival lag.Failed polls and schema changes
A failed poll leaves the polling point where it was. The polling point is the time up to which Kameleoon has already imported records, so the next scheduled poll starts from the same point and retries the same window. Kameleoon imports the records in that window once the query runs again. A schema change affects results in one of two ways, depending on what changed:- A column that your query expects is missing or renamed: The poll fails and Kameleoon logs the failure. The polling point doesn’t advance, so later runs retry the window until the query matches the table again.
- A column keeps its name but its meaning changes: The poll succeeds, and the polling point advances. Kameleoon doesn’t check that a same-named column still holds the visitor identifier, timestamp, or revenue that you expect, so results can be wrong without an error.
Visitor identifier
Kameleoon uses thevisitor_id value to identify which visitor receives credit for each conversion, so the value must match an identifier Kameleoon already has for that visitor. By default, the value is the Kameleoon visitor code. If you set up cross-device history reconciliation, Kameleoon matches visitor_id against the custom data you flagged as the unique identifier for cross-device history reconciliation instead of the visitor code.
Run your query before ingestion
Before saving your goal, you can test your query directly in Kameleoon. Testing allows you to:- Verify the connection in real time.
- Confirm your credentials and access rights are correct, which helps detect issues immediately, without having to wait for the first data import.
- Validate your data’s structure and accessibility.
Using your BigQuery goal
With the goal created, you can incorporate it into your Kameleoon campaigns. When you set up an experiment or personalization, you can select this goal in the Goals section of the finalization page. The goal then lets you track and analyze specific conversions from your BigQuery database. To learn how to set up a goal in an experiment, see Set up a goal in a campaign.Data retention
Conversion data imported from BigQuery becomes visit data in Kameleoon and follows Kameleoon’s standard retention policy. For the retention period, see the Data retention policy.Delete visitor data
You can remove Kameleoon’s copy of visitor data with the Data API endpointPOST /visit/forget, which requires authentication. For the request format, see Remove data for several visitors. The endpoint doesn’t delete tables in your BigQuery project.