Documentation Index
Fetch the complete documentation index at: https://docs.kameleoon.com/llms.txt
Use this file to discover all available pages before exploring further.
About the Bayesian method
The Bayesian approach considers the parameters you want to estimate as unknown constants. Kameleoon models each parameter as a random variable. The method takes its name from Bayes’ rule and enables you to compute quantities unavailable in the Frequentist framework. Bayes’ rule combines experiment results with any available prior information; however, A/B testing uses non-informative priors. By choosing the a priori distribution, you can leverage the posterior distribution to compute values such as the probability that a variant improves upon the original.Access Bayesian results
When you click Results on the dashboard, Kameleoon displays the classic Results page by default. To access results generated by Bayesian statistics:- Click the Actions menu at the top right of the page.
- Select Enable Bayesian.
You cannot access the Bayesian results page if you divert 100% of traffic to the original or if the experiment has zero visitors.
The Bayesian results page
The structure of the Bayesian reporting page is similar to the classic results page. However, some elements differ:- New indicators appear, such as the probability of beating the original variation and result reliability according to Bayes.
- Several graphs do not appear on the page, and the page displays only the conversion rate.
Definitions
Probability to beat the original
This value represents the probability that a variation will outperform the original page with a higher conversion rate for a given goal. If you allocate 0% of traffic to the original, the variations receive 100% of the traffic and do not compete with the original. In this case, the indicator becomes the Probability of being the winning variation.Reliability of results according to Bayes
This value represents the confidence rate of the results. Kameleoon calculates this rate on a three-level scale. The Reliability column of the results table includes a legend for easy interpretation. Results are fully reliable when the three boxes are full, indicating the reliability rate has stabilized over time. Avoid analyzing results before they reach a sufficient reliability rate to prevent trend reversals.
Bayesian continuous metrics
When you hover over non-binomial metrics (such as Revenue per visit/visitor or Average cart value), an overlay appears.
- Improvement rate: This value indicates the rate of improvement for the metric. It appears in green if positive and red if negative, alongside a credible interval.
- Probability to win over reference: This value shows the probability that the variation will outperform the control variation. It appears in green if it is higher than the reliability threshold set in the configuration; otherwise, it appears in red.
- Credible Interval Table: The min and max values define the bounds of the credible interval around the improvement rate. There is a 95% chance for the improvement rate to fall between these values. The values appear in green if positive and red if negative.