Understanding visit count mismatches in capped experiments

When you use display limits (capping) in your feature flags or experiments, the total number of visits recorded for your variations often appears uneven compared to the Original version. If you notice that the Original version has significantly more visits than the Variation, you can expect this behavior, which doesn’t indicate a failure in your experiment setup, such as a Sample Ratio Mismatch (SRM). Use this guide to understand exactly why capping creates this difference in visit counts and why you should focus on the unique visitor count instead.

​

The effect of display limits on experiment data

In advanced setups, a display limit is often achieved by dynamically adding a user to an exclusion segment after they have seen the variation once, which is commonly done by setting a custom data or local storage value for the visitor. You can also configure display limits in the Code editor for experiments and set similar capping limits for Personalizations.

​

Example: Pop-up display limits and visit counts

Suppose you configure an experiment where the Variation displays a marketing pop-up, but the Original doesn’t. To prevent annoying users, you use a display limit to show the pop-up only once per month. This configuration often causes a discrepancy in the total number of visits (sessions) recorded, even though the number of unique visitors remains balanced.

​

Original: No display limit triggers

When Kameleoon allocates a visitor to the Original version:

• The pop-up never displays.
• Because the pop-up never displays, the once-per-month limit never activates.
• The system tracks this visitor on every subsequent page view or return visit.

Result: The Original records a high number of total visits because nothing restricts the tracking.

​

Variation: Display limit applies

When Kameleoon allocates a visitor to the Variation:

• On the first visit, the visitor sees the pop-up.
• The system records that the visitor reached the display limit (1/1).
• On the next visit, the display limit prevents the pop-up from showing. Depending on your configuration, the system may stop counting these subsequent visits as part of the active experiment exposure.

​

The visit count disparity

This scenario creates a balanced split at the unique visitor level, but a significant imbalance at the total visit level.

• Original: One visitor might generate 10 visits.
• Variation: One visitor might generate only one tracked visit.

​

Why capped visit counts aren’t a sample ratio mismatch

It’s common to suspect a setup error when you see a large difference in visit counts. However, if the unique visitor count is equal, the difference isn’t a Sample Ratio Mismatch (SRM).

• What’s SRM? SRM is a statistical error where the initial allocation of unique visitors doesn’t match your configured split (50/50, for example). SRM usually indicates a tracking or technical issue.
• What’s happening here? The display limit is working as intended. It caps the exposure of the Variation, which naturally reduces the total visit count for that variation. The Original remains uncapped because it has no element to limit.

Always confirm your experiment split by looking at the number of unique visitors. If the unique visitor counts match your configured split, your experiment is running correctly.

​

FAQ