A/B Testing Experiments Dashboard
The A/B Testing Experiments dashboard shows how each of your experiments is performing, comparing your control against every variation you chose to test. It covers all Rebuy experiment types, including Widget, Smart Cart, Checkout Offer, General, Global Smart Flow, and Smart Search experiments. If you have not set up an experiment yet, follow the install instructions to get started.
The dashboard organizes your experiments across five tabs: Live for experiments running now, Scheduled for experiments set to start on a future date, Draft for experiments you created but have not launched, Completed for experiments that have ended (grouped by how strong the result is), and Archive for completed experiments you have set aside. This article explains each tab and every metric column you will see.
The metrics on this dashboard are measured at the page level, not the individual widget level, so they reflect how each variation affects the entire shopping experience. To review performance for a specific widget instead, open your Performance Report page from the Rebuy Admin Reports. Revenue and order figures are scoped to Rebuy-tracked orders, so they are built to compare variants against each other and will not match your Shopify store totals.
Live Experiments
Variation Names
These names are the aliases you chose during the installation process. If you did not set any specific alias for each of the widgets or control, then it will default to the widget name.
(Image above: Experiments dashboard row showing the Variation Names column, where each variant is labeled with the alias set during setup, for example, Control and Best Sellers, defaulting to the widget name when no alias was entered.)
Traffic
This is the allocated traffic you elected to assign to each variation type. The traffic percentages you have chosen for each variation type will be assigned randomly, similar to flipping a coin. As a result, the traffic distribution among the variations may not be exactly equal. This approach is implemented to prioritize the speedy loading of the widgets. By randomizing the traffic allocation, the widgets can be loaded quickly and efficiently, providing a seamless user experience.
(Image above: Experiments dashboard row showing the Traffic column, where each variant displays its allocated traffic percentage. For example, 50% to the control and 50% to the variation).
Visitors
The total number of visitors or views for each specific variation is tracked using cookies upon page load. Once a visitor sees a particular variation, they will consistently see the same variation throughout their browsing session, unless they clear their cookies or start a new tab in private browsing mode. This cookie-based tracking ensures that visitors have a consistent experience and allows for accurate measurement of the performance and impact of each variation based on the number of views received.
(Image above: Experiments dashboard row showing the Visitors column, where each variant displays its total visitor count from cookie-based page-load tracking. For example, 8,196 visitors for the control and 8,254 for the variation).
Conversions
The Conversions column in the Experiments dashboard shows the number of orders placed by shoppers who were shown a given variant, counted only when Rebuy's experiment tag was recorded at checkout. It is a raw order count, which is different from the Conversion column (the conversion rate percentage) shown elsewhere in the same table.
Because Rebuy counts only orders that carried the experiment tag at checkout, the Conversions number tracks a scoped subset of your store's orders and will be lower than your Shopify order count. This is by design: counting only tagged orders is what makes the comparison between variants valid. Use Conversions to compare one variant against another within the same experiment, not to reconcile against your Shopify store totals.
(Image above: Experiments dashboard variant rows with the Conversions column highlighted)
Conversion
The "Conversion Rate" option is used to determine the winning variation in an A/B test based on which variation, including the control, achieves the highest percentage of order conversions. In other words, it assesses the success of each variation by comparing the conversion rates, which represent the proportion of visitors who complete a desired action.
The conversion rate is based on page views of the widgets or controls. Not the individual widget conversion. If you want to see widget conversion rates, visit the performance reporting section. The "winning" widget is chosen based off of the page load conversion that is shown on the A/B testing dashboard and not the specific widget that made the most conversions during the A/B test.
(Image above: Experiments dashboard variant rows with the Conversion column highlighted, showing the dollar total and the per-visitor value beneath it.)
Total Revenue
The Total Revenue column in the Experiments dashboard shows the total store revenue from orders attributed to a given variant, including revenue beyond what Rebuy generated. It reflects the overall business impact of that variant, not only the revenue Rebuy drove. Beneath each figure, the dashboard also shows a per-visitor value, which is the variant's total revenue divided by its visitor count.
For example, a variant with $19,232.32 in Total Revenue across 8,196 visitors shows $2.35 per visitor ($19,232.32 ÷ 8,196 = $2.35).
(Image above: Experiments dashboard variant rows with the Total Revenue column highlighted, showing the dollar total and the per-visitor value beneath it.)
The way revenue is calculated differs depending on the type of experiment you're running:
Widget experiments display Rebuy-attributed revenue — the revenue from items added through Rebuy-powered recommendations during the experiment period.
Smart Cart experiments display total order revenue — the full cart value for sessions where the Smart Cart was active, not just Rebuy-attributed items.
This means the revenue figures across different experiment types are not directly comparable. If you're reviewing results from a widget test and a Smart Cart test side by side, keep in mind they are measuring different things.
Rebuy Revenue
The Rebuy Revenue column in the Experiments dashboard shows the revenue Rebuy generated for a given variant. This is the value attributed to Rebuy for the orders tracked in that variant, not the full value of every store order, so it is a subset of the variant's Total Revenue. Beneath each figure, the dashboard also shows a per-visitor value, which is the variant's Rebuy Revenue divided by its visitor count.
For example, a variant with $1,211.40 in Rebuy Revenue across 8,196 visitors shows $0.15 per visitor ($1,211.40 ÷ 8,196 = $0.15).
(Image above: Experiments dashboard variant rows with the Rebuy Revenue column highlighted, showing the dollar total and the per-visitor value beneath it.)
Rebuy AOV
The Rebuy AOV column in the Experiments dashboard shows the average order value Rebuy attributes to each variant, calculated as Rebuy Revenue ÷ Conversions. Rebuy Revenue is the value attributed to Rebuy for the orders tracked in that variant, not the full value of every store order, and Conversions is the count of those tracked orders.
For example, a variant with $1,524.28 in Rebuy Revenue across 98 conversions shows a Rebuy AOV of $15.55 ($1,524.28 ÷ 98 = $15.55).
Because Rebuy AOV is calculated from Rebuy-tracked orders only, it will differ from your Shopify store AOV. Use it to compare the average order value between variants in the same experiment, not as a store-wide benchmark.
(Image above: Experiments dashboard variant rows with the Rebuy AOV column highlighted, showing the info tooltip that reads "Rebuy AOV = Rebuy Revenue ÷ Conversions.")
Scheduled Experiments
The Scheduled tab of the Experiments dashboard lists experiments you have set to start automatically at a future date. A scheduled experiment has not started collecting data yet, so its card shows the variants, traffic split, and start and end dates you configured, but performance columns such as Conversions and Total Revenue show a dash until the experiment goes live.
Open the Scheduled tab from Experiments to review or change a test before its start date. From a scheduled experiment's actions menu (the three-dot icon) you can do the following:
Edit the experiment settings, variants, or schedule.
Preview a variant on your storefront (available for Widget, Smart Cart, General, and Global Smart Flow experiments).
Start Experiment to launch it immediately instead of waiting for the scheduled date.
Delete the experiment.
Only one Smart Cart, Global Smart Flow, or Smart Search experiment can run at a time. If you schedule one of these while another of the same type is already active, Rebuy warns you that the active experiment will automatically end when the scheduled one starts.
When the Scheduled tab is empty, it reads "No scheduled experiments" with the note "Schedule an experiment to start automatically at a future date."
(Image above: The Experiments dashboard with the Scheduled tab selected, showing a scheduled experiment card with its start and end dates and dashes in the performance columns.)
Draft Experiments
The Draft tab of the Experiments dashboard lists experiments you have created but not yet launched or scheduled. A draft is a work in progress: it has no start date and has collected no data, so its performance columns stay empty until you start it.
Open the Draft tab from Experiments to finish setting up a test. From a draft's actions menu (the three-dot icon) you can do the following:
Edit the draft to finish configuring its variants, traffic split, and goal.
Preview a variant on your storefront (available for Widget, Smart Cart, General, and Global Smart Flow experiments).
Start Experiment to launch it now, or open it to set a schedule.
Delete the draft.
When the Draft tab is empty, it reads "No draft experiments" with the note "Drafts are experiments you've created but haven't launched yet."
(Image above: The Experiments dashboard with the Draft tab selected, showing a draft experiment card with editable variants and no performance data.)
Completed Experiments
The Completed tab of the Experiments dashboard lists experiments that have ended. Rebuy groups each completed experiment by how strong its result is, so you can see at a glance which tests produced a winner and which need another run. Each experiment appears as a card you can expand to see the full variant comparison.
How are completed experiments grouped?
Rebuy sorts every completed experiment into one of four groups based on the winning variant's probability of being the best performer, shown in the Prob. Best column. The following table explains each group and what qualifies an experiment for it:
Group | What it means | Requirement |
Clear Winners | A variant is a statistically clear winner you can deploy with confidence. | The best variant has a 95% or higher probability of being the best performer. |
Strong Signals | A variant is very likely the best, but has not yet reached the clear-winner bar. | The best variant's probability is between 70% and 95%. |
Needs More Data | No variant has reached a confident result yet. | The best variant's probability is below 70%, or there is not enough data to calculate one. |
No Data Collected | The experiment recorded no usable results. | No variant received any visitors, or the experiment had no valid test variant. |
What does a completed experiment card show?
A completed experiment card summarizes the result before you expand it. Each card includes the following:
A result badge: "Clear Winner: [variant name]" for a clear winner, "Strong Signal: [variant name]" for a strong signal, "Not Enough Data", or "No Data".
A confidence badge for clear winners and strong signals, such as "95% Confident", showing the winning variant's probability of being the best performer.
A revenue advantage badge when the winner earned more than the control, such as "+$18,893 Revenue Advantage", showing how much more revenue the winning variant drove than the control.
A question and answer summary: the experiment name as the question, and a plain-language result such as "[Variant] increased conversion rate by +8% with 95% confidence."
A summary line showing the total visitors, the conversion-rate lift over the control, and how many days the experiment ran, such as "155,644 visitors · +8% lift · 19 days."
Select Show Details on a card to expand the full variant comparison table, which lists each variant's Conv. Rate, Conversions, Visitors, Total Revenue, Rebuy Revenue (hidden for Smart Cart experiments), Rebuy AOV, and Prob. Best. The winning variant's row is highlighted, and each metric shows its change compared with the control.
From a completed experiment you can select Archive to move it to the Archive tab, or use the actions menu to Delete it.
(Image above: A Clear Winners card on the Completed tab, expanded to show the Control and winning variant rows with conversion rate, revenue, and Prob. Best columns.)
Why does my completed experiment say "Not Enough Data"?
Your completed experiment says "Not Enough Data" because none of its variants reached a 70% probability of being the best performer, so Rebuy could not name a statistically significant winner. Rebuy shows the message "Not enough data has been collected to determine a statistically significant winner. Consider extending the test duration or increasing traffic." To get a conclusive result, run the experiment again for a longer period or send more traffic to it so each variant collects enough visitors and orders.
Archived Experiments
The Archive tab of the Experiments dashboard stores completed experiments you have moved out of the Completed tab to keep your active results focused. Archiving does not delete an experiment or its data; it moves the experiment into a separate, searchable list.
To archive an experiment, open the Completed tab and select Archive on any completed experiment card. To move an experiment back, open its actions menu on the Archive tab and select Unarchive, which returns it to the Completed tab.
The Archive tab displays experiments in a table with the following columns:
Column | What it shows |
Experiment | The experiment name and ID. Select a row to expand its full variant comparison. |
Type | The experiment type, such as Widget, Smart Cart, Checkout Offer, General, Smart Flow, or Smart Search. |
Signal | The result signal: Clear Winner, Strong Signal, Not Enough Data, or No Data. |
Archived | The date the experiment was archived. |
Above the table, you can narrow the list with the following controls:
A Search archived... box that matches the experiment name and type.
An All signals filter with the options Clear Winner, Strong Signal, Not Enough Data, and No Data.
An All types filter with the options Widget, Checkout Offer, Smart Cart, General, Smart Flow, and Smart Search.
A sort control with the options Date archived (newest), Date archived (oldest), Experiment name (A–Z), Experiment name (Z–A), and Experiment type.
Expand any row to see the same variant comparison table shown on completed experiment cards. From a row's actions menu you can Unarchive the experiment or Delete Permanently, which removes the experiment and its data and cannot be undone.
When the Archive tab is empty, it reads "No archived experiments" with the note "Archive experiments from the Completed tab to keep your view focused. Click 'Archive' on any completed experiment card to move it here."
(Image above: The Archive tab showing an archived Smart Cart experiment with a Not Enough Data signal, the search and filter controls above the table, and an expanded variant comparison.)
A/B Testing Experiments Dashboard & Reporting FAQ
Why is my Conversions number lower than my Shopify order count?
Your Conversions number is lower than your Shopify order count because Rebuy counts only the orders where its experiment tag was present at checkout, which is a subset of all your store orders. Shopify counts every order your store receives, while the Conversions column counts only orders attributed to shoppers who were shown that variant. The two numbers are measuring different things, so they are not expected to match. Compare Conversions across the variants in your experiment rather than against your Shopify dashboard.
What is the difference between Total Revenue and Rebuy Revenue in an experiment?
The difference between Total Revenue and Rebuy Revenue is what each column counts. Total Revenue is the full store revenue from orders attributed to a variant, so it reflects the variant's overall business impact. Rebuy Revenue is only the portion of that revenue attributed to Rebuy, so it is always a subset of Total Revenue. Use Total Revenue to gauge the variant's total effect on your store and Rebuy Revenue to isolate the revenue Rebuy specifically drove. Rebuy Revenue is also the value divided by Conversions to produce the Rebuy AOV column.
Why does Rebuy AOV differ from my Shopify store AOV?
Your Rebuy AOV differs from your Shopify store AOV because it is calculated only from Rebuy-tracked orders, not from every order your store receives. Rebuy AOV divides the Rebuy Revenue for a variant by that variant's Conversions count, so it reflects the average value of the scoped subset of orders Rebuy tracked. Your Shopify store AOV includes all orders across your store. Use Rebuy AOV to compare variants against each other within an experiment, not as a measure of your overall store performance.














