A/B Testing with Rebuy is an easy and powerful way to test different versions of your store to optimize for engagement and revenue metrics. Setting up experiments is a breeze, requiring under 5 minutes. After completing a test, the victorious widget, determined according to the Experiment Goal chosen during setup, will be automatically employed for all incoming traffic. Conveniently, reports can be accessed and reviewed directly from the A/B Testing Dashboard.

Before setting up an experiment, create at least 2 Rebuy widgets (or 1 if testing a "no widget" scenario). If you don't already have widgets built, then check out our documentation on how to get started with the various widgets that we offer.

Step 1: Navigate to "A/B Testing" from the main left-hand Rebuy menu.

Step 2: Click the “Create New Experiment” button.

Step 3: In the "New A/B Test" modal that opens, enter a name for your experiment in the "Name" box to help you keep track of it. Choose "Widgets" as the "Experiment Type" and click "Add Experiment" to proceed to the experiment editor.

Step 4: After completing the initial steps, users will be directed to the Experiment settings page. In the "Select Control and Variations" sections, choose your control widget and a maximum of 9 widget variations to experiment with. You must select at least 1 variation to experiment against. Then set the percentage that the control widget and each variation of the other widgets should be shown.

Step 5: Select the Experiment Goal (Revenue or Conversion Rate). Scroll down to the "Experiment Goal" section. The selected goal will determine the criteria used to determine the winner of your experiment.

The default option is "Revenue," where the control or variation that generates the most revenue will win. The "Conversion Rate" option determines the winner based on which variation (or control) that leads to the highest percentage of order conversions.

Step 6: (Optional) To set a specific test duration, navigate to the "Test Duration" section. If you prefer manual control over the experiment's start and end, there's no need to set a specific schedule. However, if you want an automated start and end, utilize the "Start Date" and "Start Time" selectors to initiate your experiment in the future. Similarly, use the "End Date" and "End Time" selectors to specify the desired end time (which must be after the scheduled start).

Remember, you can add or modify the scheduled start or end at any time before the experiment begins or concludes, providing flexibility in your testing approach. ⏰✨

Step 7: Once all required elements have been completed, the "Save Experiment" button will be highlighted.

Step 8: Your experiment is now in the "Draft" phase. Before starting the experiment, you will need to install a widget placeholder in your store and select a control and at least 1 variant.

On the right side of the editor, you'll see a "Placeholder ID" value which will be the widget ID that must be used when installing a placeholder widget on the desired page of your store. Make sure to use this "Placeholder ID" as the widget ID and avoid directly installing one of the actual widgets being tested in your experiment. This widget will not be visible to anyone visiting your store until the experiment is started.

If you are using app blocks, you can add the placeholder ID directly into the app block where you would normally place your widget IDs. Follow the instructions in the app blocks help doc.

Depending on the type(s) of widgets you are A/B testing, you have the option to directly install this code into different files: theme.liquid, checkout.liquid, additional script files, or via the Smart Cart Settings. The specific placement of the code depends on the widget type being tested. For instance, if you are testing a widget for the home page, you would insert the code into the theme.liquid file. Similarly, if you are testing a widget for the checkout page, the code would go into the checkout.liquid file. If you are testing a widget for the thank you page, you would add the code to the additional scripts section in the Shopify admin settings. Please refer to the instructions for each widget to determine the exact location for inserting this code.

If you are manually inserting this code into your theme.liquid files, you can access your Shopify theme files by going to your online store settings within the Shopify admin portal, clicking the horizontal ellipsis, and clicking "edit code." Once there, you can search for "checkout" and you will find your file.

If you are manually inserting this code into your checkout.liquid files, you can access your Shopify theme files by going to your online store settings within the Shopify admin portal, clicking the horizontal ellipsis, and clicking "edit code." Once there, you can search for "checkout" and you will find your file.

The process of installing a placeholder ID for a Smart Cart Cross Sell differs from installing app blocks or manually adding code to specific pages such as the home page, product pages, and checkouts. Cross Sell Widgets can be installed directly within the Smart Cart settings.

Before proceeding with the installation process, please ensure that you have already created your cross sell widgets and set up your experiment, saving it in draft mode (see below for save instructions). Once these steps are complete, follow the instructions below to toggle the placeholder widget in the smart cart on.

If you saved and continued to stay on the edit A/B test screen, you can start the experiment immediately from that page. Find the "Start Experiment" button near the top right-hand side of the experiment editor (in the main heading) and click "start experiment." A pop-up will present itself for you to confirm. Select "Start Experiment" once again and your A/B test will be live.

If you saved and left the edit screen, you can start the experiment immediately from the drafts tab on the A/B testing dashboard. Find the ellipsis menu next to the word "draft" on the experiment you are interested in. From there, click "Start Experiment" button. A pop-up will present itself for you to confirm. Select "Start Experiment" once again and your A/B test will be live.

If you have a scheduled end time for your experiment, then it will end according to your previously configured settings. Using the vertical ellipsis menu, you can immediately end an experiment or edit it to schedule an end time if desired. Once your experiment has concluded, the results will be available under the "completed" tab on the A/B testing dashboard. The experiment that "won" the test will be live.

Once the experiment has started, you can find it in the "Active" table on the "A/B Testing" page under "Manage Your Experiments." Using the vertical ellipsis menu, you can immediately end an experiment or edit it to schedule an end time if desired.

Performance metrics are visible in the summary blocks of experiments under the "Active" and "Completed" tabs. After an experiment ends, the embedded placeholder element in your store will automatically transform to match the configuration of the winning widget. It will be permanently visible to customers without any further action.