All Collections
A/B Testing
Rebuy's A/B Testing
Rebuy's A/B Testing

Make data-driven decisions for a customized customer experience, driving revenue growth, and exceeding customer expectations! πŸ“ˆπŸ›οΈ

Christian Sokolowski avatar
Written by Christian Sokolowski
Updated over a week ago

OVERVIEW

Rebuy's A/B Testing is a tool for optimizing conversion rates (CRO) that enables you to experiment with on-site widgets and the underlying data sources. It empowers them to make decisions based on data, ensuring a customized customer experience that drives scalable revenue growth and meets you customers' expectations for personalization.

We offer two distinct types of experimental tools: one that drives widgets and another that serves as a general tool, allowing you to manage elements within your storefront through CSS and JS. πŸ› οΈβœ¨


BENEFITS

Rebuy encourages you to utilize this tool to enhance you customer experience, boost average order value (AOV), increase revenue, extend customer lifetime value (LTV), and potentially reduce customer acquisition costs (CAC).

Those who are seeking to achieve the following goals can benefit from A/B Testing:

  • Enhancing product discovery by offering a wider range of relevant or unique products.

  • Expand and refine the Ideal Customer Profile (ICP).

  • Improving conversion rates and enhancing the customer experience by implementing personalized language within widgets.

  • Evaluating the impact of Rebuy by comparing widgets against the absence of widgets.

  • Increasing revenue and gaining insights into the customer journey by testing widgets in different locations to boost average order value (AOV), customer lifetime value (LTV), and identify areas where customers are adding products to their cart.

Want to learn more about Rebuy features? Book a demo with our team today!


FEATURE LIST

USER-FRIENDLY EXPERIENCE

The setup process for experiments is effortless, taking less than 5 minutes. Once a test is completed, the winning widget, determined based on the selected Experiment Goal during setup, will automatically be used for all incoming traffic. Accessing and reviewing reports is always convenient, directly from the A/B Testing Dashboard.


COMPREHENSIVE DASHBOARD

The A/B Testing dashboard provides a centralized view of ongoing, draft, and completed experiments, along with their respective metrics. From the dashboard, creating new experiments and getting started is simple. Completed tests can be accessed and reviewed at any time without the worry of them being automatically archived.


POWERFUL EXPERIMENTATION

Test up to 9 variations of widgets against a control widget. Variations can consist of different widgets, the same widgets with different data sources, or even no widget at all. Note: If "Do Not Show Anything" is selected, certain metrics may be unavailable (e.g., comparing conversion rates across variations requires a widget for conversions).


FLEXIBLE SCHEDULING

Schedule experiments to start and run for a specific duration or initiate an experiment immediately, allowing it to run indefinitely until manually ended. If there's a need to conclude a scheduled test earlier, it can be easily ended with a simple click.


LIMITATIONS & FAQ

What widgets are supported by A/B Testing?

Currently, you have the option to conduct A/B testing on all widget types except for the following:

  1. Third Party Widgets (Recharge, Tapcart, Malomo, Wonderment, etc.)

  2. Shopify Checkout Extensions

  3. Pre-purchase Cross-Sell

  4. Smart Links

  5. Klayvio Events

  6. Attentive Events

  7. Reorder / Reactivate landing pages

We are actively working on enhancing our capabilities for future iterations of A/B testing to accommodate a broader range of widget types. Keep an eye out for new releases with additional A/B testing capabilities!

Can I disable Attribution Tracking for the widgets in the A/B test?

When conducting an A/B test, it is necessary to track attributions in order to receive the maximum benefit from your test. Attribution tracking supports conversion optimization in a number of ways, such as helping you determine the effectiveness of different widgets, providing an understanding of user behavior on each page, and optimizing ad spend with the most profitable channels.

Because of the importance of attribution tracking for A/B tests, it is not recommended to disable the attribution tracking for widgets in your A/B test. Doing so will cause the test to fail, and will not provide the information you need from your test.

For more information on A/B testing with widgets, please see our user guide on A/B testing with widgets.


What are the Best Practices for A/B Testing Experiments?


Although A/B testing offers extensive customization options to meet your specific requirements, we have compiled a best practices guide to provide helpful tips and tricks for conducting a successful A/B test.


How many variations can I test against one another?

A/B Testing feature allows for up to 10 total variations (1 control and 9 variations).


Can I test different widget types against one another?

Yes you can test different widget types against one another.

NOTE: If a merchant installs the A/B placement widget ID in app blocks or theme code, it will not appear as a Cart Cross Sell if they later choose to add it in that capacity. Regular cross sells can be placed as app blocks and in the Smart Cart simultaneously, but the A/B testing placeholder widget cannot. To display the A/B testing placeholder widget in the cart, merchants must remove it from their theme.


How do I A/B test Post Purchase?

Currently Post Purchase A/B testing is housed within the Post Purchase flows. You can follow the Post Purchase Flow guide to set up your A/B testing experiment. A/B testing for post purchase is a bit different due to the nature of the widget type so it remains separate at this time.


How do I A/B test Smart Cart?


You have the option to test Smart Cart using the general A/B experiments feature within Rebuy. Alternatively, if you prefer a third-party solution, you can utilize Experiments.js, a JavaScript package developed by our team. With Experiments.js, you can choose your preferred A/B testing suite, whether it's Google Optimize, VWO, Optimizely, or even your own custom solution, providing you with flexibility in your testing approach.


Can I A/B Test Data Sources?

Yes, you can technically A/B test data sources. If you wish to do so, you will need to build out a duplicate widget and have each widget pointed to the respective data source. You can duplicate the widget by following these steps.

Duplicating Widget

  1. Navigate to the widgets page within Rebuy Admin.

  2. Locate the widget you wish to duplicate.

  3. Click on the vertical ellipsis menu and click duplicate.

Once you have the duplicate widget, you can adjust your data source rules to power each widget. From there you can use those two widgets to A/B test against each other, which will ultimately A/B test each data source.


Can I A/B test different placements of the widget on my page?

Yes, if you would like to A/B test the placement of a specific widget. If you wish to do so, you will need to build out a duplicate widget and have each widget pointed to the respective dynamic placement. You can duplicate the widget by following these steps.

Duplicating Widget

  1. Navigate to the widgets page within Rebuy Admin.

  2. Locate the widget you wish to duplicate.

  3. Click on the vertical ellipsis menu and click duplicate.

Setting up the widget's dynamic placement

First you will need to make sure you set up each widget as a dynamically placed widget which you can do by following the steps in the Dynamic Placement Selectors guide.

In this example, we will outline building two PDP widgets. One that will live above the add to cart button, and the other to be under the add to cart button.

  • Create widget A - place it above add to cart using the Dynamic Placement Selector.

  • Duplicate widget A - rename it widget B - place it under add the cart button using the Dynamic Placement Selector.

Once you have the two widgets built out with the dynamically placed selector configured, you can proceed to building out your A/B test.


How is the "winning" A/B widget test calculated?

The "winning" A/B test is chosen by the revenue or conversion (whichever you have chosen during the setup of the A/B test) of the page load results and not the specific performance of a widget. To learn more about the reporting structure for A/B testing, check out the Dashboards & Reporting article.


Does a Widget need to be in live mode during A/B testing?

Yes, the widget(s) should be in live mode for the A/B test. However, as a reminder the widgets are not to be installed themselves. You must use the placeholder ID for experiments. Installing the placeholder ID is simple and you can follow the steps in this guide.


How do I turn off A/B testing attributions (Rebuy-Experiment: true)

Regrettably, it's not possible to disable the A/B testing attributes, unlike the Smart Cart and Widgets, which offer that option. These attributes are essential for accurately tracking user sessions and obtaining reliable A/B testing outcomes.


How can I verify if my A/B test is running?

If you want to verify your widget A/B testing experiment is firing you can follow these steps:

  1. Inspect the page

  2. Click application

  3. Click the cookies

  4. Search _r_experiment

If the _r_experiment cookie is not present, then the A/B test did not load on the page you are on. This could be due to various reasons such as incorrect implementation or targeting settings. However, if the _r_experiment cookie is present, that means that an experiment did load in. You can verify which experiment by viewing the entire cookie to find the experiment ID. If you see multiple experiment cookies, this can happen if there are multiple A/B tests running on the same page.

Did this answer your question?