Rebuy Monetize powered by Fluent allows Shopify merchants to monetize their Thank You Page (TYP) or Order Status Page (OSP) with personalized, targeted Monetize. This document provides an overview of the A/B testing capabilities and limitations for this feature.
Step-by-Step Guide to A/B Testing Rebuy Monetize Offers
Step 1: Create Rebuy Monetize Offers
Before you begin your A/B test, you need to create Monetize offers in the Rebuy Admin.
Important Note: Rebuy Monetize offers should be created in the Rebuy Admin and NOT installed as an app block in Shopify before enabling an A/B test. This ensures that the Monetize widget works properly within the A/B testing framework.
Step 2: Access A/B Testing
Open the Rebuy app and go to the main menu.
Select A/B Testing.
Click Create New Experiment.
In the pop-up, enter a descriptive name for your experiment (for easy tracking).
Choose Checkout Offers as the Experiment Type.
Click Add Experiment to enter the experiment editor.
Step 3: Set Control and Variations
On the experiment settings page, select your control widget (the baseline for comparison).
Add up to 9 widget variations that you want to test. These could include different product recommendations or content blocks along with Monetize widgets. However, you can only include one Monetize widget in the A/B test as you are only allowed to create one at this time.
Set the display percentage for each widget (control and variations). This determines how often each variation will be shown to users during the experiment.
Important Note: You cannot currently A/B test two Monetize widgets against each other, as only one Monetize widget can be created at a time.
Step 4: Select the Experiment Goal
Scroll down to the "Experiment Goal" section.
Choose either "Revenue" (default) or "Conversion Rate" as the primary metric for determining the winner:
Revenue: The variation that generates the most revenue wins. This goal is ideal if you’re optimizing for total income.
Conversion Rate: The variation with the highest percentage of order conversions wins. This goal is recommended if you’re focusing on improving the number of completed purchases.
Step 5: Set the Start and End Time (Optional)
You can schedule the experiment to start and end at specific times or run it manually:
In the Test Duration section, set your Start Date and Start Time.
Set the End Date and End Time to automatically end the experiment at a specified time.
If you prefer to manually start and stop the experiment, skip this step.
Important Note: A/B tests cannot be scheduled to start more than 30 days in advance or end more than 30 days after the start date.
Step 6: Save the Experiment
Once all required elements have been configured, the “Save Experiment” button will become active. Click it to finalize your experiment setup.
Step 7: Get the Placeholder ID
After saving, your experiment will be in Draft mode.
Find the Placeholder ID on the right side of the editor.
Use this Placeholder ID to install the widget on the relevant page (Checkout, Thank You, or Order Status). This ensures that the A/B test runs correctly.
Important Note: Do not install the original widget IDs directly—only use the Placeholder ID to prevent errors.
Step 8: Install the Placeholder ID
Installing the Placeholder ID via Shopify's checkout editor
Open the checkout editor in your Shopify portal.
Add Rebuy Engine app block.
Add the Placeholder ID directly into the app block.
For more detailed instructions on installing via app blocks, refer to Rebuy’s app blocks help documentation.
Please note: If you install the placeholder ID in your checkout and your A/B test is not yet live, an error message will appear stating: Invalid Widget ID (incorrect type or possibly deleted): Verify your widgets before updating extension in the Checkout Editor
Once you start your A/B test and refresh the checkout editor page, the error message will disappear, and a new one will appear stating: Experiment: The selected variant will display here
Starting an Experiment
Option 1: Start from the Edit Screen
If you remain on the experiment’s edit screen after saving it, you can start the experiment immediately.
Locate the "Start Experiment" button at the top right-hand side of the experiment editor (in the main heading).
Click "Start Experiment." A pop-up confirmation will appear.
Click "Start Experiment" again to confirm, and your A/B test will go live.
Option 2: Start from the Dashboard
If you leave the edit screen, you can start the experiment from the Drafts tab on the A/B Testing dashboard.
Find the experiment you want to start and click the vertical ellipsis (three dots) next to the "Draft" label.
Select "Start Experiment" from the menu. A pop-up confirmation will appear.
Click "Start Experiment" again to confirm, and your A/B test will go live.
Ending Your Experiment
Scheduled End Time
If you configured a scheduled end time during setup, the experiment will automatically conclude based on that schedule.
Manually Ending the Experiment
If you wish to end the experiment early, go to the Active tab on the A/B Testing dashboard.
Click the vertical ellipsis (three dots) next to the active experiment.
Select "End Experiment" to immediately stop the test, or "Edit" to adjust the scheduled end time.
After the Experiment Ends
Once the experiment concludes, the results will be available under the Completed tab on the A/B Testing dashboard.
The winning widget configuration will automatically replace the placeholder in your store, making it permanently visible to customers without any further action required.
Monitoring Experiment Performance
Monitoring Active Experiments
After an experiment starts, you can view it under the Active tab on the A/B Testing page within Manage Your Experiments.
Use the vertical ellipsis menu to end the experiment immediately or edit it to modify the scheduled end time if desired.
Viewing Performance Metrics
Summary performance metrics for both active and completed experiments are visible within their respective tabs.
Metrics like revenue, conversion rate, and engagement help you assess how well each variation performs against the control.
Limitations of A/B Testing Rebuy Monetize Offers
Rebuy Monetize, powered by Fluent, allows Shopify merchants to display personalized offers on the Thank You and Order Status pages. However, there are some key limitations to keep in mind when using A/B testing for Monetize offers:
One Monetize Widget per Experiment:
You can only include one Monetize widget in an A/B test. Multiple Monetize widgets cannot be tested against each other at this time.Use Experiment Widget ID (Placeholder ID):
Always use the Experiment Widget ID (Placeholder ID) in Shopify, not the original Monetize widget ID. The Placeholder ID must be added to the Rebuy Engine app block in Shopify to avoid errors.Testing with "Do Not Show Anything":
You can A/B test Monetize offers against the option to Do Not Show Anything or other checkout extension widgets like product recommendations.
FAQs
Q: Can I A/B test multiple Monetize widgets against each other?
A: No, currently only one Monetize widget can be used in an A/B test at a time. You can test a Monetize widget against other widgets (e.g., product recommendations, content blocks), but not against multiple Monetize offers.
Q: Can I A/B test Monetize offers against other widgets or show nothing?
A: Yes, you can A/B test Monetize offers against “Do Not Show Anything” or other checkout extension widgets, like product recommendations or content blocks.