ابرووا
شروع رایگان
☰ فهرست مستندات · آزمایش A/B
به‌زودی. Abrova Push به‌زودی فعال می‌شود. این صفحه توضیح می‌دهد چطور کار خواهد کرد؛ هنوز قابل استفاده نیست.
این صفحه هنوز ترجمه نشده — نسخه‌ی انگلیسی نمایش داده می‌شود. English

A/B Testing

Test up to 10 notification variants with control groups, early stopping, statistical significance. Created in Abrova Console; mobile SDKs fetch variants and track events. Auto click tracking on mobile.

به‌روزرسانی: ۵ آبان ۱۴۰۵

Test different push notification variants to optimize engagement and conversion rates. Compare up to 10 variants with statistical significance analysis.

Included.

A/B testing, control groups, early stopping and the statistics are included with Abrova Push. Every variant you send counts toward your monthly messages like any other push: 10,000 messages a month are free, and the price above that is set when Abrova Push launches.

Overview#

AbrovaPush A/B testing lets you send different notification variants to different user segments and measure which performs best. Tests are created and run in Abrova Console.

Key Features

  • • Up to 10 variants per test
  • • Control groups (no message sent)
  • • Early stopping when significance reached
  • • Statistical confidence analysis
  • • Automatic click tracking on mobile
  • • Target by segment or percentage

Tracked Metrics

  • • Sent — Messages dispatched
  • • Delivered — Successfully received
  • • Opened — Notification viewed
  • • Clicked — User tapped notification
  • • Converted — Custom goal achieved

Creating Tests via Dashboard#

Create A/B tests in the AbrovaPush A/B Testing dashboard.

  1. Create New Test — Click "Create A/B Test" and enter test name and description
  2. Configure Variants — Add 2-10 variants with different titles, bodies, images, and action buttons. Set traffic percentage for each
  3. Set Target Audience — Choose "All users", a specific segment, or a percentage of your audience
  4. Configure Options — Enable control group, early stopping, select winning metric, set test duration
  5. Start Test — Review and start. Send notifications to begin collecting data

Control Groups#

A control group receives no notification, letting you measure the true impact of your messaging versus no messaging at all.

Example: If your control group has 5% conversion and Variant A has 12%, the notification drove a 7% lift — not just that 12% of users converted.

Early Stopping#

When enabled, the test automatically completes when statistical significance is reached, saving you from running tests longer than necessary.

OptionTypeRequiredDescription
Significance Threshold--Confidence level required (90%, 95%, or 99%)
Minimum Sample Size--Minimum deliveries per variant before checking significance (default 100)

SDK Integration — Auto Click Tracking#

Mobile SDKs support A/B testing with automatic click tracking. When users tap A/B test notifications, clicks are tracked without additional code.

Android (Kotlin)

kotlin
// Get variant for a test (with caching)
AbrovaPush.getABTestVariant(
    testId = "test_id",
    forceRefresh = false,
    onSuccess = { variant ->
        // Display content based on variant
        showBanner(
            title = variant.content?.title ?: "Default",
            backgroundColor = variant.content?.data?.get("backgroundColor")
        )

        // Track that variant was displayed
        AbrovaPush.trackABTestDisplay(variant)
    },
    onError = { error -> /* handle */ }
)

// Track click manually (automatic for push notifications)
AbrovaPush.trackABTestClicked(testId, variantId)

// Track conversion
AbrovaPush.trackABTestConverted(testId, variantId)

iOS (Swift)

swift
// Get variant for a test
AbrovaPush.shared.getABTestVariant(testId: "test_id") { result in
    switch result {
    case .success(let variant):
        showBanner(
            title: variant.content?.title ?? "Default",
            color: variant.content?.data?["backgroundColor"]?.value as? String
        )
        AbrovaPush.shared.trackABTestDisplay(variant: variant)

    case .failure(let error):
        // handle
    }
}

// Track click manually (automatic for push notifications)
AbrovaPush.shared.trackABTestClicked(testId: testId, variantId: variantId)
// Note: the iOS SDK has no conversion-tracking method yet

// Handle notification tap (automatic click tracking)
func userNotificationCenter(_ center: UNUserNotificationCenter,
                           didReceive response: UNNotificationResponse,
                           withCompletionHandler completionHandler: @escaping () -> Void) {
    // This automatically tracks A/B test clicks
    AbrovaPush.shared.handleNotificationResponse(
        userInfo: response.notification.request.content.userInfo,
        actionIdentifier: response.actionIdentifier
    )
    completionHandler()
}

Flutter

dart
final variant = await AbrovaPush.getABTestVariant('test_id');
if (variant != null) {
  showBanner(
    title: variant.content?.title ?? 'Default',
    color: variant.content?.data?['backgroundColor'],
  );

  await AbrovaPush.trackABTestDisplay(variant);
}

await AbrovaPush.trackABTestClicked(testId, variantId);
// Note: the Flutter SDK has no conversion-tracking method yet

React Native

typescript
const variant = await AbrovaPush.getABTestVariant('test_id');
if (variant) {
  showBanner({
    title: variant.content?.title ?? 'Default',
    color: variant.content?.data?.backgroundColor,
  });

  await AbrovaPush.trackABTestDisplay(variant);
}

await AbrovaPush.trackABTestClicked(testId, variantId);
// Note: trackABTestConverted is declared in the React Native package but its
// native method is missing, so calling it fails. Conversions are Android-only today.

Automatic Click Tracking#

When you send A/B test notifications from the dashboard, the SDK automatically tracks clicks when users tap on the notification or action buttons. No additional code required.

How it works: A/B test notifications include abTestId and variantId in the payload. When the user taps the notification, the SDK extracts these values and sends a click event to the server automatically.

Winner Metrics#

OptionTypeRequiredDescription
delivery_ratedelivered / sent-Testing delivery optimization
open_rateopened / delivered-Testing notification appeal
click_rateclicked / delivered-Testing engagement (recommended)
conversion_rateconverted / sent-Testing business outcomes

Best Practices#

Test One Variable at a Time

Change only one element (title, body, or image) per test to understand what drives the improvement

Use Sufficient Sample Size

Wait for at least 100-1,000 sends per variant before drawing conclusions

Include a Control Group

10-20% control group helps measure true lift vs. no messaging at all

Enable Early Stopping

Save time by automatically completing tests when statistical significance is reached

Wait for Statistical Significance

95%+ confidence before selecting a winner. The dashboard shows real-time confidence levels

قبلی
وب‌هوکبعدی
تیم‌ها
این صفحه مفید بود؟
گزارش مشکل