A/B Subject Line Testing

Last updated: 2026-05-28

For every reminder step, Duefy lets you set a second subject line that runs head-to-head against the first. After enough sends, the statistical winner gets promoted automatically.

Why this is worth setting up

The subject line is the single highest-leverage piece of the whole email — it decides whether the recipient opens at all. Industry data: a good subject beats a bad one by 2-3x on open rate. Without A/B testing, you're guessing.

How to set it up

Go to Settings → Reminder schedule.
Expand any step.
Below the main Subject field, click Add A/B variant.
Enter your challenger subject.
Save.

That's it. Duefy now sends 50/50 between A (the original) and B (the challenger). The pick is per-send, not per-client, so over time both variants get balanced exposure.

How the winner is decided

Behind the scenes, a daily cron (promoteAbWinners.php) checks each step that has an A/B variant:

Minimum sample: 30 sends per variant.
Significance: chi-square test on open rate, p < 0.05 (statistic > 3.841).

When a variant clears both bars, the winner's subject is copied to the primary slot and the B field is cleared. The dashboard then prompts you to enter a new challenger.

If neither variant clears the bar after 30 sends, the test keeps running. If both perform similarly (no winner), the test continues until you manually change one.

What gets measured

The decision is based on open rate (the strongest signal that the subject worked). Click rate + paid-after-click are also tracked in the "What's working" dashboard so you can spot variants that "win" opens but lose conversions (a classic clickbait failure mode).

Tips for good A/B subjects

Test one thing at a time. "Quick check on INV-123" vs "Quick check on invoice INV-123" tests one word. "Quick check on INV-123" vs "URGENT: PAYMENT OVERDUE FOR INV-123" tests tone, urgency, casing, capitalization — you'll know it won but won't know why.
Try formats, not just words: question vs. statement, name vs. no name, emoji vs. no emoji.
Don't bench-mark against industry norms, A/B against your last winner. What works for invoice reminders to dentists may not for B2B SaaS bills.

"What's working" dashboard

Settings → Reminder schedule shows per-step open / click / paid rates with green/red indicators vs. industry benchmarks (open 32% / click 8% / paid 6% for invoice reminders). Use it to spot underperforming steps to A/B first.