Systematic A/B Testing Framework for Email Marketing

Create a complete A/B testing framework for email marketing for [COMPANY NAME], operating in the [INDUSTRY] sector, with [NUMBER] subscribers in the database.

Context:
- Platform: [PLATFORM: Mailchimp/ActiveCampaign/HubSpot/Klaviyo/SendGrid]
- Emails sent per month: [NUMBER]
- Current open rate: [PERCENTAGE]%
- Current click rate: [PERCENTAGE]%
- Primary goal: [GOAL: opens/clicks/conversions/revenue]

**1) Statistical fundamentals (simplified):**
- Minimum sample size per variant for [NUMBER] subscribers
  - Calculator: What % of the list to test before sending to the rest
  - Confidence level: 95% (standard)
  - Minimum detectable effect: 10% (minimum significant difference)
- Minimum test duration before declaring a winner
- When NOT to test (list too small, urgency situations)

**2) Prioritized test backlog (12-week roadmap):**

Organize by potential impact:

**Weeks 1-4: Subject lines (highest impact on opens)**
- Test 1: Length (short <30 chars vs. long 50+ chars)
- Test 2: With emoji vs. without emoji
- Test 3: Question vs. statement
- Test 4: With [NAME] personalization vs. no personalization

**Weeks 5-8: Content and layout (impact on clicks)**
- Test 5: 1 CTA vs. multiple CTAs
- Test 6: Hero image vs. plain text at top
- Test 7: Short email (150 words) vs. long email (400+ words)
- Test 8: CTA button: Color A vs. Color B / Text A vs. Text B

**Weeks 9-12: Timing and frequency (impact on engagement)**
- Test 9: Tuesday vs. Thursday send
- Test 10: Morning (9 AM) vs. afternoon (2 PM) vs. evening (7 PM)
- Test 11: 1 email/week vs. 2 emails/week
- Test 12: Sender (brand name vs. personal name)

**3) Test documentation template:**
```
Test #: [NUMBER]
Date: [DATE]
Hypothesis: [If we do X, we expect Y because Z]
Variable tested: [ONE SINGLE variable]
Variant A (control): [DESCRIPTION]
Variant B (test): [DESCRIPTION]
Primary metric: [opens/clicks/conversions]
Secondary metric: [OTHER]
Sample size: [NUMBER per variant]
Duration: [HOURS/DAYS]
Result: A=[X]% vs B=[Y]% | Difference: [Z]% | Confidence: [W]%
Winner: [A/B/INCONCLUSIVE]
Learning: [INSIGHT]
Action: [What changes going forward]
```

**4) Common mistakes to avoid:**
- Testing multiple variables at the same time
- Declaring a winner too early (<24h or <1000 sends/variant)
- Not documenting results
- Testing the same thing repeatedly
- Ignoring context (seasonality, day of week)

**5) Cumulative learnings dashboard:**
- Knowledge base format: What works vs. What doesn't work
- Updated after each test
- Shareable with the team
- Patterns identified over time