Learn

Learning Loop

Decisions without review are guesses without grading. Every reviewed outcome makes the next recommendation sharper.

Decisions on file
3
Reviewed
2
1 still open
Prediction hit rate
50%
Calibration improving
Lessons captured
4
Feeding future prep

Decision ledger

  • Priya (PM) · over 56 years ago
    Prediction met

    Move the upgrade prompt out of the editor surface

    Prediction

    Removing the inline upgrade prompt will lift editor session length 8–12% without hurting paid conversion.

    Expected outcome

    Session length +10%, paid conversion within ±5% of baseline.

    Baseline (min)
    6.2
    Target
    6.8
    Actual
    7.07
    Actual outcome

    Session length lifted 14%. Paid conversion dropped 6% — within the tolerance band but worth a follow-up test on a contextual paywall.

    Lessons
    • Friction-removal wins were under-predicted; we now baseline editor changes against +12%, not +8%.
    • Conversion impact lagged 10 days — read paywall changes on a 14-day window, not 7.
  • Marcus (PM) · over 56 years ago
    Prediction missed

    Add a 'What changed' digest to the dashboard

    Prediction

    A daily digest of changes will become the #2 entry point within 30 days and lift weekly active days by 0.4.

    Expected outcome

    WAD +0.4, digest is #2 entry point.

    Baseline (days)
    3.1
    Target
    3.5
    Actual
    3.3
    Actual outcome

    Digest is the #1 entry point. WAD moved +0.2 — half the prediction. Heavy users loved it; new users ignored it.

    Lessons
    • Cohort matters more than aggregate — split predictions by tenure next time.
    • Entry-point dominance does not imply engagement lift.
  • Priya (PM) · over 56 years ago06Running
    Awaiting review

    Soft-skip SSO in onboarding (50% rollout)

    Prediction

    Decoupling SSO from invites will lift Day-1 activation by 6–9 points without raising security incidents.

    Expected outcome

    Day-1 activation back to 45%+ on the treatment cohort within 14 days.

    Baseline (%)
    38
    Target
    45
    Actual
    —