Aaron Ginn from StumbleUpon.com
October 10th, 2013 @ Miami Science Museum / Refresh Miami
Most important metrics?
Number of installations.
Revenues.
Types of metrics:
– Quantitatie vs -Qualitative.
Idea:
– On Wizard, let them know how to be in touch for Tech Support
via @frostwire and facebook.com/FrostWire Official.
Task for Marcelina to design that screen on the wizard.
QUANTITATIVE
The funnel (Pirate Metric)
Acquisition > Activations > Retention > Referral > Revenue.
QUALITATIVE
User Testing (In Progress)
Surveys
Study Groups
User Interviews
—
Stages for companies:
Early Stage involves discovery, validation. Then get Seeded out of pocket and angels.
Qualitative metrics help here.
Later Stage creation and building – Series A, profitability.
Quantitative metrics.
During creation, you want to start doing A/B to grow and see what works.
PRODUCT DASHBOARD
Engagement
User’s worth (LTV)
Growth rate
Qualities of a Good Metric
Measurements that are aggressively attacking your product.
Actionable – short feedback loop.
Product Centric – attack core assumptions of your product
Ratio – Compare against competitors performance, industry numbers.
Explainable – simpler the better
Timeless
(Checkout “Flurry” reports
Key Metrics per business
Marketplace: fulfillment rate, WOW (week over week) growth on both sides, % repeat purchase
Social Product: DAU/MAU, WOW daily registration, % ad impressions
B2B – sales funnel metrics, LTV
B2C – sales funnel metrics, LTV, WOW growth
LTV = churn * avg revenue per customer
Virality = measure the share cycle time is important. The faster, the deeper impact at sub-levels. Time between invites and activations.
Look for vitality equations.
Create your own AARRR
Acquisition – Number of new downloads, newsletter signups, twitter followers, Facebook page likes, community forum signups, Number of displays of our thank you page
Activation – Number of first uses. Displays of our promotions.
Retention – Number of sessions, Number of core actions, Newsletter unsubscribes, Facebook unlikes
Referral – Measure social activity, buzz, cult (pictures, fan art)
Revenue – Daily reports from all revenue channels (some hourly)
Qualitative Testing Schedule
One user a day. Watch them.
Two user sessions a month.
Three usability studies a month.
Important Factors in Quantitative Testing
is it Statistically Significant?
sample size.
sensitivity of the test.
duration of the test.
velocity. How many tests can you run in a period, pretty important. You need to test and get results as fast as possible, to learn more and more.
conversion.
Good habits
Test the hell out of MVPs.
Kill tech that slows you down.
Validate Instrumentation First. (make sure your test is running well)
The Goal is testing velocity.
Think about your next developer/employee.
Tools we use
Qualitative Testing Tools:
FT User Researcher
UserTesting.com
MouseOver Videos
User Studies
Quantitative Testing Tools
AB testing
Daily metrics emails
Product dashboards
Team metrics
Rapid A/B Testing Workflow
Test Iteration Implementations -> Rapid implementations.
Full Implementations -> Winner only (clean losers)
Code Fragmentation -> Clean unused test code.
Branch Based Workflow (git based)
Test Iteration Implementation -> Rapid
Full Impl -> Winner only
What will we do differently after this?
Added some new metrics I was missing thanks to this in relationship to acquisition and activations.
Will try new tools like UserTesting.com
Will look at what tech might be slowing us down.
Will not build more visualization tools of test data.