Showing posts with label Jason Cohen. Show all posts
Showing posts with label Jason Cohen. Show all posts
Wednesday, 22 January 2014
Why you must watch this video before doing A/B Tests
When it comes to A/B testing, there is a lot of content on the interweb. Like everything else about 90% of this is just total shit, from people that don't really know what they are doing, and totally miss the point.
If you like Jason's talk then you can find more at his blog:
http://blog.asmartbear.com/
Twitter: @asmartbear
Read More
Step in Jason Cohen!
Jason does a brilliant job of describing why so many business get it wrong when it comes to A/B split testing (if you don't know what A/B testing is check out this article).
If you are doing A/B testing, and you do only one thing today, watch this video!!! Yes it is almost 1 hour long. However, 1 hour is less time than 1 year, which is the time you will probably waste doing tests, especially if you are new to this stuff.
After watching it clock-off work, go home, put on your slippers and get a cup of hot chocolate... Hell, put on a Michael Buble CD and relax. You have just saved yourself about 12 months of pointless work.
Here is a quick summary of probably one of the most informative talks on the internet. To be fair nothing Jason says is new, its just simple statistical analysis. However, too many "growth-hacking" hipsters forget to take Data Science 101 and so they are miss something important in their mysterious and elusive hack-kit.
1. Have a testable hypothesis,
2. Keep A/B tests simple,
3. Avoid 47 Shades of Blue - see point 1.
4. Ensure you have a decent enough sample size - no 100 visitors is not enough.
5. Have a high "confidence" level. 95% confidence is acceptable, anything less is not.
6. Small gains (1% - 15 %) are not enough to justify change especially if you are a low volume (< 1million visits/month) /a start up. Look for changes that double something or increase by 1/3 etc.
7. Check for statistical significance using this method.
8. Run the test for an appropriate amount of time.
9. Find 1 - 3 business critical "metrics" to measure and focus on them. Jason gives a broken down example.
Watch the video here:
If you like Jason's talk then you can find more at his blog:
http://blog.asmartbear.com/
Twitter: @asmartbear
Subscribe to:
Posts (Atom)
