Website Testing Strategy – 6 Things to Consider

So you’re thinking about or have a new website. Now, for a really effective website,  you need a website testing strategy.

Awesome!

So many decisions made. Most of the time you were guessing, or said more charitably, using your instincts.

Well, there is an alternative.

A website testing strategy lets you see what the best choices are. Testing leads to constant improvement, which leads to a more effective website.

But testing is hard, you need to know math and stuff.

Well not really. Common sense will get you most of the way.

Here are 6 things you should consider when developing a website testing strategy. Use them as guides when testing your website.

1) Test for the right reason.

Don’t use testing as a surrogate for backbone:

“I’m not sure which direction is best so let’s test them both.”

This isn’t a good reason to test. First, it suggests you have no opinion. If you don’t have an opinion, then marketing isn’t your field. Testing is often used to avoid conflict. If you can’t handle conflict, then marketing isn’t your field. Seeing a pattern here. Get some backbone and make a decision.

2) Test to make it better.

If you have a successful website don’t succumb to boardroom boredom, don’t stop using what’s working. If it’s not broken, don’t fix it. I can assure you, your customers are not as bored with the website as you are.

But you should always try to make it better. You lowered the bounce rate. That’s fantastic. How low can you go, or alternatively, how low is too low?

Beat the control should be a key part of your website testing strategy.

3) Test big things.

Test things that matter, that make a difference. Test big things. Blue versus green isn’t a big thing. Don’t test this, just decide which is best. See “test for the right reasons” above. A new Call to Action or landing page design, these are big things, so is a new offer or a new form. Long copy versus short copy? This is a big thing.

Test big things like these to make your website even better.

4) Test things in isolation.

Be sure to isolate the element you are testing or, said another way, don’t test multiple things simultaneously. Unless you are a very sophisticated mathematician who thoroughly understands multi-variant testing, and test design, you won’t know which element caused the success or failure of the test.

Pick one big thing and test it.

 5) Consider the sample size.

Don’t test a larger sample than you need. Determine what sample you need for statistical validity. Work back up the response waterfall to determine the sample size. For example, if you need 200 sales to have a statistically valid comparison then apply your conversion rate and response rate to 200 to determine how many need to be in the test cell. If your test cell is too large you are putting valuable marketing resources at risk needlessly. If it’s too small you will be making decisions based on sketchy data. Neither outcome gets you where you need to be.

There are many tools available to help you execute your website testing strategy, you don’t need to know the math. The folks at Optimizely have a handy calculator: A/B Test Sample Size Calculator. Visual Website Optimizer also has a tool: A/B Split Test Significance Calculator. As you can tell from the very similar names, they do pretty much the same thing.

6) Test what you can afford.

When you test, be sure what you test will scale. Don’t test an offer you can’t afford to roll out, or a media channel you can’t afford to use. Guess what, it might work, but so what? If you can’t afford to use it for the full effort, don’t test it.

Conclusion – Website Testing Strategy

So have at it, test your website for fun and profit. Test for the right reasons, big things, things that will make it better, and that you can afford. On the more technical side, be sure sample sizes are giving you statistically significant results and test in isolation. A/B split tests are the way to go. Multi-variant testing, not so much.