GoFuckYourself.com - Adult Webmaster Forum - How to improve your clicks, sales and profits

Most people know that in order to increase their revenue they need to make test, nevertheless they don’t do it properly, example they have heard about A/B split testing and they set up something like this:

http://i.imgur.com/7taMnQj.png

These two ads are completely different from each other, so you can’t really know what element is making the difference when one perform better, when you are making an A/B split testing you need to change just one element of the control ad and evaluate it to see if the modification have a significant impact on the results; it is very important that you allow at least 30 elements of the measure you want to improve to declare a winner (if you want to know what ad has a better CTR at least get 30 clicks on each one of them, if you are measuring sales, wait to have 30 sales and so on and so forth).

So the first thing to increase the revenue is to make a proper A/B split testing.

http://i.imgur.com/J2jV6RZ.png

As you can see I have just changed one element on the ads, the second description line, the Ad A has “Get your first opportunity here” the Ad B has “Here are some well paid jobs” in this case we know for sure what element is making the difference.

What happens if you want to test more variations of the same ad (more elements)? example you want to test “The best paid jobs in town”, then you will have 3 different ads:

http://i.imgur.com/5ZsagZL.png

How does this impact your testing efforts? What does it mean to have an extra add? Well it will take you 50% more resources (traffic, time and money) to test one more variation; Here is where thigs starting to get interesting, because what if I want to test more elements on the ads?, if testing one more combination requires 50% more efforts How much resources I need to invest to test more elements? How can I manage it? Example testing 2 different set of titles such as “Looking for student jobs?” versus “Looking for Managing jobs?” then I will have all these combinations:

http://i.imgur.com/Nufgd7W.png

As you can see the number of combinations have increased and the resources to find a winner have grown too, so far the effort we require to properly test all these combinations to know which one is the better have increased exponentially, so every element I want to test is taking more and more resources.

As you can see things can get more and more complicated the more elements we want to test, the fancy name for this type of testing is “Full Factorial Model” that means you are testing all the combinations of the factors, once you have made the tests you record the results in a table like this one:

http://i.imgur.com/DmCCTA2.png

So this is how you read those numbers in the table:

If you see the columns comparing the titles one of them has a response of 2.7% (Looking for managing jobs) versus 1.93% (Looking for student jobs) while the row average don’t seem to really vary this means that the most important factor to get a better response are the titles, the other elements really don’t vary too much the response.

Here the most important conclusion to get is the information that is highlighted, this means that from all the possible combinations the one with the “Looking for managing jobs?” and the “Here are some well paid jobs” description is the one that worked the best, BUT we would not have been able to figure this thing out if we would have tested just one element at the time, all you have seen would have been the row and column average data shown in the above table.

As you can see one of the benefit of this model is to be able to see the exact combination that brings the best results, of course this model it is not limited just to 2 titles and 3 different descriptions it can be expanded as much as you want, you can to test as many elements as you want (Colours, display urls, calls to action, faces, button sizes, locations, etc…); this is exactly what most people try to do, but they really don’t have an organized approach to do it and get not significant results that is why they can’t consistently improve their sales, and even if you are trying to do it right here are some of the problems you will face:

a) You have to had a lot of traffic to test all the combinations (Remember how I had to increase in 50% more my efforts just by increasing one of the variables?, see above).

b) Because you need a lot of traffic you require a lot of time and money to make these tests.

Is there a way to make this process better? Cheaper? Faster? YES THERE IS!!! And here is how to do it, the process is an old technique used in the manufacturing world in order to increase the quality of the processes, it is a mathematical model called “Taguchi Model”, the best part is that all the mathematical mambo jambo has been already done for you ;)

As we saw the main benefit of the Full factorial model is that you test all the possible combinations, and your results are very precise, you know what title work the best with what description, with what Call to action, and you are already ahead of your competition because they are testing one element at the time at best because sometimes a title just works in the presence of a specific URL or a particular copy.

We know the advantages and disadvantages of that method as well, now lets see how we can handle the alternative, the Taguchi Method, translating all the math for you what the Taguchi model says is:

“The Full factorial model gives you very precise information, but it requires a lot of resources to get the information, what if we sacrifice a little bit of precision in order to get the information faster?”

Now, how is it done?, well Taguchi see’s all the combination model and they take a very special subset of those combinations which allows them to predict what combination will work the best without the need of running them all; as you can see there is some precision that is sacrificed so there is a little extra step you need to make at the end, but follow me please, it is not that difficult to do and it is worth it to learn it.

Let's continue working with the example we started, Imagine this is your control ad (the one you want to beat)

http://i.imgur.com/Og3Vgpj.png

Now here is the list of all the elements we want to test:

1. Headline1: Looking for student jobs? Vs Looking for managing jobs?
2. Headline 2: The best vs The greatest .
3. Body Text 1: 60000 vs 100000
4. Body Text 2: job opportunities vs carreer opportunities.
5. Body text 3: Get your first opportunity vs Get your new opportunity.
6. Body text 4: opportunity here vs opportunity now
7. Display url: bestjobs.com vs bestjobs.com/highest-salaries

As you can see we have we have too many elements, if we use the old “Full Factorial Model” this will be a total of 2^7 = 128 different ads to test (those are all the possible combinations) and that just make the things very, very complicated, now let's see how we can do this faster with the Taguchi Model.

Go to Taguchi (It is a free website), click the “Start test now” button.

http://i.imgur.com/NvLZ65T.png

Select the number of elements you are going to test (in this case 7) and click continue.

http://i.imgur.com/CbB4B5Y.png

We name each one of those elements as we did it above, now that you enter a name for the titles for each element, put the description for each one of them so the software can give you the combinations you need to test.

Once you click continue you will see the different combinations of ads you need to create and test in order to get faster results, o instead the 128 combinations we would have required in the “Full Factorial Model”, we have just these 8 combinations that the Taguchi model have given us:

http://i.imgur.com/m3E5yCL.png

If you see the Taguchi method gave me just 8 different ads instead of the 128 of the previous model and it puts in blue the elements that need to be changed from the control ad, so what you need to do is go, create those ad combinations, create 8 different ads, test them and measure the actions for each one of them; it does not matter what the action you want to measure is (it can be clicks, sales, time on the site, etc…) it needs to be statistically significance, which i means that you need at least 30 actions of whatever you are measuring to have data that is worth it.

So far you are testing 16 times faster and getting 16 times the savings with this technique compared with the old model, so far so good, now once you have done your tests you need to enter your results in the screen.

http://i.imgur.com/DJGUHPU.png

And then you are done!, here is the results screen:

http://i.imgur.com/N0b6BiI.png

You can see in the influence column how important each element of the ads is (the bigger the number the bigger the influence), in this case the headline, the second word and the domain name are the most important elements, in this way in your next text you can focus only in these elements.

Then it makes all the complex math behind and it gives you what it thinks is the best ad, something very important to point out is that the winning combination is showing is not the best from the 8 combinations we entered, but from the 128 original combinations!!!, it is already given you the winner of all if you had run all the tests; with this information you then create the ad that will be the winner among all the others, something like this:

http://i.imgur.com/WQvpJ9E.png

Then the last step is to put this ad vs the control

http://i.imgur.com/tP7hzSv.png

Happy testing