Applying scientific method to performance

There is still a lot of talk about optimization and marketing performance testing. Justifiably so. Technology helps with this, of course, but it’s the non-technical expertise that really makes the difference.

I do a lot of performance testing with my clients and I always follow the scientific method. Here’s an introduction to the scientific method, along with a deep dive into one of its key elements: hypotheses.

The scientific method

Using the scientific method offers a structured approach to performance optimization. Critically, it involves developing and testing hypotheses about what strategies or tactics will improve results, such as revenue, conversions or engagement.

Scientific method:

• It provides data-driven insights for decision making.
• It reduces risk by testing changes on a small scale before they are fully implemented.
• It enables continuous improvement by helping you identify not only what works, but also why it works.

This approach allows you to base your marketing testing work on evidence rather than assumptions.

There are eight stages of the scientific method:

Researching and developing your hypothesis is a key part of the scientific method. Here are some tips and tricks to do it successfully.

What is a hypothesis?

In this context, “hypothesis” is a convenient word for an idea of ​​what might improve performance—but it’s also more than that.

You might say, “I want to test the color of our call-to-action buttons. They’re currently red, let’s try making them green and see if that improves performance.”

It is not a hypothesis. There is no proposed explanation as to why this change might be effective.

Case study: Hypothesis building

Here’s a real-life example of how I built a hypothesis for a test years ago.

I was at my local Barnes & Noble bookstore reading about color psychology in a book I found on the discount rack.

It has been debated why red is the color of stop signs – that it might subconsciously send a stop signal. I realized that I had a promotional email with red buttons, because the brand colors were dark purple (it was almost black) and red.

The product was a financial advice publication (we gave advice on which stocks to buy to improve your financial portfolio), and that’s when I started thinking about what the color red means in the financial world. ‘In the red’ is a bad thing there — it means you own more money than you have.

Well, I thought, maybe the red buttons are a depressive reaction.

But what to test against red?

I thought of traffic lights, where red means stop and green means go. Then I thought about the financial world. Green is the color of money, and that’s what we promised stock referrals would earn them.

The hypothesis that came out of this was: “Changing the color of the CTA button from red to green will increase response and revenue for the above reasons.”

See the difference? Tests based on hypotheses supported by common sense are more likely to perform well.

Getting inspiration for hypotheses

Inspiration can come from internal or external sources. Here are some ideas for finding inspiration for your tests.

Inner inspiration

Case Study 1: Failed Performance Tests

Even if you fail the test, you should look for knowledge that you can use in the future. For example, a few years ago we faced a control email with a version where we changed a number of different elements. Our KPI was conversions; recipients should have completed a conversion form.

Control won firmly. But while the test version lagged behind in conversions, the click-through rate (CTR) on its top CTA button was nearly double that of the control group’s top CTA button. As a result, we retested only the elements around the CTA button — including the location and the message that this is an exclusive offer.

Hypothesis here: “These things seem to have increased CTR in the previous test; maybe if we isolate them from the other elements in that test and apply them to the control, they will increase not only CTR but also conversions.”

This time the test won.

Case Study 2: Campaign Metrics

Much of my work with clients involves multi-effort email campaigns, where we send a series of two or more emails to the same list over a period of time. I often get inspiration from data, for example…

In this example, we saw that Effort 5, the last effort in the series, still generated more than $18,000 — so perhaps we could raise an additional $9,000 or $10,000 by adding Effort 6. Here they assume, “Since Effort 5 done well, we should be able to raise additional revenue by adding effort 6.”

We also saw that effort 4 did well with $0.45 in revenue generated per email address, compared to just $0.32 for effort 3. Efforts sent earlier in the sequence perform better. So we ran a test with the hypothesis “Since earlier efforts tend to perform better, we should be able to increase revenue by reordering efforts 3 and 4, since effort 4 generates more revenue per email address .”

Dig deeper: Why we care about performance marketing

External inspiration

Your inbox

Your inbox can be a treasure trove of performance test inspiration. Are you getting any testing ideas from this Walgreens email?

These are the hypotheses I derived from this email:

• “Including name personalization at the top of the email should attract more people to read it, increase clicks. and increase revenue.”
• “Putting an offer behind the ‘removal’ will increase engagement and conversion rates.”
• “Showing recipients a visual of their reward status will motivate them to want to continue earning reward points, which will increase revenue.”

Articles, blog posts, webinars, presentations and other resources

Any online resource can be a great source of inspiration; even better if it presents case studies of performance tests that others have done.

Last on this list, Really Good Email, is a swipe file. They have screenshots of more than 15,000 emails; you can search a number of different variables. Browsing a drag list like this is a great way to get inspiration for hypotheses.

Now is the time to start performance testing or up your performance testing game. I hope this initial step in developing hypotheses helps you.