User Testing the SERPS: How Do Your Customers Really Search?

How do you know what your customers are searching for online? Most SEOs scream: MONTHLY SEARCH VOLUME

Don’t worry, it isn’t their fault; even Google thinks SEO research is synonymous with Keyword research (The top 10 results for “SEO Research” are all about keyword research).

The correct answer is: Ask your users! Historically, SEO recommendations are full of assumptions based on data that most marketing professionals take at face value.

But, how many times have you done extensive keyword research then landing page optimizations and didn’t hit your goals? When the client or your boss asks, what happened? Did you answer go something like this:

“But Google’s data said…”

In my experience, there is a problem within the SEO industry of making assumptions with little-to-no testing. Our assumptions are validated only when our optimizations have succeeded or failed.

Well, it is time to reframe what SEO research means…starting with simple user tests to help validate assumptions.

Why should you user test the SERP?

Maybe because “user testing” has 12k MSV and Seer wants to rank for it with this blog post!

…just kidding.

Here are a few things that we learned and gained from conducting user tests on the SERPs:

• Empathy leads to solutions. As search marketers, the queries that we are optimizing for are rarely our own; therefore, understanding the people behind your organic traffic is a great way to identify opportunities to truly solve their problems.
• The thought process behind searchers. Prompting users to conduct searches out loud gives us insights into their thought process of choosing certain keywords, which can lead to strategic insights.
• Why users don’t click results. Hearing users say what they like or dislike about a certain search result may give you insights on how to restructure your content or create more appealing call-to-actions within your meta data.
• Underlying assumptions. When a user puts a term into Google, they are assuming Google knows EXACTLY what they mean. User tests allow you to understand what the user is REALLY searching for when they use generic head terms.
• User testing is cheaper (and more insightful) than just brainstorming. This technique may save you time and money. Five people in an hour long brainstorm could cost your company between $400 – $1200 to write assumptions on a white board; five user tests range from $100-$600 for REAL, human insights straight from your audience.

How to User Test the SERP

As a starting point to begin running your own user tests, I will walk through the process that I’ve used; however, feel free to remove or add steps based on your research needs.

Step 1: Gather Assumptions

Simply put, what are you testing?

With any research project, you need to form a hypothesis. I’m not going to tell you to create a full-blown research plan, PowerPoint or proposal, but I always approach research with a few assumptions that I want to test.

Here are a few examples of SEO related assumptions to test:

• Does ranking position matter in my industry? This is a good way to help clients or your boss from focusing on ranking at position 1 for some crazy keyword. While many SEO experts say that position impacts clicks and conversions, I’ve run tests that suggest other factors, such as brand recognition and content type, may be more important to users. (Also, check out additional information on CTR data predictions).
• Do people search for [Insert Keyword] when looking for my product?. Probably my favorite test. This shows you exactly what users type in the search box to find the solution that you offer, instead of assuming what they search for based off of keyword MSV.
• What influences people to click a result in my industry? Keywords and CTAs are super important for meta descriptions and titles, right? Maybe not! Other factors, such as including the current year, may be more important to users than seeing keywords or CTAs in your specific industry
• Does device impact what results user click? This test allows you to see the difference in user behavior based on device.

These are just a few examples of questions that user testing the SERP can answer for you.

Step 2: Recruit Participants

Hopefully, you or your marketing team has a good understanding of the audience that you are targeting. If not, I suggest conducting an audience analysis to identify who can validate or nullify your test assumptions.

(Sidenote: The same people who tell you to do an entire research plan, may also suggest building out complex personas or audience profiles. While detailed demographic personas do serve a purpose in research and SEO, they aren’t a necessity for this process.)

Here are a few places I look for research participants:

• One of the most honest places on the internet. There is a subreddit for almost every vertical and I am willing to bet you can find members of your audience here.
• When conducting research, I prefer face-to-face interactions with my audience! Meetups are a great place to recruit study participants, but your selection of Meetup groups may be limited based on your location.
• Social Media. According to a2016 article by Adweek, there are about 2 billion monthly active users on Facebook, Twitter and LinkedIn combined.
• Your Network. Don’t be afraid to reach out to your friends, families or coworkers for participant referrals.
• Recruiting Services. If you have an extremely niche or hard to reach audience, then you may want to use a recruiting service to help find participants.

Lastly, I recommend some kind of incentive to motivate and thank your participants. I have found that most people are willing to participate in research for a $10 gift card, but this may vary depending on your users.

Step 3: Conducting the Test

Once you have your participants on board, it is time to set up the test! There are multiple ways to conduct user tests, but here are the two methods that I use:

Option 1: In-person/Moderated

As the name suggests, these are done in the same room as the participant. In-person tests give you the opportunity to talk to participants as they are searching. I recommend following these steps:

1. Agree on a location. If you can, try to schedule the test in the environment that the user would be searching for your product or service; this helps you get an understanding of other factors that may influence their mindset.
2. Prepare for the test.
   • Mindset: It helps to set all your biases aside. Personally, I wear clear lens glasses to get into “character.” Once my phony glasses are on, I am in full research mode.
   • Tools: For in person tests, bring a laptop with a good screen recording software. Personally, I useJing, but there are many other options for you to choose.
   • Assumptions: Remember to review your assumptions and have a clear understanding of what you are testing.
3. Running the user test.

• Build Rapport: When you meet with the participant, give an introduction and make small talk for a few minutes. The more comfortable they are, the more information they will share.
• Interview (Optional): I like to take 10-15 minutes to conduct a user interview and get a deeper understanding of their relationship to the product or service.
• Get to it! : Begin the screen recording and start the test. Be sure to ask follow up questions and have participants talk through their thoughts and decisions. The more they talk the better.

Option 2: Remote/Un-moderated

If you can’t do in person tests, remote tests may be the go-to option for you. For remote testing, try following these steps:

1. Selecting a tool. Any remote testing platform should work for user testing SERPS, but be sure to do your research to find a platform that works best for you. Platforms such asValidately and Userlytics will also recruit participants for you.
2. Writing the Prompt. This is the most important step in the un-moderated process. You need to make sure that your test prompt is clear and includes follow-up questions for every task. Also, be sure to remind the participant to talk out loud throughout the entire test.
3. Send the test. Once you have written the prompt and set up the tasks, send out your test to the participants!

Once the testing is complete, be sure to thank your participants and give them the incentive that you agreed on.

Step 4: Analyzing the Results

Sooo, what happened?

After you have completed your study, which should include at least 4-5 user tests, it is time to synthesis the data.

This step really depends on your hypothesis and goals, but here are some tips that should get you on the right track for pulling out insights:

• Watch and listen to the videos closely for eye-opening quotes
• Categorize trends and recurring themes using a detailed notes or a spreadsheet
• Compare the insights that you have to other data points, including quantitative data and any previous qualitative research from industry studies or your own previous work
• Send the videos to your team and see what insights they find from the tests

Now take the insights you gained from your user tests, make some cool asset and go win a fancy marketing award!

If you are interested in learning more about using an audience-first approach for digital marketing projects, check out our services page.

If you have any questions about user testing the SERPs, let me know in the comments or connect with me on Twitter @DarrylG