My Learning
→
User Surveys
→
Project: User Survey
Playbook
⌛
minute read
Completed [date]

Project: User Survey

Learning Objectives

Product surveys provide us data about each of our customer segments. We use this data to create actionable items for acquisition strategies. For example, we can target better on ad channels and write compelling copy for landing pages.

Phase 0: Setup

Start a blank document (in Dropbox Paper or Google Docs).

Phase 1: Figure out incentives

We find that offering some form of incentive to would-be responders helps with the following:

  • Receiving an adequate number of responses
    • Without an incentive, many people will ignore the email because there’s nothing in it for them.
  • Receiving higher quality responses
    • A free survey will mostly attract bored people. Highly-skilled people like designers will often not bother to respond unless they really love or hate the product. The incentive helps attract people across levels of engagement.
  • A boost in sign ups/revenue
    • If the incentive is tied to some sort of a product discount (instead of flat cash) it can lead to increased sales as the survey reminds people you exist and engages them further (and helps pitch why the product is worth trying).
  • Improving or saving brand image
    • Asking for something without offering something back often leaves a negative sentiment in the asked party. By offering something in return you’re showing you respect their time and effort, even if they don’t decide to take you up on the offer.

For B2B products, avoid plan upgrades as incentives in surveys.

Upgrades don’t incentivize businesses well since businesses don’t care about saving a few bucks, across the business; they generally care about having their time saved and giant dollar amounts.

Instead, the trick is to use Amazon gift cards: then, the person answering the survey gets to keep that selfishly for themselves (very compelling) as opposed to save a few bucks for the business they work for.

Ideally, offer at least $25 worth of credit to get good response rates.

Write down your chosen incentive, along with two backup alternatives, into your doc, under a section called “Incentives”.

Phase 2: Figure out the audiences

Common segments include:

  • Leads
  • Free users
  • Users at different priced plans
  • Customers (for products with one-time purchase)
  • Users who churned (used your product and left)
  • Users who never came back to the site

Actually go in and make sure you can actually get these lists of users with their email address; this has been a difficult task for some clients, since there’s sometimes work involved to figure out ways to create these lists.

Then, add them to your doc under a section called “Audiences”.

Phase 3: Figure out mechanics

Ask yourself the following questions:

  • What is your budget/discount to offer incentives for the survey?
    • We always aim to have a minimum of 100 responses. Work with any budget you can allocate to make that happen.
  • Do you have customer personas written somewhere?
    • Get those ready.
  • Do you already use a messaging platform (such as MailChimp or Intercom) that you can use to send the emails to customers to introduce the surveys?
    • If not, sign up for one of those two.
  • Do you already use a survey platform (such as Survey Monkey)?
    • If not, sign up for Survey Monkey. We’ve written step-by-step instructions on how to set up surveys using it (you’ll see these soon).
  • Are there any specifics you'd personally like to gather from this survey?

Add these questions, along with your answers, to your doc under a section called “Initial Questions”.

Phase 4: Come up with questions

For each audience, create your survey questions. Here’s an example.

Add them to your doc under a section called “Survey Questions”.

Phase 5: Write Survey Email

Craft an email to be sent to potential respondents. Check out User Surveys – Additional Resources > Template: Survey Email Outreach for templates to use.

Add them to your doc under a section called “Survey Email”.

Phase 6: Set up Survey

Port surveys from your doc to the survey platform (Survey Monkey). See User Surveys – Additional Resources > Survey Monkey Surveys on step-by-step instructions on how.

Then check for typos on the survey preview.

Phase 7: Send Surveys

Phase 7a: Build contact lists on the email platform.

Now, go onto MailChimp/Intercom/etc. and actually create the contact lists for each segment.

Build contact lists for each segment on your email/messaging platform.

Phase 7b: Build live email

On MailChimp/etc., copy-paste in the email that will go live from your survey report doc.

Do one last passthrough for typos and make sure the email and the audience match up.

Phase 7c: Test

Test it first! Send the email and survey to yourself and a team member. Click on everything and fill it out.

Make sure there are no crazy errors.

Phase 7d:

Go live!

Hit send.

Set a reminder to switch the survey off in a week. We like to use Slack to create reminders, but a calendar event will work too.

Create a reminder in 3 days to check the survey, and another to close the survey after a week.

Phase 7e:

Follow-up checks

Check on the reminders.

Keep an eye throughout the first few days for email open rates and survey response rates. If a survey is not performing well, send a reminder email as a new thread to encourage more responses.

  • Use the same email template
  • Use a different subject line to encourage immediate action
  1. Reminder: [[old subject line]]
  2. 3 days left to give us your feedback

Phase 7f: Follow up on incentives for users

If there’s an incentive, log into your survey tool and pull the email list of who responded. Send the gift cards/cash/etc.

Phase 8: Collect Results

Let surveys run for at least 7 days.

Then, go through the results. See if you can draw meaningful conclusions from them (see phase 9).

Phase 9: Summarize Suggestions and Actionable Insights in Final Report

This is something most agencies do to present to clients. It helps you pitch the rest of your company on changes you want to make — so they understand the data behind your recommendations.

After individual reports are done, we collect action items, insights, and suggestions in a single report. This document is focused on goals, and the insights we collected from surveys.

[[Customer Name]]: Insights from Surveys

Structure the document in the way shown in this template (you may need to make a different document for each survey if the results are quite different).

Phase 10: Generally applicable feedback

Keep these in mind when coming up with insights and conclusions for surveys.

Come up with a conclusion that brings it all together and briefly summarizes the biggest lessons learned.

  • This summary should come at the top of the document, so it can be easily referenced at a later date.
  • When doing this, continue thinking of the larger implications of what you’re suggesting, and try to tie larger ideas back together.

Not every question will yield meaningful conclusions. And some questions might yield multiple conclusions. So, whenever there’s an opportunity for insights, come up with them.

Be as specific and as descriptive as possible with your suggested insights as possible.

  • If you’re recommending a referral program to be put in place, how would this referral program be set up?
    • Incentives
    • Audience
    • Technical requirements
      • Thinking through these things is a thought exercise that makes you realize the true relevance and feasibility of your suggestions.
  • If you’re recommending content creation, suggest the type of content that should be created. And provide the reference data to back up the suggestion.

When including direct quotes from open-ended responses, make it clear that they are quotes.

  • Include quotation marks around individual quotes.
  • Create a table to group large numbers of quotes

Make sure what you’re saying is extra clear.

Write out the obvious, and don’t expect the reader to make any necessary logic jumps.

  • There’s a lot of material presented in survey docs. They’re very prone to skimming.
  • Always finish your thoughts. Don’t assume people will piece together the dots about what you’re implying.

When you’ve identified the value propositions that users care about, use that insight to analyze what you currently have for Ad and Landing Page copy.

  • Value propositions usually clearly identified with the question: Which use cases make [[product]] most valuable to you? These value propositions are the ones we need to communicate to audiences in ad and landing pages.
  • Analyze your Landing Pages to see how well those value propositions are communicated.
  • For Ads
    • Identify if current ad copy hits these value propositions.
    • How should ad copy change in response to these value propositions?
    • What other new ad copy can we test from these value propositions?

When you’ve identified the most valuable features of a service/tool, or the most valuable characteristics of a product, use that insight to structure Drip Message Flows.

This insight usually comes from the question: Which feature in [[product]] do you value most?

  • These are particularly important for Post Sign Up Message Flows sent to new users. Because once users experience these features, they will better understand the value provided by the product/service/tool.

Focus on frequent answers.

Some of the open-ended questions with long text responses will give you interesting insights. But be wary about acting on the opinions of one person. Maybe they’re not your target market. Maybe they wouldn’t be happy or buy from you even if you implemented their recommendations. Maybe they just want to complain.

Base your insights off of a wide range of people. If 50 people are saying something, you should definitely pay attention.

Open search
💬