This is the first part of a three-part blog series. It is December and that often means looking back over the past year. I’ve been doing a bit more than that. I’ve been looking back over my 20+ years of various research consulting roles within MaritzCX. During that time, I’ve continuously fielded research questions from our clients. In this blog series, I’m going to focus on the three questions I’ve seen the most. Probably the most frequent question I’ve seen (and the focus of this blog post) is a two-part question: “How do I increase response rates? Should I shorten my survey?”
Redefining the Question
My first response to this question is to ask, “Are you really interested in only increasing response rates, or are you interested in getting more responses?” Those are two different things. Response rates are the percentage of responses you receive from the survey invitations you send out. Responses are the absolute number of responses you receive, regardless of response rates. In many cases you can actually increase the number of responses you receive while decreasing response rates by sending out more survey invitations.
Response rates matter very little in terms of your sample providing representation of a population. What is of the utmost importance is the absolute number of responses you have. For example, if I am trying to represent the United States population of approximately 325 million people, I only need a little over 1000 respondents for a confidence level of +/- 3 percentage points. It doesn’t matter if those 1000 respondents are acquired from sending a survey invitation to 5000 people (20% response rate) or 100,000 people (1% response rate). The only caveat here is that lower response rates may be an indicator that some sort of response bias is occurring (certain types of people may be responding more in comparison to other types). If that is the case, it doesn’t matter how many responses you have. Your sample will still not represent the population. If you fear response bias, you should do a response bias study – but that is a topic for another blog post.
Usually when I point out to clients that they are probably more interested in increasing the absolute number of responses they receive rather than just increasing response rates, they agree. That opens the door to starting at the beginning of the survey life-cycle to obtain more responses.
Increasing Outgoing Survey Invitations
To increase the number of responses you receive, the first place to start is to see if you are missing out on the opportunity to send more survey invitations. Some CX programs still engage in sampling instead of sending survey invitations to all eligible customers (conducting a census). If your program is sampling, consider doing a census. This will both increase the number of responses you receive and give you the opportunity to identify and rescue more at-risk customers.
Another factor to consider is the quality of your customer contact information. Are a significant portion of your records getting removed because contact information is either missing or wrong? If you obtain customer contact information from business units such as stores, hotels, dealerships, etc., it is important to look at sample quality at the unit level. Similar to looking at sample quality, it is also helpful to examine the amount of sample records received from business units compared to their number of transactions. Units with low sample in proportion to their transactions probably need to focus on better ways to obtain customer contact information.
Another way to increase the number of outgoing surveys is to determine if there are some segments of your customer population that you are missing. Not obtaining contact information for specific customer segments often has to do with information systems issues. For instance, in the earlier days of automotive customer experience research, most companies only surveyed warranty-service customers. They did not survey customers that went to a dealership and paid for the repair/service themselves (customer-pay events). The reason was simply a systems issue. Companies did not receive those transaction records from their dealerships. Now, most automotive companies have remedied that issue and they survey both warranty and customer-pay service customers. Are there system issues that are stopping you from getting customer records for certain subsets of your customers?
Revising Your Survey Invitation
The next step is to look at your survey invitation process and the survey invitation itself. You should look for two general things. First, is there anything that might be prohibiting customers from receiving the invitation and second, is the invitation compelling enough that customers will open it and act on it? More specifically, investigate these issues:
- Are you triggering spam filters? – Sending out too many invitations in too short a time frame can trigger spam filters. Sending out too many invitations with invalid email addresses can also trigger spam filters or even get your project’s IP address black-listed at internet service providers. Therefore, make sure to check that email addresses are correctly formatted. If you are really worried about the quality of your contact information, there are services available to pre-identify valid email addresses.
- Are you sending survey invitations to the wrong customers? – Outdated databases can cause you to send surveys to people that are no longer customers. Obviously, these people probably won’t respond to your survey, thus reducing response rates.
- Is the subject line of the email engaging to the customer? – The subject line is the first thing the customer sees. If it is not engaging, the customer won’t open the invitation email. It is helpful to test various versions of the invitation with different subject lines to determine which yields the highest open rates.
- Does the invitation display well on a smartphone? –Over half of MaritzCX’s survey respondents are now completing their surveys on smartphones. Make sure your invitation (and the survey itself) displays well on these smaller devices. You should also check to see how well your invitation and survey display in all major browsers.
- Do you include a realistic time estimate for how long the survey will take to complete? – This is especially important for shorter surveys so that potential respondents know there will be only a small time commitment. It is also a good idea for longer surveys because respondents will know what time commitment they are getting into and they will be less likely to abandon the survey.
- Is the ability to respond “above the fold”? – For younger readers, “above the fold” is a newspaper term that describes a headline that is visible when the newspaper is folded in two. In our case it means that when a customer opens the invitation, is the link or button to respond to the survey visible without having to scroll down? Remember, this should be the case on a smartphone as well as on a tablet or computer.
- Is there a call to action? – Your invitation should specifically ask customers to respond and tell them why responding is important.
Revising the Survey Itself
Finally, revising the survey itself may help increase responses. However, remember that revising the survey will only increase responses by reducing the number of people that abandon the survey after starting it. Typically, that number is quite small (about 5% for most CX surveys), so reducing abandonment probably won’t lead to a meaningful increase in the absolute number of responses. That being said, some of the things you should look for, in addition to the possibility that your survey is too long, are:
- Is your survey simple and easy to use? – You should keep your survey focused on the topic it is intended to measure and avoid “nice to know questions.” In addition, avoid mixing response scales as much as possible, as this can lead to confusion for the respondent.
- Does your survey look engaging? – Your CX survey represents your brand. It should have the same professional look you strive for in your other customer communications.
- Is the language in your survey easy for customers to understand? – Don’t use industry jargon. That turns off respondents and can lead to confusion.
- Does your survey follow a logical flow to walk the customer through the experience being measured? – This not only helps in reducing abandonment, also helps customers recall the event accurately.
The big point I’m trying to make here is that when you want to increase the number of responses you receive, you should look beyond increasing your response rate and you should look beyond shortening your survey. There are much more effective ways to increase the number of responses to your survey.
Please check back with us next month when I’ll publish my answer to the second most frequent question I’ve received.