In nearly every survey project, three questions inevitably come up: Whom do I need to survey? How do I know if I’m surveying the right people? And how many survey responses do I need to ensure that my survey results will be accurate?
These are all very worthy questions. And to help you answer them, following are ten tips for developing and conducting a survey, as well as getting better survey data results:
1. Determine what your goals are for the project. Before you begin your survey project, take the time to sit down and think about what it is that you specifically want to discover. For instance, are you looking to find out how your current products or services stack up against the competition? What your employees think about your company’s culture and leadership? What your employees think about the products they support? Etc. If your goals for the survey aren’t clear, the results you get back from your survey will most likely be unclear, too. So, the more specific you are about your goals and what you’re trying to accomplish, the more likely you are to get useable answers.
2. Identify your target population. Decide who your target population is for your survey (i.e. what group of people do you want to measure?). Is it customers? Potential customers? Employees? Or some other group? Clearly identify in words who should be taking the survey and who should not.
3. Assess your survey sample frame. One of the greatest challenges that researchers face is not having all of the contact information that they need for their target population. For example, you might be limited to contacting only customers who have e-mail addresses (i.e. many companies don’t have e-mail addresses for a significant portion of their customer base), prospects from a purchased list, or only employees who have e-mail. The portion of the target population for which you do have contact information is the sample frame. Thus, determine whether or not you may need to augment your approach with another survey method (e.g. paper-based surveys, e-mail surveys, phone surveys, etc.) to ensure you have an accurate survey sample frame.
4. Consider potential target population biases. Researchers have found that different groups of people are prone to have certain biases, which means that some survey samples may not accurately represent an entire population. As a for instance, certain populations may be more aware of contemporary ideas, some may have a better historical perspective, some may reflect an intimate knowledge of your products whereas others may have a casual acquaintance. Recognize that age biases, gender biases, geographic biases, product biases, department biases, etc., may exist. To ensure your survey results are accurate, keep these biases in mind and try to write your questions in a neutral, objective tone.
5. Be wary of non-response bias. Sometimes those who do not respond to your survey have strong opinions whose absence would skew results. The portion of invitees who respond is known as the response rate. There are a lot of strategies you can use to improve response rates and reduce non-response bias. Some examples include: reminder e-mails, incentives, and using multiple mediums to contact potential respondents.
6. Aim for a 70% survey response rate. Even if thousands of people are invited to take your survey, it’s rare that everyone will complete it. Response rates vary depending on the nature of your target population, the vehicle through which your survey is conducted (e.g. whether via e-mail, paper, or phone), how engaged (or disengaged) your target population is, and the amount or size of your survey incentives (i.e. whether or not you pay or reward someone to take your surveys), etc. For some populations, it is reasonable to expect a response rate as low as 5%. However, in general, you should aim to have at least a 70% response rate on your surveys in order to minimize the potential for non-respondents’ surveys to differ in meaningful ways from survey respondents. Try to get as many invitees to become respondents as possible.
7. Determine your survey sample size. Whenever companies invite only a portion of their entire customer base to participate in a customer survey, there is always a chance that the opinions of those who didn’t get invited to participate could be vastly different from those who did. Thus, the more you increase your survey sample size, the more you reduce the chance of an incorrect finding. To answer a categorical question (yes / no, agree, disagree, ethnicity/race, etc.) with adequate certainty, gather at least 300 responses.
8. Survey a representative sample of your target population. It’s not always necessary to gather data from your entire target population. In fact, oftentimes, a simple representative sample of your target population will do. There are a lot of resources available online that can help you in determining the appropriate sample size for your survey. For example, is it 10% of customers, 100% of your purchased list, a third of all of your employees, etc.?
9. Select your survey sampling strategy.Before your survey goes out, decide what your budget is for conducting your survey and select a survey sampling strategy accordingly. Some of the most common survey sampling strategies include:
10. Simple random sample (inexpensive)
Make a randomized list of customers, select an adequate number
- Undersample (inexpensive)
Throttle down invitations to high-response groups
- Oversample (less expensive)
Use an alternative medium to gather extra responses from low-response groups (example: use paper in addition to online media to gather needed responses from elderly people)
- Oversample (more expensive)
Avoid ‘media bias’ by sending proportional media to all groups (example: if you need to gather 50% paper from elderly people then you should try to get paper responses from 50% of young people, too)
Design and create your survey. If you are using e-mail, make sure the subject and header information are accurate and compelling. Start with a short introductory or welcome message that outlines your goals. Let invitees know how long the survey will take and the value of any survey incentives. In the closing, give people contact information for a real person. Finally, assure people that their information is private and confidential. High-quality survey invitations make a huge difference in improving response rates.
When done successfully, surveys can provide your company with powerful insights about your target audience, and give you the confidence you need to design and develop even better measurement instruments in the future.
For more information on Allegiance’s survey design, creation, and survey Best Practices consulting services, contact Allegiance at (801)617-8000 or visit: