Some time ago, we brought a guide to designing a better survey structure and how to avoid making various survey mistakes. In this blog, I want to address the fact that no matter how much you try, your survey will almost always contain bias in one way or the other. However, there are some things you can focus on to minimise survey bias. But first let’s look at the bias pitfalls. In the following, we are solely considering surveys in the physical space, like a survey station or exit interviews/surveys.
How do you present your survey?
If you are running a kiosk-based survey, maybe fixed in a tablet case or a kiosk stand which is placed conveniently somewhere, then solely because of the technology that you choose, you are creating a certain bias, often related to either age or gender.
In the same way, if people complete your survey via exit interviews, then it’s pretty much the same deal. In any survey where you approach the respondent to learn about their experience or get their feedback, there’s bias. And it’s two-fold: The interviewer will consciously or subconsciously select her respondents from certain criteria (other than the ones defined in pre-segmentation). This could either be the personal appearance, personality type or something completely different. But the respondent’s answer also depends on the interviewer. For example, if the interviewer is a young extrovert with a lot of energy as opposed to a person who is less engaging, then responses may vary.
So, no matter if you are doing your on-site surveys via interviews or survey stations, there’s bias!
Where do you place your survey?
An entirely different issue is where people complete your survey. Assume that you are bringing your car in for service. Do you think that your answers would differ, if you took the survey while in the waiting area, drinking coffee or if you were on your way out of the dealership? Chances are that you would make more of an effort to give a more accurate and descriptive answer, if you were sitting in the waiting area, right? I mean, if you’re heading out the door, you’re not going to invest as much time in answering because mentally you are already leaving the establishment.
Most organisations are interested in the service experience and hence they want to ask their customers when the experience is complete, but there’s always a trade-off between how much data you will get vs. how full a response you get.
Survey design is important in order to minimise survey bias
The survey design itself is important. How you ask your questions, what conditional clauses are built into the questions etc. – i.e. is the respondent even able to answer the question properly?
Firstly, simply putting the survey questions in an order which is logical for the respondent is important. For instance, if the survey is about a service experience, then it’s important to ensure that the respondent has had an experience of some kind, meaning have they browsed, purchased, or been in contact with service staff. Therefore, it will always be key to ensure that the respondent has the fundamental ability to answer a given question. You can solve this problem easily by filtering the respondents so that they only see with relevant questions. In tabsurvey, we ensure this by introducing a simple flow in the survey.
Another typical pitfall in a survey is the question phrasing. Phrasing a question with bias, is unfortunately quite common. Consider an employee survey, that examines employee satisfaction and maybe the level of stress in the organisation. Now consider the question: “On a scale from 1 to 5, how stressed do you currently feel?”. Well, the employee who doesn’t feel stressed at all, has no way of answering this question accurately. Even a 1-score, will indicate that she feels a little stressed out. So, the lesson is to be attentive to the phrasing of the question, as it otherwise won’t minimise survey bias.
The right predefined choices
Along the same lines, creating a question with predefined choices (single or multiple choice questions) can create bias. It could be as simple as “Where did you hear about us?” followed by a list of choices. But if the respondent’s preferred choice is not among them, then there’s bias. The solution is to always have an “Other” field, where the respondent can fill in a text-based answer. It may seem trivial, but nonetheless the “Other” choice is often not included in the list of choices. As a result, the respondent is unable to express her true opinion.
So, there it is. Use these simple guidelines when you conduct your surveys and you’re sure to minimise survey bias to the extent possible.
Being an exhibitor on an event or trade fair can be an exhausting experience. Sore feet, a used up voice, and fatigue from speaking with hundreds of visitors takes its toll on you. At the same time you want to reap the commercial benefits from your efforts. Therefore, it’s important to make sure that all correspondence and potential customer engagement is documented well and archived for later follow-up. In this post you can read about how a Danish startup accomplished this using tabsurvey on the trade fair TechBBQ.
TechBBQ is hosted in Copenhagen and is a two-day international tech-startup summit by and for the startup community. Every year a dedicated team works tirelessly to link entrepreneurs with the rest of world, providing startup ecosystems with cutting edge insights, business opportunities and networking.
In 2018 the summit outgrew itself for the fifth year in a row. Hence, more than 6,000 local and international startups, tech talents, innovative minds, visionary corporates, prominent investors and pioneering speakers participated.
The next generation of content writing
One of the participating startups was JumpStory – a Danish company co-founded by Jonathan Løw and Anders Thiim Harder. JumpStory rethinks digital communication and marketing as they assist small and large organizations in writing blog posts, SEO articles, Newsletter and SoMe posts. According to Jonathan Løw, what makes JumpStory different to many others is that they use artificial intelligence in the process instead of content writers.
As an exhibitor on the TechBBQ event, JumpStory wanted to maximize customer engagement and therefore they approached tabsurvey to make use of its lead-generation tool for trade fairs and events.
How to manage sales leads on events
If you’ve ever been an exhibitor at a trade fair or event, maybe you know the feeling. The traffic and interest for your booth varies. At times it’s really slow and you and your colleagues have a hard time keeping focus. At other times, the booth is over-crowded with visitors who have all sorts of questions or comments, and you don’t really get to engage with everyone. Annoying right?
To solve this problem, tabsurvey delivered a complete solution to help JumpStory manage sales leads on events. The solution included two iPads, two iPad stands, chargers and cables and a digital signup form and survey to engage with visitors and potential new customers.
In addition to the two lead capture stations (stands), JumpStory had two employees, each with a handheld device harvesting leads. With four active channels active throughout the two days, they were able to capture all potential interest at all times.
According the Jonathan Løw, the TechBBQ event was a massive success. Not only did JumpStory engage with hundreds of visitors, they also managed to capture a lot of sales leads. During the two days of the event they managed to get 500+ signups. Of the 500 signups, a staggering 96 percent were positive towards being contacted again after the event. Almost 500 leads isn’t bad for two days work, right?
Do you a trade fair coming up, and want manage sales leads on events? Don’t hesitate to sign up or contact us for more information.
In this blog post I want to shed light on factors that can influence the response rate if you are performing Surveys On The Go in general.
What are Surveys On The Go?
By Surveys On The Go I mean the type of survey which are available in the physical space and captures responses quickly and here and now. Obviously, the survey itself matters: the questions you ask and how your engage with your visiting audience. If you are interested in survey design in general and which survey strategy to adapt, you can read about it here and here. However, there are some other, external factors that can affect your response rate. One of these elements are the respondent turnover rate.
What is your respondent turnover rate?
By this term I mean how quickly the population of respondents (surveyees) changes over time in the physical location you are surveying. For instance, if you are a retailer, and you are surveying your visiting customers, then your respondent turnover rate would be the number of new visitors your store gets divided by the number of total visitors. Imagine that your store receives 1,000 monthly visitors. Of these 300 visited your store the month before, and so they are repeat visitors. Your respondent turnover rate would be (300/1,000) x 100 = 30 percent.
A place where the turnover rate is even higher would be in a Tourism Information Center or Welcoming Center. Naturally, the turnover rate would be high here because very few tourists would visit a welcoming center more than once. Therefore, the turnover rate would be close to 100 percent, as almost every visitor is new and unique.
At the other end of the scale, consider the cafeteria of a company. The employees here are asked about the satisfaction with the warm meal – every day. The respondent turnover rate here is close to zero.
In the examples given above, the overall motivation for answering a survey is, among other factors, dependent on the respondent turnover rate. I.e. the more times the respondent sees the same survey, the less motivated she would be to answer it. This is why you cannot apply the same kind of logic when performing, say, an employee engagement survey and a visitor center survey. Apart from the fact that there is a big difference in the composition of the respondents, there are some other factors that could affect your response rates.
Asking someone raises their expectations
Consider your company was surveying you in your cafeteria every day and that they asked the same question everyday: ”How did you like the warm dish today?”. The first time they ask, you rate it mediocre and comment that the chicken was dry. The following week, you get chicken again, and guess what: It’s still dry. When experiencing the dry chicken once again the third week, you simply don’t answer anymore, because what’s the point, right?
The lesson to be learned is that the lower the turnover rate of the respondents is, the more important it is for you as survey creator to ensure that the surveys you present are variated. Please see the below graph depicting a simplified correlation between survey variation and respondent turnover rate.
Ways to increase the response rate if respondent turnover rate is low
Areas where the respondent turnover rate is typically low includes: Internal employee engagement surveys, facility management services (like printing, cleaning, cafeteria etc.), educational institutions, and food businesses that have a very frequent, regular customer base.
For these and related segments, we recommend some of the following ways to increase the response rate for your surveys.
Stop-go approach. One solution is not to monitor continuously, but instead apply a stop-go approach. For instance, to survey for 1-2 weeks and then stop the survey for another 1-2 weeks.
Acknowledge and visualise results. In itself or in conjunction with the stop-go approach, a good way to spend the weeks of not surveying would be to acknowledge the feedback received, for instance by sharing the feedback with the respondents (customers, visitors, employees). This would create a common understanding of what feedback was given and identify what, if anything, needs to be changed.
Act. Acting on the results from any survey ensures high motivation from the respondents and raises the chances of higher completion rates in the future.
Change some questions. Keep some overall base questions but vary others to keep the survey from being too boring.
Dry chicken example revisited
So if you are asking about the warm meal of the day. Then follow up in the coming weeks with a different question, like: ”Is the chicken still dry?” or “If we were to change the menu away from chicken, what would you prefer, we changed it to?” (with alternatives). That way, you are signaling that you are actually listening and want to improve the services you provide.
Anyway, would love to hear your thoughts on this subject. Have you or are you performing surveys on the go in areas where the same people are passing through? How do you engage your respondents?