The following transcript is from an interview between Brandon Laws and Kim Smith on the podcast episode entitled: “Building and Analyzing Employee Engagement Surveys.”

Brandon: I’d like to welcome Kim Smith to the podcast today. She is a Human Resource Business Partner at Xenium with over 13 years of HR experience. We are lucky to have here with us today, thank you Kim.
Kim: It’s nice to be here, thanks Brandon.
Brandon: You’re welcome. Today we are going to discuss something you are pretty excited about and that is Employee Satisfaction Surveys. Let’s dive right in. When you are considering developing a survey for a company, say it’s your company, a small business, what is typically the common purpose behind doing a survey?
Kim: It’s really to get a sense of the pulse – what are employees’ feelings about the workplace, what might we need to focus on to make things better, to engage employees more, or to lead to better satisfaction, and therefore retention at the company.
Brandon: So there is definitely some sort of purpose, whether it’s in terms of metrics, or trying to hypothesize something within the company.
Kim: We want to find out what the high points are; what do people like about the company? There might be some focus around compensation or benefits, or maybe we want to offer a flexible workplace and want to find out if employees would be receptive to telecommuting. There can be some real targeted aspects to the survey. We might be considering increasing our benefits, or maybe there’s some training and development we want to do. We can look at questions around those topics specifically, in addition to overall satisfaction with the company.
Brandon: It sounds like there are a bunch of different types of surveys you can do. In terms of the duration between surveys, do you see employers do one every 6 months, every year? Are they doing various types of surveys? Give us your perspective on that.
Kim: We see everything. Every 6 months would be great but they are pretty time intensive, so company’s today, especially in our current economic environment, may not have time to do them every 6 months. In general I’d say that annually is a great time frame. If there’s a point in the company’s calendar year or fiscal year that is slower, that is a really good time to consider doing a survey. A time when we aren’t in the middle of the holiday rush or benefits enrollment or things like that, so that employees can really focus on the questions and helping the company get better.
Brandon: At the very basic level, what are some of the tools that you can use to develop a survey?  Are we handwriting them, are we developing them electronically, what’s your experience?
Kim: Typically nowadays they are electronic because it is so much simpler to roll up the data, whether you have 5 employees or 200. Survey Monkey is something that most people are familiar with and it’s a really great tool that I’ve used before because you can really get aggregate common themes pretty quickly.

Brandon Laws

Brandon Laws


Brandon: So that tool is good for the data analysis at the very back end. In terms of collecting the data with a tool like that, do you find it easier to collect the data digitally versus doing it on paper?
Kim: Much easier to collect if it’s electronic. I’ve done some by paper, and if we think about it, even a suggestion box is a very focused,small paper survey. But the biggest issue with paper surveys is the shear amount of time it takes to roll up the data and put it into common themes for the management team to look at.
Brandon: Once you’ve defined what kind or survey you are going to do and what tool you are going to use, how do you get started from there? With developing the questions, is it a bunch of people coming up with questions? How do you typically see something like that?
Kim: We might have a sense of the basic questions we want to ask, but even before we get to questions, we want to make sure that the management team and the leadership team really buys into this process. The biggest issue is asking questions that the management team will want to fix. If there’s no desire to change things, then why bother asking? It’s really important to have managements’ buy-in as well as their commitment and willingness to take positive steps toward improving anything that might come out that’s less than desirable. So getting management buy-in to the process is the first place to start. Then it is a matter of planning. When are we going to do this, what tools are we going to use (like survey monkey), and then drilling into some of the details around the questions. It’s important to ask questions that are focused on one thing. To ask about pay and benefits for example, if they gave us a 1 and a 5 we wouldn’t know if it’s the pay they are dissatisfied about or the benefits. It’s really important to construct a question that’s focused on one particular topic.
Brandon: I’d imagine the way you ask certain questions is all about how you are going to get the data on the back end. The one thing I worry about with survey and data collection is confirming biases. If you want an outcome and you want to confirm that, you might structure the questions in a certain way that would gear towards getting the answer that you really want as a management team. How important do you think it is to have, not necessarily a third party, but somebody like an HR person within a company to help facilitate and craft the questions in a way that it’s not written like it’s coming from management.
Kim: It’s really helpful to have an HR person or third party of some sort to look at the questions. If every question is written positively then it looks like we are trying to lean and skew the data to our benefit. It’s really important to have a neutral third party to not only write the questions but to analyze them as well.
Brandon: I imagine sometimes you want open-ended comments and feedback, but you also want something you can analyze in terms of rating scales where you can actually have metrics on the back end. Where is that balance? You don’t want a whole survey full of comments boxes, that’s harder to collect.
Satisfaction SurveyKim: Yes, that is really hard to analyze, and you will get some long-winded employees. We typically see a 5 point scale, 1 being the lowest to 5 being the highest score that you can give on that particular question. It works well to have mostly 5-point scale questions, just for ease of rolling up the data and looking at it quickly, and then balancing it with a few open-ended “What else have we missed?” questions or some essay questions to delve into areas where there might be some specific, personal or generational issues that you will want to dive into later. Sometimes essays or “Tell us more” kinds of questions are important as well. It’s just a balance of how much time you have to roll up the data and how many resources you have to analyze the data in the end. That’s how I decide how many rating scale questions to ask versus open-ended.
Brandon: In terms of how many questions to ask, what’s the lowest and the highest amount you’ve seen? I imagine it varies all over the place.
Kim: It does vary all over the place. I’ve seen as few as two or three, with a question such as, “Would you recommend X company to your friends?” The max that I would recommend is around 50-75 questions. You don’t want to take more than 20 or 30 minutes of their time to fill out the survey because people have to work, too.
Brandon: It’s funny that you mention the 2 or 3 questions because I’ve seen some recent surveys where you are just capturing 2 or 3 data points. Even though it’s very broad, you are able to get some quick snapshots in terms of satisfaction between employees, or customer satisfaction, or whatever it may be. I’ve also seen the 50 or 60 question ones, and those actually concern me a little. As someone who is doing analysis on the back end, there is sort of that paralysis by analysis. With 60 data points, how could you honestly expect to have some sort of action items on the back end to fix something or just mine the data?
Kim: It’s challenging. A lot of those questions might be the same question asked a couple different ways to address that bias. I’ve seen 75 in large companies, but for a small to medium size company, 25-30 questions would be more the norm there. It is a bit challenging when it gets up to the 75 range.
Brandon: What about anonymity? Do you find that employers want to know who is filling out the survey? On the other side of it, are employees even comfortable putting their name down?
Kim: Very rarely, to be honest.
Brandon: How do you handle something like that?
Kim: Keep it anonymous. To me, that is the best practice. At the end of the day, it shouldn’t be managements’ motivation to find out who said what. Our intent should be a good faith that we are management, we are here to make things better, and we want your open feedback. That feedback is truly a gift that is going to make us better as a management team and as an organization. We are not going to worry about where it came from and rather focus on the fixes and making things better.
Brandon: I don’t know if this is a direct comparison, but the analogy for this (behind being anonymous) would be that people can hide behind a persona. You see it on the internet all the time, where people hide behind some persona they have created and they are overly negative or they make more rude comments than they normally would. With a survey like this, would you expect there to be some more negative comments than there normally would be?
Kim: You would find some more negative comments. Because it is anonymous, it’s an easy way to blow that steam that sometimes employees have. As we are analyzing the data, we want to look for those strong common themes. You will have those outliers that are super happy or really unhappy, but we look for more of the common themes. In the surveys I’ve done, it is usually pretty easy to get to those common themes that the majority of the people are giving you feedback on.
Brandon: It seems like that’s the reason you’re taking the time to do the analysis. Even though tools like Survey Monkey or Zoomerang actually do some good analysis and put them into categories, I think that you really have to look through and filter out the outliers.
Kim: Exactly, that’s how you do good analysis.
Brandon: What about encouraging participation? Being anonymous is one thing, but I’ve also seen employers offer incentives around completing surveys because they want all of the feedback, but how do you know who is taking the survey if it’s anonymous in the first place?
Kim: Well you can look for 100% participation. It’s probably not likely, although I have seen some 90-95% completion, often in organizations that have a lot to say and really see surveys as an opportunity. In that case we just said, “100% participation and we will land everyone a pizza party,” so pizza was brought in at lunch and everybody won; it was really Employee Partypositively received. That success was tied into a really strong, positive message from management that they were doing this to improve and they valued feedback, so it seemed like that made participation super easy. And pizza is always nice because then you don’t have to pack your lunch!
Brandon: Absolutely.  It seems like an easy way to get people on tune because if there’s not 100% participation then everybody loses. Let’s go back to the analysis part of it. Who should be collecting the data? If it’s an in-house person, maybe an HR person or somebody on the executive team, who is doing the analysis? If there are other people who need to have some kind of visibility to the data, how should that be represented and what have you seen done in the past?
Kim: What I’ve seen work well is having someone in human resources or someone who is viewed well and respected in the company that employees trust. The worst thing to do would be to have the manager who is always out to find the person that’s complaining. Finding that person who is well respected and who role models confidentiality and respect for feedback should be the one digging through all the details, rolling up all those common themes, and showing the percent of employees who are satisfied or not. I recommend that the information be shared with a number of different populations. First of all, the executive team is going to be very interested in seeing that information and they should probably be the first people to see it. They are going to want to dig into the details and ask the data analyzer “What do we do next?” That is a great conversation to have and to be a part of. After there’s a little bit of analysis it should go out to other management below the leadership team, and ultimately to employees as well. If we’ve asked for their feedback we really owe them the opportunity to see the results and to speak to what we are going to do.
Brandon: Do you think each level of person, if you have an executive team, a management team and an employee group, should see the same sets of data? Or is it broadened a little bit for employees to where you’re not seeing specific comments, but maybe grouping by theme and showing high level charts.
Kim: It’s going to depend on the culture of the organization. Some are pretty open and honest, but especially in an employee meeting, I would avoid individual comments because that would erode the trust and the promise that things would be confidential. This would be an opportunity to show common themes at a high level and focus on what management and leadership are going to do to improve things, whereas the executive team and the management team are going to want to dig into the details to make sure that they are putting in place the right fix.
Brandon: So we have built the survey, we’ve got it out to the employees, we’ve done the analysis, now what about action? What do you do as a management group? You have this feedback, now what do you do with it?
Kim: That is the key, putting it into action. What I’ve seen work well, and what I’ve actually been given awards for doing, is putting together an action plan with the management and leadership team involved. As an HR person, I’ve put that together and been the one to roll up the data and suggest what I think the action steps need to be. I have gotten the management and executive team to buy-in to it so we can then take it to the employees. It’s basically a project plan of the steps we are going to put in place by X time.  I’ve even held focus groups, which got rolled out into smaller departments. The payroll team might have had issues that the benefits team didn’t, so we would have focus group with specific employee groups or departments to really hone down on their issues and to ask is this is the fix they are looking for, do they think this will help, and then get their stamp of approval.
Brandon: I understand the need for focus groups if there’s a lot of negative feedback and if employees are saying things need to be fixed, but what about an organization that has glowing and positive feedback? You obviously want to stay ahead of the curve and always make sure you are improving the culture and that employees are happy, so you always have to stay ahead. But with survey feedback that is overly positive… what do you do in that case?
Kim: You might want to do focus groups around that, too. It depends on what you think the pulse is; maybe throw a party if everyone really is that happy! I haven’t been in a position to see too many of those honestly, I’d love to be. That would lead me to ask, “Is this really valid data and are we really doing that great of a job?” and if so then we should celebrate!  Maybe there’s something else other places in the organization we should be working on, knowing that we have a really satisfied bunch.
Brandon: Earlier when we were talking about the surveys, you mentioned there are different types of surveys you can take. Some of them are satisfaction based, some of them are engagement based, and some are compensation. What do you think the difference is between a satisfaction survey and an engagement survey is? If there even is a difference.
Kim: Depending on what book you read, there are some differences. I see that satisfaction is an old word that was used around 10, 15, 20 years ago, and now the word we are seeing more is engagement. I have seen some books reference to satisfaction being more tied to job content or an employee’s feelings and attitudes, and engagement more tied to their engagement with the organization – how do they like their manager, do they like the job that they do. My feeling is that it’s a fine line, and rather than mince words, I would say engagement is the word of the day and ultimately we are getting at the same thing – how happy are folks with their current job, the organization they are a part of, and what can we work on to improve their desire to stay with our company.
Survey MonkeyBrandon: What are some resources for listeners out there? What are some ways they can either get started on a survey or learn more about how to do surveys and how to set them up? I imagine it’s sort of technical, you have to know what you’re doing and how to set it up, and then you have to ask the right questions. What are some resources out there?
Kim: I think the Internet is an amazing resource and it can be a good way to get an overview of what’s involved in a survey. Then also, I’d say your local university. Especially in the Portland area, Metro area and Vancouver, we have a lot of great colleges. They often have statistic majors, HR majors and math majors who are looking for projects that will build their resume. Sometimes that can be a really great and free resource!
Brandon: I never even thought about that, that’s interesting.
Kim: Yes, and psychology majors and counselors are great at asking questions, so that’s a great resource. Then of course there are Human Resources companies and consultants out there that can help as well.
Brandon: Our guest today has been Kim Smith. Thank you for being a part of the program, I appreciate it.
Kim: Nice to be here, thank you Brandon.
Join the conversation on Twitter @XeniumHR with #hrsmbPodcast and follow Brandon Laws