Your CEO has asked you to conduct an organization-wide employee engagement survey. You have compared vendors and chosen a reputable firm. The survey results indicate worktime flexibility has a strong, positive correlation with employee engagement in your organization. Your leadership decides to invest major effort and funds into an overhaul of policy and technology to enable more work flexibility. A year after the overhaul, analyses show increased flexibility has not improved employee engagement. What happened?
A different factor was more important than flexibility.
Strategy can fall flat or even backfire when survey results are misinterpreted or incomplete. To avoid costly mistakes and get the most value out of employee engagement surveys, it's crucial to understand how your provider handles several issues:
- The limitations of simple correlations and how the provider finds the rest of the story
- Which conceptualization of “engagement” is measured and the implications of it
- Whether the survey focuses on conditions, sentiments or both
- The basis for results interpretations or action recommendations
The limitations of simple correlations
Incomplete information. Perhaps the most important limitation of correlational data is that a simple correlation can rarely tell the whole story. There are usually many, maybe dozens or hundreds, of variables at play in any work context. A simple correlation only measures the relationship between two of the variables. Other analysis tools (such as multiple regression or structural equation modeling) can better explain the influences of many variables at the same time. Those analyses will likely provide a more accurate picture of what employees are thinking and feeling.
Causality. Anyone who has attended an introductory science course (and most of those who haven’t!) has likely heard that “correlation does not equal causation.” Many survey results and the action recommendations that follow, however, are presented in terms that suggest some causal relationship.
Although a single survey can rarely establish support for any causal relationship, the research literature does include some evidence that certain engagement factors can cause changes in other factors. In addition, a provider might have some proprietary evidence for causal findings. What is important for the leader to understand is the provider’s basis for causal findings or recommendations.
Multiple report formats. Although graphics can be arranged in many ways that help interpret findings, it's important to recognize when the underlying result is a simple correlation.
No matter what a graphic seems to imply, however, if the underlying result is a simple correlation, the limitations remain the same. No causality can be inferred, and the story is likely incomplete.
Correlation findings gone awry
Consider our example of the failed implementation of worktime flexibility. If a survey indicates more work flexibility is correlated with higher engagement, that correlation alone does not reveal how other variables are working. Your employees with more flexibility may feel more autonomy in their work, and the autonomy feelings are the actual cause of higher engagement. If new policies for flexibility were rolled out in a top-down, dictatorial style and included strict controls for how employee hours will be tracked, employees could actually feel less autonomy and lower engagement. The real story the survey should have told was that autonomy was more important than flexibility, and work flexibility was just one possible way to address autonomy feelings. Employee autonomy could have been enhanced with a simpler, less costly intervention, which would actually have been more effective for improving engagement.
The definition of engagement itself
Interestingly, among practitioners there is a wide variety of views on what employee engagement actually is. How it is defined should determine how it is measured on a survey, which in turn has important implications for any findings.
There is general consensus among researchers that engagement is qualitatively different than job satisfaction or organizational commitment (two other concepts that are sometimes called engagement). A clear definition of what is measured, and the implications, is vital to your survey results.
Conditions vs. sentiments
Employee engagement surveys differ in their focus on actual workplace conditions vs. employee perceptions, feelings or motivations (which can collectively be referred to as sentiments). Understanding the difference is important for getting clear, interpretable and actionable results.
Measuring actual conditions can diagnose the management practices in different departments or gather information about employees’ understanding of policies or programs. This type of tactical intelligence is sometimes included in employee surveys.
The central benefit of surveys that focus on employee sentiments is more than 130 years of psychological research. This research has uncovered many basic properties of human thought, emotion and motivation, and it has established some sentiments, feelings and motivations that affect engagement levels. If a survey does not measure sentiments, these insights cannot be directly brought to bear on your results.
A Leader's Guide to Change Management (Whitepaper)
Learn how many corporate change initiatives ultimately fail despite good intentions.
Effective Team Building (Whitepaper)
Building effective teams is an intentional process and often a mindset shift.
Leadership Excellence Through Self-Awareness (Whitepaper)
Explore the increasing importance of self-awareness as it pertains to subtle motivations that can sabotage your success as a leader.
A Leader's Guide to Overcoming Obstacles (Whitepaper)
Learn the steps it takes to become more effective to understanding and acting on obstacles.
Take the lead. Talk with us today.
Let's discuss your specific goals and needs.