The lure of numbers–employee engagement is good for business
Gallup has done a survey of 1000 US employees investigating the relationship between engaged employees and innovation. At first glance it seems impressive. There are lots of numbers, a couple of graphs and even a statement at the bottom of the article describing the survey limitations. The results, however, hinge on their definitions of employee engagement (see pretty graphic).
So how did they determined who fell into which engagement category? This seems to be a vital missing piece. There is no indication of the questions they asked or the scales they used. Without this information the rest of the ‘data’ is nonsense to me. Here are some of the findings.
When GMJ researchers surveyed U.S. workers, 59% of engaged employees strongly agreed with the statement that their current job “brings out [their] most creative ideas.” On the flip side, only 3% of actively disengaged employees strongly agreed that their current job brings out their most creative ideas.
The study also showed that engaged workers were much more likely to react positively to creative ideas offered by fellow team members. When asked to rate their level of agreement with the statement “I feed off the creativity of my colleagues,” roughly 6 in 10 engaged employees (61%) strongly agreed, while only about 1 in 10 actively disengaged employees (9%) gave the same answer.
In the race for evidence-based management I imagine people are taking these results and believing what they read and quoting the figures (fully referenced of course) in business cases as if they are gospel. Perhaps I’m missing something but without an understanding of how these categorisations are made it’s difficult to assess the results’ veracity.
I would love to hear what Bob Sutton thinks of these types of ‘evidence-based’ pronouncements masquerading as research.
Shawn, author of Putting Stories to Work, is one of the world's leading business storytelling consultants. He helps executive teams find and tell the story of their strategy. When he is not working on strategy communication, Shawn is helping leaders find and tell business stories to engage, to influence and to inspire. Shawn works with Global 1000 companies including Shell, IBM, SAP, Bayer, Microsoft & Danone. Connect with Shawn on:
Comments
Comments are closed.
Hi Shawn
Gallup’s work on employee engagement is documented in a number of places including “First Break All The Rules” (1999) by Marcus Buckingham and Curt Coffman. Based on indepth surveys (at the time 1,000,000 employees and 80,000 managers in over 400 companies), Gallup identified the factors that determine whether people are engaged, not engaged, or actively disengaged. They came up with 12 questions: known as Gallup’s Q12. (At the very foot of the GMJ article you link to is a reference to the Q12.). Appendices in the book explain how they determined the 12 questions, and includes an excerpt from their meta analysis.
The 12 questions are
1. Do I know what is expected of me at work?
2. Do I have the materials and equipment that I need in order to do my work right?
3. At work, do I have the opportunity to do what I do best every day?
4. In the past seven days, have I received recognition or praise for doing good work?
5. Does my supervisor, or someone at work, seem to care about me as a person?
6. Is there someone at work who encourages my development?
7. At work, do my opinions seem to count?
8. Does the mission or purpose of my company make me feel that my job is important?
9. Are my coworkers committed to doing quality work?
10. Do I have a best friend at work?
11. In the past six months, has someone at work talked to me about my progress?
12. This past year, have I had opportunities at work to learn and grow?
(c) 1992-1999, The Gallup Organization, Princeton, NJ. All rights reserved.
If people rate a very low answer to all the questions, they are actively disengaged.
Since 1999, Gallup has carried out many more surveys using their G12, so millions have now been exposed to their survey around the world. Gallup carried out an Australian survey in 2004.
Although Gallup is not so well known here in Australia, they are very well known and highly regarded in other countries. We find the Gallup work resonates with clients; the questions make sense to them. We refer clients to the Gallup research selectively, depending on their interest and needs.
I look forward to others’ views – it would be great to hear other perspectives!
Amanda
I’m with you! “…without an understanding of how these categorisations are made it’s difficult to assess the results’ veracity.”
Yet the subject is a vital one. Daily I ponder, puzzle and muse over what it is to have a fully engaged workforce or at least a difference making percentage of employees bringing their creative best.
I’ve been a bit busy lately and have not been reading your blog as faithfully in the past. That was my mistake.
Thanks for keeping such helpful conversations alive and well here at Anecdote!
Thanks Amanda for shedding light on what seemed to be a massive oversight. I see now that you had to be ‘in the know’ to make sense of this article. You would think they would put a link explaining the Q12 on every article that used the concept of employee engagement.
Do you know whether Gallup has done any ‘ground truthing’ of the categorisation? That is, gone out and seen whether those people the survey say are engaged or otherwise are really like that.
The questions make sense to me. They would also form the basis, with modification, of story-eliciting questions. For example, tell me a time when you regularly received recognition from your boss and when you didn’t; what happened?
Shawn – I’m with you on your initial reaction. I think you’ve pointed out one of the basic problems with communication via the web – it tends to be disjointed and doesn’t always have enough referrals to other sources that help people evaluate the credibility of some fairly complex topics.
Hi Shawn
I think the ‘ground truthing’ comes from Gallup’s starting point: indepth interviews, and then analysed against performance indicators to derive their Q12. However I don’t konw if they ‘ground truth’ in relation to these recent research projects, such as the innovation one you cite.
You are absolutely right: the Q12 are great story-eliciting questions. The whole process in the 90s started out as lots of anecdotes, stories etc.
Nb: Hewitt and the Corporate Leadership Council are two other major players in the Employee Engagement measurement field.
Amanda
A couple of years ago I did a project for a bank that had Hewitt conduct their employee engagement survey and they found people didn’t understand how they were connected to the larger organisation. How does my job affect other parts of the organisation? How do I fit in?
We did a narrative project to tease out the issues and they designed and implemented a range of interventions.
I’d highly suggest that everybody take the claims of Gallup, or those of any other consulting company for that matter, with a very large grain of salt. Virtually every consulting company out there– Gallup included– has derived some measure of engagement that, “through extensive analysis of rigorous research that spans over x-million employees across x number of firms”, will assess a given firm’s level of employee engagement with the promise of providing massive returns in terms of the company’s financial performance. While the correlates that become apparent through these consulting firms’ analyses are empirically supported, the causation that consulting firms imply are not.
Specifically, virtually no academic research indicates a causal relationship between anything on the Q12 list and the bottom line performance of a firm. In fact, a substantial amount of literature indicates that the casual relationship between engagement and performance is likely reciprocal and, at least in some cases, financial performance predicts engagement and not the other way around (see Schneider et al. (2003). “Which comes first: employee attitudes or organizational financial and market performance?”. Journal of Applied Psychology, Vol. 88, No. 5, 836-851).