“Not everything that can be counted counts, and not everything that counts can be counted.”

When Einstein uttered these words little did he know that he was stating the case for techniques like Most Significant Change (MSC).

MSC is a simple process for helping senior decision-makers develop a gut feel for what an initiative has achieved. It’s not a replacement for gathering and analysing the numbers. Rather is a supplemental evaluation that helps to systematically develop decision-makers’ intuitive knowledge. And research shows that many of the decisions we make are based on our judgements and intuitive, so it’s a part of our knowledge we mustn’t ignore.1 2

Here’s how MSC works. It can be done in 4 steps.

STEP 1 – COLLECT STORIES OF SIGNIFICANT CHANGE

The process starts by asking two simple questions of the people affected by the initiative of interest.

1. What is the most significant change that happened since the initiative started?

2. Why is this change significant for you?

STEP 2 – IDENTIFY AND ASSEMBLE THE DECISION-MAKERS WHO NEED TO KNOW WHAT REALLY HAPPENED AS A RESULT OF THE INITIATIVE.

This step is crucial to the success of the evaluation and consists of the evaluation designers asking the question, “Who needs to know, in their gut, the impact this initiative is having?”. These decision-makers could be at any level in the organisation, in any location. The evaluation designer then arranges the decision-makers into groups of 6-8 people and arranges for these groups to meet for 90 minutes or so to consider the significant change stories collected in Step 1.

STEP 3 – SELECTING THE MOST SIGNIFICANT CHANGE STORY

When you have your decision-makers in a room the facilitator guides the group in a discussion about 4 to 6 of the stories that where selected. In fact, we encourage the group to read each story then argue why they think a story is most significant. This discussion helps embed the stories in the minds of the participants while raising issues of strategies and implementation. The participants experience a lively debate and get to know one another and the issues affecting people in the field. Most importantly they develop an intuitive understanding of the impact the initiative is having. At the end of the session the group agrees on a most significant story and describes why they selected it. They also identify actions they will take to reinforce the good things that are happening and disrupt the undesirable outcomes.

The result is communicated to the original storytellers. The most significant change story from each group is then made available to the next level in the organisation, such as an executive group, who repeats the process with the subset of stories.

STEP 4 – MAKING THE STORIES AND WHAT WAS SELECTED AVAILABLE

The evaluation concludes by collating all the stories and creating a document that includes which stories were selected and why.

Invariably lessons are learned during the process and these ideas can be then fed into a continuous improvement process.

The selection process is frequently scheduled to occur on a regular cycle. Organisations that use MSC often select a period of between selections of 3-6 months to evaluate ongoing change.

Additional resources

How to conduct an MSC selection workshop

Dart, J. and R. Davies (2003). “A Dialogical, Story-Based Evaluation Tool: The Most Significant Change Technique.” The American Journal of Evaluation 24(2): 137.

A short history of MSC

Zahmoo – software for supporting MSC projects

References

1. Klein, G. (2003). Intuition at Work. New York, Currency Doubleday.

2. Westen, D. (2007). The Political Brain: The Role of Emotion in Deciding the Fate of the Nation. New York, PublicAffairs.