Saturday, 23 August 2014

Recognising the Counterfactual

When we make changes to (try to) improve productivity, we subsequently measure results and assess impact.  (Or we should if we want to evaluate our own performance and impact.)

However, in many complex situations, we cannot be clear that what we have done has resulted in the changes we observe.  What would have happened if we had not intervened?  This is the 'counterfactual'.  How do we measure it - so we know the true impact of our changes?  Well, often, of course, we can't.  But sometimes we can extrapolate from observations we made earlier - and make an informed (and hopefully intelligent) 'guess' at what might have happened.

Sometimes, we might have a 'parallel' situation elsewhere we can continue to observe (rather like a 'control group').

The important point is to realise that in complex situations,we have to be careful about assuming (or claiming) that all change has resulted from our actions. Otherwise we can over- or under-estimate the impact we have ... and subsequent decisions might be based on this imperfect - and incorrect - knowledge.

No comments: