3 min and 37 sec to read, 905 words

A key problem in public policy work is to articulate the case for intervening and for engaging in an issue – and this is mirrored by how challenging it is to actually report on and measure impact. A classical failure mode is the policy team working hard to get a proposal to a good place, succeeding, and no one realizing what the actual outcome would have been but for the intervention the team made – massively discounting the work.

This ambiguity is natural – politics and policy are messy systems – but there are a few things we can do to improve.

One important tool to add to your craft is the outcome scale. This is a list of outcomes from most impact (100) to least impact (0) of some kind of upcoming decision or process. Let’s say that we are looking at a new bill about the technology our company is developing. The bill is in early stages, maybe it has not even been formulated as a bill yet, but we do know that legislation is likely to emerge in the coming months — what should we then do to prepare, and ensure that we can share with leadership how we should be thinking about this?

The answer is to list the possible outcomes from almost no effect to severe impact on the company’s optionality and viability. This list of outcomes is your scale – and what you will be managing.

The different packages here – A to E – are collections of regulatory interventions and legislative restrictions you can predict are likely to be a part of the overall deal. The way you should think about this is that these packages represent the spectrum of the possible – we are not looking at probabilities at this stage – and so you need to be expansive. E – the maximum outcome – may be a ban on the technology, strict liability for any deployment et cetera, and once you have done a best effort ordering of the outcomes you use this to validate with the leadership of your business, and ensure that the business agrees with your assessment of impact, the bundling you will have done and the order of the bundles. Then the decision you should seek is what bundle to aim for.

In order to make this decision, however, you need to add a piece of data for leadership to consider, and that is the cost it will take to get to the outcome in question. This is another assessment you need to make, and one way of doing that is to map all the stakeholders and try to imagine their relative power over the issue, and where they would prefer to land. This will give you a sense of the probability distribution over the different outcomes – and that can look very different for different issues. As you overlay the probability you can get something like this:

What you then need to understand is if the probable outcome is within the acceptable limit, and if there is anything you can do to affect the overall probability distribution here. These curves aggregate all the different stakeholders, but you can also break them out to understand their preferences:

Here we model 3 different stakeholders: A strongly prefers outcome B, C prefers outcome C or D, but is hesitant to go all to E, and B has a much weaker engagement overall over the different outcomes. This models both salience of the issue and preferences for the different stakeholders — and one analysis you will want to make at some point is how robust these preferences are (i.e. can you make B care much more about getting outcome C?).

Finally, you also want to make it clear how much it will cost in time, resources and effort to get to the different outcomes. Here, again, issues have different profiles. Look at issue X and Y in these diagrams:

Issue X is really costly to get out of outcome E, but it is almost equally costly to get to B and A. Note that this does not need to be the same as the probability — the probability that an issue ends up somewhere may be due to simple things, and there may be low-cost interventions that can shift the outcome. But when the cost looks like this, the probability is reasonably higher for an E outcome.

Issue Y is different:

Here the worst case outcomes is fairly easy to avoid — but it gets tricker getting from B to A.

These very simple models serve two different purposes. First, they allow you to get to a shared view on the spectrum of outcomes (this avoids surprises), the desired outcome and the cost and probability of getting there. Modeling other stakeholders also gives you an engagement map (for more on stakeholder mapping see here).

Techniques like these can also be used to predict the likely outcomes of different decisions. A robust version of this method can be found in the works of Bruce Bueno de Mesquita’s interesting book Predictioneer. He takes a model like this and adds game theory to model the likely outcomes — and this makes it technically harder and more demanding. The use of outcome scales I prefer is to help create this shared model of reality – and then prioritize work, and decide when to engage and how.