Defence CommitteeWritten evidence from Mike Young, Decision Workshops Ltd
HOW TO UNDERSTAND, PLAN, AND FORECAST FUTURE POLITICS:
EVIDENCE TO SUPPORT THE USE OF ROLE PLAYING WORKSHOPS USING CONFRONTATION ANALYSIS
1. Workshops using role-play supported by Confrontation Analysis can be invaluable in understanding planning and forecasting complex political situations, such as that in Libya.
2. We submit as evidence to support this in the form of the results of a workshop that was held at Cranfield University on 19 July.
3. The workshop participants found that they could get a good understanding of the behaviours and reactions of the other parties involved in the conflict by using this method.
4. The use of role play to produce accurate forecasts of the outcome of conflicts is a fully disclosed evidence based procedure. A summary of the academic evidence supporting this is included.
5. Because history is genuinely chaotic the workshop cannot be expected to totally mirror future history, but it will produce output that will seem a plausible alternative in the light of subsequent events, and will be an excellent guide to decide on the effectiveness of policy and look at unforeseen consequences.
6. We recommend the use of such workshops to test out and develop political strategies against a realistically behaving opposition, to help understand the potential future history and long-term consequences of political actions.
7. This submission has been made by the company “Decision Workshops”. The company has been working with Cranfield University; the Defence Academy of the UK and DSTL on workshops to understand, plan and forecast the future history of Libya. The workshop reported on here was held a Cranfield University on 19 July. A follow up workshop will be held at DSTL on 6 October.
8. This document will support the final four of the points requested by the enquiry: those relating to the future of Libya.
The “end game”: what would a successful outcome look like and how do current operations contribute to achieving this?
The extent to which the UK and NATO are interacting with and supporting the opposition forces in Libya;
Whether the necessary planning is being done to ensure the long-term stability of Libya when the military effort is complete; and
What is our exit strategy?
9. These are interesting questions because they require an understanding of and ability to forecast the future political situation in Libya, and knowing how to “play the politics” correctly.
10. This document presents evidence that role playing workshops supported by Confrontation Analysis would be a way to do this. This will help the UK to avoid involving itself in foreign ventures without understanding the long-tern consequences of its moves.
The Present Weak State of Political Forecasting
“A politician needs the ability to foretell what will happen tomorrow, next week, next month and next year. And to have the ability afterwards to explain why it didn’t happen.”—Winston Churchill
11. Influential experts and the political leaders they advise believe that they can make useful forecasts about complex and uncertain situations, such as the one we currently see in Libya. But simply observing the disagreements of the experts should make us question whether relying on their opinions is a good approach to making important policy decisions.
12. What can be done about it? How can we better understand and forecast the behaviours of those involved in confrontations such as the one in Libya?
Evidence Supports Role Play Rather than Judgement for Forecasting Accuracy
13. There has been much academic study that supports the view that just relying on people’s judgement is a very poor method of forecasting outcomes of confrontations. The evidence shows that judgement, is only marginally better than chance at predicting outcomes: Game Theory, a mathematical method popular at the time of the cold war, fares no better. However when the participants take part in a role play then it was found the ability to forecast was dramatically increased. This evidence is summarised at Appendix 1.
A War is a Chaotic System
14. But future history cannot be forecast totally as it is what is known mathematically as a “chaotic” system—one where a small, usually insignificant event just could have immense consequences further down the line. This means the “prediction” of history in detail, like that of the weather, is not possible. The analogy often quoted to describe a chaotic system is that a butterfly flapping its wings can eventually determine if a hurricane forms or not.
15. Examples of small, almost random, events that may alter the future history of the war in Libya (at the time of writing 9/9/2011) would be the capture or escape of Gaddafi or the assassination of members of the NTC or their supporters. Obviously the success or not of these events may depend on very minor unpredictable previous events, such as a delay of ten minutes, and cannot be forecast with certainty. This makes the detailed history of the war intrinsically uncertain.
16. What can be done, though, is to understand how others will react to different events should they occur. Even against this chaotic background, moves will only be successful if they make the correct assumptions about the behaviours and reactions of other parties.
17. A historical example of this is the 1914 assassination of Franz Ferdinand. Before the event we could not have predicted with certainty if the assassination would be successful, (in fact it was very nearly unsuccessful). However the reactions of the other parties involved should have been thought through more carefully. The subsequent unfolding of the political events that led to the First World War is often looked on as inevitable.
18. A method known as Confrontation Analysis,
19. Confrontation Analysis acts to keep the workshop participants focused on the important issues and acting in role. The role play is not “free-form”, but rather structured by this process.
20. Developing a scenario for a Confrontational Analysis workshop is an inherently useful exercise as it helps quantify the factors involved in decision making. It is an excellent vehicle to incorporate the opinions of subject matter experts.
21. The behaviour of somebody like Gaddafi may be irrational in our eyes, but it is not random, in the way throwing a dice would be. It can be forecast provided that you understand his constraints, his values, and nature of the dilemmas he is under. Confrontation Analysis is a tool that specifically helps to help build this understanding.
22. The major benefit is in understanding how and why others will react in the way they do. Unlike some other methods such as game theory a “rational actor” is not assumed, but rather the emotions and irrationalities are seen by Confrontation Analysis as a product of the particular dilemmas the character is under.
23. Participation in interactive activities such as role-play workshops or war-games is recognized by the American government as a useful educational exercise to help prepare decision makers for the different scenarios.
Cranfield University Role-Playing Workshop
24. Cranfield University hosted a workshop on 19 July at the Defence Academy of the United Kingdom at Shrivenham. This workshop used Confrontation Analysis supported by role play to investigate the possible future of Libya over next couple of months. In it a selection of military, ex-military and civilian subject matter experts role-played the political and military moves of the different parties that were involved in the Libyan Civil War.
25. We find this form of workshop to be potentially a very powerful tool for the Ministry of Defence, the FCO and the diplomatic service to develop and rehearse political strategies.
26. To support this claim we submit as evidence the output of the workshop, which can be compared with the actual unfolding of events.
Output from Confrontation Analysis Role-Play Workshop on 19 July
27. At the workshop we developed a forecast of the future history of Libya, in terms of decisions and actions made by the different parties involved. This has been compared with actual events as they unfolded, to see how accurate it was, and if the parties behaved in ways we had expected or foreseen.
What was different?
Cease fire and evacuation of Tripoli
28. The main difference between the workshop and reality was that in the workshop Gaddafi was able to get the UN and the NTC to agree to a cease fire early on. He did this by offering to evacuate Tripoli as a precondition for that cease fire. The deal was that in exchange for a cease fire Gaddafi would withdraw his forces from Tripoli to his heartland in Sirte and Fezzan (the southern desert of Libya). This is almost exactly the areas he still controls at the time of writing, (9 September).
29. The ceasefire was accepted because the prize offered (Tripoli, which was at the time still in Gaddafi’s control) was large enough to be “worth” the cost of the cease fire to the NTC. Although Gaddafi has always been willing to accept a cease fire, at no time has he offered a large enough additional incentive alongside to actually get it.
30. A couple of days after the workshop William Hague started hinting that a cease fire and internal exile may be an acceptable solution for the UK.
31. With the benefit of hindsight, it can be seen that if Gaddafi had done that, then he would be in a much better position politically than he is now. With a cease fire Gaddafi would still be in de facto control of his heartland and able to negotiate from a position of much greater strength than he has now.
32. An interesting point is that the man playing Gaddafi had actually taken part in a rehearsal workshop a week or so before (and played Gaddafi), and so it is reasonable to assume he had learnt from that rehearsal what the best political moves would be. Perhaps this illustrates the way lessons from this process could alter political policy. Interestingly, the workshop inadvertently found the best solution for Gaddafi, rather than the best solution for the western powers!
Some of the names were different
33. Sometimes some the names were different. In particular the workshop predicted in fighting within the NTC with clear winners and losers, but got the names of those winners and losers wrong. In the workshop general Abdul Fatah Younis managed to survive, and indeed increased his power, it was other members of the NTC that lost power to him. In real life he was assassinated a couple of days after the workshop. Without wanting to stretch a point too much, the analyst who played Abdul Fatah Younis also happened to be the only other participant at the workshop who had also took taken part in the rehearsal!
34. Similarly in the workshop the Berbers attacked and took the town of Zuwarah, rather than the almost identically named town of Zawiya, a few km down the road, which was actually taken on 13 August. Both of these moves had exactly the same strategic effect, cutting off Gaddafi’s supply line to Tripoli from Tunisia.
What was similar?
35. Many of the news stories of the few weeks after the symposium resonated as paraphrases of things that happened both in workshop and in real life. In particular all the following events happened in both the workshop and in real life:
36. Successful Berber military offensive northwards towards the coast. In both the workshop and real life the Berbers struck north from the Nafusa mountains to capture a town on the coast, cutting off Gaddafi’s supply line from Tunisia.
37. Tripoli falls with little bloodshed. Unlike other times when cities were involved in civil wars (such as Beirut or Sarajevo) the capture of Tripoli was achieved without major bloodshed, and relatively swiftly. In the workshop it was evacuated, in real life it fell relatively easily. In both the workshop and real life most of Gaddafi’s forces melted away.
38. Tripoli falls to Berber/Misrata forces rather than rebel forces from Benghazi. In both the workshop and in real life, Tripoli fell to rebel forces from Misrata and Berbers from the Nafusa mountains. The army from the major rebel held area, Cyrenaica, was not able to reach Tripoli as Gaddafi still held the town of Sirte.
39. Hard core of Gaddafi forces retreat to heartland. In both the workshop and real life, Gaddafi’s forces abandoned Tripoli and fled to his hinterland, (Sirte and the Fezzan desert in the south) where he still maintains considerable popular support.
40. Forces from Benghazi that do arrive in Tripoli come by boat. In both the workshop and real life a small force came from Benghazi to Tripoli by boat.
41. Some armed citizens of Tripoli join liberation of Tripoli, but no large massed unarmed protests. The citizens on Tripoli knew that Gaddafi will not tolerate even unarmed protest, therefore their only option was armed rebellion, but until rebel forces entered the city, there were a large number of Gaddafi paramilitary to deter open rebellion.
42. Power vacuum in Tripolitania. The workshop ended with a power vacuum in Tripolitania, identical to the one that has formed in the last couple of days.
43. Internal exile by Gaddafi seen as acceptable to the UK. Gaddafi put forwards the concept of himself going into internal exile. A couple of days after the symposium this was suggested as acceptable by William Hague.
44. Clear winners and losers in the NTC. The symposium predicted the power struggle within the NTC and its supporters with some members sacrificed at the expense of others. This has also come true with a vengeance. The assassination of General Abdul Fatah Youanis, and the subsequent reshuffling of the council indicates this has happened.
45. Increase in numbers in the NTC, but this is not matched by devolution of power, which continues to be at the top. The NTC is continuing to increase in numbers, believed to be about 40 at the moment, with plans according to the economist, to expand to 95.
46. NTC members supported by the West in exchange for oil and mineral contracts. The NTC has said that it will honor these western contracts even though they were set up under Gaddafi.
Pie Chart of Results
Every individual decision made in the workshop was recorded and the success of the predictions (as of 9 Sept 2011) are shown below. Full details are available at http://www.decisionworkshops.com/#/forecasts/4553769632.
EVIDENCE THAT ROLE PLAY IS THE BEST WAY OF FORECASTING OUTCOMES FROM CONFLICT SITUATIONS.
47. The evidence for this is outlined below:
48. The experiment was based on people trying to estimate the results from a set of eight obscure real life historical confrontations where the actual outcome was known.
49. To see how good a prediction method was, the researcher mixed the actual outcome with other plausible outcomes that could have occurred, to see how good the prediction method was at selecting the actual outcome. Names were changed to further prevent recognition.
50. Four methods were used:
51. Chance (guessing): As there are six possible outcomes for the “Artist’s protest” there is a one in six (17%) chance of getting the right answer just by chance: as there were four outcomes for “55% Pay Plan”, “Telco Takeover” and “Personal Grievance” there is a 25% chance of getting the right answer by chance, and three outcomes for the others giving a 33% chance of chance. Altogether this gave an average score of 28% just by guessing.
52. Unaided Judgement: Working from a brief and asking somebody what they think actually happened. This produced worse results than guessing for several scenarios, and averaged only a couple of percent better than just guessing. It confirms the common feelings about the usefulness of the kind of meetings known as BOGSATs (“Bunch of guys sat around a table”).
53. Game theory: Asking people with an expertise in game theory usually yielded a slightly better result than unaided judgement did, but the game theorists scored 0% in the “Telco Takeover” scenario and as a result their average was diminished.
54. Role-play: People asked to participate in role playing produced considerably better results than the other two methods, and always better than guessing.
55. In conclusion, taking the modal result of a series of role-plays (the one that occurs most often) will give a very high probability of forecasting the actual outcome of a confrontation.
56. Note: The “Water dispute” scenario was a disguised military/political confrontation, representing the conflict in 1990 between Turkey and Iraq under Saddam Hussein over Turkey reducing water flow to the Tigris and Euphrates rivers.