Home

Decision Maker User Guide - Defence Science and Technology

1. Maximise ae Figure 29 Wizard example add criterion 3 to Maximise Reliability When the information has been entered click the Add Criterion button Next click the New Criterion button Enter the information from Table 10 into the Wizard It should appear as shown in Figure 30 Table 10 Wizard example Criterion 4 information Objective Minimise Supply Time Criterion Title Minimise Supply Time Type A Spares Criterion Description The upper limit of the 95 confidence interval of the Mean p Time To Supply MTTS of Type A spares in hours hrs Unit of Measure MTTS Preferred Direction UNCLASSIFIED 26 UNCLASSIFIED DSTO GD 0681 Add Criteria to Minimise Supply Time Objective Title Objective 3 of 3 Criterion Title Minimise Supply Time ape Minimise Supply Time Type A Spares SS Vavigation ae a Objective Description Criterion Description The best spare part supply arangement shall have a short supply The upper limit of the 95 confidence interval of the Mean Time time To Supply MTTS of Type A spares in hours hrs Criterion Control Sub Objective OF Unit of Measure Enable Subjectivity 0 1 Maximise Perfomance wt MTTS hrs w e Direction of Preference Figure 30 Wizard example add criterion 4 to Minimise Supply Time When the information has been entered click the Add Criterion button Next click the
2. Refine Problem Published Processes a PrOACT CRITIC Process Technique v Semi pa Structured amp Problem Decision a Structured _ p i _ gt m Interpretation Objective Formation Analysis a Problem Decision Support Solution Facilitator OO Build User Build Qualitative Generate Profile Model Reports Provides Tractability Figure 1 Overview of Decision Maker Decision Maker is a software implementation of the Criteria Importance Through Intercriteria Correlation CRITIC objective decision making technique 1 This technique has a wide range of applications in all areas of the ADO and other government and public organisations It can be utilised in most multi criteria decision making problems that use a set of criteria objectives to identify the most preferred alternative in a set of alternatives In most problems there are conflicting criteria objectives and therefore complex trade offs need to be made between the alternatives Decision Maker can be used in these situations to support the selection process for example in choosing a preferred product or service satisfying the requirements criteria objectives or ranking the performance of systems components items using a combination of several performance measures The key features Decision Maker are 1 the implementation of an objective decision making technique the cardinal and ordinal ranki
3. Objective 2 This sub objective is the first of two objectives that decompose O Sub Objective tj Objective 2 in the Element Tree This objective is further C Criterion 3 decomposed by Sub Sub Objective 1 pe Shera Each objective in this branch has two criteria to evaluate the Sub Sub Objective 1 pertormance of the objectives in the decision problem Crtenon 9 C Criterion 10 Sub Objective 2 C Criterion 5 f Criterion 6 9 Objective 3f C Criterion i Prefered Direction C Criterion 3 Maximise Altematives Cardinal Score Criterion 9 Citenond Akemawa t 0 936 a Altemative 2 Altemative 3 Weight Ready o Quick access menu buttons Data Grid The data grid is used to add data to the problem s criteria and to view the results of analysis Refer to Section 6 3 1 for more information on using the data grid Element Properties This window provides quick access to the key properties of the element selected in the Element Tree You can edit the properties in this window Figure 54 Data Model to add data and analyse your decision problem 6 2 1 Adding Data using the Data Grid The Data Grid in the Data Model can be used to add data to your decision problem This erid functions like a table in a database and data may be copied or pasted between applications such as Microsoft E
4. Supplier 14 A O 0 1 Maximise Performance 2 nthe O 0 1 1 Maximise Reliability Sg Supplier 2 Supplier 3 Supplier 5 Supplier 12 Supplier 3 Supplier 1 Supplier 8 Supplier 6 Supplier 11 Supplier 15 Supplier 13 Supplier 10 Supplier 7 3 ied N Shows the objective Maximise Reliability where a 2uppHer P P p a3 Supplier 16 Supplier 7 dominates Supplier 9 For the objective a i 4 Minimise Supply Time Supplier 9 dominates Supplier 4 Suppler Supplier 16 Supplier 14 Minimise Supply Time For Supplier 13 there is a case of partial domination Supplier 2 occurring against Supplier 7 and Supplier 9 Supplier 3 Supplier 5 Supplier 12 Supplier 9 gt gt Supplier 1 3 Supplier 13 3 Supplier 7 Supplier 4 Supplier 10 Supplier 1 Supplier 8 Supplier 6 Supplier 11 Supplier 15 Supplier 13 Supplier 10 Supplier 7 Supplier 4 Supplier 16 Supplier 14 w x 1 gt Q G rev ew gt amp G ie SSeS amp amp VOV OY Cr Cv Oe Ov j hm Ooy po Tee amp OHHH ri 5 Lat ef ses Ua amp er er or er er er er SS SF Fe Fe O ov ov Figure 114 Partial domination example Only Dominated This indicates the selected node Alti fi a o FE Supper 2 Supplier 14 Supphet Supplier 11 1 1 9 Suppe mmm When the mouse is moved over an alternative
5. Criteria Maximise Reliability Type B Spares e Figure 47 Set Uncertainty Values dialog box changing uncertainty UNCLASSIFIED 39 UNCLASSIFIED DSTO GD 0681 s Data Model Select best supply arrangement i O New Objective New Criterion 3X Delete Refresh F Calculate Chart 5 Select best supply arrangement Element Properties B E C 1 Minimise Total Cost Tite b A pie a Uncertainty in Type Spares Reliability 5 0 0 1 1 Maximise Reliability Description B E C 2 Maximise Reliability Type A Spares A Uncertainty in Type 4 Spares Reliability B E C 3 Maximise Reliability Type B Spares A Uncertainty in Type 4 Spares Reliability O 0 1 2 Minimise Supply Time B E C 4 Minimise Supply Time Type A Spares A Uncertainty in Type A Spares MTTS B E C 5 Minimise Supply Time Type B Spares A Uncertainty in Type A Spares MTTS Uncertainty Type Uncertainty in Alternatives Type 4 Spares Reliability Supplier 1 0 1 Supplier 2 0 1 Supplier 3 at Supplier 4 at Supplier 5 0 1 Supplier 6 0 1 Supplier 7 0 1 Supplier 8 0 1 Supplier 9 0 1 Supplier 10 0 1 Supplier 11 0 1 Supplier 12 o1 Supplier 13 a1 Figure 48 Changing uncertainty values for all alternatives 5 5 Manually Structuring your Problem Manually structuring your decision problem in Decision Maker provides flexibility since this method allows you to build your pr
6. H Define the axe l for each Sed Problem Analysis Step 6 Enter Data for each Criteria H Analyse the Decision Problem Figure 7 The Decision Maker Wizard problem structuring process The Wizard is only accessible when all projects are closed To begin using the Wizard close any open project To do this select the File Close menu option as shown in Figure 8 A prompt will then appear on the screen asking if you wish to save the current project At this point take the appropriate action to save the project or to close without saving Then run the Wizard by selecting the File New Run Wizard menu as shown in Figure 9 Edit _ New View Project Tools Close Project m Save Save os Chrl 5 Windows Figure 8 Closing a project Advanced Snalysis Help LJ Pew EA Open Project Chrl a Tools Pun wizard Be Windows Advanced Analysis New Project Figure 9 Running the Decision Maker Wizard Help The Wizard s navigation dialog box as shown in Figure 10 will guide you through the steps for structuring your decision problem Each step creates a different set of structural element types Section 5 3 Upon completion your problem will be structured and ready for data and analysis The Wizard can only be run when all projects are closed 4 2 1 Step 1 Create a new Decision Project The first step in structuring your problem in Decis
7. Supplier 5 5 73 Suppher 13 Supplier 1 3 Suppher 16 ars oe Nee a ry Supplier 4 am i a Supplier 10 Supplier 14 Supplier 9 73 Supplier 1 3 Supplier 17 5 Supplier 8 Supplier 6 Supplier 1 Supplier 11 Supplier 8 Supplier 10 Supplier 15 Supplier 13 Supplier 7 Supplier 14 Supplier 16 Supplier 4 0 1 Maximise Performance l Alletnatives te Suppher 5 es as Supplier 2 i H3 Suppher 12 ni 93 Supplier 6 3 Suppher 10 Ea Suppher 15 F es a Supplier 9 4 i H 3 Supplier 8 T Show Top Level Analysis is not selected This section of the Domination Tree is built based on Maximise Performance and Minimise Total Cost Weights for selected elements are displayed in the Non Dominated Sets Graphical representation of the weights ranges in the Search Results table Bi ee E Or Or r UY GH Ce E E ir ti i k 4 4S a A lt a Sgr Oe 1 Atiega i pig siig a i i a a iy a ee This section of the domination tree is based on Maximise Reliability and Minimise Supply Time Figure 120 Weights Sensitivity sub level analysis 12 9 Minimum Change Distribution The Minimum Change Distribution is a graphical display of the search results It is designed to assist the decision maker in determining the range of weights that can cause a ranking reversal The Minimum Change Distribution graphical display is shown in
8. oO Maximise Reliability i Hae a supply trangement C Maximise Reliability Type A Spares Discription Maximise Reliability Type B Spares The Supply Manager is in the process of reviewing the existing lea O Mrimise Supply Time f a that supply lyps and Type hese parts to a CO Minimise Supply Time Type A Spares se Contracts aie a to expire needs to consider C Hivimies Supp Tene Type ff Spaes fuente Aai just renew the existing contracts of not pele tebe Select best supply arrangement spara pats how Spe contracta which Supply Manager problem as Kee contracts nto 44 Pretered Direction Mavimise Atematives Cordial Score Minimis Total Manmre gt Se oes yoo aw Supplier 2 0746 Soong ose 0 50 Cadinal Score Supplies 3 0 628 55000 0 073 Sones 4 0 448 75000 0 636 Sunpler 5 1 e100 H Supplier 0 702 69350 0 649 Supplier 7 0 63 20350 0 982 Supplier 8 0 72100 0153 0 168 0 038 0 202 Supplier 3 Supplier 10 Supplier 11 Supplier 12 0 67 63850 0 425 Supplier 13 a3 72100 0407 Suppher 14 0 008 80350 0 455 Suppher 15 Supplier 16 Weight Ready o Figure 6 Decision Maker s revised data model for the Supply Manager s Dilemma Figure 4 presented the structure and data that will be used in this example Also it is assumed that Decision Maker is installed and ready for operation 4 1 Using Decision Maker
9. 3 33 46 2 Supplier 2 5p Supplier 3 47 48 8 4 Supplier 4 1 6 23 49 Supplier 5 Supplier 6 F 32 IAS 2 n F Supplier 7 2 B 98 45 a Supplier 3 1 i a Ff TE 466 Supplier 3 eS ee ee a 0 AO Supplier 10 2 BW af a A Supplier 11 ee e a E A 1 A l2 Supplier 12 a Un CS Se AAS Supplier 13 1 2 8 46 ee 4 Atd Supplier 14 AVS Supplier 15 Suppher 16 o Two views are available Tabular and Graphical o The Alternative column contains the names of all the available alternatives The column headers contain numbers that correspond to the ordinal ranking O This is the uniquely generated identifier created when a Decision Maker element is added to a project e 5 times it ranked 27d e 24 times it ranked 3rd e 15 times it ranked 4th and e 6 times it ranked 5th e As a working example Supplier 5 scored the following rankings This table area shows the ranking distributions for each alternative for all the simulation runs in this example 50 simulation runs were conducted Figure 74 Ranking Distribution tabular format UNCLASSIFIED 56 Searing Range Rankin Distribution Weights Summary Ranking Occurence UNCLASSIFIED DSTO GD 0681 8 4 Ranking Distribution Graphical The Graphical tab in the Ranking Distribution window contains a graphical representation of the ranking distribution table as shown in Figure 75 The two different views shown in Figures 75 and Figure 76 are examples
10. 74262107 74262107 F426 2107 f426 2107 426 2107 paes 9274 7016 3274 7016 9274 7016 9274 7016_ 927 4 7016 9274 7016 S274 7016 9274 7016 92747016 9274 7016 _32 4 7016 9274 7016 S274 7016 S274 7016 9274 7016 9274 7016 92747016 92747016 92747016 92747016 92747016 9274 7016 9274 7016 g274 7016 92747016 S274 7016 9274 7016 92747016 274 701 T Spares tae Toe 6 56 _ te F 6 92 3 93 Spares 717 4 7 26 6 59 zat 854 835 Oe032 0 6032 0 6033 g 000101 o 0n0049 0 000038 0 000047 0 000087 0 0000ga 0 000074 0 000074 0 000075 0 00000a o 000084 0 000028 o 000002 6 000087 000085 0 000088 0 000073 Higher cost for better supply times Mid range costs with a mix of average supply times Lower costs but higher supply times Figure 131 Criteria Sensitivity trade off analysis UNCLASSIFIED 107 UNCLASSIFIED DSTO GD 0681 14 Database Management Interface Decision Maker uses a Microsoft SQL Express compact database engine for the recording and reporting of simulation results As the data in the database increases performance will deteriorate It is therefore important to perform regular maintenance of the simulation database A user interface provides a simple method for deleting all or selected simulations It also facilitates the export of simulation data for storage or transfer T
11. Diseription Maximise Reliability Type B Spares Pe ee pol 1 ey ri T Minimise Suppl i anne E Mmimce Supply Time Type amp Spares FH E Minimise Supply Time Type H Spares c New Giterion Supplier 2 Supple 3 Supplier 4 Supplier 5 Supplier B E Supplier 7 Sie u E ae Dis oO 0 a Supplier Suppler 3 Suppler 10 NaN Supplier 11 NaN Suppilier 12 NaN Supplier 1 a aN l i 1 3 a s o oO o Ig wo oo co a io a ep cH la Supplier 14 Supplier 15 Supplier 16 Weight Ready io w om i Figure 56 An incomplete calculation produces a NaN result 6 2 4 Charting Results in the Data Model Within the Data Model Decision Maker provides a tool to chart the output of the decision analysis process This is achieved by first selecting the Problem element or any Objective element in your problem s structure and opening the chart window To chart the data for any of the problem s elements follow these steps 1 in the Element Tree select the element to be charted and 2 select the View Chart menu option as shown in Figure 57 or click on the chart button in the Data Model window The scaling of the chart will be automatically adjusted by Decision Maker If you are charting your problem or an objective the chart will be scaled between 0 00 and 1 00 on the vertical axis The problem s alternatives will be listed on the h
12. Figure 4 The structure of the Supply Manager s Dilemma where MTTS is the Mean Time To Supply and MTBF is the Mean Time Between Failure 3 3 Alternatives The aim of this step is to generate many possible alternative solutions to the problem while not limiting the range of alternatives that could be considered Do not evaluate or eliminate any alternatives during this step This will occur later Some suggestions 4 for generating alternatives include UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 using your objectives and asking how they might be achieved being creative and thinking outside the square challenging the constraints and setting high aspirations a ee EXAMPLE CONTINUED Supply Manager s Dilemma PrOACT STAGE Generating Alternatives The Supply Manager is ready to generate some alternatives He starts his list of possibilities with the types of contracts that are available 1 separate contracts for the supply of each type of spare part or 2 acombined contract to supply both Type A and B spare parts He has two companies in mind Widget Inc and Gadget Inc However after talking to some of his colleagues and telephoning a few different companies he is able to add two more companies to his list Gismo Inc and Turtle Supplies Inc All four companies can supply Type A and Type B spare parts The Supply Manager now has 16 possible alternatives a combined contract for each company or separate contracts with two
13. Figure 71 Missing Hi Lo range bars To overcome this and display the missing data the plot gallery needs to be changed to the bar or line plot The line plot is simple to understand and is accessed as indicated by the call out in Figure 72 To open scoring plot gallery right click the mouse in the plot area and the menu shown in Figure 72 will appear The plot will then change to a line plot of the same data and the missing alternatives will now be visible as indicated by the call out in Figure 73 Toolbar fal Data Grid iJi Legend Box a Edit title Point Labels A a Font Properties Figure 72 Scoring plot gallery Ordinal Scoring Ranges 18 00 i i oo W a m gt 15 00 o ee 4 12 00 S o gh A a 7 Q a v i m Ordinal Score 9 00 e a a 3 00 3 0 00 O e A A O O amp DD SB o d _ Ae i a Te FE EE EF EEE EF EF EOE EEL EEF ES Figure 73 Line plot showing missing alternatives UNCLASSIFIED 55 UNCLASSIFIED DSTO GD 0681 8 3 Ranking Distribution Tabular The Ranking Distribution tab indicated by 0 in Figure 64 is shown in Figure 74 Within the Ranking Distribution tab are two sub tabs Tabular and Graphical Figure 74 presents the ranking distribution in tabular format Section 8 4 presents the ranking distribution in graphical format Altemative Ly ee ee Be Pa ay Ta TE TS Supplier 4 4
14. For example 0 0001 would be almost halfway between these two values UNCLASSIFIED 105 DSTO GD 0681 UNCLASSIFIED 13 3 Analysing Search Results The results from a search can be used for different types of analyses with the most simple being what if questions The data presented can also be used for trade off analysis since it presents ranges of values for the various criteria When sorted by column trade off analysis is simple This is explained in Section 13 3 2 Figure 130 shows an example of the search results Figure 130 Search results table 106 ota oe pass O aad Eee ee ses 62 0 02 64590 77 62459 28 5671943 6363815 6147926 TALTE 59082 87 6116372 0263 44 s9447 4 50695 56 047 27 61352 45 39630 38 SIES STAA 64 5940371 60561 66 59540 25 59517 5565383 56829 54 5063676 Tazo 2107 74262107 ra 26 2107 razo 2107 F426 210 7425 2107 74262107 7426 2107 7426 2107 74286 M07 7426 2107 7425 2107 7426 2107 7425 2107 7426 107 razo 2107 Taz H0 Ta26 2107 420 2007 Taz F426 2107 TAZE 2107 7Tiz75 207 F426 2107 F426 2107 7426 2007 7426 107 7426 2107 F426 2107 Mainio TREIN F Reishi Tops FRefstdiy Tope B Spares Spares 9274 7016 92747016 a2 47016 Fara 7016 92747016 9274 7016 9274 7016 o27 4 7016 3274 7016 074 7016 g2 706 a274 7016 gaa 7016 9274 7016 S274 7016 S274 7016 92747016 S274 7016 S274 7016 92747016 92747016 gyra 7016 92747018 9274 701
15. Step 4 Define the Possible Decision Alternatives 4 2 11 Consequence Definition In this final stage of using the Wizard you will be required to define and assign the appropriate attributes scales or measures needed to evaluate each of the objectives This is done in Decision Maker using criterion elements Section 5 3 4 Each of the objectives you defined earlier must have at least one criterion element for the problem to be complete UNCLASSIFIED 22 UNCLASSIFIED DSTO GD 0681 Define Alternative Humber 1 Define Alternative Humber 1 Alternative Title Alternative Title New Alternative 1 Supplier 1 Alternative Description Alternative Description Figure 23 Wizard example Define Alternative Number 1 Figure 24 Wizard example Define Alternative Number 1 For detailed information on using the dialog box presented in this stage refer to Figure 11 and Section 4 2 5 Use the Objective Navigation buttons to navigate to each of the objectives you defined earlier For each objective define the appropriate criteria and create each criterion using the information presented in Tables 7 8 9 10 and 11 To commence the definition stage click the Next button until the dialog box shown in Figure 25 is visible Step 5 Define the Criteria to Measure your Objectives In this step the benefits of each ofthe competing alternatives defined earlier are considered by assessing how well they fulfil the objectives of your probl
16. Supplier 15 0 1650 0369460 05460 Supplier 14 o1740 0 367560 0 5760 Supplier 10 0 1200 0273360 0 4630 Supplier 13 0 0550 0 209560 0 4560 Supplier 7 0 0000 O4A476e0 0 S300 Supplier 16 0 0070 0163720 0 2920 Supplier 4 0 00000 o 040080 01590 Supplier 14 0 000 0024620 013620 in g E aa w Ai feb d D 4 oS amp hi t EET g ER Cardinal Scores Ordirval Scores Ordinal Score The table view contains the best worst and average ordinal scores for each alternative for a set of simulation runs The range or ordinal scores obtained by an alternative can be read from the Ordinal Score axis This range bar indicates that most scores were in the lower part of the range i e a better ordinal score Q This range bar indicates the scores were more or less evenly spread across the range a range bar indicates that most scores were in the upper part of the range i e a worse ordinal score Figure 70 Ordinal scoring ranges 8 2 3 Hi Lo Range Bars Not Appearing At times the Hi Lo range bars for an alternative will not appear in the graphical display This is due to the best worst and average scores all being equal as in highlighted by O in Figure 71 UNCLASSIFIED 54 UNCLASSIFIED DSTO GD 0681 Ordinal scoring Ranges 18 00 15 00 12 00 nm 3 AW a g nA n Aa b amp O a Go AAD 8 sh oh oO 0 2 x Co oD FE FES KP CLS CS KS SE EC SF SEES Ordinal Score L5
17. When complete the Finished export complete dialog box will appear as shown in Figure 139 3 Decision Maker Il Options ax J General Model Views Database Management Simulations Database File Administration AttachDBFileN ame C Program Files 03TO Decision Maker I sinneporhing mdi Database Hame simreporting DataSource SLILESPRESS Integrated Security True Connect Timeout 0 User Instance No eek A Export ToZip Archive app ln Eron Zip Archive Import From Zip Archive J Create Stas ie ak TO Siors seniorie amenant OOO O O Erer mma omore eeure dante Seenaa TO Operan esing dette Sesten a20 FO orearen atmet sena Figure 138 Database Management Database File Administration Finished FA Export Complete Figure 139 Export Complete dialog box 14 2 4 Import from Zip Archive To import a zip archive click the Import From Zip Archive button as indicated by in Figure 138 An Import DMZip Archive dialog box will appear as shown in Figure 140 Browse to the location of the archive file and click Open When the import is complete a UNCLASSIFIED 111 UNCLASSIFIED DSTO GD 0681 dialog box will appear as shown in Figure 141 The details of the current database indicated by in Figure 138 will update to reflect the newly imported file Import DMZip Archive Look in my DM Data Eiron E My Recent Documents Desktop My Do
18. eo gt Supplier 4 element within the tree the alternative is enabled 1 lt anna and a pop up text will appear It displays the node l r Bappa 13 name along with two numbers in brackets The two Supple 14 numbers have the following meaning The oe Cher JE number of alternatives that dominate this alternative the number of alternatives Enis alternative dominates Figure 115 Domination Tree icon examples UNCLASSIFIED 93 UNCLASSIFIED DSTO GD 0681 12 5 View Options When selecting Show Top level Analysis the problem is viewed using the relative weights and all criteria that are children of the selected objective Relative weights are the equivalent weights required for each criterion when the problem is analysed at the root node or branch of the decision tree Another term absolute weights is used only when comparing criteria at the same branch level within the decision tree Figure 116 shows the different view options View Options Absolute Change e Show Top Level Analysis e O Percentage Change Show Altemative Rankings Actual Value Show Criteria in Tree q ___ Q r i Show Levels wih Mouse Number of Searches 25 v o These are the types of value that will be displayed Actual Value is recommended ry This enables the mouse over node information display as shown in Figure 115 Enabling this activates the Show Top Level Analysis using all criteria an
19. 0 9580 5 0 0 1 2 Minimise Supply Time 0 5310 0 694220 0 9020 iF Se aaa nome Teas Te AA E nszwreo oreo i mN sOi 0 3290 0472120 0 7520 04310 0 523140 07020 05340 0 48449n OBR Supplier 15 01850 0 363460 05460 Supplier 11 0 1740 0 367560 0 5130 Supplier 10 0 1200 0 273360 0 4630 Supplier 13 0 0550 0 209560 0 4560 Supplier 0 0000 0 147680 0 3309 Supplier 16 o oo 0 169720 0 2920 Supplier 4 0 0000 0 040080 0 1590 Supplier 14 0 0000 0 024820 0 1620 o m r wh ne wWwwWN n DD io oo ew oO Nw F amp F WN Ww nu a a u u ad om 8 So i Cardinal Scores Ordinal Scores Cardinal Scoring Ranges 5 1 000 0 800 Bg 0 600 0 400 Cardinal Score 0 200 o Decision Report Selection Section 8 1 1 o Simulation Details Section 8 1 2 Simulation Reporting Decision Tree Section 8 1 3 Simulation Report Tabs e Scoring Range Section 8 2 e Ranking Distribution Sections 8 3 and 8 4 and e Weights Summary Section 8 5 Figure 64 Simulation Reporting interface overview This drop down box contains all the available decision problem simulations by Decision Repon Selection problem name Existing DecisionMaker Il Simulations Q selec best supply arrangement Simulation Sets 0 2 For each decision problem there may be one or more simulation sets This drop down box contains the list of their unique identifying numbers Each simulat
20. 2 Define your Decision Problem tis important to start the decision making process with the right problem In this first step you will be required to define your problem This will be done by entering your problems title and Description A tip to help you define your problem is to think about what or who triggered the decision problem and why itis a problem Figure 14 Wizard example Step 2 Define your Decision Problem Step 2 Define your Decision Problem tis important to start the decision making process with the right problem In this first step you will be required to define your problem This willbe done by entering your problems title and Description A tip to help you detine your problem is to think about what or who triggered the decision problem and why itis a problem While defining your problem you must also define a preferred Direction for the solution you re trying to find The preferred direction for your solution may either be Minimise or Maximise For example 1 you are trying to selectthe best alternative out of many altematives you ll wantto Maximise your results lf you re solution presents arisk to you or your business you need to Minimise your result Figure 15 Wizard example Step 2 Define your Decision Problem continued UNCLASSIFIED 17 UNCLASSIFIED DSTO GD 0681 Mew Decision Problem Problem Title Select Best Supply Arrangement pa Pr
21. Dilemma a Problems 7 Structural Elements Element ID PRJ 1 Objectives Date Created Monday 19 June 2006 _j Altemnatwes 9 Criteria Created By Powersh Disenptior This i a test project created to demonstrate a Supply Managers Dilemma Ready Figure 52 The Structure Model used to modify your problem s structure To open the Data Model window select the View Data Model menu option as shown in Figure 53 Figure 54 describes each element of the Data Model window Items in the Element Tree can be dragged and dropped onto other elements to change the structure of the problem This allows for quick and easy manipulation during trade off analysis Editing data in the Data Grid can only be performed on criterion data All other data in the Data Grid other than the alternative s titles is generated by Decision Maker The titles for the alternatives may be changed in the Structure Model Section 6 2 File Edit Project Tools Windows Advanced Analysis Help E Structure Model a Data Model E Show Weights Show Ordinal Results Show Cardinal Results Eal Refresh Figure 53 Opening the Data Model chart UNCLASSIFIED 43 UNCLASSIFIED DSTO GD 0681 a3 Data Model Sample Problem i E Refresh ij Calculate Chart O New Objective C New Criterion X Delete P Sample Problem Element Properties E Canepon 3 H Title C Gtiterion 2 2 Criterion 11 Discription lt a
22. E S RA 19 4210 Alermatve Deltion anes oat nauetieos tie atau eea 21 4 2 11 COnSeGIE Mee SiO ia iets arene alae iaetss N ieedey dec uss 22 4 3 Assign Data to Each of the Problem s Criteria cssssssssecsssessseeeseceeeeees 28 4 4 Calculate the Scores for Each Alternative in the Problem ee eeseeees 28 4 5 Chart the Results of the Decision Analysis Process sscsssssssesssseeseees 29 4 6 Revise and Amend the Problem s Structure eesesseeesoeseessessoeseessossesesossesssossee 30 AT Remark nena Lie Sonceoaasas ss A A A 31 5 STRUCTURING YOUR PROBLEM IN DECISION MAKER eessesesseeseeseesesssesse 31 Ol PIOA TD Proces scctsiasscicuei rete a E T caeecbinnasee 31 5 2 Structuring Your Decision Problem seesssesseesoesssesssesssosssosssosssosssosssosssosssos 31 593 The PIES Ele periei E aucun ease tueiua de doeteaees 31 UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 5 3 1 HEALING Ail FO CCL aeren aA ob csshaaddnscsesanesiesnaidot 32 5 4 Decision Maker Structural Elements e seesesesoeseeesoessossoessessoessossosssossoessossesssoee 33 5 4 1 Problem Element einion i 33 54 2 ODJ ctive EICCA T arsane ein a E R REEE 33 5 4 3 Altematve Elemen bereme A 34 5 4 4 se ON Elemen a Toner eter eee pre een ne 35 5 4 5 Uncertain i Eerie gl eea N N 36 DAO ANON Uncertainties miorina a 37 5432 Changme Uncertainties sanninna a 37 5 5 Manually Structuring your Problem seeesoeesoesso
23. Search Selector example 1 13 1 Criteria Sensitivity Search Options Figure 126 describes the search options used in the Criteria Sensitivity Search search Options J ji a C Show Calulation Columns C Non Dominated Search earch Vanation 0 0005 Auto Tune J Best Show Weights Columns Number of Search Matches 20 Average iE Searches Performed Wars o These options show some hidden details i Non Dominated Search enabling this will provide only non dominated results For almost all searches this should not be used since it will limit the number of sets returned and not provide enough data for trade off analysis Stop Search this button can be used to stop a search Search progress bar Section 13 1 2 Search Variation this is the maximum difference between the Target Cardinal Score and the Search Result Cardinal score Section 13 1 1 Number of Search Matches the total number of results to find and return in the Search Results table Section 13 1 4 z Search relaxation slider Section 13 1 3 Auto Tune Section 13 2 These three boxes show the best worst and average variations in Target and Search Cardinal Scores Section 13 1 7 Searches Performed the number of searches that have been performed Section 13 1 5 Figure 126 Criteria Sensitivity Search Options 13 1 1 Search Variation Search Variation is the absolute difference between the upper alternative s cardinal score and the lowe
24. Type A and Type E spare parts to his fitm These contracts are about to expire and he needs to consider a Select best supply arrangement _ eMAX C Administration Days pply BAX Refresh_ _ Options Alternatives Cardinal Score 4 gt Supplier 70 re T3 Te Cardinal Score Supplier 2 9 31 60 Supplier 3 0 831 55 E s S A Supplier 4 0 803 75 Supplier 5 0 526 7 l e Supplier 6 0 36 68 ___ r Su Supplier 7 0 359 7g Supplier 8 0 7 a se m N OO o wo e oomoo KF NQ DO D Supplier 9 0 061 63 oS Sb c S c N CBC iC HT N c UU E ES S64 6 oo 3 oe 3 ao 5 BS RB FB SB SB BS Supplier 10 0 06 74 S S 95 amp amp amp amp amp amp amp hk w w w w w wn w w wn in a a 7 a an Supplier 11 0 097 68 Supplier 12 0 324 63250 Supplier 13 0 158 71500 Supplier 14 0 032 79750 Supplier 15 0 259 74250 0 Supplier 16 0 094 71500 Weight 0 282 0 374 0 344 Ready Figure 5 Decision Maker s data model for the Supply Manager s Dilemma Note if the scales or measures you chose for your consequences are descriptive in nature then you will need to convert them to numbers Use the Even Swap method given in 4 or use an alternative decision making technique 2 3 Note outputs from multi criteria decision making techniques can be categoris
25. a as a _ m Scoring Range Ranking Distribution Weights Summary Tabular Graphical aul E l Fo ri r 7 i ar a B HE F PEU j a g r j 4 7 mF J mis l f ba Supplier Supplier 35 4 Supolier3 0 Supplied 3 Supplier 5 Supplier 6 o 30 Supplier i 0 Supplier Supplier 8 25 iii oe P gt Supple 3 Supplier 10 E 20 0 Supplier 11 Supplier 12 15 Supplier 13 Supplier 14 10 Supplier 15 Supplier 16 a 5 F 5 7 B 5 10 11 te 13 14 15 16 Ranking E j o Toggle 2 Dimensional 3 Dimensional View ry Ranking Count Cumulative Total Figure 76 Overview of ranking distribution 2D graphical analysis 8 4 1 Filtering Alternatives To select one or more alternatives to display in the graph area click the left mouse button in the row header area of the table shown in Figure 77 To remove the alternative from the graphical display click the right mouse button in the row header area of the alternative that is to be removed Figures 77 78 and 79 show the filtering of alternatives in tabular three dimensional graphical form and two dimensional graphical form respectively UNCLASSIFIED 58 UNCLASSIFIED DSTO GD 0681 Scoiing Range Ranking Distribution weights Summa 11 12 13 14 15 16 Row header used to select deselect alternatives displayed in the graphical view In this example Supplier 2 Supplier 3 and S
26. an inverse relationship or a direct relationship between adjacent elements This is shown in Figure 110 UNCLASSIFIED 87 UNCLASSIFIED DSTO GD 0681 3 Cobweb Plot Decision Report Selection Existing DecisionMaker Il Simulations Select best supply arrangement Simulation Sets Simulation Run rs de ee ee A TT Boo amp C1 Minimise Total Cost 0 1 Maximise Performance 0 1 1 Maximise Reliability C ies Maximise Reliability Type A Sp Maximise Reliability Type B Sr B 0 1 Ys repek Supply Time C 4 gt Minimise Supply Time Type A C4 Minimise Supply Time Type B cam i 2 Gl w P1 Select best akii Bianwen C1 Minimise Total Cost e 0 1 Masamnise Petformance I 0 1 1 Maximise Reliability E C 2 Maximise Reliability Type A Spares C 3 Maximise Reliability Type B Spares H 0 1 2 Minimise Supply Time e C 4 Minimise Supply Time Type Spares C5 Minimise Supply Time Type B Spares Beed bert Minimis himimips Mamie ears Minimize Minimise el a supply Total Cini Performance Reliabiliny Kal Rekati Supply Time duppi T airangement aaay Type B dares Typa A Saris iy Pe A sharp spike indicates an inverse relationship The location of the spike gives an indication to the region of values ry An even distribution over a region or an entire plot indicates a direct relationship This is an example of two regions of inversion
27. archiving and importing exporting simulation database files 14 2 1 Simulations The Simulations tab shown in Figure 134 provides an interface to clear i e delete individual simulations or all the simulations contained within a database file Database File Options Select best supply anangement Select best supply arrangement Select best supply arrangement 3 Be Clear all Simulation Data A fe Clear Selected Simulation Set Action Selection Problem Mame select best supply anangement Simulation Date 22 05 2007 1 06 AM Created By CostolC Number of Simulation Runs 10 Simulation Set ldentiter 1 oga Available simulation sets selecting one item in the list will display summary details in the Action Selection area of the Simulations tab o This shows details of the selected simulation set in d This will clear all simulation sets shown in Q o This will clear the selected simulation set in Q o This performs the selected action 8 9 Figure 134 Database Management Simulations THIS IS A PERMANENT DELETION AND CANNOT BE REVERSED When a selection has been made click the Proceed with Clear Selected Simulation button and the confirmation prompt shown in Figure 135 will appear Alternatively if Clear all UNCLASSIFIED 109 UNCLASSIFIED DSTO GD 0681 Simulation Data is selected click the Proceed with Clear All button to erase the entire contents of the databas
28. columns indicate low scoring alternatives the colours have the opposite meanings and the value represents the fraction of simulation runs the alternative scored lower For example in the cell with row selection Supplier 3 and column selection Supplier 16 this can be interpreted as Supplier 3 scored lower than Supplier 16 in 64 of the simulation runs which is the same as Supplier 16 scoring higher 36 of the time The matrix can also be read along the row and then the column If it is used in this manner then Figure 85 Scoring Matrix Viewer The colour coding identifies the relative performance of the problem alternatives For example columns that are predominantly green indicate alternatives that scored highly for the selected problem objective or criteria the opposite is true for predominantly red columns The colour coding is based on a range of values shown in Table 12 Table 12 Scoring Matrix colour coding ranges Scoring Range UNCLASSIFIED 65 UNCLASSIFIED DSTO GD 0681 10 Domination Scoring Matrix The Domination Scoring Matrix is an advanced view that encompasses the features of the Scoring Matrix into one single view with only Criteria elements It is a useful stand alone tool and is also useful when used with the CWViewer Section 11 It allows for a more comprehensive understanding of the level of performance of alternatives within the simulation context The Domination Scoring Matrix can be started from t
29. contents of the Alternative ID column or define unique descriptors of your choosing and the objectives and criteria columns of the consequence table Table 1 Here MTBF is the lower limit of the 95 confidence interval of the MTBF Here MTTS is the upper limit of the 95 confidence interval of the MTTS UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 into Decision Maker When complete click the Calculate button Decision Maker will give a cardinal and ordinal ranking to all the alternatives EXAMPLE CONTINUED Supply Manager s Dilemma PrOACT STAGE Trading off the Alternatives The Supply Manager now enters the relevant parts of his consequence table into Decision Maker Open the file named Supply Mangers Dilemma xml to see the Supply Manager s table shown in Table 1 Now click the Calculate button and the result will appear as shown in Figure 5 ial See Uu Refresh Bi calculate We chart O Mew Objective g New Criterion X Delete Select best supply arrangement Element Propienies E Minimise Total Cost Title i Maximise Performance Maximise Reliability Select best supply arrangement Maximise Reliability Type A Spares Maximise Reliability Type B Spares O Minimise Supply Time C Minimise Supply Time Type A Spares I C Minimise Supply Time Type B Spi Disecription The Supply Manager is in the process of reviewing the existing ja contracts that supply
30. have been selected The Minimise Total Cost criterion has also been selected since cost is an important factor in this decision problem UNCLASSIFIED 83 UNCLASSIFIED DSTO GD 0681 aH CWViewer Decision Report Selection Plot Options Existing DecisionMaker II Simulations Top Percentage Limit 0 9 C Show Top Scoring Alternatives V Show Axis Values It is highly recommended that the root element P 1 which shows the overall ranking is always visible in the plot since it provides immediate understanding of the selected alternative preference over the others This is demonstrated by in Figure 106 Figure 106 is taken from the Supply Manager s Dilemma and for demonstration purposes three suppliers have been selected in the viewer In this example the Cardinal Scoring Option is also selected 9 Select best supply arrangement iv Low Percentage Limit 0 1 C Show Mid Range Alternatives iv Cardinal Scoring Option Simulation Sets Simulation Run C Show Low Scoring Alternatives a i C 1 2 Minimise Total Cast 0 1 Maximise Performance O 0 1 1 Maximise Reliability i beg C 2 Maximise Reliability Type A Spares beg C 3 Maximise Reliability Type B Spares 0 1 2 Minimise Supply Time it C 4 Minimise Supply Time Type A Spares inn C5 Minimise Supply Time Type B Spares P 1 Select best supply arrangement C1 Minimise Total Cost 0 1 Maximis
31. longer required setting all the values for each element to zero 0 will have the same effect as having no uncertainty Each criterion can only have one uncertainty However the values for each alternative can be different Criterion Title Use this field to enter a short title for your criterion The title of each criterion must be unique F o m r New Criterion Jeg Criterion Description Enter a detailed description of F the criterion in this field the more detail the better Criterion Title New Criterion Unit of Measure This field allows you to select or enter a unit of measure for your criterion For example the minimise expenditure objective might have a criterion Labour Cost The unit of measure for Labour Cost may be the dollars measure which has the symbol Alternatively you can type in your own unit of measure and define your own symbol Direction of Preference Select the preferred direction of preference for your criterion Like the Problem and Objective elements the criterion s direction of preference field provides a simple method used to inform Decision Maker whether the criterion will have a positive or negative influence on its parent objective Unit of Measure For example the sub objective maximise reliability can Default w be evaluated by assigning Criterion elements representing Mean Time To Repair MTTR and MTBF Criterion Discnpton Direction of Preference pazila L
32. tE Supper 12 f Supplier 5 ri Supplier 16 Supplier 3 tc Suppher 6 t Supplier 1 Supplier 11 gt Supplier 8 Suppher 10 Supplier 15 Suppker 13 Suppher 7 Supplier 14 Supplier 16 Supplier 4 0 1 Maximise Performance 4 Altematives 5 Supplier 5 I Supplier 2 Supplier 12 a Supplier 6 Supplier 10 Supplier 15 Supplier 3 Supplier 8 Supplier 1 Supplier 11 Supplier 7 gt Supplier 3 Supplier 14 Supplier 13 Supplier 16 Supplier 4 0 1 1 Maximise Reliability 0 1 2 Minimise Supply Time EELEE 2 L 4 2 2 L3 7 QOS FFP HHHHHHH HHH UNCLASSIFIED Simulation Details View Options Number of Alternatives 16 LJ Absolute Change xj Number of Simulation Runs 10 Percentage Change Actual Value ZI Simulation Date 8 05 2007 8 38AM Show Levels with Mouse Created By CostollC Non Dominated Sets Saacdh Rasibe Maximise Mamise Reliability TypeA Reliability Type B Spares Spares ans oas 0 203 o2 0 0 334 0 242 0107 0 434 Minimise Total ost Upper Alternative Lower Alternative aaa 0178 0214 oig 0 062 0 064 0198 0 051 Supplier 1 Supplier 10 Supplier 11 Supplier 13 Supplier 14 Supplier 15 Supplies 4 Supplier 3 Supplier 3 eae Supplier 3 Supplier 3 Supp er 3 Miniman Changa Disibadion Dec Relations a a old Wla ptos 1 000 0 800 0 600 0 400 0 200 0 00
33. tana 2 2 2 Installing Decision Maker from CD sssesssseeseeerrerierisrrrerrerresresrrersses 2 22 Unimstalline Decision Waker sniisawaeuncnuinmantienhitnuue naman 3 2 3 Starting Decision Maer j cpciiscesasscngsoueses oadesichandestareedenavnas dete aiaia ia sasa iia 3 24 Exiting Decision Maker sciscsicssecsctecteisteeastaseeen eases names 3 3 A GUIDE TO MAKING BETTER DECISIONS qu tcccseecsceeceesecescceeseseseesseeenes 3 Sih PO DIC ge sec sesute E e 5 322 ODS CUV CS sianet eneinio ai a aina aea aaa 5 Bid PIPER ATIV eS cecrn shedeestaieaies 6 34 CONSE GUCICES iscri E R E seins 7 Di Trade o Susan a E E EN 8 30 REVISE uir EE E E 10 4 QUICK START USING THE DECISION MAKER WIZARD eeseesessssseeseesesssssssese 10 4 1 Using Decision Maker and the Wizard esoeosooesooesoeesosssosssosssosssosssosssosssos 11 42 Usmo ANS VV IZA soie a A A a saiives 12 4 2 1 Step 1 Create new Decision PrOjeCt essercene 12 422 Step 2 Denne your Problemisccacsd ase se tsar Sees 13 4 2 3 otep o Denne Vy Our ODJeC HVE Snina a iat ae aati 13 4 2 4 Step 4 Define your Alternatives eeeeeeseeseereereerrrerrrsrrererrrrerresrses 14 4 2 5 Step 5 Define the Criteria for each of your objectives 0 0 14 4 2 6 Step 6 Adding data to your decision problem n se 15 4 2 7 Create a New Project Using the Wizard eee eeeeeseeeseeeeeeeeeees 15 4 2 8 Problem Denm 6 9 emer ene mene ere a fern eer ae 16 4 29 ODecHive Denm ON srian ena e a
34. target value which are the sets of result where the scores change from lower to higher than the target as indicated by 9 in Figure 130 13 3 2 Trade Off Analysis Figure 131 shows an example of how the search results can be used for criteria sensitivity trade off analysis If a search is run and the number of matches is significant to give a range of solution sets then the results can be sorted for each criterion by clicking on the column header in the search results table The sorted results can then be used to determine what trade offs can be made within the alternative In the example shown in Figure 131 three regions are highlighted The green and red arrows indicate the criteria value directional movement in the preferred green and non preferred red directions Minimise Suppl Time Type Minimise Supply Tine Type B Resul Vanaton Score Target 16033 Minimise Total Cost 39926 68 S972505 59694 67 567613 sob 2 Sa303 74 Sag 96 58291 56 Srei 74 Maxine Maxinnene Reliability Type ag Type E EGEN 560593 4 56034 16 567 26 01 36409 16 36130 56 Sga S524231 55070 85 IAS EAN A 53755 14 5m5 Sauti Fz S202136 S277566 7 ne yo as Fata a a 52274 Sa Spares 7426 2107 7426 2107 7426 07 7426 407 7426 2107 1426 2107 7426 07 7426 407 7426 107 7426 2107 14262107 7426 2107 74262107 7426 2107 74262107 74262107 7426 2107 7426 2107 ZEN 7426 D107 7426 2107 7426 2107
35. the user e criteria and objectives do not have to be independent e acardinal and ordinal ranking of alternatives e the removal of dependency on personal preferences of the decision maker which results in informed and quantifiable decision choices e a unique sensitivity analysis component using leading edge Artificial Intelligence techniques to provide analysis for what if situations e a simulation engine that allows the decision to be modelled with uncertainties and tolerance levels for criteria e advanced simulation reporting analysis in tabular and graphical format that provides a method to quickly identify the superior alternatives e advanced graphical analysis tools that allow the decision maker to compare alternative criteria and objectives in a meaningful and timely manner and e graphical analysis tools that provide a method of determining complex relationships and trends within the criteria objective data This user guide will assist and inform users on how they can structure their decision problems in the required format for Decision Maker UNCLASSIFIED UNCLASSIFIED This page is intentionally blank UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 Contents GLOSSARY t INTRODUCTION onre E ET E 1 2 GETTING STARTED WITH DECISION MAKER eeesssesseeseessoeseessoesoessoesesssoesesesossees 2 2L Installing Decision Maker casiccissiecansaiiniaueiietnaoenieie alsin 2 211 DY SEEM Requirement S asuisi an A tice samee sen iudavaansh
36. type in a meaningful name do not use spaces or any punctuation characters and click Save A progress bar will indicate the progress of creating the new database file When complete it will appear as shown in Figure 145 Click on the dialog box and it will close The details of the current database indicated by Qin Figure 138 will update to reflect the newly created file 15 Importing Data from File Data for alternatives can be imported from a formatted text file which has been delimited by commas tabs or semi colons This can be used when there are a large number of alternatives and or a large number of criteria to input This utility can also be used to add alternatives to an existing decision problem UNCLASSIFIED 113 DSTO GD 0681 Save As Save irr 3 Decision Maker UNCLASSIFIED lx otom E simreporting mdf IR test_1 mdf My Recent Documents Desktop My Documents 43 My Computer File name NewS imulationD atabasel May Network Save s type SQLExpress Database mal m P s Figure 144 Save As dialog box create new database file nave al Cancel Database Creation Complete Figure 145 New Database Creation Complete dialog box For the import utility to work it is necessary to have a project open that contains the criteria that will be mapped against the values in
37. understood Criteria Sensitivity analysis can be started from the Advanced Analysis Criteria Sensitivity menu as shown in Figure 123 UNCLASSIFIED 99 UNCLASSIFIED DSTO GD 0681 File Edit View Project Tools Windows Advanced Analysis Simulation Reporting Scoring Matrix Cobweb Plok viewer Weights Sensitivity Criteria Sensitivity Fg Domination Scoring Matrix Fil Figure 123 Starting Criteria Sensitivity analysis Unlike Weights Sensitivity analysis Criteria Sensitivity analysis does not involve domination Instead the aim is to determine the amount that one or more criteria values need to change in order to make one alternative more preferable than another This is achieved by finding the point where the two alternatives are equal and hence any further criteria value changes in the direction of preference will only make the search alternative more superior Decision Report Selection within the Criteria Sensitivity analysis is based on the selection of individual simulation runs Figure 124 shows the Criteria Sensitivity Analysis window Details of specific components and their uses are explained in the following subsections When the Criteria Sensitivity Analysis window is open selecting a simulation run in the Decision Report Selection panel will populate the tree view Other tables will be populated after selections are made and searches performed When searching for criteria the upper alternatives criteria val
38. will appear as shown in Figure 85 The matrix is used by selecting a column and a corresponding row from the Scoring Matrix Viewer Where the two intersect the cell will contain a number and be colour coded The number UNCLASSIFIED 64 v am njene A Ea Hl UNCLASSIFIED DSTO GD 0681 represents the fraction of simulation runs that the column alternative scored higher than the row alternative fa oring Viewer l Decision Report Selection Simulation Details Existing DecisionMaker I Simulations Humber ofAlematives 16 Select best supply arangement Number of Simulation Runs 50 Simulation Sets Simulation Run 1000 gr Simulation Date 1 06 2007 10 49AM Created By CostollC l E C1 Minimise Total Cost O 0 1 Maximise Performance I gt Supplier 1 88 0 28 C C2 Maximise Reliability Type A Spares Supplier 2 E C3 Maximise Reliability Type B Spares 0 1 2 Minimise Supply Time 5 e ECE C C4 Minimise Supply Time Type A Spares f _ Supplier 4 C C5 Minimise Supply Time Type B Spares Supplier 5 fp P 1 Select best supply arrangement Suppke Supplier Supplier Supplier Suppligt Supplier Supplier Suppker Supplier Supplier Supper Supplier Supplier Supplier Suppher Supplier 1 2 3 4 6 7 8 5 10 11 12 13 14 15 16 Jees Supplier 6 Supplier 7 Supplier 8 Supplier 9 Supplier 10 Supplier 11 Supplier 12 Supplier 13 Supplier 14 Supplier 15 Supplier 16 A 5 Predominantly red
39. 0 Maximise Reliability Type A Spares Minimise Total Cost Maximise Rehabilty Type B Spares EE DSTO GD 0681 oE Show Top Level Analysis C Show Altemative Rankings _ Show Criteria in Tree Number of Searches 25 m Minimise Supply Time Type B Spares 0 141 02239 0125 0204 010 0162 0214 0 081 Minimise Supply Time Type A Spares 0 191 0235 0 106 02 0 223 0 043 Minimise Supply Time Type A Spares Minimise Supply Time Type B Spares o Show Top Level Analysis is selected ry This section of the domination tree is built based on all the criteria in the problem Selected criteria weights are shown in the Non dominated sets table view Graphical analysis of the search results showing the range of weights that will produce a change This section of the domination tree is based on all criteria below Maximise Performance in the decision tree In this example the criteria are C 2 C 3 C 4 and C 5 Figure 122 Weights Sensitivity top level analysis 13 Criteria Sensitivity Criteria Sensitivity analysis is Decision Maker s is most advanced tool It is important to understand how to interpret the information presented since in some instances the predictions can produce spurious results This is an inherent problem with the Artificial Intelligence AI technique that has been used to perform the searches However if some simple guidelines are followed these problems can be easily identified and
40. 0 1 1 Maximise Reliability i beg C 2 Maximise Reliability Type A Spares 1 hep C 3 Maximise Reliability Type B Spares 0 1 2 Minimise Supply Time S HE C 4 Minimise Supply Time Type A Spares im C4 Minimise Supply Time Type B Spares gt UNCLASSIFIED DSTO GD 0681 Scoring Range Ranking Distribution Weights Summary Tabular Data Weights Ranges Graphical Display Criteria Weights Ranges 0 600 0 570 0 540 0 510 0 480 0 450 0 420 0 390 0 360 0 330 0 300 0 270 0 240 0 210 0 180 0 150 0 120 0 090 Ea Weight elie SS oie i C 1 Minimise Total Cost C 3 Maximise Reliability Type B Spares C 2 Maximise Reliability Type A Spares C 5 Minimise Supply Time Type B Spa C 4 Minimise Supply Time Type A Spares Figure 81 Weights Ranges graphical example 1 P 1 Select best supply arrangement E C 1 Minimise Total Cost 3 0 0 1 1 Maximise Reliability i he C 2 Maximise Reliability Type A Spares Ln E C 3 Maximise Reliability Type B Spares 3 0 0 1 2 Minimise Supply Time Lal i m C C 4 Minimise Supply Time Type A Spares i im C5 Minimise Supply Time Type B Spares gt Scoring Range Ranking Distribution Weights Summary Criteria Weights Ranges hip C 2 Maximise Reliability Type Spares C 4 Minimise Supply Time Type Spares C 3 Maximise Reliability Type B Spares C 5 Minimise Supply Time Type B
41. 00 0 040080 0 1590 Supplier 14 0 0000 0 024620 0 1620 on A A E Ww OW mM hm bl Cardinal Scores Ordinal Scores Cardinal Scoring Ranges Cardinal Score The table view contains the lowest average and highest cardinal scores for each alternative for a set of simulation runs The Cardinal Score axis is used to determine the range of scores for each alternative This range bar indicates that most scores were in the lower part of the range This range bar indicates that most scores were in the upper part of the range This range bar indicates the scores were more or less evenly spread across the range Figure 69 Cardinal scoring ranges 8 2 2 Ordinal Scoring Ranges The top half of Figure 70 shows some basic information collected from a simulation set namely the range of ordinal scores that each alternative was assigned and the average ordinal score Here the ordinal scores are indicated by the red bracket In ordinal scoring the lower the score is the better option UNCLASSIFIED 53 UNCLASSIFIED DSTO GD 0681 Scoring Range Ranking Distribution weights Summary Lowest Cardinal Average Cardinal Highest Cardinal Best Ordinal Average Ordinal Worst Ordinal 1 0000 41 000000 1 0000 1 0 5900 0715740 0 6590 0 5930 0773580 0 9580 1 Oe 5310 D 684220 0 2020 0 3580 0 620760 07660 Supplier amp 0 3230 O472120 0 7520 Supplier 1 o4310 0 523140 0 7020 Supplier 6 0 3340 0 494420 0 6670
42. 1 4 1 Filter by Objectives The viewer can be used to display objective elements only This assists identification of how alternatives perform in each of the objective areas Figure 104 shows an example of filtering by objectives Decision Report Selection Plot Options Existing DecisionMaker I Simulations Top Percentage Limit 0 9 _ Show Top ScoringAltematives X Show Axis Values Select best supply arrangement vi Eaa Parcent iaaa C Show Mid Range Altematives Cardinal Scoring Option SiidionSee Smiddior ni Ea Cance Ape C Show Low Scoring Altematives Minimise Total Cost Maximise Performance Laf 0 3 Maximise Reliability Type B Spares S O 0 1 2 Minimise Supply Time mE C 4 Minimise Supply Time Type A Spares iw C5 Minimise Supply Time Type B Spares Iv P1 Select best supply anangement C C1 Minimise Total Cost 0 1 Maximise Performance MIOT T Maximise Reliabity n C C2 Maximise Reliability Type A Spares C C3 Maximise Reliability Type B Spares 0 1 2 Minimise Supply Time C C4 Minimise Supply Time Type A Spares C C5 Minimise Supply Time Type B Spares jiona JAN Aternatives Suppl 1 Supplier 2 cigars Supplier 4 Suppli 5 Supplier amp Sugplier 7 f Supplier 8 Suppiier 3 Suplier 10 i t i slayer 11 i aRar T t 3 In the Plot Element Selector only th
43. 14 2 6 CT Cate Databas Espiri a A T A 113 15 IMPORTING DATA FROM FIL E nnsa konnen a ated 113 15 1 Setting the DelimMite osneniiiorane ia E E 116 15 2 Mapping Column Names ssiroriisiraeiuisisieaekasi ddaa eniin ni aieiaa 116 16 ACKNOWLEDGMENT Sisirin EEEE EEE ES 118 REFERENCES ennan aa N N T 119 APPENDIX A KNOWN FAULTS IN DECISION MAKER esesseeeseescesseessessessessoe 120 UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 Glossary ADO Australian Defence Organisation Al Artificial Intelligence CD Compact Disc CRITIC Criteria Importance Through Intercriteria Correlation CWViewer Cobweb Plot Viewer DGMARSPT Director General Maritime Support DMO Defence Materiel Organisation DSTO Defence Science and Technology Organisation GUI Graphical User Interface MTBF Mean Time Between Failure MTTR Mean Time To Repair MTTS Mean Time To Supply NaN Not a Number PrOACT Problem Objectives Alternatives Consequences Trade offs RCM Reliability Centred Maintenance RMIT Royal Melbourne Institute of Technology SOAP Simple Object Access Protocol SPO Systems Program Office XML Extended Mark up Language The following two symbols are used throughout this User Guide to highlight points that are helpful or are essential to know when using Decision Maker The Light Bulb symbol is used throughout this User Guide to indicate information that is useful to know when using Decision Maker The Exclamation symbol is used throughout this User
44. 3 24 5235 6813241 8243 47 68183 24 8183 34 82 D 0 01420284355 6810 24 8185 31 6 0 01120284856 6618 241 8157 27 D 0 11220284956 G2 Maximise Reliabit C A Minimise Supply C5 Minimise Suppk The tree view alternatives are displayed in ordinal ranking order of the parent objective or root problem KJ Criteria Search selector Options panel Simulation Results table Search Results table Figure 124 Criteria Sensitivity analysis What values of Minimise Total Cost and Minimise Supply Time Type A Spares given that the Reliability of Type A spares is 7426 and Reliability of Type B spares is 9274 and Minimise Supply Time Type B Spares is 6 57 will make Supplier 5 equal to Supplier 2 To solve the above search problem using the Criteria Search Selector the desired criteria are selected as shown in Figure 125 The criteria that will remain constant are the ones that have been selected in both columns The search settings then need to be set and in most searches the default setting will be sufficient as explained in Section 13 1 UNCLASSIFIED 101 UNCLASSIFIED DSTO GD 0681 Upper Lower Criteria Name Altemative Alternative C1 Minimise Total Cost C2 Maximise Reliability Type A Spares C3 Maximise Reliability Type B Spares C4 Minimise Supply Time Type amp Spares C5 Minimise Supply Time Type B Spares Figure 125 Criteria
45. 5 Minimise Supply Time Type B Spares V P1 Select best supply arrangement 7 C1 Minimise Total Cost 7 0 1 Maximise Performance 0 1 1 Maximise Rebabilty 7 C 2 Maximise Reliability Type A Spares V C 3 Maximise Relabiity Type B Spares F 0 1 2 Minimise Supply Time V C 4 Minimise Supply Time Type A Spares ol Supplier 3 Supplier 4 Supplier 5 Supplier 6 Supplier 7 Supplier 8 0000000 Supplier 9 Supplier 10 Supplier 11 Supplier 12 Supt Wer 13 poonon Supple ta beui supplier 15 om Supplier 16 To uniquely plot an alternative check the box as indicated The alternative name will now appear black The colour used to identify the alternative is shown in the box in the colour column Note in this view the Cardinal Scoring Option is not selected To change the colour double click in the colour cell as indicated A colour dialog box d will appear as shown in Figure 100 Figure 99 CW Viewer filtering alternatives example 1 UNCLASSIFIED 78 UNCLASSIFIED DSTO GD 0681 Select a predefined colour Select a custom colour E m i A m A 5 Hue 0 Red 255 Sat 240 Green 0 ColoiSolid Lum 120 Ble O Figure 100 Colour chooser a an aes m ad CWViewer a Decision Report Selection Plot Options 3 Existing Decision aker I Simulations Top Percentag
46. 6 92747016 9274 7015 o274 7016 3274 7016 9374 7015 Spares 56 744 6 5 oa GES TEB 585 Tal g 7 05 7 eo 355 58 5 B35 BBE BE B43 EE B2 B72 336 Minimise Suppl Minnie Supply Tine Tope A Time Type B Spares 65755 6B 3738 6 5786 ETE 6 5798 a 65756 S5796 6 5798 65796 6 57 88 Srog 65798 6 4780 6 5 8 6 5755 6 57 S68 6 5788 6 5 Pao 5 5736 6 5798 EST 5 5708 Score aipet iawa go sa EEEn jeas Des O83 oen Dass 0 8033 06033 WERE 0 8053 Maren 08053 0 8053 geass 0 8033 0 8033 0 ea oR032 peos 0 8032 oana DEUS oeg 0802 a oe 0 6052 08032 Negative values indicate criteria values produce a cardinal score slightly lower than the target Positive values indicate criteria values produce a cardinal score slightly higher than the target This is the region of change These two sets of criteria values are the closest to the target The results variation table showing the constant and varying criteria values UNCLASSIFIED 0 DoE 0000 0 mogga UNCLASSIFIED DSTO GD 0681 13 3 1 The Result Variation Column The actual variation of each result from the target score is shown in the result variation column A negative value indicates a slightly lower score and a positive value indicates a slightly higher score Sorting on this column has the same effect as sorting by the Target Score column When sorted it can be used to locate the closest solutions to the
47. 8 38 048 046 i eg C2 Maximise Reliabllity Type A Spares Supplier 2 M 0 3 Maximise Reliability Type B Spares Suppi 3 076 052 O 0 1 2 Minimise Supply Tine a eters ae mE C4 Minimise Supply Time Type A Spares a L E hee C5 Minimise Supply Time Type B Spares Supplier 5 Supplier 6 Supplier 7 Supplier 8 Supplier 9 Supplier 10 0 82 OSE 8 018 016 046 Supplier 11 j 388 054 O TAE 14 024 02 054 022 Supplier 12 Supplier 13 Supplier 14 Supplier 15 Supplier 16 2 Supplier 2 ranked higher than Supplier 5 in 92 of the simulations From the example Cobweb analysis it appears that on several occasions Supplier 14 may have ranked higher than Supplier 5 However when using the Domination Scoring Matrix it shows that Supplier 5 scored better than Supplier 14 100 of the time for the objective Minimise Supply Time Figure 109 Scoring Matrix interpretation minimise supply time 11 6 Criteria Relationships The CWViewer can be used to determine the relationships between one or more criteria This is especially useful when large complex decision trees with many criteria are being analysed Elements can be selected and filtered in the view to determine factors such as what are the trends in criterion C 1 when criterion C 3 is low and so on The green Stem Plots shown above the main plot can be used to quickly determine the direction of the relationship such as
48. EASE UNCLASSIFIED UNCLASSIFIED Decision Maker User Guide Executive Summary Decision Maker is an objective decision making tool that can be used in most multi criteria decision making problems in particular where a holistic approach is required in preference to the subjective and ad hoc processes that are often applied Decision Maker will help defence planners deal with the complexities of military planning by providing a sound scientific platform to assist in decision analysis in a timely manner as well as providing the decision maker with a documented quantifiable justification for their decision basis Examples where Decision Maker can be applied include e selecting the most preferred product tender service or company that best satisfies the specified requirements criteria or objectives i e procurement and e ranking the performance of systems components or items using combinations of several performance measurements Unlike most other products the key to Decision Maker is that it does not use subjective criteria to aid in the decision making process At the time of writing there is no other software on the market that incorporates all of the features of Decision Maker in particular the use of the objective criteria weight method and genetic algorithms for sensitivity analysis The essential elements of Decision Maker include e the use of sophisticated mathematical analysis techniques to identify conflicts in data input by
49. Figure 121 UNCLASSIFIED 97 UNCLASSIFIED DSTO GD 0681 Simulation Details View Options Existing DecisionMaker Il Simulations p Number of Altematives 16 Absolute Change Show Top Level Analysis Select best supply arrangement Number of Simulation Runs 35 Percentage Change C Show Altemnative Rankings Simulation Sets Simulation Rur A A Actual Value C Show Catena in Tree oT Simulation Date 28 06 2007 4 07PM SAA Number of Searches Created By CostollC s0 Tx P Selectbest supply arrangen Non Dominated Sets Search Results Ad Altematiy Supplier 2 aximise Maximise Minimise Supply Minimise Supply Cost Relay TypeA Reliability Type B Time Type Time Type B Spares e re spas opis Supplier 12 Supplier 9 Supper 3 Supplier 6 Supper 11 Supplier 7 Supper B Supplier 15 Supplier 10 Supplier 13 Supplier 16 Supplier 7 Supplier 4 Supplier 14 1 gt Maximise Performa Minimum Change Distribution Direct Relations 4 N 7 LOSS ETOCS 9 OST Minimum Weights Distribution For Rank Change P 1 Supplier 5 vs Supplier 10 wi a z ES Search Numbe Minimise Total Cost gt Maximise Reliability Type A Spares gt Maximise Reliability Type B Spares Mrimse Supply Time Type A Spares Minimise Supply Time Type B Spares Show Top Level Analysis is selected Show Top Level Analysis is selected si Level aay is selected Ci eeadenaniaens
50. File name Acquisition Contract Renewal Project v D EE My Network Save as type DDMS Project Files 0 Figure 12 Wizard example Defining anew project Figure 13 Wizard example Saving a project When you have defined your project click OK in the New Project dialog box and the Wizard will then ask to save your project as shown in Figure 13 Give the project a name and click on Save The Decision Maker Wizard will now guide you through defining the problem elements using the PrOACT process discussed in Section 3 This process requires you to define your problem define your objectives and alternatives and finally the consequences defined using decision criteria The Trade offs occur when the Wizard has structured your problem and you have assigned data to the criteria Navigate through the Wizard using the Next and Back buttons The following subsections will guide you through providing the information required at each step Images of the completed dialog boxes are also presented The example used is the Supply Manager s Dilemma 4 2 8 Problem Definition After creating and saving the new project the dialog box shown in Figure 14 will appear Click Next and the dialog box will then present advice for defining the problem as shown in Figure 15 Clicking the Next button again will open the New Decision Problem dialog box shown in Figure 16 UNCLASSIFIED 16 UNCLASSIFIED DSTO GD 0681 Step
51. Guide to indicate information that is essential to know when using Decision Maker Disclaimer Decision Maker is a functional prototype and as such there remain formatting spelling and grammatical errors in the user interface and this is reflected in the screen shots presented in this user guide Decision Maker will function as described but at the time of writing D7 there are no future development plans UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 This page is intentionally blank UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 1 Introduction Decision Maker is a prototype software tool that has been developed by the Defence Science and Technology Organisation DSTO for potential application in decision making problems within the System Program Offices SPOs of the Defence Materiel Organisation DMO and in other areas of the Australian Defence Organisation ADO Decision Maker is a tool that supports users throughout their decision making activities This includes problem formation decision analysis and result interpretation Structuring a problem within Decision Maker allows users to evaluate their problem using qualitative and quantitative models The process for developing these models in Decision Maker as shown in Figure 1 inherently provides tractability and documentation of the problem structuring process that users have undertaken This provides for an enhanced validation process when evaluating Decision Maker s output
52. In the example shown in Figure 106 Supplier 14 is clearly dominated by both Supplier 2 and Supplier 5 UNCLASSIFIED 84 UNCLASSIFIED DSTO GD 0681 Therefore Supplier 14 does not perform well on the sub objectives and or sub criteria that are the child elements of the decision tree i e Maximise Performance is comprised of the objectives Maximise Reliability and Minimise Supply Time Supplier 14 scored poorly in these two objectives and there is a direct relationship between the three Comparing Supplier 2 and Supplier 5 for the objective Maximise Performance requires investigation of the plot area indicated by Here Supplier 2 scores better than Supplier 5 and to determine the exact amount requires the use of the Scoring Matrix Viewer 11 5 3 Maximise Reliability Comparing Supplier 2 and Supplier 5 is a little more interesting for Maximise Reliability where Supplier 5 appears to dominate Supplier 2 In this example of 50 simulation runs Supplier 5 dominated Supplier 2 49 times for the objective of Maximise Reliability The Scoring Matrix Viewer can also be used to assist in determining the degree of domination for a single element Whilst the CWViewer provides a graphical display of domination across multiple elements simultaneously it is necessary to use the CWViewer or Domination Scoring Matrix to obtain a complete picture 11 5 4 Using Domination Scoring Matrix with CWViewer Example 1 Figure 107 presents a Domination S
53. LASSIFIED 67 UNCLASSIFIED DSTO GD 0681 CERPEN Lepr ea es Pra amina ion Sca ie a Decision Report Selection Existing DecisionMaker Il Simulations i Select best supply arrangement a Simulation Sets Simulation Run 1000 ei Supplie Supplie Supple Supplie Supplie Supplies Supple Supplie ye 13 Je VEI r B 13 it x fu Fi Ease eee xb a fom l a a we i f ie cafe LEE i an Pi JE pa E e a i zE a E z Drs f i zF on i si F E oI eh T 4 jt LJ L a La a a T Te c _ 3 E a lg 8 Ta ao wl 4 tae i 7 Des ler a 4 r lie i i S i ks Ta fs B gym nm p EE f Fi I a o This cell has been selected o A zoomed view of this cell is now displayed The column and row selections will be updated to show the two alternatives being compared Figure 88 Domination Scoring Matrix cell selection Table 13 Colour codes and corresponding range of performance UNCLASSIFIED 68 UNCLASSIFIED DSTO GD 0681 Woe Selection Column Supplier 13 iw Row Supplier 3 x Grid Zoom Scale J This shows an example of the appearance when the alternative criteria never scores better than the other alternative i e here Supplier 13 scores better 0 of the time for C 1 Minimise Total Cost This is not a surprising result since there was no uncertainty fo
54. Maximise Performanca dtemative oew y Maamise Reliability 3 o ec ive 0 500 VPP Qtat ee Ree eS i ene a eceer heat ale B P B E i Sunes 5 te Suppbet 7 E 5 Supple 6 E 00 95 Suppher 7 2 3 Supper 10 Suppher 13 A 3 Suppher 020 5 Supper 4 E g Supple 8 l 45 Suppber 2 aow a uppie E ci i Maanice Aehabilty Type A Spares Mirae Supply Time Typa Spaes 8 5E Minimise Total Cost Maxine Aielabity Typa B Spares Mirer se Supple Time Tipe B Spares o Decision Report Selection o Domination Tree View 0 300 E HE AE i i e fN O R F View Options Non Dominated Sets o Direct Relations Figure 113 Weights Sensitivity analysis window after selections 12 1 Total Domination An alternative completely dominates another alternative when a change of weights to any other value will not cause a change in ranking This is due to the criteria values of the dominating alternatives being more preferred than the lower alternatives Table 15 shows an example where Alternative A completely dominates Alternative B A consequence of this is that any change of weights will not cause Alternative B to rank better than Alternative A UNCLASSIFIED 91 UNCLASSIFIED DSTO GD 0681 Table 15 Total domination Alternative A 567 20 86 0 12 2 Partial Domination Partial domination occurs when some but not all criteria values for an alternative are more pr
55. New Criterion button Enter the information from Table 11 into the Wizard It should appear as shown in Figure 31 Table 11 Wizard example Criterion 5 Information Objective Minimise Supply Time Text The upper limit of the 95 confidence interval of the Mean Time To Supply MTTS of Type B Spares in hours hrs Preferred Direction Add Critena to Minimise Supply Time Objective Title Objective 3 of 3 Criterion Title Minimise Supply Time traed Minimise Supply Time Type B Spares SS ees avigation SS rea ae Objective Description Criterion Description The best spare part supply amangement shall have a short supply ees ae The upper limit to the 95 confidence interval of the Mean Time time to Supply MTTS of Type B spares in hours hrs Criterion Control Sub Objective Of Unit of Measure Enable Subjectivity 10 1 Maximise Performance MTTS hrs s Criteria assigned to Objective Direction of Preference c4 Minimise Supply Time Type A Spares 4 Minimise ka Figure 31 Wizard example add criterion 5 to Minimise Supply Time UNCLASSIFIED 27 UNCLASSIFIED DSTO GD 0681 All the criteria have now been assigned to the objectives Click the Finish button on the Wizard s dialog box The Wizard will now display a dialog box indicating that the process is complete This is shown in Figure 32 Click on Finish and the Data Model view will then appear as shown in Figure 33 Step 5 Problem Definition Comp
56. SIFIED DSTO GD 0681 Add Criteria to Select Best Supply Arrangement Problem Title Criterion Title Objective Navigation N gt Problem Description Criterion Description Select Best Supply Arrangement What is the best arrangement for the supply of Type 4 and B spare parts Criterion Control Unit of Measure Criteria assigned to Problem Direction of Preference C 1 Minimise Total Cost m Figure 27 Wizard example add criterion 1 to Select Best Supply Arrangement continued Table 8 Wizard example criterion 2 information Objective Maximise Performance Criterion Title Maximise Reliability Type A Spares Criterion Description The lower limit of the 95 confidence interval of the Mean p Time Between Failures MTBF of Type A spares in hours hrs Unit of Measure MTBF Preferred Direction Add Criteria to Maximise Performance Objective Title Objective 1 of 3 Criterion Title Maximise Performance Objective Maximise Reliability Type 4 Spares a a Navigation a E Objective Description Criterion Description The best spare part supply arrangement shall be able to supply Back The lower limit of the 9622 confidence interval of the Mean Time the parts with the highest performance in reliability and Supply Between Failures MTBF of Type 4 spares in hours hra time Next gt Criterion Control Hew Criterion Add Criterion Sub Objective Of Unit of Measure Enable Subjectivity Select Best Suppl
57. Spares C 0 1 2 Minimise Supply Time C 4 Minimise Supply Time Type A Spares C 5 Minimise Supply Time Type B Spares m eG SSS SEE T Ne gt n 7 SS a gs a s a Pl em wt a Sa aa _ re m a 5 7 e ri A AN n p 7 v s a a G A A cs a ee A Altemative Show Colour All Aternatives E Supplier 1 A gt i LD fi C 4a P 4 A ae ee A Ta Za lt lt S Supplier 2 Supplier 2 Supplier 4 Supplier Supplier 6 Supplier 7 Supplier 6 Supplier Supplier Jyplar i2 HEM 13 Supplier 3 T 4 ier 1 oooooooonoon aok 3 o In the Plot Element Selector only the criteria are selected Structural element P 1 is also selected This is highly recommended at all times The plot area now updates reflecting the selections made in Q The Cardinal Scoring Option is selected Figure 105 Filtering criteria elements 11 5 Interpreting Domination Alternatives Dominant alternatives are ones that score better in all areas when compared to another alternative Regardless of the changes made in the weights a ranking preference of such an alternative will not change The CWViewer is a useful tool in visualising dominant alternatives and for short listing the possible choices In the first example shown in Figure 106 the objectives root problem and top level criteria
58. Spares 0 190 0 180 0 170 0 160 0 150 0 140 Weight 0 130 0 120 0 110 0 100 0 090 Figure 82 Weights Ranges graphical example 2 8 5 3 Graphical Display Figure 83 shows a graphical representation of the weight data Here the weights are shown for each simulation run In this example the weightings used for Maximise Reliability Type B Spares are almost always greater than for Maximise Reliability Type A Spares The Weights Distribution plot has a toolbar where changes can be made to the format of the plot One item of particular use is the Gallery Chooser on the right hand side of Figure 83 where a different form of plot can be selected UNCLASSIFIED 63 UNCLASSIFIED DSTO GD 0681 Pee eee eee eee eee ee eee eee eee eee eee eee eet ee Cc ee ee To fp PJ Select best supply arrangement Scoring Range Ranking Distribution Weights Summary be Gi Minimise Total Cost 6 a Tarehe ben 0 1 Maximise Performance LLebular Dala Weights Ranges p a MA S DT T Maximise Bekiability Weights Distribution For P 1 Select best supply arrangement By C 2 Maximise Reliability Type 4 Spares i hee 0 3 Maximise Reliability Type B Spares El 0 1 2 Minimige Supply Time bef CA Minimise Supply Time Type A Spares ba o r o va ek P a 9 a A Fua hs a fi w E a p i LM Fa 1 py fy L N Fa 7 hall YS 2 CS5 Minimise Supply Time Type B Spares gt
59. T oe em Be Pa oe weet Y pA AN ngendntansiedsicessge ri iest pesati ta ai ett 10 15 20 25 40 C1 Minimize Total Cost C2 Masimise Reliability Type 4 Spares C3 Masimise Reliability Type E Spares S C4 Minimise Supply Time Type Spares C5 Minimise Supply Time Type B Spares Figure 83 Graphical display of the Weights Distribution plot 9 Scoring Matrix The Scoring Matrix Viewer can be used independently or in conjunction with the other analysis windows It is a useful guide in the initial analysis stages to assist in short listing alternatives for a more thorough analysis It provides a rapid method for identifying the more preferred solutions using a simple pivot table This table compares alternatives directly against one another for all objectives and criteria values based on an entire set of simulation runs The Scoring Matrix calculates the fraction of simulation runs that one alternative has scored better than another alternative for the overall problem or similarly for any objective or criteria The Scoring Matrix viewer can be opened from the Advanced Analysis Scoring Matrix menu as shown in Figure 84 File Edit View Project Tools Windows Advanced Analysis Simulation Reporting Scoring Matrix Cobweb Plot viewer Weights Sensitivity Criteria Sensitivity FQ Domination Scoring Matrix Fil Figure 84 Starting the Scoring Matrix When the Scoring Matrix Viewer is opened it
60. This is shown in Figure 96 UNCLASSIFIED 73 UNCLASSIFIED DSTO GD 0681 0 14 Lie 6 67 m 6 31 Select best ore p stably tank Ninos binimise A peh ries pele supply otal Cost erformmance eliability Reliability Reliability upply Time upply Time Supply Time aTargemeni Type A Spares Type B Spares Type A Spares Type B Spares C 5 o The top 10 scoring alternatives are displayed by default with a dark green colour 7 The Mid Range scoring alternatives are displayed by default with a pale blue colour The Lowest 10 scoring alternatives are displayed by default with a pale rust colour Stem Plots Cross Densities These are explained in Section 11 2 3 z Objectives and criteria can be selected as required ry Values on the axes can be turned on or off as desired Figure 95 CWViewer basic view without Cardinal Scoring Option UNCLASSIFIED 74 UNCLASSIFIED DSTO GD 0681 E ai og oT os 05 Di a o2 1 wae 1 a 0 5 07 9567 96 7 38 0 6 240 11 0 5 4 LTI 7012 25 8 1 TA J N i j j D4 70210 04 75844 04 85o Pd 8 46 N d Alis ri ri 0 3 R T2746 pos 7256 54 po3 ee Pe i 8 82 P o E i i J af E i Fd Ea j i P r Ke A a O24 ee PTRD ri f 0928 60 0 2 T nS ee 9 18 ee ee Be ieee cx Seat ake LSE 0 Prot Teh j 20 1 E a9 tab GGOO B3 0 1 es 9 53 ah 9 54 J y 3 E Ti saat 2 T a T S O o a am a EE kT sr i gt a N A A i
61. UNCLASSIFIED Australian Government Department of Defence Defence Science and Technology Organisation Decision Maker User Guide Carl Costolloe Moya Tyndall and Anthony Woolley Maritime Platforms Division Defence Science and Technology Organisation DSTO GD 0681 ABSTRACT Decision Maker is a prototype software tool developed by Maritime Platforms Division MPD of the Defence Science and Technology Organisation DSTO that can be applied to most multi criteria decision making problems Decision Maker is a software implementation of the Criteria Importance Through Intercriteria Correlation CRITIC decision making technique The decision making problems use a set of criteria and objectives to select the most preferred alternative in a set of alternatives In most problems there are conflicting criteria objectives and therefore complex trade offs have to be made between competing alternatives This is where Decision Maker can be used This user guide will assist and inform users on how they can structure their decision problems in the required format for Decision Maker RELEASE LIMITATION Approved for public release UNCLASSIFIED UNCLASSIFIED Published by Maritime Platforms Division DSTO Defence Science and Technology Organisation 506 Lorimer St Fishermans Bend Victoria 3207 Australia Telephone 03 9626 7000 Fax 03 9626 7999 Commonwealth of Australia 2012 AR 015 300 April 2012 APPROVED FOR PUBLIC REL
62. UNCLASSIFIED DSTO GD 0681 New Objective 3 leg Objective Title Use this field to enter a short title m Ml for your objective Objective Title Objective Description Enter a detailed 1 New Objective description of your objective in this field the Objective Description more detail the better Sub Objective Of This field allows you to select any other Objective element including your gt Problem element which your objective decomposes Setting this field will establish a hierarchical tree of objectives in your decision problem By default each new Objective element is set as a sub objective of your Problem element Preferred Direction of Objective This has been disabled It is recommended that the preferred direction for objectives is described in the Sub Objective Of Objective Title or Objective Description For 3 example one objective may be to maximise New Decision Problem cual profit This objective may be decomposed into bap etrd Cremation eo bl baie two other sub objectives such as minimise 1 expenditure and maximise reliability These sub objectives will have their own set of criteria as discussed in Section 5 3 4 to evaluate their Cancel performance in your decision problem Figure 38 The New Objective dialog box F ii New Alternative ax Alemative T ithe New Alteratrve Akematve Disenplion Alternative Title Use this field to enter a short title for y
63. Whilst it might be convenient and or expedient to skip this step it is unwise since this step forms the basis for all the other steps in the decision making process It is important to think creatively and look at how the problem could be turned into an opportunity Do not constrain your problem definition by including possible alternatives at this stage since this could prevent consideration of other alternatives that may not be as visible It may take time to define your problem and it is advisable to re examine your problem definition as you work through the remaining steps Time spent here may prevent undesirable delays and consequences later The following example will be used to demonstrate each of the steps in the PrOACT decision making process as shown in Figure 3 You may choose to follow this example or substitute your own decision problem as you work through the steps EXAMPLE Supply Manager s Dilemma PrOACT STAGE Problem Definition The Supply Manager is in the process of reviewing the existing contracts that supply Type A and Type B spare parts to his firm These contracts are about to expire and he needs to consider whether he should renew the existing contracts or not Currently Widget Inc has the contract to supply Type A spare parts and Gadget Inc has the contract to supply Type B spare parts to his firm Both companies have the ability to supply Type A and B spare parts however they did not have this ability when the con
64. and the Wizard Now that the Supply Manager has an understanding of the PrOACT decision making process from reading Section 3 he has an initial and possibly incomplete list of objectives alternatives and criteria The Supply Manager is now ready to use Decision Maker to analyse the supplier selection problem The Supply Manager has chosen to use Decision Makers Wizard to expedite the initial problem structuring process The steps the Supply Manager must perform to complete the decision making activities using Decision Maker are 1 create a new project using the Wizard create the problem s structural elements using the Wizard assign data to each of the problems criteria calculate the scores for each alternative in the problem chart the results of the decision analysis process and revise and amend the problem s structure oO oS Sy UNCLASSIFIED 11 DSTO GD 0681 4 2 Using the Wizard UNCLASSIFIED Decision Maker s Wizard is a tool that will guide you through the process of structuring your decision problem in Decision Maker as shown in Figure 7 The Wizard is best used when you know the PrOACT elements to your problem and you want to quickly structure the problem in Decision Maker for further analysis Wizard Process Step 1 Step 2 Create a new Decision Project Define the Decision Problem Step 3 H Define the Objectives Step 4 H Define the Alternatives Step 5
65. arge problem structured in Decision Maker the process may take a few seconds to complete The results from the calculation are presented in the Data Grid under the column heading Cardinal Score The information in this column is the normalised result of the CRITIC 1 decision analysis process The calculated results are displayed against each of the problem s alternatives This enables you to quickly assess the performance of each alternative 6 2 3 Calculation Errors Decision Maker requires data for at least one alternative within each criterion If you create a new Criterion element the data values will be set at the default value of zero 0 If all data are of the same value Decision Maker cannot calculate a result and NaN will be displayed for the objective as shown in Figure 56 If the criterion is far down the hierarchy the error will propagate up the hierarchical tree to the top most Problem element To remove this error you are required to add data to your decision problem This requires you to ensure all Criterion elements have data for each alternative which will then ensure that Decision Maker s output is valid based on the data you have provided 4 Not a Number UNCLASSIFIED 45 UNCLASSIFIED DSTO GD 0681 New Criterion Qe Delete Element Properties E Minimise Total Cost Title 2 Masimnice Performance loian Supply Tene g Maximise Reliability sat lt Maximise Reliability Type A Spares
66. at Type B Change Uncertainty Values Spates Refresh Supplier 1 7200 Supplier 2 9100 Supplier3 6600 Suppier4 6800 Supplier 5 3100 B opet eco TE Supplier 6800 Select the Criteria to which uncertainty is to be added and then click the right mouse button to open the menu Select Add New Element Uncertainty Figure 41 Adding uncertainties UNCLASSIFIED 37 UNCLASSIFIED DSTO GD 0681 LY Would you like to set an uncertainty value for all alternatives Yes Figure 42 Uncertainty confirmation prompt Uncertainties are defined as a percentage and will be applied Uncertainties are defined as a percentage and will be applied to all altematives for this Criteria to all altematives for this Criteria CrteralD E3 CrteialD C3 Criteria Maximise Reliability Type B Spares Criteria Maximise Fieliabiity Type B Spares Value Value 0 05 5 0 0 1 Maximise Performance OO pete de Maximise Relabity Type B Spares A Uncertainty in Maximise Reliability Type B Spares Description C2 Maximise Reliability Type A Spares The lower limit of the 95 confidence interval of the Mean Time A Uncertainty in Maomise Reliability Type A Spares Between Failures MTBF of Type B spares in hours hrs 0 1 2 Minimise Supply Time C 4 Minimise Supply TimeType A Spares A Uncertainty in Minimise Supply Time Type A Spares C 5 Minimise Supply Time Type B Spares A Uncerta
67. atives Decision Maker uses a genetic search algorithm technique to locate the smallest variation on weights that will cause a ranking reversal between two alternatives The weights sensitivity view also displays Pareto domination in a tree view to assist in multidimensional analysis of weights sensitivity The Weights Sensitivity analysis window can be started from the Advanced Analysis Weights Sensitivity menu as shown in Figure 111 File Edit View Project Tools Windows Advanced Analysis Simulation Reporting Scoring Matrix Cobweb Plok viewer Weights Sensitivity Criteria Sensitivity Domination Scaring Matrix Fil Figure 111 Starting Weights Sensitivity analysis The Weights Sensitivity analysis window will appear as shown in Figure 112 After selections have been made in the window s various components it will appear as shown in Figure 113 gt This is an Optimisation Search Algorithm which is based on the principles of biological genetic evolution After the Pareto principle also known as the law of the vital few or the principle of factor sparsity UNCLASSIFIED 89 UNCLASSIFIED DSTO GD 0681 Simulation Details z Number of Altematives 16 Number of Simulation Runs 50 Simulation Date 1 06 2007 10 43AM CI Show Akematne Rankings CI Show Crteria in Tree Show Levels with Mouse Number of Searches E lt Allernatives E G 01 Maximise Penormance Decision Report Sel
68. blic release OVERSEAS ENQUIRIES OUTSIDE STATED LIMITATIONS SHOULD BE REFERRED THROUGH DOCUMENT EXCHANGE PO BOX 1500 EDINBURGH SA 5111 16 DELIBERATE ANNOUNCEMENT No Limitations 17 CITATION IN OTHER DOCUMENTS Yes 18 DSTO RESEARCH LIBRARY THESAURUS Decision Making CRITIC PROACT 19 ABSTRACT Decision Maker is a prototype software tool developed by Maritime Platforms Division MPD of the Defence Science and Technology Organisation DSTO that can be applied to most multi criteria decision making problems Decision Maker is a software implementation of the Criteria Importance Through Intercriteria Correlation CRITIC decision making technique The decision making problems use a set of criteria and objectives to select the most preferred alternative in a set of alternatives In most problems there are conflicting criteria objectives and therefore complex trade offs have to be made between competing alternatives This is where Decision Maker can be used This user guide will assist and inform users on how they can structure their decision problems in the required format for Decision Maker Page classification UNCLASSIFIED
69. can be achieved by adding more objectives adding more alternatives adding uncertainties running a simulation and analysing the results oe a a defining more criterion elements for your objectives and or moving elements throughout the problem structure When you have gained understanding of your problem you may identify additional objectives that you want to include in your project Both the Structure Model and Data Model views provide the tools to add and delete the elements for your problem s structure UNCLASSIFIED 30 UNCLASSIFIED DSTO GD 0681 4 7 Remarks Do not forget to save your work regularly Decision Maker saves your project using Simple Object Access Protocol SOAP which is an eXtended Mark up Language XML file format This file format is readable using most text reader applications 5 Structuring Your Problem in Decision Maker Before you begin entering your decision problem it is important to understand the decision making process that you will undertake while using Decision Maker 5 1 PrOACT Process To begin solving your decision problem in Decision Maker it is recommended that you first think about your problem in terms of the PrOACT process Section 3 This process begins with steps where you are required to clearly define for your Problem and specify your Objectives The next activities undertaken in the PrOACT process include the construction of lists of any possible alternatives and their Con
70. companies either supplying Type A or Type B spare parts 3 4 Consequences In this step the benefits of each of the competing alternatives are considered by assessing how well they fulfil the objectives of your problem To do this appropriate attributes scales or measures are needed for each objective These may include 1 costs such as operating costs and expected profit loss 2 measures of performance such as failure rate and fuel efficiency and 3 characteristics of the objectives The inclusion of accurate and appropriate data will enable you to make better choices If the attributes scales or measures you wish to use are descriptive in nature such as those you might use to describe comfort or colour then you will either need to convert them to numbers or use an alternative decision making technique 2 3 and 4 in the Trade offs step Section 3 5 Note Decision Maker uses a mathematical process to determine the ranking of alternatives hence numerical measures are required When you have determined how you will measure each objective it is then time to collect the data and organise it in a consequence table Using a spreadsheet build a table with the alternatives list on the left hand side and the objectives along the top Note this differs from the consequence table described in 4 where the alternatives are along the top and the objectives down the left hand side The consequence table presented in this user guide
71. coring Matrix to assist in determining the degree of domination for individual elements CECA es Decision Report Selecion Smnulstion Detais Existing DecissonMaker I Srmulahans Number of Akematives 16 Select best supply arrangement Number of Simulation Runs 50 Simulation Sets Simulation Run 000 x SmmulationDate 1 06 2007 1049 4M Created By CostoliC P1 Select best supply anangement C C1 Minimise Total Cost O 0 1 Maximise Performance O 0 1 1 Maximise Reliability C C2 Manmise Retianility Type A Spares gt C C3 Mammise Reliability Type B Spares O 0 1 2 Minimise Supply Time e C4 Minimise Supply Time Type A Spares t C C5 Minimise Supply Time Type B Spares t The analysis of Supplier 2and Supplier 5 shows that Supplier 5 scored better than Supplier 2 in 52 of the simulations Analysis of the objective Maximise Performance in the CWViewer indicated that Supplier 14 scored lower than Supplier 2 and Supplier 5 in all simulation runs This is confirmed by the value that indicates the column alternative i e Supplier 2 and Supplier 5 scored higher than Supplier 14 100 of the time ry Similarly Supplier 2 scored higher than Supplier 5 in 48 of the simulation runs Figure 107 Domination Scoring Matrix interpretation maximise performance UNCLASSIFIED 85 UNCLASSIFIED DSTO GD 0681 AN Reminder Ensure that the correct element is selected 11 5 5 Using Domination Scoring Matrix with CWViewe
72. cuments My Computer File name Decision M aker D B_ mreporting dmzip My Network Files of type Decision Maker Zip Archive dmzip x Figure 140 Import DMZip Archive dialog box Finished EA Import Complete Figure 141 Import Complete dialog box 14 2 5 Open Database To open an existing database click the Open Database button as indicated by in Figure 138 An Open Simulation Database dialog box will appear as shown in Figure 142 Browse to the location of the database file select the file and click Open When the file has been opened the dialog box shown in Figure 143 will appear The details of the current database indicated by Q in Figure 138 will update to reflect the newly opened file UNCLASSIFIED 112 UNCLASSIFIED DSTO GD 0681 Open Simulation Database Look in E Debug SG ie e y l i My Recent Mysimbatabse mdf Documents Asimreporting mdf IF test_1 mdf a Desktop m DM Logs My Documents ya My Computer File name hi ySimDatabse milf iv My Network Files of type SOLE xpress Database mdF Figure 142 Open Simulation Database dialog box Figure 143 Database File Opened dialog box 14 2 6 Create Database To save a database click the Create Database button as indicated by in Figure 138 A Save As dialog box will appear as shown in Figure 144 To save the database
73. d In this example the MTTR criterion will have a Direction of Preference set to minimise and the Cancel _ o alternative with the lowest MTTR will be the preferred alternative Conversely the MTBF criterion will have a Direction of Preference set to maximise and the alternative with the greatest MTBF will be the preferred alternative Decision Maker will analyse your decision problem based on the MTTR and MTBF data and their preferred direction Figure 40 The New Criterion dialog box UNCLASSIFIED 36 UNCLASSIFIED DSTO GD 0681 5 4 5 1 Adding Uncertainties To add uncertainties to your criterion follow the steps shown in Figure 41 After completing the steps in Figure 41 the confirmation prompt shown in Figure 42 will appear Selecting No will set all uncertainties to a default value of zero This is useful if each alternative has a different value to set or if the values will be copied from a Microsoft Excel spreadsheet If every alternative is to have the same value click Yes If Yes is selected a dialog box will appear on the screen asking for the value to set as shown in Figure 43 In the Value field enter a number between 0 and 1 For example enter 0 05 for a 5 uncertainty as shown in Figure 44 Click OK to confirm and the Data Model will be updated as shown in Figure 45 5 4 5 2 Changing Uncertainties Uncertainties can also be changed for all alternatives simultaneously as show
74. d by a maximise the reliability of the spare parts and b minimise the supply time of the spare parts Note that the second objective is decomposed into two sub objectives The two sub objectives are used to evaluate the overall performance objective for the spare parts Click the Next button until the dialog box shown in Figure 18 is displayed Change the Number of Objectives to 3 When the number of objectives has been changed click Next until the dialog box shown in Figure 19 is displayed Tables 4 5 and 6 present the information required for the three objectives Enter the information presented in Table 4 into the Define Objective Number 1 dialog box The completed dialog box should appear as shown in Figure 19 When the information is entered click the OK button Step 3 Define your Decision s Objectives kal x This step requires you to define your objectives Objectives help to explain your Member of Objectives choices to others and itis importantto spend time considering and defining them some of the ways that may help you to identify your objecthves include 1 Making a wish list Making alist of what you wantto avoid Flease enter the number objectives that you currently know of you don t have any objectives atthis stage this is okay as you can enter them later Figure 18 Wizard example Step 3 Define your Decision s Objectives continued UNCLASSIFIED 19 UNCLASSIFIED DSTO GD 0681 Table 4 Wi
75. d out of the Plot Element Selector 71 UNCLASSIFIED DSTO GD 0681 File Edit View Project Tools Windows Advanced Analysis Simulation Reporting Scoring Matrix Cobweb Plot Viewer Weights Sensitivity F8 Criteria Sensitivity F9 Domination Scoring Matrix Fil Figure 92 Starting the CWViewer Decision Report Selection Plot Options Existing DecisionMaker Simulations Top Percentage Lirit 0 9 Show Top Siconng Altematives F Show Axis Values nbeere Gi Zl Show bide tomate Canal Sorin Opt SNARES Srn Rw E Show Low Scoring Altematives 1000 iv Pp PA Select best supply arrangement Eg C1 Minimise Total Cost t i 0 1 Maximise Performance o3 0 1 1 Maximise Reliability a1 ped C2 Maximise Reliability Type A Spares on bef C3 Maximise Reliability Type B Spares Da 0 1 2 Minimise Supply Time at pf C4 Minimise Supply Time Type A Spares t a E 0 5 Minimise Supply Time Type B Spares t 1 Select best supply arrangement C1 Minimise Total Cost v 0 1 Maximise Performance 0 1 1 Maximise Reliability a ety tS _ ion i ey C3 Maximise Reliability Type B Spares 7 0 1 2 Minimise Supply Time H C4 Minimise Supply Time Type A Spares 7 C5 Minimise Supply Time Type B Spares Alternative 7 i ee i Al I Alten Tatives Supplier 1 Suppli
76. d their relative weights Shows the rank of the criteria that are selected o Displays the criteria in the tree view This is the number of searches that will be performed Usually 20 50 will give a good range of results Large search numbers will increase the processing time Figure 116 Weights Sensitivity view options 12 6 Non Dominated Sets Table When a node in the Domination Tree is selected the Non dominated Sets table is automatically populated This provides a quick summary of the change required in the weightings of the criteria to produce a ranking reversal of two alternatives The Upper Alternative is the alternative with the higher cardinal score and the Lower Alternative is the alternative with the lower cardinal score Remember the best ordinal score is 1 and the worst ranking is equal to the number of alternatives in the problem The list of lower alternatives contains all the alternatives that are not dominated by the upper alternative based on the selected criteria or objective values used to determine domination Figure 117 shows an example of top level analysis where all problem criteria are considered Remember a high cardinal score translates to a low ordinal ranking preferred alternatives while a low cardinal score translates to a high ordinal score non preferred alternatives UNCLASSIFIED 94 UNCLASSIFIED DSTO GD 0681 Maximise Maxine Minimise Siupony Minimize Supply Reliability Type a Relabdi
77. e E vie w Do gm pasi 5796 2 16272 08 eG p 9 86 AE ae e Ladja ae Lp hvfaocinni se tha pina ol wantin pia Sup otal Cost erfonmance eliabaltt y Reltability Relabilit Supply Time Supply Time _ Supply Time arrancerreent Type A Spares Type rA Spares Typ oo Typ k an The top 10 scoring alternatives are displayed by default with a dark green colour The Mid Range scoring alternatives are displayed by default with a pale blue colour The Lowest 10 scoring alternatives are displayed by default with the pale rust colour Stem Plots Cross Densities These are explained in Section 11 2 3 Note the Minimise Total Cost axis now has the lowest cost at the top ry Minimise Supply Time shortest supply times are now at the top of the axis Figure 96 CWViewer plot area with Cardinal Scoring Option 11 2 3 Stem Plots Cross Densities Stem Plots are histograms of the crossing point between a pair of axes and are used to determine the relationships between criteria within a decision This is shown in Figure 97 UNCLASSIFIED 79 UNCLASSIFIED DSTO GD 0681 7 9 88 P 9 52 9 17 8 81 6 45 75 167 94 6927 TF Wie 7 02 a A _ B w a ki l i a R Tae a 5982 17 0509 O07 hoe 665 8 5796 4 G27 1 72 h 6 E 6 33 Select best viinartii biarimise Wiaximise bimimis hitaximmise hinimise hinimise hinamise Supple Performance Reliability Reliability Reliabaity Supply Tite Supply Time Supply Time airangermernt T
78. e or all of the criteria for the various alternatives in the Data Model view To run a simulation select the Tools Run Simulation menu option as shown in Figure 58 The Simulation Settings dialog box shown in Figure 59 will then appear File Edit View Project Windows Advanced 4nalysis Help Run Simulation Generate Executive Summary Import Alternatives From File F10 Options Figure 58 Running a simulation UNCLASSIFIED 47 UNCLASSIFIED DSTO GD 0681 s Simulation Settings 5 Ed Number of Simulation Runs 10 Run Simulation Figure 59 The Simulation Settings dialog box Number of Simulation Runs is the number of times the simulation will calculate the problem It is recommended that at least 50 runs are selected since this provides better reporting data and data sampling When set click on the Run Simulation button to proceed The confirmation prompt shown in Figure 60 will then appear Confirm 2 Start Simulation Figure 60 Run simulation confirmation prompt Click the Yes button to start the simulation Clicking No will return to the Simulation Settings dialog box Figure 59 When the simulation process has begun the progress will be visible as shown in Figure 61 5 Simulation Settings La E Humber of Simulation Auns 200 j Ti ee Cancel Figure 61 Simulation progress dialog box When the simulation is complete the dialog box shown in Figure 62 will appear Click on the OK bu
79. e Limit og show Top Scoring Altematives Show Axis Values select best supply arrangement iv ee int or Show Mid Range Alternatives L Cardinal Scoring Option Simulation Sets Simulation Aun Show Low Scoring Altematives P 1 Select best supply arrangement a 0 1 Maximize Performance G 0 1 1 Maximise Reliability a C C2 Maximise Reliability Type A Spares BO 0 1 2 Minimise Supply Time ut C C4 Minimise Supply Time Type A Spares Ci H C5 Minimise Supply Time Type B Spares t P 1 Select best supply arrangement C1 Minimise Total Cost 0 1 Maximise Pertormance 0 1 1 Maximise Reliability F C2 Maximise Reliability Type 4 Spares e 0 3 Maximise Reliability Type B Spares 0 1 2 Minimise Supply Time C4 Minimise Supply Time Type Spares F C5 Minimise Supply Time Type E Spares Alternative Show Colour All Alternatives Supplier 1 Supplier 10 I Ny a e i _ alk Uh f 55000 amp f agiagi O Select best Minimise Waximise heximise haximise haximise Minimise hinimise Minimise supply Total Cost Performance Reliability Reliability Reliability Supply Time Supply Time Supply Time arrangement EJ Type paris Type o Type te Type isha Figure 101 CWViewer filtering alternatives example 1 continued UNCLASSIFIED 79 UNCLASSIFIED DSTO GD 0681 Decision Report Selection 1 Plot Options Existing DecisionMak er II Simulat
80. e Performance 0 1 1 Maximise Reliability 0 C 2 Maximise Reliability Type A Spares C 3 Maximise Reliability Type B Spares 0 1 2 Minimise Supply Time C C 4 Minimise Supply Time Type Spares C C 5 Minimise Supply Time Type B Spares gt Alternative Show Colour All Alternatives O Supplier 1 BE al Supplier 2 Supplier 3 Supplier 4 Supplier 5 Supplier 6 Supplier 7 Supe 6 Supplier 9 Supplier 10 Supplier 11 Supplier 12 Supplier 13 Supplier 14 Supplier 15 Select best Minimise Maximise S d supply Supplier 16 arrangement oosoooooododsoloso Minimise hae ee Performance Reliability Supply Time max Figure 106 CWViewer interpreting dominant alternatives Each of the selected elements and of Figure 106 are described in the following subsections 11 5 1 Minimise Total Cost It is important to recognise that this is a criteria axis and the preferred direction is to minimise In the example shown in Figure 106 the Cardinal Scoring Option is selected and this means the more preferred criteria values occur at the top of the Minimise Total Cost axis Therefore Supplier 2 dominates Supplier 5 which in turn dominates Supplier 14 11 5 2 Maximise Performance This is an objective and as a consequence the preferred alternatives will intersect the axes near the top regardless if the objective is to minimise or maximise
81. e file Figure 136 shows this confirmation prompt Confirm es re you sure you Wank to Delete the Selected Simulation This action is Permanent and cannot be undone Cancel Figure 135 Confirm delete selected simulation Confirm re you sure vou want to clear the Entire Simulation Database This action is Permanent and cannot be undone Figure 136 Confirm delete all simulations In both cases No and Cancel will return you to the Database Management options tab Clicking on Yes will perform the requested action s When the data has been cleared a notification will display as shown in Figure 137 Complete l All Data has been cleared Figure 137 Deleting data completed 14 2 2 Database File Administration An example of the Database File Administration tab is shown in Figure 138 14 2 3 Export to Zip Archive This utility compresses the current database file and creates a new file on the user desktop with the file extension dmzip The file name will be created automatically and include the name of the database file When the simreporting database is exported the following file will be created Decision Maker_DB_simreporting dmzip UNCLASSIFIED 110 UNCLASSIFIED DSTO GD 0681 AN DO NOT RENAME THIS FILE If it is renamed there will be problems when attempting to import the file To create an export file click the Export To Zip Archive button as indicated by O in Figure 138
82. e objectives are selected Structural element P 1 is also selected This is highly recommended at all times The plot area now updates reflecting the selections made in 9 The Cardinal Scoring Option is selected however given that the filtering is by objective this has no effect on the display Figure 104 Filtering objective elements 11 4 2 Filter by Criteria The viewer can be used to display Criteria elements only This assists identification of how alternatives perform in each of the criteria areas Figure 105 shows an example of filtering by criteria UNCLASSIFIED 82 UNCLASSIFIED DSTO GD 0681 Decision Report Selection Plot Options Existing DecisionMaker II Simulations Top Percentage Limit 0 9 C Show Top Scoring Altematives Z Show Asis Values i lt p 5 n Select best supply arrangement Low P tage Link 01 C Show Mid Range Altematives Scoring Option Simulation Sets Simulation Run C Show Low Scoring Altematives foo Ss f oe Ci Minimise Total Cost 3 O 0 1 Maximise Performance 0 1 1 Maximise Reliability C C 2 Maximise Reliability Type A Spares i amp C3 Maximise Reliability Type B Spares 3 0 0 1 2 Minimise Supply Time b C4 Minimise Supply Time Type A Spares V P 1 Select best supply arrangement C 0 1 Maximise Performance Se O 0 1 1 Maximise Reliability V C 2 Maximise Reliability Type A Spares y V C 3 Maximise Reliability Type B
83. ection o Domination Tree View Section 12 2 View Options Section 12 3 oO Non Dominated Sets Section 12 4 A tabular view of the search results Minimum Change Distribution Section 12 7 Direct Relations Section 12 5 Figure 112 Weights Sensitivity analysis window UNCLASSIFIED 90 UNCLASSIFIED DSTO GD 0681 fs Weigh eniti l i oe m Deciso Hepat Selecton mulation Dehat Wiem Omit Existing DecitonMaker I Simulations Kumis of bananer JE Abtohte Change E Show Top Level Analysis Select best supply anangement v humbe of Simulation une 50 Percentage Change C Show Akemnatine Rankings Simulain Sets Simulation Fun T Actual Valhe CI Show Criteria in Tres D 1 10494M_ 7000 I ly Simulation ae 06 2007 40 F Show wath Mousa Number of Searches Created By Costa a 5 x P Selectpest supply arrangen Non Dominated Sets 4 Abemar r eit Supper 2 Supober 11 3 Suppher 12 i Supple 14 Supober 13 Suppker 15 Suppker 16 Suppber 4 E of Suppher 5 B Guppies 13 Suppber 14 Suppb r 15 Supplier 16 Supper 4 occ sd B Suppber 9 5 Supper 12 Suppber 14 Se Suppber 1 in eo ete Direct Fetstons Supplier 6 3 Suppber 6 Minimum Weights Distnbution For Rank Change P 1 Supplier 3 vs Supplier 11 Supper 10 1000 Suppber 11 Suppher 15 Supplier 16 im Suppber 7 Suppher 13 tew Suppber 14 Supple 4 n700
84. ed as either cardinal or ordinal An ordinal ranking only provides an order ranking of the alternatives A cardinal ranking gives the order ranking and how much the alternatives differ e g alternative A is preferred twice as much as alternative B UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 3 6 Revise The Revise step has been included as a separate step to remind users that at any stage of the decision making process as presented in this user guide see Figure 3 you need to check recap and reconsider your problem definition objectives alternatives and consequences If you are not satisfied with the preferred choice and ranking as a result of using Decision Maker then you need to reconsider whether you have captured your decision problem adequately The omission of an important objective can lead to an unsatisfactory result Other factors to include and consider are uncertainties in the consequences the level of risk you are willing to accept and whether the decision is linked to others A full explanation of these factors and suggestions on how to include them in your decision problem are given in 4 EXAMPLE CONTINUED Supply Manager s Dilemma PrOACT STAGE Revise your decision problem The Supply Manager is reasonably satisfied with the ranking presented in Figure 5 but wonders whether he should have minimise administration paperwork as an objective given that the monetary savings could more than compensate for the addi
85. eferred than the second alternative s criteria values When this occurs there is a possibility of the ranking order changing when the weights are changed In Table 16 Alternative A ranks higher than Alternative B Now if Criteria Zs weighting was increased Alternative B would be the more preferred alternative In this example the weights would have to be unrealistic for this to occur hence the reason for weights sensitivity analysis Table 16 Partial domination AlternativeA 56 7 160 8 10 12 3 Some Final Notes on Dominating Alternatives Within a problem s decision tree it is feasible that at various levels two alternatives can dominate each other at different points in the decision tree However at the root problem level top level they would not dominate each other This is best understood using an example from the Supply Manager s Dilemma as shown in Figure 114 12 4 Domination Tree View The Domination Tree View has several features to assist in determining the level of domination for an alternative i e the structure of the tree indicates the hierarchy of domination For example in Figure 114 Supplier 9 dominated Supplier 10 and Supplier 1 Supplier 1 in turn dominated Supplier 13 which dominated Supplier 7 which dominated Supplier 4 The icons also indicate the basic domination categories of alternatives Figure 115 describes the Domination Tree icons UNCLASSIFIED 92 UNCLASSIFIED DSTO GD 0681
86. em Todo this appropriate attributes scales or measures are needed for each of the problem s objectives The following stages will require you to specify measures for each objective Fossible measures may include Cost Measures Operating Cost Lite Cycle Cost Expected Profit Loss Performance Measures Failure Rate Litres per 100km Travel Time For each measure you define you must also define a direction of preference The direction for each measure may ether be Minimise or Maximise For example 1 An Operating Cost measure you would tyoically want to Minimise 2 Whereas an Expected Protit measure you will want to Maximise Flease now step through each objective and define their measures Deere tn ee Figure 25 Wizard example Step 5 Define the Criteria to Measure your Objectives Clicking the Next button again will display the Add Criteria dialog box as shown in Figure 26 Figure 26 steps you through entering the criterion information using the information presented in Table 7 When all the information is entered click the Add Criterion button UNCLASSIFIED 23 DSTO GD 0681 UNCLASSIFIED Add Criteria to Select Best Supply Arrangement Problem Title Select Best Supply Arrangement Problem Description What i the best arrangement for the supply of Type A and B spare parts Q citeion Title Objective Minimise Total Cost Navigation a Criterion Description _ 2 The total cost
87. en wee tne wot cent iver opal Tewosenensacinepenee O Peseta cent are independent and Sly proportione Tecra sements are independent and Sc proportione O Pesca sements are independent and Sc proportione Figure 98 Cross Density Plots The data displayed in the Stem Plots is for all alternatives and does not change even when ltering is applied 11 3 Filtering Alternatives The CWViewer can filter alternatives for one to one comparison or many to many comparisons Figure 99 illustrates how to filter alternatives while Figure 100 shows the colour chooser to manually change the colours for each alternative The plot area should now appear as shown in Figure 101 Figure 102 shows the CWViewer Filtering Options when various options have been deselected to make it easier to view the plot UNCLASSIFIED 77 UNCLASSIFIED DSTO GD 0681 t Decision Report Selection E jons r r Exasting DecisionMakes Il Simulations i Soi Show Top Scoring Alternatives V Show Axis Values Select best supply arrangement ae Z Show Mid Range Altematives inal Sconng Option Simulation Sets Simulation Run _ Show Low Scoring Attematives m O 5 fp l E C1 Minimise Total Cost 5 0 0 1 Maximise Performance S O 0 1 1 Maximise Reliability om C2 Maximise Reliability Type A Spares E C3 Maximise Reliability Type B Spares 5 0 0 1 2 Minimise Supply Time mo C4 Minimise Supply Time Type A Spares im C
88. er 2 Show Colour Supplier 3 Supplier 4 E Supplier 5 E Select best Minimise Maximise Warimise Mamimise hiimise Minamnaise binaries Minimis es s Supply Total Cost Performance Reliability Reliability Reliability Supply Time Supply Tire Supply Time Supplier 6 El arrangement S Type e Type 8 Spanes Type i Spares Type stares Simulation Selector Simulation Decision Reporting Tree _ Q PlotArea Figure 93 CWViewer overview 11 1 Plot Options Figure 94 presents a closer view of the Plot Options area of the CWViewer Figure 93 UNCLASSIFIED 72 UNCLASSIFIED DSTO GD 0681 oN Top Percentage Limit oa show Top Scoring Altematnves Show Akts Values Low Percentage Limit 0 1 Show Mid Range Altematves Cardinal Scoring Option Show Low Scoring Altematnes Top Percentage Limit any alternatives with a cardinal score for element P 1 that is above this value will be deemed to be Top Scoring alternatives o Low Percentage Limit any alternatives with a cardinal score for element P 1 that is below this value will be deemed to be Low Scoring alternatives Cancels any change to the Top Percentage Limit or Low Percentage Limit Applies changes to the Top Percentage Limit or Low Percentage Limit and redraws the plot e Alternatives with a cardinal score above the Top Percentage Limit will be coloured green e Alternatives with a cardinal score below the Low Pe
89. es o Use the 2D 3D switching option if it becomes difficult to interpret the graph or to select Figure 79 Filtering alternatives Ranking Distribution Graphical 2D 8 5 Weights Summary Weights Summary can be used to view the weights distribution calculated for all the criteria and objectives in the decision problem There are three sub tabs in the Weights Summary tab Tabular Data Weights Ranges and Graphical Display Each is described in the following subsections 8 5 1 Tabular Data Selecting an objective or problem node in the decision tree will update the weights summary to show the weightings used for the child criteria and objectives Figure 80 shows an example of the Tabular Data tab within the Weights Summary tab UNCLASSIFIED 61 UNCLASSIFIED DSTO GD 0681 z Tt Minimiae Total Cost 01 Maximise Pertornance C Maamise Aelian aal Coat oy ie Reb Foe Surah Te Typa Sup i ine C3 Mamie Rella l PE S ee i 4 2 Minimise Supply T C4 Minimise Supp CS Minimise Suppl 013700 GARET 011333 042829 ALEP O4275 012557 013656 gin dizi 01352 O12 011560 ona TECTITI 012674 013304 0 1 e2 O15700 01202 giiia OAZ 23 0 13066 0 1405 giz UAFU 0 115 O12 0 14353 0 14500 0 15065 012702 DINH UREHIEI 012508 0 11566 0 14834 iza LEELEE 014285 0 10415 120 0 12706 01307 0 14524 012937 10501 0 15321 0 14750 0 1251 O12 EA i The tree view is
90. essosssoeesosesoeesosssosesoesssessosssos 40 6 ANALYSING YOUR PROBLEM IN DECISION MAKER ossesseesesseoseesessessessessesssss 41 6 1 The Structure Model Modifying your Problem s Structure eee 42 6 2 The Data Model Adding Data and Analysing your Problem 42 6 2 1 Adding Data using the Data Grid sec ss ictitintieiad eis Mee 44 6 2 2 Calculating Results in the Data Model 45 6 2 3 CANAAN On EFON S aeria ea a atte tae E See 45 6 2 4 Charting Results in the Data Model eeeeeeeeeerererererrrerrerren 46 7e SIMULATIONS sats ccswkicte hati teus ates ees a a E 47 7141 MUNNING aSim laitoM nnui a a 47 5 SIMULATION REPORTING osineen E 49 8 1 The Simulation Reporting Interface seeosoeosoossosssosssosssoessoessoessosssesssesssesssoss 49 8 1 1 Decision Keport selecHo neiuna iraa a E ES 49 8 1 2 Simulanom Details S ennea a T S 51 8 1 3 Simulation Reporting Decision Tree eeeeeeeerrereererrerrrererrerrerees 51 8 1 4 PLORO e Da eaae a E R 51 BZ Scorn RINGE orison a ase teds 52 8 2 1 Cardinal Scoring Ranges seesesseseereriereererrerressrrsrrsrrsresresresresreseesees 52 8 2 2 Ordinal SC OLIN Ranges misesani a 53 8 2 3 Hi Lo Range Bars Not Appearing eee cess cesseeceseeesseeseseesesaees 54 8 3 Ranking Distribution Tabular e sooosoeesoossoessoessosesoessoessosssoessoessoesssosssossssss 56 8 4 Ranking Distribution Graphical eesooosoessoeesosssoessoessoessoees
91. for the supply of the spare parts or Next gt Criterion Control T TEF Add Criterion Unit of Measure Enable Subjectivity O di 15 Criteria assigned to Froblem Direction of Preference o Minimize Oe men O Setete diceson or preference tor campie minimise o Ensure Select Best Supply Arrangement is selected If the problem or objective parent of the criterion needs changing use the Objective Navigation Next and Back buttons Cost to Select Best Supply Arrangement When all the settings are correct click Add Criterion to add Minimise Total Figure 26 Wizard Example add criterion 1 to Select Best Supply Arrangement Table 7 Wizard example criterion 1 information Objective Minimise Total Cost of Supply Criterion Title Minimise Total Cost Criterion Description The total cost for the supply of the spare parts Unit of Measure Preferred Direction When step Q of Figure 26 is complete the Wizard will appear as shown in Figure 27 Now click the New Criterion button and the remaining criteria can be entered by following the same steps as used for Criterion 1 Ensure that the correct Problem Title is selected for each criterion Remember this is done using the Objective Navigation Back and Next buttons For more help using this interface refer to Section 4 2 5 Next enter the information presented in Table 8 into the Wizard It should appear as in Figure 28 24 UNCLASSIFIED UNCLAS
92. hat has a decision problem you would ike to resolve ore detail the better Figure 36 The New Project dialog box The following subsections introduce the structural elements used within Decision Maker with the associated dialog boxes that you will use to create the elements for your problem s structure 5 4 1 Problem Element A Problem element is used to represent your decision problem in Decision Maker Before you can begin structuring or analysing any decision you must identify your problem and clearly define it using a Problem element This step is important since it will ensure that you are starting your decision making process with the correct problem see Section 3 1 for more information on defining your problem The Problem element has three fields that are used to define your decision problem as shown in Figure 37 5 4 2 Objective Element An Objective element is used to represent each of your objectives and sub objectives within your decision problem see Section 3 2 for more information on objectives You can have many Objective elements within your problem The Objective element has four fields that are used to define the objective it represents or sub objective if required as shown in Figure 38 UNCLASSIFIED 33 DSTO GD 0681 UNCLASSIFIED 5 4 3 Alternative Element An Alternative element is used to represent each of the possible alternatives to your decision problem see Section 3 3 for
93. he Database Management interface can be opened from the Tools Options menu as shown in Figure 132 The following subsections describe the Decision Maker Database Management interface File Edit View Project Windows Advanced 4nalysis Help Run Simulation F4 Generate Executive Summary FQ Import Alternatives From File F10 Figure 132 Opening the Decision Maker database management interface 14 1 General The initial window to appear when selecting the options interface is shown in Figure 133 Ax m Decision General Model Views Databaze Management Clear the Current Froblem s DataSet _Ciear Now 4 Add the defaut sample problem to the current project _ Add Now JE O cmt oh Clears all data from the problem this deletes all alternatives Creates the Supply Manager s Dilemma problem as given in the example Database management is divided in two sections Simulations Section 14 2 1 and Database File Administration Section 14 2 2 e Model Views this is not enabled and is part of further development Figure 133 Decision Maker Options window UNCLASSIFIED 108 UNCLASSIFIED DSTO GD 0681 14 2 Database Management The Database Management tab contains two sub tabs Simulations and Database File Administration Both tabs contain options and tools to manage the database files that Decision Maker uses to store simulation data The Database File Administration options tab contains tools to assist in
94. he Advanced Analysis Domination Scoring Matrix menu as shown in Figure 86 File Edit View Project Tools Windows Advanced Analysis Simulation Reporting Scoring Matrix Cobweb Plot viewer Weights Sensitivity Criteria Sensitivity Domination Scoring Matrix Fil Figure 86 Starting the Domination Scoring Matrix 10 1 Basics The base view for the Domination Scoring Matrix is a blank window A Decision Problem or SimulationID selection must be made to populate the view Once a selection is made the view will appear similar to Figure 87 Cell selection is shown in Figure 88 10 2 Using the Domination Scoring Matrix As mentioned the Domination Scoring Matrix can be used individually or in combination with the CWViewer On its own the Domination Scoring Matrix provides an overview of how the criteria values of various alternatives compare over an entire simulation This is the same method employed by the Scoring Matrix except in the Scoring Matrix interface all the criteria are visible at once including a graphical and numerical representation providing more holistic detailed information Each cell in the Domination Scoring Matrix view is comprised of five components namely a set of colour coded meter bars each with a superimposed number This indicates the fraction of times that the column selected alternative criteria value was better than the row selected alternative criteria value For example if the criterion direction
95. he Simulation Reporting Decision Tree Figure 67 Simulation Reporting Decision Tree 8 1 4 Hi Lo Range Bars The Hi Lo range bars in Figure 64 show the maximum minimum and the ranges of the top 25 middle 50 and lower 25 scoring ranges Figure 68 describes elements of the Hi Lo range bars An interpretation of the example in Figure 68 is that in relation to all the simulation runs e the cardinal scores range from 0 550 to 0 965 e 25 of the time the alternative scored between 0 550 and 0 615 e 50 of the time the alternative scored between 0 615 and 0 810 e 25 of the time the alternative scored between 0 810 and 0 965 UNCLASSIFIED 51 UNCLASSIFIED DSTO GD 0681 e 75 of the time the alternative scored between 0 550 and 0 810 and e 75 of the time the alternative scored between 0 615 and 0 965 1 000 isi 0 900 1 0 850 The upper narrow blue bar shows the range ant MEn for the top 25 of scores 0 750 The thicker red bar shows the range for the 0 700 gt middle 50 of scores 0 650 The bottom narrow blue bar shows the range ri e for the bottom 25 of scores a lt 0 500 0 450 Figure 68 Hi Lo range bars The location of the red bar can also be used to identify the region within the range of scores that an alternative was most likely to score if further simulations were run In Figure 68 the red bar is located closer to the bottom of the scoring range This indicates that the alterna
96. hey are on the list and whether they capture your interest will also help to further refine and add to your list The list of concerns and wishes need to be converted into succinct objectives such as a short phrase consisting of a verb and an object 4 For example minimise cost maximise profit and maximise safety EXAMPLE CONTINUED Supply Manager s Dilemma PrOACT STAGE Defining Objectives It is time for the Supply Manager to consider what he means by best in his problem definition His wishes are 1 low cost 2 fast supply time for each part request 3 minimum amount of administration and paperwork managing the contract and 4 good quality of parts note that these are non repairable consumable spares His concern 1s 1 slow supply time These wishes and the concern are then converted into the following objectives 1 minimise the total cost and 2 maximise the performance of the spares defined by a maximise the Reliability and b minimise the Supply Time The structure of the problem is shown in Figure 4 A Find the Best Arrangement for Supply of Spare fr Problem Parts A amp B 2 Objective Critereon O e a Minimise Total Cost of Supply Maximise Spare Part Performance Total Cost of Q ae Q aoa Supply Maximise Spare Part Reliability Minimise Spare Part Supply Time i a E E Type A Part MTBF Type B Part MTBF Type A Part MTTS Type B Part MTTS
97. ially if some of the middle range alternatives score well at times in the simulation or if many alternatives score well and it is difficult to determine which is more preferred UNCLASSIFIED 70 UNCLASSIFIED EEEE EE EF Be Fo E EE EB EERE ERLE E EEEE EEE F R P Fe ETEJE JEJEJE JEEE EEEE E EEEE EEEE EE EEE ZEEE EEE EEEE E Digitol Colas FECECCec ECEEEEE FeEER EER kekeke REEER EEC Re Eeee FCCC EEEE B2535353773 Here is an example of what the column should look like for the higher scoring alternatives i e the column should have cells that appear predominantly g The columns that are predominantly white red and or have small amounts of green indicate poorly performing alternatives Figure 91 Domination Scoring Matrix holistic view maximum zoom out 11 Cobweb Plot Viewer UNCLASSIFIED DSTO GD 0681 The CWViewer is used for a cobweb graphical analysis of the decision problem It can be started from the Advanced Analysis Cobweb Plot Viewer menu as shown in Figure 92 The Window shown in Figure 93 will then appear The features and functionalities for the CWViewer are presented in the following subsections For a large number of simulation runs the plot area can be slow to refresh and because of this the plot will only redraw based on some special selection events If you have made a selection change and the plot does not redraw itself it can be forced to redraw by moving the mouse in an
98. ights Show Ordinal Results ae Show Cardinal Results A Chart Fal Refresh Figure 51 Opening the Structure Model The Structure Model window shown in Figure 52 consists of three panes each presenting a different level of information from within your decision problem The three panes from left to right are the Folder List the Element List and the Properties Hierarchy tabs Decision Maker organises your problem s structural elements into folders with each folder containing all the elements of that particular type Therefore all objectives within your problem s structure can be found in the Objectives folder similarly for your problem s Alternative and Criterion elements 6 2 The Data Model Adding Data and Analysing your Problem The Data Model window in Decision Maker provides the tools that enable you to add data to your decision problem and undertake analysis The Data Model window also provides the tools to create new Objective and Criterion elements so that you may further improve your problem s structure Furthermore an overview of properties for the currently selected element is provided UNCLASSIFIED 42 UNCLASSIFIED DSTO GD 0681 a Structure Model Select best supply arrangement EAX Refresh YX Delete P Problem Alternative Objective Criterion eae ain bia mie haps het et es Folder List Element List Properties Hierarchy A Project Suppi Manager s Dilemma Element Name Supply Manager s
99. ime Type A Spares _ C5 Minimise Supply Time Type B Spares Supplier 2 60000 6600 i 7 Supplier 3 55000 5200 8 8 Supplier 4 75000 6100 8 9 Supeser 5 zaw 7300 g 7 Supplies 6 69350 7300 8 8 Super 7 bess 7300 fe E Supplier 8 72100 S800 7 e Senos 8 ba 5600 6 7 8 Supplier 10 74850 6600 6800 7 8 Supplier 11 69350 6200 7200 ig 8 Supplies 12 63950 6200 s100 lg 7 Supper 13 72100 200 6800 G 3 Supplier 14 90950 6100 7200 g E Erei 7 5100 S100 7 Supple 16 72100 6100 6600 3 het This shows that Auto Generate Alternative Titles has been disabled since the data file already contains titles for the alternatives ry This row has been mapped to the column indicated by 9 Figure 150 Alternative Input Mapping Step 2 In Figure 151 the data file did not contain the alternative titles so in this example they will be created using the Auto Generate Alternative Titles option indicated by Q UNCLASSIFIED 117 UNCLASSIFIED DSTO GD 0681 3 Alternative Input Mapping 4 Delimitets Element ID Element Name Element Description C Comma F Tab Semienion sh a ie O S i lqnore This Column will Be Ignored v Auto Generate Alternative Titles Recommended a Altemative Title The Column wil be the Tile of the Alternative Element Title Base Supple A Alternative Description This Column will be the Altemative Description E Minim
100. in Figure 109 This example demonstrates the necessity of using the Domination Scoring Matrix in conjunction with the CWViewer for a thorough analysis of a decision simulation This also shows that even though Supplier 5 obtained a low score for the objective Minimise Supply Time for an individual simulation run Supplier 14 scored lower in the same simulation run For example in simulation run number 12 Supplier 5 scored higher than Supplier 14 while in simulation number 32 Supplier 14 scored higher than Supplier 5 This is shown in Table 14 UNCLASSIFIED 86 UNCLASSIFIED DSTO GD 0681 This is not a comparison in which we are interested However it is important to be aware of this issue when analysing a simulation Table 14 Domination interpretation Supplier 5 Supplier 14 oring Matrix Viewe Decision Report Selection Simulation Details Existing DecisionMaker II Simulations Number of Alternatives 16 Select best supply arrangement Number of Simulation Runs 50 Simulation Sets Simulation Run ee Simulation Date 1406 2007 10 49 4M 1000 x Uatn Created By CostollC fe P 1 Select best supply arrangement Supplier Supplier Supplier Supplier Supplier fSupplier Supplier Supplier Supplier Supplier Supplier Supplier Supplier Supplier Supplier Supplier C C1 Minimise Total Cost 2 C 2 3 4 5 6 7 8 3 10 11 12 13 14 15 16 Q 0 1 Maximise Performance a sA i O 0 1 1 Maximise Reliability Supplier 1 J 0 4
101. inty in Minimise Supply Time Type B Spares B E C 1 Minimise Total Cost A Uncertainty in Minimise Total Cost Supplier 11 Supplier 12 Figure 45 Uncertainties added UNCLASSIFIED 38 UNCLASSIFIED DSTO GD 0681 Ap Select best supply arrangement Element Properties B E C1 Minimise Total Cost a A Uncertainty in Total Cost i SIS ER EEM a 0 1 Maximise Performance Uncertainty in Maximise Reliability Type B Spares Q 0 1 1 Maximise Reliability Description Si C 2 Maximise Reliability Type A Spares A Uncertainty in Type A Spates Flefiability C 3 Maximise Reliability Type B Spares i H ype b spat My TYPES D apar ertainty in Maximis ity in Maxin a o 0 1 2 Minusen Sugly feu arena is feo Mien Reedy Vener T espa 4 Add New Element Select the uncertainty element has Uncertainty in Type 5 pales MTTs S E C 5 Minimise Supply Time Type B Spares Se Copy Siete a a A Uncertainty in Typa Spates MTTS M Paste and click the right mouse Open in New Window sa button Sensitivity Analysis aa Change Uncertainty Uncertainty in Re Number Objectives Maximise aus Lainie AD EE Alternatives Reliability Type B Change Uncertainty Values Refresh Al Uncertaint ies are defined as a percentage and will be applied to all alternatives tor this Criteria Criteria ID E3
102. ion Maker is to create a Project to contain your decision problem Therefore in the first step the Wizard requests you to enter a title and description for your project You must save your project before continuing Refer to Section 5 2 2 for an overview of the dialog box the Wizard presents for creating a new project 12 UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 Step 1 Decision Maker Wizard ax This wizard will help guide you through a process for structuring your decision problem to help you make a better decision The process you will undertake consists of multiple stages These stages include defining the problem your objectves and allemnetives and any consequences that these ney have Step i Create a new Decision Project Step 2 Define your Problem step J Define your Objectives Steg 4 Define your Alternatives Step 5 Define the Critena for each of your objectives Step 6 End the Witerd and begin adding data to vour decision problem ee aaz Figure 10 The Decision Maker Wizard navigation dialog box 4 2 2 Step 2 Define your Problem The second step requires you to define your decision problem see Section 3 1 This is achieved by creating a Problem element You must create a Problem element before the Wizard will allow you to continue to the next step Defining your problem is simple and only requires a short title for your problem a description and a direction of preference Section 5 3 1 give
103. ion are uncertainty risk tolerance and linked decisions Tips and techniques to clarify the uncertainties to consider the effect risk tolerance will have on the decision and the implications of linked decisions are briefly considered as part of the Revise step in Figure 3 The choice of the acronym PrOACT is deliberate by the authors of 4 to remind us to be proactive and not wait until a decision is forced upon us when we may not have time to consider all alternatives and consequences UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 Problem h Objectives h Alternatives H Consequences h Trade offs h Revise Figure 3 The PrOACT decision making process A structured approach such as PrOACT is recommended because it enables decision makers to effectively communicate clarify and consolidate organise their decisions However a structured approach is not required for all decisions especially those that are regarded as simple or present an obvious or clear choice These types of decisions invariably do not involve complex trade offs and therefore tools such as Decision Maker are not required Each step of the PrOACT process in conjunction with a worked example is described and discussed in the following subsections These descriptions with the exception of the worked example and the use of Decision Maker are short summaries of the key points made in 1 Decision Maker is a software impleme
104. ion set contains one or more simulation runs In this example it is disabled however at other times it is enabled to allow selection of a single simulation from a set Figure 65 Decision Report Selection UNCLASSIFIED 50 UNCLASSIFIED DSTO GD 0681 8 1 2 Simulation Details Figure 66 shows an enlarged view of the Simulation Details area of Figure 64 Simulation Delad Problem Name Select best supply anangenent E P gt Humbera Atematives 16 Simulation Date 1 06 2007 12 49 25 AM a9 Number of Simulation Runs 50 Created By Costolil 4 o rean O Sopa nshepon MEST SS comparison in the problem o The name of the person who ran the simulation the Decision Maker user login Figure 66 Simulation Details 8 1 3 Simulation Reporting Decision Tree Figure 67 shows an enlarged view of the Simulation Reporting Decision Tree area of 9 A cyan coloured highlight indicates that this is the selected node for viewing o Criteria are visible however they do not Figure 64 af H CA i Minimise Total Costi 5J 0 1 Maximize Performance F 0 1 1 Maximece Reliability hf 0 2 Maximise Reliability Type A Spares wa i C3 Maximise Reliability Type B Spares A 0 1 2 Minimise Supply Time H C4 Minimise Supply Time Type A Spares C4 Minimise Supply Time Type B Spares 3 provide any information when selected Problem and Objective elements are used to provide information in t
105. ions Top Percentage Limt 09 CI Show Top Scoring Altematives M Show Ais Values Select best supply arrangement Lew Peesrtage Lin 0 1 C Show Mid Range Alternatives dinal Scoring Option Simulabon Sets Simulation Fun C Show Low Scoring Altematives 1000 iv L Aa A Pii Select bestsupplp arangemeni C1 Minimise Total Cost H 0 1 Maximise Performance 5 9 0 1 1 Maximise Reliability p E C3 Maximise Reliability Type A Spares E K C3 Maximise Reliability Type B Spares 0 1 2 Minimise Supply Time Me C4 Minimise Supply Time Type A Spares i H C5 Minimise Supply Time Type B Spares 7 P 1 Select best supply anangement C 1 Minimise Total Cost F 0 1 Marimise Perfomance 0 1 1 Maximise Fehabiliby C 2 Maamise Relisbity Type A Spares w C 3 Maximise Reliability Type B Spares 0 1 2 Minimise Supply Time C4 Minimise Supply Time Type A Spares C 5 Minimise Supply Time Type B Spares Altemative All Aerneiyves Supplier 1 Neuer 11 a Di ae Ler Tw LLL Ww deselected o To make the plot area clearer the top mid and low level plot groups have been ry Cardinal Scoring Option has not been selected Note the axis scales of the minimise criteria Figure 102 CWViewer filtering alternatives example 1 continued When the Cardinal Scoring Option has been selected the view will change as shown in Figure 103 This view can assist i
106. is to facilitate the use of Decision Maker in the Trade offs step An example of the consequence table is shown in Table 1 Note Decision Maker provides the tools to structure decision problems and provides the user with a consequence table for data entry UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 EXAMPLE CONTINUED Supply Manager s Dilemma PrOACT STAGE Evaluating Consequences The Supply Manager s objectives were to 1 minimise the total cost and 2 maximise the performance of the spares defined by a maximise the Reliability b minimise the Supply Time He now needs to determine how he might measure each of these objectives He decides that the expected number of days per month spent on administration of the contracts and the total cost of the contract or contracts are appropriate measures for the first two objectives respectively For the remaining two objectives he decides to utilise the 95 confidence intervals for the Mean Time Between Failures MTBF and the Mean Time To Supply MTTS respectively The resulting measures for each of these objectives are the lower limit of the 95 confidence interval for the MTBF and the upper limit of the 95 confidence interval for MTTS Both measures use the less desirable outcome as a way of determining how well each alternative performs against the objectives The Supply Manager then asks each company to provide the relevant data and subsequently constructs the consequence table
107. ision Problem Structural Modef gt Data Model eal Chats Decision Model Figure 49 Decision Maker s Structural Model and Data Model input output The following sections will introduce you to Decision Maker s Structure Model and Data Model forms Within these forms Decision Maker uses a common colour code for the four different types of problem elements These colour codes enable you to quickly identify the type of the element therefore making the analysis process visually easier to comprehend The colour codes are presented in Figure 50 j roblem O jective A lternative Figure 50 Decision Maker element colour codes UNCLASSIFIED 41 UNCLASSIFIED DSTO GD 0681 6 1 The Structure Model Modifying your Problem s Structure The Structure Model window provides the tools that will enable you to build and modify your problem s structure This includes adding and deleting your problem s structural elements and editing their properties The Structure Model window is only accessible when a project is open To open or create a project refer to Section 5 2 2 To open the Structure Model window when a project is open select the View Structure Model menu option as shown in Figure 51 Use the data model discussed in Section 6 3 to add data to the problem and begin analysis File Edit View Project Toole Windows Advanced Analysis Help BF Structure Model lal Data Model Show We
108. iterion Criterion i Criterion i if pT E Criterion Criterion Criterion Sub Objective G ee G 5 F fa f q SS Sub Objective Criterion Criterion oe Criterion a oo a a a ee ae ee eee eee eee ae Figure 35 Structure of a Decision Maker project 5 3 1 Creating a Project Every decision problem must belong to a project and a project may contain more than one decision problem Before you can begin structuring your problem in Decision Maker you must create a project to store your decision problems Creating a project is achieved by using either the Wizard Section 4 or by creating a new empty project manually from the File menu Section 5 4 Figure 36 presents an example of the dialog box presented to you when creating a new project UNCLASSIFIED 32 UNCLASSIFIED DSTO GD 0681 5 4 Decision Maker Structural Elements Decision Maker uses structural objects generically referred to as elements It is a combination of these elements that you will use to structure your problem in Decision Maker The elements represent various attributes within the structure of your decision problem These elements can either be a Problem Objective Alternative or a Criterion and you may have multiples of each of these elements in your problem s structure see Figure 35 aS New Project Project Title New Project Project Discnption roject Title Use this field to enter a short itle for your project For example this may be ask t
109. ize Total Cost Stant Numbering ez Maximise Reliability Type A Spares IC Maximize Reliability Type B Spates E thot a Tine thoes A sina Create Alematives or Format Flaw File Contents tion se Total Maximise Maximise Minimize Supply Minimize Supply Post ae Type A Aelability Type B Time Type 4 Time Type B Spares Spares Spares mo 7300 7200 E a 6600 3100 7 7 eee 6200 o E E 7500 6100 E ig 72100 7300 9700 E 7 69350 7300 6600 B 8 80350 7300 6200 E E 7100 B600 7200 7 f 63850 5600 7 8 74850 5600 s800 7 E 69350 6200 8 a 63850 6200 Bee E i 72100 6200 E800 g 80350 6100 E 74850 6100 7 77100 6100 8 ry The Element Title Base used here will be the word Supplier Start Numbering At is the starting suffix to add to the Element Title Base In this example the starting index is 7 Also the alternatives will be named Supplier 1 Supplier 2 Supplier 3 Supplier n where n is the total number of alternatives being added This is a useful input when alternatives already exist and new ones are added to the problem For example in the Supply Manager s Dilemma new alternatives would be numbered from 17 onwards hence the Element Title Base would be Supplier and the Start Numbering At would be 17 Figure 151 Alternative Input Mapping generate titles 16 Acknowledgments The authors acknowledge Director General Maritime Support DGMARSPT DMO as sponsor of this research Car
110. jective ts defined by two critena MTBF and MTTR Sub Objective Of New Decision Problem Direction of Objective L Maximise Criteria Assigned to Objective Mean Time To Repair Bosra ax Cntenon Title Objective Mean Time Between Failure Navigation D ae Se Catenon Disenption Criterion Control Add Crternon 4 measure of the products reliability that is given in units of hours The higher the MTBF the more reliable the product will be Unit of M easure MTBF hrs Direction of Freference c Maximise Refer to the Problem and Objective elements in Sections 5 3 1 and 5 3 2 Criteria Assigned to Objective This field lists the Criteria that have been assigned to the currently selected objective You can use this field to verify that the criterion you have just created has been added to the objective Refer to the Criterion element in Section 5 3 4 Displays the numerical index of the currently selected Objective element Objective Navigation The Back and Next buttons enable you to navigate to your desired objective so that you may then add its criteria Criterion Control The New Criterion button creates a new Criterion element so you can edit and add to the Objective element as indicated by Q by using the Add Criterion button Figure 11 Adding criteria to your decision problem using the Wizard 4 2 7 Create a New Project Using the Wizard Run
111. l Costolloe would like to thank Dr Moya Tyndall DSTO for her support feedback and understanding during this project Thanks also go to Tom Whitehouse DSTO for ongoing feedback during the software development stages The ANZAC SPO is acknowledged and thanked for providing data and Reliability Centred Maintenance RCM cases for testing and verification of the software Finally Associate Professor Louis Doukas Royal UNCLASSIFIED 118 UNCLASSIFIED DSTO GD 0681 Melbourne Institute of Technology RMIT is acknowledged for his association with the project 17 References 1 Diakoulaki D Mavrotas G and Papayannakis L 1995 Determining Objective Weights in Multiple Criteria Problems The CRITIC Method Computers and Operations Research 22 7 Pergamon 2 Pomerol J C and Barba Romero S 2000 Multicriterion Decision in Management Principles and Practice Kluwer Academic Publishers Norwell Massachusetts United States of America 3 Mollaghasemi M and Pet Edwards J 1997 Technical briefing making multiple objective decisions IEEE Computer Society Press Los Alamos California United States of America 4 Hammond J S Keeney R L and Raiffa H 1999 Smart choices a practical guide to making better decisions Harvard Business School Press Boston Massachusetts United States of America UNCLASSIFIED 119 UNCLASSIFIED DSTO GD 0681 Appendix A Known Faults in Decision Maker Mathematical Rounding Erro
112. lem in Decision Maker using the information provided 5 3 The First Step The first step is to decide whether you will use the Wizard to structure your problem or whether you will structure your problem manually If you know the basic structure of your decision problem you can use the Wizard to guide you through the steps that will UNCLASSIFIED 31 UNCLASSIFIED DSTO GD 0681 structure your problem in Decision Maker These steps include the creation of the project that will contain your decision problem If you choose the manual method for structuring your decision problem you must first create an empty project to store the decision problem Then you can begin defining your problem and its structure The following subsections provide an introduction to the various elements that you will need to use to structure your decision problem in Decision Maker including 1 Decision Maker Project 2 Decision Maker Structural Elements a Problem Element b Objective Element c Alternative Element and d Criterion Element Figure 35 presents the basic structure of a project in Decision Maker using these elements The figure shows that any one project may contain more than one Problem and that each Problem has its own structure consisting of Alternatives Objectives and Criteria Decision Maker Project i or Pr a gt Pr a Altematives Problem 7 Alternatives i Problem i f E i i Ovjpetive p Objective Cr
113. leted Congratulations you have now completed your decision model ou may now view sour decision model and begin entering your decision data Figure 32 Wizard example Problem Definition Completed 4 3 Assign Data to Each of the Problem s Criteria Once the Wizard has concluded the assignment of data to the criteria has to be entered This is done using the Data Model view At this stage the Data Model view will look like the example in Figure 33 For detailed information on the Data Model view refer to Section 6 3 To open the Data Model view select from the View Data Model menu To assign data to the problem s criteria select an objective or criterion element in the Element Tree The Data Grid window below the Element Tree will display the data fields for the selected element Use the data presented for the Supply Manager s Dilemma in the consequence table shown in Table 1 This data is also available from the installation CD in a Microsoft Excel spreadsheet which can then be copied and pasted into Decision Maker Do this for each criterion in the problem s structure When completed and upon selection of an objective the Data Grid will show the ranking of each supplier The ranking is calculated automatically and does not require the user to initiate the calculation Figure 34 shows the rankings for each supplier for the object Maximise Performance 4 4 Calculate the Scores for Each Alternative in the Problem When the da
114. lysis and Decision Maker s output This is a difficult stage since it requires you to specify the quantifiable attributes that define each of your objectives Refer to Section 3 4 for more support on this stage The dialog box used to create Criterion elements in the Wizard is shown in Figure 11 This dialog box incorporates the same fields as the Objective and Criterion element dialog boxes as presented in Sections 5 3 2 for the Objective element In addition buttons are provided to navigate between the objectives created in Step 3 of the Wizard and also for the creation of the Criterion elements You can have as many Criterion elements as you desire in your decision problem The greater the number of criteria the greater the resolution of Decision Maker s output UNCLASSIFIED 14 UNCLASSIFIED oF 4 2 6 Step 6 Adding data to your decision problem DSTO GD 0681 You may edit the properties of the currently selected objective at any time in the dialog box and the changes will be added to your project automatically When the problem has been defined by following Steps 1 through 5 it is then possible to add data to each of the criteria This is described in Section 4 3 _ 10 Add Criteria to New Objective 1 Objectrve Title Objectiveal2 of 4 Maximise Reliability Objective Discrption Reliaility i the measure of the products ability to maintain functional operation In this case the reliability ob
115. ment shall have a short supply Description time Sub Objective Of UNCLASSIFIED 20 UNCLASSIFIED DSTO GD 0681 Define Objective Number 7 Objective Title Maximise Reliability Objective Description The best spare part supply arangement shall be able to supply the parts with the highest reliability Sub Objective OF ee S 1 Maximise Perfomance Figure 20 Wizard Example Define Objective Number 2 Define Objective Number 3 Objective Title Minimise Supply Time Objective Description The best spare part supply amangement shall have a short supply time Sub Objective OF 0 1 Maximise Performance aaa ae E r f Figure 21 Wizard example Define Objective Number 3 4 2 10 Alternative Definition In the alternative definition stage you are required to enter the number of alternatives you currently have for your decision problem The Supply Manager had defined an initial list of 16 possible alternative arrangements for the supply of Type A and Type B spare parts Section 3 3 For this example use the data given in Table 1 Note that the Wizard is not the best method for entering a large number of alternatives for a problem If you do wish to enter a large number of alternatives set the number of UNCLASSIFIED 21 UNCLASSIFIED DSTO GD 0681 alternatives to zero 0 and when the decision structure is complete using the Wizard the alternatives can be added using the I
116. more information on alternatives Each Alternative element in your decision problem must be unique and you must have at least two alternatives in your decision problem for Decision Maker to analyse your problem The Alternative element has two fields as shown in Figure 39 AN Each alternative element in your decision problem must be unique and you must have at least two alternatives in your decision problem for Decision Maker to analyse your problem New Deciston Problem Problem Tithe New Decsion Problem Problem Disciption Prefered Drechon of Solution Maximize w Canes Figure 37 The New Decision Problem dialog box 34 UNCLASSIFIED Problem Title Use this field to enter a short title for your decision problem Problem Description Enter a detailed description of your problem the more detail the better Preferred Direction of Solution Select the direction of preference for the solution of your decision problem The preferred direction field provides a simple method that will be used by Decision Maker to calculate and sort the priority of your decision problem s analysis output For example if your problem requires you to find the best alternative the preferred direction for your decision problem will be maximise Otherwise if you are making a risk based decision and you are searching for the alternative that has the least impact in your problem the direction of preference will be minimise
117. mplement a ranking change Figures 118 and 119 are some examples of the search results viewed graphically Figure 118 shows that significant changes in Minimise Total Cost and Supply Time Type B Spares are required for a change in ranking i e to make Supplier 14 more preferable than Supplier 8 Figure 119 shows that changes for all criteria are relatively minimal This indicates that it would not require a significant change in weighting to change the ranking UNCLASSIFIED 95 UNCLASSIFIED DSTO GD 0681 Weighting Weighting 1 000 0 300 0 800 0 700 0 600 0 500 0 400 0 300 0 200 0 700 0 000 Minimum verghts Distribution For Rank Change F 1 Supplier 8 vs Supplier 14 Maximize AReliability Type 4 Spares Minimise Supply Time Type 4 Spares Minimize Total Cost Maximize Reliability Type B Spares Minimize Supply Time Type B Spares Figure 118 Weights Search plot example 1 1 000 0 300 0 800 0 700 0 600 0 500 0 400 0 300 0 200 0 700 0 000 Minimum Weights Distribution For Rank Change F 1 Supplier 1 vs Supplier 6 Maximize Reliability Type 4 Spares Minimise Supply Time Type 4 Spares Minimize Total Cost Maximize Reliability Type B Spares Minimize Supply Time Type B Spares Figure 119 Weights Search plot example 2 12 8 Sub Level Weights Analysis Sub level weights analysis provides a method for analysing the weights sensitivity at various levels within the decisio
118. mport Alternatives from File facility Section 15 For this example each alternative will be manually created Click Next until the dialog box shown in Figure 22 is visible Enter the number of alternatives i e 16 in the Number of Alternatives field Click Next twice reading the Decision Maker Wizard dialog boxes as you proceed The Define Alternative Number 1 dialog box should now be open as shown in Figure 23 Change the Alternative Title to Supplier 1 as shown in Figure 24 For this example a description will not be used however it is worth noting that one can be added at this stage When the alternative title has been entered click the Ok button Now repeat the process for the 16 alternatives entering Supplier 2 through to Supplier 16 in the Alternative Title field for each of the 16 alternatives Step 4 Define the Possible Decision Alternatives The aim of this step is to generate many possible alternatives not limiting the Mule 20 el Alisiya range of alternatives that could be considered 16 Dio not evaluate or eliminate any alternatwes during this step you can do that later some suggestions for generating alternates include 1 Use your objectives and ask how they might be achieved 2 Be creative and think outside the square ie don t limit your alternatives 3 Challenge your constraints and set high aspirations Flease enter the number of known alternatives you have atthis stage Figure 22 Wizard example
119. n in Figure 46 Then when the dialog box shown in Figure 47 appears enter a value between 0 and 1 Here 0 1 has been used indicating 10 uncertainty When you have entered a meaningful value click OK The Data Model will be updated to reflect the changes made as shown in Figure 48 Uncertainties for individual alternatives can be set manually by editing the appropriate cell in the data view This is discussed in Section 6 3 1 4 Data Model Select best supply arrangement i O New Objective New Criterion 3X Delete 2 Refresh F Calculate Chart Select best supply arrangement Element Properties C1 Minimise Total Cost Title A Uncertainty in Total Cost Sra oc pu fat NTE 0 1 Maximise Performance Maximise Reliability Type B Spares O 0 1 1 Maximise Reliability Description Si C 2 Maximise Reliability Type A Spares A Uncertainty ir Type Spates Aeliabilily C 3 Maximise Reliability Type B Spares 0 1 2 Minimise Supply Time Se C4 Minimise Supply Time Type A Sparegr A Uncertainty in Type Spares MT TS SE C 5 Minimise Supply Time Type B Spares A Uncertainty in Type Spaies MTT Open in New Window Add New Element O Objective Criterion Uncertainty Delete Chart refered Direction Sensitivity Analysis Maximise Re Number Objectives Ahematives o
120. n locating dominant alternatives across multiple criteria Using the Cardinal Scoring Option in the CWViewer enables identification of alternatives based on performance Regardless of the direction of preference the more preferable alternatives intersect each axis near the top The ideal alternative would be a straight line across the top of the plot UNCLASSIFIED 80 UNCLASSIFIED DSTO GD 0681 a iewer 4 Decision Report Selection Plot Options 3 Existing DecisionMaker II Simulations Top Percentage Limit 0 9 C Show Top Scoring Alternatives X Show Axis Values ae menee fd Low Percentage Lin 01 E Show Mid Range Alternatives dinal Scoring Option Simulaton Sets mulation Run Show Low Scoring Altematives 1000 v n HE C1 Minimise Total Cost S O 0 1 Maximise Performance 5 0 1 1 Maximise Reliability iB C C 2 Maximise Reliability Type A Spares C C 3 Maximise Reliability Type B Spares 5 0 0 1 2 Minimise Supply Time l E 4 Minimise Supply Time Type A Spares e C 5 Minimise Supply Time Type B Spares 1 Select best supply arrangement 1 Minimise Total Cost 1 Maximise Performance 1 1 Maximise Reliability 2 Maximise Reliability Type A Spares 3 Maximise Reliabdity Type B Spares 1 2 Minimise Supply Time v C 4 Minimise Supply Time Type A Spares 5 Minimise Supply Time Type B Spares vj r v Sal 3 3 63 ES S S IESE 3
121. n tree In Figure 120 a sub level analysis is shown for the Supply Manager s Dilemma Supplier 3 has been selected at the element level of P 1 Select Best Supply Arrangement Since this is a sub level analysis only the immediate children of P 1 will be selected for analysis These are C 1 Minimise Total Cost and O 1 Maximise Performance as shown in the column headings for the Non dominated Sets The Domination Tree View will be populated based on these two elements alone The search results shown indicate that in general the weighting for Minimise Total Cost needs to be decreased and the weighting for Maximise Performance increased proportionally This is clearly visible in the plot of the Direct Relations The black line in the plot shows the current weighting assigned to the objective and criteria selected UNCLASSIFIED 96 UNCLASSIFIED DSTO GD 0681 i a Weights s Sensitivi pem Reput Selection Simulation Details View Options Existing DectsionMaker I Simulations Number of Alternatives 16 Absolute Change CI Show Top Level Analysis Select best supply anangement x Number cf Simulation Runs 10 C Percentage Chanoe CI Show Altemative Rankings Simulation Sets Simulation Run P MEE Actual Value O Show Criteria in Tree 1160 3 x Simulation Date on nee ey F Show Levels with Mouse Number of Searches Created By Casto r i P Selectbest es arranger A A lhariaines Paes Suni a Supplier 12 Ha
122. natives The fourth step requires you to define all the possible alternative solutions to your decision problem see Section 3 3 This is done by creating an Alternative element for each alternative in your problem You must have a minimum of two 2 alternatives in your problem Do not limit the number of alternatives that could be considered Also let Decision Maker support you by ranking your alternatives based on your criteria at which stage you can then select amongst the best alternatives for your decision problem Refer to Section 5 3 3 for an overview of the dialog box the Wizard presents to you to create each alternative element Each alternative element in your decision problem must be unique and you must have at least two alternatives in your decision problem 4 2 5 Step 5 Define the Criteria for each of your objectives The fifth step in the Wizard is the final step required before your problem is structured and ready for data entry and analysis This step requires you to define the criteria that you will use to measure the performance of each objective in your decision problem see Section 3 4 This is done by creating a Criterion element for each quantifiable attribute belonging to each objective in your decision problem Each objective in your problem including the problem itself can have as many Criterion elements as needed The greater the number of Criterion elements the greater the resolution of your decision ana
123. ng i e increasing the search variation Search Options Show Calulation Columns Show Weights Columns Stage 1 Search Options Show Calulation Columns Show Weights Columns Stage 2 Search Options Show Calulation Columns Show Weights Columns Stage 3 Non Dominated Search Search Variation 0 0004 Auto Tune J Best 0 00036 Number of Search Matches 20 Average 0 01186 Searches Performed 28 Worst 0 05564 C Non Dominated Search Search Variation 46 05 Auto Tune Best 4E 05 seesees umber of Search Matches 20 Average 0 01075 Searches Performed 232 a Auto Tune Best 46 05 Non Dominated Search Search Variation 0 00024 Number of Search Matches 20 Average 0 01079 Searches Performed 321 Worst 0 07323 A suitable starting search variation value The search variation is too small that the search variation value needs to be increased This can be o A high number of searches with slow progress is an indication accomplished using the relaxation slider oO The relaxation slider has been moved to the right o A suitable search variation value is set again Figure 129 Criteria Sensitivity Search Options using Auto Tune When Auto Tune has found some best worse and average scores Auto Tune can be turned off and the values used In Stage 3 of Figure 129 the best worse and average values indicate a variation value between 0 00066 and 0 00005 could be used
124. ng of alternatives the criteria objectives do not have to be independent objective calculation of criteria objectives weights the provision of a guide to structuring decision problems aE Yas UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 6 an easy to use Graphical User Interface GUI 7 asimulation module and 8 advanced sensitivity analysis The Decision Maker User Guide will assist and inform users in the decision making process The Decision Maker User Guide will also assist users to structure their decision problems in the required format for Decision Maker The outcome of using Decision Maker will enable users to make a better decision Better in this context means a decision that has been thoroughly considered and explored It does not mean that a better decision will necessarily be the right decision or a good decision A right or good decision usually refers to the outcome of the decision and not the process that was used to make the decision This user guide is structured to help users install and run Decision Maker Section 2 to help users structure their decision problem Section 3 to give users a quick start in using Decision Maker Section 4 and to enable users in structuring and analysing decision problems in Decision Maker Sections 5 to 15 Known problems and faults with the software are presented in Appendix A 2 Getting Started with Decision Maker 2 1 Installing Decision Maker Decision Maker i
125. ntation of the CRITIC decision analysis technique 1 CRITIC is an objective decision analysis technique and therefore does not require the decision maker s preferences The underlying principle in objective decision analysis techniques is that attributes can be viewed as information sources and that weights of importance reflect the amount of information contained in each of them 1 Weights are derived using the CRITIC technique and incorporates both the contrast intensity within each criterion objective and the conflict between the criteria objectives These weights are then used to combine the criteria objectives into a single cardinal ranking of the alternatives 1 An important feature of the CRITIC technique is that the criteria objectives do not have to be independent Note the word criteria used in the description of Decision Maker can be used interchangeably with the word objective Criteria is a more general term including all those attributes objectives and goals which have been judged relevant in a given decision situation by a particular decision maker individual or group 1 Decision Maker uses criteria to measure the performance of each objective Therefore each objective must have at least one criterion to complete the decision analysis and obtain a result UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 3 1 Problem It is important to start the decision making process with the right problem
126. oblem s structure element by element This method is also a good option if you are lacking information about your problem or the intended structure is unknown This method begins by creating a new blank project followed by the creation of all the structural elements for your problem as they are needed To begin structuring your problem manually follow these steps 1 Create a new project by selecting the File New New Project menu 2 After creating the new project open either the Structure Model Section 6 2 or Data Model Section 6 3 window depending on your modelling needs Do this by selecting View Structure Model or View Data Model and 3 Create a Problem element by selecting the Project Set Current Problem menu and then define your decision problem using the dialog box presented Section 5 3 1 When these steps are complete you can begin structuring your problem in Decision Maker It is highly recommended that the PrOACT process be followed Section 3 while structuring your problem However Decision Maker does not require you to follow this process The PrOACT recommended process includes the following sequence for structuring your decision problem 1 define your Problem UNCLASSIFIED 40 UNCLASSIFIED DSTO GD 0681 specify your Objectives create many Alternatives define the Criteria for each Objective and calculate results and Trade off the alternatives OT Ae W N Refer to Section 5 3 for mo
127. oblem Description What i the best arrangement for the supply of Type A and B spare parts Figure 16 Wizard example New Decision Problem Using the information presented in Table 3 complete the fields in the New Decision Problem dialog box so that it appears as shown in Figure 16 Table 3 Wizard example problem information Problem Title Select Best Supply Arrangement ae What is the best arrangement for the suppl of Type A Problem Description g pply yp and B spare parts When your problem has been defined click the OK button in the New Decision Problem dialog box The Wizard then requests that you define the objectives as shown in Figure 17 Step 3 Define your Decision s Objectives This step requires you to define your objectrves Objectives help to explain your choices to others and itis importantto spend time considering and defining therm neat acto m Figure 17 Wizard example Step 3 Define your Decision s Objectives UNCLASSIFIED 18 UNCLASSIFIED DSTO GD 0681 4 2 9 Objective Definition The Wizard will request you to enter the number of objectives for your decision problem The Supply Manager defined an initial list of three objectives to be used in support of selecting the best arrangement for the Type A and B spare parts Section 3 2 The objectives were to 1 minimise the total cost this will be treated as a criteria and 2 maximise the performance of the spare parts define
128. oesssessseessesssessssss 57 8 4 1 Filtern AHernnatUVeS iicsasiuisdinsiasiesaarisutenstcainc as A 58 99 Weis MS SUNMA aonais e n eC Ta 61 8 5 1 Tabuta r B Fei e memes feet tanner a a E etre eter 61 8 5 2 Lae ea gi oil RINGES mene ee ene ene ROR oer rere entre A 62 8 5 3 Sree aie URIBic o Fea repent E ner eater EON 63 Os SCORING MA TRUM Ara ER T E aetint N nannies 64 10 DOMINATION SCORING MATRIX seeseesseeseessoeseessoessossosseossosseessosssossosssossosssossosso 66 MOM Basies uenon an E N E R T 66 10 2 Using the Domination Scoring Matrix ee seeesoesoessoeesseessesssosssosssosssosssoessosssos 66 10 3 Domiuivant Alterna try S aeren E E 69 UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 10 4 ONS GC V eW anacan nnan chunieastesaeadoanacs enue 70 If COBWEB PLOT VIEWER sresiessiesiietion tetiabiontetaiiathe ieee E nee awih tas 71 TT PO OU ONS ia ce ecseasesticttcahoiesenetciaae eatasestacit ceo neaeternevessecetuucsea toes 72 DTZ PIO Alean suiacts tas a aa a aere iE 73 11 2 1 Plot Area Without Cardinal Scoring Option eeeseseeeeeereee 73 11 2 2 Plot Area with Cardinal Scoring Option eseese 73 TL2 gt Stenr Picts Cross Demsities arisini oa Ni 75 TLS Filterine A Cr atl VCS aissi aea ie i ai N eia 77 114 Filtering Elements ucioenecnnenindn a eee eens 81 TAL Fiter Dr ODJEVEN 82 1142 Pilte by Citeridsusnco Tree men arene T a 82 11 5 Interpreting Domination AlternativeSs seeesoossossseessesssesssos
129. of preference was to maximise then the alternative with a higher value would be the better alternative for this criterion alone Figure 89 shows an example of the Domination Scoring Matrix and the colour coding indicates common ranges of performance as presented in Table 13 UNCLASSIFIED 66 UNCLASSIFIED DSTO GD 0681 simulahon Detaiks Selection Number of Altematives 16 Columrt Supplier 1 A ee j E Number of Simulation Runs 50 Row Supplier 1 ka Simulation Date 1706 2007 10 49AM Grid Zoom Scale 1 Supple Supple Supple Supple Supple Supple Supple Supple Supple Suppe Supple Supple Supple Supple Suppke Supple Ph ah de da ae Le ee Le 3 4 5 B 7 a rg 4 gt Supplier 3 Supplier 10 Supplier 17 Suppher 12 Supplier 13 Supplier 14 ee Fg This region of the display shows an enlarged view of a cell when the cell is clicked with the mouse or when a selection is made in the Selection drop boxes seen in Figures 88 and 90 The Selection drop boxes can be used to select two alternatives directly without having to locate them in the grid view this can be seen in Figure 88 The Grid Zoom Scale can be used to increase decrease the scale of the grid view to enable more or less cells to be displayed When a selection is made the cell will be highlighted with a pink square Figure 87 Domination Scoring Matrix UNC
130. of when to take advantage of the two and three dimensional plots If there are only a small number of alternatives being viewed then the two dimensional plot may be more preferable and easier to read If there are a large number of alternatives in the view simultaneously then the three dimensional view may be more preferable The button visible in the plot area toolbar indicated by in Figures 75 and 76 is used to toggle between the views E Supplier E Supper 2 WS Suppie3 Suppter 4 Ei Suppher 5 C Supper 6 HB Supplier Suppter s Supplier EB SuppSer 10 HB Supplier 11 Supper 12 ME Supplier 13 E Supper 14 Supplier 15 Supplier 16 o Toggle 2 Dimensional 3 Dimensional View ry Ranking Count Cumulative Total Figure 75 Overview of ranking distribution 3D graphical analysis Each graph has two components a bar section and a line section The bar indicates the number of times an alternative was ranked in the corresponding ordinal rank This is indicated by along the bottom axis of the graph in Figures 75 and 76 The line component indicated by in Figures 75 and 76 is a cumulative indicator for the alternative in percentage terms When all of the ranking occurrences have been displayed in the bars the line will plateau to indicate 100 of the values for that alternative have been displayed UNCLASSIFIED 57 he p aE 2 B fie hoe af y a pa UNCLASSIFIED DSTO GD 0681 a i ee oe Maa ae
131. ononnaan cog Hoja 900000000 l SUDI Wer 10 APPT 11 1 30350 g 0 5792 6272 98 p 9 38 99 Select best Minimise Maximise Maximise Maximise Maximise Minimise Minimise Mine supply Total Cost Performance Reliability Reliability Reliabahy Supply Time Supply Tene Sup im arrangement i Type aorar Type 3 pen Type A Spares Type 5 Spes x To make the plot area clearer the top mid and low level plot groups have been deselected Cardinal Scoring Option is selected Note the axis scales of the minimise criteria Now the preferred intersection of the alternative is at the top of the axis i e the lower or more preferred values for these criteria are now located at the top of the axis while the higher values less preferred are located at the bottom of the axis Note that the lowest values of the criteria where the direction of preference is minimise now appear at the top of the axis Figure 103 CWViewer filtering alternatives example 2 11 4 Filtering Elements The CWViewer can also be used to filter Decision Makers structural elements There are two primary types of filtering however any combination can be used to suit various problems or reporting requirements When filtering elements it is highly recommended that the root element P 1 is always selected since this element indicates the overall performance of each alternative UNCLASSIFIED 81 UNCLASSIFIED DSTO GD 0681 1
132. or shows the number of searches that have been run If this number increases significantly and very few results have been added it is probably necessary to relax the search variation using the slider This can also be performed manually by entering a value in the Search Variation text box 13 1 6 Enable Auto Tune Using Auto Tune allows Decision Maker to automatically determine the search variation If you are unsure of the initial values to use for the search variation use the Auto Tune function However if the results are very close to the target score the variation will become very small and usually it will be necessary to relax the variation using the slider Auto Tune uses the values from the best worst and average variation boxes 13 1 7 Best Worst and Average Variation These three boxes can be very helpful when performing searches It is recommended that the first search be for a small number of searches such as the default of 10 When the search is complete the Best Worst and Average variation boxes will be populated These UNCLASSIFIED 103 UNCLASSIFIED DSTO GD 0681 values can be used to manually set the variation for a more thorough search of for example 30 to 50 or more results 13 1 8 Starting a Search To start a search click on a row header of the applicable selection In the example shown in Figure 127 the values of Supplier 12 are being compared to Supplier 2 as indicated by Decision Report Selection Sim
133. orizontal axis If you are charting a Criterion element the vertical axis will be scaled between the minimum and maximum values of the criterion s data UNCLASSIFIED 46 UNCLASSIFIED DSTO GD 0681 File Edit WieHal Project Tools Windows Advanced Snalsis Help EF Structure Model ala Data Model s Show weights Show Ordinal Results Show Cardinal Results Fa Refresh Figure 57 Charting scores 7 Simulations Decision Maker uses simulations to calculate the decision a number of times while allowing some or all of the criteria values to vary This provides a method for analysing a range of the likely outcomes that may be encountered for each alternative To use the simulation component in Decision Maker it is important to understand some basic concepts Simulation Run This is a unique number that identifies an individual simulation run For example if the Decision Problem is simulated 50 times each run will be identified by a number from 1 to 50 e Simulation Set This is a collection of Simulation Runs SimulationID This is a unique identifier generated when a Simulation Set is run and therefore identifies a group of Simulation Runs The remainder of this section describes how to run a simulation Later sections in the user guide will describe simulation reporting 7 1 Running a Simulation Running a simulation is simple However it does require that uncertainties have been entered for som
134. our alternative The title of each alternative must be unique Alternative Description Enter a detailed description of your alternative in this field the more detail the better ee Figure 39 The New Alternative dialog box 5 4 4 Criterion Element A Criterion element is used to represent each quantifiable consequential attribute scale or measure in your decision problem see Section 3 4 for more information on consequences Criterion elements represent each quantifiable attribute in your decision problem and they include attributes such as operating costs expected profit loss performance measures and UNCLASSIFIED 35 UNCLASSIFIED DSTO GD 0681 other characteristics of the objective they define The Criterion element has four fields as shown in Figure 40 5 4 5 Uncertainty Element The Uncertainty element represents the variation and or tolerance in a Criteria element This value may be known from documentation such as the tolerances of machinery parts or may be uncertain and the expected value may vary by a certain percentage All uncertainties in Decision Maker are represented as a percentage of the criterion value It is not a requirement to have any Uncertainty elements in the decision problem However to take advantage of the simulation components of Decision Maker one or more criteria require some level of uncertainty to be set When an uncertainty has been added to a criterion it cannot be removed If it is no
135. ovides a visual representation of Decision Maker s output as a graph of the cardinal and ordinal scores for each alternative To chart the results select an element in the Element Tree and select the View Chart menu located in Decision Maker s main menu bar If you select a criterion element Decision Maker will chart the results for that element UNCLASSIFIED 29 UNCLASSIFIED DSTO GD 0681 Data Model Select Best Supply Arrangement E A Select Best Supply p TETEE up D O O11 Maximise Reliability i be C3 Maximize Reliability Type B Spares ion ff C2 Maximize Reliability Type A Spares ae o 0 1 2 Minimise Supply Time i C4 Minimise Supply Time Type A Spares i 5 Minimise Supply Time Type B Spares E 7 Minimise Total Cost 4 Minimise Supply Minimise Total Cost seskea ie Jo es e ses r ps esos Spier im um Supplier 12 Supplier 15 9 Supplier 16 14 Se Figure 34 Wizard example the data model without data 4 6 Revise and Amend the Problem s Structure PEK Element Properties Title The best spare part supply armangement shall be able to supply the parts with the highest performance in reliability and supply time Preterent Direction Mams Now that you have structured your decision problem and have entered the problem s data for the criteria and or uncertainties you can calculate some scores You can also revise your problem This
136. ply arrangement 7 Number of Simulation Runs 10 0 C Show Weights Columns C UA Item Data Only J Bes 0 0002 Simulation Date 9705 2007 39am L Show Mod Columns No Search Matches 50 Average 0 05641 Created By Coston C Show Upper Alternative Data wi Besa Searches Performed 50 Wot 0 0492 P4 Select best supply arranger Alenas Supplier 2 Supplier 3 63850 0000 6295 2165 Supplier 9 i i 5933 4523 Supplier 5 Supplier 12 Supplier 3 63850 0000 68182410 Supplier 1 i 7426 2107 Supplier 8 7235 7109 7153 409 ee rf 6454 7988 58919 0462 Suppher 10 Sn Maximise M e Minmise Supply Minimise Supply Sead j Supplier 13 Cost Rehabilly TypeA Rebability Type E Time Type Time Type B Target 0 655 Spares Spares Spares aa 5 LA Item Value LA Item Value LA Item Value imi 1 S Minimise Supply Time Type B Spares LA Item Value ii Upper Alternative Lower Alternative Minimise Total rica Type A aiet 165 B Cog Spates Spares Supplier 11 pares Supplier 7 Supplier 16 Suppliet 4 5818 241 3503 57 0 004797150491 581a 241 8481 04 0 003797150431 5818241 8470 84 D 0 002797150431 Critena Name i 6518 241 8395 51 000020284956 C1 Minimise Total C 6816 241 8406 77 0 00020264956 C 2 Maximise Peliebi 5818 241 6336 36 0 00320284955 nH 6818 24 8327 63 0 00420284956 6818 241 832412 000420284956 6318 241 8307 63 6818 241 8289 43 6818241 828532 618 241 8261 76 6818 241 823967 6318 241 8238 65 681
137. r Example 2 To determine the extent of domination on a particular element the Domination Scoring Matrix Viewer can be used with the CWViewer This is shown in Figure 108 Smndabon Dask Existing DecisionMaker II Simulations Number of Atematves T5 See nee aeoea x Number of Simulation Runs 50 Simulation Sets Simulation Run Simulation Date 1 06 2007 1049 4M Created By CostoliC G1 Minimise Total Cast 082 eee see S Sei a Reading down the column Supplier 5 and along the row Supplier 2 the cell containing the value 1 is located This indicates Supplier 5 scored better than Supplier 2 in 100 of the simulation runs Conversely along the row Supplier 5 and down the column Supplier 2 the cell containing the value 0 is located This indicates that Supplier 5 scored lower than Supplier 2in 0 of the simulation runs Also shown is the comparison of Supplier 5 and Supplier 14 The value of 1 indicates that in 100 of the simulations Supplier 5 scored better than Supplier 14 This observation is more apparent when viewed in the CWViewer plot analysis Figure 108 Scoring Matrix interpretation maximise reliability 11 5 6 Using Domination Scoring Matrix with CWViewer Example 3 For the objective Minimise Supply Time it is necessary to determine the number of times or the percentage of time that one particular alternative ranks higher than another This is shown
138. r this criterion and so for every simulation run the values remained the same For C 2 Maximise Reliability Type A Spares Supplier 13 s reliability was better than Suppler 3 s reliability 52 of the time For C 5 Maximise Reliability Type B Spares Supplier 13 s reliability was only better than Supplier 3 s reliability 8 of the time For C 4 Minimise Supply time Type A Spares Supplier 13 s supply time was better than Supplier 3 s supply time 50 of the time For C 3 Maximise Reliability Type B Spares in all simulations Supplier 14 s cost value was better than the other Suppliers This shows the selection of alternatives Here the comparison being made is what fraction of simulations Supplier 13 s criteria scored better than Supplier 3 s criteria Figure 89 Domination Scoring Matrix example 10 3 Dominant Alternatives One of the uses of the Domination Scoring Matrix is to determine dominant alternatives A completely dominant alternative is one where all of its criteria values score better than all the criteria values of another alternative This has the effect that the dominant alternative will always be more preferred than any other alternative regardless of the decision tree or weights that are used provided all directions of preference for each and every criterion remains the same UNCLASSIFIED 69 UNCLASSIFIED DSTO GD 0681 Within the Domination Scoring Matrix for every criterion any alternative that scores 1 0 agains
139. r alternative s cardinal score For example if the target cardinal score is UNCLASSIFIED 102 UNCLASSIFIED DSTO GD 0681 0 8033 a search variation of 0 0005 indicates that any search results between 0 80328 and 0 80338 will be accepted and added to the search results table 13 1 2 Search Progress Bar The Search Progress Bar progresses as the number of search matches increases It is complete when the total number of search matches has been reached 13 1 3 Search Relaxation Slider There are times when a search will not return any results When this occurs it can be due to two factors 1 Most commonly search results may fall just outside the search variation range When this happens moving the Search Relaxation Slider to the right will increase the search variation This relaxes the search constraints and allows matches to be added to the results table and 2 The constant criteria value of the lower alternative is significantly inferior i e it is impossible to find any results unless the variation is set to a larger value This signifies that the variable criteria will not cause a ranking reversal 13 1 4 Number of Search Matches The Number of Search Matches is the desired number of results to search When performing searches for the purposes of trade off analysis a large number will provide more clarity Usually a minimum number of 50 search matches is recommended 13 1 5 Searches Performed The Searches Performed indicat
140. rcentage Limit will be coloured rust e Alternatives with a cardinal score between the Top Percentage Limit and Low Percentage Limit will be coloured pale blue colour Toggles the axis labels on the plot e Selecting or unselecting any of these checkboxes will hide display the applicable group Displays plot with the cardinal scoring option Section 11 3 2 Figure 94 CW Viewer Plot Options 11 2 Plot Area The Plot Area is used for a graphical analysis of the decision problem A special feature available on the plot area is a context menu to enable the plot graphic to be copied to the Microsoft Windows clipboard This will allow for pasting the copied area into other applications such as Microsoft Word or Excel The menu for copying the plot graphic is opened by clicking the right mouse button in the Plot Area 11 2 1 Plot Area Without Cardinal Scoring Option The CWViewer without the cardinal scoring option facilitates the identification of correlations between the criteria in the decision problem This is shown in Figure 95 11 2 2 Plot Area with Cardinal Scoring Option The CWViewer with the cardinal scoring option provides a view where each axis is rescaled so the most preferable point of crossing for any alternative is at the top most point This provides a way to identify the more preferable alternatives since they will intersect each axis at or near the top even if the criteria s direction of preference is minimise
141. re information on creating the structural elements for your decision problem 6 Analysing Your Problem in Decision Maker Decision Maker provides for both quantitative and qualitative modelling of your decision problem Both models are equally important for analysis of your decision problem although they have different roles in the decision analysis process Section 5 presented Decision Maker s tools for developing a structural model of your decision problem The structural model is a qualitative problem model constructed using various descriptive elements including Objective Alternative and Criterion elements that when combined form the structure of your decision problem The structural model is a powerful tool that enables traceability throughout your decision making process The structural model also provides a method to manage the complexity of hierarchical decision problems and their numerous problem components Despite its usefulness the structural model discussed so far does not provide any method to support quantitative analysis of your decision problem For quantitative analysis real data must be collected for the criteria defined in your problem s structure Decision Maker provides a method for you to enter the data collected analyse your problem and chart the output Decision Maker uses a data model to do this as shown in Figure 49 Objectives Alternatives Criteria Data Y Y E Cardinal Ranking Dec
142. rs At times rounding errors may appear The magnitude of these errors is minimal and the maximum error that can be expected is approximately 1 2 Use of the Single Quote Decision Maker may produce errors if the Single Quote is used in fields This includes element titles and descriptions It is highly recommended to avoid their use File Import Values When importing data from files numerical values must not contain a comma or dollar sign UNCLASSIFIED 120 Page classification UNCLASSIFIED DEFENCE SCIENCE AND TECHNOLOGY ORGANISATION DOCUMENT CONTROL DATA 1 PRIVACY MARKING CAVEAT OF DOCUMENT 2 TITLE 3 SECURITY CLASSIFICATION FOR UNCLASSIFIED REPORTS THAT ARE LIMITED RELEASE USE L NEXT TO DOCUMENT Decision Maker User Guide CLASSIFICATION Document Title Abstract cee m Ner 4 AUTHOR S 5 CORPORATE AUTHOR Carl Costolloe Moya Tyndall and Anthony Woolley DSTO Defence Science and Technology Organisation 506 Lorimer St Fishermans Bend Victoria 3207 Australia 6a DSTO NUMBER 6b AR NUMBER 6c TYPE OF REPORT 7 DOCUMENT DATE DSTO GD 0681 AR 015 300 General Document April 2012 8 FILE NUMBER 9 TASK NUMBER 10 TASK SPONSOR 11 NO OF PAGES 12 NO OF REFERENCES 2007 1139073 1 NAV 07 084 DGMARSPT 120 4 13 DSTO Publications Repository 14 RELEASE AUTHORITY http dspace dsto defence gov au dspace Chief Maritime Platforms Division 15 SECONDARY RELEASE STATEMENT OF THIS DOCUMENT Approved for pu
143. s an overview of the dialog box the Wizard presents to you for creating a Problem element Section 5 3 1 also provides an example to help understand and choose the direction of preference 4 2 3 Step 3 Define your Objectives The third step requires you to define your objectives see Section 3 2 This is achieved by creating an Objective element for each objective you have in your decision problem The Wizard will ask you to Please enter the number of Objectives you know of If you do not know your objectives at this stage let the Wizard know by entering zero 0 in the appropriate field otherwise enter the number of objectives When you have entered the number of objectives for your decision problem the Wizard will request you to create an Objective element for each objective This requires you to enter a short title a description and a direction of preference for each objective UNCLASSIFIED 13 UNCLASSIFIED DSTO GD 0681 Refer to Section 5 3 2 for an overview of the dialog box the Wizard presents to you to create each Objective element Section 5 3 2 also presents an example to assist in choosing a direction of preference When creating each Objective element you can specify if the objective is a sub objective of Q another Do this by setting the Sub Objective Of field in the element s creation dialog box By default each new Objective element is set as a sub objective of your Problem element 4 2 4 Step 4 Define your Alter
144. s installed from an installation Compact Disc CD 2 1 1 System Requirements Mandatory requirement 1 Microsoft Windows XP Operating System Minimum requirements 1 Microsoft Internet Explorer 5 01 2 Microsoft NET Framework 2 0 and 3 Microsoft SQL Server 2005 Express Edition NOTE Microsoft NET Framework 2 0 is available as a free of charge update from Microsoft If your system has an earlier version of the NET framework download the latest NET framework from Mircosoft s website 2 1 2 Installing Decision Maker from CD 1 Insert the Decision Maker installation CD into your computer 2 Start the installation by double clicking setup exe 3 Follow the on screen instructions to install the Decision Maker software If SQL Server 2005 Express Edition is not installed on the computer it will be installed first 4 If you have a previous version of Decision Maker installed on your computer the Installation Wizard will ask you whether to Repair or Remove your current version If you want to install the current version from the CD select Remove and continue following the Installation Wizard s instructions UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 2 2 Uninstalling Decision Maker 1 From the Start button on the Microsoft Window s taskbar open the Control Panel and select Add or Remove Programs 2 Within the Currently installed programs and updates list find the Decision Maker application and select Remove 3 Thi
145. s process will uninstall the Decision Maker application from your computer but it will not remove any project files you have created 2 3 Starting Decision Maker 1 Open the DSTO program folder from the Programs DSTO Decision Maker II menu within your taskbar Start button 2 Alternatively the Decision Maker Installation Wizard has placed an icon as shown in Figure 2 on your desktop Double click the icon to start the application Decision Maker II lnk Figure 2 Decision Maker desktop icon 2 4 Exiting Decision Maker 1 Prior to exiting Decision Maker ensure that you have saved your project by using the File Save menu 2 Decision Maker can be exited by using the File Exit menu 3 A Guide to Making Better Decisions There are many processes and techniques that can be used to assist decision makers 2 3 The PrOACT structured decision making process developed in 4 is recommended and used in this user guide to assist users in structuring their decision problems The acronym PrOACT is named after the key elements within the structured decision making process i e Problem Objectives Alternatives Consequences Trade offs In this context Decision Maker is used throughout the PrOACT process to support problem structuring and is used in the Trade offs step instead of the methods suggested in 4 An overview of the PrOACT process is given in Figure 3 Other elements of the PrOACT process that may require considerat
146. sequences Decision Maker provides the tools so that you can follow this process to structure your decision problem Once you have structured your problem you can then begin adding data and performing Trade off analysis studies using the decision analysis tools provided by Decision Maker It is important to note that during the initial stage of defining your problem in Decision Maker you do not need to be overly thorough It is more important that you begin structuring your problem with the information you have and then refining your problem later through an iterative structuring and reviewing process Furthermore prior to commencing any decision making it is also important to understand what your problem really is Objectives alternatives and the consequences i e the decision criteria can be added or removed at any stage using the tools provided by Decision Maker 5 2 Structuring Your Decision Problem Decision Maker provides two methods for structuring your decision problem The first method uses the Wizard and the second method is a manual structuring process The manual method is more flexible since you can add remove elements to from your decision problem in your own time when you have the information available If you choose the Wizard it is best that you have available some elementary objectives alternatives and criteria before you begin so that this information can be entered into the Wizard The Wizard will then structure your prob
147. shown in Table 1 Table 1 Consequence table for the Supply Manager s Dilemma Problem Alternatives Problem Objectives amp Criteria Potential Companies Minimise Maximise Reliability Minimise Supply Time Contract Type Alternative Total Cost Type A Type B Type A Type B pee pee Ez Ez MTBF MTBF MTTS MTTS Hrs Hrs Days Days suppleri Corones waga widget 7o00 vso 720 8 8 suppier2 Combined Gadget Gadget 60 000 6600 9100 7 7 Suppiers Combined cismo cismo s5000 620 e60 s 8 supplera Combined Tue Tue 75 000 s10 e80 o 9 suppliers Separate Widget Gadget 7150 7 900 9100 e 7 Suppiers Separate widget ismo 68750 730 e60 s 8 suppier7 Separate Widget Tute 79750 7900 6e00 e o Suppiers Separate Gadget waget 7150 6600 720 7 s C suppiero Separate Gadget_ Gismo 69 250 6600 6600 7 s Supplier 10 Separate Gadget Turie 74250 6600 e80 7 9 suppliers Separate Gimo Waget 68 750 6200 720 s suppliers Separate Gismo Gadget 69 250 6200 9100 e 7 Sumteri soaae f como me f riso f eeo sa f 6 fo Supplier14 Separate Tute Waget 70750 610 7200 o s supplier1s Separate Turie Gadget 7a250 610 o0 o 7 Supplier 16 Separate Tue Gismo 71500 6100 6600 o 8 3 5 Trade offs In this step enter the
148. ssosssosssosssosssosssos 83 11 5 1 Mirarmise Total Cost aaran E N 84 11 5 2 Maximise Perr orman E esgerin a a N ues 84 Ikos Maximise Renabli yersen a n 85 11 5 4 Using Domination Scoring Matrix with CWViewer Example 1 85 11 5 5 Using Domination Scoring Matrix with CWViewer Example 2 86 11 5 6 Using Domination Scoring Matrix with CWViewer Example 3 86 TL6 Criteria RelATIONSINI PS anionen aaa ct siecerssceuss tat deaueeesiues tepeucee res S 87 11 7 Producing Meaningful Plots yi sciccistectsesusccoesevacessexacseiosvicsacsendncseatviesassedecabiodsessaceive 88 12 WEIGHTS SENSITIVITY varecscogiteceicteticssreaacecg ieee a enue nee eee 89 T21 Total Dominationis aone ei ae aaia aae iaa aeaa 91 122 Partial Dominatio meenen a A eine seta 92 12 3 Some Final Notes on Dominating Alternatives essseossoossoessosssosssoessoessoessos 92 124 Domination Iree Vite sonenneciene oioi a 92 1293 View OPHoOnS acas a a a e 94 12 6 Non Dominated Sets Table ooooeooseossoossessosssosssosssosssoesssesssosssosssosssosssosssosssos 94 127 Direct RelINONS orri e EE E AENA EAE EOE 95 12 8 Sub Level Weights Analysis sisisccscsscessssvesscssscsscssastcoesss susseessuc coupes esecsseuecseedstesenss 96 129 Minimum Change Distr Ditton veciiciescdescccccnisvaonstestevcenssestacvoacevessteeoeseventeceeneeeanes 97 P2410 Top Level Analysis sce iainetuiwie niin ieee alan mentee eteeies 98 CRITERIA SENSITIVITY rira ane cccpiseceuvscevvaveyacnsa
149. t another alternative is a dominant alternative in every simulation Due to the nature of simulations there may be a scenario where no alternative is completely dominant Hence the Domination Scoring Matrix can assist in finding the alternatives that are almost dominant and for all practical purposes should be considered as dominant Figure 90 shows some examples of dominant alternatives using criteria that correspond to the Supply Manager s Dilemma presented in Section 2 o In these examples the column alternative could safely be considered dominant Here C 1 has a weighting of approximately 50 not shown in diagram while it scores better on performance it could be premature to dismiss this alternative as being superior In this example C 4 has a weighting of approximately 13 not shown in diagram This is a case of not complete dominance in the true sense however for practical purposes could be considered to be so Figure 90 Domination Scoring Matrix examples 10 4 Holistic View When the Grid Zoom Scale is set to the maximum zoom out value the grid appears as shown in Figure 91 This view can initially be used after running a simulation to assess the various alternatives and eliminate some of the poor scoring alternatives before analysis of the remainder using the CWViewer It is important to use the Domination Scoring Matrix and Scoring Matrix to determine if alternatives should be considered for short listing espec
150. ta for each of the Supply Manager s Dilemma criteria has been entered a score can be calculated to evaluate the performance of each alternative To calculate the scores select the Project Calculate menu or click the Calculate quick access button on the form UNCLASSIFIED 28 UNCLASSIFIED DSTO GD 0681 Data Model Select Best Supply Arrangement TRE GG 0 1 Maximise Performance 0 1 1 Maximise Reliability C3 Maximise Reliability Type B Spares ion ff C2 Maximize Reliability Type A Spares Description fl o 0 1 2 Minimise Supply Time What is the best arangement for the supply of Type A and B Doo be C4 Minimise Supply Time Type A Spares spare parts bag C5 Minimise Supply Time Type B Spares fw gf C1 Minimise Total Cost H Prefered Direevian Ordinal Score Cardinal Score ee ea Smmea fen en o sexes f n NaN NaN NaN Supplier NaN NaN NaM Supplier 8 NaN NaN NaN T E saen p n Supplier 13 Supplier 14 Figure 33 Wizard example the data model upon Wizard completion Note that when selecting between elements in the Element Tree Decision Maker will automatically re calculate and update your decision problem scores To disable the automatic calculation feature un check Background Calculating in the Project menu located in Decision Maker s main menu bar 4 5 Chart the Results of the Decision Analysis Process Charting the results calculated in the previous section pr
151. teria Sensitivity Fa Domination Scoring Matrix Fil Figure 63 Starting simulation reporting AN If an alternative has the lowest average and best scores all being the same value then it graph A work around for this is outlined in Section 8 2 3 8 1 The Simulation Reporting Interface The following subsections describe the four elements shown in Figure 64 8 1 1 Decision Report Selection Figure 65 shows an enlarged view of the Decision Report Selection area of Figure 64 When Simulation Reporting is selected option 9 Figure 64 will always be disabled UNCLASSIFIED 49 UNCLASSIFIED DSTO GD 0681 53 Simulation Reporting Select best supply arrangemen 1 alpa porting pply Decision Report Selection Simulation Details Existing DecisionMaker I Simulations eo Problem Name Select best supply arrangement Select best supply anangement v i f rere Number of ANematives 16 Simulation Date 1 06 2007 1249 25 AM 1000 Number of Simulation Runs 50 Ceated Ry CostollC _ _ S EE A P 1 Select best supply arrangement Scoring Range Ranking Distribution Weights Summary b A 4 0 aa at et 4 Allemative Lowest Cardinal Average Cardinal Highest Cardinal Best Ordinal Average Ordinal Worst Ordinal 0 1 1 Maximise Reliability 1 0000 1 000000 1 0000 1 i Hg C2 Maximise Reliability Type A Spares i 0 5980 0 715740 0 8590 iu amp C3 Maximise Reliability Type B Spares 0 5930 0 773580
152. the Decision Maker Wizard as shown in Figure 9 Running the Wizard from the File menu will open the Wizard s navigation dialog box shown in Figure 10 In the first step you are required to create a project for your problem When you click the Next button on the Wizard you will be presented with a New Project dialog box Section 5 2 2 Enter the UNCLASSIFIED 15 UNCLASSIFIED DSTO GD 0681 information presented in Table 2 into the New Project dialog box The completed New Project dialog box should appear like that shown in Figure 12 Table 2 Wizard example project information Field Text Project Title Acquisition Contract Renewal Project This project has been created to demonstrate that an organisation or department can use Decision Maker to Support multiple decision making activities for any one project or task Project Description mi f w z gt a New Proj ect F Save Project File Save in My Documents Project Title Pe ete Rh biit Boat 4 G dm Acquisition Contract Renewal Project D Grex My Recent My eBooks Documents 4 my Music Emy Pictures 2 SQL Server Management Studio visual Studio 2005 Project Description This project has been created to demonstrate that an organisation or department can use Decision Maker to support multiple decision making activities for any one project or task Desktop My Documents 3 My Computer J
153. the Non Dominated Sets The The Search Results tab can be selected to view the results as a table Ua tab can be selected The Search Results tab can be selected to view the results as a table view the results as a table o a meae aie Epa EA oints corresponds to a row in the search results table o The tite of the Minimum Weights Distribution indicates the alternatives that are being compared Figure 121 Example of the Minimum Change Distribution 12 10 Top Level Analysis The top level analysis only considers criteria elements It also uses relative weights values This means that if the root problem is selected then all criteria in the decision problem will be used in the analysis Similarly if an objective is selected then all the criteria that exist in the decision tree below that objective will be used in the analysis Figure 122 shows an example of top level analysis If the decision tree is small this is the preferred weights analysis to use when analysing the tree structure of the problem For large decision problems with complex trees it may be more preferable to use the sub level analysis at various sections within the tree to assist in decision structure development UNCLASSIFIED 98 fy Weights Sensitivity Decision Report Selection Existing DecisionMaker II Simulations Select best supply anangement Simulation Sets Simulation Run mea w B P Select best supply arrangen 4 Akemative t Supplier 2
154. the data file Then select the Tools Import Alternatives From File menu option This is shown in Figure 146 The dialog box shown in Figure 147 will then appear Browse to the data file location select the file and click Open The Alternative Input Mapping window will then appear Figure 148 provides an overview of the functions that can be performed File Edit View Project Windows Run Simulation Generate Executive Summary Import Alternatives From File Options Figure 146 Starting the data import utility UNCLASSIFIED 114 Advanced Analysis F4 FQ Help Fi UNCLASSIFIED DSTO GD 0681 Look in 9 Decision Maker Data E ce AE My Recent Documents 4 Desktop My Documents hy Computer File name Supply Managers Dilemma_Data ceyv Mu Network Files of type Figure 147 Open data file for import dialog box se Element Description Comma Tab CI Semicolon ec oe The Column wilBelgnored 4 Auto Generate Altemative Titles Recommended a 4 native Ti This Column will be the Title of the Altemative tEeanaiiefee oo A Altemative Descnptic This Column will be the Altemative Description Stet Numbering At 1 Maximise Reliab iy Type A Spares Maximise Rebabdity Type B Spares Minimise Supply Time Type A Spares Minimise Suppl Time Type B Spares There are three types of delimiter selection available and they can be combined to format the input into individual columns The u
155. tility can automatically generate alternative titles The table view shows the available items that can be used for column mappings These two tabs show the import process The Importing Format tab is the main tab used in mapping the columns to criteria and or alternative titles and descriptions The Raw File Contents tab provides a view of the contents of the data file and is for viewing purposes only When the mapping is complete and ready for importing this button will process the mappings that have been set Figure 148 Overview of Alternative Input Mapping UNCLASSIFIED 115 UNCLASSIFIED DSTO GD 0681 15 1 Setting the Delimiter Figure 149 shows the window after the delimiter selection has been made In this example the data file was tab delimited indicated by in Figure 149 Also the contents of the file already contained the title for each alternative Hence Auto Generate Alternative Titles is unchecked indicated by O in Figure 149 Finally the table is shown at the location indicated by O in Figure 149 In Figure 149 no mapping has yet been performed This is evident by the default column names in the importing table Mapping column names is described in Section 15 2 15 2 Mapping Column Names The process of mapping column names is shown in Figure 150 The mapping was performed by selecting with the left mouse button and holding it down and dragging the mouse to the column indicated by When the symbol appears ne
156. tional two days of administration paperwork for the separate contracts He considers employing a temporary staff member to do the additional two days of administration for the separate contracts The cost of a temporary staff member is 300 per day and so he adds 600 to the total cost of the separate contracts He deletes the administration column in his consequence table and clicks the Calculate button Figure 6 shows the result 4 Quick Start using the Decision Maker Wizard The example in this section makes use of the hypothetical Supply Manager s Dilemma that was introduced in Section 3 The following subsections present a step by step example on how to structure and analyse your decision problem in Decision Maker The scenario assumes the Supply Manager has not previously used Decision Maker but does have an understanding of the problem s structure from using the PrOACT process discussed in Section 3 This includes the identification of the problem s objectives and criteria It is assumed the Supply Manager has identified the problem s objectives criteria and alternatives and has access to relevant data for each criterion from the possible suppliers UNCLASSIFIED 10 UNCLASSIFIED DSTO GD 0681 v ay Data Model Select best supply arrangement s fm X i gt Select best supply arrangement Element Properties E Minimise Total Cost lt Title O Maximise Performance
157. tions Panel and Search Results Table will update the progress and provide feedback As mentioned earlier this information can be used to further refine searches as shown in Figure 128 Search Options Show Calulation Columns Non Dominated Search Search Variation 0 00124 Auto Tune Show Weights Columns f a J a Best 0 00038 Number of Search Matches Average 0 00454 YW ED o Manual search variation is set i e not Auto Tune Simple analysis of the best worse and average results indicate that the Search Variation could be smaller for example 0 0001 Figure 128 Criteria Sensitivity Search Options in progress UNCLASSIFIED 104 UNCLASSIFIED DSTO GD 0681 13 2 Using the Auto Tune Option The Auto Tune option assists in finding the optimal search variation range However it can slow the progress of a search by refining the variation to a range which is impractically too small Figure 129 shows a three stage process to deal with this issue 1 Stage 1 shows the search soon after it has started 2 Stage 2 shows the view after 232 searches have been run and only approximately 7 results found as indicated by the green progress bar This is due to the very low variation value assigned by Auto Tune As the search progresses the variation rapidly decreases and locating new matches begins to slow down 3 Stage 3 shows the view when the relaxation slider has been used and it can be seen that progress has increased by relaxi
158. tive scored more often somewhere in the lower range of scores in the simulation runs 8 2 Scoring Range The Scoring Range tab provides immediate information about how each alternative performed across all the simulation runs in a simulation set The graphical window provides for cardinal scoring and ordinal scoring analysis Figure 69 presents the elements of the Scoring Range tab 8 2 1 Cardinal Scoring Ranges The top half of Figure 69 shows some basic information collected from a simulation set namely the range of cardinal scores that each alternative was assigned and the average cardinal score Here the cardinal scores are indicated by the red bracket In the case of cardinal scoring the higher score is the better option UNCLASSIFIED 52 UNCLASSIFIED ia Alternative Lowest Cardinal Average Cardinal Highest Cardinal Best Ordinal Average Ordinal Worst Ordinal DSTO GD 0681 scoring Range Ranking Distribution Weights 5 Supplier 1 0000 7 000000 1 0000 1 Supplier 5 0 5960 0 715740 0 6590 Supplier 12 0 5930 OFF 3560 0 9560 Supplier 3 0 5310 0 694270 0 9020 Supplier 9 0 3660 0 620760 0 7860 Supplier amp 0 3290 0 472120 0 7520 Supplier 1 0 43510 0 525140 o 7020 Supplier 6 0 3340 0 494420 0 6610 Supplier 15 0 1650 0 369460 0 5460 Supplier 11 0 1740 0 367560 0 5160 Supplier 10 0 14200 0 273560 0 4630 Supplier 13 0 0550 0 209560 0 4560 Supplier 7 0 0000 0 147660 0 3500 Supplier 16 o 0070 0 169720 0 2920 Supplier 4 0 00
159. tracts which are about to expire were originally signed The Supply Manager is considering his options and has defined his problem as Keep the current arrangement or amalgamate both contracts into a single contract to supply both Type A and B spare parts and offer it to Widget Inc and Gadget Inc and then choose the best proposal The trigger for this decision problem is that the existing contracts are about to expire However the Supply Manager has already limited his problem by including possible suppliers in his problem definition There may be other companies that could supply the spare parts A better problem definition that would keep his options open is What is the best arrangement for the supply of Type A and B spare parts and who can provide them The key word in this definition is best By considering our objectives in the next step we will be able to define best for this example 3 2 Objectives Let objectives be your guide 4 They help to determine the information required and assist you to explain your choices to others Objectives become the decision criteria and it is therefore important to spend time considering and defining your objectives Some of the ways that may help to identify your objectives include 4 1 making a wish list 2 deciding what you want to avoid and 3 brain storming UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 When you have a list of concerns and wishes asking why t
160. tton to close the dialog box Decision Maker will then return to the main user interface Simulation results can now be viewed using any of the following tools e Simulation Reporting Section 8 e Scoring Matrix Section 9 e Domination Scoring Matrix Section 10 e Cobweb Plot Viewer CWViewer Section 11 e Weights Sensitivity Analysis Section 12 or e Criteria Sensitivity Analysis Section 13 UNCLASSIFIED 48 UNCLASSIFIED DSTO GD 0681 Confirm x a The Simulation has Completed Successfully Figure 62 The simulation completed successfully dialog box 8 Simulation Reporting The simulation reporting interface provides the summary for a simulation run and is the first analysis that should be viewed to provide a summary of the performance for each alternative It presents summary information that can be used as a starting point for further elimination of poor performing alternatives and as a means of sorting the top performing alternatives before a more thorough analysis is performed The simulation reporting interface can be started from the Decision Maker main user interface and does not require a project to be open Do this by selecting the Advanced Analysis Simulation Reporting menu option as shown in Figure 63 The window shown in Figure 64 will then appear File Edit View Project Tools Windows Advanced 4nalysis Simulation Reporting Scoring Matrix Cobweb Plot viewer Weights Sensitivity Cri
161. ty Type E Time Type Time Type B Spates Spares Spares Spares Mirunze Total Upper Alternative Lower Allemnalive Cost o2 e2 ogs ow mz Supplier 9 Supplier 1 UR 0 234 nazas as om Supplier 3 Supple i0 085 022 0373 0283 00 Supplier 9 Supplier 11 Oo aos DEN oo on Suppher Supplier 13 0 14 0 082 0728 O04 agza Supplier 9 Supplier 14 0 055 O07 0 365 o052 0 477 Supplier 9 Supplier 15 0185 116 0 328 0 248 124 Supplier 9 Supplier 5 0 205 0 147 O16 0354 0134 Suppher 9 Supplier 6 0 066 0 334 0 258 0 133 0203 Supplier 3 Supplier 7 5 034 0 36 0 258 0 202 0 086 Supplier 3 Supplier 0 05 0 042 O44 O27 0 198 3 4 o Upper Alternative the selection in the domination tree Lower Alternative all non dominated alternatives scoring worse than the upper alternative Decreases in weights are coloured red Increases in weights are coloured blue Each column corresponds to one of the selected criteria O The first row shows the actual weights that are used Clicking on the row header activates a search Figure 117 Non dominated sets table 12 7 Direct Relations To start a search click on the row header in the Non dominated Sets table The search will find a range of weights that will result in a ranking change for the selected alternatives The results are easily interpreted when viewed graphically The Search Results Table contains the individual weights sets that would be required to i
162. ues are kept constant and some or all of the lower alternatives values are allowed to change Generally some of the lower alternatives criteria can be kept constant For example the alternatives Supplier 2 and Supplier 5 are compared in Table 17 This example shows how the criteria values for Supplier 5 would have to change to make it more preferred than Supplier 2 Table 17 Criteria Sensitivity analysis example Minimise Maximise Maximise Minimise Minimise Supplier Total Cost Reliability Reliability Supply time Supply time Type A Spare Type B Spares Type A Spares Type B Spares Supplier2 60000 6847 8729 Supplier5 72100 7436 9274 In Table 17 consider the directions of preference for each criterion the superior values have been coloured blue To search for changes in the values of Supplier 5 that will make it more preferred than Supplier 2 only the lower values would be allowed to change These are Minimise Total Cost Minimise Supply Time Type A Spares and possibly Minimise Supply Time Type B Spares This last criterion could also be kept constant since the two values are almost equivalent This comparison allows for the formulation of the following search problem UNCLASSIFIED 100 UNCLASSIFIED DSTO GD 0681 Simulation Details Options Existing DecisionMaker II Simulations Number of Atematives 16 C ShowCalulationCoumns C Non Dominated Search SearchAocwacy 000M2 Z AutoTune Select best sup
163. ulation Details Search Options S Existing DecisionMaker I Simulations Number of Atematives 116 C Show Calulation Columns C Non Dominated Search Search Variation 0 0005 C Auto Tune Select best supply anangement ett upp erate Number of Simulation Runs 50 C Show Weights Columns A Best 0 00171 Simulation Sets Simulation Ruri 1000 C1 Minimi Simulation Date 1 06 2007 10 49AM C Number of Search Matches 20 Average Infinity mh i Created By CostollC Stop Search Searches Performed iT j Worst 10 00171 d LA Item Vals LA Item Value LA Item Value LA Item Value LA Item Value g 3 Maximise Maximise Minimise Supply Minimise Supply Minimise Total Reliabiliy TypeA Reliability TypeB TimeTypeA Time Type B an Spares Spares Spares Spares 60000 0000 6557 7456 8885 6509 6 6211 6 8568 Supplier 2 Supplier 5 72100 0000 7482 0333 9423 6208 8 1421 7 4945 Supplier 2 Supplier 3 55000 0000 6085 4914 6671 9032 7 8976 8 1862 Supplier 9 6794 6271 6597 0123 Supplier 12 850 00 6093 9995 8824 5025 Supplier 1 7531 3288 7418 9127 Supplier 6 72100 0000 6812 0240 7367 4565 Al Supplier 15 i Supplier 6 69350 0000 7448 4291 6676 5683 o eo m nnlie nnlie DES ARDER dig Agqa on Criteria Upper Lower Name Alternative Alternative alier 12 ater JLL C 2 Maximi a C 3 Maximi Figure 127 Criteria Sensitivity starting a search When the search begins the Op
164. upplier 6 have been selected ry Supplier 3 ranking in tabular form Supplier 6 ranking in tabular form Figure 77 Filtering alternatives Ranking Distribution Tabular UNCLASSIFIED 59 UNCLASSIFIED DSTO GD 0681 Scoring Range Ranking Distribution Weights Summa cmd Bi E ni of 50 45 E Supplier 3 wt I Suppter 2 g 3 MH Suppliers 5 30 VIALLLLLT LERURA AARAA A 11 13 15 16 Fi Bid example Supplier 2 Supplier 3 and Supplier 6 have been selected o Legend used to show the alternatives being displayed in the graphical view In this Supplier 3 ranking in graphical form Supplier 6 ranking in graphical form o Use the 2D 3D switching option if it becomes difficult to interpret the graph or to select fewer alternatives Figure 78 Filtering alternatives Ranking Distribution Graphical 3D UNCLASSIFIED 60 UNCLASSIFIED DSTO GD 0681 _Sconng Range Ranking Distribution Weight Summan Tabular Graphical a 50 m a 45 i ri 35 fi j F o a i 30 Wes S Suppher 3 T F 5 Suppher 2 Supplier 6 a 20 15 Pe k ii nik i Legend used to show the alternatives being displayed in the graphical view In this example Supplier 2 Supplier 3 and Supplier 6 have been selected Supplier 3 ranking in graphical form o Supplier 6 ranking in graphical form fewer alternativ
165. used to select the level for analysis Here the top level is shown so all criteria weights will be displayed o All the weights for the problem P 1 Select Best Supply Arrangement are shown However if 0 1 2 Minimise Supply Time was selected only the weights for C 4 and C 5 would be displayed The top three rows summarise the data with the minimum average and maximum weights for each criterion The weights calculated for each simulation run is displayed in the table There are three tabs available for viewing the weights data in various ways tabular as shown in the figure as a weight a range which graphically displays the data presented in the first three rows Section 8 5 2 and graphically which is a graphic representation of the weight data Section 8 5 3 Figure 80 Overview of Weights Summary 8 5 2 Weights Ranges Figure 81 shows an example of the Weights Ranges display for the top level element in the Simulation Reporting Decision Tree If some criteria weights have a small range and it is desirable to examine them more closely this can be done by selecting an element further down the tree For example to compare all the weights excluding C 1 Minimise Total Cost select O 1 Maximise Performance in the Simulation Reporting Decision Tree The display will change as shown in Figure 82 UNCLASSIFIED 62 P 1 Select best supply arrangement HE C 1 Minimise Total Cost 3 0 0 1 Maximise Performance O
166. useunyacouaevvereyauseersecssseseaoearvteiars 99 13 1 Criteria Sensitivity Search OptionsS essseesseessessseessesssosssosssosssossosssoessosssoesso 102 13 1 1 Searc V ata Oei a A E E 102 VG AZ CARCI ROC CSS Dar eee E N 103 13 13 Search Relaxation SUP nisciroriernni enori ae nnn aa a E 103 AA INuimberor Search Matches cicac acts crcnses dents ccusstcet sue actuatomvadatsceeardee 103 T95 Searches RemOrmied sacs eed sen auntiie iat teteneeetdteelaacaneaudaten eects cca 103 bO Enable Ato T UNE iue A ener rere a eee 103 13 1 7 Best Worst and Average Variation eeseseeereereerererrerrrrerrerere 103 tS Otaring a SOAKED orian a 104 13 2 Using the Auto Tune Option osecctcesiscescseectelcesecadeceusdecesvess sbcaaiaacectuesecsnseeedecassanerss 105 13 3 Analysing Search Results sissies ances canes ese a E 106 13 3 1 The Result Variation Column ss cesysceessnassnnieseceadeuedudtecustndecestieseoduaoete 107 UNCLASSIFIED UNCLASSIFIED DSTO GD 0681 Boz WraGde Ore Analysis nennen tach oad s EES 107 14 DATABASE MANAGEMENT INTERFACE cssssstessesessccssccessceessesessesesceeees 108 14 1 Generalac nesie eia a a aaia 108 14 2 Database Manaceme nt iiis a 109 14 2 1 SATO Seaia N 109 14 2 2 Database File Administration sccccccscssserceesssvesntavsecevesaneeeasevens 110 1429 PEPOT OZP ANINE anaa leat aa inte 110 1424 Tmportifrom Zip Archiv snesreiecons heiren aia 111 VAD D Ope Data bI E ere O E E a 112
167. with some degree of direct relationship Figure 110 Criteria relationships 11 7 Producing Meaningful Plots The CWViewer is a useful tool however it can also be cumbersome The CWViewer works best with an appropriate number of simulations but too many simulation runs can produce a plot that is meaningless It will also take time to generate the plot for larger numbers of simulation runs when there are a large number of alternatives or criteria Alternatively too few simulation runs may not provide enough data for a thorough analysis However when comparing data with no uncertainties this cannot be avoided The selection of Show Top Scoring Alternatives Show Mid Range Alternatives and Show Low Scoring Alternatives options can be used to show how all alternatives have performed during a simulation and may act as a guide for eliminating some alternatives when deciding on the final shortlist For example if you are analysing a set of alternatives and turn on the Show Top Scoring Alternatives option and only one alternative appears then UNCLASSIFIED 88 UNCLASSIFIED DSTO GD 0681 perhaps you have omitted an alternative you should be considering During the final stages of analysis this may not be the case i e a top scoring alterative may be omitted at the discretion of the decision maker 12 Weights Sensitivity Weights sensitivity analysis involves investigating the effect that changing weights will have on the ranking of altern
168. xcel or Access The data in the Data Grid can only be changed for your problem s Criterion elements All data other than the titles for the alternatives is generated by Decision Maker To help you identify your criterion data the Data Grid column header text is coloured using the same colour code used throughout Decision Maker Figure 50 UNCLASSIFIED 44 UNCLASSIFIED DSTO GD 0681 To copy or paste data between the Data Grid and any other application using your mouse select the data you want to copy or replace then right click on the selected area and choose the appropriate command When pasting data into the Data Grid ensure that the number of cells being pasted matches the number of cells in the Data Grid An error will occur if there is no match Data can only be added to Criterion type elements The data in all other elements is generated by Decision Maker excluding Alternative elements 6 2 2 Calculating Results in the Data Model Calculating results in Decision Maker may be performed at any time within the Data Model Do this by selecting the Project Calculate menu option as shown in Figure 55 File Edit View Project Tools Windows Advanced Analysis Help Set Current Problem H Add Mew Element Background Calculating E Calculate Founding Figure 55 Calculating results When the calculate function has been selected Decision Maker calculates the result for the entire problem If you have a l
169. xt to the mouse release the left mouse button and the column title will change to show the mapping selected This process is repeated until all the desired columns have been mapped and show another mapping pair In the example all columns have been mapped Element ID Element Name ee a ae Thi Cc hail wall Be Ignon d ltemative Tile This Column wall be the Title of the Altemative Chen mrm Alternative Description This Column wil be the Aematve Descaption 25 mem IC Minimise Total Cost poa laa A i Maximise Reliability Type A Spares Maximise Relability Type B Spares Minimise Supply Time Type Spares Minimise Supply Time Type B Spares Column 2 7300 Suppler 3 550 E200 Supplies 4 Supplier 5 rand Supplier 6 5935 7300 Supplier 7 Supplier 3 Supplier 3 Supplier 10 Supple 11 Supplier 12 Supple 13 Supplier 14 Supplier 15 supplier 6 172100 ee ic Figure 149 Alternative Input Mapping Step 1 UNCLASSIFIED 116 UNCLASSIFIED DSTO GD 0681 RI Ite native Inpu Mat pine _ n Nol Delmer a 1 C Comma Tab J Semicolon liner 3 This Column wil Be Ignored LI Auto Generate ANematre Tales Recommended i iD i This Column will be the Title of the Alter Element Title Base a ive Descripti This Column will be the Altemative Des Start Numbeung At 1 M P a Ri Me eh 1m Masimse Rebabiity Type E Spares Minimise Supply T
170. y Arrangement a MTBF hrs Criteria assigned to Objective Direction of Preference Maximise Cancel Finish Figure 28 Wizard example add criterion 2 to Maximise Performance UNCLASSIFIED 25 UNCLASSIFIED DSTO GD 0681 When the information has been entered click the Add Criterion button Next click the New Criterion button Enter the information presented in Table 9 into the Wizard It should appear as shown in Figure 29 Table 9 Wizard example Criterion 3 information Objective Maximise Reliability Criterion Title Maximise Reliability Type B Spares ChilaHion Description The lower limit of the 95 confidence interval of the Mean p Time Between Failures MTBF of Type B spares in hours hrs Unit of Measure MTBF Preferred Direction Add Criteria to Maximise Reliability Objective Title Objective 2 of 3 Criterion Title Maximise Reliability Objective Maximise Reliability Type B Spares SS nn SS Navigation Objective Description Criterion Description The best spare part supply arangement shall be able to supply The lower limit of the 95 confidence interval of the Mean Time ithe parts with the highest reliability gt Between Failures MTBF of Type B spares in hours hrs Crtenon Control Sub Objective Of Unit of Measure Enable Subjectivity o eS ae PA MTEF hrs SS Se el Criteria assigned to Objective Direction of Preference
171. ype a Type i ine Type oo Type grae This is the vertical crossing point used to populate the cross densities The number indicates the largest count in the stem plot In this example the 50 indicates that there are 50 crossing points at approximately 60 The vertical axis indicates the percentage at the crossing point of the two criteria plots These two plots indicate the criteria are proportional These two plots have a region of relatively directly proportional relationships and another region of somewhat inversely proportional relationships This is an example of an inversely proportional relationship between the two However since the count is 35 as opposed to 50 as indicated for the Select Best Supply Arrangement and Minimise Total Cost it can be concluded that the higher the number the greater the magnitude of the relationship Figure 97 Cross Density Plot example To reinforce the concept of the relationships indicated by the Cross Density Plots some ideal Cross Density Plot examples are shown in Figure 98 Across the top row in Figure 98 the Cross Density Plots show the ideal shapes representing inversely proportional independent and directly proportional elements respectively However the overall cross UNCLASSIFIED 76 UNCLASSIFIED DSTO GD 0681 densities for any two criteria can of course be a combination of the ideal types and examples are presented in the second row of Figure 98 Renting ar
172. zard Example Objective 1 information Field Text Objective Title Objective The best spare part supply arrangement shall be able to supply the Description parts with the highest performance in reliability and supply time Sub Objective Of Select Best Supply Arrangement Define Objective Number 1 Objective Title Maximise Perfornance Sircais Description The best spare part supply arrangement shall be able to supply the i with the highest perfomance in reliability and supply i time Sub Objective OF 75 select Best Supply Amangement Figure 19 Wizard example Define Objective Number 1 Next for Objective 2 enter the information presented in Table 5 into the Define Objective Number 2 dialog box The completed dialog box should appear as shown in Figure 20 When the information is entered click the OK button Finally for Objective 3 enter the information presented in Table 6 into the Define Objective Number 3 dialog box The completed dialog box should appear as shown in Figure 21 When the information is entered click the OK button Table 5 Wizard example Objective 2 information T Title Objective The best spare part supply arrangement shall be able to supply the Description parts with the highest reliability Sub Objective Of Table 6 Wizard Example Objective 3 information Objective Title Objective The best spare part supply arrange

Decision Maker User Guide - Defence Science and Technology

Contents

Download Pdf Manuals

Related Search

Related Contents