Just the Job

- MCDA for a personal decision

by Elizabeth Atherton

This article gives an overview of multi-criteria decision analysis (mcda) and the advantages of using it to structure decision problems. It includes a description of the use of mcda for a personal decision problem. The analysis was carried out informally on the job offers available to S. The decision problem was then modelled using Analytica, Bayesian updating and sensitivity analysis. The results modelled the decision maker’s opinions exactly. This resulted in him being able to negotiate much better working conditions, due to an increased understanding of his situation and what was important to him.

-oo0oo-

The aim of decision analysis is to help to improve decision makers’ understanding of their problem and to guide them towards the ‘best’ decision. mcda helps people to evaluate complicated situations in terms of their values (what they care about) and uncertainty (what they know or do not know). mcda can often give insight into how alternatives differ from one another and help people to identify new and improved alternatives.

Decisions are often difficult to make because:

They are complicated -
there are too many things to think about at once or there are different opinions which need to be considered.

There is uncertainty -
the decision maker does not know what will happen in the future or does not really know how they feel about parts of the decision.

Multiple objectives -
the decision maker is trying to achieve several goals at once and this may involve conflicting goals which need to be traded off.

Multiple perspectives -
if there are several decision makers they may have different views and opinions about the situation. Small changes in opinion may change the decision made.

mcda involves several evaluation processes which help to solve the problems identified above. The processes often overlap and may need to be performed several times to obtain a model that captures the decision makers’ thoughts and beliefs clearly enough for a decision to be made. This is because as the procedure is undertaken the decision makers’ understanding of the problem is increased and their opinions and thoughts develop. The order in which each of the processes is conducted, and how they are conducted is determined by the decision and its structure. There is no definite sequence which can be applied to all decisions, and even if decisions are classified to be of a certain ‘type’ the analysis will still depend on the specific nature of the decision and those involved in the process.

The steps of mcda

mcda involves the following steps:

Identify the stakeholders

These are people who have resources to influence how the decision is made or are affected by the decision. This can clarify where people may have different perspectives and who needs to be involved in the decision process. It can help to identify things that the decision makers want to achieve or avoid.

Specify objectives

The objectives are what the decision makers are trying to achieve or what they want to avoid. Specifying them helps the decision makers to determine what is important to them. It can identify where conflicts might arise and can help to clarify the complicated issues which are involved in the decision. It can also help the decision makers to create new alternatives to meet their aims.

Generate alternatives

Decision makers have to identify the alternatives available to them, and it also gives the chance to create new alternatives which they could implement. If there are lots of alternatives it may be difficult to analyse them without using a computer package, using a package can help to decrease the time needed to do the analysis.

Specify attributes

This stage involves identifying what is going to be used to evaluate how well an alternative achieves an objective. It breaks down the objectives into achievable and measurable goals, it can highlight conflicts in objectives and may identify alternatives which are not feasible. Examples of an attribute tree and influence diagrams are given in figures 1 to 4. Analytica has a graphical user interface which allows decision makers to see their problem pictorially, this can help them to understand their problem better. Analytica also allows decision makers to have multiple level influence diagrams which stops the screen from being over-crowded if there are lots of attributes. For example, figures 2 to 4 are all displayed on separate screens within Analytica. It also gives decision makers the ability to focus on one aspect of the problem at once, and can help to group the attributes and identify any conflicts.

Identify scenarios

Scenarios are states of the world that might occur after the decision is made, identifying them can help to show where there is uncertainty about the future and about the outcomes of a decision. This can help decision makers to identify areas that they need to investigate and can help to quantify the uncertainty.

Collect data and/or consult experts

The data necessary to make the decision may already be available, or the previous processes may have identified areas that need investigating. This process involves determining how well each alternative performs on each of the attributes.

Determine decision makers’ preferences

This involves identifying the relative importance of the objectives and therefore the attributes to the decision makers. It helps to identify what really matters in the decision and how much it matters. It can help to eliminate alternatives and can clarify what the decision makers are trying to achieve.

Rank alternatives

Determining which decision alternative is the ‘best’ can be difficult if there are lots of alternatives available, involving many attributes and if there is uncertainty surrounding the decision. In these cases computer packages can be used as they have the ability to analyse numerous alternatives against lots of attributes quickly. The uncertainty can be included via probability distributions and alternative scenarios of the future can be built into the model.

Sensitivity analysis

This process helps to determine how ‘robust’ the decision is, and whether it changes when the inputs are changed. It can be used to perform what if analysis of the future and can determine what affects the decision. Sensitivity analysis can identify if the uncertainty in the model affects the decision and can show where it might be useful to gather more information. The decision can be analysed from different perspectives to see if the decision changes under different view points. Any input values which the decision makers are unsure of can be modified to determine the effect of the changes. Again, computer packages can help, they allow users to consider many different scenarios quickly and easily and allow them to change values quickly. Analytica allows decision makers to change several input values at once and so the combined effects of changes can be analysed.

It may at first glance seem that mcda is very time consuming, but the rewards generally far out weigh the extra time required. Decision makers comment that they have a clearer understanding of the issues and feel more confident with their decisions. It is also easier to justify the decision made as all the thought processes involved have been clearly mapped out. The decision model itself is more likely to include all the important aspects and because the decision makers have thought about what might happen in the future, they are less likely to be surprised by an unexpected event. More detailed descriptions of how to perform decision analyses can be found in Edwards and Von Winterfeldt (1986) and Watson (1987).

To illustrate that mcda can help, even for personal decisions the following sections outline a decision problem that was analysed using mcda.

S’s decision problem

S is an actuary who had been offered two jobs. He already had a job and needed to decide which, if either, of the jobs he would like to take. I offered to help S to think about the issues involved in his decision and to help him to think about how important each of them were to him. At first S did not seem to realise that staying with his current firm was in itself a choice. After I had explained to him that this was an active decision and that we needed to analyse his current job in the same way as his job offers, we had three alternatives to consider. The act of defining his current job as a choice may have also decreased the status quo bias, as S had to evaluate it as a new possibility.

As this was a personal decision, S and his family were the only stakeholders, and he thought that his opinions reflected his wife’s and would take into consideration the needs of his one year old son. The analysis was carried out very informally at S’s house, which helped S to feel relaxed.

The whole process consisted of two meetings, approximately one week apart. The first meeting was the longest. At this meeting I helped S to identify the attributes, evaluate the alternatives with respect to each of the attributes and identify the relative weights of the attributes. The second meeting was used to update the model in light of new information which S had obtained and to perform sensitivity analysis on some of the attribute weights and values about which S was unsure.

At the beginning of the first meeting I explained carefully the aim of the meeting. I described the concept of multi-criteria decision analysis (MCDA) and how we would elicit S’s values and opinions. S found it easy to understand the ideas behind MCDA as he had a mathematical background. We discussed how S could include uncertainty in the model, and he mentioned that this might be useful. I outlined the stages of MCDA that we would go through, namely identifying attributes, evaluating the alternatives with respect to the attributes, evaluating scenarios of the future, weighting the attributes, optimising the decision, iterating and performing sensitivity analysis.

The whole process was carried out very informally and at a pace that S was happy with. At several points during the analysis I reviewed what we had achieved, to ensure that I had understood S clearly and to help him to clarify things in his mind.

The problem

The problem was such that the alternatives were already known and the main issue was helping S to think about the elements in the jobs that were important to him, and why. I started by getting S to think about all the attributes of the jobs. S already had lots of ideas about what each of the alternatives entailed and he came up with lots of attributes that he wanted to measure them on. Each attribute was written on a separate piece of paper and put on the floor. When S thought he had identified most of the attributes, we recapped, and S helped me to group the attributes into super-sections. This helped him to think about the broader picture and identify attributes which he had missed out. The new attributes were added to the original ones and put into their relevant groups. Figure 1 shows the attribute tree (Challenge was only added at the second meeting).

S’s salary increases when he passes exams and so Pay Rises one and two are controlled by the probability that he does, in fact, pass. S thinks that he has a 50:50 chance at each exam that he sits. At his present company he has a company pension, which will be lost if he moved to a new company. He also has an interest free loan with his present company, which would have to be repaid.

S believed that the pressure he was under was very important. This would be affected by the number of hours he had to work (and whether they were flexi-hours), the number of days holidays he had and the stress he would be put under at work. Some of these were clearly objective while some of them were subjective and depended on how S viewed the company and information he had gathered on them.

Figure 1: S’s Attribute T

Some attributes under Working Environment were very important while others were more aesthetic. Rank in the company, management and office structure were all related to S’s position in the company and how he would be managed. The idea of having a clean slate appealed to S as he thought it may be good to start again, without any prejudices. Variety of work was also important to S as he thought it would help his job prospects in the future and also give him more job satisfaction.

When the analysis was carried out S had just taken an exam. If he passed it he would have another two to take. The number of study days he is given depends on whether he has already taken and failed an exam, and the company will only pay fees for exams he has not already taken while working for them. If S changed companies it would be as if he was starting again, as the new company would give him a fresh start exam-wise and pay his fees and give him all his study days. At his current company he would have to pay his own fees for one of his exams and not have as many study days.

Firm Image covered many aesthetic attributes such as the office comfort and location, and though S did not think that these were very important he wanted to include them. Company Reputation, however, was important as this determines which clients the firm has and the prestige of working for them.

S thought that the Social aspect of work was important, as he enjoys playing football and interacting with the people he works with. As with most people, S does not like the idea of change and therefore included an attribute to reflect the stress of moving.

The Long Term attribute looked at S’s career prospects three years down the line, as this is when S thought he would probably review his situation. He thought that the outcome would be uncertain and would depend on how well he did in the company he was working for. He assigned probabilities to the possibility of doing well or badly and to the possibility of leaving the company at the end of three years. He also assigned a value to each of the good and bad outcomes, depending on which company he was at.

Twenty-nine attributes may seem like too many to use, but each of them were important to S and captured some aspect of his decision. As S was the only decision maker involved in the process, it was also relatively easy to obtain values for each of the attributes from him and perform the analysis. The computer package (Analytica) used to model the problem also makes it easy to model many attributes clearly, by having a network of influence diagrams which link together. The model had a top level influence diagram which had all the super-groups on it, and then a separate, linked influence diagram containing all the attributes in the super-groups. This made the computer presentation of the problem very easy and understandable and allowed S to look at one group of attributes at once. By including all of the attributes S mentioned, the model included all of the aspects of the decision that S thought were important. This made him feel comfortable with the model and that it covered everything it needed to.

Evaluating the alternatives

After all the attributes had been identified, I asked S to score each of the job alternatives on them. For some of the attributes, for example salary, it was easy to score the alternatives. The scores were then converted into a scale from 0-100 to give the relative scores of the alternatives for each attribute. For more subjective attributes, for example Company Reputation, S gave the values which he thought were appropriate to each option. The alternative with the highest score was given a value of 100, the alternative with the lowest score was given a value of 0. The other alternative was given a value in between 0 and 100 either in proportion to the original range, or where S thought that it should be valued. The financial attributes were measured over a three year period as S thought that he would review his job situation after that time.

Once each alternative had been scored on all the attributes, I asked S to rank the attributes. He arranged the pieces of paper so that the order reflected the relative importance of the attributes. When he had an order he was quite happy with we began to give weights to the attributes. We started by giving the salary attribute a weight of 1000. 1000 was used so that S could assign small differences between attribute weights. In the computer model the attribute weights were divided by 10, so that they were 100 or less. From this start point we then looked at other attributes which were measured in monetary terms. In this way I began with the easy comparisons to help to give S a feel of the procedures involved and to make him familiar with the idea of comparing attributes. When assigning a weight to an attribute the range of the scores of the alternatives was visible to S and I kept reminding him that the weight should be related to the range of the attribute. S found this easy to understand and could assign the weights without any difficulties.

Once all the monetary attributes had been weighted, we began to look at other attributes which were easy to convert into money terms, for example days holiday. S thought that he would be willing to work two days to have one day of holiday. Thus, a day of holiday was worth two days pay and so we converted it to its monetary equivalent and assigned the relevant attribute weight to it. We then looked at the more subjective attributes, for example Company Reputation. The analysis had therefore progressed from relatively straightforward comparisons to more complex ones.

The process of giving weights to the attributes resulted in some re-arranging of the rank of the attributes, as S realised that some of the attributes were worth less to him than he had originally thought. S found it relatively easy to assign weights to the non-monetary attributes, partly because the weights of the monetary attributes had already been worked out and the subjective attributes lay between these in the rank order. This gave S a ‘ball park’

figure for their weights, which made them easier to assign. Again I reminded S to concentrate on the ranges of the attributes as well as the ‘importance’.

Modelling the decision

Once all of the attributes had been weighted I went away and modelled the problem in Analytica and ran the decision problem. This gave an initial solution which I explained to S. In the meantime S thought again about the attribute weights and even gave the weights to his wife to think about to see if she thought he had been ‘misguided’. S himself felt confident with the weights because he said that they were based on logic and that they made sense to him.

The following diagrams show some of the package influence diagrams which were used to model the decision. Figure 2 shows the top level influence diagram from the package model. Each bold box links to another influence diagram, which contains all the attributes for that particular super-group. Figure 3 and Figure 4 are examples of the lower level attribute trees for the Working Environment and Study super-sections. The figures show how package allows users to create multiple, linked influence diagrams, which stop the diagrams from becoming too crowded.

Figure 2: Top level influence diagram

The diagrams show some of the package influence diagrams which were used to model the decision. Figure 2 shows the top level influence diagram from the package model. Each bold box links to another influence diagram, which contains all the attributes for that particular super-group. Figure 3 and Figure 4 are examples of the lower level attribute trees for the Working Environment and Study super-sections. The figures show how package allows users to create multiple, linked influence diagrams, which stop the diagrams from becoming too crowded.

Incorporating changes in the situation

After a few days S and I met up again, this time with the computer model. Between the first and second meeting S had received some extra information. This meant that some of the jobs scored differently on the attributes. S also introduced a new attribute (Challenge) to reflect the challenge that he felt he would face at work by working with people who were better than him. S had also indicated some weights which he wanted to do some sensitivity

Figure 3: Pay package influence diagram

Figure 4: Study influence diagram

analysis on. We adjusted the weights and looked at the results. The decision changed owing to the new information. Thus, S had performed a sort of Bayesian update without really realising what he had done. We found that the sensitivity analysis did not change the result of the model and he therefore felt comfortable that the model was stable and would not change even if his situation altered slightly.

S’s reflections

The new result agreed with the decision S had, himself, made. After the analysis we talked about the techniques I had used to help him in his decision problem and how he felt about them. S said that structuring the decision had helped him to think clearly about the issues involved. It had made it clear to him that some of the issues were less important than he had at first thought. By identifying the issues clearly he was able to then discuss them with friends and colleagues and analyse the jobs more clearly. This helped him to gather more information and to make his decision. S has a background in mathematics and was therefore very numerate. This helped him to understand the concepts of the decision modelling process and helped him to assign weights. He felt that the whole method was very logical and that the end decision model matched his opinions and thoughts, which made him confident in the decision made and content with the analysis.

Structuring the problem took the most time. The whole process took about four and a half hours. About half of this was spent structuring the decision and determining the attributes. The process showed that problem structuring was the vital stage in the decision making process. If this had not been done carefully then S would not have been satisfied with the results.

A final twist

After making his decision and beginning the process of accepting one of the new job offers S was taken to lunch by his current employers. They had heard that he was thinking of leaving and wanted to try and persuade him to stay. Having performed a detailed decision analysis process S knew exactly where the current job fell short of his other offer. In this way he was able to negotiate clearly the terms and conditions he would require if he was going to stay at the company. His employers agreed to meet his requirements and S has stayed with his original firm, but on much better terms.

For the interested reader

• Edwards W and Von Winterfeldt D (1986), Decision Analysis and Behavioural Research, Cambridge: Cambridge University Press.
• Watson S (1987), Decision Synthesis: The Principles and Practice of Decision Analysis, Cambridge: Cambridge University Press.
• Analytica is published in the UK by Roderick Manhattan Group Ltd, Manhattan House, Disraeli Road, London SW15 2DZ. http://www.rmg.co.uk

ELIZABETH ATHERTON is a PhD student at the School of Informatics at the University of Manchester. Her main research interest is intertemporal decision analysis. She has created an interactive web site which encourages people to think about their views of the long term issues involved in siting nuclear waste at http://www.informatics.man.ac.uk/da_expt.htm

First published to members of the Operational Research Society in OR Insight April - June 1998