Just the Job
- MCDA for a personal decision
by Elizabeth Atherton
This article gives an overview of multi-criteria decision analysis
(mcda) and the advantages of using it to structure decision problems. It
includes a description of the use of mcda for a personal decision problem.
The analysis was carried out informally on the job offers available to S.
The decision problem was then modelled using Analytica, Bayesian updating
and sensitivity analysis. The results modelled the decision maker’s opinions
exactly. This resulted in him being able to negotiate much better working
conditions, due to an increased understanding of his situation and what was
important to him.
The aim of decision analysis is to help to improve decision makers’ understanding
of their problem and to guide them towards the ‘best’ decision. mcda helps
people to evaluate complicated situations in terms of their values (what they
care about) and uncertainty (what they know or do not know). mcda can often
give insight into how alternatives differ from one another and help people
to identify new and improved alternatives.
Decisions are often difficult to make because:
They are complicated -
there are too many things to think about at once or there are different opinions
which need to be considered.
There is uncertainty -
the decision maker does not know what will happen in the future or does not
really know how they feel about parts of the decision.
Multiple objectives -
the decision maker is trying to achieve several goals at once and this may
involve conflicting goals which need to be traded off.
Multiple perspectives -
if there are several decision makers they may have different views and opinions
about the situation. Small changes in opinion may change the decision made.
mcda involves several evaluation processes which help to solve the problems
identified above. The processes often overlap and may need to be performed
several times to obtain a model that captures the decision makers’ thoughts
and beliefs clearly enough for a decision to be made. This is because as the
procedure is undertaken the decision makers’ understanding of the problem is
increased and their opinions and thoughts develop. The order in which each
of the processes is conducted, and how they are conducted is determined by
the decision and its structure. There is no definite sequence which can be
applied to all decisions, and even if decisions are classified to be of a certain
‘type’ the analysis will still depend on the specific nature of the decision
and those involved in the process.
The steps of mcda
mcda involves the following steps:
Identify the stakeholders
These are people who have resources to influence how the decision is made
or are affected by the decision. This can clarify where people may have different
perspectives and who needs to be involved in the decision process. It can help
to identify things that the decision makers want to achieve or avoid.
The objectives are what the decision makers are trying to achieve or what
they want to avoid. Specifying them helps the decision makers to determine
what is important to them. It can identify where conflicts might arise and
can help to clarify the complicated issues which are involved in the decision.
It can also help the decision makers to create new alternatives to meet their
Decision makers have to identify the alternatives available to them, and it
also gives the chance to create new alternatives which they could implement.
If there are lots of alternatives it may be difficult to analyse them without
using a computer package, using a package can help to decrease the time needed
to do the analysis.
This stage involves identifying what is going to be used to evaluate how well
an alternative achieves an objective. It breaks down the objectives into achievable
and measurable goals, it can highlight conflicts in objectives and may identify
alternatives which are not feasible. Examples of an attribute tree and influence
diagrams are given in figures 1 to 4. Analytica has a graphical user interface
which allows decision makers to see their problem pictorially, this can help
them to understand their problem better. Analytica also allows decision makers
to have multiple level influence diagrams which stops the screen from being
over-crowded if there are lots of attributes. For example, figures 2 to 4 are
all displayed on separate screens within Analytica. It also gives decision
makers the ability to focus on one aspect of the problem at once, and can help
to group the attributes and identify any conflicts.
Scenarios are states of the world that might occur after the decision is made,
identifying them can help to show where there is uncertainty about the future
and about the outcomes of a decision. This can help decision makers to identify
areas that they need to investigate and can help to quantify the uncertainty.
Collect data and/or consult experts
The data necessary to make the decision may already be available, or the previous
processes may have identified areas that need investigating. This process involves
determining how well each alternative performs on each of the attributes.
Determine decision makers’ preferences
This involves identifying the relative importance of the objectives and therefore
the attributes to the decision makers. It helps to identify what really matters
in the decision and how much it matters. It can help to eliminate alternatives
and can clarify what the decision makers are trying to achieve.
Determining which decision alternative is the ‘best’ can be difficult if there
are lots of alternatives available, involving many attributes and if there
is uncertainty surrounding the decision. In these cases computer packages can
be used as they have the ability to analyse numerous alternatives against lots
of attributes quickly. The uncertainty can be included via probability distributions
and alternative scenarios of the future can be built into the model.
This process helps to determine how ‘robust’ the decision is, and whether
it changes when the inputs are changed. It can be used to perform what
if analysis of the future and can determine what affects the decision.
Sensitivity analysis can identify if the uncertainty in the model affects the
decision and can show where it might be useful to gather more information.
The decision can be analysed from different perspectives to see if the decision
changes under different view points. Any input values which the decision makers
are unsure of can be modified to determine the effect of the changes. Again,
computer packages can help, they allow users to consider many different scenarios
quickly and easily and allow them to change values quickly. Analytica allows
decision makers to change several input values at once and so the combined
effects of changes can be analysed.
It may at first glance seem that mcda is very time consuming, but the rewards
generally far out weigh the extra time required. Decision makers comment that
they have a clearer understanding of the issues and feel more confident with
their decisions. It is also easier to justify the decision made as all the
thought processes involved have been clearly mapped out. The decision model
itself is more likely to include all the important aspects and because the
decision makers have thought about what might happen in the future, they are
less likely to be surprised by an unexpected event. More detailed descriptions
of how to perform decision analyses can be found in Edwards and Von Winterfeldt
(1986) and Watson (1987).
To illustrate that mcda can help, even for personal decisions the following
sections outline a decision problem that was analysed using mcda.
S’s decision problem
S is an actuary who had been offered two jobs. He already had a job and needed
to decide which, if either, of the jobs he would like to take. I offered to
help S to think about the issues involved in his decision and to help him to
think about how important each of them were to him. At first S did not seem
to realise that staying with his current firm was in itself a choice. After
I had explained to him that this was an active decision and that we needed
to analyse his current job in the same way as his job offers, we had three
alternatives to consider. The act of defining his current job as a choice may
have also decreased the status quo bias, as S had to evaluate it as a new possibility.
As this was a personal decision, S and his family were the only stakeholders,
and he thought that his opinions reflected his wife’s and would take into consideration
the needs of his one year old son. The analysis was carried out very informally
at S’s house, which helped S to feel relaxed.
The whole process consisted of two meetings, approximately one week apart.
The first meeting was the longest. At this meeting I helped S to identify the
attributes, evaluate the alternatives with respect to each of the attributes
and identify the relative weights of the attributes. The second meeting was
used to update the model in light of new information which S had obtained and
to perform sensitivity analysis on some of the attribute weights and values
about which S was unsure.
At the beginning of the first meeting I explained carefully the aim of the
meeting. I described the concept of multi-criteria decision analysis (MCDA)
and how we would elicit S’s values and opinions. S found it easy to understand
the ideas behind MCDA as he had a mathematical background. We discussed how
S could include uncertainty in the model, and he mentioned that this might
be useful. I outlined the stages of MCDA that we would go through, namely identifying
attributes, evaluating the alternatives with respect to the attributes, evaluating
scenarios of the future, weighting the attributes, optimising the decision,
iterating and performing sensitivity analysis.
The whole process was carried out very informally and at a pace that S was
happy with. At several points during the analysis I reviewed what we had achieved,
to ensure that I had understood S clearly and to help him to clarify things
in his mind.
The problem was such that the alternatives were already known and the main
issue was helping S to think about the elements in the jobs that were important
to him, and why. I started by getting S to think about all the attributes of
the jobs. S already had lots of ideas about what each of the alternatives entailed
and he came up with lots of attributes that he wanted to measure them on. Each
attribute was written on a separate piece of paper and put on the floor. When
S thought he had identified most of the attributes, we recapped, and S helped
me to group the attributes into super-sections. This helped him to think about
the broader picture and identify attributes which he had missed out. The new
attributes were added to the original ones and put into their relevant groups.
Figure 1 shows the attribute tree (Challenge was only added at the second meeting).
S’s salary increases when he passes exams and so Pay Rises one and two are
controlled by the probability that he does, in fact, pass. S thinks that he
has a 50:50 chance at each exam that he sits. At his present company he has
a company pension, which will be lost if he moved to a new company. He also
has an interest free loan with his present company, which would have to be
S believed that the pressure he was under was very important. This would be
affected by the number of hours he had to work (and whether they were flexi-hours),
the number of days holidays he had and the stress he would be put under at
work. Some of these were clearly objective while some of them were subjective
and depended on how S viewed the company and information he had gathered on
Figure 1: S’s Attribute T
Some attributes under Working Environment were very important while others
were more aesthetic. Rank in the company, management and office structure were
all related to S’s position in the company and how he would be managed. The
idea of having a clean slate appealed to S as he thought it may be good to
start again, without any prejudices. Variety of work was also important to
S as he thought it would help his job prospects in the future and also give
him more job satisfaction.
When the analysis was carried out S had just taken an exam. If he passed it
he would have another two to take. The number of study days he is given depends
on whether he has already taken and failed an exam, and the company will only
pay fees for exams he has not already taken while working for them. If S changed
companies it would be as if he was starting again, as the new company would
give him a fresh start exam-wise and pay his fees and give him all his study
days. At his current company he would have to pay his own fees for one of his
exams and not have as many study days.
Firm Image covered many aesthetic attributes such as the office comfort and
location, and though S did not think that these were very important he wanted
to include them. Company Reputation, however, was important as this determines
which clients the firm has and the prestige of working for them.
S thought that the Social aspect of work was important, as he enjoys playing
football and interacting with the people he works with. As with most people,
S does not like the idea of change and therefore included an attribute to reflect
the stress of moving.
The Long Term attribute looked at S’s career prospects three years down the
line, as this is when S thought he would probably review his situation. He
thought that the outcome would be uncertain and would depend on how well he
did in the company he was working for. He assigned probabilities to the possibility
of doing well or badly and to the possibility of leaving the company at the
end of three years. He also assigned a value to each of the good and bad outcomes,
depending on which company he was at.
Twenty-nine attributes may seem like too many to use, but each of them were
important to S and captured some aspect of his decision. As S was the only
decision maker involved in the process, it was also relatively easy to obtain
values for each of the attributes from him and perform the analysis. The computer
package (Analytica) used to model the problem also makes it easy to model many
attributes clearly, by having a network of influence diagrams which link together.
The model had a top level influence diagram which had all the super-groups
on it, and then a separate, linked influence diagram containing all the attributes
in the super-groups. This made the computer presentation of the problem very
easy and understandable and allowed S to look at one group of attributes at
once. By including all of the attributes S mentioned, the model included all
of the aspects of the decision that S thought were important. This made him
feel comfortable with the model and that it covered everything it needed to.
Evaluating the alternatives
After all the attributes had been identified, I asked S to score each of the
job alternatives on them. For some of the attributes, for example salary, it
was easy to score the alternatives. The scores were then converted into a scale
from 0-100 to give the relative scores of the alternatives for each attribute.
For more subjective attributes, for example Company Reputation, S gave the
values which he thought were appropriate to each option. The alternative with
the highest score was given a value of 100, the alternative with the lowest
score was given a value of 0. The other alternative was given a value in between
0 and 100 either in proportion to the original range, or where S thought that
it should be valued. The financial attributes were measured over a three year
period as S thought that he would review his job situation after that time.
Once each alternative had been scored on all the attributes, I asked S to
rank the attributes. He arranged the pieces of paper so that the order reflected
the relative importance of the attributes. When he had an order he was quite
happy with we began to give weights to the attributes. We started by giving
the salary attribute a weight of 1000. 1000 was used so that S could assign
small differences between attribute weights. In the computer model the attribute
weights were divided by 10, so that they were 100 or less. From this start
point we then looked at other attributes which were measured in monetary terms.
In this way I began with the easy comparisons to help to give S a feel of the
procedures involved and to make him familiar with the idea of comparing attributes.
When assigning a weight to an attribute the range of the scores of the alternatives
was visible to S and I kept reminding him that the weight should be related
to the range of the attribute. S found this easy to understand and could assign
the weights without any difficulties.
Once all the monetary attributes had been weighted, we began to look at other
attributes which were easy to convert into money terms, for example days holiday.
S thought that he would be willing to work two days to have one day of holiday.
Thus, a day of holiday was worth two days pay and so we converted it to its
monetary equivalent and assigned the relevant attribute weight to it. We then
looked at the more subjective attributes, for example Company Reputation. The
analysis had therefore progressed from relatively straightforward comparisons
to more complex ones.
The process of giving weights to the attributes resulted in some re-arranging
of the rank of the attributes, as S realised that some of the attributes were
worth less to him than he had originally thought. S found it relatively easy
to assign weights to the non-monetary attributes, partly because the weights
of the monetary attributes had already been worked out and the subjective attributes
lay between these in the rank order. This gave S a ‘ball park’
figure for their weights, which made them easier to assign. Again I reminded
S to concentrate on the ranges of the attributes as well as the ‘importance’.
Modelling the decision
Once all of the attributes had been weighted I went away and modelled the
problem in Analytica and ran the decision problem. This gave an initial solution
which I explained to S. In the meantime S thought again about the attribute
weights and even gave the weights to his wife to think about to see if she
thought he had been ‘misguided’. S himself felt confident with the weights
because he said that they were based on logic and that they made sense to him.
The following diagrams show some of the package influence diagrams which were
used to model the decision. Figure 2 shows the top level influence diagram
from the package model. Each bold box links to another influence diagram, which
contains all the attributes for that particular super-group. Figure 3 and Figure
4 are examples of the lower level attribute trees for the Working Environment
and Study super-sections. The figures show how package allows users to create
multiple, linked influence diagrams, which stop the diagrams from becoming
Figure 2: Top level influence diagram
The diagrams show some of the package influence diagrams which were used to
model the decision. Figure 2 shows the top level influence diagram from the
package model. Each bold box links to another influence diagram, which contains
all the attributes for that particular super-group. Figure 3 and Figure 4 are
examples of the lower level attribute trees for the Working Environment and
Study super-sections. The figures show how package allows users to create multiple,
linked influence diagrams, which stop the diagrams from becoming too crowded.
Incorporating changes in the situation
After a few days S and I met up again, this time with the computer model.
Between the first and second meeting S had received some extra information.
This meant that some of the jobs scored differently on the attributes. S also
introduced a new attribute (Challenge) to reflect the challenge that he felt
he would face at work by working with people who were better than him. S had
also indicated some weights which he wanted to do some sensitivity
Figure 3: Pay package influence diagram
Figure 4: Study influence diagram
analysis on. We adjusted the weights and looked at the results. The decision
changed owing to the new information. Thus, S had performed a sort of Bayesian
update without really realising what he had done. We found that the sensitivity
analysis did not change the result of the model and he therefore felt comfortable
that the model was stable and would not change even if his situation altered
The new result agreed with the decision S had, himself, made. After the analysis
we talked about the techniques I had used to help him in his decision problem
and how he felt about them. S said that structuring the decision had helped
him to think clearly about the issues involved. It had made it clear to him
that some of the issues were less important than he had at first thought. By
identifying the issues clearly he was able to then discuss them with friends
and colleagues and analyse the jobs more clearly. This helped him to gather
more information and to make his decision. S has a background in mathematics
and was therefore very numerate. This helped him to understand the concepts
of the decision modelling process and helped him to assign weights. He felt
that the whole method was very logical and that the end decision model matched
his opinions and thoughts, which made him confident in the decision made and
content with the analysis.
Structuring the problem took the most time. The whole process took about four
and a half hours. About half of this was spent structuring the decision and
determining the attributes. The process showed that problem structuring was
the vital stage in the decision making process. If this had not been done carefully
then S would not have been satisfied with the results.
A final twist
After making his decision and beginning the process of accepting one of the
new job offers S was taken to lunch by his current employers. They had heard
that he was thinking of leaving and wanted to try and persuade him to stay.
Having performed a detailed decision analysis process S knew exactly where
the current job fell short of his other offer. In this way he was able to negotiate
clearly the terms and conditions he would require if he was going to stay at
the company. His employers agreed to meet his requirements and S has stayed
with his original firm, but on much better terms.
For the interested reader
- Edwards W and Von Winterfeldt D (1986), Decision Analysis and Behavioural
Research, Cambridge: Cambridge University Press.
- Watson S (1987), Decision Synthesis: The Principles and Practice
of Decision Analysis, Cambridge: Cambridge University Press.
- Analytica is published in the UK by Roderick Manhattan Group Ltd, Manhattan
House, Disraeli Road, London SW15 2DZ.
ELIZABETH ATHERTON is a PhD student at the
School of Informatics at the University of Manchester. Her main research interest
is intertemporal decision analysis. She has created an interactive web site
which encourages people to think about their views of the long term issues
involved in siting nuclear waste at http://www.informatics.man.ac.uk/da_expt.htm
First published to members of the Operational Research Society in OR
Insight April - June 1998