This note is a shortened and simplified version of an article in the Journal of the Operational Research Society (2010) 61, 1459–1470, written by M Tavana and M A Sodenkamp. For full details, including contact details and references, please see the full article.
Introduction
A large body of scoring, economic and portfolio methods have evolved over the last several decades to assist decision makers (DMs) in project evaluation. Scoring methods use algebraic formulas to produce an overall score for each project. Economic methods use financial models to calculate the monetary pay-off of each project. Portfolio methods evaluate the entire set of projects to identify the most attractive subset. Decision analysis methods compare various projects according to their expected value. Finally, simulation, a more specific decision analysis method, uses random numbers and simulation to generate a large number of problems and picks the best outcome.
Project evaluation problems are group MCDA (multiple-criteria decision analysis) problems that embrace both qualitative and quantitative criteria. MCDA methods provide a structured framework for information exchange among the group members and thus reducing the unstructured nature of the problem. The obvious obstacle when multiple persons are involved in a group decision problem is the fact that each group member has his/her own perception of the problem that accordingly affects the decision outcome. MCDA frameworks permit group members to explore their value system from multiple viewpoints and modify their perceptions by obtaining knowledge of the other group members’ preference structure and beliefs. A number of decision methodologies in the group decision-making context have been presented in the MCDA literature.
DMs often use verbal expressions and linguistic variables for subjective judgments, which lead to ambiguity in human decision making.
- DMs often provide imprecise or vague information due to lack of expertise, unavailability of data, or time constraint.
- Meaningful and robust aggregation of subjective and objective judgments causes problems during the evaluation process.
MCDA problems involve the ranking of a finite set of alternatives in terms of a finite number of conflicting decision criteria. More often, decision criteria can be grouped into two contradictory categories, called the ‘opportunities’ and the ‘threats’. Alternatively, opportunities may be called ‘benefits’ or ‘returns’ and threats may be called ‘costs’ or ‘risks’.
Software
Fuzzy Euclid is a MCDA model that captures the DMs’ beliefs through a series of intuitive and analytical methods such as the analytic hierarchy process (AHP) and subjective probabilities. The concept of fuzzy sets is often used to reflect the inherent subjectivity and imprecision involved in the evaluation process. Fuzzy numbers have been widely used in decision problems where the information available is subjective or imprecise. We use a defuzzification method to obtain crisp values from the subjective judgments provided by multiple DMs.
A decision-making committee of three division chiefs at the Shuttle Project Engineering Office is responsible for the evaluation and selection of advanced-technology projects at the Kennedy Space Center (KSC). The proposed projects are independent and non-additive requests for engineering changes to the space shuttle that are generally initiated by the contractors or different divisions within KSC. Fuzzy Euclid, developed at KSC, considers the importance of each project relative to the longevity of the space-shuttle program and enhances the committee's decision quality and confidence.
Mathematical model and procedure
The evaluation process begins with a preliminary review of a number of advanced-technology projects submitted to KSC for funding. The Shuttle Project Engineering Office selects divisions to participate in the evaluation process. Division chiefs, called DMs in this study, are responsible for the evaluation of the advanced-technology projects. Initially, DMs use AHP independently to weight their importance of the participating divisions. Next, the DMs collectively decide what criteria (opportunities and threats) should be considered in the evaluation process. Once the DMs agree on a set of opportunities and threats, they use AHP independently to weigh their importance of the opportunities and threats. Then, the DMs consult with the experts and specialists within their divisions to assign probabilities of occurrence to the opportunities and threats. Next, a defuzzification method is used to obtain crisp values from the subjective judgments and estimates provided by the DMs for the projects. These crisp values are then synthesized in an MCDA model to produce an overall performance score for each of the M projects under consideration.
Traditionally, AHP is used to estimate the relative importance weight of the criteria and the relative performance of the alternatives in MCDA problems. However, in Fuzzy Euclid, we only use AHP to determine the importance weight of the opportunities and threats. Instead of using AHP to find the relative performance of the alternatives (projects) on each criterion, we use subjective probabilities of occurrence to capture these scores.
Fuzzy Euclid is a normative MCDA model with multiple factors representing different dimensions from which the projects are viewed. When the number of factors is large, typically more than a dozen, they may be arranged hierarchically. Fuzzy Euclidassumes a hierarchical structure by initially identifying the divisions at NASA who are responsible for the evaluation of the advanced technology projects. Following this identification, each division is asked to identify the relevant factors in their decision-making process and group them into opportunities and threats. This hierarchical structure allows for a systematic grouping of decision factors in large problems. The classification of different factors is undoubtedly the most delicate part of the problem formulation because all different aspects of the problem must be represented while avoiding redundancies.
A series of weights and probabilities are used in Fuzzy Euclid to estimate the importance weight of the selection criteria and their probabilities of occurrence for each alternative. Decision-making theory generally deals with three types of uncertainty: stochastic uncertainty, subjective uncertainty and informational uncertainty. Stochastic uncertainty is treated by probability theory and subjective and informational uncertainties are the target of fuzzy set and fuzzy logic theory.
Although fuzzy logic and probability theory are similar, they are not identical. Probability refers to the likelihood that something is true and fuzzy logic establishes the degree to which something is true. Probability is not a special case of fuzziness, but leads us to consider probability of fuzzy events.
Defuzzification is the translation of fuzzy values into numerical, scalar, and crisp representations. The process of condensing the information captured by fuzzy sets into numerical values is similar to that of transformation of uncertainty-based concepts into certainty-based concepts. Intuitively speaking, the defuzzification process in Fuzzy Euclid is similar to an averaging procedure. Many defuzzification techniques have been proposed in the literature. The most commonly used method is the Center of Gravity (COG). Other methods include: random choice of maximum, first of maximum, last of maximum, middle of maximum, mean of maxima, basic defuzzification distributions, generalized level set defuzzification, indexed centre of gravity, semi-linear defuzzification, fuzzy mean, weighted fuzzy mean, quality method, extended quality method, center of area, extended centre of area, constraint decision defuzzification, and fuzzy clustering defuzzification.
Once the model is developed, sensitivity analyses can be performed to determine the impact on the ranking of projects for changes in various model assumptions. Some sensitivity analyses that are usually of interest are on the weights and probabilities of occurrence. The weights representing the relative importance of the divisions, opportunities, and threats are occasionally a point for discussion among the various DMs. In addition, probabilities of occurrence that reflect the degree of belief that an uncertain event will occur are sometimes a matter of contention.
A case study
In the fullpaper we illustrate the application of Fuzzy Euclid to a disguised actual case study at NASA—KSC. In this case, the DMs are a committee of three division chiefs for Safety, Reliability, and Operations considering requests for funding 10 advanced technology projects. The following are the projects and anticipated expenditures: Hubble ($1,778,000), Photovoltaic ($1,908,000), Airlock ($1,515,000), Babaloon ($1,949,000), Planet-Finder ($1,266,000), Nebula ($1,348,000), Solar ($1,176,000), Truss ($1,347,000), Centrifuge ($1,790,000) and Tether ($961,000). A budget of $15,038,000 is needed to fund all 10 projects. However, budgetary constraints limit spending to $10 million.
The process began with an initial meeting of the three DMs. They used Expert Choice to weight the importance of each division. Next, the DMs worked with their divisions to identify a set of opportunities and threats to be used in the evaluation process. Each division held separate meetings and developed their set of opportunities and threats. Then, they used Expert Choice to weight these opportunities and threats.
The Safety division identified seven opportunities and seven threats, the Reliability division identified eight opportunities and five threats, and the Operations division identified 10 opportunities and seven threats to be included in the evaluation process.
Conclusions
Global competition and the rapid development of computer and information technology have made strategic decision making more complex than ever. Fuzzy Euclid is a MCDA model that uses AHP, subjective probabilities, defuzzification, entropy, and the theory of displaced ideal to reduce these complexities by decomposing the project evaluation process into manageable steps. This decomposition is achieved without overly simplifying the evaluation process.
Fuzzy Euclid is not intended to replace human judgment in project evaluation and selection at KSC. In fact, human judgment is the core input in the process. Fuzzy Euclid helps the DMs to think systematically about complex project selection problems and improves the quality of their decisions.
Full version first published to members of the Operational Research Society in JORS 2010
