Multicriteria Analysis, Multicriteria evidential reasoning

Multicriteria evidential reasoning

This article introduces concepts of evidential reasoning (ER). Here, ER is used to generate priorities for customers and their views, when these are expressed through customer satisfaction surveys. It serves as an introduction to the JORS paper Multicriteria evidential reasoning decision modelling and analysis—prioritizing voices of customer, authored by J-B Yang, D-L Xu, X Xie and A K Maddulapalli. The paper is available to ORSociety members in JORS (2011) 62, 1638–1654. The paper is freely quoted in this article, and its authors have approved this note.

Evidential reasoning approach

Data generated from different customer satisfaction surveys are often measured on different scales. Some of them are qualitative, such as ‘satisfied’, ‘neutral’ and ‘dissatisfied’ and some are quantitative. They may not be in a format that can be directly used to prioritize the voices of customer (in this field, expressed customer views are referred to as customer voices). So they need be transformed to a common scale in an appropriate format before they can serve as evidence to support voice prioritization. Such data transformation is domain specific and task dependent, so it requires expert judgments and domain specific knowledge.

In decision theory, the evidential reasoning approach (ER) is a generic evidence-based multi-criteria decision analysis (MCDA) approach for dealing with problems having both quantitative and qualitative criteria under various uncertainties including ignorance and randomness. It has been used to support various decision analysis, assessment and evaluation activities, such as environmental impact assessments. The evidential reasoning approach was developed on the basis of decision theory, in particular utility theory, artificial intelligence, statistical analysis and computer technology. It uses a belief structure to model an assessment with uncertainty, a belief decision matrix to represent an MCDA problem under uncertainty, evidential reasoning algorithms to aggregate criteria for generating distributed assessments, and the concepts of the belief and plausibility functions to generate a utility interval for measuring the degree of ignorance. A conventional decision matrix used for modeling an MCDA problem is a special case of a belief decision matrix.

Priority levels

One of the interesting issues raised in the paper is whether the marks submitted by different customers against a characteristic of a product should be averaged, when the data set is analysed. The authors argue that a characteristic which attracts a wide-range of views may be more important to a firm than one which attracts more middle marks.


In the paper, a new methodology is investigated to support the prioritization of the voices of customers through various customer satisfaction surveys. This new methodology consists of two key components: an innovative evidence-driven decision modelling framework for representing and transforming large amounts of data sets, and a generic reasoning-based decision support process for aggregating evidence to prioritize the voices of customer on the basis of the Evidential Reasoning (ER) approach. Methods and frameworks for data collection and representation via multiple customer satisfaction surveys were examined first and the distinctive features of quantitative and qualitative survey data are analysed. Several novel yet natural and pragmatic rule-based functions are then proposed to transform survey data systematically and consistently from different measurement scales to a common scale, with the original features and profiles of the data preserved in the transformation process. These new transformation functions are proposed to mimic expert judgement processes and designed to be sufficiently flexible and rigorous so that expert judgements and domain specific knowledge can be taken into account naturally, systematically and consistently in the transformation process. The ER approach is used for synthesizing quantitative and qualitative data under uncertainty that can be caused due to missing data and ambiguous survey questions. A new generic method is also proposed for ranking the voices of customer based on qualitative measurement scales without having to quantify assessment grades to fixed numerical values. A case study is examined using an Intelligent Decision System (ES software) to illustrate the application of the decision modelling framework and decision support process for prioritizing the voices of customers for a world-leading car manufacturer.

The maths made simple

Suppose in a survey, a customer is asked to provide marks for each of a set of characteristics. There results an array of views Sq, where q=1 to q' are the various characteristics. The numbers Sq are numerical values, representing the “marks” the customer gives for each characteristic. In some cases the customer is asked to mark the characteristic only qualitatively, by “ticking” an appropriate quality. For example, they may be asked to rate the characteristic good, neutral or bad. In this case three sub-qualities are introduced, q(1), q(2) and q(3), where-

q(1) =1 if the characteristic is good, =0 otherwise
q(2) = 1 if neutral, =0 otherwise
q(3) = 1 if poor, =0 otherwise.

Thus Sq(i) is an array of marks for each characteristic q. For a full survey of customers c=1 to c', the resulting data array is Sc,q(i). This can be generalised by adding other dimensions, x, y,z etc, representing other factors (e.g. time, location or characteristics of the customers). This will result in a more complicated array of data points Sc,q(i),x,y... .

These data can be analysed in many ways, using the full range of statistical techniques. The authors propose two extensions of this. Firstly, allowing the customer to tick more than one quality variable q(i), and/or by allowing values in the q(i) other than 1 or 0. This effectively introduces belief distribution concepts. This looks a little like fuzzy methods, but is in fact an extension of probability.

The other extension is by applying the evidence based reasoning process for information aggregation, an extension to Bayesian Reasoning, not to Sc,q(i),x,y, but to W c,q(i),x,y Sc,q(i),x,y, where the W are a set of weights, that can either be externally set, or defined as a function of some other variables of the model.

These changes enable a vast extension of normal methods in handling mixed arrays of uncertainties.

Latest News

March 2023

SW23 – Registration closes 17th March!

The OR Society’s Simulation Workshop (SW23) brings together our community of experts working in the field of modelling and simulation. Hosted at the National Oceanography Centre, Southampton, the conference runs from Monday 27 - Wednesday 29 March 2023 and will provide an opportunity for practitioners and academics to exchange ideas on the current and future state-of-the-art in modelling and simulation.

Read More

December 2022

AI may help protect firefighters

New research suggests that artificial intelligence (AI) may help predict 'flashovers’, a nightmare scenario for firefighters in which all combustible material in a room suddenly ignites, due to extremely high local temperatures.

Read More

November 2022

Machine learning, oil prices and cryptocurrencies

Springer has published a new Open Access article which looks at the role of cryptocurrencies and the US dollar in predicting oil prices pre and during COVID-19 pandemic.

Read More