by Emmanuel Thanassoulis and Gary Simpson
Data Envelopment Analysis can be used to set targets for individual attainments that are demanding but realistic. The targets are based on the best attainment levels realised by other individuals, after allowing for factors beyond an individual’s control. The approach will be illustrated using the example of school children but it has wider applicability in monitoring and improving the performance of individuals in any context where large numbers of individuals are engaged in a given activity but under varying conditions. For example salespersons may be set targets which reflect the varying sales environments in which they operate.
In many areas of life we attempt to meet targets or to set targets; whether as school children attempting to meet the expectations of parents and teachers, or as bank tellers attempting to reach a target level of transactions. In this paper we will describe a Data Envelopment Analysis (DEA) based technique introduced in Thanassoulis (forthcoming) to set attainment targets for individual school children, generalising it to other contexts. Targets at individual person level can play a very important role in helping individuals improve performance when the targets are suitably set to be challenging but not demotivating. For example, the Department for Education and Employment (DfEE 1996, p6) notes that effective target setting ‘helps schools to articulate clearly what is being aimed at, whether by schools as a whole, or by groups or individuals within the school’. Often our success in meeting targets depends not only on our own effort but also on factors that are beyond our control such as our aptitude for the task. The method discussed here endeavours to set realistic targets that are not unduly influenced by very exceptional performance by a small number of individuals or by random noise. Our belief is that this methodology can be readily applied in many contexts where the targets set for individuals should take into account extraneous factors and must be realistic to have the desired motivational effect.
Outline of the approach
Let us take a situation where we wish to set targets for each individual within a set, all individuals performing the same task. The targets will refer to levels to be attained on given variables over a specified period of time. For example, in the case of sales persons the targets might be in terms of the volumes of sales achieved while in the case of a tax officer the target might be in terms of the number of tax cases assessed.
In order to deploy the method we need recent data at individual person level on two sets of variables. The first set is that of attainment variables on which future targets are to be set. The second set is that of control variables. The latter relate to factors beyond the control of the individual but which nevertheless impact his/her ability to attain the targets set. For example, in the case of sales persons the attainment variables might be the volume and the value of sales of each type of product, while the control variables might be the corresponding total volume and value of sales in the geographical region covered by the sales person, the number of competitors s/he faced and so on.
The identification of the control and attainment variables is crucial if the targets are to have the desired motivational effect. The attainment variables must capture the essential aims of the activities of the individuals concerned, while the control variables must make allowances for key causal but uncontrollable factors influencing attainments by individuals. We do not wish to allow for causal but controllable factors in setting targets as we wish to eliminate controllable causes which might otherwise lead to targets which do not reflect the full potential for achievement. The approach assumes that attainment variables are continuouswhile control variables may be a mixture of continuous and categorical variables. Once attainment and control variables are decided upon, the approach proceeds in three steps as follows:
- Partition the individuals into groups which share a common context;
- Allow for the possibility of random noise inflating the estimated attainment targets;
- Estimate attainment targets after having allowed for random noise.
These steps are carried out as follows:
Partition the individuals into groups which share a common context
This step is only used if not all the individuals are strictly comparable on attainment because they belong to different categories. Whether this is or not the case will be borne out by the existence or otherwise of categorical control variables. For example if we believe that all else being equal a salesperson covering a rural area would attain different sales levels on average than one covering an urban area then our control variables would include the categorical variables urban and rural market. In such a case the first step in the target setting procedure would be to separate salespersons into those covering rural and those covering urban areas so that targets can be estimated separately for the individuals of each group.
Allow for the possibility of random noise inflating the estimated attainment targets
In principle, once we have partitioned individuals into groups sharing a common context, we can proceed to set targets for each individual in a group at the best estimated feasible attainment levels which correspond to his/her values on the non categorical control variable(s). However, before estimating targets in this manner we must allow for the fact that the observed attainment levels we are using are subject to random noise. Such noise might have influenced observed attainment levels either upwards or downwards. As we set targets at the best observed attainment levels, we are only concerned not to permit random noise to lead to overestimated feasible attainment levels as this would be demotivating to individuals. We can allow for the possibility of an upward bias in observed attainment levels by discounting what we deem as exceptional attainments. This we can do using an index of attainment exceptionality defined as follows:
An individual’s index of attainment exceptionality is the percentage his/her attainment level(s) are of what we would expect for his/her non categorical control variable levels within the ‘incumbent’ group.
The incumbent group is initially the entire group, but subsequently it consists of the individuals left in the group after those deemed to have too high an exceptionality index are dropped. For any given referent group DEA software (eg Warwick DEA Software) can be used to compute an exceptionality index. The Appendix details how we can compute an individual’s index of attainment exceptionality. The flow chart in Figure 1 summarises how we use the index to progressively drop individuals from the incumbent group, stopping when some user – specified percentage p1 of individuals of the initial group have either been dropped or have attainment exceptionality index to within 100 –p2 %, p2 being user – specified as explained below.