Sign Out
Logged In:
 
 
 
 
 
Tab Image

Risk Assessment of Gas Transportation Invoices

- business change facilitation using risk modelling

by Sallyann Blackett and Chris Smith

Introduction of a complex new process needs careful consideration to ensure full understanding by all parties. It is not always easy to see which contributing processes have most influence or how accurate the overall process is in practice. It can also be difficult to foster acceptance and confidence in the new process. In such cases risk modelling can be an effective method of analysing the potential variation and errors, and providing a method of monitoring performance of the process. This paper describes the approach adopted for resolving issues raised by a new method for invoicing gas shippers by Transco.

-oo0oo-

Transco is a part of BG plc and is a UK gas transporter. It transports gas for its customers, the 50 or so gas supply companies known as Shippers, through a gas transportation network designed and built on an integrated basis across the UK. An element of the invoicing of Shippers to domestic customers for transporting gas on their behalf utilises a new process, Reconciliation by Difference (RbD).

Estimates of gas transported for Shippers are calculated on a daily basis (known as deeming). When meters are read for large customers the meter reading is used to calculate a ‘true’ consumption that is compared with the deemed consumption. The difference between the two readings is a reconciliation that will lead to a charge or refund to the Shipper supplying the consumer. This process is accurate but intensive and was thought to present high risk and costs, to both Transco and Shippers, if implemented for 19 million domestic consumers.

Very large consumers will have meters that are read on a daily basis (DM). The remaining large industrial and commercial meters (I&C) are read either monthly or six monthly. RbD works on the basis that as daily total gas flows are known, then when a reconciliation takes place for DM or I&C customers an equal, but opposite, amount should be allocated to domestic consumers to maintain the balance in gas. This amount is apportioned to domestic Shippers based on their market share.

RbD is obviously a complex process involving algorithms based on data from several feeding processes. To build confidence in the methodology, Transco and Shippers required an objective assessment of potential variation and errors and a means of monitoring performance. Cap Gemini was commissioned by Transco and Shippers to undertake this work.

Risk modelling approach

The approach adopted by Cap Gemini was to work with both Shippers and Transco in identifying their requirements. A list of all processes that impacted RbD was compiled. Using this list as a basis for discussion, the processes were split into groups depending on whether they were felt to have a major, moderate or minor impact. To simplify the model, only processes felt to have a major impact, or those critical to calculation of the charges, were included in the model.

A qualitative model was produced using data flow diagrams and Ishikawa diagrams. These techniques allowed the impacts to be visualised and promoted further discussion with Transco and Shippers. Once a consensus had been achieved on the processes that would be modelled the quantitative model could be developed. This stage proved to be an advantage in gaining acceptance of the final model as all parties involved felt they had a chance to influence its content. Figure 1 is an example of the type of model produced at this stage. RbD risk is defined as the difference between domestic consumption and consumption invoiced through RbD.


Figure 1: Example of Qualitative Model of RbD

The model was developed to mimic the RbD process as far as possible. The model also calculated an estimate of charges that would occur if each domestic meter was read and reconciled in a similar manner to large sites. This allowed a measure of risk to be determined based on the difference in charges that Shippers would pay under RbD and the charges that would have been invoiced by reading and reconciling all domestic meters.

Where possible, actual data from the past twelve months was used to provide the input to the model. In some cases data was not available or was not appropriate in typical, ongoing conditions. In these cases a consensus assessment was made after discussion with Transco and Shippers.

The risk assessment relied on both analysed and subjective views of how each input can vary. Again, Transco and Shippers subjective assessment was sought and consensus values used to augment hard data. Each of the inputs that are considered to vary were associated with a risk distribution. The statistical distributions provide a picture of how likely the variable is to achieve a range of values.

How the model works

In essence, each input variable has a new value assigned to it during each run of the model. The values are selected at random (Monte Carlo simulation) from the input risk distribution and will mirror the ‘likelihood’ of them occurring. This means that the most common value to be selected is the input mode value. Extreme values are selected occasionally to match the input risk distribution. The risk model samples each input variable separately and then recalculates the output variables. This allows complex interactions between the variables to be modelled and produces a range of output values with a ‘likelihood’ of each value occurring. It is equivalent to running a spreadsheet with thousands of ‘what if..’ queries. The model was developed in spreadsheet format and the risk applied using the @RISK package.

The seasonal profile of gas demand was incorporated within the model. However, the start month of the profile was selected at random during the simulation to provide results that were not specific to any time of year.

The amount of variation in the final results of the model gives an indication of the risk involved in RbD. If the spread of modelled monthly RbD values is wide, there is potentially a larger range of differences and less predictability in the results of the algorithm than if the values are tightly clustered.

The model was demonstrated to all involved parties prior to commencing any analysis and an explanation given on how the model worked and what type of results could be expected. This encouraged ownership of the model by both Transco and Shippers, and managed expectations from the model. Risk modelling can often be seen as a ‘black box’ to users who may not fully understand what a risk distribution is showing. It was helpful to talk through the basic concepts using the model as a basis for discussion.

Validation

In order to be certain that the model was correctly calculating RbD values, a validation took place. The model was set up with initial estimates of variables. The model was then run to produce a distribution for a number of outputs which could be compared with reliable data giving the average, minimum and maximum values observed over the last twelve months. The risk distributions were adjusted until the model output was in line with known values in each case.

A good match was obtained in every case (or discrepancies could be explained from known issues around the data or modelling assumptions). It was agreed by Shippers and Transco that this analysis provided sufficient validation of the model prior to the risk analysis taking place.

Is RbD Accurate?

RbD, as a process, would be seen as accurate if the charges it generated were comparable with reconciliation of individual meter points. The model was built to assess risk in any one month. Due to the nature of reconciliation (charges relate to periods of up to the previous 6 months) it is unlikely that risk will be zero in any specific month. In making the model independent of the time of the year a risk distribution centred on zero would show that there was no bias in the RbD calculations. The actual distribution shape is shown in Figure 2.


Figure 2 : Shape of the risk distribution

The resulting risk distribution is centred on zero and confirms that over a period of time RbD will give the same charges as meter point reconciliation, even though in any one month there is likely to be a difference. The shape of the distribution was influenced by the seasonality of gas demand which is a feature of the reconciliation process in general.

The analysis was repeated many times looking at the sensitivity to the input variables, particularly the subjective distributions. To achieve these comparisons the random number stream used by @RISK for simulations was fixed. Each variable altered was investigated to see the effects on the results while being sure that any differences are not due to the variation in random numbers used.

Performance indicators

One method of providing confidence in the RbD methodology is by monitoring and publishing performance indicators. Prior to this analysis there was little basis for comparing RbD invoices as it was uncertain what variation in size of invoice could be expected.

Performance indicators were derived by determining the main variables that influence RbD risk. The model was set up to save a variety of distributions as output variables. This data was exported to SPSS and a standard regression analysis provided the main variables that influenced RbD risk. Analysis showed that five variables could be combined to provide a method of estimating RbD risk with an R2 value of 0.92.

Estimates of RbD risk, the performance indicator, were derived for each LDZ using the regression analysis and actual values for the five input variables. To assist with monitoring of the performance indicator, control charts have been setup where monthly values are plotted on a chart and assessed against control limits. If the plotted points cross the limits there may be a change in the system that needs investigating. Careful maintenance and improvement of the process should lead to these limits becoming narrower over time.

Benefits

Risk modelling highlighted how each process interacted to influence RbD results. It was also possible to show which processes had most influence. This was particularly useful in tackling individual perceptions on which parts of the process were of most risk to them. It was certainly the case that some areas that had been a major concern were identified as not having a significant impact on the final outcome.

Involving both Transco and Shippers in defining and using the model produced a better working environment for moving forward. It demonstrated that working together can generate common ownership and responsibility. All parties now have a shared understanding of how RbD works and can have more confidence in the process. This will help provide a firm basis for future discussions.

The future

Transco can now use the model on its own or Shippers’ behalf as a method of addressing further issues. The model can be used for ‘What If’ analysis where possible changes or improvements are foreseen to the RbD process.

The work on performance indicators will help to build further confidence in the RbD process as it demonstrates that risk is being reduced through process enhancements.

The model is being extended to look at how risk impacts individual Shippers. Results for this are still being validated but we can expect further understanding of how RbD calculations and associated risk depends on Shippers behaviour in the competitive domestic market.

Acknowledgements

The authors acknowledge the co-operation of the members of the Audit Subcommittee who provided the Shipper perspective during the project.

SALLYANN BLACKETT works as a Senior Operational Research Consultant for Cap Gemini UK. She is based in the Public and Process Division and has many years experience in the Gas Industry, having been previously employed by British Gas West Midlands. She holds a BSc in Mathematics with Statistics from Nottingham University and is a Chartered Statistician with the Royal Statistical Society.

CHRIS SMITH works as a Project Development Manager for Transco. Based in the Billing department, he leads the Analytical Services team. He previously worked for Cap Gemini as a Principal Operational Research Consultant and, prior to that, for British Coal’s Operational Research Executive. He holds a BSc in Mathematics and MSc in Operational Research from Southampton University.

First published to members of the Operational Research Society in OR Insight July- September 1999