President's Medal Previous Awards


  • 2022 - Tesco data science team led by Dr Ramon Fuentes
  • 2021 - Antuela Tako, Stewart Robinson, Anastasia Gogi, Zoe Radnor
  • 2020 - Harminder Mann, Jordan Low, Kim Brett, Lizzie Baggot, Mary McKee, Sara Smith, Sam Rose 
  • 2019 - Martine Wauben, Phil Macdent, Adam Booker and Ben Marshall
  • 2018 - Robert Leese, Jakob Blaavand, Claudia Centazzo, Andrei Bejan and Paul Munday
  • 2017 - Hara Papachristou, Chris Legge, Martin Clocherty, Morgan Knapton, Chris Henzel, Oliver Buhlinger
  • 2016 - Gilbert Owusu, Sid Shakya, Raphael Dorne, Ahmed Mohamed, Ali McCormick, Anne Liret
  • 2015 - EURO conference, no award this year
  • 2014 - D Lane, E Munro, E Husemann
  • 2013 - S Mardle, L Fildes, T Lewins
  • 2012 - C Marston, P Rose
  • 2011- S Hammond, K Slater
  • 2010 - P Frangos, S Hughes
  • 2008 - M Allwright, J Ball, D Pankhurst, I Wright
  • 2007 - T Lewins, M Sykes
  • 2006 - A Swain, A Ross
  • 2005 - P Loucopoulos, K G Zagrafos, Y Pyrgiotis, D Beis
  • 2004 - Peter Loader, Mark Bryant, Paul Bates, Chris McGuckin, Deborah Kelly
  • 2003 - M Acutt
  • 2002 - M Neighbour
  • 2001 - J Race, D Burnell
  • 2000 - F Vasko, H Gilles, D Bright, V Kocatulum, E Wolf, J Moffat
  • 1999 - G Laporte, F Semet, V V Dadeshidze, L J Ollson
  • 1998 - J H Powell
  • 1997 - H Tsoukas, D B Papoulias
  • 1996 - J J Glen
  • 1994 - A Khakee, K Stromberg
  • 1993 - C P Thunhurst, C J Ritchie
  • 1992 - M B Wright
  • 1991 - B C Dangerfield, C A Roberts
  • 1990 - A H Christer, K A H Kobbacy

Citation for President's Medal 2022

The Tesco data science team, led by Dr Ramon Fuentes (Lead Data Scientist)

Clearance Pricing Optimisation for UK’s Largest Supermarket.

The President’s Medal has been awarded for a project that combined technical expertise in a solution that delivered both the necessary scale, speed and variety of application, and has been seen through to full implementation into operational systems. The issue of balancing the need to maximise revenue whilst minimising waste is seen in both food and non-food items, and in both seasonal and non-seasonal goods. The scale of the solution is impressive enough in itself, with more than half a million pricing decisions across more than 3,000 UK stores being made daily.

The clearance optimisation problem is handled as a Markov decision process, with a series of interlinked demand forecasting and optimisation steps. The team had to overcome two sides to the optimisation challenge – the sheer number of decisions presented a computational challenge and having to optimise over fundamentally uncertain estimates which propagates further over the stages. This is book-ended with familiar challenges of bringing all the necessary data together and implementing the final models into store systems.

The results were impressive too. The initial trials in the Summer of 2019 were positive, but the second phase was able to deliver increased revenue and reduced waste. Whilst the total financial impact wasn’t shared (being highly sensitive, of course), the improvements in the levels of waste were 3-7% and that clearly represents significant savings. There are knock-on effects for the local communities too, with a much more accurate forecast of the amount of goods to be donated to food banks and other projects.

The project involved combining different approaches based on soft systems and lean service improvement methodologies. It ensured that core OR techniques were at the heart of evaluation processes that brought together service providers and users (in the form of patients). The combination of excellence of process, technical challenge, impressive impact and the longevity of the solution made this project a worthy winner in the judges’ eyes, mirroring the sentiment of the audience at the conference.

Tesco_Logo.svg.png

The team:

Ekaterina Arafailova, Francesco Bucci, Tim Butler, George Dikas, Sivaji Doguparthi, Ramon Fuentes, Ross Hart, Akshay Kishan, Can Kocer, Aleksandar Kolev, Sebastian Lautz, Stephen Logan, Fabio Milano, Edwin Reynolds, Himanshu Singh, Stephen Spurri, Hamish Teagle, Benjamin White, Da Wei Wong.

tesco_ds_team_photo.png

Citation for President's Medal 2021

Antuela Tako, Stewart Robinson, Anastasia Gogi, Zoe Radnor

The SIMTEGR8 project: ’Helping the Lightbulb Service Deliver improved housing support services to the Frail and Vulnerable in Leicestershire and Rutland’

In a highly competitive field, this project impressed the judges in several ways.

The enhancements to housing support services delivered as a result of this project significantly improved the lives of the frail and vulnerable beneficiaries at the same time as providing significant cost savings for the service provider, the local council in this case.

The cost savings were highly impressive, halving the service costs per case. The project involved combining different approaches based on soft systems and lean service improvement methodologies. It ensured that core OR techniques were at the heart of evaluation processes that brought together service providers and users (in the form of patients).

The longevity of the project was also key, from supporting the initial business case, through service optimisation to savings over several years. It is for this combination of technical challenge, degree of innovation and scale of benefit that the judges unanimously selected this project as a worthy winner.

PresidentsMedal2021.png

Left to right: Anastasia Gogi, Antuela Tako, Stewart Robinson, Zoe Radnor

Citation for President's Medal 2020

Harminder Mann, Jordan Low, Kim Brett, Lizzie Baggot, Mary McKee, Sara Smith, Sam Rose      

Creating new population movement insight to shape the UK Government’s response to the Coronavirus pandemic

The judges were very impressed by this project’s rapid development approach that delivered previously unavailable mobility information to policy makers and managers against the intensely demanding timescales of the pandemic crisis, with the first results coming through only 48 hours after a new unit was established to produce them. The solution combined existing modal mobility data with mobile telecoms and additional open source or government data sets, and results from the analytics were critical in highlighting movement patterns around hotspots of infection.  

The project went on to produce evidence to support significant decisions at the national and local levels as the crisis has unfolded. In addition, while meeting the mobility information needs of the crisis would be impressive enough, the team has ensured that the benefits of improved insight into mobility will be sustainable, meeting the longer-term needs of Government.

The President’s Medal 2020

Runner-up (1): Nav Mustafee and John Powell from the University of Exeter Business School

Project Title: NHSquicker: shaping demand for urgent care through real time data and digital nudges – presented by 

The President’s Medal 2020

Runner-up (2): Stefan Sadnicki (Copperleaf Technologies) and Sri Rao (National Grid)

Project Title: Electric transmission circuit optimisation

PresidentsMedal2020.png

Citation for President's Medal 2019

Martine Wauben, Ministry of Justice, Phil Macdent, Ministry of Justice, Adam Booker, Ministry of Justice and Ben Marshall, Ministry of Justice 

A Safety Diagnostic Tool for Harnessing Operational Data to Make Prisons Safer

The President’s Medal has been awarded for a project to develop and implement a safety diagnostic tool to support front-line prison staff in managing violence and self-harm in their institutions.

In a highly competitive field, this project impressed the judges in several ways.

Technically the project combined problem structuring, statistical methods, open-source coding, data wrangling, app design and data visualisation.  This included an innovative statistical approach to predicting prisoners’ likelihood of committing further violence.  The solution was then delivered via an app utilising R shiny alongside bespoke html, javascript and d3.js charts.

The tool has been rolled out across the prison estate and there are now about 45,000 users across 120 prisons with 250 daily unique users.  User feedback has included: ‘it is the go-to tool for incident management’, ‘I wouldn’t let it go’ and ‘it allows us to be proactive instead of reactive’.

Benefits have included more bespoke interventions, a big saving in staff time and far better data quality.

It is for the combination of technical challenge, innovation, effective embedding in prison operations and impact delivered that the judges (and the OR61 audience) agreed that the safety diagnostic tool was a worthy winner.

Citation for President's Medal 2018

Robert Leese, Jakob Blaavand, Claudia Centazzo and Andrei Bejan (Smith Institute) and Paul Munday (Bank of England)

Repurposing the Radio Spectrum: Delivering on the Promise of Next-Generation Mobile Services

The President’s Medal has been awarded for a project that provided optimisation and algorithmic support to a unique $19.8 billion two-way auction of part of the radio frequency spectrum in the United States.

In a highly competitive field this project impressed the judges in several ways.

There were tough technical challenges to overcome to ensure that the optimisations were error free, would maximise the amount of spectrum that could be made available and would run fast enough to meet the needs of the auction. If ever there was a project that had to get it right first time this was it – and it did.

The project shaped the industry for the next ten years, involving 3,000 TV stations and 62 mobile operators. The complexity of this was significant, requiring technically innovative solutions. It was also of critical importance to the US Government, generating a $7 billion surplus for the US Treasury.
An immovable and publicly announced end-date required extremely effective and agile project management, with continuous cooperative communication with the other contractors on the project.

It is for this combination of technical challenge, degree of innovation, scale of benefit delivered and quality of project management that the judges agreed that this OR support to a unique two-way auction was a worthy winner.

The President’s Medal 2018 Runner-up (1):

Ian Griffiths and James Adamson (decisionLab); Richard Wakelen and Sheena Wilson (Canal & River Trust)

Project Title: Transforming a National Institution: A Case Study in Bringing Together OR Best Practice and Engineering Expertise to Improve the Waterways of England and Wales

The President’s Medal 2018 Runner-up (2):

Max Moullin, John Soady

Project Title: Improving Sheffield’s Health with the Public Sector Scorecard

Winners of 2018 President's Medal

Robert Leese, Jakob Blaavand, Claudia Centazzo, Andrei Bejan and Paul Munday with The OR Society President, John Hopes

Citation for President's Medal 2017

Hara Papachristou and Chris Legge of Lanner Group Ltd; Martin Clocherty, Morgan Knapton, Chris Henzel and Oliver Buhlinger of Hayward Tyler Ltd.

A SIMULATION PLATFORM TO CREATE A ‘VIRTUAL FACTORY’ FOR HAYWARD TYLER’S STRATEGIC MANUFACTURING OPERATIONS

The President’s Medal has been awarded for a project that used simulation to help a mid-sized manufacturer of performance-critical pumps and motors for the energy industry, Hayward Tyler, transform their manufacturing capability.

In a highly competitive field, this project impressed the judges in several ways. Technically, it showed an exceptional range and depth of processes and entities simulated, together with innovative collaboration with a video-gaming company to develop an immersive reality simulation ‘digital twin’, which enabled the model and plans to get buy-in and commitment from staff, investors and customers as well as top management.

This modelling innovation matched and enabled the innovations in the factory itself that led it to win the 2016 UK Manufacturing Award for ‘Smart Factory of the Year’. As a consultancy project, it demonstrated the willingness of the external consultants to adapt their activities to the client’s needs, wants and timing, moving from high-level prototyping to detailed modelling as the customer's requirements evolved. The appreciation of the customers was also shown through their participation in the presentation at the OR conference.

The work was ambitious in supporting a bold ambition of the client: to double output in 6 years, without production risk. It achieved enormous business benefit: an immediate saving of £500k in capital costs; a two-thirds reduction in ‘days to order fulfilment’, and complete elimination of late fulfilment. Finally, the delivered solution was flexible enough to be readily and cheaply adapted by the client to be used in every single one of their factories around the world.

It is for this combination of practice and impact that the judges agreed that this Simulation of a Virtual Factory was a worthy winner.

Winners of 2017 President's Medal

Recipients of the President's Medal 2017

Citation for President's Medal 2016

Gilbert Owusu, Sid Shakya, Raphael Dorne, Ahmed Mohamed, Ali McCormick and Anne Liret

This was a particularly competitive year for the President’s Medal. Like all the finalists, the BT entry addressed both technical issues and culture change, demonstrated impact that shows every promise of being long-lasting; showed a flexible and adaptable approach to the presenting challenge; tackled an important problem and had a ringing endorsement from the top managers who had commissioned the work.

This award to the BT team recognises the sheer excellence of the all-round OR process, working with stakeholders from shop floor to top management, providing strong technical solutions adapted to the needs of the users, and really making a difference. Congratulations go to all participants.

The Telecom Team winners of 2016 President's Medal with Ruth Kaufman

The Telecom Team with Ruth Kaufman (President of The OR Society)

Citation for President's Medal 2014

David Lane, Eileen Munro and Elke Husemann

Using Systems Thinking to Undertake a Review of Child Protection in England

The winner of The OR Society President’s Medal for 2014, for the submission judged to be best practical, implemented, application of OR, goes to David Lane (Henley Business School), Eileen Munro (LSE) and Elke Husemann, for “Using Systems Thinking to Undertake a Review of Child Protection in England.” The work formed a central part of ‘the Munro Review’ (Munro, 2010, 2011a, b), a high-profile review of state-managed child protection activities in England, conducted for the Department for Education.

Child protection in England encompasses a wide range of services which aim to secure the safety and welfare of children and young people. This includes, for example, the provision of foster care and adoption services, and support for families. The child protection system is also responsible for the investigation of possible cases of child maltreatment, and intervention in such cases.

David Lane, Eileen Munro and Elke Husemann winners of 2014 President's Medal

David Lane, Eileen Munro and Elke Husemann

Drawing on the field of system dynamics, causal loop diagramming was used to map the sector. Among the evidence used in support of the map were published research, expert interviews, and comments from relevant professions (eg health, social work, judiciary). This systems mapping unearthed a prescriptive approach to child protection that contributed to a culture of a compliance.

A second phase of work used group model building to examine the functioning of the sector. A group of professionals and experts worked over a number of sessions and created a large and complex systems map of current operations. By considering the causal mechanisms that were in operation, it became apparent that the sector was in the grip of a set of reinforcing effects, or ‘vicious circles’. The map was subsequently used to give structure to the issues the review had to address and ultimately provided an organising framework for the recommendations that were made. Of the 15 recommendations in the report, ten were accepted in full and five were accepted in principle.

Using systems thinking and causal loop diagrams the work addressed a vital area of public policy and it had a major influence on the recommendations of the Munro Review. That influence continues through on-going changes in government policy for child protection. The work was therefore judged a worthy winner of the President’s Medal for 2014.

References

Munro, E. 2010. The Munro Review of Child Protection Part One: A Systems Analysis. TSO: London.
Munro, E. 2011a. The Munro Review of Child Protection Interim Report: The Child's Journey. TSO: London.
Munro, E. 2011b. The Munro Review of Child Protection Final Report: A child-centred system. TSO: London.

Citation for President's Medal 2013

Simon Mardle, Louise Fildes and Tony Lewins (Ernst & Young)

Optimising the Retail Network for New Zealand Post

The winner of The OR Society President’s Medal for 2013, for the submission judged to be best practical, implemented, application of OR, goes to Simon Mardle, Louise Fildes and Tony Lewins, all of Ernst and Young, for “Optimising the Retail Network for New Zealand Post”.

The work concerned how to cope with major change in New Zealand post, faced with a decline in its traditional business of letter handling, a country becoming more urbanised and a shift from high-street to out-of-town shopping.

This involved developing models to optimise, within operational, financial and social constraints, New Zealand Post’s retail network under a wide range of scenarios for possible change. These included changes in customer habits, new types of outlet, new products and various assumptions about future business volumes. 

Tony Lewins, Simon Mardle and Louise Fildes winners of 2013 President's Medal with Geoff Royston

Tony Lewins, Simon Mardle and Louise Fildes with Geoff Royston (President of The OR Society)

The team developed models employing an innovative combination of a simulated annealing algorithm and a geographic information system, allowing analysis of the complex interplay between changes in the network and business flows. The models were designed to be flexible and easy to refresh with data from routine operational systems, work (which was split between New Zealand and the UK) which involved close collaboration between analysts and managers.

The work has been used on a weekly basis for the last two years by New Zealand Post’s strategy team for a range of purposes which included designing a national transformation strategy, planning new outlet types and recovery from exceptional events. Overall the work successfully identified how to evolve the business into a radically new operation while improving annual performance by tens of millions of pounds.

This work was innovative, showed careful attention to process, and clearly achieved a high level of demonstrable and enduring benefit. It was therefore judged a worthy winner of the President’s Medal for 2013. 

Citation for President's Medal 2011

Stephen Hammond and Keith Slater (National Air Traffic Services)

Air traffic control, business regulation and CO2 emission

The presentation described the environment of air traffic control over the U.K. The project concerned the development of performance metrics to show how the actions from air traffic control impact the fuel use of flights. The use of fuel has a direct impact on costs and also an environmental impact in terms of the production of greenhouse gas emissions.

The development of these new metrics has led to benefits to National Air Traffic Services in managing their business, as well as leading to reduced costs and reduced environmental impact for the airlines operating in UK air space.

The judges particularly commended the clear presentation and the range of benefits of the work.

Citation for President's Medal 2010

Panos Frangos and Simon Hughes (Sellafield Limited)

A model future for the UK’s nuclear legacy

The presentation described the work of the Sellafield OR Group to support decisions for safely delivering reprocessing, waste management and decommissioning responsibilities on behalf of the Nuclear Decommissioning Authority. The group follows an interactive project lifecycle framework involving the client the supplier and the OR Group.

A particular project was described involving the decommissioning of a storage facility. A simulation model was developed and a visual display of the model was demonstrated showing how the operation had been modelled. Use of the model resulted in substantial savings to the company of the order of millions of pounds.

The judges particularly commended the independent validation procedures for the modelling and the clear presentation.

Citation for President's Medal 2008

Mike Allwright, Jenny Ball, Dave Pankhurst and Iain Wright, Department for Work and Pensions

Optimising the Department for Work and Pensions' estate

The Department of Work and Pension's estate is large and complex, with over 1,000 buildings costing over £600m per year. Central management could see the case for efficiencies, but local decision-makers did not have the incentives to make the necessary changes.

The OR team was instrumental in securing substantial cost reductions whilst meeting needs for the 3-year period to 2011. Key features of the project were:

  1. Collecting accurate information on the costs and usage of all buildings
  2. Developing an integer linear-programming model in SAS, which illustrated the scale of the savings possible and won the backing of senior managers for the project. The model set out to minimise the cost of the estate over a 3-year period, subject to operational constraints based mainly on localised staffing projections.
  3. Developing spreadsheets for each of the 50 districts, so as to focus the attention of key local decision-makers on the inefficiencies in their area
  4. Engaging with the multitude of stakeholders, to understand the operational context in which estate is used and the decisions that needed to be made.
Jenny Ball, Dave Pankhurst and Iain Wright winners of 2008 President's Medal

Jenny Ball, Dave Pankhurst and Iain Wright

The project resulted in 120 buildings being designated for closure, with savings estimated to be £50m over the three years. At the time of the presentation around 100 buildings had already been closed. The approach is being repeated in 2008, and is being developed for application to other resources within the DWP.

The judges were impressed with the clear exposition of this problem and its context and the use of both hard and soft approaches to achieve an implemented solution. The presentation stressed the importance of collecting relevant and accurate information and bringing together the various stakeholders within a potentially intractable organizational environment. Implementation was facilitated by providing planning tools which enabled each district manager to adjust their LP solutions so as to take advantage of detailed local knowledge and also to see the financial consequences of these changes. This resulted in shared ownership of the eventual solution.

Citation for President's Medal 2007

Tony Lewins and Martin Sykes, PA Consulting Group; Amanda Moon, Nissan UK

The President’s Medal for 2007 is awarded to Tony Lewins and Martin Sykes of PA Consulting Group and Amanda Moon of Nissan UK for their work in developing a production-scheduling tool to support Europe’s most efficient car plant at Nissan UK in Sunderland.

The original aim was to enable a third car model, the Almera, to be built alongside the already produced Primera and Micra, but without adding a third production line. The main challenges were the requirement to schedule three car models simultaneously on two production lines and the need to respect some 2,500 operational constraints governing how the different vehicles and their features can be sequenced. Failure would have resulted in the closure of the plant and the loss of 1200 jobs.

PA designed, developed and implemented a highly visual, PC-based scheduling tool to generate a buildable sequence for each week’s production of up to 12,000 cars, which has achieved an extra 30% in production capacity without the need for a further production line and cost savings estimated at £2m per week. Additional benefits include a reduction of materials stock holding by up to 40%. In mid-2006, the scheduling system was enhanced to support the building of a 4 th car, the Qashqai, on the two production lines and eventually a 5th – in fact the solution was generalised so as to solve n models on m production lines. The first Qashqai rolled off the production line as planned in December 2006 and the scheduling system continues to plan all production at the plant. The judges were impressed with both the impact of the project, the degree of innovation involved, the simplicity of the new system (which can be operated by a placement student) and the clarity of the presentation.

Tony Lewins and Martin Sykes winners of 2007 President's Medal with Jeff Griffiths

Tony Lewins and Martin Sykes with Jeff Griffiths (President of The OR Society)

Citation for President's Medal 2006

Alison Swain and Alex Ross, British Airways OR End-to-end Scheduling Team

The 2006 President’s Medal is awarded to Alison Swain and Alex Ross of the British Airways OR End-to-end Scheduling Team for their innovative and practical approach to Improving British Airways Short-haul Punctuality.

Departure delays on short-haul flights are a key area of customer dissatisfaction, and can be difficult to eliminate due to the number of different parties involved and the inherent susceptibility of aircraft schedules to knock-on effects, such as a delay in an incoming aircraft. Several analytical approaches, some quite complex, had been tried to tackle these problems but without success. The team recognised the need for a new approach that took into account the chain of events leading up to an aircraft departure. Their approach decomposed and measured each event in the chain, drawing on disparate databases to present an integrated picture. New metrics related to ‘On-Time Achievable’ departures were invented and a simple graphical presentation of the whole chain of events devised. This ‘waterfall’ presentation is used by senior management to identify weak links in the chain and also to make detailed comparisons of different cases such as Heathrow vs Gatwick.

Application of the approach has already led to improvements in scheduling saving the equivalent of £5m. Notably, senior management have, for the first time, a tool that allows assessment of schedules and their deliverability in a focussed way, providing the basis for continuous improvement in short-haul punctuality.

The team is to be commended for carrying out a simple but extremely effective piece of work in an area of British Airways’ operations which presented the results in a clear and novel way, capturing management attention, and facilitating focussed action to improve schedules.

Citation for President's Medal 2005

Professor P Loucopoulos, Professor K G Zagrafos, Mr Y Pyrgiotis and Mr D Beis

The 2005 President’s Medal is awarded to Professor P Loucopoulos, Prof K G Zagrafos, Mr Y Pyrgiotis and Mr D Beis for their robust practical application of an integrated range of OR methods in the planning of the 2004 Olympic Games in Athens.

The team initiated and directed the development of PLATO (Process Logistics Advance Technical Optimisation), a highly systematic approach to the design of venue operations. This was based on process engineering, scenario development, knowledge management and IT, underpinned by System Dynamics and Simulation. It greatly assisted decision-makers in planning and managing resources cost-effectively, and influenced many areas from flow of spectators around venues to athletes’ transportation. The work undoubtedly made a major contribution to the successful production of highly reliable venue operations at reasonable cost and will also be of great benefit to such events in years to come. It is estimated that use of the PLATO approach may have saved the organisers some 70 million US dollars.

The team is to be commended for carrying out an excellent piece of work in such a high risk, high profile arena, thereby helping to place OR on the world’s stage.

Citation for President's Medal 2003

M Acutt

The 2003 President's Medal is awarded to the Yorkshire Water team for their lucid presentation of work on a Cost-Benefit Approach to Capital-Maintenance Planning. The project was an excellent example of OR at the strategic level in a company. An important aspect of the project was the need to manage several parallel streams of activity, such as asset- failure prediction, business-risk assessment and market research on customers’ willingness to pay. The economic level-of-service model developed uses a variety of classic OR techniques to identify the optimal portfolio of capital expenditure.

The project was characterised by effective involvement of the client, thorough data collection, innovative development of IT-based models and co-operative working between internal and external consultants.

Citation for President's Medal 2001

David Burnell and Julia Race, Thames Water

The President's Medal for 2001 is awarded to David Burnell and Julia Race of Thames Water for their work on network asset performance modelling. The work dealt with major issues facing the company in its management of the network. The issues centred round the problem of leakage and its possible remedies. Political pressure from the highest levels and pressure from the regulatory authorities led to the work's being commissioned, and subsequently supported, by members of the company's board. Large-scale models were developed, which involved a judicious balance between detail and timeliness, some ingenious technical modelling and some particularly imaginative data collection. Whilst there were measurable benefits in terms of increases in revenue and reductions in costs, the work's larger impact may well lie in the enhanced understanding it offers of the processes at work, and the consequent improvement of the company's general performance, including its negotiating power.

The presentation was to be applauded for its imaginative use of two screens and alternating presenters, though this did occasionally lead to some measure of confusion. Nevertheless, the presentation conveyed clearly enough the quality and effectiveness of the work.