Rail 670: Project assessment is a fiddle

In one of those obscure but important announcements that trickle out of government, Philip Hammond, the increasingly prominent transport secretary, has changed the way that  transport projects are to be assessed. While not surprisingly this does not make headline news, especially as the announcement was slipped out on April 27 a couple of days before a rather notable event that for some reason was attracting all the attention, this is important stuff and may have a significant impact on the way that rail projects are considered.

The previous system of assessing projects, NATA, has been used since 2003 and was supposed to be an improvement on previous appraisal methods which were thought to concentrate too much on narrow financial benefits rather than wider social considerations. Under NATA there were five criteria: environmental impact, safety, economy, accessibility and integration.

However, NATA is widely thought to have failed in its objectives. Somehow, economic factors  always seemed to win through over the environment and any other considerations, despite the intention behind the method. Moreover, there were some real nonsenses such as when people were induced to leave their cars at home and travel by rail, the assessment had to take into account as a negative the fact that there would be a reduction in the amount of tax from fuel duty. Only a Treasury mandarin of the worst kind could have thought that one up.

Under the new system, there will be five new ways in which schemes will be assessed:  strategic, economic (value for money), commercial (viability), financial (affordable) and achievable (management case). As expected from a Tory dominated government, this sounds much more business-like and in a way appears to be a return to the overt concentration on economic factors in the old system.

There is, though,  some good news in that the ridiculous analysis of tax income has been scrapped and according to some insiders there is a feeling that the new way of assessing schemes will be helpful to rail projects. I am not so optimistic.  As with all these systems, it is a matter of how they are used, and given that rail schemes for the most part need government funding, the analysis may be skewed against them. Roads, with their ability to attract vast volumes of traffic, may look better given the emphasis on money which seems to be built into the new methodology which still relies on the notion that small time savings, monetised and aggregated up through counting millions of users, is the main benefit for most  projects. I have mentioned often before that this narrow approach to assessing the value of project is a nonsense and ought to be replaced with a much more sophisticated analysis.

There is considerable academic work going on in this respect.  I recently met  Harry Dimitriou of University College London who has been running a five year study into what they called Mega Urban Transport Projects which are major infrastructure schemes costing billions. The study which is not quite complete encompasses thirty schemes, including,  in Britain,  the Channel Tunnel Rail Link, about which he interviewed me a couple of year ago,  and the Jubilee Line Extension.  Their preliminary presentations show that very little proper appraisal of the value of such schemes is ever carried out despite the huge amounts of public money which go into them. Most amazingly, no one ever seems to go back to the schemes and looks at whether the promised benefits, in terms of time savings and other improvements, were actually delivered. If nothing else, it would be a way of assessing the viability of the methodology and that’s perhaps why its never done. I remember asking people at the old Strategic Rail Authority if they ever did such analysis after the event, and received the same negative answer.

Interestingly, their study will highlight factors that both question the value of these schemes and, on the other hand, which suggest their benefits are often underplayed.  On the first point, governments tend to propose these schemes without a proper strategic context. They do not look sufficiently at alternatives or at the wider aims.

On the other hand, there is, according to Professor Dimitriou, far too much focus on the narrow time frame of the construction period when the main issue is whether the scheme finished on time and on budget. That is spot on. Moving away from transport, one can point to, for example, the British Library which was the subject of enormous fuss because of cost overruns but is now recognised as a magnificent building that is universally liked (well, inside anyway, which is fabulous, but outside it still looks like Tesco mock town hall architecture that you find on ring roads) and the O2, formerly the Millennium dome, which has become Europe’s most successful music venue after being widely derided as a white elephant when it was the Millennium dome.

This should inform the HS2 debate because this is the biggest flaw in the support for it which is the question of ‘what is HS2 for?’ That has always been my main objection to it. There is a lot of woolliness about the reasons why it should be built – capacity of the rail network, the environment, regeneration of the north, connectivity and so on – but none of this has been put in the wider context.

Indeed, the one attempt to do so, the Eddington report into Britain’s transport infrastructure needs  published in 2006 was highly sceptical of the benefits of a new high speed line. On carbon emissions, for example, he said: ‘On a project of this size, carbon saving benefits of £2.1 -£5.4 billion [his undiscounted estimate] under the most optimistic assumptions, are never likely to play a major role in the business case’.

Eddington’s scepticism came, too, from the realisation that there was a big opportunity cost to building a high speed line. In other words, it would soak up huge amounts of money available for investment in the railway. Whatever ministers say about this now, I remember Theresa Villiers in opposition making it quite clear that the much of the cost of the line would come out of existing railway funds and she was speaking before the economic crisis in which we are all still struggling.

There are numerous rail schemes that are crying out for far smaller sums than HS2 but which would have, quite probably, a bigger impact pound for pound. The BML2 scheme for a second line to connect Brighton with London which was the basis of Steve Broadbent’s article in the last issue is an excellent example. The Brighton line is one of the most congested in the country and I wrote recently in this column (Rail 662) about Southern’s efforts to put on an extra hourly train which were stymied by the regulator because of fears about the effect of these services on reliabilty.

The case for BML2 looks, on the face of it, strong but, as Steve so aptly pointed out, there is a very different approach between plans for the West Coast Main Line and for London – Brighton. Whereas the projected increased demand for West Coast is presented as the main reason why HS2 is needed, nothing is being suggested to cope with the rise in passenger numbers on the Brighton line. Now of course in an ideal world, both situations should be addressed, but we live in a time of severe cash constraint.

There are lots of similar cases, especially on lines used by commuters both in the London area and in major provincial cities, where there is a real need for extra capacity. Some of it could be delivered cheaply with extra trains or longer platforms, while for others new – and inevitably expensive – infrastructure will be needed.  If Hammond is going to follow his own new rules, therefore, he must produce a strategic vision of the railways for the next 30 years, and balance all these investment decisions. If HS2 can stand up to that scrutiny, then it would be worth proceeding with it, but my instincts suggest it will not.

McNulty to sound a damper to the rail industry

By the time most people receive this magazine, the McNulty report on making the railway more efficient will probably be out. Various leaks suggest that he will push for increased fares, especially at peak times, to damp down demand and that he will put into question the viability of rural routes and branch lines, though I doubt very much whether he will use the C word as, when I met him, he did not seem to want to take on Beeching’s mantle and said such decisions were up to government. Nevertheless, this will all be a damper on the industry at a time when it is celebrating, yet again, substantial growth. My alter ego, Mystic Wolmar,  suspects, too, that any suggestions of increasing fares even more than the RPI + 3 per cent promised for the next three years or on closures will fall on deaf ears at the ministry, given their unpopularity.

  • Anonymous

    So called consultations are not worth the paper they are printed on at the end of the day – time and time again we have seen that government ministers and managements have already decided what they want to do – they then “consult” with the stakeholders because they are legally or morally obligated to, then they carry on regardless anyway with maybe a token modification here and there to make it look as though they actually listened. 

    As Ken Livingstone famously said….”if voting changed anything they’d abolish it…”

  • Paul Holt

    “…increased fares, especially at peak times, to damp down demand …” sounds like congestion charging.   If http://www.christianwolmar.co.uk/2011/06/congestion-zone-madness/ is anything to go by, CW must be in favour!

  • Richard Crompton

    ” there would be a reduction in the amount of tax from fuel duty. Only a Treasury mandarin of the worst kind could have thought that one up.”Funny you should say that Christian, because when it comes to selecting a bidder to manufacture the IEP, the Treasury Mandarins pay absolutely no regard to companies with existing UK factories who could design and manufacture better trains than the high cost, stupid one designed by the civil service.  They are not concerned that it would keep mangers, professional engineers, skilled manual workers and suppliers in profitable work, paying taxes and not swallowing unemployment benefit.  They care not a fig that if they supported UK manufacturers, they would  also given them a better chance of winning overseas orders with further benefit to the economy.
    Perhaps they employ a idiots with a different skill set in the DfT compared to the ones in the Treasury.

  • ” the assessment had to take into account as a negative the fact that there would be a reduction in the amount of tax”.

    Funny you should mention that Chris. I recall that the good people of Coventry wanted to build a bus lane for their citizens to travel in. It failed to get consent on precisely that basis.


    In that article, there is mention of a motorway crawler lane being refused consent because it would have a negative effect on tax revenue.

    What a screwed up system of government we have in the UK.

  • Christian Schmidt

    Probably a bit late, but here goes:
    I’m a transport planner. and I have undertaken many apraisals over the last decade (as well working on evaluation methods – NATA/STAG/WelTAG are not useful for smaller schemes, yet when looking at results smaller schemes so often turn out better value for money).
    I see two benefits of appraisal. Firstly it helps me to understand the proposals better. Most experienced transport planner should be able to spot good projects (definately worth doing), poor once (definately not) and ‘it depends’ ones. But there are still times when a bit back-of-the-envelope quanitfied assessment helps to clarify your own thinking. Especially sensitivity test (which variable influence the outcome of the appraisal most) can be useful, in particular for ‘it depends’.
    HS2 is of corse an ‘it depends’ scheme – most of the benefits are indirect. That doesn’t mean they are not real, the line will make a difference to people’s lives and to the economy. But any quantification benefits could be so easily be out by 50% (or even more) either way, that it doesn’t really prove anything. Furthermore the realisation of these benefits very much depends on other factors that have nothing to do with the project. (But such wider factors could equally make the project much more worthwhile – who knows.) Which overall makes it just about possible to assess whether such a project would have a positive benefit (in my views yes for HS2) but quite impossible to state whether there aren’t other (even similar) schemes that would provide a better value for money. (Again, my view is yes, but that’s because of the current structure of the UK rail industry. A better structure may make the scheme relatively more worthwhile.)
    The second benefit of appraisal is that quantification (of a the benefits of a scheme that I have already decided is worthwhile) may help convince others. And note these two treasons may require two different types of evaluation.
    Oh, and more comment, whether projects work out well in the end is often due to attention to detail (esp. for smaller schemes), which of course is never captured by any assessment.