This is the first of a series of notes which provide more detail about, and comment on, the many attempts – some successful, most not – to ‘reform’ the UK Civil Service. This note summarises reform programs through to just before Tony Blair’s government was elected in 1997.
I recommend that you read the Civil Service Reform: Introduction and Efficiency Programs sections of this website before reading these more detailed notes. You might also like to read about Civil Service Reform Syndrome in order to understand why Ministers and senior officials fail, time and time again, to establish a change program with a realistic chance of success. Then read on for more detailed information about the following key reports and initiatives, as well as about many more less prominent government initiatives, external reports and analysis.
Links to these reports will be found in the detailed text below:
- 1853 Northcote Trevelyan (published 1854)
- 1918 Haldane
- 1955 Priestley
- 1961 Plowden
- 1968 Fulton
- 1982 Financial Management Initiative
- 1987 Ibbs (Next Steps) Report (published 1988)
- 1993 Oughton
- 1994 Continuity and Change
- 1999 Modernising Government
- 2004 Civil Service Reform: Delivery and Values
- 2004 Gershon
- 2009 Putting the Frontline First
- 2012 Civil Service Reform Plan
Despite all the pressure for change, the 21st Century UK civil service retains many of the characteristics of the service that was created as a result of the1854 Northcote-Trevelyan report on ‘the Organisation of the Permanent Civil Service’.
The foundations of a detached civil service based on sound finance to run a modern imperial country can be found in the reforms implemented by William Pitt and George Grenville in the mid 1700s. But the need for reform had become clear by the mid-1800s, driven by circumstances which have immediate resonance today:
'The great and increasing accumulation of public business, and the consequent pressure on the Government'.
The result of Northcote-Trevelyan was a civil service appointed on merit and through open competition, rather than patronage, with the following core values:
- integrity - putting the obligations of public service above your own personal interests;
- honesty - being truthful and open;
- objectivity - basing your advice and decisions on rigorous analysis of the evidence; and
- impartiality - acting solely according to the merits of the case and serving equally well Governments of different political persuasions.
The first Civil Service Commissioner was appointed in 1855.
The Macaulay Report on the selection and training of entrants into ‘the Civil Service of the East India Company’ was also published in 1854. This report was in some ways a 19th century version of the 1968 Fulton Report on the structure of the UK Civil Service. Like the Northcote Trevelyan Report, the text is mercifully short and to the point, and the authors were delightfully honest when not 100% sure of their recommendations:-
“… we are inclined, though with much distrust of our own judgment, to think that …”.
They were also well aware of how their recommendations could be perverted:
“We propose to include the moral sciences in the … examination … Whether this study shall have more to do with mere words or with things, whether it shall degenerate into a formal and scholastic pedantry, or shall train the mind for the highest purposes of active life, will depend, to great extent, on the way in which the examination is conducted.”
And the training was to be thorough:
“[The new recruit] should study [Indian history], not merely in the works of Orme, of Wilks, and of Mill, but also in the travels of Bernier, in the odes of Sir William Jones, and in the journals of Heber. … He should understand the mode of keeping and checking accounts, the principles of banking, the laws that regulated the exchanges … [etc.].”
The Civil Service that developed through the rest of the 1800s was far from stuffy - or at least no more stuffy than was normal for that period. David Price, in Office of Hope, describes the 1909 creation of Labour Exchanges by politicians Beveridge and Churchill:
Charles Rey, the new General Manager, [was] the antithesis of Beveridge, with no pretence to academic qualifications but 'excitable and most practically energetic. We had for ever to be raiding the Treasury for new staff and and premises and Rey, always spoiling for a fight, was the spearhead of our forays.' ... while [Rey's] team was regarded [by other officials] as engaged in 'a great gamble ... a company of pirates' the young men saw themselves as 'high adventurers giving shape to one of the great social reforms of the quinquennium'.
Another novelty was the extent of external recruitment of managers and staff, breaking through the established practice of selection by academic written examination, which tended to favour young inexperienced people ... successful candidates included businessmen, trade unionists, a regular soldier, and an American gold speculator who claimed to have run a labour exchange in Chicago, with a revolver provided as part of the office equipment.
The next major set of reforms came about as a result of the 1918 Haldane Report published at the end of the First World War. Haldane recommended the development of deeper partnerships between Ministers and officials so as to meet the more complicated requirements of busier government as substantial executive ministries emerged from the first world war. The report's impact came through the interaction of two closely-linked ideas:
- Government required investigation and thought in all departments to do its job well: "continuous acquisition of knowledge and the prosecution of research" were needed "to furnish a proper basis for policy". Gone were the days when government Bills and decisions could be based merely on the expertise of ministers, MPs and outside opinion. Ministers could not provide an investigative and thoughtful government on their own. Neither could civil servants; but a partnership between both could.
- The partnership must be extended, however, from the cluster of officials round a minister, typical of 19th century government, to embrace whole departments as the repositories of relevant knowledge and opinion. Haldane did not spell out how such investigation and thought were to be developed, except to recommend they should be based on a split of functions between government departments that essentially has continued to this day.
The relationship between civil servants and Ministers thus became one of mutual interdependence, with Ministers providing authority and officials providing expertise. And that is how it is supposed to work today, undisturbed by a number of proposed reforms, summarised below.
The 1919 Bradbury Report created a new establishment branch of the Treasury (it would now be called 'human resources') to oversee pay and civil service organisation.
Although there has been little if any fundamental reform of the UK civil service since the 1920s, there have been numerous far-reaching efficiency initiatives, summarised here and below, and in later notes in this series. These have led to real improvements in the way in which the big departments are managed, including much faster internal communication and much greater use of the internet for handling transactions with the public.
The 1949 Handbook for the New Civil Servant provides a pretty good portrait of the post-war profession.
The 1955 Priestley Royal Commission led to the Government accepting that Civil Service pay should be determined by fair comparison with the earnings of comparable work outside the Service. Click here for more information about Civil Service pay.
(The Plowden Committee on Control of Public Expenditure: control of civil service pay, conditions and establishments - reported in 1961)
According to Bristol's Rodney Lowe (emphasis added): 'The Plowden committee on the control of public expenditure has been described as a milestone in the modernization of postwar British government. Certainly it effected major changes in both the Treasury's structure and personnel and, by securing the establishment of the public expenditure survey committee, gave subsequent governments the opportunity to plan public expenditure rationally in relation to prospective resources. Ultimately, however, the committee was a failure. The civil service was re-examined by the Fulton committee within five years and public expenditure soon escalated out of control.
The Plowden committee thus represented a major lost opportunity. The time had been ripe for a fundamental political and administrative adjustment to the needs of the extended postwar state; but the committee failed to build the necessary political, parliamentary or public support for its recommendations. The reason for failure was its restricted nature as an internal enquiry with largely ineffectual ‘outside’ members, which enabled vested Treasury interests increasingly to dictate its deliberations. A more open enquiry would have stimulated and brought the best out of the ‘modernisers’ within the Treasury.'
The Wilson and Heath Governments
Harold Wilson was Prime Minister from 1964 to 1970 and then again from 1974 to 1976. He was a great moderniser in many ways, and a fan of technology, and in 1966 tasked the Fulton Inquiry with over hauling the structure of the UK Civil Service.
The background was that the demands created by the 1939-45 Second World War had required substantial change in the civil service, and within Government more generally, including the employment of large numbers of strong characters and experts who would otherwise have remained outside government. This trend was, however, put into reverse after the war but the experience appears to have informed those who wrote the 1968 Fulton Report. They identified the following weaknesses in the Civil Service:
- It was too much based on the philosophy of the 'generalist' or 'all-rounder'.
- Scientists, engineers and other specialists were not being given the responsibilities, opportunities and authority they should have.
- There were too few skilled managers.
- There was not enough contact between the service and the community it serves.
- There was inadequate personnel management and career planning.
The report was taken seriously and led to significant change within the civil service. But it left in place the Westminster/Haldane Model of Government with non-expert 'Mandarins' working most closely with Ministers.
Edward Heath's 1970-1974 Conservative Government imposed pay restraint and other measures which led to the first ever national civil service strike in 1973. This in turn led to the Wider Issues Review whose report Civil Servants and Change was published under the second Wilson Government in 1975. It is an interesting document in all sorts of ways. For a start, it was written in close collaboration with the civil service staff associations acting through the National Whitley Council. There is no overt Ministerial involvement, and not even a Ministerial foreword to the report.
The Whitley Council's own foreword to the report notes that '... the confidence and effectiveness of the Service depend very much on Ministers ... there is a risk of imposing greater burdens and stresses on the Service than it can in practice bear.' The report notes that both pay and 'deeper more complex reasons' had caused the recent industrial action. '... civil servants feel that they have been mucked about a lot in the last five or ten years. So there is an atmosphere of sourness in many parts of the Service, and we have found it at every level.' ... 'Ministers can help ... by recognising their responsibility as employers ... they can avoid discrimination against the public service in the application of their economic and social policies'.
Here are some other interesting extracts from the report:
... the Service has changed in several respects over the years ... Older civil servants joined when recruitment was highly competitive; before the war some schools would inscribe on the honours-board the name of a boy who was accepted into the Civil Service as an executive officer ... But other jobs have become more attractive ... and today the very much larger numbers of executive officer entrants do not regard the Civil Service or themselves as very special ... the majority [of civil servants] do not have traditional white collar attitudes and do not aspire to them.
... the economic rewards must be fair, but they will never be excessive;
- The main text of the report is here.
- The Joint Statement by the National Whitley Council, List of Contents and Annexes are here.
More generally, though, some perceptive commentators felt that Edward Heath thought too much like an official and too little like a sceptical politician and, as a result, both allowed the civil service too much power to prevent things being done, and exaggerated what could be achieved by the official machinery. One example was said to be Heath's unfounded belief in the power of the Department of Trade and Industry to enforce the Industrial Relations Act.
Wilson's post-1974 second government was marked by increasingly angry clashes with the left wing of his party, represented in particular by Secretary of State for Industry, Tony Benn. Senior officials at the Industry Department found themselves caught up in these quarrels and occasionally required Mr Benn to sign Ministerial Directions before they would implement his decisions. But he and his Permanent Secretary parted on reasonably good terms - see their exchange of letters here.
Thatcher Government Reforms
Mrs Thatcher's Conservative Government came into power in 1979. Hugh Stephenson noted, in his book Mrs Thatcher's First Year, that '[her] arrival ... was the biggest jolt that that Civil Service had experienced in living memory. For a while the whole Whitehall system almost visibly juddered ... It was a culture shock. The elite administrative grade had come to think of itself as the guardian and trustee of national continuity .. The Prime Minister and a small group of sympathetic ministers ... were arguing that its ideas and advice had proved bankrupt, that now was the time for an entirely new approach.'
But there was nevertheless no great change in the service's fundamental culture or characteristics. The Whitehall system proved itself able to absorb the changes at the very top, with the staff in the Prime Minister's office in No. 10 Downing Street continuing to serve Mrs Thatcher as they had served her predecessor. Sir Kenneth Stowe as her Principal Private Secretary managed a smooth take-over, while Sir Robert Armstrong, the new Secretary to the Cabinet, appeared effortlessly to take on the role as her most senior closest official adviser. The role of Head of the Civil Service, which had been designed by Fulton to be held by the head of the Civil Service Department, was held by Sir Ian Bancroft. It was to him that the task of “de-privileging” the Civil Service was given, together with that of controlling Civil Service pay and pensions. But the Prime Minister’s Policy Unit, an organisation established by Harold Wilson, but now headed by Sir John Hoskyns and close to Margaret Thatcher, was looking closely over his shoulder.
The Thatcher Government accordingly concentrated on reforming the economy, and institutions outside government, and on improving the management of government. But even the efficiency drive got off to a rocky start. A brief biography of Sir Ian Bancroft records that:
Bancroft's experience and achievements made him a strong candidate to succeed Sir Douglas Allen as head of the home civil service when Allen retired at the end of 1977. But by then the continuing problems of morale and increasing militancy over pay issues in the civil service were having to be handled in a political environment which had become far more critical of all aspects of public administration. On appointment as head of the civil service Bancroft found that the economic situation and policies of the time gave very little room for manoeuvre.
His problems increased with the change of government in 1979. The new prime minister, Margaret Thatcher, was determined to improve the efficiency of the civil service by bringing in private sector methods and by cutting staff numbers. She was very critical of what she regarded as the negative attitudes of senior civil servants when they were made to contemplate new ideas. Bancroft accepted that the civil service would have to be reduced substantially and that the Civil Service Department would have to play a key role in that reduction, but he felt that the management problems with which his senior colleagues would have to grapple as numbers were reduced and posts eliminated were insufficiently understood in no. 10. His attempt to persuade the prime minister that some of her criticisms of senior civil servants were unfair was seen as an example of the weakness which the prime minister believed permeated government departments generally and the Civil Service Department in particular.
He persuaded the prime minister to give a dinner to permanent secretaries at which he hoped that a better understanding might be achieved, if problems could be discussed informally in a relaxed atmosphere. It proved to be a disaster. The explanation of difficulties merely persuaded the prime minister that very few of the permanent secretaries were ‘one of us’, and she ended the dinner unexpectedly early. From that moment Bancroft's position became increasingly untenable.
The Cabinet Office file for April/May 1980 makes fascinating reading. It covers the preparation for the ill-fated dinner and includes the Permanent Secretaries' subsequent 'thank you' letters as well as preparation for a Parliamentary Statement about cuts in civil service numbers.
You might also like to refer to an annotated version of a slightly odd senior officials' 2013 Tribute to the late Baroness Thatcher.
And Douglas Wass' 1983 Reith Lecture The Privileged Adviser is another good read, especially bearing in mind that it was almost certainly seen as the Mandarin's riposte to Mrs Thatcher.
The Financial Management Initiative (1982-) sought important improvements in the allocation, management and control of resources.
Next Steps - Executive Agencies
The Efficiency Unit triggered much welcome change throughout government, initially through Rayner Scrutinies and then very noticeably through the 1987 Ibbs Report Improving Management in Government; The Next Steps (published in 1988) which led to much of the executive work of Government being devolved to Executive Agencies.
The authors of the Next Steps Report identified seven points of diagnosis:
- 95% of the civil service are delivering services; they generally welcome the management changes to date
- senior management is dominated by policy staff with little experience of service delivery
- senior civil servants are ruled by ministerial and parliamentary pressures
- Ministers are overloaded and inexperienced in management
- Departments still focus upon activities and not on results
- there are insufficient pressures to improve performance
- “the Civil Service is too big and diverse to manage as a single entity”.
and recommended that:
- “agencies should be established to carry out the executive functions of government within a policy and resources framework set by a Department”
- “a full Permanent Secretary should be designated a ‘Project manager’ to ensure that the change takes place”
- “there should be clearly defined responsibilities between the Secretary of State and the Permanent Secretary
on the one hand and the Chief Executive of the agency on the other”.
Diana Goldsworthy's Setting Up Next Steps and the House of Commons Report The Accountability Debate: Next Steps Agencies include later perceptive analyses of the background to, and creation of, Executive Agencies.
Sir Geoffrey Holland praises the creation of Agencies, but offers a pretty scathing review of privatisation, market testing and contracting out, at pp 45-46 of his 1995 lecture "Alas! Sir Humphrey, I Knew Him Well".
A broadly supportive review of executive agencies (Better government services: Executive agencies in the 21st century) was published in 2002. As ever, however, it found that the civil service did not value 'delivery'. The report noted that 'Delivery experience is rarely found at the heart of departments. Departments’ leadership structures must be built around the skills and experience of delivery as well as policy in order to plan and manage all aspects of achieving outcomes for customers.' And 'The different skills needed for excellence in policy advice and in service delivery are not yet valued equally. Mutual support is essential and both are integral to achieving outcomes effectively. Departments and agencies must work together to bridge the gulf between policy development and implementation and to fill high-level skills gaps in departments and agencies.'
Career Management and Succession Planning Study: The Oughton Report
This influential 1993 Efficiency Unit report led to significant improvements in the career management of senior officials whilst challenging some Ministers' prejudices - for instance that it would be good to recruit much more from the private sector: 'The clear evidence of the private sector is that developing staff from within is right in most cases; recruiting from outside necessarily involves higher risk as well as expense'.
It also challenged Permanent Secretary's preference for promoting those officials who had done lots of different jobs rather than those who had built a more focussed career: 'In appraisal more emphasis must be placed on proven achievement, rather than on the range of posts held. Over 60 per cent of the Senior Open Structure have been in their current posts for 2 years or less.'
Comment: Over 20 years on, there is little sign that these two of Oughton's criticisms had been taken seriously. There is still considerable Ministerial pressure to recruit from outside the civil service, whilst the 'four year rule' ensures that most senior officials move far too frequently.
Ministers and Mandarins
This influential 1994 IPPR report, written by William Plowden, was published half way through John Major's steadily weakening premiership. Addressed, in effect, to an incoming Labour government (which arrived in 1997) Plowden reported that 'the civil service as a whole [was] in a state of crisis unprecedented in its history ... the government ... does not regard them as indispensable ... A significant number of senior officials feel that their professional skills are being ignored or abused.'
Plowden noted that the Oughton Report (see above) could be the springboard for many worthwhile changes including recruitment from outside the SCS to counter 'the slow disappearance, without replacement, of the gifted outsiders brought in during the war'. To guard against this damaging the essential apolitical professionalism of the civil service, he also recommended the introduction of a statutory 'code of behaviour' - and the Civil Service Code was indeed promulgated in Continuity and Change - see below - and introduced in 1996.
Key extracts from the report are here.
Continuity and Change
- the delegation of further management flexibility and freedoms, including delegation of pay and grading decisions, to individual departments
- the establishment of the Senior Civil Service, and greater use of recruitment from outside the civil service ('open recruiting') and more flexible remuneration arrangements at senior levels.
- the promulgation of the Civil Service Code, and
- an enhanced role for the Civil Service Commissioners in recruitment and selection on merit.
Sir Geoffrey Holland's lecture "Alas! Sir Humphrey, I Knew Him Well" is a good entertaining read and a useful review of many issues that faced government and the civil service in 1995.
The Institute for Government's 2012 report Reforming the Civil Service provides an excellent description and review of the work of the Efficiency Unit in the early 1980s, and of the Next Steps Report.
And do read Note 2 in this series to see what happened after 1997.