Statistics and Open Data: Harvesting unused knowledge, empowering citizens and improving public services - Public Administration Committee Contents

Conclusions and recommendations

Barriers to accountability

1.  Stephan Shakespeare's proposal that the Government should adopt a "twin-track" approach to data release is a practical and realistic way of maintaining the momentum on open data, which recognises that "the perfect should not be the enemy of the good: a simultaneous 'publish early even if imperfect' imperative AND a commitment to a 'high quality core'". Regular publication of imperfect government data will provide Departments with a powerful incentive to improve it. We recommend that the Government should adopt the twin-track approach to data release advocated by Stephan Shakespeare. Government should 'publish early even if imperfect', as well as being committed to a 'high quality core'. As long as Government is clear about its limitations, there will always be a role for data that is imperfect but improvable. (Paragraph 16)

2.  It is very difficult to assess the performance of Government in enhancing accountability through opening up its data. The concept of open data is poorly defined and there are no accepted measures of what is published. This allows supporters of open data to claim the revolution is well under way and the sceptics to say nothing has changed. (Paragraph 18)

3.  It is often pointed out that more than 13,000 datasets can now be found on, but it is unclear how many of these represent simple republishing of data already published on other government sites. Some data sets are small and others large. And it is possible for departments to get more data out by publishing it in smaller bundles or updating it more frequently, in such a way that there is little or no extra public benefit. In these circumstances, measuring progress on this important agenda is difficult if not impossible. Simply putting data "out there" is not enough to keep Government accountable. (Paragraph 19)

4.  We invite the Government to publish a clear list of open data, indicating when each data series became open in each case. (Paragraph 20)

Procurement and Open Data

5.  Open data principles should be applied not only to government departments but also to the private companies with which they make contracts (Paragraph 28)

6.  We recommend that companies contracting with the Government to provide contracted or outsourced goods and services should be required to make all data open on the same terms as the sponsoring department. This stipulation should be included in a universal standard contract clause which should be introduced and enforced across Government from the beginning of the financial year 2015-16. (Paragraph 29)

The right to data?

7.  There is confusion about the concept of the 'right' to data held by Government. On the one hand, the Minister told us that there is no right to data, but there is evidence to suggest that, in effect, a presumption already exists that government data will be published in an open format. (Paragraph 37)

8.  The Government needs to recognise that the public has the inherent 'right to data', like Freedom of Information. The Government should clarify its policy and bring forward the necessary legislation, without delay. (Paragraph 38)

Privacy and open data: managing the risks

9.  When releasing data, it is the responsibility of Government to avoid risk that individuals may be identified against their will. There has been an effective campaign to highlight unease about the release of anonymised NHS patient data for academic and pharmaceutical research as part of the programme. There is a clear need to reassure the public about personal privacy. However, it is also important to explain what open data can do to make public services more accountable and responsive to the needs of society. The recent controversy over demonstrates the danger that concerns about privacy will unduly undermine the case for open data (Paragraph 47)

Increasing engagement

10.  There is no sign of the promised emergence of an army of armchair auditors. There is little or no evidence that the Cabinet Office is succeeding in encouraging greater public engagement in using data to hold the public sector to account. (Paragraph 54)

11.  Open data is important and touches people's lives at many points. Yet Government and some of the experts sometimes make too much use of jargon and so can alienate and confuse people who do not have expert knowledge of the technical terms. This can undermine efforts to encourage more people to get involved in holding Government to account. (Paragraph 55)

12.  The Government should adopt a star-rating system for engagement, as recommended by Involve, for measuring, and reporting to Parliament on, Departments' progress on increasing accountability through open data. The Government should expect Departments to set out plans to move towards Five Star Engagement for all their data releases. (Paragraph 56)

General conclusions on accountability

13.  We welcome the clear lead on open data that has come from successive Governments. There have been some useful moves to improve accountability and engagement in recent years, with positive developments such as the establishment of the Open Data User Group. However there is much still to be done. (Paragraph 57)

14.  There should be a presumption that restrictions on government data releases should be abolished. It may be necessary to exempt certain data sets from this presumption, but this should be on a case-by-case basis, to provide for such imperatives as the preservation of national security or the protection of personal privacy. (Paragraph 58)

15.  The Cabinet Office must give a much higher priority to ensuring that more interesting and relevant data is made open, and that the release mechanisms encourage people to use it and, where appropriate, hold Government and local authorities to account. Beginning in April 2014, targets should be set for the release of totally new government datasets - not the republishing of existing ones (Paragraph 59)

Understanding the data marketplace - the two cultures

16.  The Government has set up or supported a number of initiatives and bodies which are intended to help UK business make the most of public sector open data. It is too early to say how effective they will be, but there is evidence that their work will be hampered unless Government acquires a better understanding of trends in the rapidly-moving marketplace for open data, where international competition to realise the economic benefits of key datasets is increasingly fierce. (Paragraph 68)

Charging for data

17.  A radical new approach is needed to the funding of government open data. Charging for some data may occasionally be appropriate, but this should become the exception rather than the rule. A modest part of the cost to the public of statutory registrations should be earmarked for ensuring that the resultant data - suitably anonymised if necessary - can become open data. Data held by the Land Registry and car registration data held by DVLA and, indeed, held by the NHS are among relevant examples. (Paragraph 75)

The value of core reference data

18.  Some government datasets are of huge direct value to the economy. Ministers and the Royal Mail have made a number of promises about the continued accessibility to small businesses and others of the Postcode Address File (PAF). Evidence we have received casts doubt on the credibility of such assurances. The Postcode Address File (PAF) was included in the sale to boost the Royal Mail share price at flotation. This takes an immediate but narrow view of the value of such datasets. The PAF should have been retained as a public data set, as a national asset, available free to all, for the benefit of the public and for the widest benefit of the UK economy. Its disposal for a short-term gain will impede economic innovation and growth. This was an unacceptable and unnecessary consequence of privatisation, and is at odds with the Minister's general argument that open data should not be "swallowed up [...] by big global companies." (Paragraph 88)

19.  The sale of the PAF with the Royal Mail was a mistake. The Government must never make a similar mistake. Public access to public sector data must never be sold or given away again (Paragraph 89).

Ensuring fair access

20.  There is concern about the attitudes of the research councils, and academic researchers in general, to government data. The government needs to make the case for giving privileged academic access to the new government data, when it should be more widely available. It has, after all been funded by tax payers. (Paragraph 93)

General conclusions on open data and economic growth

21.  The UK Government was an early mover on government open data, but other Governments, watching the UK with interest, are catching up fast. If the Government does not take the opportunities offered, there is a risk in the UK that businesses with growth potential will be deterred by fees for data, and by legal and administrative barriers, while other countries are developing their data industrial base and stealing a lead over the UK. It is short-sighted in the extreme for Government to seek to maximise fee income from data while those fees penalise in particular small companies that can prove the most innovative, and which could establish the UK as global leader in this new economic sector. (Paragraph 94)

22.  Core data needs to be released fast and, above all, free so that businesses (for example apps developers) can use it along with other data to make progress. To this end the Government should in particular pledge that the data held by GeoPlace LLP, a company owned by Ordnance Survey and the Local Government Association, will remain in public ownership. (Paragraph 95)

23.  Departments should be required to list all the surveys conducted and administrative systems in operation to allow the public to see what data might be produced, and should provide to Parliament and the public a prompt and clear account of all revenues from any data sale. (Paragraph 96)

24.  The Government must work closely with business and nurture new open data enterprises by providing the environment they need to grow. The Open Data Institute is a welcome recent development. It has worked to help develop some start-up businesses based on open data and has been a hub for knowledge, but its impact is far from clear and now needs to be felt more widely. (Paragraph 97)

Missing opportunities to improve effectiveness

25.  There is little evidence to suggest that the Government is consistently making the most of the opportunities to improve policy and performance via the use of its own data. Departments need to make full use of the records and information they possess to ensure they are running effectively. Opening up that data to other departments will boost the Government's evidence base and can improve policy making. The benefits of making data open include not just an increase in openness and accountability, but also the opportunity for outside experts to verify, and suggest improvements in the quality and accuracy of, the data itself. (Paragraph 103)

26.  The Cabinet Office should be much more active in ensuring Departments maximise the social and economic potential of open data, not least in increasing their own efficiency and effectiveness. To this end, it should:

a)  require Departments to produce, by the end of 2014, a detailed and timetabled plan for using data to enhance their performance,

b)  ensure that the data which is used to underpin policy work in all public announcements is published alongside the policy statements, and

c)  bring an end by January 2015 to the outdated and wasteful practice whereby Departments have to pay for access to data that has been produced by Government itself. (Paragraph 104)

27.  The Office for National Statistics, directed by the UK Statistics Authority Board, should also be at the forefront of this movement and showing the way forward by example. (Paragraph 105)

The role of statisticians

28.  Many civil and public servants lack the skills to interpret data properly and some civil servants do not seem to share the Government's desire for openness. While bearing fully in mind the needs of national security and personal privacy, civil servants need to be much more aware of the presumption to publish. They should stop being gatekeepers, guarding government data, and become enablers encouraging its wider use; key to this will be the development of a wider understanding of data issues among policy staff.(Paragraph 116)

29.  Government statisticians have the skills to do much more with government data, for example, through producing new series of statistics. But statisticians have chosen to adopt a low profile when they need to be active in producing new data sets and collaborating with their colleagues in other Civil Service professions to bring more sense and usability to open data initiatives. Government statisticians should become champions of open data.(Paragraph 117)

30.  We recommend above that the Government adopt the "five-star" system along the lines proposed by Involve, for open data engagement. A second "five-star" rating system, developed by Full Fact for assessing the usability of government statistics, would support the efforts of statisticians to play a more active role in open data. This system should also be adopted by the Cabinet Office in assessing departmental progress on open data. (Paragraph 118)

31.  The Government needs to move fast to encourage training of more data scientists. We therefore recommend that the Government should bring forward a practical timetable for training data scientists, with target numbers, to be announced before the end of July 2014. The Government should also include data skills and open data awareness sessions in the training of the policy profession in the Civil Service. (Paragraph 119)  

Who is responsible for making the Government's open data plans work?

32.  There is much to be gained from open data, but the Government's direction of travel is not clear. The ODI has set out some important elements of a more considered and coherent approach, and it is a foundation for the Government to use as the basis for further work towards a strategy. The National Action Plan provides little incentive for the wider public to get involved in open data.(Paragraph 138)

33.  There has been a lack of coordination on open data at Ministerial and official level, though this is improving. No clearer indication of the lack of strategy on open data is required than the inconsistency of the decision to sell the Postcode Address File with the Royal Mail. The Cabinet Office leads on the policy, but its mechanisms to hold Departments to account are weak. The sale of the PAF with the Royal Mail demonstrates that important Departments such as the Treasury and BIS do not appreciate the value of open data. Despite the enthusiastic rhetoric emanating from the Cabinet Office, our evidence indeed indicated something more serious - a lack of understanding of open data among most Ministers and apparently most officials. (Paragraph 139)

34.  The Information Commissioner described how the public sector's commitment to the Freedom of Information Act slackened over time. Under present arrangements, it is all too possible to foresee a repeat of this experience in the case of open data, with the issue slipping gradually down the list of public sector priorities as apparently more pressing matters come to the fore. The stakes on open data are arguably higher than those on freedom of information, and the UK has great opportunities if it gets it right - but Government needs to take a determined lead or the opportunities will slip away. (Paragraph 140)

35.  There is an unwieldy plethora of open data bodies which tends to slow both decision-making and consultation. The structure of the government web sites also make it very difficult to see what government policy is towards open data, and to identify the progress being made (Paragraph 141)

36.  To overcome departmental apathy and resistance, open data needs to be treated as a major government programme in its own right, which will only bring substantial benefits if it is subject to active leadership and management by Ministers and officials. The Minister for the Cabinet Office should be given explicit responsibility for all aspects of open data policy, including the commercial aspects. We believe that Civil Service accountability for progress needs to be much clearer, and that the Cabinet Secretary should be given the overall responsibility for pushing open data through Whitehall and beyond. A single Senior Responsible Owner should be appointed at Deputy Secretary level in the Cabinet Office, to be directly and personally responsible for delivering the benefits of the open data strategy. The Public Sector Transparency Board is too large to be effective in driving progress. A small group from that Board should work as a Programme Implementation Board. (Paragraph 142)

37.  The Government should, by the end of June 2014, submit to the Committee a detailed report on progress on the actions related to open data. This should include a list of all plans and actions from recent relevant documents, reports and committees on open data, including but not limited to the Open Government Partnership Action Plan and the National Information Infrastructure. The Cabinet Office should report to Parliament at least every six months on progress made with a consolidated list of actions. (Paragraph 143)


38.  Today there are unparalleled opportunities to harvest unused knowledge that otherwise goes to waste, which can be used to empower citizens, to improve public services and to benefit the economy and society as a whole. (Paragraph 144)

previous page contents next page

© Parliamentary copyright 2014
Prepared 17 March 2014