Publications on the internet
Science & Technology - Minutes of EvidenceHC 1538
Taken before the Science and Technology Committee
on Wednesday 26 October 2011
Andrew Miller (Chair)
Examination of Witnesses
Witnesses: Professor Paul Hardaker, Chief Executive, Royal Meteorological Society, Professor Ed Hill OBE, Director, National Oceanography Centre, and Professor Alan Thorpe, Director General, European Centre for Medium-Range Weather Forecasts, gave evidence.
Q1 Chair: For any members of the public who are a little confused about the rapid change of subject, there is no connection between our previous set of witnesses and our next set of witnesses, who are here to help us in our exploration of the effectiveness of the Met Office. I would be grateful if the three new witnesses could, first of all, introduce themselves.
Professor Hardaker: I am Paul Hardaker. I am the Chief Executive of the Royal Meteorological Society. I am a visiting professor in the Department of Meteorology at the University of Reading.
Professor Hill: I am Professor Ed Hill. I am the Executive Director of the National Oceanography Centre, which is owned by the Natural Environment Research Council.
Professor Thorpe: I am Alan Thorpe. I am the new Director General of the European Centre for Medium-Range Weather Forecasts.
Q2 Chair: Before we proceed, I have to declare a small interest in that my daughter is employed by the National Oceanography Centre. There is inherent uncertainty to any "forecast". How often can the Met Office and other organisations be expected to get their weather forecasts right?
Professor Thorpe: The skill of forecasts depends how far into the future the predictions are made for. We make an assessment of how accurate weather forecasts are using a set of metrics compared with what actually happens. We track, collectively as a meteorological community, the accuracy of the forecasts. The distance that you can predict into the future has been advancing at about a day per decade. For example, a five-day weather forecast today is as accurate as a three-day forecast was 20 years ago. This demonstrates the development of the science, computational power and observations of the atmosphere and oceans that go into making up a weather forecast. Which aspects of the weather you can predict depends on which aspects you are talking about and on what time ranges.
Professor Hardaker: Some of the things that define the improvements in accuracy are the quality and amount of observations that you have, how appropriate they are for the forecast and the resolutions of the model that you are using. Most of the developments that we have seen, that Alan was talking about, in terms of decade on decade, have been the result of more resolution, which has come with more computing resources, better data and better use of that data with the models.
Q3 Chair: One of the frustrations of living in the north-west, as I do, is that when one listens to the television forecast it tends to be a bit London-centric, like many things. Do forecasters take into account local factors that influence weather patterns? What level of local or regional information is it reasonable to expect national forecasters to provide, or is it just that the media are very selective in what they take off you?
Professor Hardaker: It is very much the case that you would expect forecasters to be building into their forecasts local knowledge about the nature of the area, the geographic features and how that will affect the weather. That is very much taken into consideration. The other significant thing we have seen in the Met community over recent years is that the Met Office models have now come down to a much higher resolution. That has happened only in the last 12 to 24 months; so the ability to forecast at that high resolution has been very much improved. It sounds a trivial thing to say that we take something that is, say, a 10 km grid and bring it down to 1 km-that is just more grid points-but, in doing that, a lot of science goes into being able to predict individual features like thunderstorms at high resolution. That move down to this high resolution is probably as big a step as the introduction of some of the early modelling capabilities that we had many years ago. Our ability to forecast at a local level is much improved now.
Professor Thorpe: The European Centre is now forecasting globally and the effective resolution of the model is about 16 km globally. The Met Office, for example, has a fine scale model that is embedded just over the UK. At the moment it is 4 km resolution, and they are looking to improve that down to about 1½ km. As Paul has said, over the years more and more local specificity in terms of the forecast is available, but also there are more local observations as we brought in new technology, such as radar measurements, which are able to define more accurately the starting conditions for these forecast models.
Professor Hill: That is about the weather. Of course, the weather has impacts and some of those relate to the seas, in particular storm surges, which can be generated by weather conditions. It is very important to be able to forecast the impacts of those locally. Coupling storm surge models of the right scale with the right resolution enables one to get forecasts of some of the impacts of that weather on events like storm surges.
Q4 Chair: To what extent does the Met Office use forecasts from the European Centre for Medium-Range Weather Forecasts and other groups? Is it common for different groups to contradict each other?
Professor Thorpe: Perhaps I had better pick this one up. The European Centre for Medium-Range Weather Forecasts exists to provide forecasts in the time range from three days ahead to 10 days and beyond. The Met Office, like many countries in Europe that are part of this consortium, gets these forecast products which complements what the Met Office’s own models are producing. We provide a complementary part of what the Met Office needs to look at the slightly longer range, going forward, and the forecasters regularly use ECMWF’s predictions. They also see the forecasts from other international forecasting offices, such as the United States, Japan, Australia and so on. The advantage of having this range of predictions is being able to look at the risk that any one model is more or less extreme. By having that range of models, you can start to assess what the most likely weather will be but assess the risk of it being more or less extreme. Having this range of rather competitive quality models is very helpful to the Met Office in being able to forecast most accurately.
Professor Hardaker: Meteorology is a very collaborative science, so it is quite common for the national met services in different countries to share the data and model information with each other, because the heart of it is about providing warnings for the protection of life and property. By exchanging all of this information, we are able to inter-compare with each other and it is a valuable part of that process. In addition, in the UK, most people will probably have heard of the Met Office and they will have seen the forecasts on the BBC, but they may not know that there are over 30 private sector providers in the UK. From time to time, it is not uncommon that among that broader community you might get some contradiction in the forecasts.
Q5 Chair: In terms of the sharing of data, one point was made to me by a research fellow who was paired with me under the Royal Society’s Pairing Scheme this year-I was out in Liverpool Bay with him-and we were talking about one of the wind farms. I said, "What’s that mast over there?", and he said, "That’s their data collection point." I asked him, "Do they share it with you?", and the answer was "No." Is that common? Are there data sets out there that you ought to get your hands on to help improve your capacity to do your job?
Professor Hardaker: It is not common to my knowledge that there are data sets out there that are not exploited, but others might know.
Professor Thorpe: There is a global collection of measurements of the atmosphere and oceans that is taken every day and is shared by all countries that are within the world meteorological organisation network. There is a tremendous international co-operation in sharing the raw observations.
Q6 Chair: This is private sector information.
Professor Thorpe: Okay. Those data-the raw observations-are, I believe, freely available1.
Chair: That is interesting.
Professor Hardaker: I was looking at some statistics, trying to estimate broadly what percentage of data is collected by the Met Office compared with what they use. About 2% or 3% of what is used in producing these forecasts is collected by the Met Office. The rest of it is exchanged with other organisations. A tremendous amount of leverage is going on as part of that process.
Professor Hill: The private sector certainly contributes to the collection of weather data. For example, out in the oceans, ships of opportunity regularly collect met data. It is an important part of the data stream. It would not be possible without them. They do contribute.
As to the particular case of the wind farm that you were talking about, I do not know whether that data are available via the Met Office or not, but it would appear that it was not directly available to the researchers concerned. That is not to say that it is not available indirectly.
Q7 Stephen Mosley: May I ask about seasonal forecasts, because I know that Philip Eden, who is a former Vice President of the Royal Meteorological Society, has asked: "Are they a public service or entertainment?" What are seasonal forecasts for?
Professor Hardaker: Seasonal forecasts are used for many things. We tend to think of their value in a UK context, but the nature of how seasonal forecasts are produced means, at the moment, that they are much more useful in certain parts of the world like the Tropics where they have much more inherent skill. As you come towards our latitudes, the skill level does tail off. That is, in large part, because we have these weather systems in the mid latitude that can change the seasonal characteristics quite markedly. We have had a good example of that in the past couple of weeks where we had this high pressure that was blocking the weather patterns coming in and changed the nature of our weather for two or three weeks. Because of this they do have less skill in our latitudes. We often forget that seasonal forecasts are used quite a lot around the world in British interests: in defence, international development, aid programmes and trade events in which we are involved. They have a significant value in those activities.
In the UK they are valuable provided the people who are using them-the end users-understand the limitations for our latitudes and how to use them. They tend to be more probabilistic in nature. That can be harder for people to interpret and understand without some clear guidance on how to make use of that information. They are much more than entertainment value, for sure.
Professor Thorpe: It is not a static situation. We are doing a lot of research internationally on exactly what is predictable on that seasonal time scale. Predictability comes from patterns, for example, in sea surface temperature, which influences the atmosphere not only locally but globally. For example, one big source of potential predictability comes from the El Niño phenomenon in the Pacific, which is a big change to the sea surface temperature. That affects weather patterns around the world. As Paul says, it does not necessarily mean that it affects north-west Europe particularly to give predictable signals. A lot of research is going on to understand those remote connections and whether they allow us to be able to predict on seasonal time scales. It is not a static position. It is something that the research is developing all the time.
Professor Hill: They certainly are more than entertainment. If one could have reliable seasonal forecasts, the potential is enormous. The kinds of users would be everything from the insurance sector, the power generation industry, construction, agriculture, tourism, the retail industry-understanding what products to put on the shelves at what time-manufacturing and transport. The potential is enormous. Particularly where large investments are at stake, any information that can add some level of insight into what is going on on those time scales is worth having. In particular, if you are into activities which are a little akin to betting, then something that is a good deal better than evens may well help out with those investments. I am thinking of some of the insurance business in that respect. This is why it is a really important area for research, to see if one can improve the skill and be able, as Professor Thorpe has said, to try and understand how we can do this in some of the more difficult regions of the world, including our latitudes.
Professor Hardaker: The interesting conclusions that we have come up with in fairly recent times in the research community is that resolution, although it is not the whole answer, is going to help us a lot in terms of understanding some of these global connections, both in the oceans and the atmosphere.
Q8 Stephen Mosley: Can I pick on something you said in your first answer about there being less skill at our latitudes? Did you mean less skill, or did you just mean that it is a lot more difficult to predict in our latitudes?
Professor Hardaker: We are looking at patterns and relationships. Those patterns and relationships are much stronger in certain parts of the world than they are at our latitudes. In the meteorological language, you call it a signal. You are looking for a signal above the noise. The signal is much weaker at our latitudes for some of these connections.
Q9 Stephen Mosley: There is a perception among the general public that the Met Office’s seasonal forecasts are not all that reliable. We all remember 2009, was it, when they were predicting a barbeque summer and it poured down with rain pretty much the whole time? You have explained the difficulties at our latitudes. Are there any organisations, for instance, the ECMWF, that provide better seasonal forecasts than the Met Office?
Professor Thorpe: We are working in collaboration. There is a project that the Met Office, ECMWF and some other met services are involved with called EUROSIP, which is a seasonal prediction project, to bring together seasonal forecasts from our model, from the Met Office’s model, from Météo France and from others as well. We are trying not only to inter-compare the predictions from different systems, but there is benefit in having that range of models because it gives you a more risk-based prediction. From time to time, some models will be better in certain regions. By having a range of models, you can get a better feeling for the risk of certain outcomes. There is regular interaction and collaboration on this, but I emphasise the fact that this area of forecasting is still emerging and a lot of research is still being done on it. Whatever state we are in at the moment in terms of the skill of those models and forecasts, the potential exists for this to improve as research results come on stream. Whatever the position is today in terms of the relative capability, it is an area that, I am hopeful, can improve in the future. We will have to wait and see what the research shows.
Q10 Stephen Mosley: Is the effort going in to make this happen?
Professor Thorpe: Absolutely. A big effort is going on in the research community, in the UK, internationally and among the major meteorological services, including ECMWF and other major centres, such as Canada, United States, Japan, France, Germany and the UK. Quite a bit of activity is going on.
Professor Hill: You prefaced your remarks by referring to the public perception. This is an issue that is both about the science needed to improve seasonal forecasts and an issue about communication. Many of these seasonal forecasts are offering probabilities such as a 60%-a two in three-chance of this kind of event happening. That is useful for some people for some industries. When the third event does not happen, that is very easily misinterpreted as, "They got it wrong". It was a two in three chance of the circumstances arising. One does have to understand the situation in those terms. For example, if someone offered that they could give you a chance that two out of every three horses that you bet on would come in as a winner, you would, probably, take that pretty seriously, and you would feel a bit churlish, if that was indeed the success rate, to go and complain about the one third of your horses that were predicted to win but did not. That is the kind of setting. You need to understand it. Where sophisticated industries are prepared to operate at those levels of probability, they will not perceive the situation in quite the same way.
Q11 Stephen Mosley: The probability issue is something we have picked up in previous inquiries. It might be worthwhile our taking that off separately afterwards for consideration.
Professor Hardaker: May I come back to your question about whether one is better than the other, because there are a number of organisations out there providing seasonal predictions? A number in the private sector do that as well. One of the challenges of assessing whether one technique is better than the other is that you do not have many seasonal predictions. You only get four a year. The spring and autumn ones tend not to be as reliable as the summer and winter ones. We do not get many and it takes a long time to collect some sensible statistics to do a comparison that has any real science in it which says, "This method is better than that one." It is a long haul in understanding where significant improvements are coming in this area.
Q12 Roger Williams: We have heard already how important it is for private individuals and commercial organisations to have some indication about extreme weather that they may be going to experience. How robust are the forecasts of extreme weather events as done by the Met Office?
Professor Hardaker: One of the challenges of extreme weather events is, thankfully, that they happen far less frequently than much of our weather. You have less hazardous weather to deal with and, therefore, fewer case studies that help you improve your prediction in these areas.
The point we mentioned before about improving the resolution of the models, to make them much higher resolution, is helping a lot in terms of improving the capabilities to forecast hazardous weather. It is a priority for the Met Office science programme. I have seen some research runs of the model looking at the Boscastle floods that show that with the very high resolution models you can pick up the convergence and the formation of the rainfall much easier. That is not to suggest that we will capture all such cases, but at these higher resolutions we are more likely to be able to pick up these local features that can often cause significant weather events.
As you move to dealing with these significant weather events, you have to do two things. The first is that you become more probabilistic in terms of what you are trying to provide in terms of information. There is a deal of uncertainty that needs to be represented in a probabilistic fashion. The second is that you need to work closely with other agencies. It is not just about the Met Office being accurate in terms of its forecast unless it is joined up with the other agencies involved that are issuing those warnings. An example is the bringing together of the National Flood Centre with the Met Office Operations Centre. That is a good example in terms of getting those warnings more effectively and efficiently out to the public. Again, there is a close relationship between the Met Office and the Highways Agency, which is also helpful for those cases that are impacting on the road network. It is a real partnership in predicting and communicating hazardous weather.
Professor Thorpe: I would entirely agree with that. It is important to say that the Met Office is one of the world-leading weather services in the world in terms of the skill of its forecasts from extreme weather right up to other aspects of the weather that we have been talking about. There is regular intercomparison of the international and national weather services, and the Met Office is one of the leading met services. Therefore, the general answer to how good their forecasts are is that they are as good as anybody else in terms of being close to world leading. It is improving all the time as well, as I was trying to say at the beginning. Of course, we would like the forecasts to be better. As the computational power, the observations and the models improve, it is improving. As Paul said, it is something that the research is focusing on, because it is critical for the public.
Q13 Roger Williams: I sat on a previous inquiry into the flooding that we experienced in this country a couple of years ago. One of the criticisms then of the Met Office was that it was not well integrated into the Environment Agency and other agencies. Has that integration been achieved? Is it working well?
Professor Hardaker: It is early days. My perception from the outside is that it is working very well. The establishment of the new National Flood Forecasting Centre, which is a partnership between the Environment Agency and the Met Office and bringing all that together into the operations centre in Exeter, has had significant benefits. Sitting hydrologists and meteorologists next to each other has been a real positive. The Met Office and the Environment Agency have come to the Society to ask us if we can provide help and support to them in terms of professional development, to give a development path for those working in hydrology to come through to gain professional charter status in the meteorological qualifications and vice-versa. That is a real positive and there is a real ambition to do more with it.
Q14 Roger Williams: Are there any limiting factors within the Met Office as to future improvements in forecasting extreme or hazardous events?
Professor Thorpe: There are limitations for all centres in terms of the ability to have the computational power to analyse the increasing volumes of measurements and also to make the predictions using the numerical weather prediction models. It is a challenge for all centres, including mine and the Met Office, to keep competitive in terms of the amount of computer power that we have to devote to this. In many respects, we can have more finely resolved models and, therefore, a better description of smaller-scale events, such as extreme weather, if we have the best and most powerful computers. Of course, this comes at a cost. There is an issue about being able to afford the most powerful computers. Meteorology and weather forecasting has been a prime driver of super-computer capability worldwide. It remains a challenge to be able to afford the amount of computational power we need.
Q15 Roger Williams: It is the computation of the data rather than the collection of the data that is the more limiting factor.
Professor Hardaker: I would say that it is a mixture of the two.
Professor Thorpe: It is a mixture. There are many more observations than there used to be, particularly because of the satellite component. We are getting access routinely in the met offices to the majority of those data. Of course, then it is a matter of analysing those data and doing the weather forecasting. That is where the computational power comes in. As Paul says, we need both. I was just highlighting that to give you an example when you asked where the limitations are, and that is certainly one of them.
Professor Hardaker: The lack of computing resources is by far a greater limitation than the lack of data in the current environment, although there are some areas, particularly in oceanography, which Ed might want to say something about, where we are a bit too data- sparse. We have fallen a long way behind the curve in terms of the computer capability keeping pace with the science. The problem as well is that it does not scale linearly. Every time we increase the resolution, we must do that in three dimensions. There is a time element to this as well. It is an exponential increase in the requirement of computing power.
Professor Hill: I would like to comment on ocean data. It is probably more relevant not so much for extreme and short-term events as it is for longer time scales, such as climate time scales and seasonal forecasting that we were referring to before. Ocean observations, particularly on those longer time scales, to which you might want to return, is an important limiting factor and will be going forward.
Q16 Roger Williams: Professor Hill, in your written contribution you stated that the insurance and re-insurance industries benefit from robust forecasts of extreme events. Do they contribute to the cost of those forecasts in terms of data or computational processes?
Professor Hill: I do not know the details of what they do and don’t contribute in terms of observations. It would be pretty fair to say that it is very unlikely that they contribute significantly to the underpinning observations. There is a much wider issue here. You picked on one particular sector, but the beneficiaries of weather information, whether it be on short time scales right out even to climate time scales, are numerous and varied. Some of the beneficiaries probably do not even know that they are benefiting and possibly care even less about where the data are coming from. We probably have a situation where, if left to those who directly benefit, you would end up with a classic market failure if you were going to expect them to pay. Indeed, you can very rarely, if ever, draw a golden thread from some particular measurement system to a particular forecast that has some indirect benefit. This is an area where, traditionally, the public purse is called upon to provide the underpinning datasets, on the understanding that there is benefit, including real economic growth benefit, somewhere downstream from the direct observation. That is ultimately where the cost is recouped from through taxation and so forth.
Q17 Roger Williams: The further you can look forward, the greater mitigation you can put in. It is not just the insurance industry, is it?
Professor Hill: Absolutely. It is health, retail, transport, energy and the whole thing. Indeed, the further one can forecast ahead, the greater is the ultimate prize. Clearly, there is a lot of economic benefit to be had if those forecasts are reliable and actionable.
Professor Hardaker: A lot of focus is going into decadal prediction at the moment. That is the time scale where some of these big capital expenditure issues are most important. Replacing the capital asset and how you design and equip that is a big challenge.
Q18 Graham Stringer: Professor Hardaker, I will read a short sentence from your submission: "However, for climate prediction and its application to Climate Change Risk Assessment (CCRA), there remains a fundamental unsolved question of whether the estimated" United Kingdom Climate Projection 09 "probabilities are actually reliable (for example, does an estimated 90% probability of an event mean the event is somehow very likely to happen?)." Whatever does that mean?
Professor Hardaker: The point we were trying to make there is that the way the predictions were produced this time was based on variations of a single model with a statistical generator of weather associated with it. We think a better approach going forward for the next generation of these scenarios might be to look at a combination of a range of models that are representing the different physics of what is going on in the atmosphere, and that will give you a more realistic representation of what the probabilities are rather than using just a single model. I agree that the language is a bit confusing.
Q19 Graham Stringer: I am no statistician, but it says "an estimated 90% probability". Does that mean it is likely to happen? I would have thought it was self-evidently true that it was.
Professor Hardaker: The point is about how you are creating that probability distribution. Are you doing that in the most efficient way that is representing the full range of probabilities of what you are trying to get to the bottom of? I can create a probability distribution by tossing a coin and getting heads and tails. The more coins I use or the more intricate method I use, the more chance I have of sampling the full range of probabilities. Have we really exploited our current understanding, knowledge and models to represent the full range of probabilities?
Q20 Graham Stringer: Is it fair to say that you are looking to put together more models in order to get a better prediction?
Professor Hardaker: More models and, perhaps, a more robust way of creating our probability distributions than we did last time round. To say "last time round" sounds a bit over-critical of UKCP09, but those model runs were the first time we moved to probabilistic predictions. That was a major step forward and we learned a great deal from that process. This time round we can move that on more significantly.
Q21 Graham Stringer: How would that be moved on?
Professor Hardaker: Because we have a lot more understanding about how to work with these probabilistic projections than perhaps we had when we began that process some time ago. The report in 2009 was a three or four-year programme in creating those scenarios. We are a long way on. In terms of the science, we have moved on. Now there is a move to be much more engaged with the communities who will use those predictions to get a sense from them about what they look to the science community to provide them.
Q22 Graham Stringer: Is any work being done on the theoretical limits to predictions? You are dealing with a linked chaotic system, are you not?
Professor Hardaker: Yes.
Q23 Graham Stringer: Is there any theory that says you can go so far but no further?
Professor Hardaker: The chaos theory suggests there are some limits to predictability. There is no suggestion, interestingly, that we are anywhere near reaching those at the moment. The other complication of that process is that some weather conditions are more predictable than others. There is not a single limit that bounds everything. The nature of the atmosphere and the oceans is such that sometimes there is a greater degree of predictability in the atmosphere than at others. It does vary depending on the state of-
Q24 Graham Stringer: Do you know when that is?
Professor Hardaker: There are obvious examples. If you are getting very varied local conditions, lots of thunderstorms and the weather is changing rapidly, the predictability is less easy compared with when you get stable pressure patterns in place which give you more long-term predictability.
Professor Thorpe: One of the scientific advances in my centre has been to create what is called an "ensemble prediction", which means that, by looking at the chance that the forecasts could be more or less extreme than the central estimate, depending on the shape of that distribution, we are able to give a prediction of whether the future weather is predictable or not. So we can predict the predictability. As Paul was saying, the signals on some occasions are inherently stronger and it is more predictable. By using these techniques we can distinguish those events where it is more predictable from those events where it is less predictable. That is a big advance in the science. It is critical to know, not only regionally but locally and, from time to time, that the predictability varies hugely in terms of what you can predict.
Q25 Graham Stringer: That predictability of the predictions is done on an empirical basis and not a theoretical basis.
Professor Thorpe: No. It is done in terms of using the mathematical physical models that we use for weather forecasting. The European Centre, for example, every day produces 50 parallel forecasts where we have slightly different starting conditions for the forecast, and also we vary within the range of uncertainty some of the representations of the physics in the model. We then start to build up a much better description of the uncertainty and are able to predict when it is predictable and when it is not. It is not empirical in that sense. It is using the physics that we know about and the range of uncertainty in that physics that we know about.
Q26 Graham Stringer: Both you and Professor Hardaker talked about recording and testing how accurate the predictions had been on three, four and five-day weather forecasting and on seasons. Have you tested the predictions since you have been using models for climate change? Have you tested whether those models are predicting what is happening on an annual or a biannual basis? I do not know when they started, but have they predicted what has been happening over the last 10 years?
Professor Thorpe: Absolutely. The Hadley Centre for Climate Prediction and Research, which is part of the Met Office, has been producing detailed hindcasts of the climate of the 20th century, for example, to look at the fluctuations in the climate. We know what has happened, but we can use our prediction models to see how accurate they are relative to what happened.
Q27 Graham Stringer: I am asking something slightly different from hindcasts. Since 2000, the climate has shifted a bit over that period of time. In 2000, were the predictions accurate or to what level were they accurate?
Professor Thorpe: I will give an initial answer and others may want to contribute. As you know, there have been several IPCC scientific assessments on climate change going right back to 1990. In 1990, when the scientific assessment was made, there were real-time predictions of what the climate, subsequent to 1990 going forward, would be. We are now in a position of having a record of what actually happened relative to the predictions that were made then of the climate from 1990 to the present time. Those comparisons show that the models of the day-of course, the models have improved since then-if anything, under-estimated the amount of global warming that has subsequently happened. We are able now, because we have done this climate prediction for a number of years, to start to assess that.
Q28 Graham Stringer: It is often said that they did not predict the flattening out that happened after 2000. Is that true or not true?
Professor Thorpe: Two processes are going on here. There is the slow trend due to the fact that greenhouse gases are increasing, which is the trend that we are interested in in terms of global warming. Then there is the year-to-year variability that comes from a phenomenon such as the El Niño. We know that El Niño variations can lead to fluctuations in the warmth of the planet on these time scales of decades. What we think has been happening is the slow trend of global warming but superimposed on that are the fluctuations on year-to-year.
Q29 Graham Stringer: That is only part of the point. Did you predict the change in the gradient?
Professor Thorpe: Not me personally.
Graham Stringer: But as a profession.
Professor Thorpe: I think I am speaking too much.
Professor Hardaker: As Alan said, the continuation of the assessment report studies have shown that what the early models predicted is largely what has come to pass in terms of our observations. In the very early days of the climate models, that intercomparison with the observations was very important. By doing that intercomparison in the early stages of the process is how we discovered that we were missing the aerosol component in our models, which was a cooling effect on the climate.
Professor Thorpe: The slow trend of global warming is a forcing from something we are doing to the atmosphere gradually and in a secular way over time. Some of these fluctuations from year to year, to come back to our earlier discussion about seasons, are more predictable using these models than others. Seasonal to decadal is an area where we are doing a lot of research. To answer your specific question of whether we can predict year on year the variation of El Niño going forward five or 10 years into the future, probably, not yet, but this is an area on which there is active research. We can get the sort of statistical variation, but the precise future chance of a particular El Niño or La Niña happening is still very challenging.
Q30 Pamela Nash: The Government Chief Scientific Adviser in the review of the Government’s needs for the Climate Science Service has recommended that a step-change increase in supercomputing capacity is required within the UK. Would each of you agree with that statement? Professor Hardaker, you are nodding.
Professor Hardaker: Those who use supercomputers will say more, but yes, absolutely. I recognise that there is an affordability issue and we have to make priorities, but it is a significant limitation on our capability at the moment, in terms of what we can do with the modelling of the oceans and the atmosphere.
Professor Hill: I would certainly endorse that. I am particularly concerned with the ocean parts of the problem, which on these longer time scales is a crucial component of it. We believe that increasing ocean resolution will take out a number of well-known biases that exist in things like sea surface temperature, which, when coupled to atmospheric models on longer time scales, produce biased effects. One classic example is that low resolution models tend to cause the Gulf Stream to break off from the coast at lower latitudes than they should, leaving that area of the north Atlantic anomalously cold compared with what we observe. This then feeds through when you couple it into atmospheric models. As you increase model resolution, as we are at the moment, to a quarter of a degree and, in due course, to one twelfth of a degree and Gulf Stream separation happens in the right place, the sea surface anomaly issue starts to resolve itself. There are many other examples like that where increased resolution, being able to resolve eddies in the system, has an important effect as you start to couple it with atmospheric models. To achieve those resolutions certainly needs increasing computing power.
There are things that we would like to do with ocean models in the future which are a little beyond weather forecasting but are important for marine forecasting. For example, to forecast harmful algal blooms, which is very important, would involve coupling ecosystem models to ocean models. Once you start to do that, you get into a whole new ballgame in terms of the computing power required to represent the different biogeochemical processes and plankton species as well. From both the climate and weather point of view but also from ocean forecasting, the answer is most certainly yes. We also need-this is a crucial point-the data to go with it. You cannot separate these two issues.
Professor Thorpe: The answer from me is yes.
Q31 Pamela Nash: So that I am clear, you mentioned funding being a limiting factor. Are we talking about funding for the purchase of technology or purely for research and development in creating that supercomputing capability?
Professor Hardaker: It is always a combination of the two. You need to have the technology in the first place and then you need to be able to exploit that by putting the science on to that technology platform. It is a mix of the two. I sense that the biggest cost in this is in the physical computing itself because, as we mentioned before, when you start trying to look at the whole-earth system, the biosphere, the ecosystem, the chemistry and the carbon cycle, the complications of that are such that you need to make a step-change in current capabilities.
Professor Thorpe: You can see very easily that it is the technology. There was a time when weather forecasting applications in terms of the use of the world’s top computers were high upon that list. It has slid gradually further and further down from the highest capability to much lower. That change has prevented us from advancing as quickly as we could have done. I fully endorse the fact that this is a real limitation now. It is very challenging from a financial point of view because of things as mundane as exchange rates, for example. When purchasing computer power in pounds, it depends on the relative exchange rate with respect to the dollar and so on, because many of the computer manufacturers are international. Our purchasing power is also a factor as well.
Professor Hill: One should not underestimate what a complex problem this is. It is not just about more computer power and more calculations per second. It is also about disc storage, mass storage for storing the results, post-processing facilities to analyse them, high bandwidth communications to move data around, and there is also a people dimension to it. In respect of the oceans, one of our problems is having the manpower to be able to process, analyse and interpret the data. Then there is the question of re-engineering some of the software, the codes in the models, so that they are able to run on massively parallel machines. It is not just a question of buying a big box with lots of processors in it. All of those dimensions require the investment.
Q32 Pamela Nash: Thank you for those answers. The partnerships that the Met Office has with international meteorological organisations have already been alluded to in this session. To what extent is the Met Office working and pooling its human and computing resources with those of institutions across the world?
Professor Thorpe: From my point of view, the Met Office is incredibly connected internationally to all of the most important networks in weather prediction, forecasting and meteorological science more generally. It is a major member state within my own organisation, which is a co-operation among 19 European countries. The Met Office is a major partner in that capability. That is on the operational weather forecasting side. On the research side, there is strong collaboration between Met Office scientists and, for example, the academic community in the UK, supported by the Natural Environment Research Council. There is a relatively new initiative called the Joint Weather and Climate Research Programme, which joins together, in a much more structured way than before, research in the Met Office and in UK institutes and academia. My answer to your question is that the Met Office is highly connected and clued into that international science and forecasting network.
Professor Hill: One particular example of how they have been working internationally and strategically is that a number of countries, including Korea and Australia in particular, have adopted the unified model, which is their workhorse code for weather forecasting. This has a number of benefits for those countries that have adopted the model, which is a very good computer code. Also, because those are being operated in different weather and climate environments from our own country, they get tested in different regimes. That means there are more hands, eyes and experience in using that particular code to be able, collectively, to learn the lessons. That is a very good example of working strategically with other countries to get leverage and benefit from that development.
Professor Hardaker: I mentioned earlier about the percentages of data that are collected locally versus those that are used. I said that 97% or 98% of the data that we use is collected elsewhere. If you look at the modelling function, only about 50% is developed in-house by the Met Office. The rest is obtained through the types of collaborations that we have talked about in terms of driving and developing these models. You can get a sense, both from the data and modelling, that it is a very collaborative programme. The Met Office, as Professor Thorpe said, is one of the world’s leading met services. You can imagine that as part of this international network it plays a very important role not just in the UK but in international meteorology.
Q33 Pamela Nash: Just to move on from that, Professor Hill, the evidence from the National Oceanography Centre states: "Supercomputing is a fundamental requirement for understanding ocean and atmospheric processes and will require continued funding, collaboration and ‘joined-up’ working to maximise potential." As you said, Professor Hardaker, there are challenges. What are the particular challenges in encouraging this collaboration and how can we overcome them?
Professor Hill: A number of these are being addressed already. It is fair to say that the Met Office, over the last five or 10 years, and five years in particular, has shifted in being much more open to collaboration. For example, in the ocean area, we have worked together on lining up the model code we use. The oceanographic community and the meteorologists were using completely different models. We have now converged on that, which is a European model, so we have a wide community of users. We have a joint ocean-modelling programme, which is part of the Joint Weather and Climate Research Programme, where scientists are defining a clear set of objectives and programme of work. They are working together in order to maximise the impact across the science community. That is a very good example of things that are happening already.
Could we do more of it? Probably, yes. There are barriers. There are funding issues. There are also, I guess, some real cultural barriers as well. The Met Office is an operational forecasting agency and works to rigid timelines and works in a very methodical way, which is what you would expect from an operational agency. The research community operates on different time scales. Bringing these things together does not always quite mesh. The research community has to work in more disciplined ways when working with the Met Office, which is good, but perhaps that is at the expense of some innovation at times and having to work with existing operational systems. For me this is an example of something which, in other circles, is called "translational research". It is the process by which basic research-a fundamental understanding of the system-gets turned in a systematic way into something that is usable and operationalised. Probably, a much more explicit recognition of the role of translational research would do a lot to break down the barrier that sometimes exists, which is as much cultural as anything in terms of the basic science getting into operational use in a timely way. That is one area to work on.
Chair: We have three more very important questions that we need to get through fairly quickly.
Q34 Stephen Metcalfe: I will try to be as quick as possible. I would appreciate it if you could keep your answers quite brief. I want to talk about access to the historical data that are held by the Met Office. We have had conflicting reports about how readily available they are. I would be interested in your comments on what you think. Also, regardless of how readily available they are now, do you think they should be made freely available to all at no charge?
Professor Hardaker: For some context, the Met Office is operating within the rules bounded by the Government’s policy on data, and it is also guided by some international agreements that exist on this subject. Of all the European met services, I would say that the Met Office makes more of its data available than any other European met service, and it has tried to go a long way to do that. We are working with the Public Weather Service at the moment to run a consultation for them with the wider community in the UK about what will be the requirements for more of that historical data online and how could it best be provided. The Met Office and the Public Weather Service components are mindful of the fact that there is a demand for more data out there. Of course, there is a cost in providing that. It would be a valuable thing to make more of that data available, but somewhere someone needs to pick up the cost of making that possible. That is a barrier in itself.
Putting much more data out into the community may help to grow the markets for meteorological services, but we, in the Society, have a real concern that it might help to grow very poor, low-quality services, because technology is such that you can easily take low- quality raw data and produce products and services of not very good quality. We want to focus on the idea of developing a quality standard for the sector that would identify those quality providers. There has been a strong push from the big customers of weather services in the UK to establish, in partnership with us, a quality mark for the sector.
Professor Hill: The principle is that publicly funded basic data should be available at minimal cost and that the value is generated by the creation of added value products. The caveats to which Paul has referred do apply. The creation of the Public Data Corporation, which is being trialled, is about doing that, trying to make more data more accessible.
In terms of accessibility to the research community, we have very effective means of working with the Met Office. That is generally not an issue. Their data are generally widely available for research purposes.
Professor Thorpe: I am not sure that I have anything to add.
Professor Hardaker: I would like to make a quick point about the Public Data Corporation. I am not up to speed on the full detail of it, but I have seen some of the developments. I have some concern that we need a clear policy and statement on how the Government are dealing with data. I get the sense from the outside that bits of Government are pulling in different directions in terms of where they want to go with this. My hope is that the Public Data Corporation is not going to add an unnecessary and unhelpful level of bureaucracy. We need some simple form of clarity that does not add an unhelpful layer of bureaucracy.
Q35 Stephen Metcalfe: You said that, by making the data more freely available, there are potential services and spin-offs, and commercial activity could come from that. Is there any way of putting a value on that commercial activity, in making sure that a quality standard is in place to make sure that it is not abused? How would one even begin to understand how someone might make money and what that would be worth to us as an economy?
Professor Hardaker: There have been a couple of goes at this over recent times. We had a piece of work published fairly recently by the European grouping of private sector meteorologists. I cannot recall the figures, but I could certainly send the Committee the paper-it was published about two years ago-which highlighted some figures associated with what was expected from this. In some regard, there was a vested interest. It was created by a community that stands to benefit from that process. There is no reason not to trust the figures and they are a helpful guideline. It would be helpful to have an independent review of the value that this data might add.
Q36 Graham Stringer: I will roll all my questions into one, if I may. First, will the Met Office move to BIS and its new research structure help with collaboration with the research base in this country and elsewhere? What will the changes be? Secondly, you have talked glowingly about the collaboration that goes on. There must be problems. Where could that be improved? Where is there a lack of collaboration?
Professor Thorpe: The BIS move will help from two points of view. One is the connection into the research base, which is located, as you know, in BIS. At some level, it seems a more natural home. As we were saying just now, there is great potential from weather forecasts for commercial and business activity in the economy in general, including opportunities to reduce costs and mitigate risks, as well as genuinely new business that can emerge, which is a clear focus of BIS as a Department. For me, it seemed to be a good move to bring the Met Office into BIS. It will help with the connection to the research base. I would not want to characterise that to mean that, in any sense, there was a major problem before. We worked hard, as Ed has mentioned, during the last five years or so to up our game in connecting the Met Office research with the general research base in the UK, even when it was in the MOD. That is already bearing fruit. I can only see the move to BIS as helping that process.
Professor Hill: In terms of lack of collaboration, I would point to one issue that is, probably, not lack of collaboration but it is a lack of join-up. As we have said before, important new observations are crucial in order to improve seasonal and climate issues, which, essentially, centre around the oceans. Ocean science has become sufficiently mature now that we have the technologies to make reliable measurements, whereas probably even a decade ago we did not. There is a whole new stream of data that is potentially available through new technology such as Argo floats and other automated systems, which will make, in due course, a significant impact.
The problem is that we do not yet have a sufficiently joined-up mechanism in Government to do a number of things. First, there are multiple Departments that are interested in the outcomes of this, so the idea of sharing the costs of this is not well developed. Second, we have the issue of translating measurement techniques which, essentially, have been in the domain of basic science into the operational arena. Some of these ought to fall, ultimately, within the remit of the Public Weather Service, but that simply cannot be at the expense of displacing existing vital measurements. We have something new coming on stream here. There is a real challenge about how to resource new things that are adding value. The ultimate answer, in my view, is that they are adding new value to the forecasts. An increased economic value can be put on them. We have not yet discovered, as a nation, the mechanism to translate these measurements from basic science to operational and, indeed, to share the resources fairly across multiple Departments of Government that benefit.
Professor Hardaker: We mentioned earlier the Joint Weather and Climate Research Programme (JWCRP). That is helping to bring these two different communities and their cultural differences together. It is early days for that. Some of the limiting factors within that framework are that we need to be able to get the models and data on to a common platform so that those two communities can work together effectively. We have some resources to do that, but it is the start of a process. Until we can get all of that on to those platforms and being used actively, we are not leveraging that partnership as strongly as we could be. However, we are moving very much in the right direction.
Q37 Roger Williams: I was brought up in a farming family and the weather forecast was sacrosanct. Everybody would be quiet and listen to it. It is not quite so much a national treasure now but it is still a very important part of our lives. Who is responsible for accurate communication of the forecast? Is it the Met Office or the broadcasting company?
Professor Hardaker: The answer to that is both of them and as well as organisations like us in terms of educating the public about how to get best value from those forecasts. We talked a lot about accuracy and how good we are in the UK at producing those forecasts, but it is wasted if we cannot communicate that to the public. It means that organisations have to work together to get the message across.
Q38 Roger Williams: Is there a place for jokes? Some of the tourist operators get very upset when they hear, "Here comes another Bank Holiday. Here comes another downpour", or, "Don’t go to Wales because it’s always wet."
Professor Hardaker: I am not going to comment on the Wales bit because my wife is Welsh. I will not make any comment on that. What is interesting is the development of the internet as a way of providing weather information and the move to digital channels. I know that organisations like the BBC are exploring the potential that has to add a whole layer of new information that will provide a detail that they cannot offer in a single broadcast. The public will be able to watch a national and a local forecast and then use the red button functionality or the internet to drill down into more and more detailed information that might be useful and specific to them. That is a really interesting innovation. Many countries make much greater use of probabilistic information in their forecasts than we do, even in their broadcasts. Perhaps with some of this digital technology and the internet, we have not been bold enough to explore the potential of that in terms of delivering more probabilistic information and then working to help the public better understand it and how better to make use of it. That is a role that, perhaps, we could play a part in as well.
Professor Hill: It is certainly the case that there is no shortage of weather information. That is not the issue. Indeed, the Met Office website is excellent in terms of the sheer depth and range of information you can get if you are looking for it. The issue is about communication, using new technologies and so forth to be able to get information to people in the form that they want and when they want it. This is a social problem as much as one of information, generation and dissemination.
Q39 Roger Williams: Professor Hardaker, you are the chairman elect of Sense About Science, are you not?
Professor Hardaker: Yes, that is right.
Q40 Roger Williams: Has that organisation something to contribute in terms of public understanding about the things we are talking about-probabilities and those sorts of issues?
Professor Hardaker: I hope so. That group has been working a lot on trying to help the public with statistics. We have published a book on weather and climate. We are hoping to publish something next year on working with uncertainty. That is a big issue. Science looks at uncertainty as knowledge and focus, whereas often the public look at uncertainty as "Well, they don’t know. They haven’t got an answer for me." That is a big gap to bridge. A better explanation of how science works and how the public can make better use of uncertainty information is of real importance in terms of their making use of it and the media communicating it more effectively.
Chair: Gentlemen, this has been a very informative session. We are very grateful for your time this morning. We are looking forward to our visit to Exeter to see how you produce all this fantastic data. Thank you.
 Note by witness: I believe that on the day I misheard or misinterpreted what the Chair was asking here. I thought he was asking whether the observations were available to the private sector - but on reading this transcript it appears he may have been asking about the availability of observations collected by the private sector.