[Update 15 April 2015]
Summaries online
The summaries of the Climate Dialogue on Regional Modelling are now online (see links below). We have made two versions: an extended and a shorter version.
Both versions can be downloaded as pdf documents:
Summary of the Climate Dialogue on Regional Modelling
Extended summary of the Climate Dialogue on Regional Modelling
[End update]
The third Climate Dialogue is about the value of models on the regional scale. Do model simulations at this level have skill? Can regional models add value to the global models?
We have three excellent participants joining this discussion: Bart van den Hurk of KNMI in The Netherlands who is actively involved in the KNMI scenario’s, Jason Evans from the University of Newcastle, Australia, who is coordinator of Coordinated Regional Climate Downscaling Experiment (CORDEX) and Roger Pielke Sr. who through his research articles and his weblog Climate Science is well known for his outspoken views on climate modelling.
Climate Dialogue editorial staff
Rob van Dorland, KNMI
Marcel Crok, science writer
Bart Verheggen
First comments on the guest blog of Bart van den Hurk:
Bart begins by discussing the ability of regional climate models to add value (over global models) to a regional climate projection. Despite some confusion about whether an RCM is providing a prediction or a projection (I suggest in this context we are only talking about projections), I largely agree with his assessment. There are plenty of publications that demonstrate added value from RCMs. As he points out, this doesn’t guarantee a better regional climate simulation however, and some aspects, such as trends, can be quite sensitive to the lateral boundary conditions from global models (an inherent limitation for RCMs). So RCMs are not perfect but in many cases are good enough to be useful.
Bart goes on to discuss some other applications of RCMs that come from the fact that they are “collections of our understanding of the regional climate system”. As a climate scientist, one of the main ways I use RCMs is to explore questions around various climate processes in a manner similar to that described. I would even take this one step further and suggest that even when using RCMs to provide regional climate projections, they can be used to identify the climate processes most responsible for any changes seen in the projection. Thus, they can help us better understand why some aspect of the climate may be changing, which may be more important than knowing the change itself.
First comments on the guest blog of Roger Pielke Sr.:
Roger presents a case for abandoning climate model projections altogether – or at least not allowing the impact/adaptation community and policy makers to see them as they will gain “an erroneous impression on their value.” In my experience this is certainly the case if you talk about the simulations as predictions rather than projections – the climate models are not predicting what the weather will be on the 5th of May 2051 – they are providing projections of the climate based on emission scenarios and initial conditions. When dealing with those outside the climate science community I have certainly found the distinction to be important.
The climate modelling community has gone to great lengths to explore, understand and quantify the uncertainty associated with climate model projections. For GCMs, the CMIP range of international collaborative experiments have produced large ensembles of simulations which are freely available for people to download and investigate. For regional models, similar ensembles of simulations have been assembled through projects like PRUDENCE, ENSEMBLES, NARCCAP, and CORDEX. In each case a significant amount of work has gone into quantifying and trying to understand the causes for the uncertainty within the ensembles.
When Roger says “the output of these models are routinely being provided to the impact communities and policymakers as robust scientific results” … I don’t think he means the large model ensembles with their included uncertainty.
I have worked with impacts/adaptation researchers for some time now. At first they were very resistant to dealing with an ensemble of future climate projections that embodied significant uncertainty. They wanted a single model projection that was the “truth” to apply in their own impacts models. In the early impacts literature you do indeed see many circumstances where a single climate projection is used as the “truth” to explore climate change impacts. My guess is that this is closer to the situation that Roger is referring to. The impacts research community is becoming much better at dealing with large ensembles that embody uncertainty however, and the one model “truth” is becoming a thing of the past.
After a summary of his 2012 paper, Roger argues that climate models perform poorly in terms of modelling a realistic recent past and presents quotes from some papers to support this claim. While there are certainly problems with GCMs, and these papers point out a few of them, there were 100s of relevant papers published over the same period and many of them show good performance. In fact, the majority of published works will point out both when the models perform well and when they perform poorly. This is a necessary step in the model development and improvement process. In this process, pointing out the problems is the easy part, finding the solutions is where the hard work is done.
He finishes by summarizing more of his 2012 paper and concluding “It is therefore inappropriate to present type 4 results to the impacts community as reflecting more than a subset of possible (plausible) future climate risks.” Indeed by producing large ensembles, for multiple different emission scenarios, we do present the climate projections as a set of possible future climate risks, with associated uncertainty. Providing such projections also does not preclude the use of a bottom-up vulnerability assessment either as shown by the inclusion of climate model projections in the Nederlands Meteorologisch Instituut report that Roger quotes.
In the end, climate models are our best tools for understanding how the climate system works. As climate scientists, we will continue to use these tools to improve our understanding of the climate system, and use our understanding of the system to improve these tools. Part of this includes exploring the impact of changing levels of greenhouse gases on the climate by creating future climate projections.
So long as climate projections that embody our best understanding of the climate system exist, it is not surprising that others involved in climate change impacts/adaptation research, and policy makers, would want to consider these projections in their work. Because of the open nature of climate research and the freely available model data, they can do this (and have done this) without directly working with climate scientists or understanding the uncertainties and limitations associated with the data. I think the real way forward is for closer collaboration between climate scientists and the impacts/adaptation and policy making communities – not less. In this way the climate scientists can ensure that the projections are understood with their associated uncertainties, and the impacts/adaptation researchers can ensure that the climate scientists are aware of the parts of the climate system they are most sensitive to, providing a focus for climate model development efforts.
I have a remark to Jason and the limitations of downscaling,
“…they assume that the derived statistical relationship will not change due to climate change.
Dynamical downscaling, or the use of Regional Climate Models (RCMs), does not share the limitations of statistical downscaling.”
I think that RCMs very much share the potential caveat of non-stationarity, as they too involve statistically fitted models, known as parameterisation schemes. The bulk effects of unresolved small-scale processes are essentially represented by statistical models, e.g. clouds and surface fluxes. What’s more, imperfections in these feed back into the calculations in RCMs, whereas such discrepancies merely cause a growing bias in the statistical schemes.
It is also possible to evaluate the downscaling for these effects, e.g. by applying them to the past. Furthermore, since the RCMs and statistical downscaling draw information from independent sources, and one need to look at both (do they converge/diverge) before using the results for decision making.
First comments on the guest blog of Jason Evans:
In Jason Evans’ blog the methodological and conceptual components of downscaling and skill assessment are nicely illustrated. However, I tend to disagree with the underlying rationale that a continuous development of models and their frequent confrontation with observations is going to reduce the uncertainty in the projections. First of all, it is not uncertainty that is the driving principle behind depicting a plausible range of future conditions, it is the range of likelihoods that is of interest. For many decision making processes (certainly those that dominate in the public political agenda) uncertainty is not a bottleneck for decision making. Instead, this process aims at keeping all options open as long as possible (that’s why most “important” decisions are taken near or past a deadline: to ensure that as many as possible options can be considered).
Second, I think climate projections do not only affect decisions when they are “out of range of current conditions”, but merely when they provide better insight in the processes that are responsible for the decision. Also a system component (climate, population, water requirement….) that does not change can affect the decision taken, if the underlying picture of (lack of) change is convincing enough.
And third, development of models not rarely is associated with the realization that new interfaces or boundary conditions, that appear to play a role, need to be included. This drives the evolution from atmospheric models to physical climate models to earth system models. Adding new boundary conditions or feedbacks does generally increase the number of degrees of freedom, which is difficult to reconcile with the notion of “reduced uncertainty”. Indeed, the range of temperature projections of CMIP3 does not strongly differ from CMIP5, although the modellers do claim that their models have improved and the level of understanding has increased (and thus inherent uncertainty has decreased).
First comments on the guest blog of Roger Pielke Sr.:
Roger Pielke provides an impressive list of studies demonstrating the lack of skill of (global and regional) climate models to predict changes in climate characteristics at the regional scale. But, as I have illustrated in my blog, a systematic bias in current day climate models – which is indeed well documented and agreed – does not make them useless for future assessments, which I think is the crucial element in this discussion.
Roger considers projections and predictions to be synonyms, but I disagree with this, and also feel that this assumption is an important element in the controversy under discussion. A prediction is a statement on the expected situation in the near future starting from a current state of the system, while a projection is a possible evolution of the system given an assumed (external) driver. A climate prediction focuses at the predictable time scale, which depends a lot on the spatial scope. Making a prediction of the future global mean temperature is a lot easier than predicting it at the regional scale. And even there, I agree with the statement of v Oldenborgh et al (2012) that the prediction of mean temperature at the regional scale can be done fairly well given the robust temperature trend.
The targets set out by CMIP5, listed by Roger, do not speak of predictions, and that is not without reason. I agree that the ambition to make (regional) climate predictions even at decadal or longer time scales can not be supported by the current apparent feasibility, given the studies that demonstrate the lack of predictive skill. There are some source of predictability that are still not fully resolved (including those dealing with improving climate models, but also related to unexplored initial conditions or driving conditions), and a great benefit of these predictability studies is that they mimic the practice of weather prediction by confronting models with observations at the relevant time and spatial scales, leading to the necessary inspiration for this model improvement. But even with perfect models, climate predictability will always be constrained by unknown evolutions of climate forcings, both anthropogenic and non-anthropogenic.
In his summary he claims that regional climate downscaling fails to provide additional scale. As is illustrated by Jason Evans’ blog (and also mine) this is not fully true: there where local information (mountains, coasts, land use) gives new information, a more reliable projection can be produced.
The purpose of a projection is to depict the possible (plausible) evolution of the system. To my opinion, the process of decision making is not dependent on the (quantitative) predictions provided by climate models, but by the plausibility that the future will bring situations to which the current system is not well adapted. For that a set of imperfect projections is far more useful than a single optimized prediction (which, like a weather forecast with a perfect model, can still be wrong). Decision making does make use but does not entirely rely on this climate information.
Maybe more important is the ability to feed the imagination, and to denote the relevance of the projection for the system under study. I fully embrace Pielke’s plea for a system analysis that takes the vulnerability of the system as a starting point. But from this kind of analyses, frequently the stakeholders are the participants that ask for support from (regional) climate models to illustrate the possible alternative future conditions. An approach where “real” weather events, in combination with their local impact on the sector or area at steak is increasingly adopted in climate research, including at KNMI, partly inspired by Rogers recommendations in his role as member of the recent review committee. But also this “future weather” events need to be framed, to avoid that the producers of these event based scenarios apply cherry picking to depict the situation that best suites their interests or assumptions. A framing of these individual events is needed, and a top-down scenario approach is well capable of providing this framework, certainly when complemented by scenarios of other drivers than climate, such as population, water requirements, land use etcetera.
An increasing number of studies succeed in adequately combining the valuable pieces of information coming from both climate and non-climatic models and assessments. A strong degradation of (regional) climate models by pointing at their lack of skill does not help the intelligent application of the inherent level of understanding that is generally captured by these models.
First comments on the guest blog of Jason Evans (I added my comments to Jason’s article):
Are climate models ready to make regional projections?
Global Climate Models (GCMs) are designed to provide insight into the global climate system.
They have been used to investigate the impacts of changes in various climate system forcings such as volcanoes, solar radiation, and greenhouse gases, and have proved themselves to be useful tools in this respect.
The growing interest in GCM performance at regional scales, rather than global, has come from at least two different directions: the climate modelling community; and the climate change adaptation community.
Due, in part, to the ever increasing computational power available, GCMs are being continually developed and applied at higher spatial resolutions. Many GCM modelling groups increasing the resolution from ~250km grid boxes 7 years ago to ~100km grid boxes today.
This model resolution increase leads naturally to model development and evaluation exercises that pay closer attention to smaller scales, in this case, regional instead of global scales. The Fifth Coupled Model Intercomparison Experiment (CMIP5) provides a large ensemble of GCM simulations, many of which are at resolutions high enough to warrant evaluation at regional scales. Over the next few years these GCM simulations will be extensively evaluated, problems will be found (as seen in some early evaluations1,2), followed hopefully by solutions that lead to further model development and improved simulations. This step of finding a solution to an identified problem is the hardest in the model development cycle, and I applaud those who do it successfully.
Probably the stronger demand for regional scale information from climate models is coming from the climate change adaptation community. Given only modest progress in climate change mitigation, adaptation to future climate change is required.
Some sectors, such as those involved in large water resource projects (e.g. building a new dam), are particularly vulnerable to climate change. They are planning to invest large amounts of money (millions) in infrastructure, with planned lifetimes of 50-100 years, that directly depend on climate to be successful. Over such long lifetimes, greenhouse gas driven climate change is expected to increase temperature by a few degrees, and may cause significant changes in precipitation, depending on the location. Many of the systems required to adapt are more sensitive to precipitation than temperature, and projections of precipitation often have considerably more uncertainty associated with them. The question for the climate change adaptation community is whether the uncertainty (including model errors) in the projected climate change is small enough to be useful in a decision making framework.
From a GCM perspective then, the answer to “Are climate models ready to make regional projections?” is two-fold. For the climate modelling community the answer is yes.
GCMs are being run at high enough resolution to make regional scale (so long as your regions are many 100kms across) evaluations and projections useful to inform the model development and hopefully improve future simulations.
For the climate change adaptation community, whose spatial scale of interest is often much lower than current high resolution GCMs can capture, the answer in general is no.
The errors in the simulated regional climate and the inter-model uncertainty in regional climate projections from GCMs is often too large to be useful in decision making. These climate change adaptation decisions need to be made however, and in an effort to produce useful regional scale climate information that embodies the global climate change a number of “downscaling” techniques have been developed.
It is worth noting that some climate variables, such as temperature, tend to be simulated better by climate models than other variables, such as precipitation. This is at least partly due to the scales and non-linearity of physical processes which effect each variable. This is demonstrated in the fourth IPCC report which mapped the level of agreement in the sign of the change in precipitation projected by GCMs. This map showed large parts of the world where GCMs disagreed about the sign of the change in precipitation. However this vastly underestimated the agreement between the GCMs3. They showed that much of this area of disagreement is actually areas where the GCMs agree that the change will be small (or zero). That is, if the actual projected change is zero then by chance, some GCMs will project small increases and some small decreases. This does not indicate disagreement between the models, rather they all agree that the change is small.
Regional climate models
Before describing the downscaling techniques, it may be useful to consider this question: What climate processes, that are important at regional scales, may be missing in GCMs?
The first set of processes relates directly to how well resolved land surface features such as mountains and coastlines are. Mountains cause local deviations in low level air flow. When air is forced to rise to get over a mountain range it can trigger precipitation, and because of this, mountains are often a primary region for the supply of fresh water resources. At GCM resolution mountains are often under-represented in terms of height or spatial extent and so do not accurately capture this relationship with precipitation. In fact, some regionally important mountain ranges, such as the eastern Mediterranean coastal range or the Flinders Range in South Australia, are too small to be represented at all in some GCMs. Using higher spatial resolution to better resolve the mountains should improve the models ability to capture this mountain-precipitation relationship.
Similarly, higher resolution allows model coastlines to be closer to the location of actual coastlines, and improves the ability to capture climate processes such as sea breezes.
The second set of processes are slightly more indirect and often involve an increase in the vertical resolution as well. These processes include the daily evolution of the planetary boundary layer, and the development of low level and mountain barrier jets.
A simple rule of thumb is that one can expect downscaling to higher resolution to improve the simulation of regional climate in locations that include coastlines and/or mountain ranges (particularly where the range is too small to be well resolved by the GCM but large enough to be well resolved at the higher resolution) while not making much difference over large homogeneous, relatively flat regions (deserts, oceans,…).
So, there are physical reasons one might expect downscaling to higher resolution will improve the simulation of regional climate. How do we go about this downscaling?
Downscaling techniques can generally be divided into two types: statistical and dynamical. Statistical techniques generally use a mathematical method to form a relationship between the modelled climate and observed climate at an observation station. A wide variety of mathematical methods can be used but they all have two major limitations. First, they rely on long historical observational records to calculate the statistical relationship, effectively limiting the variables that can be downscaled to temperature and precipitation, and the locations to those stations where these long records were collected. Second, they assume that the derived statistical relationship will not change due to climate change.
Dynamical downscaling, or the use of Regional Climate Models (RCMs), does not share the limitations of statistical downscaling. The major limitation in dynamical downscaling is the computational cost of running the RCMs.
This generally places a limit on both the spatial resolution (often 10s of kilometres) and the number of simulations that can be performed to characterise uncertainty. RCMs also contain biases, both inherited from the driving GCM and generated with the RCM itself. It is worth noting that statistical downscaling techniques can be applied to RCM simulations as easily as GCM simulations to obtain projections at station locations.
The RCM limitations are actively being addressed through current initiatives and research.
Like GCMs, RCMs benefit from the continued increase in computational power, allowing more simulations to be run at higher spatial resolution. The need for more simulations to characterise uncertainty is being further addressed through international initiatives to have many modelling groups contribute simulations to the same ensembles (e.g. CORDEX – COordinated Regional climate Downscaling EXperiment http://wcrp-cordex.ipsl.jussieu.fr/). New research into model independence is also pointing toward ways to create more statistically robust ensembles4. Novel research to reduce (or eliminate) the bias inherited from the driving GCM is also showing promise5,6.
Above I used simple physical considerations to suggest there would be some added value from regional models compared to global models. Others have investigated this from an observational viewpoint7,8 as well as through direct evaluation of model results at different scales9,10,11. In each case the results agreed with the rule of thumb given earlier. That is, in the areas with strong enough regional climate influences we do see improved simulations of regional climate at higher resolutions.
Of course we are yet to address the question of whether the regional climate projections from these models have low enough uncertainty to be useful in climate change adaptation decision making.
To date RCMs have been used in many studies12,13 and a wide variety of evaluations of RCM simulations against observations have been performed. In attempting to examine the fidelity of the regional climate simulated, a variety of variables (temperature, precipitation, wind, surface pressure,…) have been evaluated using a variety of metrics14,15, with all the derived metrics then being combined to produce an overall measure of performance. When comprehensive assessments such as these are performed it is often found that different models have different strengths and weaknesses as measured by the different metrics. If one has a specific purpose in mind, e.g. building a new dam, one may wish to focus on metrics directly relevant to that purpose. Often the projected climate change is of interest so the evaluation should include a measure of the models ability to simulate change, often given as a trend over a recent historical period16,17. In most cases the RCMs are found to do a reasonably good job of simulating the climate of the recent past.
Though, there are usually places and/or times where the simulation is not very good. Not surprising for an active area of research and model development.
Given the evaluation results found to date it is advisable to carefully evaluate each RCMs simulations before using any climate projections they produce. Being aware of where and when they perform well (or poorly) is important when assessing the climate change that model projects. It is also preferable for the projections themselves to be examined with the aim of understanding the physical mechanisms causing the projected changes. Good process level understanding of the causes behind the changes provides another mechanism through which to judge their veracity.
Finally we come to the question of whether regional climate projections should be used in climate change adaptation decisions concerning infrastructure development? In the past such decision were made assuming a stationary climate such that observations of the past were representative of the future climate. So the real question here is will the use of regional climate projections improve decisions made when compared to the use of historical climate observations?
If the projected regional change is large enough that it falls outside the historical record even when considering the associated model errors and uncertainties, then it may indeed impact the decision. Such decisions are made within a framework that must consider uncertainty in many factors other than climate including future economic, technological and demographic pathways. Within such a framework, risk assessments are performed to inform the decision making process, and the regional climate projections may introduce a risk to consider that is not present in the historical climate record. If this leads to decisions which are more robust to future climate changes (as well as demographic and economic changes) then it is worthwhile including the regional climate projections in the decision making process.
Of course, this relies on the uncertainty in the regional climate projection being small enough for the information to be useful in a risk assessment process. Based on current models, this is not the case everywhere, and continued model development and improvement is required to decrease the uncertainty and increase the utility of regional climate projections for adaptation decision making.
“In most cases the RCMs are found to do a reasonably good job of simulating the climate of the recent past.” True. Why? There is a huge difference between past and future; the past is much easier to forecast. I was tempted to say “or to model”, but the past is an important constraint for models, so they carefully adjust (many) parameters to match the past. So an acceptable performance of most global climate models for the past is almost guaranteed. That regional climate models can take past data from global climate models and produce a “reasonably good simulation of regional climate” is quite an achievement – almost a miracle to me.
I looked into workings of NCAR’s CAM 5 – Common Atmospheric Model 5. They use a simplifying assumption that the latent heat of vaporization of water is a constant, independent of temperature. That causes an error of about 3% in the calculation of heat transfer by evaporation from tropical seas – probably where most of water on this planet evaporates. Do they know how this affects their iterative model? No. Why should I – or you – trust Global Climate Models?
First comments on the guest blog of Bart van den Hurk (I added my comments to Bart’s article):
Regional downscaling of climate information is a popular activity in many applications addressing the assessment of possible effects of a systematic change of the climate characteristics at the local scale. Adding local information, not captured in the coarse scale climate model or observational archives, can provide an improved representation of the relevant processes at this scale, and thus yield additional information, for instance concerning topography, land use or small scale features such as sea breezes or organisation of convection. A necessary step in the application of tools used for this regional downscaling is a critical assessment of the quality of the tools: are regional climate models (RCMs), used for this climate information downscaling, good enough for this task?
It is important to distinguish the various types of analyses that are carried out with RCMs. And likewise to assess the ability of the RCM to perform the task that is assigned to them. And these types of analyses clearly cover a wider range than plain prediction of the local climate!
Regional climate prediction
Pielke and Wilby (2012) discuss the lack of potential of RCMs to increase the skill of climate predictions at the regional scale. Obviously, these RCM predictions heavily rely on the quality of the boundary conditions provided by global climate models, and fail to represent dynamically the spatial interaction between the region of interest and the rest of the world. However, various “big brother” type experiments (in which the ability of RCMs to reproduce a filtered signal provided by the boundary conditions (Denis et al, 2002), for instance carried out by colleagues at KNMI) do show that a high resolution regional model can add value to a coarse resolution boundary condition by improving the spatial structure of the projected mean temperatures. Also the spatial structure of changes in precipitation linked to altered surface temperature by convection can be improved by using higher resolution model experiments, although the relative gain here is generally small (Di Luca et al, 2012).
Van Haren et al (2012) also nicely illustrate the dependence of regional skill on lateral boundary conditions: simulations of (historic) precipitation trends for Europe failed to match the observed trends when lateral boundary conditions were provided from an ensemble of CMIP3 global climate model simulations, while a much better correspondence with observations was obtained when reanalyses were used as boundary condition.
Thus, a regional prediction of a trend can only be considered to be skilful when the boundary forcings represent the signal to be forecasted adequately.
And this does apply to mean temperature trends for most places in the world, but not for anomalies from these mean trends, nor for precipitation projections.
For regional climate predictability, the added value of RCMs should come from better resolving the relationship between mean (temperature) trends and key indicators that are supposedly better represented in the high resolution projections utilizing additional local information, such as temperature or precipitation extremes. Also here, evidence of adding skill is not univocally demonstrated. Min et al (2013) evaluate the ability of RCMs driven by reanalysis data to reproduce observed trends in European annual maximum temperatures, and conclude that there is a clear tendency to underestimate the observed trends.
For Southern Europe biases in maximum temperatures could be related to errors in the surface flux partitioning (Stegehuis et al, 2012), but no such relationship was found for NW Europe by Min et al (2013).
Thus indeed, the limitations to predictability or regional climate information by RCMs as discussed by Pielke and Wilby (2012) and others are valid, and care must be taken while interpreting RCM projections as predictive assessments.
But is this the only application of RCMs? Not really. We will discuss two other applications, together with the degree to which limitations in RCM skill apply and are relevant.
Bottom up environmental assessments
A fair point of critique to exploring a cascade of model projections ranging from the global scale down to the local scale of a region of interest to developers of adaptation or mitigation policies is the virtually unmanageable increase of the range of degrees of freedom, also addressed as “uncertainty”. Uncertainty arises from imperfect models, inherent variability, and unknown evolution of external forcings. And in fact the process of (dynamical) downscaling adds another level of uncertainty, related to the choice of downscaling tools and methodologies. The reverse approach, starting from the vulnerability of a region or sector of interest to changes in environmental conditions (Pielke et al, 2012), does not eliminate all sources of uncertainty, but allows a focus on the relevant part of the spectrum, including those elements that are not related to greenhouse gas induced climate change.
But also here, RCMs can be of great help, not necessarily by providing reliable predictions, but also by supporting evidence about the salience of planned measures or policies (Berkhout et al, 2013). A nice example is a near flooding situation in Northern Netherlands (January 2012), caused by a combined occurrence of a saturated soil due to excessive antecedent precipitation, a heavy precipitation event in the coastal area and a storm surge with a duration of several days that hindered the discharge of excess water from the area. This is typically a “real weather” event that is not necessarily exceptional but does expose a local vulnerability to superfluous water. The question asked by the local water managers was whether the combination of the individual events (wet soil, heavy rain, storm surge) has a causal relationship, and whether the frequency of occurrence of compound events can be expected to change in the future. Observational analyses do suggest a link between heavy precipitation and storm surge, but the available dataset was too short to explore the statistical relationships in a relevant part of the frequency distribution. A large set of RCM simulations is now explored to increase the statistical sample, but – more importantly – to provide a physically comprehensive picture of the boundary conditions leading up to an event like this. By enabling the policy makers to communicate this physically comprehensive picture provides public support for measures undertaken to adapt to this kind of events. This exploration of model based – synthetic – future weather is a powerful method to assess the consequences of possible changes in regional climate variability for the local water management.
Process understanding
Apart from a tool to predict a system given its initial state and the boundary forcings on it, a model is a collection of our understanding of the system itself. Its usefulness is not limited to its ability to predict, but also to describe the dynamics of a system, governed by internal processes and interactions with its environment. Regional climate models should likewise be considered as “collections of our understanding of the regional climate system”. And can likewise be used to study this system, and learn about it. There are numerous studies where regional climate model studies have increased our understanding of the mechanism of the climate system acting on a regional scale. A couple of examples:
Each of these examples (and many more that can be cited) generates additional insight in the processes controlling local climate variability by allowing to zoom in on these processes using RCMs. They thus contribute to setting the research agenda in order to improve our understanding of drivers of regional change.
Climate predictions versus climate scenarios
The notion that a tool – an RCM – may possess shortcomings in its predictive skill, but simultaneously prove to be a valuable tool to support narratives that are relevant to policy making and spatial planning can in fact be extended to highlighting the difference between “climate predictions” and “climate scenarios”. Scenarios are typically used when deterministic or probabilistic predictions show too little skill to be useful, either because of the complexity of the considered system, or because of the fundamental limitations to its predictability (Berkhout et al, 2013). A scenario is a “what if” construction, a tool to create a mental map of the possible future conditions assuming a set of driving boundary conditions.
For a scenario to be valuable it does not necessarily need to have predictive skill, although a range of scenarios can be and are being interpreted as a probability range for future conditions.
A (single) scenario is mainly intended to feed someone’s imagination with a plausible, comprehensible and internally consistent picture.
Used this way, also RCMs with limited predictive skill can be useful tools for scenario development and providing supporting narratives that generate public awareness or support for preparatory actions.
For this, the RCM should be trustworthy in producing realistic and consistent patterns of regional climate variability, and abundant application, verification and improving is a necessary practice.
Further developments of RCMs as a Regional Earth System Exploration tool, by linking the traditional meteorological models to hydrological, biogeophysical and socio-economic components, can further develop their usefulness in practice.
I appreciate Bart and Jason’s thoughtful replies to my post. I want to here discuss a few issues that they raise:
First, the attempt to discriminate between a “prediction” and a “projection” is not appropriate. The use of the term “projection” with respect to added greenhouse gases, for example, is just a “what if” prediction (i.e. given that forcing). These models can be (and must be) tested in hindcast runs to assess their level of skill at predicting what actually occurred. This skill must be shown before the results of future runs on multi-decadal time scales (as “projections” can be trusted.
I discuss the relationship (and misuse) of the terms “prediction” and “projection” im my paper
Pielke Sr., R.A., 2002: Overlooked issues in the U.S. National Climate and IPCC assessments. Climatic Change, 52, 1-11. http://pielkeclimatesci.wordpress.com/files/2009/10/r-225.pdf
See, for example, Figure 6 in that paper. In retrospect, I should have also emphasized that the small box” in the figure (i.e. a model result) can fall outside of the larger box (i.e. reality).
Bart also writes
“Roger considers projections and predictions to be synonyms”
They are. The only difference is that a “projection” is a prediction if a certain forcing(s) is applied. These are more properly called “sensitivity” experiments. The only way we know they have skill is running them in hindcast where we know what the forcings are (within some level of uncertainty). This is where ensemble runs should be focused.
Bart further writes
“The purpose of a projection is to depict the possible (plausible) evolution of the system. To my opinion, the process of decision making is not dependent on the (quantitative) predictions provided by climate models, but by the plausibility that the future will bring situations to which the current system is not well adapted. For that a set of imperfect projections is far more useful than a single optimized prediction (which, like a weather forecast with a perfect model, can still be wrong).”
Running the models to produce “plausbile” scenarios is reasonable, but it is very expensive in terms of time and other resources. Even more importantly, unless they can actually be shown to be “plausible”, it is not appropriate to present to the impacts community without the disclaimer that they have not shown skill at predicting the climate metrics of interest when the models are run in hindcast.
My second comment is with respect to Jason’s post
“After a summary of his 2012 paper, Roger argues that climate models perform poorly in terms of modelling a realistic recent past and presents quotes from some papers to support this claim. While there are certainly problems with GCMs, and these papers point out a few of them, there were 100s of relevant papers published over the same period and many of them show good performance.”
Please list this models (those that show skill at predicting CHANGES in climate statistics on the globa, regional and local scales).
With respect to Jason’s comment
“When Roger says ’the output of these models are routinely being provided to the impact communities and policymakers as robust scientific results’ … I don’t think he means the large model ensembles with their included uncertainty.”
I am referring to these model runs. How can one quantify uncertainty without comparing against real world observations? This can only be done in hindcast runs. The impacts communities would like skillful changes in climate statistics of interest to them (e.g. changes in the tne year average precipitation for the Netherland, ect). Uncertainty cannot be properly assessed just by intermodel and intramodel comparisons.
Jason/Bart – please list examples of the clmiate metrics that are required by the impacts communities (water resources, food, energy, human health, ecosystem function), and the level of skill that the models have shown in providing them when tested against real world data in hindcast runs.
Bart writes
“In his summary he claims that regional climate downscaling fails to provide additional scale. As is illustrated by Jason Evans’ blog (and also mine) this is not fully true: there where local information (mountains, coasts, land use) gives new information, a more reliable projection can be produced.”
What is missed in the discussion is how just local information can improve the skill of Type 4 regional model simulations when they are so dependent on the lateral boundary conditions from the parent model? Skill can be added from Type 1 through 3 regional model runs because there is a real world constraint on the lateral boundaries. This is not the case for Type 4 runs. If the global model has significant errors in the larger scale field, such as the behavior of ENSO, the PDO, the NAO ect, there is no way a regional model can correct for these errors.
Finally (for this post :-)), Jason writes
“In the end, climate models are our best tools for understanding how the climate system works.”
I disagree. Observations of the real world behavior of the climate system provide the best tool for understanding how the climate system works.
This all seems a bit silly. Roger Pielke Sr.’s arguments seem to focus on semantics rather than any substance. As far as using regional model outputs as input to decision-making, surely every useful piece of information, together with associated uncertainties *AND* any arguments for why they might be wrong, should be used in making decisions about handling possible changes in climate. As regional modeling gains skill it should be able to reduce uncertainty and direct resources in a more focused way on those assets at greatest risk of loss, but no matter what the level of skill, the information they provide should not be ignored!
And I disagree with both Jason and Roger – neither climate models nor observations provide the best tool for understanding – rather, an understanding of the essential physical components and how they interact and respond is what should provide the greatest understanding. A recent example of this is the debate that has arisen about the interactions between the jet stream and blocking events and the degree of ice cover in the Arctic. The apparent increase in blocking behavior was (as far as I know) not predicted by models, and before 10 years ago or so there was no observational evidence at all on the matter. Theoretical physical understanding of the interacting components – informed by both models and observations, is essential to understanding what’s going on. And I would up that theoretical understanding – applied through modeling and corroborated by comparison to observations, is what we are striving for especially in regional applications. Increases in extreme events, both precipitation and drought, changes in tropical storm behavior, etc. are all crucial regional climate change effects that we need to better understand, and neither models nor observation are sufficient by themselves to tell us what to expect.
Mr. Smith – With respect to your statement that this is all about semantics, you are missing the point. One of the three uses of models is to invesitgate “processes”. I have used models as such a tool thoughout my career. I agree an observation/model mix is required (e.g. such as with the reanalyses), but the absence of observational validation is what is a fundamental flaw in the use of Type 4 downscaling for multi-decadal impact assessments, as I have explained in detail in my posts.
To present regional multi-decadal climate projections to the impact communities as part of their driving forces and boundary conditions (for their models and process studies), when there is NO skill on this time scale at predicting changes in climate statistics, is a serious misleading application of the scientific method.
I am certain that Roger Peilke needs no support from me; I am sure Roger has forgotten more about how the earth’s climate works than I know. But I do have a dog in this fight; my Canadian taxpayer dollars and the Keystone XL pipeline. So let me state my opinion that Roger is absolutely correct when he writes “Observations of the real world behavior of the climate system provide the best tool for understanding how the climate system works.”
Athur Smith writes “And I disagree with both Jason and Roger – neither climate models nor observations provide the best tool for understanding – rather, an understanding of the essential physical components and how they interact and respond is what should provide the greatest understanding.”
It is hard to disagree with the sentiments of what Athur wites, but he is being idealistic, and totally impractical. It will be at least decades, and maybe even centuries, before we understand the “essedntial physical proceesses” of how our climate works. There are dozens of reasons for this. Let me take but one.
Please visit
http://www.atmos-chem-phys.net/13/1039/2013/acp-13-1039-2013.html
and
http://judithcurry.com/2013/01/31/condensation-driven-winds-an-update-new-version/
Anastassia Makeireva has a highly controversial theory about what causes winds. Her paper took two years before it was published; a process which normally takes two months. In the end, the editor published the paper, not because Anastassia had proved that she was right, but because nobody could pove that she was wrong.
Now if we do not understand what causes winds to blow; if the models have the wrong physics as to why winds blow, then there is a fundamental error in our whole understanding of how our climate works. I have seen no estimate of cost as to how anyone would go about trying to find out if Anastassia is correct, but it must be in the millions of dollars, and there is no sign that this sort of money will be available for decades.
So while Arthur is right on principle, he is totally wrong from the practical point of view. Roger Pielke is completely and utterly correct.
I would like to come back to the usefulness of (imperfect) models for depicting possible future climatic conditions, and illustrate my case around the concept of Future Weather.
Roger states that one cannot consider climate model predictions (his type 4) at the regional scale when their predictive skill in hindcast mode is not demonstrated. The kind of (model supported) scenario construction that is deserving a lot of attention nowadays is the generation of synthetic weather events using a climate model, but cast in a future setting by adjusting the boundary conditions driving the climate system (greenhouse gas, aerosol, land use, …). It allows to imagine conditions to which society may be vulnerable (e.g., a combination of extreme precipitation and a storm surge that hinders discharging the abundant water) but have never occurred yet in the historical record. Running a large ensemble of atmospheric model simulations for present day climate conditions can help to build statistics of this kind of complex multivariate events, and these statistics can to some extend be validated by comparing the generated distribution to the climatological distribution. “To some extend”, because the limited length of the observational record does not allow any estimate of very rare events, which in contrast can be done by a long model integration. Highly relevant for impact and vulnerability assessment, since these statistics enter the norms and design criteria of protection measures, for instance. During this design process the question of the consequence of climate change frequently emerges, as one desires to avoid over-, re- or underinvestment. A rerun of the instrument to generate synthetic weather under future conditions may give insight to the consequence of changes in large scale forcings on the relevant statistics. The weather model used can and has been tested very frequently when applied in its weather forecasting application mode (type 1) and statistics of these in hindcasts (type 2), but are now extended to an application that is to my opinion a blend of type 2 and type 4 forecasts. It is not the pure prediction of the (regional) trends that is aimed for (type 4), but the occurrence of certain (relevant) weather types conditioned on an assumed change in the large scale forcing.
This is an illustration of where predictive skill at the regional scale can be limited (in terms of predicting the accurate trends of the local climate conditions), but where still useful information can be extracted from models applied to these future conditions. The models are credible to the extend they are skillfull in making good weather forecasts, or rather, in representing the climate processes that play a dominant role in generating (regional) climate variability, and this credibility is clearly present and on the rise.
“In the end climate models are our best tools for understanding how the climate system works”. This is disputed by Roger, who refers to observations that best describe the working of the real world. To my opinion, any observation is supported by a conceptual model. The notion of a spatial structure of temperatures, for instance, makes no sense without even a faint idea of the notion of latitude, seasonality, altitude, land, weather.
I don’t think we build our phyical comprehension of the real world solely on observations, we build it on observations that feed a conceptual model of this real world. And in that sense, a climate model is nothing more or less than a conceptual model, necessary to give a frame of reference to the state of the real world, which can never be generated by observations alone (which are incomplete, may be inconsistent, contradicting each other, biased, non-representative, …).
The theory of Makarieva has been proved wrong by Meesters et al (2009), HESS (see http://www.hydrol-earth-syst-sci.net/13/1299/2009/hess-13-1299-2009.html).
Bart van den Hurk, you write “The theory of Makarieva has been proved wrong by Meesters et al (2009),”
I cannot comment on the science of whether Makarieva has been proven wrong or not. I merely note that her paper was published in January 2013, so the editors of the journal did not consider your reference, published in 2009, to be sufficient grounds to warrant the paper not being published.
Hi Bart – Thank you for your further comment. You write
“I don’t think we build our physical comprehension of the real world solely on observations, we build it on observations that feed a conceptual model of this real world. And in that sense, a climate model is nothing more or less than a conceptual model, necessary to give a frame of reference to the state of the real world, which can never be generated by observations alone (which are incomplete, may be inconsistent, contradicting each other, biased, non-representative, …).”
I agree. We need a modeling construction, based on the accepted principles of physics, to permit the understanding of the interelationship between variables. The climate models are “concepts”, as you wrote, but they are then, therefore, hypotheses. With hypotheses, the following requirements are needed as I wrote in my weblog post http://pielkeclimatesci.wordpress.com/2010/11/15/hypothesis-testing-a-failure-in-the-2007-ipcc-reports/.
A useful summary of the scientific method is given on the website sciencebuddies.org.where they list six steps
■ Ask a Question
■ Do Background Research
■ Construct a Hypothesis
■ Test Your Hypothesis by Doing an Experiment
■ Analyze Your Data and Draw a Conclusion
■ Communicate Your Results
While one cannot falsify models, one can falsify model predictions. With respect to climate, multi-decadal model predictions can be falsified as models themselves are hypotheses and can be tested. They are, after all, engineering code, as large parts of the physics, chemistry and biology are parameterized using tunable parameters. Only the dynamic core of these models (i.e. advection, the pressure gradient force, gravity) are expressed in terms of fundamental physics.
As I document in my post in our current debate, the multi-decadal regional and local climate model predictions have been falsified in terms of lacking skill at predicting (in hindcast) important climate variables. Even in terms of the global annual average surface temperature trends, they are close to failing as recent data is examined (e.g see http://rankexploits.com/musings/2013/individual-model-tests-jan-2001-march-2013/)
Finally, I reiterate my request for you and Jason to present papers that document a skill of the multi-decadal (Type 4) regional climate models to predict (in hindcast) the observed CHANGES in climate statistics over this time period.
Until they can show skill (which they have not, in any study that I am aware of), their use to provide multi-decadal predictions (projections) of climate conditions in the future for the impacts and policy communities is inappropriate and misleading.
Hi Bart
You write
“The kind of (model supported) scenario construction that is deserving a lot of attention nowadays is the generation of synthetic weather events using a climate model, but cast in a future setting by adjusting the boundary conditions driving the climate system (greenhouse gas, aerosol, land use, …). It allows to imagine conditions to which society may be vulnerable (e.g., a combination of extreme precipitation and a storm surge that hinders discharging the abundant water) but have never occurred yet in the historical record..”
In my view you are making an assumption without a robust foundation. You have not demonstrated that these “synthetic weather events” are actually possible in the real world. They are accepted only because a model produces these events. Yet, if these models cannot skillfully predict changes in climate statistics in the past few decades, there is no basis to accept their prediction of changes in weather events (as synthetic weather) in the future as realistic.
If the impact community insists on using them, they should have the clear disclaimer sent along with the results that states “these climate results are presented without any demonstration of skill at predicting (projecting) changes in climate statistics when run over the last few decades”.
In terms of generating synthetic weather events, I presented an example by Kerry Emanuel –
Emanuel, K., S. Ravela, E. Vivant and C. Risi. 2006: A Statistical-Deterministic Approach to Hurricane Risk Assessment. BAMS. ftp://texmex.mit.edu/pub/emanuel/PAPERS/hurr_risk.pdf
This uses the historical record (such as reanalyses) to introduce many more plausible landfalling hurricanes than actually occurred, in order to assess better risk.
The proper question we should ask, of course, should be:
What are the thresholds of climate changes that must occur before a negative effect occurs for particular key resources?
As one of the next step in our debate, please list what are some of the requirements needed in terms of changes in weather events (e.g length of growing season; days above 35C, etc) needed by specific impact communities? Then, what evidence is there that the models can skillfully predict plausible changes in these events? Generating synthetic weather events without showing a foundation of skill is not scientifically robust.
I propose,as a robust method, using our bottom-up approach, where we seek to assess these thresholds, and then ask the question, could real-world weather events occur to cause such a threshold to be passed? These real-world events can be constructed not only from the historical record, but, for example, from the paleo-record and by sequencing different historical time periods together (e.g, the driest 10 years in the historical record, etc). Even arbitrary scenarios could be constructed, built on reanalyses; for instance, adding 10% more water vapor on the lateral boundaries of a Type 2 downscaling run.
We do not need to spend all of the money and time running the climate models to create synthetic weather events, which may have no basis in reality, when we can create these scenarios much more efficiently using the robust approach I outlined above.
I looked into the CET variability, as the most scrutinised and longest regional set of data, also having good correlation with N. Hemisphere and some with global variability. Graphic results and extrapolation into future (with very short comments) are shown here:
http://www.vukcevic.talktalk.net/CET-NV.htm
Below is a ‘guest’ comment from Gerbrand Komen, who is sitting in the advisory board of Climate Dialogue. Gerbrand is the former research director of KNMI and has a special interest in the current discussion about RCM’s. He emailed it to us but with his permission we bring it into the dialogue because we think it can help to structure the discussion. Marcel
Guest comment by Gerbrand Komen:
Hi Marcel,
I am following the discussion between Bart, Jason and Roger with great interest. This is a very important topic indeed, and I hope that this discussion will not only lead to better mutual understanding but also indeed to a better insight in the way in which model results should be interpreted.
Perhaps you could benefit from some of my observations.
First of all, Bart, Jason and Roger seem to agree that combining observations and models is the best way forward if you try to understand the climate system.
But there is also some real disagreement.
Roger repeatedly makes the following points:
· There is no difference between predictions and projections.
· Climate models have no skill in a hindcast mode and are therefore not useful for ‘type 4’ predictions.
· Adaptation (always important, with and without climate change) should be done on the basis of a vulnerability analysis.
Roger also feels that the use of RCMs for the development of adaptation strategies is cost-inefficient.
Jason and Bart both maintain that RCMs can (or may?) provide useful additional information. As an example Bart mentions the case where excessive precipitation and an extreme storm surge coincide.
I hope the discussion will clarify a few questions that came to my mind.
Question 1a: What exactly is a prediction?
I tend to agree with Roger that there is no sharp distinction between predictions and projections. They are both modelled constructions of the state of the climate system at a future time. The real question is about the belief in the possibility that these predictions may verify. This is not black or white, because the prediction may be approximately correct for some variables and wrong for some other variables, and because one can distinguish different degrees of beliefs about the likelihood that a prediction will (or might) come true [very uncertain, possible, very unlikely, . . very likely . .]. Roger seems to assume that skill in hindcasts is necessary for usefulness. I can see that skill strengthens our belief in the likelihood, but in my essay [http://bit.ly/cCb1n2 or http://home.kpn.nl/g.j.komen/Uncertainties.pdf] on Rogers website I have made two points that may be relevant for the present discussion: 1. One should realize that there is ALWAYS a chance that predictions do not come true, even if the model has shown skill in hindcast studies; 2.There are a number of tests (but more than just the skill in a hindcast) that feed our belief in the usefulness of a model, indeed in a sort of Bayesian manner.
Question 1b: What exactly is meant by the word skill?
There are technical definitions, such as
http://www.ecmwf.int/products/forecasts/guide/Measure_of_skill_the_anomaly_correlation_coefficient.html, but, more loosely speaking, the word skill could also be used to indicate agreement between modelled and observed quantities. It seems important that Bart, Jason and Roger use the same definition.
Question 2: How robust is the distinction Roger makes between 4 different downscaling types?
I have my doubts. Bart also, apparently. He somewhere called his scenarios as ‘between type 2 and 4’. To me it seems that there are different ‘dimensions’ to the problem:
1. how are we nesting, initializing and forcing?
2. how are we validating?
3. what prediction horizon is considered?
4. how are we using the results?
Roger seems to project this on a one-dimensional space (type 1 – 4), but perhaps a more subtle division could benefit the present discussion.
Question 3: (for Bart and Jason): Would it be possible to be more specific about the added value of RCMs by giving a list of references and quotes?
This is exactly what Roger has been asking for several times. Roger gave specific lists of model failures, but I suspect that he has been cherry picking. It would be very useful therefore to also have a list of model successes. From my own extensive modelling experience I know that there always are aspects that are not well-described by a model. By developing one’s belief in a model one has to weigh failures and successes. Roger – and many others – seem to have the outdatedPopperian idea that you can refute a model of a complex system. This is simply not true. For example, in our ocean wave prediction model the wave height was generally better computed than the wave period. So we knew that reality was more complex than our model, but this did not make our model useless. In fact, the results have been widely and successfully used. So for the present discussion, it seems essential to have lists of both successes and failures of RCMs
Question 4: Would it be feasible to focus on one specific vulnerability case, and to explore what would be the difference between the approaches of Roger on the one hand, and of Bart and Jason on the other hand?
An interesting case might be the precipitation/storm surge case mentioned by Bart, but any other specific case would also be helpful.
So, these are my questions. It would be great if Bart, Jason and Roger could come up with answers.
To end, it may be helpful when I indicate how I try to understand the different viewpoints in a simplified, daily life example. In daily life we make scenarios all the time. For example, I can imagine getting involved in a traffic accident at some future time. So this is a realistic scenario, and a statement about what might happen. As a result I take insurance. Fortunately, I do not need to run a sophisticated agent-based model of traffic in the Netherlands to generate this scenario. The observation that traffic accidents occur and the fact that I am not very different from other people suffices. This is more or less what Roger says about RCMs. However, I can imagine that an insurance company would want to run sophisticated models to investigate whether the accident rate goes up, if, for example, the traffic density increases. And this is the position of Bart and Jason. This suggests that the value of using models not only depends on the trust one has in the model but also on the particular question that is being asked. Of course, I am not sure whether this analogy stands, but perhaps reactions to question 4 can shed light on this.
So these are my observations. I hope they are helpful, and, above all, I really hope that the discussion between Bart, Jason and Roger on climatedialogue will bring us forward.
All the best,
Gerbrand
Hi Gerbrand
Thank you for the opportunity to further clarify. I will answer each of your questions.
Thanks Gerbrand for this useful attempt to stratify the discussion. Let me, like Roger, go through your questions.
What is a prediction? What is skill?
I agree with Gerbrand that it is not the technical definition of a prediction that is of importance, but the credibility associated to it. For predictions at the decadal time scale, as Roger identifies in his Type 4 application, assessment of skill is actually barely possible. Even a perfect model can deviate significantly from past observed trends or changes, just because the physical system allows variability at decadal time scales; the climate and its trend that we’re experiencing is just one of the many climates that we could have had. I don’t know how to assess skill of decadal trends, and so do not require models to reproduce the past trends. A measure of skill of predictions thus should be that the observed climate trends fall within the range of an ensemble of hindcast predictions. Similarly, a “prediction” of a future state can be very uncertain, and is in fact useless when interpreted as a deterministic forecast. It is a projection of a possible range of future conditions that is of interest, and also is the guidance for adaptation measures. Measures that are able to adapt to the range of projected futures are considered robust. And again: I don’t think that real-world adaptation measures are solely based on model projections of future climate, nor that the only role of climate predictions is to support adaptation design. They help to get insight in the working of a complex system, and part of this knowledge is used for adaptation related planning.
How robust is the type typology?
I agree with Gerbrand that Roger’s typology is too strict, and that an application dimension should at least be added. Is it designed as a deterministic forecast? Or a probabilistic one? Or one that generates understanding underlying our decisions? Those applications give very different criteria for skill and consistence.
Examples of skill
In terms of a contribution to the understanding of the working of the climate system I have already given a few examples of skill in my initial post. But some examples where regionalized or higher resolution models do provide more skill in generating relevant (changes in) climate statistics are for instance
* Changes in extreme wind conditions related to small scale hurricane-type storms can not be skillfully detected in models that have a resolution to coarse to resolve this storms. Haarsma et al (http://onlinelibrary.wiley.
* I was just recently pointed at a study by Cretat et al (http://link.springer.com/
I will continue to look for more examples.
A test case
The role of models to explore implications of the combined occurrence of surges and heavy precipitation events has already been discussed before. Roger suggests to start from the vulnerability perspective and take the example of Amsterdam, where sea level rise is an issue of concern.
I agree that the application of higher resolution model equipment does not help to reduce the uncertainty range of the sea level in the Amsterdam harbours: assumptions about changes in the heat storage, icecap melting and changes in the gravity field dominate this uncertainty range (although some regiona features related to oceanic circulation and heat redistribution may be better resolved in higher resolution models). The assistence of higher resolution projections to the vulnerability assessment of Amsterdam may lie in the analysis of consequences of an assumed sea level change on the probability that a major storm or inland precipitation event (or a combination of these) lead to water levels that are disruptive for the city. One can take an historic event, reproduce this in a state-of-the-art weather forecasting model (which is shown to have skill), but alter the boundary conditions related to the sea level. Likewise, such a scenario study can be done with different boundary conditions related to population or building density, traffic, capital, … This kind of scenario studies is being used in planning the mitigation of disasters by localizing weak spots in evacuation routes, communication nodes etc. To my opinion a useful application of high resolution models set in the context of a future condition.
Hi Bart
Thank you for your follow on comment. I have a few responses:
I agree with you that
“a ‘prediction’ of a future state can be very uncertain, and is in fact useless when interpreted as a deterministic forecast.”
However, you add
“It is a projection of a possible range of future conditions that is of interest, and also is the guidance for adaptation measures.”
This is not consistent.
First, “probability” predictions (projections), as applied in multi-decadal climate simulations, are deterministic. They are being based on an ensemble run of model realizations (with different models, different initial conditions etc) and their spread is then used to estimate a “probability”. This spread results because the model equations provide a deterministic set of results that each can be different since the climate is a chaotic nonlinear system both in the model, and even more so in the real world.
Thus these probability forecasts must also be validated in hindcast runs as a necessary condition to at all claim they should be used for input to impact studies for the coming decades. Multi-decadal predictions of climate probabilities, as well as all climate statistics based on the global and regional and global climate models are deterministic model exercises.
On the typology of downscaling, please be more specific with examples in the literature. I know of no study that cannot be assigned to one of our four types.
In terms of the examples of “skill” that you provide, none are with respect to type 4 downscaling which is the type used by the impact communities for their planning for the coming decades. Indeed, the title of the paper you list
“Changes in extreme wind conditions related to small scale hurricane-type storms can not be skillfully detected in models that have a resolution to[o] coarse to resolve th[ese] storms”
appears to contradict your statement!
I agree that the ECMWF has provided outstanding improvements in forecast skill, but this is on the one or two weekly time periods, NOT on the multi-decadal time scale. ECMWF weather forecasts involve Type 1 downscaling.
On the paper Cretat et al (http://link.springer.com/article/10.1007/s00382-013-1796-7
they are not looking at CHANGES in climate statistics, so their statement that
“The RCM performs at least as well as the most accurate global climate model, demonstrating a clear added value to general circulation model simulations and the usefulness of regional modeling for investigating the physics leading to intense events and their change under global warming.”
overstates their findings. Moreover, if the RCMs performs
“at least as well as the most accurate global climate model”
is hardly an endorsement for the RCM adding substantial skill.
They also write
“The majority of the AGCMs and AOGCMs greatly overestimate the frequency of intense events, particularly in the tropics, generally fail at simulating the observed intensity, and systematically overestimate their spatial coverage.”
and
“Both RCM simulations provide similar results. They accurately capture the spatial and temporal characteristics of intense events, while they tend to overestimate their number and underestimate their intensity.”
This example that you provide actually further makes my point that the regional projections of climate on multi-decadal time periods are challenged with
1. even reproducing current climate when run in hindcast,
and
2. their remains NO skill at predicting in hindcast CHANGES in climate statistics
On the test case, I agree. High resolution models can be applied to better understand the climate processes which could amplify a rise in sea level. For example, running a regional model with lateral boundaries from a reanalysis which has the jet stream wind associated with an oncoming historically observed storm increased by 10% to see to if the resulting winter storm could produce an even higher storm surge event.
This is, however, quite a distinct approach from running a multi-decadal global climate model with added CO2 to present to the impacts community (such as the government of the Netherlands) as a robust projection.
I still retain my request for specific detailed documentation when you write
“To my opinion a useful application of high resolution models set in the context of a future condition.”
You still have not presented examples of skillful predictions (in hindcast) of changes in climate statistics. Since this has not yet been provided, the answer to the question
Are regional models [with respect to robust multi-decadal projections of changes of climate statistics] ready for prime time?
is clearly still NO.
Hi Roger,
In an earlier comment you wrote:
In your blog post you mentioned a lot of recently published papers that show model simulations don’t do well compared to observations even in hindcast. However did any of these studies determine the skill in the way you defined above? Are you aware of any studies that followed your definition? If not then apparently you use a definition of skill that is not being used at all. What would be the reason?
I ask these questions because I think different definitions are used for models being skillful, so we should get clearness about this issue.
Thanks
Marcel
Hi Bart,
In your last comment you wrote about “skill” (bold mine):
In an earlier comment you wrote:
Together these two paragraphs are quite confusing. And I think it is relevant for the discussion whether the models have “skill”. On the one hand you say “I don’t know how to assess skill of decadal trends” and on the other hand you also claim that the “prediction of mean temperature at the regional scale can be done fairly well given the robust temperature trend”. Now the last claim sounds like “models have skill to me”. Could you clarify this?
Marcel
Hi Marcel
The authors’ of the papers that I listed do indeed discuss skill at predicting (in hindcast runs) the ability of multi-decadal climate model runs to simulate the real world observed climate. A few of them also look at changes in climate over the past decades. Even with respect to current climate, however, as I document in the listed papers, the models are having serious issues with faithfully reproducing the climate over this time period.
In terms of climate change, except perhaps for the global annual average surface temperature trend over the last century, the models’ are not showing evidence of an ability to accurately predict other changes. Even the observed trends in the global annual average surface temperature during the last decade are falling below most of the model predictions.
The burden, of course, is for authors that present hindcast multi-decadal climate projections, to provide quantitative documentation of the ability of their model to predict changes in the climate metrics that are requested by the impact and policy communities. To present results for future decades without such an examination of skill is either naïve on the part of the modelers, or disingenuous.
My challenge to Bart (and Jason if he comes back into the discussion) is to present their precise definition of what should be the formal tests of skill for multi-decadal climate projections. These tests must be performed using the framework of Type 4 downscaling.
Roger Sr.
Hi Roger and Bart,
Bart, in your first reaction to Roger you wrote:
How could something “plausible” not being quantitative? I think Roger’s main point is that a first condition for making plausible projections is that the models have skill in hindcast. That would increase the plausibility of the model output. Roger is just saying that as long as there is no skill in hindcast why bother to spend so much money and research time on doing projections. Better spend the money on improving the hindcast skill or – if you really want to do something for the adaptation community – follow the bottom up approach.
Let’s make it more specific and talk about the KNMI scenario’s for The Netherlands. The four scenario’s generated so far say that The Netherlands will warm 1 or 2 degrees in 2050. Now I would say this is a quantitative prediction/projection. I suppose you would say 1 or 2 degrees of warming is more plausible than 3 degrees of warming or 0 degrees of warming. Is that right? So there is some quantification?
Now Roger, are such regional model scenario’s for 2050 meaningless to you? Would you say we cannot say anything meaningfull about the temperature in The Netherlands in 2050? If so does that mean that for you 1 or 2 degrees of warming in 2050 is not more plausible than other amounts of warming?
Marcel
Hi Roger and Marcel
I do agree that the examples I gave do not demonstrate skill of changes in climate statistics. However I still feel uncomfortable with your definition of required skill. I think we should conclude that we agree on the fact that on shorter (decadal) time scales GCM/RCM have shown little regional skill to predict/hindcast observed changes. But that does not necessarily imply that they are useless or have no skill on longer time scales.
Let’s try to define the information needs required to assess changes in risks that adverse events strike society and lead to the need to adapt to these. A high-impact event can be considered as an expression of natural variability, that acts randomly, on short time scales, and thus gives quite different criteria for defining skill in their prediction than the gradual change in the background climate from which these extreme events emerge. Even while one cannot exactly assess the risk for such events in present day conditions, one can assess that the risk for being exposed to particular events changes when background conditions change. Just like the fact that the chance for a temperature higher than 25 degrees is higher in summer than in winter, one can deduce that flood risk increases with sea level rise, or with enhanced precipitation in the upstream headwaters. Projections of these changes of risk using models in which changes in the background climate are incorporated, and applied using models that do a fair job at the short time scale (like high resolution weather prediction, or hydrological discharge, or…) is thus a viable procedure, and does yield added value.
Another confusing component in the discussion is the time scale issue. Any extreme, any risk, any trend comes with the definition of a time scale. And natural variability plays a role at nearly any time scale as well. As I indicated in an earlier blog, the natural variability at decadal time scales hinders the validation of any projection against observations, as these observations reflect just one possible trend. At relatively short time scales (say, a couple of decades), natural variability will dominate over systematic climate change, and there is more added value to expect from ensuring that the projections reflect natural variability adequately than to assess the degree to which the background climate changes. This is what you would demand if you say the models in hindcast simulations should show skill; you primarily demand that we can predict the phase of natural variations. We agree that at the moment models have little skill to do so (although for some areas initialized GCM runs show skill in some areas).
But at longer time scales the change of the probability of a very rare extreme event is dominated by the change in the background climate, where small changes in the mean can lead to large changes in the probability of extremes (see e.g. the scaling between surface temperature and 99 percentile precipitation by Lenderink et al (2011), doi:10.5194/hessd-8-4701-2011, or the non-linear relation between trends and extremes by Rahmstorf and Coumou (2011), http://www.pnas.org/content/early/2011/10/18/1101766108.abstract). Again, superposition of this changing background climate on projections with high resolution models resolving the relevant process does give valuable insights of the possible changes in risks and vulnerability.
The assumed change of the background climate, as is embedded in for instance the Dutch climate change scenarios (+1 or +2 K increase by 2050), can be interpreted as a quantitative assessment of how much the climate system will warm. We derive these values from the spread of IPCC model responses, realizing that these models do have systematic errors, but reproduce historic variations well enough to have confidence about the approximate range of warming in 2050. We don’t make scenario’s for a 0 degree warming, both because these models (and the theory that is embedded in them) show that this is quite unlikely, and because making a scenario for a situation that is similar to the present condition does not pose an unforeseen challenge relative to present conditions. But we also do not give scenarios for a 10, or even 5 degrees warming in 2050, because also this is not supported by our understanding of the climate system response, and thus also not by the climate model integrations. Although they are quantitative numbers, these scenarios are not to be interpreted as (type 4) forecasts, but as a set of conditions that is relevant to consider when making future assessments.
Hi Bart
Thank you for your further comments. It appears we agree on the inability of the global models to simulate the natural variations on decadal and, presumably longer time scales, of large scale circulation patterns such as ENSO, the PDO, the NAO etc. It is these features, of course, which have so much influence on the weather in western Europe and elsewhere.
Where we still disagree, however, is the claim that a long term background climate can be skillfully predicted. What is your basis for this claim? Over the last decade or so, the models have not shown an ability to predict the lack (or very muted) change in the annual average global surface temperature trend. The lower troposphere has similarly not warmed since ~2000 [e.g. see http://www.ssmi.com/msu/msu_time_series.html%5D When do you assume the warming would restart?
Moreover, if we agree the climate models cannot skillfully predict natural variations, why would you conclude that they can skillfully predict changes in climate statistics needed by the impacts and policy communities? The assumption of a global annual average increase in the coming decades +1C and +2C, is of little use in defining changes in climate impacts at the regional and local scale, which are so dependent in how large scale circulation features would change in the coming decades. The impacts communities care about such things as the growing season, warm season precipitation, snow coverage, soil moisture, etc, for their locations, not an annual average change in the global surface temperature. These climate metrics are much more dependent on the large scale circulation features than a global annual average surface temperature trend.
If one wants to assess the risk due to +1C and +2C, the impact models themselves can be run with this increase prescribed directly into their models. There is no need to run a computationally expensive multi-decadal global climate model downscaled to a region to make such a prescription to the impact models. Indeed, any results that are provided of changes in regional and local climate statistics have no skill as we have been discussing in our dialogue, and we both seem to agree on.
My bottom line is that while the global climate models, when run with added CO2 and other greenhouse gases, show that this is a warming effect, they are inadequate tools to assess the consequences of these human climate forcings on the regional and local scale. Even worse, the models inadequately include the diverse myraid effects of aerosols and land use/land cover change on the climate system, so they are already hindered in their ability to accurately represent the real world spectrum of human climate forcings.
You wrote with respect to the results from the multi-decadal global climate model projections that
“Although they are quantitative numbers, these scenarios are not to be interpreted as (type 4) forecasts, but as a set of conditions that is relevant to consider when making future assessments.”
and
“The assumed change of the background climate, as is embedded in for instance the Dutch climate change scenarios (+1 or +2 K increase by 2050), can be interpreted as a quantitative assessment of how much the climate system will warm.”
However, you did not present evidence as to why these results (this “set of conditions”) should be considered as the primary framework to communicate regional and local climate conditions expected in the coming decades to the impacts and policy communities. At most, they are just sensitivity studies that provide documentation that the climate system is affected by adding CO2 and other greenhouse gases to the atmosphere. Beyond this, they have shown no skill at predicting changes in climate statistics, as we now both appear to agree on.
Thus, to write that we know
“how much the climate system will warm”
overstates where we actually are in terms of climate science. This sentence should be written, in my view, as
“The assumed change of the background climate, as is embedded in for instance the Dutch climate change scenarios (+1 or +2 K increase by 2050), can be interpreted as ONE assessment of how much the climate system COULD warm.”
And, of course, we do not need to global climate models to run impact models with an annual average increase in the mean surface air temperature of +1C and +2C prescribed for the Netherlands. We can prescribe these directly into the impacts models, as I wrote above.
Thank you again for the opportunity to discuss. We are making excellent progress in showing where we agree, and where we still disagree.
Roger Pielke Sr. in his latest comment seems to be what I can only describe as trolling. Does he deny the standard WMO definition of climate?
“Climate in a narrow sense is usually defined as the “average weather,” or more rigorously, as the statistical description in terms of the mean and variability of relevant quantities over a period of time ranging from months to thousands or millions of years. The classical period is 30 years, as defined by the World Meteorological Organization (WMO). These quantities are most often surface variables such as temperature, precipitation, and wind. Climate in a wider sense is the state, including a statistical description, of the climate system.”
http://www.wmo.int/pages/prog/wcp/ccl/faqs.html
Skill in describing the statistical distribution of surface variables is very different from skill in predicting specific values of those variables over a limited period of time (like a decade). This is very fundamental. Boundary-value vs initial-value problems. Pielke seems to be deliberately confusing the two here. Why?
Dear Roger
Indeed, there is a number of issues that we can agree on, and a few where we remain to have a different point of view.
You do conclude that we tend to agree that current models are unable to simulate the natural variations on decadal (and longer) time scales. However, I think there remains a disagreement here, and that refers to the difference between simulation and prediction. Models are increasingly good in simulating ENSO, NAO, but the predictability of these phenomena remains to be limited owing to unpredictable components in the forcings and dynamics.
Indeed, we also disagree in the ability to predict regional patterns of changing background climate. Van Oldenborgh et al (2012), to which I referred before, clearly demonstrate that most models do a fair job in simulating the observed spatial patters of warming; what is lacking is a good predictability of the variability around these trends.
Even with imperfect predictions of regional trends, there is credibility in assuming global warming to be somewhere between 1 and 2 K in 2050. Indeed, there will always be the possibility that – owing to known and unknown model deficiencies and to natural variability – the true warming will be well outside this range, similar to smaller or stronger deviations at the regional scale. But the confidence that this is the right order of magnitude comes from that combination of models, basic undestanding of the greenhouse mechanism and involved feedback, and observed trends. Again, we agree that the probability that the true trends will be outside this range will be larger than zero, but I do think that the phrase that these climate projections are not credible is not justified. With our scenarios we do not attach probabilities to any of these scenario values, but we do consider these values to be valuable benchmarks for assessing the impact at regional scale. For that reason, I fully embrace your proposed rephrasing, that the numbers embedded in the Dutch Climate Scenarios can be interpreted as ONE assessment of how the climate COULD warm. But an assessment that, given the evidence we see, does make sense.
Your statement that complex climate models are not needed for assessing local impacts of changing climate statistics and can be replaced by simple permutation of the forcing conditions is sometimes true and sometimes it clearly isn’t. It really depends on the application at hand. We do have a wide experience with climate impact assessment people (see for instance my overview in the volume on Society Vulnerability you recently edited; Van den Hurk et al, 2013: Vulnerability Assessments in the Netherlands Using Climate Scenarios. Climate Vulnerability: Understanding and Addressing Threats to Essential Resources. Elsevier Inc., Academic Press, 257–266 pp.). And for some applications a simple scaling may be sufficient, as long as all relevant variables can be well scaled with a single temperature measure. However, precipitation intensity changes nonlinearly with temperature (Lenderink, and van Meijgaard, 2008; doi:10.1038/ngeo262), circulations and the frequency of dry/wet spells may change (Haarsma et al, 2009; doi:10.1029/2008GL036617), simultaneous occurrence of floods and storm surges changes nonlinearly (Kew et al, in review), and this is just the “meteorological” part of the spectrum. Even seemingly “straightforward” applications like the assessment of the impact of changes in the mean annual cycle of temperature on shifting butterfly populations did find more climate variables than just temperature to be of importance (see e.g. WallisDeVries et al, 2011; http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3172409/), and this generally applies to many sectors and applications. So, a simple scaling is ok, as long as the relevant covarying variables can be changed adequately with this scaling.
We get repeated requests from users not to provide as realistic as possible predictions of the future, but to provide physically realistic pictures that can be understood and interpreted as a possible consequence of altering climate statistics. And to make this realistic pictures, a simple scaling is often not adequate to depict all consequences of an altered background climate, and models are useful tools for this.
To my feeling this discussion is much more than a repeteated rephrasing of statements and opinions. It helped me to realize even stronger that it is the society stakeholder that should be put in the middle of our work, and with respect to this you and your colleagues have echoed a very convincing and true sound in the debate around climate science and its applications. Also we invest most of our research resources not in climate model development per se, but in assessing the information that can be extracted from the existing model archive, in novel applications that are at the heart of interaction with stakeholders, in producing pictures of how our future climate COULD look like, which help feeding the imagination of those that have to take the real decisions. And we fully understand and acknowledge that for most decisions climate is just one of the many entries that is affecting them. By this attitude we surely hope and expect to serve society optimally with our expertise and dedication.
Dear participants, thanks for the interesting discussion. I’d like to make some observations and pose some questions.
Roger and BartH apparently use very different definitions of “model skill”
BartH:
“A measure of skill of predictions thus should be that the observed climate trends fall within the range of an ensemble of hindcast predictions.”
Roger:
“In the context of multi-decadal climate predictions of climate change, skill is defined as an ability to produce model results for climate variables that are at least as accurate as achieved from reanalyses (averaged to correspond to the time period of the model prediction).”
The definition most common in meteorology (correct me if I’m wrong) is that skill is the ability of a model to simulate the observations better than a naïve forecast (e.g. of “no change”).
BartH notes that
“At relatively short time scales (say, a couple of decades), natural variability will dominate over systematic climate change, and there is more added value to expect from ensuring that the projections reflect natural variability adequately than to assess the degree to which the background climate changes.”
Presumably this means that for timescales shorter than, say, two decades it is extremely difficult for a model to show skill in the ‘traditional’ meaning as I gave just above. However,
BartH gave some examples in this discussion however that these sorts of projections can still be useful and also examples where higher resolution and/or RCM’s in comparison with GCM’s resulted in increased model skill.
Roger’s definition is very strict: The model simulations should be better than the (model interpolated) observations themselves (which is what reanalyses could be described as). Would any model simulation (e.g. also a weather forecast) have skill according to this definition? Isn’t this an impossible expectation to have of model simulations?
Hi Bart
Thank you for your comments. My response is as follows.
You wrote
“The definition most common in meteorology (correct me if I’m wrong) is that skill is the ability of a model to simulate the observations better than a naïve forecast (e.g. of ‘no change’).”
This definition of skill is not what is generally applied even in weather forecasting. While “no change” is certainly an option, more generally persistence or climatology are used. As one example of how skill has been defined, see
Landsea,C.W., Knaff,J.A., 2000: “How much skill was there in forecasting the very strong 1997-98 El Niño?” Bulletin of the American Meteorological Society, 81. http://www.aoml.noaa.gov/hrd/Landsea/LandseaKnaff_bulletinAMSSept2000.pdf
In that study, they used
“A previously developed simple statistical tool—the El Niño–Southern Oscillation Climatology and Persistence (ENSO–CLIPER) model—is utilized as a baseline for determination of skill in forecasting this event.”
For multi-decadal climate forecasts, we need to specify what is the measure of skill.
I have proposed that, in order to provide value to the impacts communities, the projections must have skill in predicting changes in regional climate statistics in hindcast runs. Otherwise, we should use the reanalyeses for these impact studies, with, as one set of applications, perturbations prescribed in the impact models to assess at what level of change in climate statistics would result in a negative threshold being crossed in one or more key societal and/or environmental resources.
If the projections have no skill at predicting changes in regional climate statstics, but we give these changed forecasts to the impact and policy communities implying they are robust, we are not being honest with them.
My question to you is
“What is your quantitative measure of skill (value) to justify the cost and expense of running multi-decadal climate projections and providing these results to the impact and policy communities with the claim they are robust?
Makarieva et al. is falsified by looking at the behavior of cloud formation as shown in any number of videos on YouTube (this is not a joke). For example this one which shows that as clouds form they expand as would be expected based on simple thermodynamical arguments (condensation provides heat energy to the surrounding atmosphere, which expands carrying the aerosols in the cloud outward)
The following questions may be central to the current discussion (as suggested by Gerbrand Komen). Hopefully the discussants could give their opinion on each:
1. How do we develop belief in scientific results, in particular in the ability of models to generate information about relevant possible futures?
2. Which global and regional climate model properties support our belief in the usefulness of models as a tool for the exploration of possible futures?
3. What are the relative advantages of the methods proposed by Bart vdH and Roger PSr for advising policymakers about climate (and other) vulnerability risks?
Bart, you ask “1. How do we develop belief in scientific results, in particular in the ability of models to generate information about relevant possible futures?”
Simple. Go back to Physics 101, and rely ONLY on empirical data. Forget about hypotheses like CAGW, and the output of non-validated models, and rely solely on what can actually be measured.
Gerbrand Komen asked me to submit the following:
Hi Roger,
In my previous contribution to this dialogue, I wrote:
“There are a number of tests (but more than just the skill in a hindcast) that feed our belief in the usefulness of a model, indeed in a sort of Bayesian manner.”
This prompted you to formulate the following questions:
A. “On your second comment, what tests are you referring to?”
B. “Also, the use of the word ‘belief’ is not a robust scientific demonstration of added value. Please be more specific.”
Re question A. An example of ‘other tests’ , would be to test to what extent a model is able to represent ENSOs. Even if it is not able to skillfully predict the exact timing of ENSOs it may or may not be able to simulate the (random) occurrence of ENSOs. Many other examples of this type of test can be found in chapter 8 (Climate Models and Their Evaluation) of IPCC/AR4, which assesses both model successes AND model failures.
Re question B. The word ‘belief’ as used here should not be seen as ‘religious belief’, but rather as a technical term used by philosophers, psychologists, and social scientists. There is an extensive body of literature from such fields as the philosophy of science and a field called social epistemology, that study the role of belief in science. It is my guess that this is also relevant for the present discussion.
I hope this clarifies my earlier statement, and I very much look forward to the continuation of your discussion with Bart van der Hurk.
Gerbrand
Bart van der Hurk wrote:
“We get repeated requests from users not to provide as realistic as possible predictions of the future, but to provide physically realistic pictures that can be understood and interpreted as a possible consequence of altering climate statistics. And to make this realistic pictures, a simple scaling is often not adequate to depict all consequences of an altered background climate, and models are useful tools for this.
To my feeling this discussion is much more than a repeteated rephrasing of statements and opinions. It helped me to realize even stronger that it is the society stakeholder that should be put in the middle of our work, and with respect to this you and your colleagues have echoed a very convincing and true sound in the debate around climate science and its applications. Also we invest most of our research resources not in climate model development per se, but in assessing the information that can be extracted from the existing model archive, in novel applications that are at the heart of interaction with stakeholders, in producing pictures of how our future climate COULD look like, which help feeding the imagination of those that have to take the real decisions. And we fully understand and acknowledge that for most decisions climate is just one of the many entries that is affecting them. By this attitude we surely hope and expect to serve society optimally with our expertise and dedication”
This is frightening. Science should not feed the fear of their stakeholders by “producing pictures of how our future climate COULD look like”. This statement provides good insight in how a hype like “global warming” develops
I agree with Arthur Smith that it is all about the physical understanding and this has not been adressed on this site yet.
Hi Bart
You have asked several questions –
“1. How do we develop belief in scientific results, in particular in the ability of models to generate information about relevant possible futures?”
The only robust approach, in my view, is to show that the models have skill at predicting CHANGES in climate statistics on multi-decadal time scales. This must be done, of course, using hindcast runs. Otherwise, the appropriate approach is to use the models within the framework of reanalyses which permits the incorporation of real world observed data.
“2. Which global and regional climate model properties support our belief in the usefulness of models as a tool for the exploration of possible futures?”
I have reported on this in my earlier posts. If the models show a lack of skill and need tuning with respect to predicting (in hindcast)even the current climate statistics on multi-decadal time scales (much less than CHANGES in climate statistics), they are not ready to be used as robust projection tools for the coming decades.
“3. What are the relative advantages of the methods proposed by Bart vdH and Roger PSr for advising policymakers about climate (and other) vulnerability risks?”
The bottom-up, resource-based (contextual) approach to assess vulnerability is an inclusive approach which can accommodate the top-down projections as one plausible set of future environmental conditions. However, it permits the broad assessment of the entire spectrum of risks to key resources, and does not presuppose that the climate part of the risk can be skillfully predicted by the climate models.
Best Regards
Roger Sr.
Hi Gerbrand
You wrote
“There are a number of tests (but more than just the skill in a hindcast) that feed our belief in the usefulness of a model, indeed in a sort of Bayesian manner.”
and, as you noted, I asked
A. “On your second comment, what tests are you referring to?”
B. “Also, the use of the word ‘belief’ is not a robust scientific demonstration of added value. Please be more specific.”
Your first response is
“Re question A. An example of ‘other tests’ , would be to test to what extent a model is able to represent ENSOs. Even if it is not able to skillfully predict the exact timing of ENSOs it may or may not be able to simulate the (random) occurrence of ENSOs. Many other examples of this type of test can be found in chapter 8 (Climate Models and Their Evaluation) of IPCC/AR4, which assesses both model successes AND model failures.”
My Reply- Even IF the climate model could predict the current statistics of the occurrence of ENSO, the NAO, the PDO, etc [which is proving quite a daunting challenge], this, by itself is not sufficient to justify their use to provide projection of CHANGES in the statistics of ENSO, the NAO, the PDO, etc events in the coming decades. The IPCC/AR4 failed to discuss this requirement, and thus, misled the impacts and policy communities on what is the current state of multi-decadal climate prediction modeling.
Your second response is
“Re question B. The word ‘belief’ as used here should not be seen as ‘religious belief’, but rather as a technical term used by philosophers, psychologists, and social scientists. There is an extensive body of literature from such fields as the philosophy of science and a field called social epistemology, that study the role of belief in science. It is my guess that this is also relevant for the present discussion.”
My Reply – I suggest we defer using the word “believe”. The model results should be accepted as robust ONLY when model predictions (i.e. the hypotheses) cannot be shown to be falsified with real world data. This is the scientific method. I have presented peer reviewed papers that do, in fact, falsify the models even with respect to their ability to predict (in hindcast) the current climate.
Thank you for continuing to discuss!
Best Regards
Roger Sr.
Arthur Smith – You wrote
“Roger Pielke Sr. in his latest comment seems to be what I can only describe as trolling. Does he deny the standard WMO definition of climate?
Climate in a narrow sense is usually defined as the “average weather,” or more rigorously, as the statistical description in terms of the mean and variability of relevant quantities over a period of time ranging from months to thousands or millions of years. The classical period is 30 years, as defined by the World Meteorological Organization (WMO). These quantities are most often surface variables such as temperature, precipitation, and wind. Climate in a wider sense is the state, including a statistical description, of the climate system.”
My Comment:
As you noted, this is a “narrow” definition. In the AMS Glossary, climate is defined as
The system, consisting of the atmosphere, hydrosphere, lithosphere, and biosphere, determining the earth’s climate as the result of mutual interactions and responses to external influences (forcing). Physical, chemical, and biological processes are involved in the interactions among the components of the climate system.”
The Glossary defines “climate change” as
“Any systematic change in the long-term statistics of climate elements (such as temperature, pressure, or winds) sustained over several decades or longer. Climate change may be due to natural external forcings, such as changes in solar emission or slow changes in the earth’s orbital elements; natural internal processes of the climate system; or anthropogenic forcing.”
I am unclear why you feel compelled to use the term “trolling” but hope you are now clearer on what is meant by the term “climate”. This broader view was what was adopted in the NRC report
National Research Council, 2005: Radiative forcing of climate change: Expanding the concept and addressing uncertainties. Committee on Radiative Forcing Effects on Climate Change, Climate Research Committee, Board on Atmospheric Sciences and Climate, Division on Earth and Life Studies, The National Academies Press, Washington, D.C., 208 pp.http://www.nap.edu/openbook/0309095069/html/
which I recommend you read.
Hi Roger,
Thanks for your elaborate answers.
In your answer to 1 (24-06 12:32) you state that unless models have proven skill in predicting changes in climate statistics on multi-decadal timescales, they should not be used for projections or predictions. Given your very strict definition of skill and your statements that models, in your view, do not possess given skill, the question arises what alternative sources of information there might be to inform society and stakeholders about the potential or likely (given a certain scenario) trajectories of future climate? E.g. I would argue that even amidst the very large uncertainty, some trajectories are more likely than others (e.g. given a continued positive radiative forcing, warming is more likely than cooling).
Regarding tests of a model’s usefulness (24-06 12:50) you state that models are unable to project changes in e.g. ENSO. You seem focused on what models do poorly, while Gerbrand gave an example here of what models do half-decent: Models do produce the characteristic ENSO behavior at irregular time intervals, even though this is not entered into the model as an input. It arises as an emergent property of the model. Does that not count as a test of usefulness to simulate some of the system’s physical processes, which aids our physical understanding?
Finally, in your reply to Arthur Smith you mentioned that climate change is defined over timescales “over several decades or longer”. These timescales, I believe, is the issue that Arthur wanted to draw attention to: 10-15 years is relatively short in climate terms, so the apparent hiatus in surface warming (while the oceans keep accumulating heat at a fast rate) is not necessarily proof of model failure. Moreover, as you rightly pointed out, the timing of ENSO can not be predicted, so the real-world predominance of La Nina conditions over the past 10 to 15 years is not reflected by the model simulations.
Dear Roger and Andries
The use of the word “belief” in the discussion tends to give it another twist: to what sense can we “believe” that altering climate statistics might give rise to reconsider some of our society decisions when we have nothing more than (imperfect) modls to feed this “belief”? Andries Rosema even finds it “frightening” that generating pictures of how climate statistics could change is feeding these considerations, as they introduce a large risk of alarmistic messages and overinvestments in mitigation or adaptation measures. I tend to turn the argument around: ignoring the possibility that the climate could look different in the future while making these society decisions, is given the evidence we get from a large source of scientific studies (including, indeed, imperfect models) a bit naive.
Let’s try to bring some focus in the discussion around the extend to which “believe” supports decision making, and how the arguments for these decisions are justified. I participated in a writing team lead by Prof Frans Berkhout that analyses the extent to which one can use plausible (model-derived) storylines for decision making processes in society. After appreciating that there are many types of decisions that do have very different dependence on the notion of changing climate statistics, we formally discerned a number of mind frames that form the starting point for this decision making process. From the (draft) manuscript (in review by Regional Environmental Change) I cite
“In their role as decision support tools, scenarios have to compete with other ways in which decisionmakers may reason about the future. According to Rumelhart (1989), there are three common processes for reasoning about novel situations. They are:
* Reasoning by similarity: a problem is solved by seeing the current situation as similar to a previous one in which the solution is known. In this category fall intuition, reasoning by example or experience, generalization and analogical reasoning.
* Reasoning by simulation: a problem is solved by imagining the consequence of an action and making explicit the knowledge that is implicit in our ability to imagine an event. This category includes story-building to mentally simulate the events leading up to a certain ending.
* Formal reasoning: a formal symbolic system, such as logic or mathematics, is employed in the solution of a problem. Examples are formal mathematical models of biophysical or social systems, including climate scenarios.
Rumelhart DE (1989) Toward a microstructural account of human reasoning. In Vosniadou S, Ortony A (Eds.) Similarity and analogical reasoning. New York, NY, Cambridge University Press, pp. 298-312.”
The discussion about the implication of using imperfect models in this discussion making comes down to comparing the second and third ways of reasoning. Roger (and Andries) tend to stress the need for formal reasoning, while I (and Gerbrand) make a plea for the second way of reasoning. Roger could reply again by stating that models that don’t show skill in projecting changing statistics cannot be used for this reasoning by simulation, but I remain to disgree with him: the skill of climate models to project changing climate statistics at decadal time scales can formally not be established due to large role of natural variability, but is also not always required for generating useful information that enters the imagination process.
My main point is: we should ask ourselves what kind of arguments decision makers use when deciding on large investments in climate change mitigation or adaptation. To my opinion it is not true that the only way of reasoning for these decisions is the formal way of reasoning: like any decision an individual or society takes, it is based on a blend of evidence, gut feeling, windows of opportunity, compounding consequences etc. The formal arguments that are used for justification of decisions come from more sources than imperfect models. Arguments coming from the simulations can help to decide on no-regret measures, compounding consequences, and focus at vulnerable sectors or areas, and – I repeat myself – are thus a useful source of information.
Bart Verheggen – Thank you for your response. I have several comments.
With respect to your statement that
“Models do produce the characteristic ENSO behavior at irregular time intervals.”
Please provide the literature citation for this finding, along with a summary of its observational validation.
I do agree with you that running the climate models “aids our physical understanding”.
Indeed, I have published quite a few of these process studies; e.g. see our papers cited in
Pielke Sr., R.A., A. Pitman, D. Niyogi, R. Mahmood, C. McAlpine, F. Hossain, K. Goldewijk, U. Nair, R. Betts, S. Fall, M. Reichstein, P. Kabat, and N. de Noblet-Ducoudré, 2011: Land use/land cover changes and climate: Modeling analysis and observational evidence. WIREs Clim Change 2011, 2:828–850. doi: 10.1002/wcc.144. http://pielkeclimatesci.files.wordpress.com/2012/01/r-369.pdf
Mahmood, R., R.A. Pielke Sr., K. Hubbard, D. Niyogi, P. Dirmeyer, C. McAlpine, A. Carleton, R. Hale, S. Gameda, A. Beltrán-Przekurat, B. Baker, R. McNider, D. Legates, J. Shepherd, J. Du, P. Blanken, O. Frauenfeld, U. Nair, S. Fall, 2013: Land cover changes and their biogeophysical effects on climate. Int. J. Climatol., http://pielkeclimatesci.files.wordpress.com/2013/02/r-374.pdf
However, such applications of the models are distinct from using them to make predictions.
You also write
“….the apparent hiatus in surface warming (while the oceans keep accumulating heat at a fast rate) is not necessarily proof of model failure.”
Actually, it is evidence of a failure of the models to skillfully predict the evolution of the climate system. This transfer of heat to deeper levels (if it in fact is really occurring) was not predicted by the models beforehand. Indeed, the upper ~750m of the oceans has not been warming in recent years as clearly shown in the figure at this url [http://oceans.pmel.noaa.gov/].
To document the failure of the models with respect to this climate metric, see this response in 2005 from Jim Hansen with respect to the GISS climate model –
http://pielkeclimatesci.files.wordpress.com/2009/09/1116592hansen.pdf
“Our simulated 1993-2003 heat storage rate was 0.6 W/m2 in the upper 750 m of the ocean. The decadal mean planetary energy imbalance, 0.75 W/m2, includes heat storage in the deeper ocean and energy used to melt ice and warm the air and land. 0.85 W/m2 is the imbalance at the end of the decade.”
The real-world heating in this layer is not now in agreement with the GISS model projections, even if you claim the heat is now at deeper levels. This discrepancy is only a little over a decade but the obvious question is how much more time would have to pass with such a disagreement between the models and the real world before you would be prepared to reject them as credible tools to create regional climate scenarios?
With respect to other multi-decadal climate model failings, see also the informative analyses by Bob Tisdale in his posts
http://bobtisdale.wordpress.com/2013/06/20/model-data-comparison-daily-maximum-and-minimum-temperatures-and-diurnal-temperature-range-dtr/
http://bobtisdale.wordpress.com/2013/06/15/model-data-comparison-hemispheric-sea-ice-area-2/
http://bobtisdale.wordpress.com/2012/12/27/model-data-precipitation-comparison-cmip5-ipcc-ar5-model-simulations-versus-satellite-era-observations/
http://bobtisdale.wordpress.com/2013/02/28/cmip5-model-data-comparison-satellite-era-sea-surface-temperature-anomalies/
http://bobtisdale.wordpress.com/2013/04/20/model-data-comparison-with-trend-maps-cmip5-ipcc-ar5-models-vs-new-giss-land-ocean-temperature-index/
http://bobtisdale.wordpress.com/2013/06/27/model-data-comparison-alaska-land-surface-air-temperatures/
Bart – Thank you for engaging in this constructive discussion.
Bart van den Hurk – Thank you for your responses.
You write
“In their role as decision support tools, scenarios have to compete with other ways in which decision makers may reason about the future. According to Rumelhart (1989), there are three common processes for reasoning about novel situations. They are:
* Reasoning by similarity: a problem is solved by seeing the current situation as similar to a previous one in which the solution is known. In this category fall intuition, reasoning by example or experience, generalization and analogical reasoning.
* Reasoning by simulation: a problem is solved by imagining the consequence of an action and making explicit the knowledge that is implicit in our ability to imagine an event. This category includes story-building to mentally simulate the events leading up to a certain ending.
* Formal reasoning: a formal symbolic system, such as logic or mathematics, is employed in the solution of a problem. Examples are formal mathematical models of biophysical or social systems, including climate scenarios.”
However, you did not define what is to be the output of such scenarios. I suggest that what the impacts and policymakers need to start with is an identification of what are the consequences of specified changes in climate statistics.
As just one specific example, if the average July maximum temperature in Amsterdam were to rise by 1C, 2C etc in the coming decades what would be the effect in terms of electric energy demand.
Once the risk to demand from such a temperature increase is identified, can we assess the probability, or less stringently, the plausibility, of such an increase occurring over the next several decades?
Then, what is the cost, for each scenario, to make the electric system resilient to such increases in temperature. Policymakers need to determine what they want to pay for each scenario methodology given the uncertainty associated with them. The multi-decadal global climate model predictions can certainly be used as one option for scenario generation.
However, the question is whether the large time and human resources that are used to create these climate model runs could be more effectively used for other scenario methodologies [which would be much less costly] and, even more importantly, in developing responses to climate (and other environmental threats) so as to reduce the risks we face.
The climate models lack of skill when run in multi-decadal hindcast runs makes them a very poor choice as the primary tool to assess how the climate will behave in the coming decades. Not only are they expensive but their skill is lacking what is required.
Thank you for continuing our informative discussion!
On the website http://klimazwiebel.blogspot.com/2013/07/prediction-or-projection-nomenclature.html, I have also been discussing the issue of what is meant by “prediction” and “projection” with respect to multi-decadal climate runs. I am presenting my latest comment below, in response to Hans von Storch’s comment as the size of my reply seems too large for their website.
My comment follow –
Hi Hans –
The attempt to distinguish between the terms “projection” and “prediction”, whether by the IPCC or others, has introduced an unnecessary confusion to the impacts and policy communities regarding the skill of regional and local multi-decadal climate model runs.
The 2007 IPCC defines “projection”, “climate projection” and “climate prediction” as [http://www.ipcc.ch/publications_and_data/ar4/wg1/en/annexes.html]
“Projection – A projection is a potential future evolution of a quantity or set of quantities, often computed with the aid of a model. Projections are distinguished from predictions in order to emphasize that projections involve assumptions concerning, for example, future socioeconomic and technological developments that may or may not be realised, and are therefore subject to substantial uncertainty.
Climate prediction – A climate prediction or climate forecast is the result of an attempt to produce an estimate of the actual evolution of the climate in the future, for example, at seasonal, interannual or long-term time scales. Since the future evolution of the climate system may be highly sensitive to initial conditions, such predictions are usually probabilistic in nature.
Climate projection – A projection of the response of the climate system to emission or concentration scenarios of greenhouse gases and aerosols, or radiative forcing scenarios, often based upon simulations by climate models. Climate projections are distinguished from climate predictions in order to emphasize that climate projections depend upon the emission/concentration/radiative forcing scenario used, which are based on assumptions concerning, for example, future socioeconomic and technological developments that may or may not be realised and are therefore subject to substantial uncertainty”
Clearly a “climate projection” IS a “climate prediction, using the IPCC definitions, when the emission/concentration/radiative forcing scenario actually has occurred. This can be done using hindcast model runs, and these model runs can be quantitatively tested.
The source of much of this confusion appears to be in the assumption by the IPCC and others, that while weather prediction is an initial value problem, climate prediction (on multi-decadal time periods) is a boundary value problem; the term “projection” then being reserved for the later.
However, as succinctly stated by F. Giorigi
F. Giorgi, 2005 : Climate Change Prediction: Climatic Change (2005) 73: 239. DOI: 10.1007/s10584-005-6857-4
“….because of the long time scales involved in ocean, cryosphere and biosphere processes a first kind predictability component also arises. The slower components of the climate system (e.g. the ocean and biosphere) affect the statistics of climate variables (e.g. precipitation) and since they may feel the influence of their initial state at multi decadal time scales, it is possible that climate changes also depend on the initial state of the climate system (e.g. Collins, 2002; Pielke, 1998). For example, the evolution of the THC in response to GHG forcing can depend on the THC initial state, and this evolution will in general affect the full climate system. As a result, the climate change prediction problem has components of both first and second kind which are deeply intertwined.”
You clearly adopt the assumption that the multi-decadal climate runs represent a boundary value problem when you use the term “simulation” and write
“Technically, most if not all projections are conditional/what-if forward simulations, as we run a GCM or RCM with assumed forcing and a random initial state consistent with climatology.”
The 2007 IPCC Glossary has no definition for “simulation”.
The AMS Glossary defines a “numerical simulation” as [http://glossary.ametsoc.org/wiki/Numerical_simulation]
“A numerical integration in which the goal is typically to study the behavior of the solution without regard to the initial conditions (to distinguish it from a numerical forecast).
Thus the integrations are usually for extended periods to allow the solution to become effectively independent of the initial conditions.”
This is a view of this definition “simulation” that is being shown to be flawed with respect to the real world climate; e.g. see
S. Lovejoy, 2013: What Is Climate? EOS 2 JAN 2013 DOI: 10.1002/2013EO010001. http://onlinelibrary.wiley.com/doi/10.1002/2013EO010001/pdf
and
Rial, J., R.A. Pielke Sr., M. Beniston, M. Claussen, J. Canadell, P. Cox, H. Held, N. de Noblet-Ducoudre, R. Prinn, J. Reynolds, and J.D. Salas, 2004: Nonlinearities, feedbacks and critical thresholds within the Earth’s climate system. Climatic Change, 65, 11-38. http://pielkeclimatesci.wordpress.com/files/2009/10/r-260.pdf
Much more important, however, than the name we use is how do you test the quantitative accuracy of the multi-decadal climate model results, regardless of whether they are called “projections”, “predictions”, “simulations” or “forecasts”?
I have presented evidence from the peer reviewed literature; see my post at
https://mwenb.nl/are-regional-models-ready-for-prime-time/
that shows they are not yet robust tools to be used to make multi-decadal climate (predictions; projections) of changes in regional and local climate statistics.
Do you have peer reviewed results that counter this conclusion? I also would value your response to summarize how you conclude we should assess the skill of multi-decadal climate predictions of changes in climate statistics.
Best Regards
Roger Sr.
I posted this comment at http://klimazwiebel.blogspot.com/2013/07/prediction-or-projection-nomenclature.html
Hi Hans
I have another query for you to address.
You write
“But the definition [of a projection] is more of the sort “description of a possible outcome, independent of the method”. A prediction, also independent of the methodology, is an ‘effort for describing a probable outcome’.
This distinction, possible vs. probable is an important difference for dealing with the objects, which may look rather similar, and the IPCC is to be applauded for working the difference out. The advantage is that we can use the same terminology; if you do not want to follow this terminology, this is fine, but if others do, they are not ‘false’.”
What you are telling us, in my view, are that the impact and policy communities are basing their studies and decisions on model results that are “possible” based on the multi-decadal climate model runs. This means, of course, that any of the results I present in my papers on land use change, as just one example, are “possible” and should be used as part of the assessment of risk. I have stated, however, that such model runs are “sensitivity studies” only.
This framework that the results are “possible” and thus are a subset, perhaps a small subset of what will occur in the coming decades, is not a view that is communicated to the impacts and policy communities. Indeed, a review of multi-decadal climate impact papers makes frequent use of verbs such as “will” (rather than “could”), etc when presenting the model results for the coming decades. Figures are usually presented as, for example, the decade 2050-2059, etc.
If you agree that a “projection” is a “sensitivity study” than we are actually in closer agreement than I thought. 🙂
However, you also write that, with respect to the distinction between a “projection” and a “prediction”,
“the IPCC is to be applauded for working the difference out. The advantage is that we can use the same terminology; if you do not want to follow this terminology, this is fine, but if others do, they are not “false”.
It is fine, of course, to use a different word (i.e. “projection” instead of a “sensitivity study”) as long as the meaning is clear and they mean the same thing.
The IPCC, unfortunately, has failed to properly communicate that the “projections” are only providing, at best, what is possible, and does not in any way, provide a demonstration of quantitative likelihood or evidence of any actual regional and local prediction skill on multi-decadal time periods.
I appreciate you engaging in this discussion, as you have permitted a clear statement of what the IPCC is actually doing when they introduce the terminology “projection”.
With Best Regards
Roger Sr.
Dear Roger,
I applaud your comment re. projection vs. prediction and I offer the following paragraph from the paper Koutsoyiannis et al. (2011), which is related to this issue:
Best regards,
Demetris
Reference
Koutsoyiannis, D., A. Christofides, A. Efstratiadis, G. G. Anagnostopoulos, and N. Mamassis (2011), Scientific dialogue on climate: is it giving black eyes or opening closed eyes?, Hydrological Sciences Journal, 56 (7), 1334–1339.
The discussion on the issue of multi-decadal climate prediction skill, and the meanings given to the terms “prediction”,”projection” and “scenario” have continued also in the comments to the posts
http://klimazwiebel.blogspot.com/2013/07/prediction-or-projection-nomenclature.html
http://klimazwiebel.blogspot.com/2013/06/vortragsankundigung-nacht-des-wissens.html [note you can use googe translator to translate the post but most of the comments are in English].
My response to the claim by Hans von Storch regarding his proposed 4th type of climate modeling application is
“Hi Hans
Your fourth type of model application is using the process model application type and just renaming them as scenarios. There is really no fourth type.
I have already documented the inadequacies of this approach in my guest post and comments at
https://mwenb.nl/are-regional-models-ready-for-prime-time/
as well as on your weblog post
http://klimazwiebel.blogspot.com/2013/07/prediction-or-projection-nomenclature.html
What are being provided as “scenarios” to the impacts and policy communities are just model results with no demonstrated skill with respect to the climate metrics of climate change on multi-decadal time scales that they are requesting.
I have shown that the use of scenarios as you define, with any claim that they have skill, is misleading those communities.
Best Regards
Roger Sr.”
Roger A. Pielke Sr.
In the July 2013 issue of the Bulletin of the American Meteorological Society, there are two articles on the issue of regional multi-decadal climate predictions (projections). They are
Pielke Sr., R.A. 2013: Comment on “The North American Regional Climate Change Assessment Program: Overview of Phase I Results.” Bull. Amer. Meteor. Soc., http://journals.ametsoc.org/doi/pdf/10.1175/BAMS-D-12-00205.1
and
Mearns, L.O. , R. Leung, R. Arritt, S. Biner, M. Bukovsky, D. Caya, J. Correia, W. Gutowski, R. Jones, Y. Qian, L. Sloan, M. Snyder, and G. Takle 2013: Reply to R. Pielke, Sr. Commentary on Mearns et al. 2012. Bull. Amer. Meteor. Soc.,. doi: 10.1175/BAMS-D-13-00013.1. http://journals.ametsoc.org/doi/pdf/10.1175/BAMS-D-13-00013.1
In the Mearns et al article, there is a remarkable admission. They write
“We do argue, however, that regional climate models can provide useful information about climate change as long as there is some value in the large-scale infor¬mation provided by the multimodel GCM ensembles. This statement is a logical extension of the fact that regional climate predictability can be derived from regional forcing as well as the large-scale conditions. Hence, one would expect a fraction of the model skill demonstrated by the numerical experiments described in Mearns et al. (2012) to be retained in future pro¬jections given the role of regional forcing remains, and there is some skill in the large-scale conditions derived from the multimodel ensemble of GCM projections.”
These authors ignore that there are major deficiencies in the GCM model runs (and thus in the ensemble) which I provided examples of in my guest post. Adding regional forcing cannot correct errors that are obtained from the larger model through lateral boundary conditions and interior nudging. The regional climate models are slaves to the parent global model results.
Mearns et al write that
“…one would expect a fraction of the model skill demonstrated by the numerical experiments described in Mearns et al. (2012) to be retained in future pro¬jections.”
but fail to show what is the magnitude of this “fraction”. Their use of the words “one would expect” is just an hypothesis, but they present as a matter of faith. This is not the scientific method. It is incumbent on them to quantity this “fraction” both for replicating the current regional climate statistics, but also the CHANGES in the statistics over time.
Simply, stating that
“Approximately 100 articles have now been published using the NARCCAP simulations, with most articles by researchers other than NARCCAP principal investigators (PIs).”
just means that these investigators have been mislead on the actual skill of the NARCCAP results if they are being used in lieu of reanalyses for current climate, and for impact and policy studies of future climate.
In summary, I see that the regional climate modeling community is at a crossroads. They can either ignore the failings of their downscaling approach when applied to multi-decadal regional climate change impact studies and continue to mislead those communities, or they can reassess and focus on the quantification of the predictability of regional climate statistics and their changes on different time and space scales.
If they continue to mislead, however, without quantifying their level of skill at predicting changes in regional climate statistics on multi-decadal time scales, they are not being honest to the impact and policy communities.