Models Blur Science and Advocacy: A Note from Ian Read
Posted by jennifer, July 31st, 2009 - under Opinion.
Tags: Climate & Climate Change
MANY mainstream media science, economic and environmental journalists are not sufficiently trained to be aware of the limitations of models when they present climate-modelled output computated projections not only as data but also advocate this output as supposed proof of the threat posed by anthropogenic global warming, particularly with regard to runaway or catastrophic climate change. This disjunct between the scientific and media presentation when contained within the paradigm of advocacy represents a threat to the integrity and falsifiability of science.
Science seeks the truth in knowledge; (some) media advocacy seeks to propagandise this knowledge. The impact is reinforced if a climate scientist/modeller is directly quoted as an expert, further blurring the line between science and advocacy. This has societal repercussions as the science of anthropogenic global warming (AGW) and the perceived impact of runaway or catastrophic climate change is so model-dependent that the citizenry is not always able to differentiate between the science and advocacy – the implications of which, as regards policy development in term of climate change mitigation, are likely to have a profound effect on society.
Climate models are used, in part, to determine future climate change scenarios related to anthropogenic global warming (AGW) and are described by the Intergovernmental Panel on Climate Change (IPCC) as “mathematical representations of the climate system, expressed as computer codes and run on powerful computers.”
Furthermore, the IPCC states that climate models:
“Are derived from fundamental physical laws (such as Newton’s laws of motion), which are then subjected to physical approximations appropriate for the large-scale climate system, and then further approximated through mathematical discretization. Computational constraints restrict the resolution that is possible in the discretized equations, and some representation of the large-scale impacts of unresolved processes is required (the parametrization problem).”
In other words a climate model is a numerical model or simplified mathematical representation of the Earth’s climate system, or parts thereof. It includes data from real world observations and creates parameters or variables for the unresolved or unknown processes.
The ability of a model to simulate interactions within the climate system depends on not only the level of understanding of the physical, geophysical, chemical and biological processes that govern the climate system but on how accurately these processes are expressed as algorithms within the model, and how closely they represent real-world data. These models do contain some well-established science but they also contain implicit and explicit assumptions, guesses, and gross approximations, referred to as parameters (the parametrization problem mentioned above), mistakes in any of which can invalidate the model outputs when compared to real world observations. In other words computer models are just concatenations of theoretical calculations; as such they do not constitute evidence.
Climate models are data and parameters dependent. Data is based on direct or indirect observations from the environment; parameters (or parametrizations) are defined by the IPCC as:
“Typically based in part on simplified physical models of the unresolved processes . . . Some of these parameters can be measured, at least in principle, while others cannot. It is therefore common to adjust parameter values (possibly chosen from some prior distribution) in order to optimise [author’s italics] model simulation of particular variables or to improve [author’s italics] global heat balance. This process is often known as ‘tuning’.”
Tuning is considered justifiable if two conditions are met: that parameter ranges do not exceed observational ranges where applicable (though this does not necessarily constrain parameter values, which could lead to model output problems); that adjusted (or tuneable) parameters are allotted less degrees of freedom than in the observational constraints used in the model’s evaluation. The IPCC states that,
“If the model has been tuned to give a good representation of a particular observed quantity, then agreement with that observation cannot be used to build confidence in that model. However, a model that has been tuned to give a good representation of certain key observations may [author’s italics] have a greater likelihood of giving a good prediction than a similar model . . . that is less closely tuned.”
Herein lies a problem with modeling: that last sentence implies a subjective judgment on the part of the modeler regarding the greater likelihood of the model providing a good prediction than a less closely tuned similar model. In other words there is the possibility that tuneable parameters can be used as a ‘fudge’ factor, in either model prediction or hindcasting (making the model fit already observed data).
Prominent climatologist Richard Lindzen, writing in “Climate Science: Is it currently designed to answer questions?” a paper he presented at the “Creativity and Creative Inspiration in Mathematics, Science, and Engineering: Developing a Vision for the Future” conference held in San Marino, August 2008, summarises the problem thusly:
“Data that challenges the [AGW] hypothesis are simply changed. In some instances, data that was thought to support the hypothesis is found not to, and is then changed . . . Bias can be introduced by simply considering only those errors that change answers in the desired direction. The desired direction in the case of climate is to bring the data into agreement with models, even though the models have displayed minimal skill in explaining or predicting climate. Model projections, it should be recalled, are the basis for our greenhouse concerns. That corrections to climate data should be called for is not at all surprising, but that such corrections should always be in the ‘needed’ direction is exceedingly unlikely. Although the situation suggests overt dishonesty, it is entirely possible, in today’s scientific environment, that many scientists feel that it is the role of science to vindicate the greenhouse paradigm for climate change as well as the credibility of models. Comparisons of models with data are, for example, referred to as model validation studies [author’s italics] rather than model tests.”
It needs to be kept in mind that computer climate models do not output data: their results are simply computations of the input data. Obviously then, the accuracy or otherwise of the computated output is dependent upon the accuracy of the input data. Furthermore, a climate model’s output is only reliable to the degree that the model’s performance can be validated, not necessarily by comparisons with other models but from raw data recorded or observed from the real world. Of course, tuned parameter corrections may be legitimate but only if they include both those corrections that bring observations into agreement with the model, and those that do not – to exclude the latter is to obfuscate the model’s outcome through omission.
In climate science the most notorious example of obfuscation through omission is what has become known as Mann’s Hockey Stick. Lindzen again:
“In the first IPCC assessment (IPCC, 1990), the traditional picture of the climate of the past 100 years was presented. In this picture, there was a medieval warm period that was somewhat warmer than the present as well as the little ice age that was cooler. The presence of a period warmer than the present in the absence of any anthropogenic greenhouse gases was deemed an embarrassment for those holding that present warming could only be accounted for by the activities of man. Not surprisingly, efforts were made to get rid of the medieval warm period . . . The most infamous effort was that due to Mann et al . . . which used primarily a few handfuls of tree ring records to obtain a reconstruction of Northern Hemisphere temperature going back eventually a thousand years that no longer showed a medieval warm period. Indeed, it showed a slight cooling for almost a thousand years culminating in a sharp warming beginning in the nineteenth century. The curve came to be known as the hockey stick, and featured prominently in the next IPCC report, where it was then suggested that the present warming was unprecedented in the past 1000 years. The study immediately encountered severe questions concerning both the proxy data and its statistical analysis.”
The Mann Hockey Stick has since been discredited by two independent assessments, both statistically and by reference to historical and archeological records, though his initial claim that the current (late 20th century) warming is unprecedented remains within the lexicon of adherents to the AGW hypothesis.
There is a problem here for the reliability of science when models fail, either through prediction or hindcasting, but are still given the same validity as observed or model input data. One could suspect that advocacy is overriding science in this instance. While advocates and politicians might think that the science of AGW is settled scientists and climate modelers need to be able to, and be seen to, separate clearly what is science and what is advocacy otherwise their research may be subjected to political manipulation.
The computated output of climate models, often used in conjuction with models from outside the field of climate science, have been used to construct climate change scenarios, often abbreviated as SRES, an acronym for Special Report on Emission Scenarios. SRES was developed by the IPCC to develop scenarios with which to analyze, according to SRES:
“How driving forces may influence future [greenhouse gas] emission outcomes and to assess the associated uncertainties. They assist in climate change analysis, including climate modeling and the assessment of impacts, adaptation, and mitigation. The possibility that any single emissions path will occur as described in scenarios is highly uncertain . . . Any scenario necessarily includes subjective elements and is open to various interpretations.”
The output of SRES models, alternative views of how the future may unfold, are termed projections. Projections are often stated or implied erroneously, particularly in the media in connection with runaway climate change, as forecasts. This creates the impression that the SRES model output is new data, even proof, as opposed to being simply a projection of computated input data and parameters from a number of sources within and beyond the field of climate science.
Multi-model SRES climate change scenarios are said to create an ensemble of climate change projections. Modellers then consider the spread of these SRES projections, upon which has been built the notion that if the spread is close together then they can have confidence in the projections while if the spread significantly differs then there is uncertainty about the projections even though they may offer a range of possibilities.
The SRES approach is problematic: it is assumptive; prone to exaggerated errors; unscientific. Firstly, assumptions are made about unresolved processes by using tuned parameters while, secondly, errors may be exaggerated by (i) using the computational outputs as new ‘data’ by reintroducing that ‘data’ as inputs into a new model, upon which new projections are predicted and (ii) it assumes not only that a close spread within an ensemble raises the confidence of the prediction but also that a broad spread, rather than disproving the accuracy or otherwise of the models, indicates a range of possibilities, though with a lower (assumed) confidence. Thirdly, multi-model SRES outputs are based on so many assumptions that its use is inherently unscientific because many of the model elements are not falsifiable. It is, nonetheless, a good tool for advocacy though, especially when presented in the guise of science.
This SRES approach has no place in the scientific processes as its outputs can not be verified with real world data: projections are not records and models are not data generators. As yet there is no scientific principle that says that one can derive valid estimates from model outputs until the model output resembles the observed non-modelled data. The uncertainty of an ensemble of climate change projections will always depend on the accuracy of the raw data input irrespective of the spread of projections.
This is not to say that there is no place for models in climate science: even though they are a tool and not data generators there are there are many examples of statistical climate forecasting models providing good projection examples over short time frames. It is important that models, in this context, remain a tool of climate science and not a tool of advocacy.
A problem with climate modelling is that of replication and validation. Because models are tools used in order to calculate, usually complicated, data inputs it is important that the computated outputs can be tested (the models are validated, or not) and repeated (the models’ results are replicated, or not) – by doing so helps remove bias (the models’ outputs are easily influenced by data inputs, flux adjustments and parameterisation) thus increasing the confidence that the models do in some way represent what they are seeking to show. Not to do so means that the models’ computated outputs could be used for non-scientific purposes, such as advocating a predetermined position, without the ability for others, perhaps affected by this advocacy, having the ability to ascertain that the models’ computated outputs actually represent what they seek to show.
This is especially true for climate science, and the repercussions that the computated outputs have on public policy as regards AGW, climate change, tipping points, emissions trading schemes, etc. Furthermore, as much climate science is publicly funded through government grants, etc then it is even more imperative that the funders, ie the public, receive information that can be trusted. It is unfortunate, then, that there is a reticence for some climate scientists and modellers not to share data, especially all the codes or algorithms used that would allow the models to be fully replicated and, if necessary, challenge the validity of the models. This reticence goes to the core of scientific thought and process.
******************
Notes and Links
Ian Read is a researcher, author and geographer with a special interest in climatology and vegetation. He has written over twelve books including The Bush: A Guide to the Vegetated Landscapes of Australia and Australia: The Continent of Extremes – Our Geographical Records.
This article was previously published at On Line Opinion on 23 June 2009 and is reproduced here with permission from the author.
“Even a cursory look at climate models shows exactly the same thing. Just count up the number of variables affecting climate, that you have to get right to make your model work – laughable.”
What variables. What model are you talking about?
Well Louis, think of it this way, you could shove it where the sun don;t shine. Same effect.
Thank you for playing
Eli,
What has been shown is your descent into ad hominems – and hence the bankrupty of your argument.
Thanks for proving the AGW proselytiser’s intellectual vacuity yet again.
SJT “But it is, that is a very useful model to use for understanding electrical current”.
It might be in ‘Electronics for Dummies’, but not for the serious student of electronics. For one, that analogy applies very loosely to copper conductors, not to semiconductors or waveguides. In other words, it has a very limited scope.
A water pipe model leads students into visualizing current in a conductor as if electrons are a stream, like water. The thing that needs to be understood is that electric current travels via charges and not just electron motion per se. The charges travel much faster than the electrons, much like a row of marbles on a grooved ruler transfers the energy from a tapped marble at one end to dislodge the marble on the other end. None of the marbles in the middle move, but the end marble shoots off with good energy (oops…a model).
The point I’m trying to make is that many models are introduced supposedly to make learning easier. It has been my experience that many such models get in the way of understanding what is actually going on. I may come across as being opposed to climate models, but I’m not. I just don’t think we should be trying to understand atmospheric processes based solely on immature models, and certainly not through modelers lacking in physics theory or an in-depth understanding of atmospheric processes..
Eli Rabett “Thank you for playing…”
I get the impression that playing is all you’re doing. You’re the king of the castle, and if people agree with your POV, you’re a benevolent king. If anyone disagrees, you get your nose out of joint.
Ego imposes a severe limitation on intelligence, since both can’t exist in the same mental space at the same time. You need to decide which is more important, ego or intelligence. You can’t have both.
Actually, it’s not a choice, it’s awareness, which is the begining of intelligence.
SJT wrote:
Validation is a mathematical exercise and does not rely on observations of the system you are attempting to model. Verification involves comparisons to real-world data. As I stated, you cannot verify a model until it is validated. I would suggest that if you are ignorant of the process, then perhaps you should study it a little before you make such inane comments.
Eli wrote:
You are right that models do not need to be complete to be useful. They must be validated to be useful though – an incomplete model can be shown to be calculating exactly what it should be, yet due to incompleteness, fail verification. Can you show me a validated climate model?
For those who are lost in this conversation, as an example of why validation is required I will give the following example of the convergence test: take a high resolution digital photo of, say, a persons face. Run a “pixelisation” filter on it at various numbers of in to out pixels. At low values, the face still looks like a face, just blurry. But at some point, it stops looking like a face and is just a pile of coloured blocks. This change is quite easily seen, because a small change in the value makes a huge change in the output. In terms of climate models, we do not yet, as far as I am aware, know the value where that change takes place – in fact, while I do not have the references to hand, it is my recollection that there are published papers showing significant changes in model output for very minor changes in temporal step sizes. This would seem to indicate that those particular parameters are producing little other than numerical noise. Surely Eli, if we DO know these values, you can both quote them to me (or give a cite that shows them), and show model output using the same initial conditions but with slightly different spacial and temporal resolutions and thus demonstrate that a sequence of minor changes produces very similar output – say, 100km, then 90km, then 80km etc grid size with the same initial conditions should show simply more detail, not radically changed output. Ditto for the time step. Can you show this to me please, or cite a published paper that shows it? This would be an important and required step on the way to a demonstration that climate models are useful, but without it, they are simply too unreliable IMO for decisions involving trillions of dollars and the lives and lifestyles of billions of people. Probably still useful as a tool for understanding and/or learning though.
“Validation is a mathematical exercise and does not rely on observations of the system you are attempting to model. Verification involves comparisons to real-world data. As I stated, you cannot verify a model until it is validated. I would suggest that if you are ignorant of the process, then perhaps you should study it a little before you make such inane comments.”
You are ignorant, you mean. The models have been demonstrated to be useful and accurate to a reasonable extent. What you are talking about is a concept for models that is an abstract ideal that is nice to know about, but not really relevant to the issue we are dealing with.
cohers,
A bit o/t but I wonder if Luke, eli et al have seen this great example of peer reviewed “robustness” at JGR?
http://climatesci.org/
Eli, you are clearly blinded by your faith….and judging by so many of your posts an egotistical driven one as well. Oh and apparently not a very nice person either!
“A bit o/t but I wonder if Luke, eli et al have seen this great example of peer reviewed “robustness” at JGR?”
It is not peer reviewed but it is bleeding edge. I think they have tried to push the technology too far with these medium range forecasts, but that is immaterial to the issue of long range climate change models. They do not pretend to claim the accuracy needed to make the forecasts of actual weather into the long range future. Pielke dodges their defence that their forecasts are still reasonably accurate for such new technology, by focussing on a specific example. If he was being honest in his criticism he would acknowledge.
* These forecasts are not related to climate modeling
* He has not looked at their overall success rate, only cherry picked one that was wrong.
SJT wrote:
Then educate me, sir!
I ask nothing more than for you to show me evidence that this is so.
Any engineer knows that such “ideals” are vital to their continuance in their chosen profession – they are held accountable for their decisions. Surely it is not too much to ask that the same information is available WRT climate? Surely you are not suggesting that climate change is of lesser importance than designing a car or a bridge? Standards for proof exist – I ask only that such standards are adhered to. Show me your evidence, and should it prove compelling, then you will have my support. Speculation and expert opinion are nice, but hardly at the level required for public policy, especially when you assert that the proof exists yet cannot or will not supply it.
“There are several other gratuitous claims and errors in Benestad and Schmidt’s paper. However, the above is sufficient for this fast reply. I just wonder why the referees of that paper did not check Benestad and Schmidt’s numerous misleading statements and errors. It would be sad if the reason is because somebody is mistaking a scientific theory such as the “anthropogenic global warming theory” for an ideology that should be defended at all costs.
“Nicola Scafetta, Physics Department, Duke University”
SJT,
That paper was refereed!
Benestad and Schmidt [as in God Gavin from RC] made many errors including a simple one by misapplying an accepted algorithm.
This is a huge and obvious blunder by someone who considers himself to be above scepticism.
But don’t hold your breath waiting for apologies.
Not to mention the huge and obvious blunder by referees who you, Luke et al tout as being so essential to published cred.
Spangles; the Schmidt effort is par for the course; lucia has a repeating thread dealing with Schmidt’s statistical contortions; perhaps the most egregious peered paper recently is this one which simply says because estimates of climate sensitivity have a high degree of uncertainty climate sensitivity must be higher rather than less;
http://wattsupwiththat.com/2009/07/19/insufficient-forcing/#comment-161653
OK Spanglers – you explain what they are debating? (SJT wait for crickets!)
“Surely you are not suggesting that climate change is of lesser importance than designing a car or a bridge? Standards for proof exist – I ask only that such standards are adhered to.”
No, but it is not like designing a car or bridge. It is like modeling a climate.
SJT wrote:
I did not ask if it was “like” it, I asked if it was of lesser importance.
Clearly, the evidence that I require – that the climate models have been validated – is not available, which is as I have said all along. In short, there is no evidence that these models are producing anything other than numerical noise. If such evidence exists, post it, or a link to it, or cite a paper that shows it. Put up or shut up – and you will notice that Eli, who, in other forums incites people to violence and relies on ad hom attacks, has dropped this issue like a hot potatoe. Why? Because he knows I am speaking the truth and that no such evidence exists. In fact, the only published research on the convergence issue for the 3D N-S equations (Ye et al) is that no such convergence exists. The evidence for numerical stability of the models is also negative – most require an “island” at the north pole or they simply cannot perform their calculations – this is unphysical and the “fix” for such things as negative mass is to add to the code and “constrain” values to what’s physically possible. Further, you will never see such model output in absolute temperatures, only as “anomolies”. Why? Because they are off by up to 10C or more in absolute terms!
Climate models are clearly not fit for the purpose of public policy decisions where peoples lives, jobs and lifestyles are at stake. They may have some academic uses, but that’s it. If you wish to believe otherwise, then that is your concern – I do not and will not accept that these things are anything other than cute toys unless and until someone can show me that they have been formally validated. That any govt. would accept these “projections” as being in any way related to reality is disturbing and disgusting.
spangled drongo…. re Benestad and Schmidt paper and Scafetta reply.
This closing commentary from Scafetta is telling:
“I just wonder why the referees of that paper did not check Benestad and Schmidt’s numerous misleading statements and errors. It would be sad if the reason is because somebody is mistaking a scientific theory such as the “anthropogenic global warming theory” for an ideology that should be defended at all costs”.
Roy Spencer has made similar accusation of the modern peer review process as has Lindzen. Spencer went so far as to claim the reviewer did not seem to understand what he was saying. This is also not the first time Schmidt has been called out for his lack of understanding of basic principles. Jeffrey Glassman nailed him on his understanding of feedback and solubility of CO2 in water.
http://www.rocketscientistsjournal.com/2006/11/gavin_schmidt_on_the_acquittal.html
Glassman concludes:
“Nowhere does Schmidt suggest that the models on which he relies to frighten the public might have been validated. He relies instead on an incompetent tutorial to support the AGW conjecture.
….The burden remains on the GCM operators advocates to revise their models. They need to abandon CO2 as a forcing, and instead make the atmospheric CO2 concentration respond to global temperature as dictated by the solubility of CO2 in water. This should be a fatal blow to anthropogenic global warming”.
“Clearly, the evidence that I require – that the climate models have been validated – is not available, which is as I have said all along. In short, there is no evidence that these models are producing anything other than numerical noise. If such evidence exists, post it, or a link to it, or cite a paper that shows it. Put up or shut up – and you will notice that Eli, who, in other forums incites people to violence and relies on ad hom attacks, has dropped this issue like a hot potatoe. Why? Because he knows I am speaking the truth and that no such evidence exists. In fact, the only published research on the convergence issue for the 3D N-S equations (Ye et al) is that no such convergence exists. The evidence for numerical stability of the models is also negative – most require an “island” at the north pole or they simply cannot perform their calculations – this is unphysical and the “fix” for such things as negative mass is to add to the code and “constrain” values to what’s physically possible. Further, you will never see such model output in absolute temperatures, only as “anomolies”. Why? Because they are off by up to 10C or more in absolute terms!”
Just a bunch of rumours.
SJT wrote:
Then you should have no trouble citing evidence that I am wrong. And yet you do not.
C’mon – show me a validation study on any climate model of your choice. Show me ANY peer reviewed paper that demonstrates convergence between the discrete and continuous solutions to 3d N-S. Show me ANY climate model output in absolute values that matches real world measurements. It should be simplicity itself to show these things if, as you insist, I am “wrong” or spouting “rumours”.
The truth is that you cannot show these things to be wrong. The truth is that the models are not up to the standard required for public policy decisions, and what’s more, those who run such models are well aware of this fact, yet propose we change the whole basis of our economy based on their output! The reality is that any engineer who designed something based on an unvalidated model would lose accreditation even if the resultant artifact caused no problems, no deaths and no injuries. This would be so even if later studies showed the artifact to meet or exceed the design specifications in every respect. And yet, oddly enough, we seemingly need to meet no such standard for climate models, and we seem to be going down the road of having faith in these models and making huge changes based on their output – changes that affect more people in more ways than most engineers would dream their product could. And you stand around cheering them on! It boggles the mind.
“Then you should have no trouble citing evidence that I am wrong. And yet you do not. ”
You are the one making the claim, you provide the evidence. All I have seen is a random collection of denialist rumours that are floating around.
SJT wrote:
Alas, I am not making the claim – modellers are. *They* must provide the evidence, and I have asked you to cite it. You have not and cannot because it doesn’t exist. In any case, I *have* provided a cite re: convergence (Ye et al – 2005 IIRC). Do you have a counter-cite? Clearly not, or you would have provided it. AFAIK, it’s the ONLY published work in this area in more than 40 years.
All I have seen from climate modellers is numerical noise. If they wish to be taken seriously, they need to show the models meet standards. They have not and do not. What you believe is no concern of mine, except where it affects me. Provide evidence I need to change, or go away and leave me be.
“All I have seen from climate modellers is numerical noise. If they wish to be taken seriously, they need to show the models meet standards. They have not and do not. What you believe is no concern of mine, except where it affects me. Provide evidence I need to change, or go away and leave me be.”
Model E is available for downloading. You can run a cut down version on your PC.
Neil Fisher “All I have seen from climate modellers is numerical noise. If they wish to be taken seriously,…”
Unfortunately they are taken seriously, and by influential people who either don’t know better or influential people with agendas. Sir John Houghton, co-chair of the IPCC, has obviously had a major impact on the UK government. He’s an Oxford Old Boy, and from what I can gather his only backround in climate science is as a modeler. The IPCC has been steered by modelers since its inception.
We have a serious problem in climate science. Traditional observational science has been pushed aside by virtual science, and for no good reason. The only apparent reason is an agenda of some kind that is more political than anything. Greens and extreme environmentalists, who can’t get voted in on a platform, have found a way to manipulate science to do its bidding.
I find it deeply disturbing that arrogant people would be so dishonest as to force their agendas on the public under the guise of science.