Confirmation Bias at the Australian Bureau of Meteorology
Posted by jennifer, March 26th, 2009 - under News, Opinion.
Tags: Climate & Climate Change
THROUGH his ongoing study of US weather stations, Anthony Watts has uncovered some remarkable examples of poor placement resulting in a warming bias. I am increasingly of the opinion that the problem in Australia is not so much placement of weather stations, but rather how the data is manipulated post collection. It is also difficult to reconcile the increasing interest in, and funding for, climate related research in Australia with the extraordinary deterioration in the actual recording of data. The important official weather station at Willis Island, off the north east coast of Australia, is a case in point with no recordings made for long periods over recent years.
The following note from Steve Jones explains in some detail his concerns regarding the quality of data collection and processes at the Australian Bureau of Meteorology.
Hi Jennifer,
It is my view that the Australian Bureau of Meteorology is currently tampering with its climate data to fit in with the global warming theory.
Here is a very simple exercise you can do yourself to judge the truth (or otherwise) of my claim:
Go to: Weather Station Data:
http://www.bom.gov.au/climate/data/weather-data.shtml Long record temperature sites,
Open (in Microsoft Excel), Click top left box between 1 & A which will highlight all boxes,
Choose: Data, Sort, Column F, OK which will sort them from oldest to newest. Note some have closed so chose ones with the most complete datasets. Also chose ones which are most remote or which you know or figure have been subject to the least change over time e.g. population growth, land clearing etc.
Then go and find them here:
Go to: Climate statistics for Australian sites:
http://www.bom.gov.au/climate/averages/tables/ca_site_file_names.shtml
Choose e.g. (in Monthly Statistics when you go back to chose another one), New South Wales, Gunnedah Pool, Monthly statistics, Click (VERY IMPORTANT)under map All Available, then notice Highest temperature, Lowest Maximum Temperature, Lowest Temperature, Highest Minimum Temperature for each month and also note the date in Column ‘Annual’ when the record occurred.
You will notice for Gunnedah that even though the 19th century comprises only 24 out of 132 years of data that (counting 1900) it includes 6 out of 12 of the hottest days and 8 out of 12 of the coldest days! You will also notice that the period 1975 to 2009 when we have been told the world has been warming most and when extremes of weather have been increasing most that NONE of the hottest days have been in that period but 3 of the coldest have been! Since that period comprises b32 out of 132 of the years of data you should expect a like proportion to that of extreme days of all sorts or 24.24% or about one in four (3 out of 12)as we find with the coldest days. But as NONE of the hottest days occurs in that period (though we should expect one quarter of them to – again 3 out of 12)we can only conclude that Gunnedah at least has cooled in the last 32 years! If you look at the four Annual climate extremes for Gunnedah you will see 24 Jan 1882 (hottest), 4 Aug 1921 (lowest maximum), 27 June 1881 (coldest), 2 Jan 1905 (warmest night).
In other words far from extreme weather events increasing recently it has been 87 years since Gunnedah experienced an extreme weather event! Perhaps the REST of Australia (or the world) has warmed and Gunnedah is the exception?
I have looked at a selection of the more remote sites on the Australian Weather Bureau’s site which have long-term records (e.g. Cape Otway Vic ) and found the same pattern i.e. very few of the hottest days have occurred in the period 1975-2008 and a very high proportion of the hottest days occurred in the 19th century.
You have to choose sites which have long-term records comprising the last quarter of the nineteenth century up to 2008 otherwise some relatively warm years recently will appear to be the warmest when they were not. Obviously large cities WILL have warmed because of the urban heat island effect NOT because the climate has changed.
You have to be VERY careful to notice what years have been averaged in the column ‘Years’ as even though the station may have data from e.g. 1869 to 2008, it may be that the data averaged is only 1970-2008 or the like.
Some sites have moved too and so or not very reliable e.g. Melbourne Regional Office has averages from 1855-2008 but the site only BEGAN in 1908 which begs the question of where the earlier data comes from and is it comparable? Even so, for that site (a very warm site because of the huge growth of the city) only two out of 12 of the hottest days were in that period and they were both back in the 1980′s. The three hottest days by a long shot were in 1938-1940. Sydney Observatory has 6 out of 12 of the hottest days in the defined period but no-one would have thought Sydney had not warmed.
Deniliquin is one of the oldest continuous sites: 2 out of 12 hottest days; Bourke 1 out of 12; Boulia 0 out of 12; Marble Bar 3 out of 12.
I suspect what the ‘global warmers’ have done is to chose the average of ALL sites and since few sites have old records the warmest years on record will be quite recent and therefore the average of ALL sites will show the same pattern but ONLY BECAUSE the older data is missing or drowned out by the more recent data. So it will APPEAR as if there has been global warming when in fact there has not been ANY.”
When I went back to check Cape Otway recently I found that suddenly there had been an increase in the warmest days in recent times.
The BOM’s explanation can be found here:
http://www.bom.gov.au/climate/cdo/about/about-stats.shtml where they say:
“We will soon finish reprocessing many of the data which are used to calculate the climate statistics. This will improve the quality of the statistical information provided to you. In the meantime, the statistics will be based on the same (updated) datasets as used previously. This will mean: You may find small differences when comparing the superseded (pre 2007) version of the climate statistics with the most recent. These changes are likely to be associated with the calculation process or the removal of unsuitable data. Some of the previously identified climate extremes may disappear in this new product. This is associated with data quality issues.”
I view this behaviour on the part of the BOM as unacceptable and serious.
I tried the internet archive to see if I could get back to the earlier version where pretty much ALL the hottest weather had been long ago at Cape Otway as with Gunnedah which will no doubt soon change too. But you can’t get back to the earlier version.
This process is really creating a new reality.
Another strange thing I noticed about the BOM’s data is this: I was looking for data for where I live (Yinnar, Victoria). You can’t have temperature data only rainfall data. (Incidentally you can’t get average DAILY rainfall data anywhere either which was what I was looking for. It would be very helpful to know e.g. whether the first five days of April are wetter or drier on average e.g. for planning future holidays). I looked at the rainfall data for Yinnar here:
http://www.bom.gov.au/jsp/ncc/cdio/weatherData/av?p_nccObsCode=18&p_display_type=dataFile&p_stn_num=085100
when I noticed that 1928 & 9 were incredibly wet (50% more than the greatest BOM recognises (notice 525mm in October!) yet the data for those two years is not included in the averaging or the statistics because there is a month missing from the data (presumably – the Postmaster probably went on holidays!).
The same thing probably happens to the temperature data which would no doubt cause all sorts of distortions to the record.
Cheers,
Steve Jones
Yinnar, Victoria
************************************
Notes
Anthony Watt’s project, started on June 4th 2007, is designed for the express purpose of photographically surveying every one of the 1221 USHCN weather stations in the USA which are used as a “high quality network” to determine near surface temperature trends in the USA. USHCN is a subset of the larger COOP network of stations in the USA, of which there are about 9000. The USHCN subset has been hand picked by the National Climatic Data Center to be more regionally representative due to their placement, length of service and minimum station moves. Unfortunately, the network has fallen into neglect, and the temperature data produced by it is suspect due to microsite biases. See what has been learned so far here in this slide show. http://gallery.surfacestations.org/UCAR-slides/index.html From, http://wattsupwiththat.com/test/
Marked deterioration in Willis Island temperature data quality, December 1, 2007 by Warwick Hughes, http://www.warwickhughes.com/blog/?p=144
BOM Annual Report
My first impression of this threads topic was that it was but rough criticising an organisation for having just a few bad stations and/or processes, that one could imply a confirmation bias or similar.
The only valid way was to either get more data of specific examples, and/or see if there may be anything systemic involved.
One way to start to look for systemic matters is to see how well the place is run, and what the tax payer gets for its money. You know the shareholder returns sort of stuff that one can get for any publically listed entity, good or bad.
Well for starters, there is quite a lot of information on their site, but it is seriously out of date, most up to date data is for 2003-4, nearly 6 years behind.
http://www.bom.gov.au/inside/eiab/reports/html/overview.shtml#P61_18619
Having looked through this stuff cannot one have confidence that there isn’t a conformation bias, or some other deficiency affecting the quality of its outputs.
It’s just not possible to tell, but if their own documentation is anything to go by, they don’t know either, for example:
1. They are 5-6 years behind in their Annual Reports, so neither they nor the Parliament knows what they are up to.
2. Their Corporate Planning is pathetic and involves Goals and Objectives that are just not measurable.
3. There is absolutely no indication as to how the top level Goals and Objectives break down and become measurable entities.
Nick
Paranoid am I? Seems I have rge facts of the situation just about right.
“U.N. CLIMATE PLAN WOULD SHIFT TRILLIONS OF DOLLARS TO FORM NEW WORLD ECONOMY
A United Nations document on “climate change” that will be distributed to a major environmental conclave next week envisions a huge reordering of the world economy, likely involving trillions of dollars in wealth transfer, millions of job losses and gains, new taxes, industrial relocations, new tariffs and subsidies, and complicated payments for greenhouse gas abatement schemes and carbon taxes – all under the supervision of the world body.
Those and other results are blandly discussed in a discreetly worded United Nations “information note” on potential consequences of the measures that industrialized countries will likely have to take to implement the Copenhagen Accord, the successor to the Kyoto Treaty, after it is negotiated and signed by December 2009. The Obama administration has said it supports the treaty process if, in the words of a U.S. State Department spokesman, it can come up with an “effective framework” for dealing with global warming.
The 16-page note, obtained by FOX News, will be distributed to participants at a mammoth negotiating session that starts on March 29 in Bonn, Germany, the first of three sessions intended to hammer out the actual commitments involved in the new deal.
In the stultifying language that is normal for important U.N. conclaves, the negotiators are known as the “Ad Hoc Working Group On Further Commitments For Annex I Parties Under the Kyoto Protocol.” Yet the consequences of their negotiations, if enacted, would be nothing short of world-changing.
Getting that deal done has become the United Nations’ highest priority, and the Bonn meeting is seen as a critical step along the path to what the U.N. calls an “ambitious and effective international response to climate change,” which is intended to culminate at the later gathering in Copenhagen.”
http://www.foxnews.com/story/0,2933,510937,00.html
Paranoid seems more than appropriate.
Jen “This forum has a huge and diverse audience including thousands of Australians….”
…and the odd Canadian, transplanted from Scotland at an early age, completely against his will.
Malcolm,the BOM 2007-08 annual report is available online.
Hey, thanks for the link, Nick!
Your silence is deafening.
Mal and Wes talking sans bum yet again. Plenty of reports – except drongos can’t comprehend them. Took a whole 10 seconds to find the latest report Wes Wonk. Gee I thought dodos were extinct.
Time for Wes to reel off another thesaurus busting rant about nothing. zzzzzzzzzzz
Thanks,
As it turns out, the latest is not much better at revealing their performance and the measures used. It weasels out of the hard stuff with a few well chosen paras of consultant speak.
Probably done so as to not potentially embarrass the Minister.
I’m shocked Malcolm. shocked. Yawn.
Luke, your irrelevance is deafening.
Oh, and Luke, it’s nice you’re stretching your coprolalian oeuvre to include (sort of) grammatically correct sentences. Yet, we’re going have to start a metaphor watch if you continue to mix four metaphors every four sentences. Mal and I are san bums or drongos or wonks or extinct dodos, but it’s discombobulating to combine those images in the same comment.
Try to extrapolate on a single metaphorical motif through out a comment. For example, perhaps Mal and I as uncomprehending drongos squawk and strut around like we’re in heat during the big wet, flapping our mouths like stubby beaks….laying dud egg comments, poop in our own nests, etc. But drongos aren’t extinct dodos or wonks or australorps. You sound like a galah, mate. Don’t mix your metaphors.
You grok?
Malcolm exposes as an ignoramus does his best to save face. You pathetic crow-eating moron.
Of course the ever obsessive Walker (Bsc Hons) demonstrates to all that he has read countless Balance Sheets and Annual Reports, Government and commercial, never mind being a party to producing same
Given your record of referencing material that you clearly have never read you have cheek trying to have a go at anyone, you pretentious fraud.
See if you can do a days work for your employer for change–us tax payers are getting fed up under writing the brown cardigan brigade.
http://www.larouchepub.com/eiw/public/2007/2007_20-29/2007-25/pdf/33-37_725.pdf
Another example of “data” that has been “adjusted” to agree with expectations. Nils-Axel Morner on sea level.
Hey Malcolm Dill – have you found the 2007/08 report yet? hahahahahahahaa
Yes thank you –
And I have read and understood it.
Which is lot more that you have ever done with your incessant Googling and posting of any old rubbish that you dont read.
What’s your rate fee for posting these days Wakka boy?
Still getting $200 a post from your GW cronies and greeny lobbyists.?
Struggling to keep it up are we.?
Well gramps – we’re glad we could be of assistance – given those BoM reports are just soooooooooooooo hard to find.
Golly they’re just SOOOOOOOOOOOOOO secretive. We had to google for what – oh – a whole 20 seconds to find the latest report. SOOOOOOOOOOOOOOOOOOOOOOO hard.
Now Malcolm – you’ve never shown much nous on here – and always have been a bit of a ranter – it’s just easier to make stuff up isn’t it Mally boy -
like – “They are 5-6 years behind in their Annual Reports, so neither they nor the Parliament knows what they are up to” -
Who got caught bluffing ? eh – come on – tell us. We won’t laugh for too long. hahahahahahahahaha – ROTFL LOL LMAO
so tell us – are you sure you have the right document. Are you sure it wasn’t the telephone directory. I mean for someone like yourself the phone book might be easily confused with an annual report.
Mally welly – I’m sorry you have never understood my googles. Perhaps you could consider repeating Grade 8 so you might have some of the basic skills required. Never too late for some remedial effort.
$200 a post – don’t insult me. That’s cheap.
Well Walker you obnoxious little creep there is no one–I repeat No One who has inhabited this blog over the last few years who has displayed a greater level, of ignorance and bluffery, combined with abuse and ridicule.
My reading of these posts and your 1000′s of so called contribitions is that the general view of others is also that you dont read most of what you google/post.
As for not finding the BOM Annual Reports first time that was an error on my part, which I have acknowledged. As Annual Reports go they well done. But that doesnt change the criticism of the lack of published performance measures–as far as I can tell.
Only someone with too much time on their hands or suffering from some pscychotic delusions the way you have done over the years. Whats your number for postings up to now.
2659 is my guess–what a screwball.
Now to do something more useful.
Australian data should be very suspect. I report to BOM 4 years ago about errors in the min/max temperatures for Tewantin Queensland, that, for example, showed a low of -8C one year. BOM replied that, yes there were errors for Tewantin but that it might take them some time to correct the data. Checking the data 3 years later it was still uncorrected.
At Casino airport, there is a manual weather station and an automatic weather station some 300m apart. The MWS is surrounded by buildings (a house has just been built recently only some 20m away) and it is also near a tarred road (about 5m away). The AWS is on a grassed oval with no buildings within 70m of it. The AWS temps are recorded in the BOM’s daily weather observations but because the MWS has been there for much longer, those temp measurements are used in their monthly climate statistics (I suppose for continuity’s sake). The interesting thing is that the MWS always records higher for both average monthly max and min temps, sometimes up to 1C. ‘Urban warming’ in action. I guess.
I also read a report that Jim Hansen adjusted the world temps a few years ago and by dropping pre-1980 temps and raising post-1980 temps (someone had overlaid his new graph with one he had used in a prior IPCC report). Any truth in that?
The point Malcolm is that you gleefully put the boot in with no regard in the slightest. It took seconds to find that report. You didn’t bother looking. You were simply willing to abuse them over a fabricated assertion. Do you think that’s good form? Although typical.
The fact you think I have “randomly” googled contributions honestly reinforces my opinion of your critical faculties. Go and do something useful yourself like total the quantity of abuse from AGW haters. Vile abuse. Add it up Mal !
Well el creepo the quantities have been added up, and if one includes all your aliases such as Phil Done, then you win hands down by a factor of 10:1.
No one can beat you for the sheer volume of vile abuse extending over a long period.
Now Mal – are we going to take the word of a bloke who couldn’t find the 2007-08 report. You just put that little anti-BoM rant straight out of thin air didn’t you. Just make up a bit of crap and see if you could slip it in.
Now mate we’d love to believe you but I don’t think old ranters who make up stories have a lot of maths credibility.
Well El Creepo its not hard to do even retrospectively, but for you I will start the meter running today.
Its already got one hit.
Nut cases like you wont be able to help yourself. Whats the betting on yet another change in persona is in the offing.
As for you wanting to keep harping on about my initial comments re the BOM, based upon what I had at the time, I now actually think their reporting in total is well done- but still weak in the area of KPI’s etc.
Steve Jones
I just checked Melbourne RO and found 8 of the hottest days for each month were before 1941, 2 were in the cooling period (’57 and ’75), one in 1982 and one in 2009. After doing research about the 1851 bushfires in Victoria, The Argus newspaper reported that the temp on 6th Feb, 1851 was 117F (47.2C) – higher than 7th Feb, 2009 (46.4C). I suppose that doesn’t count but you don’t think they would be that far out. I agree with you that when they start manipulating (or eliminating) the data and smoothing the numbers while calling it ‘quality control’, it is unacceptable.
For instance, Casino has records back to 1908 but the data for hottest day in each month goes only back to the 60′s giving the impression that the second part of the century was the hottest as previous data has not been taken into account.
Thanks for pointing this out.
[...] I wrote on a related issue in March 2009, what I’m now calling part 1, http://jennifermarohasy.com/blog/2009/03/confirmation-bias-at-the-australian-bureau-of-meteorology/ [...]