Averaging by its very nature smooths: removing peaks and troughs. Temperature data tends to be cyclical, whether on a one-minute, or thousand-year scale. The Australian Bureau of Meteorology has made a habit of smoothing when it is convenient and using extreme values otherwise. Take their one-minute temperature data from Canberra Airport: super-sensitive electronic equipment now records the highest, lowest, and last second of each minute and reports the highest second as the daily maximum temperature. Back in 2019 I purchased some of this data to test the Bureau’s claim that averaging the data would make no difference. I found that averaging the last one-second of each minute always gave me a lower maximum temperature. This is because the difference between the the highest and the last second could typically be 0.7 degrees Celsius as shown in Figure 17 – that is from a comprehensive report I co-authored in 2020. I have so far been unable to get this report published in a suitable peer-reviewed journal perhaps because it contradicts the Bureau’s much lauded Ayers and Warne (2020) analysis that comes to the opposite conclusion.
The Bureau claim that there is no need to average all 60-seconds in each minute as recommended by the World Meteorological Organisation when using resistance probes hooked up to data loggers. Ideally the Bureau would at least collect each of these seconds, and test this claim, but they never do. It was after meeting with Carl Otto Weiss for a drink at the Sunshine Beach Surf Club back in 2017 that I decided to at least test the concept by averaging the last second of each minute. This data can be purchased from the Bureau at some cost and with some delay.
In September 2017, I did met with Carl Otto Weiss. He is an Advisor to the European Institute for Climate and Energy and a former President of the German Meteorological Institute, Braunschweig. He was not particularly interested in my work on how the Australian Bureau of Meteorology measures temperatures, he had come to Noosa to meet with me and John Abbot to discuss our research newly published in the journal GeoResJ on the application of artificial intelligence, for evaluating anthropogenic versus natural climate change (GeoResJ, Vol. 14, Pgs 36-46 published in July 2017).
Our GeoResJ paper had been pilloried on Twitter, and we had been defamed by Graham Readfearn in The Guardian. So, it was a relief that contrary to everyone else in mainstream climate science at the time, who wanted our GeoResJ paper retracted/destroyed/burnt, that Otto Weiss praised it.
He thought it a most wonderful contribution to science showing not only what many suspect, that natural climate cycles drive the more significant changes in temperature over hundreds and thousands of years, but most importantly how the latest advances in artificial intelligence could be used to quantifying these effects.
I knew that Otto Weiss had a particular interest in measurement, after all, he had just attended the Australasian Measurement Conference (MSA2017) in Brisbane with Jane Warne from the Australian Bureau of Meteorology.
I wanted to know what he thought about the Bureau recording Australian temperatures as the highest, lowest and last second in every minute rather than taking the average of all the seconds over each minute.
The World Meteorological Organisation recommended that with the transition to more sensitive resistance probes hooked up to data loggers, to maintain some consistency with temperatures historically measured by mercury thermometers that have more inertia, sampling is best averaged over at least one minute.
At that time the Bureau had just finished and published its ‘Review of the Bureau of Meteorology’s Automatic Weather Stations’ in direct response to a front-page article by Graham Lloyd in The Australian newspaper on 1st August 2017. That article, with a photograph of Lance Pidgeon and me at the Goulburn airport, explained the Bureau had been forced to admit it had been caught out setting a limit of minus 10 degrees Celsius on how cold temperatures could be recorded; the limit had been in place for some 15 years since the transition to data loggers and the many ways the algorithms can be pre-programmed.
Side-stepping the issue of the cold limits, Otto Weiss queried whether it really was the case that the Bureau took spot-readings, rather than numerically averaging. I showed him the Bureau’s newly published AWS review and quoted from pages 22 where it explains:
One-minute maximum air temperature is the maximum valid one-second temperature value in the minute interval.
I also explained that the Bureau takes the lowest one-second spot reading as the minimum, but that until recently the Bureau had sent a limit of minus 10 degrees Celsius on how cold a temperature could actually be recorded.
I explained that the Bureau also records the last one-second temperature value in each minute interval. Otto Weiss explained this was the value that was perhaps most useful, the last second in each minute. He suggested that if the Bureau’s new resistance probes with data loggers had time constants that accurately mimicked mercury thermometers as the Bureau claimed, then this could be tested by averaging the last second that is recorded in each minute.
Perhaps the highest of these would then be recorded as the maximum temperature for each day? This is the method since used to calculate the daily maximum temperatures in the much quoted paper by Jane Warne and Greg Ayer published in the Journal of Southern Hemisphere Earth Systems Science (Vol 70, Pgs 160-165) in 2020.
Except there are three key problems with their method, never mind the dearth of data they actually compare:
1. Ayer and Warne claim to compare this last-second with the average of all 60-seconds in each minute except they compared the last second with just 5 one-second values from each one-minute interval incorporating the highest and lowest.
2. They used data from Darwin Airport (Site No. 14015), that is one of the 38 sites that still has mercury thermometers recording temperatures. So why not record the last-second from the probes with the value recorded from the mercury thermometer. If the objective of the Ayer and Warne study is to determine whether the time constant of the resistance probes is equivalent to a mercury thermometer, why not make a direct comparison.
3. While Ayer and Warne conclude that it is appropriate for the Bureau to record the value at the last second of each minute as satisfying WMO requirements, the Bureau don’t ever actually use this value. To reiterate, the Bureau use the highest one-second and the lowest-one second. It is nonsense and dishonest for Ayer and Warne to suggest otherwise.
According to page 17 of the AWS review:
The Almos DAS can provide one-second, one-minute, and 10-minute messages, as well as various other standard format meteorological messages.
So, the probe at Darwin Airport could have been reprogrammed to record a true one-minute average of all 60 one-second measurements. Then the comparison would at least have been consistent with WMO guidelines. This average could then have been compared with the manual recordings from the mercury at Darwin Airport, at least as a check of the WMO guidelines. Alas, and to reiterate, to justify the method currently used by the Bureau, Greg Ayers and Jane Warne would also have needed to make the comparison with the highest and lowest one-second value in each 24 hour period. Ayer and Warne never did this.
Yet, the Ayers and Warne paper has been held up as proof that temperature measurements from the Bureau’s probes in automatic weather stations are equivalent to readings from traditional mercury thermometers. Further, for me to suggest otherwise has been labelled a conspiracy theory.
Meanwhile, I can only characterise the Ayers and Warne paper as a ‘fake’ because it uses this different method of recording temperatures (the highest last-second of all the last-seconds each day) while claiming to be using the Bureau’s method that records the highest second within each minute each day as the maximum temperature. Detail can be tedious, and in this case is important. So I reiterate.
Nevertheless, Ayers and Warne are cited in The Guardian, by the Australian Broadcasting Corporation and the Agency France-Press, as reason to disregard my concerns about the Bureau hyping maximum temperatures.
It was two years after Otto Weiss visited, in 2019, and after purchasing batches of daily one-second data for Canberra, Adelaide and Melbourne from the Bureau, that I tested Otto Weiss’s hypothesis, that is essentially the Ayers and Warne methodology of using the last second in each minute.
I co-authored a 27-page report that sets out our method, results and conclusions. We test a lot more data points than Ayers and Warne, and in different ways. We could not calculate a proper minute average because the Bureau never collects every second of each minute. And we were unable to compare against a mercury thermometer, because the Bureau will not provide us with the parallel data for Canberra Airport, or any of the other locations.
Like the Ayers and Warner paper, our analysis was ready for publication in 2020. Entitled ‘One Minute Surface Air Temperature Observation – Adelaide, Canberra, Melbourne’ it, however, remains unpublished. Unlike Ayers and Warne, I no-longer have any colleagues willing to risk publishing me in a mainstream climate science journal. The last editor who published me had his journal shutdown: GeoResJ was discontinued in 2018.
My co-author of this report, testing the last one-second hypothesis as discussed with Otto Weiss all those years ago, cannot be named. My co-author also lives in Australia that is purportedly a secular democracy, but he risks losing his day job for assisting me with the analysis and report given there is no tolerance of dissent in Australia when it comes to issues of science and climate change.
Our unpublished manuscript begins:
Resistance temperature detectors (RTDs) in the Australian Bureau of Meteorology (BoM) automatic weather station (AWS) network provide temperature data at a rate of 1 Hz (sample per second). For every clock minute, three surface air temperature (SAT) observations are recorded:
• T , the last one-second reading (taken at 00 seconds of each minute)
• Tmax, the highest one-second reading over the last 60 seconds
• Tmin, the lowest one-second reading over the last 60 secondsThe BoM, however, only publishes the daily extreme values and associated statistics, e.g. the monthly and annual means. The one-minute data can be requested from the BoM for a given station, typically at a cost and processing delay.
The BoM has published statements indicating that their RTD and historical liquid-in- glass (LiG) measurements are equivalent, and specifically that the response times are similar. Every one-second reading is viewed as a time-averaged value (integrating over the past 40 to 80 seconds), effectively describing the moving-average temperature leading up to the given second, due to design of the RTD. High-frequency temperature fluctuations should therefore not be seen from second to second in the data, and also not from minute to minute (although more fluctuation could be expected at longer time scales).
Evidence that high-frequency fluctuations are indeed present in the measurements is given in this report, questioning the equivalence between RTD and LiG data.1 This can be seen by evaluating the time series consisting of all the last-second observations (a temperature series with constant sample spacing of 60 seconds), and also the difference between the last second and extreme measurements (Tmin and Tmax) for every minute, which indicates the measure of fluctuation possible, as measured with an RTD, within one minute. ENDS.
This is technical speak for let’s compare the last second reading from the resistance probe (RTD) with the highest and lowest reading each minute and the average.
When we did the analysis for Canberra airport – the example I am using in this note – we found that within the one-minute interval the difference between the last second reading and the highest second reading (maximum temperature archived by the Bureau) in any one minute interval was often more than 0.5C, and sometimes as high as 2.1C, as shown in Figure 2.3 chart B and table bottom left.
We concluded our analysis of the Canberra, Melbourne and Adelaide one minute data with comment:
The approach of the BoM to measure SAT [surface air temperature] is to record the highest, lowest and last second of every minute, as discussed before. The last-second data with the daily extremes are published and updated every 10 minutes on the “Latest Weather Observations” page for a given AWS [automatic weather station]. The data from the last 72 hours are updated every 30 minutes. The one- minute Tmin and Tmax data are also used to determine the daily ADAM Tmin and Tmax.
The WMO recommends averaging RTD [resistance probe] data over one minute. However, the BoM does not average at all, which is the reason for the spikiness of the data analysed in this report. Another example is shown in Fig. 16, displaying the last-second data observed at Canberra Airport (70351) on 17 Jan 2019.
If the WMO recommendation were followed, the BoM would provide the mean of 60 values — instead of only the single last value — for each minute. This would smooth the time series, similarly to what the averaging process depicted in Fig. 16 would do.
For illustration purposes, the moving average (MA) series over the last 5 samples (or 5 minutes, with only 1 sample per minute) is shown over the spiky last-second data. Al- though this illustration is not perfect (more data is needed to smooth over every minute), it does show that the daily Tmax would likely be lower, as it would be based on an average and not an instantaneous observation. ENDS.
Numerical averaging will drop the daily maximum temperature by almost a full one degree Celsius relative to taking the last second in each minute and by more than one degree when recording the highest one-second in each minute.
It is Bureau policy to record the highest one-second in each minute and the highest of these becomes the maximum temperature for that day for that location.
This last chart (Figure 16) from my unpublished report, shows that contrary to the hypothesis of Carl Otto Weiss, which is also a central thesis of the fake paper by Greg Ayers and Jane Warne, recording just the last second of the minute is not equivalent to the numerical average of even just the five last-one second readings. At least this was the situation at Canberra on 17th January 2019.
This is part 6 of ‘Jokers, Off-Topic Reviews and Drinking from the Alcohol Thermometer’. In part 7 I will explain why it is imperative that Greg Ayers and Jane Warne provide the A8 reports for Darwin Airport for April 2018 – that is the parallel data on which their analysis is based. The highest, lowest and last second records for each minute for the months of March, April and May of 2018 also need to be made public. You can read some of my criticism of Warne and Ayer at the popular climate blog WattsUpWithThat.com. I am grateful to Anthony Watts and Charles Rotter for republishing this series.
Jennifer Marohasy BSc PhD is a critical thinker with expertise in the scientific method.
It is highly disappointing if not scandalous the way Jennifer is treated by a public body responsible to all. However, silencing has been and is the preferred modus operandi of climate activists and the behaviour of the BoM certainly shows it to be part of that sector.
Jemmifer, I know you wish this dicsussion to be science focussed.
But as a retired practical farmer, I want to point out that the recent actual weather shows a trend towards climate cooling.
For example my fruit trees are severely effected by t e cold weather conditions during this past Spring, Summer & Autumn. I had Feijoas flowering in January rather than November. I am harvesting them now rather than in Early April ! My fig trees usually fruit from December till February. This year I am getting a few ripe fruits in May !
The cold is being felt by everyone here in SA’s Adelaide Hills. And many of us have been lighting fires at night since March. )(Though we had fire in January on a couple of nights as well )
We have not had a proper Heat wave here in SA since 2018 ! Only the NOM’s new 2-3 day “Heat Spikes ”
Ordinary people notice all these things in their immediate life !
And there is a real increase in popular doubt & scepticism about the BOM. In a word they’ve cocked things up !
I’ve given up talking to friends about the BOM idiotic Climate change ( warming science. Instead we talk about the cold Summer weather, the need for fires and how out gardens are doing.
Thanks Bill. Your observations are relevant. What I am doing is laying out a case against the Bureau, I am moving towards an end point and a call for action.
I ask your patience.
My comment is also not exact science but, here in northern Tasmania, at our house we rely entirely on electricity for heating as well as hot water and cooking. Heating is far and away our major use of electricity. The past year has shown a notable increase in our electricity usage and, as cooking use is relatively small, it is highly probable that heating and hot water usage is the cause. We are not increasing the temperature at which our house is operated, in fact the contrary with jumper use more prevalent. It is consistent with a cooling climate.
On the topic of fruit trees, we chopped down our fig tree as few figs ripened over the past two years.
Good point the resistance temp sensor can have little thermal mass, depending on how the analoug to digital data its possible to have misleading actual reading averaged over 60 seconds, is it in software and how its done , I would hope the sensor would have thermal rise behaivor like mercury so the data would be the same after software calculation
Definitely very chilly in the Adelaide Hills over the last few years. Tony
Thanks again Jen, for all your hard work and revelations.
Wouldn’t you think that it would be perfectly rational and logical in a world that is spending trillions to address this “problem”, particularly when both sides of the debate are at such odds, that a more thorough, public climate audit, allowing both sides to produce and demand ALL the facts, might be carried out before we lurch down this foreseeably dubious and crazy path?
Just filing this here:
https://www.nst.com.my/opinion/columnists/2023/05/909419/meteorologists-targeted-climate-misinformation-surge?utm_source=nst&utm_medium=mostpoplatest
By Roland Lloyd Parry, Benedicte Rey, Adria Laborda, Kate Tan
ONCE trusted faces on the news, meteorologists now brave threats, insults and slander online from conspiracy theorists and climate change deniers who accuse them of faking or even fixing the weather.
Social media users falsely accused Spain’s weather agency of engineering a drought, Australia’s of doctoring its thermometers and France’s of exaggerating global warming through misplaced weather stations.
“The coronavirus is no longer a trend. Conspiracy theorists and deniers who used to talk about that are now spreading disinformation about climate change,” Alexandre Lopez-Borrull, lecturer in Information and Communication Sciences at the Open University of Catalonia, said.
ADVERTISING
“These scientific bodies are seen as part of the establishment, so anything they say may get disputed. They are providing evidence against what the climate-deniers claim, so the latter try to discredit them.”
MORE NEWS
Record sea surface heat sparks fears of warming surge
Djokovic defies controversy to surge into Australian Open final
Magnitude 6.1 quake shakes Japan’s Hokkaido, no tsunami warning
Germany misses 2022 climate target on Ukraine war fallout
Spain’s State Meteorological Agency (AEMET) spoke out after its members were threatened on Twitter, phone calls and emails.
“Murderers”, “Criminals”, “You’ll pay for this”, “We’re watching you”, the messages shouted.
They came from people who believe the widely debunked theory that aeroplane condensation trails are “chemtrails” sprayed by the authorities to poison people or create weather disasters.
Some referred to the “2030 agenda”, a debunked theory that global elites are plotting to subjugate people through Covid-19 and climate policies.
“Do you want us to publish your contact details and those of your family?” read one Tweet aimed at an AEMET employee.
“Crooks! You are destroying nature on the orders of the damned 2030 agenda,” said another.
“We have seen an increase in insulting messages as a result of a thread we published about condensation trails” on April 10, said AEMET spokesman Estrella Gutierrez-Marco.
“What makes no sense is that they are insulting an institution that is constantly watching out for their interests, whose aim is… to contribute to people’s safety.”
Lopez-Borrull noted a “significant increase” in climate change denial — particularly among far-right supporters who see it as a leftist cause, and oppose reforms aimed at curbing its impacts.
“People distrust politicians, judges and the media, and the cost of living is rising,” he said.
“In this context people feel alienated and end up listening to people they never listened to before, with messages appealing directly to their emotions.”
In another case investigated by AFP Fact Check, conservative media and Facebook users shared unfounded claims that Australia’s Bureau of Meteorology (BOM) had doctored its temperature readings.
In an analysis of data obtained via a freedom of information request, prominent climate sceptic Jennifer Marohasy said: “BOM’s electronic probes returned readings up to 0.7°C warmer than those of its older mercury thermometers.”
Experts who analysed the data said the claims were inaccurate.
Monash University emeritus environment professor Neville Nicholls said the difference between most readings on the electronic probes and the mercury thermometers was negligible — between 0°C and 0.1°C.
“This difference is very small compared to the strong warming trend in average temperature over Australia” — about 1.4°C over the past century — Nicholls said.
The World Meteorological Organisation said BOM’s measurements were in line with its standards, contrary to Marohasy’s allegation…
A few nights in hospital unable to sleep, and debate with clowns on twitter, helped crystalize my rambling but good faith view on this, invoking Klein and others.
The very foundation of the scientific method which has led to our incredible explosion of enlightenment, advancement, improvement for the human condition is “empiricism”. Those ancient babylonians, looking at the sky, with no laws of physics to guide them, invented maths to help describe and understand what they saw. Funny thing is, the mathematics was right (by definition), and Ptolemy, a Greco-Egyptian mathematician, astronomer, and geographer who lived in the 100s CE, claimed that Earth was stationary and at the centre of the universe, and that all the planets and the stars revolved around it. This view was generally accepted by Christians (non deniers) because it put Earth (Gaia), God’s “greatest creation,” at the centre of the universe, which was considered unmoving and perfect, and also because it seemed to accurately describe what we see in the skies every day: when the sun “rises” and “sets” each day, it certainly seems like the sun is moving and Earth is standing still. Sound familiar?
Along comes Nicolaus Copernicus who used mathematics to try to prove or disprove the Ptolemaic theory. Copernicus, a Polish astronomer, could not prove the truth of Ptolemy’s theory, and the mathematics actually yielded that the geocentric theory (which held that Earth was at the centre) was actually less likely than the heliocentric theory (which held that the sun was at the centre). The believers banned this deniers work from 1616 – 1835, over 200 years in the dumpster.
Along came Galileo, with his inconvenient telescope, observing moons revolving around larger planets, and discovered the incontrovertible truth which is so obvious to us today. The believers banned Galileo’s books, sentenced to house arrest until his death in 1642.
Worth noting, that this fulfils the hypothesis of Nicholas Klein, a trade union activist that you mentioned https://jennifermarohasy.com/2023/05/the-coronation-the-guardian-temperatures-misinformation-part-2/. Galileo has monuments scattered around the world now, and it is powerful to ponder them especially in juxtaposition to those other monuments (aka Cathedrals), and that this man eventually changed the believers to the thinking of his denialism, and yet there monuments remain (for now).
It’s a pity, none of these greats had W. Edwards Deming to quote ”In God we trust. All others must bring data”. Interesting to note that Deming was a descendant of the puritans (definitely believers). Deming made a significant contribution to Japan’s reputation for innovative, high-quality products, and for its economic power. He is regarded as having had more impact on Japanese manufacturing and business than any other individual not of Japanese heritage. Despite being honoured in Japan in 1951 with the establishment of the Deming Prize, he was only just beginning to win widespread recognition in the United States at the time of his death in 1993. He was a devout believer, yet overcame the shackles of belief and tradition. More on this later.
Heading back down to the depths of decaying organic matter, held in the sedimentary layers over 4000m below the surface, I reflect on my own experience with measuring temperature. For 10 years I was a field engineer for Schlumberger (30 years working with them), and much of my life revolved around getting a very high quality pressure sensor into the bowels of the earth, to measure this property of the fluids contained with the permeable rocks below. The pressure measurement, and its response to various perturbations, allow many properties of the fluid to be derived, including density, viscosity, permeability of the rock, etc etc. Companies would pay literally many millions of dollars for this measurement, as the result could be used to calculate the 10 of billions of dollars of oil or gas in these rocks, and allow design of the equipment to produce them. Fundamental to any of this, was the measurement of the humble temperature, which was one of the first measurements every made subsurface (with a max reading mercury thermometer – sound familiar?), which was now replaced by a CQC gauge
https://www.slb.com/-/media/files/testing-services/product-sheet/signature-cqg-gauge-ps-2
which measured temperature with an accuracy of +/-0.2C, and resolution of 0.001C. Remember this is at 16000 psi, up to 350F, kilometres underground in a hole that has just been drilled. It is calibrated in facilities that correct for local gravity variations. I managed one of these facilities in the Netherlands. Note, to this day, we cross check the million dollar temperature with the 20c mercury gauge! In order to measure pressure accurately, the first thing you need to do was achieve thermal equilibrium of the equipment. Each psi (excuse the imperial units but that is the language of the industry, American at its core) added or subtracted millions from the interpretation of the value of the discovery. This often meant hours staring at a humble but very accurate temperature measurement in order to monitor for the temperature stabilisation of this gauge with its environment, in order to ensure the best result. There will be a series of measurements taken, and the slope of the pressure versus depth will imply the density of the fluid within the rocks. In short, I did spend a decade staring a temperature gauge, and became familiar with its meanderings and behaviour in a changing environment. It was contained in a 20cm thick stainless steal vessel, that had come from surface temp to 350F in a matter of hours, and we became very familiar of the characteristics of this stabilisation. Every minute of this operation would be costing 1000s of dollars in the rig, people and organisation standing by, so often, someone with that in mind would intervene and tell me to hurry up and get the answer. Ok, I would say, do you want wrong and now, or right in another x minutes, and $x0000, but getting it right will add x-billions to the end result. I knew I could apply any manner of filtering, averaging to this temperature data, in order to get an answer immediately. But I knew that looking at the rawest data over the most time, and as a whole, resulted in the most correct answer, and was worth achieving.
Even shorter – I feel your pain. The BOM are taking measurements which will result in trillions of dollars of expenditure, yet argue about the cost of providing you with the rawest of data.
I clearly would be defined as being on the wrong side of this argument, as denier. I’m actually not, I’ve come to the conclusion that the production of producing and burning fossil fuels sequestered over billion years, in a period of 100 years, is something that needs to change for obvious reasons, and the behaviour of the companies involved will make it difficult. I also think nuclear power will end up being essential, even if not in Australia, and that seems to infuriate people more. I observed that the simple desire or request to people, that perhaps we should just see the data, invoking Ptolemy, Galileo, and Deming immediately has everyone trolling your life to ensure you can be ignored.
Even Shorter again – the BOM should just release the data so empiricists can observe, and science will apply. I do not have your skills or experience at collating, analysing temperature data for temperatures sake. But I’ve got decades of experience of knowing its importance to everything else.
And I am empathic to your cause. They (the BOM) are invoking God, when they aren’t, and should bring the data.