Jennifer Marohasy

a forum for the discussion of issues concerning the natural environment

How Scientific Ideas Become Fashionable (Part 2)

By

MICHAEL Crichton wrote the Oscar-winning science fiction adventure Jurassic Park. But screen writing was not his first career, he studied medicine at Harvard, and later in life became very concerned about environmentalism and science, and the difficulty of sorting fact from fiction. In a lecture to the Commonwealth Club in San Francisco in September 2003 he said:

“The greatest challenge facing mankind is the challenge of distinguishing reality from fantasy, truth from propaganda. Perceiving the truth has always been a challenge to mankind, but in the information age (or as I think of it, the disinformation age) it takes on a special urgency and importance.”

For sure every day we are bombarded with information from the internet, radio and television and making sense of it can be difficult.

Scientists are meant to know the difference between fact and fiction and as a first check of the reliability of a source of information they will often ask if it has been “peer-reviewed”. Peer-review means that research findings are conducted and presented to a standard that other scientists working within that field consider acceptable. This is normally achieved through publication in a scientific journal and involves the editor of the journal asking for comment on the validity, significance and originality of the work from other scientists before publication. In short, the system of peer-review means scientific research is subject to independent scrutiny but it doesn’t guarantee the truth of the research finding.

In theory rebuttals play an equal or more important role than peer review in guaranteeing the integrity of science. By rebuttals I mean articles, also in peer-reviewed journals, that show by means of contrary evidence and argument, that an earlier claim was false. By pointing out flaws in scientific papers that have passed peer-review, rebuttals, at least theoretically, enable scientific research programs to self-correct. But in reality most rebuttals are totally ignored and so fashionable ideas often persist even when they have been disproven.

Consider, for example, a paper published in 2006 by marine biologist, Boris Worm, and coworkers, in the prestigious peer-reviewed journal Science [1]. The study was based on the meta-analysis of published fisheries data and predicted the collapse of the world’s fisheries by 2048. Publication of the article by Worm et al. was accompanied by a media release entitled “Accelerated loss of ocean species threatens human well-being” with the subtitle “Current trend projects collapse of all currently fished seafoods before 2050”.

Not surprisingly, given the importance of the finding, the article attracted widespread attention in the mainstream media and also within the scientific community. But not everyone agreed with the methodology used in the Worm study. Eleven rebuttals soon appeared, many within the same journal Science, and within months of the original article.

Ray Hilborn, a Professor in the School of Aquatic and Fishery Sciences, University of Washington, described the projection that all the world’s wild fish would be collapsed by 2048 as “fallacious and inappropriate to appear in a scientific journal” [2]. Professor Hilborn objected, in particular, to the use of catch data as an indication of fish abundance. You see, rather than estimating the actual number of fish of a particular species in a particular part of the ocean the Worm study defined fisheries “collapse” as a drop in the number of that fish species caught below 10 percent of the recorded maximum number of that fish species ever caught for that locality. Professor Hilborn explained that number of fish caught may not in fact be an accurate reflection of abundance, because a low catch could be the result of management restrictions, for example, the introduction of a fishery exclusion zone or even a ban on catching that species of fish. So, Professor Hilborn explained, a healthy, well-managed stock may in fact be classified as collapsed using the Worm et al. criteria.

Other rebuttals made similar comment, with Steven Murawski, the Director of Scientific Programs and Chief Scientific Advisor, National Marine Fisheries Service in the USA, explaining that [3]:

“A variety of biological, economic, and social factors and management decisions determine catches; low catches may occur even when stocks are high (e.g., due to low fish prices or the effects of restrictive management practices), and vice versa. The inadequacy of Worm et al.’s abundance proxy is illustrated by the time series of data for [the fish species] Georges Bank haddock (Melanogrammus aeglefinus).

“The highest catch for haddock occurred in 1965 at 150,362 tons. This catch occurred during a period of intense domestic and international fishing. In 2003, haddock catch was 12,576 tons, or 8% of the time series maximum. Under the Worm et al. definition, the stock would be categorized as collapsed in 2003. However, stock assessment data estimate the total magnitude of the spawning biomass in 2003 to be 91% of that in 1965. Comparing the estimate of spawning stock biomass in 2003 to the level producing maximum sustainable yield, the stock was not even being overfished in 2003.”

Franz Holker and coworkers, wrote that the Worm study extrapolated far beyond the range of available observations and that using a similar rational it was possible to predict 100 percent unemployment in Germany by 2056 [4].

Dr Holker agreed with Dr Worm that there had been an increased in the number of overexploited or depleted fish stocks from about 10 percent in the mid-1970s to around 25 percent in the early 1990s, but suggested that the situation had stabilized since the early 1990s.

Given these very significant errors in the Worm study documented in the many rebuttals, it would seem reasonable to conclude that the original research paper as published in the peer-reviewed journal Science would have been quickly and totally discredited. But this was not the case at all. Dr Worm became quite famous and his paper claiming the imminent collapse of the world’s fisheries was cited on average 72.3 times each year for the three years after its publication in other peer-reviewed publications, while the 11 rebuttals were cited on average just 1.5 times per year [5].

In reality, the rebuttals scarcely altered the scientific perception of the original article.

In a comprehensive study of this, and six other high-profile original articles and their rebuttals, Jeannette Banobi, Trevor Branch and Ray Hilborn, found that at least in marine biology and fishery science rebuttals are for the most part ignored [5].

They found that original articles were cited on average 17 times more than rebuttals and that annual citation numbers were unaffected by rebuttals. On the occasions when rebuttals were cited, the citing papers on average had neutral views on the original article, and incredibly 8 percent actually believed that the rebuttal agreed with the original article.

Dr Banobi and coworkers commented that:

“We had anticipated that as time passed, citations of the original articles would become more negative, and these articles would be less cited than other articles published in the same journal and year. In fact, support for the original articles remained undiminished over time and perhaps even increased, and we found no evidence of a decline in citations for any of the original articles following publication of the rebuttals…

“Thus the pattern we observed follows most closely the hypothesis of competing research programs espoused by Lakatos (1978): in practice, research programs producing and supporting the views in the original papers remained unswayed by the publication of rebuttals, thus significant changes in these ideas will tend to occur only if these research programs decay and dwindle over time while rival research programs (sponsored by the rebuttal authors) gain strength.”

But there is no guarantee that a rival research program will gain strength. In fact given the increasing politicization and shrinking funding-base for many scientific disciplines it is perhaps more likely that popular research programs, however flawed, will become entrenched and with them mistaken but popular ideas about the natural environment. And this can only hinder our capacity to respond in a timely and effective manner to environmental issues.

As Michael Crichton said in his 2003 lecture:

“What constitutes responsible [Environmental] action is immensely difficult, and the consequences of our actions are often difficult to know in advance. I think our past record of environmental action is discouraging, to put it mildly, because even our best intended efforts often go awry. But I think we do not recognize our past failures, and face them squarely.”

Indeed it is the naive view that scientific communities learn from obvious mistakes. And as past failures become more entrenched it can only become increasingly difficult to distinguish truth from propaganda, including in the peer-reviewed literature.

*******
[1] Worm, B. et al. 2006. Impacts of Biodiversity Loss on Ocean Ecoystem Services, Science, Vol. 314, pp. 787-790
[2] Hilborn, R. The Projection from B. Worm Et Al. Science, Vol 316, pp. 1281-1282
[3] Murawski, S. et al. 2007. Biodiversity Loss in the Ocean: How Bad is it? Science, Vol 316, p. 1281
[4] Holker, F. et al. 2007. Comment on “Impacts of Biodiversity Loss on Ocean Ecoystem Services”, Science, Vol. 316, p. 1285
[5] Banobi, J. et al. 2011. Do Rebuttals Affect Future Science? Ecosphere. Vol 2, pp 1-11. http://www.esajournals.org/doi/abs/10.1890/ES10-00142.1

Reader Interactions

Comments

  1. Neville says

    Perhaps the first requirement by all govts and scientific groups would be the exposure of any new study to any group or individual who wanted 100% of the info involved in the new study.

    All data must be publicly available to anyone who wants to test that data and conclusions derived from the data. All costs for access should be minimal or preferably free to interested parties.

    Look at the nonsense of the hockey stick graph and the eager uptake and promotion by govts and the IPCC.
    But for the tireless work of McIntyre and McKitrick this rubbish would still be fooling govts and scientists and costing billions of $ more with a zero change to climate and temp.

    The mitigation of AGW is one of the greatest frauds and con tricks of the last 100 years and any govt anywhere that persists with this deceit should be voted out at the first available opportunity.

  3. bazza says

    I would like to see a bit more than one weak analogy from fisheries to support your ambit claim : “But in reality most rebuttals are totally ignored and so fashionable ideas often persist even when they have been disproven.”. This would enable you to cast your net a bit wider and show Arrhenius was erroneous! I am still waiting to see a useful rebuttal by analogy.

  5. Rob JM says

    A classic example of fashion over fact was the first study that identified a link between mobile phones and brain cancer. All they did was identify a correlation between increased cancer rate and an increase in global phone use. Problem was the people who were getting brain cancer were elderly people who had never used a mobile.
    We all know where this BS study has led us now.

  6. spangled drongo says

    The fashion to claim so many “scientific” predictions are “worse than we thought” and the frightening cost that is then generated by these widely advertised and quickly acted upon claims without any serious checking, shows what a crazy world we live in.

    Individuals, families or small communities could never dream of operating in such mindless haste but many govts have no qualms.

    The more remote the foot that you shoot, I suppose, the less it hurts….

  9. Ian says

    It will be interesting to see if the Watts et al paper, which essentially is rebutting some claims of temperature increases in the USA, gets as much publicity as the claims it rebuts. From this piece by Dr Marohasy it seems very unlikely.

  10. Neville says

    Ross McKitrick isn’t impressed with Muller or the BEST study. As a reviewer he found too many problems with the study and like Pielke he doesn’t like Muller’s blitzing of the media.
    The study so far hasn’t passed peer review.

    http://www.rossmckitrick.com/

  11. Tony Price says

    Bazza said: I would like to see a bit more than one weak analogy from fisheries to support your ambit claim : “But in reality most rebuttals are totally ignored and so fashionable ideas often persist even when they have been disproven.”

    Is a lot “a bit more” than one?
    http://retractionwatch.wordpress.com/

    Even withdrawn papers continue to be cited decades after withdrawal.

  12. Larry Fields says

    Here’s another quote from the late Michael Crichton:

    “The ‘precautionary principle,’ properly applied, forbids the precautionary principle. It is self-contradictory. The precautionary principle therefore cannot be spoken of in terms that are too harsh.”

  13. el gordo says

    A bit more on the late Michael Crichton…

    ‘On March 14, 2007, Intelligence Squared held a debate in New York City entitled Global Warming is Not a Crisis, moderated by Brian Lehrer. Crichton was on the for the motion side along with Richard Lindzen and Philip Stott against Gavin Schmidt, Richard Somerville, and Brenda Ekwurzel.

    ‘Before the debate, the audience were largely on the Against the motion side at 57% vs 30% in favor of the for side, with a 12% undecided. At the end of the debate, there was a notable shift in the audience vote at 46% vs 42% in favor of the for the motion side leaving the debate with the conclusion that Crichton’s group won.

    ‘Schmidt later reflected on the debate in a RealClimate blog posting, conceding that his side’s presentation was “pretty dull” and calling Crichton’s debating skills “extremely polished.”

  15. Bob_FJ says

    Bazza,

    Did you really mean to say the following?

    I am still waiting to see a useful rebuttal by analogy

  19. cohenite says

    Is bazza asking for pro-AGW papers which have been disproved but still have currency, at least in the media and possibly subsequent papers? Try Gergis, Steig and Muller, BEST 1 & 2, noting BEST 1 was submitted to peer review and not accepted.

  21. Neville says

    Even Judith Curry gets stuck into Muller and seems to think that co2 is not the main driver of recent climate change.

    http://judithcurry.com/2012/07/30/observation-based-attribution/

    She is right that observation must trump some doubtful modeling. Of course same goes for SLR as well.

    Luke , if the Watts team is correct it will bring the recent warming into line with the satellite measurement.
    If that’s all they achieve it will be congratulations all round for all the members of his team.

  23. Luke says

    You’re easily bluffed Neville – measurements of what? – and satellite measures different things. Sucked in. And not even peer reviewed yet – hohohoho – science by press release – a new sceptic low.

  25. cohenite says

    Unlike Muller Watts’ methodology creates a temperature record which correlates with the satellite record so it passes that important test.

    Statistically, BEST is a mess. The BEST methodology is wrong because it does not do the job it was designed to do. BEST had millions of data points which had to be standardised because some stations had only a few bits of data while others had many, and many stations had varying quantities of data at different times; to reduce confidence intervals a weighting standard had to be applied with subsequent averaging and then removal of data which was an outlier to the average.

    BEST use a Jacknife method with a spatial correlation, height above sea level, supposedly to eradicate UHI, and NO time correlation.

    The Jacknife is a statistical method used to produce accurate confidence intervals where data is sparse but relies on an equal weighting of the data; this did not happen because of the lack of a time correlation; a station may have had sparse data at one time but more data at another. As a result, as Jeff Id notes, the following happened:

    “Another way to think of it is if you have 5 stations. Say one is weighted approximately 10 times greater than the other 4 which are equally weighted. This would be the same as 10 copies of 1 and 4 other copies of something else. If you eliminate one station which happens to be one of the 4 and the weights are recalculated so that one station is say 10 times weighted to 3 at similar weights to the original, the difference in the temperature reconstruction would be about 1/14th of change from error rather than 1/5th as would be assumed by jackknife.

    These values are arbitrary of course but if you expand the effect across thousands of stations, you can see that the subsampling and reweighting give artificially low CI’s.”

    Watts uses the methodology from Leroy, which, while simple, is also elegant because it focuses on the temperature effects and sources. By comparison the Muller Jacknife method is flawed because it covers a non-temperature indice, height above sea level, to distinguish site micro-effects.

    Muller has released this blurb:

    Berkeley Earth has just released new results, showing that the average temperature of the Earth’s land has risen by 1.5 °C over the past 250 years. The good match between the new temperature record and historical carbon dioxide records suggests that the most straightforward explanation for this warming is human greenhouse gas emissions.

    The new analysis from Berkeley Earth goes all the way back to 1753, about 100 years earlier than previous groups’ analysis. The limited land coverage prior to 1850 results in larger uncertainties in the behavior of the record; despite these, the behavior is significant. Sudden drops in the early temperature record (1750 to 1850) correspond to known volcanic events.

    In its 2007 report the IPCC concluded only that “most” of the warming of the past 50 years could be attributed to humans. It was possible, according to the IPCC, that increased solar activity could have contributed to warming prior to 1956. Berkeley Earth analyzed about 5 times more station records than were used in previous analyses, and this expanded data base along with its new statistical approach allowed Berkeley Earth to go about 100 years farther back in time than previous studies. By doing so, the Berkeley Earth team was able to conclude that over 250 years, the contribution of solar activity to global warming is negligible.

    This is unmitigated crap which will be devoured over the next few days. But for starters the declaration that “The good match between the new temperature record and historical carbon dioxide records suggests that the most straightforward explanation for this warming is human greenhouse gas emissions” when combined with this “the Berkeley Earth team was able to conclude that over 250 years, the contribution of solar activity to global warming is negligible” is astounding and contradicted by other AGW luminaries such as Mann and Schmidt:

    http://pubs.giss.nasa.gov/docs/2001/2001_Shindell_etal_1.pdf

    You can bet that the Fairfax and ABC reporting of Muller will not delve into such internal contradictions of the AGW scam.

    And re: gav’s link to the ABC; Muller was never a sceptic. This is what he says in 2003:

    “Let me be clear. My own reading of the literature and study of paleoclimate suggests strongly that carbon dioxide from burning of fossil fuels will prove to be the greatest pollutant of human history. It is likely to have severe and detrimental effects on global climate. I would love to believe that the results of Mann et al. are correct, and that the last few years have been the warmest in a millennium”

    And another thing, BEST 1 was submitted to peer review and was knocked back; it still hasn’t been published other than in the public arena where the usual media groupies can promote it. BEST has been submitted to peer review but its methodology is worse than BEST 1.

    This is just a fraud; and more strength to Watts for showing that.

  26. toby says

    “By doing so, the Berkeley Earth team was able to conclude that over 250 years, the contribution of solar activity to global warming is negligible.”…Yes Cohenite, when I read that line I also shook my head and laughed and figured the paper must be crap. thankgod for human “pollution” bringing us out of the mini ice age!

    i wonder how many data points they had for 1750?…I believe it was 10?…enough for warmers to determine its all evil humans…but when we point to teh MWP, RWP,MWP…we are told …not enough evidence for the southern hemisphere to support these warmings being global.

    what a joke.

    I certainly hope watts paper is proved correct because it will mean we have little to worry about and show those extreme heat anomalies to be yet another fabrication…….

  27. Robert says

    You see, there are these aerosols we squirt, and they make it cold, but they have to fight against the carbon dioxide we squirt, which makes it hot. But the ocean soaks up the carbon dioxide in these trough things, but when the troughs get filled up they kind of blow out, so our aerosols lose, though Schneider and the early CRU used to think they would win and our CO2 would lose…

    It’s terribly simple, except when it’s fantastically complex. There’s still so much to learn and research, but the science is nonetheless settled. All findings are absolutely certain, it’s just that future findings will find that stuff is worse than we thought.

    Anyone doubting this (almost-dead white males) will be subjected to a relentless bombardment of smirks from the ABC’s army of matronising convent girls.

  31. gavin says

    Re Watt & Muller -Washington Post July 30

    “So-called blockbuster climate change studies prove little”

    “Both studies staged high-profile releases and represent concerted efforts to influence public perception about what we know about climate science. But neither has been published in a peer-reviewed publication and there is cause to question their legitimacy”

  32. cohenite says

    “But neither has been published in a peer-reviewed publication and there is cause to question their legitimacy”

    That is not accurate; BEST 1 was submitted for peer review and knocked back; BEST 2 has not been submitted.

    Watts’ first Subsurface paper was published after peer review and his 2nd has yet to be submitted.

  33. gavin says

    cohenite above in his long post claims, Muller was never a sceptic

    “I was not expecting this, but as a scientist, I feel it is my duty to let the evidence change my mind,” Muller, a professor of physics at the University of California, Berkeley, said in a statement on Monday. (Herald Sun today)

    You guys will be working overtime from here on and it has little to do with Muller

  34. John Sayers says

    Oh Gavin – you are so blinkered!

    Most of your posts I can’t understand so I don’t reply – but your last post reflects your blinkered view.

    Muller clearly was NOT a sceptic and all his hot air is just short of a good fart!

  35. John Sayers says

    What really concerns me is that Professor of Physics at a leading US University is so stupid!

  36. Robert says

    Muller’s skepticism consisted in criticising the Hockeystick, along with McIntyre and McKitrick. That just means his reasoning is somewhere above “brute” or “Mann” level. It doesn’t make him a skeptic.

  37. Debbie says

    http://www.bbc.co.uk/news/science-environment-19047501
    This report has definitely NOT been accepted yet….but it is certainly gaining traction in the media.
    That wouldn’t be because Muller and BEST have deliberately released it to the media would it?
    Gavin,
    You guys will be working overtime?
    Working overtime for what in particular?
    Isn’t the ‘peer review’ process supposed to work on this first?
    So far….it is all just PR spin.
    And yes….Watt’s has been spinning PR too….but his approach is rather different….he is inviting criticism….Muller is claiming that this report is above criticism and the fact that he once thought he was a sceptic is somehow proof that he must be right????
    It’s also interesting to note that Judith Currie is not willing to become involved and has criticised Muller’s approach.
    Why would that be the case do you think Gavin?
    And BTW Gavin,
    The problem is actually the USE of the data….it isn’t the ‘Science’.

  39. Johnathan Wilkes says

    Comment from: John Sayers July 31st, 2012 at 5:56 pm
    “Most of your posts I can’t understand so I don’t reply “

    Thank you, thank you John Sayers.
    So? I’m not losing my marbles at an early age after all.
    I’ve given up on him some time ago because most of the time I couldn’t make head or tail of his posts either.

    Mayhap it’s to do with his ever increasing qualifications since he started posting here?
    (early on, instrument technician -> you name it these days?)

  40. sp says

    Watts cant really be criticised for his press release, Muller does his business the same way.

    Watts paper has not been “reviewed” by anybody – I believe the full paper and ALL supporting data has not been released yet?

    Watts has promised to release ALLsupporting data with the paper to allow others to replicate the method and test his results. I believe that is the accepted scientific method.

    Its a bit premature to criticise something before studying it.

    Maybe Watts will be proven to be correct, maybe not. Time will tell.

    Muller is a clown. Even Mann thinks Muller is s show pony and only in it for the money

  42. kuhnkat says

    Gavin,

    From a 2008 Grist interview of Richard Muller:

    Do you consider yourself an environmentalist?

    Oh yes. [Laughs.] In fact, back in the early ’80s, I resigned from the Sierra Club over the issue of global warming. At that time, they were opposing nuclear power. What I wrote them in my letter of resignation was that, if you oppose nuclear power, the U.S. will become much more heavily dependent on fossil fuels, and that this is a pollutant to the atmosphere that is very likely to lead to global warming.

    grist.org/article/lets-get-physical/

    So, in the early 80’s, before there was enough “adjusted” temperature data to indicate a problem, he already knew that CO2 was an issue!! He has NEVER been a sceptic. Makes him a blatant liar and propagandist who apparently is also doing it to line his own pockets. Here is one of his companies:

    mullerandassociates.com/

    I would suggest he has become involved as Gorebull Warming is losing adherents and the public no longer believes.

  43. Debbie says

    Gavin,
    There is no need to guess, the report clearly says where.
    Just because it starts with a ‘global warming’ theme doesn’t prove or disprove anything.
    It could just as easily be a local phenomenon or entirely from a natural adjustment.
    There is NOTHING in this report that either proves or disproves that human activity is the CAUSE of this nice run of luck.

  44. Debbie says

    Actually,
    To stay within the parameters of this debate, I should have said a nice run of luck with the WEATHER.