Climategate (2/3): be careful what you model for because you might get it

In the previous post I showed how James Hansen at GISS NASA clearly over estimated global warming in the late 80’s due to the modeling choices he made. To make a point on how influential the choice of a model is, in this post I will make modeling choices that will allow us to claim that global warming can be explained as a fluke in a random process.

I like to explain the relationship between data and models saying that data is the shadow reality casts, and models are what we believe is casting the shadow. So once we have a model  we can use it to cast shadows (make predictions) like the one James Hansen did and could be read in 1986 newspapers:

Hansen predicted global temperatures should be nearly 2 degrees higher in 20 year. “Which is about the warmest the earth has been in the las 100,000 years.”

Interestingly James Hansen downgraded his prediction in a 1988 paper from the nearly two degrees higher to a one degree higher. Though to be fair I would not be surprised if media misquoted him; I might not trust scientists but I absolutely distrust media.

Anyhow, let’s now compare NASA’s prediction in this 1988 paper (in red) to what actually happened years later (in blue):

Yearly Average Global Temperature Change
1988 NASA predictions on top of yearly average global temperature changes with major volcano activity and parts per million levels of CO2

Data for this plot comes from the B.E.S.T and N.O.A.A.

Hansen works in his paper with three scenarios, and this is how Hansen describes scenario A:

Scenario A assumes that growth rates of trace gas emissions typical of the 1970s and 1980s will continue indefinitely; the assumed annual growth averages 1.5% of current emissions, so the net greenhouse forcing increases exponentially.

The previous plot shows that this model overestimates reality; it steps away from the real temperatures right from the beginning and it seems that it sticks closely to the expected exponential growth of CO2 gas emissions. Let’s check the plot with Hansen’s prediction scenarios up to 2060:

Hansen 1988 prediction for scenario A
1988 NASA scenarios for global temperatures behavior (The “we do nothing” scenario A shows exponential growth)

So there you go, if Hansen’s model is right and we continue to pollute I guess we are pretty much… how to say in English… screwed, yeah, that’s the word. Fortunately Hansen’s model diverges from reality right from the start and for the last decade or so we have experienced a mild decline in the global temperatures. In other words, Hansen’s model casts the wrong shadow, and models pay this scientific sin of ugliness with their lives… or plastic surgery if they are pretty enough, which is usually what happens since scientists love their babies.

But this is anyway how science works if you agree with Karl Popper; we design models to describe reality with falsifiability features, we make predictions with it, they fail, we upgrade our models to fit new data and we make new predictions. Nonetheless some climatologists have the nasty habit to shape models that always overestimate temperatures, which makes the rest of us wary of the next end-of-the-world prediction coming from them.

So now that I have devilish planted a seed of doubt on the models used by climatologists, I am going to test the hypothesis that the increase in temperatures in the last century is nothing more than a random fluke.

Come on! It’s growing! That can’t possibly be random!

Well actually a random walk can easily show growing trends and yet it is a complete random process. So do temperatures behave like a random walk?

Whatever temperature we have today is likely to influence the temperature that we have tomorrow, in other words, it can be reasoned that temperatures can be describe with an autoregressive model of the form:

X_t = c + \sum_{i=1}^p \phi_i X_{t-i}+ \varepsilon_t

So let’s fit the simplest version of this model using the yearly aggregated temperatures (not to confuse with the yearly running average) instead the monthly aggregated temperatures (since monthly seasonal patterns are of no interest in this analysis). Let’s also use temperatures just from the 1880 on since this is when measurements are more reliable. The fitting yields the following results:

ARIMA(1,0,0) with non-zero mean

         ar1  intercept
      0.8671     0.0364
s.e.  0.0449     0.1317

sigma^2 estimated as 0.04435:  log likelihood=17.63
AIC=-29.26   AICc=-29.07   BIC=-20.61

We can see that the autoregressive coefficient (ar1) is dangerously close to one, and I say “dangerously” because if it actually was one the autoregressive model would be describing a random walk. We can actually test if this coefficient is one with a unit root test.

 Augmented Dickey-Fuller Test

Test Results:
    Lag Order: 1
    Dickey-Fuller: -1.5483

So the p-value for this unit root test is low but not low enough to reject the hypothesis that the temperatures in the context of an autoregressive model have a unit root and, therefore, considering data from 1880 we cannot reject the possibility that global warming is caused by a natural random process with no underlying cause.

NASA model vs Random model

Okay, so how good would be a simple autoregressive model predicting temperatures? Would it yield much worse predictions than James Hansen models? After all, a lot of science has been accounted for in NASA simulations. Well… Let’s see, considering the results of the unit root test we can fit a simple AR model embedded in a simple AR seasonal model with a period of eleven years (solar wind cycle). If we now fit the model with temperatures from 1880 to 1988 when Hansen published his predictions we have:


          ar1     sar1
      -0.3386  -0.4803
s.e.   0.0969   0.0887

sigma^2 estimated as 0.05727:  log likelihood=-0.44
AIC=6.89   AICc=7.15   BIC=14.58

And if now we plot side by side the forecast of this autoregressive model with the forecast of NASA’s model on top of the actual values from 1988 until 2011 we have the following:

1988 James Hansen (NASA) Forecast vs ARIMA(1,1,0)(1,1,0)[11] Forecast
1988 James Hansen (NASA) Forecast vs ARIMA(1,1,0)(1,1,0)[11] Forecast

Well, turns out that the autoregressive model does a better job predicting temperatures than the NASA simulations! 

I find very interesting that the eleven years seasonal period in the ARIMA model yields such a good fitting considering eleven years is the average time it takes to the Sun to reach maximum solar activity (periods outside the solar winds 9 -13 years range yield bad predictions)… Serendipity feeling here.

So is global warming a random fluke?

I personally don’t think it is. Climatologists overestimation is most likely due to their natural bias towards end-of-the-world scenarios when modeling global interactions.

On the other hand it is a bit embarrassing that a simple ARIMA model does a better job predicting temperatures than a full fledge scientific NASA model. It makes you wonder how much climatologists really know about climate.

Sure now climatologists probably have new models that cast more precise shadows; models that can explain the past and present really well but, until they don’t predict the future really well they are not something we should bet our houses on. You see, models are like witches from the middle ages; if you torture them enough you can get them to say anything you want.

The Future

I was going to end this post by making a point about how by simply making modeling choices we could justify global warming as a random fluke, but since the ARIMA model fits so well I think I will use it to predict the future global temperatures until 2060 just as Hansen did.

It’s worth to note that even though the ARIMA predictions are very good the really important thing in this case are the prediction intervals since they tell us how bad or good things can be. The ARIMA model shows a mean yearly trend of 0.028 ºC  increase in global temperatures but, if the trend changes, so will the prediction intervals when updating the ARIMA model.

Global Temperatures Predictions until 2060 (NASA vs ARIMA(1,1,0)(1,1,0)[11])
Global Temperatures Predictions until 2060 (NASA vs ARIMA(1,1,0)(1,1,0)[11])

So, will the ARIMA model in the near future still have the upper hand over 1988 NASA fancy models? Time will tell… Any bets?🙂

7 thoughts on “Climategate (2/3): be careful what you model for because you might get it

  1. I’ve read most of your posts about this and it does seem thorough and as though you understand what you’re doing. Can I just clarify what you have established. If I understand it properly, some of the data analysis and modelling hasn’t been done as well as it could have been done and some of the predictions are not quite what wer’e seeing, but the bottom line is still that we are undergoing warming that is largely due to CO2 released by our use of fossil fuels. What you’ve done here hasn’t really changed the general conclusions we should be drawing from the results of climate science? Have I understood this properly?

  2. Hi to.the.left, thanks for reading and your comment.

    The fact that the rising trend of temperatures is not clearly incompatible with a random process does not mean I believe it is.

    Though there are conflicting studies on what role CO2 plays (from those saying that the warming we are experiencing is mainly due to CO2, to those claiming it is just an amplifier, or simply a byproduct of the warming) I believe CO2 plays a role, it just makes sense to me.

    On the other hand it is important to notice that data does not show CO2 rendering the catastrophic impact that many have lead us to believe. NASA played with CO2 driven exponential temperature growth models and data shows that, at much, that growth is linear; which is great news for us anyway.

    So answering your question, no, I am not changing the general conclusion of climate sciences, just questioning the magnitude of the claims.

    • Thanks for the reply. I think that is what I understood you to be saying and it makes a certain amount of sense. Something that is of interest to me is the issue of the energy imbalance. In other words, the greenhouse gases in the atmosphere are resulting in less energy leaving the Earth than we receive. We normally then assume that this should result in a rise in surface temperatures. If we want to reach equilibrium, then this has to happen (assuming that the energy imbalance remains). However, if a lot of that energy is going into heating the deep oceans and/or melting the ice, we may not initially see as large a rise in surface temperatures as we might expect if we didn’t model the energy flow correctly.

      I guess what I’m suggesting is that looking only at the variation in surface temperatures may not be giving us the whole picture. If (as seems to be happening) a significant fraction of the energy is going into the oceans and melting ice then the surface temperatures would rise more slowly, but the outcome might eventually be the same once the oceans have increased their temperature sufficiently and once the melting of the polar ice has slowed. Have you given this any thought? If so, do you think it could change what you conclude from your analysis here?

      • Well, though I am a physicist at heart I am truly a data expert so please take my opinion with a grain of salt when it comes to physics/biology, specially if these are fields of your expertise.

        Nonetheless, those making climate models at NASA not only are supposed to be good physicists, but the cream of the cream in their field. I know they consider in their models the ocean’s temperatures but you are saying energy might be hiding in deep ocean waters. Well, I would expect geniuses at NASA to know about it and reflect that knowledge in their models. Since their models are overly catastrophic compared to what reality is showing us, and thanks God for that to be so, there are just a few options left:

      • 1 – They are not so good at physics and their models did not contemplate this “stored” energy.
      • 2 – They are good at physics but their environmentalist bias is reflected in their models.
      • 3 – They did contemplate the raise in deep ocean temperatures but Earth is way too complex for their models.
      • I would say reality lies between options 2 and 3. Anyhow, please do consider this; energy cannot only hide in the form of warm water ready to bounce back one day, for instance, the fact that we have life in Earth seemingly goes against the Physics principle of “entropy can only increase in a system” then, of course, when we take into account the energy from the Sun the principle holds and the decreased entropy in Earth stops being a mystery. What I am trying to say is, how about if this excess of energy goes into building life? I have read news linked to climate change like Giant red crabs invade the Antarctic abyss. You need lots of energy to “build” these fellas.

        As a matter of fact, we could devise mechanisms where the energy gone into developing life might actually decrease global temperatures. Earth is complex, and I have the feeling that climatologists are playing politics way too much. I also believe that they are doing so with good intentions, but I guess that at a point they should decide if they are scientists or activists; the mixture is not working well.

        So, nope, these hiding energy considerations do not alter my conclusions but thank you very much for your suggestion.

      • I wasn’t really meaning that energy is being hidden, what I was meaning is that the heat content of the oceans and the latent heat of ice are quite large. If there is an energy imbalance then to reach equilibrium, the surface temperature does have to rise. The energy required, however, to change the temperature of the oceans by a set amount is much larger than the energy to change the surface temperature by the same amount. Therefore, a relatively small error in how the models treat energy deposition in the oceans (or treat the melting of polar ice caps) could have quite a large effect on the predictions for the rise in surface temperature.

        I guess my point was a little subtler than maybe I made out. We seem to look at changes in surface temperature as being fundamental. To a certain extent it is and what you’ve done here does seem quite thorough and I don’t disagree with your conclusions. However, it seems to me (as a physicist as well) that the truly fundamental thing is the energy imbalance. This tells us how much excess energy the Earth is receiving from the Sun. This excess energy will then lead to changes in ice coverage, ocean temperatures and surface temperatures. If the surface temperature rises to reach equilibrium then the energy imbalance disappears. If the imbalance remains, then we know that the surface temperature has to continue rising. The rate at which it does so may be different to what the models predict if the models aren’t properly tracking this energy deposition and your analysis suggests exactly this. This could simply mean that we have more time, but then again maybe not.

        I certainly don’t know the answer. I guess I was just suggesting that basing so much on analysis of the surface temperature data may not be quite good enough and that we shouldn’t forget that the fundamental issue is the energy imbalance. Of course, if this energy could be used for life that might be fine. If, however, it’s simply going into heating the oceans and melting the polar ice caps then we may still see an exponential surface temperature rise once the ice has melted and the ocean temperature has risen sufficiently.

        As far as politics goes, you may have a point. I think slightly better of climate scientists than you seem to. There is no question that this is clearly a political issue and neither side of the debate is faultless.

        • By “hiding” energy I meant “unaccounted for” so I believe we were in the same page.

          Also I think climatologists should do as yourself and say “I certainly don’t know the answer” to then offer possible scenarios. Had they been cautious in their predictions now people would take them seriously, instead general public have the feeling of “oh yeah, another end-of-the-world-is-nigh-guy… whatever”. In science the best strategy should be to be right, not to be loud, but I guess among climatologists cautious scientists don’t make headlines or win Nobel Peace prizes.

      • Yes, you may well have a point. I don’t think that climate scientists are any worse than other types of scientists but their field is clearly politically relevant. What they have done badly is work out how best to engage with this “politics”. Given the level of debate that seem to be taking place at the moment, whatever they’re doing is not working very well.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s