Mailbag pits Modeller vs Modeller

3 Big data security analytics techniques

Andrew's Mailbag "El Reg, it is very disappointing tthta you blatantly run a denialist agenda, then do not allow debate. I wouldnt piss on Orlowski if he was on fire." Register reader Dave 14

"that useless half wit Orlowski has again been allowed to post his climate denial bullshit, without any comments allowed as a right of reply. I am sick of ElReg supporting fruit loop deniers and not allowing comment on their irrelevant rantings. Fuck you El Reg." Register reader Dave 14

Welcome to Andrew's Mailbag, which is a bit like the Comments, but without the death threats. It also gets contributions you don't get, possibly because I insist on getting my mail via a mailto, rather than being typed into a web form. This is the response to my Climategate analysis. For whatever reason, we're I'm blessed with two contributions you're going to enjoy comparing.

First you'll hear from a Climate Scientist from a top university. A PhD. He didn't request anonymity but given the bitchfest that we know it is, from the CRU leak, we don't want to get him into trouble. Then there's one from an experienced computer modeller. I got a lot of email after my Climategate analysis, but these two are worth a standalone, so compare and contrast. I think you'll enjoy them.

Interesting article, that makes some good points. As a climate scientist I have some opinions. First a quick pedantic comment.

The statement that models have no predictive value and that correlation does not demonstrate causation is false. An example of a model with predictive value is Newtons law of motion, Force = Mass x Acceleration. The high correlation between the pain you receive and putting your hand into fire demonstrates that the fire is hot. The only way we can demonstrate cause and effect is via a high value of correlation.

What I think you mean is:

(1) Sometimes models are believed more than the evidence suggests. (2) Often correlations are used as evidence without estimating how likely the correlation is arrived at by chance. (3) Often correlations are used as evidence without having an understanding of why such a correlation should take place.

Regarding the main thrust of the article:

I feel that the main problem is the pressure scientists are under to publish rather than any conspiracy theory.* If you don't publish more papers, then that is the end of your career and you can't do any more science. In fact, at each step up the ladder the majority of scientists have their career ended by a lack of publications. It is not just a few sackings it is the majority that loose their jobs. This has lead to an explosion in shoddy papers that do nothing to aid understanding.

Peer reviewers are not paid and do not have the time or energy to review the stack of papers that arrive on their desks every day. They hardly ever check calculations and assume that the writer is honest. Being difficult to understand, papers are given to people in closely related fields to review.

If you have worked for years to obtain data and you give it away, other scientists will make the discovery and get the credit (and papers) and you will loose your job. Clearly this system rewards secrecy, which is not good for science.

A climate scientist has to be an expert mathematician, statistician, physicist and programmer with extensive knowledge of the climate system. Is it any wonder that a jack of all trades is master of none? You are funded and usually work as an individual which does not encourage teamwork.

I feel that we can all complain about the system and shout how imperfect it is. Coming up with a solution however is the really difficult task.

But, wait.

I am a statistical modeller by profession and an engineer by education. I find the climate scientists argument about positive feedback the most disingenuous of them all.

If you want a square wave generator, or a triangle wave generator, then you use positive feedback. Otherwise the system has to be controlled with negative feedback. The reliance on positive feedback is always going to create a runaway system.

And what is most non-intuitive about the whole thing is that temperatures on the earth have remained remarkably stable. Yes, the equilibrium point has changed, but the stability hasn't. Even during the Carboniferous period with carbon dioxide levels at twice that of now, we still had a stable equilibrium. Any model relying on positive feedback is going to result in no equilibrium.

Furthermore, just as when you are setting up positive feedback circuits (like the square wave generator), a tiny change in inputs can have a huge change in outputs. The same appears to be the case with most climate models. If you tweak their constant for how water precipitates into clouds, then you get a massive difference in results.

They then tune the constants for a very short period of tuning data (usually the last 20 or 30 years where we have more observations of the temperature) and then try to forecast 100+ years. Anyone who needs an accurate forecast knows that to forecast the next week or month you often need years of data to get a statistically significant answer. The climate modellers are forecasting 100 years with 20 years of training data. And doing so with over-fitted constants that massively change results if you get them wrong.

The climate modellers are using 20 or 30 years of detailed data, but then supplementing it with additional data from further back produced by things like tree rings and ice cores. The core data for things like tree rings is often dodgy - taken from cherry-picked tree cores. Freeman Dyson has it right when he gets on his soap box about how the models are generally flawed.

Furthermore, the analysis is based on very poor statistical methods. They used a Principal Component Analysis to back out the temperature record from the last 1000 years. Unfortunately, they failed to carry out PCA correctly. If you generate random but auto-correlated data for that period, you can actually generate a hockey stick from it because of the flaws in their analysis. Wegman et al have thoroughly discredited the process they used, yet Mann et al continue to peddle it and it is still used to demonstrate the "consensus" opinion by many journalists.

As pointed out at El Reg, CO2 climate forcing has no smoking gun to link cause and effect, and as such, it is hard to accept such a minor factor affects the climate and that other forcings, such as the sun, cannot be key factors.

I will be interested to see what happens when the greenhouse gas hypothesis is shown to be the charade that it is.

More Climategate mail over the page.

Top three mobile application threats

More from The Register

next story
So, just how do you say 'the mutt's nuts' in French?
Vital linguistic question interrupts LOHAN spaceplane mission
95 floors in 43 SECONDS: Hitachi's new ultra-high-speed lift
Guangzhou skyscraper denizens to hold on to hats
Most Americans doubt Big Bang, not too sure about evolution, climate change – survey
Science no match for religion, politics, business interests
KILLER SPONGES menacing California coastline
Surfers are safe, crustaceans less so
Discovery time for 200m WONDER MATERIALS shaved from 4 MILLENNIA... to 4 years
Alloy, Alloy: Boffins in speed-classification breakthrough
LOHAN and the amazing technicolor spaceplane
Our Vulture 2 livery is wrapped, and it's les noix du mutt
Liftoff! SpaceX Falcon 9 lifts Dragon on third resupply mission to ISS
SpaceX snaps smartly into one-second launch window
STEALTHY NANOROBOTS dress up as viruses, prepare to sneak into YOUR BODY
Cloaking techniques nicked from viruses tackle roadblocks on way to medical frontier
prev story


Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mainstay ROI - Does application security pay?
In this whitepaper learn how you and your enterprise might benefit from better software security.
Combat fraud and increase customer satisfaction
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.