Feeds

Big Blue prototypes software for big, big data

Coping with the astronomy info-glut

SANS - Survey on application security programs

IBM has prototyped a software architecture for the huge data demands of astronomy projects such as the SKA (square kilometer array).

One of the many problems created by a project as large as the SKA is that wherever it’s built – we’ll know next year if the South Africa bid or the Australia / New Zealand bid wins – it’s going to generate too much data to store.

With as much as Exabyte of data as its daily dump, the SKA will demand new techniques just so astronomers can use the facility’s output (cue: Mission Impossible theme).

Working with New Zealand-based radio astronomer, Dr Melanie Johnston-Hollitt from Wellington’s Victoria University, IBM has created the Information Intensive Framework prototype.

Under the framework, data will be classified into astronomical concepts, and overlaid with a guided search facility for faster data access and fewer errors. IBM says the prototype has also suggested further improvements to achieve the SKA’s performance demands.

"Undertaking research on exa-scale datasets will force radio astronomers into a new, as yet, unexplored regime of automated processing, imaging and analysis,” Dr Johnston-Hollitt said in IBM’s announcement.

“Surveys on even SKA precursor telescopes such as ASKAP and MWA are expected to produce catalogues of tens of millions of radio sources. How we organise and classify these data, which we will have in the next three years, is a significant challenge. We will need new solutions to fully realize the vast scientific potential of these datasets and it's fantastic that organisations like IBM are prepared to take up that challenge.”

If the A/NZ team wins the SKA contract, data will have to be pre-processed close to the telescopes (most of which would be in the remote north-west of Western Australia), then sent back to the Pawsey Supercomputing Centre in Perth for storage and analysis.

Since contracts are now open for the next phase of the Pawsey Centre’s implementation, The Register wouldn’t be surprised if IBM isn’t the only company trying to position itself with relevant software architectures. ®

Top three mobile application threats

More from The Register

next story
Fancy joining Reg hack on quid-a-day challenge?
Recruiting now for charity starvation diet
Red-faced LOHAN team 'fesses up in blown SPEARS fuse fiasco
Standing in the corner, big pointy 'D' hats
KILLER SPONGES menacing California coastline
Surfers are safe, crustaceans less so
LOHAN's Punch and Judy show relaunches Thursday
Weather looking good for second pop at test flights
Discovery time for 200m WONDER MATERIALS shaved from 4 MILLENNIA... to 4 years
Alloy, Alloy: Boffins in speed-classification breakthrough
Curiosity finds not-very-Australian-shaped rock on Mars
File under 'messianic pastries' and move on, people
Elon Musk's LEAKY THRUSTER gas stalls Space Station supply run
Helium seeps from Falcon 9 first stage, delays new legs for NASA robonaut
Top Secret US payload launched into space successfully
Clandestine NRO spacecraft sets off on its unknown mission
prev story

Whitepapers

Designing a defence for mobile apps
In this whitepaper learn the various considerations for defending mobile applications; from the mobile application architecture itself to the myriad testing technologies needed to properly assess mobile applications risk.
3 Big data security analytics techniques
Applying these Big Data security analytics techniques can help you make your business safer by detecting attacks early, before significant damage is done.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.