Hey, software snobs: Hardware love can set your code free
Comment The two go hand in hand when scaling data-crunching systems
In computing there are many, many different ways to run down other people’s work, not the least of which is: “OK, so they removed the bottleneck, but only by throwing faster hardware at it.”
The implication is that tackling an issue just with software is intrinsically better. When did you ever hear anyone say: “OK, so they …
Big Data versus small data: Unpicking the paradox
Can NoSQL and relational both be adaptable?
NoSQL and Big Data crashed into the ordered world of relational architectures a few years back, thanks to services like Twitter, Facebook and LinkedIn.
But while concepts such as key value stores and content-specific stores have certainly enriched our environments, the downside to their arrival is that it has created quite a bit …
Big Data's big issue: Where are all the data scientists coming from?
Analysis This personnel gap isn't just a job-title change
Plug “data scientist” into Google and it is clear the job title has finally come of age and, suddenly there is a huge skills shortage.
An oft-quoted source about this shortage is a McKinsey Global Institute study, here. This predicts a talent gap of 140,000 to 190,000 people by 2018 in the US alone. I am always sceptical of IT …
Big Data bites back: How to handle those unwieldy digits
When you can't just cram it into tables
Data is easy. It comes in tables that store facts and figures about particular items – say, people. The columns define the data to be stored about each item (such as FirstName, LastName) and there is one row for each person. Most tabular database engines are relational and we use SQL for querying. So this "Big Data" thang must …
Microsoft shakes SQL Server 2012's business end at big data
Review Still holding out for Hadoop
Those not in “Denali Denial” will be interested in SQL Server 2012, which has just been unwrapped and is currently being delivered to manufacturers.
In this review (I used Release Candidate 0) I’ll focus on how the new database looks, feels and compares with the existing server. I’ll also take a look at how SQL Server's big data …
SQL Server 2008 - from semi-relational to sublime
Review Inside Microsoft's R2 preview
SQL Server 2008 R2 is a step closer to reality.
On the heels of August's first code drop, Microsoft has released a second, more-fully-featured community technology preview (CTP) of its next database server. It promises a number of things, including improved business intelligence through database changes and integration with …
Teradata unveils SSD Blurr appliance
Feels need for speed
Teradata has previewed a concept appliance based on Solid State Disks (SSDs) and codenamed Blurr to partners.
Unveiled at the company's Partners User Group Conference in Washington, DC, the first device - called Teradata Extreme Performance Appliance - is going onto a customer site in two weeks time.
Before you ask, yes, it is …
What your database needs is a good thermometer
Data blows hot and cold
In the very early days of databases, when they contained relatively small quantities of information, the distinction between frequencies of access was pretty immaterial.
Now that we collect and store data in massive quantities, however, the distinction is becoming increasingly important. And apposite terms have evolved to …
In the spin of SSDs on database servers
The future is static
Interest in solid state drives (SSDs) is growing as their initially terrifying price plummets: Intel recently announced a 60 per-cent reduction in the cost of its X25-M Mainstream SATA SSD since its launch last year.
SSDs are attractive because they radiate less heat than traditional spinning disks, they're smaller, quieter and …
Sarah Palin's words get data mined
USA '08 Whoah, there's data here? Business analysis gets to work on VP transcripts
Business Intelligence (BI) is about extracting information from data. The name implies that it is only applicable to business information, but that’s misleading. Given the right techniques, information can be found in the most unexpected places - even in the speeches of vice-presidential candidates.
Sarah Palin’s meteoric rise …
Beware the innocent systems 'health check'
D'oh! Cough, please
This column has generated a lot of feedback, some indicating that the answers to the problems were obvious. Quite right - they are obvious to people with years of practice, and some of our readers are blessed with that experience.
But the whole point of the series is that none of us start with 20 years of experience. We all have …
SQL Server 2008 moves on, wants to stay friends
Review It’s not you, it’s the BI and database engine
Microsoft's SQL Server 2008 is now upon us. More than a year after its initial planned delivery date, Microsoft Wednesday released to manufacturing code for a new version of its database that's almost as significant as its last great release - SQL Server 2005, three years ago.
Microsoft is, with SQL Server 2008, bucking its …
Lateral thought saves sizzling server
D'oh! Game, set and crash
I learned a long time ago that generating random numbers (really, truly random numbers) is a non-trivial exercise.
However, I completely failed to apply that computer science lesson to the real world of computing and continued to believe that events in the Newtonian world could happen without a cause. Such a belief system is not …
What I learned from a dumb terminal
D'oh! Economy of language meets an excess of context
There is a (probably apocryphal) entry in a naval officer's fitness report that reads: "This officer never makes the same mistake twice. However, he appears to be attempting to make them all once."
As developers we should try to avoid collecting the full set, but making some mistakes are inevitable; the trick is to learn from …
Windows experiment meets the bottom line
Project Watch: Microsoft 2008 Resist anything but temptation
This project I began writing about in late January, the prototyping of a large database project using the latest versions of a Microsoft software stack, has been an unusual exercise.
Unusual because, from the start, both the executive and the technical personnel were fully co-operative throughout the entire undertaking. The …
Virtual Earth puts human face on data
Project Watch: Microsoft 2008 Lightweight programming, minus Google
Those who have been following Project Watch will know that I have been leading the development in a large database project using SQL Server 2008, Windows 2008 and Visual Studio 2008.
Relatively heavy stuff. Part of that project, though, involves the creation of a mashup that displays our spatial data on a map - yes, I know, all …
Frustration and joy - Microsoft's CTP in action
Project Watch: Microsoft 2008 Thank God for file streaming
OK, so choosing and installing the hardware - that was easy. I wish I could say the same for the software.
On the face of it all I had to do was install the beta version of Windows Server 2008, the production version of Visual Studio 2008 and the beta SQL Server 2008. Sorry, by beta of course I mean community technology preview …
Windows hardware challenge draws on resources
Project Watch: Microsoft 2008 Things get heavy
Here's a question for you: what hardware does it take to run an entirely new, pre-release Windows operating system and 1TB-worth of SQL Server 2008 community technology preview?
This question seems simple to answer, but the challenge comes in locating the requisite hardware. One problem that always arises when using any beta …
Microsoft measures up
Project Watch: Microsoft 2008 A meter is a meter is a meter. Not
The next task in rolling out our 1TB SQL Server 2008 application using Visual Studio 2008 and Windows Server 2008 is to import signed spatial data into the spatial data type. This is the code:
UPDATE
tblSpatialData
SET
SpatialLocation =
geography::STGeomFromText('POINT …
Back to basics for SQL Server 2008
Project Watch: Microsoft 2008 Hand feeding
When I asked: "How do we convert more than 12,000 location items - by hand?" we had almost completed the process as part of our move to Microsoft's up-coming SQL Server 2008. The question was, in fact, rhetorical. Nevertheless, we received a lot of advice and suggestions from Reg Dev readers. This, for example, from …
Experience overcomes Microsoft's broken promises
Project Watch: Microsoft 2008 People-unready database ready
Before we go on, let's just talk briefly, in a quiet voice, about the delay to SQL Server 2008.
The major issue here is that whilst Microsoft conveniently forgets the past, most of us can still remember SQL Server 2003, er...2004, oh, actually, that was eventually 2005. So Microsoft is turning into a serial offender when it …
The 'blem wit' error messages
Things get terse
When I was young I built up a collection of system error messages. Ok, look, it's not as sad as collecting stamps! It is? Really? Oh well, never mind. Anyway, my recent piece about Borland putting rude words in Quattro Pro got me thinking it was time to revisit that collection.
Some of them date back to the days of the mainframe …
Inside the Windows 2008 stack experience
Project Watch: Microsoft 2008 Laughter, tears, upgrades
Register Developer regular Mark Whitehorn is rolling out a significant new IT project. This has, at its heart, a database of around 1TB and is ultimately expected to support several thousand users.
The project is using nothing less than upgrades to a trio of big-ticket products from Microsoft - Visual Studio 2008, Windows …
'Draconian' Microsoft promises to make Office work again
Update Busting through SP3
Microsoft has threatened to release a fix for an Office 2003 update that may well have the productivity suite work as intended once again.
As reported earlier Service Pack 3 for Office takes the unprecedented step of barring access to files created with earlier versions of the product. Install the Service Pack and your stash of …
Why Borland trashed its spreadsheet
Myths and legends Finds hidden meaning
Always remember, the compiler is your friend. Programming is stressful but no matter how many f**ks and b***ocks you might occasionally feel the need to insert into the comments, the compiler will always strip them out. It's great.
Of course, it doesn't actually go looking for rude words. So it would be a mistake to use them …
Mind your languages with Microsoft LINQ
Data-access sticking plaster?
We've been hearing a growing amount this year about LINQ - Microsoft's Language Integrated Query. You can expect a lot more next year, starting in February as Microsoft launches Visual Studio 2008 and SQL Server 2008. LINQ promises to close the skills and knowledge gap for developers using C# and VisualBasic trying to connect to …
Running queries on the HMRC database fiasco
Comment Dis-information systems management
When it comes to talking about last week's data loss by the HMRC, I was told not to use precious words outlining my feelings of rage and bafflement that a government body can be so cavalier with so much data because, presumably, we all feel the same.
So I will simply note, for the record, that my gob has been totally smacked by …
Oracle 'cheated' in TPC benchmarks
Myths and legends Denies wrong doing
When database folks gather to sup ale and chat, as they invariably will at Oracle's OpenWorld in San Francisco this week, talk occasionally turns to benchmarking. As soon as it does, someone will say: "Of course the trouble is you can't trust the vendors. Wasn't Oracle caught cheating at the TPC benchmarks?"
The public has a …
Hands on with MDX
Dimension and measures (yes, but can it handle cubic furlongs?)
Following our introduction to MDX (to be found here) this follow-up article is a get-you-started guide to using this powerful language to manipulate multi-dimensional data.
The basics
Relational databases store data in two-dimensional tables, a familiar concept that mimics grids of data on paper. Multi-dimensional data is …
MDX: Why it will matter to application developers
That's Multi-Dimensional eXpressions to you, guv
Since when did application developers need to know about an analytical querying language like MDX? Since February 27th, next year. That’s when Microsoft will launch SQL Server 2008. Of course, you’ll still have breathing space because, as we are painfully aware, launching and actually delivering products are, in Microsoft’s …
Evolutionary vs. traditional database design
DBA fights back
We recently published an article on the advantages of evolutionary database design (EDBD), a process which has its roots in the agile/extreme programming world. To provide a little balance, some yang for the yin, we asked Mark Whitehorn to comment on the article and give his views on EDBD vs. the more traditional database design …
Oracle 11g’s new toys
It may be Oracle, but it is shiny
Oracle has just announced onto a suspecting world the latest release of its flagship database engine.
It is packed full of additional features. If you are an Oracle DBA and/or developer, then you will be keen to find out what’s coming because, for good or ill, these will be the features you will work with in the coming years. …
Remember the ‘Service’ in SOA
But what if firms don't know what 'service' means?
There is, of course, no widely agreed definition of Service Oriented Architecture – I know this to be true because I read it on the Web. However most people might agree, if they happened to be chilled out and mellow at the time, that, as my Web source (Wikipedia as it happens) continues “Service-orientation describes an …
The great migration debate: Essbase to Analysis Services
Moving Cubes
There may be multiple political/managerial reasons behind a decision to move from Hyperion's Essbase to Microsoft's Analysis Services – not least of which might be the recent purchase of Hyperion by Oracle.
Whatever the reason, from a developer's viewpoint, moving a set of multi-dimensional databases from one engine to another …
MUST is a must
A good addition for migrating Access DBs
The task of migrating or upsizing Microsoft Access databases is a common one, particularly for small businesses that are growing well and feel a real need to move. Microsoft's SSMA (SQL Server Migration Assistant) is an obvious candidate, but here is an alternative that compares well.
Microsoft's SSMA
In October last year I …
eBay's phishy old problem
Comment We were wrong, but is eBay wronger?
Reg Developer recently published a story about listings on eBay that point users to phishing sites. We thought we'd uncovered a new security issue on eBay, but it turns out we were wrong.
Not wrong about the security issue, there certainly is one. Our error was in assuming that it was new and/or that eBay didn't know about it …
It was Microsoft wot done it
Database myths and legends (Part 9) What did happen to that Bloor Report?
In this series we're looking at the myths and legends of the database world - some are true, some false. The myth under the spotlight today is: Bloor Research once produced a report that slammed SQL Server so much that Microsoft had it suppressed.
OK, let's start at the top. Bloor Research is a highly respectable analyst …
Gone phishing with eBay
Why is there a naked woman in that Bentley?
There I was, on Monday night, scanning eBay for car bits. This is not a problem. I have this under complete control. I can give up buying worn out parts and rusty bits of bodywork at any time. Really.
Anyway, I spotted a real bargain, a 2007 Bentley Continental for 0.01 GBP. Since these usually retail for something in the …
Steve Ballmer may have said something interesting; we couldn’t possibly comment...
A non-report from the third and final day of Microsoft’s BI conference
To criticise Microsoft is, of course, to attack not only an easy target but also a popular one. Many people hate the big M for the simple (and undeniable) reason that it is successful. This is a comfortable reason to hate a company because it guarantees an unending supply of hate figures; if/when the big M falls from grace …
The four pillars of Katmai
Microsoft BI conference Dispatches from the front
The keynote on the second day of Microsoft's BI conference was given by Ted Kummert – corporate vice president of the data storage and platform division at Microsoft.
As Jeff Raikes before him, he devoted considerable time to Katmai, the next version of SQL Server. He too stressed that the release date will be 2008 (so perhaps …
Microsoft opens kimono for 'Katmai' SQL Server
Marks datebook at BI Conference
Microsoft's first-ever BI Conference kicked off this morning in Seattle with a keynote that promised the next version of SQL Server will hit the streets sometime next year.
Jeff Raikes, president of the Microsoft Business Division outlined/reiterated Microsoft's strategy for "delivering pervasive BI and performance management" …
Microsoft vs. Teradata
Column Data Warehousing – there really isn't just one answer
Microsoft and Teradata are both significant players in the BI market but they have wildly different approaches to the challenges of extracting information from data. The reason lies in the fact that the two companies elected to solve two very different, but equally intractable, computational problems in order to get their BI …
A tribute to Jim Gray
Lost, but not forgotten
As we post this, Jim Gray has been missing for three months. It now seems very unlikely that he is still alive, although his family has not given up hope.
We decided to publish the following tribute to him, because his life deserves to be celebrated.
James 'Jim' Nicholas Gray (born 1944)
Photo of Jim Gray. of Microsoft …
Is Gartner's Magic Quadrant really magic?
Database myths and legends (Part 8) The disappearing act
In this series we're looking at the myths and legends of the database world; some are true, some false. The myth under the spotlight today is whether Gartner's Magic Quadrant really is magic.
(True, Gartner's Magic Quadrant (MQ) isn't just applied to databases, but its recent application to the area of BI platforms, for me at …
MySQL – is this database fit for the Enterprise?
Part 2: Carving a niche
MySQL has recently appeared as an Enterprise edition. We have already looked at whether MySQL (the company) is enterprise ready, but we can also ask whether the product itself is suitable for enterprise use.
Some Reg Dev readers clearly have strong views about this already.
First up, remember that Enterprises come in all kinds …
Don't Dodge this Viper
Analysis IBM's DB2 gets native XML support
On the 6th Feb 2007 IBM announced Viper. Yes, we know that it announced Viper in July 2006, but this is a different Viper, it just happens to have the same name. Really, it all makes perfect sense; you just have to think like IBM…..
DB2, the company’s database engine, comes in three tangy, zesty flavours:
DB2 for z/OS ( …
MySQL is the company's SQL now...
So what must the world's nicest database do to drum up some enterprise sales?
Let’s face it; MySQL is a fabulous database engine. Not only is it free, it’s small, powerful and easy to drive. It also runs happily on free operating systems and so it can be used to create incredibly cost-effective database servers.
Of course, like all database engines, it polarizes those in the computing world. Some people …
OLAP and the need for SPEED
Database myths and legends (Part 7) In another dimension
In this series we're looking at the myths and legends of the database world; some turn out to be true, others false. This myth is about why we use OLAP.
If you follow the Inmon model, you use a relational data warehouse for flexibility and OLAP cubes in the data marts for the speed. On the other hand, if you follow Kimball, you …
Building a data warehouse on parallel lines
Kognitio ergo something-for-nothing?
Never look a gift horse in the mouth, especially if there are many of them running in parallel…
There are various structures we can use in a data warehouse – each with its pros and cons. For example, if you use a relational structure for the core of the warehouse then you gain very high flexibility but lose out on speed. …
Ancient pyramids discovered in Bosnia
Natural or man-made?
The Great Pyramid of Giza is the sole survivor of the Seven Wonders of the World. An Arab proverb says that: "Man fears time, yet time fears the Pyramids", a reference to the fact that the pyramid has survived for about 4,500 years and, in that time, has lost a mere 10 metres off its incredible 145 metre height.
Composed of two …
