Tell me, professor, what is big data?
Big Data may be misunderstood and overhyped - but the promise of data growth enabling a goldmine of insight is compelling. Professor Mark Whitehorn, the eminent data scientist, author and occasional Register columnist, explains what big data is and why it is important.
Sometimes life is generous and hands you an unexpected gift …
We gave SQL Server 2012 one year to prove itself: What happened?
I reviewed SQL Server 2012, codenamed Denali, just over a year back and highlighted the major improvements in Microsoft's relational database. After a year in production, was I right, or have other features proved more important in practice?
I'm bound to agree with my last review, so I polled colleagues who also work with SQL …
Hey, software snobs: Hardware love can set your code free
In computing there are many, many different ways to run down other people’s work, not the least of which is: “OK, so they removed the bottleneck, but only by throwing faster hardware at it.”
The implication is that tackling an issue just with software is intrinsically better. When did you ever hear anyone say: “OK, so they …
Big Data versus small data: Unpicking the paradox
NoSQL and Big Data crashed into the ordered world of relational architectures a few years back, thanks to services like Twitter, Facebook and LinkedIn.
But while concepts such as key value stores and content-specific stores have certainly enriched our environments, the downside to their arrival is that it has created quite a bit …
Big Data's big issue: Where are all the data scientists coming from?
Plug “data scientist” into Google and it is clear the job title has finally come of age and, suddenly there is a huge skills shortage.
An oft-quoted source about this shortage is a McKinsey Global Institute study, here. This predicts a talent gap of 140,000 to 190,000 people by 2018 in the US alone. I am always sceptical of IT …
Big Data bites back: How to handle those unwieldy digits
Data is easy. It comes in tables that store facts and figures about particular items – say, people. The columns define the data to be stored about each item (such as FirstName, LastName) and there is one row for each person. Most tabular database engines are relational and we use SQL for querying. So this "Big Data" thang must …
Microsoft shakes SQL Server 2012's business end at big data
Those not in “Denali Denial” will be interested in SQL Server 2012, which has just been unwrapped and is currently being delivered to manufacturers.
In this review (I used Release Candidate 0) I’ll focus on how the new database looks, feels and compares with the existing server. I’ll also take a look at how SQL Server's big data …
SQL Server 2008 - from semi-relational to sublime
SQL Server 2008 R2 is a step closer to reality.
On the heels of August's first code drop, Microsoft has released a second, more-fully-featured community technology preview (CTP) of its next database server. It promises a number of things, including improved business intelligence through database changes and integration with …
Teradata unveils SSD Blurr appliance
Teradata has previewed a concept appliance based on Solid State Disks (SSDs) and codenamed Blurr to partners.
Unveiled at the company's Partners User Group Conference in Washington, DC, the first device - called Teradata Extreme Performance Appliance - is going onto a customer site in two weeks time.
Before you ask, yes, it is …
What your database needs is a good thermometer
In the very early days of databases, when they contained relatively small quantities of information, the distinction between frequencies of access was pretty immaterial.
Now that we collect and store data in massive quantities, however, the distinction is becoming increasingly important. And apposite terms have evolved to …
In the spin of SSDs on database servers
Interest in solid state drives (SSDs) is growing as their initially terrifying price plummets: Intel recently announced a 60 per-cent reduction in the cost of its X25-M Mainstream SATA SSD since its launch last year.
SSDs are attractive because they radiate less heat than traditional spinning disks, they're smaller, quieter and …
Sarah Palin's words get data mined
Business Intelligence (BI) is about extracting information from data. The name implies that it is only applicable to business information, but that’s misleading. Given the right techniques, information can be found in the most unexpected places - even in the speeches of vice-presidential candidates.
Sarah Palin’s meteoric rise …
Beware the innocent systems 'health check'
This column has generated a lot of feedback, some indicating that the answers to the problems were obvious. Quite right - they are obvious to people with years of practice, and some of our readers are blessed with that experience.
But the whole point of the series is that none of us start with 20 years of experience. We all have …
SQL Server 2008 moves on, wants to stay friends
Microsoft's SQL Server 2008 is now upon us. More than a year after its initial planned delivery date, Microsoft Wednesday released to manufacturing code for a new version of its database that's almost as significant as its last great release - SQL Server 2005, three years ago.
Microsoft is, with SQL Server 2008, bucking its …
Lateral thought saves sizzling server
I learned a long time ago that generating random numbers (really, truly random numbers) is a non-trivial exercise.
However, I completely failed to apply that computer science lesson to the real world of computing and continued to believe that events in the Newtonian world could happen without a cause. Such a belief system is not …
What I learned from a dumb terminal
There is a (probably apocryphal) entry in a naval officer's fitness report that reads: "This officer never makes the same mistake twice. However, he appears to be attempting to make them all once."
As developers we should try to avoid collecting the full set, but making some mistakes are inevitable; the trick is to learn from …
Windows experiment meets the bottom line
This project I began writing about in late January, the prototyping of a large database project using the latest versions of a Microsoft software stack, has been an unusual exercise.
Unusual because, from the start, both the executive and the technical personnel were fully co-operative throughout the entire undertaking. The …
Virtual Earth puts human face on data
Those who have been following Project Watch will know that I have been leading the development in a large database project using SQL Server 2008, Windows 2008 and Visual Studio 2008.
Relatively heavy stuff. Part of that project, though, involves the creation of a mashup that displays our spatial data on a map - yes, I know, all …
Frustration and joy - Microsoft's CTP in action
OK, so choosing and installing the hardware - that was easy. I wish I could say the same for the software.
On the face of it all I had to do was install the beta version of Windows Server 2008, the production version of Visual Studio 2008 and the beta SQL Server 2008. Sorry, by beta of course I mean community technology preview …
Windows hardware challenge draws on resources
Here's a question for you: what hardware does it take to run an entirely new, pre-release Windows operating system and 1TB-worth of SQL Server 2008 community technology preview?
This question seems simple to answer, but the challenge comes in locating the requisite hardware. One problem that always arises when using any beta …
Microsoft measures up
The next task in rolling out our 1TB SQL Server 2008 application using Visual Studio 2008 and Windows Server 2008 is to import signed spatial data into the spatial data type. This is the code:
UPDATE
tblSpatialData
SET
SpatialLocation =
geography::STGeomFromText('POINT …
Back to basics for SQL Server 2008
When I asked: "How do we convert more than 12,000 location items - by hand?" we had almost completed the process as part of our move to Microsoft's up-coming SQL Server 2008. The question was, in fact, rhetorical. Nevertheless, we received a lot of advice and suggestions from Reg Dev readers. This, for example, from …
Experience overcomes Microsoft's broken promises
Before we go on, let's just talk briefly, in a quiet voice, about the delay to SQL Server 2008.
The major issue here is that whilst Microsoft conveniently forgets the past, most of us can still remember SQL Server 2003, er...2004, oh, actually, that was eventually 2005. So Microsoft is turning into a serial offender when it …
The 'blem wit' error messages
When I was young I built up a collection of system error messages. Ok, look, it's not as sad as collecting stamps! It is? Really? Oh well, never mind. Anyway, my recent piece about Borland putting rude words in Quattro Pro got me thinking it was time to revisit that collection.
Some of them date back to the days of the mainframe …
Inside the Windows 2008 stack experience
Register Developer regular Mark Whitehorn is rolling out a significant new IT project. This has, at its heart, a database of around 1TB and is ultimately expected to support several thousand users.
The project is using nothing less than upgrades to a trio of big-ticket products from Microsoft - Visual Studio 2008, Windows …
'Draconian' Microsoft promises to make Office work again
Microsoft has threatened to release a fix for an Office 2003 update that may well have the productivity suite work as intended once again.
As reported earlier Service Pack 3 for Office takes the unprecedented step of barring access to files created with earlier versions of the product. Install the Service Pack and your stash of …
Why Borland trashed its spreadsheet
Always remember, the compiler is your friend. Programming is stressful but no matter how many f**ks and b***ocks you might occasionally feel the need to insert into the comments, the compiler will always strip them out. It's great.
Of course, it doesn't actually go looking for rude words. So it would be a mistake to use them …
Mind your languages with Microsoft LINQ
We've been hearing a growing amount this year about LINQ - Microsoft's Language Integrated Query. You can expect a lot more next year, starting in February as Microsoft launches Visual Studio 2008 and SQL Server 2008. LINQ promises to close the skills and knowledge gap for developers using C# and VisualBasic trying to connect to …
Running queries on the HMRC database fiasco
When it comes to talking about last week's data loss by the HMRC, I was told not to use precious words outlining my feelings of rage and bafflement that a government body can be so cavalier with so much data because, presumably, we all feel the same.
So I will simply note, for the record, that my gob has been totally smacked by …
Oracle 'cheated' in TPC benchmarks
When database folks gather to sup ale and chat, as they invariably will at Oracle's OpenWorld in San Francisco this week, talk occasionally turns to benchmarking. As soon as it does, someone will say: "Of course the trouble is you can't trust the vendors. Wasn't Oracle caught cheating at the TPC benchmarks?"
The public has a …
Hands on with MDX
Following our introduction to MDX (to be found here) this follow-up article is a get-you-started guide to using this powerful language to manipulate multi-dimensional data.
The basics
Relational databases store data in two-dimensional tables, a familiar concept that mimics grids of data on paper. Multi-dimensional data is …
MDX: Why it will matter to application developers
Since when did application developers need to know about an analytical querying language like MDX? Since February 27th, next year. That’s when Microsoft will launch SQL Server 2008. Of course, you’ll still have breathing space because, as we are painfully aware, launching and actually delivering products are, in Microsoft’s …
Evolutionary vs. traditional database design
We recently published an article on the advantages of evolutionary database design (EDBD), a process which has its roots in the agile/extreme programming world. To provide a little balance, some yang for the yin, we asked Mark Whitehorn to comment on the article and give his views on EDBD vs. the more traditional database design …
Oracle 11g’s new toys
Oracle has just announced onto a suspecting world the latest release of its flagship database engine.
It is packed full of additional features. If you are an Oracle DBA and/or developer, then you will be keen to find out what’s coming because, for good or ill, these will be the features you will work with in the coming years. …
Remember the ‘Service’ in SOA
There is, of course, no widely agreed definition of Service Oriented Architecture – I know this to be true because I read it on the Web. However most people might agree, if they happened to be chilled out and mellow at the time, that, as my Web source (Wikipedia as it happens) continues “Service-orientation describes an …
The great migration debate: Essbase to Analysis Services
There may be multiple political/managerial reasons behind a decision to move from Hyperion's Essbase to Microsoft's Analysis Services – not least of which might be the recent purchase of Hyperion by Oracle.
Whatever the reason, from a developer's viewpoint, moving a set of multi-dimensional databases from one engine to another …
MUST is a must
The task of migrating or upsizing Microsoft Access databases is a common one, particularly for small businesses that are growing well and feel a real need to move. Microsoft's SSMA (SQL Server Migration Assistant) is an obvious candidate, but here is an alternative that compares well.
Microsoft's SSMA
In October last year I …
eBay's phishy old problem
Reg Developer recently published a story about listings on eBay that point users to phishing sites. We thought we'd uncovered a new security issue on eBay, but it turns out we were wrong.
Not wrong about the security issue, there certainly is one. Our error was in assuming that it was new and/or that eBay didn't know about it …
It was Microsoft wot done it
In this series we're looking at the myths and legends of the database world - some are true, some false. The myth under the spotlight today is: Bloor Research once produced a report that slammed SQL Server so much that Microsoft had it suppressed.
OK, let's start at the top. Bloor Research is a highly respectable analyst …
Gone phishing with eBay
There I was, on Monday night, scanning eBay for car bits. This is not a problem. I have this under complete control. I can give up buying worn out parts and rusty bits of bodywork at any time. Really.
Anyway, I spotted a real bargain, a 2007 Bentley Continental for 0.01 GBP. Since these usually retail for something in the …
Steve Ballmer may have said something interesting; we couldn’t possibly comment...
To criticise Microsoft is, of course, to attack not only an easy target but also a popular one. Many people hate the big M for the simple (and undeniable) reason that it is successful. This is a comfortable reason to hate a company because it guarantees an unending supply of hate figures; if/when the big M falls from grace …
The four pillars of Katmai
The keynote on the second day of Microsoft's BI conference was given by Ted Kummert – corporate vice president of the data storage and platform division at Microsoft.
As Jeff Raikes before him, he devoted considerable time to Katmai, the next version of SQL Server. He too stressed that the release date will be 2008 (so perhaps …
Microsoft opens kimono for 'Katmai' SQL Server
Microsoft's first-ever BI Conference kicked off this morning in Seattle with a keynote that promised the next version of SQL Server will hit the streets sometime next year.
Jeff Raikes, president of the Microsoft Business Division outlined/reiterated Microsoft's strategy for "delivering pervasive BI and performance management" …
Microsoft vs. Teradata
Microsoft and Teradata are both significant players in the BI market but they have wildly different approaches to the challenges of extracting information from data. The reason lies in the fact that the two companies elected to solve two very different, but equally intractable, computational problems in order to get their BI …
A tribute to Jim Gray
As we post this, Jim Gray has been missing for three months. It now seems very unlikely that he is still alive, although his family has not given up hope.
We decided to publish the following tribute to him, because his life deserves to be celebrated.
James 'Jim' Nicholas Gray (born 1944)
Photo of Jim Gray. of Microsoft …
Is Gartner's Magic Quadrant really magic?
In this series we're looking at the myths and legends of the database world; some are true, some false. The myth under the spotlight today is whether Gartner's Magic Quadrant really is magic.
(True, Gartner's Magic Quadrant (MQ) isn't just applied to databases, but its recent application to the area of BI platforms, for me at …
MySQL – is this database fit for the Enterprise?
MySQL has recently appeared as an Enterprise edition. We have already looked at whether MySQL (the company) is enterprise ready, but we can also ask whether the product itself is suitable for enterprise use.
Some Reg Dev readers clearly have strong views about this already.
First up, remember that Enterprises come in all kinds …
Don't Dodge this Viper
On the 6th Feb 2007 IBM announced Viper. Yes, we know that it announced Viper in July 2006, but this is a different Viper, it just happens to have the same name. Really, it all makes perfect sense; you just have to think like IBM…..
DB2, the company’s database engine, comes in three tangy, zesty flavours:
DB2 for z/OS ( …
MySQL is the company's SQL now...
Let’s face it; MySQL is a fabulous database engine. Not only is it free, it’s small, powerful and easy to drive. It also runs happily on free operating systems and so it can be used to create incredibly cost-effective database servers.
Of course, like all database engines, it polarizes those in the computing world. Some people …
OLAP and the need for SPEED
In this series we're looking at the myths and legends of the database world; some turn out to be true, others false. This myth is about why we use OLAP.
If you follow the Inmon model, you use a relational data warehouse for flexibility and OLAP cubes in the data marts for the speed. On the other hand, if you follow Kimball, you …
