Feeds

Gartner spells out magic behind quadrants

Blogger smackdown leads to reveal of research methods

Using blade systems to cut costs and sharpen efficiencies

Analyst group Gartner has detailed how it prepares its sometimes-controversial magic quadrants, revealing that a two-hour demo is sometimes part of the research process.

Gartner already offers a detailed explanation of how it compiles its Magic Quadrants here.

But in an exchange with governance, risk management and compliance consultant Michael Rasmussen, the firm has released more information about the spells it casts to make a Magic Quadrant.

The firm has done so in response to Rasmussen's complaints about the latest Magic Quadrant for Enterprise Governance, Risk and Compliance Platforms (paywalled). Rasmussen thinks it's a dud and blogged this strident opinion on the document:

My first impression is that it is best for the compost pile to be used as fertilizer for the garden next spring and not used in organizations that may rely on it to make misinformed GRC technology decisions.

The blog post lays out several other objections about the document, including:

Gartner has a script and gives a vendor a short time period to demo their GRC product to Gartner. They do not allow you to go off script – I have heard this from multiple vendors frustrated with the process. A vendor may have an absolutely amazing differentiator but if it is off script you have to kick and scream to get even passing attention. In other words, Gartner has their rigid view of the GRC capabilities of EGRC vendors and if you approach it differently then you are outside their myopic vision.

In a blogged response to Rasmussen's post, Gartner Fellow and vice president French Caldwell says he considers Rasmussen's complaint valid, an offers this description of the Magic Quadrant research process:

“For two hours of one of 365 days in the year, we hold the vendors to a demonstration script.  The vendors have the other 364 days of the year to communicate to us all the breadth and depth of the capabilities that they have and those are considered in the magic quadrant. A vendor does not have to be a Gartner client to do a vendor briefing.  They talk, they show us what they do, and we listen to them the  rest of the year — but for two hours of one day, they follow our script.  Michael, is that really unreasonable?

When establishing a baseline of capabilities for evaluation, we compare apples to apples.  But the demo script is fairly open — any vendor who can’t find a place in it to demonstrate their best differentiators is slipping up.”

The post goes on to say that Gartner will allow its clients to access and adapt the scripts and that the two-hour session is not the only research that goes into a Magic Quadrant.

Caldwell also says the firm gives “vendors plenty of time afterwards to demonstrate additional capabilities that were not in the script”.

Ian Bertram, Gartner's head of research for Asia/Pacific and global manager of the firm's Analytics and Business Intelligence Research team, told The Reg Caldwell's blog was largely accurate.

Bertam said the term “'demonstration script' is not the right phrase” and explained that demonstrations are not a compulsory or universal part of the research process to create a Magic Quadrant.

“For the Business Intelligence Magic Quadrant that my team manages we send a survey. How vendors want to respond to that is up to them,” he said. “They can do a demonstration, or they can do it in some sort of script format.”

Demonstrations aren't required, and indeed in some years Gartner's teams don't request them as they feel insufficient change in a market makes them superfluous.

Bertram added that research methods can be flexible, so long as each team is consistent in the way it assesses the vendors it considers. The net is cast wide for participants, with a “reference survey” sent to 1400 vendors for the business intelligence Quadrant. Bertram said that's the largest such survey the firm conducts.

Not every respondent ends up in the magic quadrant, as the survey weeds out some who don't belong. But Bertram said the exercise is not futile, with the information gathered by the survey informing numerous other reports and conversations analysts create or conduct. ®

The Essential Guide to IT Transformation

More from The Register

next story
BBC goes offline in MASSIVE COCKUP: Stephen Fry partly muzzled
Auntie tight-lipped as major outage rolls on
iPad? More like iFAD: We reveal why Apple fell into IBM's arms
But never fear fanbois, you're still lapping up iPhones, Macs
White? Male? You work in tech? Let us guess ... Twitter? We KNEW it!
Grim diversity numbers dumped alongside Facebook earnings
Bose says today is F*** With Dre Day: Beats sued in patent battle
Music gear giant seeks some of that sweet, sweet Apple pie
Amazon Reveals One Weird Trick: A Loss On Almost $20bn In Sales
Investors really hate it: Share price plunge as growth SLOWS in key AWS division
Dude, you're getting a Dell – with BITCOIN: IT giant slurps cryptocash
1. Buy PC with Bitcoin. 2. Mine more coins. 3. Goto step 1
There's NOTHING on TV in Europe – American video DOMINATES
Even France's mega subsidies don't stop US content onslaught
You! Pirate! Stop pirating, or we shall admonish you politely. Repeatedly, if necessary
And we shall go about telling people you smell. No, not really
Too many IT conferences to cover? MICROSOFT to the RESCUE!
Yet more word of cuts emerges from Redmond
prev story

Whitepapers

Top three mobile application threats
Prevent sensitive data leakage over insecure channels or stolen mobile devices.
Implementing global e-invoicing with guaranteed legal certainty
Explaining the role local tax compliance plays in successful supply chain management and e-business and how leading global brands are addressing this.
Boost IT visibility and business value
How building a great service catalog relieves pressure points and demonstrates the value of IT service management.
Designing a Defense for Mobile Applications
Learn about the various considerations for defending mobile applications - from the application architecture itself to the myriad testing technologies.
Build a business case: developing custom apps
Learn how to maximize the value of custom applications by accelerating and simplifying their development.