Original URL: https://www.theregister.com/2008/03/18/when_google_does_evil/
Google's riches rely on ads, algorithms, and worldwide confusion
'Please ignore the cash machine behind the curtain'
Posted in Channel, 18th March 2008 18:44 GMT
Special Feature Why did millions of money-making post-holiday clicks suddenly disappear from the world's largest search engine? It depends on who you ask. Some people blame a soft economy, insisting that Google is well on its way to piddling quarterly revenues. But Google disagrees. CEO Eric Schmidt and company claim that they planned the whole thing - that this dip in paid clicks indicates that both Google and its advertisers are richer than ever.
We'd like to tell you who's right and who's wrong. But that's impossible. Google's ad system is a riddle wrapped in a mystery inside an enigma. That's how Google likes it. If you can't see inside Google's ad system, Google has the power to make more money. When the company says it's richer than ever before, it may be telling the truth. But it may be fudging the truth when it comes to advertisers - especially the small guys.
On February 25, the online researchers at comScore unveiled a new report detailing Google's "paid click rate." According to the report, already the stuff of legend, seven per cent fewer people clicked on Google advertisements from December to January.
Web pundits everywhere were sure that Google had fallen victim to an anemic economy, and the company's stock price took the requisite tumble, dropping almost 10 per cent over the next two days. Then comScore "clarified" its report, presenting the vanishing clicks as part of Google's master plan to improve the "quality" of ads and therefore increase revenue.
Naturally, Google backed them up, telling everyone it's on a mission to eliminate "unintentional clicks." "Conversions actually go up for advertisers, which is positive, but there are less clicks overall," Tim Armstrong, Google's president of North America advertising and commerce, told MarketWatch.
Believe it or not, comScore's Google-placating argument is at least partially true. In all likelihood, Google's first quarter numbers will look just fine. The company has far more control over revenues than the world realizes.
The Google Enigma
The question is whether Google has too much control. The truth is that Google's commitment to quality ads goes only so far. Google isn't above boosting revenues at the expense of advertisers - particularly the small and inexperienced kind. The company can afford to do so. It controls almost 60 per cent of the search market.
Automatic for the People
Late last month, just as comScore's report hit the web, Google sent an email to at least a few members of its flagship adverting program, AdWords. With AdWords, online businesses can arrange for their very own ads to appear in response to keyword searches. The program is billed as "an auction." If a business bids high enough, its ad will appear each time someone searches on a particular term, or collection of terms. And each time someone clicks on the ad, the business pays Google somewhere south of its bid - until the campaign reaches a daily budget.
Of course, there are days when a campaign doesn't reach its daily budget, when an ad receives fewer clicks than normal. And that's where Google's late February email comes into play. The email notified certain AdWords users that Google was tweaking their ad campaigns to include something called "automatic matching." With this AdWords "feature," Google automatically grabs an advertiser's excess budget and uses it to post ads against keywords other than those the advertiser is actually bidding on.
"Automatic Matching automatically extends your campaign's reach by using surplus budget to serve your ads on relevant search queries that are not already triggered by your keyword lists," Google's email reads. "For example, if you sold Adidas shoes on your website, Automatic Matching would automatically crawl your landing page and target your campaigns to queries such as 'shoes,' 'adidas,' 'athletic,' etc., and less obvious ones such as 'slippers' that our system has determined will benefit you and likely lead to a conversion on your site."
In other words, Google will automatically take the money an advertiser isn't spending and spend it on searches the advertiser hasn't explicitly approved. For the average advertiser, this is not a good idea.
A Click is Enough
"They're offering you the exciting opportunity to bleed every penny of your budget every day, advertising against keywords that you didn't want to bid on," says Dan Thies, a Frisco, Texas-based keyword guru who makes a living counseling search engine advertisers. "Sure, if I sell Adidas shoes, why wouldn't I want to get some traffic from people who searched for slippers? I mean, it's not like I'm trying to turn a profit or anything, right?" If you couldn't tell, that last bit is sarcasm.
On the other hand, if you're Google, "automatic matching" is a wonderful idea. It's a sure way to boost the bottom line. There are cases where Google only gets paid if an ad leads to a sale or some other customer "conversion" - the company recently introduced a "pay per action" program - but typically, a click is enough.
Yes, "automatic matching" will turn out well for some advertisers. Increased clicks will lead to increased conversions. But it won't turn out well for all.
Google Simplifies AdWords
“If you're a new advertiser, either you fail or Google makes a bunch of money off you.”
The trouble is that Google seems to include users in this new program without their approval. According to Thies, a Google support representative told one of his customers that the onus is on the user to turn the program off. And most users don't know any better.
To be fair, Google bills "automatic matching" as a beta. And Google is certainly committed to keeping "unintentional clicks" down. But the company bills almost everything it does as a beta. And there's no arguing it's interested in boosting its own revenue.
"Automatic matching" is just the latest in a long list of ways the company misleads advertisers who don't quite understand the way the system works. And it has to be said: An awful lot of Google customers don't understand the way the system works.
"Even for experienced advertisers, setting up an AdWords account is a real challenge," says Andrew Goodman, the Toronto-based paid search consultant who wrote the book on AdWords. "If you're a new advertiser, either you fail or Google makes a bunch of money off you."
What Google Doesn't Want You to Know
Only Google understands AdWords entirely. And that includes the self-proclaimed experts.
For one thing, AdWords is epically complicated - as you'd expect from a system that juggles bids from more than a million advertisers across the globe. But even if you're blessed with a brain hardwired for search engine marketing, you can't know all the ins and outs. Google won't divulge the system's core algorithms.
There's a good reason for this. If Google exposed everything, the experts say, advertisers could turn the system itself into a money maker - and ad relevance would go out the window. "If Google was completely transparent, the bad guys in my industry would game the system," says David Szetela, owner and CEO of the online ad consultant Clix Marketing. "And that would lead to irrelevant ad results for the person doing a search."
The end result is that inexperienced advertisers have no hope of grasping what's what - or even coming close. Naturally, when these advertisers sign up for the system, Google gives them an interface that "simplifies" much of that epic complexity. But at the same time, Google stacks the deck in its own favor. This may or may not be justified. But it's true.
"Hiding the complexities of AdWords makes Google the most money. There's the potential for a conflict of interest."
"You're kind of at the mercy of Google," says Adam Audette, founder of AudetteMedia, a boutique search marketing shop out of Bend, Oregon. "They make everything as intuitive and as easy as possible right out of the gate - and that's a nice bonus for people who don't know have a lot of experience with Adwords. But on the flip side, hiding the complexities of AdWords makes Google the most money. There's at least the potential for a conflict of interest."
By default, AdWords saddles users with several features that behave a lot like the new "automatic match." They serve Google's needs without necessarily serving the advertiser's. What's more, in fashioning algorithms that weed out so-called "bad actors," Google may force legitimate users into bidding ridiculously high amounts just to participate in the AdWords auction.
"I've got a whole staff of Cal Tech and Rice and Baylor engineers who sit here and do nothing but track what Google is doing," says Erick Herring, vice president of engineering at Adapt, a Pasadena, California company that offers a online service for confused AdWords users. "What is the poor end marketer to do? Many only do what Google wants them to do, which is tell Google what their site is about, give Google money, and hope they don't go out of business before the clicks start converting."
The Broad Jump
"Automatic matching" is just an extension of another AdWords feature known as "broad matching." Broad matching works in exactly the same way, except it doesn't target accounts with excess budget. It targets just about everyone.
Google CEO Eric Schmidt
Ordinarily, Google's platform offers three ways of matching ads to keywords: exact match, phrase match, and broad match. With an exact match, an ad won't appear unless you type in a keyword character-for-character - and nothing else. Phrase match also requires character-for-character entry, but the ad will still appear if you tack on a few extra words.
Meanwhile, with a broad match, Google will serve ads against keywords it decides are somehow related to the terms an advertiser is bidding on. "Google really takes liberties with broad match. If you're bidding on 'dog gifts,' you may show up for 'hot dog buns,'" says Adam Audette.
AdWords account records from Mulesource, the San Francisco-based open source outfit that has spent close to $90,000 on the ad system since November 2006, show the unpredictability of broad matches. When the company bids on a word like "mule," Google may broad match on "muele," "mula," "mula spain," "mulapelada," "riding mules for sale," "trainer mules," and "yamaha mules." And the list goes on.
"With broad match, the Google AdWords system automatically runs your ads on relevant variations of your keywords, even if these terms aren't in your keyword lists," Google told us. "Keyword variations can include synonyms, singular/plural forms, relevant variants of your keywords, and phrases containing your keywords.
"For example, if you're currently running ads on the broad-matched keyword web hosting, your ads may show for the search queries web hosting company or webhost. The keyword variations that are allowed to trigger your ads will change over time, as the AdWords system continually monitors your keyword quality and performance factors. Your ads will only continue showing on the highest-performing and most relevant keyword variations."
It wasn't always this way. Google seemed to overhaul its broad match back in July of last year. "In the past, a broad match was when someone typed any of your keywords - in any order - with any other keywords, but the keyword you were bidding on had to show up in the query," explains Don DeVange, a search engine consultant who blogs on SEM issues at a site called Serpzone.com. "But then Google started expanding that to other 'relative' terms. So, there are cases where this 'expanded' or 'advanced' broad match trigger ads for terms you don't want to be bidding on."
And broad matching is turned on by default.
Yes, advertisers can turn it off, adding exact and phrase match qualifiers to the words they're bidding on. And when used in tandem with something called "embedded matching" - a way of determining which keywords are actually driving conversions - broad matching can be a very good thing. But so many advertisers don't even realize that broad matching exists. And if they do, they're given little help with the various ins and outs behind the system.
"Google can't get paid if someone does a search and there's not an ad in place that a person can click on."
When Mulesource CEO Dave Rosenberg first started using AdWords, he was unaware of broad match. And even now, nearly a year and a half later, he doesn't understand it. "Google does give an explanation for broad match," he told us. "But I don't get it." And he doesn't use embedded match.
"Embedded match is a way to refine what keyword matching works, but it's really time consuming, and it's not well documented. In fact, some experts call it the least documented functionality on AdWords," Audette explains. "And unless you use embedded match correctly, you're going to be wasting money. You'll be showing up on all sorts of keyword matches you don't want to show up on. And you won't necessarily know."
But there's no doubt that broad matching suits Google. In a big way. It puts more ads on more pages. And that makes Google more money. "If you're Google, the way to expand revenue is to expand 'coverage.' Google can't get paid if someone does a search and there's not an ad in place that a person can click on," says Adapt's Erick Herring. "Google has to have a system that encourages a diversity of advertising, so that there are ads in more places. One way to do that is to use various forms of broad matching."
AdNonSense
The same logic applies to the Google "content network," a collection of third-party websites that ties into the AdWords program. In addition to serving ads onto its own search site, Google serves them onto other sites, offering those sites a cut of the AdWords pie.
Google co-founder Sergey Brin
In this case, ads aren't matched to keywords. They're matched to a site's content, and ultimately, it's Google that decides what's a match and what's not.
Many advertisers have great success on the content side. There are valid reasons to drive ad traffic through third-party sites. But it's a very different game than pure search advertising. People browsing content sites aren't actively searching for anything. They're just reading. This means that click-through-rates and conversion rates are often much lower. And since Google has complete control over matches, it's even harder to control where ads show up and where they don't.
Nonetheless, when advertisers sign up for AdWords, they're automatically included in the content network. And again, many don't realize it. They think they're bidding for nothing but keywords, when in fact they're bidding for placement on third-party content sites as well.
"All new search campaigns are automatically run on the Google content network unless an advertiser opts out," Google told us. "Contextual targeting is an extension of search advertising. Just as users can search for keywords on Google and see ads related to those keywords, visitors to content network sites can see ads related specifically to the content they are viewing on a web page."
Using something called "smart pricing," Google automatically tweaks bids for use on the content network. But once again, this doesn't always work in the advertiser's favor. "Google says that smart pricing will figure it all out, taking your search bids and turning them into content bids," says Aaron Wall, another well-known search marketing consultant. "But if you don't take the time to separate those bids, a lot of times you're going to get hosed."
"I do feel like Google duped me into the content network."
Chances are, an advertiser will inadvertently serve ads onto sites that have nothing to do with their ads. And by the time they realize this is going on - sometimes months down the road - they've already burned some serious cash. Six months passed before MuleSource's Rosenberg discovered that a majority of his ads were being served onto content sites, and a good portion of these sites clearly had nothing to do with his product.
In fact, many of the sites were obvious domain parkers, sites that do nothing but make money from ads. "You could tell just from their names," Rosenberg told us. These domain names included "1000jogos.com," "23qqw.cn," and "193.164.132.164."
"I do feel like Google duped me into the content network," Rosenberg, who spent close to $7,500 a month before discovering his ads were appearing on the content network, said. "You can get completely screwed - unless you go in and exclude irrelevant sites manually for every campaign you run." And of course, if you don't make such exclusions, Google reaps the benefits.
The Anti-eBay
But let's say an advertiser knows his stuff. Let's say he sidesteps the pitfalls of the content network, broad matches, and other "features" Google automatically turns on. This advertiser still faces another hurdle - one that even the cleverest marketers can't overcome without forking over some serious dough.
The closed doors of the Googleplex
AdWords isn't like eBay, where the highest bidder always wins. With the AdWords auction, each bid is multiplied by something called a "quality score" - which varies from advertiser to advertiser, and from keyword to keyword - and it's this calculation that determines who places high on the page and who places low. On each keyword search, the auction winner gets the top ad spot on the page. The second place finisher gets the next spot. And so on. Naturally, the higher an ad is placed, the more likely it is to get a click.
The rub is that Google won't tell the world what goes into "quality score." "Quality Score," the company told us, "is determined by your keyword's click-through rate (CTR) on Google, relevance of ad text, historical keyword performance, the quality of your ad's landing page, and other relevancy factors."
The "other relevancy factors" bit means that Google reserves the right to do whatever it wants. This is how the company weeds out so-called bad actors. But it also means that many small advertisers are forced to pay a premium just to get into the game.
On any given keyword auction, Google also gives every advertiser a separate "minimum bid." For one advertiser, the minimum bid might be 30 cents. For another, it might be $5. And this minimum bid is a function of quality score.
An advertiser can't even join the auction unless they bid their minimum. And even if their minimum is $5 higher than the competition's, they can still drop to the bottom of the page - or off the page entirely and onto the next.
That may sound like a paradox - but it's the way things work.
A Google ad
The Score on Quality Score
New advertisers may start out with very poor quality scores, so they're often forced to pay through the nose just to get into the game. If their quality is poor, their minimum bid is high. And if they want to improve their quality score, they have no choice but to fork over that minimum bid and see what happens.
For some, improving quality score is a Sisyphean task. Remember: quality score is tied to click through rate. If you're low in the rankings, your click through rarely tends to drop, which takes you further down the rankings. And so on.
Again, this is a problem that typically hits small advertisers. The big players - companies likely to spend large amounts of money anyway - typically get help with their quality score. "Large advertisers and large accounts get preferential treatment, just because of the way the system works," Audette says. "If I'm Mercedes Benz, I'm going to get great treatment from the beginning. I'm going to get dedicated account reps, hand-holding through the entire process, and much higher quality scores - right out of the gate.
"But if I'm a small account, say a Mercedes Benz dealership in LA or something, I won't necessarily get an account rep right from moment one. I'll get quality scores that won't default automatically to a green light in terms of getting good placement. I'll have to work my way through the system and know how to play the game. And if I don't know that, then I may not have a favorable experience."
Audette points out that all advertisers are entitled to a free account "optimization." But he also says that most are unaware of this.
"Google can obviously earn a lot more revenue if they convince advertisers that there's something wrong with them and they need to bid large amounts of money per click."
Yes, Google has good reasons for putting this minimum bid in place, but it's still a plus for the company's revenues. It not only allows Google to scrape extra revenue from inexperienced advertisers. It allows Google to control who has access to the market and who doesn't.
"We can understand the reasons for bad quality scores and high minimum bids in certain situations," says Andrew Goodman. "Obviously, there might be cases where you deserve a low quality score because you have low click through rates. Or you're guilty of web site violations of some sort.
"But Google can obviously earn a lot more revenue if they convince advertisers that there's something wrong with them and they need to bid large amounts of money per click. And there may be cases where they're cracking down on business models that conflict with Google." In other words, if Google thinks someone is competing with them, they can cut them out of the equation.
Turning the Knob
Behind the AdWords curtain, Google has the power to do just about whatever it wants. Which begs the question, How far does the company actually go?
Naturally, Google refuses to answer this. We first approached the company about an Adwords interview in early January. Two months and many requests later, we're still waiting - though the company did provide answers to a long list of questions over email.
Google always stops short of answering the biggest questions. After all, keeping AdWords a mystery suits the company just fine. During the fourth quarter of 2007, Google-owned sites raked in $3.12bn in revenue, and revenue from partner sites topped $1.6bn.
And the questions Google refuses to answer are far bigger than the world realizes. Considering Google's use of clever tricks like quality score and minimum bid, you might ask, is the AdWords auction is really an auction? But more on that later.
Yes, Google is interested in keeping ads relevant. But it's also interested in making large amounts of money. It's a business like any other - a publicly traded business under enormous pressure to grow revenue from quarter-to-quarter, from year-to-year. And though AdWords is designed to keep ads relevant, this design also gives Google tremendous latitude to adjust advertiser spending.
Erick Herring's company, Adapt, is run by a man named Michael Harris, who was once a senior VP at Overture, the company that pioneered search engine marketing and was eventually purchased by Yahoo!.
"We know for a fact - because we know what happened at Overture - that when a quarter runs short, you turn the knob and more money comes in," Herring says. "That happens all the time at Overture, and I'm sure it happens at Google. Why wouldn't it? Like you said, it's a publicly traded company." ®