So who exactly was to blame for Marketo losing its dotcom?
Company points finger at registrar; registrar points right back
Billion-dollar online marketing company Marketo had a bad week when it failed to renew its main dotcom domain name.
On Wednesday morning, marketo.com was moved by the organization's registrar, Network Solutions, to its "pending deletion" pile after it failed to pay the $35 required for another year's registration.
The result was not only the disappearance of the biz's website and email but also the instant loss of millions of tracked hyperlinks, forms and client reports that all rely on the internet address to work.
The company discovered the problem with a few hours and corrected it – although it took more than a day for the change to fully propagate across the 'net, resulting in enormous problems for Marketo's clients and the company itself.
But in among all that, the question remains: how did this actually happen?
Anyone who has ever bought a domain name will be familiar with the fact that every registrar gives you the option to auto-renew your domain when its registration runs out. In fact, most put it on by default because it saves both them and the customer enormous amounts of time and trouble, and, of course, brings in additional funds.
Even with auto-renew, registrars will email the named administrator of the domain prior to the expiration to let them know the registration is soon ending. In fact, they are contractually obliged to do so.
Registrars don't want to lose a domain registration so if auto-renewal isn't on, they will typically contact a customer several times before giving up and moving it to the deletion pile. Likewise, if auto-renewal is enabled but payment fails, the registrar will also email several times to let the customer know.
When it comes to big companies and important names, for an additional fee, all large registrars offer additional services such as having a named contact and refusing to make any domain name changes until someone has specifically spoken to that person. That service was largely created for security reasons following a spate of domain name thefts but it also serves as a point of contact that registrars used to try to upsell more services.
So what happened?
All of which points to the failure of Marketo to register its main domain name as extremely unusual.
We speculated that the failure to carry out such a simple but important task was down to the two most common reasons that domain names expire when the registrant didn't want them to:
- An out-of-date credit card, or
- An out-of-date admin email
But Marketo denied outright that an expired credit card was the reason. Maybe the corp's staffers failed to set the dotcom on auto-renew? No, not according to them.
In an email Marketo CEO Steve Lucas sent to customers once the issue had been resolved, he said that the domain was set on auto-renewal. "We renew thousands of domain name properties we own every year with precision," he wrote, "yet the auto renew process for registering our main domain, Marketo.com, failed. This catalyzed a cascading series of issues, but ultimately human and process error are to blame and again, we take full responsibility."
So, if it was set to auto-renew and the credit card was fine: why did the domain name not renew and cause two days of pain?
We went back to Marketo and asked again. This time we were told: "Someone who left Marketo had arranged for the domain to be auto-renewed but the system failed," adding that "the system did not auto renew as it claimed it would."
That implies that it was Network Solutions – and not Marketo – that was at fault. If so, you have to admire Steve Lucas' willingness to take the blame for the temporary collapse of his company's operations when it was, apparently, another organization's fault.
So back to Marketo a third time: was this Network Solutions fault? "The system failed due to Network Solutions' auto renewal," we were told.
Which, if true, is enough to give every company in America heart palpitations. Network Solutions remains one of the largest registrars in the world and looks after many of the most valuable and important address that exist online. If its systems can suffer from a glitch that allows auto-renewals to fail, then that is a huge, multi-billion-dollar problem.
And so, of course, we turned to Network Solutions, explaining that, according to Marketo, it was responsible for the failure of a critically important domain name to renew. Network Solutions promised a quick response.
When, the next day – Thursday – we had heard nothing back, we emailed again. And when the day after that still nothing had turned up, we called and told Network Solutions we were planning to write a story that said they may have a fundamental flaw in their registration systems affecting millions of domains.
They sent us the following response:
"We don't comment on individual accounts. However, it is our standard protocol with any customer as their account approaches renewal to make multiple attempts to contact them to remind them about their upcoming renewal and to assist them should they have any processing issues."
So Marketo is saying it had the domain set to auto-renewal but Network Solutions' systems failed. And Network Solutions is claiming that it tried multiple times to contact Marketo but the biz failed to get back.
And, according to Marketo, there was no issue with a valid credit card.
So if you have a solution to this puzzle for how a billion-dollar online company managed to lose its domain name, let us know in the comments below. We'll start asking questions again come Monday. ®
Sponsored: Becoming a Pragmatic Security Leader