Original URL: https://www.theregister.com/2010/07/21/drupal_7_changes/
Drupal looks beyond open source zealots
Trade-offs, cocoons, and broken APIs
Posted in Channel, 21st July 2010 17:31 GMT
Every few years, Drupal violates one of the industry's most sacred rules: don't break your APIs.
The next version of the popular open source content management system – Drupal 7 – will do just that. And more. It will offer a redesigned user interface that targets – gasp! – non-technical users. It will hide features that devs know and love.
Also in the works: Drupal Gardens, for hosting sites built with Drupal 7. It's analygous to Wordpress.com, and it's set to enter public beta "very shortly".
Such changes come at a critical juncture for Drupal. They're intended to help increase uptake among content creators – those outside Drupal's core audience of open sourcers.
After 10 years, Drupal runs about one per cent of all sites on the net, according to estimates, a stat that gave delegates at this April's DrupalCon in San Francisco, California a sense of satisfaction. Drupal has built a niche powering micro sites for big corporations, such as Britney.com and Justintimberlake.com from Sony, and it scored a high profile win with Whitehouse.gov.
But Drupal creator Dries Buytaert says it must do better. Drupal still isn't as popular as WordPress (8.5 per cent of websites) or Joomla (just over one per cent). And while Drupal has strong name recognition at developer events, it's WordPress that has successfully crossed into the consciousness of mainstream bloggers and content creators – people without a background in PHP programming.
"We are doing really well. We are growing rapidly and have a lot of big wins like Whitehouse.gov... but at the same time, I like to paint the picture of where we could go," Buytaert told The Reg during a recent interview.
"Drupal 7 will take a very big step forward in terms of usability. We made it the number-one priority...We made a big transformation where Drupal used to be a tool for developers by developers for developers. We are starting to switch to make Drupal...much more optimized for the end users versus the person building the site."
Drupal 7 has benefited from weekly end-user usability sessions. For the first time, developers have watched real people from behind two-way mirrors in usability labs as they interacted with the system. A design expert - Mark Boulton, who's helped re-design the BBC's site - has also been hired to work on the Drupal 7 UX and Drupal.org.
The results: there will be one-click access to features like creating posts and moderating content, while developer features will need a few more clicks to be uncovered, Buytaert said.
Big - the new small
There's one problem with going mass market. Buytaert recognizes that for real growth – and we presume money too, given that Buytaert is chief technology officer and co-founder at Drupal software and services company Acquia – Drupal needs wider uptake among paying enterprise customers like Sony. Therefore, Drupal 7 will introduce a database abstraction layer, out-of-the box, so that big sites can use Drupal with multiple databases. This will mean a performance hit for smaller sites – a calculated gamble for growth.
"I think we made the right trade-offs. If we want Drupal to grow it has to grow in the enterprise," Buytaert said. "While there's a small performance hit for the smaller users, it's worth that penalty because small sites can grow big over time – you don't want people to migrate off the platform over time."
Drupal Gardens also gets enterprise attention. One of Gardens' first enterprise features will be Site Duplication – recently added to the private beta – for a site owner to clone an existing site's design, functionality, information architecture, and content onto a new site.
The desire to straddle both small and big users suggests there's a dynamic tension inside the Drupal camp. Buytaert admits those participating and adding features in the past haven't necessarily been representative of the typical Drupal user.
People manning Drupal's issue queues and driving feature requests and bug reports actually come from bigger sites because their devs can take the time to contribute. Drupal users running smaller sites typically don't have time to participate so download and go away.
White-House win blackened
"Sometimes we have to make trade-offs – we go for big sites, and make a compromise that effects the smaller sites, and when we do, we have to be careful. Right now, there's a bias towards the bigger sites because of the core development team," Buytaert said.
Security has been another source of tension. Three weeks after Druplers crowed that Whitehouse.gov gave them credibility by releasing new open modules, a potentially serious XSS hole was spotted in the Drupal Context module used by the Whitehouse.gov and 10,000 other sites.
XSS vulns are not uncommon in web software, but the discovery touched off a disagreement between the security researcher who'd documented this and other vulns, Justin Klein Keane, and Drupal's security team. Klein Keane complained that his report went unanswered for ages and that Drupal's security team didn't make it clear who researchers should contact or how such are incidents handled. Drupal's security team consists of 32 individuals, including Buytaert.
The result was that Drupal's security team announced that it could only work to fix Drupal modules considered finished and that release-candidate modules won't get fixed. The Whitehouse.gov module had been a release candidate piece of software – but the White House installed it anyway.
Klein Keane, the senior information security specialist at the University of Pennsylvania and a Drupal user, told The Reg in reaction to the White-House-inspired changes that there exists "tremendous opportunity" to improve the situation – though he was generally complimentary of the project.
"This is one of their biggest challenges to becoming a widely used piece of enterprise software," he told us.
"I'd really love to see them encourage more participation by security researchers - they have a tremendous opportunity here and they are making it difficult and painful," he said.
Buytaert said in response that in addition to clarifying what code Drupal will and won't fix, the security team's polices and workflows have been improved as a result of the White House incident. There was no word on opening the process to external security researchers.
Another hurdle to Drupal's uptake is reliable delivery. Like most open source projects, Drupal's timely completion is hostage to participation. At DrupalCon, the goal was for Drupal 7 to arrive by June. Code was frozen in September 2009, and 114 bugs still stood in the way of a shipping product. Completion has now slipped to summer/fall – possibly a year after code was finished.
Illustrating the participation problem: half of the patches in Drupal 7 by April came from just 25 contributors.
Buytaert acknowledged the painful process of completing Drupal in a recent blog here. "Seeing Drupal 7 getting steadily closer to its release, is like watching a cocoon grow into a butterfly," he wrote. "The inevitable results are going to be spectacular."
Spectacular they may be, but about those breaking changes in Drupal 7?
Users implementing the core Drupal modules will find the move to the latest version easy. But who doesn't modify? Things will be trickier if you've custom coded so it'll come down to manual coding. Sometimes, the changes will be minor, like re-naming a function or adding a new argument, Buytaert tried to reassure us. He also stressed peoples' data won't get lost.
"People will have to update their codes," Buytaert said. "That's going to piss people off, but it also makes developers really happy because it means we have no legacy and we have really clean and consistent APIs."
Does this thing go to 8?
And with Drupal 7 crawling towards completion, Buytaert's already looking towards changes with Drupal 8. To crack the enterprise, Buytaert believes configuration management and staging need more work on usability. They are not as easy in Drupal 7 as they could be. "That will require some API changes across the board," he cautioned.
Also up for discussion is whether Drupal and Drupal Gardens should include more tools out-of-the-box such as in traffic monitoring – something competitor Wordpress provides.
What ever the pain these or other changes involve, Buytaert reckons Drupal needs to become even more focused on the needs of the end user.
"I do think that one of the things that has been changing is to have a culture of user testing. We have to go back and observe what the pain points are - the existing UIs and the features that people want," he told us.
Expect more broken APIs. ®