Support and community
Evaluate the level of community backing your chosen CMS. The number of active developers in the community will directly influence the quality of the CMS, as they will be able to add features, fix bugs and provide some level of support. A good CMS should have valuable support channels like official forums, mailing lists, wikis and knowledge bases. In short, the bigger and more active the community, the better the CMS.
Users and clients
See who's using your chosen product. Some users have added their support to CMS communities through donations, which can help promote growth and maturity; others have contributed to development, testing, translations, themes and skin design, or simply promoted and spread the word.
If and when something does go wrong, often the fastest way to solve a problem is to check out the documentation rather than wait for a quick response from the community. Documentation - installation and upgrade guides, user manuals, and administration and developer manuals - should be so simple that non-technical staff should understand them, yet detailed enough that every step can be followed easily.
Age, simplicity and complexity
Maturity matters. Forget a CMS with a version number less than one, as this is a good sign of its stability and completeness. The CMS that's been on the market more than eight to ten years might also have problems, because it can become very big, and start to pack in too many features and add-ons for your needs. However, more features may be what you want - it just depends on your requirements: a simple website with just news, event and pages support versus a portal with membership, discussion board, photo gallery, blog, auction and payments.
Open source is opening up CMS, putting the ability to filter and serve information in the hands of more developers and organizations with big plans but tight budgets. Hopefully, this guide will have given you some helpful tips when it comes to choosing the open source CMS that suits your needs.
A specialist in Java, .NET and open source for distributed online development, Bayarsaikhan Volodya blogs frequently about his experiences while providing comparative analysis at www.plentyofcode.com
... Drupal is what I personally use - having made use of a variety of CMS systems including Joomla, Wordpress and others - and it's up there with the best.
Drupal is more flexible than you think.
Drupal gives you all the control you need.
The theme system gives you complete control over every aspect of presentation.
The software design is amazing modular consisting of an elegant system of hooks and API calls. Your modules can step in at any point and replace default behavior with its own functions without having to hack the core Drupal modules.
Granted to write a sohpisticated module you are going to have to be a competent PHP coder and maybe read a 200 page book.
Simply creating a theme is a bit easier, but still the learning curve might be higher than with other CMSs.
However I found that studying Drupal's inner workings to be rewarding for its own sake. The code is so well designed that I it was almost like reading a text book on how to apply modern software designed methodology to the PHP programming language.
I am not saying Drupal right for everyone, or even for most. People have a wide variety of needs which calls for a wide variety of of feature sets and characteristics.
I wanted a PHP based CMS with well written code, and a highly flexible modular design can be easily extended.
So far Drupal is the best I found.
evaluate whether there are xml / css templates????
> Also, evaluate whether there are XML and CSS templates available that allow you to change the skin of the site, for a different look and feel.
This is a content management system. A content management system is a system that manages content. It doesn't need to manage layout or function or interactivity because it manages content. If a content management system was going to manage these other things it wouldn't be called a content management system.
Why the rant?
I tried Joomla / Mambo, phpnuke, drupal. They are all interesting systems but they all had one failing, they dictated layout of code, they dictated layout of the site. It might be a good, sensible, well thought out usable layout but it's their layout. I may want to build something that doesn't fit into their rigid layout. I may want something that will spit out unstyled content between two tags and leave the rest of the site up to me to dictate. I may not want to have to mess with the source code (which is probably in a source-management system so if I upgrade it will get over-written) in order to use the system.
If you get a chance have a look at CMS Made Simple, www.cmsmadesimple.org. It supports a very sane separation of code, style and content, it is fairly light on the web server, the code is well written and maintained and it has a very active development community and good support. You can leave questions on their forum and people will actually answer them and their IRC channel has always had people there willing to help. Not bad for free. Crucially there are a lot of people using the system on live sites, in production.