No, Agile does not 'equal' DevOps: Examining complexity and the long haul
No one talks about the network
Two of the hottest buzzwords in the industry at the moment are DevOps and microservices – both are riding high on the hype cycle and we're starting to see gurus emerge telling us that they will save the IT industry (or at least save the industry a great deal of money).
One of the ideas of DevOps is that software deployment becomes so easy it can be done many times a day, almost without thought, removing one of the most stressful moments of the software cycle and hopefully making long deployment weekends a thing of the past.
Through automation deployment can be achieved through the press of a button, if you do that every day to systems very similar to the production environment, then an actual deployment to production is no big thing.
DevOps practitioners develop deployment pipelines to automate the entire process with carefully designed continuous feedback to ensure that only working software gets deployed to end systems. Press the button and the deployment starts, wait a short while and then you know if it has succeeded or not.
The problems start as the systems get bigger and more monolithic, the time from the button press to deployment can grow and feedback can slow down.
This is not only a philosophical problem, though, as DevOps aims to be fast and return fast feedback on a build for a good reason. Being only slightly more developed monkeys, we find it hard to keep track of what we are doing if there is an interruption or delay – which impacts our ability to correct problems when they go wrong. Removing these delays is paramount.
Enter microservices that are designed to replace large monolithic systems with much smaller systems that are very fine-grained (sometimes so small they do one job and one job really well).
The entire system is built of these small systems that communicate with each other using well-defined interfaces. The small systems themselves can be changed or replaced without changing the entire system ... indeed the small systems need not be written in the same language or use the same database technologies or run on the same operating system, as long as the interface remains within specification.
Teams developing individual microservices might well be completely autonomous, defining their own stack of software using the most appropriate technology, and that sound like an ideal for DevOps.
Your small teams now have a small stack of software to look after, so pressing the deploy button should never going to get to that lengthy process stage (and if it does, you need to split that service into smaller services).
Each of your microservices team should operate in a DevOps environment (or as Netflix has it, NoOps, as there is no operations) and each team's deployment will be separate from everybody else’s, and hopefully your automated testing will catch any changes that breaks the interfaces.
Suddenly the number of deployments you system can make will increase as each of your teams will be making many deployments a day. Problem solved, let's go home.
But hang on, haven’t we just added another layer of complexity? Haven’t we just broken our complexity down to the network? Worse, haven’t we just increased the amount of work the development of the pipeline needs?
Sure, each team should be responsible for their own pipeline (because they may be developing in different languages). That means each team will need deployment expertise.
Automated testing will also need to be more complicated, and the interface from the service absolutely cannot break or you risk breaking the whole system. What happens if your customer produces a requirement that needs a major change to the whole system, how are we going to synchronise the individual deployment systems (or can your teams automate that as well)?
Clever readers will have spotted the flaw in this plan: it assumes your monolithic system can be split up. There are classes of applications that can’t be easily broken out of the monolithic model, those that need very fast user interaction or fast storage of user data, think music or video creation systems.
Development can still benefit from DevOps processes, deployment to customers can still be sped up, new features added, bugs fixed and distribution through new purchase models. Deployment probably won’t be in terms of many a day, but it could be on a weekly or monthly schedule.
So if you’ve started down the DevOps path, microservices may be the next right move for you. But beware what you are getting into: more complicated interfaces, complicated integration tests, smaller teams that will need their own deployment scripts and most of all, an overall town plan to define borders and lines of communication.
The path is beginning to become a road, and the haul a long one. ®
Want to learn more about DevOps, Continuous Delivery, and Agile? Head to our Continuous Lifecycle Conference from May 3-5. Full details here.