What is DevOps?
DevOps is a relatively new term in the Agile world, its often not understood correctly or misinterpreted, hopefully this post can help you understand the fundamentals of this concept. Is it a culture? Is it a job title? Is it a way of organising a team? Is it a way of thinking? Is it a fancy term for something that already exists?
Teams who adopt agile are always evolving and trying new ways to find new ways to adapt and improve, so there are always new trends and methodologies that come out of these teams.
I would define DevOps as the following.
In its most basic term, its the merger of the following:
The development part is covered by the coders, the guys that build the functionality. Testing is covered by a QA team, who test the functionality and ensure quality. Finally, operations is covered by the IT operations team who manage releases, deployments and coordinating deployments. As a Venn Diagram it would look something like this:
Why use DevOps?
As agile teams where improving interpreting and developing new functionality, it became clear that there was still a bottle neck in some organisations. The IT operations or service delivery team or whatever you want to call it were not able to keep up – and still being technically a different team, communication and shared understanding was not often there.
DevOps brings these teams together, just as in Agile we do with the developers, QA and product owners; we now incorporate more roles into the team such as a SBA, SysAdmin or Middleware to improve the communication, collaboration and integration through the full software delivery lifecycle. Now, rather than having two separate teams with a hand over, this part if the process is integrated into your team. Using DevOps allows you to enforce the dependance on development and operations and ensure rapid delivery is actually there.
A DevOps team might look something like this:
Probably around 2009, this was a hot topic of discussion in many conferences and social media, teams could deliver quickly through agile, but the last step was often an issue. Naturally agile teams and IT operations teams when discussing their issues came up with the DevOps movement. Even though the introduction of the agile methodology allowed software development to make a massive step forward in efficiency, the delivery method was still very ‘waterfall’. As teams started to move into continuous delivery; this was the obvious bottle neck or last hurdle from achieving this. Adapting DevOps allowed these teams to become fully cross functional, share responsibility and also incorporate shared goals, DevOps encourages automation of the change, configuration and release processes.
So who is using DevOps?
Well who you might expect:
Google, Amazon, Twitter, Facebook, Netflix and Etsy are known to do deployments multiple times a day. In order to deploy that often, you have to know 100% you’re not going to break what’s already working. DevOps helps to ensure frequent deployments can be made with a low failure rate. But its not just these companies, smaller companies are using it too. DevOps is especially big with companies pushing cloud storage. Below is a little information on how these companies are using Dev Ops:
Amazon ran into an issue when they were running on dedicated servers, it struggled to meet traffic demands and estimate when they would see spikes in tragic. Its documented that about 40% of Amazons server capacity was wasted. The company then moved to Amazon Web Services (AWS) cloud, which allowed them to scale up or down incrementally, whenever they needed. Not only did the company save money, but it allowed them to move to continuous deployment; meaning any developer had the capacity to deploy their own code to any server, at any time. This move also reduced outages and increased revenue
Within a year the developers were deploying code every 11.7 seconds, on average. .
There is a good article here on Amazons move to DevOps: http://servicevirtualization.com/how-amazon-made-the-leap-to-a-devops-culture/
You might not remember, Netflix’s original business model was shipping out DVDs, which it eventually evolved to the streaming service we know and love today. At the time it did this, there weren’t any enterprise tools which would allow them to smoothly stream video, and keep its cloud infrastructure in tact. It had to turn to its own and open source solutions, which evolved into something called the Simian Army. This is a set of tools that stress test Netflix’s infrastructure and identify vulnerabilities before they become a real issue. You can find these here: https://github.com/Netflix/SimianArmy
Netflix embraces automation and open source, and its developers deploy thousands of times a day.
Facebook, not just as an industry leader in social media, but also software development. Some the the techniques implemented by Facebook helped to reinvent the way we build software. Some of the staple points in agile where installed into Facebook’s culture from the beginning such as responsibility and ownership of code, iterative development and change management, automation and continuous delivery.
These values allow Facebook to adapt and continuously improve, speeding up the software development lifecycle, you might have noticed that you get a notification now to update your Facebook applications every two weeks now….
Etsy, in its early days had a lot of problems with slow, painful updates that made the site often go down for long periods of time. This obviously impacted customers massively, thus making revenues decline. Obviously Etsy had to do something about it, so they integrated a new tech team management system which allowed it to move from its slow, inefficient waterfall methodology to a more agile one.
The company now has a full automated deployment pipeline, and its continuous delivery process has more than 50 deployments a day.
A good article on how Easy transitioned to DevOps can be found here: http://www.networkworld.com/article/2886672/software/how-etsy-makes-devops-work.html
For most of us who use Adobe software, many years ago will remember the packaged CD-ROM you had to buy. Over time, the company moved away from this to a cloud infrastructure, and rather than big annual software updates to smaller more frequent updates.
Adobe uses CloudMunch’s end-to-end DevOps platform to automate deployments, because its very central and integrates with other software – users can easily see the impacts it has any any other software they are using. This move obviously enabled faster delivery, meaning it was easier to maintain its software and keep users using it for longer.
LinkedIn started to user deployment automation all the way back from its beginnings, to manage releasing hundreds of applications/services on a 1000+ node environment. Now they are cultivating the worlds DevOps communities.
This is a good article on the tools they use: https://devops.com/deployment-and-monitoring-automation-glu/
Moving to DevOps
A lot of development teams will go through a long, tedious and methodical process to deliver a project, and by the time the software is ready; have already amassed a list of changes and fixes to apply to the next major release.
The downsides to this are obvious; and since the mobile and smartphone revolution this has completely changed the way we use software. Companies like the ones above have reengineared user interfaces, and how we navigate and how quickly users want these features. Because of this shift, companies needed to meet demand and ultimately alter how software is developed and deployed. Think about it, every time you load up your App Store, you are expecting there to be updates to your applications, new features and functionality to enhance how you currently use the application now. Moving to DevOps allows any company to keep up with the demand of their user bases expectations, and at the same time maintain scalability and performance.
Lets put it into more perspective, the average time that an mobile application stays in the top 10 list of an App Store is one month. In one month your application that was topping the list of most downloaded applications is no longer on the initial page of apps in the chart. I read a great description online (sorry can’t remember the source), but they compared it to like driving a car. When you are learning everything is a sequence of events, in a methodical fashion – fasten your seatbelt, check the mirrors, start the car, hands on the steering wheel in the correct position. Even a small process like turning a corner takes some small tasks to complete. This would be compared to the old development methodologies, where as DevOps would be compared to the natural way you drive after you have been doing it for a long time, everything happens simultaneously and by intuition. You can get to where you want to go without much conscious thought.
In summary, because of the way the market is now, especially the mobile market – companies need to move past the slow, methodical process of development and deployment and use DevOps to deliver applications quicker, whilst still maintaining a stable, high quality product.
According to data from the US Small Business Administration (https://www.sba.gov/sites/default/files/FAQ_Sept_2012.pdf). Only 16% of companies starting out today will make it past a generation, so for smaller companies or companies who only develop mobile applications, they don’t really have a choice than to have in place a methodology to deliver quickly. If not, a company is almost guaranteed to fail, an app can be released with nice new functionality and almost instantly be copied by a competitor and released at a higher quality. How many people are using WhatsApp now rather than Viber?
I have mentioned mobile a lot, but its not just restricted to that market, any company that needs develops software should be adhering to these principles.
How to Move?
In order to innovate quickly, maintain profitability and deliver high quality, stable applications in the modern day 3 things are required:
- Cloud Architecture
- Test Automation
Dev Ops is not software, a tool or a service – the only way a company can adopt DevOps is to have a culture change. How teams work together, and how you get from idea to production in the quickest most effective way.
This is a very long process, and there is no silver bullet. It takes a lot of trust and vision, there will be people who don’t like change or are afraid of the unknown, not just on development but on the business side too. Both often have conflicting goals and targets. The only way this can be realistically done is through small measurable steps; make a change, did it work? If results are good then more buy in is easier, if not, why? Review and try again. The move to DevOps takes a lot of investment and you can not change the whole company over night. Prioritise where DevOps might be most useful, and start from there.
I also think automation is a key factor to the move to DevOps, for sure you need to invest in technology. There are a lot of overheads in testing, industrialisation and deployment. You should look to start to automate the areas that need less human intervention. Look at build and deployment automation, test automation.
Finally, consider your architecture and code base. New companies can roll out DevOps easier because they use the latest technologies and don’t have to deal with legacy code that larger, older companies do. Simultaneously, you will need to fix issues with architecture and get it into a place that can deliver faster business innovation.
To close, the important think is to understand why companies are moving towards DevOps, how it benefits them and how it can benefit you. Realise that its not an overnight silver bullet, and there can be a lot of pain in moving to the methodology. Its important to have a clear goal and vision in place to drive continuous improvement and iteratively change and improve.
Process hacker, optimizer and team motivator. Blogging my ideas, initiatives and innovations. Certified Scrum Master/ Product Owner & Agile Project Manager
Please don’t forget to add or follow me on the following social media!
If you like, take something away or reference any of my content, please consider leaving a small donation to help with the running of the site! To make a small donation simply click on the button below. Thank you very much in advance!