1 Introduction

DevOps is the practice of operations and development engineers working together collaboratively to improve productivity by automating infrastructure, automating workflows and continuously measuring application performance.

To achieve better collaboration, it does the following to help development and operations teams be more efficient, and faster, and deliver higher value to businesses and customers:

• Automate everything, code testing, infrastructure and workflows
• Agile and continuous delivery - They would write software in small chunks, that are integrated, tested, and deployed usually in hours. It enables them to adopt an iterative process to monitor, measure and improve the code and operation every day.
• They have identical environment for development, testing and production based on the same configuration.
• Use a source control system to help manage, track and document all changes to both application code and configuration management code.
• Adopt a discipline of application performance monitoring and optimization in almost real time. This allows developers to understand the performance impact of their changes.

Benefits of DevOps:

• Faster time to market
• Higher productivity
• Better quality
• Better collaboration
• More frequent releases

The concept of DevOps is founded on building a culture of collaboration between teams that historically functioned in relative siloes.

The goal of DevOps is to deliver features, fixes, and updates faster and frequently in close alignment with business objectives.

At its essence, DevOps is a culture, a movement, a philosophy.

A primary corollary to this is that part of the major change in practice from previous methods is:

DevOps is also characterized by operations staff making use many of the same techniques as developers for their systems work.

In details, see:

https://www.youtube.com/watch?v=_I94-tJlovg

https://www.atlassian.com/devops

https://theagileadmin.com/what-is-devops/

https://aws.amazon.com/devops/what-is-devops/?nc1=h_ls

https://www.kartar.net/2010/02/what-devops-means-to-me.../

1.1 Relation to Agile Development

DevOps can be interpreted as an outgrowth of Agile – agile software development prescribes close collaboration of customers, product management, developers, and (sometimes) QA to fill in the gaps and rapidly iterate towards a better product.

DevOps says “yes, but service delivery and how the app and systems interact are a fundamental part of the value proposition to the client as well, and so the product team needs to include those concerns as a top level item.” From this perspective, DevOps is simply extending Agile principles beyond the boundaries of “the code” to the entire lifecycle of the software product.

1.2 Barriers between teams

开发者和运维人员之间最大的问题在于：都是企业中IT部门不可或缺的，但他们有着截然不同的目的（Objective）。开发者和运维人员之间目的上的差异就叫做混乱之墙（Wall of Confusion）。

DevOps是为了填补开发端和运维端之间的信息鸿沟，改善团队之间的协作关系。不过需要澄清的一点是，从开发到运维，中间还有测试环节。DevOps其实包含了三个部分：开发、测试和运维。

DevOps是一种融合了一系列基本原则和实践的方法论（并从这些实践中派生出了各种工具），意在帮助这些人员向着一个统一的共同目的努力：尽可能为公司提供更多价值。

1.3 Goals

• Improved deployment frequency;
• Faster time to market;
• Lower failure rate of new releases;
• Shortened lead time between fixes;
• Faster mean time to recovery (in the event of a new release crashing or otherwise disabling the current system).

1.4 运维人员之困

生产团队（运维团队）有将近一半（47%）的时间花在了与部署有关的工作中：

• 执行实际的部署工作，或
• 修复与部署工作有关的问题

以手工操作的数量所表示的成功部署概率：

这些统计告诉我们：

• 只需手工运行5条命令的情况下，成功部署的概率就已跌至86%。
• 如需手工运行55条命令，成功部署的概率将跌至22%。
• 如需手工运行100条命令，成功部署的概率将趋近于0（仅2%）！

1.5 让生产端变得敏捷起来

令人惊奇的是，这个问题竟然有一个非常简单的“银子弹”：让生产端变得敏捷起来！

DevOps consists mostly in extending agile development practices by further streamlining the movement of software change thru the build, validate, deploy and delivery stages, while empowering cross-functional teams with full ownership of software applications - from design thru production support.

DevOps encourages communication, collaboration, integration and automation among software developers and IT operators in order to improve both the speed and quality of delivering software.

DevOps teams focus on standardizing development environments and automating delivery processes to improve delivery predictability, efficiency, security and maintainability. The DevOps ideals provide developers more control of the production environment and a better understanding of the production infrastructure.

DevOps encourages empowering teams with the autonomy to build, validate, deliver and support their own applications.

1.6 Core Principles

• Infrastructure as Code
• Continuous Delivery
• Learn from the field
• Culture of Collaboration

A DevOps Manifesto

1.6.1 Infrastructure as Code（基础架构即代码，IaC）

IaC是通用的DevOps实践的前提要求，如version control, code review, continuous integration and automated testing，它在于通过机器可处理的定义或脚本，管理，供应（provision）和配置我们的infrastructure，交互式配置工具和手工命令的使用已经不合时宜了。We should consider and handle infrastructure concerns just as we handle coding concerns! 也就是，将软件开发实践应用于infrastructure。

Versioning, Continuous Integration and Automated testing of infrastructure components:

The ability to version the infrastructure - or rather the infrastructure building scripts or configuration files - as well as the ability to automate test of them are very important.

DevOps consists in finally adopting the same practices XP to the production side.

Even further, Infrastructure elements should be continuously integrated just as software deliverables.

1.6.1.1 Overview of Tools

In a very summarized way, the levels of infrastructure and operation concerns at which automation should occur is represented on this schema:

The tools proposed as examples on the schema above are very much oriented towards building the different layers. But a devops toolchain does much more than that.

1.6.1.2 Toolchain

There is no single tool, rather a set of tools fitting into the following categories of DevOps circle, so called toolchain:

• Code : Code development and review, version control tools, code merging
• Build : Continuous integration tools, build status
• Test : Test and results determine performance
• Package : Artifact repository, application pre-deployment staging
• Release : Change management, release approvals, release automation
• Configure : Infrastructure configuration and management, Infrastructure as Code tools
• Monitor : Applications performance monitoring, end user experience

Tools such as Docker (containerization), Jenkins (continuous Integration), Puppet (Infrastructure building) and Vagrant (virtualization platform) among many others are often used and frequently referenced in DevOps tooling discussions as of 2016.

1.6.1.3 Benefits

• Repeatability and Reliability: building the production machine is now simply running that script or that puppet command. With proper usage of docker containers or vagrant virtual machines, a production machine with the Operating System layer and, of course, all the software properly installed and configured can be set up by typing one single command - One Single Command. And of course this building script or mechanism is continuously integrated upon changes or when being developed, continuously and automatically tested, etc. Finally we can benefit on the operation side from the same practices we use with success on the software development side, thanks to XP or Agile.
• Productivity: one click deployment, one click provisioning, one click new environment creation, etc. Again, the whole production environment is set-up using one single command or one click. Now of course that command can well run for hours, but during that time the operator can focus on more interesting things, instead of waiting for a single individual command to complete before typing the next one, and that sometimes for several days...
• Time to recovery ! : one click recovery of the production environment, period.
• Guarantee that infrastructure is homogeneous: completely eliminating the possibility for an operator to build an environment or install a software slightly differently every time is the only way to guarantee that the infrastructure is perfectly homogeneous and reproducible. Even further, with version control of scripts or puppet configuration files, one can rebuild the production environment precisely as it was last week, last month, or for that particular release of the software.
• Make sure standards are respected: infrastructure standards are not even required anymore. The standard is the code for infrastructure.
• Allow developer to do lots of tasks themselves : if developers become themselves suddenly able to re-create the production environment on their own infrastructure by one single click, they become able to do a lot of production related tasks by themselves as well, such as understanding production failures, providing proper configuration, implementing deployment scripts, etc.

1.6.2 持续交付

持续交付是一种可以帮助团队以更短的周期交付软件的方法，该方法确保了团队可以在任何时间发布出可靠的软件。该方法意在以更快速度更高频率进行软件的构建、测试和发布。

足够简单直接并且可重复的部署流程对持续交付而言至关重要。

The key ideas behind continuous deliveries are:

• The more often you deploy, the more you master the deployment process and the better you automate it. If you have to do something 3 times a day, you will make it bullet proof and reliable soon enough, when you will be fed up of fixing the same issues over and over again.
• The more often you deploy, the smallest will be the changesets you deploy and hence the smallest will be the risk of something going wrong, or the chances of losing control over the changesets
• The more often you deploy, the best will be your TTR (Time to Repair / Resolution) and hence the sooner will be the feedback you will get from your business users regarding that feature and the easier it will be to change some things here and there to make it perfectly fit their needs (TTR is very similar to TTM in this regards).

But continuous delivery is more than building a shippable, production-ready version of the product as often as possible. Continuous delivery refers to 3 key practices:

• Automation
• Deploy more often

1.6.2.1 Deploy more often

The DevOps credo is:

"If it hurts, do it more often !"

This idea of doing painful things more frequently is very important in agile thinking.

There are three good reasons for that:

• Firstly most of these tasks become much more difficult as the amount of work to be done increases, but when broken up into smaller chunks and dealt with them more often, they will be done easily.
• The second reason is quick feedback. Much of agile thinking is about setting up feedback loops so that we can learn more quickly.
• The third reason is practice. With any activity, we improve as we do it more often. With software development, there's also in addition the potential for automation. Once one has done something a few times, it's easier to see how to automate it, and more importantly one becomes more motivated to automate it. Automation is especially helpful because it can increase speed and reduce the chance for error.

How often to deliver with DevOps ?

There is no straight answer to that. It really depends on the product, the team, the market, the company, the users, the operation needs, etc.

My best answer would be as follows: If you don't deliver at least every 2 weeks - or at the end of your sprint duration period - you do not even do Agile, not to speak of DevOps.

DevOps encourages to deliver as frequently as possible. In my understanding, you should train your team to be able to deliver as frequently as possible. A sound approach, the one I'm using with my team is to deliver twice a day on a QA environment. The delivery process is fully automated: twice a day, at noon and at midnight, the machinery starts, builds the software components, runs integration tests, builds the Virtual Machines, start them, deploys the software components, configures them, runs functional tests, etc.

1.6.2.2 Continuous Delivery requirements

What does one need before being able to move to Continuous Delivery?

My checklist, in a raw fashion :

• Continuous integration of both the software components development as well as the platform provisioning and setup.
• TDD - Test Driven Development. This is questionable ... But in the end let's face it: TDD is really the single and only way to have an acceptable coverage of the code and branches with unit tests (and unit tests makes is so much easier to fix issues than integration or functional tests).
• Code reviews ! At least codereviews ... pair programming would be better of course.
• Continuous auditing software - such as Sonar.
• Functional testing automation on production-level environment
• Strong non-functional testing automation (performance, availability, etc.)
• Automated packaging and deployment, independent of target environment

1.6.2.3 Zero Downtime Deployments

Zero Downtime Deployment (ZDD) consists in deploying a new version of a system without any interruption of service.

I'll mention 4 techniques:

• Feature Flipping
• Dark launch
• Blue/Green Deployments
• Canari release

1.6.2.3.1 Feature flipping

Feature flipping allows enabling / disabling features while the software is running. It's really straightforward to understand and put in place: simply use a configuration properly to entirely disable a feature from production and only activate it when its completely polished and working well.

1.6.2.3.2 Dark Launch

The idea of Dark Launch is to use production to simulate load and test a feature in background, invisible to users without disrupting usability! It is used where it's difficult to simulate load of a software used by hundreds of millions of people in a testing environment.

1.6.2.3.3 Blue/Green Deployments

Blue/Green Deployments consists in building a second complete line of production for version N + 1. Whenever the version N + 1 is ready to be used, the configuration is changed on the load balancer and users are automatically and transparently redirected to the new version N + 1.

At this moment, the production line for version N is recovered and used to peacefully build version N + 2. And so on.

This is quite effective and easy but the problem is that it requires to double the infrastructure, amount of servers, etc.

1.6.2.3.4 Canari release

Canari release is very similar in nature to Blue/Green Deployments but it addresses the problem to have multiple complete production lines.

The idea is to switch users to the new version in an incremental fashion: as more servers are migrated from the version N line to the version N + 1 line, an equivalent proportion of users are migrated as well.

This way, the load on every production line matches the amount of servers.

At first, only a few servers are migrated to version N + 1 along with a small subset of the users. This also allows testing the new release without risking an impact on all users. When all servers have eventually been migrated from line N to line N + 1, the release is finished and everything can start all over again for release N + 2.

1.6.3 Learn from the Field

There is no truth in the development team, the truth lies in the head of the business users. This is why Agility attempts to put the feature in the hands of the users to get their feedback as soon as possible.

But the truth doesn't come out in the form of a formal user feedback. One should trust its own measures and measure everything: response times, user think times, count of displays, count of API calls, click rate, etc. Finding the right metrics enabling the team to learn about the success or failures of an approach, about what would be better and what has the most success.

1.6.4 Communication/Collaboration

The primary characteristic of DevOps culture is increased collaboration between the roles of development and operations.

1.6.4.1 Ops as Users

Below is a simplified view of how the Agile Software Development Process usually looks like.

• Initially the business representatives work with the Product Owner and the Architecture Team to define the software, either through Story Mapping with User stories or with more complete specification.
• Then the development team develops the software in short development sprints, shipping a production ready version of the software to the business users at the end of every sprint in order to capture feedback and get directions as often and as much as possible.
• Finally, after every new milestone, the software is deployed for wide usage to all business lines.

The big change introduced by DevOps is the understanding that operators are the other users of the software ! and as such they should be fully integrated in the Software Development Process.

• At specification time, operators should give their non-functional requirements just as business users give their functional requirement. Such non-functional requirements should be handled with same important and priority by the development team.
• At implementation time, operators should provide feedback and non-functional tests specifications continuously just as business users provides feedback on functional features.
• Finally, operators become users of the software just as business users.

通过采用DevOps方法，运维可以全面融入软件开发流程中。

1.6.4.2 Share Tools

1.6.4.3 Work together

A fundamental philosophy of DevOps is that developers and operations staff must work closely together on a regular basis. An implication is that they must see one other as important stakeholders and actively seek to work together.

Inspired from the XP practice "onsite customer", which motivates agile developers to work closely with the business, disciplined agilists take this one step further with the practice of active stakeholder participation, which says that developers should work closely with all of their stakeholders, including operations and support staff.

This is a two-way street: operations and support staff must also be willing to work closely with developers.

Collaborations could be:

• Have operators taking part in Agile rituals (Daily scrum, sprint planning, sprint retro, etc.)
• Have devs taking part in production rollouts
• Share between Dev and Ops objectives of continuous improvement

1.7 好处

• Puppet和DevOps Research and Assessment (DORA) 主办了2016年DevOps调查报告，根据全球4600位各IT公司的技术工作者的提交数据统计，得出高效公司平均每年可以完成1460次部署。与低效组织相比，高效组织的部署频繁200倍，产品投入使用速度快2555倍，服务恢复速度快24倍。在工作内容的时间分配上，低效者要多花22%的时间用在为规划好或者重复工作上，而高效者却可以多花29%的时间用在新的工作上。所以这里的高效不仅仅指公司产出的效率提高，还指员工的工作质量得到提升。

• DevOps另外一个好处就是会改善公司组织文化、提高员工的参与感。员工们变得更高效，也更有满足和成就感；调查显示高效员工的雇员净推荐值（eNPS:employee Net Promoter Score）更高，即对公司更加认同。

• 快速的部署其实可以帮助更快地发现问题，产品被更快地交付到用户手中，团队可以更快地得到用户的反馈，从而进行更快地响应。而且，DevOps小步快跑的形式带来的变化是比较小的，出现问题的偏差每次都不会太大，修复起来也会相对容易一些。