Development Operations, commonly referred to as DevOps, is one of the most influential trends in software development today. The term itself reflects what it does – merge development and operations processes, practices and responsibilities into a single DevOps software development approach.
In a traditional organisation, the software development team designs, codes and tests (QA is sometimes considered a third distinct team) a software application and then hands it over to the operations team, which is responsible for running it for users. DevOps unifies the development team and operations team into a single DevOps team with shared performance metrics.
What’s the relationship between DevOps and Agile?
DevOps teams are almost always also described as ‘agile teams’ but there can be some confusion around the exact nature of the relationship between DevOps and Agile software development. The easiest way to think of the relationship between DevOps and Agile is to consider the former an extension of the latter’s principles.
A DevOps approach applies similar principles to those outlined by the Agile Manifesto but extends them across a single team that unifies development and operations roles, responsibilities and performance metrics.
DevOps solves a bottleneck problem associated with Agile development. When Agile frameworks like Scrum or Extreme Programming (XP) speed up the pace at which new iterations are developed, distinct operations teams can often struggle to test and deploy the software in a timely manner, and the value of rapid development is lost. Extending an Agile approach across the entire software development life cycle (SDLC) to deployment removes that bottleneck.
What’s the best definition of DevOps?
Despite its popularity, the concept of DevOps often lacks clarity for even many within or connected to the IT sector. That can see it mistaken as a buzzword that lacks real substance.
The fact its nature is described in so many different ways, interchangeably as a philosophy, culture, methodology, practices, set of tools and team roles undoubtedly has a lot to do with that. But there is no contradiction in saying DevOps can be all of those things simultaneously. In fact, to see its full benefits, DevOps should be:
- A philosophy
- A culture
- A methodology
- Practices
- Involve the use of tools with an emphasis on automation, and;
- Often involves distinct team roles
To answer the question posed by the title of this section,“what’s the best definition of DevOps?”, we need to introduce why how DevOps can be described in so many different ways. And it also makes sense to explore what DevOps is not.
DevOps as a philosophy
At its most abstract level, DevOps can be, and often is, described as a philosophy: “a theory or attitude that acts as a guiding principle for behaviour.”
The starting point of a successful DevOps approach to software development is that the team (many would even argue this should be at an organisational level, especially in a tech company) buy into the concept of why unifying development (including testing), and operations will lead to better outcomes across stakeholder groups.
DevOps as a culture
While DevOps as a philosophy introduces it at a conceptual level, DevOps as a culture implies the concept manifesting itself in the approach through processes and practices.
A DevOps team culture implies habitually doing things, to the point they become ingrained, which support the realisation of the DevOps philosophy of unifying development, testing and operations as one continuous feedback loop.
A DevOps culture means that the approach, decisions and practices of software development teams, and other members of the organisation they work with directly or whose decisions affect them, are, as a default, intrinsically informed by the philosophy.
Instilling a DevOps culture involves cultivating multidisciplinary teams responsible for the entire lifecycle of a product. And fostering transparency, communication, and collaboration between roles with titles that would have traditionally been considered as belonging to the development or operations teams.
Pillars of a DevOps culture include:
- Collaboration, visibility, consensus – different team roles must plan work together to align individual and team responsibilities, goals and metrics in a way that supports the DevOps philosophy.
- Cross-team responsibilities and accountability – role and team-level responsibilities and accountability are redefined to empower and create shared performance metrics that bridge dev and ops.
- Shorter release cycles – are shorter release cycles part of DevOps culture or the result of DevOps culture? It’s a bit of a chicken and egg question but at K&C we consider them so intrinsic they are part of the culture.
Gene Kim, one of the authors of the 2013 DevOps book The Phoenix Project as well as The DevOps Handbook (2016), formulated three guiding principles that should underpin a DevOps culture:
- Systems Thinking: emphasises the performance of the entire system, as opposed to the performance of a specific silo of work or department and focuses on all business value streams that are enabled by IT. In the context of a software development lifecycle, Systems Thinking starts when requirements are identified (e.g., by the business), through planning and development, before an application is transitioned into production, where the value is then delivered to the customer as a form of service.
- Amplification of Feedback Loops:as a process improvement initiative, a goal of DevOps is to shorten and amplify feedback loops so corrections and improvements can be continually made. The outcome of amplified feedback loops should be improved understanding of and response to all customer needs and then embedding knowledge where it is needed.
- Continual Experimentation and Learning:a DevOps culture involves a clear focus on finding new and better ways of working. A DevOps transformation should, says Kim, “create a culture that encourages continuous experimentation, risk-taking and learning from failure, and an understanding that repetition and practice are essential to mastering a task.”
DevOps as a methodology
A methodology is a specific approach and order of doing things to reach a particular goal. Software development methodologies are no different and any new software product passes through the same 6 broad stages of a development life cycle:
- Planning
- Design
- Development
- Testing
- Deployment
- Maintenance
DevOps life cycles are often presented as slightly more granular, adding two additional stages:
- Plan
- Code
- Build
- Test
- Package
- Release
- Deploy
- Operate
- Monitor
A DevOps methodology passes through these eight stages of software development and operations in an infinite loop.
DevOps as processes and practices
If DevOps as a methodology is an order to doing things approximately represented by the infinite loop of the eight life cycle stages detailed, DevOps processes and practices are the series of actions, performed in a certain order, that make up those stages.
DevOps processes and practices are designed to speed up, automate, and streamline a phase. And some can span multiple stages. DevOps processes and practices include:
- Creating code repositories
- Source control
- Code compiling
- Code validation tests
- Package creation
- Continuous testing
- Continuous integration (CI)
- Continuous delivery
- Continuous deployment
They might also include the processes and practices of a particular Agile framework DevOps acts as an extension of like Scrum, such as:
- Daily stand-ups
- Product backlog creation
- Sprint planning meetings
- Working on the Sprint
- The impediment list
- Sprint review and Sprint retrospective
Or Kanban, for example:
- Visualize the flow of work
- Limit WIP (Work in Progress)
- Manage Flow
- Make Process Policies Explicit
- Implement Feedback Loops
- Improve Collaboratively, Evolve Experimentally
[totb title=”Can We Help You With Your Next Software Development Project?” subtitle=”Flexible models to fit your needs!” buttonlink=”https://kruschecompany.com/service/software-development-company/” buttonlabel=”Get in touch!”][/totb]
DevOps tools
DevOps is highly associated with the ecosystem of tools that have been developed in line with the cultural mantra “if you can automate it, you should automate it”.
DevOps is so pro-automation because it means:
- Teams can focus more on the product
- Results are repeatable and predictable
- Teams can respond much quicker and flexibly to problems
It’s also why CI/CD pipelines are such an integral part of DevOps. But there are fantastic tools available to help automate or make more efficient every stage of the DevOps life cycle, as well as many of its processes and practices.
Octopus Deploy has put together a great list of tools commonly used by DevOps teams. Some, like those in the planning phase for example, are more general while others are more specific to the DevOps context:
Planning
Collaboration and project management tools
- Slack
- Zoom
- Trello
- Teams
- Confluence
- com
Code repositories and source control
Source control is managed with Git and code repositories are hosted services for Git-managed code.
- Git
- GitHub
- GitLab
- Bitbucket
Build and test
DevOps teams use CI platform, or build servers, to save time through the automation of code compiling, validation tests and package creation. Among the most popular are:
- Jenkins
- GitHub Actions
- TeamCity
- Circle CI
- Atlassian Bamboo
- Azure DevOps
Package
The packaging tools are used to convert code into deployable artefacts hosted and deployed by package repositories. Among the most popular are:
- Docker Hub
- JFrog
- Codefresh
- Red Hat Quay
Releases and deployments
Release management and deployment tools help automate CI/CD pipelines, with popular options including:
- Octopus
- Atlassian Bamboo
- AWS CodeDeploy
- GitLab CI/CD
- DeployHQ
- ElectricFlow
Operations
Popular DevOps tools used for the setup, running, and pipeline infrastructure maintenance include:
- Chef
- Puppet Enterprise
- Saltstack
- Octopus Runbooks
Monitoring
DevOps tools use in monitoring scrape code and related systems for data that helps inform performance and customer usage decisions with popular options including:
- New Relic
- Splunk
- Dynatrace
- Datadog
DevOps as a role or job title
DevOps is less about the introduction of new roles and job titles as it is about a redistribution of responsibilities across a team, with a more inter-disciplinarian approach. The entire team shares accountability for application and infrastructure planning, development, testing and release.
However, the title of DevOps Engineer is also now common to the point several studies pinpointed it as the most in-demand software development role in the world in 2021.
What does a DevOps engineer do?
While DevOps engineers are in extremely high demand, the title can apply to a relatively wide variety of roles. A DevOps engineer working on one software development project can have significantly different responsibilities from another working on a different project. It’s even perfectly possible for a single DevOps team to contain multiple DevOps engineers with different focuses and responsibilities.
DevOps engineers are most commonly focused on building and maintaining CI/CD pipelines and have experience with setting up and using popular DevOps tools for:
- Version control
- Continuous integration (CI) servers
- Configuration management
- Deployment automation
- Containers and container orchestration (especially Kubernetes)
- Infrastructure orchestration
- Monitoring and analytics
- Testing and cloud quality control
- Infrastructure as Code (IAC)
For more details on the role of DevOps engineer you can check out our blogs Profile of a DevOps engineer and International DevOps engineer salary ranges and rates.
What’s the benefit of DevOps?
The benefit of a combined DevOps team is seen as higher quality, more reliable software that is iterated on faster. Which is why DevOps is such a major trend in contemporary software development.
The cloud services provider Microsoft Azure lists the benefits of DevOps as the business case deliverables:
- Shorter time to market
- Adaptation to the market and competition
- Maintaining the stability and reliability of systems
- Improvement in MTTR (Mean Time To Recover)
In this blog, we’ll look at different but mutually compatible definitions of DevOps and how it is practically implemented to deliver the organisational benefits that make it such an influential trend.
What problem does DevOps solve?
DevOps realises benefits for software development teams and at an organisational level by helping to eliminate weaknesses inherent in the functional separation of those who write and test the code that forms a software application and those that deploy and support it.
The early proponents of DevOps like Patrick Debois, Andrew Shafer and John Allspaw argued there were inherent flaws in this separation which compromised the overall quality of software delivered to users. These stemmed from the fact that separate teams judged on different performance indicators were responsible for the same end goal; providing users with the highest quality software experience possible.
If the software development team has no involvement in or accountability for the successful deployment of the software they have built, human nature dictates they will be less invested in the quality and efficiency of that deployment. DevOps breaks downs the development and operations silos to share accountability and recognition for high quality software, defined as the user experience.
DevOps KPIs
Any organisation that commits to a DevOps transformation needs KPIs to track and measure its impacts on strategic goals. These most often include:
- Deployment frequency: deployments should become smaller and more frequent after DevOps adoption.
- Frequency of failed deployments: this metric should show a decline.
- Features released per quarter: the number should rise.
- MTTD (Mean-Time-To-Discovery): the average time it takes for something going wrong with software to be noticed should fall.
- MTTR (Mean-Time-To-Recovery): the average time it takes to fix errors when something has gone wrong should also fall.
- MLT (Mean Lead Time): the average amount of time between a request for new code or features to implementation should fall.
- Uptime: the availability of software, including planned downtown and outages, should increase.
- Defect Escape Rates: the number of bugs and errors escaping detection during testing and QA and making it into production should drop.
- Application Performance: there should be a more pronounced improvement in the performance of software following the release of new iterations should improve. Software performance should be measured by its own set of KPIs.
What’s the connection between DevOps and cloud development?
DevOps is most associated with cloud and cloud-native development, though is by no means only applicable in that context. Broadly defined, cloud computing “allows infrastructure to be defined as code or templates, hence enabling the creation of repeatable processes that are not possible without cloud technologies.”
These “repeatable processes” are what allow the automation which is, as we’ll go on to explain in greater detail, central to DevOps development. In short, the speed, scale, and efficiency of the cloud help agile and DevOps teams to increase the speed and quality of their work.
Cloud-native development is most commonly described as “developed in the cloud for the cloud”. That refers to the reality that the development environment of cloud-native applications is, or should be, an exact replica of the production environment operations will run software in.
There are two core advantages to development and production infrastructure mirroring each other:
- Sofware should run in the production environment in the exact same way it has run in the development environment. That means operations should not need to separately provision deployment infrastructure for it to smoothly and reliably run the software built by development. Nor return it to development for updates to make it compatible.
- Matching development and deployment environments allows for the continuous iteration/continuous deployment (CI/CD) pipelines that make software development faster and more flexible, improving agility and, ultimately, quality.
DevOps takes a cloud-first approach to any new application, which flips the question “should we use the cloud for this?”, instead asking “is there a reason not to use the cloud for this?”
Common reasons why an app should not be exclusively built and run in a cloud environment, or even at all, include security and regulatory requirements as well as sometimes broader infrastructure factors at an organisational level.
But not using the cloud, or only the cloud, doesn’t preclude DevOps. A DevOps mindset of development and operations working collaboratively towards the shared end goal of the highest quality software experience possible for users is also applicable in the context of hybrid and bare-metal server deployment. So while DevOps and cloud share a close relationship, it is not an exclusive one.
DevOps engineers and teams can also work together with site reliability engineers (SREs), with a well-defined and managed approach offering significant synergies.
DevOps architecture
When the term DevOps architecture is used it refers to the specific infrastructure created to support DevOps processes and practices across the different phases of an application’s life cycle. Part of that infrastructure is likely to consist of some of the tools mentioned in the previous section, while others are used in its creation.
Cloud platforms, virtualization, containerisation, microservices, CI/CD pipelines, security practices and Infrastructure as Code all typically come together in a DevOps architecture.
The role of containers in a DevOps architecture
Containers, most often orchestrated by Kubernetes, are a key pillar of a DevOps architecture. Containers consolidate source code, configuration files, libraries, binaries, and all dependencies into a single package to quickly and reliably run an application. Packaging code modules within one or multiple containers allows for easy transportation between computing environments, like development, testing and production environment.
Containers insulate code from any differences in the operating system distribution, security policies, network topologies and underlying infrastructure, which is vital to smooth CI/CD pipelines and much of the other automation that DevOps introduces. By providing a self-contained virtual infrastructure (IaC), containers can run the code packaged within them smoothly in any environment.
Breaking an application down into smaller containerised modules known as microservices is also common to a DevOps architecture. Each microservice runs independently and is loosely coupled with other microservices, with which it communicates via APIs. This decoupling of features and functionalities has two core advantages which align with the goals of DevOps:
- Higher availability – if one microservice encounters a problem and suffers an outage, it is unlikely to impact the overall running of the application in the way an error in code in a monolithic architecture would because the loose coupling of services minimises dependencies.
- Faster iteration – separating functions into microservices makes it much easier to add new functions to deployed software without downtime, making the CI/CD practices central to DevOps possible.
What is DevSecOps?
The DevOps approach is increasingly being extended to integrate security processes, practices and tooling and that is a trend that can be expected to strengthen as security grows as a priority for all software systems and applications. The integration of cross-team accountability for security has been termed DevSecOps.
If the goals of DevOps are deliver better quality software, faster, the goals of DevSecOps are to deliver more secure, better quality software faster.
The 2012 DevSecOps manifesto outlines three core principles of DevSecOps:
- Introduce and integrate security thinking early in the development process.
- Introduce security thinking as a core development responsibility, not a separate team’s concern.
- Automate security processes to maintain the efficient flow of DevOps.
DevOps as a software development industry standard
Understanding how DevOps is defined in the different contexts we have covered here, its goals and how they are achieved by the processes, practices and tools we’ve covered here is important for anyone in the IT and software development space. That holds true whether your role is directly connected to software development lifecycles or more loosely as a stakeholder in a technology-first organisation.
Why? Because DevOps is becoming as much of a technology standard as leveraging cloud computing and adopting Agile development practices. In fact, it’s becoming a standard as a direct result of those two trends. All three are not only here to stay for the foreseeable future but will become increasingly the default standards for contemporary software solutions.
