Role of Enterprise Architecture in DevOps Adoption

Introduction

In traditional development, the development team and operations team always work in silos. Development teams focus on functionality, features, user experience, architecture, and non-functional requirements. The focuses of operations teams are cost, reliability, security, risk, and manageability.

DevOps is the set of concepts, cultural philosophies, practices, team organization structures and tools that increases an organization’s ability to deliver applications and services at high velocity to the clients. It helps in responding to new requirements quickly, or to problems that occur in production. This enables organizations to better serve their customers and compete more effectively in the market.

DevOps aims to decrease the time between applying a change to a system and the change transferred to the production environment.

Challenges in Agile

Agile was all about speed to market, but DevOps brings in the quality and reliability aspect. In Agile, we skip a lot of the architectural considerations and then perform sprint after sprint until they come together and realize the pieces do not fit.

This technical debt grows from sprint to sprint, and there is minimal time to go back and fix the problem because they focused on getting new features out.

DevOps promotes the Agile methods to operations. It provides the collaboration between development and operations teams.

What is DevOps?

DevOps aims at unifying software development and software operation. DevOps emphasizes people and culture to improve collaboration between development and operations groups. 

DevOps is not:

• A role, person, or organization

• Something only system administrator can only do

• Something only developers can do

• Writing Chef and Puppet scripts

• Tools

Organizations today are looking at faster delivery, quick feedback from customers, providing quality software, less recovery cycle time from any crashes, and fewer defects from more and more automation. Thus, one needs to ensure that with all the tools used and about the Integrations for the Development and Operations team to collaborate or communicate better.

DevOps is about principles, practices, and creating a collaborative environment that improves software delivery and increases business value. It encompasses people, processes and tools. It connects development, quality assurance and technical operations, working together with engineers to get things done faster in an automated and repeatable way.

The objective of DevOps is to maximize the predictability, efficiency, security, and maintainability of operational processes. Automation supports this objective.   

DevOps targets product delivery, quality testing, feature development, and maintenance releases in order to improve reliability, security, and faster development and deployment cycles.

The goals of DevOps are to make improvements across all components in the product and service delivery. They include:

• Improved deployment frequency

• Faster time to market

• Lower failure rates of new releases

• Faster recovery time from crashes or failures

DevOps implementation utilizes technology, especially automation tools, and a big part of automation is continuous integration and delivery. With this automation, we can go from concept to production very quickly, but only with proper architecture.

DevOps EA Framework

Planning

DevOps Readiness

This is the first step where we conduct an assessment of the organization's readiness for the DevOps adoption.

The DevOps architect and the DevOps SMEs engage in conversation with the respective customer teams to analyze the current DevOps capabilities and the tool usage across enterprise. This provides the DevOps adoption maturity level of an organization.  

A questionnaire and attributes analysis is used to gather information about the nature of the application portfolio and their architecture. A properly filled questionnaire provides a good understanding of the current state at the customer organization.

DevOps Planning and Road-mapping

The DevOps architect and stakeholders will finalize the DevOps objectives for the implementation and an appropriate timeline for achieving those objectives.

Document the current processes, tools and systems used across the enterprise.

The DevOps architect and the team analyze the existing applications for CI/CD scope and estimate the enterprise applications that can be part of CI/CD. Identification of applications for the DevOps adoption is carried out based on the inputs provided by stakeholders and high-level analysis.   

A roadmap is developed to implement new DevOps processes, to improve the existing processes and the list of tools is identified as part of the planning stage.   

Baseline Measurement

As part of baselining, the following data needs to be agreed on and captured across the enterprise by an architect in agreement with the development teams:

• Number of releases in a year
• Average time to fix a defect
• Time taken to complete full software build
• Time taken to complete execution of Unit tests suite
• Average time to build and test  etc

Pilot Applications Identification

During this stage, the business impact, user base, criticality of the application being assessed and documented for the pilot implementation.

A pilot shows the benefit of DevOps process by prototyping and demonstrating one of the areas identified for improvement.

DevOps Solution Blueprint

The solution blueprint architecture addresses the implementing Continuous Integration (CI), Continuous Testing (CT), Continuous Deployment (CD) and Continuous Change Management (CCM) subsystems.

DevOps solution blueprint architecture supports Agile or non-Agile development environments. It supports physical, virtualized, and mixed physical/virtualized host and lab network environments. It also supports configuration and control across multiple product boundaries, service layers, and software versions.  

A DevOps architect with the support of DevOps Engineer prepares this solution blueprint.

Architecture Principles

A DevOps architect along with the enterprise stakeholders define the DevOps architecture principles. The core architecture principles are:

• Architecture has to be delivered as frequently as necessary and preferably before a decision is made
• Frequency of the change can vary within an enterprise and across the enterprises
• Delivery of the architecture and use of architecture should match the needs of customers
• Enterprise has multiple divisions, like IT, Finance, HR, Manufacturing, and Operations. Agile requires all divisions of the enterprise to be Agile.
• Enterprise architecture covers all business units and guides delivery of solutions by all business units

Solution Definition

During this phase, the DevOps process definition of domain architectures covering business, application, data, and technology are covered. A set of domain architectures work to clear the gaps understood by the stakeholders.

DevOps Process Definition

DevOps processes will transform the enterprise from its current to future state across all dimensions. The solution approach would be multi-dimensional across people, process, tools and automation and data to enable enterprise objectives.

The DevOps architect and other stakeholders of DevOps work with enterprise IT and Business to design the target state capabilities and DevOps processes that would help an enterprise achieve its business and IT objectives.

Tools Selection

The following are the DevOps tools selection design principles,

• Maximize tool usage and coverage across all Services and focus on value
• Utilize fit for purpose Contractor tools, or leverage Customer tools where required
• Integration with various sources for analytics
• Focus on automation and productivity tools for operations
• Application performance and currency through combination of dynamic and static analysis tools

DevOps Tool Chain Identification

DevOps implementation does not mean throw away existing tools in favor of new tools. It involves integrating the existing tools and migrating the non-compliant tools to fit the chosen practice/process. In addition, no single product in the market that can fulfill the DevOps tool capabilities across the enterprise.   

A DevOps toolchain is a set or combination of tools that aid in the delivery, development, and management of applications throughout the systems development life cycle. DevOps toolchain improves productivity, speeds up time to market, reduces risk and increases quality.

The DevOps architect has to assess the demand for a collection of tools, potentially from a variety of vendors used in one or more stages of the lifecycle and recommend the tool chain. 

Tools Integration

In an Agile environment, developers and testers need to ensure not only that code changes work and integrate well, but also that frequent iterations do not affect the stability of the product. Hence, seamless cross-tool collaboration is very important in DevOps implementation. An integrated tool system saves a considerable amount of time and effort.

Just deploying the right set of tools to the right users is not enough. Proper connectivity between and across the DevOps tools chain is necessary to create a collaborative work environment around the Dev and Ops teams.

Software Change Management

Software change management supports software development by providing software configuration control and identification, visibility into the status of software changes, and management of versions of the tools needed to implement the build process.

In a best-practices DevOps system, the SVM includes a well-designed, flexible repository structure and supports effective software branching and merging capabilities. This capability is critically important for Agile development processes, as developers merge changes back into the repository frequently.

Cloud

A DevOps hosted environment may be private, public or hybrid cloud services (e.g. AWS, Rackspace). The cloud is an abstract term used to generalize various types of remote storage, computation, and user access services. While cloud-computing infrastructures may consist of many types of services, most implementations provide one or more of the following service models:

• Infrastructure-as-a-Service (IaaS)

• Platform-as-a-Service (PaaS

• Software-as-a-Service (SaaS)

The increasing number of organizations adopting the cloud model indicates the value they are finding in the cloud. Cloud computing is an extremely cost-effective method of deploying, maintaining and securing desktop and server applications. Rather than purchase hardware and software licenses for each member, an organization can serve them up based on demand, not only reducing costs but also enhancing scalability.

Containerization Tools

Containerization tools help in maintaining consistency across the environments where the application is developed, tested, and deployed. It eliminates the failure in a production environment by packaging and replicating the same dependencies and packages used in development/testing/staging environment.

Platform Implementation

Reflection of Changes

In traditional software development, the changes in code are done quickly, but the reflection of those changes in a production environment takes time and it delays the product release. DevOps architect can solve this issue by implementing a CI/CD pipeline to speed up development, testing, and releases.

Repositories

Artifact repository is a collection of binary software artifacts and metadata stored in a defined directory structure. A repository stores two types of artifacts, releases and snapshots. Release repositories are for stable, static release artifacts and snapshot repositories are frequently updated repositories that store binary software artifacts from projects under constant development.  As an example, GitHub is the central repository where the code maintained. Bitbucket and Nexus are the other repository tools.

Configuration Management

Configuration Management helps in establishing and maintaining the consistency of applications' functional requirements and performance. Configuration Management tools work based on the master-slave architecture.

Tool Configuration and Tool Architecture

In this phase, the system architect supports test automation and continuous integration goals. The tool architecture has to support the application changes without release (e.g., late binding). The architecture has to be scalable, secure, available, and reliable.  It has to address the implementation of High Availability Infrastructure.

During this phase, the setting up of a sandbox environment for these tools is established. In addition, the identification and creation of modular and reusable components done.

Operations

Feedback and Monitoring

In traditional software development models like Waterfall, feedback takes too much time for validation, modification, and implementation.  However, in a DevOps approach, due to the automated delivery pipeline, constant feedback and monitoring become easy across the organization.

Frequent Releases

Version controls, updates, and releases are continuously monitored by using DevOps tools. DevOps architect recommends the strategies for tool adoption that provides greater efficiency.

Deployment automation solves the problem of deploying an application with an automated and repeatable process.

Metrics

An architect along with the organizational leadership defines Metrics or KPI’s. These are generated and updated to provide timely status report to organizational leadership.

The measurable KPI’s that are relevant and derived from DevOps deployments are:

• Percent of tests automate
• Percent availability of automation test infrastructure
• Product Velocity: release frequency
• Total Capex and Opex cost reduction compared to other approaches (e.g. ROI case study)

Service Monitoring

Operation Management Tools enables a monitoring solution to monitor performance and availability of the Infrastructure and Applications that comprise their Services and to monitor events coming from the Infrastructure devices.

Operations management provides a converged view of application and infrastructure performance across cloud environments. Management information data picked for probable cause analysis, behavior learning and dynamic baselining. It also provides behavioral and performance trends with automated event and log analytics, and prevent future impact with proactive notifications.

Monitoring services will ensure the Hardware components, OS, middleware, application, database and networking covered for performance, availability, utilization, capacity, trends, and recommendations on operational improvements.

DevOps Analytics

As the complexity of the as-is infrastructure environment and operations process grows, without applying the IT Analytics, we may lead to perform repetitive operations, time-consuming tasks in order to close these automation gaps.

DevOps leverage analytics and machine learning to automate the manual tasks involved in incident detection, troubleshooting, and root cause analysis.

The IT analytics tools can help in IT operations. The various categories of the analytics are:

• Statistical pattern analytics
• Textual pattern analytics
• Configuration analytics

DevOps Foundation

Continuous Planning

DevOps implementation and approach for the development and transitioned into Operations is done. The plan to deploy DevOps that involves people, process, and tools are done. Definition of target stage, transformation plan and execution plan are set during this phase.  Continuously plan, measure and bring business strategy and customer feedback into the development lifecycle.

Continuous Integration

Continuous Integration (CI) helps the developers to integrate code into a shared repository several times a day. It allows teams to detect problems early and verifies each check-in. By integrating regularly, it can detect errors quickly, and locate them more easily.

Continuous Deployment

Continuous Deployment is when every change goes through the pipeline and is automatically put into production, resulting in many production deployments every day with greater delivery speed and frequency for complex applications.

Continuous Monitoring

Continuous monitoring across all phases of the application development, testing, and deployment is crucial for a successful DevOps implementation. Capabilities to improve service quality by monitoring application performance and log management solves the problem of aggregating, storing, and analyzing all logs in one place.

DevOps EA Governance

To be effective, EA has to merge into a DevOps methodology. Instead of being the province of architects alone, it must get regular input from other teams, particularly engineering, operations, and business. Moreover, instead of laboring for years to create the ultimate architecture, architects should be prepared to deliver interim architectures in a matter of weeks.

Rather than being a lifeless stack of PowerPoint slides, EA should be deployable and testable.

DevOps EA Roles

Enterprise Architects are a part of the Agile team that helps in develop, improve and sustain enterprise architecture. Agile architects are active members of development teams, developing software where appropriate and acting as architectural consultants to the team.

• Design of the enterprise Cloud environments
• Playing a vital role in the design of production, staging, QA and development cloud infrastructures running in 24x7 environments
• Delivery of the enterprise cloud strategies aligned with the organization business objectives
• Provide leadership in infrastructure and drive cloud adoption

DevOps architects promote the DevOps adoption approach across the enterprise. The roles and responsibilities of a DevOps architect are similar to a DevOps Engineer, except for some broader leadership qualities. Each organization has a different structure for this depending on people, projects, and processes at their organization.

The responsibilities of the DevOps architect broadly classified as:

• Analyzing, executing, and streamlining DevOps practices
• Automating processes with the right tools
• Facilitating development process and operations
• Establishing a suitable DevOps channel across the organization
• Setting up a continuous build environment to speed up software development and deployment process
• Architecting overall, comprehensive, and efficient practices
• Guiding developers and operation teams in case of an issue
• Monitoring, reviewing, and managing technical operations
• Ability to manage teams with a leadership mindset

Automation and Tools Lead drives all the tool implementations across the enterprise, ensures ITSM tools and processes integrate with the service delivery model, and follows CI/CD to increase the level of resilience.

DevOps Engineers manage the entire application lifecycle while responding to feedback and analytics collected from the running application.

Benefits of DevOps

Some of the key benefits when we adopt DevOps in our Organization,

• Develops and verifies against production-like systems
• Reduces cost/time to deliver - deploy often, deploy faster with repeatable, reliable process
• Increases quality with automated testing that reduces the cost/time to test
• Increased service quality
• Increased reliability of service delivery
• Reduces defect cycle time
• Increases virtualize environments utilization
• Increased customer value through responsiveness to change
• Reduces deployment-related downtime
• Minimizes rollback
• Holistic value creation increasing employee satisfaction
• Constant learning and improvement

Summary

A DevOps journey should start with an analysis of the enterprise architecture and be based on a flexible foundation so that the benefits of rapid and frequent deployments can indeed bring in benefits to the business users.

Adjust enterprise EA practice to be consultative in the context of cultural change. The success of DevOps in an organization be achieved by, 

• High-trust cultural norms
• Architecture that lends itself to testability and deployment ability
• Cluster of technical practices that includes continuous integration, continuous delivery, automated deployment, and proactive production monitoring

To get the maximum benefit from the DevOps implementation, the focus shall be on three key areas — change of culture, connection of processes, and common tooling. This is crucial to reduce development-to-operations costs and minimize change-related outages.

DevOps architect plays a major role in enterprise DevOps adoption. They examine the big picture and recommends the model on how Development and Operations teams work together. They promote real-time, rapid, automated, and efficient development culture among the software development and operation teams. DevOps architect possesses knowledge about each tool and concept in a detailed manner.

From the business perspective, analytics reports help clarify the user base, location of users, features used by the users, and impact on the performance. It helps to receive the feedback from end users. End users are the key partners for any organization to strengthen DevOps products/services. Therefore, the valuable feedback from customer/end-user helps to optimize the performance and add features to the applications.

References

Scaled Agile Framework

Krutchen Engineering Services

Acknowledgements

The author would like to thank David Kenner and Raju Alluri of GEA Practice of Wipro Technologies for giving the required time and support in many ways in bringing up the article as part of GEA practice efforts.