The Ultimate DevOps Tools Ecosystem Tutorial

With the development process “shifting left,” DevOps is becoming a crucial part of any software product or service. Therefore, it’s no surprise that the number of tools for DevOps keeps growing to answer all of our working needs. 

However, the existence of so many DevOps tools can be confusing. To assist in organizing and choosing the right tools for the right needs, we’re going to cover the top tools over the upcoming weeks and their pros and cons. The tools will be divided into the five stages of the work cycle: plan, develop, test, release, and operate. We will cover a different section each time, for a total of six blog posts (including this introductory one).

For your convenience, you can see the complete DevOps ecosystem infographic here:

So, what does each stage mean?

Plan

Planning is the initial stage, and it covers the first steps of project management. The project and product ideas are presented and analyzed, in groups, alone, or on whiteboards. The developer, team, and organization decide what they want and how they want it and assign tasks to developers, QA engineers, product managers, etc. This stage requires lots of analysis of problems and solutions, collaboration between team members, and the ability to capture and track all that is being planned.

Develop

Developing is the stage where the ideas from planning are executed into code. The ideas come to life as a product. This stage requires software configuration management, repository management and build tools, and automated Continuous Integration (CI) tools for incorporating this stage with the following ones.

Test

Testing is a crucial part that examines the product and service and makes sure they work in real time and under different conditions (even extreme ones, sometimes). This stage requires many different kinds of tests, mainly functional tests, performance or load tests, and service virtualization tests. It’s also important to test compatibility and integrations with third-party services. The data from the tests needs to be managed and analyzed in rich reports for improving the product according to test results.

Release 

Once a stage that stood out on its own and caused many a night with no sleep for developers, now the release stage is becoming agile and integrating with the Continuous Delivery process. The Release stage takes place through all the layers of the infrastructure and the company. It includes deployment tools, containers, and release tools, as well as configuration management and Cloud foundation for scalability. The release stage takes us from code to working product.

Operate

We now have a working product, but how can we maximize the features we’ve planned, developed, tested, and released? This is what this stage is for. Implementing the best UX is a big part of this, monitoring infrastructure, APMs, and aggregators, and analyzing Business Intelligence (BI). This stage ensures our users get the most out of the product and can use it error-free.

Obviously, this work cycle isn’t one-directional. We might use tools from a certain stage, move on to the next, go back a stage, jump ahead two stages, and so on. Essentially, it all comes down to a feedback loop. You plan and develop. The test fails, so you develop again. The test passes, you release it, and you get information about customer satisfaction through measurement tools like google analytics or A/B testing. Then, you re-discuss the same feature to get better satisfaction out of the product, develop it again, etc.  

Top Tools to Use in the Planning Stage

Assignment Tracking and Management: JIRA, Confluence, and CA Agile Central

JIRA is a tracking tool that offers Kanban-style, Scrum, or fully customized boards that show the progress of each task, the task assignees, and their development. By using JIRA boards, managers and team members can reflect on the status of different assignments, identify work bottlenecks, and monitor development progress as a whole. JIRA is also helpful for bug management.

JIRA can be complemented by Confluence for optimized assignment management and analysis. Another recommended tool is CA Agile Central, which provides feature tracking and team management abilities, as well as the choice between Kanban, Scrum, or customized dashboards.

• Pro: All tools are easy to work with and enable teams to be in sync.

• Con: Free open-source tools are sufficient for small teams.

Chatting and Instant Communication Between Teams: Slack

Slack is a team communication tool that enable developers, product managers and any company member to chat and share screens. The tool enables personal messaging as well as group chats, channels or flows. Creating groups dedicated to different projects makes it an ideal tool when instant communication is needed while working on assignments. 

• Pros: The tool is user-friendly and visually pleasant.

• Cons: Group messages are sometimes hard to track.

Additional Recommendations

In addition to the aforementioned tools, you might also find the following useful:

• Mixpanel and Google Analytics for measuring mobile and web user engagement. These tools assist you in characterizing user behavior and user flows in your system and analyzing them. This helps you determine which features need to be added and/or fixed, and how. They can be used for A/B testing different features as well.

• Aha! for product roadmap planning. With this tool, you can build a roadmap, create features, distribute tasks, grade tasks and prioritize. The planning work starts with this tool, which can then be integrated with JIRA (see above).

Choosing the right tools is crucial for ensuring that the primary stage of development, i.e., planning, provides a strong base for the rest of the process. You will keep going back to this stage and using these tools, so make sure you choose a tool that best fits your own needs.

Top Tools to Use in the Development Stage

Version Control System Management: BitBucket and GitHub

Version control systems let you host your code, share it with other developers, and collaborate on them. BitBucket and GitHub are two of the most popular version control system management services. Both were launched in 2008, while Bitbucket was acquired by Atlassian in 2010.

Which one should you choose? That depends on your needs. The main differences are that GitHub is open-source friendly, as it enables an unlimited number of collaborators for public repositories. However, if you need private repositories, BitBucket offers them for free for up to five collaborators. GitHub also supports only Git revision control, while BitBucket supports Git and Mercurial. Git, however, is usually enough for developers.

Package Management: NPM 

Package management enables developers to reuse and share code through repositories, which store “packages.” Packages are files or directories that are grouped together to provide a service or functionality. We recommend using NPM, which is a Node.JS package manager. Founded in 2014, NPM is an open-source product.

• Pros: Most packages in one place; easy to use.

• Cons: Requires all packages to be on the NPM website. 

We also recommend using Yarn, which is faster.

Continuous Integration: Jenkins

Jenkins is an open-source Continuous Integration tool that enables developers to automate their tests every time they commit a new build.

• Pros: Open-source; easy to get started with; runs tests in parallel; integrates with many tools, including JMeter and Taurus.

• Cons: Plain UI.

Webpack 

In addition to these tools, we also recommend using webpack. Webpack is a build tool for managing your modules and deciding how all of your assets are processed in your final code bundle. 

• Pros: Supports modules from different syntaxes, which enables running server modules on the client; compresses the code after the build, which saves resources.

• Cons: Not user-friendly.

Top Tools to Use in the Testing Stage

Load Testing: JMeter

JMeter is the most popular open-source load testing tool. JMeter enables users to create a test scenario that contains the actions and requests they want to test on their website or app. Then, the can modify it according to their business needs with elements like ramp-up time and timers, run it for multiple users, and examine the results.

• Pros: Open-source, easy to use, robust, has a vibrant and involved support and development community, has multiple plugins, and integrates with Continuous Integration tools like Jenkins.

• Cons: Requires downloading, limited scalability, limited reporting, and creating a test scenario can be meticulous. 

Load Testing: CA BlazeMeter

CA BlazeMeter has all the abilities JMeter has since it’s an enhanced and upgraded version of JMeter.

• Pros: Scalability (JMeter in the cloud), advanced reporting, collaboration options, and multiple plugins and test recording features.

• Cons: Not open-source. 

CA BlazeMeter reports: 

Functional Testing: Selenium

Selenium is an open-source functional testing tool that automatically tests browsers and enables the testing of web applications.

• Pros: Open-source, supports a variety of languages, operating systems and browsers, records scripts (through Selenium IDE), and works with JMeter.

• Cons: Requires high expertise to use, does not support windows based applications, and lacks online support. 

Functional Testing: Perfecto Mobile

Perfecto Mobile is a functional testing tool for mobile apps that runs automated app tests on real devices.

• Pros: High optimization and accuracy due to running tests on real devices.

• Cons: Testing speed can be slow.

Service Virtualization: CA

CA Service Virtualization creates virtual assets that enable developers and teams to work in parallel on their systems without having to write code to duplicate infrastructure or dependencies.

• Pros: Quick setup and integrates with testing and Continuous Integration tools.

• Cons: On-premise and heavy-weight. 

Top Tools to Use in the Release Stage

Configuration Management: Chef, Puppet, and Ansible

Configuration management lays the groundwork for the app and its dependencies, by tracking and updating information about hardware and software. It can also be used for continuous deployment strategies by automatically distributing changes.

Chef, Puppet, and Ansible are three similar tools that help you build strong and automated infrastructure by IT automation that operates and deploys your infrastructure. 

Which One Should You Choose?

• In Chef and Puppet, the language/DSL is more flexible than Ansible’s YAML files. But, it’s hard to manage multi-DC masters and replication. In addition, these systems need to manage another PKI (public key infrastructure), unlike Ansible which uses an existing infrastructure. This makes the infrastructure management more complex.

• Ansible can run ad-hoc commands, it enables simple reading and writing using YAML files, it’s agentless, and the PKI is already there if you have ssh access to the machines. However, in Ansible, YAML files are strict and prone to indentation errors.

In general, for all config management tools, every change has to run dynamically on the servers, but they might break at runtime. The solution for the runtime issue is immutable infrastructure, so read on to the next section: containers.

Container Runtimes and Orchestrators: Docker, rkt (Rocket), Kubernetes, and Nomad

Containers are tools that assist in keeping releases organized, by delivering the changes in an isolated structure. Once, releasing required sending a code in a separate file that needed to be uploaded to the server. Today, with containers, developers can prepare a container image that contains all the needed dependencies. The container image is distributed to developers and to QA, who don’t need to install anything but can use it straight from the dev laptop to production. 

Container runtimes are tools that run the containers on a single machine. Orchestrators run containers on clusters through the runtimes. Container runtimes and orchestrators help you eliminate the need for single-purpose machines (called pets). Instead, a cluster of machines is treated as one giant machine, thus using the full capacity of your machines. All you need to know is how much compute/disk you need to run your app, and the framework will do the rest to distribute your app and make it available somewhere in the cluster. In any case, you need to add capacity to the cluster you just add new machines.

Docker and rkt are both runtime tools for containers. They leverage Linux kernel features such as CGroups and namespaces, for achieving isolation, security and resource management (CPU, memory, etc.). In addition, they are both very useful for building immutable infrastructure and for pushing the artifacts to a central repository (registry). With them, you build once and run anywhere. Finally, they enable wrapping the application and its dependencies into the same container, with the config mgmt scheme the organization used to separate dependencies (ops) from the app (dev). This ensures scalability of code since the image is immutable and runs the same way, whether it’s one container or 200 containers.

Which One Should You Choose?

• Docker and Rocket have very similar features, even though Docker is more mainstream and it has a large supporting open-source community.

• Rocket pushes to standardize containers, while Docker keeps breaking APIs compatibility with each version as it tries to build its own platform.

Kubernetes is by far the largest distributed computing framework, i.e orchestrator. It orchestrates containers and helps interconnect containers across machines and hosts. It includes secret, config management, deployments, and service discovery.

Nomad is an orchestrator that is a scheduler. It can schedule and distribute binaries, jars, and containers for managing the workload across a cluster of machines. Nomad compares to the Kubernetes scheduler (which is a part of the complete Kubernetes framework). All of the Hashicorp (Nomad’s creators) tools stand in their own right, but can also be tied together. With Nomad you can achieve service discovery mechanism by attaching a consul.io cluster. For secret management, you will need Vault.

• Both tools are very strong and there is no clear choice.

• Kubernetes is supported by a strong open-source community. However, unlike Nomad or any other orchestrator, Kubernetes supports containers-only infrastructure, whether it’s Docker, rkt (by CoreOS), or Hyper through its CRI interface. This means that every process has to be in containers.

Cloud Infrastructure: AWS, Google Cloud, and Microsoft Azure

Cloud infrastructure platforms offer infrastructure as a service. They enable compute, i.e., virtual machines and database services, as well as storage and availability. Through their infrastructure developers and teams can develop, provide services and manage and store their code, products, and applications. One of their main advantages is that they enable companies to lease compute that had to be purchased in the past. This enables companies that grow or diminish to scale according to their needs, without an in-house DevOps engineer.

AWS (Amazon Web Services), Google Cloud, and Microsoft Azure are the largest and most stable cloud infrastructure platforms. AWS is the leading provider that invented the market and has the largest market share, but both Google Cloud and Microsoft Azure are making progress and leveling the playing field when it comes to features and abilities.

Which One Should You Choose?

• AWS has the most functionalities.

• Google Cloud is rapidly improving the number of its features and has the most user-friendly interface.

• Microsoft Azure is improving the number of its features as well as its reliability and stability, their geographical reach is the widest and they have the most special certifications.

Top Tools to Use in the Operating Stage

Application Monitoring (APM): New Relic, CA APM, AppDynamics, and DynaTrace

Application monitoring (APM) monitors the inner parts of the application, including the language’s garbage collection performance, stack traces and the time it took to query databases and additional micro-services. This ensures smooth operation and the ability to track usage trends and system health.

Though there are many successful APMs that we recommend, including AppDynamics, DynaTrace, and CA APM and New Relic, we will focus on New Relic. 

New Relic has a very strong user interface and detailed reports, and above all, it is very comfortable to use. New Relic provides plugins to any language and platform, thus automatically scraping metrics without the developer having to do anything. All the developers need to do is add the New Relic agent to the app they’re developing, and New Relic will automatically extract platform-specific metrics, whether in Java, Node, PHP, etc.

For example, to load the New Relic agent on NodeJS app, all you need to do (after installing New Relic) is to add this code: require('newrelic'); to the app’s bootstrap, while the relevant environment variables: NEWRELIC_APP_NAME, NEWRELIC_API_KEY 
are loaded on that app.

That’s it! New Relic reports will automatically be created and shown to you on the app’s APM page on New Relic’s website. 

Infrastructure Monitoring: Prometheus, Sensu, Zabbix, and sysdig 

Infrastructure monitoring provides insights into your underlying infrastructure and monitors server load, CPU, memory, and more. It also examines connectivity between services, databases, and external systems. If you need to drill down to the root cause, you can also use aggregated logging services such as ELK/Splunk for visualizing events.

The top tools in the market are Prometheus, Sensu, Zabbix, and sysdig. In this post, we will focus on Prometheus.

Why? Because Prometheus is adapted for the container movement, uses service discovery to discover new applications, i.e users don’t need to manually configure new applications for monitoring, and supports the pool-based model. But above all, it is an open-source tool that is owned by the Cloud Native Computing Foundation, and not by a large company, making it a tool by users, for users.