How Your Secrets Management Maturity Can Impact Your DevOps Research and Assessment Metrics

Most folks managing or working within a DevOps organization are already familiar with the book "Accelerate" and DevOps Research and Assessment, commonly abbreviated as DORA, metrics. For those who are not familiar or just need a quick refresher, DORA metrics categorize the performance of a DevOps organization based on four key leading indicators:

1. Deployment Frequency: The cadence of an organization’s successful releases to production. The more often, the better.
2. Mean Lead Time for Changes: How long it takes a developer's commit to get into production. The more, the better.
3. Mean Time to Recover: How long it takes a team to restore service in the event of an unplanned outage or another incident. The shorter the time interval, the better.
4. Change Failure Rate: How often making changes lead to failures in production. The smaller the percentage, the better.

DORA metrics help you quickly gauge the overall health of your team and output. There are a lot of metrics that influence these indicators, they are a handy way to think about DevOps performance from a high level. This methodology groups organizations into one of three tiers for overall performance: Low, Medium, or High.

You can see the matrix of metrics to performance in the table below.

Secrets Management Metrics

You might be asking, "Where does code security, specifically secrets management, show up in DORA metrics?" Perhaps a better question might be, "How can we make our code more secure with regard to secrets while also delivering our applications as a High performer?" The secret here, like so many things in DevOps, is in "shifting left."

But first, we need a way to measure code security posture. GitGuardian has come up with the Secrets Management Maturity Model, a quick roadmap to help your organization make sense of its security posture around credentials and how to improve.

In our model, we map four major areas of the SDLC:

• Developer environment
• Source code management
• CI/CD pipeline and artifacts
• Runtime environments

From there, one is able to evaluate a team's current maturity on a five-level scale for each of these areas, with the intention of moving to the next level. The full report goes into much greater detail, but here is a quick summary of each level:

0. Uninitiated: Secrets are in plaintext, and no secrets detection is in place

1. Beginners: Secrets are unencrypted in config files, and occasional secrets detection scanning is manually triggered.

2. Intermediate: Secrets are stored in a vault and shared through a secret manager, and continuously scanned at the pull/merge requests stage.

3. Advanced: Secrets are stored in a vault and shared exclusively through a secret manager, and there is a well-defined secrets rotation policy in place. Scanning before pushing code (pre-commit/pre-push) is adopted by security champions with developers systematically involved in the remediation process.

4. Expert: Secrets are stored in a vault with access controls and logging. Dynamic secrets have limited scopes, and all repositories are continuously monitored. Remediation workflows are automated, and developers directly fix issues.

How Does This Relate to DORA?

Now that we have a common vocabulary to communicate about secrets management, how does your it impact DORA?  Let's look at each level of the SMM and walk through some ways it can impact your flow, which is another way DORA talks about performance.

Level 0: No Plan in Place

Since the uninitiated are not watching for it, it might not seem to affect the bottom line at first. That changes suddenly when there is a security incident. Not having a plan around secrets management in place can impact your Mean Time to Recover profoundly as the team scrambles to deal with the incident.

The time it takes to "focus on security" after an incident is going to be a giant drag on your Deployment Frequency and Mean Lead Time for Changes. Determining how to rotate the affected keys and how to refactor the code to prevent subsequent breaches takes a lot of attention and is stressful, especially after an incident.

Level 1: Beginner

At this level of maturity, when secrets detection scanning does take place, it is late in the software development lifecycle, in the pre-launch testing phase. When detected this late, it is going to cause the flow of work to reverse, as the code failing the tests will be sent back to the developer to refactor and deal with the exposed secret. This backlog of fixes coming from failed tests means Deployment Frequency and Mean Lead Time for Changes are both going to suffer.

Here is a quick illustration of what failed tests do to a work backlog:

Level 2: Intermediate

At this stage, since you are securely storing secrets regularly and scanning for secrets with each PR, then you are catching any errors earlier than the testing phase, lessening the impact on your team's flow. However, inconsistency means there will be times errors are caught too late, blocking work and hindering your backlogged items from progressing. Organizations in this situation typically hover in the Medium performer category as they strive to become High performers.

Level 3: Advanced

While even an occasional secret-related incident will affect Deployment Frequency and Mean Lead Time for Changes, overall, you are going to be dealing with them much faster and likely either already a High performer or well on your way. The goal for folks at this level is to standardize practices throughout all teams and to automate work like secrets rotation and the developer feedback cycle when incidents do pop up.

Level 4: Expert

At this stage in your journey, you might occasionally hit sporadic secrets-related incidents, but those will be exceptions to an otherwise smoothly-flowing DevOps organization. At this level of maturity, each developer will automatically scan for secrets, using a tool like ggshield run via git hooks before each commit is completed. This practice means secrets very rarely make it further than the developer's terminal.

At the same time, your org will have the proper processes in place, so it will only take a few short moments to appropriately modify the code and manage their secrets, making no noticeable impact on Deployment Frequency or Mean Lead Time for Changes. This is the ideal we should all be working toward.  

Secrets Are Part of the Larger Equation

There are many factors that affect your DORA metrics score. Secrets management maturity is just one of the factors, but an important one for any DevOps organization to consider. Beyond just velocity, it can also impact your company's bottom line when breaches occur. It is critical to get a handle on hardcoded credentials throughout your code, no matter where it is in the software development lifecycle.