The Differences Between CI/CD and CR and How to Instantly Pinpoint Issues in Pre- and Post- Production

It seems like we’ve entered the era of continuous everything – Continuous Integration (CI), Continuous Delivery/Continuous Deployment (CD), Continuous Reliability (CR). The list goes on… But what exactly is the difference between them? What do they actually look like in practice? What are the popular tools that make it all happen? 

Development teams are under immense pressure to push daily code changes to keep users happy with new features. With each code push to the repository, or development branch, CI will automatically and continuously test your apps by running them through various code exercises to make sure this continuous code push doesn’t break anything in production.

Let’s back up a step. Before new developer code gets pushed to the repository and through CI, development teams will conduct manual code reviews and run the code through static analysis tools.

Some common DevOps tooling with testing includes: CI (e.g., Jenkins, Bamboo), static testing (e.g., Sonarqube, Coverity), and GitLab (repository, CI, issue tracker, etc.). The point of CI is simple, your code must pass all tests, guidelines, compliance, and regulatory checks you have set up for your application before it goes to production. Despite all the testing in CI, code still breaks because tests just can’t detect 100% of errors. If CI could detect 100% of errors, the term hot fix would not exist. What a world that would be!

Now we have the application, or feature update, built and tested - so who is going to push it to production? If you want manual intervention and require a human to make the final call, continuous delivery is the next step in the process. For those looking to automatically push CI-approved code straight to production, continuous deployment is the next step in the process. To confirm, the main difference between C-delivery and C-deployment is that the former is a manual push and the later is an automatic code push to production.

Now we are live. Production! Nothing bad can happen here, right? If only. DevOps teams must deal with logged exceptions, swallowed exceptions, and uncaught exceptions (defects) so they turn on the logs, error trackers, and APM tools. 

Did you know that, of the billions of daily Google searches, Google hasn’t seen 15% of those before? Change is eternal my friends. This is why code breaks. The world delivers the unexpected into our applications, and we hope our logs, error trackers, and APM tools are good enough to help us fix what breaks.

Even a logged exception doesn’t guarantee that your application is working well though; only that the exception is logged. At least your developer wrote the code properly to catch and log the exception!

But what about when they don’t? Swallowed exceptions are caught and handled, but no data is emitted so the error is essentially invisible. Who wrote that code anyway? For uncaught exceptions, your developer wrote code improperly and did not prepare for an exception, handle, and way to log it. Let’s call a spade a spade here, not all developers are equal and sometimes a bad developer can make things worse and the logs, error trackers, and APM tools can’t magically make them better at coding. 

Yes, most of us have Splunk, Elastic (ELK Stack), Sumo Logic or a handful of others for logging errors. AppDynamics, Datadog, Dynatrace, or New Relic for APM. Sentry, Rollbar, Raygun, Bugsnag, Airbrake, and others for error tracking.

Despite all of it, developers can struggle to find and fix errors. We can get lost in the logs, APM tools, and error trackers due to a lack of context. When applications are emitting thousands, millions, perhaps even billions of data points, it can be impossible to sift through all the noise to find the signal.

That’s where OverOps helps: with pinpoint accuracy of the issue cause. OverOps makes your developers better with instant, deep insights on where, when, and why code breaks — and who wrote it. OverOps provides complete context to resolve every error and can even Slack the developer who wrote the code. 

OverOps is a dynamic code analysis solution that seamlessly integrates with your existing CI/CD pipeline to uncover all of the unknown errors in your applications and help you achieve the broader goal of continuous reliability. Unlike static testing, log aggregators and APMs that rely on foresight, OverOps automatically analyzes code anywhere it runs, all without code changes in an effort to identify, prevent, and resolve critical software issues, before the customer is impacted. 

For our visual friends, I offer this graphic:

An OverOps infused CI/CD pipeline offers increased developer velocity. In other words, your team will literally do more with the same number of devs! You will knock out more story points to work on value-add and not remediation & legacy code. Reduction in MTTI (average OverOps customer sees a 70% reduction) and MTTR means that customers are happier and you’re saving money by not wasting developer time sifting through the logs and delivering multiple hotfixes in hopes of fixing the same problem.