Dead Letter Queue for error handling in Apache Kafka: Alternatives, best practices, and case studies from Uber and Crowdstrike.

14 First Principles to Help Your Team Succeed

The agile testing methodology approach in software testing aims to enhance the flexibility and efficiency of the software development process.

Remember mainframes? Serverless is pretty much that: we own the machine and you rent out time on our big iron. We went full circle on progress!

10 Simple Principles of a Critical Scrum Success Factor

Software companies can accelerate their business and deliver reliable products by adopting site reliability engineering principles.

Ian Nowland of DataDog speaks with me about how to be a stellar engineering manager and gives some tips on why "mistakes" are more valuable than you realize

Agile Math The basic math of team-based agile is pretty simple. You can slice it several ways, but at the end of the day, one of these three basic formulas has to hold true. It’s all about time, cost, and scope… you get to decide which two constraints you want to lock, but then you have to derive the third. 1. backlog size / velocity = duration 2. duration * velocity = backlog size 3. backlog size / duration = velocity I generally suggest that agile is all about fixing time and cost, and deriving scope… but it doesn’t have to be that way. Feel free to derive time-based on a fixed backlog and known velocity. You can even derive a planning velocity based on fixed scope and time. This one is the riskiest, so be prepared to measure, adjust, and negotiate as the plan unfolds. Limiting WIP But here's the rub… when a team has too much work to do, and not enough time to do it, there is a cognitive dissonance between the messages of agile and what they see on the ground. We can say all day long that the PO gets to decide the “what” and the team gets to decide “how” and “how much”… but if management is fixing all three variables, the team isn’t going to buy in. Putting the Right People in Place One of the biggest mistakes that people make when working on any coding project is not having the right people in the right spots to help them out. It is absolutely necessary to use the talent and resources that are available to you in the most effective ways possible. Doing anything short of that can lead to major issues that you do not want to deal with. Instead of taking a risk, make sure you look at the pool of talent you have beforehand and begin to reassign people based on the skills that they clearly possess. You may be able to connect just the right pieces where they need to go in order to place people in the correct spots where they can be the most effective possible for you. If this is the case, then you will be in good shape when the time comes to use those people to get certain missions completed. You are responsible for putting people in a position where they can be as helpful and useful to you as possible. Observe their strengths and weaknesses to try to figure out exactly where that spot is. This may take some time, so make sure you have budgeted enough time for yourself to get these kinds of things figured out. It won’t always be easy, but it is the kind of work that you need to do to see real results on your projects. Rushing the Backlog Generally, here is what I ask from management out of the gate… give us three sprints to help the team come up with a backlog and establish a velocity, afterwards we’ll see what we have and decide how to proceed further. We’ll start by doing just enough backlog planning to identify a sprint or two worth of work and get the team working to establish a velocity. While the team begins work to establish their velocity, the PO aggressively moves to create the backlog. Almost never do I see a PO that can create a backlog all by themselves. Very often we need Product Managers, Architects, and Analysts to paint the complete picture. More often than not, I’ll ask these folks to work full time for as long as it takes to get the backlog together. I’ve got one PO team that has been at it for 8 weeks just to get ahead of the team, and define the release. Initially, the PO team is focused on feeding the team's high-value, high-risk stories… but as the backlog emerges we start rounding out the app. If all goes well, after several sprints we have a decent idea of what we have to build and the rate at which the team can complete the work. At that point, we apply one of our three formulas, baseline the plan, and go. Emergence or Convergence How far ahead of the team you need to be, largely depends on your business goals for the release. If you are highly uncertain about what you need to build, smaller backlogs are probably better, and the release planning process can be more nimble. Trying to predict stuff you just don’t know is a waste. In this case, agile is helping support an emergent outcome. Not all companies are going for an emergent outcome… Some want stability and predictability. In these cases, the PO team needs to plan further ahead of the team and adjust as the product is developed. The better we know where we are going, and what it is going to take to get there, the further out we can plan the backlog, and the more certain we can be about outcomes. Here agile is supporting a convergent outcome with a focus on risk reduction and predictability. One of the biggest problems I see with teams new to agile is that they act as if they are going for stability and predictability, when their product requires an emergent approach. Either requirements are not well understood or because of high technical risk or a ton of unknowns around how to implement them. Either way, you have to act as if the project is emergent until you gain enough knowledge to establish a more predictable plan. Not Knowing What You Don’t Know I’ve met a few teams lately where everyone is new and unfamiliar with the product and the code base. How do you set a schedule in this environment? The short answer is… you don’t. It’s okay not to know, but it’s not okay not to know forever. In this case, you better have a plan to get it figured out fast… It’s not reasonable to indefinitely ask the business to invest with no strategy for getting it done.

I wanted to explore the possibility of using JBehave to formalize scrum's definition of done. The idea being to encapsulate a definition of done as a JBehave scenario. So in true scrum style I decided to timebox 4 hours of work dedicated to JBehave. From a scrum point of view BDD can be used to turn the definition of done into a test artifact. The team produces scenarios for each task. With JBehave a scenario file describes the required behavior and test steps it will need to pass to be considered done. I.e Given some prerequisites, perform some action and expect some results. See the JBehave project for more detail as this is only a simple example. Why Examine This Topic? It is useful to examine anything that a programmer may use in the course of doing their work. Not only should we challenge the assumptions that are floating around out there about different programs, but it may also be useful to look at the resources that are available to us and ask ourselves if we are truly using them in the most effective ways possible. There is always the chance that just taking a hard look at some of the elements of the work that we do may help us understand where it can be improved upon when the time comes. Indeed, many people find the concept of examining such things quite appealing, and that leads to major research into these topics. There are a lot of people making incredible strides towards getting things done that the world never believed was possible, but those changes are happening rapidly, and we should all be thankful that this is the case. BDD in 4 hours? So scenarios are just text files describing the required functionality in terms of expected behavior. Even before development though, it's possible to run them and see the pending expectations. Let's wizz through a simple example showing who in a scrum team is responsible for what. I already have a project that I use to explore test patterns so I thought I would just introduce it to that. If you want the working version just take a fork from GitHub. Nb. I have a data fixture used in testing a complex page component. So whilst not a pure scenario I can make its existence part of the Givens. In this project, it's likely that no real given is required, but a developer would add it during the build. First the product owner and tester produce a simple scenario file describing behavior and expectations. Given a data fixture When a page is created with 20 items on a page Then expected lines on a page is 4 with data items in a line 5 Next developers write a harness and flesh out the steps required to meet the behavior. We need to bootstrap the scenario and add steps to it. The general idea is to have a scenario test, that then includes the steps. This is one way of bootstrapping in the steps, other ways are available including a spring system that uses SpringStepsFactory. I ran into problems with the spring system as my project uses spring 3 and JBehave is still locked into spring 2.5.6. In addition, I am using junit 4.8 and spring 2.5.6 requires 4.4. This is ultimately an issue as on many projects. I would not want to tie in those older versions. In fact, other ideas in my project prevent me from doing this too. So in my timebox, I avoid the issue and bootstrap manually. The Bootstrap test class public class SimplePageScenario extends JUnitScenario { public SimplePageScenario() { super(new MostUsefulConfiguration() { @Override public ScenarioDefiner forDefiningScenarios() { return new ClasspathScenarioDefiner( new UnderscoredCamelCaseResolver(".scn"), new PatternScenarioParser()); } }); addSteps(new SimplePageSteps()); } } Before adding the call to addSteps()) running the scenario shows the steps that are pending. (org/testpatterns/scenarios/example_scenario.scn) Scenario: Given a spring data fixture (PENDING) When a page is created with 20 items on a page (PENDING) Then expected lines on a page is 4 with data items in a line 5 (PENDING) Finally the developers fill in the Steps class adding sections that match against the scenario keywords, Given, When and Then. Development is done until the behavior passes the test. public class SimplePageSteps extends Steps { private SimplePage page; private final PageDataFixture dataFixture = new PageDataFixture(); @Given("a data fixture") public void givenASpringDataFixture() { notNull(dataFixture); } @When("a page is created with $itemsOnAPage items on a page") public void createPage(int itemsOnAPage) { page = SimplePage.newInstance(dataFixture.getDataItem(), itemsOnAPage); } @Then("expected lines on a page is $linesOnAPage with data items in a line $itemsInALine") public void validatePage(int linesOnAPage, int itemsInALine) { Map expectedLines = dataFixture.createPageExpectation(linesOnAPage, itemsInALine); Map actualMap = page.getMap(); ensureThat(actualMap, is(expectedLines)); } } Good Points I like the potential to get key scrum members interacting. The collaboration between the testers and the product owner is an important one as it should flesh out questions about behavior at an early stage. Similar benefits occur when the development team starts to get to grip on the behavior. Having a simple scenario artifact means the product owner can get involved with something that actually becomes part of the continuous integration. Bad Points The bootstrap is clumsy. Having to add steps means that after producing the new scenarios development has to get involved before the pending steps appear. There are other ways to bootstrap. The spring system is a step in the right direction, in fact I would like to use injection throughout the application. This still does not solve the problem of having to edit something before the steps appear. I would prefer that once the scenario is in place, steps are automatically found as they are written, and appear as not started until they are cut. Extending the Junit Testcase is also a problem. It's highly likely that in more complex projects we would need mocking techniques or spring-based tests. These have their own runners brought in with the @RunWith annotation. It would be better if there were some way to bootstrap the behavior into any other test framework. In my 4-hour timebox, I did not get the reporting working. Unfortunately, the reporting requires more coding, and setup and is far from being just a bit of maven configuration. When running from maven, things didn't work until I configured the maven-resource-plugin to copy over the scenario files. This should have just worked out of the box. Most of my 4 hours were taken up with configuration which was compounded by the slim documentation. Conclusion Attempting a behavior-driven approach using scenarios is worth doing. I could see the benefits. Not sure if JBehave is the solution for me though. Difficult to configure, and incompatibilities with other frameworks would put me off at the moment. I will find some time to have a look at easyb another Java BDD system next.

How do you get ahead of tech debt before it piles up? And how do you deal with tech debt you already have, without sacrificing velocity on new projects?

This blog is an extended insight on the real purpose of Sprint Goals and how clearly defining and utilizing them can stack a business with benefits.

Making Your Scrum Work #27

When things go wrong, who is in charge? And what does it feel like to do that role?

Making Your Scrum Work #26

A major challenge facing organizations is on-call. Incidents are inevitable in complex systems. Automated incident response can’t handle every problem.

In this article, we will define the “bare minimum” set of metrics, talk about product success metrics and discuss what can be considered a true agile measure of success.

Learn about crucial things about compatibility testing that you should know. Here are key points to remember about compatibility testing.

Making Your Scrum Work #25

This article will help you understand the top software methodologies, their advantages, pitfalls, and the difference between them.

This article hopefully gives you ample working knowledge on the concept of the DaD framework, history, lifecycles, principles, pros, cons, and more.