Identifying Non-Functional Requirements (NFR) As Part of Your Agile Project Inception

 NFR workshop in a glance:

NFRs:

In addition to the customer value-adding Epics and User stories you typically brainstorm in story writing workshops, the team needs to consider & plan for how to meet critical non-functional requirements that are also essential to the success of the product. These include things like performance, security, reliability, etc. To truly differentiate your product from the competition, think about NFRs not merely as compliance must-haves, but as distinguishing factors and essential contributors to the value proposition of the product. A big part of why our product is superior to the competition could be because it is more secure, more reliable, faster, etc.  

NFRs include things like performance, flexibility, usability, maintainability, audit, logging, data migration, availability, reliability, recoverability, traffic/user volume, security, globalization/localization, etc.

In practice, we need to look at each of these non-functional requirements and answer 3 broad questions:

• What is our Definition of Success for this NFR? Exploring this question is critical in order to determine how much time and effort we need to dedicate to this NFR. 

Let us take usability as an example: here is an excerpt of the Definition of Success for the Usability NFR from a team I coached recently:

1. the system should be accessible remotely via a virtual desktop
2. users should be able to customize the user interface
3. users should be able to use keyboard shortcuts to access frequently used features
4. response time for the system should be <n seconds
5. user should be able to have multiple instances of the system open at the same time
6. the system should have a usability score on the System Usability Scale (SUS) of 68 or higher.

• What specific actions do we need to take (and when) to address/fulfill the definition of success for this NFR?
• Does this NFR provide guidance regarding how customer-value features should be developed (i.e. something that applies across the board and should be considered when implementing every Epic or user story)?

Let us take information security as another example: In a recent session led by our Information security architect, we used the simple 3-question model mentioned above to brainstorm how we can satisfy the security requirement. We wanted to know what proper security meant in the context of our product (definition of success), what specific actions we needed to take (reviews to conduct, training to deliver, etc.) to meet that definition of success, and how to embed security into how we build Epics and User stories. Here is what we came up with:

• Definition of success: Industry regulations and organizational security standards define clearly (& quantitatively) what good security means (including targets for KPIs such as Mean Time to Identify, Mean Time to Repair, Number of vulnerabilities with high severity, % of features with security acceptance criteria, % security-driven development training completion, etc.)
• Specific actions to be taken (and when) to meet the NFR’s Definition of success:
  • During project inception/sprint 0: Establish high-level security requirements, threat modeling, security training, and security and privacy risk assessment
  • At the beginning of every Sprint (Sprint Planning): Add security acceptance criteria and security test cases at the feature level to ensure that security concerns are identified and addressed
  • Before releasing the MMP:  perform a final (and comprehensive) security review on the product increment
• Security aspects to be incorporated into how we develop Epics and User stories:
  • For a User story to be considered done, it must meet its security acceptance criteria, pass security tests, undergo peer review for security, and the code should be automatically analyzed for security vulnerabilities by a SAST tool.

The ‘work’ – actions, tasks, etc. – we identify as necessary to satisfy non-functional requirements that need to be added to the release plan and counted against our capacity. It is unrealistic to allocate major migration activities, for example, to a Sprint that is already at or near its full capacity (with all the User stories that have been allocated to it). Who is going to do that work? Will they have enough capacity to carry it out? What if they do not have the capacity? Do we move some User stories from the Sprint to clear some space for the NFR actions? Do we simply move the action to earlier (or later) in the release? Etc. Questions like these should be discussed during the Inception Sprint/Sprint 0. Unfortunately, what happens all too often is that teams allocate NFR fulfillment actions to sprints without giving much thought to how complex and time consuming these actions could be. With the time crunch of Sprints, many teams choose to defer these all-too-important NFR considerations to later - usually to the very end (we’ll worry about usability later, for now, let’s just focus on getting it to work), often causing major delays and re-work. 

Corporate Constraints & Enablers:

In addition to NFRs, the team is constrained by external factors that need to be covered off. Things like media and marketing for release, post-deployment support, and operations, corporate obligations, etc. Even if the MMP is working and ready, if we fail to consider things like marketing, operational readiness, etc. then we will be delayed, and we will not be able to deliver value to our customers on time. Just like NFRs, plans to address these external factors and constraints should find their way into our roadmap. We need to add work items that represent what we must do to address these constraints.

Preparation For This Workshop:

Prior to the activity, make sure that the appropriate SMEs have accepted your invitation to attend. Investing time in inviting the right SMEs and ensuring that they have the background needed to contribute effectively pays off well during the workshop.

Workshop Facilitation Guide:

• The facilitator writes the following NFRs on post-it’s (one NFR per post-it) and places them on the board: performance, flexibility, usability, maintainability, audit, logging, data migration, availability, reliability, recoverability, traffic/user volume, security, globalization/localization 
• Discuss with the team what additional NFRs to add to the list above (and what NFRs to remove because of irrelevance to the context of the project)
• The team is then split into small groups, each tackling one or more of the NFRs. Each group should be facilitated by a domain SME relevant to the NFR subject area (e.g. the small group analyzing security should be facilitated by a security expert)
• Team members self-select the NFR groups they want to be a part of – the facilitator should encourage self-organization, but stress the need for roughly equal representation in NFR groups, and also that business and Technology should both be represented in all groups
• Even though an expert in facilitating the conversation in each group, the WHOLE group (including the business, obviously) should be involved in brainstorming & decision making. Decisions about moving user stories from sprints to free up capacity, for example, cannot proceed without the business approval
• The facilitator should first work with the group to establish how success will be measured with respect to the NFR. It is best if the expert/facilitator offer their opinion last to ensure that others have an opportunity to provide perspective as well (e.g. the facilitator might choose to have the team silently brainstorm first before discussing results)
• Next, the facilitator and the group come up with a definition of success for the NFR (specifying goals/targets per success metric) – e.g. for usability, the group might choose SUS >= 68, CES >= 4.5, etc.) Targets should not be set arbitrarily. The facilitator should lead a thoughtful discussion about what kind of investment we need to put in to achieve certain targets (e.g. what does it take to have a SUS higher than 70, which is at the 90^th percentile of similar products) – providing a perspective around investment, what competitors are doing etc. will help the team set good stretch targets
• Once the group is clear on their definition of success for the NFR, the attention turns to brainstorming in two areas:
  • What actions do we need to take to achieve this definition of success? Do we need to undertake certain training? Use certain tools? Organize regular check-ins or review meetings? Perform certain operations? Etc. The group brainstorms all actions necessary to accomplish the definition of success. The group then tries to allocate these actions to Sprints or months – similar to how user stories were allocated to months/sprints. When mapping actions to sprints, the group takes into account the Sprint capacity (for all feature teams) and the complexity/effort needed to implement the action and assess whether it is realistic to assume that the Action could still be carried out despite everything else the feature teams have to do in the Sprint. If not possible, the group then considers whether the NFR action could perhaps be moved to a later sprint, one that has more capacity. If the action were deemed too critical and could not be postponed, the group notes that and brings it to the entire team to discuss and decide (the group cannot move user stories on their own without discussing with the whole team first)
  • Does this NFR provide guidance regarding how customer-value User stories should be developed? For example, for information security, the group might explore how adding security-focused acceptance criteria to every story helps achieve the definition of success of that NFR, for example
• When the timebox expires, each group presents the work they have done to analyze NFRs, the definition of success, and most importantly discuss NFR action to sprint allocations and whether the teams agree that capacity is available (or not available) in certain sprints to accommodate NFR actions. The whole team decides on the final mapping of actions to Sprints

The whole team also reviews what each group has come up with regarding how the NFR provides guidance on developing features and what each group proposes should be part of how features are developed (e.g. the security NFR group proposes that security becomes part of the acceptance criteria; the usability group proposes that before each feature is considered done it has to go through a usability test, etc.) – again, the whole project team makes the final decision regarding which of these proposals should be adopted. The proposals the team adopts will later feed into the Definition of Done they build collectively in a future exercise.