Boost Chapter Performance With the Chapter Health Diagnostic (Part 1)

Before We Start, What’s a Chapter?

A chapter is a craft community (sometimes also referred to as a Practice or CoE) whose primary purpose is to help practitioners of the same craft or discipline (e.g., development, testing, business analysis, scrum mastery, UX, etc.) elevate their mastery of that particular craft or discipline. Chapters are also typically tasked with setting the standards and guidelines for performing that craft in the organization. 

Credit: Ashley-Christian Hardy, “Agile Team Organisation: Squads, Chapters, Tribes and Guilds,” 2016

TL/DR

• In Agile organizations, chapters (Chapters, Practices, CoEs, etc.) pursue the systematic development of capability and craft. This pursuit adds a lot of value to the organization (better quality, better knowledge retention, higher employee engagement, etc.).
• Chapters often struggle to pinpoint where they need to improve or how they can add more value to their members and the organization. Many organizations don’t offer clear guidelines to chapters (and chapter leads) as to what exactly is expected of them or what good looks like.
• This article presents a simple diagnostic (click to download the diagnostic) that a chapter could use to identify areas where they must focus their improvement efforts. It defines what ‘good’ looks like in the context of a chapter and provides a tool to help the chapter assess where they stand against that definition (where they’re doing well and where they need to improve).
• In the second part of this series, I will share several experiments that could be run to optimize each dimension of the chapter's effectiveness. I will also share a case study of how this model was implemented at scale at a large organization.

Key Terms

First, let’s define some of the key terms that will be used throughout this article:

• Craft refers to the specific discipline, domain, or skill set around which the chapter is formed. e.g., QA is a craft; UX is a craft; business analysis is a craft, etc. A craftsperson is a practitioner of a craft (developer, QA specialist, marketer, business analyst, etc.)
• I use the term performing the craft to refer to the actual day-to-day carrying out of tasks by a craftsperson (chapter member) – e.g. a QA specialist performs their craft by carrying out QA tasks.
• Craft quality (quality of the craft) refers to how well the craft is being performed. I sometimes use craftsmanship, which refers to craft mastery and excellence.
• Knowledge base refers to a centralized repository or system where craft-related information, best practices, documentation, standards, etc. are stored and shared among chapter members (and others across the organization).
• Standards (craft standards) refer to the established guidelines and principles that define the expected level of quality within a specific craft.
• Learning journey refers to the ongoing formal and informal learning efforts (training programs, hands-on application of new knowledge, mentoring, etc.) intended to extend existing skills and build new skills, and how that learning is expected to be acquired over time.

Is It Worth Reading This Article?

Well, if any of the following statements resonate with you, then I would strongly suggest that you read on:

As an organization (or tribe, business unit, etc.):

• “We have a high risk of losing critical knowledge if a small number of our people leave”
• “We struggle with onboarding and, despite hiring top talent, it’s difficult for them to hit the ground running”
• “Despite hiring people with a lot of experience to mentor and grow our junior staff, we feel that knowledge sharing isn’t happening”
• “We invest a lot in training our people – sending them to courses, etc. – but we don’t feel that investment has had much impact on the overall quality of our products”
• “We have knowledge siloes even within the same discipline — there are islands of expertise that aren’t connected”
• “We fear that when the contractors and external consultants leave, our people won’t be able to deliver the same level of high craft quality”
• “Team members struggle when moving from one team to another due to a lack of consistency in how things are done”
• “Our team members seem to struggle to tap into the collective expertise of the team, leading to a lot of re-inventing the wheel and time wasted”

While these are all difficult problems that result from complex interactions of causes that affect and are affected by every aspect of how the organization works, these issues are all heavily influenced by how effective we are at developing craftsmanship — that is, how good we are at developing craft mastery and excellence. Given that in Agile organizations craft communities (chapters, practices, CoEs, etc.) are the primary custodians of developing craftsmanship, what I’m proposing here is that by systematically assessing and optimizing the effectiveness of how these craft communities work, we can make great strides at addressing these challenges.

So, Why Care About Chapters?

Effective chapters create the conditions that enable high product quality, high employee satisfaction, and low knowledge loss risk. This is because effective chapters are good at developing master craftspeople. People who feel mastery of their craft are typically happier and more engaged, their output and the speed at which that output is delivered is superior, and, due to the fact that there’s more than a small number of them (and that there’s a robust process to develop more), the departure of a few won’t be catastrophic for the organization.

Members of an effective chapter (that is, a chapter that’s good at developing the craftsmanship of its members), would typically say things like:

1. Our chapter follows a systematic approach to building our capability and craft mastery (defining what capability needs to be built, providing its members with the mechanisms to plan how to build those capabilities, and providing the resources and support needed to implement that plan)
2. Our chapter has in place the process and systems that enable us to leverage and build upon the accumulated formal knowledge that the chapter has amassed over time – the standards, playbooks, guidelines, best practices, lessons learned, etc.
3. Our chapter has nurtured a rich social network that enables us to tap into the collective informal (tacit) knowledge and expertise of the chapter – the knowledge, nuances, and highly contextual experiences that aren’t documented anywhere (most knowledge is tacit)
4. Our chapter follows a systematic approach to measuring the impact (outcomes) of craftsmanship-building and capability uplift efforts and leveraging the feedback to guide further craftsmanship-building efforts

If we improve the effectiveness of a chapter (that is, optimize the 4 factors mentioned above that are key leading indicators to chapter effectiveness), we would increase the chapter’s ability to develop its members into craftspeople, which will positively affect and improve problems such as high knowledge loss risk, siloes, ineffective knowledge sharing, and low product quality.

How Do We Improve Chapter Effectiveness?

The first step to improving chapter effectiveness is to systematically assess how the chapter is performing against the 4 key dimensions of chapter effectiveness identified above (access to documented (formal) knowledge; systematic capability building; access to tacit (informal) knowledge; and systematic craft quality measurement and continuous improvement). 

In this 2-part series, I will introduce a simple diagnostic tool to assess chapter effectiveness (Part 1 – this article), and then delve into how to use the insights from the assessment to identify areas of improvement and how to go about setting chapter effectiveness goals and planning, implementing, and learning from chapter effectiveness improvement actions (Part 2).

How Do We Measure Chapter Effectiveness?

In this section, we will first go over the dimensions comprising it in some detail, and then present the diagnostic tool itself.

Chapter Effectiveness Dimensions

Dimension #1

The comprehensiveness, fitness-for-purpose, and ease of access (and use) of our craft standards and knowledge base – this is a broad area that covers things like how good we are at leveraging (and creating/documenting) existing knowledge, the ease of access to relevant knowledge, the accuracy and relevance of the knowledge chapter members can draw from (and its alignment with industry best practices), and the effective distilment of ‘lessons learned,’ which represents how outside knowledge is contextualized to fit the unique context of the organization, among other factors.

Dimension #2

The effectiveness of the chapter’s effort to uplift the capability and craftsmanship of its members — effective chapters are good at describing what mastery of their craft means (what skills to acquire, what the levels of mastery of each skill look like, etc.), helping chapter members pinpoint where they are on that journey, and then collaborate as a team to envision what the path to mastery looks like for each member. They’re also good at taking those plans and turning them into reality: not only providing the resources and mentorship, but also the encouragement and peer support, keeping each other accountable, and measuring the outcomes of their elevated levels of craft mastery.

Dimension #3

The effectiveness of tacit (informal) knowledge sharing between chapter members – Effective chapters realize that most knowledge is tacit – that is, not documented anywhere. Tacit knowledge is difficult to extract or express, and therefore difficult to formally document or write down. How do we effectively leverage knowledge that isn’t documented? By nurturing a thriving social network that allows chapter members to feel comfortable reaching out to each other for help, seek advice, ask questions, share interesting insights, etc. Such a network doesn’t just happen – it requires conscious, persistent effort to build. The statements comprising this dimension seek to explore some of the leading indicators to building effective knowledge-sharing and advice-seeking social networks.

Dimension #4

The effectiveness of the chapter’s efforts to systematically and continuously improve craft quality – how do we know if the actions we’re undertaking to uplift the quality of our craft (committing to learning and capability uplift, fostering stronger knowledge-sharing networks, etc.) are delivering value? How do we know if the investment we’re putting into uplifting our capability into specific tools or frameworks is generating the returns expected? Effective chapters are really good at measuring and evaluating the quality of their craft across the organization (quantitatively and/or qualitatively). They’re good at setting SMART craft improvement goals given their understanding of how well the craft is being implemented and where they need to focus and invest in improvement, planning how to implement those goals, and good at implementing those plans (and learning from that implementation). This is a significant challenge area for many chapters, as it is often difficult to ‘measure’ the quality of how the craft is being implemented.

The Chapter Effectiveness Diagnostic

Introduction

The diagnostic (click here to download the pdf version) comprises a number of statements that are intended to capture what ‘good’ looks like for that particular dimension. Chapter members are expected to rate how well they believe each statement describes the reality of their chapter on a scale ranging from 'completely disagree' to 'completely agree.'

All chapter members (including their chapter lead) should take part in completing this diagnostic. One option to fill it (what many chapters do) is to send it out as a survey first, then discuss results or insights in one or more follow-up workshops.

The purpose of this diagnostic is to serve as a conversation starter. As with all diagnostic and maturity models, the questions are merely intended to prompt us to have a discussion. The comments, anecdotes, and insights chapter members share as the team moves from one item to another provide a wealth of information. That information is what’s going to guide us (as a chapter) as we attempt to optimize the outcomes our chapter is creating. There’s no particular magic to this (or any) assessment model – it simply provides a structure within which good conversations can take place. 

What’s in the Pack?

This pack contains the statements comprising this diagnostic model. Next to each statement is a brief explanation of why having a conversation about that statement is important and what to look for (and how to dig deeper and uncover insights) if the score against that particular statement is low.

In the appendix, you'll find a printable version of the diagnostic (a template with only the statements), which can be distributed as handouts.

Next Steps

If you want to run the diagnostic as a survey, copy the statements into your survey tool. You may set the response options for each statement as completely disagree — disagree — neutral — agree — completely agree. Alternatively, you might opt for a sliding scale of 1-5, for example, or use whatever rating method you prefer to enable your team to assess how well each statement describes its reality. 

OK, We Ran the Diagnostic – What’s Next?

As mentioned earlier, the conversation that follows this self-assessment is where we really get the value. As a team, the chapter gets together to reflect, explore, and try to make sense of the results of their chapter effectiveness self-assessment. They reflect on where they seem to be doing well and where they’re struggling, where they seem to all have the same experience, and where the scores reflect a difference in their perceptions. They reflect on common themes, outliers, relationships, and connections between statements, explore why some statements are not correlated even though they were expected to (and vice versa), and any other interesting insights that came out of the assessment.

In the second part of this series, we will do a deep dive into how to translate these insights and learning into experiments and actions and measure the impact they create in optimizing chapter effectiveness. We will explore how to prioritize chapter effectiveness interventions, what experiments to run to uplift each chapter effectiveness dimension, and how to establish a robust continuous improvement cycle to consistently and systematically seek higher chapter effectiveness. We will go through a case study from a large financial organization where this model was applied at scale across a large number of chapters, and share some of the learnings from that experience.