Implementing Promotion Bias Checks in Engineering

Perhaps the easiest way to reduce bias in promotions is to use a promotion bias spreadsheet. 

You can use this spreadsheet in promotion decision meetings to show, in real-time, who is getting promoted. This can help highlight bias, and result in better decision-making.

Create a Promotion Spreadsheet

To start with, you need a spreadsheet you can use during the promotion review meeting. Here’s how to set it up:

1. Create a spreadsheet with everyone in the organization being promoted.

2. Add a column for under-represented minorities (URM). Yes or No.

   • Note: this is highly imperfect. Determining if someone is a URM or not isn’t easy. There are intersectionality issues that this doesn’t account for. Gender isn’t binary. Etc. You can take that into account if you want, but you might want to do that in later iterations after you have some practice with this. 
   • You probably should consult with your HR department about this. I’ve seen companies simplify this by using preferred pronouns to indicate URM status (but that ignores too much in my opinion). There aren’t any easy ways to categorize people.
   • I’ve done it anyway, despite these complications. The aim of this is to do a real-time check on your decisions and make you think about them more carefully and consciously. 
3. Set up the spreadsheet to show the percent of promotions that are URMs, versus non-URMs. Add graphs if you want to be fancy! 
4. You can do all sorts of interesting things with this: promotions by level, promotions by tenure at the company, promotions by URM status.

The most important thing is that it should be easy to review during the meeting who is being promoted, based on factors like level, tenure, and URM status.

Use the Spreadsheet during a Promotion Review Meeting

1. Invite managers to a promotion review meeting. The form of this varies a lot between companies. But the idea is to have a place where managers come together and review all the promotion decisions. I usually devote about 3-5 minutes per promotion pitch. 
2. Invite HR and an observer from outside your organization. Ask them to be on the lookout for bias. (This is just an extra practice that helps). 
3. During your promotion review meetings, update the spreadsheet in real-time.
4. Review it during the meeting, and definitely before finalizing the promotions. Have everyone look at it! Ask the review team: what patterns are we seeing in the promotions we’re making? Is there any evidence of bias? What changes might we consider to reduce bias? Have an open discussion about it.

Why This Works

The purpose of this spreadsheet is to switch the team’s thinking from their gut to real analysis. 

You know you’ve done a good job with the spreadsheet if, during the meeting, someone can say: “Okay, so we’re proposing to promote these people, but I notice that our promotions are skewed 90% towards men when they are only 60% of the team. How do we feel about that?“. You then can dig in and do more careful thinking about who is being promoted, and make sure you’re comfortable with the results.

You may notice patterns in who is getting promoted you wouldn’t notice before. I’ve used this in the past to notice that we were promoting senior engineers more heavily than junior engineers (we ended up rethinking some of our promotions as a result), and also used it to notice some women we weren’t promoting that really deserved it. 

You may very well come out of the meeting having an unequal result. That can be okay, as long as you’ve given the results a hard look, and asked your team to think carefully about their rationale for promotion. 

For this reason, this technique works even if you don’t have statistical relevance or large sample size. The purpose is to force analysis, not to force an outcome.

Let me know about your experience with this, and if it helps!