Navigating the Challenges of Rapidly Scaling Your Engineering Team

In this article, we are going to look at the challenges faced when rapidly scaling engineering teams in startup companies as well as other kinds of companies with a focus on product development. These challenges change between different types of companies, sizes, and stages of maturity. For instance, the growth of a consultancy software company focused on outsourcing is so different from a startup focused on product development.

I've faced much team growth and also seen the growth of teams in several companies, and most of them have faced the same challenges and problems.

Challenges

The following are some of the challenges or problems that we will have to address in high-growth scenarios:

• Growth is aligned with productivity: many companies grow, but the output is unfortunately far from the goals.
• Avoid team frustration due to failure to achieve growth goals.
• Avoid too much time being consumed with the hiring process for the engineering teams.
• Avoid the demotivation of newcomers due to chaotic onboarding processes: the onboarding process is the first experience in the company.
• Maintain and promote the cultural values defined by the organization.
• The impact on delivery is aligned with the defined goals and risks.
• New hires meet expectations and goals in terms of value contribution.

Navigating the Challenges

Goals

Goals are the main drivers of the growth strategy. They need to be challenging, but also realistic, and linked to mid-term and long-term vision. 

• Challenging: Push the team to go beyond their comfort zone and strive for excellence. It requires effort, innovations, planning, and agility.
• Realistic: Ensure the goals can be achieved to avoid lead with frustration and burnout. The growth of the company and its success have to enhance the motivation and inspiration of the team.
• Long-term: Goals have to be aligned with the company's long-term vision and in a wide range. Large growth cannot be organized with the next three months in mind, because that may be the time it takes to find suitable candidates.

Goals have to be measurable, clear, and specific to:

• Promote accountability
• Evaluate and measure the goal's success
• Take data-driven decisions

All growth requires dedication and effort from the team; time that they will not dedicate to product evolution or development.

Example: Unrealistic Goal

Let's suppose we have a team of 10 engineers divided into 2 squads: backend and platform. The company set the following goals:

1. Triplicate the team in 1 year, from 10 to 30 engineers.
2. Keep the delivery performance.
3. Create three news squads: DevOps, Data Platform, and Front End.
4. Promote the culture.
5. Only hire top-tier engineers.

Most likely, the number of candidates we will have to evaluate in interviews and technical exercises will be at best four candidates for each position in addition to the time dedicated to the onboarding process. Usually, there is more than one engineer collaborating in the hiring process so we are likely to have a significant impact on delivery.

Finding a team of experienced and highly qualified people is not an easy task. It is necessary to define what we consider "talent" and the different levels at which we can hire.

Maintaining and promoting the culture in a high-growth environment where in one year there are more new people than the team we have is very complex and requires a good strategy, definition of objectives, and agility in decision-making.

With this, we want to reflect that one of these objectives would already be ambitious - but all of them together make it practically impossible to achieve success.

Talent Acquisition and Hiring Process

The talent acquisition team plays a crucial role in a company's growth strategy, but they need the support of all of the company. C-Levels and hiring managers have to provide all the support and be involved as the same team.

Clear Communication

Foster open and clear communication between teams to ensure that everyone understands the goals and the role each team plays in the process.

Review Pipeline Quality

Sometimes many candidates go through the early stages of the pipeline and are ultimately discarded, and this generates a lot of frustration in the engineering team because the analysis of each candidate requires significant effort.

It is important to adjust the requirements and search criteria for candidates in the early stages of the pipeline and this requires constant communication between the teams.

Market Knowledge

Talent acquisition teams should provide insights into market trends and competitor strategies. This knowledge provides important information to the company to define the expectations and strategy and stay ahead in the market.

Cultural Values

It is important to keep in mind that each engineer who joins our team brings his or her own culture based on factors such as work experience, personality, or the country where they live.

Although these people fit the cultural pattern we are looking for, most of the time they do not have the culture of the company, and the hiring process is not reliable. If maintaining the culture is important to the company, we need to mentor new employees starting with the recruitment process itself.

• Promote values in the hiring process.
• Promote values in the company and team onboarding process.
• Promote values during the first years through the mentoring process.

Promoting the cultural values and the company's goal are tasks that must be done continuously, but we must reinforce and review them with new hires more frequently.

On-Boarding

In my opinion, the onboarding process has a huge role in almost all companies and is not given enough attention. It is especially important in high-growth companies. The two main problems are:

• No onboarding process: Onboarding is focused on a meeting with human resources, another with the manager, and finally the team: a three-hour process. This can only be considered as a welcome meeting.
• Highly technical processes: Processes very oriented to perform your first deployment and that mainly promote knowledge silos and little engagement with the company. 

The onboarding process must be led by the organization. It must be structured and must encourage a smooth integration of new hires into the organization, minimizing disruptions and maximizing productivity over time. In addition, the entire onboarding process should be a step-by-step process with as much documented support as possible.

This would be a base structure for a complete onboarding process:

• Pre-boarding: It includes all the activities that occur between the acceptance of the offer and the new hire's first day.  Continuous communication is important because it promotes motivation and cultural values and helps to create a feeling within the company.
• Welcome Day:
  • Welcome meeting, company overview,  review of company policies and cultural values  
  • Paperwork, documentation, and enrollment processes
  • Initial equipment setup
  • Introduction to Team and Manager
  • Security training
• Company 360 (scheduled by month): 360-degree meetings with leaders from all departments provide valuable insights, foster collaboration, and help new employees understand the broader organizational context.
• Starting the first week: 
  • Cultural values and goals: The manager and the team share the same cultural values and team goals. The goals have to be clear and most of them measurable. 
  • Mentorship: Assign a mentor to support the integration process at least during the first year.
  • Engineering Tech best practices and tools: Share the vision of architecture principles, DevOps, data principles, tools, and best practices of the organization.
  • Roles-specific training
  • Team integration: Start participating in team meetings.
  • Feedback and evaluation: Feedback must always be continuous, honest, and constructive. We must put more emphasis on new hires to adjust goals, mentoring, or training. It would be better to start with one-to-one and include this evaluation and feedback in these sessions.
• Starting in the third month:
  • Performance evaluation
  • Continuous learning is part of the cultural values but at this time learning paths could be considered
  • Initiate conversations about long-term career paths.

It is important to avoid onboarding processes based solely on pairing or shadowing strategies because they require too much effort and also only generate silos and misalignment. These sessions are important but must be supported by documentation from both the organization and the team itself.

Impact on Delivery

The growth phase often requires a high investment of time, effort, and people in the hiring and onboarding process.

• Hiring process: Participating in technical sessions, reviewing candidate profiles, and reviewing technical exercises.
• Onboarding: The process of onboarding a new engineer to a team is always time-consuming and usually involves a learning curve until these people can offer more value than the effort invested in their integration.

In the case of large growth, there may be situations in which teams are formed entirely by new engineers. This also has an impact on delivery, because these teams need:

• Mentors and support to adapt to the new environment
• Transversal coordination with other squads

Talent Density

In my opinion, growth should be based on the amount of talent and not on the number of engineers. At this point, there are a number of aspects to consider:

• What does talent mean to our organization?
• Finding talent is very complicated. There is a lot of competition in the market, people specialized in hiring processes, and the pressure to grow.
• Many people mistake talent for knowledge or years of experience.
• In my case, I have always given more value to the person's potential for the new role and for the organization rather than the experience in the role or the companies in which he/she has worked.

The fit of a new hire is not only restricted to the hiring process but also to the evaluation period. Moreover, it is during the evaluation period that we can really evaluate the person. It is in this period when the decision is less painful for both parties, a person who does not fit in the organization will generate a negative impact both for him and for the organization.

Team Topology

These growth scenarios require changes in the organization and the creation of new teams or departments. Two fundamental factors must be taken into account:

• Team creation strategy
• Conway's Law

Team Creation Strategy

There are several strategies for developing the organization of teams:

• Integrate new hires into existing squads.
• Integrate new hires into existing squads and after some time, divide the team in two.
• Create entirely new teams with new hires.
• Create a new team from current leadership and new hires.

The decision to apply a single approach or a combination of several approaches depends on several factors, including the organization's specific needs, resource availability, and long-term objectives. 

Conway's Law

Conway's Law is a principle in software engineering and organizational theory:

Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.

Conway's Law suggests that the communication patterns, relationships, and team structures within an organization are reflected in the architecture, design, and interfaces of the software or systems they build.

Summary

The growth of engineering teams is one of the most complex challenges facing a growing organization, especially if this growth must be aligned with productivity and cultural goals.

Hiring the number of people we have set as a target can be easy. Hiring the right people can be almost impossible and hiring a ratio of enough talented people is very difficult. This can only be done well if you work as a team.