Ten Dos and Don'ts of Threat Modeling

Threat modeling is a structured approach to identifying and mitigating security risks early in the software design phase. This proactive practice seeks to understand the types of threats an application might face, assess their potential risk and impact, and implement controls to prevent them. As a key application security activity, threat modeling is integral to building applications that are secure by design.

As the lead of the threat modeling program at Microsoft Identity, I’ve gained first-hand experience with the practical challenges and significant benefits of threat modeling in large organizations. Early detection of security flaws during the design phase can deliver measurable business value, avoiding costly fixes later on and strengthening application resilience.

In this article, we’ll explore five essential dos and five don’ts of threat modeling, focusing on its practical application while being mindful of resource and time constraints.

#1. Do Start Early in the Development Lifecycle

Every software system is, by necessity, a security system. For this reason, threat modeling should never be an afterthought. Make it a foundational activity in your software design phase. Beginning threat modeling early in the development lifecycle is essential for building secure software from the ground up. By initiating threat modeling at the beginning of the design phase, you can identify potential design flaws and security risks before they become deeply embedded in the system. This provides a clear roadmap for developers and security teams to follow, fostering a security-first mindset and avoiding the pitfalls of fixing vulnerabilities in production, which can be both costly and time-consuming.

#2. Don't Rely Solely on Automated Tools

Automation has become a powerful asset in threat modeling, enabling faster creation of data flow diagrams, generating compliance and "what if" questions, and even providing answers to some of these questions. AI-powered tools, such as large language models (LLMs) and Copilot assistants, have expanded the capabilities of automation, making it easier to identify common security risks, suggest mitigation strategies, and highlight areas of concern. However, threat modeling should not rely solely on automation.

Human insight remains essential in threat modeling. Subject matter experts bring critical thinking and a deep understanding of the technology, business logic, and potential attack vectors, allowing them to recognize context-specific threats that may be unique to a particular application — threats that automated tools might overlook. Human judgment is crucial for prioritizing threats, assessing the severity of risks, and determining realistic mitigation strategies tailored to specific environments and scenarios. Moreover, human review is necessary to filter out false positives or false negatives generated by automated tools, ensuring more accurate threat modeling outcome.

Effective threat modeling combines advanced automation with expert analysis in a collaborative process that benefits from diverse perspectives and expertise. When security and development teams work together, they can leverage automation for efficiency while relying on human expertise for contextual understanding, resulting in a comprehensive and balanced threat model.

#3. Do Define a Clear Threat Modeling Process and Security Objectives

Establishing a well-defined threat modeling process and clear security objectives is essential for effective threat modeling. This approach guides teams in identifying, prioritizing, and protecting the organization’s assets. By understanding which assets require protection and setting specific security goals, development and security teams can streamline their efforts to focus on significant threats and achieve meaningful outcomes.

A structured threat modeling process should incorporate the Four Question Framework:

• "What are we working on?" to understand the systems, data, and assets in focus
• "What can go wrong?" to identify potential threats and risks
• "What are we going to do about it?" to determine appropriate security controls and mitigations
• "Did we do a good job?" to evaluate the effectiveness of the defences implemented

This framework provides a comprehensive structure, ensuring all critical aspects of threat modeling are thoroughly addressed.

An effective threat model begins with asset identification and classification to assess each asset’s value and risk within the organization. This approach enables security teams to apply appropriate controls, prioritizing higher-value assets with stricter protections and ensuring that security efforts are proportional to risk. Defining security requirements and controls for each asset serves as a security roadmap, helping to appropriately mitigate specific threats identified. This process ensures that all security measures align with each asset’s unique risk profile, reinforcing a robust and targeted security posture.

#4. Don't Ignore the Human Element

While technical controls and system security risks are often the main focus, human actions—whether intentional or accidental—pose significant security risks. Insider threats, whether from unhappy employees, unintentional misuse, or social engineering attacks, could expose an organization to attack vectors that bypass traditional defenses. Consider how employees, contractors, and other users might interact with sensitive systems or data, and evaluate potential security risks associated with user behavior. For example, employees may fall victim to phishing attacks, inadvertently download malicious software, or misconfigure access controls.

By addressing the human element, you can design security controls like passwordless authentication, zero trust security, regular security awareness training, fraud detection, and monitoring mechanisms for detecting unusual activity, all of which improve resilience against insider threats and social engineering tactics. This holistic approach enables a more robust threat model, ensuring that the organization is prepared to manage both technical and human-centered risks effectively.

#5. Do Document and Share Insights Across Dev Teams to Build a Security Culture

Consistent documentation of findings allows teams to monitor the status and effectiveness of threat modeling efforts while building a valuable threat modeling knowledge base. Using clear and concise language ensures that both technical and non-technical stakeholders can understand the findings, fostering a shared perspective on security risks and requirements across the organization.

Communicating these findings promotes collaboration between teams and helps establish a strong security culture. When everyone has access to the same security insights, teams are empowered to proactively identify and manage risks, making more informed decisions within their areas of responsibility. This shared knowledge encourages accountability and collective engagement with security objectives.

Over time, consistent documentation and transparent communication contribute to a culture of security awareness throughout the organization. This practice not only benefits ongoing projects but also provides a structured framework for onboarding new team members, enabling them to quickly understand the company’s security priorities and historical challenges. By having security insights readily accessible and even integrated into DevSecOps workflows, development teams are more likely to adopt a proactive approach to security, treating threat modeling as an essential component of the development and design process.

#6. Don't Overcomplicate the Process

Avoid unnecessary complexity and keep the threat modeling process as simple and straightforward as possible. Don’t get bogged down in overly complex methodologies, excessive detail, or too many steps. A streamlined process encourages broader adoption across development teams. When threat modeling is intuitive, easy to follow, and integrated into regular design workshops, teams are more likely to engage with it consistently, treating it as an integral part of the development lifecycle rather than an additional burden. A complicated threat modeling process would eventually overwhelm team members who will lose interest and motivation in the process.

In the long run, a simplified threat modeling process builds confidence, facilitates smoother collaboration, and helps teams stay agile in identifying and mitigating risks.

#7. Do Regularly Review and Update the Threat Model

Threat modeling should not be a one-time exercise but an ongoing process. The data flow diagrams and threat models should be treated as living artifacts, regularly updated to reflect system changes. Consistent reviews and revisions of the threat model are essential to keep security aligned with the system’s evolving architecture and requirements.

To ensure threat modeling remains effective and valuable, it’s crucial to establish mechanisms for regular, ongoing reviews. Updates should be prompted by a range of triggers, such as the addition of new features or improvements, architectural or infrastructure changes, changes in security requirements or policies, and integrations with other services—all of which can introduce new risks or alter the existing risk profile. Furthermore, as technology evolves, new threats emerge regularly, making it necessary for threat models to adapt in response.

Reactive triggers, such as reviews prompted by security incidents or new threat intelligence, are also valuable in maintaining an up-to-date and relevant threat model. This approach ensures that the team can learn from incidents and adjust security strategies accordingly.

This approach supports a dynamic threat modeling strategy that adapts to both internal changes and the evolving threat landscape.

#8. Don't Neglect External Dependencies

External dependencies are components outside the application that are essential to its development or operation. These include third-party libraries, frameworks, APIs, services, and even tools used in DevOps and software development lifecycle (SDLC) processes, all of which can introduce significant security risks. Since these dependencies are often beyond the direct control of your organization or development team, they can become potential attack vectors if not carefully managed.

Threat modeling should explicitly account for external dependencies. Begin by identifying and classifying all external dependencies, then evaluate their trust boundaries and the level of trust assigned to each dependency. This includes assessing the vendor’s reputation, their security practices, and the dependency’s impact to your application’s security. Consider the potential impact of each dependency, including its blast radius in the event of a security incident, to ensure the application remains resilient against attacks introduced through each external component.

#9. Do Follow the “Assume Breach" Principle

The "assume breach" principle is a fundamental security concept that limits trust in applications, services, identities, and networks. It treats all components, both internal and external, as compromised by default.

Practically, this means shifting the mindset from asking "if it gets compromised" to "assume it's already compromised." When it comes to external dependencies, each third-party service or component should be treated as a potential entry point for attackers, with data received from these sources considered inherently untrusted and malicious.

Assume breach limits trust in dependencies and aims to contain the potential damage from a successful attack. This involves implementing security controls such as network segmentation, strong authentication, the principle of least privilege, zero trust, input validation, and continuous monitoring. These security controls prevent attackers from moving laterally within the system.

By adopting the “assume breach” principle in threat modeling, teams can design systems that are even more resilient to attacks.

#10. Don't Fall Into the One-Size-Fits-All Trap: Tailor Your Threat Modeling Approach

In threat modeling, relying on a one-size-fits-all approach can result in critical oversights and ineffective security measures. Each development environment is different and each software system has unique characteristics — such as its architecture, dependencies, users and machine identities, and operational environment — that influence its specific vulnerabilities and threats. For instance, a monolithic web application handling non-sensitive user data will face different risks compared to a microservice interacting with various external APIs developed in an agile methodology with continuous deployment.

Tailoring your threat modeling approach ensures that it can be practical for your development and security teams and delivers results without compromising the business roadmap.

A customized threat modeling process also enables the identification of context-specific threats that generic models may overlook. In an IoT environment, for instance, device authentication and secure communication are critical considerations, while a financial application may prioritize regulatory compliance and data integrity. By adapting your threat modeling practices, you can incorporate industry-specific best practices and facilitate collaboration among development, security, and operations teams.

Conclusion

Effective threat modeling is a cornerstone of secure by design software development. By starting early, defining clear objectives, considering the human element, documenting and sharing your findings consistently, and regularly updating the threat model, you can significantly enhance your security posture. On the other hand, relying solely on automated tools, ignoring external dependencies, overcomplicating the threat modeling process, and assuming one size fits all can undermine your efforts.

Remember, threat modeling is an ongoing process that evolves with your system and the threat landscape.