What Is Software Testing? Why Do We Test Software?

I wanted to create a very simple, easy-to-understand overview of software testing. And that is what I have tried to do, at least, in this blog post. Let's get to it.

Why Do We Test Software?

The “Why” seems simple enough:

• People develop software. People make mistakes.
• When we develop software, we make mistakes.
• Those mistakes might become live.
• The mistakes might manifest as problems, including defects, exploitable security issues, poor UX, performance issues, etc.

If we do not want those problems to exist, we want to try and detect them early and take the necessary precautions. That is why we test.

Search the web for “software problem” and you’ll get thousands of results and reasons as to why we test.

The general reason is that we test to learn about the reality of software, rather than our beliefs about the software.

What Is Software Testing?

Let's start with the notion that we want to find problems in software.

I have to:

• Have the ability to spot a problem
• Know how to use the software
• Know various ways of finding problems

How Can I Spot a Problem?

If I look at software, e.g. going to Google Search for news, I have to have some understanding of what this is supposed to do — I search for a term, see news reports for that term, and they will display on the screen so that I can read them.

I have some model of “Google Search for News.” I then use Google Search for News in various ways and compare what I’m observing to that model. For example:

• Can I search? Yes, I appear to
• Are these the results I’m supposed to see? I don’t know
• Can I see these results?
• Currently, I’m using a wide window, what if it is smaller? Should it do that?

When I resize the browser window, I see that the rendered text does not resize, so buttons, fields, and links are cut off. Should it do that?

I’ve observed something that I think might be a problem. Based on my model of what I think Google News should do, that might not be the same model that Google shares, so they might not view that as a problem.

When we implement software testing in an organization, we have a communication process and set of expectations that we incorporate into our software testing so that we report the results of comparing the software to our model to other people.

We also saw that we are learning as we test. I learned that I don’t know if these are the correct results, all the results, etc. I learned that I don’t know if the display should resize. I’m also learning about the functionality as I test, and that expands my model.

For example, my initial model didn’t include paging. I can see paging at the bottom of the page. Now, I’m expanding my model. I assume that if I click on “1,” nothing will happen because I’m on the first page. I assume that if I click on “2,” then I will be taken to page “2” and see different results, same with “3,” etc. I’m also assuming that if I click back to “1,” I will see the same results.

I can then use the software and compare what I see with my assumptions. I’ll learn if my assumptions are valid, in which case they will form part of my model, or if the assumptions are incorrect, then I’ll have to decide — is the “System” the truth, or is my model “The Truth.” But I’m learning and expanding my model.

Problems are not the only things we look for. We try to learn as much as we can. But problems are the most obvious type of information that software testing is expected to communicate.

And that’s basically software testing.

What Type of Models Are Used?

We have multiple models that we can use:

• Requirements: what it is supposed to do
• Risks: what we fear it might do
• Issues: what we know it does and don’t want it to
• Performance: does it process quickly enough
• Physical models: what versions of what stuff
• Usage scenario models: how we expect it to be used
• And more.

And we build new models as we go:

• Investigations: what can it do that we didn’t expect
• Exploits: what can it do that can be used to do something bad
• And more. We build models to help us think about the software in different ways.

What Types of Testing Are There?

Some descriptions of testing want all the models to be defined and unchanging in advance of any testing being performed. You might hear statements like “The requirements need to be signed off.” Some descriptions of testing go even further and say that testing can’t start until all the approaches we are going to use to compare the models with the software are written down, and all the results we expect to see are agreed in advance.

This is clearly a lot of work and will slow down the learning process. And there is a high risk that our models are wrong and our descriptions of the approaches are wrong and the expected results are wrong. This would mean that when we do start testing, we have a lot of rework to do, as well as the testing.

Some organizations do test like this, and this might be referred to as:

• Structured testing
• Formal testing
• Traditional testing 
• Waterfall development

Other organizations implement testing in a more agile or exploratory way. So, as less time and detail are added to the models up front, the models are not complete when we start testing and we want to learn as much as we can as we test and expect to expand and refine our models as we test.

I’d probably call this:

• Exploratory testing, or
• Agile testing, if it was an agile project.

How Do We Know What to Test?

Some of the ways of identifying "what to test” from our models have been codified as test techniques:

• Boundary Value Analysis: takes a model of data and identifies specific data values to use
• Equivalence Partitioning: takes a model of data and identifies what data we might sample
• Path: takes a graph model of flows through the system and identifies scenarios or paths to cover
• etc.

As you learn more about testing, you’ll encounter more techniques. These generally provide guidance on how to turn a model into something that you can use to drive your testing.

And we also identify other ways of comparing our models to the system; e.g. by identifying risks, comparing with other systems, thinking of what-ifs, looking more deeply into the technologies of the system, etc.

Conclusion

This was a very high-level description of software testing. Key takeaways:

• There’s a risk when we make mistakes, and they might cause issues if they are triggered when the software is live.
• When we test, we build models of the software and we compare those to our software to change our models, find new things to test, and spot mismatches between the model and the software. And this might mean that our model needs to change or that the system might have a problem, or we may have found some other information we need to communicate. For more, you can check out the video below where I talk more about why it is important to test software. Enjoy!