Verification and Validation in Automated Testing

I’ve been asked several times recently about the difference between Verification and Validation in automated testing and for some advice on applying and documenting each kind of testing. Let me first take a minute to define these terms.

Verification vs. Validation

Verification is testing that your product meets the specifications / requirements you have written. “Did I build what I said I would?”
Validation tests how well you addressed the business needs that caused you to write those requirements. It is also sometimes called acceptance or business testing. “Did I build what I need?”

V&V together make sure that your software has delivered on its purpose in an error-free (ideally) way.

Using V&V in Your Workflow

In a more traditional Waterfall process, with Specifications and Requirements defined at the start, Validation is often performed at the end of the testing cycle. You spend tons of time defining a product, build it, make sure your software was bug-free, and then do user acceptance testing, or submit it to a client / users, perhaps even a Beta release.

In an Agile development process, ideally both verification and validation activities occur as close to simultaneously as possible. This is because you are always updating and refining your User Stories, necessitating constant small V&V loops to enable this continual feedback.

Part of the role of the Product Owner is to define what features should look like ahead of a sprint, elicit any customer feedback needed during development, and speak for the user if necessary. Validation therefore in part falls on the P.O. but is also baked into your Agile process itself. Because you are ideally using short development iterations with continual feedback from users/customers, in reality you are practicing almost Continuous Validation.

Applying Agile Verification and Validation to some industries can be a little intimidating. I’ve even heard people incorrectly say things like “my industry can’t test in an Agile way because of X regulation.” For instance, in Medical Devices you need to make sure you are in compliance with FDA regulations / guidance like 21 CFR part 11 and ISO 13485, which have more focused Validation requirements and a lot more focus on risk and documentation. Verification activities are fairly straightforward and often done via automation. Validation can be trickier.

One thing I know several Medical Device or other regulated teams do is Exploratory or Session Testing; this can be a great tool especially with mixed software/hardware devices. Structure and documentation is inherent in the way the FDA views testing, but structure does not mean that it must be pre-scripted.

Give your testers assignments on a sheet in a format similar to:

“Explore _______ With ______ To Discover _______”

An example might be: “Explore drug inventory control With a variety of products To Discover if entering the information is intuitive.”
Or better yet let them have some fun coming up with a bunch of these themselves. This can work great as a method to uncover defects in unwritten requirements and as a way to test User Experience. If you use the above structure, essentially a Test Charter, record your steps and results, and you have also generated documented Validation tests!

This article originally appeared on the SmartBear blog by Jeffrey Martin.