API Testing With Postman Collections in AWS CodePipeline

Step 1: Creating a Git Repository With AWS CodeCommit

AWS CodeCommit is a version control service hosted by AWS. You can create and manage Git repositories on CodeCommit.

For this project, we will create a new repository named postman-sample from AWS Console > Developer Tools > CodeCommit.

Clone the newly created repository on your computer.

Step 2: Exporting the Postman Collection

Next, we need to export the Postman collection so we can run this using Newman CLI.

Select the Collection v2 option from the Export Collection dialogue box and click Export.

Save the collection in the cloned repository folder from Step 1 above.

Creating a Buildspec File

We need to create a buildspec file to tell AWS CodeBuild how we want to run the collection. Before running the collection, we need to install The Newman NPM package in the pre_build phase and then call the Newman CLI with the collection we want to run in the build phase. We can also specify report options to generate an HTML file at the end. We will upload this file to S3 after CodeBuild executes the collection:

Commit and push the collection and buildspec file to CodeCommit in the master branch.

You can find the collection and buildspec file used for this example here.

Step 3: Creating an S3 Bucket to Save the Report

Let’s create an S3 bucket to save the report file generated by Newman. We can use this file to publish the results.

From AWS Console > Storage > S3, create a new bucket named postmanreport (you will need to use a unique name). You can enable version control on the S3 bucket to see historical reports.

Step 4: Creating an AWS CodeBuild Project

We will use CodeBuild to fetch the changes from CodeCommit and run the collection using Newman. We already have a buildspec file which has a sequence for execution.

From AWS Console > Developer Tools > CodeBuild, create a project named postman-sample.

Set Source Provider as AWS CodeCommit and Repository as postman-sample:

We need an environment to run the build job. Let’s configure the Ubuntu/Node.js environment and Artefacts settings as shown in the below screenshot:

Next, we need to configure a Service role. Create a new service role and click continue:

The CodeBuild project is ready. We can test this project by manually starting the build.

Step 5: Bringing It Together With AWS CodePipeline

Finally, we need to create a new AWS CodePipeline to trigger the CodeBuild when a new change is pushed to CodeCommit.

From AWS Console > Developer Tools > CodePipeline, create a pipeline named postman-sample:

Configure the Source Location as AWS CodeCommit as shown in the below screenshot:

Configure the Build provider as AWS CodeBuild with the CodeBuild project created in Step 4 above, as shown in the below screenshot:

We just want to run tests and stop there for now. We will not deploy anything in this project, so we will select the No Deployment option in the Deploy section as shown in the below screenshot:

Finally, we need to configure the AWS Service role as shown in the below screenshot:

This will take you to IAM to define the new role:

Back in CodePipeline, make sure the role is specified:

Review the CodePipeline configuration and create the pipeline.

You will see the Pipeline created success message. You can try invoking the newly created Pipeline by clicking the Release Change button:

Once the Pipeline is executed, you will see both the Source and Build stages in Green (unless there are any errors), as shown in the below screenshot:

After a successful run, you can go and check the S3 bucket to view the report.

Newman generates a nicely formatted report, as shown below:

You can also configure a Lambda function or SNS to send a notification along with the report.