System on Modules (SOM) and Its End-to-End Verification Using Test Automation Framework

SOM is an entire CPU architecture built in a small package, of size like a credit card. It is a board-level circuit that integrates a system function and provides core components of an embedded processing system - processor cores, communication interfaces, and memory blocks on a single module. Designing any product based on the SOM is a much faster process than designing the entire system from the ground up. 

There are multiple System on Module manufacturers available in the market worldwide with an equal amount of open-source automated testing frameworks. If you plan to use System-on-Module (SOM) in your product, the first thing required is to identify the test automation framework from the ones available out in the market and check for a suitable module for your requirement. 

Image/Video intensive industries face difficulty in designing and developing customized hardware solutions for explicit application, with reduced time and cost. It is linked with quick evolving processors with increasing complexity, requiring product companies to constantly introduce upgraded variants in a short span. System on Module (SOM) ensures reduced development and design risk for any application. SOM is a re-usable module embracing maximum hardware/processor complexity, leaving behind reduced work on the carrier/mainboard, thus accelerating Time-to-Market.

System-on-Module is a small PCB board having CPU, RAM, Flash, Power Supply, and various IOs (GPIOS, UART, USB, I2C, SPI, etc.). In new-age electronics, SOM is becoming a quite common part of the design, specifically in industrial, medical electronics. It reduces the design complexity and the time-to-market which is critical for a product's success. These System-on-Modules runs an OS and are mainly used in applications where Ethernet, file systems, high-resolution display, USB, Internet, etc. are required and the application needs high computing with less development effort. 

If you are building a product with less than 20-25K volume, it is viable to use a ready SOM for the product development. 

Test Automation Frameworks for SOM

A testing automation framework is a set of guidelines used for developing test cases. A framework is an amalgamation of tools and practices designed to support quality assurance experts test more efficiently. The guidelines involve coding standards, methodologies to handle test data, object repositories, processes to store test results, or information on accessing external resources. Testing frameworks are an essential part of any successful product release that goes under testing automation. Using a framework for automated testing will enhance a team’s testing efficiency, accuracy, and will reduce time and risks. 

There are different types of automated testing frameworks, each having its architecture and merits/demerits. Selecting the right framework is very crucial for your SOM application testing.

Here are a few commonly used frameworks:

• Linear Automation Framework
• Modular Based Testing Framework
• Library Architecture Testing Framework
• Data-Driven Framework
• Keyword-Driven Framework
• Hybrid Testing Framework

From above, the Modular and Hybrid testing frameworks are best suitable for SOM and their development kit verification. The ultimate goal of testing is to ensure that software works as per the specifications and in line with user expectations. The entire process involves quite a few testing types which are preferred or prioritized over others depending on the nature of the application and organization. 

Let us see some of the basic testing involved in the end-to-end testing process:

Unit Testing

The full software stack is made of many small components, so instead of directly testing the full software stack, one should cover individual module level testing first. Here unit testing makes sure to have module/method level input/output testing coverage. Unit testing offers a base for complex integrated software and provides fine quality application code, speeding up continuous integration and development process. Often unit tests are executed through test automation by developers.

Smoke Testing

Smoke testing is to verify whether the deployed software build is stable or not. To go ahead with further testing depends on smoke test results. It is also referred to as build verification testing which checks whether functionality meets its objective. There is still some development work required if SOM does not clear the smoke. 

Sanity Testing

The changes or proposed functionality that are working as expected is defined by sanity testing. Suppose we fix some issue in the boot flow of the embedded product, then it should go to the validation team for sanity testing. Once this test is passed it should not impact other basic functionality. Sanity testing is unscripted and specifically targets the area that has undergone a code change.

Regression Testing 

Every time the program is revised/modified, it should be retested to assure that the modifications didn’t unintentionally “break” some unrelated behavior. This is called regression testing; these tests are usually automated through a test script. Each time the program/design is tested, it should give a smooth result.

Functional Testing

Functional testing specifies what the system does. It is also known as black-box testing because the test cases for functional tests are developed without reference to the actual code, i.e., without looking “inside the box.”

Any embedded system has inputs, outputs, and implements some drivers between them. Black-box testing is about which inputs should be acceptable and how they should relate to the outputs. The tester is unaware of the internal structure of the module or source code. Black-box tests include stress testing, boundary value testing, and performance testing.