How can you possibly test modern software fully?

Implementing all-pairs testing

If testing all the possible combinations of values is not necessary, the question becomes how to construct the all-pairs tests. There are several applications that allow you to enter variables and generate the tests. However, it is helpful to try constructing the test cases manually once or twice so you understand exactly how all-pairs testing works.

Identifying the variables

Before you can implement all-pairs testing, you need to identify the variables. For example, single sign-on support was recently added to your company's sales management application, and the login screen was redesigned for this feature. The application runs on a variety of operating systems, supports several databases, and includes a cross-platform graphical user interface (GUI) and a web-based client. The following table (Figure 3) summarises the variables that affect the tests:

[Note: For the sake of simplicity, the following examples assume that all the variable combinations are valid.]

How do you effectively test single sign-on support based on these different variables? A simple calculation shows there are 48 possible combinations (3 x 4 x 2 x 2). All-pairs testing can significantly reduce that number.

Creating the first pair of values

After identifying the variables, use a spreadsheet to combine the values from a pair of variables. Before starting, arrange the variables by the number of values they contain from greatest to least. In our example, the variables would be placed in the following order: Database, Operating System, Client Type, and Browser. Label the first column in the spreadsheet with the name of the variable with the most values (Figure 4):

Label the second column with the name of the variable with the second-highest number of values (Figure 5):

Pair the first value in the first variable column with the first value in the second variable column (Figure 6):

Then match the first value from the first variable column with the second value in the second variable column (Figure 7):

Continue the process until the first value of the first variable is paired with each of the second variable values (Figure 8):

Skip a row in the spreadsheet to improve readability and to allow for expansion. Then match the first variable's second value with all the values from the second variable (Figure 9):

Repeat the steps until all values in the first two variables are paired up (Figure 10):