Unlocking Efficiency: Pairwise Testing

6 min readNov 11, 2023

What is Pairwise Testing

Software testing is a critical aspect of ensuring the reliability and functionality of applications. However, testing every possible combination of inputs can be impractical and time-consuming.

In this article, we explore the concept of pairwise testing, a powerful technique that optimizes test case selection while maintaining comprehensive coverage.

Understanding Pairwise Testing

Pairwise testing, also known as “all-pairs testing,” is a method that focuses on testing all possible pairs of input parameters. This approach is based on the observation that most defects are caused by the interaction of pairs of factors rather than the interaction of all factors simultaneously.

Scenario

Fed up with reading theories. Ok, let’s try to understand the concept by using real-world examples.

A few years back while I was working in a budget Airline reservation system, we had many combinations to develop test cases.

I used a budget airline especially here because as you know they sell everything like meals, seats etc.

Let’s summarize some of the inputs that we have to provide when we are filling out ticket reservation forms in budget airlines.

Departure City (Colombo, Sharjah, Casablanca)
Arrival City (Paris, Milan, Kuala Lumpur)
Departure Date (May 1 2024, May 15 2024, May 30 2024)
Arrival Date (May 5 2024, May 20 2024, June 5 2024)
Adult Passengers (1, 2, 3)
Child Passengers (0, 1, 2)
Infant Passengers (0, 1)
One Way or Return (One Way, Return)
Promotion Code (None, Promo1, Promo2)
Payment Types (Visa Card, Master Card, KIOSK)
Baggage (None, Standard, Extra)
Seat Reservation (None, Aisle, Middle, Window, Extra Legspace)
Meal Reservation (None, Vegetarian, Non-Vegetarian, Baby Meal, Diabetic Meal)
Special Services Request (None, Wheelchair, Priority Boarding, Blind Passenger, Baby Basket)
Hala Services (None, Meet and Greet, Lounge Access)
Currency (AED, EUR, GBP)
Insurance (Yes, No)

So can you imagine how many combinations we need to test on this?

3×3×3×3×3×3×2×2×3×3×3×5×5×5×3×3×2=13,608,000

Now you take a paper and try to write down some test cases that you can think of to cover all the above combinations.

Done?

Let’s compare your results with my results later in this article.

Is there a way to optimize this test case generation to cover all the possible combinations?

Is it possible to test all the combinations that we can provide while filling out the flight reservation form?

This is where Pairwise testing comes into play.

Before we move to the optimizing test cases section let’s discuss the tools that we can use for this.

Tools for Pairwise Testing

There are several tools are available to assist in implementing pairwise testing. Some popular tools include:

Pict (Pairwise Independent Combinatorial Tool): Pict is a command-line tool developed by Microsoft that generates pairwise combinations efficiently.
Allpairs: Allpairs is a command line file based on a Perl script. Using the application requires basic knowledge of console commands:
Pairwise Pict Online is a free service (version of Microsoft Pict)

All the available tools you can find here

Pairwise Testing

Combinatorial Test Case Generation

www.pairwise.org

Pairwise Testing in Action

For this example, we will use Pairwise Pict Online (https://pairwise.yuuniworks.com/) since we don’t need any setup to use that tool.

Now click on generate to generate the test cases

After test case generation copy the text and paste it into an Excel file. So you can read them easily. I have formatted it for better visibility.

Here you can see it has generated only 32 test cases to cover all the possible scenarios.

13,608,000 test cases to 32 test cases with very high test coverage.

That’s the magic of using Pairwise testing. :)

Now you can compare test cases that you have written with the list above.

Deep dive

Now you might have a question that some combinations can be invalid. Example: Infant or child travelling without an adult

An infant must be accompanied by at least one adult.
A child cannot travel alone without at least one adult.

So as per the above example, we need to introduce constraints to ignore those invalid scenarios to optimize our test cases further.

Here is how you do it.

if [Infant Passengers] >= 1 then [Adult Passengers] >= 1;
if [Child Passengers] >= 1 then [Adult Passengers] >= 1;

Since we have provided Adult Passengers (1, 2, 3) values you should get a warning. But this is how you can add constraints.

So Let’s discuss the benefits as we noticed in this example.

Benefits of Pairwise Testing

Efficiency: Pairwise testing allows you to achieve extensive coverage with a significantly reduced number of test cases, saving time and resources.
Defect Detection: By focusing on pairs of parameters, which often reveal defects, pairwise testing efficiently identifies potential issues.
Comprehensive Coverage: The method ensures that all possible pairs of input values are tested, providing confidence in the application’s behaviour.

So are there any disadvantages of using Pairwise testing? Yes.

Disadvantage of pairwise testing

Limited to Pairs of Parameters:

Pairwise testing primarily focuses on testing combinations of pairs of parameters. While this significantly reduces the number of test cases, it may not cover all possible interactions involving three or more parameters simultaneously.

2. May Miss Higher-Order Interactions:

Some defects or issues in a system may only arise when specific combinations of three or more parameters are present. Pairwise testing might not catch these higher-order interactions.

3. Requires Good Understanding of System Behavior:

Effective pairwise testing relies on a solid understanding of the system under test and the potential interactions between parameters. If the tester lacks this understanding, critical scenarios may be overlooked.

4. Doesn’t Guarantee Full Coverage:

While pairwise testing ensures coverage for pairs of parameters, it doesn’t guarantee coverage of all possible combinations. There might be specific scenarios where certain parameter combinations need to be tested individually.

5. Applicability Depends on the System:

Pairwise testing is not universally applicable to all types of software systems. In some cases, the nature of the system or the specific requirements might necessitate other testing methods.

6. Complexity Increases with Additional Parameters:

As the number of parameters increases, generating and managing the combinations becomes more complex. Beyond a certain point, the benefits of pairwise testing may diminish.

7. Limited to Input Parameter Testing:

Pairwise testing is particularly effective for testing input parameters and their interactions. It may not be as suitable for scenarios where output or state-based interactions are critical.

8. Dependent on Test Data Generation Tools:

The effectiveness of pairwise testing can be influenced by the capabilities of the test data generation tools used. Some tools may not handle certain types of constraints or requirements well.

Conclusion

Pairwise testing is a powerful approach to optimize software testing efforts, especially in scenarios with numerous input parameters. By leveraging this technique and utilizing dedicated tools, testing teams can achieve thorough coverage while maintaining efficiency. Implementing pairwise testing in your testing strategy can lead to more effective defect detection and improved software quality.

So, my conclusion regarding the drawbacks largely hinges on your familiarity with the domain and the time that you spend carefully on this test case generation and review.