Strategies for Black Box Testing

Ideally, we’d like to test every possible thing that can be done with our program. But, as we said, writing and executing test cases is expensive. We want to make sure that we definitely write test cases for the kinds of things that the customer will do most often or even fairly often. Our objective is to find as many defects as possible in as few test cases as possible. To accomplish this objective, we use some strategies that will be discussed in this subsection. We want to avoid writing redundant test cases that won’t tell us anything new (because they have similar conditions to other test cases we already wrote). Each test case should probe a different mode of failure. We also want to design the simplest test cases that could possibly reveal this mode of failure – test cases themselves can be error-prone if we don’t keep this in mind.

Tests of Customer Requirements

Black box test cases are based on customer requirements. We begin by looking at each customer requirement. To start, we want to make sure that every single customer requirement has been tested at least once. As a result, we can trace every requirement to its test case(s) and every test case back to its stated customer requirement. The first test case we’d write for any given requirement is the most-used success path for that requirement. By success path, we mean that we want to execute some desirable functionality (something the customer wants to work) without any error conditions. We proceed by planning more success path test cases, based on other ways the customer wants to use the functionality and some test cases that execute failure paths. Intuitively, failure paths intentionally have some kind of errors in them, such as errors that users can accidentally input. We must make sure that the program behaves predictably and gracefully in the face of these errors. Finally, we should plan the execution of our tests out so that the most troublesome, risky requirements are tested first. This would allow more time for fixing problems before delivering the product to the customer. It would be devastating to find a critical flaw right before the product is due to be delivered. We’ll start with one basic requirement. We can write many test cases based on this one requirement, which follows below. As we’ve said before, it is impossible to test every single possible combination of input. We’ll outline an incomplete sampling of test cases and reason about them in this section.

Requirement: When a user lands on the “Go to Jail” cell, the player goes directly to jail, does not pass go, does not collect $200. On the next turn, the player must pay $50 to get out of jail and does not roll the dice or advance. If the player does not have enough money, he or she is out of the game.

There are many things to test in this short requirement above, including:

• Does the player get sent to jail after landing on “Go to Jail”?
• Does the player receive $200 if “Go” is between the current space and jail?
• Is $50 correctly decremented if the player has more than $50?
• Is the player out of the game if he or she has less than $50?

At first it is good to start out by testing some input that you know should definitely pass or definitely fail. If these kinds of tests don’t work properly, you know you should just quit testing and put the code back into development. We can start with a two obvious passing test case, as shown in Table

Test Plan #1 for the Jail Requirement

You will also note that we should test the simplest possible means to force the condition we are trying to achieve. For example, in Test Case 5, we only have one player so we temporarily didn’t have to spend our time with Player 2. We add Player 2 in Test Case 6 so we can observe that the loss of $50 and dice roll occurs on the next turn (after Player 2 goes). We could go on and test many more aspects of the above requirement. We will now discuss some strategies to consider in creating more test cases.