Chapter 4:

On page 89 the authors make the point:

"The key to successful testing lies in the effectiveness of your test cases."

The authors also make the point (page 90) that "...test case development has a life cycle of its own." They state:

It has a Design phase in which test objectives, input specifications, and test configurations are defined. There is a Development phase in which the detailed test procedures are defined, and a Verification and Debug phase, in which the tests are reviewed and debugged."

You should be familiar with each of these phases. These notes will focus on important points from the authors discussion of the Development and Verification & Debug phases.

Developing Test Cases (pg96-98):

A good test case has the following attributes:

• High probability of finding a bug: attitude of constructively breaking the product. That is, the person developing test cases should not be interested in establishing that the product works but in trying to find ways in which the product does not work.
• Repeatable: If the product fails, then you must be able to reproduce the exact conditions of failure. That is pre-conditions (initial system state) must be defined with the test case.
• Clearly defined expected results and pass/fail criteria: Expected result and pass/fail are related notions. If actual deviates from expected then the product has FAILed.
• Should not be redundant: Testing involves inferring the behavior of all inputs from a relatively small number of inputs. There is no value in running redundant tests. Equivalence Partitioning is one technique that may be used to ensure that test cases are not redundant.

1. Developing Test Procedures:

   Test Procedures are usually developed from the artifacts created in the design phase. The authors mention black-box testing techniques as one category of techniques that may be used in software testing. They also mention equivalence partitioning and boundary value analysis as examples of two black-box testing techniques.
   Note: See below for a discussion on black-box and white-box testing techniques. Also, see week 6 lecture notes for a more detailed discussion of equivalence partitioning and boundary value analysis.

   1. Equivalence Partitioning:

      The basic idea is to partition the input domain of a program into disjoint classes of data where the partitioning is accomplished by identifying groups of inputs that should be treated equivalently by the program. Test cases may then be designed for each class of data thus reducing the total number of tests needed. Valid and invalid classes should be considered.

      e.g. Consider a simple program that computes commissions for salesmen based on monthly sales. Let us say the program simply computes commission as 10% of sales if sales is more than $1 million and 8% otherwise. The input domain is the set of integers where positive integers (including zero) are valid and negative integers are invalid. Also, the valid input domain will consist of two classes. That is, sales more than $1 million and sales less than or equal to $1 million. These partitions suggest that at least 3 test cases, one from each class, will be needed.
      Note: See the example on page 99 also.

   2. Boundary Value Analysis:

      A technique that complements equivalence partitioning. That is, rather than selecting any element of an equivalence class, boundary value analysis recommends that test cases should be taken from elements at the boundaries, or edges, of classes.

2. Define Expected Results:

   See page 100-101.

3. Setups and Cleanups:

   See page 101-102.

4. Test Case Template:

   See page 102-104.

5. Test Case Configuration Management:

   See page 104-105.

Verifying & Debugging Tests:

On page 105 the authors mention that the testing of test cases may be accomplished by using static testing techniques. In Chapter 2 (see point #3 from the Requirements section of week 2 lecture notes) the authors mention that these techniques include Inspections, Walkthroughs, and Peer Reviews (a detailed dicussion of these techniques follow). The authors suggest some things to look for:

• Does the test suite fully address the functionality stated in the requirements.
• Are all requirements covered.
• Are the tests organized efficiently.
• Are the test casess checked into a CM system.
• Are any tests redundant.
• Is there enough detail in the procedures.
• Are expected results clearly stated.
• Are automated steps adequate.

Chapter 7:

These notes focus on four important definitions presented in Chapter 7.

1. Developing Test Cases:

   On page 154 the authors discuss strategies for developing test cases. They mention that there are two distinct categories:

   1. Black Box Testing: When the software product is treated as a black box. That is the internal structure of the product is not taken into account. Test cases are determined only from knowledge of what the products functional specification.
   2. White/Glass Box Testing: When the internal structure of the software product is taken into account (hence the term glass box).

2. Static Testing (pg 156-157):

   Refers to testing that does not require executing the software artifact. That is textual or graphical artifacts are examined. Many techniques exist, some of which are automated, but the most important are Formal Reviews such as Inspections (see below).

3. Dynamic Testing (pg 157-158):

   Refers to testing that requires an executable software artifact. Usually, this means that this type of testing may only be done when code has already been developed.

Note also, that you should be aware of the four distinct phases in the testing lifecycle. On pages 160-161 the authors discuss Formal Test Phases:

• Unit Testing
• Integration Testing
• System Testing
• Acceptance Testing

Chapter 9:

1. Inspections/Walkthroughs/Peer Reviews:

   See page 188 - 192. Also, see the readings for Assignment #1.