ISTQB Certified Tester Foundation Level: - Module 4 of 6: Test Design Techniques
ISTQB Certified Tester Foundation Level: - Module 4 of 6: Test Design Techniques
February, 2017
Study Sessions
• These study packs are intended as sessions directly from the chapters in the
  syllabus
                                                                                 2
Covered in this session...
                                                  3
Test Development Process
 Test Development Process
The process can be                                          Remember
                                                            Remember Principle
                                                                     Principle 6
                                                                               6 –– Testing
                                                                                    Testing is
                                                                                            is context
                                                                                               context
   • Very informal with little or no                                     dependent
                                                                         dependent
     documentation                                          E.g. May test differently on web
   • Very formal with lots of                                  info page vs. Safety critical
                                                                     breaking system
     documentation
                                                                                                         5
How would you test software without requirements?
                          User
                         Manual     Heuristics
                                                 What we
            Experience
              d User                             know of
                                                 software
                              Software
                                                            6
How would you test software with 1,000 documents?
          Business
          Business                   Technical
                                     Technical    Architecture
                                                  Architecture
                       Use
                       Use Cases
                           Cases                                 Process
                                                                 Process Flows
                                                                         Flows   Transactions
                                                                                 Transactions
        Requirements
        Requirements               Requirements
                                   Requirements    Diagrams
                                                   Diagrams
                                                                                                7
Test Design and the Fundamental Test Process
                Planning
                Planning and
                         and
                   Control
                   Control
                Evaluate
                Evaluate Exit
                         Exit
                Criteria and
                Criteria and
                 Reporting
                 Reporting
                Test
                Test Closure
                     Closure
                                                                                       8
Test Development Process
                                                                               9
Step 1: Analyse the Test Basis
• Prioritise
   • Which test basis to use for testing first?
                                                  10
Step 2: Identify Test Conditions
• Prioritise
   • Ensure the most important test conditions are identified
• Traceability
    • From test conditions back to specifications and requirements
    • Enables impact analysis when things change
    • Helps to determine requirements coverage in both planning and execution
                                                                                11
What is a Test Condition?
• Testers usually brainstorm all the possible test conditions for the requirement/feature that they
  are testing
• Then we pick the most appropriate test conditions and build into test cases
                                                                                                      12
    Examples of Test Conditions
                                                                                          Employee Record
                                                                                                                   13
Step 3: Identify Test Cases
                                                                                           14
What is a Test Case?
                                                                                                  15
Step 4: Develop the Test Procedure Specification
                                                                                 Test
• Specifies sequence of action                                                Procedure
                                                                                  1
   • Manual test script
   • Automated test procedure when script
                                                                             1.
     executed by a tool                                                      2.
                                                                             3.
                                                                             4.
                                                                             5.
• Prioritise Test Procedure Specifications
   • Want to execute the most important ones first!
                                                                                                          16
Step 5: Develop the Test Execution Schedule
                                                                                                 Other
                                                                                                 Other structure-based
                                                                                                        structure-based
                                       Techniques
                                        White-box
                                                                                                      techniques
                                                                                                       techniques
                                                                 Structure-based
                                                                                                  Decision
                                                                                                  Decision testing
                                                                                                           testing and
                                                                                                                   and
                                                                                                       coverage
                                                                                                       coverage
                                                                                                 Statement
                                                                                                 Statement testing
                                                                                                           testing and
                                                                                                                   and
                                                                                                       coverage
                                                                                                       coverage
                                                                                          Exploratory
                                                                                          Exploratory testing
                                                                                                      testing approach
                                                                                                              approach
                                                                Experienced-
                                                                                                        Fault
                                                                                                        Fault attack
                                                                                                              attack
                                                                   based
                                         Black-box Techniques
Categories of Test Design Techniques
                                                                                                      Error
                                                                                                      Error guessing
                                                                                                            guessing
                                                                                                     Use
                                                                                                     Use case
                                                                                                         case testing
                                                                                                              testing
                                                                    Specification-based
                                                                                                  State
                                                                                                  State transition
                                                                                                        transition testing
                                                                                                                   testing
                                                                                                   Decision
                                                                                                   Decision table
                                                                                                            table testing
                                                                                                                  testing
                                                                                                 Boundary
                                                                                                 Boundary value
                                                                                                          value analysis
                                                                                                                analysis
                                                                                                  Equivalence
                                                                                                  Equivalence partitioning
                                                                                                              partitioning
Why use test design techniques?
•   Consistency
•   “Best” practice
•   Systematic
•   Testers may have different levels of experience
•   Save time
•   Impossible to test all permutations, therefore we have to choose what we will test
                                        Remember
                                        Remember Principle
                                                 Principle 2
                                                           2 –– Exhaustive
                                                                Exhaustive testing
                                                                           testing is
                                                                                   is
                                                      impossible
                                                      impossible
                                                                                         20
Specification-based or black-box techniques
• Systematically derive tests from formal or informal models (descriptions) used for the specification
  of the software
    • Analysis of the test basis
    • Based on documentation or what we know
    • “what” the system should do
    • Without reference to its internal structure
                                                                                                    21
Structure-based or white-box techniques
• The extent of test coverage can be measured (by a tool) for existing test cases and further test
  cases can be derived to increase coverage
                                                                    00
                                                                                                     22
Experienced-based techniques
                                                                                   23
 Specification-based (Black-box) Techniques
Equivalence Partitioning
Equivalence Partitioning
• Inputs are divided into groups that are expected to behave similar
                                                                       25
Equivalence Partitioning
Activity
• Let’s derive the equivalence classes for an application that lets you apply for your driver’s license
• In Victoria, Australia you must be at least 18 yrs. old to apply for your probationary license
• Let’s say theoretically you must stop driving when you are 90 yrs. old
                               17      18                   89     90
         Others?
         Others?
          Non-
          Non-
         integer
         integer
                                                                                                          26
 Specification-based (Black-box) Techniques
                                                                        28
Boundary Value Analysis
    Back to our activity…
    • In Victoria you must be at least 18 yrs. old to apply for your probationary license
    • Let’s say theoretically you must stop driving when you are 90 yrs Old
17 18 89 90
                                                                                            29
Why do both EP and BVA?
• If you test only BVA and you find an error, you have to test more options anyway – is just the
  boundary wrong or the whole partition?
     • But could be useful if there are time pressures and you are looking to find defects first, so test
       boundaries only.
     • If you do boundaries only, you have covered all the partitions as well.
     • Boundary testing requires a lot more tests and may be expensive to set up.
                                                                                                            30
Question: EP
• An airline offers pricing as follows; infants < 2 yrs. travel free, children aged between 2-11 yrs. at
  75% of adult fare and adults at full price.
• Which of the following are age values from different VALID partitions for airfare pricing?
   a) 0, 2, 10
   b) -1, 1, 3
   c) 1, 7, 45
   d) 2, 9, 12
                                                                                                           31
Question: 2-BVA
• An electricity company charges usage <1000 kW at $0.13 per kW, between 1000 and 5000 kW
  inclusive at $0.22 per kW and >5001 kW at $0.28 per kW
                                                                                            32
Question: 3-BVA
• A bank’s saving account earns an additional 2% interest when the balance of the account is
  greater than $10,000
                                                                                               33
Specification-based (Black-box) Techniques
       Decision Tables
Decision Table Testing
• The strength of decision tables is that it creates combinations of conditions, inputs situations and
  events that might be missed
• The top part of the table contains combinations of input conditions
• The bottom part contains the resultant actions as True or False (Boolean)
• Each column can then be considered as a test case of a business rule
• Coverage standard could be at least one test for each column to trigger all combinations of
  conditions.
                         Exam Tip: Decision Table Testing is good for
                             testing business rules or combinations
                                                                                                     35
Example: Shipping an online order
   Conditions
      • Cart has something in it
      • Shipping address is a PO box
      • Account balance is in credit
   Actions
      • Item is shipped
      • Account is debited
                                                                36
Construct the decision table
                                                                                       Spec
                                                                                       Spec doesn’t
                                                                                            doesn’t
                                                                                            say
                                                                                            say
                          Not
                          Not feasible,
                              feasible, so
                                        so     Same
                                               Same resulting
                                                     resulting
                                                actions…          Each
                                                                  Each column
                                                                       column is
                                                                               is aa
                             remove?
                             remove?            actions… do
                                                          do
                                             conditions matter?      test
                                                                     test case
                                                                          case
                                             conditions matter?
                                                                                                      37
Rationalise the decision table
                                                                     38
Question: Decision table
                                                                                               39
Specification-based (Black-box) Techniques
                            money inserted
                            ask for selection                             invalid selection
                                                                            beep
Model shows:
  • states the software can occupy (the circles)
  • transitions between the states (the lines)
  • events that cause the transitions
  • actions that result from the transition
                                                                                              42
State Transition Testing
• Different events may trigger a specific action and transition to another state of the system.
• Tests can be designed to cover every state, a typical sequence, every transition or to test invalid
  transitions
                                                                                                        43
State Transition Testing – Vending Machine Example
                             Cancel                                                      S6
            S1                                                                        Suspend
           start                                       Password                         Acct
                                                        invalid
                             S2                    S3                 S4             Password
     Enter                        Password
                             1st                  2nd                3rd              invalid
    Username                       invalid
                          attempt               attempt            attempt
                                                                                                45
State Transition Table
                                          Cancel             Password             S6
              S1
                                S2                       S3 invalid     S4     Suspend
             Start
                                1st                      2nd            3rd      Acct
                                     Password
          Enter              attempt                  attempt        attempt
                                      invalid                                   Password
         Username                                           Password
                                                                       Password invalid
                                Password                      valid
                                                                          valid
                                  valid                                  S5
                                                                       Access
                                                                        Acct
                                                                                           47
Specification-based (Black-box) Techniques
                                                                           49
Use Case – Table Format
                          50
Use Case – Table Format Example
                                  51
Use Case – Diagram Format
Source: http://argouml-stats.tigris.org/nonav/documentation/manual-0.22/ch04s03.html
       Actors: Bank Engineer, Customer, Loan Bank Official and Central Computer
       Further Use Cases e.g. for Withdraw Cash for more detail on this transaction
                                                                                                     52
  Structure-based (White-box) Techniques
                                                                              54
Statement Testing and Coverage
    Test Coverage
       • Here we devise test suites with information on how many statements they
         exercise to achieve a coverage target
• Usually involves a tool that measures statement coverage during test execution
                                                                                       55
How to draw a control flow diagram
                                                                           IF
• Decisions (e.g. IF) are represented by diamonds
• Be consistent
   • Eg. Always draw to the right for “true” decisions
                                                                                                   56
Statement Testing and Coverage: Example 1
                               Qty
                                                              Question: How many statements are
                                                              exercised if we set the input value of
1. Read Qty                                                   “Qty” to 10?
                                 IF
                                               True           Answer = 4 out of 5
2. IF Qty > 50                  >50                           statements
                                                              Statement Coverage = 80%
                                        Else
                                                      X=Qty
                                                       *0.8
4. End If
                              Print X
5. Print Qty
                                                                                                       57
Statement Testing and Coverage: Example 2
  1. Read Qty                    Qty                                Question: How many statements
                                                                    are exercised if we set the input
                                                                    value of “Qty” to 100?
  2. IF Qty > 50
                                                True
                                   IF                               Answer = 5 out of 5 statements
                                  >50                               Statement Coverage = 100%!!!
                                         Else
                                                       X=Qty
  3. Qty = Qty * 0.8                                    *0.8
                                End If
  4. End If
                               Print X
  5. Print Qty
                                                               But wait... Are we missing something?
                                                                                                        58
  Structure-based (White-box) Techniques
• Decision coverage is the assessment of the percentage of decision outcomes exercised (e.g. the
  True and False options of the IF statement)
• 100% decision coverage guarantees 100% statement coverage, but not vice versa
                                                                                                   60
Decision Testing and Coverage
    1. Read Qty                                            Qty            Question 1: How many tests
                                                                          are required to achieve 100%
                                                                          statement coverage?
                                                                   True
                                                                                Answer = 1 Test
    2. IF Qty > 50                                          IF                  e.g. Value Qty = 100
                                                           >50
                                                                                 Tip:
                                                                                 Tip: 100%
                                                                                      100% Decision
                                                                                           Decision Coverage
                                                                                                    Coverage means
                                                                                                               means cover
                                                                                                                     cover all
                                                                                                                           all of
                                                                                                                               of
    5. Print Qty                                                                                    the lines!
                                                                                                    the lines!
                                                         Print X
          Tip:
          Tip: 100%
               100% Decision
                    Decision Coverage
                             Coverage is
                                      is always
                                         always == or
                                                   or >> nbr
                                                         nbr of
                                                             of tests
                                                                tests needed
                                                                      needed for
                                                                             for 100%
                                                                                 100% Statement
                                                                                      Statement coverage.
                                                                                                coverage. Never
                                                                                                          Never less
                                                                                                                less
                                                              than!
                                                              than!
                                                                                                                                    61
Decision Testing: IF with Else                     Question 1: How many tests are
                          Qty
                                                   required to achieve 100%
                                                   statement coverage?
1. Read Qty                 IF
                           >50
                                    True           Answer = 2 Tests,
2. IF Qty > 50                                     e.g. values 1 and 100
                             Else          X=Qty
3.    Qty = Qty * 0.8                       *0.8
                                                                                    62
Decision Testing: Nested IF Statements
1. Read Qty
2. IF Qty > 50                       Have a go at drawing the
3.    X = Qty * 0.8                   control flow diagram in
4. Else                                   your notes 
5. IF Qty < 0
6.       Print “Error”
7. Else
8.       X = Qty * 2
9. End If
10. End If
11. Print X
                                                                63
Decision Testing: Nested IF Statements
                            Qty                          Question 1: How many tests
                                                         are required to achieve 100%
                             IF
                                        True             statement coverage?
1. Read Qty                 >50
2. IF Qty > 50                                                   Answer = 3 Tests,
                                 Else
                                                                 e.g. Values 100, -1
3.    X = Qty * 0.8                                              and 25
                                        True
                            IF                  Print    X=Qty
4. Else                                        “Error”    *0.8
                            <0
5. IF Qty < 0                    Else
6.       Print “Error”     X=Qty                                 Question 2: How many
7. Else                     *2
                                                                 tests are required to
8.       X = Qty * 2       End If                                achieve 100% decision
9. End If                                                        coverage?
10. End If                 End If                                Answer = 3
11. Print X
                           Print X
                                                                                         64
Decision Testing: WHILE LOOP
                                     Read
                                                    Question 1: How many tests
                                                    are required to achieve 100%
1. Read Nbr_Exams                                   statement coverage?
                                      WHILE
                                                                                    65
WHILE LOOP with nested IF Statement
                               Read                                Question 1: Minimum tests
    1.   Read Number of Exams
                                                                   to achieve 100% statement
    2. WHILE more exams DO
                                                                   coverage?
                                          WHILE
                                 False
                                                                           Answer = 1 Test
    3.   Mark answers                               True
   7.     End IF
                                  End If
                                                      Question 1: Minimum tests
   8. END WHILE                                       to achieve 100%
                                   End
   9. Class Result = Fail/Exams   WHILE               statement coverage?
   10. IF Fail < 30%              Class                 Answer = 2
                                  Result
   11. Print “Smart Class”
                                                      Question 2: Minimum tests
   12. ELSE                         IF < 30   Smart   to achieve 100% decision
   13. Print “Poor Trainer”                           coverage?
   14. END IF                      Poor
                                  Trainer
                                                       Answer = 2
                                  END IF
                                                                                       67
Other Structure-based Techniques
• We have been looking at detailed low-level coverage, but coverage can also be
  applied to higher-levels like…
   • System Testing: % of menu items covered by test cases
   • Integration Testing: % components exercised
                                                                                  68
   Experienced-based techniques
• Utilising tester’s skill, intuition and experience with similar applications or technologies
• Useful when complementing systematic techniques and when special tests not easily captured by
  formal techniques
• Error Guessing
    • Anticipate defects based on experience
       • Common knowledge of system, data and why system fails
   • Or develop a list of possible errors and design tests to systemically attack these errors: this
     approach is called Fault Attack
                                                                                                       70
Exploratory Testing
• Concurrent test design, execution, logging and learning how the system behaves
                                                                                   71
Exploratory Testing
                                                                                               72
Choosing Techniques
Choosing Test Techniques
                                                        74
Question 1: Which techniques would you use?
    Scenario                      Techniques
       • Banking application
       • Previous problems with       Equivalence Partitioning
           rounding errors            Boundary Value Analysis
       • Lots of combinations
                                      State transition testing
           of possible factors
       • Good documentation           Decision tables
       • Experienced testers          Use case testing
       • Time constraints
                                      Statement testing
                                      Decision testing
                                      Error guessing
                                      Exploratory testing
                                                                  75
Question 2: Which techniques would you use?
    Scenario                                Techniques
       • Web site
       • Little documentation                   Equivalence Partitioning
       • Previous problems with navigation      Boundary Value Analysis
       • Experienced testers
                                                State transition testing
       • Time constraints
                                                Decision tables
                                                Use case testing
                                                Statement testing
                                                Decision testing
                                                Error guessing
                                                Exploratory testing
                                                                            76
Question 3: Which techniques would you use?
    Scenario                             Techniques
       • Train automatic braking system
       • Lots of documentation               Equivalence Partitioning
       • Previous problems with              Boundary Value Analysis
           untested combinations
                                             State transition testing
       • Safety standards require
           decision coverage                 Decision tables
       • High risk                           Use case testing
       • Experienced testers
                                             Statement testing
                                             Decision testing
                                             Error guessing
                                             Exploratory testing
                                                                         77
 End of Module 4
QUESTIONS
End of Module Learning Check
                                                                           79
ISTQB Practice Exam Question 1
One of the coverage goals for the project is 100% decision coverage. The following 3 tests have been
executed for the control flow graph below.                                            A
                                                                                                       80
ISTQB Practice Exam Question 2
     A bank application determines the creditworthiness of customers.
     The application uses a set of rules to determine the upper limit of the
     credit amount. Which of the following black-box test design
     techniques is best for testing the application?
81