CSE455 - SOFTWARE Chapter 4

TESTING
INTRODUCTION
Analysis of work products (documentation and code) contributes measurably to
increased product quality
Static testing can be performed in a tool-based environment or manually and it is
often neglected
It can be close examination by one or more persons or can be performed using
appropriate analysis tools
In safety-critical industries like aviation and medicine, static testing is an extremely
important factor in ensuring the required product quality

2
STATIC TESTING IS ALL
ABOUT PREVENTION
Deviation from plan are identified before they can have a negative effect
All relevant work products are quality assured before anyone does any further work
on them

3
WHAT CAN WE ANALYZE AND
TEST?
Quality of each individual document contributes to the overall quality of the results
Specification errors need to be found before they are converted into code
Static testing reveals include inconsistencies, ambiguities, contradictions, gaps,
inaccuracies, redundancies.
Source code can be checked for consistency with the project’s programming
guidelines
It is recommended to also test and verify documents created during testing like test
plan, test cases, test procedures and scripts, acceptance criteria
Hence, static testing can improve the overall development process

4
STATIC TESTING TECHNIQUES
Human mental and analytical skills can be used to analyze and evaluate complex
situations
For the success of analysis process, people involved must understand the documents
they are reading and comprehend the statements and definitions they contain
Review is the most important static testing technique
Review can formal (adhere to pre-defined process and participants document a
planned set of results) or informal (no clear definition of intended results or what is
logged)
Type of review process you choose depends on development model, its maturity,
complexity of documents to be tested, and regulatory requirements

5
REVIEW PROCESS ACTIVITIES

6
PLANNING
Project management decides which type of review is required for a specific
document, its associated estimated time
Project manager selects skilled participants and assign them their roles
Formal review requires predefined entry and exit criteria. If plan includes entry
criteria, you need to check (checklist) that these have been fulfilled before review
Different viewpoints of participants increased effectiveness of review process
Often entire document is not required to be reviewed but only the part which
contains high risk defects
When and where review meeting will be conducted is also decided
The quality characteristics under investigation are defined

7
INITIATE REVIEW
All the resources are allocated; participants are provided with all necessary physical
and electronic materials
Along work products that are to be reviewed, review participants require other
material known as baseline document(s); These include any documents that help to
decide whether what they are looking at is a deviation

8
INDIVIDUAL REVIEW
PREPARATION
A review can only be successful if all participants are well prepared
Any potential defects, recommendations, questions or comments are noted

9
ISSUE COMMUNICATION AND
ANALYSIS
Findings are collated and discussed
It can take place at a review meeting or a company-internal online forum
Who is responsible for the correction of which defect, how to monitor correction
process and determine whether a follow-up review of fixed document is required
Each quality characteristic is evaluated and the results of the analysis are
documented
Review team has to provide a recommendation regarding acceptance of the review
object:
 Accepted without changes
 Revision necessary due to extensive changes
 Not accepted (rejected)

10
FIXING AND REPORTING
Defects found are communicated to responsible person
Correction of defects or inconsistencies the review has revealed
Author rectifies (corrects) the revealed defects
Formal review requires more work; you need to record the current status of a defect
or its report, check whether exit criteria met or not
Results of a review process vary considerably depending upon the type of review
and degree of formality involved

11
INDIVIDUAL REVIEW TECHNIQUES

The basic review process requires each participant to prepare for the team review
There are different individual review techniques, their effectiveness depends on the
type of review you are conducting:
 Ad hoc
 Checklist based
 Using scenarios and dry runs
 Role and viewpoint based

12
AD HOC – NO RULES

No predefined rules regarding individual preparation

Reviewer reads the work products sequentially and notes any issues, findings or
defects
If a review object is reviewed by more than one person, some issues are sure to be
flagged multiple times

13
CHECKLIST-BASED

Contains a list of questions based on potential defects that have been discovered in
the course of earlier projects
The questions can be based on formal guidelines or standards
Must be updated regularly according to newly organized issues and remove
questions related to outdated or resolved issues
Should be kept short and only include key questions
It focuses on specific class of questions ignoring other possibilities
Always try to keep an open mind when using this technique

14
USING SCENARIOS AND DRY RUNS

Scenario-based review is based on predefined, structured guidelines that dictate how

to run through the review process
Availability of specific use cases simplifies this process as all you have to do is
perform dry runs of all use cases; also check whether each use case is implemented
or not in code review
In the case of code review, scenarios can be based on classes of defects like one
reviewer focusing on error handling while other concentrates on checking
boundaries
Exclusive use of scenarios makes it difficult to identify other types of defects and
should be avoided if possible

15
ROLE AND VIEWPOINT-BASED

The reviewer is tasked with checking the review object from a specialist’s point of
view; utilize skills each role provides
If end-users or the system’s operator cannot take part in review sessions, these roles
have to be played by others
If the system is to be used within the organization, a reviewer can play the role of
system or user administrator
In role-based review, each reviewer pretends to be someone with a specific job and
checks the document from that role’s point of view
In viewpoint-based (perspective-based) review, each person looks at the document
from their own stakeholder’s point of view; understand the needs and concerns of
different groups and how they affect decision like priority, effort or risk

16
ROLES AND RESPONSIBILITIES WITHIN
THE REVIEW PROCESS
Not every role has to be filled by one person, and there are overlaps between roles that justify
having one person fill multiple roles (for example, moderator and review leader)
Which role can be combined depends on the nature of the project and quality criteria you
need to achieve
The individual activities associated with individual roles also vary according to the type of
review you are conducting:
 Management
 Review Leader
 Facilitator/Moderator
 Author
 Reviewer
 Scribe

17
MANAGEMENT

Responsible for planning and allocating the required resources (people, time, money)
and keep an eye on cost effectiveness
Selects which work products and supporting materials are to be included in review
process

18
REVIEW LEADER

Describes who participates in a review, as well as when and where it takes place
Holds the overall responsibility and needs to make sure planning, preparation,
execution, revision, and any follow-up work all contribute to achieving the review
objectives

19
FACILITATOR/MODERATOR

Ensures the smooth running of review process

Must have good degree of assertiveness and ability to detect undertones within the
conversation, good at bringing opposing viewpoints and keeping discussion brief
without hurting anyone’s feelings
Introduces the participants and their roles, and also provide brief summary of the
subject due for inspection
Asks all the reviewers if they are sufficiently well prepared (for instance, by making
sure that all checklist questions have been answered)

20
AUTHOR

Writer of the review object

Ensures that entry and exit criteria are fulfilled
Responsible for rectifying any defects found during the review
Must never take criticism personally
Hence, review process serves solely to improve the quality of the review object

21
REVIEWER

Participates in the review meeting following preparation

Identify and describe problematic places in the review object
Having different perspectives (tester, developer, end-user, system operator) aids the
effectiveness of a review
Individual reviewer concentrates on specific aspect of the review increases the
efficiency of the review process like one reviewer concentrates on adherence to a
particular standard while another checks the program syntax
Must be able to clearly differentiate between parts of the review object that pass the
review and those that require improvement

22
SCRIBE (RECORDER)

Documents the unresolved issues, open questions, any other related tasks generated
by the review
Documentation has to be brief and to include undistorted summaries of all the
discussion that took place
For practical reasons, author is often also the scribe because he knows exactly what
needs documenting in order to perform changes requested by the reviewer
As dedicated tools become more common, the job of the review scribe is slowly
becoming obsolete

23
TYPES OF REVIEW

Two types of review can be identified when looking at review objects:

 Review of documents that are created as work products during development process
 Review that analyze the project or development process itself (management review)

Main objective is to identify defects by investigating documents

Can be informal review, a walkthrough, a technical review or an inspection
All of these can be conducted as a peer review in which participation of colleagues
on the same an identical or similar level within the company hierarchy
A single review object can be reviewed using more than one type of review

24
MANAGEMENT REVIEW

Review object is the entire project; checking whether project is on schedule and how
management is doing
Often conducted when project reaches a particular planning milestone (i.e., a
development phase is completed)
In agile projects, it takes the form of retrospective meeting where it is held following
every sprint

25
INFORMAL REVIEW

Kind of soft review that nevertheless follows the standard review process without a
strict formal structure
Minor issues can be resolved using this type of review
Usually initiated by the author
Success of informal review is highly dependent on the skills and motivation of
reviewer
Different techniques can be used in informal review:
 Buddy testing – A teammate tries out your work
 Pair programming – Two people code together and review in real-time
 Code swapping – Developers swap code to review each other’s work

26
WALKTHROUGH

Used to check for conformity with required standards and project specifications
A forum for discussing alternative implementations
Result of walkthrough should be a consensus opinion
Focus of a walkthrough is a meeting that is usually led by the author; results need to
be recorded but there is no need to prepare formal transcript or summary report
Use of checklists is also optional
Little individual preparation is required as compared to technical review or
inspection

27
CONT..

Reviewers use the comments and questions raised by the participants as a basis for
identifying defects
This technique is useful for small teams (up to five people) and little or no
preparation and follow-up is required
Can be used for checking non-critical objects

28
TECHNICAL REVIEW

Main focus is forming a consensus, evaluating product quality and building

confidence in the review object
New ideas and suggestions for alternative implementations are welcome in a
technical review
Colleagues of the author who work in the same or closely related domain should
participate as reviewers; experts from other fields can help prevent from becoming
blind to their own habits
Result need to be recorded, but not by the author
If discussion is not reaching any consensus, voting can be used and log the results in
the review report

29
INSPECTION

Follow a strict predefined flow having entry and exit criteria

Here too, reaching a consensus is an important part of the process
Findings of the inspection should help the author to avoid similar mistakes in the
future
Additional objective is the improvement of software development process
Fulfillment of entry criteria is verified before the inspection begins

30
CONT..

Reviewer’s individual preparation takes place according to predefined rules

Some of the data collected during an inspection can also be used to identify the
causes of weakness in the development process and thus to improve its quality
It is often referred to as design, code, or software inspection based on the type of
documents that are being inspected
If formal review criteria exist, all type of documents can be inspected

31
SELECTING TYPE OF REVIEW

Depends on the objectives you are pursuing, the required quality, and effort this will
cost
You have to decide from project to project which type of review is best suited to the
situation at hand
Different questions can help answering which type of review is suitable:
 Does the review requires experts from different disciplines?
 Is scheduling easy or difficult?
 How much specific knowledge of the review object do the reviewers need?
 How formal is the review object?
 How much management support do you have?

32
SELECTING TYPE OF REVIEW

Depends on the objectives you are pursuing, the required quality, and effort this will
cost
You have to decide from project to project which type of review is best suited to the
situation at hand
Different questions can help answering which type of review is suitable:
 Does the review requires experts from different disciplines?
 Is scheduling easy or difficult?
 How much specific knowledge of the review object do the reviewers need?
 How formal is the review object?
 How much management support do you have?

33
BENEFITS OF REVIEW

Reviews are an efficient tool for assuring the quality of the work products under investigation
Early identification and correction of defects through reviews is usually cheaper than
resolving defects later when application has reached an executable stage
Static tests are generally cheaper than dynamic tests
Reviews often saves significant amount of development time
Effort involved in making correction depends on the nature of the issue that requires attention
Dynamic test can involve less effort if a code review has already removed faults in the test
object
Reviews can reveal inaccuracies in customer requirements that can be resolved in advance of
implementation
As reviews takes place in group environment, they encourage exchange of knowledge among
the participants

34
PEOPLE RELATED SUCCESS FACTORS

Select the right participants to reach your planned review objectives

It is useful to involve testers in the review, so they become familiar with project documents
from the start of the development process and begin specifying right test cases
Participants should given enough time to concentrate on details
If necessary, offer participants additional training in advance of the review especially for a
formal review such as an inspection
If a review fails because of insufficient preparation, this is usually due to poor scheduling
If the reviewers don’t understand the significance of the review or the effectiveness of the
review process in quality improvement, you can use figures from the current project (or from
earlier projects) to underscore the importance of the review process

35
DIFFERENCE BETWEEN STATIC AND
DYNAMIC TESTING
Static tests identify faults directly in documents and other work products while
dynamic tests usually identify failures in the source code in executable form
Dynamic testing verifies the visible, external behavior of the test object, whereas
static tests focus on improving the internal quality of the object under investigation

36
TYPES OF DEFECTS

Requirement defects – Inconsistencies, ambiguities, contradictions, gaps,

inaccuracies, or redundancies in requirements
Software design defects – Defects found in architecture documents that specifies
the system’s internal interfaces. Coupling and cohesion, algorithm and database
structures are ineffectively designed
Coding defects – Defects occur due to inexperienced programmers. Detect
redundant variables never invoked, unreachable duplicate code. It can be detected by
a compiler or by using an automated static analysis tool
Deviations from standards – Insufficient adherence to programming guidelines
leads to the poor quality; stick to them from start is crucial.

37
TYPES OF DEFECTS CONT..

Faulty interface specification - The names of the interfaces between components

and their parameters (number, data type, and sequence) must be checked
individually, as not all discrepancies will be found during integration
Security vulnerabilities – Static tests can also uncover these kinds of vulnerabilities
like buffer overflow because constraints weren’t explicitly checked or when input
data or SQL queries are deliberately manipulated
Gaps or inaccuracies in test basis traceability or coverage – Connection between
requirement and test case must be checked. A review report can point out missing
tests for fulfilled exit criteria

38
CONCLUSION

Every work product (document) can be reviewed, provided that all the participants in
the review are aware of how to read and understand the document in question
Reviews are static tests – can be applied at any stage
Documents are inspected by more than one person, and the results are discussed and
recorded in a dedicated meeting
The activities involved in the review process are planning, initiation, individual
preparation (i.e., individual review), discussion of the findings (issue communication
and analysis, usually in a dedicated meeting), fixing and reporting
In order to better identify defects at the individual review stage, there are different
review techniques you can use

39
CONCLUSION

The roles that need to be assigned for a review are manager, review leader, facilitator
(moderator), author, reviewer (expert), and scribe
An informal review doesn’t follow any formal rules, and the form of the resulting
documents is not regulated. Because it involves relatively little effort, the informal
review process has become a widely accepted and well-used technique
A walkthrough is an informal technique that sees the author presenting a document
to reviewers in a meeting. Here too, preparation is minimal. Walkthroughs are ideal
for small development teams who need to discuss alternatives and distribute know-
how within the team
A technical review follows a more strictly regulated flow in which individual
reviewer preparation is essential. Reviewers should be professional colleagues of the
author so that alternative realizations can be discussed

40
CONCLUSION

An inspection is the most formal of these types of review. Preparation is based on

checklists, and each review steps has its own entry and exit criteria. The review
session is led by a trained facilitator. Alongside checking the quality of the work
product, the objective of an inspection is to improve the development and review
processes
If you keep appropriate organizational and people-related success factors in mind,
and as long as you take care to avoid known pitfalls, a review can be a seriously
effective tool for increasing quality in software development projects
Reviews typically identify different defects than dynamic testing

41