Software Testing Techniques

Testing philosophies
Purpose

• Discuss some of the principles for best practices when testing code.

Structure of a Test

• Tests should test one and only functionality in a way as independent of outside influences as
possible.

• Tests should be as authentic as possible (don’t test with only integers if most user data will be
floating point).

• Tests should not depend on data external to the test framework (URLs, databases, etc)

• Tests should use all functionality of the testing harness to eliminate any “test smell” - just like
code smell.}

Goals of Testing

• Improve software quality

o Act as a specification

o Provide defect localization

o Bug repellant

• Understand the system under test (SUT)

o Act as documentation

• Reduce, not introduce, risk

o Provide a safety net

o Do no harm (no test logic in production)

• Be easy to run (and are run frequently)

o Automated

o Self-Checking (Hollywood Principle)

o Repeatable

o Independent

• Be easy to write and maintain

o Simple
 o Expressive

o Separation of Concerns

• Require minimal maintenance as the system evolves

Testing Principles

• Write tests first (TDD)

• Design for testability

• Use the front door first

o “api level” testing only uses the public interface

o “layer crossing” testing uses the public interface and monitors the back door

o “asynchronous” testing interacts with external services (databases, remote servers) and
should be avoided at all costs

• Communicate intent

• Don’t modify the SUT

• Keep tests independent

• Isolate the SUT

• Minimize test overlap

• Minimize untestable code

• Keep test logic out of production code

• Verify one condition per test

o No alternating exercise/verify blocks

• Test concerns separately

• Ensure commensurate effort and responsibility

o The effort to write the test shouldn’t overshadow the effort to write the production
code.

Testing strategies

Model based testing

Testing using models: Using finite state machine
Control-flow and dataflow-based testing
Domain and combinatorial testing

Software Testing Strategies

Software testing is the process of evaluating a software application to identify if it meets
specified requirements and to identify any defects. The following are common testing
strategies:
1. Black box testing – Tests the functionality of the software without looking at the internal
code structure.
2. White box testing – Tests the internal code structure and logic of the software.
3. Unit testing – Tests individual units or components of the software to ensure they are
functioning as intended.
4. Integration testing – Tests the integration of different components of the software to
ensure they work together as a system.
5. Functional testing – Tests the functional requirements of the software to ensure they
are met.
6. System testing – Tests the complete software system to ensure it meets the specified
requirements.
7. Acceptance testing – Tests the software to ensure it meets the customer’s or end-user’s
expectations.
8. Regression testing – Tests the software after changes or modifications have been made
to ensure the changes have not introduced new defects.
9. Performance testing – Tests the software to determine its performance characteristics
such as speed, scalability, and stability.
10. Security testing – Tests the software to identify vulnerabilities and ensure it meets
security requirements.

Model Based Testing

Model-based testing is a software testing technique where the run time behavior of the
software under test is checked against predictions made by a model. A model is a description of
a system’s behavior. Behavior can be described in terms of input sequences, actions, conditions,
output, and flow of data from input to output. It should be practically understandable and can
be reusable; shareable must have a precise description of the system under test.
There are numerous models available, and it describes different aspects of the system behavior.
Examples of the model are:
 • Data Flow
• Control Flow
• Dependency Graphs
• Decision Tables
• State transition machines
Model-Based Testing describes how a system behaves in response to an action (determined by a
model). Supply action, and see if the system responds as per the expectation.
It is a lightweight formal method to validate a system. This testing can be applied to both
hardware and software testing.
Types of MBT
There are two types of Model based testing frameworks-
1. Offline / a priori: Generation of Test Suites before executing it. A test suite is nothing but
a collection of test cases.
2. Online / on-the-fly: Generation of Test Suites during test execution

Finite State Machine (FSM) Testing

Finite State Machine (FSM) Testing is a type of model-based testing that uses finite-state
machines to show how a system works. An FSM is a mathematical model of a system that shows
it in terms of a set number of states, transitions between them, and the acts that happen when
these transitions happen.
For FSM testing, test cases are made based on the machine’s various states and transitions to
ensure coverage of all possible situations.
When it is used:
FSM Testing is typically used when a system can be clearly defined in terms of distinct states and
transitions. It is especially useful for systems with complex logic, such as:
• Protocol Testing: Ensuring communication protocols behave correctly.
• UI Testing: Verifying navigation between screens in an application.
• Embedded Systems: Testing devices with a set of predefined states, such as traffic lights
or vending machines.
Pros:
 • Clear Coverage: FSM testing provides a structured way to cover all possible states and
transitions, reducing the chance of missed scenarios.
• Easily Understandable Models: Finite state machines offer a visual and straightforward
representation of system behaviour, which simplifies understanding and communication.
• Automated Test Generation: Test cases can be automatically generated from the FSM,
saving time and improving consistency.
Cons:
• Complexity with Large Systems: For systems with many states or transitions, the FSM
can become large and difficult to manage.
• State Explosion: The number of possible states and transitions can grow exponentially,
making the model hard to handle.
• Limited to Well-Defined Systems: FSM testing is most effective when the system can be
accurately modelled as a finite number of states; it’s less suitable for dynamic or highly
complex systems.
Some popular tools for FSM Testing: Spec Explorer, GraphWalker, Conformiq

State charts Testing

State charts Testing is an advanced form of model-based testing that extends Finite State
Machine (FSM) Testing by incorporating hierarchy, concurrency, and communication. State
charts are graphical representations that not only capture states and transitions but also include
concepts like nested states (hierarchical states), parallel states (concurrent states), and events.
This makes State charts more expressive and capable of modeling complex, real-world systems.
When it is used:
State charts Testing is used when a system has complex behaviour that involves hierarchical
states, concurrent processes, or a need to manage events and actions across different parts of
the system.
It is particularly useful in:
• Embedded Systems: Such as automotive systems, where subsystems operate
concurrently.
• Complex User Interfaces: Where different components may be active simultaneously.
• Telecommunications Systems: Where multiple protocols and states must be managed
concurrently.
Pros:
 • Handles Complexity: State charts effectively manage complexity by allowing hierarchical
and concurrent states, making it easier to model and test intricate systems.
• Improved Test Coverage: By capturing complex behaviours in a single model, State
charts ensure more comprehensive test coverage.
• Event-Driven Testing: Supports testing of systems where behaviour is driven by external
events, which is common in real-time and embedded systems.
Cons:
• Steep Learning Curve: State charts are more complex than simple FSMs, requiring a
deeper understanding to model and test correctly.
• Tool Support: Fewer tools directly support State charts than FSMs, limiting automation
options.
• Model Complexity: As systems grow, the State chart model can become difficult to
manage and maintain, particularly with deep hierarchies and multiple concurrent states.
Popular Tools for State charts Testing include Yakindu State chart Tools, MATLAB/Simulink State
flow, Rhapsody (IBM), etc

Data-Flow Testing
A way to test software called “Data-Flow Testing” looks at the paths that data variables take
through a program. This check examines how the code creates, uses, and ends data. The main
objective is to find and test the points at which variables are given values (definitions), how
these values are then used (usage), and where they are no longer used (termination).
This testing method looks at the data flow to find mistakes in how the data is being used, like
wrong variable usage, variables that haven’t been set up, and improper variable scope.
When it is used:
Data-flow testing is particularly effective in the following scenarios:
 • Critical Data Handling: When the correctness of data usage is vital, such as in financial
systems or safety-critical software.
• Complex Codebases: In large or complex systems where data dependencies are intricate
and difficult to track manually.
• Maintenance and Refactoring: When modifying existing code, ensure that changes do
not introduce new data-related errors.
Pros
• Detects Data-Related Errors: Effectively identifies issues like uninitialized variables,
incorrect data usage, and unintended variable overwrites.
• Improves Code Quality: By ensuring proper data flow, it enhances the reliability and
maintainability of the code.
• Comprehensive Coverage: Provides detailed insights into how data is managed within
the code, leading to more thorough testing.
Cons
• Complexity: Analyzing data flow, especially in large systems with numerous variables
and data paths, can be complex and time-consuming.
• Requires Detailed Knowledge: Effective data-flow testing requires a deep understanding
of the code and its data dependencies.
• Tool Dependence: Manual data-flow analysis can be challenging, so specialized tools are
often needed, which might require additional learning and integration effort.
Popular Tools for Data-Flow Testing include Parasoft Jtest, LDRA Testbed, etc