Examining the Specification
Part 2: Testing Fundamentals
Chapter 4
Highlights
What is black-box and white box testing
How static and dynamic testing differ
What high-level techniques can be used for
reviewing a product specification
What specific problems you should look for
when reviewing a product specification in
detail
Getting Started
Product specification
Written document using words and pictures to
describe the intended product
Calculator example
Reason
Programmer’s idea != User’s idea
Thoroughly describe the product in a specification
Document is testable
Black-box and White Box Testing
Black-box testing
Tester only knows what the software is
supposed to do
Doesn’t know how
White-box testing
Access to program’s code and can examine
the program
Know how
Risk: tailor tests to match code’s operation
Static and Dynamic Testing
Static testing
Testing something that’s not running
Examining and reviewing
Dynamic testing
Normal testing
Running and using
Car testing example
Static Black-Box Testing:
Testing the Specification
Static --- document review
Know extraction process?
No
Varieties of specifications
Words
Diagrams
Self-documenting computer language
What if no spec?
Walking, talking
Test “mental” specification
Performing a High-Level Review of the
Specification
Process of requirements elicitation is an
inexact science and is prone to having
problems
Research from high level
Large fundamental problem
Oversights
Omissions
Pretend to Be the Customer
Research who the customer will be
Who will be using the software?
Understand customer’s expectations
Gaining some familiarity with the field
Tip
Don’t forget software security
It must be specified
User will assume that the software is safe
Research Existing Standards and
Guidelines
Corporate Terminology and Conventions
Industry Requirement
Government Standards
Graphical User Interface (GUI)
Security Standards
Test instead of making decision
Review and Test similar Software
Scale
Fewer or greater features?
Less or more code
Matter in your testing?
Complexity
More or less complex
Will this impact your testing
Testability
Resources, time, and expertise to test?
Quality/Reliability
Representative of the overall quality planned for your software?
More or less reliable
Security
Secure?
Advertised and actual
Compare to what you will be offering
Review and Test similar Software (2)
Tip
Reading online
Collect review and articles about the
competition
Especially helpful for security issues
Low-Level Specification Test
Techniques
Specification Attributes Checklist
Specification Terminology Checklist
Specification Attributes Checklist
Complete
Accurate
Precise, Unambiguous, and Clear
Consistent
Relevant
Feasible
Code-free
Testable
Specification Terminology Checklist
List of problem words
The appearance of these words often signifies
that a feature isn’t yet completely thought out
It likely falls under one of the preceding
attributes
Specification Terminology Checklist (2)
Always, Every, All, None, Never
Certainly, Therefore, Clearly, Obviously,
Evidently
Some, Sometimes, Often, Usually, Ordinarily,
Customarily, Most, Mostly
Etc., And So Forth, And So On, Such As
Good, Fast, Cheap, Efficient, Small, Stable
Handled, Processed, Rejected, Skipped,
Eliminated
If…Then…(but missing Else)