With the urgent demand for rapid turnaround on new software releases-without compromising quality-the testing element of software development must keep pace, requiring a major shift from slow, labor-intensive testing methods to a faster and more thorough automated testing approach. This book is a comprehensive, step-by-step guide to the most effective tools, techniques, and methods for automated testing. Using numerous case studies of successful industry implementations, this book presents everything you need to know to successfully incorporate automated testing into the development process.
Table of Contents
I. WHAT IS AUTOMATED TESTING? 1. The Birth and Evolution of Automated Testing.
Background on Software Testing.
The Automated Test Life-Cycle Methodology (ATLM).
Decision to Automate Test.
Test Tool Acquisition.
Automated Testing Introduction Phase.
Test Planning, Design, and Development.
Execution and Management of Tests.
Test Program Review and Assessment.
ATLM's Role in the Software Testing Universe.
ATLM Relationship to System Development Life Cycle.
Test Maturity Model (TMM)-Augmented by Automated SoftwareTesting Maturity.
Test Automation Development.
Software Testing Careers.2. Decision to Automate Test.
Overcoming False Expectations for Automated Testing.
Automatic Test Plan Generation.
Test Tool Fits All.
Immediate Test Effort Reduction.
Immediate Schedule Reduction.
Tool Ease of Use.
Universal Application of Test Automation.
One Hundred Percent Test Coverage.
Benefits of Automated Testing.
Production of a Reliable System.
Improvement of the Quality of the Test Effort.
Reduction of Test Effort and Minimization of Schedule.
Acquiring Management Support.
Test Tool Proposal.3. Automated Test Tool Selection and Evaluation.
Organization's Systems Engineering Environment.
Third-Party Input from Management, Staff, and Customers andUsers.
Tool Criteria Reflecting the Systems Engineering Environment.
Level of Software Quality.
Help Desk Problem Reports.
Types of Tests.
Long-Term Investment Considerations.
Test Tool Process.
Tools That Support the Testing Life Cycle.
Business Analysis Phase Tools.
Requirements Definition Phase Tools.
Tools for the Analysis and Design Phase.
Programming Phase Tools.
Other Testing Life-Cycle Support Tools.
Testing Phase Tools.
Test Tool Research.
Evaluation Domain Definition.
Hands-On Tool Evaluation.
II. INTRODUCTION OF AUTOMATED TESTING TO A PROJECT. 4. Automated Testing Introduction Process.
Test Process Analysis.
Goals and Objectives of Testing.
Case Study: Test Objectives and Strategies.
Test Tool Consideration.
Review of Project-Specific System Requirements.
Review of Project Schedule.
Test Tool Compatibility Check.
Demonstration of the Tool to the Project Team.
Test Tool Support Profile.
Review of Training Requirements5. Test Team Management.
Organizational Structure of a Test Team.
Stovepipe Test Team.
Centralized Test Team.
IV&V Test Team.
Systems Methodology and Test Team.
Test Team Summary.
Test Program Tasks.
Test Effort Sizing.
Test Team Sizing Methods: An Overview.
Development Ratio Method.
Test Procedure Method.
Task Planning Method.
Test Effort Sizing Factors.
Test Engineer Recruiting.
Test Engineer Qualities.
Test Team Composition.
Locating Test Engineers.
Test Engineer Interviews.
Distinguishing the Best Candidate.
Roles and Responsibilities.
III. TEST PLANNING AND PREPARATION. 6. Test Planning: Smart Application of Testing.
Test Planning Activities.
Test Program Scope.
Test Goals, Objectives, and Strategies.
Test Program Parameters.
Test Requirements Definition.
Test Requirements Management.
Requirements Management Tools.
Assessing the Test Requirements Risk.
Prioritization of Tests.
Requirements Traceability Matrix.
Test Program Events, Activities, and Documentation.
The Test Environment.
Test Environment Preparations.
Test Environment Integration and Setup.
The Test Plan.
Test Completion/Acceptance Criteria.
Sample Test Plan.7. Test Analysis and Design.
Test Requirements Analysis.
Development-Level Test Analysis (Structural Approach).
System-Level Test Analysis (Behavioral Approach).
Test Program Design.
Test Program Design Models.
White-Box Techniques (Development-Level Tests).
Black-Box Techniques (System-Level Tests).
Test Design Documentation.
Test Procedure Design.
Test Procedure Definition.
Automated Versus Manual Test Analysis.
Automated Test Design Standards.
Case Study: Naming Conventions.
Manual Test Design Guidelines.
Detailed Test Design.
Test Data Requirements.8. Test Development.
Test Development Architecture.
Environment Readiness Checks.
Automation Reuse Analysis.
Test Procedure Development/Execution Schedule.
Explanation of the Sample Modularity- Relationship Matrix.
Calibration of the Test Tool.
Compatibility Work-Around Solutions.
Case Study: Incompatibility Work-Around Solution.
Manual Execution of Test Procedures.
Test Procedure Inspections-Peer Reviews.
Test Procedure Configuration Management.
Test Development Guidelines.
Reusable Test Procedures.
Case Study: Navigation Using Tabs or Mouse Clicks.
Case Study: Testing Bitmaps Using a Capture/Playback Tool.
Maintainable Test Procedures.
Case Study: Automating Documentation.
Case Study: Automated Random Testing.
Table-Driven Test Automation.
PC Environment Automated Setup Script.
Automated Recording Options.
Verifying GUI Standards.
Case Study: Smoke Test Application.
Help Function Verification Script.
Timed Message Boxes Function.
Advanced Math Functions.
IV. TEST EXECUTION AND REVIEW. 9. Test Execution.
Executing/Evaluating Test Phases.
Unit Test Execution and Evaluation.
Integration Test Execution and Evaluation.
System Test Execution and Evaluation.
Test Results Analysis of Regression Tests.
User Acceptance Test Execution and Evaluation.
Defect Tracking and New Build Process.
Defect Life-Cycle Model.
Test Program Status Tracking.
Earned Value Management System.
Case Study: System Test Status Tracking.
Test Metrics Collection and Analysis.10. Test Program Review and Assessment.
Test Program Lessons Learned-Corrective Actions andImprovement Activity.
Test Program Return on Investment.
Case Study: TestProgram Return on Investment.
Case Study: Quantify Tool Return on Investment.
V. APPENDIXES. A. How to Test Requirements.
Requirements Testing Approach.
The Quality Gateway.
Make the Requirement Measurable.
Coherency and Consistency.
Requirement or Solution?
Order in a Disorderly World.
References.B. Tools That Support the Automated Testing Life Cycle.
Business Analysis Phase.
Business Modeling Tools.
Configuration Management Tools.
Defect Tracking Tools.
Technical Review Management.
Requirements Definition Phase.
Requirements Management Tools.
Use Case Generators.
Analysis and Design Phase.
Visual Modeling Tools.
Structure Charts, Flowcharts, and Sequence Diagrams.
Test Procedure Generators.
Memory Leak and Runtime Error Detection Tools.
Static and Dynamic Analyzers.
Unit Testing Tools.
Code (Test) Coverage Analyzers and Code Instrumentors.
Usability Measurement Tools.
Testing Support Tools.
Test Data Generators.
File Comparison Tools.
Test Management Tools.
Network Testing Tools.
GUI Application Testing Tools.
Load/Performance Testing Tools.
Web Testing Tools.
Year 24710 Testing Tools.
Other Test Tool Vendors.C. Test Engineer Development.
Technical Skills Stage.
Test Process Stage.
Team Effort Stage.
Technical Stewardship Stage.
Test/Project Management Stage.
Business/Product Management Stage.D. Sample Test Plan.
Roles and Responsibilities.
Project Roles and Responsibilities.
Test Task Structure.
Test Team Resources.
Test Environment Configuration.
Test Program Reporting.
Test Program Metrics.
Detailed Test Schedule.Appendixes. D.A Test Procedure Development Guidelines. D.B Test Verification Summary and Matrix. D.C Test Procedures and Test Scripts E. Best Practices.
Test Tool Compatibility Checks.
Test Tool Upgrades.
Baselined System Setup and Configuration.
Software Installations in the Test Environment Baseline.
Overall Test Program Objectives.
Keep Automation Simple.
Test Procedure Design and Development Standards.
Automated Versus Manual Test Analysis.
Test Team Communication with Other Teams.
Defect Documentation and Reporting.
Automated Test Advocates and Experts.
Test Team Assignments.
User Group Participation.
Test Tool Improvement Suggestions.
Become a Beta Testing Site.
Specialty Topic Experts. 0201432870T04062001
With the urgent demand for rapid turnaround on new software releases--without compromising quality--the testing element of software development must keep pace, requiring a major shift from slow, labor-intensive testing methods to a faster and more thorough automated testing approach.
Automated Software Testing is a comprehensive, step-by-step guide to the most effective tools, techniques, and methods for automated testing. Using numerous case studies of successful industry implementations, this book presents everything you need to know to successfully incorporate automated testing into the development process.
In particular, this book focuses on the Automated Test Life Cycle Methodology (ATLM), a structured process for designing and executing testing that parallels the Rapid Application Development methodology commonly used today. Automated Software Testing is designed to lead you through each step of this structured program, from the initial decision to implement automated software testing through test planning, execution, and reporting. Included are test automation and test management guidance for:
- Acquiring management support
- Test tool evaluation and selection
- The automated testing introduction process
- Test effort and test team sizing
- Test team composition, recruiting, and management
- Test planning and preparation
- Test procedure development guidelines
- Automation reuse analysis and reuse library
- Best practices for test automation
Elfriede Dustin is the lead author of Automated Software Testing (Addison-Wesley, 1999), and Quality Web Systems (Addison-Wesley, 2002). An acknowledged expert in software engineering and testing practices, she has assisted numerous companies in the definition and implementation of QA and testing processes. For more information, please see her Web site at www.effectivesoftwaretesting.com. Jeff Rashka
is coauthor of Automated Software Testing
. He has significant industry experience as a test/quality assurance manager, software development program manager, and software division director. In addition, he has served as a SEI CMM /ISO 9000 Process Improvement Director. John Paul
has worked as a senior programmer/analyst on financial and budgeting systems as well as a host of other information systems. His software development leadership responsibilities have included system analysis and design, application prototyping, and application development using a number of different methodologies and programming techniques. His software development responsibilities have included application testing using automated test tools. John has also assumed a lead role in the performance of year 2000 compliance testing.