| WebMan: A Basic RefEntry Servlet | ||
|---|---|---|
| <<< Previous | Next >>> | |
This chapter lays out the testing strategy, specifies the specific tests used to qualify the application, and includes the resulting test report.
The cover the project documentation, the webman client, the Format utilities, the build process, and the RefEntry servlet itself.
The tests verify that the documentation:
Reflects completely and accurately the architecture and implementation of the project
Covers all procedures for using and installing the software
Explains how to use all public features of the clients, including RefEntry development and stylesheet extension
The tests verify that the webman client:
Provides readable output for a variety of browsers, including at least Mozilla, Netscape, Internet Explorer, and links (or lynx)
Allows use of the search functionality on each of the browsers indicated
Includes the search string in the output
The tests verify that the Format utility:
Handles garbage input gracefully
Allows use of customized stylesheets
Produces useful, intelligible diagnostics when encountering problems
The tests verify that the build process:
Produces a working servlet WAR file if at all possible, including all files specified for the WAR file architecture
Provides diagnostics for missing files, corrupt configuration files, and other similar configuration problems
Allows use of customized stylesheets
The tests verify that the RefEntry servlet:
Permits multiple concurrent searches
Handles garbage requests appropriately
Returns simple search results for a single word within 2 seconds
Is straightforward to install and use
Complies with servlet standards for portability
In order to verify each of these points, functional, system, and installation tests are run for the second, third, and fourth deliveries respectively. Tests continue to be developed as the software and documentation take shape for the delivery. Before each delivery including software tests, a test result report is completed, and a list of bugs compiled. Bugs that are not to be fixed are identified as such in the release notes for the delivery.
The test results are cumulative. Also, once tests are developed, they are run regularly to check for regressions in the software.
Functional tests are developed during implementation. Their overall behavior is complete when the interface definition is complete. They verify the following.
All public and friendly methods respond as documented.
All elements are formatted as described in the documentation.
Functional tests may be installed alongside the software and may be run through the JUnit test framework.
System tests are defined and developed during components integration. Their overall behavior is complete when the components are integrated together. They verify the following.
Configuration file content has the documented effect on the build results.
Concurrent searches may be run against the servlet. The servlet does not corrupt requests or results for concurrent searches. Servlet performance does not degrade steeply for reasonable numbers of clients (given machine power and hardware).
The servlet handles garbage client requests gracefully.
System tests run alongside the servlet and clients. Some of the system tests (comparing documents) may be at least partially manual.
Installation tests are defined and developed as the installation procedures are being defined. They verify the following.
Installing over an existing copy does not overwrite what has been changed by the user.
Improper installation configuration causes the servlet to provide useful, intelligible diagnostic messages.
Installation procedures are clear and easy to follow. Given that all the prerequisite software is correctly installed, installation of the default servlet (without added RefEntry documents) should take no more than 10-15 minutes for a novice.
Development and installation of modified servlets works as documented.
| <<< Previous | Home | Next >>> |
| Servlet Packaging | Test Results |