Test Plan and Results

This chapter lays out the testing strategy, specifies the specific tests used to qualify the application, and includes the resulting test report.

Test Plans

The cover the project documentation, the webman client, the Format utilities, the build process, and the RefEntry servlet itself.

The tests verify that the documentation:

The tests verify that the webman client:

The tests verify that the Format utility:

The tests verify that the build process:

The tests verify that the RefEntry servlet:

In order to verify each of these points, functional, system, and installation tests are run for the second, third, and fourth deliveries respectively. Tests continue to be developed as the software and documentation take shape for the delivery. Before each delivery including software tests, a test result report is completed, and a list of bugs compiled. Bugs that are not to be fixed are identified as such in the release notes for the delivery.

The test results are cumulative. Also, once tests are developed, they are run regularly to check for regressions in the software.

Functional Tests

Functional tests are developed during implementation. Their overall behavior is complete when the interface definition is complete. They verify the following.

  • All public and friendly methods respond as documented.

  • All elements are formatted as described in the documentation.

Functional tests may be installed alongside the software and may be run through the JUnit test framework.

System Tests

System tests are defined and developed during components integration. Their overall behavior is complete when the components are integrated together. They verify the following.

  • Configuration file content has the documented effect on the build results.

  • Concurrent searches may be run against the servlet. The servlet does not corrupt requests or results for concurrent searches. Servlet performance does not degrade steeply for reasonable numbers of clients (given machine power and hardware).

  • The servlet handles garbage client requests gracefully.

System tests run alongside the servlet and clients. Some of the system tests (comparing documents) may be at least partially manual.

Installation Tests

Installation tests are defined and developed as the installation procedures are being defined. They verify the following.

  • Installing over an existing copy does not overwrite what has been changed by the user.

  • Improper installation configuration causes the servlet to provide useful, intelligible diagnostic messages.

  • Installation procedures are clear and easy to follow. Given that all the prerequisite software is correctly installed, installation of the default servlet (without added RefEntry documents) should take no more than 10-15 minutes for a novice.

  • Development and installation of modified servlets works as documented.