Release Notes for WebMan: A Basic RefEntry Servlet Mark Craig Last Updated: 27 March 2003 -------------------------------------------------------------------------- As described in the product documentation, this project spanned 4 deliveries. For each delivery, I've written some notes. Delivery 4 ---------- Looks okay. The outstanding "bugs" I see at this point are primarily limitations of the design. One really nice thing to have would be a native installation method, so for example, you'd have an Install Shield executable that sets registry keys on Windows, RPMs on RedHat, Solaris packages on Solaris, etc. That turns out to be lots more work than I expected. I saw specs for a sort of universal installer defined using the Java Community Process, but did not investigate whether a reference implementation is available. Fixed Bugs ---------- Bug - 4 : It appears that the performance issue was related mainly to my having forgotten something I learned in CS310 about how programming languages are handled, and in CS260 about Java. I was having my servlet needlessly implement SingleThreadModel because I was worried not that the backend would be corrupted, but that concurrent access to the servlet would spoil sets of data I was constructing inside the doGet() method of the servlet. As Josu Vegara, a colleague who does lots of Java development, explained to me -- and I'm sure I'd seen this before -- I'd only have to worry about concurrency if I'm modifying class member data *outside* the scope of the method, because the JVM is going to give each thread its own memory for the method and objects created therein. So I removed the SingleThreadModel interface, and performance increased a little more than 10x. For 100 threads: time in sec ------------------ max 2.109 min 0.012 avg 0.360 stddev 0.417 Delivery 3 ---------- Well, I had everything ready the second week of February, then deleted everything I'd done with a /bin/rm -rf * accident before making a backup. That was so depressing I stopped looking at the project for two weeks. This is the first delivery for which the servlet works in Tomcat. You still need to download Tomcat from http://jakarta.apache.org/ before you can try it out. I've run into a weird Tomcat "feature" on Windows 98. When Tomcat is started on my portable under Windows, the processor runs full blast until the fan comes on and the touchpad heats up. Maybe it's a Java/DOS thing. I've had some trouble testing whether the SingleThreadModel interface truly allows for multiple clients or not. The idea is that the servlet shouldn't slow down when multiple clients are querying it. I tried to check that using a simple Java test that runs a bunch of requests against the servlet, using clients that are Java Threads. I get the impression that once the servlet is up, it does two things to defeat my simple test. First, Tomcat seems to cache HttpServletResponses or something so that after you've requested the same thing a couple of times, it takes the servlet almost no time to respond to that request. Second, since I've modified the design to put formatted strings in the serialized backend the servlet loads on init(), everything is in memory and it takes less time for the servlet to respond to an HTTP request than it takes the Java virtual machine to instantiate a client Thread and all the objects inside it! In any case, you don't have to wait long for a response. I found that even when running the multi-threaded test on two machines, one of them an aging but big Sun multiprocessor server, to attack the servlet, response time was in the 100 millisecond range when running Tomcat on a 400 MHz workstation. That includes the time to build the Threaded requests, to send them, and to get them back over the Ethernet from a different IP subnet. So I don't think performance is a problem for small numbers of relatively small RefEntry documents at least. I'd like to take all the man pages on Solaris and massage them to work with my DTD, then build a servlet containing all of it, but I don't have the time to do that during this project. My guess is that until images are included, a servlet with static content like this won't take up much memory, and will be pretty quick. The trade off is that you have to build everything in from the start. --- After much wailing and gnashing of teeth and an occasional strong desire to rend my garments, I've more or less understood how to use Ant and get this whole thing to operate under Tomcat. During integration, I discovered some things I'd done earlier -- like add a RefEntrySearch class, or try to write my own servlet builder instead of just yielding to Ant -- that no longer made sense. This has significant documentation impact. I finally find myself having some real, hard won sympathy for my developer colleagues who are forever struggling in the straitjacket of backward compatibility with the bugs we never fixed in earlier releases (so users devised workarounds that now rely on particular aspects of existing brokenness). I also still get to sympathize with myself as tech writer: Those shifty developers never implement what they said they would! Fixed Bugs ---------- Bug - 1 : This one had me going round and round in circles trying to understand how to instantiate a real SAX parser in Java... until I finally gave up and tried just what the doc said, using "org.apache.xerces.parsers.SAXParser" as an argument to XMLReaderFactory.createXMLReader(). That worked first try ;-) Bug - 2 : I factored RefEntrySearch out of the project. For the other two, I'm using Russell Gold's ServletUnit (comes with his HttpUnit) framework to run tests on this. Bug - 3 : I was trying to use the id() XSLT function with NMTOKEN id attributes. Now, ID type attributes are used and the xref linkend attribute is of type IDREF. Appears to work fine. Known Bugs for this delivery ---------------------------- Bug - 4 : Performance for many simultaneous requests is 10 times too slow. Using the TestThreads unit test on a multi-processor server shows that for 100 threads, the average time to respond to a request is over 20 seconds. Delivery 2 ---------- Draft modules completed. The friendly methods on the objects seem to work, but I haven't checked absolutely all public and friendly methods on all the objects. The Format command for generating individual output pages seems to work relatively well. I'll be checking the Build command when I move to Delivery 3 and am working on the integration. Draft stylesheets completed. The stylesheets appear to work as required. I learned here using a trial version that XML SPY, the commercial XML editor, is nice to have when working on big XSL stylesheets. On the other hand, it didn't seem to work at all for DTDs. Maybe I got the wrong trial version. Documentation updated, including moving the DTD chapter to an appendix (too long and reference-like for a chapter). Many bug fixes concerning design errors and reference entries for DTD elements. Unit tests operational for both stylesheets and components. Using JUnit for the unit testing, to prevent regressions. I probably didn't factor the stuff well enough since implementing full unit tests for the Servlet and Search classes would involve probably lots of mock objects. Known BUGS for this delivery ---------------------------- Bug - 1 : The Format tool does not do validation. The Format tool does not check whether an input file is valid XML according to the DTD. So you can feed it garbage, and it just chokes. A workaround is to use the Multi-Schema Validator written by Kohsuke Kawaguchi, and run from the command line. After validating your XML against the refentry.dtd, Format should not choke on it. Bug - 2 : The AcceptanceTest does not really work for the RefEntryServlet, the RefEntrySearch, and the RefEntryTransformer. The Servlet and the Search objects can be debugged as things are combined into the Servlet. My sense was that it was going to be harder to use the Mock Objects -- harder to write good ones -- than to debug after the Servlet was ready to put into the container. The RefEntryTransformer problem is that I can compare two documents with JUnit, but *any two RefEntry output documents will always differ* because the date is embedded. I figure I can workaround this problem by inspecting the output run on bigtest.xml, which includes about everything. Bug - 3 : The xref element is not transformed correctly. The link from the xref to the target element works correctly, but the link text generation does work yet. I've spent a while trying to debug this, and am convinced it's something simple that I'm just missing in the XSLT. Unfortunately, I cannot find anyone local who can help me. So I'm going to put it aside for a while and come back to it when I've forgotten the context and can look again with fresh eyes. Delivery 1 ---------- Draft design completed. DTD specified. Formatting expectations specified.