Recently we organized an AppServer shootout (we call it deatmatch) on our local Java User Meeting. The three contestants would have been Sun's Glassfish v2, Redhat's JBoss and IBM's Geronimo. Unfortunately IBM guy couldn't attend on this event because of other business and JBoss presentation had technical difficulties so we had to do with Glassfish demo. Let's see the 8 tasks:
1. Deploying simple EJB3/JPA webapplication with pre-initialized DB. Checking WSDL as a bonus.
It was easy and elegant. We checked WSDL which was an ugly and large XML but it's been generated automatically. We did a test request by using the Glassfish's admin console. That means we filled an HTML form containing one of the webservice's method's parameters and submitted it.
2. Remote deploy from build script. (Usual development task.)
We did it from Netbeans using Ant and a special Glassfish deploy task. AFAIK hot deploy is also supported by Glassfish when appserver scans a pre-set directory and our task is to plump ear or war file into that place.
3. Changing logging config during runtime.
We did it by using the admin console and logs could be browsed by a fancy HTML table. Severity level of loggers could be set by logger instance as usual. The Test application was Glassfish-friendly because it used Java Util Logging and 'fortunately' Glassfish also supports that (and only that?) logger framework. I think log4j config could had been happened by editing log4j.xml by hand and we had to pass up the fancy HTML log table, nevertheless I would have been happy with that since I prefer anything else than java.util.logging.
4. Remote runtime-debugging of the application.
It was OK, however Glassfish must have been restarted each time before connecting debugger client. As I remember JBoss doesn't need restart when a certain JVM switch is set. Of course it needs when we want to change that switch. We used Netbeans as a debugger client but it would have been anything else, for example Eclipse.
5. Changing data-source.
We expected it won't work without restarting the application and of course it did (not). Other expectation could be against the appserver to not to totally crash down in this case. It didn't crash, just the application started to throw ugly excpetions with long-long stack traces. Changing datasource had been worked after restarting application.
6. Restarting appserver with the application. (How much time does it take?)
We restarted appserver, but unfortunately forgot to measure it with stopwatch. It could have been 10 seconds.
7. Integrating LDAP (if we have time)
Guys started to configure it but there were some technical difficulties. They said usually it can be done in 10 minutes. We wanted to see the stress test so we jumped.
8. Stress test with JMeter
That would have been exciting if we had the other contestant appservers running on the same hardware, unfortunately we had not. However, a plain dual-core notebook could serve 1700 requests per second without tuning, and 2200 with tuning.
Tuning tips we used.
Why didn't we use Apache Benchmark.
We used JMeter.
I'm looking forward to the next occassion where we'll have hopefully other competitors with sharpened environments.
Beyond this appserver stuff, we had a presentation about component models and a BlazeDS, Flex prototype app.