I've finally begun construction on the Entertainment Center project I've been planning for several years now.





I got the plywood a few weeks ago and have begun cutting it down into parts. Thankfully the garage hasn't been too cold.

I've always liked TCPView from SysInternals as a graphical way of viewing open network connections by process and port on a Windows machine.

Now I have a new reason to like it. I needed to do some testing of how our application handles network failures. I was figuring I'd have to pull network cables or other painful procedures to simulate this. However, I found that TCPView has an option to let you close a network connection through the right-click menu:

We recently found and resolved an issue at work that was sufficiently tricky I felt that notes should be shared for anyone else who runs into the problem.

Background

We are a Java shop using an AS/400 (iSeries V5R4) as our backend. That means we're using the jt400.jar for our JDBC driver.

The Problem

We recently encountered a problem where our database calls would start failing intermittently. The problem manifests under load. That is, when a few hundred JDBC calls are made, eventually they will start to "fail". This failure can take several forms, but most often appears as either an ArrayIndexOutOfBoundsException inside the jt400 driver code, or some other exception indicating the data passed to/returned from the AS/400 is malformed. Alternately, the call can succeed without an error, but the data is wrong. For example: - With the our login stored procedure, when it failed, it returned errors indicating the username couldn't be found. - With another stored procedure it threw SQLException: Descriptor index not valid. Debug logging indicated this was a wrapper exception around an ArrayIndexOutOfBoundsException - A simple INSERT call would eventually fail complaining that one of the integers being inserted was malformed even though all tracing indicated it was a valid integer. This behavior only appeared under a limited set of circumstances:

• JDK 1.6 (1.6.0_12) - The problem never appeared with 1.5

• Multiple prepared statements. The problem appeared was related to preparing statements repeatedly. I was able to work around the issue in one application by upgrading our version of iBATIS from 2.2 to 2.3. That release caches prepared statements. This doesn't solve the problem for another app due to the way that Spring and iBATIS interact that prevents iBATIS from caching statements.

• JVM HotSpot optimization ON (-server). One of the things I learned in tracking this down is that the HotSpot VM is enabled by default in Solaris and disabled (-client) by default on PCs. This problem went away when we disabled HotSpot optimization.

The problem does not appear related to a particular version of the jt400.jar.

Solution

One solution is to disable HotSpot entirely. We didn't want to take this drastic step and lose the performance gains, so we figured out how to disable HotSpot optimization for a specific class. [1] Our method was to turn on -XX:+PrintCompilation to have the JVM tell us every time it optimized a class/method. Then, we looked for the most recent optimizations before a failure and told the JVM to exclude those. Exclusion can be done through a command line parameter, so we used the following to exclude commonExecuteBefore() method of AS400JDBCPreparedStatement. The option is:

-XX:CompileCommand=exclude,com/ibm/as400/access/AS400JDBCPreparedStatement,commonExecuteBefore

[1] The Print Compilation Flag and how to read it's output

CodeGeist III is almost over. I just noticed there were several last minute entries, one of which looks very clever, the HTMLDiff plugin. It displays the difference between two page versions as they appear in the browser, not at the wiki source level. That seems very nice.

My plugin is ready. I put finishing touches on the plugin page for the documentation. I really wish I could have gotten more testing. People seemed interested in the plugin, but it wasn't clear how many were actually trying it. I wonder how many Confluence users have 2.7.2+ installed such that they can try it. It'll be interesting to see if activity picks up if I manage to backport it to 2.5 or earlier.

I just checked and it looks like they have until June 15th to judge the contest, so I guess I'll be waiting a while for the results. Keeping my fingers crossed for more than a month is going to make typing dffjclt. :)

I'm at JavaOne next week. I'm glad I decided to post my entry to the Codegeist competition last weekend as it has taken all week to get it to the point that it feels stable. There's a lot more tweaking that can be done to it, but I feel like it's pretty much ready to be judged (javadoc is partially done and I'd like to have more unit tests if possible).

Unfortunately, I haven't gotten much in the way of feedback beyond people who are excited by the idea. I only know of one user who tested it for sure and he only reported one bug. I know there must be more, but I'm not sure how to encourage further testing.

If you want to meet up at JavaOne, I'll be the guy with the backpack. :)

I'm submitting a Confluence plugin for the Atlassian Codegeist III competition. I've been working on and off on a plugin to allow attaching images via copy n' paste and drag n' drop and this contest motivated me to finish it, or at least get it in a working, useable state. Here's the page for my entry: Image Attacher Plugin

Quite a bit late in coming, but I wasn't chosen as a speaker for JavaOne 2008. Based on the presentations they've picked, I think I had the problem of being slightly off the topics they wanted to cover and trying to cover too much in one presentation. I may try again this November if I feel that I can craft a presentation they're more likely to accept.

Also, they never contacted me at all except to indicate I hadn't been chosen. Again, not surprising considering how many they had to review, but it suggests you should assume that you won't get a chance to clarify anything for the judges after you've submitted.

I'm putting the final touches on a paper I'm submitting for the annual JavaOne Call for Papers. In my preparation last night I had some questions about the process that I sent to j1papers@sun.com . I thought the answers might be generally useful, so they're reproduced below:

Q: What is the attachment field for? Can I submit a PDF with my presentation outline and code samples?

A: Yes, this is exactly what this "field" is for. It gives the review team more insight into your proposal.

Q: How interactive is the review process? Does my submission need to be perfect, or will the committee be willing to accept the presentation but provide feedback for tailoring it? I'm willing to narrow the focus if they think it's too broad a topic or I'm covering too much.

A: We try to make the process interactive. The reviewers have access to your contact information and may engage in discussion with you regarding details that may help in their decision making.

I hope that helps other/future presenters. I'll try to post more information as I go through the submission process.

I tracked down a very frustrating error today that turned out to be a non-issue. The problem was in Eclipse with a new project I was configuring for one of our developers. The project is supposed to produce 1.3 compatible classfiles and Eclipse was giving me the following error:

'Incompatible class files version in required binaries' followed by the path to the rt.jar file in my JDK directory.

I couldn't see what was different about this developer's system from all the others that were similarly configured without error. I finally figured out that this error has a setting under Preferences... -> Java | Compiler | Building -> "Incompatible required binaries". The developer having trouble had this setting set to "Error". On my system, it was set to "Ignore", which apparently is not an issue.

I suppose we might run into trouble in the future if we try to use the jars produced in a 1.3 JVM, but I kind of doubt it.

Java generics are cool.

This:

Collections.sort(administratableUsers,
   new Comparator<user>()
   {
       public int compare(User u1, User u2)
       {
           return u1.getUsername().compareTo(u2.getUsername());
       }
   });

is much cleaner than:

Collections.sort(administratableUsers,
   new Comparator()
   {
       public int compare(Object o1, Object o2)
       {
           User u1 = (User)o1;
           User u2 = (User)o2;

           return u1.getUsername().compareTo(u2.getUsername());
       }
   });

Someone on this Javalobby post asked me to elaborate on our use of Ant and copy filtering as a build configuration mechanism, so here it is.

The problem: projects tend to have lots of different configuration files that need to be changed based on the machine or circumstance in which they are installed. Local file paths, database passwords, and host names are just a few of the settings that typically need to be set. Frequently, the same value appears in multiple configuration files allowing for the problems always associated with dual-maintenance. Also, it's often inconvenient to locate all the configuration files in the same place, so a user (a developer or configuration engineer) needs to know where they all live in order to configure them (web.xml lives in WEB-INF, log4j.xml lives in the classpath, etc.)

The solution: We use Ant to solve this problem. To address the dual-maintenance issue, we have a single file that a user needs configure called build.properties that lives in the same directory as the build.xml file. It's typically copied from another file called build.properties.template that lives in CVS. Any values in build.properties override those in build.properties.template.

Example build.properties.template:

uk.co.ourCompany.mainApp.hostname=localhost
uk.co.ourCompany.mainApp.basedir=/home/builder/src/mainApp

Example build.properties:

uk.co.ourCompany.mainApp.hostname=bender.ourCompany.co.uk
uk.co.ourCompany.mainApp.basedir=c:/src/mainApp

In our Ant build process, we have tasks that handle configuring all of our various files. They look like this:

<target name="setupWebXml">

  <copy overwrite="true" 
      file="${uk.co.ourCompany.mainApp.basedir}/web/WEB-INF/web.xml-template"
      tofile="${uk.co.ourCompany.mainApp.basedir}/web/WEB-INF/web.xml">
     <filterset refid="build.propertiesFilter" />
     <filterset refid="build.properties.templateFilter" />
  </copy>
   
</target>

The filtersets are defined like this:

 <!-- The following two filter sets are referenced in each of the following config targets.
     First the build.properties tokens are replaced, then the build.properties.template
     tokens are replaced (only if they didn't already exist in build.properties) -->
<filterset begintoken="{" endtoken="}" id="build.propertiesFilter"
         description="Used to parse tokens in config files into their associated values in build.properties.">
  <filtersfile file="build.properties"/>
</filterset>

<filterset begintoken="{" endtoken="}" id="build.properties.templateFilter"
         description="Used to parse tokens in config files into their associated values from build.properties.template.">
  <filtersfile file="build.properties.template"/>
</filterset>

Here's a snippet of the web.xml-template file that becomes web.xml:

 <context-param>
     <param-name>Hostname</param-name>
     <param-value>{com.lollipoplearning.devEditHost}</param-value>
 </context-param>

That's an artificial example for the purposes of this article; there are better ways to get the hostname, if you should be getting it at all.

Tip: At the top of each template file, put in a note (we actually have a token that also gets replaced) that warns users that the file has been generated and they should not edit it. It's very frustrating to make changes to a file and find they don't take effect because they've been clobbered by the build process.

I wanted to make a note of a solution I found today in the hopes that it will be helpful to others and easier to find.

Until today, hot-syncing my Sony Clie PEG-SJ22 with my Dell Inspiron laptop was so slow as to be useless (more than 20 minutes before I gave up and cancelled the operation). I finally was able to track down a fix via a lot of googling and came up with the following.

The problem is the with the SMC IR driver used by default in WinXP. Apparently, it tries to communicate much faster than the Clie can handle. Throttling it back via the driver controls doesn't fix it. The trick is to replace the SMC driver with the generic Windows IR driver. This means the following:

1. Go to Control Panel -> Wirelss Link -> Hardware tab -> Properties for the IR device -> Driver tab
2. Click Update Driver...
3. Install from specific location. Next.
4. Don't Search. I will choose the driver to install.
5. Uncheck Show Compatible Hardware
   
6. Choose Manufacturer (Standard Infrared Port)
7. Device: Built-in Infrared Device

That's it. Reboot and everything should be groovy. Of course, groovy means a poky transfer speed of 115Kbps, so even a useless hotsync (right after a complete hotsync) takes 1-2 minutes. But that's useable if you don't have a USB cable handy.

I wanted to post the following "instructions" in case they'd be of use to anyone else.

My new hard drive came today (80GB Hitachi Deskstar SATA) and I was able to image the old parallel IDE drive to the new one without a hitch. Just to be clear, what I wanted to end up with was the new drive exactly replacing the old one, but with more space. I may try to use the old one as a secondary drive later, but not right away.

Here's what I did (expanding on these Instructions)

1) Installed the SATA drivers for my Motherboard (K8T-NEO FSR) into Windows.

2) Burnt the latest Knoppix Linux-on-CD (3.4) to disc.

3) Rebooted with the CD in the drive. Gave it the "knoppix26 lang=us" start option. If you go with the default, you get the 2.4 Kernel and it won't support SATA.

4) Opened the root shell and unmounted the C: drive ' umount /mnt/hda1'. Double-checked that this worked by running 'mount' and verifying neither hda nor hdg (the device assigned to my Sata drive) appeared. BTW, I didn't expect it to be /hdg. I only noticed the name during the boot sequence.

5) From the root shell, I ran 'dd if=/dev/hda of=/dev/hdg'. That basically says, copy every byte from /dev/hda to /dev/hdg. This took about an hour to do 20GB. You can check on the process while it's running by doing a 'kill -SIGUSR1 <pid of the dd process>'. If you don't know Unix, don't worry about it. Just be patient. For reference, I was getting about a 9MB/s transfer rate.

6) At this point, the new drive is exactly like the old one, plus a bunch of unusable wasted space. To enlarge the 20GB partition to 80GB, I used QtParted (partition editor) which comes with Knoppix.

7) I disconnected the old IDE drive and then had to tell my BIOS to use the SATA drive at boot time.

That's it.

I want to mention a cool plugin I recently found for Eclipse. It's called implementors and it addresses a frustration I've had for a while. It has to do with interfaces and the implementors of those interfaces. We try to write all our code to interfaces and then write implementing classes to handle the actual processing. So if you need to interact with the database, you obtain an instance of IDataService. But what happens when you want to look at the code behind one of those methods? Pressing F3 (go to definition) takes you to the interface, no terribly helpful. What I really want is to go to the concrete class, DataServiceImpl, that is always behind that interface. The implementors plugin gives you Alt-F3 that takes you straight to the implementation. Beautiful.

I had a need today to dynamically add an onload handler to a web page, if a certain block of <script> was included in the page. Not wanting to overwrite any previously assigned onload handlers, this page had the solution.

Last month I built a JSP and Servlet-based list paging framework to take long lists of items and display them over multiple pages, much like google. Before I did this, I of course looked around for free, public implementations of the same thing as I see no reason to re-invent the wheel. However, I wasn't able to find one to fit my needs. JSP Tags has a pager library that seems to be very popular, but it requires roundtrips to the server for each page and wants to manipulate the URL for its refreshes. I wanted to avoid (obvious) roundtrips and have more control over the refresh behavior.

It should be noted that I only had IE 5.5+ (IE6, really) as a target platform, although I usually make reasonable attempts to keep the latest Mozilla happy as well. This is a nice luxury I know and it likely made my solution more workable.

There are essentially two parts to my framework. On the server side, there's the ListPager object that handles breaking a list into "pages" as well as filtering items out of the list that don't match a given Predicate. This object is built by a ListPagerFactory and then stuffed into the request. The request itself is initiated in a hidden IFRAME that is created by the other half of the framework, the listPager.js javascript file. This script is responsible for initiating requests to the server for new pages and then handling a "callback" indicating the latest page has loaded and should be integrated into the visible page.

[...blah blah, more explanation how it works...]

The reason I felt I should blog this is that I'm debating internally if I should release my framework to the public, probably via SourceForge. I feel like I need to justify why the world needs another paging framework. I guess I'm unsure if the answer "well, I needed one" is good enough.

I'll keep thinking about it...

Something recently pointed me to the following article about the Spring Framework. Although it's a long read that I'm still digesting, it sounds very intriguing. Definitely something I'm going to consider the next time I start a new project.

I just want to write what a good experience I've had recently with HTMLArea, the WYWSIWYG HTML editor that replaces textarea elements in your web page. This thing is so easy to use and configure. I was able to drop it in in less than 15 minutes including unpacking the .zip and choosing which buttons I wanted to appear. I also tried FCKEditor, but it doesn't allow the setting of text colors, which I felt was a requirement. I understand that <font> is deprecated and I should use styles, but these WYSIWYG controls just don't work well for that paradigm, IMHO. The users they are targeted at understand Word, not HTML/CSS.

Sweet! Someone just added a comment which means my blog is somehow reachable. I've done nothing to advertise it, even to friends, and I was curious to see if it would get any traffic. Apparently, it's not only reachable but helpful too, which is even cooler.

Just started setting up Tomcat 5 today and wanted to leave some notes on a few things I had to do to get it to work.

I had to change the doctype of my web.xml. This page had an excellent example of what to do.

I added this to enable EL in JSPs

<jsp-config>
<jsp-property-group>
<url-pattern>*.jsp</url-pattern>
<el-enabled>true</el-enabled>
<scripting-enabled>true</scripting-enabled>
</jsp-property-group>
</jsp-config>

I had to replace my Jakarta Taglibs Standard 1.0 library with the 1.1 version. 1.0 is for Servlet 2.3, 1.1 is for 2.4. I discovered this when the JSP compiler complained about an EL expression in the value attribute of the <c:param> tag.