I've always been a big fan of Java, especially since its promise "Write once, run everywhere" comes very close - at least if you compare the situation with the C/C++ world 8 years ago (and take all the libraries into account).

However, this promise nearly killed my Christmas last year. Ever used the new fancy ScheduledThreadPoolExecutor? Nice class. Easy task handling in a multi-threaded application design (at least if you dont try to extend it, but this is another story). I've developed my application on Windows, we did extensive tests in a Linux environment and when we delivered the application to our customer's location Friday before Christmas it crashed quite instantly. Within the logger! What's that?

Well, the application opens a port for remote control, that enables to manage running tasks. If it has to shutdown, it interrupts its running threads, so that the internal thread pool of the ThreadPoolExecutor is cleared and all running tasks are informed if they monitor their interrupt flag. If a severe error occurres the application will initiate itself such a shutdown sequence. Fine.

Now, the initial problem has been a simple misconfiguration that caused the shutdown sequence, but this has been completely hidden by a thrown InterruptedIOException when the application tried to log about the problem. But why on earth was this exception thrown?

As it turned out, this is a difference in the behaviour between the different Java platforms. If a thread is interrupted normally an InterruptedException is thrown when the thread is already waiting or calls such a method like wait(), Thread.sleep() or join(). However on some platforms every call performing an IO operation is cancelled with an exception as well: an InterruptedIOException is thrown. The JVM on Solaris is such a case. And unfortunately every dumb log statement in your code will perform an IO operation ... :-/

What was my solution? In the end I did a quite ugly hack. I subclassed the Thread class used in the pool and overwrote the isInterrupted() and interrupt() method to support some kind of soft-interrupt mode. Additionally I noticed that those InterruptedIOException were thrown out of the native code of the JDK and isInterrrupt() was never called, so the JDK checks here the native state of the Thread. Same applies to Thread.interrupted(), it will also not call the isInterrupted() method of a Thread instance. Unfortunately I could not switch completely to the soft-interrupt mode, since all idle threads in the pool of the ThreadPoolExecutor will no longer terminate at the end of the application, if the interrupt() call is not passed-through to the original implementation. Therefore I subclassed the Runnable and Callable instances that where called periodically be the ScheduledThreadPoolExecutor to enable the soft-interrupt mode before the task is executed and disabled that mode again after the task finished and the thread is turned back into the pool again. The called code must check the thread's state with Thread.currentThread.isInterrupted() though.

A major help to test this was an own MemoryLog implementation that simply also throws an InterruptedIOException if the native thread has been interrupted (Thread.interrupted() returns true).