Topic
  • 10 replies
  • Latest Post - ‏2015-02-04T00:36:57Z by KLangman
FSuchy
FSuchy
4 Posts

Pinned topic Multi-catch can cause finally block to be skipped.

‏2014-06-30T21:23:24Z |

I have discovered a bug with multi-catch in the IBM Java 7 implementation that can lead to a finally block being skipped.

For example, in the following code snippet, the call to finallyChecker.inFinally() may not be executed.

    FinallyChecker finallyChecker = new FinallyChecker();
    try {
        throwIt();
    }
    catch (BindException | NoRouteToHostException | PortUnreachableException e) {
        throw e;
    }
    finally {
        finallyChecker.inFinally();
    }

I've done some testing with variations and have narrowed the conditions to:

  • A multi-catch block is used to catch more than one exception.
  • An exception is thrown from the try block that matches anything but the first exception in the multi-catch.
  • An exception is thrown from the catch block.  It can be a new exception, or the caught exception.

Some observations:

  • It appears to work for a while, then will fail consistently.
  • When the first exception in the multi-catch list is thrown, the finally block is always executed.
  • This can lead to some rather serious problems, including memory and resource leaks, and unpredictable behavior.  The only work around is to not use the multi-catch feature of Java 7, which may not be an option when using 3rd party libraries.

I have been testing with the latest Linux version (V7 Release 1, 07 May 2014).

This behavior is not present in the Oracle implementation.

The following code can be used to verify the defect.  When run, it will begin looping over the try/catch/finally test.  Every 5 seconds it will report how many tests it has run so far, how many times the finally block was executed, and how many times it was skipped.

import java.net.BindException;
import java.net.NoRouteToHostException;
import java.net.PortUnreachableException;

/**
 * Tests for finally block execution.
 * 
 * I found that under certain conditions, the finally block will not be executed. The conditions are:
 *  - IBM JRE version 7, up to at least V7 Release 1 (07 May 2014).
 *  - A multi-catch block is used to catch more than one exception.
 *  - An exception is thrown from the try block that matches anything but the first exception in the multi-catch.
 *  - An exception is thrown from the catch block. It can be a new exception or the caught exception.
 */
public class FinallyFailProof {

    private final static int ACTIVITY_INTERVAL = 5000;

    private final static String HEADER_FORMAT = "%20s%20s%20s" + System.lineSeparator();

    private final static String RESULT_FORMAT = "%20d%20d%20d" + System.lineSeparator();

    private final Counts counts = new Counts();

    public static void main(String[] args) {
        new FinallyFailProof().runTest();
    }

    public void runTest() {

        long now = System.currentTimeMillis();
        long nextActivityTime = now + ACTIVITY_INTERVAL;

        System.out.printf(HEADER_FORMAT, "Tests Executed", "Finally Success", "Finally Fail");
        System.out.printf(HEADER_FORMAT, "--------------", "---------------", "------------");

        for (int i = 0; i < Integer.MAX_VALUE; i++) {
            try {
                catchIt();
            }
            catch (Exception e) {
                // Expected
            }

            now = System.currentTimeMillis();
            if (now > nextActivityTime) {
                reportCounts();
                nextActivityTime = now + ACTIVITY_INTERVAL;
            }
        }

        reportCounts();
    }

    private void reportCounts() {
        System.out.printf(RESULT_FORMAT, counts.created, counts.inFinally, counts.finalized);
    }

    private void catchIt() throws BindException, NoRouteToHostException, PortUnreachableException {
        FinallyChecker finallyChecker = new FinallyChecker();
        try {
            throwIt();
        }
        catch (BindException | NoRouteToHostException | PortUnreachableException e) {
            throw e;
        }
        finally {
            finallyChecker.inFinally();
        }
    }

    private void throwIt() throws BindException, NoRouteToHostException, PortUnreachableException {
        throw new NoRouteToHostException();
    }

    private class Counts {

        long created = 0;

        long finalized = 0;

        long inFinally = 0;
    }

    private class FinallyChecker {

        private boolean closed = false;

        FinallyChecker() {
            counts.created++;
        }

        @Override
        protected void finalize() throws Throwable {
            super.finalize();
            if (!closed) {
                counts.finalized++;
            }
        }

        void inFinally() {
            closed = true;
            counts.inFinally++;
        }
    }
}

 

  • KLangman
    KLangman
    7 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2014-07-18T20:28:41Z  

    Does the problem still occur when you use the -Xint Java command line option?

  • FSuchy
    FSuchy
    4 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2014-07-18T21:10:33Z  
    • KLangman
    • ‏2014-07-18T20:28:41Z

    Does the problem still occur when you use the -Xint Java command line option?

    The problem stops with the -Xint option.  Interesting...

    IBM JIT bug then I suppose?

     

  • KLangman
    KLangman
    7 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2014-07-18T21:13:01Z  
    • FSuchy
    • ‏2014-07-18T21:10:33Z

    The problem stops with the -Xint option.  Interesting...

    IBM JIT bug then I suppose?

     

    Seems likely. Let me take a look and get back to you. I assume the test-case provided is something I can just compile and run? I'll try it out.

  • KLangman
    KLangman
    7 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2014-07-18T22:32:50Z  
    • FSuchy
    • ‏2014-07-18T21:10:33Z

    The problem stops with the -Xint option.  Interesting...

    IBM JIT bug then I suppose?

     

    It has something to do with putting multipal object types in the catch statement:

            catch (BindException | NoRouteToHostException | PortUnreachableException e) {
                throw e;
            }

    If you expand it out to use different catch statements for each type then it works:

            catch (BindException e) {
                throw e;
            }
            catch (NoRouteToHostException e) {
                throw e;
            }
            catch (PortUnreachableException e) {
                throw e;
            }

    Not sure if this is a JIT issue or an interaction between the JIT and the VM. The problem still occurs when the JIT is only allowed to compile without optimizations.  I'll keep looking at this next week.

  • FSuchy
    FSuchy
    4 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2014-07-22T14:03:40Z  
    • KLangman
    • ‏2014-07-18T22:32:50Z

    It has something to do with putting multipal object types in the catch statement:

            catch (BindException | NoRouteToHostException | PortUnreachableException e) {
                throw e;
            }

    If you expand it out to use different catch statements for each type then it works:

            catch (BindException e) {
                throw e;
            }
            catch (NoRouteToHostException e) {
                throw e;
            }
            catch (PortUnreachableException e) {
                throw e;
            }

    Not sure if this is a JIT issue or an interaction between the JIT and the VM. The problem still occurs when the JIT is only allowed to compile without optimizations.  I'll keep looking at this next week.

    Yes, these were my findings also.

    Also note that, in a multi-catch statement, it only seems to fail when an exception other than the first in the list is thrown.  IE, in my example, it would work correctly if BindException was thrown instead of NoRouteToHostException or PortUnreachableException.  This may be a clue to the underlying issue.

  • KLangman
    KLangman
    7 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2014-07-22T17:22:35Z  
    • FSuchy
    • ‏2014-07-22T14:03:40Z

    Yes, these were my findings also.

    Also note that, in a multi-catch statement, it only seems to fail when an exception other than the first in the list is thrown.  IE, in my example, it would work correctly if BindException was thrown instead of NoRouteToHostException or PortUnreachableException.  This may be a clue to the underlying issue.

    Yep.. I discovered that as well.

    The problem is this:  The JIT likes to have one catch block for each type. In the case of the new Java7 syntax (and for rare cases in where non-standard bytecode generators are used) it's possible for one block of code to be the target for two different exception types. In these cases the JIT will clone the exception handler block(s) for each additional type, but (in some cases) it does not seems to clone the "exception out" set meaning that the implied "try" region around the exception catcher block(s) used to insure that the finally blocks are executed in the case an exception in thrown by the catcher is not copied to the cloned handler block(s).


    In any case, this is a bug in the JIT very early in the compile, during the time when the JIT converts the bytecodes into our intermediate representation.  We are working on determining how best to fix this right now, it's a problem is a rather sensitive area of the code so we need to do a lot of testing to insure we don't muck up anything for other areas of the JIT that might be expecting a particular cloning behaviour.


    I'll let you know when we have a final solution.

  • FSuchy
    FSuchy
    4 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2014-07-22T17:38:33Z  
    • KLangman
    • ‏2014-07-22T17:22:35Z

    Yep.. I discovered that as well.

    The problem is this:  The JIT likes to have one catch block for each type. In the case of the new Java7 syntax (and for rare cases in where non-standard bytecode generators are used) it's possible for one block of code to be the target for two different exception types. In these cases the JIT will clone the exception handler block(s) for each additional type, but (in some cases) it does not seems to clone the "exception out" set meaning that the implied "try" region around the exception catcher block(s) used to insure that the finally blocks are executed in the case an exception in thrown by the catcher is not copied to the cloned handler block(s).


    In any case, this is a bug in the JIT very early in the compile, during the time when the JIT converts the bytecodes into our intermediate representation.  We are working on determining how best to fix this right now, it's a problem is a rather sensitive area of the code so we need to do a lot of testing to insure we don't muck up anything for other areas of the JIT that might be expecting a particular cloning behaviour.


    I'll let you know when we have a final solution.

    I'm not sure if you noticed in my first posting, but there are a few more interesting things to note that might help you zero in on the problem. For instance, the error doesn't seem to occur if an exception is not thrown from within the catch.  Also, it doesn't fail 100% from the beginning.  It starts out working correctly, then changes to failing consistently.

    Quote from my first post:

    I've done some testing with variations and have narrowed the conditions to:

    • A multi-catch block is used to catch more than one exception.
    • An exception is thrown from the try block that matches anything but the first exception in the multi-catch.
    • An exception is thrown from the catch block.  It can be a new exception, or the caught exception.

    Some observations:

    • It appears to work for a while, then will fail consistently.
    • When the first exception in the multi-catch list is thrown, the finally block is always executed.
  • KLangman
    KLangman
    7 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2014-07-22T18:20:04Z  
    • FSuchy
    • ‏2014-07-22T17:38:33Z

    I'm not sure if you noticed in my first posting, but there are a few more interesting things to note that might help you zero in on the problem. For instance, the error doesn't seem to occur if an exception is not thrown from within the catch.  Also, it doesn't fail 100% from the beginning.  It starts out working correctly, then changes to failing consistently.

    Quote from my first post:

    I've done some testing with variations and have narrowed the conditions to:

    • A multi-catch block is used to catch more than one exception.
    • An exception is thrown from the try block that matches anything but the first exception in the multi-catch.
    • An exception is thrown from the catch block.  It can be a new exception, or the caught exception.

    Some observations:

    • It appears to work for a while, then will fail consistently.
    • When the first exception in the multi-catch list is thrown, the finally block is always executed.

    Yes, all of your observations are consistant with what we know about the JIT issue that cased it. The working cases early in the run are due to the fact the methods are executed by the interpreter for some time before the JIT gets around to compiling the method. Once compiled the problem happens consistantly. If you use the "-Xjit:count=0" option you'll notice that it failes 100% of the time.

  • KLangman
    KLangman
    7 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2014-09-10T20:33:43Z  
    • FSuchy
    • ‏2014-07-22T17:38:33Z

    I'm not sure if you noticed in my first posting, but there are a few more interesting things to note that might help you zero in on the problem. For instance, the error doesn't seem to occur if an exception is not thrown from within the catch.  Also, it doesn't fail 100% from the beginning.  It starts out working correctly, then changes to failing consistently.

    Quote from my first post:

    I've done some testing with variations and have narrowed the conditions to:

    • A multi-catch block is used to catch more than one exception.
    • An exception is thrown from the try block that matches anything but the first exception in the multi-catch.
    • An exception is thrown from the catch block.  It can be a new exception, or the caught exception.

    Some observations:

    • It appears to work for a while, then will fail consistently.
    • When the first exception in the multi-catch list is thrown, the finally block is always executed.

    OK, we have settled on a perminate fix and are making the changes in the next day or so. We will open an APAR to define this problem and describe where the fix can be found in Java 7 and Java 7.1.

    I'll update this thread again when I have the APAR number to share and we have a better idea of what service refresh levels will contain the fix.

  • KLangman
    KLangman
    7 Posts

    Re: Multi-catch can cause finally block to be skipped.

    ‏2015-02-04T00:36:57Z  
    • FSuchy
    • ‏2014-07-22T17:38:33Z

    I'm not sure if you noticed in my first posting, but there are a few more interesting things to note that might help you zero in on the problem. For instance, the error doesn't seem to occur if an exception is not thrown from within the catch.  Also, it doesn't fail 100% from the beginning.  It starts out working correctly, then changes to failing consistently.

    Quote from my first post:

    I've done some testing with variations and have narrowed the conditions to:

    • A multi-catch block is used to catch more than one exception.
    • An exception is thrown from the try block that matches anything but the first exception in the multi-catch.
    • An exception is thrown from the catch block.  It can be a new exception, or the caught exception.

    Some observations:

    • It appears to work for a while, then will fail consistently.
    • When the first exception in the multi-catch list is thrown, the finally block is always executed.

    This problem was fixed with APAR IV68110

    http://www-01.ibm.com/support/docview.wss?uid=swg1IV68110

    The fix will be available in:

    Java 7.0 SR9

    Java 7.1 SR2 FP11

     

    Please let me know if you have any further questions.