IC5Notice: We have upgraded developerWorks Community to the latest version of IBM Connections. For more information, read our upgrade FAQ.
Topic
  • 19 replies
  • Latest Post - ‏2012-12-05T09:01:48Z by SystemAdmin
SystemAdmin
SystemAdmin
2736 Posts

Pinned topic Immature finalization issue - an active object is being finalized

‏2012-11-29T12:13:13Z |
It seems that IBM JDK 6/7 can kick up the finalization of an object even if there is a reference to it on the stack.

In the reproducer of this bug, see [1], there is a simple test method which looks as follows:

@Test 

public 

void testLookup() 

throws Exception 
{ localContext.bind(
"test", 
"TestValue"); assertEquals(
"TestValue", getRemoteContext().lookup(
"test")); localContext.unbind(
"test"); 
}


The issue is that a RemoteContext object created by getRemoteContext() method is finalized before the lookup method finishes. The issue can be clearly seen with more logging added to the RemoteContext constructor and its lookup method, see [2]. The logging includes this.toString() output for tracking particular object instance.

The logs reveal that only one RemoteContext object is created during the test. Then lookup() method is invoked on that RemoteContext instance. After that 'J9VMInternals.runFinalize' invokes finalize() method on the RemoteContext object and the lookup() method subsequently fails as the finalize method of RemoteContext triggers cleaning of some other objects being used within the lookup(). So the RemoteContext object is finalized several milliseconds before the failure, but the stack trace of the failure still shows the RemoteContext object on the stack. See attached full logs.

I've tried it with different GC policies, i.e. gencon, optthruput, optavgpause, with the same result.

More info can be found at
https://issues.jboss.org/browse/AS7-5967
The issue affects both IBM JDK6 SR12 and IBM JDK7 SR3:
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64 jvmxa6460sr12-20121024_126067 (JIT enabled, AOT enabled)
IBM J9 VM (build 2.6, JRE 1.7.0 Linux amd64-64 20121024_126071 (JIT enabled, AOT enabled)

Steps to reproduce:
1. git clone git://github.com/istudens/jboss-remote-naming.git
2. cd jboss-remote-naming && git checkout AS7-5967
3. mvn clean test -Dtest=FinalizerTest -Dtest.level=TRACE
4. check logs in target/surefire-reports/TEST-org.jboss.naming.remote.FinalizerTest.xml and target/test.log

[1] https://github.com/istudens/jboss-remote-naming

[2] https://github.com/istudens/jboss-remote-naming/compare/master...AS7-5967
Updated on 2012-12-05T09:01:48Z at 2012-12-05T09:01:48Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T12:13:39Z  
    Attaching another log with -verbose:gc.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T12:30:11Z  
    Could anybody take a look at it what is happening there?
    Thank you.
  • Ian_Partridge
    Ian_Partridge
    23 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T12:54:45Z  
    I haven't examined your testcase in detail, but it's quite likely that this is the issue discussed at http://thevirtualmachinist.blogspot.com/2011/07/subtle-issue-of-reachability.html Have a read.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T13:48:27Z  
    I haven't examined your testcase in detail, but it's quite likely that this is the issue discussed at http://thevirtualmachinist.blogspot.com/2011/07/subtle-issue-of-reachability.html Have a read.
    Mmm, that's inspiring. Thanks for pointing that out.

    If I understand it correctly a method inlined by JIT can cause such a failure. The issue, however, occurs even with JIT disabled. I will re-check that again.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T14:07:23Z  
    I haven't examined your testcase in detail, but it's quite likely that this is the issue discussed at http://thevirtualmachinist.blogspot.com/2011/07/subtle-issue-of-reachability.html Have a read.
    It does the same even with -Xint.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T14:40:31Z  
    It does the same even with -Xint.
    Yes, it is in fact the issue that Peter details on his blog. Peter was the original developer of IBM's GC, so he knows what he's talking about. The issue is a consequence of the way the GC is designed. It's best to take Peter's advice and avoid finalization whenever possible.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T14:49:07Z  
    Yes, it is in fact the issue that Peter details on his blog. Peter was the original developer of IBM's GC, so he knows what he's talking about. The issue is a consequence of the way the GC is designed. It's best to take Peter's advice and avoid finalization whenever possible.
    Is this issue related to the JIT compiler then? Should it disappear when JIT is disabled?
    I thought only the JIT compiler can decide to inline a method, not the javac compiler.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T15:32:13Z  
    Is this issue related to the JIT compiler then? Should it disappear when JIT is disabled?
    I thought only the JIT compiler can decide to inline a method, not the javac compiler.
    Peter uses in-lining in his code example only as a tool to show explicitly how/when the problem can occur. In-lining is not necessary for this problem to occur.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-30T09:23:00Z  
    Peter uses in-lining in his code example only as a tool to show explicitly how/when the problem can occur. In-lining is not necessary for this problem to occur.
    I still tend to consider this being a bug as I have not experienced such problems with OracleJDK or OpenJDK. Are you sure IBM JDK does its part right at this case?
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-30T11:21:23Z  
    Peter uses in-lining in his code example only as a tool to show explicitly how/when the problem can occur. In-lining is not necessary for this problem to occur.
    In the example of the blog:
    
    Foo f = 
    
    new Foo(); 
    
    byte[] array = f.getData(); System.out.println(
    "array={" + array[0] + 
    ", " + array[1] + 
    ", " + array[2] + 
    "}");
    


    Where does the caller lose the reference to f during the method call?
  • Ian_Partridge
    Ian_Partridge
    23 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-30T12:58:07Z  
    In the example of the blog:
    <pre class="jive-pre"> Foo f = new Foo(); byte[] array = f.getData(); System.out.println( "array={" + array[0] + ", " + array[1] + ", " + array[2] + "}"); </pre>

    Where does the caller lose the reference to f during the method call?
    It depends on what the bytecodes are. But if "this" is no longer used inside a method, the GC is allowed to infer that the object is no longer referenced.

    Don't use finalizers. They are broken by design. Also, since they are processed serially on a separate thread you cannot predict when they'll run, or if they'll run at all.

    https://www.securecoding.cert.org/confluence/display/java/MET12-J.+Do+not+use+finalizers
  • Ian_Partridge
    Ian_Partridge
    23 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-30T13:09:07Z  
    It depends on what the bytecodes are. But if "this" is no longer used inside a method, the GC is allowed to infer that the object is no longer referenced.

    Don't use finalizers. They are broken by design. Also, since they are processed serially on a separate thread you cannot predict when they'll run, or if they'll run at all.

    https://www.securecoding.cert.org/confluence/display/java/MET12-J.+Do+not+use+finalizers
    Another blog post discussing similar issues:

    http://jeremymanson.blogspot.com/2010/01/garbage-collection-references.html
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-30T13:31:58Z  
    It depends on what the bytecodes are. But if "this" is no longer used inside a method, the GC is allowed to infer that the object is no longer referenced.

    Don't use finalizers. They are broken by design. Also, since they are processed serially on a separate thread you cannot predict when they'll run, or if they'll run at all.

    https://www.securecoding.cert.org/confluence/display/java/MET12-J.+Do+not+use+finalizers
    I now understand it is worth avoiding the use of finalizers. However, I still cannot see any clue, even in the posted links, how a JDK could think of an object A as unreachable while still being inside of invocation of one of its method, e.g. inside of A.lookup().

    I might accept it can happen sometimes if the in-lining is in the game, but without that the JDK still has an active reference to the object and cannot safely decide on its finalization.
  • Ian_Partridge
    Ian_Partridge
    23 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-30T13:41:17Z  
    I now understand it is worth avoiding the use of finalizers. However, I still cannot see any clue, even in the posted links, how a JDK could think of an object A as unreachable while still being inside of invocation of one of its method, e.g. inside of A.lookup().

    I might accept it can happen sometimes if the in-lining is in the game, but without that the JDK still has an active reference to the object and cannot safely decide on its finalization.
    The key part of Peter's post is this:

    "In the VM, the receiver of a function (i.e. this) isn't very special at all. It's just the first argument of a virtual function."

    Once a function is executing, if the VM notices that "this" is no longer used then the GC can collect the object.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-30T17:59:12Z  
    The key part of Peter's post is this:

    "In the VM, the receiver of a function (i.e. this) isn't very special at all. It's just the first argument of a virtual function."

    Once a function is executing, if the VM notices that "this" is no longer used then the GC can collect the object.
    The Java^tm^ Language Specification 7, 12.6.1:
    A reachable object is any object that can be accessed in any potential continuing computation from any live thread.

    The fact that I am calling a method implies that I have a live thread, ergo the object is reachable. A naive JIT implementation might assume that I would not need the object anymore and just finalize it. Thus doing the wrong thing.

    While https://www.securecoding.cert.org/confluence/display/java/MET12-J.+Do+not+use+finalizers sports valid points which should be guarded against when using finalizers. http://jeremymanson.blogspot.nl/2010/01/garbage-collection-references.html is bogus on the part of "1 0 0". Because the usage of j in the finalize method, it can never be expunged by a JIT.
    Given the description in JLS it is valid to GC FinalizableObject after setJ, but never during setJ.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-12-01T01:46:15Z  
    The Java^tm^ Language Specification 7, 12.6.1:
    A reachable object is any object that can be accessed in any potential continuing computation from any live thread.

    The fact that I am calling a method implies that I have a live thread, ergo the object is reachable. A naive JIT implementation might assume that I would not need the object anymore and just finalize it. Thus doing the wrong thing.

    While https://www.securecoding.cert.org/confluence/display/java/MET12-J.+Do+not+use+finalizers sports valid points which should be guarded against when using finalizers. http://jeremymanson.blogspot.nl/2010/01/garbage-collection-references.html is bogus on the part of "1 0 0". Because the usage of j in the finalize method, it can never be expunged by a JIT.
    Given the description in JLS it is valid to GC FinalizableObject after setJ, but never during setJ.
    "A reachable object is any object that can be accessed in any potential continuing computation from any live thread."

    The object is not reachable because there is no possible path from the current point which will use the object. The last use of the object was to look up the method you're invoking. Once that look up completed the object is never used again and is eligible for garbage collection.

    The Java specification isn't referring to any conceivable program (you could write a program where the object is used again), but to the actual program it's executing.
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-12-03T13:25:53Z  
    "A reachable object is any object that can be accessed in any potential continuing computation from any live thread."

    The object is not reachable because there is no possible path from the current point which will use the object. The last use of the object was to look up the method you're invoking. Once that look up completed the object is never used again and is eligible for garbage collection.

    The Java specification isn't referring to any conceivable program (you could write a program where the object is used again), but to the actual program it's executing.
    While I applaud your efforts to counter what is commonly called 'invisible' references, it basically breaks down because the current execution is not taken properly into account.

    
    
    
    import java.util.HashSet; 
    
    import java.util.Set;   
    
    public 
    
    class FinalizeBug 
    { 
    
    private Set<String> set = 
    
    new HashSet<String>();   
    
    protected 
    
    void finalize() 
    { System.out.println(
    "finalize"); set.clear(); 
    }   
    
    public 
    
    static 
    
    void main(String[] args) 
    { FinalizeBug bug = 
    
    new FinalizeBug(); bug.work(); 
    // allowed for GC from this point on 
    }   
    
    public 
    
    void work() 
    { 
    
    final Set<String> set = this.set; set.add(
    "Hello world"); System.gc(); System.runFinalization(); System.out.println(set); 
    
    if (set.isEmpty()) 
    
    throw 
    
    new Error(
    "bug!"); 
    // as a work-around: 
    // assert this != null; 
    } 
    }
    


    That I have to forcibly hold a reference to this in my hands during a computation running on FinalizeBug is just silly.
    1. The current frame holds this in variable #0. While technically it is possible to set local variable #0 to null, I find that an extremely dirty action. (Which the compiler also hints at.)
    2. The previous frame is also holding a reference to FinalizeBug (or at least should be ;-) ) in its stack.
    Once the method returns it is however elligible for GC as per JLS spec and thus the 'invisible' reference problem can be countered.

    So at the end of the day it really boils down to the question: does a method invocation on an object imply a strong reference to said object?

    Since neither JLS 7 or JVMS 7 give a clear resolution I think we should forward it onto the EG.
  • Ian_Partridge
    Ian_Partridge
    23 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-12-03T14:21:23Z  
    While I applaud your efforts to counter what is commonly called 'invisible' references, it basically breaks down because the current execution is not taken properly into account.

    <pre class="jive-pre"> import java.util.HashSet; import java.util.Set; public class FinalizeBug { private Set<String> set = new HashSet<String>(); protected void finalize() { System.out.println( "finalize"); set.clear(); } public static void main(String[] args) { FinalizeBug bug = new FinalizeBug(); bug.work(); // allowed for GC from this point on } public void work() { final Set<String> set = this.set; set.add( "Hello world"); System.gc(); System.runFinalization(); System.out.println(set); if (set.isEmpty()) throw new Error( "bug!"); // as a work-around: // assert this != null; } } </pre>

    That I have to forcibly hold a reference to this in my hands during a computation running on FinalizeBug is just silly.
    1. The current frame holds this in variable #0. While technically it is possible to set local variable #0 to null, I find that an extremely dirty action. (Which the compiler also hints at.)
    2. The previous frame is also holding a reference to FinalizeBug (or at least should be ;-) ) in its stack.
    Once the method returns it is however elligible for GC as per JLS spec and thus the 'invisible' reference problem can be countered.

    So at the end of the day it really boils down to the question: does a method invocation on an object imply a strong reference to said object?

    Since neither JLS 7 or JVMS 7 give a clear resolution I think we should forward it onto the EG.
    One person's "dirty action" is another person's elegant optimisation :-)

    Note that the spec. says:

    "Optimizing transformations of a program can be designed that reduce the number of objects that are reachable to be less than those which would naively be considered reachable. For example, a Java compiler or code generator may choose to set a variable or parameter that will no longer be used to null to cause the storage for such an object to be potentially reclaimable sooner."
  • SystemAdmin
    SystemAdmin
    2736 Posts

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-12-05T09:01:48Z  
    One person's "dirty action" is another person's elegant optimisation :-)

    Note that the spec. says:

    "Optimizing transformations of a program can be designed that reduce the number of objects that are reachable to be less than those which would naively be considered reachable. For example, a Java compiler or code generator may choose to set a variable or parameter that will no longer be used to null to cause the storage for such an object to be potentially reclaimable sooner."
    I've forward the question to the EG. See http://mail.openjdk.java.net/pipermail/java-se-8-spec-comments/2012-December/000001.html