Topic
19 replies Latest Post - ‏2012-12-05T09:01:48Z by SystemAdmin
SystemAdmin
SystemAdmin
2736 Posts
ACCEPTED ANSWER

Pinned topic Immature finalization issue - an active object is being finalized

‏2012-11-29T12:13:13Z |
It seems that IBM JDK 6/7 can kick up the finalization of an object even if there is a reference to it on the stack.

In the reproducer of this bug, see [1], there is a simple test method which looks as follows:

@Test 

public 

void testLookup() 

throws Exception 
{ localContext.bind(
"test", 
"TestValue"); assertEquals(
"TestValue", getRemoteContext().lookup(
"test")); localContext.unbind(
"test"); 
}


The issue is that a RemoteContext object created by getRemoteContext() method is finalized before the lookup method finishes. The issue can be clearly seen with more logging added to the RemoteContext constructor and its lookup method, see [2]. The logging includes this.toString() output for tracking particular object instance.

The logs reveal that only one RemoteContext object is created during the test. Then lookup() method is invoked on that RemoteContext instance. After that 'J9VMInternals.runFinalize' invokes finalize() method on the RemoteContext object and the lookup() method subsequently fails as the finalize method of RemoteContext triggers cleaning of some other objects being used within the lookup(). So the RemoteContext object is finalized several milliseconds before the failure, but the stack trace of the failure still shows the RemoteContext object on the stack. See attached full logs.

I've tried it with different GC policies, i.e. gencon, optthruput, optavgpause, with the same result.

More info can be found at
https://issues.jboss.org/browse/AS7-5967
The issue affects both IBM JDK6 SR12 and IBM JDK7 SR3:
IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux amd64-64 jvmxa6460sr12-20121024_126067 (JIT enabled, AOT enabled)
IBM J9 VM (build 2.6, JRE 1.7.0 Linux amd64-64 20121024_126071 (JIT enabled, AOT enabled)

Steps to reproduce:
1. git clone git://github.com/istudens/jboss-remote-naming.git
2. cd jboss-remote-naming && git checkout AS7-5967
3. mvn clean test -Dtest=FinalizerTest -Dtest.level=TRACE
4. check logs in target/surefire-reports/TEST-org.jboss.naming.remote.FinalizerTest.xml and target/test.log

[1] https://github.com/istudens/jboss-remote-naming

[2] https://github.com/istudens/jboss-remote-naming/compare/master...AS7-5967
Updated on 2012-12-05T09:01:48Z at 2012-12-05T09:01:48Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    2736 Posts
    ACCEPTED ANSWER

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T12:13:39Z  in response to SystemAdmin
    Attaching another log with -verbose:gc.
  • SystemAdmin
    SystemAdmin
    2736 Posts
    ACCEPTED ANSWER

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T12:30:11Z  in response to SystemAdmin
    Could anybody take a look at it what is happening there?
    Thank you.
  • Ian_Partridge
    Ian_Partridge
    23 Posts
    ACCEPTED ANSWER

    Re: Immature finalization issue - an active object is being finalized

    ‏2012-11-29T12:54:45Z  in response to SystemAdmin
    I haven't examined your testcase in detail, but it's quite likely that this is the issue discussed at http://thevirtualmachinist.blogspot.com/2011/07/subtle-issue-of-reachability.html Have a read.
    • SystemAdmin
      SystemAdmin
      2736 Posts
      ACCEPTED ANSWER

      Re: Immature finalization issue - an active object is being finalized

      ‏2012-11-29T13:48:27Z  in response to Ian_Partridge
      Mmm, that's inspiring. Thanks for pointing that out.

      If I understand it correctly a method inlined by JIT can cause such a failure. The issue, however, occurs even with JIT disabled. I will re-check that again.
    • SystemAdmin
      SystemAdmin
      2736 Posts
      ACCEPTED ANSWER

      Re: Immature finalization issue - an active object is being finalized

      ‏2012-11-29T14:07:23Z  in response to Ian_Partridge
      It does the same even with -Xint.
      • SystemAdmin
        SystemAdmin
        2736 Posts
        ACCEPTED ANSWER

        Re: Immature finalization issue - an active object is being finalized

        ‏2012-11-29T14:40:31Z  in response to SystemAdmin
        Yes, it is in fact the issue that Peter details on his blog. Peter was the original developer of IBM's GC, so he knows what he's talking about. The issue is a consequence of the way the GC is designed. It's best to take Peter's advice and avoid finalization whenever possible.
        • SystemAdmin
          SystemAdmin
          2736 Posts
          ACCEPTED ANSWER

          Re: Immature finalization issue - an active object is being finalized

          ‏2012-11-29T14:49:07Z  in response to SystemAdmin
          Is this issue related to the JIT compiler then? Should it disappear when JIT is disabled?
          I thought only the JIT compiler can decide to inline a method, not the javac compiler.
          • SystemAdmin
            SystemAdmin
            2736 Posts
            ACCEPTED ANSWER

            Re: Immature finalization issue - an active object is being finalized

            ‏2012-11-29T15:32:13Z  in response to SystemAdmin
            Peter uses in-lining in his code example only as a tool to show explicitly how/when the problem can occur. In-lining is not necessary for this problem to occur.
            • SystemAdmin
              SystemAdmin
              2736 Posts
              ACCEPTED ANSWER

              Re: Immature finalization issue - an active object is being finalized

              ‏2012-11-30T09:23:00Z  in response to SystemAdmin
              I still tend to consider this being a bug as I have not experienced such problems with OracleJDK or OpenJDK. Are you sure IBM JDK does its part right at this case?
            • SystemAdmin
              SystemAdmin
              2736 Posts
              ACCEPTED ANSWER

              Re: Immature finalization issue - an active object is being finalized

              ‏2012-11-30T11:21:23Z  in response to SystemAdmin
              In the example of the blog:
              
              Foo f = 
              
              new Foo(); 
              
              byte[] array = f.getData(); System.out.println(
              "array={" + array[0] + 
              ", " + array[1] + 
              ", " + array[2] + 
              "}");
              


              Where does the caller lose the reference to f during the method call?
              • Ian_Partridge
                Ian_Partridge
                23 Posts
                ACCEPTED ANSWER

                Re: Immature finalization issue - an active object is being finalized

                ‏2012-11-30T12:58:07Z  in response to SystemAdmin
                It depends on what the bytecodes are. But if "this" is no longer used inside a method, the GC is allowed to infer that the object is no longer referenced.

                Don't use finalizers. They are broken by design. Also, since they are processed serially on a separate thread you cannot predict when they'll run, or if they'll run at all.

                https://www.securecoding.cert.org/confluence/display/java/MET12-J.+Do+not+use+finalizers
                • Ian_Partridge
                  Ian_Partridge
                  23 Posts
                  ACCEPTED ANSWER

                  Re: Immature finalization issue - an active object is being finalized

                  ‏2012-11-30T13:09:07Z  in response to Ian_Partridge
                  Another blog post discussing similar issues:

                  http://jeremymanson.blogspot.com/2010/01/garbage-collection-references.html
                • SystemAdmin
                  SystemAdmin
                  2736 Posts
                  ACCEPTED ANSWER

                  Re: Immature finalization issue - an active object is being finalized

                  ‏2012-11-30T13:31:58Z  in response to Ian_Partridge
                  I now understand it is worth avoiding the use of finalizers. However, I still cannot see any clue, even in the posted links, how a JDK could think of an object A as unreachable while still being inside of invocation of one of its method, e.g. inside of A.lookup().

                  I might accept it can happen sometimes if the in-lining is in the game, but without that the JDK still has an active reference to the object and cannot safely decide on its finalization.
                  • Ian_Partridge
                    Ian_Partridge
                    23 Posts
                    ACCEPTED ANSWER

                    Re: Immature finalization issue - an active object is being finalized

                    ‏2012-11-30T13:41:17Z  in response to SystemAdmin
                    The key part of Peter's post is this:

                    "In the VM, the receiver of a function (i.e. this) isn't very special at all. It's just the first argument of a virtual function."

                    Once a function is executing, if the VM notices that "this" is no longer used then the GC can collect the object.
                    • SystemAdmin
                      SystemAdmin
                      2736 Posts
                      ACCEPTED ANSWER

                      Re: Immature finalization issue - an active object is being finalized

                      ‏2012-11-30T17:59:12Z  in response to Ian_Partridge
                      The Java^tm^ Language Specification 7, 12.6.1:
                      A reachable object is any object that can be accessed in any potential continuing computation from any live thread.

                      The fact that I am calling a method implies that I have a live thread, ergo the object is reachable. A naive JIT implementation might assume that I would not need the object anymore and just finalize it. Thus doing the wrong thing.

                      While https://www.securecoding.cert.org/confluence/display/java/MET12-J.+Do+not+use+finalizers sports valid points which should be guarded against when using finalizers. http://jeremymanson.blogspot.nl/2010/01/garbage-collection-references.html is bogus on the part of "1 0 0". Because the usage of j in the finalize method, it can never be expunged by a JIT.
                      Given the description in JLS it is valid to GC FinalizableObject after setJ, but never during setJ.
                      • SystemAdmin
                        SystemAdmin
                        2736 Posts
                        ACCEPTED ANSWER

                        Re: Immature finalization issue - an active object is being finalized

                        ‏2012-12-01T01:46:15Z  in response to SystemAdmin
                        "A reachable object is any object that can be accessed in any potential continuing computation from any live thread."

                        The object is not reachable because there is no possible path from the current point which will use the object. The last use of the object was to look up the method you're invoking. Once that look up completed the object is never used again and is eligible for garbage collection.

                        The Java specification isn't referring to any conceivable program (you could write a program where the object is used again), but to the actual program it's executing.
                        • SystemAdmin
                          SystemAdmin
                          2736 Posts
                          ACCEPTED ANSWER

                          Re: Immature finalization issue - an active object is being finalized

                          ‏2012-12-03T13:25:53Z  in response to SystemAdmin
                          While I applaud your efforts to counter what is commonly called 'invisible' references, it basically breaks down because the current execution is not taken properly into account.

                          
                          
                          
                          import java.util.HashSet; 
                          
                          import java.util.Set;   
                          
                          public 
                          
                          class FinalizeBug 
                          { 
                          
                          private Set<String> set = 
                          
                          new HashSet<String>();   
                          
                          protected 
                          
                          void finalize() 
                          { System.out.println(
                          "finalize"); set.clear(); 
                          }   
                          
                          public 
                          
                          static 
                          
                          void main(String[] args) 
                          { FinalizeBug bug = 
                          
                          new FinalizeBug(); bug.work(); 
                          // allowed for GC from this point on 
                          }   
                          
                          public 
                          
                          void work() 
                          { 
                          
                          final Set<String> set = this.set; set.add(
                          "Hello world"); System.gc(); System.runFinalization(); System.out.println(set); 
                          
                          if (set.isEmpty()) 
                          
                          throw 
                          
                          new Error(
                          "bug!"); 
                          // as a work-around: 
                          // assert this != null; 
                          } 
                          }
                          


                          That I have to forcibly hold a reference to this in my hands during a computation running on FinalizeBug is just silly.
                          1. The current frame holds this in variable #0. While technically it is possible to set local variable #0 to null, I find that an extremely dirty action. (Which the compiler also hints at.)
                          2. The previous frame is also holding a reference to FinalizeBug (or at least should be ;-) ) in its stack.
                          Once the method returns it is however elligible for GC as per JLS spec and thus the 'invisible' reference problem can be countered.

                          So at the end of the day it really boils down to the question: does a method invocation on an object imply a strong reference to said object?

                          Since neither JLS 7 or JVMS 7 give a clear resolution I think we should forward it onto the EG.
                          • Ian_Partridge
                            Ian_Partridge
                            23 Posts
                            ACCEPTED ANSWER

                            Re: Immature finalization issue - an active object is being finalized

                            ‏2012-12-03T14:21:23Z  in response to SystemAdmin
                            One person's "dirty action" is another person's elegant optimisation :-)

                            Note that the spec. says:

                            "Optimizing transformations of a program can be designed that reduce the number of objects that are reachable to be less than those which would naively be considered reachable. For example, a Java compiler or code generator may choose to set a variable or parameter that will no longer be used to null to cause the storage for such an object to be potentially reclaimable sooner."