Topic
5 replies Latest Post - ‏2014-03-21T20:52:34Z by Mathias Mamsch
matt_hall
matt_hall
3 Posts
ACCEPTED ANSWER

Pinned topic Is there a memory leak with object(int, module)?

‏2014-03-18T22:55:08Z |

I'm experiencing a memory leak when I use this function:

Object object(int, [module])

I use it within a loop and the module input is the baseline of the current working copy.  The intent is to do a buffer comparison between the working copy and the baseline copy to flag a difference.

I have this call embedded in a function that I call within the loop.  When I put a return statement prior to this call, everything works very fast.  When I put a return statement directly after this call (to rule out the possibility of poor garbage collection on the Buffers), we noticed a significant performance degradation over the life of the loop.

Has anyone seen this?  Does anyone know why this happens?  I was not expecting a leak on this function.

  • Mathias Mamsch
    Mathias Mamsch
    1937 Posts
    ACCEPTED ANSWER

    Re: Is there a memory leak with object(int, module)?

    ‏2014-03-20T15:27:42Z  in response to matt_hall

    I can't imagine your code well at the moment. I can only say, that I used this function in the past for scripts that processed over a million objects and I have never experienced any problems with it. There is the possibility that there might be a problem for a very new DOORS client version, but this is very unlikely.

    Can you please post some example code?

    Memory Leaks and Performance in DXL is a very tricky subject and there are lots of mistakes that one can make, especially when it comes to processing large buffers (e.g. rtf read directly from an attribute). A very common mistake is for example leaking buffers through the substring operation on a buffer, e.g.:

    Buffer buf2 = "{" buf1[0:pos] "}"
    

    It is not obvious, but in the above line the substring operation creates a temporary buffer, that is leaked after the statement instead of creating a temporary string. A line like this will make your performance drastically decrease. And there are a lot of those pitfalls.

    And you talking of garbage collection (DOORS has no garbage collection!) it seems there might be another non-obvious problem with your code.

    Regards, Mathias

    • matt_hall
      matt_hall
      3 Posts
      ACCEPTED ANSWER

      Re: Is there a memory leak with object(int, module)?

      ‏2014-03-20T15:53:45Z  in response to Mathias Mamsch

      Thanks for the reply.  My code runs a loop on the objects within a module.  Within this loop, I call this function to determine if there is a difference between the module I'm executing the DXL from v. a baseline defined by mBaseline for a given attributeName.  I define global buffers mpTempBuffer1 and mpTempBuffer2.  See code below:

      bool changeExists_(Object o, Module mBaseline, string attributeName) {
          AttrDef ad  = null
       
       bool isChange = false
       
       int absNum   = 0
      
       Object bO  = null
      
       // If null Object, error.
       if (null o) {
        print "Error:  Null Object passed into changeExists_ function."
        return false
       }
       
       // If null, this means the user did not set a value from the GUI.
       if (null mBaseline) {
        print "WARNING:  A baseline was not selected (or stored within the settings file) to generate change bars.\n" //-
           "Please select a baseline and rerun the tool."
        return false
       }
       
       // Get absolute number ID of Object o.
       absNum = o."Absolute Number"
       
       // Search baseline for baselined version of Object o.
       bO = object(absNum, mBaseline)
       
       // If Object o doesn't exist in the baseline, this indicates Object o is new and is a valid change.
       if (null bO) return true
       
       // Reset buffers.
       setempty(mpTempBuffer1)
       setempty(mpTempBuffer2)
       
       // Determine if attributeName is a valid attribute, not a DOORS defined value (i.e., "Object Level").
       if (isObjectAttribute_(o, attributeName)) {
        // Get Object o attribute value.
        mpTempBuffer1 = (o.attributeName)
        
        // Determine if Module mBaseline has attributeName defined.
        ad = find(mBaseline, attributeName)
        
        // Get baselined Object o value.  Set to blank if attributeName isn't defined within Module mBaseline.
        if (null ad) { 
         mpTempBuffer2 = tempStringOf("")
        } else {
         mpTempBuffer2 = (bO.attributeName)
        }
       }
       
       // If current object and baseline object are different, return there is a valid change.
       if (mpTempBuffer1 != mpTempBuffer2) {
        isChange = true
       }
       
       // Reset buffers.
       setempty(mpTempBuffer1)
       setempty(mpTempBuffer2)
       
       // Set default.
       return isChange
      }
      

      When I call this function from my loop, I see it degrade.  I've never seen this before.  When I put a return statement prior to the object(int, module), performance is as expected.  Right after it, I see the same level of degradation as if I let the function run.  So, it's right there.

      My colleague and I have scrubbed the code for leaking buffers and other known issues.  We've narrowed it down to here - thinking we did something with the buffers in this function.  But, like I said, the degradation occurs at the object function, which doesn't make sense.

      As for my comment on garbage collection, my apologies if I confused you.  I was merely saying that I'm clearing and/or deleting the structures I can to free up resources (i.e., delete(buffer), delete(skip), etc.) when it's appropriate to do so.

      Last data point - I've used a version of this function (without the object call) for other things like string manipulation.  I have no issues with performance or any signs of leakage.

      • Mathias Mamsch
        Mathias Mamsch
        1937 Posts
        ACCEPTED ANSWER

        Re: Is there a memory leak with object(int, module)?

        ‏2014-03-21T20:52:34Z  in response to matt_hall

        Your code seems fine. I am still positive, that there is no leak/performance issue with the object function ... Can you please measure the timing of your function with the following code snippet? Taking a variant of your code I get no performance degradation and of course no allocated object leaks inside the code, however I do not have the isObjectAttribute_ function (your own implementation?).

        Please check if you can reproduce the problem with your own code, but the below profiling code.

        pragma runLim, 0
        int *::+(int *ptr1, int ofs) { int *ptr2 = ptr1; ptr2+=ofs; return ptr2 }
        int *::@(int *ptr, int ofs) { int ad = *(ptr + ofs); int *ptr2 = addr_ ad; return ptr2 }
         
        int *getCurrentDXLContextPtr () {
            DB x = create ""
                int *ptr = addr_ x
                int *result = ptr @ 48
                destroy x
                return result
        }
         
        int *getMemoryBlockNodes (int *cc) { return cc @ 0x74 }
        int *nextNode      (int *memNode) { return memNode @ 8 }
         
        int countAllocatedObjects() {
                int *memBlocks = getMemoryBlockNodes getCurrentDXLContextPtr() 
                int count = 0
                while (!null memBlocks) {
         
                        memBlocks = nextNode memBlocks
                        count++
                }
                return count
        }
        
        Buffer mpTempBuffer1 = create(); 
        Buffer mpTempBuffer2 = create(); 
        
        bool changeExists_(Object o, Module mBaseline, string attributeName) {
            AttrDef ad  = null
            bool isChange = false
            int absNum   = 0
        
            Object bO  = null
        
            absNum = o."Absolute Number"
            bO = object(absNum, mBaseline)
         
            if (null bO) return true
         
            setempty(mpTempBuffer1)
            setempty(mpTempBuffer2)
        
            AttrDef adO = find(module o, attributeName); 
         
            if (!null adO && adO .object) {
               mpTempBuffer1 = (o.attributeName)
               ad = find(mBaseline, attributeName)
          
               if (null ad || !ad.object) { 
                  mpTempBuffer2 = tempStringOf("")
               } else {
                  mpTempBuffer2 = (bO.attributeName)
               }
            }
         
            if (mpTempBuffer1 != mpTempBuffer2) isChange = true
         
            setempty(mpTempBuffer1)
            setempty(mpTempBuffer2)
         
            // Set default.
            return isChange
        }
        
        Module mod = current;
        Module modBase = current // shouldn't matter for a leak?
        Object o = current;      // shouldn't matter for a leak?
        
        for (i = 0; i < 100; i++) {
            int iStart = getTickCount_ (), iEnd = 0, count = 0;
            while (true) {
                changeExists_(o, modBase, "Object Text")
                iEnd = getTickCount_ (); 
                count++;
                if (iEnd - iStart >= 1000) break; 
            }
            print count "/" ((iEnd - iStart)) " Repetitions/ms. Allocated Objects: " countAllocatedObjects() "\n";
        }
        

        Regards, Mathias

  • This reply was deleted by Mathias Mamsch 2014-03-21T15:48:59Z.
  • This reply was deleted by Mathias Mamsch 2014-03-21T15:49:03Z.
  • This reply was deleted by Mathias Mamsch 2014-03-21T15:49:07Z.
  • This reply was deleted by Mathias Mamsch 2014-03-21T15:49:12Z.
  • llandale
    llandale
    2943 Posts
    ACCEPTED ANSWER

    Re: Is there a memory leak with object(int, module)?

    ‏2014-03-20T15:53:17Z  in response to matt_hall

    Mathias is right, post the code.

    I would imagine some kind of performance difference between this function actually returing the basline Object found and not returning it.  Presumably, if it doesn't return a found object then you subsequently bypass your 'diff" function, since you have nothing to compare; and if so then you probably have an ineffieciency somewhere in your diff processing.

    In any event, I never use this "object(int)" function, since it only finds objects that are currently "displayed" (even if invisible) which puts you at the mercy of the current view filtering et tal settings.  I routinely put the objects into a Skip, and then search the skip:

    • Skip skpBase = create() // KEY 'int' AbsNo; DATA 'Object' handle
    • for oBase in mBase do
    • {  iAbsNoBase = oBase."Absolute Number"
    •    put(skpBase, iAbsNoBase, oBase)
    • }
    • for oCurr in mCurr do
    • {  iAbsNoCurr = oCurr. "Absolute Number"
    •    if (find(skpBase, iAbsNoCurr, oBase))
    •    then compare
    •    else there is no corresponding Baseline object
    • }
    • delete(skpBase)

    -Louie

  • llandale
    llandale
    2943 Posts
    ACCEPTED ANSWER

    Re: Is there a memory leak with object(int, module)?

    ‏2014-03-20T15:53:52Z  in response to matt_hall

    Mathias is right, post the code.

    I would imagine some kind of performance difference between this function actually returing the basline Object found and not returning it.  Presumably, if it doesn't return a found object then you subsequently bypass your 'diff" function, since you have nothing to compare; and if so then you probably have an ineffieciency somewhere in your diff processing.

    In any event, I never use this "object(int)" function, since it only finds objects that are currently "displayed" (even if invisible) which puts you at the mercy of the current view filtering et tal settings.  I routinely put the objects into a Skip, and then search the skip:

    • Skip skpBase = create() // KEY 'int' AbsNo; DATA 'Object' handle
    • for oBase in mBase do
    • {  iAbsNoBase = oBase."Absolute Number"
    •    put(skpBase, iAbsNoBase, oBase)
    • }
    • for oCurr in mCurr do
    • {  iAbsNoCurr = oCurr. "Absolute Number"
    •    if (find(skpBase, iAbsNoCurr, oBase))
    •    then compare
    •    else there is no corresponding Baseline object
    • }
    • delete(skpBase)

    -Louie