Topic
  • 11 replies
  • Latest Post - ‏2013-04-22T18:41:52Z by benray
SystemAdmin
SystemAdmin
47290 Posts

Pinned topic ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

‏2013-03-15T21:58:23Z |
Hello. When we upgrade our ClearCase Clients from 7.0.1.7 to 7.1.2.4 we have seen the average time to check out/check in files increase 7x (3.5 seconds versus 24.5 seconds per file). The client OS is Windows XP. The 7.1.2.4 client version also averages about 24.5 seconds per file when running on Windows 7. The server version is 7.1.2.3. Has anyone else seen this issue. This is my first post to this forum. Please let me know if this isn't the right place for this type of question. Thank you for your help.
  • brcowan
    brcowan
    763 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-01T20:18:42Z  
    I'm afraid that there are far too many questions to ask about your environment than can be done in a forum thread.

    My initial suspicion is that more than just the client version changed. Did the server version change? Did the server change? Were any Windows domain changes done at about the same time?

    I would recommend opening a PMR for more detailed analysis.

    =================================================================
    Brian Cowan
    Advisory Software Engineer
    ClearCase Software Advisory Team (SWAT)
    Rational Software
    IBM Software Group
    550 King St
    Littleton, MA 01460

    Phone: 1.978.899.5436
    Web: http://www.ibm.com/software/rational/support/
  • SystemAdmin
    SystemAdmin
    47290 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-02T23:34:35Z  
    • brcowan
    • ‏2013-04-01T20:18:42Z
    I'm afraid that there are far too many questions to ask about your environment than can be done in a forum thread.

    My initial suspicion is that more than just the client version changed. Did the server version change? Did the server change? Were any Windows domain changes done at about the same time?

    I would recommend opening a PMR for more detailed analysis.

    =================================================================
    Brian Cowan
    Advisory Software Engineer
    ClearCase Software Advisory Team (SWAT)
    Rational Software
    IBM Software Group
    550 King St
    Littleton, MA 01460

    Phone: 1.978.899.5436
    Web: http://www.ibm.com/software/rational/support/
    The server, server version and windows domain did not change. The server is running version 7.1.2.3.

    We have done a few more tests since my initial post. We did tests with 32-bit XP, 32-bit Win7 and 64-bit Win7 running version 7.1.2.4. With a normal user account checking out 5 files took around 64 seconds while using a super user account checking out the same 5 files on the same dynamic view took 3 seconds. When the same XP computer was downgraded to 7.0.1.7, a normal user account took about 13 seconds to check out the same 5 files while using a super user account still took around 3 seconds. The difference between the normal user account and the super user account is the super user account in on an exclusion list for cleartriggers.

    In summary, without using cleartriggers it takes about 3 seconds to check out 5 files but with cleartriggers it takes about 13 seconds on 7.0.1.7 and 64 seconds on 7.1.2.4.

    We tried commenting out everything in the clearbits file but we didn't see a performance improvement. We also tried turning off the anti-virus, but we didn't see a performance improvement.

    I'll post an update if we get it figured out. Let me know if there is something else we should look at.
  • Tgefen
    Tgefen
    727 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-03T12:03:26Z  
    The server, server version and windows domain did not change. The server is running version 7.1.2.3.

    We have done a few more tests since my initial post. We did tests with 32-bit XP, 32-bit Win7 and 64-bit Win7 running version 7.1.2.4. With a normal user account checking out 5 files took around 64 seconds while using a super user account checking out the same 5 files on the same dynamic view took 3 seconds. When the same XP computer was downgraded to 7.0.1.7, a normal user account took about 13 seconds to check out the same 5 files while using a super user account still took around 3 seconds. The difference between the normal user account and the super user account is the super user account in on an exclusion list for cleartriggers.

    In summary, without using cleartriggers it takes about 3 seconds to check out 5 files but with cleartriggers it takes about 13 seconds on 7.0.1.7 and 64 seconds on 7.1.2.4.

    We tried commenting out everything in the clearbits file but we didn't see a performance improvement. We also tried turning off the anti-virus, but we didn't see a performance improvement.

    I'll post an update if we get it figured out. Let me know if there is something else we should look at.
    sab193, have you checked if it occurs on certain VOBs only?

    Regards,
    Tamir Gefen, GoMidjets
  • brcowan
    brcowan
    763 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-03T14:34:10Z  
    The server, server version and windows domain did not change. The server is running version 7.1.2.3.

    We have done a few more tests since my initial post. We did tests with 32-bit XP, 32-bit Win7 and 64-bit Win7 running version 7.1.2.4. With a normal user account checking out 5 files took around 64 seconds while using a super user account checking out the same 5 files on the same dynamic view took 3 seconds. When the same XP computer was downgraded to 7.0.1.7, a normal user account took about 13 seconds to check out the same 5 files while using a super user account still took around 3 seconds. The difference between the normal user account and the super user account is the super user account in on an exclusion list for cleartriggers.

    In summary, without using cleartriggers it takes about 3 seconds to check out 5 files but with cleartriggers it takes about 13 seconds on 7.0.1.7 and 64 seconds on 7.1.2.4.

    We tried commenting out everything in the clearbits file but we didn't see a performance improvement. We also tried turning off the anti-virus, but we didn't see a performance improvement.

    I'll post an update if we get it figured out. Let me know if there is something else we should look at.
    Now we're getting somewhere...

    Regarding the triggers:
    1) What operations are they performing? Are certain operations within the triggers slower than others?
    2) how large is the -nusers list?
    3) Are the -nusers lists domain-qualified?
    4) Are ONLY checkouts (or other operations with triggers applied) impacted? Or are other operations impacted?

    =================================================================
    Brian Cowan
    Advisory Software Engineer
    ClearCase Software Advisory Team (SWAT)
    Rational Software
    IBM Software Group
    550 King St
    Littleton, MA 01460

    Phone: 1.978.899.5436
    Web: http://www.ibm.com/software/rational/support/
  • SystemAdmin
    SystemAdmin
    47290 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-03T14:35:50Z  
    The server, server version and windows domain did not change. The server is running version 7.1.2.3.

    We have done a few more tests since my initial post. We did tests with 32-bit XP, 32-bit Win7 and 64-bit Win7 running version 7.1.2.4. With a normal user account checking out 5 files took around 64 seconds while using a super user account checking out the same 5 files on the same dynamic view took 3 seconds. When the same XP computer was downgraded to 7.0.1.7, a normal user account took about 13 seconds to check out the same 5 files while using a super user account still took around 3 seconds. The difference between the normal user account and the super user account is the super user account in on an exclusion list for cleartriggers.

    In summary, without using cleartriggers it takes about 3 seconds to check out 5 files but with cleartriggers it takes about 13 seconds on 7.0.1.7 and 64 seconds on 7.1.2.4.

    We tried commenting out everything in the clearbits file but we didn't see a performance improvement. We also tried turning off the anti-virus, but we didn't see a performance improvement.

    I'll post an update if we get it figured out. Let me know if there is something else we should look at.
    Hi, ClearTrigger on the largest busines VOB does not place that kind of burden on checkin/out, but perhaps you have placed the depot or the binaries at a location on a slow part of the network. Is ClearTrigger loggin on? If some user are on the special list and still significanly outperform other that would point to other issues.. Are you using cleartrigger <groups>?

    Feel free to contact ABS Support or me directly for any ClearTrigger issues.

    Hope this helps...

    -Charles

    Try ClearTrigger or ClearReplica at http://www.abs-consulting.com

    A Better Solution, Inc.

    Charles Clarke III
    ClearCase Consultant
    A Better Solution, Inc.
    4303 Cedar Lake Cove
    Conley, Ga. 30288

    Email: charles@abs-consulting.com
    http://www.abs-consulting.com
  • SystemAdmin
    SystemAdmin
    47290 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-03T19:31:22Z  
    • Tgefen
    • ‏2013-04-03T12:03:26Z
    sab193, have you checked if it occurs on certain VOBs only?

    Regards,
    Tamir Gefen, GoMidjets
    Thank you for your post. We checked a couple other VOBs and saw about the same performance as our initial test VOB. The issue doesn't appear to be related to certain VOBs only.
  • SystemAdmin
    SystemAdmin
    47290 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-03T19:34:43Z  
    • brcowan
    • ‏2013-04-03T14:34:10Z
    Now we're getting somewhere...

    Regarding the triggers:
    1) What operations are they performing? Are certain operations within the triggers slower than others?
    2) how large is the -nusers list?
    3) Are the -nusers lists domain-qualified?
    4) Are ONLY checkouts (or other operations with triggers applied) impacted? Or are other operations impacted?

    =================================================================
    Brian Cowan
    Advisory Software Engineer
    ClearCase Software Advisory Team (SWAT)
    Rational Software
    IBM Software Group
    550 King St
    Littleton, MA 01460

    Phone: 1.978.899.5436
    Web: http://www.ibm.com/software/rational/support/
    Thank you for your post. Here are the answers to the questions posed:

    1&4) We experience the slowness on check-outs, check-ins, creating elements and creating symlinks.
    2) The -nusers list only has 2 users.
    3) The -nuser list names are not domain qualified. This is a dual Windows/Unix peer configuration, so there are two domains involved.
  • SystemAdmin
    SystemAdmin
    47290 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-03T19:41:02Z  
    Hi, ClearTrigger on the largest busines VOB does not place that kind of burden on checkin/out, but perhaps you have placed the depot or the binaries at a location on a slow part of the network. Is ClearTrigger loggin on? If some user are on the special list and still significanly outperform other that would point to other issues.. Are you using cleartrigger <groups>?

    Feel free to contact ABS Support or me directly for any ClearTrigger issues.

    Hope this helps...

    -Charles

    Try ClearTrigger or ClearReplica at http://www.abs-consulting.com

    A Better Solution, Inc.

    Charles Clarke III
    ClearCase Consultant
    A Better Solution, Inc.
    4303 Cedar Lake Cove
    Conley, Ga. 30288

    Email: charles@abs-consulting.com
    http://www.abs-consulting.com
    Thank you for your post. Both the ClearTrigger executables and the depots are located on a NAS filer. We may do a test in the future where we copy them to the ClearCase server. We are not currently using CT groups.

    Can you describe how I can check to see if ClearTrigger logging is on? I don't know if ClearTrigger logging is on.
  • SystemAdmin
    SystemAdmin
    47290 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-04T14:46:08Z  
    Thank you for your post. Both the ClearTrigger executables and the depots are located on a NAS filer. We may do a test in the future where we copy them to the ClearCase server. We are not currently using CT groups.

    Can you describe how I can check to see if ClearTrigger logging is on? I don't know if ClearTrigger logging is on.
    It should be noted (I'm working with sab193 on the testing of different scenarios regarding this problem) that the environment is UNCHANGED.

    In a nutshell, we are at 7.0.1.7, and users (with Cleartriggers enabled) takes on average about 13 seconds to check out 5 files (and a little longer to undo those checkouts). When you upgrade to version 7.1.2.4 on the SAME machine, it then takes 64 seconds to check out those SAME 5 files.

    It's very confusing, because alot of people want to point at environment variables, but in this case, the environment, aside from the client version upgrade, is unchanged. The ONLY way to get anything resembling decent performance is to disable cleartriggers, however that is not to say that cleartriggers is the issue...Although it does stand to reason that perhaps there is some issue with Cleartrigger and this particular version of Clearcase.

    For what its worth, we are experiencing very similar slowness with ClearQuest (as a test, I did a simple modify of 5 ClearQuest records...62 seconds on 7.1.2.04, and 16 seconds on 7.0.1.7).

    We only upgraded to 7.1.2.4 because we moved to Windows 7, and 7.1.x is the only version that supports that OS.

    If anyone has any ideas to try, we are all ears!
  • sab193
    sab193
    1 Post

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-22T18:23:23Z  
    The server, server version and windows domain did not change. The server is running version 7.1.2.3.

    We have done a few more tests since my initial post. We did tests with 32-bit XP, 32-bit Win7 and 64-bit Win7 running version 7.1.2.4. With a normal user account checking out 5 files took around 64 seconds while using a super user account checking out the same 5 files on the same dynamic view took 3 seconds. When the same XP computer was downgraded to 7.0.1.7, a normal user account took about 13 seconds to check out the same 5 files while using a super user account still took around 3 seconds. The difference between the normal user account and the super user account is the super user account in on an exclusion list for cleartriggers.

    In summary, without using cleartriggers it takes about 3 seconds to check out 5 files but with cleartriggers it takes about 13 seconds on 7.0.1.7 and 64 seconds on 7.1.2.4.

    We tried commenting out everything in the clearbits file but we didn't see a performance improvement. We also tried turning off the anti-virus, but we didn't see a performance improvement.

    I'll post an update if we get it figured out. Let me know if there is something else we should look at.

    We were running ClearTrigger 12.2. When we upgraded to 12.7 the check out time of those same 5 files went from ~64 seconds down to ~11 seconds. Upgrading to ClearTrigger 12.7 seems to have fixed our performance issue with ClearCase 7.1.2.4. The upgrade also sped up the check outs of those same 5 files using ClearCase 7.0.1.7 from 13 seconds down to 6 seconds.

    BTW.. I wrote the original post for this thread. When I look at the thread now it says "SystemAdmin" posted the original message.

  • benray
    benray
    74 Posts

    Re: ClearCase Client 7.1.2.4 7x Slower than 7.0.1.7

    ‏2013-04-22T18:41:52Z  
    • sab193
    • ‏2013-04-22T18:23:23Z

    We were running ClearTrigger 12.2. When we upgraded to 12.7 the check out time of those same 5 files went from ~64 seconds down to ~11 seconds. Upgrading to ClearTrigger 12.7 seems to have fixed our performance issue with ClearCase 7.1.2.4. The upgrade also sped up the check outs of those same 5 files using ClearCase 7.0.1.7 from 13 seconds down to 6 seconds.

    BTW.. I wrote the original post for this thread. When I look at the thread now it says "SystemAdmin" posted the original message.

    Thanks for the update on this. If you need to get more stats on time spent in trigger logic this will also help:

    http://www-01.ibm.com/support/docview.wss?uid=swg21126919