Topic
9 replies Latest Post - ‏2013-01-18T14:47:36Z by SystemAdmin
SystemAdmin
SystemAdmin
47293 Posts
ACCEPTED ANSWER

Pinned topic One user stops ClearCase for ALL users by selecting View -> Refresh view

‏2013-01-14T08:44:29Z |
Hi

This is a problem which has nagged me for months. We even hired a ClearCase consultant to fix it, but no luck. Hopefully someone in this forum can help me.

When a Windows users open ClearCase Explorer and select "View -> Refresh view shortcuts" the program will just hang. But the real problem is that ClearCase also stop working for ALL other users (both Window and Linux users). Then we can wait some minutes and ClearCase starts to work again. I'm not sure how long ClearCase is down but I estimate it to 3-5 minutes.

I suspect that it is the albd process on our vob server which stops responding. I have attached some logs at the end of this mail.

The strange thing is that this does not happen for all windows users, some may run "View -> Refresh view shortcuts" successfully. I have not been able to find out why.

We are running the view and vob server on an Solaris 10 machine (SunOS 5.10). The Windows client are running Windows 7 64-bit OS. And ClearCase version is 8.0.0.4 both on the client and server. And some of the VOBs are multisite.

Does anyone have a similar problem?
All suggestion to what may be wrong are greatly appreciated.

/Atle

Here are some log info from the vobserver, during a hang situation (ip addresses are replaced by x.x.x.x):
>cleartool getlog -around now 10 albd db vobrpc ccfs
cleartool: Error: Unable to contact albd_server on host 'x.x.x.x'
cleartool: Error: Cannot contact license server host "x.x.x.x" defined in file /var/adm/rational/clearcase/config/license_host.
cleartool: Error: You do not have a license to run ClearCase.

>cleartool getlog -around now 10 albd db vobrpc ccfs
=============================================================================
Log Name: albd Hostname: smncc Date: 2012-11-05T11:29:13+01:00
Selection: Lines between 2012-11-05T11:19:13+01:00 and 2012-11-05T11:39:13+01:00 displayed

2012-11-05T11:20:00+01:00 albd_server(629): Job 14 "Daily MultiSite Receive" (7663) Started.
=============================================================================
Log Name: db Hostname: smncc Date: 2012-11-05T11:29:13+01:00
Selection: Lines between 2012-11-05T11:19:13+01:00 and 2012-11-05T11:39:13+01:00 displayed

=============================================================================
Log Name: vobrpc Hostname: smncc Date: 2012-11-05T11:29:13+01:00
Selection: Lines between 2012-11-05T11:19:13+01:00 and 2012-11-05T11:39:13+01:00 displayed

2012-11-05T11:27:44+01:00 vobrpc_server(8191): Error: albd_sched_info call failed: RPC: Timed out
2012-11-05T11:22:58+01:00 vobrpc_server(8191): Error: albd_sched_info call failed: RPC: Timed out
2012-11-05T11:22:03+01:00 vobrpc_server(8191): Error: albd_sched_info call failed: RPC: Timed out
=============================================================================
Log Name: ccfs Hostname: smncc Date: 2012-11-05T11:29:13+01:00
Selection: Lines between 2012-11-05T11:19:13+01:00 and 2012-11-05T11:39:13+01:00 displayed

=============================================================================
>
Updated on 2013-01-18T14:47:36Z at 2013-01-18T14:47:36Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    47293 Posts
    ACCEPTED ANSWER

    Re: One user stops ClearCase for ALL users by selecting View -> Refresh view

    ‏2013-01-14T10:32:22Z  in response to SystemAdmin
    Sorry, this question was double posted.
    /Atle
  • SystemAdmin
    SystemAdmin
    47293 Posts
    ACCEPTED ANSWER

    Re: One user stops ClearCase for ALL users by selecting View -> Refresh view

    ‏2013-01-15T13:02:33Z  in response to SystemAdmin
    It seems it is the albd_server of your license server that has crashed.
    Is your vob server also your license server ?

    What happens if you execute from such a client the commands:
    cleartool lsview
    clearlicense

    Marc
  • brcowan
    brcowan
    673 Posts
    ACCEPTED ANSWER

    Re: One user stops ClearCase for ALL users by selecting View -> Refresh view

    ‏2013-01-15T16:48:33Z  in response to SystemAdmin
    View/Refresh view shortcuts will pull the entire view registry and attempt to get the view owner information for every view. If your registry server is also a view server, this could have "unfortunate" effects on the environment. it would help to know just how many views are in your environment, and on the server in question...

    =================================================================
    Brian Cowan
    Advisory Software Engineer
    ClearCase Software Advisory Team (SWAT)
    Rational Software
    IBM Software Group
    550 King St
    Littleton, MA 01460

    Phone: 1.978.899.5436
    Web: http://www.ibm.com/software/rational/support/
    • SystemAdmin
      SystemAdmin
      47293 Posts
      ACCEPTED ANSWER

      Re: One user stops ClearCase for ALL users by selecting View -> Refresh view

      ‏2013-01-18T11:46:48Z  in response to brcowan
      Hi

      Thanks for the comments.

      We have the same server running as: vob server, view server, registry server and license server. The reason that we run all the servers on one machine is that it is less then 10 developers using ClearCase at the same time, and it should not be necessaries to have a big distributed ClearCase installation for such a small group. It might affect ClearCase performance but it should not cause hang situations(?).

      Here are some machine info about our ClearCase server:
      From top:
      load averages: 0.02, 0.03, 0.03
      83 processes: 82 sleeping, 1 on cpu
      CPU states: 99.4% idle, 0.1% user, 0.5% kernel, 0.0% iowait, 0.0% swap
      Memory: 8192M real, 6076M free, 291M swap in use, 15G swap free

      >uname -a
      SunOS smncc 5.10 Generic_120011-14 sun4u sparc SUNW,Sun-Fire-V215

      Running a ClearCase commando on an client when we have a hang situation generates outputs similar to this:
      > ct ls
      cleartool: Error: Unable to contact albd_server on host 'smncc'
      cleartool: Error: Cannot contact license server host "smncc" defined in file /var/adm/rational/clearcase/config/license_host.
      cleartool: Error: You do not have a license to run ClearCase.

      We are running with ClearCase (Atria) licensing, not FlexLM.

      We have been using ClearCase for a while now so its starting to accumulate some views. I guess I'm the worst case with 10 views. More info:

      >ct lsview -reg unix |wc -l
      88
      >ct lsview -reg windows | wc -l
      106

      Unix views are also visible in the windows region, in addition there are some snapshot views in the windows region.

      On the view server:
      smncc>ps -ef | grep view|wc -l
      19

      Brian: what do you mean by: "If your registry server is also a view server, this could have "unfortunate" effects on the environment."

      Hope this helps, I really want to get rid of this error. The users are quite annoyed with the ClearCase behaviour for the time being. Today we have had multiple hangs, even though no one have been using "View -> Refresh view shortcuts". I suspect one Windows machine caused it, since the albd_service died on that machine. But I have no idea why.

      /Atle
      • SystemAdmin
        SystemAdmin
        47293 Posts
        ACCEPTED ANSWER

        Re: One user stops ClearCase for ALL users by selecting View -> Refresh view

        ‏2013-01-18T12:30:12Z  in response to SystemAdmin
        Hi,

        Is the albd_service still running on your server when such locks happen ? And can you execute at that moment the command "cleartool ls" on the server itself ?

        We have more than 500 views registered on a server which acts as vob/registry/license server and it runs fine in our case (ClearCase 8.0.0.3). We are also using Atria Licensing, which works quite well for us.

        Maybe it has something to do with your interop solution ? If it's samba, could you share (part of) your smb.conf ?

        Marc
        • SystemAdmin
          SystemAdmin
          47293 Posts
          ACCEPTED ANSWER

          Re: One user stops ClearCase for ALL users by selecting View -> Refresh view

          ‏2013-01-18T13:26:45Z  in response to SystemAdmin
          > Is the albd_service still running on your server when such locks happen ?

          Yes, it is.

          > And can you execute at that moment the command "cleartool ls" on the server itself ?

          No, I can not. Not 100% certain, but as far as I remember I have tested this and I could not run any ClearCase commands on the server. I could login and run normal unix commands, but not ClearCase commands.

          > Maybe it has something to do with your interop solution ? If it's samba, could you share (part of) your smb.conf ?

          We do not use Samba. I don't have details about the interop solution right now, but I can find out if it's relevant and interesting.

          Nice to hear that you also run vob/view/registry/license-server on one server, and it works quite well for you. Even though you have more views than us, and probably more users also.

          /Atle
          • SystemAdmin
            SystemAdmin
            47293 Posts
            ACCEPTED ANSWER

            Re: One user stops ClearCase for ALL users by selecting View -> Refresh view

            ‏2013-01-18T14:38:48Z  in response to SystemAdmin
            > Nice to hear that you also run vob/view/registry/license-server on one server, and it works quite well for you. Even though
            > you have more views than us, and probably more users also.

            There is one difference however: most of our views are hosted on the clients themselves and not on the vobserver. So our workstations serve as viewserver.

            Marc
            • SystemAdmin
              SystemAdmin
              47293 Posts
              ACCEPTED ANSWER

              Re: One user stops ClearCase for ALL users by selecting View -> Refresh view

              ‏2013-01-18T14:47:36Z  in response to SystemAdmin
              Ok, then we are back to Brian's comment about view server and registry server. Maybe it will help to separate them on different machines?

              Lets see if Brian can elaborate a bit about what he means with: "If your registry server is also a view server, this could have "unfortunate" effects on the environment".

              /Atle
  • SystemAdmin
    SystemAdmin
    47293 Posts
    ACCEPTED ANSWER

    Re: One user stops ClearCase for ALL users by selecting View -> Refresh view

    ‏2013-01-18T11:57:45Z  in response to SystemAdmin
    Changed question back to unanswered. It was set to answered since it was a double post. But people started to comment on this post instead of the original post.