Topic
  • 14 replies
  • Latest Post - ‏2012-11-16T11:02:56Z by SystemAdmin
SystemAdmin
SystemAdmin
47293 Posts

Pinned topic Newbie ClearCase Questions

‏2012-11-10T00:49:49Z |
Warning new to ClearCase but I have RTM. I was given the task of getting ClearCase up and running for a group of 20 programmers managing 10 projects that are in maintenance mode. I am hoping someone would be kind enough, and that has some real world experience with ClearCase, to give me their opinions.

1. I chose component VOB’s over Project VOB's and decided against using admin VOB’s. I am all ears on this one, this for me is the hardest decision to make, VOB design. I would like to keep it fairly simple though having an admin VOB (to allow global metadata) might be better I am not sure. If someone could offer me their opinion I sure would appreciate it.

2. I am stuck with trying to figure out how to handle chain of command promotion within ClearCase. Meaning a low level programmer can check out a copy of the main branch but cannot merge their changes back into the main branch, this would be their supervisor’s responsibility. Basically my idea so far is this; I am planning on using "user branching" (each user has their own branch to do their check out, check in and changes), I would lock the main branch except for supervisory staff exclusions. Once the user checks in their branch their supervisor would do the merging of the user’s changes into the main branch? Is this the approach to take with a chain of command type of scenario or is there a better way to do things?

Thanks for any help.
Updated on 2012-11-16T11:02:56Z at 2012-11-16T11:02:56Z by SystemAdmin
  • Tgefen
    Tgefen
    724 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-10T19:20:23Z  
    Number two makes sense. I'd suggest you to consider UCM approach as well.

    Regards,
    Tamir Gefen, GoMidjets
    Have you tried Visual Annotate 2.0 for ClearCase?
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-11T21:00:49Z  
    foobarbaz wrote:
    > 1. I chose component VOB’s over Project VOB's and decided against
    > using admin VOB’s.

    This vocabulary already smells UCM... although Tamir didn't notice, as it seems...?
    Admin vobs are a bad idea because they freeze early design decisions. The result is artificial inflexibility in multisite environments.

    UCM is bad in general. The technical reason is that it builds on a radical design error concerning branching: that one should deliver by merging back to some kind of integration branches.
    This defeats the advantages which were sought by branching off in the first place (in short: not depending on the accidental state of the system)
    It also creates artificial duplication of data, artificial mastership issues, irreversibility, and delays. It jeopardizes configuration records produced before delivery.

    > 2. I am stuck with trying to figure out how to handle chain of command
    > promotion within ClearCase.

    Try to decouple data changes from state changes: promotion should take place without data changes, in-place. I.e. it should not use merging.
    This is easy to implement using labels, and locking the types.

    I have been developing and maintaining over the past 5 years a free cleartool wrapper in perl, available through CPAN, and which supports the design principles mentioned here above:
    ClearCase::Wrapper::MGi
    The package builds upon a few other CPAN packages, a full platform developed by David Boyce.
    Note that I also offered all my ideas to IBM as an RFE.
    Marc
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-12T21:49:10Z  
    When you say 10 projects, I suppose you mean projects as in "Java Projects", and not as in parallel development streams
    for different releases (l'll call the latter "release"-projects) ?
    Depending on the size of these projects, you can put all the code in a single VOB, or you could create one VOB per project.
    You also have to decide what historical data you want to import from your current SCM-system into ClearCase.

    Then you should choose between UCM and Base ClearCase. If you choose for UCM, you should have one UCM project VOB, and one
    or more UCM component VOBs. The latter is were your source code goes. If you choose Base ClearCase, you need only one type of VOBs:
    source code VOBs, which are the equivalent of the component VOBs in UCM.

    In our environment we are using UCM now for about 4 years. It helps us a lot to manage our parallel releases and it provides
    out-of-the-box support for private development branches.
    In your post, you propose that only a supervisor should merge changes into the main branch (in UCM, it's called the Integration branch).
    In a typical UCM-process, this is however the responsibility of the developer.
    If you want to use Base ClearCase with private branches, then you have to teach the developers to setup a configspec file, or
    you should provide templates for configspec files for them.

    You should also think about your branching (Base ClearCase) or Stream (UCM) architecture.
    Standard practice in both Base and UCM, is that you define a main branch (or stream) that you only use for putting the stable
    versions from your actual "release"-projects. You do this when the work for that release is mostly finished, and you want to
    start a new release.

    Note also that the term "checkout" in ClearCase has a different meaning than in tools like SVN etc.
    Finally, I would advice to read the first chapters of Kevin Lee's book
    "ClearCase, Ant, and CruiseControl: The Java Developer's Guide to Accelerating and Automating the Build Process".
    It provides a step-by-step approach for setting up your first ClearCase projects. The book covers both Base and UCM.

    HTH,
    Marc
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-13T17:42:38Z  
    Sorry for such late responses, the forum was down for maintenance this weekend when I checked on my question. I responded to everyone below. One thing I would like to say is I have tried twice to attend the ClearCase boot camp class but IBM keeps canceling, not enough attendance. Also some clarification when I said "managing 10 projects" what I meant was "10 different environments", example; financial environment, customer support environment, document management environment, etc., that are all completely separate from each other (not counting SOA). Sorry for the confusion. Also worth mentioning is we are a mix of many different languages; plsql, "Oracle Forms and reports" java, python, shell, php, JSP, etc.

    @Tgefen
    Thanks for the confirmation on my second question.

    @314
    I will take a second look at UCM per your suggestion, if I could just understand UCM I would be able to come to a conclusion. When you say "Try to decouple data changes from state changes" I am left confused, I will try and figure this one out with more thought. Thanks for the heads up on the wrapper script I will take a look. Thanks.

    @marcdb
    >When you say 10 projects, I suppose you mean projects as in "Java Projects", and not as in parallel development streams
    >for different releases (l'll call the latter "release"-projects) ?
    Sorry for the confusion, what I meant was 10 different environments, these are custom built in house systems that have been developed over many years and are completely separate from each other. Your response helped me understand some additional things about ClearCase, thanks.
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-13T17:52:25Z  
    Actually what I should have said was "managing 10 different systems", using the word environment is also misleading now that I think about it.
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-15T08:30:40Z  
    Sorry for such late responses, the forum was down for maintenance this weekend when I checked on my question. I responded to everyone below. One thing I would like to say is I have tried twice to attend the ClearCase boot camp class but IBM keeps canceling, not enough attendance. Also some clarification when I said "managing 10 projects" what I meant was "10 different environments", example; financial environment, customer support environment, document management environment, etc., that are all completely separate from each other (not counting SOA). Sorry for the confusion. Also worth mentioning is we are a mix of many different languages; plsql, "Oracle Forms and reports" java, python, shell, php, JSP, etc.

    @Tgefen
    Thanks for the confirmation on my second question.

    @314
    I will take a second look at UCM per your suggestion, if I could just understand UCM I would be able to come to a conclusion. When you say "Try to decouple data changes from state changes" I am left confused, I will try and figure this one out with more thought. Thanks for the heads up on the wrapper script I will take a look. Thanks.

    @marcdb
    >When you say 10 projects, I suppose you mean projects as in "Java Projects", and not as in parallel development streams
    >for different releases (l'll call the latter "release"-projects) ?
    Sorry for the confusion, what I meant was 10 different environments, these are custom built in house systems that have been developed over many years and are completely separate from each other. Your response helped me understand some additional things about ClearCase, thanks.
    foobarbaz wrote:
    > @314
    > When you say "Try to decouple data changes from state changes" I am left confused

    I meant that delivery should be a state change, a promotion.
    You want to deliver something as it is, without modifying it.
    You want to deliver what you tested.
    Ideally (in ClearCase) you'd want deliver not to invalidate your test, and its configuration record.

    UCM, and the adverse branching strategy this one implements, forces you to create a duplicate of the original version, to make a data change.
    This forces a rebuild of anything that referenced the version prior to delivery, including any test.
    The operation is of course also much heavier, and its success depends on multiple conditions, first of all that the latest version on the delivery branch is not reserved by anybody else, but also that the branch is mastered locally.
    All this prevents the delivery from being atomic, other than by using a locking strategy which impacts other developers even more.

    Even UCM would benefit from the changes I propose (by reimplementing streams).
    The real benefit is however to reestablish the value of clearmake, especially in the java building case, with .JAVAC

    Marc
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-15T17:55:26Z  
    @314 - Thanks for your reply. I think I understand what you are telling me now. Basically the idea of "chain of command" wont work with how Clearcase is designed and that I am stuck with an open system where anyone at any level of their job title can check out, check in, delete, move or rename an element. My idea was a programmer would check an element out into their own branch and when their work was completed a head programmer would merge their work back into the main branch. I am just having a hard time understanding why in theory this is a bad idea and why Clearcase cant handle this.
  • jeff98air
    jeff98air
    817 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-15T19:39:38Z  
    @314 - Thanks for your reply. I think I understand what you are telling me now. Basically the idea of "chain of command" wont work with how Clearcase is designed and that I am stuck with an open system where anyone at any level of their job title can check out, check in, delete, move or rename an element. My idea was a programmer would check an element out into their own branch and when their work was completed a head programmer would merge their work back into the main branch. I am just having a hard time understanding why in theory this is a bad idea and why Clearcase cant handle this.
    foobarbaz (OP), since you are new to ClearCase and also to this forum, I feel the need to make you aware that 314 (Marc) is somewhat of our resident anti-UCM extremist IMHO. Marc has been a valuable contributor to this forum for many years and I do respect his knowledge, but his anti-UCM view does not reflect the whole of this users group. Many members of this forum have vast experience in SCM/Release Engineering/ClearCase Administration and have been using ClearCase UCM successfully for many, many years (20 yrs SCM, 14 yrs BaseCC, 12 yrs UCM, for me personally, but many have even more).

    Like many other "options" that exist in our industry (Java vs. C++, UNIX vs. MacOS vs. Windows, iPhone vs. Android, etc.), the correct answer is "...it depends...". Of course there are patterns, anti-patterns and best practices for doing just about anything, but you need to pick a solution that works well in YOUR environment. IMHO, based on your experience with ClearCase, UCM would be a worthwhile consideration.

    -Jeff Ng
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-15T19:57:50Z  
    @jeff98air - Thanks for you honest reply, much appreciated. It matters not to me weather or not I go with base or UCM I just need need it to get the job done simply and efficiently with as little amount of my intervention as an administrator as possible. I have already spend considerable time trying to learn base so that is why I have been hesitant. I will look at UCM per your suggestion but my issue still exists, no matter the solution I pick, I need to implement some sort of control where only the lead programmers can commit into main (marc said its called Integration branch in UCM) and it needs to work with CCRC because that is what the users will be using. Does this sound reasonable to you?
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-15T20:55:07Z  
    • jeff98air
    • ‏2012-11-15T19:39:38Z
    foobarbaz (OP), since you are new to ClearCase and also to this forum, I feel the need to make you aware that 314 (Marc) is somewhat of our resident anti-UCM extremist IMHO. Marc has been a valuable contributor to this forum for many years and I do respect his knowledge, but his anti-UCM view does not reflect the whole of this users group. Many members of this forum have vast experience in SCM/Release Engineering/ClearCase Administration and have been using ClearCase UCM successfully for many, many years (20 yrs SCM, 14 yrs BaseCC, 12 yrs UCM, for me personally, but many have even more).

    Like many other "options" that exist in our industry (Java vs. C++, UNIX vs. MacOS vs. Windows, iPhone vs. Android, etc.), the correct answer is "...it depends...". Of course there are patterns, anti-patterns and best practices for doing just about anything, but you need to pick a solution that works well in YOUR environment. IMHO, based on your experience with ClearCase, UCM would be a worthwhile consideration.

    -Jeff Ng
    Jeff Ng wrote:
    > 314 (Marc) is somewhat of our resident anti-UCM extremist IMHO.
    ...
    > his anti-UCM view does not reflect the whole of this users group.

    I never chose to be an extremist! I have a clear demonstration, and do not understand
    why the discussion is impossible.
    Now, the moment is poorly chosen: I'll be on vacation for a couple of weeks.

    Marc
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-15T21:04:37Z  
    @jeff98air - Thanks for you honest reply, much appreciated. It matters not to me weather or not I go with base or UCM I just need need it to get the job done simply and efficiently with as little amount of my intervention as an administrator as possible. I have already spend considerable time trying to learn base so that is why I have been hesitant. I will look at UCM per your suggestion but my issue still exists, no matter the solution I pick, I need to implement some sort of control where only the lead programmers can commit into main (marc said its called Integration branch in UCM) and it needs to work with CCRC because that is what the users will be using. Does this sound reasonable to you?
    foobarbaz wrote:
    > only the lead programmers can commit into main
    ...
    > it needs to work with CCRC

    These two requirements weaken the advantages of my proposals.

    To me, SCM is a tool to organize freedom, i.e. to allow to get rid of bottleneck roles, such as 'lead programmers'. The point is to make freedom possible while avoiding chaos.
    It is an issue of bottom-up communications.

    CCRC is extremely restricted. It doesn't support clearmake, hence it deepens the gap between source files and derived objects, when the graal of MVFS and dynamic views was to fill it, and ultimately to make of the derived object the first class citizen, the prototypical configuration item.

    Marc
  • jeff98air
    jeff98air
    817 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-15T21:33:33Z  
    @jeff98air - Thanks for you honest reply, much appreciated. It matters not to me weather or not I go with base or UCM I just need need it to get the job done simply and efficiently with as little amount of my intervention as an administrator as possible. I have already spend considerable time trying to learn base so that is why I have been hesitant. I will look at UCM per your suggestion but my issue still exists, no matter the solution I pick, I need to implement some sort of control where only the lead programmers can commit into main (marc said its called Integration branch in UCM) and it needs to work with CCRC because that is what the users will be using. Does this sound reasonable to you?
    Yes, UCM can easily support your usage model with minimal intervention from you as the ClearCase administrator after the initial creation of the PVOB, component VOBs, components and projects.

    You will follow a very standard usage of UCM with the exception that the project integration stream will remain locked to all except for certain users (your lead programmers). This will allow any user to create development (child) streams off of the project integration stream and checkout/checkin any element in any modifiable component you have configured for the project. All of those developer changes exist only on their development stream and are captured in a UCM Activity (aka. task). When work is ready to be delivered from a development stream to the integration stream, a "lead programmer" can deliver the desired UCM Activities from the dev stream to the integ stream. If desired, the "lead programmer" can also make a new baseline on the integ stream which other dev streams can rebase from to get the delivered changes into their dev stream (Note that only elements modified by a dev stream are potentially "merged". All other element versions just become "visible" by reference of the new baseline).

    The most important part of setting up a new development environment is properly defining your software components. A UCM component will match to some level of your software architecture, usually at the CSCI level. Here is a very simplified example:

    Product name: EasyBank
    Project (release) name: EasyBank-1.0
    PVOB: /vobs/easybank/pvob_easybank
    UCM Components:
    - easybank_CBL (rootless component for composite baseline)
    - easybank_common (common code ie. schemas, properties, build system)
    - easybank_database (database code ie. oracle, sybase, etc.)
    - easybank_ui (user interface code ie. java, flash, etc.)
    - easybank_server (app server code)
    - easybank_installer (installer code)

    Again, this is a simple example. Some components might be at a slightly lower level (e.g. db-oracle, db-sybase). Also, each component can either be at the VOB level (e.g. /vobs/easybank/database) or sub-VOB level (e.g. /vobs/easybank/database/oracle, /vobs/easybank/database/sybase). You may also wish to control some third-party software (3PP) as separate components in a separate PVOB (e.g. /vobs/3pp/pvob_3pp) that can be referenced "read-only" to other projects. Examples would be JDK, Ant, JUnit, ActiveMQ, Xerces, etc.

    It sounded like you have around 10 different "products" that you are controlling, although that wasn't very clear. In any case, be sure to take some time up front and define all of your components for each product. See if there are any that are common/shared across multiple products.

    -Jeff Ng
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-16T00:28:39Z  
    @314
    >To me, SCM is a tool to organize freedom, i.e. to allow to get rid of bottleneck roles, such as 'lead programmers'.
    >The point is to make freedom possible while avoiding chaos.
    >It is an issue of bottom-up communications.
    I am not saying I disagree completely, I am admittedly ignorant on the subject, but none of the large FOSS project that I know of allow just anyone to commit into "main" (clearcase terminology). There always seems to be some maintainer that is responsible for this. Though these project use different tools (GIT,SVN) the theory is the same IMO, there should be someone controlling what gets committed into "main".

    @jeff98air
    Awesome, thanks, just what I was looking for. I tried doing this with base clearcase but I couldn't get past the config rules. For example I locked the main branch and created a user branch. Then as a user I checked some elements out of the main branch into the user branch, made some changes and then checked in. I was lost after that because I couldn't figure out the rules that would allow the lead developer to see both the main branch and user branch to do the merging. Then later on I could no longer see the main branch from the users end, I could only see the user branch. If I could get past the rules it might just work.
  • SystemAdmin
    SystemAdmin
    47293 Posts

    Re: Newbie ClearCase Questions

    ‏2012-11-16T11:02:56Z  
    @314
    >To me, SCM is a tool to organize freedom, i.e. to allow to get rid of bottleneck roles, such as 'lead programmers'.
    >The point is to make freedom possible while avoiding chaos.
    >It is an issue of bottom-up communications.
    I am not saying I disagree completely, I am admittedly ignorant on the subject, but none of the large FOSS project that I know of allow just anyone to commit into "main" (clearcase terminology). There always seems to be some maintainer that is responsible for this. Though these project use different tools (GIT,SVN) the theory is the same IMO, there should be someone controlling what gets committed into "main".

    @jeff98air
    Awesome, thanks, just what I was looking for. I tried doing this with base clearcase but I couldn't get past the config rules. For example I locked the main branch and created a user branch. Then as a user I checked some elements out of the main branch into the user branch, made some changes and then checked in. I was lost after that because I couldn't figure out the rules that would allow the lead developer to see both the main branch and user branch to do the merging. Then later on I could no longer see the main branch from the users end, I could only see the user branch. If I could get past the rules it might just work.
    foobarbaz wrote:
    > none of the large FOSS project that I know of allow just anyone to commit into "main"
    > (clearcase terminology). There always seems to be some maintainer that is responsible for this.

    That's right, apart for 'main' being ClearCase terminology.
    Or rather, it is ClearCase terminology but for something completely unrelated, and low-level implementation.

    The 'FOSS' projects (they too have terminology problems... I liked 'free software', and dislike 'open source' and 'FOSS', but this would lead us astray--even if this is all related...) do not know/admit/understand that there could be an alternative to their 'control' model, which is surprisiingly 'closed' and traditional!

    There are many other ways to close software than keeping source proprietary (even if this is undeniably one way).

    This is ironic: clearmake-like tools could offer a much more open development model than most of what is mispresented as its competition, be it CVS, subversion, Git, maven, ant, Perforce or UCM!

    True, ClearCase always was a proprietary tool, and we all suffer from this. Not only because of the price of licenses. It is certainly not a free beer, but it goes very far in supporting free speech, farther and in different directions/dimensions than even Richard Stallman ever considered.

    BTW, the book I coauthored on ClearCase goes far in showing the technical details of what I mean.

    Good luck.
    Marc