Topic
15 replies Latest Post - ‏2012-04-19T06:36:00Z by raygear
raygear
raygear
17 Posts
ACCEPTED ANSWER

Pinned topic Automation script to checkin/checkout, build and deploy using ClearCase

‏2012-03-28T10:03:24Z |
Wonder if it is possible by means of scripting to automate certain processes e.g. check-in and check out.

Let's say that i have created certain structure in the repository
ABC
- SRC
- BIN

And i wish to have a script to check out to a common folder and from the common folder check the files back in. Is there such way to do it? And what if I am using ClearCase Remote Client?

Next is there a way to compile the files (e.g. Java) and generate a war file using ClearCase? Read somewhere mentioning "clearmake", does it do such function?

Seek you advise on the matter. automation can mean scripting, basically minimise human intervention.
Updated on 2012-04-19T06:36:00Z at 2012-04-19T06:36:00Z by raygear
  • martina
    martina
    1025 Posts
    ACCEPTED ANSWER

    Re: Automation script to checkin/checkout, build and deploy using ClearCase

    ‏2012-03-28T16:22:34Z  in response to raygear
    clearcase was born on the command line in a time way before there was an internet and everything you can do in a full client GUI you can do with cleartool on the command line.
    Now CCRC is a different story. There is a rcleartool that has beta quality. I think it went with some 7.1 version. I would not recommend using it for production purposes.
    To automate a build, you want to have the thick client and be on a box that is on the same LAN as the CC servers.

    as a side note: you do not want to check-in build results in your source VOB. It will get very big very soon and when updating your view in CCRC it will slow down things as well. I suggest a separate build result vob that you can "prune", i.e. remove unneeded versions, occasionally.
    Martina
    Don't Postpone Joy - Have Fun

    ReleaseTEAM Inc
    http://www.releaseteam.com
    IBM Rational Certified Consultants
    IBM Business Partner
    • raygear
      raygear
      17 Posts
      ACCEPTED ANSWER

      Re: Automation script to checkin/checkout, build and deploy using ClearCase

      ‏2012-03-29T02:12:50Z  in response to martina
      Thanks Martina for replying.

      I think I mentioned the wrong product. I actually meant ClearCase Team Explorer (CTE) if it matters.

      "To automate a build, you want to have the thick client and be on a box that is on the same LAN as the CC servers."
      > Do you mean if the compilier, common folders and CTE are on the same box and the CC servers are on the same LAN, i will be able to automate the build?
      > If that is so, is there any link that i can refer to develop the automation build script?

      Next, is it possible to deploy files (war files, artifacts, etc) to target appl server via CTE/CC?
      • raygear
        raygear
        17 Posts
        ACCEPTED ANSWER

        Re: Automation script to checkin/checkout, build and deploy using ClearCase

        ‏2012-03-29T02:15:05Z  in response to raygear
        Adding on my question - is it possible for the checkin and checkout automation? any links i can refer to? thanks.
      • martina
        martina
        1025 Posts
        ACCEPTED ANSWER

        Re: Automation script to checkin/checkout, build and deploy using ClearCase

        ‏2012-03-29T02:50:16Z  in response to raygear
        clearcase is mostly a version control system. It has a special make, clearmake. That is as far as it goes, It gives you commands. You get to write the script that executes the commands in a sequence that makes sense. You get to use your favorite scripting language.

        clearmake is mostly suited for C/C++ code although there is some stuff in there for java. It gets its power from dynamic views and great build avoidance features.
        To build java, these days ant or maven is what is commonly used. And if you want some sort of continuous integration, Jenkins is very popular.
        If you use Visual Studio, you get to look into the command line build for it.

        You don't say whether you are on windows or Linux or what your programming language is, but the above should cover the most common cases.

        The ClearCase command reference is at http://publib.boulder.ibm.com/infocenter/cchelp/v8r0m0/index.jsp?topic=%2Fcom.ibm.rational.clearcase.cc_ref.doc%2Ftopics%2Fintro.htm

        Now to CTE. IT is a common gui for web views (formerly CCRC) and dynamic views (thick client). So just from that info I cant tell whether you have the thick client or not. If you have cleartool, you have the thick client.
        Do
        cleartool -version
        and if you get a version, you have the thick client.

        Big warning: the thick client does not work over the WAN, the performance is very horrible. You need to use it on a client that is on the same LAN as the ClearCase (VOB) server.

        p.s. there is also a new java api, the CM API (unless they changed the name). Its documentation is very fuzzy and it doesn't have all commands implemented yet, so I would not really recommend it for you. It also requires that the thick client is installed.

        hth

        Martina
        Don't Postpone Joy - Have Fun

        ReleaseTEAM Inc
        http://www.releaseteam.com
        IBM Rational Certified Consultants
        IBM Business Partner
        • SystemAdmin
          SystemAdmin
          47293 Posts
          ACCEPTED ANSWER

          Re: Automation script to checkin/checkout, build and deploy using ClearCase

          ‏2012-03-29T11:19:52Z  in response to martina
          martina wrote:
          > clearcase is mostly a version control system.
          > It has a special make, clearmake.

          Sorry, I cannot let this pass. This may be your opinion, it may even be the mainstream opinion, it is not mine.

          I could say:

          ClearCase is mostly a build management system.
          This requires and motivates an original underlying version control system.
          Sorry for the original questions: I would ignore CTE. For WAN, I recommend MultiSite. If you are able to automatize checkout, edit and checkin, then probably the files do not need to be elements and can be derived objects (hence neither checkout nor checkin). The hard part is usually edit. Checkout and checkin are ways to communicate intentions to others: it doesn't make sense to automatize them at large.

          Marc
          • martina
            martina
            1025 Posts
            ACCEPTED ANSWER

            Re: Automation script to checkin/checkout, build and deploy using ClearCase

            ‏2012-03-29T15:17:39Z  in response to SystemAdmin
            well, I can come around to: in 1994, it was a state of the art build system (and underlying version control system) (on Unix systems, since this was when windows was in diapers).

            If you look at what is state-of-the art today with agile development and continuous integration, adding something like Jenkins to the mix is necessary.
            And while I certainly love my dynamic views and clearmake and build auditing, the way it works with java is cludgy, so ant or maven is a much more elegant way of building java.

            You can try and stem the tide, but I don't see how you can stop it.
            Martina
            Don't Postpone Joy - Have Fun

            ReleaseTEAM Inc
            http://www.releaseteam.com
            IBM Rational Certified Consultants
            IBM Business Partner
            • SystemAdmin
              SystemAdmin
              47293 Posts
              ACCEPTED ANSWER

              Re: Automation script to checkin/checkout, build and deploy using ClearCase

              ‏2012-03-29T19:33:07Z  in response to martina
              martina wrote:
              > well, I can come around to: in 1994, it was a state of the art build
              > system (and underlying version control system) (on Unix systems, since
              > this was when windows was in diapers).

              OK. We are not far away from each other.
              And it has not been developed much since then.
              Apart for the .JAVAC support...

              > If you look at what is state-of-the art today with agile development
              > and continuous integration, adding something like Jenkins to the mix
              > is necessary.

              It takes some courage to state that the emperor is naked, and that all this stuff is just thin air with big words.
              I know it may sound presumptuous, but that's my deep belief.

              > And while I certainly love my dynamic views and clearmake and build
              > auditing, the way it works with java is cludgy, so ant or maven is
              > a much more elegant way of building java.

              There, we disagree.
              The reason which makes .JAVAC look kludgy is purely bound to perverse branching strategies, which boil down to delivering by merging back.
              This introduces an artificial, spurious, instability in the versions referenced by the config records!
              It takes some work to support branching off, yet delivering in-place by labeling, but this is enough to make .JAVAC not kludgy at all, and in fact less kludgy than either ant or maven!

              Restoring the pristine economy of clearmake builds makes continuous integration completely spurious. It is only motivated by the need to hide horrendous kludges from the naive and cheap young graduates.

              All the essential work, I have done in my CPAN wrapper: ClearCase::Wrapper::MGi

              I keep making additions, still some significant ones... But I'll submit this as an RFE as well.

              > You can try and stem the tide, but I don't see how you can stop it.

              I know by experience. Only this tide is bigger yet: it is not only our house which will be removed to build a new road, the whole planet will also go because of an intergalactic motorway.

              I believe that I am suggesting the only way to salvage something of ClearCase, and the gem it was hiding. A revolution which did not happen.

              My company didn't want to fund my presenting this at the next conference (as: 'Waking Up Sleeping Beauty: base ClearCase').
              I hope they will support my (forthcoming) RFE (or set thereof).

              Marc
              • marct22
                marct22
                488 Posts
                ACCEPTED ANSWER

                Re: Automation script to checkin/checkout, build and deploy using ClearCase

                ‏2012-03-29T20:01:07Z  in response to SystemAdmin
                I too disagree about maven being a more elegant solution. Maven, while simple, takes 3rd party libraries out of clearcase unless you work at keeping the maven repository stored in clearcase (which it does not do by default). In the maven way, out-of-the-box, 3rd party jars and plugins are downloaded from various external websites and stored in your home directory. Consequently, out-of-the-box, you've lost reproducibility because they aren't labeled.

                Plus you are at the mercy of some external site removing a third-party jar you are dependent on from their website. this has bitten me where I needed version a.b.f, and only a.b.c and a.d.e were available on the various maven repositories I searched through/googled for.

                Nexus isn't really a solution neither, since it's repository can also be outside the VOBs. While you could serve stuff up out of Nexus that's fetched from clearcase, still, ultimately, the repository is in your home directory unless you do some configuration. This is probably easier done on *nix, where most people log into *nix to work (set maven to work globally) as opposed to windows, where I can have my own install of maven which could be configured differently than my fellow developer with her own maven install.

                While it is entirely possible to make maven work purely inside a VOB, you have to work at it, and should a developer configure things differently, you can run into the situation that we should have left years ago, where "I don't know why the build breaks for you, it builds for me".

                Marc Towersap
                Sr. Consultant
                ReleaseTEAM
                mtowersap@releaseteam.com
                • marct22
                  marct22
                  488 Posts
                  ACCEPTED ANSWER

                  Re: Automation script to checkin/checkout, build and deploy using ClearCase

                  ‏2012-03-29T20:17:35Z  in response to marct22
                  But anyhoo, regardless of using ant, maven, clearmake, msbuild, devenv, nant, gmake, whatever, and regardless of your CI tool (BuildForge, Hudson/Jenkins, TeamCity, ElectricCommander, CruiseControl, etc.), you still have to write some scripts to make your code build. We can't see your codebase. We don't know your test environment. You are going to have to take some initiative and write some scripts, be it something simple like just doing cd /to/some/directory, then ant clean build or mvn install, clearmake, etc., some scripts to deploy the code (copy war or ear here, bounce this, rename that, register dll, gacutil, etc.). Talk to your developers, they should tell you how to build it and someone should tell you how to uninstall the old then deploy the new. then write some scripts to do so. Whether you do it in a shell script, perl, python, a batch file, powershell, whatever, pick the language that works for you and that others can follow/modify. How you execute those scripts (manually or via a CI tool), well, try to figure it out.

                  Marc Towersap
                  Sr. Consultant
                  ReleaseTEAM
                  mtowersap@releaseteam.com
                  • raygear
                    raygear
                    17 Posts
                    ACCEPTED ANSWER

                    Re: Automation script to checkin/checkout, build and deploy using ClearCase

                    ‏2012-03-30T03:21:46Z  in response to marct22
                    from what were mentioned earlier, am i right to say that there is no out of the box scripts from CC/CCRC/CTE that will help to automate check in/check out/build/ deploy?

                    can i write scripts then to automate check in/ check out/ build/ deploy?

                    if yes, must i use a 3rd party tool like ant, maven, etc to develop the automation script for check in/ check out/ build/ deploy? is there anything in CC/CCRC/CTE that i can use at all understanding that there are the CLI commands?
                    • SystemAdmin
                      SystemAdmin
                      47293 Posts
                      ACCEPTED ANSWER

                      Re: Automation script to checkin/checkout, build and deploy using ClearCase

                      ‏2012-03-30T04:27:50Z  in response to raygear
                      You got dragged into a bit of an off topic war there :-)

                      I take it you are new to ClearCase? First thing you need to be aware of - there is no need to check out to build in ClearCase. In ClearCase you create a view. Views have rules (config specs) that make file/versions visible in the local file system (or something that looks like the local file system if you use dynamic views). The only files that you need to check out are those you are going to change. So generally if you can build your software in the local file system you can also build it in ClearCase.

                      You can use any appropriate tool to do the build. If you want to deploy - same rules apply. It is a file system. Use whatever tools you use today to deploy from a file system. If you want to automate a check in the results of the build or automate a deployment - then you will need do something extra. Exactly what will depend on the tools you use and what you need to deploy and where. There was discussion about a variety of tools in the thread. All of them are possible. Not everyone agrees on exactly what is best as you can tell from the other posts.

                      Some of the tools have built in integrations for ClearCase. In others you can just use the command line. There are command line tools for just about everything in ClearCase. You probably need to read up about command line capabilities. Maybe start here:
                      http://publib.boulder.ibm.com/infocenter/cchelp/v8r0m0/topic/com.ibm.rational.clearcase.tutorial.doc/topics/a_use_cli.htm
                      • raygear
                        raygear
                        17 Posts
                        ACCEPTED ANSWER

                        Re: Automation script to checkin/checkout, build and deploy using ClearCase

                        ‏2012-03-30T05:21:37Z  in response to SystemAdmin
                        Mark: very new to CC to be exact.
                        > there is no need to check out to build in ClearCase.
                        > So generally if you can build your software in the local file system you can also build it in ClearCase.

                        what do you mean by no need to check out and build it in CC?

                        Not familiar with the View you mentioned here. How does it work?
                        • SystemAdmin
                          SystemAdmin
                          47293 Posts
                          ACCEPTED ANSWER

                          Re: Automation script to checkin/checkout, build and deploy using ClearCase

                          ‏2012-03-30T07:07:07Z  in response to raygear
                          If you don't know what a view is you can't use ClearCase. You need to get a basic intro to ClearCase before you try doing things like automated builds using it. Nothing anyone says here will help you unless you do that first. If there is no one at your site that can help you at least go through the basic tutorials.
                          Start here: http://publib.boulder.ibm.com/infocenter/cchelp/v8r0m0/index.jsp
                          and look at the tutorials first. Then if you still need help come back here.
              • SystemAdmin
                SystemAdmin
                47293 Posts
                ACCEPTED ANSWER

                Re: Automation script to checkin/checkout, build and deploy using ClearCase

                ‏2012-03-30T21:12:39Z  in response to SystemAdmin
                Marc wrote:
                > it is not only our house which will be removed to build a new road,
                > the whole planet will also go because of an intergalactic motorway.

                It stroke me that I ought to mention that this was a reference to 'Hichhiker's Guide to the Galaxy'.
                And that I might remind I coauthored a book in which we go deeper on ClearCase, and thus on what I summed up previously. And once there, that I might announce the existence of an errata wiki:

                http://code.google.com/p/errata/wiki/WikiTop

                Marc
                • raygear
                  raygear
                  17 Posts
                  ACCEPTED ANSWER

                  Re: Automation script to checkin/checkout, build and deploy using ClearCase

                  ‏2012-04-19T06:36:00Z  in response to SystemAdmin
                  I think i might have found what i've been asking, but have yet to implement it.

                  "Automating the build/deploy process for Web applications managed with UCM"
                  http://www.ibm.com/developerworks/rational/library/jun06/tao/