Topic
15 replies Latest Post - ‏2012-04-20T17:43:58Z by SystemAdmin
E12590
E12590
23 Posts
ACCEPTED ANSWER

Pinned topic Regarding RAA and distributed scanner - Automating scanning

‏2012-04-04T12:33:58Z |
We are running RAA on the host(zos), and are now in the process of
getting RAA running on the distributed platform as well.

We are at the moment running RAA v6.0.0.7, and are planning to go to
v.6.0.0.8 later in April 2012.

The questions I have is regarding automating the scanning on the distributed
platform:
1) What should the bat script or bat command look like to add source scanning requests
to RAA on the Distributed platform?
2) What should the bat script or bat command look like to request analysis of the distributed assets?
3) Which user is the bat scripts or bat commands using, with regards to DmhScan.cfg
LocalAuthentication and RemoteAuthentication?
And what if both of these are set to False?
Updated on 2012-04-20T17:43:58Z at 2012-04-20T17:43:58Z by SystemAdmin
  • jcdelmo
    jcdelmo
    343 Posts
    ACCEPTED ANSWER

    Re: Regarding RAA and distributed scanner - Automating scanning

    ‏2012-04-09T17:21:23Z  in response to E12590
    What should the bat script or bat command look like to add source scanning requests to RAA on the Distributed platform?
    What should the bat script or bat command look like to request analysis of the distributed assets?

    I'm not sure which scenario you mean...
    • You have installed the remote-components on Windows connecting to RAA on z/OS.
    • You have installed a full-blown instance of RAA on Windows. z/OS is not involved.

    Since I don't know which it is, I'll describe both.

    If you have installed the remote-components on Windows connecting to RAA on z/OS:
    Running batch files is not required to load distributed assets. Distributed container scan requests can be queued up like any other container scan request using whatever mechanisms are available for z/OS containers up on z/OS. As long as the site and resource manager on the queue entry is set to the distributed site and NTFS, the Java Queue Processor will forward the container scan request to the remote scanner daemon and download and load the results.

    It's all transparent from the user's perspective, and fundamentally not different from z/OS containers. They really never need to touch the remote machine after it's been installed. It's just a dummy service listening for container scan requests from the z/OS machine. The user does everything from the z/OS UI or DMH0700-style batch inserts.

    If you have installed a full-blown instance of RAA on Windows. z/OS is not involved:
    There is not a distinct command for submitting container scan requests from a command line in Windows.

    The closest we have would be dmhrest.bat, where you would post DMH0700-style import files to /raarest/import
    dmhrest.bat POST /raarest/import -user me -pw mypw -i importFile.txt

    ...followed by starting the queue processor...
    dmhstartqp.bat -user me -pw mypw

    But again, you're not touching the scanner at all. The user is doing everything through the RAA REST interface. The scanner is just a dummy service waiting for scan requests from the queue processor.
    Which user is the bat scripts or bat commands using, with regards to DmhScan.cfg: LocalAuthentication and RemoteAuthentication? And what if both of these are set to False?

    The LocalAuthentication settings are no longer used. This setting was the first attempt at securing the scanner daemon's REST interface. It simply set a user/pw on the interface. We stopped using it because that user/pw would have to be stored in the database in order to access the scanner daemon.

    The RemoteAuthentication setting simply authenticates against the RAA REST interface (/raarest/admin/authenticateUser). So the user/pw to access the scanner daemon is the same as the user/pw to access the RAA REST API. It's a preferred solution since you don't need to store the user/pw in DmhScan.cfg or the database. The RAA REST engine can simply use it's own credentials to authenticate to the scanner daemon REST interface. The only downside to this approach is that you cannot connect to the same scanner daemon from two different instances of RAA (unless they're using the exact same credentials).

    If both are set to false, then there is no security enabled on the scanner daemon, and anyone with a browser can look at all the files on your file system. So it's important that security be enabled.
    • E12590
      E12590
      23 Posts
      ACCEPTED ANSWER

      Re: Regarding RAA and distributed scanner - Automating scanning

      ‏2012-04-10T09:28:00Z  in response to jcdelmo
      We have installed the remote-components on Windows connecting to RAA on zOS.
      • E12590
        E12590
        23 Posts
        ACCEPTED ANSWER

        Re: Regarding RAA and distributed scanner - Automating scanning

        ‏2012-04-10T10:47:50Z  in response to E12590
        I haven't been able to find a description of a distributed scan request, I have found out that the type is DIR but what about the other parameters you specify when you request a scanning via the WEB interface?

        We have 3 containers (distributed), but they are still lying in the ANALYSIS queue and look like they aren't procssed even though we have run a analysis scan (from Q17 to Q14).
    • E12590
      E12590
      23 Posts
      ACCEPTED ANSWER

      Re: Regarding RAA and distributed scanner - Automating scanning

      ‏2012-04-11T08:15:47Z  in response to jcdelmo
      Thank you for your answer.

      We have installed the remote-components on Windows connecting to RAA on zOS.

      In response to your answer I haven't been able to find a description of the input format, for requesting an analysis of a distributed asset scan request - when adding items to the zOS analysis queue (as a batch job on zOS).

      I have found out that the type is DIR, but what about the other parameters you specify when you request a scanning via the WEB interface?
      • How should the request input string look like on zOS for distributed assets?
      • SystemAdmin
        SystemAdmin
        849 Posts
        ACCEPTED ANSWER

        Re: Regarding RAA and distributed scanner - Automating scanning

        ‏2012-04-11T14:19:34Z  in response to E12590
        This is what I would suggest, which has been used successfully at some installations I have been involved

        1) For a given JAVA application, create some kind of script that when the application is promoted, it creates a ZIP file containing the byte code archive (EAR, WAR..) and a directory structure with the source used to build the part. Name the ZIP file with the application name, no spaces, lets say "myJavaAppl.zip". Same script should drop the file in what I call the distributed root directory (2) and create the scan request card (3)
        2) Create the distributed scan root in the scanning server. Make it as short as possible, no spaces, lets say D:\JVA_ASSETS. The ZIP file from (1) should drop in it.
        3) Scan request. The format is, which applies to z/OS and distributed assets is

        C rmType siteName containerName memberFilter [parentApp,]app N) N) N)

        For JAVA file schema in (1) it would be

        C NTFS siteName D:\JVA_ASSETS\myJavaAppl.zip * MYJAVAAPPL SUBDIR=Y UTSCAN=Y SCANZIPS=Y

        That should result in scanning all the files in container "D:\JVA_ASSETS\myJavaAppl.zip", opening ZIPs and recursing directories, with symbol scanning.

        Make sure you have good network bandwidth. Make sure you perform progressive RUNSTATS on z/OS a few times as you scan a few applications. A single JAVA application can insert many hundred thousands or even millions.
        • SystemAdmin
          SystemAdmin
          849 Posts
          ACCEPTED ANSWER

          Re: Regarding RAA and distributed scanner - Automating scanning

          ‏2012-04-11T14:32:03Z  in response to SystemAdmin
          Sorry, the format got mangled, I will try again

          "C rmType siteName containerName memberFilter [ parentApp] ,app N) N) N) "
          • SystemAdmin
            SystemAdmin
            849 Posts
            ACCEPTED ANSWER

            Re: Regarding RAA and distributed scanner - Automating scanning

            ‏2012-04-11T14:38:07Z  in response to SystemAdmin
            Goody !

            C rmType siteName containerName memberFilter appls subDirOpt symScan zipScan

            where appls, subDirOpt symScan and zipScan are optional and like

            parentApp,app (parentAppl optional, must exist already. Child application will be created if not exists
            SUBDIR=Y|N
            UTSCAN=Y|N
            SCANZIPS=Y|N
            • E12590
              E12590
              23 Posts
              ACCEPTED ANSWER

              Re: Regarding RAA and distributed scanner - Automating scanning

              ‏2012-04-16T15:38:32Z  in response to SystemAdmin
              Thank you for your answer.

              We have tried it in RAA v6.0.0.8 but it seems that it doesn't recognize then SUBDIR, UTSCAN and SCANZIPS parameters, but it doesn't give any error messages. We have tried with SUBDIR=Y and also just an Y but with the same result.
              • SystemAdmin
                SystemAdmin
                849 Posts
                ACCEPTED ANSWER

                Re: Regarding RAA and distributed scanner - Automating scanning

                ‏2012-04-16T19:43:07Z  in response to E12590
                What are you using to submit the request and from where? The format I have provided is supposed to be consumed via the REST API. I am not totally familar with the parts that ship with the remote component for the z/Series version. Look for a dmhload.bat file. It is a REST interface for submission.
                • E12590
                  E12590
                  23 Posts
                  ACCEPTED ANSWER

                  Re: Regarding RAA and distributed scanner - Automating scanning

                  ‏2012-04-17T15:49:48Z  in response to SystemAdmin
                  I have tried to submit it from zOS using the DMH0700 program and analysing it with the DMH6000 program.
                  • SystemAdmin
                    SystemAdmin
                    849 Posts
                    ACCEPTED ANSWER

                    Re: Regarding RAA and distributed scanner - Automating scanning

                    ‏2012-04-17T15:55:17Z  in response to E12590
                    As far as I know 700 only supports z/OS assets. You have to use the REST API path.
                    • rogern
                      rogern
                      5 Posts
                      ACCEPTED ANSWER

                      Re: Regarding RAA and distributed scanner - Automating scanning

                      ‏2012-04-17T16:55:00Z  in response to SystemAdmin
                      I spoke to the former developer/owner of the distributed scanner support. He mentioned there is a dmhstartqp.sh script that could be executed in a USS process that would signal the servlet to start
                      the queue processor. We have never tested such a scenario in-house so I'm checking with a couple of
                      my colleagues to see if they have sample JCL we can model for running USS batch scripts.

                      I'm not sure if RAA will officially support this solution, but I will at least see what I can
                      dig up
                      • SystemAdmin
                        SystemAdmin
                        849 Posts
                        ACCEPTED ANSWER

                        Re: Regarding RAA and distributed scanner - Automating scanning

                        ‏2012-04-19T16:00:50Z  in response to rogern
                        What are you trying to scan on distributed side?
                        Are those development side artifacts, or the deployed artifacts?
                        • E12590
                          E12590
                          23 Posts
                          ACCEPTED ANSWER

                          Re: Regarding RAA and distributed scanner - Automating scanning

                          ‏2012-04-20T09:13:32Z  in response to SystemAdmin
                          It is .EAR files that are ready for deployment.
                          • SystemAdmin
                            SystemAdmin
                            849 Posts
                            ACCEPTED ANSWER

                            Re: Regarding RAA and distributed scanner - Automating scanning

                            ‏2012-04-20T17:43:58Z  in response to E12590
                            Make sure that the requests for scanning have the EAR file name as part of the container with members '*', and not a base container with a member name of earFilename.EAR. For RAA an archive file is really a container.