Topic
  • 9 replies
  • Latest Post - ‏2018-09-10T06:47:36Z by niceseb38
Lionel Butler
Lionel Butler
1 Post

Pinned topic RDi V9.6 Code Coverage License implications

‏2017-10-09T11:09:28Z |

Based on the announcement for V9.6:

Code coverage: Initially delivered only in the Rational Developer for i desktop tools, the code coverage engine is now being delivered as part of the Rational Development Studio, callable only from the command line. This new support allows code coverage to be a part of the build process in order to get solid measurement of the effectiveness of tests being executed.  RDi is required only to view the results.

 

Does this mean any user can run the code coverage command to gather the dat?  i.e. the base Rational Development Studio for i license  covers them?

Or is the command linked to requiring an ADTS or one of the compiler licenses per user running the command?

 

Many thanks

Lionel

 

 

  • EdmundReinhardt
    EdmundReinhardt
    269 Posts

    Re: RDi V9.6 Code Coverage License implications

    ‏2017-11-14T14:27:22Z  

    Hi Lionel,

    I am sorry that I did not see your question until now.

    I have good news for you.  The CODECOV command will be available in the RDS option 60.  Option 60 is free of charge and does not require the purchase of any of the compilers or tools.  So this is basically free.

    The output is in a .cczip file in the directory/file you specify.  You can point to it from RDi or copy it to your workstation and view the results from RDi, but no license of RDi is required to gather the information.

    This is all available via PTF now

     

    V7R2M0 MRM PTF
    SI65228|5050|approved|2017-10-23 19:47:21.0

    V7R2M0 MRI PTFs
    SI64544|2924|approved|2017-07-27 20:17:54.0
    SI64547|2902|approved|2017-10-23 19:55:33.0
    SI64548|2903|approved|2017-10-23 19:55:58.0
    SI64549|2904|approved|2017-10-23 19:56:20.0
    SI64550|2905|approved|2017-10-23 19:56:41.0
    SI64551|2906|approved|2017-10-23 19:57:03.0
    SI64552|2909|approved|2017-10-23 19:57:24.0
    SI64553|2911|approved|2017-10-23 19:57:50.0
    SI64554|2912|approved|2017-10-23 19:58:13.0
    SI64555|2913|approved|2017-10-23 19:58:37.0
    SI64556|2914|approved|2017-10-23 19:59:02.0
    SI64557|2922|approved|2017-10-23 19:59:25.0
    SI64558|2923|approved|2017-10-23 19:59:48.0
    SI64559|2925|approved|2017-10-23 20:00:14.0
    SI64560|2926|approved|2017-10-23 20:00:38.0
    SI64561|2928|approved|2017-10-23 20:01:01.0
    SI64562|2929|approved|2017-10-23 20:01:26.0
    SI64563|2930|approved|2017-10-23 20:01:49.0
    SI64564|2931|approved|2017-10-23 20:02:12.0
    SI64566|2932|approved|2017-10-23 20:02:39.0
    SI64567|2933|approved|2017-10-23 20:03:03.0
    SI64568|2937|approved|2017-10-23 20:03:25.0
    SI64569|2938|approved|2017-10-23 20:03:50.0
    SI64570|2939|approved|2017-10-23 20:04:15.0
    SI64571|2940|approved|2017-10-23 20:04:34.0
    SI64572|2942|approved|2017-10-23 20:04:52.0
    SI64573|2954|approved|2017-10-23 20:05:10.0
    SI64574|2956|approved|2017-10-23 20:05:28.0
    SI64575|2957|approved|2017-10-23 20:05:47.0
    SI64576|2958|approved|2017-10-23 20:06:05.0
    SI64577|2961|approved|2017-10-23 20:06:23.0
    SI64578|2962|approved|2017-10-23 20:06:41.0
    SI64579|2963|approved|2017-10-23 20:06:59.0
    SI64580|2966|approved|2017-10-23 20:07:17.0
    SI64581|2972|approved|2017-10-23 20:07:34.0
    SI64583|2974|approved|2017-10-23 20:07:52.0
    SI64584|2975|approved|2017-10-23 20:08:10.0
    SI64585|2976|approved|2017-10-23 20:08:28.0
    SI64586|2978|approved|2017-10-23 20:08:46.0
    SI64587|2979|approved|2017-10-23 20:09:04.0
    SI64588|2980|approved|2017-10-23 20:09:22.0
    SI64589|2981|approved|2017-10-23 20:09:40.0
    SI64590|2984|approved|2017-10-23 20:09:59.0
    SI64591|2986|approved|2017-10-23 20:10:18.0
    SI64592|2987|approved|2017-10-23 20:10:37.0
    SI64593|2989|approved|2017-10-23 20:10:55.0
    SI64594|2992|approved|2017-10-23 20:11:13.0
    SI64595|2994|approved|2017-10-23 20:11:31.0
    SI64596|2995|approved|2017-10-23 20:11:49.0
    SI64597|2996|approved|2017-10-23 20:12:07.0
    SI64598|2998|approved|2017-10-23 20:12:25.0


    V7R3M0 MRM PTF
    SI65229|5050|approved|2017-10-23 19:48:47.0

    V7R3M0 MRI PTFs
    SI64655|2924|approved|2017-07-27 20:19:17.0
    SI64656|2902|approved|2017-10-23 20:37:19.0
    SI64657|2903|approved|2017-10-23 20:37:35.0
    SI64658|2904|approved|2017-10-23 20:37:53.0
    SI64659|2905|approved|2017-10-23 20:38:11.0
    SI64660|2906|approved|2017-10-23 20:38:26.0
    SI64661|2909|approved|2017-10-23 20:38:44.0
    SI64662|2911|approved|2017-10-23 20:39:01.0
    SI64663|2912|approved|2017-10-23 20:39:17.0
    SI64664|2913|approved|2017-10-23 20:39:33.0
    SI64665|2914|approved|2017-10-23 20:39:47.0
    SI64666|2922|approved|2017-10-23 20:40:02.0
    SI64667|2923|approved|2017-10-23 20:40:14.0
    SI64668|2925|approved|2017-10-23 20:40:30.0
    SI64669|2926|approved|2017-10-23 20:40:44.0
    SI64670|2928|approved|2017-10-23 20:40:56.0
    SI64671|2929|approved|2017-10-23 20:41:12.0
    SI64672|2930|approved|2017-10-23 20:41:26.0
    SI64673|2931|approved|2017-10-23 20:41:40.0
    SI64674|2932|approved|2017-10-23 20:41:56.0
    SI64675|2933|approved|2017-10-23 20:42:10.0
    SI64676|2937|approved|2017-10-23 20:42:24.0
    SI64677|2938|approved|2017-10-23 20:42:39.0
    SI64678|2939|approved|2017-10-23 20:42:56.0
    SI64679|2940|approved|2017-10-23 20:43:13.0
    SI64680|2942|approved|2017-10-23 20:43:29.0
    SI64681|2954|approved|2017-10-23 20:43:46.0
    SI64682|2956|approved|2017-10-23 20:44:01.0
    SI64683|2957|approved|2017-10-23 20:44:15.0
    SI64684|2958|approved|2017-10-23 20:44:33.0
    SI64685|2961|approved|2017-10-23 20:44:46.0
    SI64686|2962|approved|2017-10-23 20:44:58.0
    SI64687|2963|approved|2017-10-23 20:45:13.0
    SI64688|2966|approved|2017-10-23 20:45:30.0
    SI64689|2972|approved|2017-10-23 20:45:48.0
    SI64690|2974|approved|2017-10-23 20:46:04.0
    SI64691|2975|approved|2017-10-23 20:46:20.0
    SI64692|2976|approved|2017-10-23 20:46:34.0
    SI64693|2978|approved|2017-10-23 20:46:46.0
    SI64694|2979|approved|2017-10-23 20:46:58.0
    SI64695|2980|approved|2017-10-23 20:47:12.0
    SI64696|2981|approved|2017-10-23 20:47:25.0
    SI64697|2984|approved|2017-10-23 20:47:39.0
    SI64698|2986|approved|2017-10-23 20:47:51.0
    SI64699|2987|approved|2017-10-23 20:48:05.0
    SI64700|2989|approved|2017-10-23 20:48:17.0
    SI64701|2992|approved|2017-10-23 20:48:33.0
    SI64702|2994|approved|2017-10-23 20:48:47.0
    SI64703|2995|approved|2017-10-23 20:49:00.0
    SI64704|2996|approved|2017-10-23 20:49:16.0
    SI64705|2998|approved|2017-10-23 20:49:33.0

  • barbara_morris
    barbara_morris
    24 Posts

    Re: RDi V9.6 Code Coverage License implications

    ‏2017-11-20T23:25:23Z  

    About the "MRI PTFs", you probably only need one or maybe two of those additional PTFs. You only need the one(s) for the languages currently installed on your system.

    To find out which languages are installed on your system

    • Use command "GO LICPGM"
    • Choose option 20 (Display installed secondary languages)
      • The primary language for your system is listed in the first line of the output. For my 7.3 system, it is "2924" (English).
      • If you have secondary languages, they will be listed below. You're interested in the second column where it lists the numbers.
    • You only need the PTFs that have your installed language numbers next to the PTF number in PTF lists above. For example, on my system, I would need PTF SI64655, because it says "SI64655|2924|".

    So on my 7.3 system, I would need the "MRM" PTF SI65229 and the "MRI" PTF SI64655.

    If my primary language were Danish (2926), and I had English (2924) installed as a secondary language, I would need 3 PTFs: the "MRM" PTF SI65229 and the "MRI" PTFs SI64669 (for 2926) and SI64655 (for 2924).

  • niceseb38
    niceseb38
    4 Posts

    Re: RDi V9.6 Code Coverage License implications

    ‏2018-08-22T05:40:45Z  

    Hi Edmund or anyone,

     

    What is the attribute hits="PPPPJPP" within the ccdata xml file mean?

     

    Thanks

     


    Sebastian

  • AlanBoxall
    AlanBoxall
    9 Posts

    Re: RDi V9.6 Code Coverage License implications

    ‏2018-08-22T16:31:55Z  
    • niceseb38
    • ‏2018-08-22T05:40:45Z

    Hi Edmund or anyone,

     

    What is the attribute hits="PPPPJPP" within the ccdata xml file mean?

     

    Thanks

     


    Sebastian

    The hits attribute is an encoded string that records which lines were hit, it works together with the lines= attribute.

     

    Rather than document the format (which can change) we make the information available through an API.

    The API is in a jar (called ccapi.jar) that can process various code coverage formats and then you can use java code to get the results without needing to know the format of the files. 

     

    You can find the documentation here on the interface to the ccapi
    https://www.ibm.com/support/knowledgecenter/SSQ2R2_14.1.0/com.ibm.codecoverage.javadoc.doc/index.html

    The ccapi is versioned and should be close to the one supplied with RDi. 

    To get the one that matches your installed version of RDi see the following:

    You can find the ccapi in the plugin called 

    com.ibm.debug.pdt.codecoverage.core.results_<version>.jar in the shared install directory.
    The default shared install directory will be called something like 
    C:\Program Files\IBM\IBMIMShared\plugins

    Jar files are in a zip format.  So if you have a favourite tool that opens zip files you can use it on the above .jar

    In the jar you will find the ccapi.jar

    META-INF
    ccapi.jar  <-- this is the one you want to copy out.
    ccapiext.jar

     

    Create a test java program (I've supplied a snippet below that shows using the api to read in the data)

    and then invoke it passing the ccapi.jar in the classpath

     

    The following snippet illustrates how you get a result object from an existing cczip file.

    The argument to createResult is a path to the cczip or ccresult file (it can also process .clcoveragedata files)

     

    ICCResult results = null;
    try {
    results = CCResultsFactory.getInstance().createResult({"path to file"});
    } catch (CCResultException e) {
    e.printStackTrace();
    }

    Now that you have an ICCResult object you can use the javadoc to see what you can extract from it.

    for example results.getPercentCoverage()

    will give you a numeric value for the overall percent coverage.

     

    In summary, rather than document the raw formats of the cc data files that we generate, we supply a public api to read and process the coverage data.

    This shields you from any changes that we make to the raw data.

    Hope this answers your original question!

     

    Alan Boxall, Code Coverage Architect.

  • niceseb38
    niceseb38
    4 Posts

    Re: RDi V9.6 Code Coverage License implications

    ‏2018-08-23T05:48:30Z  

    The hits attribute is an encoded string that records which lines were hit, it works together with the lines= attribute.

     

    Rather than document the format (which can change) we make the information available through an API.

    The API is in a jar (called ccapi.jar) that can process various code coverage formats and then you can use java code to get the results without needing to know the format of the files. 

     

    You can find the documentation here on the interface to the ccapi
    https://www.ibm.com/support/knowledgecenter/SSQ2R2_14.1.0/com.ibm.codecoverage.javadoc.doc/index.html

    The ccapi is versioned and should be close to the one supplied with RDi. 

    To get the one that matches your installed version of RDi see the following:

    You can find the ccapi in the plugin called 

    com.ibm.debug.pdt.codecoverage.core.results_<version>.jar in the shared install directory.
    The default shared install directory will be called something like 
    C:\Program Files\IBM\IBMIMShared\plugins

    Jar files are in a zip format.  So if you have a favourite tool that opens zip files you can use it on the above .jar

    In the jar you will find the ccapi.jar

    META-INF
    ccapi.jar  <-- this is the one you want to copy out.
    ccapiext.jar

     

    Create a test java program (I've supplied a snippet below that shows using the api to read in the data)

    and then invoke it passing the ccapi.jar in the classpath

     

    The following snippet illustrates how you get a result object from an existing cczip file.

    The argument to createResult is a path to the cczip or ccresult file (it can also process .clcoveragedata files)

     

    ICCResult results = null;
    try {
    results = CCResultsFactory.getInstance().createResult({"path to file"});
    } catch (CCResultException e) {
    e.printStackTrace();
    }

    Now that you have an ICCResult object you can use the javadoc to see what you can extract from it.

    for example results.getPercentCoverage()

    will give you a numeric value for the overall percent coverage.

     

    In summary, rather than document the raw formats of the cc data files that we generate, we supply a public api to read and process the coverage data.

    This shields you from any changes that we make to the raw data.

    Hope this answers your original question!

     

    Alan Boxall, Code Coverage Architect.

    Thanks Alan, this is great response as we looked everywhere but not much info for this explanation.

    But I need to parse this xml first before it can be applied to the createResult method?

    How to parse it (in Windows RDi environment), with CBLCALLER_xxxx.zip in project root:

     

        results = CCResultsFactory.getInstance().createResult({"/testCCzip/CBLCALLER_xxxx.zip"});

     

    Exception The method createResult(String[] in the type CCResultsFactory is not applicable for the arguments ()

     

    Also tried:
     

    String [] ary = new String[] {"C:\\Users\\xxxx\\IBM\\rationalsdp\\workspace\\testCCzip\\CBLCALLER_xxxxxxxcc.zip"};

    try {

    results = CCResultsFactory.getInstance().createResult(ary);

    }

     

    CCResultException: CRRDG7215E Errors occurred during import. Details are included in this exception:

    CCResultException.<init> CCResultException.java:32

    CCResultsFactory.createResult(CCResultsFactory.java:163)

    CCResultsFactory.createResult(CCResultsFactory.java:84)

    test.test.main(test.java:25)

    Seb

    Updated on 2018-08-23T10:15:17Z at 2018-08-23T10:15:17Z by niceseb38
  • AlanBoxall
    AlanBoxall
    9 Posts

    Re: RDi V9.6 Code Coverage License implications

    ‏2018-08-23T14:46:54Z  
    • niceseb38
    • ‏2018-08-23T05:48:30Z

    Thanks Alan, this is great response as we looked everywhere but not much info for this explanation.

    But I need to parse this xml first before it can be applied to the createResult method?

    How to parse it (in Windows RDi environment), with CBLCALLER_xxxx.zip in project root:

     

        results = CCResultsFactory.getInstance().createResult({"/testCCzip/CBLCALLER_xxxx.zip"});

     

    Exception The method createResult(String[] in the type CCResultsFactory is not applicable for the arguments ()

     

    Also tried:
     

    String [] ary = new String[] {"C:\\Users\\xxxx\\IBM\\rationalsdp\\workspace\\testCCzip\\CBLCALLER_xxxxxxxcc.zip"};

    try {

    results = CCResultsFactory.getInstance().createResult(ary);

    }

     

    CCResultException: CRRDG7215E Errors occurred during import. Details are included in this exception:

    CCResultException.<init> CCResultException.java:32

    CCResultsFactory.createResult(CCResultsFactory.java:163)

    CCResultsFactory.createResult(CCResultsFactory.java:84)

    test.test.main(test.java:25)

    Seb

    1) The ccapi does all the parsing.  No need to do anything with xml :-)

    2) It is interesting that using {} directly didn't work.  I'm using java 1.7, I thought I tried it before completing the append.

    Initializing the String[] first is another way I do it too.

    2.1) ccapi requires at least a 1.7 java

    3) On to the exception

    The file extension might be the problem.   If the zip contains a ccdata file then the ccapi expects it to end with .cczip

    Try renaming the file to have a .cczip extension 

    During development I use 7-Zip and tell windows to open .cczip with that tool.  Makes it much easier to see what is inside.

    4) There is a trick to seeing what the exceptions are.

    During "import" (createResult() imports the data) anything that goes wrong is gathered and attached to the global CCResultException (that's why the message says the details are included in the exception)

    This allows the import to continue where it can and at the end it throws the CCResultException.

    In some cases you can retrieve the partial results calling CCResultException.getResult().  It might not be complete but at least you can see what did work.

     

    CCResultException has a method to get all the exceptions/messages generated.
    I usually put something like this in the catch clause

    } catch (CCResultException e) {
    for(CCAbstractException ie : e.getExceptions())
    System.out.println(ie.getMessage());
    }

    You can also use CCResultException.getMessages() to see if there were any informational messages logged.

    We'll get this working!

  • niceseb38
    niceseb38
    4 Posts

    Re: RDi V9.6 Code Coverage License implications

    ‏2018-09-05T09:28:47Z  

    1) The ccapi does all the parsing.  No need to do anything with xml :-)

    2) It is interesting that using {} directly didn't work.  I'm using java 1.7, I thought I tried it before completing the append.

    Initializing the String[] first is another way I do it too.

    2.1) ccapi requires at least a 1.7 java

    3) On to the exception

    The file extension might be the problem.   If the zip contains a ccdata file then the ccapi expects it to end with .cczip

    Try renaming the file to have a .cczip extension 

    During development I use 7-Zip and tell windows to open .cczip with that tool.  Makes it much easier to see what is inside.

    4) There is a trick to seeing what the exceptions are.

    During "import" (createResult() imports the data) anything that goes wrong is gathered and attached to the global CCResultException (that's why the message says the details are included in the exception)

    This allows the import to continue where it can and at the end it throws the CCResultException.

    In some cases you can retrieve the partial results calling CCResultException.getResult().  It might not be complete but at least you can see what did work.

     

    CCResultException has a method to get all the exceptions/messages generated.
    I usually put something like this in the catch clause

    } catch (CCResultException e) {
    for(CCAbstractException ie : e.getExceptions())
    System.out.println(ie.getMessage());
    }

    You can also use CCResultException.getMessages() to see if there were any informational messages logged.

    We'll get this working!

    Thanks Alan got it to work, but something I am not too sure, when working with one xxx.cczip file, the extracted results.getPercentageCoverage() is of a certain value say 85%, but when I have 

    xxx.cczip

    and

    xxy.cczip

    now iterating through in a loop, the value of the extracted results.getPercentageCoverage() from previous run, sometimes changes and sometimes it doesn't.

    This I understand is related to unit test overlapping, but just couldn't work out why even if that is the case, why would a result from a given xxx.cczip ever change when it was already

    previously generated? The idea is to extract coverage from a list of xxx.cczip files but this puzzles me why results seems to dynamically change.

     

    Is this related to how xxx.cczip is generated, with the fMerged=true flag?

     

     

    Seb

    Updated on 2018-09-06T01:45:56Z at 2018-09-06T01:45:56Z by niceseb38
  • AlanBoxall
    AlanBoxall
    9 Posts

    Re: RDi V9.6 Code Coverage License implications

    ‏2018-09-05T15:41:28Z  
    • niceseb38
    • ‏2018-09-05T09:28:47Z

    Thanks Alan got it to work, but something I am not too sure, when working with one xxx.cczip file, the extracted results.getPercentageCoverage() is of a certain value say 85%, but when I have 

    xxx.cczip

    and

    xxy.cczip

    now iterating through in a loop, the value of the extracted results.getPercentageCoverage() from previous run, sometimes changes and sometimes it doesn't.

    This I understand is related to unit test overlapping, but just couldn't work out why even if that is the case, why would a result from a given xxx.cczip ever change when it was already

    previously generated? The idea is to extract coverage from a list of xxx.cczip files but this puzzles me why results seems to dynamically change.

     

    Is this related to how xxx.cczip is generated, with the fMerged=true flag?

     

     

    Seb

    Checking my implementation... 

    The calculation is simply hitlines/executable lines * 100

    I'm not entirely sure I understand your question so perhaps it would help if I explain how it works.

    First... when you "merge" >1 result it will check if the files match and only do the merge if the files are the same (that assumes it has access to the source)

    Assuming the files match, the executable lines should stay constant (that isn't entirely true in the case of C/C++ header files) and only the hitlines will change based on the testcase.

    So the ccapi tracks the number of executable lines AND the lines that were hit.   When merging > 1 result the number of executable lines won't change but the number of hit lines can increase as the hit lines are merged.   They should not decrease!

    When you call ICCFile.getPercentCoverage() it will return the coverage based on the merged hitlines.    

    To get the individual percentcoverage in a merged result you would have to do the calculation yourself but it is easy.

    ICCFile.getNumExecutableLines() will remain constant

    ICCFile.getHitLines(ICCTestcase) will return the hit lines for the individual testcase.  (typically a result == testcase unless the result was already merged)

    so you can do the following:

    ICCCoverageData.getNumExecutableLines()/ICCCoverageData.getHitLines(<specific testcase>) * 100

     

    The design/architecture of the ccapi is that while it merges results (testcase data) the individual results can still be retrieved.

     

    NOTE: the ccapi uses interfaces to describe artifact behaviour

    anything that implements ICCPercentCoverage can calculate coverage

    anything that implements ICCCoverageData can return detailed coverage information

     

    Not sure I answered your question but I hope the background explains what it is intended to do.   if you think it isn't behaving as described let me know.

  • niceseb38
    niceseb38
    4 Posts

    Re: RDi V9.6 Code Coverage License implications

    ‏2018-09-10T06:47:36Z  

    Checking my implementation... 

    The calculation is simply hitlines/executable lines * 100

    I'm not entirely sure I understand your question so perhaps it would help if I explain how it works.

    First... when you "merge" >1 result it will check if the files match and only do the merge if the files are the same (that assumes it has access to the source)

    Assuming the files match, the executable lines should stay constant (that isn't entirely true in the case of C/C++ header files) and only the hitlines will change based on the testcase.

    So the ccapi tracks the number of executable lines AND the lines that were hit.   When merging > 1 result the number of executable lines won't change but the number of hit lines can increase as the hit lines are merged.   They should not decrease!

    When you call ICCFile.getPercentCoverage() it will return the coverage based on the merged hitlines.    

    To get the individual percentcoverage in a merged result you would have to do the calculation yourself but it is easy.

    ICCFile.getNumExecutableLines() will remain constant

    ICCFile.getHitLines(ICCTestcase) will return the hit lines for the individual testcase.  (typically a result == testcase unless the result was already merged)

    so you can do the following:

    ICCCoverageData.getNumExecutableLines()/ICCCoverageData.getHitLines(<specific testcase>) * 100

     

    The design/architecture of the ccapi is that while it merges results (testcase data) the individual results can still be retrieved.

     

    NOTE: the ccapi uses interfaces to describe artifact behaviour

    anything that implements ICCPercentCoverage can calculate coverage

    anything that implements ICCCoverageData can return detailed coverage information

     

    Not sure I answered your question but I hope the background explains what it is intended to do.   if you think it isn't behaving as described let me know.

    Thanks Alan, it seems to work.

     

    But could this ccapi.jar be placed in Maven Central? As I am not sure how to add this jar in Intellij in a Maven archetype project, but only worked in Eclipse project.Thanks