Topic
  • 10 replies
  • Latest Post - ‏2013-05-27T15:53:08Z by Glen.S
SystemAdmin
SystemAdmin
1959 Posts

Pinned topic Cognos Transformer: file size exceeds 2GB

‏2012-12-03T04:49:28Z |
Dear Cognos gurus,

It's an old problem coming back.. Cognos Transformer 10.1.1 encounters this error:
(TR1901) PDS-PPE-0197 The local data cache size exceeded the allowed maximum of 2147418112 bytes.

Partition size has been changed from 1000000 to 500000 then 200000 - and I can see the cube build log reflecting this value (as shown below), however there will be a point where a certain *.mdp file exceeds 2GB, resulting in the same problem.

Anyone has an effective solution to this problems?

Many thanks,
timtam18

Mon 03 Dec 2012 14:21:59 4 00077D39 --- Performing Pass 2 with 23361408 rows and 611852 categories remaining.
Mon 03 Dec 2012 14:21:59 4 00077D39 Start Write leaving 611852 categories remaining.
Mon 03 Dec 2012 14:21:59 4 0039C207 Updating the PowerCube data.
Mon 03 Dec 2012 14:21:59 4 0039C207 Updating the PowerCube data.
Mon 03 Dec 2012 14:22:01 4 0039C207 Performing DataBase Commit at record number 250000. << reflecting the value mentioned above
Mon 03 Dec 2012 14:22:04 4 0039C207 Performing DataBase Commit at record number 500000.
Updated on 2013-02-08T04:03:24Z at 2013-02-08T04:03:24Z by SystemAdmin
  • whatever890
    whatever890
    6 Posts

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2012-12-06T14:44:08Z  
    Try changing the MultiFileCubeThreshold in the cogtr.xml file to something like 500000. If this value isn't in the cogtr.xml file then I believe you can add it.
  • SystemAdmin
    SystemAdmin
    1959 Posts

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2012-12-07T00:24:09Z  
    Try changing the MultiFileCubeThreshold in the cogtr.xml file to something like 500000. If this value isn't in the cogtr.xml file then I believe you can add it.
    Hi whatever890 - thanks for this

    MultiFileCubeThreshold helps managing the size of the *.mdp files but there is a point all the data is put into one single mdp file before being allocated to other partition. If the volume of data exceeds 2GB it still fails at this point.

    We've found a workaround by manually patitioning the cube using a level in a dimension where the data is evenly distibuted into its categories.

    Regards -
  • SteveW.
    SteveW.
    56 Posts

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2012-12-10T16:43:28Z  
    Hi whatever890 - thanks for this

    MultiFileCubeThreshold helps managing the size of the *.mdp files but there is a point all the data is put into one single mdp file before being allocated to other partition. If the volume of data exceeds 2GB it still fails at this point.

    We've found a workaround by manually patitioning the cube using a level in a dimension where the data is evenly distibuted into its categories.

    Regards -
    What kinds of performance are you getting by doing this?
  • SystemAdmin
    SystemAdmin
    1959 Posts

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2013-01-03T14:11:25Z  
    • SteveW.
    • ‏2012-12-10T16:43:28Z
    What kinds of performance are you getting by doing this?
    Please try to user Preference file while building the cube because by touching the XML file the change will be reflecting for every Cube build.

    Thanks & Regards,
    Vasudev Chavan
  • SystemAdmin
    SystemAdmin
    1959 Posts

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2013-01-10T01:39:38Z  
    Please try to user Preference file while building the cube because by touching the XML file the change will be reflecting for every Cube build.

    Thanks & Regards,
    Vasudev Chavan
    Hi Vasudev Chavan - yes this is correct thanks. We use the -f option and place these preference into the preference xml file instead of changing the global value of the whole Transformer environment.
    We've also found a more sustainable solution using time-based partitioned cube.
    Thanks,
  • SystemAdmin
    SystemAdmin
    1959 Posts

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2013-02-06T02:00:30Z  
    Hi Vasudev Chavan - yes this is correct thanks. We use the -f option and place these preference into the preference xml file instead of changing the global value of the whole Transformer environment.
    We've also found a more sustainable solution using time-based partitioned cube.
    Thanks,
    Hi TimTam18,
    It seems -f option to be very useful. But I can't find any docs about how to build the preference xml file. Could you give me some suggestion on it?
    Thanks.
  • SystemAdmin
    SystemAdmin
    1959 Posts

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2013-02-06T23:03:17Z  
    Hi TimTam18,
    It seems -f option to be very useful. But I can't find any docs about how to build the preference xml file. Could you give me some suggestion on it?
    Thanks.
    Hi JJBS_Ping_Zhi_Hu,

    Basic information about the Cognos transformer preference xml file can be found here

    http://129.35.224.116/infocenter/cbi/v10r2m0/topic/com.ibm.swg.ba.cognos.ug_cogtr.10.2.0.doc/c_cogtr_xml_file_settings.html?path=5_7_13_6#cogtr_xml_file_settings

    Note that you can also specify the preference for each cube build (as opposed to the global settings) by keeping a seperate version of cogtr.xml and only point to it using the -f option of the command line.

    Hope this helps,
    timtam18
  • SystemAdmin
    SystemAdmin
    1959 Posts

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2013-02-08T04:03:24Z  
    Hi JJBS_Ping_Zhi_Hu,

    Basic information about the Cognos transformer preference xml file can be found here

    http://129.35.224.116/infocenter/cbi/v10r2m0/topic/com.ibm.swg.ba.cognos.ug_cogtr.10.2.0.doc/c_cogtr_xml_file_settings.html?path=5_7_13_6#cogtr_xml_file_settings

    Note that you can also specify the preference for each cube build (as opposed to the global settings) by keeping a seperate version of cogtr.xml and only point to it using the -f option of the command line.

    Hope this helps,
    timtam18
    got it. Thank you, timtam18.
  • AdityaUpadhyay
    AdityaUpadhyay
    1 Post

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2013-05-18T07:43:27Z  
    Hi whatever890 - thanks for this

    MultiFileCubeThreshold helps managing the size of the *.mdp files but there is a point all the data is put into one single mdp file before being allocated to other partition. If the volume of data exceeds 2GB it still fails at this point.

    We've found a workaround by manually patitioning the cube using a level in a dimension where the data is evenly distibuted into its categories.

    Regards -

    Hi ,

    We are facing the same issue, the mdp files are exceeding 2 GB, almost daily we have to change the threshold setting as the data volume is growing  to make the cubes live, how did you do the partition to solve this problem?

  • Glen.S
    Glen.S
    20 Posts

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2013-05-27T15:53:08Z  

    Hi ,

    We are facing the same issue, the mdp files are exceeding 2 GB, almost daily we have to change the threshold setting as the data volume is growing  to make the cubes live, how did you do the partition to solve this problem?

    This limitation in PowerCubes is fairly fundamental, and as you have found, makes life extremely difficult once your data has grown to this size. This sort of thing is why the Dynamic Cube technology was created, and released in Cognos 10.2.