Topic
10 replies Latest Post - ‏2013-05-27T15:53:08Z by Glen.S
SystemAdmin
SystemAdmin
1959 Posts
ACCEPTED ANSWER

Pinned topic Cognos Transformer: file size exceeds 2GB

‏2012-12-03T04:49:28Z |
Dear Cognos gurus,

It's an old problem coming back.. Cognos Transformer 10.1.1 encounters this error:
(TR1901) PDS-PPE-0197 The local data cache size exceeded the allowed maximum of 2147418112 bytes.

Partition size has been changed from 1000000 to 500000 then 200000 - and I can see the cube build log reflecting this value (as shown below), however there will be a point where a certain *.mdp file exceeds 2GB, resulting in the same problem.

Anyone has an effective solution to this problems?

Many thanks,
timtam18

Mon 03 Dec 2012 14:21:59 4 00077D39 --- Performing Pass 2 with 23361408 rows and 611852 categories remaining.
Mon 03 Dec 2012 14:21:59 4 00077D39 Start Write leaving 611852 categories remaining.
Mon 03 Dec 2012 14:21:59 4 0039C207 Updating the PowerCube data.
Mon 03 Dec 2012 14:21:59 4 0039C207 Updating the PowerCube data.
Mon 03 Dec 2012 14:22:01 4 0039C207 Performing DataBase Commit at record number 250000. << reflecting the value mentioned above
Mon 03 Dec 2012 14:22:04 4 0039C207 Performing DataBase Commit at record number 500000.
Updated on 2013-02-08T04:03:24Z at 2013-02-08T04:03:24Z by SystemAdmin
  • whatever890
    whatever890
    6 Posts
    ACCEPTED ANSWER

    Re: Cognos Transformer: file size exceeds 2GB

    ‏2012-12-06T14:44:08Z  in response to SystemAdmin
    Try changing the MultiFileCubeThreshold in the cogtr.xml file to something like 500000. If this value isn't in the cogtr.xml file then I believe you can add it.
    • SystemAdmin
      SystemAdmin
      1959 Posts
      ACCEPTED ANSWER

      Re: Cognos Transformer: file size exceeds 2GB

      ‏2012-12-07T00:24:09Z  in response to whatever890
      Hi whatever890 - thanks for this

      MultiFileCubeThreshold helps managing the size of the *.mdp files but there is a point all the data is put into one single mdp file before being allocated to other partition. If the volume of data exceeds 2GB it still fails at this point.

      We've found a workaround by manually patitioning the cube using a level in a dimension where the data is evenly distibuted into its categories.

      Regards -
      • SteveW.
        SteveW.
        56 Posts
        ACCEPTED ANSWER

        Re: Cognos Transformer: file size exceeds 2GB

        ‏2012-12-10T16:43:28Z  in response to SystemAdmin
        What kinds of performance are you getting by doing this?
        • SystemAdmin
          SystemAdmin
          1959 Posts
          ACCEPTED ANSWER

          Re: Cognos Transformer: file size exceeds 2GB

          ‏2013-01-03T14:11:25Z  in response to SteveW.
          Please try to user Preference file while building the cube because by touching the XML file the change will be reflecting for every Cube build.

          Thanks & Regards,
          Vasudev Chavan
          • SystemAdmin
            SystemAdmin
            1959 Posts
            ACCEPTED ANSWER

            Re: Cognos Transformer: file size exceeds 2GB

            ‏2013-01-10T01:39:38Z  in response to SystemAdmin
            Hi Vasudev Chavan - yes this is correct thanks. We use the -f option and place these preference into the preference xml file instead of changing the global value of the whole Transformer environment.
            We've also found a more sustainable solution using time-based partitioned cube.
            Thanks,
            • SystemAdmin
              SystemAdmin
              1959 Posts
              ACCEPTED ANSWER

              Re: Cognos Transformer: file size exceeds 2GB

              ‏2013-02-06T02:00:30Z  in response to SystemAdmin
              Hi TimTam18,
              It seems -f option to be very useful. But I can't find any docs about how to build the preference xml file. Could you give me some suggestion on it?
              Thanks.
              • SystemAdmin
                SystemAdmin
                1959 Posts
                ACCEPTED ANSWER

                Re: Cognos Transformer: file size exceeds 2GB

                ‏2013-02-06T23:03:17Z  in response to SystemAdmin
                Hi JJBS_Ping_Zhi_Hu,

                Basic information about the Cognos transformer preference xml file can be found here

                http://129.35.224.116/infocenter/cbi/v10r2m0/topic/com.ibm.swg.ba.cognos.ug_cogtr.10.2.0.doc/c_cogtr_xml_file_settings.html?path=5_7_13_6#cogtr_xml_file_settings

                Note that you can also specify the preference for each cube build (as opposed to the global settings) by keeping a seperate version of cogtr.xml and only point to it using the -f option of the command line.

                Hope this helps,
                timtam18
      • AdityaUpadhyay
        AdityaUpadhyay
        1 Post
        ACCEPTED ANSWER

        Re: Cognos Transformer: file size exceeds 2GB

        ‏2013-05-18T07:43:27Z  in response to SystemAdmin

        Hi ,

        We are facing the same issue, the mdp files are exceeding 2 GB, almost daily we have to change the threshold setting as the data volume is growing  to make the cubes live, how did you do the partition to solve this problem?

        • Glen.S
          Glen.S
          20 Posts
          ACCEPTED ANSWER

          Re: Cognos Transformer: file size exceeds 2GB

          ‏2013-05-27T15:53:08Z  in response to AdityaUpadhyay

          This limitation in PowerCubes is fairly fundamental, and as you have found, makes life extremely difficult once your data has grown to this size. This sort of thing is why the Dynamic Cube technology was created, and released in Cognos 10.2.