Topic
  • No replies
ipapadop
ipapadop
4 Posts

Pinned topic XLC C++ large compilation times when using -O2 and up

‏2009-10-11T02:35:46Z |
Hi,

I have some really heavy templated code. I have some test cases which they all compile fine using

<table class="jive-wiki-table">
<tr>
<td>-qenum=small -qstrict -q64 -qrtti=all -qtemplatedepth=5000 -qarch=auto -qtune=auto -O0 -qmaxmem=-1</td>
</tr>
</table>

However when I set -O2 or higher either my times skyrocket or it would take longer than 4 hours (after that I just stop it).

My first test needs 1min in -O0 and 33min in O2. My other test case needs 1.45 mins (-O0) and 102 mins (-O3). The last test needs 2.3 mins (-O0) and after 4 hours didn't finish in -O2.

I didn't see any difference by changing qtemplatedepth (the default value of 300 is not enough), by changing qstrict or qarch/qtune. If I lower the qmaxmem let's say to 8192, the tests don't even compile.

Do you have any suggestions to lower compilation times? Are there any options/tools to check what part of the code is taking up so much time to compile?

Thank you
Updated on 2010-09-22T21:10:51Z at 2010-09-22T21:10:51Z by SystemAdmin
  • jrheath
    jrheath
    3 Posts

    Re: XLC C++ large compilation times when using -O2 and up

    ‏2009-10-13T14:28:43Z  

    Hello,
    Can you try compiling with the -qphsinfo option at both -O0 and -O2? This should help identify which phase of compilation is causing compilation time to increase. Also note that setting higher optimization levels will always cause compilation times to increase, though moving up to -O2 from -O0 doesn't usually cause as dramatic a difference as you are seeing.
    Also, which version of the XL C++ compiler are you using?
  • SystemAdmin
    SystemAdmin
    196 Posts

    Re: XLC C++ large compilation times when using -O2 and up

    ‏2009-11-24T03:58:42Z  
    • jrheath
    • ‏2009-10-13T14:28:43Z

    Hello,
    Can you try compiling with the -qphsinfo option at both -O0 and -O2? This should help identify which phase of compilation is causing compilation time to increase. Also note that setting higher optimization levels will always cause compilation times to increase, though moving up to -O2 from -O0 doesn't usually cause as dramatic a difference as you are seeing.
    Also, which version of the XL C++ compiler are you using?

    I suspect the problem is that the templated code is producing huge files, each of which takes a lot of time to compile. At low optimizations, many of the phases in the compiler are O(n) in their complexity. That is, they are "fast and dumb", producing code quickly, at the cost of lost opportunities to improve the code. At higher levels of optimization, such as O2, the phases in the compiler can be O(n^2). That is, the time taken to process the input increases non-linearly with the size of the files. A file twice as big can take four times as long to compile, and so on.
    There is a solution you could try to see if my suspicion is correct: having the compiler generate many small files instead of a few large ones, to contain the template instantiations. The compiler flags of interest are: -qtempincand -qtempmax.
    Try something like "-qtempinc -qtempmax=500" and see if your compilation times improve.
  • camorton
    camorton
    10 Posts

    Re: XLC C++ large compilation times when using -O2 and up

    ‏2010-01-07T14:27:09Z  

    I suspect the problem is that the templated code is producing huge files, each of which takes a lot of time to compile. At low optimizations, many of the phases in the compiler are O(n) in their complexity. That is, they are "fast and dumb", producing code quickly, at the cost of lost opportunities to improve the code. At higher levels of optimization, such as O2, the phases in the compiler can be O(n^2). That is, the time taken to process the input increases non-linearly with the size of the files. A file twice as big can take four times as long to compile, and so on.
    There is a solution you could try to see if my suspicion is correct: having the compiler generate many small files instead of a few large ones, to contain the template instantiations. The compiler flags of interest are: -qtempincand -qtempmax.
    Try something like "-qtempinc -qtempmax=500" and see if your compilation times improve.

    I would not recommend the use of -qtempinc as this require a complete reorganization of the source code. This is an old option that is kept around for legacy reasons. Please use -qtemplateregistry instead.

    -qtemplateregistry


    http://publib.boulder.ibm.com/infocenter/comphelp/v8v101/topic/com.ibm.xlcpp8a.doc/compiler/ref/ngcpp.gif

    Description


    Maintains records of all templates as they are encountered in the source
    and ensures that only one instantiation of each template is made.

  • ipapadop
    ipapadop
    4 Posts

    Re: XLC C++ large compilation times when using -O2 and up

    ‏2010-08-30T15:08:08Z  
    • jrheath
    • ‏2009-10-13T14:28:43Z

    Hello,
    Can you try compiling with the -qphsinfo option at both -O0 and -O2? This should help identify which phase of compilation is causing compilation time to increase. Also note that setting higher optimization levels will always cause compilation times to increase, though moving up to -O2 from -O0 doesn't usually cause as dramatic a difference as you are seeing.
    Also, which version of the XL C++ compiler are you using?
    I am using version 10.1.

    I am reorganizing the code to eliminate some template instantiations to make it faster.
  • SystemAdmin
    SystemAdmin
    196 Posts

    Re: XLC C++ large compilation times when using -O2 and up

    ‏2010-09-22T21:10:51Z  
    • ipapadop
    • ‏2010-08-30T15:08:08Z
    I am using version 10.1.

    I am reorganizing the code to eliminate some template instantiations to make it faster.
    Hello. Would it be possible for you to make compilable code samples available for further examination? We may be able to provide more accurate suggestions after looking at some sample code.