For WP 6.1 (uses Java 5.0), the GC policies are described in Frequently asked questions about the Garbage Collector at http://publib.boulder.ibm.com/infocenter/javasdk/v5r0/topic/com.ibm.java.doc.diagnostics.50/diag/understanding/mm_faq.html. This WP version installs with -Xgcpolicy:gencon.
For WP 6.0 (uses Java 1.4.2), the GC policies are described in Frequently asked questions about the Garbage Collector at http://publib.boulder.ibm.com/infocenter/javasdk/v1r4m2/topic/com.ibm.java.doc.diagnostics.142/html/gcfaquestions.html. This WP version installs with the default GC policy (optthruput).
You can try testing with -Xgcpolicy:optavgpause. However, if the OOM problem is occurring because the Java heap is exhausted, then that will still happen no matter which GC policy you use.
Before you test, be sure to install the latest Java SDK cumulative fix available at Recommended fixes for WebSphere Application Server at http://www.ibm.com/support/docview.wss?rs=180&uid=swg27004980.
WebSphere Portal support
The postings on this site are my own and do not necessarily represent the positions, strategies, or opinions of IBM.