IBM Support

RedHat 6 Linux Huge Page option increases runtime memory usage of InfoSphere DataStage Parallel Jobs

Troubleshooting


Problem

InfoSphere Information Server customers upgrade their installations to RedHat Linux 6 and later notice that Parallel Engine jobs are using more memory. The increased usage has been most visible for InfoSphere Information Services Director workloads but it applies to all Parallel Engine jobs.

Symptom

The increased memory usage is caused by the Linux kernel ‘Huge Page’ feature. This feature has been in Linux for years, but it was normally not enabled. Starting with Red Hat Enterprise Linux 6, the Huge Page feature is enabled by default ("Transparent Huge Pages") and customers who upgrade to Red Hat 6 are likely to see a 30% or more increase in memory usage during parallel job execution. The Huge Page feature is intended to improve performance on systems with very large amounts of RAM, but memory usage will increase unless the Linux kernel is properly tuned.

[{"Product":{"code":"SSVSEF","label":"IBM InfoSphere DataStage"},"Business Unit":{"code":"BU048","label":"IBM Software"},"Component":"Not Applicable","Platform":[{"code":"PF016","label":"Linux"}],"Version":"9.1;8.7;8.5;8.1","Edition":"","Line of Business":{"code":"LOB76","label":"Data Platform"}}]

Log InLog in to view more of this document

This document has the abstract of a technical article that is available to authorized users once you have logged on. Please use Log in button above to access the full document. After log in, if you do not have the right authorization for this document, there will be instructions on what to do next.

Document Information

Modified date:
25 April 2025

UID

swg21664196

Page Feedback