JR54950: PX ON HADOOP YARN Q4 2015 ROLLUP FOR IS 11.5

APAR status

• Closed as program error.

Error description

• This APAR is a rollup of fixes and enhancements in the PX on
  Hadoop functionality in Information Server 11.5.
  Some of the code changes are based on internal development and
  testing and others are based on feedback from Beta and
  General Release customers.
  The changes in this APAR include
  APARs:
  JR54570 Binary localization location should be configurable.
  Currently it is hardcoded to /tmp/px_yarn_binaries
  JR54792 Stopping jobmon on AIX invokes the stop-pxyarn script
  JR54389 Binaries are not getting copied after installation of
  the patch PXE_115Q4PXCC_3.151130.ispkg and
  CC_115Q4PXCC_3.151130.ispkg
  JR54763 Very large jobs fail with Internal Error: (pfdesc !=
  NULL): util/syscall_hdfs.C: 168
  
  Other Changes:
  copy-orchdist can fail to tar all files if there is a core dump
  file or other unreadable file in any of the binary directories
  copy-orchdist has 'permission denied' messages in the log
  File Connector: Implement ORC file type support
  File Connector: Implement support for reading/writing Avro
  File Connector: requiring keytab file in every node when running
  in kerberos
  Hadoop YARN: Support submitting jobs as the datastage user that
  is running the job
  Issues with JDBC/ODBC requiring keytab or credential cache file
  in every node
  PX Application Master outputs the same failure message when
  binding to default port 55555 whether the bind succeeds or fails
  PX Application Master process allocates up to 18GB of virtual
  memory on some systems
  start-pxyarn.sh reports that the yarn client started
  successfully when it doesn't
  start-pxyarn.sh without Hadoop services running the script
  reports that the client started successfully
  The ports used by the PX YARN Client and AMs should be
  configurable
  

Local fix

• Not Applicable
  

Problem summary

• ERROR DESCRIPTION:
  This APAR is a rollup of fixes and enhancements in the PX on
  Hadoop functionality in Information Server 11.5.
  Some of the code changes are based on internal development and
  testing and others are based on feedback from Beta and
  General Release customers.
  The changes in this APAR include
  APARs:
  JR54570 Binary localization location should be configurable.
  Currently it is hardcoded to /tmp/px_yarn_binaries
  JR54792 Stopping jobmon on AIX invokes the stop-pxyarn script
  JR54389 Binaries are not getting copied after installation of
  the patch PXE_115Q4PXCC_3.151130.ispkg and
  CC_115Q4PXCC_3.151130.ispkg
  JR54763 Very large jobs fail with Internal Error: (pfdesc !=
  NULL): util/syscall_hdfs.C: 168
  
  Other Changes:
  copy-orchdist can fail to tar all files if there is a core dump
  file or other unreadable file in any of the binary directories
  copy-orchdist has 'permission denied' messages in the log
  File Connector: Implement ORC file type support
  File Connector: Implement support for reading/writing Avro
  File Connector: requiring keytab file in every node when running
  in kerberos
  Hadoop YARN: Support submitting jobs as the datastage user that
  is running the job
  Issues with JDBC/ODBC requiring keytab or credential cache file
  in every node
  PX Application Master outputs the same failure message when
  binding to default port 55555 whether the bind succeeds or fails
  PX Application Master process allocates up to 18GB of virtual
  memory on some systems
  start-pxyarn.sh reports that the yarn client started
  successfully when it doesn't
  start-pxyarn.sh without Hadoop services running the script
  reports that the client started successfully
  The ports used by the PX YARN Client and AMs should be
  configurable
  
  LOCAL FIX:
  Not Applicabable
  
  There is a newer rollup patch for Hadoop issues. See JR55416
  

Problem conclusion

• ERROR DESCRIPTION:
  This APAR is a rollup of fixes and enhancements in the PX on
  Hadoop functionality in Information Server 11.5.
  Some of the code changes are based on internal development and
  testing and others are based on feedback from Beta and
  General Release customers.
  The changes in this APAR include
  APARs:
  JR54570 Binary localization location should be configurable.
  Currently it is hardcoded to /tmp/px_yarn_binaries
  JR54792 Stopping jobmon on AIX invokes the stop-pxyarn script
  JR54389 Binaries are not getting copied after installation of
  the patch PXE_115Q4PXCC_3.151130.ispkg and
  CC_115Q4PXCC_3.151130.ispkg
  JR54763 Very large jobs fail with Internal Error: (pfdesc !=
  NULL): util/syscall_hdfs.C: 168
  
  Other Changes:
  copy-orchdist can fail to tar all files if there is a core dump
  file or other unreadable file in any of the binary directories
  copy-orchdist has 'permission denied' messages in the log
  File Connector: Implement ORC file type support
  File Connector: Implement support for reading/writing Avro
  File Connector: requiring keytab file in every node when running
  in kerberos
  Hadoop YARN: Support submitting jobs as the datastage user that
  is running the job
  Issues with JDBC/ODBC requiring keytab or credential cache file
  in every node
  PX Application Master outputs the same failure message when
  binding to default port 55555 whether the bind succeeds or fails
  PX Application Master process allocates up to 18GB of virtual
  memory on some systems
  start-pxyarn.sh reports that the yarn client started
  successfully when it doesn't
  start-pxyarn.sh without Hadoop services running the script
  reports that the client started successfully
  The ports used by the PX YARN Client and AMs should be
  configurable
  
  LOCAL FIX:
  Not Applicable
  
  5724Q36DS
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WIS DATASTAGE

• Fixed component ID

  5724Q36DS

Applicable component levels

• RB50 PSY

     UP