wm.hadoop.hdfs:write

The wm.hadoop.hdfs:write service connects to the HDFS cluster and writes to a specified file. To ensure that the contents of the file are not updated by other users or services at the time of the write operation, the service changes the file extension to that of a temporary file (.tmp), writes to the temporary file, and reverts the extension to the original value after the write operation is complete. The rollover logic renames the .tmp file to have the following filename conventions if the rollover is based on the file size or time duration: 

{Filename}+{_hostname}+{_currentdatetime}.{format}

The wm.hadoop.hdfs:write service returns the status of the write operation.

Input Parameters

handle String Handle of the HDFS file to which the wm.hadoop.hdfs:write service writes data.
writeAs Format of the HDFS file to which the wm.hadoop.hdfs:write service writes data.
data String Input source to write to HDFS. Set to:
  • bytes to write a byte array.
  • stream to write a java.io.InputStream object.
  • string to write a java.io.String object.
  • localFilePath to write a java.io.localFilePath string.
  • reader to write a a java.io.Reader object.
Note: If you specify multiple input sources, the service considers the first of the specified sources for writing to the HDFS file.
writeOptions Document Details to specify for the write operation.
Option Description
encoding String Optional. Name of a registered, IANA character set. If you specify an unsupported encoding, the system throws an exception.
permissions String Optional. File permissions in either octal or symbolic format.
execution Document Optional. Details to specify for executing the wm.hadoop.hdfs:write service.
Option Description
async Boolean Optional. Flag that indicates whether or not the service is to be executed asynchronously.
Set to:
  • true to execute the service asynchronously.
  • false to execute the service synchronously.
callbackServiceName String Optional. Fully qualified name of the service to call after the wm.hadoop.hdfs:write service is executed asynchronously.

Specify this value only if the async option is set to true.

Output Parameters

status String The status of the write operation. The valid value of this parameter can either be success to indicate success of the write operation, failure to indicate failure of the write operation, or submitted to indicate that the write request is submitted to HDFS but has not yet started.
message String A detailed message specifying the reason for the success or failure of the write operation.