Examples

Specialized toolkits - release 4.3.1.0-prod20190605 > com.ibm.streamsx.hdfs 4.4.1 > com.ibm.streamsx.hdfs > HDFS2DirectoryScan > Examples

This example uses the HDFS2DirectoryScan operator to scan the HDFS directory On IBM Cloud. The hdfsUser and hdfsPassword parameters are used to provide the username and password for authentication.

(stream<rstring filename> Files) as HDFS2DirectoryScan_1 = HDFS2DirectoryScan()
{
	param
		directory :"/user/myuser/";
		hdfsUri   :"webhdfs://hdfsServer:8443";
		hdfsPassword: "password";
		hdfsUser: "biuser";
		sleepTime : 2.0;
}

This example uses the HDFS2DirectoryScan operator to scan a HDFS directory every two seconds. The hdfsUri parameter in this case overrides the value that is specified by the fs.defaultFS option in the core-site.xml.

(stream<rstring filename> Files) as HDFS2DirectoryScan_1 = HDFS2DirectoryScan()
{
	param
		directory :"/user/myuser/";
		hdfsUri   :"hdfs://hdfsServer:1883";
		sleepTime : 2.0;
}