The bigdata [1.0.2] toolkit operators HDFSFileSink/HDFSFileSource don't support csv format, so it seems like any time I need to read from/write to HDFS using csv format I need to use a Format and a Parse to convert a line (string) to a blob to parse it. It's not terrible, but it seems like there should be a more straightforward approach (especially compared to the non-HDFS FileSink/FileSource, which handle the CSV parsing/formatting natively).
Am I missing something?
Do you know if there are any plans in the future ot include CSV handling to the bigdata toolkit? CSV seems to be a very common format for biginsights data, so it seems like ti would be a natural development.
Thanks in advance for any pointers or suggestions on better ways to handle this, or any information on future direction for the toolkit.