The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.
All information submitted is secure.
The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerworks.
Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.
This series explores how to process unstructured data in parallel fashion —
within a machine and across a series of machines — using the power of IBM DB2 for Linux, UNIX and Windows (LUW) and GPFS shared-nothing cluster (SNC) to provide efficient, scalable access to unstructured data through a standard SQL interface.
In this article, see how the Java-based sashyReader framework
leverages the architectural features in DB2 LUW. The
sashyReader provides for parallel and scalable processing of unstructured data
stored locally or on a cloud via an SQL interface. This is useful for data
ingest, data cleansing, data aggregation, and other tasks requiring the scanning,
processing, and aggregation of large unstructured data sets. You also
learn how to extend the sashyReader framework to read arbitrary unstructured
text data by using dynamically pluggable Python classes.
See how unstructured data can be processed in parallel fashion. Leverage the power
of IBM DB2 for Linux, UNIX and Windows to provide efficient highly scalable access to
unstructured data stored on the cloud.
Learn how unstructured data can be processed in
parallel fashion -- within a machine and across a series of
machines -- by leveraging DB2 Linux, UNIX, and Windows
and GPFS SNC to provide efficient highly scalable access to unstructured
data, all through a standard SQL interface. Realize this capability with clusters
of commodity hardware, suitable for provisioning in the cloud or directly on
bare metal clusters of commodity hardware. Scalability is achieved within the
framework via the principle of computation locality. Computation is
performed local to the host which has direct data access, thus minimizing or
eliminating network bandwidth requirements and eliminating the need for any
shared compute resource.