![Close [x]](http://dw1.s81c.com/i/c.gif){kind=small}
Tab navigation
- Overview- selected tab,
- Requirements
- Downloads
- FAQs
Page navigation
Unstructured information management (UIM) applications are software systems that analyze unstructured information (text, audio, video, images, and so on) to discover, organize, and deliver relevant knowledge to the user. In analyzing unstructured information, UIM applications make use of a variety of analysis technologies, including statistical and rule-based Natural Language Processing (NLP), Information Retrieval (IR), machine learning, and ontologies. IBM's Unstructured Information Management Architecture (UIMA) is an architectural and software framework that supports creation, discovery, composition, and deployment of a broad range of analysis capabilities and the linking of them to structured information services, such as databases or search engines. The UIMA framework provides a run-time environment in which developers can plug in and run their UIMA component implementations, along with other independently-developed components, and with which they can build and deploy UIM applications. The framework is not specific to any IDE or platform.
This technology, the UIMA SDK (Software Development Kit), is an all-JavaTM implementation of the UIMA framework, and it supports the implementation, description, composition, and deployment of UIMA components and applications. It also supports the developer with an Eclipse-based development environment that includes a set of tools and utilities for using UIMA.
One large, but not the only, application area of text analysis is improving text search. By detecting important terms and topics within documents, semantic search engines provide the capability to search for concepts and relationships instead of keywords. IBM's enterprise search solutions, IBM OmniFind Enterprise Edition or IBM DB2 Warehouse Edition, have such semantic search capabilities. They allows UIMA annotators to be plugged into the OmniFind or DB2 Warehouse processing flow, enabling semantic search to be performed on the extracted concepts. Another large application area is information extraction. The text-analysis functions of IBM DB2 Warehouse Edition focus on information extraction that creates structured data out of unstructured data. DB2 Warehouse Edition allows UIMA annotators to be plugged into a Mining flow, enabling the extraction of information that can then be analyzed together with structured information by using business intelligence tools. Since UIMA is used and developed both by IBM research and development teams, there are two locations of the UIMA SDK:
- The UIMA SDK on alphaWorks is the "early adopter" version of the SDK. It is intended for users who don't use OmniFind or DB2 Warehouse, or who want to use features of UIMA that may not be supported by OmniFind or DB2 Warehouse. The alphaWorks SDK is also a test bed to gather feedback on new features of the UIMA SDK. Its versions may evolve more rapidly, and are not tied to specific OmniFind or DB2 Warehouse releases. The SDK is supported on a "best can do" basis, by way of the alphaWorks forum. The Java source code for core components of the alphaWorks SDK is available at SourceForge.
- The UIMA SDK on developerWorks is the "OmniFind-compatible" and "DB2 Warehouse-compatible" version of the SDK. It is intended for users who want to develop and deploy semantic search solutions with IBM OmniFind Enterprise Edition or solutions that take advantage of OmniFind's capabilities for enterprise-scale document crawling and extraction. It is also intended for users who want to develop and deploy text-analysis projects with IBM DB2 Warehouse Edition. The developerWorks SDK is tested for compatibility with a specific OmniFind and and DB2 Warehouse version and will be updated to keep in sync with new OmniFind and and DB2 Warehouse releases. As the SDK evolves, prior versions will still be available on developerWorks, to ensure that each supported OmniFind and and DB2 Warehouse version has a corresponding SDK. For customers who have an OmniFind or and DB2 Warehouse license, this SDK is supported by way of the IBM support channels and also through the developerWorks forum.
How does it work?
UIMA is an architecture in which basic building blocks called Analysis Engines (AEs) are composed in order to analyze a document. At the heart of AEs are the analysis algorithms that do all the work to analyze documents and record analysis results (for example, detecting person names). These algorithms are packaged within components that are called Annotators. AEs are the stackable containers for annotators and other analysis engines.
How Annotators represent and share their results is an important part of the UIMA architecture. To enable composition and reuse, UIMA defines a Common Analysis Structure (CAS) precisely for these purposes. The CAS is an object-based container that manages and stores typed objects having properties and values. Object types may be related to each other in a single-inheritance hierarchy. Annotators are given a CAS having the subject of analysis (the document), in addition to any previously created objects (from annotators earlier in the pipeline), and they add their own objects to the CAS. The CAS serves as a common data object, shared among the annotators that are assembled for an application.
Many UIM applications analyze entire collections of documents. UIMA supports this analysis through its Collection Processing Architecture. This part of the architecture allows specification of a "source-to-sink" flow from a collection reader though a set of analysis engines and then to a set of CAS Consumers. The collection reader's job is to connect to and iterate through a source collection, acquiring documents and initializing CASes for analysis. After the analysis engines have added their information to the CAS, CAS consumers do the final CAS processing, for example, sending the CAS contents to a search engine or extracting elements of interest and populating a relational database. A Semantic Search engine is included in the UIMA SDK; it will allow the developer to experiment with indexing analysis results, which will enable semantic searches using the the annotations in the CAS.
Learn more about it:
- Read the article "Semantic search in WebSphere Information Integrator OmniFind Edition: The case for semantic search".
- Take the tutorial Semantic search in WebSphere Information Integrator OmniFind Edition: Deploy a semantic search solution.
- Take the tutorial Semantic search in IBM OmniFind Enterprise Edition, Part 2: Semantic search with UIMA and OmniFind.
- Read the article "Hints and tips for using the Unstructured Information Management Architecture SDK with OmniFind".
- Read the article "Leverage taxonomies for enterprise search using IBM OmniFind, IBM Classification Module, and SchemaLogic".
- Read the UIMA SDK User's Guide.
- Read the UIMA SDK Readme document.
Standards and open source
IBM has started an OASIS working group to create an open standard for UIMA applications. The purpose of this working group is the creation of standards to ensure interoperability between different UIM applications and thus create an open ecosystem of unstructured analysis platforms and applications. Participation in the working group is open to all OASIS members. Concurrently, IBM has donated the source code to the Apache Software Foundation, and Apache has accepted UIMA as an Incubator project. All new UIMA development will happen on Apache, and new Apache UIMA releases are available there. It will be some time before the first release will be available from Apache. We will continue to support previous versions of UIMA through developerWorks.
What's new in UIMA release 1.4?
- XMI support has been added. There are two new chapters in the user's guide describing this support. As a part of this change, additional type system feature description information for types which are arrays or lists can now be specified, including the type of the elements of these collections.
- A new utility to merge two or more PEAR files has been added, and is described in the user's guide.
- Please see the release notes for details on other enhancements and bug fixes.
About the technology authors
The UIMA SDK is being developed by teams from IBM Research and IBM Software Group. It is a world-wide effort, with significant participation from the following IBM sites:
- IBM Thomas J. Watson Research Center (New York)
- IBM Haifa Research Laboratory (Israel)
- IBM Development Laboratory Boeblingen (Germany)
- IBM Almaden Research Center (California)