SAP BO Integration Requirements

The following are the prerequisites necessary for IBM Automatic Data Lineage to connect to this third-party system, which you may choose to do at your sole discretion. Note that while these are usually sufficient to connect to this third-party system, we cannot guarantee that the connection or integration will be successful since we have no control, liability, or responsibility for third-party products or services, including for their performance.

Requirements for Automatic Data Lineage

Up to and including R42.13: The scanner supports SAP BusinessObjects BI 4.3 support pack 1 and 2. As of R42.14: The scanner supports the SAP BusinessObjects BI 4.3 support pack 4.
Up to and including R42.13: Client Tools 4.3 with Semantic Layer Java SDK installed. Without Semantic Layer Java SDK, the extraction won’t succeed. The extraction process will download only a fraction of the required data, and the visualization will be incomplete. As of R42.14: The Windows version of the SAP BO Server (Platform) with Semantic Layer Java SDK installed can be used instead of the Client Tools as well.
- Client Tools can only be installed in Windows. Linux is not supported.
- Make sure this option is checked during Client Tools installation, as shown in the image below.
- Required JAR files installed with Semantic Layer Java SDK:
  - In the folder ${SAP_BO_INSTALLATION_FOLDER}/SL SDK/eclipse/plugins:
    
    Up to and including R42.13:
    - com.sap.sl.sdk.authoring.jar
    - com.sap.sl.sdk.framework.jar
    - org.eclipse.emf.common_2.15.0.v20180914-1817.jar
    - org.eclipse.emf.ecore.xmi_2.15.0.v20180706-1146.jar
    - org.eclipse.emf.ecore_2.16.0.v20181124-0637.jar
    - org.eclipse.equinox.registry_3.8.200.v20181008-1820.jar
    - org.eclipse.osgi_3.13.200.v20181130-2106.jar
    - org.eclipse.equinox.common_3.10.200.v20181021-1645.jar
    As of R42.14:
    - org.xmlpull_1.1.4.C.jar
    - com.businessobjects.bcm/lib/bcm.jar
    - com.businessobjects.boesdk/lib/ceaspect.jar
    - com.businessobjects.boesdk/lib/cecore.jar
    - com.businessobjects.boesdk/lib/celib.jar
    - com.businessobjects.boesdk/lib/ceplugins_core.jar
    - com.businessobjects.boesdk/lib/cesearch.jar
    - com.businessobjects.boesdk/lib/cesession.jar
    - com.businessobjects.boesdk/lib/coarsegrain.jar
    - com.businessobjects.boesdk/lib/corbaidl.jar
    - com.businessobjects.boesdk/lib/sapjce.jar
    - com.businessobjects.boesdk/lib/secSAPR3.jar
    - com.businessobjects.connectionserver.jar
    - com.businessobjects.corba/lib/ebus405.jar
    - com.businessobjects.corba/lib/freessl201.jar
    - com.businessobjects.datafederator.nativeconnection.jar
    - com.businessobjects.datafederator.sourcedefiner.jar
    - com.businessobjects.deploy.jar
    - com.businessobjects.dsl.commons.jar
    - com.businessobjects.dsl.commons.resources.jar
    - com.businessobjects.dsl.framework.jar
    - com.businessobjects.dsl.services.impl.jar
    - com.businessobjects.dsl.services.repository.cesdk.jar
    - com.businessobjects.foundation.logging.jar
    - com.businessobjects.i18n.jar
    - com.businessobjects.i18n.monofmtlib_jni.jar
    - com.businessobjects.mds.businesssecurityaccess.jar
    - com.businessobjects.mds.connection.jar
    - com.businessobjects.mds.consumption.jar
    - com.businessobjects.mds.datafoundation.jar
    - com.businessobjects.mds.datafoundationsecurityaccess.jar
    - com.businessobjects.mds.entity.jar
    - com.businessobjects.mds.integrity.rules.jar
    - com.businessobjects.mds.olap.jar
    - com.businessobjects.mds.olap.protocol.bics.jar
    - com.businessobjects.mds.olap.protocol.newdb.jar
    - com.businessobjects.mds.olap.protocol.sap.jco.jar
    - com.businessobjects.mds.parameter.jar
    - com.businessobjects.mds.publishing.jar
    - com.businessobjects.mds.repository.cms.jar
    - com.businessobjects.mds.repository.filesystem.jar
    - com.businessobjects.mds.repository.jar
    - com.businessobjects.mds.repository.metadata.jar
    - com.businessobjects.mds.repository.model.jar
    - com.businessobjects.mds.resource.jar
    - com.businessobjects.mds.sdk.plugin.desktop.jar
    - com.businessobjects.mds.securedconnection.cms.jar
    - com.businessobjects.mds.securedconnection.jar
    - com.businessobjects.mds.security.cms.jar
    - com.businessobjects.mds.security.jar
    - com.businessobjects.mds.services.datafederator.jar
    - com.businessobjects.mds.services.jar
    - com.businessobjects.mds.services.platform.jar
    - com.businessobjects.mds.sets.jar
    - com.businessobjects.mds.toolkit.jar
    - com.businessobjects.mds.universe.jar
    - com.businessobjects.plugins.slplugins/lib/SL_plugins.jar
    - com.businessobjects.sdk.core.jar
    - com.businessobjects.sdk.core.server.common.jar
    - com.businessobjects.sdk.core.server.corba.jar
    - com.businessobjects.sdk.core.server.http.jar
    - com.businessobjects.sdk.core.server.jar
    - com.businessobjects.sdk.core.server.ws.jar
    - com.businessobjects.sdk.core.session.cms.jar
    - com.businessobjects.sdk.core.session.http.jar
    - com.businessobjects.sdk.core.session.jar
    - com.businessobjects.semanticlayer.qt.jar
    - com.businessobjects.semanticlayer.qt.tools.jar
    - com.businessobjects.tp.antlr/lib/antlr-3.5.2-runtime.jar
    - com.businessobjects.universe.migration.jar
    - com.businessobjects.xi.sdk.jar
    - com.ibm.icu_63.1.0.v20181030-1705.jar
    - com.sap.bip.jcomanager.jar
    - com.sap.checkintegrity.core.jar
    - com.sap.connectivity.cis.jar
    - com.sap.connectivity.foundation.jar
    - com.sap.datafederator.consumption.jar
    - com.sap.datafederator.designer.model.jar
    - com.sap.datafederator.designer.resource.jar
    - com.sap.datafederator.embedded.bundle.jar
    - com.sap.datafederator.publisher.jar
    - com.sap.datafederator.rewriterfacade.jar
    - com.sap.js.passport.api.jar
    - com.sap.sl.core.emf.jar
    - com.sap.sl.edp.hana.jar
    - com.sap.sl.edp.relational.jar
    - com.sap.sl.parameter.formula.jar
    - com.sap.sl.qs.converter.jar
    - com.sap.sl.sdk.authoring.jar
    - com.sap.sl.sdk.framework.jar
    - com.sap.sl.sdk.jar
    - com.sap.sl.tools.jar
    - com.sap.tools.commons.jar
    - com.sap.tracelog.jar
    - com.sun.istack.commons-runtime_3.0.12.jar
    - com.sun.xml.bind.jaxb-core_3.0.2.jar
    - fr.loria.tom.runtime_2.6.0.jar
    - jakarta.activation-api_2.1.1.jar
    - jakarta.xml.bind-api_3.0.1.jar
    - json_20230618.0.0.jar
    - org.apache.commons.codec_1.10.0.v20180409-1845.jar
    - org.apache.commons.logging_1.2.0.v20180409-1502.jar
    - org.apache.felix.gogo.runtime_1.1.0.v20180713-1646.jar
    - org.apache.felix.scr_2.0.14.v20180822-1822.jar
    - org.apache.httpcomponents.client5.httpclient5_1.0.0.jar
    - org.apache.httpcomponents.core5.httpcore5_1.0.0.jar
    - org.apache.logging.log4j.1.2-api_2.17.1.jar
    - org.apache.logging.log4j.api_2.17.1.jar
    - org.apache.logging.log4j.core_2.17.1.jar
    - org.aspectj.runtime_1.6.4.20090106083800/aspectjrt.jar
    - org.glassfish.hk2.osgi-resource-locator_2.5.0.jar
    - org.glassfish.jaxb.runtime_3.0.2.jar
    - org.eclipse.core.contenttype_3.7.200.v20181107-1343.jar
    - org.eclipse.core.jobs_3.10.200.v20180912-1356.jar
    - org.eclipse.core.runtime_3.15.100.v20181107-1343.jar
    - org.eclipse.emf.common_2.15.0.v20180914-1817.jar
    - org.eclipse.emf.ecore.xmi_2.15.0.v20180706-1146.jar
    - org.eclipse.emf.ecore_2.16.0.v20181124-0637.jar
    - org.eclipse.emf_2.8.0.v20180706-1146.jar
    - org.eclipse.equinox.common_3.10.200.v20181021-1645.jar
    - org.eclipse.equinox.preferences_3.7.200.v20180827-1235.jar
    - org.eclipse.equinox.registry_3.8.200.v20181008-1820.jar
    - org.eclipse.osgi_3.13.200.v20181130-2106.jar
  - Up to and including R42.13: in the folder ${SAP_BO_INSTALLATION_FOLDER}/java/lib:
    - i18n4j.jar
    - jcmFIPS.jar
    - cryptojFIPS.jar
    - aspectjrt.jar
    - sdk.core.jar
    - bcm.jar
    - biarengine.jar
    - ccis.jar
    - ceaspect.jar
    - cecore.jar
    - celib.jar
    - ceplugins_core.jar
    - cesession.jar
    - corbaidl.jar
    - ebus405.jar
    - logging.jar
    - TraceLog.jar
    - activation.jar
    - dsl_engine.jar
    - xpp3.jar
    - ConnectionServer.jar
    - sapjce.jar
    R42.14 doesn't require libraries from the ${SAP_BO_INSTALLATION_FOLDER}/java/lib folder.
  - Up to and including R42.13: In the folder ${SAP_BO_INSTALLATION_FOLDER}/java/lib/external:
    - freessl201.jar
    R42.14 doesn't require libraries from the ${SAP_BO_INSTALLATION_FOLDER}/java/lib/external folder.
  - Up to and including R42.13: In the folder ${SAP_BO_INSTALLATION_FOLDER}/java/lib/external/axis2/1.7.9: (The following libraries were used during development. Slightly newer versions might work, but Automatic Data Lineage cannot guarantee it.)
    - axis2-adb-1.7.9.jar
    - axis2-kernel-1.7.9.jar
    - axis2-saaj-1.7.9.jar
    - commons-logging-1.1.1.jar
    - jaxb-api-2.2.6.jar
    R42.14 doesn't require libraries from the${SAP_BO_INSTALLATION_FOLDER}/java/lib/external/axis2/1.7.9 folder.
- Other dependencies:
  - Accessible BO connectivity directory at ${SAPBO_INSTALLATION_PATH}/dataAccess/connectionServer
  - The environment variable Path must contain the absolute path to the ${SAP_BO_INSTALLATION_FOLDER}/win64_x64 folder.
User having the following roles
- Universe designer users with full control permissions for the folders where the documents and universes are stored
- With full control permissions, it is possible to manually remove write access. Be aware that adding only read permissions without setting full control is insufficient.
Server must be accessible via network and running
- Server must support RESTful API (HTTP port must exist and be open)

SAP BO Scanner Overview

The SAP BO scanner extracts the JSON/XML specifications of documents, universes, and data foundations as well as database connection information. It analyzes documents and all the objects that the documents depend on and produces data lineage. If the SAP BO scanner and relevant database scanners are configured properly, Manta Viewer can display lineage starting with databases, flowing through data foundations and universes, right into the data providers or document variables, and finally to the visualization elements called report elements.

Extractor Usage

Limitations of the Extractor

Up to and including R42.13: The extractor is only fully functional with Java 11 distributions or with the Java 17 OpenJDK distribution. If the Oracle distribution of Java 17 is used, the extraction process will download only a fraction of the required data and the visualization will be incomplete.

As of R42.14: Any Java version is compatible.
The Agent instance used for extraction must be installed on the same machine as the SAP BO instance. If the Agent instance isn’t installed on the same machine, the extraction process will download only a fraction of the required data and the visualization will be incomplete.
As of R42.14: If SAP BO SDK and the Agent use incompatible versions of log4j-core and log4j-api libraries, use the following workaround:
1. Locate the log4j-core-<version>.jar and log4j-api-<version>.jar in the <manta installation folder>\cli\scenarios\manta-dataflow-cli\lib folder.
2. Copy the jars into the <manta installation folder>\agent\manta-flow-agent-dir\lib-ext or into <manta installation folder>/manta-flow-agent-dir\lib-ext folder accordingly depending on the type of the agent (default installation or separately installed agent).
3. Restart the agent.
  
  If you don't perform the workaround correctly, you might receive "Unable to run application" error message during connection validation or "Logging system failed to initialize using configuration from 'classpath:log4j2-spring.xml'" error message during extraction scenario.

How to Execute the SAP BO Extraction Scenario

Manta (Agent) is installed on the same machine as the SAP BO instance. See Manta Flow Agent Configuration for Extraction for more details about the agent.
- Before the first ever extraction is performed on the machine with SAP BO installed, the environment variable Path needs to be set. The PATH variable consist of multiple entries. Put a value as the first entry in the PATH variable.
  - Configure the environment variable as follows:
    - Add a new entry with a path to the ${SAP_BO_INSTALLATION_FOLDER}/SAP BusinessObjects Enterprise XI 4.0/win64_x64
      folder; for example, C:\Program Files (x86)\SAP BusinessObjects\SAP BusinessObjects Enterprise XI 4.0\win64_x64.
    - In the search bar located next to the Start button or at the bottom or top of the Start menu, type “Environment Variables“.
    - The search results will show Edit the System Environment Variables or Edit Environment Variables for Your Account. Click on either of these options to open the System Properties window.
    - In the System Properties window, click on the Environment Variables button located in the lower-right corner.
    - In the Environment Variables window, you’ll see two sections: User Variables and System Variables. Select the Path variable under System Variables, and click the Edit button.
    - Click on the OK button in the Edit Environment Variable window, then the OK button in the Environment Variables window, and finally, the OK button in the System Properties window.
  - If the variable isn’t set properly, the log file will contain the error java.lang.UnsatisfiedLinkError: no sapcrypto in java.library.path: ....
  - If the wrong folder, named win32_x86, is provided, the error interface requested not found : csLIB will appear in the log file.
- Create the SAP BO Connection in Manta Admin UI.
  - Fill in the required properties with information about the connection and SAP BO instance. Source System Properties
  - For the SapBo Installation Path (sapbo.installation.path) property, enter the location of the Semantic Layer Java SDK folder.
    - This folder, named SAP BusinessObjects Enterprise XI 4.0, is located in the installation folder of the SAP BO instance.
    - The path must be absolute. For example, the default value is C:\Program Files (x86)\SAP BusinessObjects\SAP BusinessObjects Enterprise XI 4.0.
    - The folder separators must be escaped (duplicated) in the window path; for example, C:\\Program Files (x86)\\SAP BusinessObjects\\SAP BusinessObjects Enterprise XI 4.0.
      - If this requirement isn’t met, the validation and extraction will fail and the logs will contain an error message like Unrecognized character escape 'P' (code 80).
- Run the extraction scenario in Manta Admin UI.
  - The extracted folder, located in <MANTAFLOW_INSTALLATION_DIRECTORY>/cli/temp/sapbo/${SAPBO_EXTRACTOR_ID} after the first successful extraction, contains four top-level folders required for further analysis.
    - Connections
    - Documents
    - Spreadsheets
    - Universes

Supported Features

Note: This is a list of SAP BO features that Automatic Data Lineage supports. There may be features that aren’t explicitly named but are included in the named items, as this list aims to be a high-level overview. Other than that, features that are not listed are primarily considered not supported.

Documents using UNX universes as their data provider
Single-source data foundations
Database connections (JDBC, ODBC)
Almost all report elements with the exception of custom elements
Business objects (dimensions, measures, attributes) and their definitions using pure SQL

Known Unsupported Features

Automatic Data Lineage does not support the following SAP BO features. This list includes all of the features that IBM is aware are unsupported, but it might not be comprehensive.

SAP BusinessObjects BI 4.3 support pack 3 is not supported due to breaking changes in the underlying SAP libraries.
SAP Business Objects BI 4.2 and older.
Data sources of data providers other than UNX universe (including UNV universes, files, etc.)
Multisource-enabled data foundations
@Functions (with the exception of @Aggregate_Aware, which is currently supported in some simple cases)
Filter business objects
Custom report elements
Free-hand SQL statements