IBM Support

Performance troubleshooting the Microsoft SharePoint connector

Troubleshooting


Problem

Collecting and processing from a very large (100,000+ items) SharePoint list might generate a variety of performance-related symptoms.

Symptom

Symptoms include one or more of the following:

  • Documents are not archived in the target repository.
  • Post-processing does not complete.
  • There are entries in the "ERROR" audit log file
    (ibm.ctms.taskrouting.auditlog-ERROR-YYYY-MM-DD.log).
  • The SharePoint connector log file (ibm.ctms.spconnector-YYYY-MM-DD.log) contains errors that might contain indications of SQL deadlocks, SharePoint Object Model API exceptions, or ASP.Net timeouts. Typically there is also a notice of retry attempts:

    An error occurred and the same task has been retried 6 time(s) without success. Last exception was:

Tips:
The default location of the log file is:
C:\Program Files\IBM\ContentCollector\ctms\Log

The default location of the audit log file is:
C:\Program Files\IBM\ContentCollector\ctms\audit

Resolving The Problem

Possible solutions for performance-related problems include the following:

Adjustments to web service configuration

  1. Navigate to the application path for Content Collector (the default path is C:\Program Files\IBM\ContentCollector\ctms).
  2. Open the file SPDiscovery.dll.config in a text editor.
  3. Increase the RetryWaitTime values to give SharePoint more time between retry attempts to complete previous operations. The values are in milliseconds.
  4. Increase the MaxRetries value to give Content Collector for SharePoint more attempts to complete operations in SharePoint. The value is in milliseconds.
  5. Increase the QueryPageSize value to collect more items per "page" from SharePoint, but note that this increase also consumes more resources. This value might need to be decreased if processing is exhausting available resources. The value is in number of documents.

Adjustments to ASP.Net configuration
Increase the timeout span for your web application.
  1. Open and edit the web.config file in the root directory of the web application. For example, this file for a default port 80 SharePoint web application in a default Internet Information Services directory would be located at: C:\Inetpub\wwwroot\wss\VirtualDirectories\80\web.config
  2. Locate the httpRuntime element under <configuration><system.web> Example: <httpRuntime maxRequestLength="51200" />
  3. Add or edit the executionTimeout attribute, giving it a higher value. The default value is 110 seconds and the maximum value is 999999 seconds. If dealing with large files, also increase the maxRequestLength, which has a maximum value of 2097151.
    Example: <httpRuntime executionTimeout="999999" maxRequestLength="2097151" />

Adjustments to post-processing configuration
If none of the above alleviates the problem, another approach is to use a post-processing option that involves fewer SharePoint operations. In order of amount of operations required for each SP Post-processing task option, highest first:
  1. Replace with Link
  2. Lock down
  3. Mark as processed
  4. Delete
[{"Product":{"code":"SSAE9L","label":"Content Collector"},"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Component":"Content Collector for Microsoft SharePoint","Platform":[{"code":"PF033","label":"Windows"}],"Version":"2.2.0","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

Document Information

More support for:
Content Collector

Software version:
2.2.0

Operating system(s):
Windows

Document number:
414723

Modified date:
17 June 2018

UID

swg21459229

Manage My Notification Subscriptions

Page Feedback