After accounting for the security and shared settings differences of the Salesforce Objects data that you want to crawl and deliver to your users, you will want to consider how your Salesforce API usage limitations will impact the Salesforce connector. Therefore, when preparing to crawl your data, consider the following:
- Your API usage limitations can unexpectedly be consumed if you are not careful as to understanding how the Salesforce connector will crawl your Salesforce data. For example: You might configure your Salesforce data collection to crawl only one Object type, with a limited number of records, in a large Salesforce organization, and be surprised to learn that you consumed a large number of API calls. This is because the Salesforce connector will still need to crawl security information (as well as any Base64 binary fields) for this object type, which may greatly increase the API calls to your Salesforce repository. Therefore, when preparing to crawl your Salesforce data, be sure that the number of API calls allowed by Salesforce is large enough to crawl all the Salesforce Objects in your search, plus the security information configured in OrganizationWideDefaults, Profiles, PermissionSets, Users, Groups, Roles, ShareObject records, and binary fields.
- Binary data and certain types of security information are crawled at different rates, which affects the number of API calls used to crawl this data. This includes the Salesforce Objects: OrganizationWideDefaults, Profiles, PermissionSets, Attachment, Documents, Document Folders, ContentVersion, and metadata required by the crawl for all Objects and Custom Objects. This also includes binary field data which may exist on Salesforce Objects such as FeedItem, Idea and Custom Object types.
- For such items, there is a limitation of 1 binary field per API call, and a maximum of 10 records per API call otherwise. An additional daily allowance of API requests may be required if your Salesforce site contains many binary or security items.
Note: For optional binary data fields, the Salesforce connector is designed to check your Salesforce Objects first to determine if binary data is present before making a separate API call to retrieve that data.