Object Stores support

File Agent supports Amazon S3 or S3 compatible object stores, Azure Blob, Google Storage and IBM Cloud Object Storage. A default configuration is provided to access S3 objects using the default Amazon S3 credential mechanisms. For more details on credentials mechanisms for each of these object stores, see Credentials.

Amazon S3 default configuration

A property file located in the File Agent config directory is dedicated to object stores configuration if some particular configuration properties should be changed from the default values or created for another provider. A default set of properties is already available to connect to Amazon S3. This configuration will enable File Agent to connect and Amazon S3 using the default credentials mechanisms.
# AWS S3 provider - default
cdfa.provider.1=scheme=S3://
cdfa.provider.1=store.providerName=S3

S3 configuration

A set of provider properties is declared by using the following syntax: cdfa.provider.setName=property=value with n the set name. All properties declared on this set will rely on the same provider. A scheme property identifies and link the scheme name in File Agent watched directory and a provider.
Credential Mechanism Scheme in providers.properties Scheme in watched directories
cdfa.provider.1=scheme=
S3://
S3://abucketname/afolder This watched directory will be linked to provider 1.
cdfa.provider.2=scheme=
OS3://
OS3://otherbucketname/otherfolder This watched directory will be linked to provider 2.

No entries for scheme X4://

X4://container/folder This watched directory will be rejected

Additional object stores configuration

More properties can be used to fine configure a provider entry. See Stores Properties. Use the cdfa.provider.setName=property=value model to declare them.

Out scheme property

When Connect Direct is configured to work with a scheme but not the same than File Agent, use outscheme property to define what scheme will be used when generating %FA_FILE_FOUND. variable. %FA_WATCHED_FILE_FOUND. content is the original name.

When Connect Direct is configured to work with a scheme but not the same than File Agent, use outscheme property to define what scheme will be used when generating %FA_FILE_FOUND. variable. %FA_WATCHED_FILE_FOUND. content is the original name.
Scheme Outscheme Discovered File %FA_FILE_FOUND. %FA_WATCHED_FILE_
FOUND.
S3:// Not provided S3://bucket/folder/name S3://bucket/folder/name S3://bucket/folder/name
S3:// OS3:// S3://bucket/folder/name OS3://bucket/folder/name S3://bucket/folder/name

Configuration example

# AMZ:// scheme will use provider S3 and will generate %FA_FILE_FOUND. With scheme S3://
cdfa.provider.AMS3=store.providerName=S3
cdfa.provider.AMS3=scheme=AMZ://
cdfa.provider.AMS3=outscheme=S3://
cdfa.provider.AMS3=s3.accessKey=xxxxxxxxxxxxxxxxxxx
cdfa.provider.AMS3=s3.secretKey=xxxxxxxxxxxxxxxxxxxxxxxx

# QA:// scheme will request properties from Connect Direct initparms
# file.ioexit name=QA must exist in initparms
# Provider type and other properties are retrieved from initparms.cfg file 
# This example is only valid for X86_64 Linux
cdfa.provider.QA=scheme=QA://
cdfa.provider.QA=store.configFromCD=YES

# SEC:// scheme will use provider S3
# CAs certificates will be retrieved from Secure Plus keystore only
# This example is only valid for X86_64 Linux and Windows
cdfa.provider.SECURE=store.providerName=S3
cdfa.provider.SECURE=scheme=SEC://
cdfa.provider.SECURE=store.keyStore=SP_ONLY

# BOTH:// scheme will use provider S3 (default)
# CAs certificates will be retrieved from Secure Plus keystore and next from JRE cacerts file
# S3 credentials are retrieved from the default S3 credentials chain
# This example is only valid for X86_64 Linux and Windows
cdfa.provider.MIXEDCAS=scheme=BOTH://
cdfa.provider.MIXEDCAS=store.keyStore=SP_JRE


# MICROSOFT:// scheme will use provider Azure Blob
cdfa.provider.AZURE=store.providerName=AZ
cdfa.provider.AZURE=scheme=MICROSOFT://
cdfa.provider.AZURE=az.connectionString=xxxxxxxxxxxxxxxxxx

# IBM:// scheme will use provider IBM COS 
cdfa.provider.IBMCOS=store.providerName=COS
cdfa.provider.IBMCOS=scheme=IBM://
cdfa.provider.IBMCOS=cos.location=ams03

# GOOGLE:// scheme will use provider Google Storage
cdfa.provider.GOOGLE=store.providerName=GS
cdfa.provider.GOOGLE=scheme=GOOGLE://
cdfa.provider.GOOGLE=gs.credentialsPath=xxxxxxxxxxxxxxx