Enabling an external content repository for source control
You can use the Administration page in the web client to enable an external content repository for source control. When enabled, source control content larger than the configured threshold is stored in an external content repository instead of in the application's database.
Before you begin
About this task
Jazz Source Control can store content externally, either on a server that supports the WebDAV protocol, or on the application server's local filesystem. If you are configuring an external WebDAV server, you must supply credentials that the CCM application can use to access that server.
Procedure
Changing the configured external content repository
You can change the configuration values for the external content repository. Any changes affect where new content is stored; it does not affect how existing content is stored or retrieved.
Procedure
Changing WebDAV access credentials
IBM® Engineering Workflow Management stores the access credentials for each Versioned Content External Repository – URI value that you enter. If you change the Versioned Content External Repository – URI value and its associated credentials, the old URI and credentials are stored so that Engineering Workflow Management can continue to retrieve content on that server while using the new credentials to access the currently configured external content repository.
Procedure
Moving content
You can use the repository tools moveContent
command to move previously
stored content in three ways.
About this task
You can use the repository tools moveContent
command to move previously stored
content in these ways:
- From the CCM server database to an external content repository
- From an external content repository to the CCM server database
- From an old external content repository to the currently configured one
The moveContent
command has arguments to specify source and destination content
repositories, as well as the minimum or maximum size of content to move. For details about how to
run the command with various arguments, see the repository tools command-line reference documentation.
Deleting content
About this task
When content stored in an external content repository is deleted from Engineering Workflow Management, the server also attempts to delete the content from the external content repository. The server attempts the deletion, but it is not guaranteed. For more information, see: Deleting Content From Source Control
If a problem occurs with deleting content from the external repository, the problem is logged on the server but does not cause the delete operation to fail.
Reporting using JMX MBeans
In Engineering Workflow Management, you can monitor content stored in SCM by using Java Management Extensions (JMX) managed beans (MBeans).
About this task
Procedure
Content Repository Metrics MBean
When the Content Repository Metrics MBean is enabled, an MBean is published that contains
a list of the content repositories in which content has been stored. The CCM database is represented
as a repository with DATABASE
as its URI.
About this task
The MBean name has the form
com.ibm.team.scm.content:name="<app-context>",type=repositoryMetrics
. The bean
contains a list of repositories. For each content repository, the following information is
published:
Property name | Description |
---|---|
externalURI | The external content repository URI for the content repository, or DATABASE
for the default database repository. |
totalNoOfContent | The number of unique content objects stored in the content repository. |
compressedDBSize | The total number of bytes stored in the database for this content repository. For the default
DATABASE repository, this is the compressed size of the content. External content
repositories store metadata in the database. |
totalSize | The total uncompressed size of the content stored in the content repository. |
Component Metrics MBean
When the Component Metrics MBean is enabled, an MBean is published for each SCM component.
About this task
The MBean names have the form
com.ibm.team.scm.content:name="<app-context>",type=componentContent,componentNameAndId="<componentName>_<componentUUID>"
.
For each content repository, the following information is published:
Property name | Description |
---|---|
componentName | The name of the component. |
componentId | The UUID of the component. |
ownerName | The name of the component owner. This is the name of the containing project area if the component is owned by a project or team area, or the name of the owning contributor. |
ownerId | The UUID of the component owner. This is the UUID of the containing project area if the component is owned by a project or team area, or the UUID of the owning contributor. |
totalNoOfContent | The number of unique content objects claimed by files in this component. |
dbSize | The total number of bytes stored in the database for content claimed by files in this component. |
totalSize | The total uncompressed size of content claimed by files in this component. |
externalStores | A list of external content repositories in which content is stored for this component. This value will be empty if all content is stored in the CCM database. For each external content repository, see the table below: |
Property name | Description |
---|---|
externalURI | The URI of the external content repository. |
totalSize | The total number of bytes stored in the external content repository. |
totalNoOfContent | The number of unique content objects claimed by files in this component stored in this external content repository. |
Engineering Workflow Management uses a hash-based storage mechanism to eliminate storing multiple copies of identical content. If identical content is checked in multiple times, only a single copy of that content is physically stored and each project, component, or file associated with that content points to the single persisted copy. The transparency to users applies to both content stored in the CCM server database and content stored in external content repositories.
The combined total of all the component metrics beans might be larger than the actual number of bytes physically stored in the database or external content repositories.
Top File Sizes MBean
The number of files reported by the Top File Sizes MBean is controlled by the CCM server
advanced property File Sizes MBean – Number of Files to report under
VersionedContentMetricsTask
. When the Top File Sizes MBean is enabled, a bean is
published for each of the largest files whose size is greater than that specified by the File Sizes
MBean – size threshold (in bytes) property.
About this task
The MBean names have the form
com.ibm.team.scm.content:name="<app-context>",type=topContent,rank="<N>"
. For
each of the largest files, the following information is published:
Property name | Description |
---|---|
contentSize | The compressed size of this content if stored in the database; otherwise, the full content size when stored externally. |
externalURI | The URI for the external content repository where this content is stored, or
DATABASE if stored in the CCM server database. |
contentClaimer | For each external content repository, see the following table. |
componentName | The name of the containing component. |
itemId | The item UUID for the claiming file. |
stateId | The state UUID for the claiming file. |
approximatePath | The approximate or most popular path for the file. The path for a file is determined by the
directory structure of each repository workspace or stream in which it appears and, in general,
cannot by computed without specifying a workspace or stream. The file might not actually exist at
the approximatePath in any specific workspace, particularly if each workspace or
stream that contains the file has it at a different path. |
Engineering Workflow Management clustering considerations
You might use Engineering Workflow Management in a clustered environment.
About this task
- Your external content repository must be accessible to all nodes in the cluster. This is true for both file-based and WebDAV-based content repositories.
-
If you have content in multiple WebDAV repositories, and need to edit the credential files manually, you must update the credentials file on each node in the cluster.
Backup
When you back up a Engineering Workflow Management configuration that uses external content repositories, it is important to back up the CCM database first, and then back up the external repositories. When data is restored, the server reflects the state immediately after the database backup regardless of when the external content repositories are backed up.
About this task
If you roll back the server, you only have to roll back the CCM database. You do not have to roll back the external content repositories. In this type of scenario, the external content repositories retain any orphaned content that was added before the database roll back, and the data integrity is not affected. The orphaned content is automatically reclaimed if the content is checked in to the server again.
The backup procedure for an external content repository depends on the type of repository. For a WebDAV content store, consult that product documentation for specific instructions. For a file system content store, any industry-standard file system backup procedure is appropriate.
During the time between the completion of the database backup and the completion of the external
repository backups, do not use the delete content feature or the repository tools
moveContent
command. The delete content feature is a rare action performed by an
administrator that removes content bytes from Jazz
Team Server: it does
not refer to the common action of checking in file deletions. Most users never use the delete
content feature.
If any content deletions are performed between the two backups, after data is restored, the content delete must be reapplied to return the database to a consistent state for those items. Content deletes, if originally performed, must also be reapplied if the CCM database is rolled back and the external content store is not.
Best practices
Note the following best practices for configuring external content repositories:
About this task
- To use the external content repository feature with certain strictly compliant WebDAV servers such as Microsoft IIS, ensure that you install Rational® Team Concert® version 6.0.6 iFix003 or later.
- Because various WebDAV servers might have specific configuration requirements, ensure that you read the server documentation before configuring the server.
- To confirm that the WebDAV server is working properly, run the curl PUT and curl GET commands.
- Before you configure the WebDAV server, know the following information:
- Maximum file upload size limit imposed by the WebDAV server
- File type and content type configuration of the server
- Whether the basic authentication is installed and enabled on the server
- Configure your WebDAV server to use
HTTPS
. Engineering Workflow Management uses basic authentication to send WebDAV login credentials. Configuring your WebDAV server to use HTTPS ensures that these credentials are encrypted and not passed in plain text. - Do not use very small values for the
Versioned Content External Content – Artifact Minimum Size
property. The added overhead of making HTTP requests to a WebDAV server negatively affects performance if very large numbers of small files need to be accessed. The external content repository feature was designed to move large files out of the CCM server database and into external content repositories: it is not intended to move all SCM files out of the CCM server database.
If you use a Microsoft IIS server, you might encounter the following issues:
Issue | Solution |
---|---|
Basic authentication for IIS is not installed or enabled by default. | Install basic authentication for IIS by using the Windows Control Panel and enable the authentication in IIS Manager. For details, see the Microsoft IIS help on basic authentication. |
IIS fails to return content that does not have file extensions. |
Add a MIME type mapping for files with no file extension. You can configure the MIME type at the virtual directory level for the virtual directory that contains the external content. Configure the MIME type in IIS Manager. For details, see Microsoft IIS help for configuring MIME types in IIS 7. |
The default maximum file upload size for IIS is only 30 MB. | You can increase the maximum file upload size at the website level in IIS Manager. For details, see Microsoft IIS help for configuring request filtering in IIS. |
IIS has a maximum upload size of 4 GB. |
If you must handle larger files, you must select a different server such as Artifactory or use the File content store type. If you configure a maximum file upload size on your WebDAV server, you must set the Engineering Workflow Management Advanced Server Property "Versioned Content Maximum Size" to the same value. If you set a value and check in a file that is larger than the set value, the Engineering Workflow Management client displays an error message instead of attempting to upload the file to the WebDAV server first. |
- Turn off anonymous access in Artifactory to prevent non-authenticated users from retrieving content from the Artifactory server.
- If you use the content delete feature in Engineering Workflow Management to delete sensitive SCM content, you must empty your Artifactory trash folder, or configure Artifactory not to use a trash folder.
- UNC paths are not supported. If you want to use network storage for a FILE external content repository, it must be mounted on the local file system on Linux or mapped as a network drive in Windows.
- Use canonical paths when you configure the content repository URI. Paths that contain spaces
should be appropriately escaped (for example,
file:////C:/Content%20Repository
).