HTTP access log entry fields
HTTP access log entry fields
| Name | Type | Required | Release introduced | Standard NCSA log field |
Monitoring & Alerting fields AT: activity tracker <Other category>: |
Description |
|---|---|---|---|---|---|---|
| type | String | Yes | 3.9.0 | No | The access log entry type, which will always be "http" for entries of this type. Used for differentiation from other access log entry types such as "s3-multi-delete", "mirror", or "proxy". | |
| server_name | String | Yes | 3.1.0 | Yes |
AT: target.hostAdress, (except S3.EXPIRE.OBJECT), responseData.bucketLocation For S3 request, NA for COS resource configuration API, S3.EXPIRE.OBJECT For activity tracker logSourceCRN, location, or message location portion etc if the container_region is not available on 4xx errors. |
The host name of the server to which the request was sent. It is the value of the part
before ":" in the "Host" header value, if any, or the resolved server name, or the server IP
address. For requests using virtual-host style addressing, this field will have the bucket name in it followed by the endpoint name. When virtual-host style addressing isn't used, this field might just be the endpoint IP address, or DNS name. It can potentially be anything because the accesser doesn't care what the value is since it can get the bucket name from the URL. If the LB exists and the requests using a status check, this field will be the IP of the LB. |
| remote_address | String | Yes | 3.1.0 | Yes | Returns the Internet Protocol (IP) address of the client or last proxy that sent the request. | |
| forwarded_for | String | No | 3.1.0 | Yes | AT: initiator.host.address, target.address, and
responseData.targetAddress.type such as "public", "private", and "direct", i.e.
translate access log "service"→ private, transfer "and"→ direct, (when not from Bluemix) |
The
HTTP X-Forwarded-For header field value from the request. This is generally the
IP address of the originating client in cases where the request has been proxied. When the client
connection mode is set to the proxy connection, and the customer plans to use IP whitelisting
capability, the rightmost proxy IP address of the forward-for is regarded as the
client originating IP if the IP access control is enforced. In IBM Cloud, the field is the client IP, if the LB exists. If the LB doesn’t exist and there is no proxy, this field shouldn’t exist too. |
| forwarded | String | no | Returns the protocol which is used to make the request ("http" or "https") | |||
| user | String | No | 3.1.0 (deprecated in 3.4.1) | Yes | Returns the login of the user who is making this request, if the user is authenticated. | |
| remote_user | String | No | 3.4.1 (replaces deprecated field "user"). | Yes | AT:initiator.id Empty for public access, REST.OPTIONS.OBJECT or S3.EXPIRED.OJECT, malformed or missing credential |
Returns the login of the user making this request, if the user has been authenticated. This
can be any of the following depending on the authentication mechanism used: basic auth: user name S3 auth: S3 credential key ID. (HMAC access key) IAM auth: IAM username (IAM token). This is the same as cloud_iam.subject_ibm_id. keystone auth: user ID from keystone token This will be blank for anonymous requests. At the time of failure, this field reflects the best knowledge of user information that has been processed so far; if the user information is not available, the field will not be shown in the access log. |
| timestamp_start | long | Yes | 3.1.0 | No | AT:eventTime | The time stamp, in seconds since epoch, when the request was received. |
| timestamp_finish | long | Yes | 3.1.0 | No | The time stamp, in seconds since epoch, when response processing was completed. | |
| time_start | String | Yes | 3.1.0 | Yes | A human-readable date string that is associated with timestampStart (ISO 8601). | |
| time_finish | String | Yes | 3.1.0 | No | A human-readable date string that is associated with timestampFinish (ISO 8601). | |
| request_method | String | Yes | 3.1.0 | Yes | AT:action | Returns the name of the HTTP method with which this request was made, for example, GET, POST, or PUT. |
| request_uri | String | Yes | 3.1.0 | Yes | The request URI | |
| protocol | String | Yes | 3.1.0 | Yes | Returns the name and version of the protocol the request uses in the form
protocol/majorVersion.minorVersion , for example, HTTP/1.1 . Moved into "https" field in ClevOS 3.14.2. |
|
| status | int | Yes | 3.1.0 | Yes | AT:reason.reasonCode | The status code that is returned in the response. |
| midstream_error | String | No | 3.1.0 | No | If a mid-stream error occurred during the processing of the HTTP response, this field shows the error that occurred. A mid-stream response error is defined as an error that occurs after initial HTTP response headers are sent. | |
| response_length | Long | Present for all primary access log entries. | 3.1.0 | Yes | The number of bytes actually written in response body. | |
| request_length | Long | No | 3.2.0 | No | For PUT or POST requests, the number of bytes read
from the request body. Note: For server-side copy requests, the request_length
indicates the number of bytes read from the copy source, even though no data is originating from the
client’s request body.
|
|
| referer | String | No | 3.1.0 | Yes | The referrer entry from the http log. | |
| user_agent | String | Yes | 3.1.0 | Yes | AT:initiator.host.agent | The user agent from the http log. |
| request_latency | Long | Yes | 3.1.0 | No | The time, in milliseconds, from when the request was received until request processing is complete. | |
| request_id | String | Yes | 3.1.0 | No | AT:responseData.request_id | The request ID, a unique identifier that is generated for each request to assist with event correlation in the debug log. |
| remote_request_id | String | No | 3.13.5 | No | The Cleversafe request ID on the remote machine, in case a proxy is configured for the vault or container, to be able to trace the request. | |
| request_type | String | No | 3.4.0 | No |
AT:dataEvent, action dataEvent: true/false, message, target.typeURI, For example, action as "cloud-object-storage.object-batch.delete, cloud-object-storage.object-acl.create,cloud-object-storage.bucket.create, and target.typeTypeURI as "cloud-object-storage/bucket/retention" |
A string that captures the S3 request type.
REST.<HTTP_method>.<resource_type>For example, in the Resource Configuration API, the CONTAINER_CONFIG in the primary access log entry represents the command itself. Activity Tracker note: refer to the request type section for covered REST.* types in the Activity tracker. In addition, S3.Expiry and BATCH.DELETE are also included in the Activity tracker. |
| interface_type | String | Yes | 3.1.0 | No |
AT:objectType in the action field for the resource-configuration API (S3 is not mentioned) serviceName.objectType.action For example: cloud-object-storage.resource-config.update and target.typeURI, for example, cloud-object-storage/resource/config |
The API used to make the request. It supports s3, service, internal, and
resource-configuration. Activity Tracker note: Activity tracking is only applied when this value is "S3" or "resource-config". |
| stat | BaseChannel InOutStat | No | 3.1.0 | No | A JSON element with more latency information. | |
| upload_id | String | No | 3.10.1 | No | AT:correlationId | Upload UUID for multipart upload operations. |
| object_length | Long | No | 3.1.0 | No | The size of the object that is associated with this request. For delete requests, it is the
size of the object before its deletion, which is equal to
previous_object_length. Note: For 3.1, object length is not provided for delete
operations.
|
|
| object_id | String | No | 3.1.0 | No | The object ID, included for all SOH API requests. | |
| version_name | String | No | 3.2.0 | No | The version ID that is associated with the version of the object created/read/deleted. | |
| version_transient | Boolean | No | 3.2.0 | No | Represents whether the indicated version is a "transient" (true) or "permanent" (false) version of an object. Permanent versions are created on write if versioning is "Enabled" on a particular vault. Transient versions are created when versioning is "Suspended" or off. Transient versions can be overwritten while the current retention mode is not set to "Enabled". | |
| delete_marker | Boolean | No | 3.2.0 | No | True if the corresponding version ID is a delete marker. | |
| selective_logging_enabled | Boolean | No | 3.5.1 | No | True if this request was selected for selective debug logging. Otherwise, this field is omitted. | |
| last_modified | String (ISO 8601) | No | 3.8.0 | No | The last time the content of this object was modified. If no overwrites are done, the value is equal to the object creation time. | |
| last_changed | String (ISO 8601) | No | 3.8.0 | No | The last time the attributes of this object were modified. If no attribute modification is done, the value is equal to the last_modified time. | |
| e_tag | String | No | An identifier for a specific version of resource. | |||
| content_embedded | Boolean | No | 3.15.4.3 | No | Indicates if an object has its content embedded in the content header. This field may be omitted when this question does not apply (e.g. non-object targeted API) or if it has not been determined yet (e.g. due to errors). | |
| x_trans_id | String | For Swift | 3.9.0 | No | Swift transaction ID header (similar to request-id). | |
| object_name | String | 3.8.2 | No | AT: target.name (manadatory for all object-level operation except Multi-Delete) | The object's name. | |
| error_code | String | For S3 | 3.8.2 | No | AT: reason.reasonType | For S3 API, if an error is encountered in
status, this field contains the English-language descriptive error code. |
| proxy_enabled | String | No | 3.8.2 | No | Will be true if proxy is enabled. | |
| proxy_source | String | No | 3.10.0 | No | The source vault that is specified in the proxy configuration. | |
| proxy_type | String | No | 3.10.0 | No | The protocol that is used for communication with the source vault. | |
| range | String | Ranged-read is performed. | 3.8.2 | No | The HTTP Range header, if a ranged read request was performed. | |
| vault_name | String | No container metadata. | 3.8.2 | No | The name of the vault that is associated with the request. | |
| previous_last_modified | String (ISO 8601) | No | 3.10.2 | No | Last time that the object content was modified before the current request. For example, if an object was created on March 5, 2016 at 10:01, the next content update to this object would create an access log entry with previous_last_modified set to March 5, 2016, 10:01. Null for non-overwrite operations. | |
| previous_object_length | Long | No | 3.10.2 | No | Object length before the current object length. That is, before the current requested operation. Null for non-overwrite operations. | |
| requested_location_constraint | String | For PUT BUCKET | 3.10.0 | No | The requested location constraint of a PUT BUCKET (vault or container) request; if none is specified, then an empty String. Appears in PUT BUCKET requests. | |
| storage_account_id | String | For StaaS | 3.9.0 | No | AT: responseData.serviceInstanceId | The Account ID of the owner of the container. In the case of a 409 (BucketAlreadyExists), it will return the Account ID associated to the context of the request. If this is using cloud IAM, then the supplied service instance is used. Otherwise, the account stored with the existing bucket is used. |
| storage_location_id | String | For StaaS | 3.9.0 | No | Storage Location ID | |
| storage_account_realm | String | for Staas | 3.11.0 | no | Storage Account Realm (will always be 00000000-0000-0000-0000-000000000002 for 3.9.0, thus not introduced in access log until 3.11.0) | |
| container_name | String | For StaaS, if container metadata exists. | 3.9.0 | No | AT: target.name | Container Name |
| container_id | UUID | For StaaS | 3.9.0 | No | Container UUID | |
| principals.identity | String | No | The field will be the storage_account_id plus the "realm”. | |||
| principals.aws | String | No | HMAC access key; blank if using IAM-style credentials or anonymous access. | |||
| container_creator_id | String | For StaaS (Present in container creation with impersonation) | 3.10.1 | No | ID of the manager local user that initiated container creation | |
| fanout_copy_count | Integer | For fanout writes | 3.7.2 | No | The number of copies that are requested in this fanout request. | |
| fanout_copy_index | Integer | For fanout reads | 3.7.2 | No | The zero-indexed fanout copy requested to be read. | |
| fanout_punch_index | Integer | For fanout individual deletes | 3.7.2 | No | The zero-indexed fanout copy requested to be deleted (punched). | |
| fanout_delete_all | Boolean | For fanout delete all. | 3.7.2 | No | Indicates request to delete all copies of fanout object. | |
| format | Integer | yes | 3.9.0 | No | The current version of the HTTP request log entry format | |
| headers | Map<String, List<String>> | No | 3.8.2 | No | Logs request headers if enabled via advance configuration. | |
| is_secure | boolean | yes | 3.10.2 | no | Indicates whether the request was made over a secure HTTPS connection | |
| cipher_suite | String | no (present if secure request) | 3.10.2 | no | If the request was secure, indicates the SSL cipher suite that was used; If not secure, this will be null. Moved into "https" field in 3.14.2. | |
| encryption_method | String | no (present if header is present and request is successful, or if the request is using key protect) | 3.10.2 | no | Returns the encryption method. | |
| key_crn_location | String | No, Only for container creation when it is Key Protect or HPCS enabled. | 3.14.8 | no | The location field of the Root Key CRN being used in container creation. | |
| container_region | String | No. Only for container creation. | 3.14.8 | no | AT: logSourceCRN, AT event routing, responseData.bucketLocation This field is not available for 4xx errors (or 5xx error), AT looks up server_name for location instead. |
The region string configured for the container being created. This applies to all operations. It will be logged for success and failure cases that can get to a point where we can acquire the container_region information (So some front end checks won't have this information). |
| bucket_protection | BucketRetentionLogObject | for PUT BUCKET_PROTECTION, GET BUCKET_PROTECTION, and bucket deletion | 3.12.0 | no | AT: responseData.bucketProtction on (status and permanent_retention_enabled) | Relevant information when an operation is performed using the retention configuration (protection) sub-resource. |
| object_protection | ObjectRetentionLogObject | for object upload with protection, PUT OBJECT_PROTECTION, GET OBJECT_PROTECTION, POST OBJECT_LEGAL_HOLD, GET OBJECT_LEGAL_HOLD, POST OBJECT_RETENTION_EXTENSION, object deletion, any operation that fails due to a 451 UNAVAILABLE FOR LEGAL REASONS | 3.12.0 | no | AT: responseData.objectProtection on (retention_period and legal_hold_count) | Relevant information when an operation is performed relating to object protection. |
| aspera | AsperaRequestInfo | for Aspera requests | 3.13.4 | no | Information corresponding to the Aspera request, specifically related to the retry attempts. | |
| sse_kms | SseKmsRequestInfo | for Key Protect requests | 3.12.1 | no | Information about the Key Protect request, specifically related to the latency of the HTTP request made to key protect. | |
| error_message | String | no | 3.12.1 | no | AT: message | This field is present when the “status” field gives insufficient information regarding the
partial or complete failure of an attempted operation, stemming from an internal or SDK error. The
error handling sections in design documentation detail the specific scenarios in which this field
should be populated. The activity tracker filters error_message by keywords to inform the user of the detail of a subset of errors, such as invalid client IP addresses, to inform the end-user of a potential security risk. |
| object_count | Integer | for POST OBJECT_MULTI_DELETE | 3.13.0 | no | The number of objects deleted in a multiple object deletion. | |
| object_lifecycle | ObjectLifecycle contains object transition and object expiration. object transition - cloud only object expiration - all modes |
for objects with a lifecycle policy | 3.13.3 | no | AT:action | Information related an object's lifecycle if a policy exists. This will contain a list of lifecycle_actions which are equivalent to the lifecycle rules associated with an object. This structure of information will appear when writing an object, overwriting an object's content, or restoring an object from the archived state. |
| expiration | ExpiryAction | Yes | A single object contains information about an object's expiration (i.e. to be deleted at
some future time) Note: Must be populated if object_lifecycle is
present.
|
|||
| notification_sent | Boolean | no | 3.14.0 | no | Set to true if the operation attempted to send a notification using the Notification Service. | |
| https | HttpsAccessLogInfo | for secure HTTP requests | 3.14.2 | no | Relevant information regarding an HTTPS request. Fields include: protocol: Returns the name and version of the protocol the request uses in the form protocol/majorVersion.minorVersion, for example, HTTP/1.1. cipher_suite: If the request was secure, indicates the SSL cipher suite that was used; If not secure, this will be null |
|
| ibm_client_originating_ip | String | no | 3.14.3 | no | AT:initiator.host.address, responseData.targetAddress.type as (as "webui") | This field represents the trusted client IP that an intermediary trusted IBM service passes to COS. This field applies to both S3 API and Resource Configuration API. It is an optional field which is required when a trusted-service is used for IP access control for a bucket. |
| auth_decisions_copy_source | AuthDecisions | no | 3.17.2 | no | (same format as auth_decisions) |
S3 API: Represents the opcode of the S3 action the client is attempting on a source. |
| object_lock_configuration | ObjectLockConfiguration |
Bucket-Level configuration for buckets enabled for Object Lock: Object Lock enablement and Object Lock default retention |
3.17.2 | no | Information related to the Object Lock configuration for a bucket including enablement, default retention and the last modified time of the configuration. | |
| object_lock_protection | ObjectLockProtection |
Object-Level protection for objects with Object Lock Protection: Object Lock legal hold and Object Lock retention |
3.17.2 | no | Information related to the Object Lock protection for an object including legal hold, last modified time for legal hold, retention and last modified time for retention | |
| resource_group_id | String | no | 3.14.11 | no | AT:resourceGroupId | The resource group id associated with the IBM account id |
| request_body | String | no | 3.15.5 | no | AT:requestData | Returns a JSON format element when a patch request is successful. The content of this element is depended on which following parameters are set in this patch request. The parameters include firewall, activity tracking, monitoring, notification, hard quota. |
| replication | Object | No | 3.20.0 | no | Contains replication-related information. Present on user-initiated requests that trigger replication(s) and replication requests that are initiated by background replication agent . | |
| trace_id | String | No | 3.20.0 | No | The W3C trace id associated with the request. |
The following code is an example of a Service API to config Container allowed_IP PATCH /b/<bucket.name> operation:
{
"server_name": "10.137.16.68",
"remote_address": "9.47.30.38",
"remote_user": "admin",
"timestamp_start": "1548093957416",
"timestamp_finish": "1548093957517",
"time_start": "21/Jan/2019:18:05:57 +0000",
"time_finish": "21/Jan/2019:18:05:57 +0000",
"request_method": "PATCH",
"request_uri": "/container/container-vault-fdffec2d-e9c6-4546-b2e9-601575ce7a40",
"protocol": "HTTP/1.1",
"status": 200,
"response_length": "589",
"user_agent": "JetEngine/1.0",
"request_latency": "101",
"request_id": "1591acac-e03d-4908-9573-6f7798c372b3",
"request_type": "REST.PATCH.CONTAINER",
"interface_type": "service",
"selective_logging_enabled": true,
"object_name": "/container/container-vault-fdffec2d-e9c6-4546-b2e9-601575ce7a40",
"storage_account_id": "container-user93d95e64-2607-43ee-91d5-a933019b145f",
"storage_location_id": "449984a3-05be-783d-011b-90a1558aad65",
"container_id": "b64984b4-6da7-494f-a836-8c0c43ab1132",
"is_secure": false,
"principals": {
"identity": "ad28f191-b2d7-724a-01cf-7722c2bba250@00000000-0000-0000-0000-000000000000",
"username": "admin@default"
},
"type": "http",
"format": 1
}The following code is an example of a Service API to retrieve allowed_IP using GET /b/<bucket.name> operation:
{
"server_name": "10.137.16.68",
"remote_address": "9.47.30.38",
"remote_user": "admin",
"timestamp_start": "1548093955482",
"timestamp_finish": "1548093955487",
"time_start": "21/Jan/2019:18:05:55 +0000",
"time_finish": "21/Jan/2019:18:05:55 +0000",
"request_method": "GET",
"request_uri": "/container/container-vault-fdffec2d-e9c6-4546-b2e9-601575ce7a40",
"protocol": "HTTP/1.1",
"status": 200,
"response_length": "499",
"user_agent": "JetEngine/1.0",
"request_latency": "5",
"request_id": "c91b941f-950f-428a-8fba-54e285b1e9cc",
"request_type": "REST.GET.CONTAINER",
"interface_type": "service",
"selective_logging_enabled": true,
"object_name": "/container/container-vault-fdffec2d-e9c6-4546-b2e9-601575ce7a40",
"storage_account_id": "container-user93d95e64-2607-43ee-91d5-a933019b145f",
"storage_location_id": "449984a3-05be-783d-011b-90a1558aad65",
"container_id": "b64984b4-6da7-494f-a836-8c0c43ab1132",
"is_secure": false,
"principals": {
"identity": "ad28f191-b2d7-724a-01cf-7722c2bba250@00000000-0000-0000-0000-000000000000",
"username": "admin@default"
},
"type": "http",
"format": 1
}