Notification format

Edit online

Note: The Notification format in ClevOS 3.14.6 and later is not backward compatible with the Notification format in prior ClevOS releases.

The following S3 operations generate a notification from the Notification Service.

Operation	Notifications `event_type` field value	Notes
PUT Object	Object:Write
POST Object	Object:Write
PUT Copy	Object:Write	The notification does not provide information about the source object.
Multipart upload complete	Object:Write	The notification is generated upon completion of the multi-part upload.
DELETE Object	Object:Delete, Object:CreateDeleteMarker, Object:CreateDeleteMarker:NullVersionDeleted	The notification event_type that is generated for a delete depends on the bucket’s version state. See S3 versioning behavior.
Multi-delete	Object:Delete, Object:CreateDeleteMarker, Object:CreateDeleteMarker:NullVersionDeleted	A separate delete notification is generated for each delete in the multi-delete operation. The notification event_type that is generated for a delete depends on the bucket’s version state. See S3 versioning behavior.

Notification key

The system uses Kafka’s default partitioning scheme to compute a hash based on the notification key. Because the notification key contains the request_id of the operation and the notification_id of the notification message, the hash is random, which allows the notifications to be evenly distributed over the topic’s partitions.

The following is a sample notification key:

{
  "format" : "2.0",
  "request_id": "1c8b6e0c-9f6a-489d-a177-a9b0ac7d1d3e",
  "notification_id": "1c8b6e0c-9f6a-489d-a177-a9b0ac7d1d35"
}

Notification payload

The following is a sample notification payload for PUT Object with versioning enabled:

{
  "format": "2.0",
  "request_id": "1c8b6e0c-9f6a-489d-a177-a9b0ac7d1d3e",
  "request_time": "2018-07-04T17:12:28.030Z",
  "event_type": "Object:Write",
  "bucket_name": "myVault",
  "bucket_uuid": "8b7bbeef-9f6a-192d-d839-a0b9a8c7f00d",
  "system_uuid": "48def817-f0a3-1949-abe2-9193be91de22",
  "system_name": "My DsNet",
  "object_version": "f3d83646-47be-4370-9557-3fa283dd0a5e",
  "object_name": "object.foo",
  "object_length": 123456,
  "object_etag": "51252794dea40abc1e5d65a47a5f806f",
  "content_type": "image/jpg",
  "meta_headers": [
    {"header":"x-amz-meta-foobar",
     "value":"foobar value"},
    {"header":"x-amz-meta-mandm",
     "value":"green"}
  ]
}

Certain fields might or might not be included in the notification payload for every event_type. The following table describes each field and the corresponding behavior for each operation.

Table 1. Payload field behavior by event_type
Payload field	event_type code
Payload field	Object:Write	Object:Delete	Object:CreateDeleteMarker	Object:CreateDeleteMarker:NullVersionDeleted
`format`	Included. The value is "2.0" by default.
`request_id`	Included.
`event_type`	Included.
`bucket_name`	Included.
`object_name`	Included.
`object_length`	Included. The value is the new object size.	Included. The value is the old object size and is 0 if the deleted object is a delete marker.	Included. The value is 0.	Included. The value is the old object size.
`object_version`	Included if versioning is enabled or suspended. The value is the object version if versioning is enabled. The value is null if versioning is suspended. Not included if versioning is disabled.	Included if versioning is enabled or suspended. The value is the object version that was deleted. Not included if versioning is disabled.	Included if versioning is enabled or suspended. The value is the object version of the delete marker if versioning is enabled. The value is null if versioning is suspended.	Included if versioning is suspended. The value is null.
`object_etag`	Included.	Not included.	Not included.	Not included.
`request_time`	Included.
`meta_headers`	Included if set in s3 . Headers starting with this prefix are user-defined metadata. Each one is stored and returned as a set of key-value pairs. The system doesn't validate or interpret user-defined metadata.	Included if found in deleted versions. Headers starting with this prefix are user-defined metadata. Each one is stored and returned as a set of key-value pairs. The system doesn't validate or interpret user-defined metadata.	Not included.	Included if found in deleted null version. Headers starting with this prefix are user-defined metadata. Each one is stored and returned as a set of key-value pairs. The system doesn't validate or interpret user-defined metadata.
`content_type`	Included if set in the s3 request.	Included if found in deleted version.	Not included.	Included if found in deleted null version.
`bucket_uuid`	Included.
`system_uuid`	Included.
`system_name`	Included if the user set a system name.

S3 versioning behavior

The notification payload fields and values change based on the version state of the bucket that is being operated on. The following payload format changes occur for buckets that are in one of the three versioning states: enabled, suspended, or not enabled.

Write event_types (PUT/POST/COPY/Multipart complete)

Table 2. Write operation payload format or behavior
Versioning state	Payload format or behavior
Enabled	The notification payload includes the new object's version, which matches the x-amz-version-id returned from the write operation.
Suspended	The object's version in the notification payload is null, which matches the x-amz-version-id returned from the write operation.
Not enabled	The object_version field is not included in the notification payload.

Delete event_types

The following behaviors are for a DELETE of an object in a versioned bucket that does not specify the versionId sub-resource in the S3 request.

Table 3. Delete event_type payload format or behavior
Versioning state	Payload format or behavior
Enabled	A delete marker is created with its own s3 version, and no actual object data is deleted from the volume. The notification payload includes an object_length of 0. The object_etag, content_type, and meta_headers fields are not included.
Suspended	A delete marker is created with the version of null and, if it exists, a version of the object which also had the s3 version of null is deleted. If a null version of the object exists and is deleted, or if the existing object is a null delete marker that is being replaced with a new one, the Notification Service uses the Object:CreateDeleteMarker:NullVersionDeleted event_type code. The object_version value is null and the object_etag field is not included. The content_type, meta_headers, and object_length fields represent the object being deleted. If a null version of the object does not exist, the Notification Service uses the object:CreateDeleteMarker. The object_version value is null and the object_etag field is not included. The object_etag, content_type, and meta_headers fields are not included and the object_length is 0. IBM Spectrum Discover uses these event_type codes to determine if an existing record needs any modification before it is replaced by the delete marker.
Not enabled	The fields that describe the object represent the object at the time of deletion. Object_length, content_type, and x-amz-meta fields match the state of the object being deleted. object_version and object_etag fields are not included.

Delete version event_type

The following behaviors are for a DELETE of a versioned bucket that specifies the versionId subresource in the S3 request.

Table 4. Delete version event payload format or behavior
Versioning state	Payload format or behavior
Enabled	When a specific version of an object is deleted, an Object:Delete notification is generated. It includes the deleted object's version, metadata, content type, and size. To delete a delete marker, use a version delete that specifies the marker's version. This generates an Object:Delete notification with the object_version set to the marker's version, which matches the version of the Object:CreateDeleteMarker notification that created the marker. The meta_headers, content_type, and content_etag fields are not included in the payload. The object_length is 0.