List of Kafka topics to be mirrored

Review the list of Kafka topics to be mirrored for each service when you implement a multicluster setup.

Catalog service

  1. Main topic:
    • {topicPrefix}-{environment}-catalog-update-attribute
    • Without environment: {topicPrefix}-catalog-update-attribute
    • Mirrored topic:
      • {mirrorTopicPrefix}-{crossDCTopicPrefix}-{environment}-catalog-update-attribute
      • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-catalog-update-attribute
      • Without environment:
        • {mirrorTopicPrefix}-{crossDCTopicPrefix}-catalog-update-attribute
        • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-catalog-update-attribute
    • Backend server: The Catalog UpdateAttribute backend server retrieves data from the main and mirrored topics and then indexes it into the appropriate Elasticsearch index.
  2. Main topic:
    • {topicPrefix}-{environment}-catalog-update-category
    • Without environment: {topicPrefix}-catalog-update-category
    • Mirrored topic:
      • {mirrorTopicPrefix}-{crossDCTopicPrefix}-{environment}-catalog-update-category
      • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-catalog-update-category
      • Without environment:
        • {mirrorTopicPrefix}-{crossDCTopicPrefix}-catalog-update-category
        • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-catalog-update-category
    • Backend server: The Catalog UpdateCategory backend server retrieves data from the main and mirrored topics and then indexes it into the appropriate Elasticsearch index.
  3. Main topic:
    • {topicPrefix}-{environment}-catalog-update-item
    • Without environment: {topicPrefix}-catalog-update-item
    • Mirrored topic:
      • {mirrorTopicPrefix}-{crossDCTopicPrefix}-{environment}-catalog-update-item
      • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-catalog-update-item
      • Without environment:
        • {mirrorTopicPrefix}-{crossDCTopicPrefix}-catalog-update-item
        • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-catalog-update-item
    • Backend server: The Catalog UpdateItem backend server retrieves data from the main and mirrored topics and then indexes it into the appropriate Elasticsearch index.
  4. Main topic:
    • {topicPrefix}-{environment}-catalog-update-classification
    • Without environment: {topicPrefix}-catalog-update-classification
    • Mirrored topic:
      • {mirrorTopicPrefix}-{crossDCTopicPrefix}-{environment}-catalog-update-classification
      • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-catalog-update-classification
      • Without environment:
        • {mirrorTopicPrefix}-{crossDCTopicPrefix}-catalog-update-classification
        • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-catalog-update-classification
    • Backend server: The Catalog UpdateClassification backend server retrieves data from the main and mirrored topics and then indexes it into the appropriate Elasticsearch index.

Rules service

  1. Main topic:
    • {topicPrefix}-{environment}-rules-data-rep
    • Without environment: {topicPrefix}-rules-data-rep
    • Mirrored topic:
      • {mirrorTopicPrefix}-{crossDCTopicPrefix}-{environment}-rules-data-rep
      • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-rules-data-rep
      • Without environment:
        • {mirrorTopicPrefix}-{crossDCTopicPrefix}-rules-data-rep
        • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-rules-data-rep
    • Backend server: The Rules dataReplication backend server retrieves data from the mirrored topics and then indexes it into the appropriate Elasticsearch index.

Audit service

  1. Main topic:
    • {topicPrefix}-{environment}-iv-common-audit-events
    • Without environment: {topicPrefix}-iv-common-audit-events
    • Mirrored topic:
      • {mirrorTopicPrefix}-{crossDCTopicPrefix}-{environment}-iv-common-audit-events
      • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-iv-common-audit-events
      • If the environment is not provided:
        • {mirrorTopicPrefix}-{crossDCTopicPrefix}-iv-common-audit-events
        • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-iv-common-audit-events
    • Backend server: The Audit logstash backend server retrieves data from the main and mirrored topics and then indexes it into the appropriate Elasticsearch index.

Search service

  1. Main topic:
    • {topicPrefix}-{environment}-iv-reservation-change-event
    • Without environment: {topicPrefix}-iv-reservation-change-event
    • Mirrored topic:
      • {mirrorTopicPrefix}-{crossDCTopicPrefix}-{environment}-iv-reservation-change-event
      • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-iv-reservation-change-event
      • Without environment:
        • {mirrorTopicPrefix}-{crossDCTopicPrefix}-iv-reservation-change-event
        • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-iv-reservation-change-event
    • Backend server: The Search logstash-reservation backend server retrieves data from the main and mirrored topics and then indexes it into the appropriate Elasticsearch index.
  2. Main topic:
    • {topicPrefix}-{environment}-iv-inventory-tag-change-event
    • Without environment: {topicPrefix}-iv-inventory-tag-change-event
    • Mirrored topic:
      • {mirrorTopicPrefix}-{crossDCTopicPrefix}-{environment}-iv-inventory-tag-change-event
      • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-iv-inventory-tag-change-event
      • Without environment:
        • {mirrorTopicPrefix}-{crossDCTopicPrefix}-iv-inventory-tag-change-event
        • Without crossDCTopicPrefix: {mirrorTopicPrefix}-{topicPrefix}-iv-inventory-tag-change-event
    • Backend server: The Search logstash-inventorytag backend server retrieves data from the main and mirrored topics and then indexes it into the appropriate Elasticsearch index.