审计日志记录采用指南

配置 IBM® Cloud Private 集群以生成审计日志并将日志路由到安全信息和事件管理 (SIEM)。

IBM Cloud Private 审计日志记录体系结构

审计日志记录组件 AA - 审计代理程序；LA - 应用程序日志代理程序

审计容器

以下是审计日志记录体系结构的关键组件：

审计容器

审计容器是一个 sidecar 容器。它使用 system-cat 命令来跟踪 audit.log 文件并通过管道输送该文件。它将审计日志发送到 systemd 日志。

每个生成审计日志的服务都会将日志写入到 /var/log/audit/<service_name>-audit.log 文件。服务容器必须与审计容器共享 /var/log/audit。

emptyDir 卷用于在服务容器间共享目录 /var/log/audit，以在 Pod 中审计 sidecar 容器。

由于审计容器必须对 systemd 日志进行写入，因此它还需要安装存在系统日志的主机文件系统。

logrotate 工具用于监视 /var/log/audit 目录中日志的大小、循环周期和其他参数，以及按照配置中所指定来回收审计日志。

Journald（systemd 日志）

systemd 是在节点上运行的一项服务。在具有审计 sidecar 容器的 Pod 中运行的 IBM Cloud Private 服务所生成的审计日志发送到 systemd 日志。systemd 日志以二进制格式存储数据。只能附加数据。在 systemd 日志接收到审计数据时，该数据由 fluentd 提取并发送到 Elasticsearch 或 SIEM。

Fluentd

Fluentd 是使用输入和输出插件从多个源收集数据并将数据分发或发送到多个目标的日志收集器。

Fluentd 使用 fluent-plugin-systemd 输入插件从 systemd 日志收集审计日志。此插件具有数据过滤能力，并且仅从 systemd 日志收集审计日志。

Fluentd 容器安装日志数据存储所在的主机文件系统。缺省位置为 /run/log/journal。

缺省情况下，Fluentd 使用 fluent-plugin-elasticsearch 输出插件将审计日志发送到 Elasticsearch Logstash Kibana (ELK) 堆栈。Fluentd 可以配置为将日志发送到企业 SIEM 工具（例如 QRadar）。

从审计日志记录角度而言，以下配置选项对于输出插件非常重要：

客户机或主机证书选项。这些选项启用 ELK 的公用密钥基础结构 (PKI) 认证。

  ca_file /path/to/your/ca/cert

  client_cert /path/to/your/client/cert
  client_key /path/to/your/private/key
  client_key_pass password

  ssl_version TLSv1_2

注：可以使用以下语句从私钥馈送密码：

  client_key_pass "#{ENV["APP_KEYSTORE_PASSWORD"]}"

Logstash 选项。此选项将审计日志记录数据放在其存储区中，以将审计日志与其他调试日志分隔开来。

  logstash_format true # defaults to false
  logstash_prefix mylogs # defaults to "logstash"
  logstash_prefix_separator _ # defaults to "-"
  logstash_dateformat %Y.%m. # defaults to "%Y.%m.%d"

ELK

ELK 堆栈用于存储、索引和表示 IBM Cloud Private 日志。

ELK 还用于审计日志记录。ELK 从 Fluentd 接收到的数据会进行解析，然后放入到不同存储区中：一个用于日志记录，一个用于审计数据。

审计日志事件

以下事件可以记录为审计事件：

认证事件（登录尝试成功或失败）
授权事件（尝试访问资源或数据成功或失败）
系统配置修改
创建、读取、删除或更新资源或数据
会话管理失败
用户特权更改
数据库访问（成功或失败）
网络或应用程序防火墙、侵入检测系统 (IDS) 或侵入防御系统 (IPS) 事件
系统使用量
系统启动、关闭或重新引导事件
应用程序或系统故障事件

审计日志中不得包含以下数据：

敏感信息
用户凭证
密码
银行帐户详细信息
访问令牌
认证令牌
文件系统路径或信息
数据库查询或字符串
加密或解密密钥

审计日志记录格式

IBM Cloud Private 遵循云审计数据联盟 (CADF) 标准。该标准定义事件模型以收集进行审计所需的数据。IBM Cloud Private 添加一些定制字段以生成综合日志。

有关 CADF 的更多信息，请参阅 Cloud Auditing Data Federation 在新选项卡中打开。

以下字段非常重要：

{
    "typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event",
    "eventType": "activity",
    "id": "icp:e97c7b00-e215-11e8-abf8-79cb75b57820",
    "action": "create",
    "requestPath": "/identity/api/v1/teams",
    "initiator": {
        "typeURI": "service/security/account/user",
        "name": "admin",
        "credential": {
            "type": "token"
        },
        "host": {
            "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Safari/605.1.15",
            "address": "icp-management-ingress:8443"
        }
    },
    "target": {
        "id": "c4e8170e90a7c01a228fbef74c22245d2665cefffac1662907a6a75e82319a74",
        "name": "icp-testing-audit-logs",
        "actions": {
            "name": "icp-testing-audit-logs",
            "teamId": "icp-testing-audit-logs",
            "users": [],
            "usergroups": [],
            "directoryList": []
        },
        "typeURI": "service/security/group"
    },
    "observer": {
        "id": "target"
    },
    "severity": "normal",
    "outcome": "success",
    "reason": {
        "reasonType": "HTTP",
        "reasonCode": 200
    },
    "eventTime": "2018-11-06T22:47:17.424Z",
    "kubernetes.container_id": "c4e8170e90a7c01a228fbef74c22245d2665cefffac1662907a6a75e82319a74",
    "kubernetes.container_name": "platform-identity-management",
    "kubernetes.pod": "auth-idp-mw2x9",
    "kubernetes.namespace": "kube-system",
    "origination": "ui",
    "version": "v1.0"
}

Go 语言 CADF 结构

type CADF struct {
    TypeURI   string `json:"typeURI"`
    Action    string `json:"action"`
    ID        string `json:"id"`
    Initiator struct {
        Name    string `json:"name"`
        TypeURI string `json:"typeURI"`
    Credential struct {
            Type string `json:"type"`
        } `json:"credential"`
    } `json:"initiator"`
    Target struct {
        ID      string `json:"id"`
        Name    string `json:"name"`
        TypeURI string `json:"typeURI"`
    } `json:"target"`
    RequestPath             string `json:"requestPath"`
    EventType               string `json:"eventType"`
    Severity                string `json:"severity"`
    Outcome                 string `json:"outcome"`
    EventTime               string `json:"eventTime"`
    KubernetesContainerID   string `json:"kubernetes.container_id"`
    KubernetesContainerName string `json:"kubernetes.container_name"`
    KubernetesPod           string `json:"kubernetes.pod"`
    KubernetesNamespace     string `json:"kubernetes.namespace"`
    Observer                struct {
        ID string `json:"id"`
    } `json:"observer"`
    Origination string `json:"origination"`
    Version     string `json:"version"`
}

NodeJs CADF 结构

 let cadf = {
      "typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event",
      "eventType": "activity",
      "id": "icp:"+uuid1,
      "action": action,
      "requestPath": path,
      "initiator": {
    "typeURI": (user ? "service/security/account/user": ''),
        "name": user,
    "credential": {
      "type": "token"
        },
        "host": {
            "user-agent": req.headers['user-agent'],
          "address": req.headers['host']         
    }
      },
    "target": {
        "id": cont_id,      
    "name": res,
    "actions": actions,
        "typeURI": (map ? map: parseUrl(path)) // pretend this app is a service
      },
      "observer": {
        "id": "target"
    },
      "severity" : severity,
      "outcome": outcome,        
      "reason": {
    "reasonType":"HTTP",
        "reasonCode": status, // like 200 or 400 
      },
      "eventTime": expT,
      "kubernetes.container_id": cont_id, 
      "kubernetes.container_name": process.env.SERVICE_NAME, 
      "kubernetes.pod": process.env.POD_NAME || process.env.HOSTNAME,
      "kubernetes.namespace": process.env.POD_NAMESPACE, 
      "origination": identifyOrig(req.headers['referer'] || req.headers['user-agent']), 
      "version": "v1.0"
    };

部署文件修改

在部署文件中 container 部分下的第一个位置添加 audit logging sidecar container：

      - name: icp-audit-service
        image: mycluster.icp:8500/ibmcom/icp-audit-service:3.1.1
        imagePullPolicy: IfNotPresent
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /run/systemd/journal
          name: journal
        - mountPath: /var/log/audit
          name: shared

注：映像路径取决于 IBM Cloud Private 版本和安装类型。以下示例是 IBM Cloud Private V3.1.1 上的脱机安装示例：

在 Pod 中的所有容器上安装 /var/log/audit/ 共享卷：

        volumeMounts:
        - mountPath: /var/log/audit
          name: shared

可以使用环境变量 kubernetes.container_name、kubernetes.pod 和 kubernetes.namespace 在容器中提供其中一些 CADF 格式的定制字段。

      - env:
        - name: SERVICE_NAME
          value: key-management-lifecycle
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: CONFIG_PATH
          value: /opt/keyprotect/config/
        - name: ICP_NAMESPACE
          value: kube-system
        - name: CLUSTER_NAME
          valueFrom:
            configMapKeyRef:
              key: CLUSTER_NAME
              name: platform-auth-idp

在服务容器中，运行中的应用程序将在装载在 /var/log/audit/ 中的目录中创建 <service_name>-audit.log 文件。
将应用程序或服务生成的所有审计日志写入或追加到 <service_name>-audit.log 文件。审计 sidecar 容器将日志转发到 journald，然后 journald 将日志转发到 fluentd。

审计日志必须是 JSON 格式，Fluentd 和 SIEM 工具才能正确解析。请参阅以下样本：

 {"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:16d5af60-4cc2-11e9-9451-57b95a7e8968","action":"update","requestPath":"/identity/api/v1/teams/test-team/resources","initiator":{"typeURI":"service/security/account/user","name":"admin","credential":{"type":"token"},"host":{"user-agent":"curl/7.47.0","address":"<cluster-ip>:8443"}},"target":{"id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","name":"test-team","actions":"crn:v1:icp:private:k8:mycluster.icp:n/default:::","typeURI":"service/security/group"},"observer":{"id":"target"},"severity":"normal","outcome":"success","reason":{"reasonType":"HTTP","reasonCode":200},"eventTime":"2019-03-22T16:46:50.198Z","kubernetes.container_id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","kubernetes.container_name":"platform-identity-management","kubernetes.pod":"auth-idp-8jdlx","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}

If there are multiple service containers in a single pod, there is a separate audit log file for each service container in the /var/log/audit/ directory and only one file for audit sidecar container. The volume of the /var/log/audit/ directory is shared by all containers in a pod.
You can enable or disable audit logs for a service. Add the AUDIT_ENABLED flag in the ConfigMap. The flag can be added as an environment variable so that the application code has access to enable or disable generation of audit records.

Following is a deployment file example:

# Please edit the object below. Lines beginning with a '#' will be ignored,
  # and an empty file will abort the edit. If an error occurs while saving this file will be
  # reopened with the relevant failures.
  #
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: kompose convert
    kompose.version: 1.17.0 ()
  creationTimestamp: null
  labels:
    io.kompose.service: think-blue-demo-app
  name: think-blue-demo-app
  namespace: jkstore
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        io.kompose.service: think-blue-demo-app
    spec:
      containers:
      - name: icp-audit-service
        image: mycluster.icp:8500/ibmcom/icp-audit-service:3.1.2
        imagePullPolicy: IfNotPresent
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /run/systemd/journal
          name: journal
        - mountPath: /var/log/audit
          name: shared
      - name: think-blue-demo-app
        image: mycluster.icp:8500/kube-system/think-blue-demo_app:0.1
        imagePullPolicy: IfNotPresent
        args:
        - npm
        - start
        ports:
        - containerPort: 30003
        env:
        - name: CLIENT_ID
          value: "9b570f23952a45099966ffa2cf7b6355"
        - name: CLIENT_SECRET
          value: "3rQwon8BrfpOEqzX2RttUXAt4CVkf8S2WuBQKiE5wPHKudMGX0FIlARejVf9"
        - name: AUTH_URL
          value: "https://<CLUSTER_IP>:8443/idprovider/v1/auth/authorize"
        - name: TOKEN_URL
          value: "https://<CLUSTER_IP>:8443/v1/auth/token"
        - name: ISSUER_ID
          value: "https://mycluster.icp:9443/oidc/endpoint/OP"
        - name: CALLBACK_URL
          value: "https://bobcat.rtp.raleigh.ibm.com/auth/liberty/callback"    
        - name: LOGOUT_URL
          value: "https://<CLUSTER_IP>:8443/v1/auth/logout"  
        - name: AUDIT_ENABLED
          value: "true"
        - name: CONTAINER_ID
          value: "think-blue-demo-app"
        - name: SERVICE_NAME
          value: "think-blue-demo-app"
        - name: AUTHZ_URL
          value: "https://<CLUSTER_IP>:8443/iam-pdp/v1/authz"  
        - name: PAYMENT_URL
          value: "https://<CLUSTER_IP>:8443/payments/payment/"    
        - name: AUDIT_ENABLED
          valueFrom:
            configMapKeyRef:
              name: "think-blue-demo-ConfigMap"
              key: AUDIT_ENABLED
        resources: {}
        volumeMounts:
        - mountPath: /var/log/audit
          name: shared
      restartPolicy: Always
      volumes:
      - hostPath:
         path: /run/systemd/journal
         type: ""
        name: journal
      - emptyDir: {}
        name: shared
status: {}

Note: icp-audit-service is at first position. The /var/log/audit volume is mounted and shared between application container and audit sidecar container. AUDIT_ENABLED flag is imported from the ConfigMap file.