Node time not in sync

Follow this procedure if the system nodes time is drifting apart.

Time sync issues might break the upgrade process.

If the time is not in sync on the system nodes, the following alert might also be raised:
412: Node time not in sync
If you suspect time sync issues:
  1. Run the following command on all three control nodes:
    Example healthy output:
    [root@e1n1 ansible]# /opt/ibm/appliance/platform/apos-comms/tools/
    No upstream time sources defined
    Time currently syncing from e1n1
    Time offset healthy. Value:0.000000013
    Example unhealthy output:
    [root@e1n2 ~]# /opt/ibm/appliance/platform/apos-comms/tools/
    Configured timesource has not been accepted as a time source by chrony. It is NOT used for time source
    Time currently syncing from e1n2.fbond
    Time offset healthy. Value:0.000001082
  2. If the time server is unreachable, or having issues, revert the time_servers entry in System_Name.yml to the template default, as in the following example:
        time_servers: ["<OPTIONAL>"]
  3. Rerun the time setup playbook:
    cd /opt/ibm/appliance/platform/apos-comms/customer_network_config/ansible
    ANSIBLE_HASH_BEHAVIOUR=merge ANSIBLE_LOG_PATH=/var/log/appliance/platform/apos-comms/house_setup.log ansible-playbook -i <System_name.yml> playbooks/set_upstream_time.yml

For more information on setting up time servers in the yml file, see Common network configuration values. Note that setting up external time source is strongly discouraged unless it is required in your system setup.