What do I do when my system posts a "Time synchronization to primary or Console has failed" system notification?
The Console appliance is responsible for maintaining time synchronization for all managed hosts in the deployment. Every 10 minutes, managed hosts request a synchronization to the Console time and if the rdate (TCP/UDP 37) request for time update is unsuccessful, then a system notification is generated to administrators. Time synchronization to the Console is critical to QRadar. Without time synchronization searches, reports, and offenses might not complete successfully or return the expected data.
NOTE: The time synchronization interval is not customizable in QRadar and multiple system notifications for time synchronization should be reviewed by administrators. Administrators investigating time synchronization issues should watch internal firewall traffic for denied communication on TCP/UDP port 37 between the QRadar Console and other managed hosts. Administrators should also review and monitor for network degradation issues and attempt to resolve this notification quickly due to the potential impact to users.
The error message defined in /var/log/qradar.log shows a failure of a managed host to synchronize with the Console or the primary appliance.
June 26 11:20:11 127.0.0.1 [ERROR] [NOT:0150003100] Time Synchronization to Console has failed - rdate: timeout
The event itself from the log file will show the loopback IP address, however, the system notification on the QRadar Dashboard will show the actual IP of the managed host that failed to synchronize.
The managed host is either port blocked or configured to synchronize with a time server that is not the Console. It is required that all managed hosts synchronize to the Console to ensure that searches, reports, and offenses complete properly.
QRadar 7.2.0 and above.
Resolving The Problem
To resolve this issue, the administrator can review the following options:
- To verify that a firewall is not blocking data on port 37, SSH to the remote host and try to connect to the Console over port 37. This can be done using netcat. To connect to the Console from a managed host over port 37, type the following command: nc -zv 172.16.77.35 37
If the connection is successful, then you know that TCP port 37 is open.
- Verify that port 37 TCP/UDP (xinetd) is listening on Console. Some QRadar Forensics systems do not have port 37 open by default in IP tables.
- Review time difference between the Managed Host and the QRadar Console. This can be done using the date command on both appliances manually or by using the all_servers script in the support folder.
For example, to list the time on all QRadar appliances, from the root directory (/) type ./opt/qradar/support/all_servers.sh -C "date"
Optionally, this command can also be run for a simplified view:
- To run time synchronization on all hosts and see if any fail to synchronize with the Console, from the root directory (/) type the following command: ./opt/qradar/support/all_servers.sh "/opt/qradar/bin/time_sync.sh"
- Restart the tunnelrdate service on Managed host (if encryption is used on the host) and xinetd service on the Console with the following commands:
- To restart the tunnel service, type: ./opt/qradar/init/tunnel restart tunnelrdate
- To restart the xinetd service, type: service xinetd restart
After services have restarted, the administrators can clear the system notification 'Time Synchronization to Console has failed' and see if the notification is regenerated. The time synchronization is checked on 10 minute intervals.
Where do I find more information?
If you have additional questions or some of this content is not clear, you can see the QRadar forum or contact customer support:
Internal Use Only
This technote was generated by Technote Kickstart 126.96.36.199 based on Internet Security Systems PMR 23222,442,000.
View the associated PMR's text via Wellspring at: http://eclient.lenexa.ibm.com:9082/DocFetcher/source/PMR/23222.442.000%20O15/03/19
10 May 2019