Forwarded events

To better understand what forwarded events are, it is helpful to understand how to configure and set up Windows Event Forwarding (WEF).

Windows Event Forwarding basics

Windows Event Forwarding (WEF) is a powerful log forwarding solution that is integrated within modern versions of Microsoft Windows. Detailed documentation of WEF is available on the Microsoft Documentation page. The following list is a summary of WEF:

  • Windows Event Forwarding provides the ability to send event logs, either via a push or pull mechanism, to one or more centralized Windows Event Collector (WEC) servers.
  • WEF is agent-free and relies on native components that are integrated into the operating system. WEF is supported for both workstation and server builds of Windows.
  • WEF supports mutual authentication and encryption through Kerberos (in a domain) or can be extended through the usage of TLS (additional authentication or for non-domain-joined machines).
  • WEF has a rich XML-based language to control which event IDs are submitted, suppress noisy events, batch events together, and configure submission frequency. Subscription XML supports a subset of XPath, which simplifies the process of writing expressions to select the events you’re interested in.

What are the WEC server’s limitations?

Three factors limit the scalability of WEC servers. The general rule for a stable WEC server on commodity hardware is “10k x 10k,” meaning no more than 10,000 concurrently active WEF Clients per WEC server and no more than 10,000 events per second average event volume.
Disk I/O
The WEC server does not process or validate the received event, but rather buffers the received event and then logs it to a local event log file (EVTX file). The speed of logging to the EVTX file is limited by the disk write speed. Isolating the EVTX file to its own array or using high-speed disks can increase the number of events per second that a single WEC server can receive.
Network Connections
While a WEF source does not maintain a permanent, persistent connection to the WEC server, it does not immediately disconnect after it sends events. This means that the number of WEF sources that can simultaneously connect to the WEC server is limited to the open TCP ports available on the WEC server.
Registry size
For each unique device that connects to a WEF subscription, a registry key (corresponding to the FQDN of the WEF Client) is created to store bookmark and source heartbeat information. If this information is not pruned to remove inactive clients, this set of registry keys can grow to an unmanageable size over time.
  • When a subscription has more than 1000 WEF sources connect to it over its operational lifetime (lifetime WEF sources), the Subscriptions node on the Event Viewer can become unresponsive for a few minutes, but will function normally afterward.
  • At more than 50,000 lifetime WEF sources, Event Viewer is no longer an option and you must use wecutil.exe (included with Windows) to configure and manage subscriptions.
  • At more than 100,000 lifetime WEF sources, the registry is no longer readable, and the WEC server might need to be rebuilt.
Large deployment limitations
For large deployments, for example, if you deploy 40,000 to 100,000 source computers, it is recommended that you deploy more than one collector that has 2,000 to not more than 4,000 clients per collector.

Also, it is recommended that you install at least 16 GB of RAM and four processors on the collector to support an average load of 2,000 to 4,000 clients that have one or two subscriptions configured. Fast disks are recommended, and the ForwardedEvents log can be put onto another disk for better performance.

For more information, see Best practice for configuring EventLog forwarding in Windows Server.

WinCollect configuration

After you configure Windows Event Forwarding, you can configure the WinCollect agent to collect WEF events:
  • Install the WinCollect 10 agent on your Windows Event Collector (WEC) servers.
  • Configure a Windows Events (default) source and select Forwarded Events (WEF) as the channel.
  • Deploy your changes.
Note:
  • The maximum EPS supported by the Agent in a WEF environment is 10,000 EPS.
  • Although the WinCollect agent displays only a single source in the user interface, the source listens and processes events for potentially hundreds of event subscriptions. One source in the agent list is for all event subscriptions. The agent recognizes the event from the subscription, processes the content, and then sends the Syslog event to QRadar®.
  • Forwarded events are displayed as Windows Auth @ <hostname> in the Log Activity tab.

Additional information

For more information about managing large Windows Event Collection implementations, see https://www.ultimatewindowssecurity.com/webinars/watch_get.aspx?Attach=1&Type=SlidesPDF&ID=1426.

For more information about using Windows Event Forwarding to help with intrusion detection, see https://docs.microsoft.com/en-us/windows/security/threat-protection/use-windows-event-forwarding-to-assist-in-intrusion-detection.