SD-WAN Versa Collector Agents Guide
About
This documents provides details for the Versa collector agents.
Dependencies
The following table provides information about the dependencies of the Versa collector agents.
Agent | Dependencies |
---|---|
ClearAlertsAgent | - |
CreateAlertsStreamingAgent | - |
DeviceDescriptionAgent | InstallerAgent |
DeviceHealthStreamingAgent | InstallerAgent |
FlowAgent | InstallerAgent |
FlowInterfaceCacheAgent | - |
InterfaceQueueStreamingAgent | - |
InterfaceStatAgent | InstallerAgent |
InterfaceStatStreamingAgent | InstallerAgent, FlowInterfaceCacheAgent |
MetadataAgent | InstallerAgent |
ObjectDescriptionAgent | InstallerAgent, DeviceDescriptionAgent |
TunnelStatStreamingAgent | InstallerAgent |
Versa Collector Agents
ClearAlertsAgent
Clear alert from NMS on a periodic interval of 5 minutes.
Data Flow
- Get event alerts from NMS which are not closed (indicated by closed=0), and plugin name is VRE.
- Get site status alerts from NMS which are not closed (indicated by closed=0), and message is site status:up.
- Get alarm alerts from NMS which are not closed (indicated by closed=0), and message is alarmClass:cleared.
- Clear alerts from NMS with message Event cleared, Site status alarm cleared, and Alarm cleared for the event, site status, and alarms respectively.
CreateAlertsStreamingAgent
Create alerts for event, alarm, and site status syslogs and push them to NMS.
Data Flow
- Get syslogs from RabbitMQ queue alarm.
- Identify type of syslog which can be either of eventLog, sdwanSiteStatusLog, or alarmLog.
- Get device ID for which alert is generated.
- Find if old alerts are present in NMS based on device ID, alert type, plugin name, and alert generated time.
- If old alerts are found, clear them and create a new one with details such as, device ID, alert generated time, plugin name, message, origin, and severity.
- Send alerts to NMS.
DeviceDescriptionAgent
Update device description with deviceName::TenantName details.
Data Flow
- Get all devices from NMS.
- Cross reference devices with nmsDeviceInfo map stored in redis cache. This cache is populated by the InstallerAgent. If the map is not present, the agent will abort.
- Update each NMS device description with the device's name.
DeviceHealthStreamingAgent
Create indicator CPU, memory, and disk utilization data points from systemLoadLog syslogs and push them to NMS.
Data Flow
- Get syslogs from RabbitMQ queue system-load with Syslog category systemLoadLog.
- Get the CPU load, disk load, and memory load metrics from the syslog entry.
- If generateTime is given in the syslog, use it as the timestamp, otherwise use the syslog's timestamp as the timestamp for the indicator metrics.
- Create a Device Object Indicator (DOI) with the collected metrics and timestamp.
- Send this DOI batched with other DOIs to NMS via barebones' batch process.
FlowAgent
Maps flow objects to NMS objects for object-types sdwan::device-interface and sdwan::tunnel.
Data Flow
- Get MSP device group for the vendor.
- Fetch all the tenants under the MSP.
- For each tenant,
- Get NMS devices for the tenant.
- Fetch the flow device mappings for these devices.
- Fetch the flow interfaces of the mapped flow devices.
- Fetch the NetFlow view id and NetFlow field id specified in tunnel_filter_view and tunnel_filter_field configuration.
- Fetch all NetFlow filters.
- Fetch interface and tunnel object types.
- Fetch tunnel and interface indicator types for rx_octets and tx_octets.
- Fetch all indicators of these indicator types.
- Fetch existing interface and tunnel flow object mappings.
- For each indicator, create flow object mapping. Recreate mapping if already exists.
- Device mapping: map NMS devices to flow devices using flow device mapping.
- Object mapping: map interface objects to flow interfaces using ifindex.
- Map indicators using directions: incoming to rx* and outgoing to tx*.
- Add NetFlow filter to tunnel object mappings.
InstallerAgent
Get MSP tenant tree from Versa Director API and transform it to device group tree, object group tree, and polled tenants. Create the missing Device Object Indicator (DOI) types in NMS like metadata schema, device types, and object types and also create device groups, and object groups based on the tenant tree. Also, create devices based on appliances which fetched from Versa Director API.
Data Flow
- Get MSP tenant tree from Versa Director API.
- Run over the child tenants and filter only the tenants with nodeType is ORG.
- If the MSP tenant has any devices and the config enable_provider_tenant_stats is true then add the MSP tenant as a child to itself.
- Get tenant appliances and their metadata from the Versa Director API.
- Get interface stats and dynamic tunnel details for each tenant appliance from the Versa Director API.
- Get access circuits for each appliance from the Versa Director API and cache those access circuits details.
- Process the tenant tree to determine the device group tree, object group tree, and polled tenants.
- Create device group data from the tenant tree.
- If the tenant is not a leaf then process all child tenants and get device group tree, object group tree, and polled tenants and merge them.
- If the tenant is a leaf, create an object group and transform the appliance for the tenant to an object group tree.
- Set metadata from device groups.
- Transform the interface stats and dynamic tunnels.
- Transform object types based on tenant access circuits.
- Update NMS
- Create or update metadata schema for all namespaces like SDWAN_TENANT, SDWAN_SITE, SDWAN_DEVICES, and SDWAN_TUNNEL.
- Create common device types.
- Create device groups based on the device group tree.
- Create common object types.
- Create object groups and rules based on the object group tree.
- Fetch devices from tenant appliances and create them.
- Cache appliance interface info, dynamic tunnels, and polled tenants.
InterfaceQueueStreamingAgent
Reads and parses syslog sdwanAccCktCosLog message, collects and then loads all interface queue metrics, including cosEfTx, cosBeTx, cosNcTx, cosAfTx, cosEfTxDrop, cosBeTxDrop, cosNcTxDrop, cosAfTxDrop, for the parsed tenant, device, and object.
Data Flow
- Get syslogs from rabbitMQ queue cos.
- Parse rabbitMQ messages and validate key values.
- Obtain IP address from redis for parsed tenant/device (dependent on the InstallerAgent).
- For each device IP, send metrics for object type sdwan::device-interface-queue to
NMS.Note: Device health name: "device-health"
Device health description: "Device health summary"
InterfaceStatAgent
Create devices in NMS based on the appliance interface stats data associated with polled tenants.
Data Flow
- Get polled tenants from cache (populated by the InstallerAgent).
- Get the appliances of the polled tenants from the Versa Director.
- Get the interface stats from the Versa Director.
- Create a CreateDeviceDataRequest from the interface stat data.
- Create the device in NMS.
MetadataAgent
Update the metadata and SLA profiles (retrieved from the Versa Director) for each device in the polled tenants in NMS.
Data Flow
- For each tenant,
- Get the appliances of this tenant from the Versa Director.
- For each appliance,
- Get metadata for that appliance from the Versa Director.
- Get the SLA profile.
- For each appliance returned in the step above by the Versa Director,
- If setting vendor.unique_hostname_per_msp is true, set the device name to <appliance_name>. Otherwise, append the tenant name to it like <appliance_name>::<tenant_name>.
- Update the metadata of the device with this name in NMS with device and appliance information.
- If setting vendor.metadata.disable_sla_constraints is false, update the NMS metadata with the SLA attributes retrieved from the SLA profiles.
TunnelStatStreamingAgent
Populate NMS devices with time series data for sdwan::TUNNELTYPE object type by syslog processing.
Data Flow
Example monStatsLog syslog
2021-02-22T07:34:55+0000 monStatsLog, applianceName=Controller2, tenantName=SevOne, mstatsTimeBlock=1613979300, tenantId=2, vsnId=0, mstatsTotSentOctets=0, mstatsTotRecvdOctets=2089433, mstatsTotSessDuration=300000, mstatsTotSessCount=61, mstatsType=sdwan-site2site-stats, localSite=Controller2, localAccCkt=Internet, remoteSite=NewYork, remoteAccCkt=Internet, localSiteId=2, localAccCktId=102, remoteSiteId=1, remoteAccCktId=1, mstatsAttribs=
Example sdwanB2BSlamLog syslog
2021-02-22T07:25:13+0000 sdwanB2BSlamLog, applianceName=NewYork, tenantName=SevOne, localAccCktName=Internet, remoteAccCktName=Internet, localSiteId=102, localSiteName=NewYork, remoteSiteId=101, remoteSiteName=Seattle, fwdClass=fc_ef, tenantId=2, delay=0, fwdDelayVar=0, revDelayVar=0, fwdLoss=0, revLoss=0, fwdLossRatio=0.00, revLossRatio=0.00, pduLossRatio=0.00, fwdSent=167, revSent=166
- Listen in on syslogs of type monStatsLog or sdwanB2BSlamLog from the tunnel queue.
- Pull a monStatsLog or sdwanB2BSlamLog syslog off of that queue.
- Get source device from NMS with objects.
- Get destination device info from NMS using the device name.
- Prepare tunnel object.
- Format into a string an object description using site details and NMS device info.
- Get the timestamp.
- If this is a monStatsLog syslog, get the timestamp from mstatsTimeBlock or syslogTime or currentTime.
- Otherwise, if this is a sdwanB2BSlamLog syslog, get the timestamp from syslogTime or currentTime.
- Get object name from cache.
- If this is a monStatsLog syslog, get the object name from cache using the values from the syslog such as, localSite, remoteSite, localAccCkt, remoteAccCkt.
- Otherwise, if this is a sdwanB2BSlamLog syslog, get the object name from cache using the values from the syslog such as, localSiteName, remoteSiteName, localAccCktName, remoteAccCktName.
- If not found in cache, fetch interface details from the Versa director.
- Get device IP from cache. If not found, default to 0.0.0.0.
- Check if the tunnel object is already in NMS. If not, create it.
- Populate the indicators and push it to NMS.