Setting up Remote Flow Collector

Optionally, install the Flow Collector Service on a remote host as a stand-alone installation to keep the collector closer to your data center. Remote Flow Collector sends captured flow packet to flow collector that uses Kafka messaging.

Before you begin

You can install Kafka version 2.4.1 on the server where you want to install Remote Flow Collector or use other Kafka cluster if available.

Make sure you have Java™ SDK v8 on the node where you are installing the Remote Flow Collector.

Procedure

  1. Download and extract the IBM® Telco Network Cloud Manager - Performance, version 1.4.6 English Multiplatform Advanced package that has M0HSWEN.tar.gz file.
    Download and extract the file to <DIST_DIR>.
  2. Copy and extract the /remote/basecamp-remote-flow-collector-2.4.5.0.tar.gz file to a location of your choice.
    You can see the following files and folders:
    • bin
    • conf
    • etc
    • lib
    • work
  3. Edit the settings in /conf/application.conf file.
    Parameter Description Default value
    collector.flow.udp.ports The UDP port from where the Flow Collector is listening. 4379
    collector.flow.sctp.ports The SCTP port from where the Flow Collector is listening. 4381
    main.zk-url URL with Zookeeper node IP address with port number http://<zookeeper_IP_add>:2181
    messaging.kafka.broker-list Kafka brokerlist with Kafka node IP address with port number <kafka_IP_add>:9092
    messaging.kafka.zk-connect Zookeeper node IP address with port number <zookeeper_IP_add>:2181
    remote.id Unique ID for the Remote Flow Collector rfc1
    remote.context Domain name of the Remote Flow Collector node.
    It is required to append the domain name to the interfaces. Currently, it is used for Flow interfaces only to avoid overlapping of IP addresses from different domains that can be uniquely identified with domain name that is appended to the IP addresses.
    Note: The remote.context parameter is introduced to represent a domain.
    		 "MY"
    remote.topic The name of the Kafka topic to which the records that are collected from the Remote Flow Collector are written to. <remote.flow.topic>
    remote.cluster.collector.protocol The cluster Flow Collector Service protocol or the Dashboard Service protocol if you are accessing with proxy server setup. http
    remote.cluster.collector.hosts The cluster Flow Collector Service host or dashboard if via proxy <host name>
    remote.cluster.collector.port

    Port number of the cluster where the Remoter Flow Collector is installed.

    Use the <proxy_port> if you have configured a proxy server.

    		 30040

    On OpenShift® Container Platform, the port is 443. Typically, the port of the Route, is 443, which is the default https port.

    If you have configured a proxy server, this port is the Dashboard Service port, which is 31443 in Kubernetes environment.
    remote.cluster.collector.proxy.path Proxy server path.
    Note: Optional, use this only if you have configured a proxy server.
    remote.cluster.collector.proxy.tokens Proxy server token.
    Note: Optional, use this only if you have configured a proxy server.

    In Kubernetes environment, the token can be obtained from here:
    <https://<server>:<dashboardPort>/remoteToken/>

    In OpenShift Container Platform environment, the token can be obtained from here:
    https://<dashboard-route>/remoteToken/<

    Note: Optional, use this only if you have configured a proxy server. 
    collector.flow.udp.ports = [ 4379 ]
collector.flow.sctp.ports = [ 4381 ]
main.zk-url = "http://<zookeeper_node_IP_add>:2181"
messaging.kafka.broker-list = "<kafka_node_IP_add>:9092"
messaging.kafka.zk-connect = "<zookeeper_node_IP_add>:2181"

remote.id = "remote-X"
remote.context = ""
remote.topic = "remote.flow.topic"
remote.cluster.collector.protocol = "http"
remote.cluster.collector.hosts = [<cluster-node>]
remote.cluster.collector.port = 30040
remote.cluster.collector.proxy.path = ""
remote.cluster.collector.proxy.tokens = [""]
  4. Use the following command to run bin/remote-flow-collector script: 
    cd <DIST_DIR>/remote_flow
./bin/remote-flow-collector
  5. Check the log files in the Flow Collector Pod in your cluster. 
    [INFO] [2020-11-16 11:22:48.720] [akka.tcp://flow-collector@<IP>:2554/user/basecamp-flow-collector/flow-collector/flow-collector-mgr/singleton] [flow-collector-flow-collector.dispatcher-2111] remote RemoteCollector [ <collector.flow.remote.id>,<messaging.kafka.broker-list>.<collector.flow.remote.topic>,,9223372030684775807) already registered
[INFO] [2020-11-16 11:22:48.716] [akka.actor.ActorSystemImpl (flow-collector)] [qtp85930990-46] Receiving RemoteCollector(<collector.flow.remote.id>,<message.kafka.broker.list>,<collector.flow.remote.topic>,,9223372030684775807)