Netcool deployment scenarios in Hybrid Cloud Environment

Body

In this article I will review possible scenarios and supplementing details for deploying Netcool/Omnibus in a Hybrid Cloud Environment.

Many customers already have Netcool/Omnibus deployed in on-premises environment independently or as part of Netcool Operations Insight. Once they start using Public Cloud to host some applications, the next step would be to be able to bring alerts from Public Cloud applications to the on-premises NOI.

Scenario 1: Direct connectivity from public cloud event sources to on-premises Netcool deployment

This scenario assumes that Security and Networks teams permitted and configured connectivity from public cloud event sources to the on premises Netcool/Omnibus probes. Event Sources in this case could be Monitoring systems such as Prometheus, Management system such as Microsoft OMS, or application logs.

Security and Network teams would have to configure entry point for event sources which will forward them to a Netcool/Omnibus Probes.

This scenario is the easiest to deploy, but it relies on the ability of Security and Network teams to allow connectivity from the public cloud.

Scenario 2: Direct connectivity from Netcool/Omnibus Probes deployed in public cloud to on-premises Netcool

This scenario rely on the same assumption as the previous one - Security and Network teams permit and configure the connectivity from public cloud to the on premises Netcool/Omnibus Object Server.

In this scenario Netcool/Omnibus Probes are deployed in Public Cloud either on assigned Virtual Machines or as Kubernetes pods (note that as of now only Netcool/Omnibus Probe for Message Bus is available for kubernetes deployment).

This scenario will require deployment and on-going administration and maintenance effort for Netcool/Omnibus Probes in Public Cloud.

The biggest concerns with this scenario is the same as with the previous one - Security.

Scenario 3: Using DMZ to deploy Collector layer Netcool environment

This scenario assumes that Security and Network teams require all connectivity to and from public networks to pass through the DMZ environment.

A dedicated server or shared server resources will be required to install Netcool/Omnibus and deploy and configure Netcool/Omnibus Collector tier Object Server.

Usually DMZ rules allow only inbound connectivity from on-premises environment to DMZ resources - Netcool/Omnibus Collector layer gateway will have to be deployed on on-premises server.

This scenario allows Netcool/Omnibus Probes to be deployed either on Public Cloud or in the DMZ.

This scenario is more secure and has better chances to be approved by Security and Networks teams.

This scenario, however, has a big operations, administrations and maintenance overhead to maintain Collector layer Object Server.

Scenario 4: Using DMZ to deploy Netcool/Omnibus Firewall Bridge Server

Netcool/Omnibus Firewall Bridge Server is the Netcool component that is designed to support deployments like the one reviewed in this article.

It is important to differentiate between Netcool/Omnibus Firewall Bridge Server and Netcool/Omnibus Proxy Server. First one serves as a tunnel between Netcool components that need connectivity through firewalls. And Proxy server's only purpose is to reduce the number of connections from probes to Object Server.

In this scenario - Netcool/Omnibus Firewall Bridge server is deployed both in DMZ environment and on-premises:

- In DMZ environment it is configured in a 'Client Access' mode - presenting connection point for Netcool components that need connectivity to Object Server

- In on-premises environment it is configured in a 'Server Access' mode - initiating connections to a local Netcool/Omnibus Object Server as well as to Netcool/Omnibus Firewall Bridge server in a 'Client Access' mode in DMZ.

This configuration adheres to DMZ rule to allow only inbound connections from local network.

In this scenario - OA&M overhead is minimal in comparison to the previous one and Netcool/Omnibus probes can be deployed in public cloud as well as withing the DMZ.

Security considerations

For all scenarios described above it is important to configure Netcool/Omnibus security appropriately to prevent data from being intercepted and more importantly to prevent from false alarms to be injected.

Configuring Netcool/Omnibus components to support SSL connectivity will encrypt all data passed through network:

- Netcool/Omnibus Object Server (including Collector layer one for scenario 3) should be configured to support SSL connectivity. It is possible to set-up SSL connectivity to be used only for connections from public cloud/DMZ and unsecure connectivity for local connections.

- Netcool/Omnibus Probes should also be configured to use require data sources to send data via encrypted connections. Not all probes support secure connection from event sources, however connection from the probe to Object Server will support SSL, so decision should be made on where the probes should be deployed - in public cloud or in DMZ.

- Netcool/Omnibus Firewall Bridge server does not provide additional connection or data security capabilities. It's just serves as a tunnel from remote Netcool component (Probe) to a Netcool/Omnibus Object Server. That's why it is important to configure SSL connectivity on Object Server.

Note that Netcool/Omnibus Firewall Bridge server will always choose unsecure connection to Object Server if 'Server Access' component is pointed to the omni.dat record for Object Server that contains both unsecure and ssl ports. This issue is resolved by adding a separate record in omni.dat that describes ssl only connectivity to the Object Server.

During SSL configuration self-signed certificate or CA issued certificate should be copied to all client deployments. That serves as an authentication mechanism and should prevent the data injection. As an additional measure, Netcool/Omnibus Object Server secure mode can be enabled, requiring all connections from Netcool/Omnibus probes and gateways to authenticate to the Object Server.

Netcool/Omnibus Firewall Bride Server Deployment Example

The following describes all steps required to deploy Netcool/Omnibus in a Hybrid Cloud environment using Firewall Bridge Server.

1. Request dedicated server or resources on a shared server in the DMZ environment

2. Request connectivity from a server that will run Netcool/Omnibus Firewall Bridge Server in "Server Access" mode on premises to the dedicated or shared server requested earlier. TCP/IP port should be assigned for this connection (e.g. 10001).

3. Netcool/Omnibus should be installed on a dedicated on-premises server, or Netcool/Omnibus Firewall Bridge component should be added to Netcool/Omnibus deployment running Object Server.

4. Install Netcool/Omnibus on a dedicated or shared server in DMZ.

5. Install required Netcool/Omnibus probes either on a dedicated or shared server in DMZ or in public cloud.

6. If not already done so - configure SSL for Netcool/Omnibus Object Server - refer to the Knowledge Base.

Make sure to distribute certificate to DMZ server and any Netcool/Omnibus Probe deployments in a Public Cloud.

7. Configure Firewall Bridge Server in 'Client Access' mode in DMZ.

$OMNIHOME/etc/NCO_BRIDGE.props:

Name: 'NCO_BRCLIENT'                            # STRING (Server name)
Bridge.ClientAP.ClientPort: 10011               # UNSIGNED (The client listening port for inbound netcool clients.)
Bridge.ClientAP.Hostname: 'host2'               # STRING (The client hostname for inbound netcool clients.)
Bridge.ClientAP.ServerPort: 10002               # UNSIGNED (The server listening port for inbound bridge clients.)
Bridge.Role: 'CLIENT_AP'                        # STRING (Which role is performed by the bridge server.)
$NCHOME/etc/omni.dat:

[NCO_BRCLIENT]
{
       Primary: host2 4300                      # "Command port"
}

7. Configure Probes in DMZ or public cloud to connect to Firewall Bridge Server in 'Client Access' mode in DMZ.

$NCHOME/etc/omni.dat:

[NCOMS]
{
       Primary: host2 ssl 10011                # Port for client connections
}
8. Configure Firewall Bridge Server in 'Server Access' mode on premises.

$OMNIHOME/etc/NCO_BRIDGE.props:

Name: 'NCO_BRSERVER'                            # STRING (Server name)
Bridge.ClientAP.Hostname: 'host2'               # STRING (The client hostname for inbound netcool clients.)
Bridge.ClientAP.ServerPort: 10002               # UNSIGNED (The server listening port for inbound bridge clients.)
Bridge.Role: 'SERVER_AP'                        # STRING (Which role is performed by the bridge server.)
Bridge.ServerAP.Server: 'NCOMS_SSL'             # STRING (The name of the ObjectServer that the bridge should connect too.)

$NCHOME/etc/omni.dat:

[NCOMS_SSL]
{
       Primary: host1 ssl 4150  
}

[NCO_BRCLIENT]
{
       Primary: host2 4300
}
Note that if "Server Access" Firewall Bridge Server runs on the same host as Object Server it connects to it should use a separate omni.dat SSL only property for Bridge.ServerAP.Server. If Object Server omni.dat property is used - Firewall Bridge Server will use non-SSL connection to Object Server and all traffic from probe to Object Server will be insecure.

$NCHOME/etc/omni.dat: (If Firewall Bridge Server runs on the same host as Object Server)

[NCOMS]
{
       Primary: host1 4100 ssl 4150
}
[NCOMS_SSL]
{
       Primary: host1 ssl 4150  
}
[NCO_BRSERVER]
{
       Primary: host1 4300
}
[NCO_BRCLIENT]
{
       Primary: host2 4300
}
8. Start all components

 

Tips

- Connectivity between Firewall Bridge components is initiated from "Server Access" Firewall Bridge Server running on premises to "Client Access" Firewall Bridge Server running in DMZ. Data is flowing in opposite direction - from "Client Access" Firewall Bridge Server running in DMZ to "Server Access" Firewall Bridge Server running on premises

- Firewall Bridge Server by itself does not provide any means to encrypt the traffic - it's just passes all data from clients connected to "Client Access" Firewall Bridge Server running in DMZ to Object Server that "Server Access" Firewall Bridge Server running on premises connected to.

It is important to configure SSL for Object Server so data is encrypted.

- "Server Access" Firewall Bridge Server running on premises should use SSL only connection entry in omni.dat. It should not use 'listening' Object Server record in omni.dat that contains both SSL and non-SSL ports.

- Tcpdump (run as root) can be used to validate that traffic is encrypted. Examples:

On DMZ node:

tcpdump -A -i lo port 10011   # to monitor the traffic on "Client Access" bridge listening port - from probes running on the same server

tcpdump -A -i eth1 port 10011   # to monitor the traffic on "Client Access" bridge listening port - from probes running on other hosts or in public cloud

tcpdump -A -i eth1 port 10002   # to monitor the traffic from "Client Access" bridge to "Server Access" bridge on DMZ side

On on premises side:

tcpdump -i eth1 -A port 10002   # to monitor the traffic from "Client Access" bridge to "Server Access" bridge on on premises side

tcpdump -i lo -A port 4150      # to monitor the traffic on Object Server ssl port if "Server Access" bridge runs on the same server

tcpdump -i eth1 -A port 4150      # to monitor the traffic on Object Server ssl port if "Server Access" bridge runs on other server
tcpdump -i lo0 -A port 4100      # to monitor the traffic on Object Server non-ssl port if "Server Access" bridge runs on other server

 

If all traffic is encrypted - the the following entries will be logged:

tcpdump -A -i eth1 port 10002
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
10:35:06.855818 IP host2.documentum > host1.54228: Flags [P.], seq 1049560419:1049560738, ack 4003248203, win 379, options [nop,nop,TS val 2608066558 ecr 2608065382], length 319
E..sE.@.@.*.    ...     ...'...>..c...K...{.......
1......z.^Y.|:....z@T.u....)............i..I.,.v.....o7v.;.....U..J.......n..6.LW............rt.G.^........u_.H.        .j......=...."~F.l.l..A..i..o2.{...&iv.c.........n...o......v...
...(/...].S...F....eS|..V........#.....=1...X..Lx........=<.8...3-.....Y.<.....b. ....gT..'.Vr~
10:35:06.857300 IP host1.54228 > host2.documentum: Flags [P.], seq 1:56, ack 319, win 1424, options [nop,nop,TS val 2608066383 ecr 2608066558], length 55
E..k..@.@.h.    ...     .....'....K>..............
.s.O.s......2................;|.f..p.....O.JU..
.ev;..}.+a....y

If "Server Access" Firewall Bridge Server running on premises fails to connect to Object Server using SSL port - it will be visible on all tcpdump commands. E.g.:

tcpdump -A -i eth1 port 10002
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
10:48:50.055370 IP host2.documentum > host1.54262: Flags [P.], seq 1802559813:1802560127, ack 1617935054, win 354, options [nop,nop,TS val 2608889758 ecr 2608888580], length 314
E..n.;@.@...    ...     ...'...kp.E`o.....b.......
......{....:....!-....insert into alerts.status values ('BeijingMachineStats3Stats',0,'Beijing','','Simnet Probe','MachineStats','Stats','76% full',3,'Diskspace alert',0,1544726930,154
4726930,0,0,0,0,3300,0,'',65534,0,0,0,'',0,0,0,'','',0,0,'',0,'',0,0,'',0,0,'','','','','','','','','',0,0,'','','NCOMS',0,'','',0,0,'');

10:48:50.056381 IP host1.54262 > host2.documentum: Flags [P.], seq 1:27, ack 314, win 1420, options [nop,nop,TS val 2608889582 ecr 2608889758], length 26
E..Nd.@.@...    ...     .....'.`o..kp.............
..~...............................