Tweet

Hi, 

 

This article describes how you can easily add a new topology layer to IBM Tivoli Network Manager (ITNM) via the collector framework without having to use stitchers.  ITNM's collectors provide a way of incorporating topology data into the discovery process from files, Element Management Systems, databases and so on. Collectors can also work in-parallel with ITNM's SNMP, Telnet & SSH based discovery capabilities that are typically used to obtain data directly from devices. ITNM v3.9 FP2 was used for this article.

 

The scenario described is one where you have the topology data you want to use already held in an external system and that you're able to create CSV text files containing the data needed by ITNM's Generic CSV collector. The business value of adding new topology layers to ITNM is that you can, for example, expedite the resolution of problems by visualising events in-context of resource and relationship information that would otherwise be outside of what ITNM provides out of the box.

 

In this article, we've got a fictional network topology layer, MyBusinessLayer, that we want to model and we want to do it without having to write any ITNM stitchers. The secret behind this technique is (a) to ensure the CSV files contain the new topology layer adjacency data and (b) to configure ITNM to directly map those adjacencies to the new topology. This technique is only really suitable if the source of topology data gives you the adjacencies explicitly.

 

The first thing to do is ensure that the CSV files contain the new adjacency data that we'll map to the topology later on. An easy way to do this is to store the new adjacency data within ITNM's "ExtraInfo" freeform data field but how can the collector do that given the CSV format is very tabular in nature?  The answer is via an XML fragment that describes the additional data to store in ExtraInfo for the record described by the line in the CSV file. In this article, I'll use interfaces.csv which describes ports and interfaces the collector should discover.

 

Here's an example line, where the columns defined in the line are:-

Note the XML fragment that's mapped to extraInfo at the end of the line and, in particular, the multiple <m_MyBusinessLayer> elements.  

device1,1,GigabitEthernet1,6,C3:22:B4:35:90:C0,Gi1,11.11.2.10,1,11.11.2.8,255.255.255.252,<m_IfAlias>Connection to other devices</m_IfAlias><m_MyBusinessLayer></m_MyBusinessLayer >< m_MyBusinessLayer >device2[ Gi1 ]</m_MyBusinessLayer >< m_MyBusinessLayer >device3[ Fa2/2 ]</m_MyBusinessLayer >

 

The <m_MyBusinessLayer> elements represent the relationships to other records in the resulting topology, regardless of whether they're discovered by the collectors or directly from the devices. There are two things to ensure:- 

If you're familiar with Perl, a handy way of creating the necessary XML fragment format is as follows. The first method fragment would be called when you create the adjacencies. It pushes a dummy adjacency and the real one into an array reference called "interfaces" within the hash called scenario. It then uses the extraInfoToXML method below to convert the Perl hash to an XML fragment. There are, of course, efficiency gains to be had here but it serves as an example of the approach.

 

sub addMyBusinessLayerLink

{

        my ($fromDevice,$fromInterfaceIndex,$toDevice,$toInterface) = @_;

        if($fromDevice && $fromInterfaceIndex && $toDevice && $toInterface)

        {

                my $entityName = $toDevice."[ ".$toInterface." ]";

                push( @{ $scenario{$fromDevice}{interfaces}{$fromInterfaceIndex}{extraInfo}{m_MyBusinessLayer} } , "" )

                        unless $scenario{$fromDevice}{interfaces}{$fromInterfaceIndex}{extraInfo}{m_MyBusinessLayer};

 

                push( @{$scenario{$fromDevice}{interfaces}{$fromInterfaceIndex}{extraInfo}{m_MyBusinessLayer} } , $entityName );

                $scenario{$fromDevice}{interfaces}{$fromInterfaceIndex}{extraInfoString}

= extraInfoToXML($scenario{$fromDevice}{interfaces}{$fromInterfaceIndex}{extraInfo});

        }

} 

 

sub extraInfoToXML

{

        my $exref = shift;

        my $xs = new XML::Simple( ForceArray => 1);

        my $xml = $xs->XMLout($exref, NoAttr => 1, NoIndent => 1, RootName => '', KeyAttr => [] );

        return $xml;

}

 

Now you've got the CSV file created, we need to tell ITNM to map the adjacencies stored in the <m_MyBusinessLayer> elements to the new topology layer.  Follow these steps...

 

1. Create a new entityType in ITNM's NCIM database for the new topology layer.  Use your favourite database client or ncp_oql to do this. For example, use entityType 90 for the new layer (check you are not using it first!).

 

ncp_oql -domain ATC -username ncim -service ncim -password ncim -query "insert into ncim.entityType (entitytype,typename,dbtable,metaclass,managername) values (90,'MyBusinessLayer','entityData','Topology','PrecisionIP');" 

 

2. Configure ITNM's ncp_model process to map the new ExtraInfo field to the new topology layer by editing the $NCHOME/etc/precision/ModelNcimDb.cfg file. (or your domain-specific version of it). 

 a. Add an insert to tell ncp_model about the new topology layer, the 606 is an example, it should just increment from previous entries - it just allows model to know about the layer in subsequent configuration.

 

b. Modify the entityData insert to reference the new field, for example add the following:-

 

The whole entry may then becomes as follows. This tells ncp_model to process any entity record and create the adjacencies for them.

insert into dbModel.entityMap

(

    EntityFilter,

    TableName,

    DisplayLabel,

    FieldMap,

    Relationships

)

values

(

    "1=1",

    "entityData",

    "eval(multibyte, 'LOOKUP(`m_DisplayLabel`, &ExtraInfo, &EntityName)')",

    {

        entityId            = "eval(int,  '&ObjectId')",

        entityName          = "eval(text, '&EntityName')",

        mainNodeEntityId    = "eval(int,  '$mainNodeEntityId')",

        entityType          = "eval(int,  '$entityType')",

        createTime          = "eval(text, 'TIMESTAMP(&CreateTime)')",

        changeTime          = "eval(text, 'TIMESTAMP(&ChangeTime)')",

        displayLabel        = "eval(text, '$displayLabel')",

        description         = "eval(multibyte, '&Description')",

        cdmAdminState       = "eval(int,'LOOKUP(`m_CdmAdminState`, &ExtraInfo, `0`)')",

        manual              = "eval(int, '0 + &Manual')"

    },

    {

        //

        // Append the desired connection type to the connects

        // table name using & as a sepatator character

        //

        connects&1 = "eval(list type text, '&RelatedTo')",

        connects&2 = "eval(list type text, '&ExtraInfo->m_RouterLinks')",

        connects&10 = "eval(list type text, '&ExtraInfo->m_PWData(0)->m_PWFarEndInterface')",

        connects&11 = "eval(list type text, '&ExtraInfo->m_OSPFAdjacencies')",

        connects&600 = "eval(list type text, '&ExtraInfo->m_MplsTeLinks')",

        connects&601 = "eval(list type text, '&ExtraInfo->m_PIMAdjacencies')",

        connects&603 = "eval(list type text, '&ExtraInfo->m_LocalVlanTopology')",

        connects&604 = "eval(list type text, '&ExtraInfo->m_IPMRouteAdjacencies')",

        connects&605 = "eval(list type text, '&ExtraInfo->m_PWData(0)->m_VPLSPWFarEndInterface')",

        connects&606 = "eval(list type text, '&ExtraInfo-> MyBusinessLayer')"

    }

);

 

3. Ensure the Collector drive file has a mapping for extraInfo by editing the /opt/IBM/tivoli/netcool/precision/collectors/perlCollectors/GenericCsv/exampleCsv.drv  (driver) file or your version of it.  For example, my entry for InterfaceData has the following entry to map the 11th item of the comma-delimited line to ExtraInfo:-

 11 =>

            {

               Name => 'extraInfo',

               Description => '',

               MapsTo => 'extraInfo'

            }, 

 

4. Finally, tell ITNM's GUIs about the new topology layer so you can use it in the hop view and filters etc. Edit the ncp_layertypes.properties file (and those for your locale) by editing the appropriate files in the /opt/IBM/tivoli/netcool/precision/profiles/TIPProfile/etc/tnm/locale directory. My file now reads:- 

 

# NLS_MESSAGEFORMAT_NONE

# NLS_ENCODING=UNICODE

# The word connectivity. should be prefixed to each of the layer's entityId

# Add only those layers which u want to display in GUI. For ex: You can skip entityType 77 and 78

connectivity.107:Layer 1

connectivity.72:Layer 2

connectivity.73:Layer 3

connectivity.81:PIM

connectivity.83:IPMRoute

connectivity.-1:IP Subnets

connectivity.default:Default

connectivity.5736254:No Connections

# article demo

 

 

Now restart ITNM, run a discovery and see if your layer is populated. Verify the OQL record in ncp_model has your new adjacencies, for example:-

ncp_oql -domain ATC -username admin -service model -query "SELECT * FROM master.entityByName WHERE EntityType = 2 AND ExtraInfo->m_MyBusinessLayer IS NOT NULL;" 

{

        EntityName='device1[ Gi1 ]';

        Address=['','98:47:D8:2F:AF:31','22.22.0.14'];

        EntityType=2;

        EntityOID='1.3.6.1.4.1.foo';

        Status=1;

        IsActive=1;

        ExtraInfo={

                m_IfIndex=1;

                m_IfDescr='GigabitEthernet1';

                m_IfType=6;

                m_IfName='Gi1';

                m_LocalNbrPhysAddr='98:47:D8:2F:AF:31';

                m_IpAddress='22.22.0.14';

                m_IfOperStatus=1;

                m_Subnet='22.22.0.12';

                m_SubnetMask='255.255.255.252';

                m_IfHighSpeed='1000';

                m_MyBusinessLayer=['device2[ Gi1 ]','device3[ Gi1 ]','device4[ Gi1 ]'];

                m_IfSpeed='1000000000';

                m_IfAlias='Connection to switch3 port GigabitEthernet3';

                m_IPData=[

                {

                        m_IpAddress='22.22.0.14';

                        m_Protocol=1;

                        m_Subnet='22.22.0.12';

                        m_SubnetMask='255.255.255.252';

                        m_NetMaskBits=30;

                }

                ];

                m_CdmAdminState=0;

                m_DisplayLabel='[ Gi1 ]';

                m_HasMainEntityIp=1;

                m_BaseName='device1';

                m_AddressSpace=NULL;

                m_AccessProtocol=NULL;

                m_RouterLinks=['switch3[ Gi3 ]'];

                ReverseRelatedTo=['switch3[ Gi3 ]'];

        };

        UpwardConnections=['device1'];

        LingerTime=3;

        ActionType=0;

        CreateTime='2013-01-30 17:01:38';

        ChangeTime='2013-02-05 15:12:25';

        ClassName='Foo';

        ClassId=2117840861;

        ObjectId=4693;

} 

 

 

Once the new topology layer is populated, you will now be able to use it in filters, the hop view and so on. 

hope this helps, 

Matt