Administering BulkLoader

To import network resources into the database, use the BulkLoader utility. You execute the BulkLoader utility by specifying the BulkLoader keyword when executing the icosutil utility.

BulkLoader utility overview

The BulkLoader utility populates the database with a set of resources, so that normal configuration management tasks can begin in ITNCM - Base.

The BulkLoader utility reads a CSV file that contains a list of network resources to import. It then uses the API to create placeholders for each resource and imports the current configuration of each. See the ITNCM API Guide for information about how the API creates resources and imports configurations.

Note: When using the BulkLoader and SSH, the Resource Access Document (RAD) needs to be predefined at the realm level.

Note: Use the BulkLoader utility only when ITNCM - Base is stand alone. Do not use the BulkLoader utility when integrated with ITNM.

Update Sequence

When invoked, the BulkLoader utility selects a record from the specified CSV file and reads the database, matches on the host name and VTMOS, and if found, updates the access method(s). If the host name is not found in the database, the BulkLoader utility attempts to match on realm. If the sub-realm under existing root realm is not found, and if the -cr parameter is specified, then only it is added. Next the BulkLoader utility adds the resource to the database and then attempts to import the resource.

The significance of this sequence (update if present and add if not present) means you can use the BulkLoader utility to make security access changes to multiple devices that are already present in the database.

Data file formatting rules

The accepted format for the file is CSV, in which each row provides resource information. The data file imported by the BulkLoader utility must comply with the following format:

NCMrealm,10.219.1.34,Cisco,Router,26*,12.2-*,"telnet,none,true,testuser1,
password1,enable1,none,23","ssh,sshtype1,true,testuser1,password1,
enable1,2600e,22"NCMrealm,10.219.1.35,Cisco,Router,26*,12.2-*,
"telnet,none,true,testuser2,password2,enable2,none,23","ssh,sshtype2,
true,testuser2,password2,enable2,2600d,22"NCMrealm,10.219.1.36,Cisco,
Router,26*,12.2-*,"telnet,none,true,testuser3,password3,enable3,none
,23","ssh,sshtype3,true,testuser3,password3,enable3,2600c,22"

To import multiple network devices the 'range' keyword must be used. In the following example all host names between 10.219.34.1 and 10.219.34.21 are specified:

NCMrealm,range:10.219.34.1-10.219.34.21,Cisco,Router,26*,12.2-*,
"telnet,none,true,testuser,password,enable,none,23",
"ssh,sshtype,true,testuser,password,enable,2600e,22"

Or alternatively, the range can be specified using a subnet suffix:

NCMrealm,range:10.219.34.1/24,Cisco,Router,26*,12.2-*,"telnet,
none,true,testuser,password,enable,none,23","ssh,sshtype,true,
testuser,password,enable,2600e,22"

It is critical that the VTMOS be properly defined for the BulkLoader entry to match the support list for that device. Otherwise, you will invoke autodiscovery and no device-level RAD will be created. The following list provides the rules for specifying VTMOS:

If a specific device is given you must have values for VTMOS.
If a range of IP addresses is used you can use an '*' to specify VTMOS.

The following list provides the rules for access data:

The date file for the import must follow the format specified.
Optional: If the access type is not specified, the default access type is used.
No access type specification is required for importing with a range of IP addresses.

Note: The field in the RAD socketConnectTimeout provides the connect timeout in milliseconds. This is used to control the socket connection timeout for Auto-Discovery.

CSV file description

ITNCM - Base supports fallback methods to communicate with each resource, as well as different user names and passwords on a per resource basis. All remaining data in this table are optional.

If login credentials are not included for a particular device, the BulkLoader utility will use authentication or RAD objects already present. The utility will also default to using telnet as the communication protocol with no fallback access method.

ITNCM - Base provides three access methods: TELNET, SSH, and alt-telnet or console. The order of the attempts is defined on a left to right basis within the data file.

The following table describes the information that must be contained within the CSV file:

Column	Description
realm	Specifies the location where the resource will reside in the database. The realm is defined by creating a path name that would exist under the main realm that was created during the ITNCM - Base installation process. This is a required field.
host name	Specifies the name of the resource being imported. This name must be resolved by a DNS server or it must be logged in the host file for the server. ITNCM - Base will use the host name for device communication. This is a required field.
vendor	Specifies the vendor name for the resource. This is a required field. Note: For the VTMOS fields, the system validates that you have provided supported values, but you still must be sure that the values are accurate for the resource you are importing.
type	Specifies the type of resource (router, switch, or firewall). This is a required field.
model	Specifies the model number of the resource. This is a required field.
OS	Specifies the version of OS running on the resource. This is a required field.
access-type	Specifies a title for the access method; it is just a descriptor placed in the access script that is defined for the resource. If the access-type defined in the data file is not the same as the default access script, the resource will use the new method.
ssh-type	Specifies a flag that determines the method used to access the resource. Select one of the following options: none — Uses TELNET. ssh1 — Uses the des cipher, which is acceptable for CISCO resources. ssh2 — Uses blowfish cipher, which is acceptable for Juniper resources.
streaming	Specifies a flag that indicates whether streaming data will be used. Select one of the following options: True — Streaming data will be used. False — ftp and/or tftp will be used.
username	Specifies the username that will be used to log onto the resource. This username needs to belong to a group with appropriate privileges to modify the resource.
password	Specifies the password associated with the username from the previous column.
enable password	Used for resources to allow modifications to the resource.
alt-hostname	Specifies an optional parameter that is used for access to the device through a console server.
port	Specifies the port that will be used to communicate with the resource.

Each column must be separated by a comma. Values can be enclosed in single or double quotes. All values that are not quoted must be composed of letters (lower or uppercase), numbers, dollar signs ($), periods (.), parentheses, dashes (-), and underlines (_). Comment lines begin with a # (pound sign).

The text file may use any other characters, but the entire value that includes the non-supported character must be enclosed in single or double quotes.

The last eight security access information values (starting with com-type) may be repeated, which is indicated by the double quote marks in the header row. Being repeatable means that access information for both SSH and telnet can be entered within the same record.

If a value does not exist for one of the fields, make sure to use a space between the commas. When viewed in a spreadsheet, each set of eight repeating values will occupy a single cell, as shown in the following example:

com-type,ssh-type,streaming,username,password,enable-password,alt-hostname,port
telnet,none,true,go,go,go, ,23
telnet,none,true,go,go,go, ,23
telnet,none,true,go,go,go, ,23
telnet,none,true,go,go,go, ,23
telnet,none,true,go,go,go, ,23
telnet,none,true,go,go,go, ,23

Client setup

To run the BulkLoader utility from a client machine, the client setup procedures as described in the ITNCM Installation Guide must be performed for the operating system that is running on the client machine.

Syntax

icosutil BulkLoader {required parameters) (optional parameters}

Parameters

BulkLoader

Specifies the keyword that instructs the icosutil utility to run the BulkLoader utility.

-u

Specifies a required option when using the Encrypt keyword.

required parameters

Specifies one of the following required parameters:

Required parameter	Description
-l	Specifies a valid username.
-p	Specifies the password associated with the username being used. This should be clear text.
-f	Specifies the path and file name of the data file containing the list of resources to import.

optional parameters

Specifies one of the following optional parameters:

Optional parameter	Description
-server	Specifies the hostname or IP address of the ITNCM - Base server to which you are connecting.
-cu[realm]	Specifies a command that enables network resources to be created where the VTMOS = unknown/unknown/unknown/unknown. This is in the event that the import fails. By default it will create the network resource in the realm specified in the CSV for the import. If a realm is specified, the unknown Network Resource will instead be created in the realm.
-port	Specifies the port number to use. 16310 is the default port for the ITNCM - Base server, but this can be changed at installation time. This user must belong to a group with add rights to the realm in which you are adding the resources, as well as add rights to resources in that realm.
-cr	Specifies that you want to create the realm (under existing root realm), if the realm you are importing to does not already exist under the existing root realm.
-poll	Specifies import status polling
–help	Specifies usage information about the `BulkLoader` utility.
-update	Updates the existing RAD.

Sample

The following example specifies a sample command line for the BulkLoader utility:

icosutil BulkLoader -server 1.2.3.4 -port 80 -l name -p pwd -f /home/icosuser/
list.csv -cr

The icosutil utility would return the following:

Auto discovery

The BulkLoader utility provides an autodiscovery function that loads devices even when you are not entirely sure what is on your network. The utility “discovers” the Vendor, Type, Model, and Operating System (VTMOS) of each host and loads it into ITNCM - Base.

The following example shows how to use the autodiscovery function that the BulkLoader utility provides:

	A	B	C	D	E	F	G	H
1	#Realm	Hostname	Vendor	Type	Model	OS	Access Method 1 (optional)	Access Method 2 (optional)
2	IPX01-customer-A	sales_lab_2600-2	Cisco	Router	26*	12.2	telnet,none,true,goo, go,go,none,23	telnet,none,true,cisco, cisco,cisco,none,23
3	IPX01-customer-A	sales_lab-7	Cisco	Router	26*	12.2
4	IPX04-customer-B	range:10.217.200.1/24	*	*	*	*
5	IPX04-customer-B	range:10.219.34.10-10.219.34.20	*	*	*	*

The following list describes the example:

Sales_lab_2600-2 will be imported as a Cisco/Router/26* regardless of its true type. It uses custom primary and secondary access methods.
Sales_lab-7 will be discovered using standard authentication resource with values for VTMOS. Primary and secondary access methods are not required.
The 10.217.200.1 subnet will be discovered using standard authentication resource and VTMOS as ‘*'. Primary and secondary access methods are not required.
Hosts 10.219.34.10 through 10.219.34.215 will be discovered using standard authentication methods. The system will find the access method for each device it attempts to import.

Results of the BulkLoader utility

After you invoke the utility, the program parses the text input file you are passing in. If any syntax errors are found, the program will return the input line in which the error was found. It then continues processing the records from the data file.

The following example shows the results of a syntax error. In this case, the Vendor name is incorrect.

Invalid Vendor, Type, Model or OS: 2600f/Cico/Router/26*/12.2-* - On line 3

If a sub-realm name in text file cannot be found, that sub-realm will be created under the given root realm, as long as you specified the -cr attribute when invoking the command. If the sub-realm does not exist under the given root realm and you did not use the -cr option, then the BulkLoader utility returns the following error:

Realm ITNCM/realm1 not found skipping: 2600e/Cisco/Router/26*/12.2-* - On line 2

If the resource being imported already exists in the ITNCM - Base database, the resource is updated with the VTMOS information from the data file.

Updating: 2600b/Cisco/Router/26*/12.2-* - On line 4

If the input file is in the correct format, a unit of work (UOW) consisting of an import is submitted for each line of the text file, meaning each resource is imported as a separate UOW. Each UOW will automatically begin processing unless approvals are required first. If approvals are required, you won't be returned to the command prompt until after each UOW has been approved.

When finished, the utility shows a summary that includes the number of sucessful and failed import submittals.

Importing: 2600c/Cisco/Router/26*/12.2-* - On line 2
Updating: 2600k/Cisco/Router/26*/12.2-* - On line 3
Updating: 2600h/Cisco/Router/26*/12.2-* - On line 4
Imported: 2600c/Cisco/Router/26*/12.2-* : Result - SUCCESS