File-based access method

The file-based access method obtains device information from a file server. Typically, a customer downloads the device information to the file server, and then Netcool Configuration Manager imports the device information from the ftp/scp server. The file is downloaded to the worker server that is processing the UOW. The worker server connects directly to the file server that contains the device info. Once the information has been retrieved, it is treated just like any other device.

NCM file-based driver setup using a single server

Overview

You can import devices using the file-based driver access method from a separate server, or you can import files (devices) from the same worker server. Using a two server approach is typical in a production environment where network policies prevent direct interaction between the Netcool Configuration Manager worker servers and the network EMS or nodes. Typically, this file server resides in a DMZ where the production devices can reach one NIC on the server while Netcool Configuration Manager can reach a separate NIC. In other situations files can be located on the same worker server and Netcool Configuration Manager transfers the file from one directory to another.

Single server implementation

You need to create a realm from the Netcool Configuration Manager UI, and within that realm you must create an authentication resource, a file transfer resource, and any network resources that will be created and imported. On the worker server that will be performing the imports, a separate directory must be created to store the files that will be imported (source files). These source files will then be transferred on the server to the destination directory specified in the File Transfer Resource.

Detailed setup process

1. Create a file directory on the worker server named /home/icosftp/single. This directory specifies the source directory for the files. Place any device configuration files in this directory ensuring that they have read permissions set as appropriate for the icosftp user.
2. Create a File Transfer Resource with the following parameters (this FTP resource specifies the destination directory for the files):

   Name

   ftpInfo

   Host

   localhost

   Username

   icosftp

   Password

   icosftp user’s password

   Path

   /home/icosftp

   Passive Mode

   checked

3. Create an Authentication Resource for the default FTP user, if it does not already exist. The following example settings use the default icosftp user:

   Username

   icosftp

   Password

   icosftp user’s password

   Enable Password

   leave blank

   Retry Count

   0 (zero)

   Retry Delay

   0 (zero)

   Ignore

   False

4. Create a Network resource specifying a Name, Vendor, Type, Model, and OS.
5. Either edit the Resource Access on the Network Resource (right-click on the Network Resource and select Resource Access) or create a Resource Access (click on File > New > Resource Access).
   Note:

   If you create a new Resource Access

   You add the file script to the resource access.

   Follow the steps described in the File-based access method topic starting with Step 3.

   If you edit the Resource Access on the network resource

   Modify the file script.

   Follow the steps described in the File-based access method topic starting with Step 10.

6. Edit the file script in the Resource Access Document (RAD) to specify the source directory for the files, as shown in the following sample.

   config.running.01.getFtpFileName=ls -t $ftpPath$/single/*$addressName$* | cut -d"/" -f5,first
   config.running.02.getSCP=$ftp_username$:$ftp_password$@$ftp_hostname$:$ftpPath$/single/$ftp_filename$
   config.running.end=$

   Note: Note the introduction of the directory name single into the path, and the change to the directory depth to 5 (was 4), highlighted.
7. Import the network device.

Writing changes to a named configuration file

You can use the NSM rest API (but not the GUI) to write or delete full device configurations or full device service configurations to named files and directories for devices to which you have file-based access. A service is a subset of the full configuration of a device. You can write and delete either device configurations or device service configurations to any one device, not both.

-
_
.
/ (for path separator only)

To write changes to a named configuration file, complete the following steps.

1. Set up file-based access as described above.
2. Create an Authentication Resource for the default FTP user, if it does not already exist. The following example settings use the default icosftp user:

   Username

   icosftp

   Password

   icosftp user’s password

   Enable Password

   leave blank

   Retry Count

   0 (zero)

   Retry Delay

   0 (zero)

   Ignore

   False

3. Create an FTP Resource to specify the destination directory for the files.
   1. Add a new entry and set the following parameters:
      • Name: ftpInfo
      • Host: localhost
      • Username: icosftp
      • Password: password of icosftp user
      • Path: /home/icosftp
      • Passive Mode: unchecked
   2. Add another entry and set the following parameters:
      • Name: altftpInfo
      • Host: address of remote host
      • Username: icosftp
      • Password: password of icosftp user
      • Path: /home/icosftp
      • Passive Mode: unchecked
4. Create a custom UOW label to use for the custom file name.
   1. Select Tools > System Properties from the Systems Manager.
   2. Set the Custom UOW label 1 property to Config_Dir_Name.
   3. Set the Custom UOW label 1 state to have the value Optional.

5. Complete either this step, Step 5, for device configuration updates, or Step 6 for service configuration updates.

   1. Write a script that generates a new configuration file. The script is run from the Resource Access device script, as shown in Step 5.b. The script must achieve the following results:

      • If you want the device to be given an initial configuration after it is first created, the script must create a new configuration file using the directory and filename that are passed to the script, if the file does not exist already, and then populate the file.
      • Do not create or populate the configuration file if it exists already for the device.
      • Do not return any data. Returned data is added to the configuration.
      • Change the permissions on the initial configuration file for the device using the chmod 777 command. The script is usually run as the icosuser, whereas FTP is usually run as the icosftp user.

   2. Edit the Resource Access that you created previously for file-based device access, select the file access type, and make the following changes:

      1. Uncheck Streaming: Put in the Transport tab.
      2. Check Update on change in the Configuration tab.
      3. Edit the device script and make the following changes:
         1. In the # Device script properties section, change putConfigFileServer to putConfigFileServer=configured_for_remote_scp.

         2. Add a new Device script removeConfigDir section:

            # New section in device script, removes all config files and directories for a device, both locally and remotely.
            removeConfigDir.01.getDeviceFileDirContent=ls $ftpPath$/$addressName$/ | cut -d"/" -f5,all
            removeConfigDir.02.removeRemoteContent=$ftp_altusername$:$ftp_altpassword$@$ftp_althostname$:$ftp_altpath$/$device_file_dir_content$
            removeConfigDir.03.exec=rm -rf $ftpPath$/$addressName$\r

         3. Add a new Device script removeConfigFile section:

            # New 'removeConfigFile' section.
            # Removes a config file for a device, both locally and remotely, this will be triggered as a result of a 'delete service' being performed.
            # 'Update on Change' should be set in the Resource access, so that changes here will be reflected in the configuration.
            removeConfigFile.01.removeRemoteContent=$ftp_altusername$:$ftp_altpassword$@$ftp_althostname$:$ftp_altpath$/$config_dir$/$ftp_filename$
            removeConfigFile.02.exec=rm -rf $ftpPath$/$addressName$/$config_dir$/$ftp_filename$\r
            removeConfigFile.03.exec=rm -rf $ftpPath$/$addressName$/$addressName$.cfg\r

         4. In the Device script config.running section, replace config.running.01 and config.running.02 in the file-based example with the following lines:

            config.running.01.exec=mkdir -p $ftpPath$/$addressName$\r
            config.running.02.exec=$ftpPath$/telus_script_for_initial_config.sh -d $ftpPath$/$addressName$ -f $addressName$.cfg\r
            config.running.03.setFileName=$addressName$.cfg
            config.running.04.setSubDirForFtpPath=$addressName$
            config.running.05.readSCPFile=$ftpPath$/$subDirForFtpPath$/$ftpFileName$

            Note: The line beginning config.running.02.exec is optional. It is provided as an example of how to set up an initial default configuration that takes effect at first import after creating a device. Replace the script name script_for_initial_config.sh with the script that you created in this step. The location of the script, and the argument names -d and -f in the preceding code can be changed.
      4. If you have a cp -i alias set up on the host, undefine the alias by using unalias cp -i, otherwise cp -f does not work in the following step.

      5. In the Device script ftp section, replace the entire ftp section in the example with the following lines:

         ftp.01.exec=mkdir -p $ftpPath$/$addressName$/$config_dir$/\r
         ftp.02.exec=mv -f $ftpPath$/$ftp_filename$ $ftpPath$/$addressName$/$config_dir$/\r
         ftp.03.exec=cp -f $ftpPath$/$addressName$/$config_dir$/$ftp_filename$ $ftpPath$/$addressName$/$addressName$.cfg\r
         ftp.04.setSubDirForFtpPath=$addressName$/$config_dir$
         ftp.05.scpFile=$ftp_altusername$:$ftp_altpassword$@$ftp_althostname$:$ftp_altpath$/$config_dir$/r

6. Complete either this step, Step 6, for service configuration updates, or Step 5 for device configuration updates.

   1. Write a script that generates a new configuration file. The script is run from the Resource Access device script. The script must achieve the following results:

      • If you want the device to be given an initial configuration after it is first created, the script must create a new configuration file using the directory and filename that are passed to the script, if the file does not exist already, and then populate the file.
      • Aggregate any service configuration files present in the directory $ftpPath$/$addressName$ up to a single device configuration file for import: $addressName$.cfg.
      • Handle any specific ordering or formatting requirements for aggregating the individual service configuration files up to a device configuration file.
      • Do not return any data. Returned data is added to the configuration.
      • Change the permissions on the initial configuration file for the device using the chmod 777 command. The script is usually run as the icosuser, whereas FTP is usually run as the icosftp user.

   2. Edit the Resource Access that you created previously for file-based device access, select the file access type, and make the following changes:

      1. Uncheck Streaming: Put in the Transport tab.
      2. Check Update on change in the Configuration tab.
      3. Edit the device script and make the following changes:
         1. In the # Device script properties section, change putConfigFileServer to putConfigFileServer=configured_for_remote_scp.

         2. Add a new Device script removeConfigDir section:

            # New section in device script, removes all config files and directories for a device, both locally and remotely.
            removeConfigDir.01.getDeviceFileDirContent=ls $ftpPath$/$addressName$/ | cut -d"/" -f5,all
            removeConfigDir.02.removeRemoteContent=$ftp_altusername$:$ftp_altpassword$@$ftp_althostname$:$ftp_altpath$/$device_file_dir_content$
            removeConfigDir.03.exec=rm -rf $ftpPath$/$addressName$\r

         3. Add a new Device script removeConfigFile section:

            # New 'removeConfigFile' section.
            # Removes a config file for a device, both locally and remotely, this will be triggered as a result of a 'delete service' being performed.
            # 'Update on Change' should be set in the Resource access, so that changes here will be reflected in the configuration.
            removeConfigFile.01.removeRemoteContent=$ftp_altusername$:$ftp_altpassword$@$ftp_althostname$:$ftp_altpath$/$config_dir$/$ftp_filename$
            removeConfigFile.02.exec=rm -rf $ftpPath$/$addressName$/$config_dir$/$ftp_filename$\r
            removeConfigFile.03.exec=rm -rf $ftpPath$/$addressName$/$addressName$.cfg\r

         4. In the Device script config.running section, replace config.running.01 and config.running.02 in the file-based example with the following lines:

            config.running.01.exec=mkdir -p $ftpPath$/$addressName$\r
            config.running.02.exec=$ftpPath$/telus_script_for_initial_config.sh -d $ftpPath$/$addressName$ -f $addressName$.cfg\r
            config.running.03.setFileName=$addressName$.cfg
            config.running.04.setSubDirForFtpPath=$addressName$
            config.running.05.readSCPFile=$ftpPath$/$subDirForFtpPath$/$ftpFileName$

            Note: The line beginning config.running.02.exec is optional. It is provided as an example of how to set up an initial default configuration that takes effect at first import after creating a device. Replace the script name script_for_initial_config.sh with the script that you created in this step. The location of the script, and the argument names -d and -f in the preceding code can be changed.
      4. If you have a cp -i alias set up on the host, undefine the alias by using unalias cp -i, otherwise cp -f does not work in the following step.

      5. In the Device script ftp section, replace the entire ftp section in the example with the following lines:

         ftp.01.exec=mkdir -p $ftpPath$/$addressName$/$config_dir$/\r
         ftp.02.exec=mv -f $ftpPath$/$ftp_filename$ $ftpPath$/$addressName$/$config_dir$/\r
         ftp.03.exec=cp -f $ftpPath$/$addressName$/$config_dir$/$ftp_filename$ $ftpPath$/$addressName$/$addressName$.cfg\r
         ftp.04.setSubDirForFtpPath=$addressName$/$config_dir$
         ftp.05.scpFile=$ftp_altusername$:$ftp_altpassword$@$ftp_althostname$:$ftp_altpath$/$config_dir$/r

7. Edit a service template to include the new parameter uowParameter with name CUSTOM_UOW_LABEL_1. Adding the parameter allows the configuration file name to be passed in REST calls. The following example service template shows a command set that adds snmp contact and snmp location elements in a configuration, with the new parameter added:

   <serviceTemplate name="SNMP_test" description="NSM Service Template to add and delete SNMP info">
   	<clientParameters>
   		<clientParameter>
   			<name>LOCATION</name>
   		</clientParameter>
   		<clientParameter>
   			<name>CONTACT</name>
   		</clientParameter>
   	</clientParameters>
       <uowParameters>
           <uowParameter>
               <name>CUSTOM_UOW_LABEL_1</name>
               <description>The configuration file name</description>
           </uowParameter>
       </uowParameters>
   	<implementations>
   		<implementation>
   			<rules>
   				<rule type="DeviceType">
   					<ruleProperty name="Vendor" value="mediatrix"/>
   					<ruleProperty name="Type" value=".*"/>
   					<ruleProperty name="Model" value=".*"/>
   					<ruleProperty name="OS" value=".*"/>
   				</rule>
   			</rules>
   			<serviceOperations>
   				<serviceOperation type="CREATE">
   					<operations>
   						<operation name="ITNCM/mediatrix/SNMP-add" type="COMMANDSET">
   							<parameters>
   								<parameter name="LOCATION"/>
   								<parameter name="CONTACT"/>
   							</parameters>
   						</operation>
   					</operations>
   				</serviceOperation>
   				<serviceOperation type="DELETE">
   					<operations>
   						<operation name="ITNCM/mediatrix/SNMP-delete" type="COMMANDSET">
   							<parameters>
   								<parameter name="LOCATION"/>
   								<parameter name="CONTACT"/>
   							</parameters>
   						</operation>
   					</operations>
   				</serviceOperation>
   			</serviceOperations>
   		</implementation>
   	</implementations>
   </serviceTemplate>
   

   The following example service template removes a device, and has the new parameter added:

   <?xml version="1.0" encoding="UTF-8"?>
   <serviceTemplate name="remove_device" description="To remove devices">
       <uowParameters>
           <uowParameter>
               <name>CUSTOM_UOW_LABEL_1</name>
               <description>The configuration file name</description>
           </uowParameter>
       </uowParameters>
   	<implementations>
   		<implementation>
   			<rules>
   				<rule type="DeviceType">
   					<ruleProperty name="Vendor" value="Mediatrix"/>
   					<ruleProperty name="Type" value=".*"/>
   					<ruleProperty name="Model" value=".*"/>
   					<ruleProperty name="OS" value=".*"/>
   				</rule>
   			</rules>
   			<serviceOperations>
   				<serviceOperation type="CREATE">
   					<operations>
   						<operation name="REMOVE" type="REMOVE">
   						</operation>
   					</operations>
   				</serviceOperation>
   				<serviceOperation type="DELETE">
   					<operations>
   						<operation name="REMOVE" type="REMOVE">
   						</operation>
   					</operations>
   				</serviceOperation>
   			</serviceOperations>
   		</implementation>
   	</implementations>
   </serviceTemplate>
   

8. When you submit a configuration update by using the GUI, enter the custom name for the configuration file in the Config_Dir_Name field on the Describe Work screen.

9. The following example NSM service post body defines the configuration file name as MAC999999/SN77777_1.xml.

   XML example:

   <serviceTemplate id="101">
   	<deviceID>402</deviceID>
     <uowParameters>
       <uowParameter>
         <name>CUSTOM_UOW_LABEL_1</name>
         <value>mediatrix_device.cfg</value>
       </uowParameter>
     </uowParameters>
   	<clientParameters>
   		<clientParameter>
   			<name>LOCATION</name>
   			<value>The Shire</value>
   		</clientParameter>
   		<clientParameter>
   			<name>CONTACT</name>
   			<value>Bilbo Baggins</value>
   		</clientParameter>
   	</clientParameters>
   </serviceTemplate>

   JSON example:

   {
          "serviceTemplateId" : "101",
          "deviceID": "402",
          "clientParameters": [
         {
            "name": "LOCATION",
            "value": "The Shire"
         },
         {
            "name": "CONTACT",
            "value": "Bilbo Baggins"
         }
      ],
      "uowParameters": [
        {
            "name": "CUSTOM_UOW_LABEL_1",
            "value": "MAC999999/SN77777_1.xml"
         }
      ]   
   }

10. The following example NSM service post body for removing a device has the delete_device_file_based_configs value that triggers the deletion of configuration files associated with the device.

    XML example:

     <serviceTemplate id="105">
    	<deviceID>3207</deviceID>
      <uowParameters>
        <uowParameter>
          <name>CUSTOM_UOW_LABEL_1</name>
          <value>delete_device_file_based_configs</value>
        </uowParameter>
      </uowParameters>
    </serviceTemplate>
    

    JSON example

    {
           "id" : "105",
           "deviceID": "3207",
       "uowParameters": [
         {
             "name": "CUSTOM_UOW_LABEL_1",
             "value": "delete_device_file_based_configs"
          }
       ]   
    }