IBM Support

Encoding converting script for Netcool/Impact 5.1.0 upgrading

Technical Blog Post


Abstract

Encoding converting script for Netcool/Impact 5.1.0 upgrading

Body

There is an encoding enhancement in Impact 5.1.1.  Impact uses UTF8 as a default encoding to handle policies.  So Impact 5.1.1 and newer release could read, write and display special character correctly, such as accent, either in file system or GUI pages without setting a specific locale/encoding.

 

 However, customer need set correct locale to display them normally in Impact 5.1.0 and older release if policy having special characters, otherwise garbled characters will display.  Customer set correct locale naturally by most chances.

 

 If you found several policies could not be imported into Impact 5.1.1/6.x from an Impact 5.1.0 export package. There is a MalformedInputException in ByteToCharUTF8.convert and then a FileNotFoundException throws. Details see following error messages in Impact product log file.  Please use attached script to solve this import issue.  

 

Now, let’s see what we need to do for this issue.  Encoding converting is a suitable way to solve it.

The coded character set used in former Impact 5.1.0 is the key for the converting.  So I write out a script to convert encoding to UTF8 which has only one parameter - original coded character set.

 Please run conv.sh on customer's environment.

Place conv.sh into follow dir $NCHOME/bin

$ cd [export_dir]/Policy

$ $NCHOME/bin/conv.sh ISO_8859-1

it will display as follow

                [impact@tivbsml3v01 Policy]$ /opt/ibm/netcool/conv.sh ISO_8859-1

                Convert AddPolicyProcessMapping.ipl from ISO_8859-1 to UTF8.

                Convert a.ipl from ISO_8859-1 to UTF8.

                Convert b.ipl from ISO_8859-1 to UTF8.

                Convert c.ipl from ISO_8859-1 to UTF8.

                ... ...

If you are not sure about the parameter - original coded character set, please select one from output of `iconv -l` and compare with the one from `locale` on Impact 5.1.0 environment.

$ iconv  -l

The following list contain all the coded character sets known.  This does

not necessarily mean that all combinations of these names can be used for

the FROM and TO command line parameters.  One coded character set can be

listed with several different names (aliases).

 

  437, 500, 500V1, 850, 851, 852, 855, 856, 857, 860, 861, 862, 863, 864, 865,

  866, 866NAV, 869, 874, 904, 1026, 1046, 1047, 8859_1, 8859_2, 8859_3, 8859_4,

  8859_5, 8859_6, 8859_7, 8859_8, 8859_9, 10646-1:1993, 10646-1:1993/UCS4,

  ANSI_X3.4-1968, ANSI_X3.4-1986, ANSI_X3.4, ANSI_X3.110-1983, ANSI_X3.110,

 

 

Try nci_import if there is no error of the script execution.

Please try nci_import once more If FileNotFoundException in UPDATED_EVENT. 

This exception is normal, because importer want to update a non-exist policy.

 

 

Exception in netcool.log

30 May 2013 16:05:48,949 DEBUG [PolicyRepository] Started importing Policies...

30 May 2013 16:05:48,950 DEBUG [VersionControlSystemManager] [SubVersion] Checkout: /opt/ibm/netcool/platform/linux/svn/bin/svn lock "etc/NCI_policylist" Username: impact

30 May 2013 16:05:48,950 DEBUG [PolicyRepository] About to import Policy: Customised1

30 May 2013 16:05:48,951 DEBUG [PolicyRepository] Failed to import Policy: Customised1.

sun.io.MalformedInputException

        at sun.io.ByteToCharUTF8.convert(ByteToCharUTF8.java:278)

        at sun.nio.cs.StreamDecoder$ConverterSD.convertInto(StreamDecoder.java:316)

        at sun.nio.cs.StreamDecoder$ConverterSD.implRead(StreamDecoder.java:366)

        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:252)

        at java.io.InputStreamReader.read(InputStreamReader.java:212)

        at java.io.BufferedReader.fill(BufferedReader.java:157)

        at java.io.BufferedReader.readLine(BufferedReader.java:320)

        at java.io.BufferedReader.readLine(BufferedReader.java:383)

        at com.micromuse.response.util.FilteringLineReader.readLine(FilteringLineReader.java:153)

        at com.micromuse.response.util.FilteringLineReader.readLine(FilteringLineReader.java:131)

        at com.micromuse.response.util.FilteringLineReader.read(FilteringLineReader.java:70)

        at com.micromuse.response.util.PolicyReader.<init>(PolicyReader.java:82)

        at com.micromuse.response.util.PolicyFilteringLineReader.<init>(PolicyFilteringLineReader.java:48)

        at com.micromuse.response.util.ImportUtils.importPolicy(ImportUtils.java:66)

        at com.micromuse.response.dblayer.PolicyRepository.importPolicies(PolicyRepository.java:1000)

        at com.micromuse.response.dblayer.PolicyRepository.importFromFiles(PolicyRepository.java:955)

        at com.micromuse.response.dblayer.Type.importFromFiles(Type.java:1108)

        at com.micromuse.response.dblayer.TypeList.importFromFiles(TypeList.java:692)

        at com.micromuse.response.Importer.importFunction(Importer.java:296)

        at com.micromuse.response.common.parser.IPLImpactNoActionInterpreterVisitor.visit(IPLImpactNoActionInterpreterVisitor.java:189)

        at com.micromuse.response.common.parser.IPLImpactActionInterpreterVisitor.visit(IPLImpactActionInterpreterVisitor.java:290)

        at com.micromuse.common.parser.internal.core.ASTFunctionCall.jjtAccept(ASTFunctionCall.java:17)

        at com.micromuse.response.common.parser.IPLImpactNoActionInterpreterVisitor.visit(IPLImpactNoActionInterpreterVisitor.java:265)

        at com.micromuse.response.common.parser.IPLImpactActionInterpreterVisitor.visit(IPLImpactActionInterpreterVisitor.java:290)

        at com.micromuse.common.parser.internal.core.ASTFunctionCall.jjtAccept(ASTFunctionCall.java:17)

        at com.micromuse.common.parser.internal.core.PolicyParserInterpreterVisitor.visit(PolicyParserInterpreterVisitor.java:777)

        at com.micromuse.response.common.parser.AbstractPolicyParserInterpreterVisitor.visit(AbstractPolicyParserInterpreterVisitor.java:383)

        at com.micromuse.common.parser.internal.core.ASTBlock.jjtAccept(ASTBlock.java:17)

        at com.micromuse.common.parser.internal.core.PolicyParserInterpreterVisitor.visit(PolicyParserInterpreterVisitor.java:976)

        at com.micromuse.response.common.parser.AbstractPolicyParserInterpreterVisitor.visit(AbstractPolicyParserInterpreterVisitor.java:299)

        at com.micromuse.response.common.parser.IPLImpactNoActionInterpreterVisitor.visit(IPLImpactNoActionInterpreterVisitor.java:556)

        at com.micromuse.response.common.parser.IPLImpactActionInterpreterVisitor.visit(IPLImpactActionInterpreterVisitor.java:290)

        at com.micromuse.common.parser.internal.core.ASTFunctionCall.jjtAccept(ASTFunctionCall.java:17)

        at com.micromuse.common.parser.internal.core.PolicyParserInterpreterVisitor.visit(PolicyParserInterpreterVisitor.java:143)

        at com.micromuse.response.common.parser.AbstractPolicyParserInterpreterVisitor.visit(AbstractPolicyParserInterpreterVisitor.java:77)

        at com.micromuse.response.common.parser.IPLImpactActionInterpreterVisitor.visit(IPLImpactActionInterpreterVisitor.java:72)

        at com.micromuse.common.parser.internal.core.ASTCompilationUnit.jjtAccept(ASTCompilationUnit.java:17)

        at com.micromuse.response.common.parser.PolicyManager$1.execute(PolicyManager.java:119)

        at com.micromuse.response.common.parser.PolicyExecutor.executePolicy(PolicyExecutor.java:168)

        at com.micromuse.response.common.parser.PolicyExecutor.executePolicy(PolicyExecutor.java:253)

        at com.micromuse.response.server.MessageProcessor.processMessage(MessageProcessor.java:330)

        at com.micromuse.response.server.MessageProcessor.processMessage(MessageProcessor.java:263)

        at com.micromuse.response.server.SynchronousMessageProcessor$1.call(SynchronousMessageProcessor.java:143)

        at com.micromuse.response.server.SynchronousMessageProcessor$1.call(SynchronousMessageProcessor.java:142)

        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)

        at java.util.concurrent.FutureTask.run(FutureTask.java:138)

        at com.micromuse.common.util.concurrent.TrackingExecutor$1.run(TrackingExecutor.java:98)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)

        at java.lang.Thread.run(Thread.java:810)

30 May 2013 16:05:48,952 DEBUG [PolicyRepository] Imported policy: Customised1

30 May 2013 16:05:48,952 ERROR [PolicyRepository] internalAddPolicy: Failed to import policy : Customised1

java.io.FileNotFoundException: /opt/ibm/netcool/impact/policy/NCI_Customised1.ipl (No such file or directory)

        at java.io.FileInputStream.open(Native Method)

        at java.io.FileInputStream.<init>(FileInputStream.java:135)

        at java.io.FileReader.<init>(FileReader.java:78)

        at com.micromuse.response.dblayer.PolicyRepository.loadPolicy(PolicyRepository.java:492)

        at com.micromuse.response.dblayer.PolicyRepository.internalAddPolicy(PolicyRepository.java:267)

        at com.micromuse.response.dblayer.PolicyRepository.importPolicies(PolicyRepository.java:1024)

        at com.micromuse.response.dblayer.PolicyRepository.importFromFiles(PolicyRepository.java:955)

        at com.micromuse.response.dblayer.Type.importFromFiles(Type.java:1108)

        at com.micromuse.response.dblayer.TypeList.importFromFiles(TypeList.java:692)

        at com.micromuse.response.Importer.importFunction(Importer.java:296)

        at com.micromuse.response.common.parser.IPLImpactNoActionInterpreterVisitor.visit(IPLImpactNoActionInterpreterVisitor.java:189)

        at com.micromuse.response.common.parser.IPLImpactActionInterpreterVisitor.visit(IPLImpactActionInterpreterVisitor.java:290)

        at com.micromuse.common.parser.internal.core.ASTFunctionCall.jjtAccept(ASTFunctionCall.java:17)

        at com.micromuse.response.common.parser.IPLImpactNoActionInterpreterVisitor.visit(IPLImpactNoActionInterpreterVisitor.java:265)

        at com.micromuse.response.common.parser.IPLImpactActionInterpreterVisitor.visit(IPLImpactActionInterpreterVisitor.java:290)

        at com.micromuse.common.parser.internal.core.ASTFunctionCall.jjtAccept(ASTFunctionCall.java:17)

        at com.micromuse.common.parser.internal.core.PolicyParserInterpreterVisitor.visit(PolicyParserInterpreterVisitor.java:777)

        at com.micromuse.response.common.parser.AbstractPolicyParserInterpreterVisitor.visit(AbstractPolicyParserInterpreterVisitor.java:383)

        at com.micromuse.common.parser.internal.core.ASTBlock.jjtAccept(ASTBlock.java:17)

        at com.micromuse.common.parser.internal.core.PolicyParserInterpreterVisitor.visit(PolicyParserInterpreterVisitor.java:976)

        at com.micromuse.response.common.parser.AbstractPolicyParserInterpreterVisitor.visit(AbstractPolicyParserInterpreterVisitor.java:299)

        at com.micromuse.response.common.parser.IPLImpactNoActionInterpreterVisitor.visit(IPLImpactNoActionInterpreterVisitor.java:556)

        at com.micromuse.response.common.parser.IPLImpactActionInterpreterVisitor.visit(IPLImpactActionInterpreterVisitor.java:290)

        at com.micromuse.common.parser.internal.core.ASTFunctionCall.jjtAccept(ASTFunctionCall.java:17)

        at com.micromuse.common.parser.internal.core.PolicyParserInterpreterVisitor.visit(PolicyParserInterpreterVisitor.java:143)

        at com.micromuse.response.common.parser.AbstractPolicyParserInterpreterVisitor.visit(AbstractPolicyParserInterpreterVisitor.java:77)

        at com.micromuse.response.common.parser.IPLImpactActionInterpreterVisitor.visit(IPLImpactActionInterpreterVisitor.java:72)

        at com.micromuse.common.parser.internal.core.ASTCompilationUnit.jjtAccept(ASTCompilationUnit.java:17)

        at com.micromuse.response.common.parser.PolicyManager$1.execute(PolicyManager.java:119)

        at com.micromuse.response.common.parser.PolicyExecutor.executePolicy(PolicyExecutor.java:168)

        at com.micromuse.response.common.parser.PolicyExecutor.executePolicy(PolicyExecutor.java:253)

        at com.micromuse.response.server.MessageProcessor.processMessage(MessageProcessor.java:330)

        at com.micromuse.response.server.MessageProcessor.processMessage(MessageProcessor.java:263)

        at com.micromuse.response.server.SynchronousMessageProcessor$1.call(SynchronousMessageProcessor.java:143)

        at com.micromuse.response.server.SynchronousMessageProcessor$1.call(SynchronousMessageProcessor.java:142)

        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)

        at java.util.concurrent.FutureTask.run(FutureTask.java:138)

        at com.micromuse.common.util.concurrent.TrackingExecutor$1.run(TrackingExecutor.java:98)

        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:665)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:690)

        at java.lang.Thread.run(Thread.java:810)

30 May 2013 16:05:48,952 DEBUG [PolicyRepository] About to import Policy: Customised2

[{"Business Unit":{"code":"BU053","label":"Cloud & Data Platform"},"Product":{"code":"SSSPFK","label":"Tivoli Business Service Manager"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"","Edition":"","Line of Business":{"code":"LOB45","label":"Automation"}}]

UID

ibm11276240