IBM Support

tacmd sudden login failure.

Technical Blog Post


Abstract

tacmd sudden login failure.

Body

image

tacmd sudden login failure


This was an issue seen with one customer, suddenly the tacmd login failed with the message


tacmd login fails - Error message "KUIC00003E"


Investigation found that this would suddenly start to happen in the running environment when up to that point in time tacmd commands had been running fine.

Tracing was set on the tacmd script:

 
/opt/IBM/ITM/bin/tacmd

 
KBB_RAS1=ERROR(UNIT:KDY ALL)(UNIT:KSH ALL)(UNIT:KGL ALL)

and on the TEMS

KBB_RAS1: ERROR (UNIT:KSH ALL) (UNIT:KUI ALL) (UNIT:KDY ALL)

At the next failure a tacmd login was done and the pdcollect was collected .

This showed the tacmd error of:

 kuiras log

(Wed Mar 30 16:20:00 2016.1340-1:kuitacmdmain.cpp,5144,"getPortFromCMS")
Response ==><<HTML><HEAD><META HTTP-EQUIV='Refresh' CONTENT='2;
URL=login.htm'></META><TITLE>Error</TITLE></HEAD><BODY><H1>Error
401</H1> <P>Invalid user ID or password.</P></BODY></HTML>>
(Wed Mar  30 16:20:00 2016.1341-1:kuitacmdmain.cpp,5195,"getPortFromCMS")
Unable to get Service Point for retrieval of TEMS from Response
(Wed Mar  30 16:20:00 2016.1342-1:kuitacmdmain.cpp,5203,"getPortFromCMS")
Could not get alternate port number from TEMS

and TEMs logs of:
:kdhscsv.c,137,"login_attempts_exceeded") failedLogins 5
maxFailedLogins 5 currentTime 1454944800 lastLoginFailure 1454944252
(Wed Mar  30 16:20:00 2016.033A-1A:kshstrt.cpp,127,"default_service")
Login attempts exceeded

This indicated that the tacmd command had hit the number of login attempts allowed by one ip and a lock out for the id had been set.

Investigation found that tacmd commands were being run every minute from scripts.

This was found to be too high a level, once this was lowered to every 10 minutes that problems were no longer seen.

It should be noted that running tacmd commands use a SOAP login and therefore are checked for how many over a period is done and if there are more than one script running this issue can be hit.
Also the requirement to run tacmd commands on the system should be reviewed as these,  like any other requests do take resources and so running  many requests can cause unforeseen issues due to the load on the environment.

[{"Business Unit":{"code":"BU004","label":"Hybrid Cloud"},"Product":{"code":"","label":""},"Component":"","Platform":[{"code":"","label":""}],"Version":"","Edition":""},{"Business Unit":{"code":"BU004","label":"Hybrid Cloud"},"Product":{"code":"SSZ8F3","label":"IBM Tivoli Monitoring V6"},"Component":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Versions","Edition":""}]

UID

ibm11084107