Viewing the logs (OCP)

You can see the logs for all Agile Service Manager OCP pods using the kubectl logs command. You can change the logging level to 'debug' (or a number of other levels) to troubleshoot in more detail.

About this task

All Netcool Agile Service Manager OCP containers are deployed in pods. You can identify logs either by their names or their labels.

Network Discovery service logs: In an OCP installation, the logs for the Network Discovery service are held in the container, and tailed by the nasm-net-disco-worker-logs sidecar container, which outputs to stdout.
Application Discovery service logs: For each component (primary storage server, discovery server and secondary storage server), a sidecar container is designated as log-collector.

To increase or decrease the log levels for a specific observer to allow you to investigate issues, you first use the kubectl get deployments command to display a list of all OCP deployments, then use the kubectl edit deployment command to change the logging levels to, for example, 'debug'.

Procedure

Viewing logs

To list all Agile Service Manager pods and the labels applied to the pods, run the kubectl get pod --show-labels command, as in the example below.

Display the logs using either the pod names or labels.

When running the kubectl logs command, you must also specify the namespace.

View logs for a specific pod

kubectl logs asm-layout-88bd88bdb-htjfp --namespace=netcool

View logs using the label

kubectl logs -l app=layout --namespace=netcool

Example

When using the kubectl get pod --show-labels command to list all pods, the system will display the pod names, their status, number of restarts, age, and labels. The labels retrieved will also contain additional information, such as, for example, the release name, which is important if more than one Agile Service Manager deployment exists.

$ kubectl get pod --show-labels
NAME                                               READY     STATUS    RESTARTS   AGE       LABELS
deploy-test-cassandra-0                            1/1       Running   0          2d        app=cassandra,chart=cassandra,controller-revision-hash=deploy-test-cassandra-7d8f56b884,heritage=Tiller,release=deploy-test,statefulset.kubernetes.io/pod-name=deploy-test-cassandra-0
deploy-test-ciscoaci-observer-8755fcc94-7cp7h      1/1       Running   0          2d        app=ciscoaci-observer,chart=ciscoaci-observer,heritage=Tiller,pod-template-hash=431197750,release=deploy-test
deploy-test-contrail-observer-54f49cdc8c-bjdnq     1/1       Running   0          2d        app=contrail-observer,chart=contrail-observer,heritage=Tiller,pod-template-hash=1090578747,release=deploy-test
deploy-test-dns-observer-7ffc598847-kdqlr          1/1       Running   0          2d        app=dns-observer,chart=dns-observer,heritage=Tiller,pod-template-hash=3997154403,release=deploy-test
deploy-test-elasticsearch-0                        1/1       Running   0          2d        app=elasticsearch,chart=elasticsearch,controller-revision-hash=deploy-test-elasticsearch-5bb4857dff,heritage=Tiller,release=deploy-test,statefulset.kubernetes.io/pod-name=deploy-test-elasticsearch-0
deploy-test-file-observer-76c5869d8b-tm7xj         1/1       Running   0          2d        app=file-observer,chart=file-observer,heritage=Tiller,pod-template-hash=3271425846,release=deploy-test
deploy-test-ibmcloud-observer-65497f5478-4nsr6     1/1       Running   0          2d        app=ibmcloud-observer,chart=ibmcloud-observer,heritage=Tiller,pod-template-hash=2105391034,release=deploy-test
deploy-test-itnm-observer-85d7cc7878-2xdhf         1/1       Running   0          2d        app=itnm-observer,chart=itnm-observer,heritage=Tiller,pod-template-hash=4183773434,release=deploy-test
deploy-test-kafka-0                                2/2       Running   0          2d        app=kafka,chart=kafka,controller-revision-hash=deploy-test-kafka-5c78c96dbc,heritage=Tiller,release=deploy-test,statefulset.kubernetes.io/pod-name=deploy-test-kafka-0
deploy-test-kubernetes-observer-66cb697d7-ktgqt    1/1       Running   0          2d        app=kubernetes-observer,chart=kubernetes-observer,heritage=Tiller,pod-template-hash=227625383,release=deploy-test
deploy-test-layout-66656685bd-wkp6b                1/1       Running   0          2d        app=layout,chart=layout,heritage=Tiller,pod-template-hash=2221224168,release=deploy-test
deploy-test-merge-688468b7-q4ckq                   1/1       Running   0          2d        app=merge,chart=merge,heritage=Tiller,pod-template-hash=24402463,release=deploy-test
deploy-test-newrelic-observer-7c879f5545-72k4p     1/1       Running   0          2d        app=newrelic-observer,chart=newrelic-observer,heritage=Tiller,pod-template-hash=3743591101,release=deploy-test
deploy-test-openstack-observer-5df8fffd56-kfdjc    1/1       Running   0          2d        app=openstack-observer,chart=openstack-observer,heritage=Tiller,pod-template-hash=1894999812,release=deploy-test
deploy-test-rest-observer-789cd8699d-zb8vx         1/1       Running   0          2d        app=rest-observer,chart=rest-observer,heritage=Tiller,pod-template-hash=3457842558,release=deploy-test
deploy-test-search-5dc4ccc99b-nmqz2                1/1       Running   0          2d        app=search,chart=search,heritage=Tiller,pod-template-hash=1870777556,release=deploy-test
deploy-test-servicenow-observer-854bbff7dc-sl75x   1/1       Running   0          2d        app=servicenow-observer,chart=servicenow-observer,heritage=Tiller,pod-template-hash=4106699387,release=deploy-test
deploy-test-taddm-observer-79dd5b556-bnzp2         1/1       Running   0          2d        app=taddm-observer,chart=taddm-observer,heritage=Tiller,pod-template-hash=358816112,release=deploy-test
deploy-test-topology-75688cfc48-jjkfd              1/1       Running   0          2d        app=topology,chart=topology,heritage=Tiller,pod-template-hash=3124479704,release=deploy-test
deploy-test-vmvcenter-observer-675bd88f5c-8kgnk    1/1       Running   0          2d        app=vmvcenter-observer,chart=vmvcenter-observer,heritage=Tiller,pod-template-hash=2316844917,release=deploy-test
deploy-test-vmwarensx-observer-dc96946f4-7jcxj     1/1       Running   0          2d        app=vmwarensx-observer,chart=vmwarensx-observer,heritage=Tiller,pod-template-hash=875250290,release=deploy-test
deploy-test-zookeeper-0                            1/1       Running   0          2d        app=zookeeper,chart=zookeeper,controller-revision-hash=deploy-test-zookeeper-d65f46875,heritage=Tiller,release=deploy-test,statefulset.kubernetes.io/pod-name=deploy-test-zookeeper-0

Setting log levels to 'debug'

Run the following command to change a pod's debug level.
Topology pod example:
```
kubectl exec -it deploy-test-topology-75688cfc48-jjkfd -- python3 /opt/ibm/netcool/asm/bin/log-level.pyc --level DEBUG
```
Example system output:
```
Configured logging level for com.ibm to DEBUG
Response code: 200
```
Note: The UI API pod uses a shell script instead of the python script used by all the other pods. When changing the logging level for the UI API pod to DEBUG, use the following command:
```
log-level.sh --level DEBUG
```
Tip: For information on changing log levels for observers, see the Setting observer log levels (OCP) topic.
Verify that the log is in DEBUG mode.
DEBUG will log all request/response operations to provide detailed diagnostic data.

What to do next

Tip:

A number of log levels are available, in addition to 'debug'.

You can change your current log levels to the following values (INFO is the default log level):

OFF
ERROR
WARN
INFO
DEBUG
TRACE
ALL

When setting the log level, you may want to consider disk usage. The ALL log levels uses the most space.