Monitoring Python applications in Kubernetes environment

Before you monitor Python applications in IBM® Cloud Private or OpenShift, you must connect the data collector to the server by creating a secret. Then, you update your application deployment to monitor the Python applications.

1. Update the Dockerfile to add the following lines to install Python data collector for your Python application, and get write access to the root directory.

   ADD ibm_python_datacollector.tgz root_of_application
   RUN chmod 777 root_of_application
   RUN pip install --no-index --find-links=root_of_application/python_dc ibm_python_dc

   Where root_of_application is the Python application root directory of the context of the build (the Dockerfile).

2. From IBM Cloud App Management V2019.4.0, you can use the Python data collector to monitor Python applications running on the web server uWSGI V1.9.0 or later versions. But some configurations are needed to make the Python data collector work well.
   • If you start uWSGI without threads, the threads that are generated by your application and the Python data collector will never run and thus the Python data collector cannot work normally. You must enable uWSGI threads by adding the --enable-threads option in the uwsgi command. This option is applied automatically when you specify the --threads option to configure the number of threads.
   
   
   • If you run the uwsgi command with the --master option, Python scripts and modules are preloaded in the parent master process, and worker processes are forked from the parent master process. In addition, background threads that are created in the master process are killed in worker processes. To make the Python data collector work normally, you must add the --lazy-apps option in the uwsgi command to use the lazy loading mode.
   Command example:

   uwsgi --enable-threads --master --lazy-apps --processes 4 --http :8002 --wsgi-file=flask_hello.py --callable app

   Note: You can also add the options in a .ini configuration file and run the uwsgi command with the .ini file. For more information, see uWSGI documents.
3. Integrate the installed data collector in your Python application:
   • If your application is based on Django V1.10 or later versions, open settings.py of your Django application, and add the following content into first line of section MIDDLEWARE in that file:

      'ibm_python_dc.kpg_dc_django.ResourceMiddleware',

   • If your application is based on Django V1.9 or older versions, add the following content into the first line of the section MIDDLEWARE_CLASS in that file:

     'ibm_python_dc.kpg_dc_django.ResourceMiddleware',

   • If your application is based on Flask, add the Python data collector wsgi middleware in your Python application file, for example, if you run export FLASK_APP=run.py, then edit the run.py file and ensure that the Python data collector wsgi middleware are added in front of other middlewares. Example:

     from flask import Flask
        from flask_restful import Api
        from api.board import Article
        from api.auth import Login, Register, RefreshToken
        from middleware import Test
        from werkzeug.middleware.dispatcher import DispatcherMiddleware
        
        api.add_resource(Login, '/login')
        api.add_resource(Register, '/register')
     
        from ibm_python_dc.kpg_dc_wsgi import ResourceMiddleware
        app.wsgi_app = ResourceMiddleware(app.wsgi_app)
        
        app.wsgi_app = DispatcherMiddleware(serve_frontend, {
          '/test': test,
          '/admin': admin,
        })
     
        ......
        ......
        

4. Go to the ibm-cloud-apm-dc-configpack directory where you extract the configuration package in Obtaining the server configuration information, and run the following command to create a secret to connect to the server, for example, name it as icam-server-secret.

   kubectl -n my_namespace create secret generic icam-server-secret \
   --from-file=keyfiles/keyfile.jks \
   --from-file=keyfiles/keyfile.p12 \
   --from-file=keyfiles/keyfile.kdb \
   --from-file=global.environment

   Where my_namespace is the namespace where you want to create the secret. If you want to create the secret in the default namespace, remove -n my_namespace from the command.
5. Update the application yaml file to mount the secret. See the following example.

   apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
    name: djangoapp
    labels:
        app: djangoapp
    spec:
    selector:
        matchLabels:
        app: djangoapp
        pod: djangoapp
    replicas: 1
    template:
        metadata:
        name: djangoapp
        labels:
            app: djangoapp
            pod: djangoapp
        spec:
        containers:
        - name: djangoapp
            image: mycluster.icp:8500/default/djangoapp:v1
            imagePullPolicy: Always
            ports:
            - containerPort: 8001
            protocol: TCP
            env:        
            - name: NAMESPACE_DEFAULT
              value: "default"
            - name: KPG_LOG_TOCONSOLE
              value: "True"
            - name: KPG_LOG_LEVEL
              value: "INFO"
            volumeMounts:
            - name: serverconfig
              mountPath: /opt/ibm/apm/serverconfig
        volumes:
        - name: global-environment
          secret:
            secretName: icam-server-secret
            optional: true

6. Build the new Docker image.
7. Update the application yaml file to use the new Docker image.