September 16, 2019 By Glynn Bird 4 min read

Cloudant is now rolling out customer-facing logging for its Cloudant services.

In this post, we’ll explore how to set it up and how it can help you keep tabs on your IBM Cloudant service.

Note: At the time of writing, this service is only available on our Dallas, London, and Frankfurt-based cloud clusters. Other services will follow in due course. Setting up logging

First, we need to set up an IBM Log Analysis with LogDNA service in the IBM Cloud. Begin by choosing the service from the Developer Tools section of the IBM Cloud catalog:

Configure the service in the same region as your Cloudant service (e.g., Dallas) and choose and appropriate a resource group:

Once provisioned, choose the Configure Platform Services button to direct your IBM Cloud platform logs to your LogDNA service:

That’s it! Click the View LogDNA button to see a live stream of your logs:

You should see all of your API calls listed in the live stream, with a short delay as the data makes its way to LogDNA.

Logging data structure

The data is logged as a JSON object containing everything you need to know about the request and how it was handled:

{
    "accountName": "mycloudantaccount",
    "httpMethod": "GET",
    "httpRequest": "/cities/_all_docs?limit=10",
    "responseSizeBytes": 1221,
    "clientIp": "94.194.94.13",
    "clientPort": 63072,
    "statusCode": 200,
    "terminationState": "----",
    "dbName": "cities",
    "dbRequest": "_all_docs",
    "userAgent": "curl/7.54.0",
    "sslVersion": "TLSv1.2",
    "requestClass": "query",
    "parsedQueryString": {
        "limit": "10"
    },
    "rawQueryString": "limit=10",
    "timings": {
        "connect": 142,
        "request": 0,
        "response": 10,
        "transfer": 1
    },
    "meta": {},
    "logSourceCRN": "crn:v1:bluemix:public:cloudantnosqldb:us-south:a/someid::",
    "saveServiceCopy": false
}

In the free version of LogDNA, only the live stream of data can be viewed, with optional filtering by source. In the paid plans, the JSON data is parsed and is retained for a period of time, with optional archiving in IBM Cloud Object Storage.

With the data parsed, it’s easy to compose queries, alerts, and graphs based on the data you see in the object above. You may wish to keep an eye on the following:

  • The response timing of query requests on a particular database
  • The count of the occurrence of HTTP 429 responses, indicating that you have exceed your provisioned throughput
  • The range of IP addresses that are accessing your Cloudant service
  • The number of requests hitting each of your databases
  • The version TLS being used by your client code to access client: Do you have traffic originating from an out-of-date source?
  • The values of responseSizeBytes that are very high: Do some of your requests return lots of data?

The LogDNA documentation shows how you can explore the incoming stream of data, creating named views which can be retrieved later. Saved views can be combined to create visual dashboards to give you an at-a-glance view of your Cloudant traffic.

Example graphs

Request counts by HTTP request code, grouped into successes (statusCode < 300) and failures (statusCode >= 400):

Request counts by request “class” (i.e., whether IBM considers the request a read, write, or query for billing purposes):

Request counts by HTTP method (get or post, in this case):

95th percentile response times broken out by each of the timing metrics:

Creating a graph—step by step

1. Log into your Log Analysis with LogDNA service, choose Boards from the left-hand menu and click New Board.

2. Click Add Graph.

3. LogDNA then needs to know which field in the JSON you wish to plot:

4. Enter statusCode, the < operator, and 300. This will only show status codes of 299 and below, indicating success:

5. You should now see a plot of the counts of successful API calls against time. To add a second plot, click Add Plot:

6. This time, choose statusCode >= 300 to plot the errors:

7. The graph will change to add a second plot, showing the error counts alongside the success counts.

8. Repeat this process to add further graphs on metrics that interest you.

Timing metrics

The logged JSON contains an object with the timings of each part of the request:

"timings": {
  "connect": 142,
  "request": 0,
  "response": 10,
  "transfer": 1
}
  • connect: The time it took for your HTTP client to make the initial HTTPS connection (the handshakes required to negotiate a secure connection between client and server). Programmatic API calls using the official Cloudant libraries will reuse existing connections, eliminating this connection delay.
  • request: The time taken to receive the request data. For bulk uploads, this may be significant, especially from clients with limited uplink bandwidth.
  • response: The time taken for the database to do its work.
  • transfer: The time taken for the response to be transferred to the client.

Further information

More from Announcements

IBM Hybrid Cloud Mesh and Red Hat Service Interconnect: A new era of app-centric connectivity 

2 min read - To meet customer demands, applications are expected to be performing at their best at all times. Simultaneously, applications need to be flexible and cost effective, and therefore supported by an underlying infrastructure that is equally reliant, performant and secure as the applications themselves.   Easier said than done. According to EMA's 2024 Network Management Megatrends report only 42% of responding IT professionals would rate their network operations as successful.   In this era of hyper-distributed infrastructure where our users, apps, and data…

IBM named a Leader in Gartner Magic Quadrant for SIEM, for the 14th consecutive time

3 min read - Security operations is getting more complex and inefficient with too many tools, too much data and simply too much to do. According to a study done by IBM, SOC team members are only able to handle half of the alerts that they should be reviewing in a typical workday. This potentially leads to missing the important alerts that are critical to an organization's security. Thus, choosing the right SIEM solution can be transformative for security teams, helping them manage alerts…

IBM and MuleSoft expand global relationship to accelerate modernization on IBM Power 

2 min read - As companies undergo digital transformation, they rely on APIs as the backbone for providing new services and customer experiences. While APIs can simplify application development and deliver integrated solutions, IT shops must have a robust solution to effectively manage and govern them to ensure that response times and costs are kept low for all applications. Many customers use Salesforce’s MuleSoft, named a leader by Gartner® in full lifecycle API management for seven consecutive times, to manage and secure APIs across…

IBM Newsletters

Get our newsletters and topic updates that deliver the latest thought leadership and insights on emerging trends.
Subscribe now More newsletters