Search LogDNA Records from the Command Line

After provisioning the LogDNA instance as part of the IBM Cloud observability portal, you can launch its UI. There, it is possible to define views, perform searches, and manage instance settings. An important setting is the configuration of log archiving (i.e., the automatic backup of activity data to cloud object storage—compliance and more).

LogDNA allows you to ingest (link resides outside ibm.com) and export (link resides outside ibm.com) events through APIs. The required API keys can be obtained via UI in the settings for organization.

The export API requires to and from parameters to limit the time range and supports event filters (e.g., by hosts, event level, and, most importantly, by query string). Authentication is performed by API service key.

With that information, I had everything I needed to write a small Python script—logdna-search—that serves as a foundation for a more capable command line tool or package for some serverless activity analytics. The script requires a configuration file with the instance region and the service key. The number of hours for the time range and a query string are optional. The following searches for log events from the past 24 hours, where the initiator of that security-related event has a name starting with “hloeser“:

searchLogDNA.py logConfigEU.json 24 'initiator.name:hloeser'

The above returns events as JSON lines (link resides outside ibm.com); each event is a JSON record on its own.

If you are familiar with jq (link resides outside ibm.com), a command line JSON processor, you appreciate its filtering and formatting capabilities. Because of the nature of the activity events and LogDNA logs, the JSON-based events are embedded into JSON records with additional metadata, making it harder to read. With the help of jq, it is easy to filter down:

searchLogDNA.py logConfigEU.json 24 'initiator.name:hloeser' | jq -r '._line'

To extract just the initiator part of the events, apply another filter:

searchLogDNA.py logConfigEU.json 24 'initiator.name:hloeser' | jq -r '._line' | jq -r '.initiator'

Want to test it on your own? The Python script is available on GitHub in the repo logdna-search. Information on Activity Tracker with LogDNA on IBM Cloud is in the documentation.

If you have feedback, suggestions, or questions about this post, please reach out to me on Twitter (@data_henrik) (link resides outside ibm.com) or LinkedIn (link resides outside ibm.com).