Problem

Tracing Solr HTTP requests

The DataStax Enterprise Help Center also provides troubleshooting information.

• Enable probabilistic tracing .
• Pass an explicit cassandra.trace=true request parameter in the HTTP query.

After running the example of using a join query , you can trace the join query by adding the cassandra.trace parameter to the HTTP request:

http://localhost:8983/solr/internet.songs/select/?q={!join+from=song+to=id+fromIndex=internet.lyrics+force=true}words:love&indent=true&wt=json&cassandra.trace=true

The Solr response includes a cassandra.trace.session value, the unique session id of the tracing session for the request:

{
  "cassandra.trace.session":"3e503490-bdb9-11e3-860f-73ded3cb6170",
  "responseHeader":{
    "status":0,
    "QTime":1,
    "params":{
      "indent":"true",
      "q":"{!join from=song to=id fromIndex=internet.lyrics force=true}words:love",
      "wt":"json",
      "cassandra.trace":"true"}},
  "response":{"numFound":2,"start":0,"docs":[
      {
        "id":"8a172618-b121-4136-bb10-f665cfc469eb",
        "title":"Internet Love Song",
        "artist":"John Cedrick"},
      {
        "id":"a3e64f8f-bd44-4f28-b8d9-6938726e34d4",
        "title":"Dangerous",
        "artist":"Big Data"}]
}}

To see the information from the trace, query the system_traces.events, using the session id to filter the output.

cqlsh> select * from system_traces.events where session_id = 3e503490-bdb9-11e3-860f-73ded3cb6170;

 session_id  | activity                                                                  | source_elapsed  
-------------+---------------------------------------------------------------------------+----------------
 3e503490... |          Parsing SELECT * from "internet"."songs" WHERE "id" = 8a172618...|           2607 
 3e503490... |                                                       Preparing statement |           3943 
 3e503490... |                                 Executing single-partition query on songs |           4246 
 3e503490... |                                              Acquiring sstable references |           4261 
 3e503490... |                                               Merging memtable tombstones |           4305 
 3e503490... |                                               Key cache hit for sstable 1 |           4388 
 3e503490... |                         Seeking to partition indexed section in data file |           4399 
 3e503490... | Skipped 0/1 non-slice-intersecting sstables, included 0 due to tombstones |           4873 
 3e503490... |                                Merging data from memtables and 1 sstables |           4954 
 3e503490... |                                        Read 1 live and 0 tombstoned cells |           5162 
 3e503490... |        Parsing SELECT * from "internet"."songs" WHERE "id" = a3e64f8f...  |           6160 
 3e503490... |                                                       Preparing statement |           7424
 . . .

For example purposes, the event_id, node IP address, and thread id have been deleted from this output to fit on the page.

In the case of distributed queries over several nodes, the same tracing session id is used on all nodes, which makes it possible to correlate database operations on all the nodes taking part in the distributed query.