Topic
11 replies Latest Post - ‏2014-02-17T08:01:40Z by Karex
Karex
Karex
7 Posts
ACCEPTED ANSWER

Pinned topic ClearQuest REST API problem

‏2014-02-12T12:44:01Z |

Hello.

I want to send a query via REST to cq server using cUrl.

curl -D- --user user:pass -H 'Content-Type: application/json' 'http://165.213.147.58/cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500' > file.txt

Unfortunately it doesn't work... Response is:

HTTP/1.1 401 Unauthorized
Date: Wed, 12 Feb 2014 12:34:29 GMT
Server: IBM_HTTP_Server/6.1.0.25 Apache/2.0.47 (Win32)
WWW-Authenticate: OAuth realm="TN_STANDARD", oauth_problem="token_not_authorized"
WWW-Authenticate: Basic realm="TN_STANDARD"
Content-Length: 52
Content-Type: application/x-www-form-urlencoded;charset=UTF-8
Content-Language: ko-KR

HTTP%20status=401&oauth_problem=token_not_authorized


Is it needed to perform also oAuth authentication? From browser I can get to the http://165.213.147.58/cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500 with just giving my username and password. Is it impossible using cURL?

 

CQ 7.1.2.12

Updated on 2014-02-12T13:36:23Z at 2014-02-12T13:36:23Z by Karex
  • pdubovitsky
    pdubovitsky
    374 Posts
    ACCEPTED ANSWER

    Re: ClearQuest REST API problem

    ‏2014-02-12T13:45:47Z  in response to Karex

    Hello,

    Basic authentication should work.
    Could it be some special character in the password that caused the issue? Could you try to enclose credentials in quotes, like 'user:pass'?

    Pavel

    • Karex
      Karex
      7 Posts
      ACCEPTED ANSWER

      Re: ClearQuest REST API problem

      ‏2014-02-12T13:53:34Z  in response to pdubovitsky

      Login contains dot, however it changes nothing...

       

      curl -D- --user 'u.ser:pass123' -H 'Content-Type: application/json' 'http://165.213.147.58/cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500' > file.txt

       

      HTTP/1.1 401 Unauthorized
      Date: Wed, 12 Feb 2014 13:52:08 GMT
      Server: IBM_HTTP_Server/6.1.0.25 Apache/2.0.47 (Win32)
      WWW-Authenticate: OAuth realm="TN_STANDARD", oauth_problem="token_not_authorized"
      WWW-Authenticate: Basic realm="TN_STANDARD"
      Content-Length: 52
      Content-Type: application/x-www-form-urlencoded;charset=UTF-8
      Content-Language: ko-KR
       
      HTTP%20status=401&oauth_problem=token_not_authorized

      • pdubovitsky
        pdubovitsky
        374 Posts
        ACCEPTED ANSWER

        Re: ClearQuest REST API problem

        ‏2014-02-12T14:26:54Z  in response to Karex

        401 means that credentials are not accepted. Usually, it is wrong username/password.
        It looks like you are using 7.1.x release. Could you check correspondent messages in the WAS cmprofile logs?
        {RATIONALSDLC_DIR}\common\CM\profiles\cmprofile\logs\server1\SystemOut.log

        It might provide some additional information.

        Pavel
         

        • Karex
          Karex
          7 Posts
          ACCEPTED ANSWER

          Re: ClearQuest REST API problem

          ‏2014-02-12T14:53:41Z  in response to pdubovitsky

          It is not my server. I'm trying to get some information from it. Administrators are in "far away land" and aren't very responsive. I should figure it out alone or forget about it.

          username and password are the same as I give them in browser. In my oppinion browser is somehow disabling OAuth, while cUrl tries to login using my login and password as OAuth parameters, not using Basic Authentication.

          • pdubovitsky
            pdubovitsky
            374 Posts
            ACCEPTED ANSWER

            Re: ClearQuest REST API problem

            ‏2014-02-12T15:03:02Z  in response to Karex

            It would not hurt to ask admins for log files.
            In interim time, you can take an IP traces using tcpdump or wireshark when you sending this request from a browser and when you are using curl. Comparing request headers could give you some information too. You can also try using curl from a different host.

            Pavel

  • DonaldN
    DonaldN
    145 Posts
    ACCEPTED ANSWER

    Re: ClearQuest REST API problem

    ‏2014-02-13T00:03:58Z  in response to Karex

    You should use the HTTP header for the Basic Authentication. For the sample 'user:pass', instead of using -u 'user:pass', you should use -H 'Authorization: Basic dXNlcjpwYXNz' (Base64 encoded). So the simplest form for you to access CQWeb using cURL is:

    curl -H 'Authorization: Basic dXNlcjpwYXNz' 'http://165.213.147.58/cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500'

    You should get a feed as the response. To get a JSON response, add the HTTP Accept header: (I think you mistakenly used Content-Type)

    curl -H 'Authorization: Basic dXNlcjpwYXNz' -H 'Accept: application/json' 'http://165.213.147.58/cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500'

    Since the current CQWeb supports OSLC 2.0, it's better to specify as such. If you also like to dump the response header, the complete and proper command should be like:

    curl -H 'Authorization: Basic dXNlcjpwYXNz' -H 'Accept: application/json' -H 'OSLC-Core-Version: 2.0'
    'http://165.213.147.58/cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500' -D file.txt

  • Karex
    Karex
    7 Posts
    ACCEPTED ANSWER

    Re: ClearQuest REST API problem

    ‏2014-02-13T07:30:12Z  in response to Karex

    Funny thing. Today I cannot open this page even in browser. Response is the same as for cUrl:

    HTTP/1.1 401 Unauthorized
    Date: Thu, 13 Feb 2014 07:22:26 GMT
    Server: IBM_HTTP_Server/6.1.0.25 Apache/2.0.47 (Win32)
    WWW-Authenticate: OAuth realm="TN_STANDARD", oauth_problem="token_not_authorized"
    WWW-Authenticate: Basic realm="TN_STANDARD"
    Content-Length: 52
    Keep-Alive: timeout=15, max=99
    Connection: Keep-Alive
    Content-Type: application/x-www-form-urlencoded;charset=UTF-8
    Content-Language: ko-KR
     

    =================EDIT==============

     

    Now I can open http://165.213.147.58/cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500 from browser. Request sent is:

    GET /cqweb/oslc/repo/TN_STANDARD/db/D01/query/79033663?oslc_cm.pageSize=500 HTTP/1.1

    Host: 165.213.147.58

    User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0

    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

    Accept-Language: en-US,en;q=0.5

    Accept-Encoding: gzip, deflate

    Connection: keep-alive

    Authorization: Basic asddasdasdasdasdasdsadsa==

     

    Normal cUrl request is:

    GET /cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500 HTTP/1.1

    Authorization: Basic asddasdasdasdasdasdsadsa==

    User-Agent: curl/7.19.7 (x86_64-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15

    Host: 165.213.147.58

    Accept: */*

     

    After changing cUrl to

    curl -D- -H 'Connection: keep-alive' -H 'Accept-Encoding: gzip, deflate' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' --user-agent 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0' -H 'Authorization: Basic asddasdasdasdasdasdsadsa==' http://165.213.147.58/cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500 > file.txt

    Request is:

    GET /cqweb/oslc/repo/TN_STANDARD/db/DB01/query/79033663?oslc_cm.pageSize=500 HTTP/1.1

    User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0

    Host: 165.213.147.58

    Connection: keep-alive

    Accept-Encoding: gzip, deflate

    Accept-Language: en-US,en;q=0.5

    Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

    Authorization: Basic asddasdasdasdasdasdsadsa==

     

    So it is the same as from browser, but it doesn't work... Still the same 401 response:

    HTTP/1.1 401 Unauthorized
    Date: Thu, 13 Feb 2014 12:59:33 GMT
    Server: IBM_HTTP_Server/6.1.0.25 Apache/2.0.47 (Win32)
    WWW-Authenticate: OAuth realm="TN_STANDARD", oauth_problem="token_not_authorized"
    WWW-Authenticate: Basic realm="TN_STANDARD"
    Content-Length: 52
    Keep-Alive: timeout=15, max=100
    Connection: Keep-Alive
    Content-Type: application/x-www-form-urlencoded;charset=UTF-8
    Content-Language: ko-KR
     

    Updated on 2014-02-13T13:55:50Z at 2014-02-13T13:55:50Z by Karex
    • DonaldN
      DonaldN
      145 Posts
      ACCEPTED ANSWER

      Re: ClearQuest REST API problem

      ‏2014-02-13T22:59:02Z  in response to Karex

      At this stage, it's quite hard to continue troubleshooting without knowing the server side of things. Since you cannot gain access to the server itself, I would suggest you use a different OSLC client such as the Firefox add-on RESTClient to verify all the HTTP headers that you are using before passing on to cURL. Bottom line is, it works for me when I use the command in my previous post.

      Another possibility is that the CQWeb server has not been configured to output XML format (or formats other than HTML), but that should easily be verified.

      • Karex
        Karex
        7 Posts
        ACCEPTED ANSWER

        Re: ClearQuest REST API problem

        ‏2014-02-14T07:48:43Z  in response to DonaldN

        I've created a loop in AutoIt scripting language in which, using WinHttp I request response from the server. Result of this test is very interesting.

        Most of tries are timeouts (after 60 seconds)

        Sometimes, there's a response that is correct. According to wireshark - response is always after about 30 seconds from request.

        Sometimes, there's a response 409 - Conflict.

         

        HTTP/1.1 409 Conflict

        Date: Fri, 14 Feb 2014 07:11:00 GMT

        Server: IBM_HTTP_Server/6.1.0.25 Apache/2.0.47 (Win32)

        X-Frame-Options: SAMEORIGIN

        Set-Cookie: JSESSIONID=0000v8iNVaeVMjaaZizYfMXEzDK:-1; Path=/

        Expires: Thu, 01 Dec 1994 16:00:00 GMT

        Cache-Control: no-cache="set-cookie, set-cookie2"

        Keep-Alive: timeout=15, max=100

        Connection: Keep-Alive

        Transfer-Encoding: chunked

        Content-Type: application/json; charset=UTF-8

        Content-Language: ko-KR

         

        {"oslc_cm:message":"CRVSV0072E \uc694\uccad\uc744 \ucc98\ub9ac\ud558\ub294 \uc911\uc5d0 \uc11c\ubc84 \uc624\ub958\uac00 \ubc1c\uc0dd\ud588\uc2b5\ub2c8\ub2e4.","oslc_cm:statusCode":"409","reasonCode":"conflict","nestedExceptions":[{"oslc_cm:message":"CRVSV1090I RPC server has unexpectedly exited. Any un-saved work may has been lost.","oslc_cm:statusCode":"409","reasonCode":"conflict","nestedExceptions":[{"oslc_cm:message":"","oslc_cm:statusCode":"409"}]}]}

         

        When I open the requested page in browser response comes in about 2 seconds (according to firebug).

         

        Wireshark shows that after opening page in browser, there are also started few TCP connections (please see file CQ1.png)

        With connection started by AutoIt there's much less connections. Why is that? (please see file CQ2.png)

        Attachments

        • DonaldN
          DonaldN
          145 Posts
          ACCEPTED ANSWER

          Re: ClearQuest REST API problem

          ‏2014-02-16T23:27:43Z  in response to Karex

          Always keep in mind that a script can behave differently to a browser. Sometime even IE can behave differently to Firefox. A browser will take care everything (HTTP return code, HTTP headers, cookies and etc) behind the scene while you need to deal with every bit of information in a script. Unless you use a script to "simulate" a browser, don't expect that you will always get the same result as a browser would.

          As for the 30 second time gap (possibly a timeout), it could be caused by the way the script (AutoIt) interacts with network (DNS or something else).

          To be honest, it is way out of the scope of ClearQuest already. If you can get correct results when using a browser, but not with a script, the fault is likely with the client, not the server.

          • Karex
            Karex
            7 Posts
            ACCEPTED ANSWER

            Re: ClearQuest REST API problem

            ‏2014-02-17T08:01:40Z  in response to DonaldN

            Today I've received such a response when using Mozilla:

            HTTP/1.1 401 Unauthorized

            Date: Mon, 17 Feb 2014 07:57:53 GMT

            Server: IBM_HTTP_Server/6.1.0.25 Apache/2.0.47 (Win32)

            WWW-Authenticate: OAuth realm="TN_STANDARD", oauth_problem="token_not_authorized"

            WWW-Authenticate: Basic realm="TN_STANDARD"

            Content-Length: 52

            Keep-Alive: timeout=15, max=100

            Connection: Keep-Alive

            Content-Type: application/x-www-form-urlencoded;charset=UTF-8

            Content-Language: ko-KR

             

            and I receive it all the time...

            Request:

            GET /cqweb/oslc/repo/TN_STANDARD/db/D01/record/?rcm.name=550898 HTTP/1.1

            Host: 165.213.147.58 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:27.0) Gecko/20100101 Firefox/27.0

            Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

            Accept-Language: pl,en-us;q=0.7,en;q=0.3 Accept-Encoding: gzip, deflate 

            Connection: keep-alive Authorization: Basic dsadsadasdsadas=

             

            I can login to web panel at 165.213.147.58 without any problems...

            Updated on 2014-02-17T08:02:38Z at 2014-02-17T08:02:38Z by Karex