Contents


Do more with the IBM Blueworks Live REST API, Part 3

Use Blueworks Live REST API resources interactively with cURL

Comments

Content series:

This content is part # of # in the series: Do more with the IBM Blueworks Live REST API, Part 3

Stay tuned for additional content in this series.

This content is part of the series:Do more with the IBM Blueworks Live REST API, Part 3

Stay tuned for additional content in this series.

In Part 2 of this series, you explored the Representational State Transfer (REST) application programming interface (API) for IBM® Blueworks Live by using the SoapUI test tool. Now, in Part 3, you learn how to retrieve data through the API and how to process it to get the outcomes described in the three use case scenarios. (See use case scenarios for document downloads, user analytics, and tagged artifact lists, in Part 2. Parts 2-4 of the series use the same three use case scenarios as examples.)

Client for URLs (cURL) is a command-line program that allows you to send REST requests. It is available under the open MIT license and is ported for several operating systems. With a UNIX or Linux shell, you can process REST responses in a very flexible and interactive way. Also, you can gather commands in a script that you can run at any time.

You can follow the same approach with Windows if you use Cygwin, a collection of open source tools that provide similar capabilities as a Linux distribution. The following examples are run with Cygwin on Windows, but they also can be run on a standard shell on Linux or any other UNIX system. For details and download information, see https://cygwin.com.

cURL is not part of the default installation set of Cygwin, but it is available as an installable package. Therefore, it must be manually selected during an initial installation , as shown in Figure 1, or added at a later point.

Figure 1. Installation of cURL with the Cygwin Setup tool.
Screen                     capture of the the Cygwin                     Setup tool
Screen capture of the the Cygwin Setup tool

Use cURL with Blueworks Live

First, start a console or workstation and check if you can call the curl command. Then, you can try a first request, as shown in Figure 2.

Figure 2. Example of using cURL in a Cygwin console or workstation on Windows
Screen                     capture of a Cygwin                     console or workstation on Windows
Screen capture of a Cygwin console or workstation on Windows

The first part is always the same, specifying the user name, password, and the base address. Type the following command: curl --basic --user user_name:password https://www.blueworkslive.com

Instead of /api/Auth, you use other resources and parameters, as shown in the following examples. The following examples show only the right part of the command after you type curl --basic --user user_name:password https://www.blueworkslive.com. For example, …/api/Auth means you must type curl --basic --user user_name:password https://www.blueworkslive.com/api/Auth.

However, as you start to use other resources, the original address is forwarded to the local data center that hosts your account. For, example, the local data center in Amsterdam is https://ams001.blueworkslive.com.

So you need to extract the specific address from the REST response data as you did in Part 2. For example, type: curl --basic --user westphal@de.ibm.com:XXX "https://www.blueworkslive.com/api/Auth?account=IBM70&version=20110917"

See Figure 3 for an example output.

Figure 3. Example output for determining the service endpoint
Screen                     capture of the example output
Screen capture of the example output

You also can tell cURL to extract the specific address from the REST response data for you, so for all future calls you can use the following constant: curl --location-trusted --basic --user user_name:password https://www.blueworkslive.com.

With the previous arguments, a request is sent with the GET method (see the Overview of REST resources table in Part 2). If you need to send a request with the POST method, cURL requires the following extra arguments:

curl --location-trusted --basic --user user_name:password -X POST –post302 https://www.blueworkslive.com.

First information retrieval example

Start with the same example that you used with SoapUI in Part 1, and store the response data in a local file named users.json. Use the following command:

…/scr/api/UserList?account=IBM70 > users.json

You can look at the file content with the cat file_name command, for example: cat users.json in this case.

Now you can process the content. A simple way to process the content is to use shell commands. You can hook extra commands with a pipe symbol, | directly to the previous command (instead of the > users.json file output). The pipe hands the output of one command to the next so that each command only must do simple processing. With the following commands, you remove the JavaScript Object Notation (JSON) formatting to extract the text values and get one line per user. These commands must be appended to the commands you have used so far. Use the up arrow key to get your previous command input and append the following code example:

| sed 's/},{/\n/g' | sed 's/.*{//' | sed 's/}.*//g'

The result looks like the following example:

"archived":false,"email":"user1@de.ibm.com","name":"Max","role":"Author","locked":false,"licensed":true,"date":"2014\/01\/21 06:05:26 -06:00","invited":false

"archived":false,"email":"user2@fi.ibm.com","name":"Moritz","role":"Author","locked":false,"licensed":true,"date":"2015\/02\/03 11:42:17 -06:00","invited":false

"archived":false,"email":"user3@de.ibm.com","name":"Fritz","role":"Author","locked":false,"licensed":true,"date":"2014\/07\/30 03:50:14 -05:00","invited":false

Finally, remove the names of the fields such as "archived", "email", "name". The result is a comma-separated list (a .csv file) that you can read and use with Excel. Use the following command:

| sed 's/"[^"]*"://g'

The result looks like the following example:

false,"user1@de.ibm.com","Max","Author",false,true,"2014\/01\/21 06:05:26 -06:00",false
false," user2@fi.ibm.com","Moritz","Author",false,true,"2015\/02\/03 11:42:17 -06:00",false
false," user3@de.ibm.com","Fritz","Author",false,true,"2014\/07\/30 03:50:14 -05:00",false

If you have not done so, put all the commands together (in a single line) and store the result in a file like the following example:

cat users.json | sed 's/},{/\n/g' | sed 's/.*{//' | sed 's/}.*//g' | sed 's/"[^"]*"://g' > users.csv

Now, getting a list of all users with the author role is as simple as typing grep Author users.csv.

Use Case 1 example: Attached documents

For the first use case, you want to get all attached documents from a specific Blueworks Live account and download them to a local file system. Then, you want to get only the files that were added to the Blueworks Live repository since your last call.

Similar to the information retrieval example, start by adding the retrieved data in an output file. Because you need more than one parameter, you need to separate the parameters. However, in shell commands, the & character has special meaning, so protect it with a leading \ as an escape character.

As an alternative, you can frame the complete URL with quotation marks. See the following example:

…/scr/api/ListFiles?account=IBM70\&from=2015-03-01 > files.json

As in the information retrieval example, you create a .csv file from the JSON output. Type the following command:

cat files.json | sed 's/},{/\n/g' | sed 's/.*{//' | sed 's/}.*//g' | sed
                's/"[^"]*"://g' > files.csv

The first lines of the output that you get with the head files.csv command should look similar to the following example:

8714,"30000d4ea8b6c7e","westphal@de.ibm.com","process","Company Guidelines.xlsx","30000d4ea8b6c39","2015-02-10T15:39:59.363-06:00"
287009,"30000d6ea8b36f8","westphal@de.ibm.com","process","Werkstatt.png","c83e8fdd25","2015-02-10T16:53:29.995-06:00"
18113,"30000d6ea8b4125","westphal@de.ibm.com","process","Antrag.png","c83e8fdd25","2015-02-10T16:55:02.481-06:00"

To download the files from the Blueworks Live account, you need only the following two data fields from each line of the .csv file:

  • Column 2 – file ID
  • Column 5 – file name

This time, you need to address the characters with special meaning two times. The awk command removes any single \ character, so you need another \ as an escape character to protect the following & character. Therefore, you use \\& in the following example :

awk -F"," '{printf("curl --location-trusted --basic --user westphal@de.ibm.com:XXX
https://www.blueworkslive.com/scr/api/FileDownload?account=IBM70\\&fileItemId=%s > %s\n", $2, $5);}' files.csv > download_files.sh

The script that you create contains only \&. When you run this script later, it behaves like the commands that we used so far in this tutorial series.

The script looks like the following example:

curl --location-trusted --basic --user westphal@de.ibm.com:XXX
https://www.blueworkslive.com/scr/api/FileDownload?account=IBM70\&fileItemId="30000d4ea8b6c7e" > "Company Guidelines.xlsx"
curl --location-trusted --basic --user westphal@de.ibm.com:XXX https://www.blueworkslive.com/scr/api/FileDownload?account=IBM70\&fileItemId="30000d6ea8b36f8" > "Werkstatt.png"
curl --location-trusted --basic --user westphal@de.ibm.com:XXX https://www.blueworkslive.com/scr/api/FileDownload?account=IBM70\&fileItemId="30000d6ea8b4125" > "Antrag.png"

The last step is to run the script, as shown in the following example.

chmod u+x download_files.sh
./download_files.sh

The downloaded documents are stored in the local folder.

Use Case 2 example: User activities

In the second use case, you want to determine the last time that a user logged in during a specific time period (if at all).

You start by getting the login activities for a certain period. Note that the supported request period is for a maximum of 31 days. For longer periods, you need more than one request call.

As in the previous examples, you clean the output and create one line per entry. Field number 4 and 6 contain endTime and user (compare the SoapUI example in Part 2). Then, you switch, sort in reverse order (so that the last login date comes first), and remove multiple user entries. With one command prompt, you can retrieve the information, as shown in the following example:

curl --location-trusted --basic --user westphal@de.ibm.com:XXX 'https://www.blueworkslive.com/scr/api/activity?account=IBM70&type=LOGINS&startDate=2015-02-02T00:00:00.000-00:00&endDate=2015-02-08T23:59:59.999-00:00' | sed 's/},{/\n/g' | sed 's/.*{//' | sed 's/}.*//g' | sed 's/"[^"]*"://g' | awk -F"," '{print $6,$4}' | sort -r | awk '{if ($1 != prev) print $0; prev = $1}'

The output should look like the following example:

"westphal@de.ibm.com" "2015-02-03T10:16:21.992-06:00"
"sebastian.schmidt@de.ibm.com" "2015-02-06T04:25:18.065-06:00"
…

Use Case 3 example: Tagged artifacts list

In the third use case, you want to produce a report that lists all artifacts that are tagged with a specific keyword and that contains detailed information like artifact type, name, and ID.

In principal you apply the same approach as the previous example:

  1. Request the information.
  2. Create one line per item.
  3. Do further processing if needed.

However, the structure of the search response is more complex. In the previous example, you processed the JSON object with simple text filters. More complex objects require "real" JSON parsing, which means that the parser must understand the nesting of the object that is represented by brackets. Also, a deeper analysis of the returned object might be needed. For more complex scenarios, see Part 4 and Part 5 of the series.

For this Part 3 tutorial, you use a simple, one line command that extracts all of the spaces that are tagged.

Request the information

Run the following command:

curl --location-trusted --basic --user westphal@de.ibm.com:XXX https://www.blueworkslive.com/scr/api/Search?account=IBM70\&version=20120130\&searchFieldName=tag\&searchValue=demo\&returnFields=tag > search.json

Look at the following example result to understand the structure of the response:

{
  "spaces": [
    {
      "tags": [{"id": "100003bea792c7f","name": "demo"}],
      "id": "3ea6e050b",
      "name": "Car Part Company"
    },
    {
      "tags": [{"id": "100003bea792c7f","name": "demo"}],
      "id": "30000d5ea8b30c7",
      "name": "2.3.6 Warranty Management",
      "parentSpaceId": "30000d4ea8be6c7",
      "processes": [
        {
          "tags": [{"id": "100003bea792c7f","name": "demo"}],
          "id": "c83e8fdd25",
          "name": "Warranty Claim",
          "milestones": [
            {
              "id": "c83e8fe11e",
              "name": "Request Creation",
              "activities": [
                {
                  "id": "cc3e8f5cc3",
                  "name": "Fill Form"
                },
                {
                  "id": "cc3e8f5ca2",
                  "name": "Review Response"
                },
                …

The example shows the first part of a JSON response that is formatted and sometimes reordered for better readability. You see two spaces that are both tagged as "demo". The second space also contains a process called "Warranty Claim" that is also tagged. There might be spaces that are not tagged but that contain tagged artifacts. The structure goes as deep as necessary and contains subelements such as the process details that are shown in this example. Child spaces of tagged spaces are also listed.

Create one line per item

To get one line per Blueworks Live space, remove the outer parts of the JSON object with the following command:

cat search.json | sed 's/^{\"spaces\":\[{//'  | sed 's/}\]}$//'

Replace the list with tags (if it contains the tag that you want) with "tag\":\"demo\" as shown in the following example:

| sed 's/\"tags\":\[[^]]*demo[^]]*\]/\"tag\":\"demo\"/g'

Replace all other lists framed by [ and ] with "…" for three levels, as shown in the following example:

| sed 's/\[[^][]*\]/\"...\"/g' | sed 's/\[[^][]*\]/\"...\"/g' | sed 's/\[[^][]*\]/\"...\"/g'

Finally, break to a new line whenever you find },{ and filter out all lines that contain demo, as shown in the following example:

| sed 's/},{/\n/g' | grep demo

See the following example result:

"tag":"demo","id":"3ea6e050b","name":"Car Part Company"
"processes":"...","tag":"demo","id":"30000d5ea8b30c7","name":"2.3.6 Warranty Management","parentSpaceId":"30000d4ea8be6c7"

You obtained a list with the two spaces that are tagged and can see that there are more tagged artifacts, including a process in the second space. You might agree that this third use case is not really suitable for shell pipe processing, unless you can address the nested structure of the response. The example shows the limitations of this approach.

Since December 2015 there is a new Blueworks Live REST API resource called…/scr/api/LibraryArtifact that makes achieving this task a lot easier. However, the attempt that was previously shown to solve the third use case is still a good example for more complex processing. Part 4 of this series shows a more appropriate implementation.

Conclusion

Now you completed the third part in the Blueworks Live REST API series. You know how to make a REST API call to retrieve results with cURL and how to process the results with shell commands.

If you want to learn how to write an easy-to-use Java™ program to do some more complex processing of the results, continue to Part 4.

Acknowledgments

The author would like to thank Margaret Thorpe from the Blueworks Live management team for help with the tutorial concept and Daniel Treiber from the development team and Sean Roe from the technical sales team for their review and comments.


Downloadable resources


Related topics


Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Middleware, Cloud computing
ArticleID=1027825
ArticleTitle=Do more with the IBM Blueworks Live REST API, Part 3: Use Blueworks Live REST API resources interactively with cURL
publish-date=06292017