Managing Watson Conversation Workspaces
Hickmat 100000QA3T Comments (6) Visits (8910)
Over the course of my recent work with the Watson Conversation service a requirement emerged to support multiple developers working on a common Workspace "code base". Anyone who has had chance to experiment with the Watson Conversation service in anger with a team of conversation developers will have run into this challenge. Just now the tooling (great as it is) doesn't do much to help. When I discussed this with some colleagues there seemed to be two approaches:
After discussing these options with the team I'm working with the view was that they just wanted a single workspace in production and not manage transition between multiple workspaces so opted for duplicating the workspace and then merging. Based on this I decided to look at creating some tooling to help address the merging requirements and also try and help with tracing changes.
I started by looking at the structure of the workspace JSON file which can be exported from the Conversation Tooling and it became clear that it was basically a wrapper container for 3 types of JSON objects (entity, intent and dialog) which were contained in separate array objects. This meant it would be simple task to shred the workspace into the component parts, manage each individually and recreate a fresh workspace by populating arrays in an empty workspace JSON object and given its all based on JSON I could use Cloudant as the storage mechanism. So the conceptual approach I was aiming for was:
From the above you can see that I planned to detect changes in the JSON elements during the import process so as to allow multiple developers to upload the same common workspace but only changed elements would be reflected in the underlying Cloudant database. I also decided to take the approach of keeping the old JSON element document in the Cloudant database and point to it from the changed one. In addition I concluded that it would also make sense to also store the detected changes in the Cloudant document so that they could be reviewed. To support this JSON element management in Cloudant I knew I could use the identifier name in the JSON element as the _id field in the JSON doc. This meant that to archive it off to maintain a change log I just needed to generate a new random and set that as the _id for the old doc. Finally I also figured that it would make sense to add a "project" identifier to the _id for the current elements so the tooling could not only support multiple developers but also multiple projects. So my _id for the Cloudant docs would look like - <element unique id>_:_<project name> and to support the management of projects I would use a specific document in the Cloudant database with an _id of watson/projects.
With the data storage design in place and a set of supporting views defined in the Cloudant database I set about creating the UI app and supporting REST services. The following show the UI I created (warning - I'm no UI design expert )
First the login so I can identify the user
Clicking the "Upload File" button transfers the file to the upload management service which processes it and updated the Cloudant database to reflect the changes in the workspace being uploaded. In this example its the first upload so all the elements need to be inserted and the results screen shows
If I then change the JSON file and re-upload the changes will be identified and the results would look like
Where changes have been identified clicking on the "View" button displays details about the changes which were identified
As well as managing updates and inserts I needed to support deletion of elements and this is accessed by clicking on the "Delete Items" which displays the following screen
From here the individual element types can be selected to show the available elements of that time to select for deletion as shown here
Clicking the "Delete Elements" button will prompt for confirmation
and confirming here deletes the selected elements. When a new Workspace file needs to be generated, this can be done my clicking the "Generate Workspace" button on the home screen. This will present
In this screen the name and description for the workspace can be provided (NB: the name is a required field). Once enabled the "Generate" button can be clicked and the generated workspace will be displayed for review.
The JSON can be dowloaded by clicking the "Dowload" button. The final option available from the home screen is "Delete Project". Clicking this presents a confirmation screen
Clicking "OK" will result in the project and all associated JSON element documents associated with the project are deleted.
So that's it, I now have a tool which should be able to help support parallel workspace development within our team. I've still got to do some serious shake down testing but just now it all looks to be working well.