Recently we have launched baas
We had some requirements for a "man in the middle" that wraps the IBM Watson Assistant (aka Conversation) Service api.
As we started with a growth mindset, we made the choice for a server-less architecture to start quickly and scale easily.
A bit like the diagram below.
From a developer perspective this is ideal. No server, no run-time to select, just parse incoming JSON and return outgoing JSON.
And should we get more customers, it scales automatically for us !
Python or Java in our case also did not matter.
After a successful initial phase where everything worked from a functional perspective, we had to meet the non-functional requirement for response time.
As you can imagine, a chat-bot should response quickly, so the requirement was set at 2 seconds to respond.
During development we had seen different response times, so we decided to do some measurements.
Below the output form a daily and weekly perspective.
First Java in an IBM Cloud Function, on average 3 seconds, but not really consistent.
|Loadtime chatbot||Cloud function Java|
Next python, because we thought loading the JVM might be the issue here.
On average halve a second responsive time (that is great), but here also not very stable.
|Loadtime chatbot||Cloud function Python|
Next attempt, what if we had a server that was always on, like a liberty profile server.
That is more like it, see the charts below.
|Loadtime chatbot||Liberty Profile Server|
No that we have a "always on" server, we can even improve performance because we can leverage the JEE techniques to do caching and/or sessions.
The server-less architecture is also stateless, so there is no way ( that matches the intended architecture) to achieve this.
Pro's en con's
So will we never again use IBM Cloud Functions (or other server-less solutions) ?
Of course not, as always, it depends :-)
No costs (initially). This would also be the case for your first Liberty Profile Server, but we already have a few, so this is not free any more.
Testing made easy from the browser.
API-Connect only 1 click away
Use this when there is a IFTT scenario, and there is no user waiting on screen for a response
Wake up time, as you can see in the charts, there is something going on for the first time we hit our chat-bot. But this is where the first impression is, so if you really need fast responses, our experience is not to use server-less.
Limited processing time (max 300 seconds), we did not run into this, but it is a limitation (recently doubled: https://www.ibm.com/blogs/bluemix/2018/04/ibm-cloud-functions-doubling-time-limit-executing-actions/)
Since it is stateless by nature, you cannot easily cache expensive data.
So we did NOT use cloud functions for the middleman to our Watson Assistant.
We did use cloud functions to update existing workspaces (chatbots) with new SmallTalk versions.