Querying or retrieving data on cloud database
The System Implementers (SI) and Administrators working on the IBM Sterling® Order Management System solution on cloud have requirements to query or retrieve data from the database for various purposes such as examining the data populated in the database, investigating issues with business data, troubleshooting, cross checking or analyzing with other systems in their stack, extracting data for analytical purposes, providing back fills, and so on.
About this task
The requirements could be during the application development or testing phases and after the applications are available to the end-user. In some scenarios, they might want to test the database queries that are run by the application APIs to ascertain whether the application works as expected or not. Similarly, the IBM Support personnel who consider the issues reported by SI may need database data to troubleshoot the issues effectively. Such requirements are on-demand basis and generally do not fetch a huge volume of data. There are other requirements like extracting data on continuous feeds near real time or in a deferred manner. The continuous feed solutions keep track of the previous feed and extract what is newly added or modified after the last extract.
Besides that, some implementations may require pulling the entire table data out. They may have internal tools for analysis and significant value was found from pulling and analyzing such data. The characteristics of this requirement is different from the previously mentioned ones. Firstly, it does not mean as a continuous feed, users run this as and when they need it. The same requirement may or may not arise again. Secondly, these can be independent requests, have no relation to the previous attempts even if queries are the same and hence pulls from the entire table data. Thirdly, it need not be a blind extract. It is for the internal tools to analyze and they want only the relevant data to be pulled from the cloud database based on some clause or criteria. Fourthly, because of all the mentioned aspects, there is no control on the volume of matching records. It could be numbering single or double digits, a few hundreds, in thousands or may be 100 thousand or even more. It is purely depending on the clause or criteria that you want to invoke.
There are other subjects to ponder, like the delivery of the query results. There are restrictions on the cloud environment to store the results in a file system and download them separately. Though the Service Definition Framework supports many delivery mechanisms some of them are not best suited for the cloud environment. Therefore, the options are, either send the results as an HTML response to the browser where users made the request or perform it asynchronously and FTP to a remote server accessible to the SI (it can be a customer site also) using a pre-configured SFTP configuration. Another aspect is where to run these kinds of requests. It is possible that processing these requests on the application server can crash itself as there is no guarantee that these requests fetch only a feasible volume of data. Therefore, IBM Sterling Order Management System solution recommends the retrieval of huge volumes of data to run asynchronously on a separate JVM such as Agent or Integration server. These asynchronous transactions require a mechanism to either alert the user when the job is done or how the user can know whether the request is complete.