UI performance enhancement for periodic synchronization

The GUI depends on OpenStack events to keep data consistent in MongoDB. Certain activities might be performed outside of UI. For example, through REST API or through PowerVC CLI, which does not generate enough information to keep data consistent in MongoDB. In such a case, data on the UI might require 15 minutes to refresh as there is a periodic sync of data between MongoDB and PowerVC every 15 minutes.

Periodical calls from UI server continuously engage PowerVC REST APIs causing VM operational issues such as deploy failures due to request timeout in an attempt to keep data consistent in MongoDB. Listed here are some considerations.
  • Periodic syncing of all resources is removed.
  • For resources that do not have event support, all out of band changes reflect only when user clicks Refresh on the page.
  • User may have to click Refresh to view latest data on a few resource pages.
  • Resources which are not heavily used will be fetched live using REST API on ‘Click’ event from UI.
  • Resources which are in high scale will still be coming from MongoDB to free up REST traffic and to load the UI faster.
  • To handle loss of events or miss or unavailability, asynchronous project specific resource sync happens only when there is a difference of 10 minutes from the last sync as shown in the diagram here.

    This image shows the UI data consistency mechanism.

  • To keep data consistent in MongoDB for such resources (Resources in bold in the table below), following strategy has been incorporated.
Table 1. Resource matrix with event support, Data fetch and onClick background syncing strategies
Resources Events support Data population - List page Data population - Edit page Project specific syncing onClick background syncing
VMs Y Mongo Mongo Y Y
Volumes Y Mongo Live Y Y
Hosts Y Mongo Mongo, SEA call live NA Y
Images Y Mongo Mongo Y Y
Networks Y Mongo Mongo, SEA/SRIOV live N Y
Storage providers Y Mongo Live NA Y
Log Messages Partial Mongo NA Y


(15 minutes)

Users N Live Live Y NA
Groups N Live Live Y NA
Host groups Y Live Mongo NA NA
HMCs N Live + mongo for hosts Mongo NA Y
VM snapshots N Live Mongo N NA
Collocation Rules Y Live Mongo Y NA
Compute templates N Live Mongo Y NA
Deploy templates Y Live Mongo Y NA
SCGs N Live Mongo N NA
Storage templates N Live Mongo N NA
Consistency groups N Live Mongo N NA
FC ports N Live NA NA NA
Fabrics Y Live Mongo NA NA
Image backups Y Live Mongo N NA
Backup nodes Y Live Mongo NA NA
Compute plane nodes Y Live Mongo NA NA
COS Partial Live Mongo NA NA
Projects N Live NA NA NA
Cloud policies N Live NA Y NA
SSK requests Y Mongo Mongo Y Y
Listed below are the other enhancements.
  • Retained only GraphQL nested queries that are required.
  • User preference for pagination is no longer saved when the value is > 1000.
  • Improved UI server logging for better readability, logging sync execution time for resources.
  • Error handling robustness.
Note: Full sync runs initially when ui-server is started. This pulls in data for all resources across all projects. For subsequent ui-server restarts ENABLE_SYNC (/opt/ibm/powervc/new-ui/.env) can be changed to false to avoid running full sync.