Topic
  • 14 replies
  • Latest Post - ‏2014-08-20T18:17:46Z by Gordon Siegfriedt
SystemAdmin
SystemAdmin
7615 Posts

Pinned topic Table control performance issue with large number of columns

‏2013-01-29T07:56:20Z |
Hello,
I´m trying to use the Table control from Coaches Toolkit (BPM 8.0.1) for creating large editable data grid (About 500 rows and 40 columns) and I found out that the Table control is quite unusable for that purpose. It consume extremely large amount of physical memory (on client side) and loading of the Table cause freezing of the browser. Even if I reduce the number of rows to 20, the behavior of the Table is quite the same as with 500 rows with pagination setup to 20 rows. I found out that the Table is able to render only about 100 cells (10 rows with 10 columns) in reasonable amount of time but I need to render at least 800 cells (20 rows with 40 columns).

Is there any way how to optimize the Table control to be able to handle 40 columns and at least 20 row per page?
Updated on 2013-02-10T03:56:58Z at 2013-02-10T03:56:58Z by kolban
  • MaheshGollamudi
    MaheshGollamudi
    61 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-29T11:49:16Z  
    Am not sure dojo JavaScript can handle this requirement even if it uses json for rendering the grid,we have used Dhtmlx(paid version) for handling this kind of requirement,the maximum it can handle 1000 rows and 50 columns .Use json rather than XML

    Thanks & Regards
    Mahesh Gollamudi
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-29T12:43:42Z  
    Am not sure dojo JavaScript can handle this requirement even if it uses json for rendering the grid,we have used Dhtmlx(paid version) for handling this kind of requirement,the maximum it can handle 1000 rows and 50 columns .Use json rather than XML

    Thanks & Regards
    Mahesh Gollamudi
    Thank you for your answer. Clean DOJO grid looks like it should handle large amount of data without problem. It´s more look like the IBM implementation in Table control CV slow down the process of creating the grid DOM structure as every cell contains another Coach View. The question is if there is any way how to optimize the Table CV to be able to create grid with 50 columns and 20 rows per page (500 rows total)?
  • kolban
    kolban
    3316 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-29T15:49:43Z  
    Thank you for your answer. Clean DOJO grid looks like it should handle large amount of data without problem. It´s more look like the IBM implementation in Table control CV slow down the process of creating the grid DOM structure as every cell contains another Coach View. The question is if there is any way how to optimize the Table CV to be able to create grid with 50 columns and 20 rows per page (500 rows total)?
    Tomas,
    I am VERY interested in your story. I was producing a demo yesterday with a table of about 20 columns and 50 rows and all hell broke loose. The table took minutes to "release" back to the browser and during that time the CPU consumption of the browser was huge. It looks like we may be at the same point. I'm going to start studying this now.

    What version of the product are you using? I am using 8.0.1.

    I am going to start building some tests now to see if I can recreate the problem outside of my project.

    Neil
  • kolban
    kolban
    3316 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-29T19:29:59Z  
    • kolban
    • ‏2013-01-29T15:49:43Z
    Tomas,
    I am VERY interested in your story. I was producing a demo yesterday with a table of about 20 columns and 50 rows and all hell broke loose. The table took minutes to "release" back to the browser and during that time the CPU consumption of the browser was huge. It looks like we may be at the same point. I'm going to start studying this now.

    What version of the product are you using? I am using 8.0.1.

    I am going to start building some tests now to see if I can recreate the problem outside of my project.

    Neil
    I was able to reproduce the problem quite easily on 8.0.1. I have created IBM defect report 24286,004,000 to track the issue.

    Neil
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-30T06:42:36Z  
    • kolban
    • ‏2013-01-29T19:29:59Z
    I was able to reproduce the problem quite easily on 8.0.1. I have created IBM defect report 24286,004,000 to track the issue.

    Neil
    Neil,
    I´m also using version 8.0.1. It´s look like we have the same problem. Is there any way how to track the defect report as I would like to be inform about progress in this issue?
  • kolban
    kolban
    3316 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-30T15:44:15Z  
    Neil,
    I´m also using version 8.0.1. It´s look like we have the same problem. Is there any way how to track the defect report as I would like to be inform about progress in this issue?
    Hi Tomas,
    What i'd suggest is for you to create your own defect report on-line. In your own defect report give your symptoms, your impact, your description and then conclude by pointing to this forum thread and also say that this may be similar to the defect report number I posted in the last post from me.

    IBM doesn't allow other customers to see the content or track the outcome of other customer's reported defects. This is to protect confidentiality and potentially business data/knowledge that customers may share in order to track the problem (think of them like medical records).

    Once you have created your own report, it will be tracked independently of the one I raised but if they are the same problem, them the outcomes will be linked. If you do this, please post your own defect number.

    Neil
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-30T16:28:31Z  
    • kolban
    • ‏2013-01-30T15:44:15Z
    Hi Tomas,
    What i'd suggest is for you to create your own defect report on-line. In your own defect report give your symptoms, your impact, your description and then conclude by pointing to this forum thread and also say that this may be similar to the defect report number I posted in the last post from me.

    IBM doesn't allow other customers to see the content or track the outcome of other customer's reported defects. This is to protect confidentiality and potentially business data/knowledge that customers may share in order to track the problem (think of them like medical records).

    Once you have created your own report, it will be tracked independently of the one I raised but if they are the same problem, them the outcomes will be linked. If you do this, please post your own defect number.

    Neil
    The reason for the performance issue is not a bug per-se but a design problem in the stock table control.
    The table control has to create sub views for each rendered "cell", which if the views in your cells are expensive to create, already multiplies the overhead. But to make things worse, the first time the table is created, it creates/loads ALL of the subviews for each cell, unloads them ALL, then reloads them all again.

    Lastly, adding or deleting a row in a table again causes a full unload and reload of ALL subviews again every time you add or delete a row.

    If you absolutely must use the stock table control as currently implemented, your only workaround for now is to use pagination. 10 rows per "page" is OK (assuming you don't have more than ~4 or 5 columns) and provides acceptable performance for row additions/deletions too. With this approach, even a table with 5000 rows will behave decently.

    Eric Ducos
    EmeriCon, LLC
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-30T16:37:52Z  
    The reason for the performance issue is not a bug per-se but a design problem in the stock table control.
    The table control has to create sub views for each rendered "cell", which if the views in your cells are expensive to create, already multiplies the overhead. But to make things worse, the first time the table is created, it creates/loads ALL of the subviews for each cell, unloads them ALL, then reloads them all again.

    Lastly, adding or deleting a row in a table again causes a full unload and reload of ALL subviews again every time you add or delete a row.

    If you absolutely must use the stock table control as currently implemented, your only workaround for now is to use pagination. 10 rows per "page" is OK (assuming you don't have more than ~4 or 5 columns) and provides acceptable performance for row additions/deletions too. With this approach, even a table with 5000 rows will behave decently.

    Eric Ducos
    EmeriCon, LLC
    ...I just re-read the very first post on this thread. If you need to display a table with 40 columns, then you probably should forget using the stock table control altogether.

    Instead I would recommend you create your own View type, based on a Dojo grid, or even plain HTML for that matter. This isn't extremely difficult to do if you're familiar with Dojo/JavaScript/HTML.
    The table can be populated from the bound data (at load or view time) or with an attached AJAX service. Either way you would completely bypass the overhead of the current stock table control. You probably wouldn't even want to treat your cells as subviews to further reduce the overhead...

    Eric Ducos
    EmeriCon, LLC
  • kolban
    kolban
    3316 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-30T16:56:57Z  
    ...I just re-read the very first post on this thread. If you need to display a table with 40 columns, then you probably should forget using the stock table control altogether.

    Instead I would recommend you create your own View type, based on a Dojo grid, or even plain HTML for that matter. This isn't extremely difficult to do if you're familiar with Dojo/JavaScript/HTML.
    The table can be populated from the bound data (at load or view time) or with an attached AJAX service. Either way you would completely bypass the overhead of the current stock table control. You probably wouldn't even want to treat your cells as subviews to further reduce the overhead...

    Eric Ducos
    EmeriCon, LLC
    Good posts Eric. I am a fan of building Coach Views and am starting to build out an "as-is" replacement for the out of the box Table control. I have chosen to use "gridx" as the new Dojo level table control. See:

    http://oria.github.com/gridx/

    If anyone wishes to collaborate with me on it, I'd be delighted. You must have good Dojo skills or be very keen of having a new Table control.

    My high level thinking is to NOT attempt to define a table with cells being existing Coach View types. Although this is probably optimal flexibility, I am going to think tactically and have the cells be of a finite number of types:

    • String
    • Integer
    • Decimal
    • Date/Time
    • Image
    • Combo

    This should be quick and fits perfectly the intended architecture of gridX

    Neil
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-01-31T08:00:11Z  
    • kolban
    • ‏2013-01-30T16:56:57Z
    Good posts Eric. I am a fan of building Coach Views and am starting to build out an "as-is" replacement for the out of the box Table control. I have chosen to use "gridx" as the new Dojo level table control. See:

    http://oria.github.com/gridx/

    If anyone wishes to collaborate with me on it, I'd be delighted. You must have good Dojo skills or be very keen of having a new Table control.

    My high level thinking is to NOT attempt to define a table with cells being existing Coach View types. Although this is probably optimal flexibility, I am going to think tactically and have the cells be of a finite number of types:

    • String
    • Integer
    • Decimal
    • Date/Time
    • Image
    • Combo

    This should be quick and fits perfectly the intended architecture of gridX

    Neil
    Thank you for you posts Eric. The reason why I want to use the Table control is that it support all the required functionality as I need to create a grid which will be:
    • able to handle large amount of data
    • has editable cell
    • support adding new rows
    • support deleting existing row
    • can highlight the cells which contains validation errors (server validation is required) and show validation error to user when user focus the cell
    • sorting and filtering would be nice to have feature

    The Table control is not able to view large amount of cell (subviews) so as I´m not familiar with DOJO, I decided to create my own grid. I choose the SlickGrid: https://github.com/mleibman/SlickGrid which is based on the jQuery and jQuery UI.

    I was able to bind the data to grid. It is fast even with 5000 rows and 40 columns so it can handle the large amount of data. Main problem which I have with creating a SlickGrid Coach View is that the grid sometimes didn´t render well. I found out that the reason why this happened is that the SlickGrid required to link large number of files (css and js) and this files loads slower than the Coach renders so in time of creating the grid there aren´t all neccessery files. I have no idea how to force the Coach View to wait when all Included Scripts are loaded. The second problem is that the SlickGrid has very little support for validation.

    Neil,
    GridX look like quite fast grid. I would like to help you with creating of the Coach View, but I´m not sure if I will be able to help you as I´m more an Analyst then Programmer and sadly I have no experience with DOJO.
    -
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-02-08T14:15:45Z  
    • kolban
    • ‏2013-01-30T16:56:57Z
    Good posts Eric. I am a fan of building Coach Views and am starting to build out an "as-is" replacement for the out of the box Table control. I have chosen to use "gridx" as the new Dojo level table control. See:

    http://oria.github.com/gridx/

    If anyone wishes to collaborate with me on it, I'd be delighted. You must have good Dojo skills or be very keen of having a new Table control.

    My high level thinking is to NOT attempt to define a table with cells being existing Coach View types. Although this is probably optimal flexibility, I am going to think tactically and have the cells be of a finite number of types:

    • String
    • Integer
    • Decimal
    • Date/Time
    • Image
    • Combo

    This should be quick and fits perfectly the intended architecture of gridX

    Neil
    Neil - sorry I wasn't watching the thread any more.
    I am definitely happy to work on an alternate implementation with you. Creating a more efficient table implementation was actually on my near-term to-do list.
    I am very comfortable with Dojo and Coach NG. Please contact me at eric.ducos@emericon.com and we can get going on it!

    Regards,

    Eric Ducos
    EmeriCon, LLC
  • kolban
    kolban
    3316 Posts

    Re: Table control performance issue with large number of columns

    ‏2013-02-10T03:56:58Z  
    Neil - sorry I wasn't watching the thread any more.
    I am definitely happy to work on an alternate implementation with you. Creating a more efficient table implementation was actually on my near-term to-do list.
    I am very comfortable with Dojo and Coach NG. Please contact me at eric.ducos@emericon.com and we can get going on it!

    Regards,

    Eric Ducos
    EmeriCon, LLC
    Hi Eric,
    Awesome!!

    I have workable draft of the code so far here:

    http://bpmwiki.blueworkslive.com/display/samples/GridX+Table

    If you have a look at what I have started and we can get some more folks interested, we can start a Github (or other) project to co-operate. If you like what you see and you want me to just continue with it myself, feed in the bug reports and feature requests and I'll turn around as quickly as I can.

    Neil
  • monicabathla
    monicabathla
    1 Post

    Re: Table control performance issue with large number of columns

    ‏2014-08-20T13:09:40Z  
    • kolban
    • ‏2013-02-10T03:56:58Z
    Hi Eric,
    Awesome!!

    I have workable draft of the code so far here:

    http://bpmwiki.blueworkslive.com/display/samples/GridX+Table

    If you have a look at what I have started and we can get some more folks interested, we can start a Github (or other) project to co-operate. If you like what you see and you want me to just continue with it myself, feed in the bug reports and feature requests and I'll turn around as quickly as I can.

    Neil

    Hi Neil,

     

    I have created a grid X table. Was able to create dynamic columns too.

    But i am not able to display the list data in the Table.  I read somewhere I have to create Ajax Service to display the data - ( do you have any sample code) and where in Table Configuration I use this ajax Service.

    Here is my JS for dynamic columns

    tw.local.column = new tw.object.listOf.TableColumn();

    var textColumn = new tw.object.TableColumn();
    textColumn.columnType = "Integer";
    textColumn.fieldName = "tw.local.HeightInFt";
    textColumn.columnTitle = "Feet";
    textColumn.width = 20;
    textColumn.sortable = false;
    textColumn.editable = true;

    var textColumn3 = new tw.object.TableColumn();
    textColumn3.columnType = "Text";
    textColumn3.fieldName = "tw.local.shipOrder.shipDetails.Statelist";
    textColumn3.columnTitle = "State";
    textColumn3.width = 20;
    textColumn3.sortable = false;
    textColumn3.editable = true;

    var textColumn2 = new tw.object.TableColumn();
    textColumn2.columnType = "Integer";
    textColumn2.fieldName = "tw.local.HeightInInches";
    textColumn2.columnTitle = "Inches";
    textColumn2.width = 100;
    textColumn2.options =
    textColumn2.sortable = false;
    textColumn2.editable = true;

    tw.local.column.insertIntoList(0, textColumn3);
    tw.local.column.insertIntoList(0, textColumn2);
    tw.local.column.insertIntoList(0, textColumn);

    tw.local.HeightInFt = new tw.object.listOf.Integer();
    tw.local.HeightInInches = new tw.object.listOf.Integer();

    tw.local.HeightInFt[0] = 1;
    tw.local.HeightInFt[1] = 2;
    tw.local.HeightInFt[2] = 3;

    tw.local.HeightInInches[0] = 4;
    tw.local.HeightInInches[1] = 5;
    tw.local.HeightInInches[2] = 6;

     

    Any help will be highly appreciated.

    Regards

    monica

  • Gordon Siegfriedt
    Gordon Siegfriedt
    11 Posts

    Re: Table control performance issue with large number of columns

    ‏2014-08-20T18:17:46Z  

    Hi Neil,

     

    I have created a grid X table. Was able to create dynamic columns too.

    But i am not able to display the list data in the Table.  I read somewhere I have to create Ajax Service to display the data - ( do you have any sample code) and where in Table Configuration I use this ajax Service.

    Here is my JS for dynamic columns

    tw.local.column = new tw.object.listOf.TableColumn();

    var textColumn = new tw.object.TableColumn();
    textColumn.columnType = "Integer";
    textColumn.fieldName = "tw.local.HeightInFt";
    textColumn.columnTitle = "Feet";
    textColumn.width = 20;
    textColumn.sortable = false;
    textColumn.editable = true;

    var textColumn3 = new tw.object.TableColumn();
    textColumn3.columnType = "Text";
    textColumn3.fieldName = "tw.local.shipOrder.shipDetails.Statelist";
    textColumn3.columnTitle = "State";
    textColumn3.width = 20;
    textColumn3.sortable = false;
    textColumn3.editable = true;

    var textColumn2 = new tw.object.TableColumn();
    textColumn2.columnType = "Integer";
    textColumn2.fieldName = "tw.local.HeightInInches";
    textColumn2.columnTitle = "Inches";
    textColumn2.width = 100;
    textColumn2.options =
    textColumn2.sortable = false;
    textColumn2.editable = true;

    tw.local.column.insertIntoList(0, textColumn3);
    tw.local.column.insertIntoList(0, textColumn2);
    tw.local.column.insertIntoList(0, textColumn);

    tw.local.HeightInFt = new tw.object.listOf.Integer();
    tw.local.HeightInInches = new tw.object.listOf.Integer();

    tw.local.HeightInFt[0] = 1;
    tw.local.HeightInFt[1] = 2;
    tw.local.HeightInFt[2] = 3;

    tw.local.HeightInInches[0] = 4;
    tw.local.HeightInInches[1] = 5;
    tw.local.HeightInInches[2] = 6;

     

    Any help will be highly appreciated.

    Regards

    monica

    The table in the (free) BP3 Brazos UI Toolkit is built to handle large data sets using Ajax calls. It ships with a set of working examples. You may find it helpful: http://bp-3.com/brazos