• 1 reply
  • Latest Post - ‏2012-07-30T09:16:19Z by SystemAdmin
40 Posts

Pinned topic Handling huge volume of data in Catalog

‏2012-07-30T07:51:34Z |

We have a particular requirement in our project:

There are 2 Catalogs : Catalog-A with 100 items and Catalog-B with 25000 items. Now each item of Catalog-A needs to be associated/tied with all 25000 items of Catalog-B and i don't think we can either use relationship attribute or linked attribute to achieve this considering huge volume of 25000.

And even if there is a way to associate 25,000 items,we will end up with combination of 100*25000 = 2500000 items. Can a catalog handle such huge volume ?

Do we have any restrictions on Volume of items in a catalog? After building such huge catalog, we need to make sure it has low performance impact, and we should be able to perform import/export operations on this huge catalog.

Can you please suggest if anyone had come across such implementation/scenario and better way to implement this. Thanks in advance !!
Updated on 2012-07-30T09:16:19Z at 2012-07-30T09:16:19Z by SystemAdmin
  • SystemAdmin
    648 Posts

    Re: Handling huge volume of data in Catalog

    My first question is that if this can be an implicit association rather than a explicit one via some mechanisms which you talked about ?

    If there are very few which you don't associate then can only those be stored and rest assumed to be associated. This would be just a programmatic assumption and convention in your solutioning. Also, did not understand why 100*25000 = 2500000 items will be added ? You are not going to add so many new items anyway.

    If there are various categories of associations in your solution, you might had a hierarchy and map the items to signify a particular case.

    If these are not options you can take, then, I would rather use a custom DB table and have the association matrix stored where there would be 25000 records ...PK would be stored etc etc