Transferring catalog item data

The catalog item ingestion feature transfers item data from an external system into the catalog in Sterling Intelligent Promising. Any invalid input is uploaded back to you as a .csv file for errors to be addressed.

Before you begin

If there are model items, they must be ingested in a separate .csv file. Then, after they are ingested, they can be referenced by child items in another .csv file.

About this task

If you are using the catalog service API to maintain your catalog data, you don't need to upload catalog item data.

It is important to be aware of the following:
  • Upload the .csv items data in the specified format by using the file transfer adapter to make the required information available.
  • You use the .csv upload for data such as primaryInformation, inventoryInformation, inventoryInformationTag, and code fields.
  • If you want to delete items or manage categories, attributes, and components, you can use the appropriate REST APIs.
  • The order of items that are processed in the .csv file might not be in the sequence of items in the file.
  • As a secondary option, catalog item management can be done by using an API request. Sterling Intelligent Promising enables data ingestion from external system by using .csv files.

Invalid items data in the uploaded .csv file

The uploaded .csv items data is validated. However, for any of the invalid items of data in the .csv file, the itemId, unitOfMeasure and validation error descriptions are published to IBM Cloud® Object Storage. You can download the invalid items list for the .csv file from the <Input_Prefix>/itemIngestValidateFailure virtual folder on IBM Cloud Object Storage, in case any items that are uploaded contain invalid input.

Procedure

  1. To create the .csv files, complete the following steps:
    1. For the file name, use the appropriate naming with .csv format. For example, <tenantId>_Items_20240531010101.csv.
    2. For the separator, use the comma (",").
    3. Upload multiple .csv files by compressing them into a tar.gz file.
  2. You must follow the order of the fields shown in the data type table by matching the same order for the fields in the column in the CSV file. For example, if Item ID is the first field that is defined in the data type table, it must follow the same order in the column in the CSV file. Review the following fields and descriptions for the catalog item ingestion data:

Sequence Field name Required Type Description
1. itemId Yes String The item ID for data ingestion.
2. unitOfMeasure Yes String The unit of measure in the context of the item's quantity.
3. itemGroupCode No String Indicates whether the item is a Product.
Note: Only PROD type supported.
4. globalItemId No String A unique global identifier used to cross-reference the item.
5. allowGiftWrap No String An item that can be gift wrapped. Valid values include, TRUE, FALSE (or blank).
6. assumeInfiniteInventory No String An item that is assumed to be always available. Valid values include, TRUE FALSE (or blank).
7. isAirShippingAllowed No String An item that can be shipped by air. Valid values include, TRUE, FALSE (or blank).
8. isDeliveryAllowed No String An item that allows delivery. Valid values include, TRUE, FALSE (or blank).
9. isForwardingAllowed No String An item that can be forwarded for zone skipping. Valid values include, TRUE, FALSE (or blank).
10. isFreezerRequired No String An item that requires a freezer for transportation. Valid values include, TRUE, FALSE (or blank).
11. isHazmat No String An item that has hazardous material. Valid values include, TRUE, FALSE (or blank).
12. isModelItem No String An item that has variations. Valid values include, TRUE, FALSE or blank.
13. isParcelShippingAllowed No String An item that can be shipped through a parcel service. Valid values include, TRUE, FALSE (or blank).
14. isPickupAllowed No String An item where pickup is allowed. Valid values include, TRUE, FALSE (or blank).
15. isProcurementAllowed No String An item that can be procured. Valid values include, TRUE, FALSE (or blank).
16. isReturnable No String A product item that is returnable. Valid values include, TRUE, FALSE (or blank).
17. isShippingAllowed No String An item that allows shipping. Valid values include, TRUE, FALSE (or blank).
18. substitutionAllowed No String An item that allows substitution. Valid values include, TRUE, FALSE (or blank).
19. taxable No String An item that is taxable. Valid values include, TRUE, FALSE (or blank).
20. bundleFulfillmentMode No String Indicates how the bundle is fulfilled.
  • To ship independently, use code 00.
  • To ship together, use code 01.
  • To deliver together, use code 02.
21. colorCode No String The color code for the item.
22. costCurrency No String The currency in which the item cost is defined.
23. countryOfOrigin No String The item's country of manufacturing.
24. defaultProductClass No String The default product class of an item.
25. description No String The description of the item.
26. effectiveEndDate No dateTime The end date until when the item is valid in the format YYYY-MM-DDThh:mm:ss.SSSZ.
27. effectiveStartDate No dateTime The start date from when the item is valid in the format YYYY-MM-DDThh:mm:ss.SSSZ.
28. extendedDescription No String The extended multi-line description of the Item.
29. extendedDisplayDescription No String Display a description for the item.
30. itemType No String The generic type of the item.
31. keywords No String Specifies keywords to be associated with the item.
32. kitCode No String The kit code for the item. BUNDLE indicates bundle item. LK indicates logical kit item. PK indicates physical kit. DPK indicates dynamic physical kit.
33. manufacturerDescription No String The manufacturer's description of an item.
34. manufacturerItemId No String The identifier used by a manufacturer to identify an item.
35. manufacturerName No String The manufacturer of the item.
36. masterCatalogId No String The master calculator ID to which this item belongs.
37. maxOrderQty No Double The maximum order quantity.
38. minOrderQty No Double The minimum order quantity.
39. minReturnWindow No Long The minimum number of days before which return might be accepted for the item.
40. returnWindow No Long The number of days after which return might not be accepted.
41. shipmentConsolidationWindow No Long The number of days that the item can be delayed to be consolidated into a single shipment.
42. modelItemId No String The unique itemId with variations to which this item belongs
43. modelItemUnitOfMeasure No String The unit of measure of the item with variations.
44. primaryEnterpriseCode No String The primary enterprise associated with an item.
45. primarySupplier No String The primary supplier of the item.
46. productLine No String The product line associated with the item.
47. shortDescription No String The short description of the item.
48. sizeCode No String The size code of the item.
49. status No Integer Status of the item.
  • 2000 is for held Status.
  • 3000 is for published status.
50. unitCost Yes Double The cost of one unit of an item.
51. unitHeight Yes Double The height of one unit of item.
52. unitLength Yes Double The length of one unit of item.
53. unitWidth Yes Double The width of one unit of item.
54. unitWeight Yes Double The weight of one unit of item.
55. unitHeightUOM Yes String Unit of measure of item height.
56. unitLengthUOM No String Unit of measure of item length.
57. unitWeightUOM No String Unit of measure of item weight.
58. unitWidthUOM No String Unit of measure of item width.
59. imageUrl No String The image URL of the item.
60. advanceNotificationTime No Long Communication to ship the order to shipping node is made at most these many days before the expected shipment date.
61. timeSensitive No String An item that is time sensitive. Valid values include, TRUE, FALSE (or blank).
62. fifoTracked No String An item that is FIFO tracked. Valid values include, TRUE, FALSE (or blank).
63. itemBasedAllocationAllowed No String An item where the Item-Based Allocation process is allowed. Valid values include, TRUE, FALSE (or blank).
64. useUnplannedInventory No String An item that is allowed to use unplanned inventory. Valid values include, TRUE, FALSE (or blank).
65. tagControlled No String An item that is tag controlled.
  • Y indicates that the item is always tag controlled where the case inventoryTagInformation needs to be passed.
  • N indicates never tag controlled.
  • S indicates sometimes tag controlled.
66. leadTime No Long The lead time for getting the product delivered.
67. maxNotificationTime No Double Communication to ship the order to shipping node is made at most these many shipping calendar hours before the expected shipment date.
68. minNotificationTime No Double Communication to ship the order is made at least these many days before the expected shipment date.
69. onhandSafetyFactorQty No Double The quantity that is excluded from the actual on-hand availability of the item.
70. onhandSafetyFactorPercentage No Double The percentage that is excluded from the actual on-hand availability of the item.
71. futureSafetyFactorPercentage No Double The quantity that is excluded from the actual future availability of the item.
72. defaultExpirationDays No Long The default number of days after which the item expires.
73. processingTime No Long Processing time in days for item special services.
74. batchNo No String The batchNo that can be used as a tag identifier or as a tag descriptor. Valid values are
  • 01 - use as a tag descriptor.
  • 02 - use as tag identifier
  • 03 - do not use.
75. lotAttribute1 No String The lotAttribute1 that can be used as a tag identifier or as a tag descriptor. Valid values are
  • 01 - use as a tag descriptor.
  • 02 - use as tag identifier
  • 03 - do not use.
76. lotAttribute2 No String The lotAttribute2 that can be used as a tag identifier or as a tag descriptor. Valid values are
  • 01 - use as a tag descriptor.
  • 02 - use as tag identifier
  • 03 - do not use.
77. lotAttribute3 No String The lotAttribute3 that can be used as a tag identifier or as a tag descriptor. Valid values are
  • 01 - use as a tag descriptor.
  • 02 - use as tag identifier
  • 03 - do not use.
78. lotKeyReference No String The lotKeyReference that can be used as a tag identifier or as a tag descriptor. Valid values are
  • 01 - use as a tag descriptor.
  • 02 - use as tag identifier
  • 03 - do not use.
79. lotNumber No String The lotNumber that can be used as a tag identifier or as a tag descriptor. Valid values are
  • 01 - use as a tag descriptor.
  • 02 - use as tag identifier
  • 03 - do not use.
80. manufactureDate No String The manufactureDate that can be used as a tag identifier or as a tag descriptor. Valid values are
  • 01 - use as a tag descriptor.
  • 02 - use as tag identifier
  • 03 - do not use.
81. revisionNo No String Indicates is revisionNo can be used as a tag identifier or as a tag descriptor. Valid values are
  • 01 - use as a tag descriptor.
  • 02 - use as tag identifier
  • 03 - do not use.
82. commodityCode No String The commodity code of an item.
83. eccnNo No String The ECCN number of an item.
84. harmonizedCode No String The harmonized code of an item.
85. hazmatClass No String The hazardous material classification code of an item.
86. naicsCode No String The NAICS code of an item.
87. nmfcClass No String The NMFC code of an item.
88. nmfcCode No String The NMFC code for an item.
89. pickingType No String The picking the type code of an item.
90. postingClassification No String The posting classification of an item.
91. scheduleBcode No String The schedule B code of an item.
92. storageType No String The storage type code of an item.
93. taxProductCode No String The tax product code of an item.
94. unspsc No String The UNSPSC code of an item.
95. velocityCode No String The velocity code.

  1. Review the following sample .csv file:
    itemId,unitOfMeasure,itemGroupCode,globalItemId,allowGiftWrap,assumeInfiniteInventory,isAirShippingAllowed,isDeliveryAllowed,isForwardingAllowed,isFreezerRequired,isHazmat,isModelItem,isParcelShippingAllowed,isPickupAllowed,isProcurementAllowed,isReturnable,isShippingAllowed,substitutionAllowed,taxable,bundleFulfillmentMode,colorCode,costCurrency,countryOfOrigin,defaultProductClass,description,effectiveEndDate,effectiveStartDate,extendedDescription,extendedDisplayDescription,itemType,keywords,kitCode,manufacturerDescription,manufacturerItemId,manufacturerName,masterCatalogId,maxOrderQty,minOrderQty,minReturnWindow,returnWindow,shipmentConsolidationWindow,modelItemId,modelItemUnitOfMeasure,primaryEnterpriseCode,primarySupplier,productLine,shortDescription,sizeCode,status,unitCost,unitHeight,unitLength,unitWidth,unitWeight,unitHeightUOM,unitLengthUOM,unitWeightUOM,unitWidthUOM,imageUrl,advanceNotificationTime,timeSensitive,fifoTracked,itemBasedAllocationAllowed,useUnplannedInventory,tagControlled,leadTime,maxNotificationTime,minNotificationTime,onhandSafetyFactorQty,onhandSafetyFactorPercentage,futureSafetyFactorPercentage,defaultExpirationDays,processingTime,batchNo,lotAttribute1,lotAttribute2,lotAttribute3,lotKeyReference,lotNumber,manufactureDate,revisionNo,commodityCode,eccnNo,harmonizedCode,hazmatClass,naicsCode,nmfcClass,nmfcCode,pickingType,postingClassification,scheduleBcode,storageType,taxProductCode,unspsc,velocityCode
    
    pen,EACH,PROD,GTIN1,TRUE,FALSE,TRUE,TRUE,FALSE,FALSE,TRUE,FALSE,TRUE,TRUE,FALSE,FALSE,TRUE,FALSE,TRUE,,RED,USD,US,,ballpoint pen,2030-09-09T01:01:01.001Z,2024-05-09T01:01:01.001Z,red color ballpoint pen,,,,,,,,,20,2,1,1,2,,,,,,pen,,3000,10,1,4,1,1,inch,inch,gram,inch,http://image/item/1,,FALSE,FALSE,FALSE,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
    
    book,EACH,PROD,GTIN2,TRUE,FALSE,FALSE,FALSE,FALSE,FALSE,FALSE,FALSE,TRUE,TRUE,TRUE,TRUE,TRUE,TRUE,FALSE,,,USD,US,,notebook,2030-09-09T01:01:01.001Z,2024-05-09T01:01:01.001Z,notebook for students,,,,,,,,,5,3,1,1,2,,,,,,book,,3000,6,1,8,7,1,inch,inch,gram,inch,http://image/item/2,,FALSE,FALSE,FALSE,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
  2. Complete one of the following steps:
    • Upload a single file to the <Input_Prefix>/UPLOAD/ITEM/ virtual folder on IBM Cloud Object Storage.
    • Upload all files from a local directory to the <Input_Prefix>/UPLOAD/ITEM/ virtual folder on IBM Cloud Object Storage.
  3. Familiarize yourself with the file format, for example:
    • If the original file is called 2024-01-01-10-59-00.0000-3423-Availability.csv, for simplicity, you might call this file <myfilename>.csv.
    • If the data row fails to process, you receive the following error row.csv and the error summary.txt files:
      <myfilename>_file_failed_row.csv. for the error rows only (all successful are not present here)
      <myfilename>_file_failure_reason.txt are the summary file which provides the error code. Sterling Intelligent Promising automatically creates a prefix to store processed files based on their current state which includes:
      The error row.csv file is the original ingestion record that can't be processed. The error summary.txt file provides the reason for the processing failure.
  4. Understand the prefix that is created automatically by Sterling Intelligent Promising to store processed files that are based on the following current states:
    /failed
    Any CSV row that fails to process is placed in a file for download and review. As part of the monitoring process, you should review any files that are placed in this prefix. Then, you must rectify the failure and resubmit the CSV rows for reprocessing in the input stream.
    /reprocessed
    Any files that are in a queue for reprocessing due to connectivity issues. No actions are required for this prefix. Any files that are stored in the reprocessing prefix temporarily are retired automatically in Sterling Intelligent Promising. More actions are not required to handle the data.
    /completed
    All files that completed processing. It is important to clean up the completed files routinely to minimize the storage requirements on IBM Cloud Object Storage.
    /summary
    A data ingestion summary file that describes the error that occurred during execution. This summary file can be used in conjunction with the files that failed to process to help to understand the error code and the reason for failure.

Example

The system generates the following sample of the prefix generated by the data ingestion system:
  • /failed/<Input_Prefix>/sip-item/myinput
  • /completed/<Input_Prefix>/sip-item/myinput
  • /reprocess/<Input_Prefix>/sip-item/myinput
  • /summary/<Input_Prefix>/sip-item/myinput

Depending on the rules that are configured, you might see sip-data-capture included instead of sip-catalog.