Topic
  • 11 replies
  • Latest Post - ‏2013-02-18T16:56:54Z by SystemAdmin
SystemAdmin
SystemAdmin
7615 Posts

Pinned topic Export BPM Managed File

‏2013-02-17T17:44:31Z |
Dear BPM Forum,

I built a BPM demonstration that use comma separated value (CSV) loaded as managed BPM server files as a data source. How can I export the managed files so a demo user can edit the data in the CSV files? I know you can reupload the files again to overwrite the existing ones but this doesn't help me with distribution of the demo unless I include the CSVs and tell people to reupload them if they make a change to the CSV files.

Thanks
Scott
Updated on 2013-02-18T16:56:54Z at 2013-02-18T16:56:54Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T15:26:00Z  
    I'm not totally sure I understand your use case. Are you talking about designers modifying the CSV file to change a configuration like setting? Or are you talking about end users downloading and editing the CSV as part of the process?

    David
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T15:34:03Z  
    I'm not totally sure I understand your use case. Are you talking about designers modifying the CSV file to change a configuration like setting? Or are you talking about end users downloading and editing the CSV as part of the process?

    David
    Hi Dean,

    Thank you very much for responding. My last few posts were never answered at all and this is great.

    The CSV files are data, and not config. They are flat files of an Excel save as. I didn't want to have to depend on a relational database for my demo. Actually I have a check-box called simiulator and when checked data is read from the CSV flat files instead of querying a real system. As far as user experience same though. So I load them into memory in the managed files so they are included with the BPM exported snapshot. I would like users to be able to export the CSV files and change the data within if needed. Otherwise, I can include the CSV files separately with directions on how to re-add it after making a change

    Scott
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T15:34:07Z  
    I'm not totally sure I understand your use case. Are you talking about designers modifying the CSV file to change a configuration like setting? Or are you talking about end users downloading and editing the CSV as part of the process?

    David
    Hi Dean,

    Thank you very much for responding. My last few posts were never answered at all and this is great.

    The CSV files are data, and not config. They are flat files of an Excel save as. I didn't want to have to depend on a relational database for my demo. Actually I have a check-box called simiulator and when checked data is read from the CSV flat files instead of querying a real system. As far as user experience same though. So I load them into memory in the managed files so they are included with the BPM exported snapshot. I would like users to be able to export the CSV files and change the data within if needed. Otherwise, I can include the CSV files separately with directions on how to re-add it after making a change

    Scott
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T15:57:10Z  
    Hi Dean,

    Thank you very much for responding. My last few posts were never answered at all and this is great.

    The CSV files are data, and not config. They are flat files of an Excel save as. I didn't want to have to depend on a relational database for my demo. Actually I have a check-box called simiulator and when checked data is read from the CSV flat files instead of querying a real system. As far as user experience same though. So I load them into memory in the managed files so they are included with the BPM exported snapshot. I would like users to be able to export the CSV files and change the data within if needed. Otherwise, I can include the CSV files separately with directions on how to re-add it after making a change

    Scott
    I'm sorry, that's still not clear to me.

    1) I'm still not sure what you mean by "users". Since a managed file is "per app" and bundled with the snapshot, not "per instance", an end user can't change a managed file. (Any more than they could change any of the coaches or code associated with a snapshot.)
    2) What do you mean by "load them into memory"? Making something a managed file doesn't really the file into memory. Not really, anyway. I might be picking nits here, but since I'm not understanding you I want to make sure that there isn't something fundamental you are misunderstanding here about managed files.
    3) Speaking of which, how are you actually reading the CSV files as data? It may not be relevant to solving your problem, but it might help me understand what you are doing. In fact, I would think that using a managed file as "data" would be relatively challenging.

    I guess the bottom line is that you might just be looking for the "how to extract a managed file" section of the docs . http://publib.boulder.ibm.com/infocenter/dmndhelp/v7r5m1/index.jsp?topic=%2Fcom.ibm.wbpm.bpc.doc%2Ftopics%2Frdev_jsapi_startinganewprocess.html But I'm still not sure. I've had some cases where I've done something like that: included a managed file in the app, allowed a user to download it, modify it, and re-upload it as an attached file. But reading a CSV as a datasource would seem to be a very tricky proposition to begin with.

    I'd also say that "not wanting to depend on a relational database" may be more trouble than it's worth. You are guaranteed to a have a relational database because you are running BPM. And if it's just a demo, you could just write some simple services to create your tables inside of the proc database. (In a production app, you wouldn't want to use the same database as BPM.) There may be some minor challenges (such as dealing with different database types), but I think it would be much easier than dealing with CSVs as data stores.

    David
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T16:08:00Z  
    I'm sorry, that's still not clear to me.

    1) I'm still not sure what you mean by "users". Since a managed file is "per app" and bundled with the snapshot, not "per instance", an end user can't change a managed file. (Any more than they could change any of the coaches or code associated with a snapshot.)
    2) What do you mean by "load them into memory"? Making something a managed file doesn't really the file into memory. Not really, anyway. I might be picking nits here, but since I'm not understanding you I want to make sure that there isn't something fundamental you are misunderstanding here about managed files.
    3) Speaking of which, how are you actually reading the CSV files as data? It may not be relevant to solving your problem, but it might help me understand what you are doing. In fact, I would think that using a managed file as "data" would be relatively challenging.

    I guess the bottom line is that you might just be looking for the "how to extract a managed file" section of the docs . http://publib.boulder.ibm.com/infocenter/dmndhelp/v7r5m1/index.jsp?topic=%2Fcom.ibm.wbpm.bpc.doc%2Ftopics%2Frdev_jsapi_startinganewprocess.html But I'm still not sure. I've had some cases where I've done something like that: included a managed file in the app, allowed a user to download it, modify it, and re-upload it as an attached file. But reading a CSV as a datasource would seem to be a very tricky proposition to begin with.

    I'd also say that "not wanting to depend on a relational database" may be more trouble than it's worth. You are guaranteed to a have a relational database because you are running BPM. And if it's just a demo, you could just write some simple services to create your tables inside of the proc database. (In a production app, you wouldn't want to use the same database as BPM.) There may be some minor challenges (such as dealing with different database types), but I think it would be much easier than dealing with CSVs as data stores.

    David
    Sorry, that was the wrong link. I had it in my clipboard from another post. I couldn't find the infocenter page that I had before, but it was essentially an example of using the Javascript API to work with managed files. ( http://publib.boulder.ibm.com/infocenter/dmndhelp/v7r5m1/index.jsp?topic=%2Fcom.ibm.wbpm.ref.doc%2Fjavascript%2FJSAPI.html ) Basically just using findManagedFileByPath to get a managed file object and then writeDataToFile to save the file to disk.

    David
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T16:26:45Z  
    I'm sorry, that's still not clear to me.

    1) I'm still not sure what you mean by "users". Since a managed file is "per app" and bundled with the snapshot, not "per instance", an end user can't change a managed file. (Any more than they could change any of the coaches or code associated with a snapshot.)
    2) What do you mean by "load them into memory"? Making something a managed file doesn't really the file into memory. Not really, anyway. I might be picking nits here, but since I'm not understanding you I want to make sure that there isn't something fundamental you are misunderstanding here about managed files.
    3) Speaking of which, how are you actually reading the CSV files as data? It may not be relevant to solving your problem, but it might help me understand what you are doing. In fact, I would think that using a managed file as "data" would be relatively challenging.

    I guess the bottom line is that you might just be looking for the "how to extract a managed file" section of the docs . http://publib.boulder.ibm.com/infocenter/dmndhelp/v7r5m1/index.jsp?topic=%2Fcom.ibm.wbpm.bpc.doc%2Ftopics%2Frdev_jsapi_startinganewprocess.html But I'm still not sure. I've had some cases where I've done something like that: included a managed file in the app, allowed a user to download it, modify it, and re-upload it as an attached file. But reading a CSV as a datasource would seem to be a very tricky proposition to begin with.

    I'd also say that "not wanting to depend on a relational database" may be more trouble than it's worth. You are guaranteed to a have a relational database because you are running BPM. And if it's just a demo, you could just write some simple services to create your tables inside of the proc database. (In a production app, you wouldn't want to use the same database as BPM.) There may be some minor challenges (such as dealing with different database types), but I think it would be much easier than dealing with CSVs as data stores.

    David
    1) By users, I mean Process Designer users. I build demonstrations and distribute it to many users. Some users many want to change the data in the CSV file.

    2) I have a server side BPM JavaScript block that loads into a business object in the process flow.

    3) Yes, I am actually reading the CSV files in as data. In simulation mode, I wanted users to be able to run the process without having to configure and depend on relational database. Thanks for the link I believe that's all I need on how to export a managed file. Interesting suggestion on adding my own tables to the BPM's internal DB. The JavaScript code is very simple as long as you know the CSV file doesn't contain any comma's besides the field delimiter. Here's a basic example to read CSV file and move into a two dimensional list:

    /* First line in CSV file contains metadata, preferably no spaces and is used as keys for the in memory record list */
    var data = tw.system.model.findManagedFileByPath("data.csv", TWManagedFile.Types.Server).readText();
    var lines = [];
    var records = new Array();
    lines = data.split(/\r\n|\n/);
    var keys = lines[0].split(',');

    for (var i=1; i < lines.length; i++) {
    var entries = lines[i].split(',');
    var record = new Object();
    for (var j=0; j < entries.length; j++) {
    record[keysj] = entries[j];
    }
    records[i - 1] = record;
    }
    /* End JavaScript Example */

    The only thing is that code will NOT work if the CSV values contain comma's within. I removed all comma's in the field values in the CSV file. I know JQuery has a CSV file loader but don't know how to load that into a server side BPM JavaScript block. In order to correctly parse a CSV file with quoted data like: Test, "Dear, John", Value 3, Value4, is much more complicated then what I have above. Also BPM Designer syntax checker gives a warning on this line: record[keysj] = entries[j]; But, its legit JavaScript code.
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T16:32:44Z  
    1) By users, I mean Process Designer users. I build demonstrations and distribute it to many users. Some users many want to change the data in the CSV file.

    2) I have a server side BPM JavaScript block that loads into a business object in the process flow.

    3) Yes, I am actually reading the CSV files in as data. In simulation mode, I wanted users to be able to run the process without having to configure and depend on relational database. Thanks for the link I believe that's all I need on how to export a managed file. Interesting suggestion on adding my own tables to the BPM's internal DB. The JavaScript code is very simple as long as you know the CSV file doesn't contain any comma's besides the field delimiter. Here's a basic example to read CSV file and move into a two dimensional list:

    /* First line in CSV file contains metadata, preferably no spaces and is used as keys for the in memory record list */
    var data = tw.system.model.findManagedFileByPath("data.csv", TWManagedFile.Types.Server).readText();
    var lines = [];
    var records = new Array();
    lines = data.split(/\r\n|\n/);
    var keys = lines[0].split(',');

    for (var i=1; i < lines.length; i++) {
    var entries = lines[i].split(',');
    var record = new Object();
    for (var j=0; j < entries.length; j++) {
    record[keysj] = entries[j];
    }
    records[i - 1] = record;
    }
    /* End JavaScript Example */

    The only thing is that code will NOT work if the CSV values contain comma's within. I removed all comma's in the field values in the CSV file. I know JQuery has a CSV file loader but don't know how to load that into a server side BPM JavaScript block. In order to correctly parse a CSV file with quoted data like: Test, "Dear, John", Value 3, Value4, is much more complicated then what I have above. Also BPM Designer syntax checker gives a warning on this line: record[keysj] = entries[j]; But, its legit JavaScript code.
    FYI ... I noticed the web changed the CSV read and convert code some. Here's it is again:

    
    
    /* First line in CSV file contains metadata, preferably no spaces and is used as keys for the in memory record list */ var data = tw.system.model.findManagedFileByPath(
    "data.csv", TWManagedFile.Types.Server).readText(); var lines = []; var records = 
    
    new Array(); lines = data.split(/\r\n|\n/); var keys = lines[0].split(
    ',');   
    
    for (var i=1; i < lines.length; i++) 
    { var entries = lines[i].split(
    ','); var record = 
    
    new Object(); 
    
    for (var j=0; j < entries.length; j++) 
    { record[keys]] = entries[j]; 
    } records[i - 1] = record; 
    } 
    /* End JavaScript Example */
    


    Again, that code isn't a true CSV parser. It only works if the values don't contain commas. A true CSV parser would be much more complicated.
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T16:38:16Z  
    1) By users, I mean Process Designer users. I build demonstrations and distribute it to many users. Some users many want to change the data in the CSV file.

    2) I have a server side BPM JavaScript block that loads into a business object in the process flow.

    3) Yes, I am actually reading the CSV files in as data. In simulation mode, I wanted users to be able to run the process without having to configure and depend on relational database. Thanks for the link I believe that's all I need on how to export a managed file. Interesting suggestion on adding my own tables to the BPM's internal DB. The JavaScript code is very simple as long as you know the CSV file doesn't contain any comma's besides the field delimiter. Here's a basic example to read CSV file and move into a two dimensional list:

    /* First line in CSV file contains metadata, preferably no spaces and is used as keys for the in memory record list */
    var data = tw.system.model.findManagedFileByPath("data.csv", TWManagedFile.Types.Server).readText();
    var lines = [];
    var records = new Array();
    lines = data.split(/\r\n|\n/);
    var keys = lines[0].split(',');

    for (var i=1; i < lines.length; i++) {
    var entries = lines[i].split(',');
    var record = new Object();
    for (var j=0; j < entries.length; j++) {
    record[keysj] = entries[j];
    }
    records[i - 1] = record;
    }
    /* End JavaScript Example */

    The only thing is that code will NOT work if the CSV values contain comma's within. I removed all comma's in the field values in the CSV file. I know JQuery has a CSV file loader but don't know how to load that into a server side BPM JavaScript block. In order to correctly parse a CSV file with quoted data like: Test, "Dear, John", Value 3, Value4, is much more complicated then what I have above. Also BPM Designer syntax checker gives a warning on this line: record[keysj] = entries[j]; But, its legit JavaScript code.
    Ok. Clearer now.

    Now, I think you are just looking how to get the current version of the managed file from the Process Designer. To do that, you can just click on the icon of a file when you have the managed file component open in PD. It will then save it to the local desktop of the PD user. (Yes, this is not particularly obvious.)

    David
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T16:42:41Z  
    FYI ... I noticed the web changed the CSV read and convert code some. Here's it is again:

    <pre class="jive-pre"> /* First line in CSV file contains metadata, preferably no spaces and is used as keys for the in memory record list */ var data = tw.system.model.findManagedFileByPath( "data.csv", TWManagedFile.Types.Server).readText(); var lines = []; var records = new Array(); lines = data.split(/\r\n|\n/); var keys = lines[0].split( ','); for (var i=1; i < lines.length; i++) { var entries = lines[i].split( ','); var record = new Object(); for (var j=0; j < entries.length; j++) { record[keys]] = entries[j]; } records[i - 1] = record; } /* End JavaScript Example */ </pre>

    Again, that code isn't a true CSV parser. It only works if the values don't contain commas. A true CSV parser would be much more complicated.
    Sorry last post. Just noticed the key piece in the JavaScrpt code example is still missing a closing brace and inner counter.

    
    record[keys[j]] = entries[j];
    


    Complete code again:

    
    
    /* First line in CSV file contains metadata, preferably no spaces and is used as keys for the in memory record list */ 
    /* NOTE: Will not parse CSV file correctly if values contain quoted commas. More code is needed for quoted comma values */ var data = tw.system.model.findManagedFileByPath(
    "data.csv", TWManagedFile.Types.Server).readText(); var lines = []; var records = 
    
    new Array(); lines = data.split(/\r\n|\n/); var keys = lines[0].split(
    ','); 
    
    for (var i=1; i < lines.length; i++) 
    { var entries = lines[i].split(
    ','); var record = 
    
    new Object(); 
    
    for (var j=0; j < entries.length; j++) 
    { rrecord[keys[j]] = entries[j]; 
    } records[i - 1] = record; 
    } 
    /* End JavaScript Example */
    
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T16:55:35Z  
    Hi David,

    Yes that works. In BPM 8.0.1, clicking on the folder icon in the right pane above the browse button in the server file view in Process Designer is how I was able to save a local copy. This is exactly what I wanted.

    Scott
  • SystemAdmin
    SystemAdmin
    7615 Posts

    Re: Export BPM Managed File

    ‏2013-02-18T16:56:54Z  
    Sorry last post. Just noticed the key piece in the JavaScrpt code example is still missing a closing brace and inner counter.

    <pre class="jive-pre"> record[keys[j]] = entries[j]; </pre>

    Complete code again:

    <pre class="jive-pre"> /* First line in CSV file contains metadata, preferably no spaces and is used as keys for the in memory record list */ /* NOTE: Will not parse CSV file correctly if values contain quoted commas. More code is needed for quoted comma values */ var data = tw.system.model.findManagedFileByPath( "data.csv", TWManagedFile.Types.Server).readText(); var lines = []; var records = new Array(); lines = data.split(/\r\n|\n/); var keys = lines[0].split( ','); for (var i=1; i < lines.length; i++) { var entries = lines[i].split( ','); var record = new Object(); for (var j=0; j < entries.length; j++) { rrecord[keys[j]] = entries[j]; } records[i - 1] = record; } /* End JavaScript Example */ </pre>
    You might find it easier to just keep the data in a scriptlet. It depends how much data you have, I suppose. A scriptlet would let your users modify the data more easily. They can just edit it right in place. Or, if it is big enough to warrant a better tool, just cut and paste it in from a text editor. The downside is that I don't think scriptlets handle end of lines very well, if I recall correctly. So you'd probably have to have an end-of-line character like a semi-colon.

    David