Topic
  • 11 replies
  • Latest Post - ‏2014-11-06T08:42:17Z by E1F3_varnika_gupta
XIntegration50
XIntegration50
83 Posts

Pinned topic Saving export to Repository

‏2013-08-13T20:41:16Z |

Hi,

I have a SOMA export request which is returning soap response with base64 encoded binary. I am saving this in a temporary: file using dump-nodes as per suggestion from this forum. We are using ClearCase/CVS to save the export for maintaining versions. I feel if we save it as a zip file it would be convenient for users to view the changes in the files etc.  But if I have to do that I have to follow the following two steps

1) Convert base64 encoded binary to .Zip using *nix and save both SOAP response ( so to give as input for import SOMA ) and zip for the convenience of developers to view and to maintain version history.

2) Manually, we copy export ( soap response ) file from dp and load to DP ( while importing ).

I know there are some tools that can do these tasks but we are restricted. So, I have an XML firewall service that makes soma calls, a request from SOAPUI triggers those calls.

I am not sure how implementations are done outside just want to check if there is any thing that can simplify the above tasks or if I have to follow those steps only ?

 

Thanks.

  • swlinn
    swlinn
    1347 Posts

    Re: Saving export to Repository

    ‏2013-08-13T21:04:52Z  

    I take it you're doing an internal SOMA call, perhaps with a service that builds the SOMA request and then sends it to local host at port 5550.  What I've seen customers do is have a linux server issue the SOMA request, kicked off by a cron job.  The process initiated is a perl script that uses curl to send the request, gets the response, does the base 64 decode of the dp:file element of the response, and then saves that into a save directory.  Why not place this requirement on your management server instead of within the appliance?

    Regards,
    Steve

  • XIntegration50
    XIntegration50
    83 Posts

    Re: Saving export to Repository

    ‏2013-08-15T05:49:43Z  
    • swlinn
    • ‏2013-08-13T21:04:52Z

    I take it you're doing an internal SOMA call, perhaps with a service that builds the SOMA request and then sends it to local host at port 5550.  What I've seen customers do is have a linux server issue the SOMA request, kicked off by a cron job.  The process initiated is a perl script that uses curl to send the request, gets the response, does the base 64 decode of the dp:file element of the response, and then saves that into a save directory.  Why not place this requirement on your management server instead of within the appliance?

    Regards,
    Steve

    Hi Steve,

    Your expectation is correct on how I am making SOMA call. The reason why I am doing this way is here

    1) Client doesnt want to use any tools and make this task complex.

    2) Our deployment policy is very simple, we simple export and import and then save them in repository system.  We have few services only and they were doing manually till now.

    So, I have chosen this way and trying to make it as simple as possible for deployers. I believe I have to follow the steps ( converting to zip and saving both .zip and xml resp with base64) .

    Thanks

    ps: Even though we are not planning to implement it is good to learn Pearl and implementing SOMA throgh CRON job.... do we have any manual for this Steve ?

  • swlinn
    swlinn
    1347 Posts

    Re: Saving export to Repository

    ‏2013-08-15T19:17:17Z  

    Hi Steve,

    Your expectation is correct on how I am making SOMA call. The reason why I am doing this way is here

    1) Client doesnt want to use any tools and make this task complex.

    2) Our deployment policy is very simple, we simple export and import and then save them in repository system.  We have few services only and they were doing manually till now.

    So, I have chosen this way and trying to make it as simple as possible for deployers. I believe I have to follow the steps ( converting to zip and saving both .zip and xml resp with base64) .

    Thanks

    ps: Even though we are not planning to implement it is good to learn Pearl and implementing SOMA throgh CRON job.... do we have any manual for this Steve ?

    So SOAPUI as a client could be replaced by cURL, and you could send a SOMA request via cURL, get the response piped to a file, and i have a simple ruby script that extracts the base64  dp:file element to a zip file, which I've attached.  You could place that into a script to do what you want and not have an appliance based service to do this.  It would seem a lot more convenient.  Ultimately you can kick off your script with cron to run it on a scheduled basis.  Perl will allow you to dynamically build your SOMA request, and I assume Ruby could as well.  I just hack around with either as I need a particular function and have something close to use as a starting template.  I'd defer to Hermann on handling the decoded zip file in a stylesheet as I'm not sure if those characters would cause you issues using url open to send them to some destination such as a http server or a ftp server.

    Regards,
    Steve

    Attachments

  • XIntegration50
    XIntegration50
    83 Posts

    Re: Saving export to Repository

    ‏2013-08-19T02:27:18Z  
    • swlinn
    • ‏2013-08-15T19:17:17Z

    So SOAPUI as a client could be replaced by cURL, and you could send a SOMA request via cURL, get the response piped to a file, and i have a simple ruby script that extracts the base64  dp:file element to a zip file, which I've attached.  You could place that into a script to do what you want and not have an appliance based service to do this.  It would seem a lot more convenient.  Ultimately you can kick off your script with cron to run it on a scheduled basis.  Perl will allow you to dynamically build your SOMA request, and I assume Ruby could as well.  I just hack around with either as I need a particular function and have something close to use as a starting template.  I'd defer to Hermann on handling the decoded zip file in a stylesheet as I'm not sure if those characters would cause you issues using url open to send them to some destination such as a http server or a ftp server.

    Regards,
    Steve

    Hi Steve,

    Thanks for the response. I tried to run SOMA scripts with cURL and piped the response to a .zip file and it worked. But to create SOMA requests dynamically and to have control on sequence of SOMA ( for example on successful import SOMA call , initiative next call which is SaveConfig ) I have chosen to create the requests internally through a service ( styleshet ). I try to use Perl which I never tried, to create requests directly.

    I am not sure if there is a way to save binary to zip in an stylesheet. If there is one that is good. However, I see you have been stressing on moving these to management server instead of appliance. Can you please explain what would be impact ?

    Thanks,

    Venkat

  • swlinn
    swlinn
    1347 Posts

    Re: Saving export to Repository

    ‏2013-08-19T13:34:17Z  

    Hi Steve,

    Thanks for the response. I tried to run SOMA scripts with cURL and piped the response to a .zip file and it worked. But to create SOMA requests dynamically and to have control on sequence of SOMA ( for example on successful import SOMA call , initiative next call which is SaveConfig ) I have chosen to create the requests internally through a service ( styleshet ). I try to use Perl which I never tried, to create requests directly.

    I am not sure if there is a way to save binary to zip in an stylesheet. If there is one that is good. However, I see you have been stressing on moving these to management server instead of appliance. Can you please explain what would be impact ?

    Thanks,

    Venkat

    Hi Venkat,

    I'm not much of a perl writer myself, but here's a link I googled on perl with a bunch of examples.  http://accad.osu.edu/~mlewis/Class/Perl/perl.html I'd imagine you would use your input parameters which would contain the domain name, userid, password, any other dynamic data, which would be used to write out a temporary file which you could use in the curl command. The example "Generating XPost Scripts" shows examples of opening a file and using print statements to write to it.  Perhaps there are others on this forum that have already done something similar to what you need as I know these types of scripts have been discussed in this forum in the past, although search for them has recently been impacted by the forum hosting changes.  Hermann has had some luck in the past of using Google to find previous forum posts, so that may be worth a try as well.

    Regards,

    Steve

  • DataPowerDev
    DataPowerDev
    17 Posts

    Re: Saving export to Repository

    ‏2013-08-19T18:10:21Z  
    • swlinn
    • ‏2013-08-19T13:34:17Z

    Hi Venkat,

    I'm not much of a perl writer myself, but here's a link I googled on perl with a bunch of examples.  http://accad.osu.edu/~mlewis/Class/Perl/perl.html I'd imagine you would use your input parameters which would contain the domain name, userid, password, any other dynamic data, which would be used to write out a temporary file which you could use in the curl command. The example "Generating XPost Scripts" shows examples of opening a file and using print statements to write to it.  Perhaps there are others on this forum that have already done something similar to what you need as I know these types of scripts have been discussed in this forum in the past, although search for them has recently been impacted by the forum hosting changes.  Hermann has had some luck in the past of using Google to find previous forum posts, so that may be worth a try as well.

    Regards,

    Steve

    Steve, Do you've any python scripts for doing the deployments into domains ?. Thank you in advance!

  • swlinn
    swlinn
    1347 Posts

    Re: Saving export to Repository

    ‏2013-08-19T20:28:00Z  

    Steve, Do you've any python scripts for doing the deployments into domains ?. Thank you in advance!

    No, sorry I have no python scripts to provide you.

    Regards,

    Steve

  • kenhygh
    kenhygh
    1519 Posts

    Re: Saving export to Repository

    ‏2013-08-19T21:19:35Z  

    Steve, Do you've any python scripts for doing the deployments into domains ?. Thank you in advance!

    This should get you going.

  • DataPowerDev
    DataPowerDev
    17 Posts

    Re: Saving export to Repository

    ‏2013-08-19T22:02:07Z  
    • kenhygh
    • ‏2013-08-19T21:19:35Z

    This should get you going.

    Steve/kenhygh, Thank you very much to you both!.

  • XIntegration50
    XIntegration50
    83 Posts

    Re: Saving export to Repository

    ‏2013-08-21T03:29:07Z  
    • kenhygh
    • ‏2013-08-19T21:19:35Z

    This should get you going.

    Thank you so much Steve and Ken for the info you provided.

  • E1F3_varnika_gupta
    E1F3_varnika_gupta
    1 Post

    Re: Saving export to Repository

    ‏2014-11-06T08:42:17Z  

    Hi Steve,

    Thanks for the response. I tried to run SOMA scripts with cURL and piped the response to a .zip file and it worked. But to create SOMA requests dynamically and to have control on sequence of SOMA ( for example on successful import SOMA call , initiative next call which is SaveConfig ) I have chosen to create the requests internally through a service ( styleshet ). I try to use Perl which I never tried, to create requests directly.

    I am not sure if there is a way to save binary to zip in an stylesheet. If there is one that is good. However, I see you have been stressing on moving these to management server instead of appliance. Can you please explain what would be impact ?

    Thanks,

    Venkat

    Hi Venkat,

    Would you please share that soma request where you have piped the output to zip format.