IC SunsetThe developerWorks Connections platform will be sunset on December 31, 2019. On January 1, 2020, this forum will no longer be available. More details available on our FAQ.
Topic
  • 10 replies
  • Latest Post - ‏2017-09-06T17:34:23Z by Harley Stenzel
Saheev
Saheev
8 Posts

Pinned topic Reusable/Re-shareable filesystem/directories in Datapower

‏2017-08-24T11:14:30Z | datapower

Hi All,

 

I need help regarding our below queries :

 

I'hv posted a concern Regarding Datapower Filesystem's scope over the IBM RFE Community at the below link : http://www.ibm.com/developerworks/rfe/execute?use_case=viewRfe&CR_ID=109081 against a PMR 00943,001,866 .

Our concern is in below :

Description: Using IBM DataPower Integration Appliance XI52 7.1.0.4 on os Windows .
We need to create a common file system and make it share across other 
application domains for making it reusable .

Dear IBM Datapower Architect and Product Development Team -- Please suggest and advise is there any way
to achieve it . If yes, then how ? if not then why ? Does Datapower 
support this concept ? If not , then why ? 
Please provide us explanation as our development and design halt here .

Use Case :

Our 
requirement is there will be a common directory which will store some 
common codebase/logic and can be reusable and accessible from all 
application's domain codebase. A filesystem common reusable directory 
which is re-shareable across/towards other application's domains 
codebase/configurations/filesystems .

We did try to implement the same via making domain visible from one 
domain and uploading comon files to stores directory but not fulfilling our exact requirement .

The feedback we received from IBM RFE given below with our comments attached inline :

You have a couple of choices:

[1] Instead of keeping these files on DP, you can store them in a shared remote location, e.g. on a HTTP server, and simply reference them, (Our Comments Inline) This will change the present architecture and security concern may be there. [2] Create a domain say "Common" and create a MPGW in it that serves these common file from its local: folder when requested by services in other domain on the same Gateway instance.-->(Our Comments Inline) Not meeting exact requirement, I guess. [3] Make either default domain or some specified application "Common" domain as visible to the consuming domains. Defined this way, the consumers can see any files in the local: directory.-->(Our Comments Inline)We tried this but only read-only accessibility. [4] The DataPower Gateway for Docker and Linux form factors allow you to mount directories from the host OS file system, you could utilize that mechanism.-->(Our Comments Inline) It's Docker Engine and Containerization oriented design .

But after that i'm not able to get any further update from them after chasing a while . Hence posted it in public forum if anybody could throw some more light into it .

Will appreciate if anyone let us know about the below possibilities :

Regarding Option 1 : Also, does DataPower has the capability of mounting a shared NFS to one or more domains (option 1) ? If yes, then how ? --> any articles document will be highly appreciated .

having the common services in the Common domain as MPGW services that can be called from other domains in Option ? --> Could please assist us how to explore this . Are you aware of any kind of this example/implementation? If yes could you please share us ..

Please refer the post here as well : https://developer.ibm.com/answers/questions/396047/reusablere-shareable-filesystemdirectories-in-data.html

Thanks in Advance .

  • kenhygh
    kenhygh
    2522 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-08-24T12:41:06Z  

    First off, you aren't running DataPower on Windows, but maybe DataPower in Docker or a VM on Windows.

    I haven't tried this from a shared filesystem like NFS, but I've done it with an external HTTP server. We had a large cluster of DataPowers in Production, and wanted to keep files on each DP to a minimum. So for XSLs, rather than a 'local:///my.xsl', we'd use 'http://common.server/my.xsl'. This allowed us to have a single place where all these lived, so we could simply update the HTTP server and then flush all the clustered DataPower caches. This worked just fine.

    Ken

  • Saheev
    Saheev
    8 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-08-24T12:48:22Z  

    Thanks Ken,

     

    Yes you're correct it's on VMs on Windows . 

    Will appreciate if you elaborate a bit more about the approaches on the solution you've mentioned :

    we'd use 'http://common.server/my.xsl'. This allowed us to have a single place where all these lived, so we could simply update the HTTP server and then flush all the clustered DataPower caches.

     

    Thank & Regards,

    Kingshuk.

     

  • kenhygh
    kenhygh
    2522 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-08-24T13:47:54Z  
    • Saheev
    • ‏2017-08-24T12:48:22Z

    Thanks Ken,

     

    Yes you're correct it's on VMs on Windows . 

    Will appreciate if you elaborate a bit more about the approaches on the solution you've mentioned :

    we'd use 'http://common.server/my.xsl'. This allowed us to have a single place where all these lived, so we could simply update the HTTP server and then flush all the clustered DataPower caches.

     

    Thank & Regards,

    Kingshuk.

     

    In a transform action, for the stylesheet to be executed you can use any of the supported protocols, such as http, and point to off-box. I don't believe GatewayScript yet supports this.

    Setup an HTTP server to respond back with all your XSLs/XMLs/whatever. This could be on a DataPower box; you could set one up on each DataPower if you wanted, but then you'd need to deploy all files on each DataPower when doing an update. 

    In all your transform actions, you then point the stylesheet URL to the HTTP server.

    That help?

    Ken

  • Saheev
    Saheev
    8 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-08-24T14:36:51Z  
    • kenhygh
    • ‏2017-08-24T13:47:54Z

    In a transform action, for the stylesheet to be executed you can use any of the supported protocols, such as http, and point to off-box. I don't believe GatewayScript yet supports this.

    Setup an HTTP server to respond back with all your XSLs/XMLs/whatever. This could be on a DataPower box; you could set one up on each DataPower if you wanted, but then you'd need to deploy all files on each DataPower when doing an update. 

    In all your transform actions, you then point the stylesheet URL to the HTTP server.

    That help?

    Ken

    Hi Ken,

     

    Please allow me to put it as per my understanding as of now in simple manner . Please correct me wherever i'm Wrong.

    For an example :

    There is one http service , say "http1" which points  to the XML/XSL/XSD etc in a common domain . We have a different mpgw Sservice lets "A" in different application domain where according to our need will call the http1 service from any transform action of that mpgw A trough stylesheet URL??

     

    Thanks ,

    Kingshuk.

  • Saheev
    Saheev
    8 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-09-06T10:23:04Z  

    Hi Ken,

     

    Just an update it was worked using "http services" from a common domain were called from particular  services resided in different domain . Great help if could let me know , in this approach what kind of performance challenges , network laggings and latency can be involve there  . 

     

    Alternatively , any idea does Datapower Support Mounting of 3rd Party NFS directly or remotely to achieve the same goal ? 

     

    Thanks & Regards,

    Kingshuk.

  • HermannSW
    HermannSW
    8729 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-09-06T12:41:04Z  
    • Saheev
    • ‏2017-09-06T10:23:04Z

    Hi Ken,

     

    Just an update it was worked using "http services" from a common domain were called from particular  services resided in different domain . Great help if could let me know , in this approach what kind of performance challenges , network laggings and latency can be involve there  . 

     

    Alternatively , any idea does Datapower Support Mounting of 3rd Party NFS directly or remotely to achieve the same goal ? 

     

    Thanks & Regards,

    Kingshuk.

    Great help if could let me know , in this approach what kind of performance challenges , network laggings and latency can be involve there  . 
    >
    You should access the HTTP service from other domain via "http://127.0.0.1/...", then the loopback interface will be used which is much more performant than the ethernet layer.

    You should disable persistent connections on localhost as explained in this technote:
    http://www-01.ibm.com/support/docview.wss?uid=swg21417934

    InfoCenter on NFS:
    https://www.ibm.com/support/knowledgecenter/SS9H2Y_7.5.0/com.ibm.dp.doc/nfs-client_nfssettings_task.html

    Hermann.

    Updated on 2017-09-06T12:42:05Z at 2017-09-06T12:42:05Z by HermannSW
  • Saheev
    Saheev
    8 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-09-06T12:58:45Z  

    Thanks for Information,

    I am not totally clear in the second part . Could you please help me to understand in brief a bit more on "then the loopback interface will be used which is much more performant than the ethernet layer." please .

     

    Kingshuk

  • HermannSW
    HermannSW
    8729 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-09-06T14:10:26Z  
    • Saheev
    • ‏2017-09-06T12:58:45Z

    Thanks for Information,

    I am not totally clear in the second part . Could you please help me to understand in brief a bit more on "then the loopback interface will be used which is much more performant than the ethernet layer." please .

     

    Kingshuk

    I am not a networking guy, but when you access 127.0.0.1 you directly hit the service over a shortcut. When you use the IP address of one of your appliance's ethernet interfaces then the call will have to pass the ethernet card firmware layer, with higher latency, and reaching at the same service finally.

    Hermann.

    Updated on 2017-09-06T14:10:42Z at 2017-09-06T14:10:42Z by HermannSW
  • Saheev
    Saheev
    8 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-09-06T14:22:12Z  
    • HermannSW
    • ‏2017-09-06T14:10:26Z

    I am not a networking guy, but when you access 127.0.0.1 you directly hit the service over a shortcut. When you use the IP address of one of your appliance's ethernet interfaces then the call will have to pass the ethernet card firmware layer, with higher latency, and reaching at the same service finally.

    Hermann.

    Thanks Hermann,

    I never knew this before . Good to have an insight view of it from you .

     

    Thanks & Regards,

    Kingshuk.

  • Harley Stenzel
    Harley Stenzel
    253 Posts

    Re: Reusable/Re-shareable filesystem/directories in Datapower

    ‏2017-09-06T17:34:23Z  
    • HermannSW
    • ‏2017-09-06T14:10:26Z

    I am not a networking guy, but when you access 127.0.0.1 you directly hit the service over a shortcut. When you use the IP address of one of your appliance's ethernet interfaces then the call will have to pass the ethernet card firmware layer, with higher latency, and reaching at the same service finally.

    Hermann.

    You won't see any appreciable latency or performance difference using a native address vs using a loopback address. However, using a loopback address is tidier and I'd still use it for an internal chained service. Additionally, it is more portable as the config moves through its dev/test/deploy lifecycle.

    But if the service I needed to call is also an externally visible service, I would call it directly, but not by IP address. I'd use a host alias to gain the same lifecycle benefits.

     --Harley