Topic
  • 8 replies
  • Latest Post - ‏2013-12-17T02:32:01Z by ThomasTse
ThomasTse
ThomasTse
5 Posts

Pinned topic seeking suggestion on architecture setup

‏2013-11-22T08:15:45Z |

Hi all,

We have potential customer which wish to meet the following requirements by using GPFS:

- files places in local with share nothing. e.g. server A places files named ABC*, while server B places files named XYZ*

- if one of file servers (e.g. server A) has been down, files named XYZ* (located at server B) can still be available to access.

Can the above requirements be achieved by GPFS? can it set without no replication? or any other suggestion? Thanks in advance.

Thomas

  • yuri
    yuri
    282 Posts

    Re: seeking suggestion on architecture setup

    ‏2013-11-23T06:13:46Z  

    You could do something like this on a small scale by creating a storage pool per node, and using placement policy to select the desired pool.  You wouldn't have to replicate data, but you would need to replicate metadata.  However, this basic idea is counter to what GPFS is about: uniting capabilities provided by multiple nodes and disks.  If you want all data sitting on a single node, with all of the attendant performance and failure tolerance issues, you don't really need a cluster file system, NFS export of a local fs may be sufficient.

    yuri

  • ufa
    ufa
    147 Posts

    Re: seeking suggestion on architecture setup

    ‏2013-11-28T13:01:35Z  
    • yuri
    • ‏2013-11-23T06:13:46Z

    You could do something like this on a small scale by creating a storage pool per node, and using placement policy to select the desired pool.  You wouldn't have to replicate data, but you would need to replicate metadata.  However, this basic idea is counter to what GPFS is about: uniting capabilities provided by multiple nodes and disks.  If you want all data sitting on a single node, with all of the attendant performance and failure tolerance issues, you don't really need a cluster file system, NFS export of a local fs may be sufficient.

    yuri

    Hi Yuri,

    just a question: in principle is what Thomas asked for a shared-nothing setup, correct? So it might be that (given that his example was for two nodes, but actually there are a few more) a unique name space and general access from all to all is required but data are processed nearly always locally. Would you consider that a reasonable use of GPFS?

    ufa

  • HajoEhlers
    HajoEhlers
    253 Posts

    Re: seeking suggestion on architecture setup

    ‏2013-11-29T16:38:55Z  

    > or any other suggestion? 

    What is the final goal you are trying to archive ?

  • yuri
    yuri
    282 Posts

    Re: seeking suggestion on architecture setup

    ‏2013-12-06T03:10:54Z  
    • ufa
    • ‏2013-11-28T13:01:35Z

    Hi Yuri,

    just a question: in principle is what Thomas asked for a shared-nothing setup, correct? So it might be that (given that his example was for two nodes, but actually there are a few more) a unique name space and general access from all to all is required but data are processed nearly always locally. Would you consider that a reasonable use of GPFS?

    ufa

    Striping data across disks for performance and fault tolerance is the cornerstone of what GPFS does.  One could use storage pools to circumvent this, but it's just not a good fit.  There are other file systems out there, e.g. AFS/DFS, which do something like that: aggregating islands of storage into a global name space, and those would a better fit.  Using a cluster file system where an NFS-exported local file system would be about as effective just introduces unnecessary overhead, in terms of code path length and management complexity.

    yuri

  • Christian_Svensson
    Christian_Svensson
    23 Posts

    Re: seeking suggestion on architecture setup

    ‏2013-12-06T09:46:05Z  

    Hi Tomas,
    Basically what you are looking for is a RSYNC solution or a Microsoft DFS solution between 2 sites?
    My suggestion either you look on RSYNC between your 2 sites or look at SONAS/V7000 Unified that you can get Global Mirror on the data and GPFS function build in to a GUI.

    Or if this is only for home directories, then you should look at cTera version with GPFS and create your own cloud storage.

    Thanks
    Christian

  • ThomasTse
    ThomasTse
    5 Posts

    Re: seeking suggestion on architecture setup

    ‏2013-12-09T04:17:19Z  

    I am sorry I forget specifying customer's platform is RHEL and hence DFS is not a possible solution. Let me describe requirements more:

    Background:

    - solution should be non-stop approach

    - currently, they are using RHEL HA with NFS function enabled. they claims that if there is node failure, it takes times to wait failover complete.

    - customer has application to search news (they claim that the number of news will increase 1 million per day.). The application will handle how to get news files from corresponding nodes.

    - now they have several hundreds node to store news.

    - they don't want to use SAN storage and all GPFS partitions will be come from local drive.

    - recovery time and I/O should have minimum impact since they always have node down due to hardware failure.

    - I/O should have minimum impact when new node has been added.

    Any suggestions and questions are welcome. Thanks.

     

    Thomas

     

  • OeRC-GPFS
    OeRC-GPFS
    19 Posts

    Re: seeking suggestion on architecture setup

    ‏2013-12-09T09:29:06Z  
    • ThomasTse
    • ‏2013-12-09T04:17:19Z

    I am sorry I forget specifying customer's platform is RHEL and hence DFS is not a possible solution. Let me describe requirements more:

    Background:

    - solution should be non-stop approach

    - currently, they are using RHEL HA with NFS function enabled. they claims that if there is node failure, it takes times to wait failover complete.

    - customer has application to search news (they claim that the number of news will increase 1 million per day.). The application will handle how to get news files from corresponding nodes.

    - now they have several hundreds node to store news.

    - they don't want to use SAN storage and all GPFS partitions will be come from local drive.

    - recovery time and I/O should have minimum impact since they always have node down due to hardware failure.

    - I/O should have minimum impact when new node has been added.

    Any suggestions and questions are welcome. Thanks.

     

    Thomas

     

    This sounds like a GPFS FPO solution would fit. Each Node is an NSD server with all its locally owned NSDs in their own failure group. Set metadata and data copies to 2.

    You say in your earlier post that you don't want to use replication. But, if you always have a node down then you will always have some data unavailable.

    If you want non-stop, then set the file system to automatically re-stripe on NSD failure. Then it doesn't matter if you have a catastrophic node failure - your data will be replicated elsewhere and you can throw the failed node away.

    You say the application will target specific nodes for news, which will be fine if the node is up and has never failed. The data the node has generated would be located in its local NSDs (though I don't think that's always guaranteed), plus have a replica "somewhere else". If the node goes down, then your application should try another node as the data will be available elsewhere in the cluster.

  • ThomasTse
    ThomasTse
    5 Posts

    Re: seeking suggestion on architecture setup

    ‏2013-12-17T02:32:01Z  
    • OeRC-GPFS
    • ‏2013-12-09T09:29:06Z

    This sounds like a GPFS FPO solution would fit. Each Node is an NSD server with all its locally owned NSDs in their own failure group. Set metadata and data copies to 2.

    You say in your earlier post that you don't want to use replication. But, if you always have a node down then you will always have some data unavailable.

    If you want non-stop, then set the file system to automatically re-stripe on NSD failure. Then it doesn't matter if you have a catastrophic node failure - your data will be replicated elsewhere and you can throw the failed node away.

    You say the application will target specific nodes for news, which will be fine if the node is up and has never failed. The data the node has generated would be located in its local NSDs (though I don't think that's always guaranteed), plus have a replica "somewhere else". If the node goes down, then your application should try another node as the data will be available elsewhere in the cluster.

    Thank for your kindly reply, I am glad to hear FPO solution is feasible. One more question, is there any step by step guide about FPO setup? I searched official IBM website but no luck on it.

    why I need official guideline because our testing environment always down when 1 node down but no hint to solve. I suspect there is something wrong during setup

    Thanks in advance.