Topic
  • 4 replies
  • Latest Post - ‏2013-12-21T22:59:02Z by mparadis
mparadis
mparadis
33 Posts

Pinned topic Is direct storage faster than 4GB FC storage?

‏2013-12-12T15:05:01Z |

I am wondering what experience some of you may have in terms of disk access speeds when comparing direct disk to external fibre channel storage.

My understanding of FC is that because of it's nature, it is incredibly fast, sometimes even beating local disk access. That is making the assumption that your FC network speed is right up there as well.

 

Anyhow, just wondering if anyone might be able to enlighten me about this.

 

Thanks.

  • mparadis
    mparadis
    33 Posts

    Re: Is direct storage faster than 4GB FC storage?

    ‏2013-12-16T17:11:14Z  

    Am I posting this in the wrong place?

  • Josh_Corder
    Josh_Corder
    94 Posts

    Re: Is direct storage faster than 4GB FC storage?

    ‏2013-12-16T17:55:43Z  
    • mparadis
    • ‏2013-12-16T17:11:14Z

    Am I posting this in the wrong place?

    The answer to this question is quite difficult to answer based off what you have supplied.  What is the type of disk?  Spinning or SSD?  What's the interfaces?  SATA, SAS, SCSI, IDE?  How is the fibre channel configured?  What's the cache of the disks?  Is there a controller caching IO?  What's the raid level?  

    FC is very fast, but it's also very expensive.  For most applications that do not have large scale processor requirements or high end databases, SAS, NFS, and CIFS will suffice.  It's only when you get into hosts with large amounts of VMs, Oracle or DB2 database requirements, or high amount so IO such as streaming video that I consider FC to even come into play.

    With IP convergence and FCoE, I think it will eventually not matter what protocol you use.  

    This is just my opinion though.  I know in the large scale distributed environments I support, FC is way too expensive to consider it.  

  • mparadis
    mparadis
    33 Posts

    Re: Is direct storage faster than 4GB FC storage?

    ‏2013-12-19T17:08:44Z  

    The answer to this question is quite difficult to answer based off what you have supplied.  What is the type of disk?  Spinning or SSD?  What's the interfaces?  SATA, SAS, SCSI, IDE?  How is the fibre channel configured?  What's the cache of the disks?  Is there a controller caching IO?  What's the raid level?  

    FC is very fast, but it's also very expensive.  For most applications that do not have large scale processor requirements or high end databases, SAS, NFS, and CIFS will suffice.  It's only when you get into hosts with large amounts of VMs, Oracle or DB2 database requirements, or high amount so IO such as streaming video that I consider FC to even come into play.

    With IP convergence and FCoE, I think it will eventually not matter what protocol you use.  

    This is just my opinion though.  I know in the large scale distributed environments I support, FC is way too expensive to consider it.  

    You're right my question was very generic and there are too many variables.

    So let's say in the following condition then.

    I have a BladeCenter being used for esx hosting. All of the vm's operating systems and OS's are on external storage. Only the esx instance runs on local disk on each blade along with swap files. The external storage is fibre channel, so optical connection to the storage at 4GB to spinning disks set up as RAID5. The storage devices have 2GB caching. While there are many other variables still, you get a sense of what I'm running I hope. Database servers have their own direct storage and aren't on FC, only VM's are.

    As a very basic test, I did the following.

    On a vm where without local storage, where it's all over FC.

    # hdparm -tT /dev/sda1

    /dev/sda1:
     Timing cached reads:   1918 MB in  2.00 seconds = 959.42 MB/sec
     Timing buffered disk reads: 500 MB in  2.89 seconds = 172.89 MB/sec
     

    On a blade which only uses it's local SCSI drive.

    # hdparm -tT /dev/sda1

    /dev/sda1:
     Timing cached reads:   1932 MB in  2.00 seconds = 966.29 MB/sec
     Timing buffered disk reads: 176 MB in  3.00 seconds =  58.64 MB/sec
     

    Quite a difference.

  • mparadis
    mparadis
    33 Posts

    Re: Is direct storage faster than 4GB FC storage?

    ‏2013-12-21T22:59:02Z  
    • mparadis
    • ‏2013-12-19T17:08:44Z

    You're right my question was very generic and there are too many variables.

    So let's say in the following condition then.

    I have a BladeCenter being used for esx hosting. All of the vm's operating systems and OS's are on external storage. Only the esx instance runs on local disk on each blade along with swap files. The external storage is fibre channel, so optical connection to the storage at 4GB to spinning disks set up as RAID5. The storage devices have 2GB caching. While there are many other variables still, you get a sense of what I'm running I hope. Database servers have their own direct storage and aren't on FC, only VM's are.

    As a very basic test, I did the following.

    On a vm where without local storage, where it's all over FC.

    # hdparm -tT /dev/sda1

    /dev/sda1:
     Timing cached reads:   1918 MB in  2.00 seconds = 959.42 MB/sec
     Timing buffered disk reads: 500 MB in  2.89 seconds = 172.89 MB/sec
     

    On a blade which only uses it's local SCSI drive.

    # hdparm -tT /dev/sda1

    /dev/sda1:
     Timing cached reads:   1932 MB in  2.00 seconds = 966.29 MB/sec
     Timing buffered disk reads: 176 MB in  3.00 seconds =  58.64 MB/sec
     

    Quite a difference.

    Where are ya? I supplied more info! :)