Topic
5 replies Latest Post - ‏2013-02-22T17:45:16Z by SystemAdmin
B2T8_Scott_Stewart
B2T8_Scott_Stewart
2 Posts
ACCEPTED ANSWER

Pinned topic Performance Issues w/ IDS 10.00 Running On VMWare Guest

‏2012-05-23T22:03:19Z |
Hello!

We have been attempting to troubleshoot an performance issue with IDS instance hosted on VMWare (ESXi v4.1) guests. The guest OS is Red Hat Enterprise Linux, versions 4.9 and 5.8 (64-bit). During ETL operations, particularly exports from the database, we are seeing as much as a 2x increase in time required to complete the operation, as compared with the original physical hosts from which these instances were migrated. This occurs with both versions of Red Hat and two different versions of IDS ( 10.00 and 11.70.FC4EX). We've tried multiple memory and core configurations, RAID types, etc. to no avail. During test runs, monitoring at the IDS level shows a 3 sec latency per transaction in some cases, while the OS and VMWare report little if any I/O activity or latency.

Write performance from a strictly OS perspective using tools such as iozone and dd show performance in the 380 - 620 MB/sec range which is faster than the original physical hosts, yet IDS doesn't seem to be able to take advantage of the increase in capacity. We have cases open with VMWare and IBM, but thus far, nothing has budged the needle.

I know that there are quite a few variables involved in an virtualized environment and over the past couple of months, I feel like we've tried every combination in the book. Other VM-based DBMS environments, such as Oracle, do not seem to be experiencing these issues in our organization. Before we abandon the effort to move IDS into a virtualized configuration, I'd like to see if anyone else has experience with mid-sized (500 GB - 1 TB) Informix instances in VMWare.

Thanks in advance for any information that you can provide!
Scott
Updated on 2013-02-22T17:45:16Z at 2013-02-22T17:45:16Z by SystemAdmin
  • JJ@IBM
    JJ@IBM
    18 Posts
    ACCEPTED ANSWER

    Re: Performance Issues w/ IDS 10.00 Running On VMWare Guest

    ‏2012-05-24T07:17:48Z  in response to B2T8_Scott_Stewart
    Hi,

    What device type are you using at the VM level? i.e. Files, Block or Raw

    Under 11.70.FC4, have you got direct I/O set?

    Are you using KAIO or AIO?

    When using dd, what block size did you use?

    How did you split up the device(s)? i.e. Under LVM or direct

    JJ

    "You are what you is"
    • B2T8_Scott_Stewart
      B2T8_Scott_Stewart
      2 Posts
      ACCEPTED ANSWER

      Re: Performance Issues w/ IDS 10.00 Running On VMWare Guest

      ‏2012-05-24T13:49:35Z  in response to JJ@IBM
      Hi, JJ

      We've tried just about every type of structure for our extract output with no measurable difference in performance:

      • ext3 filesystem
      • ext3 filesystem with noatime parameter
      • Direct I/O
      • raw slice

      Disks were managed by Linux native LVM in all cases.

      As far the dd command is concerned, I've tried a number of different block sizes (2k/4k/8k) with the following syntax:

      time bash -c "dd if=/dev/zero of=/quickstore/ddtest bs=2k count=500000; sync; sync"

      The database in question uses a mix of 2k and 8k pagesizes (most are 2k).
      Thanks!
      Scott
      • BlinddogMS
        BlinddogMS
        10 Posts
        ACCEPTED ANSWER

        Re: Performance Issues w/ IDS 10.00 Running On VMWare Guest

        ‏2012-06-06T14:15:54Z  in response to B2T8_Scott_Stewart
        Hi.
        Have you seen this material, entitled:
        "Best Practices for Running Informix in VMWare vSphere Virtual Environments" ?
        Check it out, maybe it give you some great tips...

        https://www.ibm.com/developerworks/wikis/display/ifmxchatwithlab/Home

        Regards.
      • SystemAdmin
        SystemAdmin
        1143 Posts
        ACCEPTED ANSWER

        Re: Performance Issues w/ IDS 10.00 Running On VMWare Guest

        ‏2013-02-22T17:45:16Z  in response to B2T8_Scott_Stewart
        Scott,

        I'm curious if you were able to resolve the performance issue w/IDS 10 & 11 running on a VMware guest?

        We seem to be running into a similar issue, although it only occurs with our backups.

        Thanks,

        John
  • andreasl
    andreasl
    364 Posts
    ACCEPTED ANSWER

    Re: Performance Issues w/ IDS 10.00 Running On VMWare Guest

    ‏2012-05-24T09:18:02Z  in response to B2T8_Scott_Stewart
    Hi,

    "Write performance ... faster than the original physical hosts" sounds suspicious and could mean some sort of caching involved - something you should consider very carefully (with any (vendor's) database activity). A database write to disk needs to be 100% reliable, if data consistency is of any relevance this might be at least as important as speed.

    If Informix write performance isn't as fast this simply might mean it is smart enough (or configured well enough) to circumvent such caching and write directly to disk, so what you're sensing simply might be "physical disk speed" (whatever "physical disks" are here).

    JJ's questions regarding KAIO and/or direct i/o would shed some light on this (well, DIRECT_IO shouldn't be available yet in v10 afaik).
    There also is a conv=direct option to dd for writing through during a dd test.

    HTH,
    Andreas