Topic
9 replies Latest Post - ‏2012-06-13T10:02:34Z by KarlM
SystemAdmin
SystemAdmin
6902 Posts
ACCEPTED ANSWER

Pinned topic jfs2 - one filesystem or many for database ?

‏2012-05-29T13:53:47Z |
Hello,

we have been deploying our new AIX servers. It is going to work for Oracle (typical OLTP). The database size is about 5 TB.
We have 8 LUNs from storage array. Is it better to create one filesystem for this database (using all 8 LUNs) or a few filesystems
on this 8 LUNs ? From a performance point of view which solution is better ?
Updated on 2012-06-13T10:02:34Z at 2012-06-13T10:02:34Z by KarlM
  • UNIX-OS
    UNIX-OS
    14 Posts
    ACCEPTED ANSWER

    Re: jfs2 - one filesystem or many for database ?

    ‏2012-05-29T22:40:34Z  in response to SystemAdmin
    Here is my option.

    1. Change queue_depath to 32 or higher for the LUNs. configure MPIO to round robin, no_reserve_lock.
    2. create on VG with 8 LUNs with smaller pp size (32 or 64 MB).
    3. create multiple LV with max spread option.
    4. mount the file system with noatime option.

    This will help to spread IO to all LUNs and adapters.

    Hope this helps.

    Note:
    if your database using caching inside, you need to limit the maxperm (jfs2 file system cache) to <20%.
    • SystemAdmin
      SystemAdmin
      6902 Posts
      ACCEPTED ANSWER

      Re: jfs2 - one filesystem or many for database ?

      ‏2012-05-30T07:41:31Z  in response to UNIX-OS
      We are going to use filesystemio_options=setall (CIO at the database level) so no caching.
      Since we are new to AIX could you please tell me the commands for creating VG and multiple LV with max spread option ?
      • Tibor_B
        Tibor_B
        41 Posts
        ACCEPTED ANSWER

        Re: jfs2 - one filesystem or many for database ?

        ‏2012-05-30T10:06:15Z  in response to SystemAdmin
        Hi,

        I am no expert for this, so let me correct me if I am wrong

        First you must create VG on single hdisk (/usr/sbin/mkvg (AIX 7.1) or /usr/sbin/mkvg4vp)
        Then you must 'extendvg' to spread VG to other hdisks

        check what hdisks are currently in VG 'lsvg -p $vg'

        and location of LV over hdisks can be show with 'lslv -m $lv'

        This is all from me, I wonder if it is possible reallocate/spread PP anytime or their location on hdisks are permanent.
        • ColombianJoker
          ColombianJoker
          68 Posts
          ACCEPTED ANSWER

          Re: jfs2 - one filesystem or many for database ?

          ‏2012-05-30T19:43:58Z  in response to Tibor_B
          Almost all operations on storage systems in AIX are done ONLINE.

          To create your VG:
          mkvg -s 512 -y OracleDataVG hdisk4 hdisk5 hdisk5 hdisk7 hdisk8 hdisk9 hdisk10 hdisk11
          1. This will create a new OracleDataVG in eigth free disks named hdisk4 to hdisk11, with a physical partition size of 512MB

          To create your LVs:
          for i in 1 2 3 4 5 6 7 8
          do
          mklv -t jfs2 -y OracleData${i}LV -C 8 -S 16M OracleDataVG 1000G
          done
          1. This will create eight logical volumes marked jfs2, with names OracleData1LV to OracleData8LV, with an stripe size of 16MB, spread on eight disks of OracleDataVG, each sized 1000GB

          To create your FSs:
          for i in 1 2 3 4 5 6 7 8
          do
          crfs -v jfs2 -d OracleData${i}LV -m /oracledata/fs${i} -A yes -a log=INLINE -a options=cio
          mount /oracledata/fs${i} && chown oracle /oracledata/fs${i}
          done
          1. This will create eight JFS2 filesystems, with names /oracledata/fs1 to /oracledata/fs8, with auto mount, inline (internal) transaction log, and mounted always with the "bio" option, will change owner to "oracle" user too
          • KarlM
            KarlM
            17 Posts
            ACCEPTED ANSWER

            Re: jfs2 - one filesystem or many for database ?

            ‏2012-05-31T12:29:55Z  in response to ColombianJoker
            Oracle 10g and above will use CIO if FILESYSTEM_IO_OPTIONS
            • KarlM
              KarlM
              17 Posts
              ACCEPTED ANSWER

              Re: jfs2 - one filesystem or many for database ?

              ‏2012-05-31T12:40:29Z  in response to KarlM
              Oracle 10g and above will use CIO if FILESYSTEM_IO_OPTIONS=SETALL so cio mount option is not needed. You should be separating your redo logs and control files into filesystems created with agblksize=512 and your data file filesystems with agblksize=4096, assuming your DB_BLOCK_SIZE >= 4k. Your Oracle DBA should be able to supply you with all the Oracle tuning recommendations easily enough.
              As others have said, you need to be looking at sizing your hdisk queue_depth, HBA num_cmd_elems and max_xfer_size. The number of LUNs you have you might need to up the pv_pbuf_count when you get it running and can monitor.
              • SystemAdmin
                SystemAdmin
                6902 Posts
                ACCEPTED ANSWER

                Re: jfs2 - one filesystem or many for database ?

                ‏2012-06-01T11:14:18Z  in response to KarlM
                Hello,

                thank you all for your advice. I'd like to say that this is not our first Oracle database in our company (we have up to 50) and we have been working with Oracle for over 12 years (but this is our first Oracle database on AIX). We have dedicated filesystem for redo, data and archive logs. But my question is about the data filesystem: if it has a few TBs is it better to have one filesystem or many ? The engineer who implemented this told me that one filesystem could be potential bottleneck so it is better to have many filesystems. I don't believe that jfs2 couldn't copy as a one filesystem but I'd like to be sure ...
                • KarlM
                  KarlM
                  17 Posts
                  ACCEPTED ANSWER

                  Re: jfs2 - one filesystem or many for database ?

                  ‏2012-06-13T10:02:34Z  in response to SystemAdmin
                  Not sure if you meant "I don't believe that jfs2 couldn't copy " or "I don't believe that jfs2 couldn't cope "?

                  There should be no problem using a single filesystem but you will need to monitor pbuf and fsbuf usage and tune if necessary.
                  I suggest you take a look at this doc it you haven't already:

                  http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100883
      • UNIX-OS
        UNIX-OS
        14 Posts
        ACCEPTED ANSWER

        Re: jfs2 - one filesystem or many for database ?

        ‏2012-06-03T14:54:28Z  in response to SystemAdmin
        use this option for mklv command for maxspread.

        
        mklv –e x