Topic
5 replies Latest Post - ‏2012-10-27T17:43:25Z by SystemAdmin
vashi
vashi
11 Posts
ACCEPTED ANSWER

Pinned topic AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

‏2012-01-18T20:48:59Z |
Wanted to check if anyone experienced this on AIX with JFS2. We recently had performance problem on AIX 6.1 TL6 / p7 770, and had to reboot. But after reboot a lot of user data was truncted to zero. IBM dump analysis said there was lot of pending IO, which caused the performance issue. But any files which were created or in process of being edited before were truncted to zero bytes or files where the data was being written were corrupted.

Is this normal for AIX with JFS2 filesystems, is this something that happens all time with AIX and JFS2.
Updated on 2012-10-27T17:43:25Z at 2012-10-27T17:43:25Z by SystemAdmin
  • MarkTaylor
    MarkTaylor
    230 Posts
    ACCEPTED ANSWER

    Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

    ‏2012-01-25T14:37:38Z  in response to vashi
    What you mean by reboot, is that you forced a dump as the system stopped responding ? therefore all of your pending I/Os were never written to the relevant files .. under normal operation the data would have been written and the file closed .. Does your I/O process open the files O_TRUNC ?

    Rgds
    Mark Taylor
    • SystemAdmin
      SystemAdmin
      6907 Posts
      ACCEPTED ANSWER

      Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

      ‏2012-10-27T17:43:25Z  in response to MarkTaylor
      Hi Mark,

      Exactly same thing happend twice on two of my servers. Initially we had issues with SAN luns and finally we ended up rebooting the box. Afer rebooted, we ran fsck and mounted the file systems.

      Interestingy, whatever the files were created/moved/closed were became 0 byte size. System stopped responding eventhough it is up and runing, finally rebooted with dump option.

      In our case, java process is writing new files and moved old data to archive directories.

      do you have any idea why it is only happening on JFS2. Is there a known issue ?
  • SystemAdmin
    SystemAdmin
    6907 Posts
    ACCEPTED ANSWER

    Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

    ‏2012-10-25T15:36:50Z  in response to vashi
    Curious whatever bacame of your situation, findings, fixes, documentation, etc. I've been seeing the exact same symptom with no resolution as of yet. Unable to find any references other than your post here.
    • SystemAdmin
      SystemAdmin
      6907 Posts
      ACCEPTED ANSWER

      Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

      ‏2012-10-25T15:44:43Z  in response to SystemAdmin
      6.1.0.0
      IBM Power 770 (9117-MMB)
      EMC Symmetrix FCP VRAID
  • SystemAdmin
    SystemAdmin
    6907 Posts
    ACCEPTED ANSWER

    Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

    ‏2012-10-27T17:36:48Z  in response to vashi
    HI,

    Are you able to find out any solution or root cause ? Exact same thing happend twice on two of my servers. Initially we had issues with SAN luns and finally we ended up rebooting the box. Afer rebooted, we ran fsck and mounted the file systems.

    Interestingy, whatever the files were created/moved were become 0 byte size.

    Please let me know your findings.