Topic
  • 5 replies
  • Latest Post - ‏2012-10-27T17:43:25Z by SystemAdmin
vashi
vashi
11 Posts

Pinned topic AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

‏2012-01-18T20:48:59Z |
Wanted to check if anyone experienced this on AIX with JFS2. We recently had performance problem on AIX 6.1 TL6 / p7 770, and had to reboot. But after reboot a lot of user data was truncted to zero. IBM dump analysis said there was lot of pending IO, which caused the performance issue. But any files which were created or in process of being edited before were truncted to zero bytes or files where the data was being written were corrupted.

Is this normal for AIX with JFS2 filesystems, is this something that happens all time with AIX and JFS2.
Updated on 2012-10-27T17:43:25Z at 2012-10-27T17:43:25Z by SystemAdmin
  • MarkTaylor
    MarkTaylor
    230 Posts

    Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

    ‏2012-01-25T14:37:38Z  
    What you mean by reboot, is that you forced a dump as the system stopped responding ? therefore all of your pending I/Os were never written to the relevant files .. under normal operation the data would have been written and the file closed .. Does your I/O process open the files O_TRUNC ?

    Rgds
    Mark Taylor
  • SystemAdmin
    SystemAdmin
    6902 Posts

    Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

    ‏2012-10-25T15:36:50Z  
    Curious whatever bacame of your situation, findings, fixes, documentation, etc. I've been seeing the exact same symptom with no resolution as of yet. Unable to find any references other than your post here.
  • SystemAdmin
    SystemAdmin
    6902 Posts

    Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

    ‏2012-10-25T15:44:43Z  
    Curious whatever bacame of your situation, findings, fixes, documentation, etc. I've been seeing the exact same symptom with no resolution as of yet. Unable to find any references other than your post here.
    6.1.0.0
    IBM Power 770 (9117-MMB)
    EMC Symmetrix FCP VRAID
  • SystemAdmin
    SystemAdmin
    6902 Posts

    Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

    ‏2012-10-27T17:36:48Z  
    HI,

    Are you able to find out any solution or root cause ? Exact same thing happend twice on two of my servers. Initially we had issues with SAN luns and finally we ended up rebooting the box. Afer rebooted, we ran fsck and mounted the file systems.

    Interestingy, whatever the files were created/moved were become 0 byte size.

    Please let me know your findings.
  • SystemAdmin
    SystemAdmin
    6902 Posts

    Re: AIX JFS2 Filesystem Files Becomes Zero Due to High I/O

    ‏2012-10-27T17:43:25Z  
    What you mean by reboot, is that you forced a dump as the system stopped responding ? therefore all of your pending I/Os were never written to the relevant files .. under normal operation the data would have been written and the file closed .. Does your I/O process open the files O_TRUNC ?

    Rgds
    Mark Taylor
    Hi Mark,

    Exactly same thing happend twice on two of my servers. Initially we had issues with SAN luns and finally we ended up rebooting the box. Afer rebooted, we ran fsck and mounted the file systems.

    Interestingy, whatever the files were created/moved/closed were became 0 byte size. System stopped responding eventhough it is up and runing, finally rebooted with dump option.

    In our case, java process is writing new files and moved old data to archive directories.

    do you have any idea why it is only happening on JFS2. Is there a known issue ?