Topic
  • 4 replies
  • Latest Post - ‏2012-11-13T14:29:34Z by nima.samii
nima.samii
nima.samii
3 Posts

Pinned topic Inconsistent md5sums of big files

‏2012-09-21T15:52:45Z |
Hello,

we're using GPFS for storing the experimental data at our institute.
These files can get rather large (up to and exceeding 100GB).
We use 1 NSD-Server connected to Dell and Promise storage arrays
and 3 GPFS-Clients for data analysis.
Unfortunately on the NSD Server (running RHEL5.2 64bit) we are
getting weired file corruption.
Consecutive md5sum calculations of files approx. bigger than
20GB stand a better than even chance of repeatedly getting
different md5sums. On file of sizes larger than 25GB the likelyhood
of corruption is 100%.
Copying the files from a Backup-Location to the NSD-Server results
in a correct md5sum on the first run and invalid md5sums on following
runs.
Copying big files to an ext3 volume on the NSD-server results in
wrong but constant md5sums, so it is probably not the system/cpu/ram
triggering this.

The GPFS daemon(s) don't register this fault anywhere. As we wan't to
completely switch to GPFS for our data storage this makes us rather
nervous.

We need help debugging this problem and would greatly appreciate
any help in tracking down the issue.

With kind regards,
Nima Samii

Institute of nuclear physics
University of Cologne

Updated on 2012-11-13T14:29:34Z at 2012-11-13T14:29:34Z by nima.samii
  • YuanZhengcai
    YuanZhengcai
    9 Posts

    Re: Inconsistent md5sums of big files

    ‏2012-10-08T07:11:45Z  
    There's a thread which has the similar md5sum issue, you may can find something from it.
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14263476&#14263476
  • nima.samii
    nima.samii
    3 Posts

    Re: Inconsistent md5sums of big files

    ‏2012-10-18T11:35:19Z  
    There's a thread which has the similar md5sum issue, you may can find something from it.
    https://www.ibm.com/developerworks/forums/thread.jspa?messageID=14263476&#14263476
    Thanks for the pointer!
    I'll have a look at the proposed debugging
    in the linked thread and will respond once
    I found a common defect in the md5sum processing.

    With kind regards,
    Nima Saed-Samii
  • Tucks
    Tucks
    78 Posts

    Re: Inconsistent md5sums of big files

    ‏2012-11-13T14:22:31Z  
    Which GPFS version is this?
    We use 3.4.0-10 PTF1 on RH 5.x and 6.x.
    We have files several multiples of the size you say, but see no corruption to date.
  • nima.samii
    nima.samii
    3 Posts

    Re: Inconsistent md5sums of big files

    ‏2012-11-13T14:29:34Z  
    • Tucks
    • ‏2012-11-13T14:22:31Z
    Which GPFS version is this?
    We use 3.4.0-10 PTF1 on RH 5.x and 6.x.
    We have files several multiples of the size you say, but see no corruption to date.
    We use 3.5.0.2 on RHEL 5.8