IC4NOTICE: developerWorks Community will be offline May 29-30, 2015 while we upgrade to the latest version of IBM Connections. For more information, read our upgrade FAQ.
2 replies Latest Post - ‏2009-05-08T01:27:00Z by SystemAdmin
2092 Posts

Pinned topic No space left on device

‏2009-05-07T21:45:38Z |
Hi People,

I hope someone can help me with this.

When I try to use a newly created GPFS file system I get the following error "No space left on device"

dd if=/dev/zero of=/gpfs0r/bigfile1 bs=64k count=1

dd: writing `/gpfs0r/bigfile1': No space left on device
1+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000439 seconds, 0.0 kB/s

Details of setup:

Red Hat Enterprise Linux Server release 5.3 (Tikanga) x86_64
kernel 2.6.18-128.1.6.el5
GPFS 3.2.1-11
2 x Data centres, 2 GPFS nodes & IBM N5200 in each centre.

What I am trying to do, Put the metadata on the quicker FC disk & data on the SATA disks and replicate both.

Both nodes have access to all disks over FC.

dm-8 size=10G=SATA
dm-7 size=145G=FC
dm-6 size=145G=FC
dm-5 size=10G=SATA

I Created a file call HA.txt which contains the following:


root@gpfs-itd ~# mmcrnsd -F HATiebreaker.txt
No errors reported

mmcrfs gpfs0 -F HA.txt -B 64k -M 2 -m 2 -R 2 -r 2 -A yes -T /gpfs0r
No errors

mmlsfs gpfs0
flag value description
-f 2048 Minimum fragment size in bytes
-i 512 Inode size in bytes
-I 8192 Indirect block size in bytes
-m 2 Default number of metadata replicas
-M 2 Maximum number of metadata replicas
-r 2 Default number of data replicas
-R 2 Maximum number of data replicas
-j cluster Block allocation type
-D nfs4 File locking semantics in effect
-k all ACL semantics in effect
-a 1048576 Estimated average file size
-n 32 Estimated number of nodes that will mount file system
-B 65536 Block size
-Q none Quotas enforced
none Default quotas enabled
-F 317696 Maximum number of inodes
-V 10.01 ( File system version
-u yes Support for large LUNs?
-z no Is DMAPI enabled?
-L 2097152 Logfile size
-E yes Exact mtime mount option
-S no Suppress atime mount option
-K whenpossible Strict replica allocation option
-P system;hadata Disk storage pools in file system
-d nsd1;nsd2;nsd3;nsd4 Disks in file system
-A yes Automatic mount option
-o none Additional mount options
-T /gpfs0r Default mount point

mmmount all -a
No errors

mmdf gpfs0
disk disk size failure holds holds free KB free KB
name in KB group metadata data in full blocks in fragments

Disks in storage pool: system (Maximum disk size allowed is 1.3 TB)
nsd3 152051712 1 yes no 151662720 (100%) 108 ( 0%)
nsd2 152051712 2 yes no 151662720 (100%) 108 ( 0%)

(pool total) 304103424 303325440 (100%) 216 ( 0%)

Disks in storage pool: hadata (Maximum disk size allowed is 105 GB)
nsd1 10485760 1 no yes 10483648 (100%) 62 ( 0%)
nsd4 10485760 2 no yes 10483648 (100%) 62 ( 0%)

(pool total) 20971520 20967296 (100%) 124 ( 0%)

============= ==================== ===================
(data) 20971520 20967296 (100%) 124 ( 0%)
(metadata) 304103424 303325440 (100%) 216 ( 0%)
============= ==================== ===================
(total) 325074944 324292736 (100%) 340 ( 0%)

df -h
/dev/gpfs0 20G 4.2M 20G 1% /gpfs0r
Updated on 2009-05-08T01:27:00Z at 2009-05-08T01:27:00Z by SystemAdmin
  • dlmcnabb
    1012 Posts

    Re: No space left on device

    ‏2009-05-07T22:42:53Z  in response to SystemAdmin
    Did you create a policy to place files in the data pool? You defined the default system pool to only have metadata, so there is no space for data.

    Create a basic policy file (say placement.policy) with contents:
    'defplacement' SET POOL 

    Then install it in the filesystem:
    mmchpolicy gpfs0 placement.policy
    • SystemAdmin
      2092 Posts

      Re: No space left on device

      ‏2009-05-08T01:27:00Z  in response to dlmcnabb
      Thanks for the reply.

      You got it in one, I didn't have a policy in place.

      Thanks very much.