How To
Summary
Basic definitions and a sample program are shown to create an executable that is capable of pre-allocating blocks for a file in a JFS2 filesystem under AIX.
A program named j2wfile is used to write to sections of the file
The fileplace command is used to list on what blocks the data resides demonstrating the advantage of using pre-allocation
Objective
As files are extended new blocks will be allocated over time from different parts of the backing device Because of this file data tends to be discontinuous. One way to avoid this effect is to have blocks pre-allocated to use for a file even though they have not been written to yet. This is especially helpful if the user has some idea of what the final size can be. Then blocks can be allocated for use at the needed size and some can later be de-allocated if it turns out they are not used. This way a file with less fragmented blocks can be produced over time. JFS2 provides a function posix_fallocate() which can be used to do pre-allocation for a file. It also provide the function ftruncate64() to reduce pre-allocation if that is desired. The objective of this article is to show a sample program that makes use of these functions and demonstrates how to use the program's executable to perform the desired operations on the file.
Environment
Here is sample program sample_falloc.c
-----------------------------------------------------------------------------
#include <stdlib.h>
#include <sys/errno.h>
#include <sys/types.h>
#include <string.h>
#include <fcntl.h>
#include <j2/j2_cntl.h>
#include <unistd.h>
#include <sys/errno.h>
#include <sys/types.h>
#include <string.h>
#include <fcntl.h>
#include <j2/j2_cntl.h>
#include <unistd.h>
int main(int argc,char **argv)
{
offset_t offset, poffset;
offset_t len, plen;
char *filename;
char errmsg[120];
int rc=0;
int fd;
int i,c;
int uflag=0; /* UNDO Allocation option */
int aflag=1; /* Normal pre-allocation */
int index;
{
offset_t offset, poffset;
offset_t len, plen;
char *filename;
char errmsg[120];
int rc=0;
int fd;
int i,c;
int uflag=0; /* UNDO Allocation option */
int aflag=1; /* Normal pre-allocation */
int index;
while ((c = getopt (argc, argv, "dru")) != -1)
switch (c)
{
case 'u':
uflag = 1;
aflag=0;
break;
case '?':
switch (c)
{
case 'u':
uflag = 1;
aflag=0;
break;
case '?':
if (isprint (optopt))
{
printf( "usage: %s [ -u ] <file_name> <page_offset> <page_range> \n", argv[0] );
printf( " where no option means we pre-allocate range\n");
printf( " -u for unallocate range\n");
}
else
printf ("Unknown option character `\\x%x'.\n", optopt);
exit ( 1 );
default:
printf ("Unknown option character `\\x%x'.\n", optopt);
exit ( 1 );
}
{
printf( "usage: %s [ -u ] <file_name> <page_offset> <page_range> \n", argv[0] );
printf( " where no option means we pre-allocate range\n");
printf( " -u for unallocate range\n");
}
else
printf ("Unknown option character `\\x%x'.\n", optopt);
exit ( 1 );
default:
printf ("Unknown option character `\\x%x'.\n", optopt);
exit ( 1 );
}
if (( uflag + aflag) != 1)
{
printf("Only one option is allowed exiting\n");
exit(1);
}
if ( ( argc - optind ) != 3 )
{
printf( "usage: %s [ -u ] <file_name> <page_offset> <page_range> \n", argv[0] );
printf( " where no option means we pre-allocate range\n");
printf( " -u for unallocate range\n");
exit ( 1 );
}
i=1;
for (index = optind; index < argc; index++)
{
printf("Only one option is allowed exiting\n");
exit(1);
}
if ( ( argc - optind ) != 3 )
{
printf( "usage: %s [ -u ] <file_name> <page_offset> <page_range> \n", argv[0] );
printf( " where no option means we pre-allocate range\n");
printf( " -u for unallocate range\n");
exit ( 1 );
}
i=1;
for (index = optind; index < argc; index++)
{
if (i == 1)
filename = &argv[index][0];
else if (i == 2)
sscanf(argv[index],"%lld", &poffset );
else if (i == 3)
sscanf(argv[index],"%lld", &plen );
i++;
}
if (i == 1)
filename = &argv[index][0];
else if (i == 2)
sscanf(argv[index],"%lld", &poffset );
else if (i == 3)
sscanf(argv[index],"%lld", &plen );
i++;
}
offset = poffset*PAGESIZE;
len = plen*PAGESIZE;
len = plen*PAGESIZE;
if ( ( fd = open ( filename,O_RDWR) ) < 0 )
{
sprintf(errmsg,"%s: cannot open %s ", argv[0], filename);
perror ( errmsg );
exit ( 1 );
}
{
sprintf(errmsg,"%s: cannot open %s ", argv[0], filename);
perror ( errmsg );
exit ( 1 );
}
if ( aflag == 1 )
rc = posix_fallocate(fd,offset,len);
else if ( uflag == 1 )
rc = ftruncate64(fd,offset);
if (rc == -1)
{
rc = errno;
printf(" error: rc: %d \n",rc);
return rc;
}
return rc;
}
-------------------------------------------------------------------------------
Steps
==============================
Example of using pre-allocation program
==============================
Example of using pre-allocation program
==============================
First the result of file when pre-allocation is not used:
Over some amount of time these 3 calls are made without preallocation to create a file of 10 blocks:
We use ./j2wfile executable to write data 3 times to different offsets of the file
# ./j2wfile -?
usage: ./j2wfile <data file> <blocks> <block size> <initial seek in blocks units>
# touch test1
# ./j2wfile test1 2 4096 0
wrote to file: test1 2 blocks of size 0x1000 from initial seek 0x0
....
# ./j2wfile test1 3 4096 2
wrote to file: test1 3 blocks of size 0x1000 from initial seek 0x2
....
# ./j2wfile test1 5 4096 5
wrote to file: test1 5 blocks of size 0x1000 from initial seek 0x5
# ./j2wfile -?
usage: ./j2wfile <data file> <blocks> <block size> <initial seek in blocks units>
# touch test1
# ./j2wfile test1 2 4096 0
wrote to file: test1 2 blocks of size 0x1000 from initial seek 0x0
....
# ./j2wfile test1 3 4096 2
wrote to file: test1 3 blocks of size 0x1000 from initial seek 0x2
....
# ./j2wfile test1 5 4096 5
wrote to file: test1 5 blocks of size 0x1000 from initial seek 0x5
We use fileplace command to see the location of the data on the backing device.
# fileplace test1
# fileplace test1
File: test1 Size: 40960 bytes Vol: /dev/hd3
Blk Size: 4096 Frag Size: 4096 Nfrags: 10
Blk Size: 4096 Frag Size: 4096 Nfrags: 10
Logical Extent
--------------
00047478-00047479 2 frags 8192 Bytes, 20.0%
00048449-00048451 3 frags 12288 Bytes, 30.0%
00048467-00048471 5 frags 20480 Bytes, 50.0%
--------------
00047478-00047479 2 frags 8192 Bytes, 20.0%
00048449-00048451 3 frags 12288 Bytes, 30.0%
00048467-00048471 5 frags 20480 Bytes, 50.0%
----------------
We now use pre-allocation and then run the same commands:
We preallocate to test2 12 blocks
# touch test2
#
# ./sample_falloc -?
./sample_falloc: illegal option -- ?
usage: ./sample_falloc [-u ] <file_name> <page_offset> <page_range>
where no option means we pre-allocate range
-u for unallocate range
#
# ./sample_falloc -?
./sample_falloc: illegal option -- ?
usage: ./sample_falloc [-u ] <file_name> <page_offset> <page_range>
where no option means we pre-allocate range
-u for unallocate range
#
# ./sample_falloc test2 0 12
#
# ./sample_falloc test2 0 12
#
We use fileplace command with -a option that shows a "*" by sections of file that are pre-allocated:
# fileplace -a test2
File: test2 Size: 49152 bytes Vol: /dev/hd3
Blk Size: 4096 Frag Size: 4096 Nfrags: 12
Blk Size: 4096 Frag Size: 4096 Nfrags: 12
Logical Extent
--------------
* 00048480-00048491 12 frags 49152 Bytes, 100.0%
--------------
* 00048480-00048491 12 frags 49152 Bytes, 100.0%
We now start writing to the file as we did before:
# ./j2wfile test2 2 4096 0
wrote to file: test2 2 blocks of size 0x1000 from initial seek 0x0
wrote to file: test2 2 blocks of size 0x1000 from initial seek 0x0
....
# ./j2wfile test2 3 4096 2
wrote to file: test2 3 blocks of size 0x1000 from initial seek 0x2
# ./j2wfile test2 3 4096 2
wrote to file: test2 3 blocks of size 0x1000 from initial seek 0x2
....
# ./j2wfile test2 5 4096 5
wrote to file: test2 5 blocks of size 0x1000 from initial seek 0x5
# ./j2wfile test2 5 4096 5
wrote to file: test2 5 blocks of size 0x1000 from initial seek 0x5
We now look at fileplace -a output for test2:
#fileplace -a test2
File: test2 Size: 49152 bytes Vol: /dev/hd3
Blk Size: 4096 Frag Size: 4096 Nfrags: 12
Blk Size: 4096 Frag Size: 4096 Nfrags: 12
Logical Extent
--------------
00048480-00048489 10 frags 40960 Bytes, 83.3%
* 00048490-00048491 2 frags 8192 Bytes, 16.7%
--------------
00048480-00048489 10 frags 40960 Bytes, 83.3%
* 00048490-00048491 2 frags 8192 Bytes, 16.7%
If we don't plan to use the final 2 blocks we can unallocate them from the file:
# ./sample_falloc -u test2 10 2
#
# fileplace -a test2
#
# fileplace -a test2
File: test2 Size: 49152 bytes Vol: /dev/hd3
Blk Size: 4096 Frag Size: 4096 Nfrags: 10
Blk Size: 4096 Frag Size: 4096 Nfrags: 10
Logical Extent
--------------
00048480-00048489 10 frags 40960 Bytes, 100.0%
--------------
00048480-00048489 10 frags 40960 Bytes, 100.0%
As you can see we have been able to write 3 times to the file over a period of time and the blocks have stayed contiguous because they were pre-allocated.
The use of the above program sample_falloc() will help you to keep your filesystem from too much fragmentation as long as you can estimate correctly the correct ending size of your files. You can over estimate a little and unallocate any leftovers.
Document Location
Worldwide
[{"Line of Business":{"code":"LOB08","label":"Cognitive Systems"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SWG10","label":"AIX"},"ARM Category":[{"code":"a8m0z0000001fMuAAI","label":"AIX General Support"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"All Version(s)"}]
Was this topic helpful?
Document Information
Modified date:
04 February 2021
UID
ibm16385742