Contents
Targeting your applications – what little endian and big endian IBM XL Fortran compiler differences mean to you
To migrate from a big endian compiler to a little endian compiler, you might have to change some code to maintain program behavior or results. Vectors, storage association between items of different sizes, 16-byte reals, complex numbers, and unformatted data files are aspects of your code that should be considered when porting code from big endian to little endian. The little endian Linux® distributions on IBM Power Systems™ use a different ABI than big endian distributions. Programs that have dependencies on the old ABI will need to be updated. New intrinsic procedures make porting easier with respect to byte ordering of vectors.
This article describes the possible issues encountered when porting Fortran code for use on big endian IBM XL Fortran on Power Systems to little endian IBM XL Fortran on Power Systems. It makes some suggestions about coding changes and describes compiler features and options that help with porting code.
Comparing big endian byte order and little endian byte order
Endianness determines how data is interpreted in memory. The endianness of your platform is determined by the architecture of your processor. The two most common types of endianness used today are big endian and little endian.
On big endian platforms (often referred to as simply big endian in this article), bytes in memory are ordered with the most significant byte first (or on the "left"). On little endian platforms (often referred to as simply little endian in this article), bytes in memory and vector registers are ordered with the least significant byte first (or on the "left").
For example, Figure 1 depicts how 000102030405060716 (interpreted as an 8-byte integer) is stored in memory on big endian and little endian platforms. In Figure 1, a represents the memory address at that location.
Figure 1: Representations of big endian and little endian byte ordering in memory
Vectors
The IBM POWER® processor architecture supports 16-byte vectors containing sixteen 1-byte elements, eight 2-byte elements, four 4-byte element, or two 8-byte elements. The processor has 128-bit vector registers and instructions to load vectors into registers, operate on vectors in registers, and store vector registers to memory. The IBM XL compilers provide intrinsic procedures and language support to work with vectors.
There are differences between big endian and little endian with respect to vector element order. New compiler options and intrinsic procedures have been introduced to help with these differences. These differences and new features are explained in the following sections.
Vector element order and vector element byte order
There are two ways to lay out the elements of a vector in a vector register. You can load the elements from lowest to highest, so that element 0 is the leftmost element in the vector register. Alternatively, you can load the elements from highest to lowest, so that element 0 is the rightmost element in the vector register. The former placement is called big endian vector element order, while the latter is called little endian vector element order.
On big endian, big endian vector element order is always used.
On little endian, you have the choice between using big endian vector element order and little endian vector element order.
Regardless of the vector element order, vector elements on big endian use big endian byte order in memory. Vector elements on little endian use little endian byte order in memory by default.
To demonstrate the differences of big endian and little endian vector element ordering, consider Figure 2 and Figure 3.
Figure 2 depicts how 000102030405060708090A0B0C0D0E0F16 (interpreted as a 16-byte vector) is represented in a vector register in big endian vector element order. The b127 and b0 markings on the figure denote bit 127 and bit 0 of the register respectively. The figure shows representations for vectors filled with elements of 1, 2, 4 and 8 bytes as you move down from the top.
Figure 2: Representations of big endian vector element order in vector registers
Figure 3 depicts how 000102030405060708090A0B0C0D0E0F16 (interpreted as a 16-byte vector) is represented in a vector register in little endian vector element order. The b127 and b0 markings on the figure denote bit 127 and bit 0 of the register respectively. The figure shows representations for vectors filled with elements of 1, 2, 4 and 8 bytes as you move down from top.
Figure 3: Representations of little endian vector element order in vector registers
-qaltivec option
The -qaltivec option can be used to tell the little endian compiler how to order vector elements in the vector registers.
With -qaltivec=le, the compiler loads vectors in little endian element order and assumes vectors are loaded with little endian element order for vector stores. The compiler inserts vector permute operations, if necessary, to ensure that the load and store intrinsic procedures use little endian element order. For vector intrinsic procedures that reference specific elements, the compiler assumes that vectors were loaded with little endian element order. On little endian, -qaltivec=le is the default.
With -qaltivec=be, the compiler loads vectors in big endian element order and assumes vectors are loaded with big endian element order for vector stores. The compiler inserts vector permute operations, if necessary, to ensure that the load and store intrinsic procedures use big endian element order. For vector intrinsic procedures that reference specific elements, the compiler assumes that vectors were loaded with big endian element order.
To demonstrate, consider the following program:
program main
vector(integer(8)) v
integer(8) i(2)
equivalence(v, i)
i = [z'0001020304050607', z'08090A0B0C0D0E0F']
print 1, "First vector element: ", vec_extract(v, 0)
print 1, "Second vector element: ", vec_extract(v, 1)
1 format(A, z16.16)
end programWhen this program is compiled on a big endian platform, or when it is compiled on a little endian platform with -qaltivec=le, it produces the following output:
First vector element: 0001020304050607 Second vector element: 08090A0B0C0D0E0F
However, if the program is compiled on a little endian platform with -qaltivec=be, it produces the following output:
First vector element: 08090A0B0C0D0E0F Second vector element: 0001020304050607
The vector was loaded backwards into the vector register, while the order of elements in array i was unchanged.
A more portable way of working with vectors is to load and store using the vec_xl, vec_xl_be, vec_xst, and vec_xst_be intrinsic procedures described in the next section instead of EQUIVALENCE. Consider the following program:
vector(real(4)) v1, v2, v3, v4 real(4) a(4) v1 = vec_xl(0, [real(4) :: 1.0, 2.0, 3.0, 4.0]) call vec_xst(v1, 0, a) print *, 'v1=', a v2 = vec_neg(v1) call vec_xst(v2, 0, a) print *, 'v2=', a ! Merge high and low depend on vector element order v3 = vec_mergeh(v1, v2) call vec_xst(v3, 0, a) print *, 'v3=', a v4 = vec_mergel(v1, v2) call vec_xst(v4, 0, a) print *, 'v4=', a end
The program produces the same output on big endian platforms and little endian platforms. The output is:
v1= 1.000000000 2.000000000 3.000000000 4.000000000 v2= -1.000000000 -2.000000000 -3.000000000 -4.000000000 v3= 1.000000000 -1.000000000 2.000000000 -2.000000000 v4= 3.000000000 -3.000000000 4.000000000 -4.000000000
The vector intrinsic procedures vec_xl, vec_xst, vec_mergeh, and vec_mergel take the vector element order into account. In other words, when the program is compiled on a little endian platform with -qaltivec=le:
vec_xluses a Vector Scalar eXtension (VSX) load instruction, which always loads in big endian element order. A vector permute instruction is then used to reverse the vector in the register to use little endian element order.vec_xstassumes that the vector in the register is using little endian vector element order so it uses a vector permute instruction to reverse the vector element order to big endian vector element order. It then uses a VSX store instruction, which always stores in big endian element order, to store the vector back to memory.vec_mergehknows that vector elements start from the right. The vector registers containing v1 and v2 look as follows:
v1 4.0 3.0 2.0 1.0 v2 -4.0 -3.0 -2.0 -1.0
Because vec_mergeh starts counting from the right, it correctly uses 1.0 and 2.0 for elements 0 and 2 of the result of vec_mergeh(v1, v2).
vec_mergelsimilarly knows that vector elements start from the right. It therefore correctly uses -1.0 and -2.0 for elements 1 and 3 of the result ofvec_mergel(v1, v2).
When the program is compiled on a big endian platform, or on a little endian platform with -qaltivec=be:
vec_xluses a VSX load instruction, which always loads in big endian element order. No vector permute is necessary.vec_xstassumes that the vector in the register is using big endian vector element order. So, it directly uses a VSX store instruction, which always stores in big endian element order, to store the vector back to memory.vec_mergehknows that vector elements start from the left. The vector registers containing v1 and v2 look as follows:
v1 1.0 2.0 3.0 4.0 v2 -1.0 -2.0 -3.0 -4.0
Because vec_mergeh starts counting from the left, it correctly uses 1.0 and 2.0 for elements 0 and 2 of the result of vec_mergeh(v1, v2).
vec_mergel similarly knows that vector elements start from the left. It therefore correctly uses -1.0 and -2.0 for elements 1 and 3 of the result of vec_mergel(v1, v2).
For programs that do not use EQUIVALENCE, the –qaltivec=be option can be useful in porting code from big endian to little endian.
The POWER8 cryptography intrinsic procedures require that input vectors to be in big endian vector element order. You can do this either by using -qaltivec=be, or by using the vec_xl_be and vec_xst_be intrinsic procedures to load and store. These vector load and store functions are outlined in the following section.
New vector load and store intrinsic procedures
New vector load and store intrinsic procedures that use VSX instructions have also been added. You can find these intrinsic procedures in the XL Fortran Compiler Reference (XL Fortran for Linux documentation library).
VEC_XL(ARG1,ARG2)This function loads a 16-byte vector from the memory address specified by the displacement ARG1 and the address of ARG2 with the appropriate element order with respect to the platform and the -qaltivec option.
ARG1 is an INTENT(IN) integer.
ARG2 is INTENT(IN) of any of the following types:
REAL(4)orREAL(8)INTEGER(1),INTEGER(2),INTEGER(4), orINTEGER(8)VECTOR
VEC_XL_BE(ARG1,ARG2)This function loads a 16-byte vector from the memory address specified by the displacement ARG1 and the address of ARG2 in big endian order regardless of the platform or the -qaltivec option.
ARG1 is an INTENT(IN) integer.
ARG2 is INTENT(IN) of any of the following types:
REAL(4)orREAL(8)INTEGER(1),INTEGER(2),INTEGER(4), orINTEGER(8)VECTOR
VEC_XST(ARG1,ARG2,ARG3)This function stores the elements of a 16-byte vector specified by ARG1 into a given memory address. The address is calculated by adding the displacement specified by ARG2 to the memory address specified by ARG3 with the appropriate element order with respect to the platform and the -qaltivec option.
ARG1 is an INTENT(IN) vector.
ARG2 is an INTENT(IN) integer.
ARG3 is INTENT(OUT) that must be a vector or type of integer or real as follows:
- If
ARG3is a vector, it must be of the same type asARG1. - If
ARG3is not a vector andARG1is an integer vector or an unsigned vector,ARG3must be of theINTEGERtype with the same kind type parameter as the elements ofARG1. - If
ARG3is not a vector andARG1is a real vector,ARG3must be of the same type and kind as the elements ofARG1. - If
ARG3is not a vector andARG1is a pixel vector,ARG3must be of typeINTEGER(2).
VEC_XST_BE(ARG1,ARG2,ARG3)This function stores the elements of a 16-byte vector specified by ARG1 into a given memory address. The address is calculated by adding the displacement specified by ARG2 to the memory address specified by ARG3 in big endian order regardless of the platform or the -qaltivec option.
ARG1 is an INTENT(IN) vector.
ARG2 is an INTENT(IN) integer.
ARG3 is INTENT(OUT) that must be a vector or type of integer or real as follows:
- If
ARG3is a vector, it must be of the same type asARG1. - If
ARG3is not a vector andARG1is an integer vector or an unsigned vector,ARG3must be of theINTEGERtype with the same kind type parameter as the elements ofARG1. - If
ARG3is not a vector andARG1is a real vector,ARG3must be of the same type and kind as the elements ofARG1. - If
ARG3is not a vector andARG1is a pixel vector,ARG3must be of typeINTEGER(2).
Application binary interface (ABI)
XL Fortran for little endian Linux on Power Systems uses the new IBM Power Architecture® 64-bit ELF V2 ABI specification. This new ABI improves several areas, including function calls. This means, however, that assembly files targeting the old ABI have to be ported to the new ABI. Fortran, C, and C++ programs that follow their respective language standards do not require porting to the new ABI. Programs containing non-standard extensions need to be reviewed for ABI sensitivity. One example in Fortran is programs calling C functions with variable argument lists.
Calling C functions with variable arguments lists from Fortran
The C language allows functions to have variable argument lists. This is indicated by the use of ellipsis (…). For example, printf has the following C prototype:
int printf(const char *restrict format, ...);
The ELF V2 ABI requires callers to functions with variable argument list to set up a parameter save area on the stack. As a result, calls to functions with variable argument lists must have accurate prototypes.
The Fortran standard, however, considers C functions with variable argument lists to be non-interoperable and does not provide a way of writing interfaces for them in Fortran. Some programs try to work around this by adding an explicit interface containing the maximum number of arguments they plan to use. For example:
interface
function printf1int(format, a) bind(c, name='printf')
use, intrinsic :: iso_c_binding, only: c_int
implicit none
integer(c_int) printf1int
character, intent(in) :: format(*)
integer(c_int), value :: a
end function
end interfaceCalling printf1int using the interface above might lead to stack corruption with the new ABI due to a missing parameter save area. To allow XL Fortran know about the variable argument list, add the procedure_attribute(varargs) directive to the interface.
interface
function printf1int(format, a) bind(c, name='printf')
use, intrinsic :: iso_c_binding, only: c_int
implicit none
integer(c_int) printf1int
character, intent(in) :: format(*)
integer(c_int), value :: a
!ibm* procedure_attribute(varargs)
end function
end interfaceStorage association between different sized items
When porting a program from big endian to little endian, storage association between items of different size must be considered. In Fortran, this concerns EQUIVALENCE, common blocks, ENTRY statements, argument association, and format-directed I/O. The following sections outline those items in greater detail.
Note that the hexadecimal values are always printed in big endian order.
EQUIVALENCE
The Fortran standard restricts the types of objects that can be storage-associated together with the EQUIVALENCE statement. XL Fortran (and most compilers), however, allow objects of almost any type to appear in EQUIVALENCE. When using EQUIVALENCE with items of different sizes, the result can be different on little endian than it is on big endian. To demonstrate this, consider the following program:
integer(8) j
integer(4) i(2)
integer(2) h(4)
integer(1) k(8)
equivalence (j, i), (j, h), (j, k)
j = z'0001020304050607'
print 1, 'j = ', j
print 2, 'i = ', i
print 4, 'h = ', h
print 8, 'k = ', k
1 format(A, "z'", z16.16, "'")
2 format(A, 2("z'", z8.8, "'", 1x))
4 format(A, 4("z'", z4.4, "'", 1x))
8 format(A, 8("z'", z2.2, "'", 1x))
endIn the above program, we specified that j shares storage space with i, h, and k – all of which have different kinds. In both big endian and little endian, arrays are laid out from left to right, or more accurately, the lowest array element has the lowest memory address. The byte order of each array element is different on big endian and little endian. On big endian, the most significant byte of each array element is on the left, so j, i, h, and k look the same in memory. The output is:
j = z'0001020304050607' i = z'00010203' z'04050607' h = z'0001' z'0203' z'0405' z'0607' k = z'00' z'01' z'02' z'03' z'04' z'05' z'06' z'07'
On little endian platforms, the least significant byte of each array element is on the left, and therefore, j, i, h, and k look different in memory. The output is:
j = z'0001020304050607' i = z'04050607' z'00010203' h = z'0607' z'0405' z'0203' z'0001' k = z'07' z'06' z'05' z'04' z'03' z'02' z'01' z'00'
Common blocks
Common blocks provide storage association between the items in different compilation units. If the same common block is declared differently in two compilation units, the common block members are storage associated. If corresponding members have different sizes, extra consideration is required when porting code from big endian to little endian. Consider the following program:
subroutine sub1
integer(8) j
common /blk/ j
j = z'0001020304050607'
print 1, 'sub1: ', j
1 format(A, "z'", z16.16, "'")
end subroutine
subroutine sub2
integer(4) i(2)
common /blk/ i
print 2, 'sub2: ', i
2 format(A, 2("z'", z8.8, "'", 1x))
end subroutine
program main
call sub1
call sub2
end programIn the above code, j and i share storage space and have different kinds. The output of the program shows that the elements of i are ordered differently on big endian and little endian. On big endian, running the program results in the following output:
sub1: z'0001020304050607' sub2: z'00010203' z'04050607'
On little endian, running the program results in the following output:
sub1: z'0001020304050607' sub2: z'04050607' z'00010203'
ENTRY statements
Storage association between the results of functions and their ENTRY statements occurs when the results have different characteristics. In the following program, f and g are storage associated.
function f(a)
double precision a, f
real g
entry g(a)
f = a
end function
program main
interface
function f(a)
double precision a, f
end function
function g(a)
double precision a
real g
end function
end interface
double precision x
x = z'0001020304050607'
print '(z16.16)', f(x)
print '(z8.8)', g(x)
end programOn big endian, g, which is a single-precision real, is storage associated with the most significant half of the double precision f. As a result, the output on big endian is:
0001020304050607 00010203
On little endian, g is storage associated with the least significant half of f. As a result, the output on little endian is:
0001020304050607 04050607
Argument association
Calls in the FORTRAN 77 style do not use explicit interfaces and rely on the user to pass the correct type. On big endian, some non-conforming programs could run successfully although the actual arguments and dummy arguments do not match. Consider the following program:
program main
integer :: i = 1
character(5) :: c = 'abcd'
integer(2) :: h = 2
real(8) :: d = 1.0
call hexprint(i, 4)
call hexprint(c, 5)
call hexprint(h, 2)
call hexprint(d, 8)
end program
subroutine hexprint(buffer, size)
character(8) buffer
integer size
print 100, buffer(1:size)
100 format (z<2 * size>.<2 * size>)
end subroutineThe program is non-conforming as it associates actual arguments of various integer kinds with a dummy argument of character type. The program does not use an explicit interface, and therefore, the compiler does not detect the error. On big endian, character and non-character data use the same byte order, and therefore, the program produces the expected output. On little endian, this is not the case. To demonstrate this, consider the output of the program on big endian and little endian.
On big endian, the program produces the following output:
00000001 6162636420 0002 3FF0000000000000
On little endian, the program produces the following output:
01000000 6162636420 0200 000000000000F03F
Format-directed I/O
In format-directed I/O, the edit descriptor used to read or write an I/O item must be appropriate for the type and type parameters of the I/O item. For example, porting issues might occur if the A edit descriptor, which assumes the I/O item to have type character, is used to read or write an I/O item of type integer. For example, consider the following program:
character(4) file integer(4) a a = z'61626364' write(file, '(A)') a print *, file end
On big endian, the output is:
abcd
But on little endian, the output is:
dcba
16-byte reals and complex types
XL Fortran's extended-precision floating-point type, REAL(16), is not in the binary128 format suggested by the Institute of Electrical and Electronics Engineers (IEEE) standard. Instead, REAL(16) is composed of two REAL(8) parts with different magnitudes that do not overlap (except when the number is zero or close to zero). The high-order REAL(8) value (the one that comes first in storage) must have the larger magnitude, even on little endian. The value of a REAL(16) number is the sum of its two REAL(8) parts.
Complex types are made of a real and an imaginary part where the real part always comes before the imaginary part.
Consider the following code snippet:
real(8) d(2) real(16) q complex(8) z equivalence (q, d), (q, z) q = 1.0q0
On both big endian and little endian, the first element of d is 1.0d0 and the second element of d is 0.0d0. The real part of the complex number z is 1.0d0 and the imaginary part is 0.0d0.
The byte order of the elements of a 16-byte real number and a complex number is different between big endian and little endian, but the element order is the same.
Unformatted data files
Unformatted Fortran data files are made up of records and record markers. Record markers are 4- or 8-byte integers containing the number of bytes in the record. Record markers appear before and after each record. Consider the following example.
write(10) 10, 11 end
This program writes the following data (in binary) to the file connected to unit 10:
0x00000008 0x0000000a 0x0000000b 0x00000008
Because record markers are integers, they are affected by the byte order. Also, because 10 and 11 above are integers, they are also affected by byte order. As a result, an unformatted data file using one byte order (for example, big endian) cannot be read on a platform using a different byte order (for example, little endian) without conversion. XL Fortran provides several methods of performing the conversion. They are, in order of highest precedence:
- The
XLFRTEOPTS=ufmt_bigendian=<units>andXLFRTEOPTS=ufmt_littleendian=<units>runtime options - The
CONVERT=specifier in theOPENstatement - The
@PROCESS UFMTdirective - The
-qufmtcompiler option
These methods are outlined in greater detail in the following sections.
Environment variables for runtime conversion
The XLFRTEOPTS=xlf_bigendian=<units> and XLFRTEOPTS=xlf_littleendian=<units> environment variables can be used to specify the endianness of unformatted data files at run time for conversion.
XLFRTEOPTS=xlf_bigendian=<units> is only valid on little endian and is used for units connected to big endian unformatted data files.
XLFRTEOPTS=xlf_littleendian=<units> is only valid on big endian and is used for units connected to little endian unformatted data files.
Examples of valid values for <units> for both xlf_bigendian and xlf_littleendian (the following example shows xlf_bigendian but these <unit> values are applicable to xlf_bigendian as well) are:
XLFRTEOPTS=xlf_bigendian=5
Specifies that unit 5 is connected to a big endian file.XLFRTEOPTS=xlf_bigendian=5-10
Specifies that units 5 through 10 are connected to big endian files.XLFRTEOPTS=xlf_bigendian=1,5-10
Specifies that units 1 and 5 through 10 are connected to big endian files.XLFRTEOPTS=xlf_bigendian=1-
Specifies that all units, from 1 onward, are connected to big endian files.XLFRTEOPTS=xlf_bigendian=newunit
Specifies that all automatically generated units are connected to big endian files. Automatically generated unformatted units are created with negative identifiers.XLFRTEOPTS=xlf_bigendian=1-,newunit or XLFRTEOPTS=xlf_bigendian=*Specifies that all units are connected to big endian files.
CONVERT= specifier in OPEN statements
The CONVERT= specifier can be used to specify endianness of unformatted data files for conversion. Valid values for this specifier are:
CONVERT='NATIVE'
Tells the compiler that the endianness of the file is that of the native platform, and no conversion is necessary. This is the default.
CONVERT='BIG_ENDIAN'
Tells the compiler that the unformatted file uses big endian byte ordering. On little endian, performs conversion on data transfer operations to and from the file. On big endian, no conversion is done.
CONVERT='LITTLE_ENDIAN'
Tells the compiler that the unformatted file uses little endian byte ordering. On big endian, performs conversion on data transfer operations to and from the file. On little endian, no conversion is done.
Source directives
@PROCESS UFMT source directives apply to all OPEN statements in the compilation unit before which they appear.
@PROCESS UFMT(big endian)
Assumes that OPEN statements are connecting to big endian data files. On little endian, conversions are performed at run time on data transfer to and from these files. On big endian, no conversion is necessary.
@PROCESS UFMT(little endian)Assumes that OPEN statements are connecting to little endian data files. On big endian, conversions are performed at run time on data transfer to and from these files. On little endian, no conversion is necessary.
-qufmt compiler option
The -qufmt compiler option allows the user to specify the endianness of unformatted data files at compile time for conversion.
-qufmt=be
Tells the compiler that all unformatted data files are big endian. On little endian, conversions are performed at run time on data transfer to and from these files. On big endian, no conversion is necessary.
-qufmt=le
Tells the compiler that all unformatted data files are little endian. On big endian, conversions are performed at run time on data transfer to and from these files. On little endian, no conversion is necessary.
Loading and storing in reverse byte order
XL Fortran provides the following intrinsic procedures to help with converting byte order:
LOAD2R(X)
Loads the value of X with reverse byte order where X is an INTENT(IN) dummy argument of type INTEGER(2). The result has type INTEGER(2).
LOAD4R(X)
Loads the value of X with reverse byte order where X is an INTENT(IN) dummy argument of type INTEGER(4). The result has type INTEGER(4).
LOAD8R(X)
Loads the value of X with reverse byte order where X is an INTENT(IN) dummy argument of type INTEGER(8). The result has type INTEGER(8).
REVERSE_BYTE_ORDER(X)
Is a generic function that loads the value of X with reverse byte order where X is an INTENT(IN) dummy argument of type INTEGER(2), INTEGER(4) or INTEGER(8). It resolves to LOAD2R, LOAD4R or LOAD8R depending on the type of X. The result has the same type and type parameters as that of X.
VEC_REVB(ARG1)
Returns a vector of the same type as ARG1 containing the bytes of the corresponding elements of ARG1 in reversed byte order. ARG1 is an INTENT(IN) integer, unsigned, or real vector.
VEC_REVE(ARG1)
Returns a vector of the same type as ARG1 containing the elements of ARG1 in reversed element order. ARG1 is an INTENT(IN) integer, unsigned, or real vector.It returns a vector which is the same type as that of X containing the INTENT(IN) integer, unsigned, or real vector.
Resources
- Visit the XL Fortran for Linux product page for more information.
- Visit the Power Architecture: 64-Bit ELF V2 ABI Specification - OpenPOWER ABI for Linux Supplement for more information.
- Get the free trial download for XL Fortran for Linux.
- Get connected. Join the Rational Fortran Cafe community.