Working with pointers in ILE RPG

Practical usages on MQ with large-length data handling in ILE RPG

In today's business environment, demand for sending or receiving large data within IT systems is constantly growing and IBM i is no exception to that. To handle large data, IBM® Integrated Language Environment® (ILE) RPG can use a pointer function. This article describes the pointer concept, data segmentation method and how to use a pointer within ILE RPG. The MQGET application programming interface (API) sample code is used for our modification sample.

Share:

Yukihiro Minote (yminote@jp.ibm.com), AITS, IBM Japan

Photo of Yukihiro MinoteYukihiro Minote is a consulting IT specialist working for IBM Japan. He has 25 years of experience in the IBM i environment. His areas of expertise include network (APPN HPR and TCP/IP), security, and performance (in IBM DB2 for i and IBM WebSphere Application Server on i).



20 June 2013

RPG and large message data

The RPG language is at the heart of IBM i. There can hardly be any IBM i application developer who has never used it for building their applications. Accessing data through data description specifications (DDS) file, such as a physical file, is an efficient way to get data easily from a programming point of view. This is a concept that is well considered for developing business applications quickly.

For a long time, user data that RPG has handled were not long because of the DDS I/O size limitation (for example, the length of a physical file record must be 32766 bytes or smaller).

In V5R4 RPG, the maximum message length for a fixed-length character field is 65535, but beyond V6R1, the size changed from 65535 to 16773104. Because the I/O size is smaller than RPG limitation, we need not care about the character length issue in our programming cycle.

On the other hand, when we look into our business transactions today, there are many long message lengths data, such as XML. We see business data over 64 KB message length, being exchanged and processed. In the real world, RPG does not ignore or isolate this game change. Even though today's RPG has already extended their capacity for large data, there are much larger data existing on the customer's production environment. For example, the IBM WebSphere MQ product has been used for exchanging information between systems. RPG can also handle these data through MQ common APIs, such as MQGET. You can increase the size of the MaxMsgLength attribute up to 100 MB on all WebSphere MQ systems at V6 or later.

How can we handle these large message length data within RPG? The most common way is to use application segmentation. Figure 1 shows how to do that. In this article, we will discuss pointer in ILE RPG. Then, we will talk about pointer operation, overlaying data with a pointer-based field, and finally the data segmentation method with ILE RPG.

Figure 1. Data segmentation overview
Data segmentation overview

Using the application segmentation for large message length data

This section explains segmentation and the concept of pointer that is used for dividing data easily.

The definition for the word segmentation itself, is the act of dividing something into smaller parts, or the state of being divided. In this article, segmentation means the program logic of dividing large message length data into chunks of smaller message length data so that they can be handled by a program variable. We will divide large data into 64 KB data chunks and process with a normal RPG code logic. To do this, we use the pointer operation. What is a pointer? Which operation code should we specify in RPG and how do we write a code using it?

The pointer in ILE RPG

The concept of pointer is not a state-of-the-art technology. Many computer languages (such as C) have the same concept for handling data efficiently. Generally, a pointer is a useful piece of advice or information that helps you do or understand something. In computer terminology, a pointer is a field that contains a memory address.

Multiple occurrence data structure is a good sample of pointer. The occurrence of the database is based on an offset of the address in the data structure.

Let's take a look at the pointers in ILE RPG.

First, you define pointer data types '*' in D spec. You also need to define fields, arrays or data structures whose positioning in memory is based on the value of the pointer with the Based keyword. See Figure 2.

Figure 2. Pointer definition in D spec
fig2

With this definition, we can map the data to the location that the pointer variable addresses. This means that any message length of data can be divided into smaller chunks. Figure 3 illustrates the concept of pointer address operation.

Figure 3. Moving the pointer address
fig3

A pointer can show any address of the data. A field that is based on this pointer can refer or overlay the data. We can change the value of a pointer by adding or subtracting that value. With these operations, you will see overlaying data.

Now, we understand what a pointer is in ILE RPG programming. Next, we must understand how to get the pointer value within an ILE RPG code.

Built-in function %ADDR

In ILE RPG, the %ADDR built-in function is used for getting the address of a pointer. From an ILE programming model point of view, variable fields are referenced for processing data. So a variable's pointer value is needed when coding a pointer operation. Figure 4 shows you how to use this built-in-function.

Figure 4. Usage of the built-in function, %ADDR
fig4

In this code snippet, variable p is defined as a type pointer. The data structure, TestDS, is also declared. It contains two text fields. The first one is initialized as static text '1234' and second one as '5678'. The variable field, p, is initialized as the result of %Addr(Text1). This means that the field p retrieves the pointer value of the Text1 field, which is '1234'. This is because the variable field is based on the pointer variable p. Next, the Eval operation 'p= %Addr(Text2)' changes the values of pointer p. Now the value of pointer p is equal to the value of the Text2 pointer which is '5678'.

So far we have gone through ILE RPG pointer concept, definition procedure itself and its operand in ILE RPG codes. To put these pieces together more clearly, I will give you a practical sample program code using the MQ API, MQGET, that can handle pointers.

Sample code MQGET with pointer

The MQGET function call is used for retrieving data from the message queue. From an ILE RPG point of view, MQGET syntax is as follows.

MQGET(HCONN, HOBJ, MSGDSC, GMO, BUFLEN, BUFFER, DATLEN,

        CMPCOD, REASON)

And, its prototype definition for the call is:

        C*..1....:....2....:....3....:....4....:....5....:....6....:....7..
        C                     CALLP     MQGET(HCONN : HOBJ : MSGDSC : MO :
        C                                     BUFLEN : BUFFER : DATLEN :
        C                                     CMPCOD : REASON)
        D*..1....:....2....:....3....:....4....:....5....:....6....:....7..
        DMQGET            PR                  EXTPROC('MQGET')
        D* Connection handle
        D HCONN                         10I 0 VALUE
        D* Object handle
        D HOBJ                          10I 0 VALUE
        D* Message descriptor
        D MSGDSC                       364A
        D* Options that control the action of MQGET
        D GMO                          112A
        D* Length in bytes of the BUFFER area
        D BUFLEN                        10I 0 VALUE
        D* Area to contain the message data
        D BUFFER                          *   VALUE
        D* Length of the message
        D DATLEN                        10I 0
        D* Completion code
        D CMPCOD                        10I 0
        D* Reason code qualifying CMPCOD
        D REASON                        10I 0

You can see a parameter BUFFER that contains the whole message defined as type pointer. Also, the IBM i MQ product provides a sample code for the ILE language. AMQ3GET4 is a sample code for the MQGET function call with ILE RPG. This code is a good sample for understanding the ILE RPG pointer operation.

In the original code, the BUFFER variable is defined as a 60-byte message length field as follows.

        D BUFFER          S             60

In this article, we extend that message length from 60 bytes to 32752 bytes.

        D BUFFER          S          32752

And, we add the following definitions to operate pointer values and addresses.

        * Buffer pointer
        D BUFPTR          S               *   INZ (%ADDR(BUFFER))
        *
        *****ADD FOR TEST*****
        D BUFF            S          32752A   BASED(P_BUFF)
        D ARRAY          S          32752A   DIM(32)
        D X                S             10I 0
        D Y                S             10I 0
        *****ADD FOR TEST*****

In the above code, the pointer variable BUFPTR is declared and initialized with a message length of 32752 byte. And, the variable field, ARRAY is defined for dividing into small bytes within the ILE RPG logic.

Here is an initialization logic for allocated pointer area.

        ***ADD FOR TEST***
        /free
        buflen = 32752* 32;
        bufptr = %alloc(buflen);
        for x = 0 to 31;
        p_buff = bufptr + (x * %size(buff));
        buff = *blanks;
        endfor;
        /end-free
        ***ADD FOR TEST***

The variable field buflen is 32752 bytes multiplied by 32, which calculates to a message length of 1048064 bytes.

The built-in %alloc is used for allocating memory area. Thus, this code allocates 5244288 bytes long memory area and returns its pointer to the bufptr variable field. The %alloc function does not initialize its allocated area. So, the user should take care of its initialization (the following line shows how to do that). Each field, with a message length of 32752 bytes, is initialized as blank, and therefore, the pointer moves to the next 32752 bytes, initializing again.

By 32 times of loop processing, the whole allocated memory area is initialized as *BLANK.

The following code provides an example for retrieving data from the received message queue.

        ***ADD FOR TEST***
        /free
        for x = 0 to 31;
        p_buff = bufptr + (x * %size(buff));
        y = x + 1;
        ARRAY(y) = buff;
        endfor;
        /end-free
        
        ***ADD FOR TEST***

As stated above, the MQGET function returns the message's pointer value. In our sample code, it is represented by the variable field, BUFFER. So, with the above code, the message is retrieved in chunks with 32752 bytes each. And then each message stores into the ARRAY field. With this ILE RPG code, we can handle messages of over 1048064 bytes long. Here is the entire code for our customization for the MQ sample AMQ3GET4.

Download code


References


Conclusion

This article described ILE RPG capability for large size data by using pointers.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into IBM i on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=IBM i
ArticleID=934383
ArticleTitle=Working with pointers in ILE RPG
publish-date=06202013