Processor affinity on AIX

Comments

Regardless of what system you use and what functional and regression testing you do when your application is deployed for production, you can still get errors that are unavoidable. You might get a hang where the CPU is being used 100 percent of the time, or you might get deadlocks where a couple of threads are locked and never release a resource. In some cases, you'll see hangs in the application that are very old (even a couple of decades back), even on multiprocessor architectures and fast hardware.

In such cases, error logs from the application, or stack traces from the application dumps, often provide little or no help in pinpointing the exact line of code where the problem originates. Even if the problem is recreated, you might not be able to collect very much information about the problem.

The best you can usually do is to reproduce the problem and figure out the code that is failing. In a worst-case scenario, even after recreating the exact production environment (which is very difficult due to the potentially large load on the system and hardware configurations), you might not find the root cause of the hang problems on multiprocessor environments.

One approach that can provide relief for you, or for customers who are facing this problem, is to use the processor affinity setting methods available on the operating systems.

Binding the process to available CPUs on AIX

You can use either the bindprocessor command that's available on the AIX® platform, or the bindprocessor API available on the AIX platform, in a program. The bindprocessor command binds or unbinds the kernel threads of a process to a processor.

You can also list the available processors with the bindprocessor command. The only input required is the process identifier of the process whose threads are to be bound or unbound. Once kernel threads of the process are bound, they'll always be scheduled to run on the selected processor.

The syntax for the bindprocessor command is:

bindprocessor Process [ ProcessorNum ] | -q | -u Process{ProcessID [ProcessorNum] | -u ProcessID | -s SmtSetID | -b bindID ProcessorNum | -q }

The following flags are used in Table 1 below:

Table 1. Flags
Sample codeDescription
-bThis flag binds all threads of an application to the hardware threads of the same physical processor.
-qThis flag displays the processors that are available.
-uThis flag unbinds the threads of the specified process.

Suppose process p1 is running on multi-CPU hardware. You need to bind the process to CPU1 and check the status using the ps command.

  1. To query for the available processors on the machine, try:
    # bindprocessor -q
    The available processors are:  0 1
  2. Bind the p1 processes, whose process id is 14662 to processor 1:
    # bindprocessor 14662 1
  3. Check to make sure the process is bound to the specified processor:
    #ps -emo THREAD | grep p1
      USER   PID  PPID   TID ST CP PRI SC    WCHAN        F       TT     BND COMMAND
      root  4460  5428    -  A  0  60  7  f0145c10   240001    -    - /usr/sbin/rpc.mountd
      root  4710  5428    -  A  0  60   4     *       240001    -   - /usr/sbin/tftpd -n
      root 14662 10566    -  A 120 126 0    -     200001   pts/0   1        ./p1
  4. Unbind the process:
    # bindprocessor -u 14662
  5. Check the result of the above command:
    # ps -emo THREAD | grep p1
      USER   PID  PPID   TID   ST CP PRI   SC   WCHAN      F     TT        BND   COMMAND
      root  12672  10566   -    A  1 60    1  50a05e84   200001  pts/0      -      grep p1
      root  14662 10566  -    A  78 111  0      -             200001  pts/0    -     ./p1

APIs available on AIX

If you want to provide a tunable parameter to the binary that binds to a specified processor, bindprocessor is also a subroutine provided by AIX. To read more details about the bindprocessor subroutine, see the IBM AIX Base Operating System and Extensions Technical Reference, available through AIX man page.

Listing 1 below shows the use of the bindprocessor subroutine.

Listing 1. Bindprocessor subroutine
#include <stdio.h> 
#include <unistd.h>
#include <sys/processor.h>

int main(int argc, char ** argv)
{
        long noProcessorConf,noProcessorOnline;
        int bindparameter;
        int retValue = 0;
        int pid;

        pid = getpid();

        /*_SC_NPROCESSORS_CONF Number of processors configured.
          _SC_NPROCESSORS_ONLN Number of processors online.*/

        noProcessorConf = sysconf(_SC_NPROCESSORS_CONF);
        noProcessorOnline = sysconf(_SC_NPROCESSORS_ONLN);

        if(-1 == noProcessorConf || -1 == noProcessorOnline )
        {
                printf("sysconf API failed\n");
        }

        if(-1 != noProcessorConf || -1 != noProcessorOnline )
        {
            printf("Number of Processor %d Processor Online %d\n", noProcessorConf, 
               noProcessorOnline);

            if(noProcessorConf > 1) /* This is multi processor configured environment */
             {
                    bindparameter = atoi(argv[1]);

                    if(bindparameter > noProcessorConf)
                    {
                        printf("Processor parameter specified is out of configured 
                           processors\n");
                    }
                    else
                    {
                            retValue = 0;
                            retValue = bindprocessor(BINDPROCESS, pid, bindparameter);
                            if(-1 == retValue)
                                    printf("Failed to set the process affinity\n");
                            if(0 == retValue)
                                printf("Process affinity set to specified 
                                   CPU %d\n",bindprocessor);
                    }
             }
       }
        while(1){};
}

To compile the previous program, enter:

# cc –o setpb setpb.c

The results of running the setpb program binary of the above compiled code are shown below.

To bind setpb to CPU 1, enter:

# setpb 1

The results are:

Number of Processor 2 Processor Online 2
Process affinity set to specified CPU 1

To see if setpb is bound to CPU 1, enter:

# ps -emo THREAD | grep setpb

The results are:

USER  PID    PPID    TID     ST CP PRI   SC    WCHAN        F      TT      BND   COMMAND
Root 15080   10566   -       A  85 102   0      -           200001 pts/0    0   setpb 1

Run the setpb program with the argument below, so it tries to bind to CPU number 0.

# setpb 0

The results are:

Number of Processor 2 Processor Online 2
Process affinity set to specified CPU 0

The command to check if setpb is bound to CPU number 0 is:

# ps -emo THREAD | grep setpb

The results are:

  USER  PID    PPID    TID     ST CP PRI   SC    WCHAN        F      TT      BND   COMMAND
Root 15082   10566   -       A  99 109   0      -           200001 pts/0    0   setpb 0

Run the setpb program with the argument below so that it tries to bind to CPU number 4. CPU 4 is not available on the machine, so it will display the following error message and run the default available processors.

# setpb 4
Number of Processor 2 Processor Online 2
Processor parameter specified is out of configured processors

To check to see if setpb is bound to CPU4, enter:

# ps -emo THREAD | grep setpb

The following is displayed:

USER  PID    PPID    TID     ST CP PRI   SC    WCHAN        F      TT      BND   COMMAND
Root 15084   10566   -       A  87 103   0      -           200001 pts/0    0   setpb 4

Downloadable resources


Related topics

  • IBM trial software: Build your next development project with software for download directly from developerWorks.
  • AIX 5L Wiki: Visit this collaborative environment for technical information related to AIX.

Comments

Sign in or register to add and subscribe to comments.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=175081
ArticleTitle=Processor affinity on AIX
publish-date=11162006