Learn how to use processor affinity to restrict a process and run it only on a specified central processing unit (CPU). Processor affinity is a facility provided by operating systems, and you can use it on multi-processor hardware. All the threads within the process can be bound to run on the specified processor. Using process affinity settings to bind or unbind threads can help you find the root cause of troublesome hang or deadlock problems, especially for those hairy deadlocks and hangs that are hard to debug.

Umesh Prabhakar Gaikwad (ugaikwad@in.ibm.com), Software Engineer, IBM

Photo of Umesh GaikwadUmesh Prabhakar Gaikwad is a Staff Software Engineer for IBM India Software Labs. He has worked for IBM for the past three years, focusing on distributed computing Environment (DCE) on AIX, Windows, and Solaris. He is currently involved in Level 3 activities for IBM DCE products. Umesh holds a bachelor's degree in computer science and engineering from the University of Pune, India.



Kailas S. Zadbuke (kailashsz@in.ibm.com), Software Engineer, IBM

Photo of Kailas ZadbukeKailas S. Zadbuke is a Staff Software Engineer for IBM India Software Labs. He has worked for IBM for the past three years. He is currently working on the IBM Distributed File System product. He has worked on implementing the Security RFC for Simple Public Key Mechanism (SPKM) as well as on distributed technology including DCE and Standalone RPC (SARPC). Kailas holds a master's degree in electronics and communication from the University of Roorkee, India. You can contact him at kailashsz@in.ibm.com.



16 November 2006

Also available in Chinese

Introduction

Regardless of what system you use and what functional and regression testing you do when your application is deployed for production, you can still get errors that are unavoidable. You might get a hang where the CPU is being used 100 percent of the time, or you might get deadlocks where a couple of threads are locked and never release a resource. In some cases, you'll see hangs in the application that are very old (even a couple of decades back), even on multiprocessor architectures and fast hardware.

In such cases, error logs from the application, or stack traces from the application dumps, often provide little or no help in pinpointing the exact line of code where the problem originates. Even if the problem is recreated, you might not be able to collect very much information about the problem.

The best you can usually do is to reproduce the problem and figure out the code that is failing. In a worst-case scenario, even after recreating the exact production environment (which is very difficult due to the potentially large load on the system and hardware configurations), you might not find the root cause of the hang problems on multiprocessor environments.

One approach that can provide relief for you, or for customers who are facing this problem, is to use the processor affinity setting methods available on the operating systems.


Binding the process to available CPUs on AIX

You can use either the bindprocessor command that's available on the AIX® platform, or the bindprocessor API available on the AIX platform, in a program. The bindprocessor command binds or unbinds the kernel threads of a process to a processor.

You can also list the available processors with the bindprocessor command. The only input required is the process identifier of the process whose threads are to be bound or unbound. Once kernel threads of the process are bound, they'll always be scheduled to run on the selected processor.

The syntax for the bindprocessor command is:

Click to see code listing

bindprocessor Process [ ProcessorNum ] | -q | -u Process{ProcessID [ProcessorNum] | -u ProcessID | -s SmtSetID | -b bindID ProcessorNum | -q }

The following flags are used in Table 1 below:

Table 1. Flags
Sample codeDescription
-bThis flag binds all threads of an application to the hardware threads of the same physical processor.
-qThis flag displays the processors that are available.
-uThis flag unbinds the threads of the specified process.

Suppose process p1 is running on multi-CPU hardware. You need to bind the process to CPU1 and check the status using the ps command.

  1. To query for the available processors on the machine, try:
    # bindprocessor -q
    The available processors are:  0 1
  2. Bind the p1 processes, whose process id is 14662 to processor 1:
    # bindprocessor 14662 1
  3. Check to make sure the process is bound to the specified processor:
    #ps -emo THREAD | grep p1
      USER   PID  PPID   TID ST CP PRI SC    WCHAN        F       TT     BND COMMAND
      root  4460  5428    -  A  0  60  7  f0145c10   240001    -    - /usr/sbin/rpc.mountd
      root  4710  5428    -  A  0  60   4     *       240001    -   - /usr/sbin/tftpd -n
      root 14662 10566    -  A 120 126 0    -     200001   pts/0   1        ./p1
  4. Unbind the process:
    # bindprocessor -u 14662
  5. Check the result of the above command:
    # ps -emo THREAD | grep p1
      USER   PID  PPID   TID   ST CP PRI   SC   WCHAN      F     TT        BND   COMMAND
      root  12672  10566   -    A  1 60    1  50a05e84   200001  pts/0      -      grep p1
      root  14662 10566  -    A  78 111  0      -             200001  pts/0    -     ./p1

APIs available on AIX

If you want to provide a tunable parameter to the binary that binds to a specified processor, bindprocessor is also a subroutine provided by AIX. To read more details about the bindprocessor subroutine, see the IBM AIX Base Operating System and Extensions Technical Reference, available through AIX man page.

Listing 1 below shows the use of the bindprocessor subroutine.

Listing 1. Bindprocessor subroutine
#include <stdio.h> 
#include <unistd.h>
#include <sys/processor.h>

int main(int argc, char ** argv)
{
        long noProcessorConf,noProcessorOnline;
        int bindparameter;
        int retValue = 0;
        int pid;

        pid = getpid();

        /*_SC_NPROCESSORS_CONF Number of processors configured.
          _SC_NPROCESSORS_ONLN Number of processors online.*/

        noProcessorConf = sysconf(_SC_NPROCESSORS_CONF);
        noProcessorOnline = sysconf(_SC_NPROCESSORS_ONLN);

        if(-1 == noProcessorConf || -1 == noProcessorOnline )
        {
                printf("sysconf API failed\n");
        }

        if(-1 != noProcessorConf || -1 != noProcessorOnline )
        {
            printf("Number of Processor %d Processor Online %d\n", noProcessorConf, 
               noProcessorOnline);

            if(noProcessorConf > 1) /* This is multi processor configured environment */
             {
                    bindparameter = atoi(argv[1]);

                    if(bindparameter > noProcessorConf)
                    {
                        printf("Processor parameter specified is out of configured 
                           processors\n");
                    }
                    else
                    {
                            retValue = 0;
                            retValue = bindprocessor(BINDPROCESS, pid, bindparameter);
                            if(-1 == retValue)
                                    printf("Failed to set the process affinity\n");
                            if(0 == retValue)
                                printf("Process affinity set to specified 
                                   CPU %d\n",bindprocessor);
                    }
             }
       }
        while(1){};
}

To compile the previous program, enter:

# cc –o setpb setpb.c

The results of running the setpb program binary of the above compiled code are shown below.

To bind setpb to CPU 1, enter:

# setpb 1

The results are:

Number of Processor 2 Processor Online 2
Process affinity set to specified CPU 1

To see if setpb is bound to CPU 1, enter:

# ps -emo THREAD | grep setpb

The results are:

USER  PID    PPID    TID     ST CP PRI   SC    WCHAN        F      TT      BND   COMMAND
Root 15080   10566   -       A  85 102   0      -           200001 pts/0    0   setpb 1

Run the setpb program with the argument below, so it tries to bind to CPU number 0.

# setpb 0

The results are:

Number of Processor 2 Processor Online 2
Process affinity set to specified CPU 0

The command to check if setpb is bound to CPU number 0 is:

# ps -emo THREAD | grep setpb

The results are:

  USER  PID    PPID    TID     ST CP PRI   SC    WCHAN        F      TT      BND   COMMAND
Root 15082   10566   -       A  99 109   0      -           200001 pts/0    0   setpb 0

Run the setpb program with the argument below so that it tries to bind to CPU number 4. CPU 4 is not available on the machine, so it will display the following error message and run the default available processors.

# setpb 4
Number of Processor 2 Processor Online 2
Processor parameter specified is out of configured processors

To check to see if setpb is bound to CPU4, enter:

# ps -emo THREAD | grep setpb

The following is displayed:

USER  PID    PPID    TID     ST CP PRI   SC    WCHAN        F      TT      BND   COMMAND
Root 15084   10566   -       A  87 103   0      -           200001 pts/0    0   setpb 4

Resources

Learn

Get products and technologies

  • IBM trial software: Build your next development project with software for download directly from developerWorks.

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into AIX and Unix on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=AIX and UNIX
ArticleID=175081
ArticleTitle=Processor affinity on AIX
publish-date=11162006