IBM Support

Cleaning Up UNIX Interprocess Communication Structures for Red Brick without Rebooting

Troubleshooting


Problem

Removing message queues, shared memory segments and semaphores following a Red Brick warehouse daemon failure allows Red Brick to be restarted without rebooting the UNIX server.

Symptom


The Red Brick Warehouse is in a state that will not allow normal processing to continue and rbw.stop will not shut down the warehouse. Stopping and restarting the UNIX operating system is not an option for some reason unrelated to Red Brick.

Cause

A critical Red Brick process failed or was killed and normal processing cannot continue.

Resolving The Problem


Perform the following steps to shut down the Red Brick processes one at a time in the proper order to preserve as much data as possible. If a process or file mentioned does not exist, move on to the next step.

    1. Copy the Red Brick system catalogs of all databases to a safe location. The path to the database catalog files are listed in the $RB_CONFIG/rbw.config file under the title "Logical database name mappings"

    The catalog files are:
      • RB_DEFAULT_IDX      
      • RB_DEFAULT_LOADINFO  
      • RB_DEFAULT_SEGMENTS
      • RB_DEFAULT_INDEXES  
      • RB_DEFAULT_LOCKS    
      • RB_DEFAULT_TABLES

    2. Log in to the ADMIN database using RISQL and issue the following command:

      ALTER SYSTEM CLOSE USER SESSION ALL DATABASE ALL;

    3. Identify any remaining rbwsvr processes by executing he following command at the UNIX command line:

      ps -ef | grep rbwsvr
      Attention: Killing Red Brick processes can corrupt the catalogs and your data. To minimize damage, use the following kill commands in the order given until the process is killed:

        1. kill -2 pid       
        2. kill -5 pid 
        3. kill -9 pid
          pid
            The process ID of the process you want to kill.

    Kill any rbwsvr processes that you find.

    4. Execute rbw.stop. If it succeeds, then skip the remaining steps.

    5. Use the sequence of kill commands given in step 3 to kill the following processes, if running, in the order listed.
      1. rbwpmond
      2. rbwvcd
      3. rbwadmd
      4. rbw.servermon
      5. rbwlogd
      6. rbwapid
      7. rbwpchk
      6. Remove the semaphore set listed in the semaphore file ($RB_CONFIG/.$RB_HOST.semaphores). To do this, use the following UNIX command for each record in the semaphore file:
        ipcrm -s identifier
          identifier
            The second field of the record.

        Note: On LINUX use sem in place of the -s option.

      7. Remove shared memory segments listed in the shared memory file ($RB_CONFIG/.$RB_HOST.sharedmemory). To do this, use the following UNIX command for each record in the shared memory file where the first field contains an M:
        ipcrm -m identifier
          identifier
            The second field of the record.
        Note: On LINUX use shm in place of the -m option.


      8. Remove message queues listed in the shared memory file. To do this, use the following UNIX command for each record in the shared memory file where the first field contains a Q:
        ipcrm -q identifier

          identifier
            The second field of the record.

        Note: On LINUX use msg in place of the -q option.

      9. Delete all hidden files from the $RB_CONFIG directory. These files have names beginning with a period (.). Use the following UNIX command to show all files, including the hidden files:
        ls -a $RB_CONFIG
    Red Brick can now be started normally.

        Tip: Create a shell script to loop through the semaphores and sharedmemory files and execute ipcrm for each identifier number since the list can be lengthy. The following script may be used for this purpose:

        To use the script:

          1. Copy the file remove_RB_ipc.ksh.txt to a working directory and change the name to remove_RB_ipc.ksh.
            remove_RB_ipc.ksh.txt
          2. Run the script with the following command.
            ksh remove_RB_ipc.ksh $RB_HOST $RB_CONFIG

    [{"Product":{"code":"SSCRW7","label":"Red Brick Warehouse"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"--","Platform":[{"code":"PF002","label":"AIX"},{"code":"PF010","label":"HP-UX"},{"code":"PF016","label":"Linux"},{"code":"PF027","label":"Solaris"}],"Version":"6.3","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

    Document Information

    Modified date:
    16 June 2018

    UID

    swg21117103