Handle synchronous events from shared objects in Linux

How to use design patterns for more effective use of shared memory

Making effective use of shared memory in high-level languages such as C++ is not straightforward, but it is possible to overcome the inherent difficulties. This article describes, and includes sample code for, two C++ design patterns that use shared memory on Linux® in interesting ways and open the door for more efficient interprocess communication.

Share:

Sachin Agrawal, Staff Software Engineer, IBM Software Labs, India

Sachin AgrawalSachin has been working extensively in C++ for more than six years, including three years of research into the C++ object models of various compilers. He currently works for IBM Global Services, India. You can contact him at sachin_agrawal@in.ibm.com.



Swati P. Udas (swatudas@in.ibm.com), Software Engineer, IBM

Swati UdasSwati Udas has been working with IBM Software Laboratories, India, since February 2005. She currently works on Lotus Workplace. Contact her at swatudas@in.ibm.com.



10 November 2005

Also available in Japanese

In an object-oriented system, when an object receives a message, it can fire off a set of events. Mostly, these events are handled in a synchronous manner: the calling process or thread sending the object a message receives and handles events before the send-message call is completed. However, the situation changes slightly if the object shooting off the events is shared by more than one process and resides in shared memory.

This article describes this scenario in detail using two C++ design patterns and demonstrates the solution with sample code (get the sample code in the Download section of this article):

  1. We describe the sample code in which no shared memory is used.
  2. We make changes needed to use shared memory, using the first design pattern.
  3. We illustrate how to achieve interprocess communication (IPC) can also be achieved, using the second design pattern.

You can apply the overall concept in these design patterns to any machine architecture, operating system, and compiler combination. We used the RedHat Linux 7.1 distribution for 32-bit x86 Intel® architectures; we used GNU C++ compiler version 3.2.3 and associated tools to build and test the sample program.

No shared memory

Let's start with the sample program, the one that doesn't use shared memory:

Listing 1. common.h
#ifndef __COMMON_H__
#define __COMMON_H__

class IObjectWithEvents
{
public:
   class IEventSink
   {
   public:
      virtual void OnEvent(pid_t pid, const char * msg) = 0;
   };

   static IObjectWithEvents * getInstance();

   virtual bool AddEventHandler(IEventSink * pEI) = 0;
   virtual void SendMessage() = 0;
};

#endif //__COMMON_H__

The interface class IObjectWithEvents contains an interface class embedded in it, IEventSink, which defines the OnEvent() method. This event handler receives the sender's pid and a string message. The getInstance() method returns the object reference in shared memory, AddEventHandler() registers an event handler, and SendMessage() sends a message to the object. Without any reference to shared memory, IObjectWithEvents can be used as in Listing 2:

Listing 2. shm-client1.cpp
#include <iostream>
#include <sys/types.h>
#include <unistd.h>

#include "common.h"

#define HERE __FILE__ << ":" << __LINE__ << " "

using namespace std;

class EventSink : public IObjectWithEvents::IEventSink
{
public:
   void OnEvent(pid_t pid, const char * msg)
   {
      cout << HERE << "Message from pid(" << pid << ")\t : " << msg << endl;
   }
};

int main()
{
   IObjectWithEvents * powe = IObjectWithEvents::getInstance();

   EventSink sink;
   powe->AddEventHandler(&sink);

   powe->SendMessage();
   return 0;
}

Class EventSink provides the implementation of the event handler. The main function shows the standard sequence for sending the message and handling the events.

The typical implementation of ObjectWithEvents is shown in Listings 3 and 4:

Listing 3. ObjectWithEvents.h
#include "common.h"

class ObjectWithEvents : public IObjectWithEvents
{
public:
   // We assume singleton design pattern for illustration
   static ObjectWithEvents * ms_pObjectWithEvents;

   ObjectWithEvents();

   //the implementation for IObjectWithEvents
   void FireEvent();
   virtual bool AddEventHandler(IEventSink * pEI);
   virtual void SendMessage();

   //Collection for maintaining events
   enum { MAX_EVENT_HANDLERS = 16, };
   long m_npEI;
   IEventSink * m_apEI[MAX_EVENT_HANDLERS];
   pid_t m_alPID[MAX_EVENT_HANDLERS];
};
Listing 4. ObjectWithEvents.cpp
#include <iostream>
#include <sys/types.h>
#include <sys/shm.h>
#include <unistd.h>
#include <pthread.h>

#include "ObjectWithEvents.h"

using namespace std;

ObjectWithEvents * ObjectWithEvents::ms_pObjectWithEvents = NULL;

IObjectWithEvents * IObjectWithEvents::getInstance()
{
   // the following commented code is for illustration only.
   /*
   if (NULL == ObjectWithEvents::ms_pObjectWithEvents)
   {
      ObjectWithEvents::ms_pObjectWithEvents = new ObjectWithEvents();
   }
   */

   return ObjectWithEvents::ms_pObjectWithEvents;
}

ObjectWithEvents::ObjectWithEvents() : m_npEI(0)
{
}

void ObjectWithEvents::FireEvent()
{
   // iterate through the collection
   for (long i = 0; i < m_npEI; i++)
   {
      //Recheck for NULL
      if (0 != m_apEI[i])
      {
         // Fire the event
         m_apEI[i]->OnEvent(m_alPID[i], "");
      }
   }

   return;
}

bool ObjectWithEvents::AddEventHandler(IEventSink * pEI)
{
   // NULL check
   if (NULL == pEI)
   {
      return false;
   }

   // check if there is space for this event handler
   if (MAX_EVENT_HANDLERS == m_npEI)
   {
      return false;
   }

   // Add this event handler to the collection
   m_alPID[m_npEI] = getpid();
   m_apEI[m_npEI++] = pEI;

   return true;
}

void ObjectWithEvents::SendMessage()
{
   //Some processing
   //And then fire the event

   FireEvent();

   return;
}

You can compile the sample code in Listing 4 using the following script:

g++ -g -o shm_client shm_client1.cpp ObjectWithEvents.cpp

When you run the shm_client, you should get the following output:

$ ./shm_client shm_client1.cpp:16 Message from pid(3920) :


Using shared memory: No event caching

Now, to instantiate ObjectWithEvents in the shared memory, make the following changes to the ObjectWithEvents implementation.

Listing 5. Changes to ObjectWithEvents.cpp
// To add a declaration for the "new" operator:

class ObjectWithEvents : public IObjectWithEvents
{
public:
   void * operator new(unsigned int);
};


// To include an additional header for the Initializer class:

#include "Initializer.h"


// To overload the operator "new":

void * ObjectWithEvents::operator new(unsigned int)
{
   return ms_pObjectWithEvents;
}


// Then, FireEvent is completely changed:

void ObjectWithEvents::FireEvent()
{
   // We need to serialize all access to the collection by more than one process
   int iRetVal = Initializer::LockMutex();

   if (0 != iRetVal)
   {
      return;
   }

   pid_t pid = getpid();

   // iterate through the collection and fire only events belonging to the current process
   for (long i = 0; i < m_npEI; i++)
   {
      // Check whether the handler belongs to the current process.
      if (pid != m_alPID[i])
      {
         continue;
      }

      //Recheck for NULL
      if (0 != m_apEI[i])
      {
         m_apEI[i]->OnEvent(pid, "");
      }
   }

   // release the mutex
   if ((0 == iRetVal) && (0 != Initializer::UnlockMutex()))
   {
      // Deal with error.
   }

   return;
}


// The following are changes to ObjectWithEvents::AddEventHandler():

// 1. Before accessing the collection, we lock the mutex:

int bRetVal = Initializer::LockMutex();

if (0 != bRetVal)
{
   return false;
}

// 2. After accessing the collection, we release the mutex:

if ((0 == bRetVal) && (0 != Initializer::UnlockMutex()))
{
   // Deal with error.
}

To instantiate the object within the shared memory, define an additional class, Initializer.

Listing 6. Initializer.h
#ifndef __Initializer_H__
#define __Initializer_H__

class Initializer
{
public :
    int m_shmid;
    static Initializer ms_Initializer;
    Initializer();

    static pthread_mutex_t ms_mutex;
    static int LockMutex();
    static int UnlockMutex();
};

#endif // __Initializer_H__

Initializer defines shared memory id m_shmid and a semaphore ms_mutex for synchronous event handling.

The function LockMutex() locks the mutex and UnlockMutex() unlocks mutex.

The implementation of Initializer is defined in Listing 7:

Listing 7. Initializer.cpp
#include <iostream>
#include <sys/types.h>
#include <sys/shm.h>
#include <unistd.h>
#include <pthread.h>

#include "Initializer.h"
#include "ObjectWithEvents.h"

using namespace std;

Initializer Initializer::ms_Initializer;

pthread_mutex_t Initializer::ms_mutex = PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP;

Initializer::Initializer() : m_shmid(-1)
{
   bool bCreated = false;
   key_t key = 0x1234;

   m_shmid = shmget(key,sizeof(ObjectWithEvents), 0666);

   if (-1 == m_shmid)
   {
      if(ENOENT != errno)
      {
         cerr<<"Critical Error"<<endl;
         return;
      }

      m_shmid = shmget(key, sizeof(ObjectWithEvents), IPC_CREAT|0666);

      if (-1 == m_shmid )
      {
         cout << " Critical Error " << errno<< endl;
         return;
      }

      bCreated = true;
   }


   ObjectWithEvents::ms_pObjectWithEvents = (ObjectWithEvents*)shmat(m_shmid,NULL,0);

   if (NULL == ObjectWithEvents::ms_pObjectWithEvents)
   {
      cout << " Critical Error " << errno << endl;
      return;
   }

   if (true == bCreated)
   {
      ObjectWithEvents * p = new ObjectWithEvents();
   }

   // Create a mutex with no initial owner.


   pthread_mutex_init(&ms_mutex, NULL);


   }

   int Initializer::LockMutex()
   {
   // Request ownership of mutex.

   pthread_mutex_lock(&ms_mutex);

   if(EDEADLK == errno)
   {
      cout << "DeadLock" << endl;
      return -1;
   }

   return 0;
   }

   int Initializer::UnlockMutex()
   {
   return pthread_mutex_unlock(&ms_mutex);
   }

Shared memory is created if it does not exist, and the shared object is created within it. The object construction is skipped if the shared memory already exists. Initializer::m_shmid records the identifier and ObjectWithEvents::ms_pObjectWithEvents records the shared object reference.

The shared memory is not destroyed even after all processes detach from it. This lets you explicitly destroy it using the ipcrm command or make some quick observations with the ipcs command. The test application is built as follows:

g++ -g -o shm_client shm_client1.cpp ObjectWithEvents.cpp Initializer.cpp

The console dump for the trial run looks like this:

Listing 8. The console dump
$ ./shm_client
shm_client1.cpp:16 Message from pid(4332)        :

$ ipcs
------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status
0x00001234 327686     sachin    666        136        0

$ ./shm_client
shm_client1.cpp:16 Message from pid(4333)        :

$ ipcrm -m 327686

The ObjectWithEvents instance has a collection of events from various processes. It can fire off only the events that were registered by the current process. This design pattern illustrates these two points:

  • Any access to a collection of events is guarded by a mutual exclusion object.
  • Events before firing are filtered by process ID.

Shared memory and event caching for IPC

Now let's look at using shared memory and caching of events for interprocess communications. If the events are cached within the shared object, they can be fired later. The receiving process will have to query the shared object for events. Thus, by sticking to a synchronous model, interprocess communication can be achieved. This is the motivation behind developing the following design pattern.

Add a couple of methods to IObjectWithEvents, like so:

Listing 9. Adding methods to IObjectWithEvents
class IObjectWithEvents
{
public:
   virtual bool EnqueueEvent(const char * msg) = 0;
   virtual bool PollForEvents() = 0;
};

EnqueueEvent() simply adds to the cache of events within the shared object, and PollForEvents() will retrieve the cache.

shm_client1 will use EnqueueEvent() as follows:

powe->EnqueueEvent("Message from shm_client1");

shm_client2 (essentially a copy of shm_client1) will use PollForEvents() as follows:

powe->EnqueueEvent("Message from shm_client2"); powe->PollForEvents();

Additions in ObjectWithEvents are completed like this:

Listing 10. Additions to ObjectWithEvents
class ObjectWithEvents : public IObjectWithEvents
{
public:
   virtual bool EnqueueEvent(const char * msg);
   virtual bool PollForEvents();

   //The event cache
   enum { MAX_EVENTS = 16, MAX_EVENT_MSG = 256, };
   long m_nEvents;
   pid_t m_alPIDEvents[MAX_EVENTS];
   char m_aaMsgs[MAX_EVENTS][MAX_EVENT_MSG];
};

All this culminates in the new constructor:

ObjectWithEvents::ObjectWithEvents() : m_npEI(0), m_nEvents(0) { }

EnqueueEvent() stores events (such as message and process id for every fired event) to a queue. PollForEvents() iterates through the queue and calls OnEvent() for the enqueued events one by one.

Listing 11. EnqueueEvent
bool ObjectWithEvents::EnqueueEvent(const char * msg)
{
   if (NULL == msg)
   {
      return false;
   }

   if (MAX_EVENTS == m_nEvents)
   {
      //IEventSink collection full
      return false;
   }

   int bRetVal = Initializer::LockMutex();

   if (0 != bRetVal)
   {
      return false;
   }

   m_alPIDEvents[m_nEvents] = getpid();
   strncpy(m_aaMsgs[m_nEvents++], msg, MAX_EVENT_MSG - 1);

   if ((0 == bRetVal) && (0 != Initializer::UnlockMutex()))
   {
      // Deal with error.
   }

   return true;
}

bool ObjectWithEvents::PollForEvents()
{
   if (0 == m_nEvents)
   {
      return true;
   }

   int bRetVal = Initializer::LockMutex();

   if (0 != bRetVal)
   {
      return false;
   }

   pid_t pid = getpid();

   for (long i = 0; i < m_npEI; i++)
   {
      // Does the handler belongs to current process ?

      if (pid != m_alPID[i])
      {
         continue;
      }

      //Recheck for NULL

      if (0 == m_apEI[i])
      {
         continue;
      }

      for (long j = 0; j < m_nEvents; j++)
      {
         m_apEI[i]->OnEvent(m_alPIDEvents[j], m_aaMsgs[j]);
      }
   }

   if ((0 == bRetVal) && (0 != Initializer::UnlockMutex()))
   {
      // Deal with error.
   }

   return true;
}

Now try the build script:

g++ -g -o shm_client1 shm_client1.cpp ObjectWithEvents.cpp Initializer.cpp g++ -g -o shm_client2 shm_client2.cpp ObjectWithEvents.cpp Initializer.cpp

The output should look like this on your console:

Listing 12. Output from shm_client1 and shm_client2
$ ./shm_client1

$ ./ipcs

------ Shared Memory Segments --------
key        shmid      owner      perms      bytes      nattch     status
0x00001234 360454     sachin    666        4300       0

$ ./shm_client2
shm_client2.cpp:16 Message from pid(4454)        : Message from shm_client1
shm_client2.cpp:16 Message from pid(4456)        : Message from shm_client2

Download

DescriptionNameSize
Shared memory sample codesync_code.zip4KB

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Linux on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Linux
ArticleID=98406
ArticleTitle=Handle synchronous events from shared objects in Linux
publish-date=11102005