Store datasets directly in shared memory with PHP
A step-by-step guide on how to use shared memory in PHP as a storage option
Shared memory is an efficient way to exchange data between applications in the same machine. One process can create a memory segment that other processes can access, as long as it assigns the correct permissions. Every segment has a unique ID, called shmid that points to an area of physical memory where other processes can manipulate it. Once created, and given proper permissions, other processes in the same machine can manipulate those segments by: read, write, and delete.
This means that an application written in C can share information with an application written in other languages, such as Java™ or PHP. They can all share information, as long as they can access and understand that information. Shared memory is widely used with implementations made available for most languages, so access should not be a problem. As for understanding information, we can use a standard format like XML or JSON.
Use of shared memory is a fast method of data exchange between processes, mainly because there is no kernel involvement in passing data after the segments are created. Methods of this kind are often called interprocess communication (IPC). Other IPC methods include pipes, message queues, RPC, and sockets. This fast and reliable ability to exchange data between applications is invaluable when working with an ecosystem of applications needing to communicate with each other. The usual method of using databases to exchange information between applications often causes slow queries and even blocking I/O, depending on the size of the ecosystem. With shared memory, there's no I/O slowing a developer down.
This article's proposal is simple, learn how to create and manipulate shared memory segments with PHP and use them to store datasets that other applications can use. Even without a plan to use shared memory for data exchange, it is beneficial because it allows applications to stay away from I/O issues. Storing datasets directly in memory has many advantages, from web services data cache to session sharing. It's a very useful concept that every PHP developer should know.
Shared memory and PHP
PHP has a wide variety of extensions available, as does shared memory. With a few shared functions, and without having to install any extension, developers are able to easily manipulate segments.
Shared memory functions are similar to the file manipulation functions, but
instead of working with a stream, you'll be working with a shared memory
access ID. The first example of this is the
shmop_open function, which allows you to open
an existing segment or create a new one. This function is very similar to
fopen function that opens streams
for file manipulation, returning a resource that can later be used by
other functions that wish to read or write in to that opened stream. Let's
shmop_open in Listing 1.
Listing 1. The
<?php $systemid = 864; // System ID for the shared memory segment $mode = "c"; // Access mode $permissions = 0755; // Permissions for the shared memory segment $size = 1024; // Size, in bytes, of the segment $shmid = shmop_open($systemid, $mode, $permissions, $size); ?>
The first thing presented is the system ID parameter. This is the number
that identifies the shared memory segment in the system. The second
parameter is the access mode, which is very similar to the access mode of
fopen function. You can access a segment in
four different modes:
- Mode "a", which allows you to access the segment for read-only
- Mode "w", which allows you to access the segment for read and write
- Mode "c", which creates a new segment, or if the segment already exists, try to open for read and write
- Mode "n", which creates a new segment, and if the segment already exists, fail
The third parameter is the permissions of the segment. You must provide an octal value here.
The fourth parameter provides the segment size in bytes. Before writing in to a segment, you must allocate a proper number of bytes on it.
Note that this function returns an ID number that can be used with other
functions for manipulating the shared memory segment. This ID is the
shared memory access ID and is different from the system ID, passed as a
parameter. Be careful not to confuse the two. On failure,
shmop_open will return FALSE.
Write in to segments
shmop_write function to write in to a
shared memory block. This function is simple to use and accepts only three
parameters. This function, as shown in Listing 2, is simple to use and
accepts only three parameters.
Listing 2. Use
shmop_write to write in to a
shared memory block
<?php $shmid = shmop_open(864, 'c', 0755, 1024); shmop_write($shmid, "Hello World!", 0); ?>
There is a resemblance with the
which has two parameters: the opened stream resource, returned by
fopen, and the data you wish to write. The
shmop_write function does just that.
The first parameter is the ID returned by
shmop_open and identifies the shared memory
block you're manipulating. The second parameter is the data you want to
store and, finally, the third parameter is where you want to start
writing. By default, we always use 0 here to start writing from the
beginning. Note that this function returns FALSE on failure and the number
of bytes written on success.
Read from segments
Reading from shared memory segments is simple. All you need is an opened
segment and the
shmop_read function. This
function accepts a few parameters and works similarly to
fread. See Listing 3 to read the contents of a
file with PHP.
Listing 3. Use
shmop_read to read the contents of a file
<?php $stream = fopen('file.txt', 'r+'); fwrite($stream, "Hello World!"); echo fread($stream, 11); ?>
Reading the contents of a shared memory segment is similar, as shown in Listing 4:
Listing 4. Read the contents of a shared memory segment
<?php $shmid = shmop_open(864, 'c', 0755, 1024); shmop_write($shmid, "Hello World!", 0); echo shmop_read($shmid, 0, 11); ?>
Observe the parameters here. The
function will accept the ID returned by
shmop_open, which is already known to us, but
it also accepts two other parameters. The second parameter is where you
want to start reading from the segment; while the third one is the amount
of bytes you want to read. The second parameter might always be 0, the
beginning of the data, but the third parameter can be a problem because we
may not know how many bytes we want to read.
This is very similar to the behavior we had in the
fread function, which accepts two parameters:
the opened stream resource, returned by
and how many bytes you want to read from that stream. Use the
filesize function, which returns the amount of
bytes in a file to read it in its entirety.
shmop_size function returns the
size in bytes of a segment, similar to the
filesize function, when working with shared
memory segments. See Listing 5.
Listing 5. The
shmop_size function returns the
size in bytes of a segment
<?php $shmid = shmop_open(864, 'c', 0755, 1024); shmop_write($shmid, "Hello World!", 0); $size = shmop_size($shmid); echo shmop_read($shmid, 0, $size); ?>
Remove a segment
We learned how to open, write, and read shared memory segments. To complete
our CRUD class, we need to learn how to remove segments. Such a task can
be easily done by using the
function, which expects only one parameter: the shared memory ID that we
want to remove.
Listing 6. The
shmop_delete marks segment for deletion
<?php $shmid = shmop_open(864, 'c', 0755, 1024); shmop_write($shmid, "Hello World!", 0); shmop_delete($shmid); ?>
This doesn't actually remove the segment. It marks the segment for deletion
since shared memory segments cannot be removed while there are other
processes using it. The
will mark the segment for deletion and prevent any other process from
opening it. To remove it we need to close the segment.
Close a segment
Opening a shared memory segment "attaches" to it. After attaching to the
segment, we can read and write from it, but once we've finished our
manipulation, we must detach from it. This is done using the
shmop_close function in Listing 7.
This is very similar to the
fclose function when
working with files. After opening a stream with a file and reading or
writing from it, we must close it or locks will occur.
Listing 7. Use
shmop_close to detach from a segment
<?php $shmid = shmop_open(864, 'c', 0755, 1024); shmop_write($shmid, "Hello World!", 0); shmop_delete($shmid); shmop_close($shmid); ?>
Use shared memory as a storage option
With a basic knowledge of shared memory and basic CRUD operations on shared memory segments, it's now time to put this knowledge to work. We can use shared memory as a unique storage alternative, providing benefits such as fast read/write operations and process interoperability. For a web application, this means:
- Cache storage (database queries, web services data, external data)
- Session storage
- Data exchange between applications
Before we proceed, I would like to present a small library called SimpleSHM. SimpleSHM is a small abstraction layer for shared memory manipulation using PHP, allowing easy manipulation of segments in an object-oriented way. This library helps create much cleaner code when writing a small application that uses shared memory for storage. To get started into SimpleSHM, grab a tarball from the GitHub page.
You have three methods to work with: read, write, and delete. Simply instantiating an object from the class will control the shared memory segment opening. Listing 8 shows basic use.
Listing 8. SimpleSHM basic use
<?php $memory = new SimpleSHM; $memory->write('Sample'); echo $memory->read(); ?>
Note that an ID is not passed for the class. If an ID isn't passed, it will choose a number randomly and open a new segment with that number. We can pass a number as a parameter for the constructor to open an existing segment or to create with a specific ID as shown in Listing 9.
Listing 9. Open a specific segment
<?php $new = new SimpleSHM(897); $new->write('Sample'); echo $new->read(); ?>
The magic method
__destructor takes care of
shmop_close on the segment to unset the
object in order to detach from the segment. We'll call this "SimpleSHM
101." Now let's use this for a higher purpose: using shared memory as
storage. Storing datasets requires serialization since arrays or objects
cannot be stored in memory. While JSON is used here for serialization, any
other method, such as XML or the built-in PHP serialization features, will
suffice. See Listing 10 for an example.
Listing 10. Use shared memory as storage
<?php require('SimpleSHM.class.php'); $results = array( 'user' => 'John', 'password' => '123456', 'posts' => array('My name is John', 'My name is not John') ); $data = json_encode($results); $memory = new SimpleSHM; $memory->write($data); $storedarray = json_decode($memory->read()); print_r($storedarray); ?>
We successfully serialized an array into a JSON string, stored in a shared memory block, read from it, unserialized the JSON string, and displayed the stored array. This seems basic, but imagine the possibilities of this snippet. You can use it for storing the results of a web service request, database query, or even template engine cache. Reading and writing from memory will provide much higher performance than reading and writing from disk.
Using this storage technique is not only useful for cache, but also for data exchange between applications, as long as data is stored in a format readable by both ends. Don't underestimate the power of shared memory in web applications. There are many different ways to cleverly implement this kind of storage and the only limit is a developer's creativity and skill.
This article covers most of the tools available in PHP's toolkit for manipulating shared memory segments, and explains how shared memory works. Further, it offers suggestions for improving web applications and outlines some of the factors to consider when creating a solution for common web application problems. The concepts and implementation guidelines can help you establish a starting point. The early model we built can help you envision more complex features and solutions.
We outlined some common problems that are a perfect fit for shared memory, such as caching, session sharing, and common data exchange between applications. This introduction to shared memory provides you the opportunity to explore much more elegant solutions to common problems. Feel free to extend the current implementation of SimpleSHM to match your needs and contribute the changes back to the project.
- Michael Kerrisk's book The Linux Programming Interface has excellent chapters about interprocess communication and chapters 45 to 48 are dedicated to System V IPC.
- Dave Marshall’s "IPC:Shared Memory" article has an interesting and pragmatic approach to shared memory functions in C language.
- Richard Stevens UNIX Network Programming has great technical content and several implementations in C. Check the sample chapter.
- Expand your PHP skills by checking out IBM developerWorks' PHP project resources.
- Read other developerWorks articles and tutorials related to PHP content.
- Follow developerWorks on Twitter.