Applying memcached to increase site performance

Reduce reads from databases and data sources

The open source memcached tool is a cache for storing frequently used information to save you from loading (and processing) information from slower sources, such as disks or a database. It can be deployed in a dedicated situation or as a method of using up spare memory in an existing environment. Despite the simplicity of memcached, it is sometimes used incorrectly, or it is used as a solution in the wrong type of environment. Learn when it is best to take advantage of using memcached.

Share:

Martin Brown (troy@backstopmedia.com), Freelance writer, Freelance Developer

Martin Brown has been a professional writer for over seven years. He is the author of numerous books and articles across a range of topics. His expertise spans myriad development languages and platforms -- Perl, Python, Java™, JavaScript, Basic, Pascal, Modula-2, C, C++, Rebol, Gawk, Shellscript, Windows, Solaris, Linux®, BeOS, Mac OS/X and more -- as well as Web programming, systems management, and integration. Martin is a Subject Matter Expert (SME) for Microsoft® and regular contributor to ServerWatch.com, LinuxToday.com, and IBM developerWorks. He is also a regular blogger at Computerworld, The Apple Blog, and other sites. You can contact him through his Web site at: http://www.mcslp.com.



03 August 2010

Also available in Japanese Portuguese

Introduction

memcached is used to speed up application processes, and here we will focus on the best practices for deploying it within your applications and environments. This includes what you should store and what you shouldn't, how to handle the flexible distribution of data, and how to regulate the method for updating the memcached and stored version of the data. We will also cover the support of high-availability solutions, including IBM WebSphere® eXtreme Scale.

All applications, especially many web applications, need to optimize the speed that they access and return information to the client. Frequently, however, the same information is being returned. Loading the data from your data source (database or filesystem) is inefficient, especially if you ultimately run the same queries every time you want to access the information.

Although many web servers can be configured to use a cache to send back information, that doesn't work with the dynamic nature of most applications. This is where memcached can help. It provides a generalized memory store that can hold anything, including native-language objects, enabling you to store a variety of information and access it from many applications and environments.


The basics

memcached is an open source project designed to make use of the spare RAM in many servers to act as a memory cache for frequently accessed pieces of information. The key element is the use of the word cache: memcached provides temporary storage, in memory, of information that can be loaded from elsewhere.

For example, consider a typical web-based application. Even a web site served up dynamically probably has some components or information constant throughout the life of the page. Within a blog, the list of categories for the individual blog posts is unlikely to change regularly between page views. Loading this information each time through a query to the database is comparatively expensive, especially when the data has not changed. You can see in Figure 1 some of the page fragments within a blog that could be cached.

Figure 1. Cacheable elements of a typical blog page
Diagram shows cacheable blog elements and their layout: Page Header at top, Current post lists at left, About, Post History, External Links and Category list stacked on the right

Extrapolate this structure to other elements of the blog, poster information, comments — even the blog posts themselves — and it is possible to identify 10-20 database queries and formatting that take place just to display the content of the home page. Repeat this over hundreds or even thousands of page views each day, and your servers and applications will be executing far more queries than are necessary to show the page contents.

By using memcached, you can store the formatted information loaded from the database in a form ready to be used directly on the web page. And because the information is being loaded from RAM, and not from disk through a database and other processing, the access to the information is almost instantaneous.

To reiterate, memcached is a cache for storing frequently used information to save you from loading and processing information from slower sources, such as disks or a database.

The interface to memcached is provided over a network connection. This means you can share a single memcached server (or multiple servers, as will be demonstrated later in this article) with multiple clients. The network interface is quick, and to improve performance, the server deliberately does not support authentication or secure communication. But this shouldn't limit the deployment options. The memcached server should exist within the inside part of your network. The practicality of the network interface and the ease with which you can deploy multiple memcached instances enables you to make use of the spare RAM on multiple machines and increase the overall size of your cache.

The storage method with memcached is a simple keyword/value pair, similar to the hash or associative array available in many languages. You store information into memcached by supplying the key and the value, and recover the information by requesting the information by the specified key.

Information is retained in the cache indefinitely, unless one of the following occurs:

  1. The memory allocated for cache is exhausted— In this instance, memcached uses a least-recently used (LRU) method to remove items from the cache. Items that have not been used recently are deleted from the cache, oldest access first.
  2. The item is specifically deleted— You can always delete an item from the cache.
  3. The item is expired— Individual items can have an expiration to allow them to be flushed from the cache when the information stored against the key is likely to be too old.

These situations can be used in combination with your application logic to ensure that the information in the cache is up to date. With that in mind, let's examine how best to use memcached in your applications.


When to use memcached

There are a number of key processes and steps that can be modified when using memcached to improve your application performance.

When loading information, the typical scenario is as shown in Figure 2.

Figure 2. Typical sequence for loading information for display
Diagram shows the flow from Loading data to Processing/formatting data to Sending data to client

In general terms, the steps are:

  1. Execute one or more queries to load the information from the database
  2. Format the information suitable for display (or further processing)
  3. Use or display the formatted data

When using memcached, you change the application logic slightly to accommodate the cache:

  • Try loading the information from the cache
  • If it exists, use the cached version of the information
  • If it does not exist:
    1. Execute one or more queries to load the information from the database
    2. Format the information suitable for display or further processing
    3. Store the information into the cache
    4. Use the formatted data

This is summarized in Figure 3.

Figure 3. Loading information for display when using memcached
Diagram shows that if requested data resides in the cache it skips all the processing steps, saving time

The loading of data then becomes at most a three-stage process, loading the data from the cache or from the database and storing in the cache, as appropriate.

The first time this process occurs, the data will be loaded from the database or other source as normal, then stored into memcached. The next time the information is accessed, it will be pulled from memcached, rather than loaded from the database, saving you time and CPU cycles.

The other side of the equation is to ensure that if you ever change the information that might be stored within memcached, you update the memcached version at the same time you update the back-end information. This modifies the typical sequence from that shown in Figure 4 to the slight modification in Figure 5.

Figure 4. Updating or storing data in a typical application
Diagram shows the flow from Updating data to Processing/formatting data to Sending updated dat to the client

Figure 5 shows the modified sequence using memcached.

Figure 5. Updating or storing data when using memcached
Diagram shows expanded flow: Updating data to Processing/formatting data to Storing in memcached to Sending updated data to client

For example, using the blog as the basis, when the blog system updates the list of categories in the database, the update should follow this sequence:

  1. Update category list in database
  2. Format the information
  3. Store the information into memcached
  4. Return the information to the client

Storage operations within memcached are atomic, so the information will be updated without clients getting only partial data; they will get the old version or the new one.

For the majority of applications, these are the only two operations you need to worry about. When you access data that people use, it gets automatically added to the cache, and changes to that data are automatically updated in the cache.


Keys, namespaces, and values

An important consideration with memcached is how you organize and name the data you store within the cache. From the previous blog examples, it should be clear that you will need to use a consistent naming structure so you can load the blog categories, history, and other information, then use that when loading information (and updating the cache), or when updating the data (and, again, updating the cache).

The exact naming system you use is application-specific, but you can usually use a similar structure of your existing application, which will probably be based on some kind of unique identifier. This occurs when pulling information from the database or when collating a collection of information.

Using the blog post example, you might store the list of categories in an entry with the key category-list. The values associated with a single post against the post ID, such asblogpost-29, could be used, while the comments against the entry could be stored in blogcomments-29, where 29 is the blog post ID. This way, you can store a huge variety of information into the cache using different prefixes to identify the information.

The simplicity of the memcached key/value store (and the lack of security) means that if you want to support multiple applications using the same memcached server at the same time, you may want to consider using some other form of quantifier to identify the data as belonging to a specific application. For example, you might add an application prefix like blogapp:blogpost-29. The keys are free-form, so you can use any string you like as the key name.

In terms of storing values, you should make sure the information you store into the cache is suitable for your application. For example, with the blog system, you may want to store the object used by the blog application to format the post information, rather than the raw HTML. This can be more practical if the same basic structure is used in multiple places within your application.

Most language interfaces, including Java™, Perl, PHP, and many others, can serialize language objects for storage within memcached. This enables you to store and later recover entire objects from the memory cache, instead of reconstructing them by hand within your application. Many objects, or the structures they use, are based on some kind of hash or array structure. For cross-language environments, like when you want to share the same information between your JSP environment and a JavaScript environment, you can use an architecture-neutral format, such as JavaScript Object Notation (JSON) or even XML.


Populating and using memcached

As an open source product, and one originally developed to work within an existing open source environment, memcached is supported by a wide range of environments and platforms. The interfaces for communicating with a memcached server are numerous, often with multiple implementations for all the languages. See Resources for common libraries and toolkits.

It would be impossible to list all of the supported interfaces and environments offered, but they all support the basic API provided by the memcached protocol. These descriptions have been simplified and should be taken within the context of the different languages where errors can be indicated using different values. The main functions are:

  • get(key)— Gets the information from memcached stored the specified key. Returns an error if the key does not exist.
  • set(key, value [, expiry])— Stores the specified value using the identifier key in the cache. If the key already exists, it will be updated. The expiry time is in seconds and is taken as a relative time if the value is less than 30 days (30*24*60*60), or an absolute time (epoch) if larger than this value.
  • add(key, value [, expiry])— Adds the key to the cache if it does not exist or returns an error if the key already exists. This can be useful if you want to explicitly add a new key without updating it if it already exists.
  • replace(key, value [, expiry])— Updates the value of the specified key, returning an error if the key does not exist.
  • delete(key [, time])— Deletes the key/value pair from the cache. If you supply a time, adding a new value with this key is blocked for the specified period. The timeout allows you to ensure that the value is always reread from your data source.
  • incr(key [, value]) — Increments the specified key by one or the specified value. Only works on numerical values.
  • decr(key [, value])— Decrements the specified key by one or the specified value. Only works on numerical values.
  • flush_all— Invalidates (or expires) all the current items in the cache.

For example, within Perl, a basic set operation would be handled as shown in Listing 1.

Listing 1. Basic set operation within Perl
use Cache::Memcached;

my $cache = new Cache::Memcached {
    'servers' => [
                   'localhost:11211',
                   ],
    };

$cache->set('mykey', 'myvalue');

The same basic operation in Ruby is shown in Listing 2.

Listing 2. Basic set operation within Ruby
require 'memcache'
memc = MemCache::new '192.168.0.100:11211'

memc["mykey"] = "myvalue"

You can see in both examples the same basic structure: set the memcached server, then assign or set the value. Other interfaces are available, including those that work within Java technology, allowing you to use memcached within your WebSphere applications. The memcached interface class allows you to serialize Java objects straight into memcached so you can store and load complex structures. When deploying within an environment like WebSphere, two things are really important: the resilience of the service (and what to do if memcached isn't available) and how to increase your cache storage to improve your performance when using multiple application servers, or environments like WebSphere eXtreme Scale. We will look at both of these issues next.


Resilience and availability

One of the most common questions about memcached is, "What happens when the cache is unavailable?" As should be clear from the previous sections, the information in the cache should never be the only source for the information. You must be able to load the data stored in the cache from some other location.

Although the practicality is that not being able to access information from the cache will slow down the performance of your application, it should not stop the application from functioning. There are a few scenarios that can take place:

  1. If the memcached service goes down, your application should fall back to loading the information from the original data source and formatting it as necessary for display. The application should also continue to try to load and store the information in memcached.
  2. Once the memcached server becomes available again, your application should automatically try to store the data. There is no need to forcibly reload the cached data, you can use the standard access to load and populate the cache with information. Eventually, the cache will be repopulated with the most commonly used data.

To reiterate, memcached is a cache of information and not the only source. Losing your memcached server should not be the end of your application, although it might imply a decrease in performance until the memcached server comes back up. In practice, the memcached server is relatively simple, and although not crash-free, the simplicity leads to fewer errors.


Distributing your cache

The memcached server is just a cache storing values against keys over a network. If you have multiple machines, the temptation is to set up a memcached instance on all of your spare machines to provide a massive amount of networked RAM cache storage.

When faced with this idea, the temptation is to assume that you need some sort of distribution or replication mechanism that will copy the key/value pairs between the machines. The problem with this approach is that you are actually reducing your available RAM cache, not increasing it. If you look at Figure 6, you can see that there are three application servers, each with access to a memcached instance.

Figure 6. Incorrect use of multiple memcached instances
Diagram shows three isolated 1-GB instances of memcached supporting three application servers yields 1 GB of cache space each

Although each memcached instance is 1 GB in size (giving 3 GB of RAM cache), if each application server has only its own cache (or there was replication of data between memcached instances), your entire installation would still only have 1 GB of cache duplicated across each instance.

Since memcached provides its information over a network interface, a single client can access the data from any memcached instance it has access to. If the data is not copied or duplicated across each instance, what you end up with is 3 GB of RAM cache available to each application server, as shown in Figure 7.

Figure 7. Correct usage of multiple memcached instances
Diagram shows three interactive 1-GB instances of memcached supporting three application servers resulting in 3 GB of shared cache space overall

The problem with this approach is choosing which server to store your key/value pair on and how you determine which memcached server to talk to when you want to recover a value. The solution is to ignore complexities like lookup tables or expecting the memcached server to handle the process for you. Instead, memcached clients must think simply.

Instead of having to determine this information, the memcached clients use a simple hashing algorithm on the key you specify when storing the information. When you store or get information from a list of memcached servers, the memcached client derives a numerical value from the key using a consistent hashing algorithm. To give you an example, the key mykey gets converted to the value 23875. It doesn't matter whether you are storing or getting information, you will always be using the same key as the unique identifier to load from the memcached server, so "mykey" will always get hash to a value of 23875, in this example, anyway.

If you have two servers, the memcached client takes this numerical value and performs a simple calculation on it (for example, modulus) to determine whether it should store the value on the first, or second, configured memcached instance.

When you store a value, the client determines the hash value from the key and the server used to store it. When you get a value, the client determines the same hash value from the key and chooses the same server to get the information.

If you use the same server list (and the same order) on every application server, every application server will choose the same server when asked to store or retrieve the same key. Now you are sharing the 3 GB of memcached space, instead of duplicating the same 1 GB of space, giving you more cache to use and, in all likelihood, improving the performance of your application for even more users.

There are complexities to this process (such as what happens when a server is unavailable), but the documentation offers more information (see Resources).


How not to use memcached

Despite the simplicity of memcached, the temptation is to use the memcached instance in some unexpected, and sometimes unfortunate, ways.

memcached is not a database

Probably the most common misuse of memcached is to use it as a data store, not as a cache. The primary purpose of memcached is to improve the response times for data that can otherwise be constructed or recovered from some other source. An example is recovering information from a database, particularly if there is any formatting or manipulation of the information before it is displayed to the user. Memcached is designed to store this information in memory to save repeatedly performing that task each time the data is recovered.

You should never use memcached as the only source for the information you need to run your application; the data should always be derivable from some other source. Also, keep in mind that memcached is just a key/value store. You cannot perform a query over the data or iterate over the contents to extract information. You should use it to store blocks or objects of data that you need to use on a wholesale basis.

Do not cache database rows or files

Although you can use memcached to store the rows of data as loaded from a database, this is really query caching, and most databases provide their own query-caching mechanism. The same can be said for other objects, such as images or files from the filesystem. Many applications and web servers already have well-optimized solutions for this type of work.

You will gain more utility and performance improvements from memcached if you use it to store entire blocks of information after loading and formatting. Using our blog example, the best point to store information is once you have formatted the blog categories as an object, or even after formatting into HTML. Construction of the blog page can then be achieved by loading the components (blog posts, category list, post history, etc.) from memcached and writing the completed HTML back to the client.

memcached isn't secure

To ensure the maximum performance, memcached does not provide any form of security, either in authentication or encryption. This means that access to your memcached servers should be handled by putting them in the same private-side zone of your application deployment environment, or, if security is a must, use UNIX® sockets and only allow applications on the current host accessing the memcached server.

This removes some of the flexibility and resilience, and your ability to share a RAM cache across multiple machines over the network, but this is the only solution for securing your memcached data in this situation.


Don't limit yourself

Despite the things that you shouldn't use your memcached instances for, the flexibility of memcached shouldn't be ignored. Because memcached sits at the same architectural level as your application it is easy to integrate and connect to. And changing your application to take advantage of memcached isn't complicated. Furthermore, because memcached is just a cache, it shouldn't stop the execution of your application in the event of a problem. What it does do, if used correctly, is lower the load on the rest of the server infrastructure (reduces reads from databases and data sources) and that means supporting more clients without needing more hardware.

But remember, it's just cache!


Summary

In this article, we have covered memcached and how best to use it. In particular, we have looked at how information is stored, how to choose reasonable keys, and how to choose the information to store. We have also looked at some key deployment issues that all users of memcached experience, including the use of multiple servers, what to do when a memcached instance dies, and, perhaps most importantly, how not to use memcached.

As an open source application, and one with a very simple and straightforward aim, the power and utility of memcached comes from that simplicity. By providing a massive RAM store for information, making it available on the network, then accessible through such a huge range of different interfaces and languages, you can integrate memcached into a huge variety of installations.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Open source on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Open source, XML
ArticleID=503412
ArticleTitle=Applying memcached to increase site performance
publish-date=08032010