"State" and "persistence" are crucial "terms of art" for computing. They are concepts that arise throughout computing, but have meanings hard to understand from outside the domain. Clarity about these is essential for a developer of modern distributed applications.
In general, "state" refers to information about the current conditions of program execution—runtime data stored in memory. "Persistence," by contrast, refers to keeping data between one program execution and another. In a program which iterates through a table in a database, the database itself is "persistent," but the information about which row is being displayed is "state."
When applied to a protocol, "state" treats each series of interactions as having continuity, much like a single program's state. A "stateless" protocol is one in which there is no such continuity; each request must be processed entirely on its own merits. HTTP is conventionally considered a stateless protocol.
To call HTTP a stateless protocol (see Resources, below), as we conventionally do, means both more and less than is first apparent. HTTP is fundamentally a request-response couple: A browser requests a particular URL, perhaps with supplementary data, and the server answers with a response page. While the end-user might experience his or her surfing as a trip made up of steps in a particular sequence, at the protocol level each delivered page is independent of the others; any display is simply the output corresponding to the latest URL-plus-data input.
How is it, then, that Web developers talk about "sessions," "logins" and "logouts," "personalization," "hijacking," and other such inherently stateful ideas? HTTP is supplemented by several devices which give it state functionality; that is, standards subordinate to the HTTP definition give mechanisms that, among other functions, can be interpreted as state interfaces.
Most Web frameworks and browsers layer higher-level interfaces (a programmable Session object, for example) that simplify development. At a general
level, though, they encapsulate one or more of the following basic session-maintenance
mechanisms in an abstract interface.
Note that there is much more to be said about each of these than is possible to include in a survey such as this. For more details on each technique, please see Resources.
HTTP interaction blurs the distinction between state and persistence a bit. Developers may refer to data preserved through a given browsing session as "state," but data preserved between browsing sessions as "persistent." Alternatively, some might call data "state" when it is preserved by the user's browser, and "persistence" when it is preserved by the server. It wouldn't be modern computing without inconsistent use of basic terminology. This article uses the term "state" to refer to data kept by the client, and "persistence" to refer to data kept by the server; this loosely reflects the original "Cookie" spec, which described cookies as providing "persistent client-side state" (see Resources).
It is possible to store state in a client program, and for many applications, this is desirable. If state is always provided by the client, the server remains (at least from a programmer's viewpoint) stateless: There is no data to maintain, just incoming data to process as it shows up.
The "cookie" spec was introduced by Netscape long, long, ago. The name reflects the convention of referring to identifiers as "magic cookies"—blame MIT. Cookies allow name/value pairs to be associated with a given URL, although they can be set generically across a domain. Early browsers were vulnerable to a number of spoofing tricks that could reveal personal information, and to this day many users disable cookies, or throw them out regularly. By default, cookies expire when the browser is closed, but a cookie with a specified expiration is supposed to survive until that expiration, even if the user closes the browser and reopens it. The use of these "persistent" cookies is discouraged by many best practices specs. They do not reliably work (many browsers can be configured to clear all cookies on exit, even persistent ones), and they increase security risks. If you must use them, be aware that they will not always work. So, you will need a workaround for the cases where persistent cookies are not available. This, in turn, means that you do not really need them.
While cookies are now specified by a standard (RFC 2965, "HTTP State Management Mechanism"—see Resources for a link), users continue to undermine their functionality, and often do so for very good reasons. Cookies are essentially insecure, requiring browsers to provide additional restrictions for security purposes. Furthermore, they're sent in plain text, making it dangerous to store private data (such as user names or passwords) in cookies. As a result of these security concerns, cookies cannot reliably be used across multiple Web sites. Many browsers refuse to process cookies not in the domain of the current page. Furthermore, cookies don't distinguish between multiple users of a shared computer, making them even more dangerous. Much as it pains me to say it, cookies are a "sometimes food." Used well, they can be a wonderful part of a state-handling mechanism. Used all the time for everything, they are not a good thing at all. As cookies won't always work, you may need to know about other state mechanisms, too.
Forms, methods, and actions, oh my!
There are two historical ways to provide state through the client; both are commonly seen, even today, with forms. The first is to append state information to a URL, as parameters to a CGI program or something similar. The second is to use HTTP POST to submit parameters.
Appending parameters to the URL turns out to be, in many cases, exceptionally simple. Furthermore, it has one obvious advantage over POST parameters: You can create an arbitrarily complicated URL and use it in a standard link tag, without the need for a surrounding form and submit button, or some similar device.
On the other hand, POST has advantages, too. One of the most significant is that POST data does not become part of a user's browser history. Furthermore, as the HTTP specification recommends strongly that GET operations be "idempotent" (multiple retrievals have no more lasting effects than single retrievals), users may be very painfully surprised if a GET operation has side effects, such as charging a credit card or cancelling a reservation. For anything which has side effects, POST is superior.
Both POST parameters and URL suffixes are, of course, vulnerable to user editing. There are many horror stories of shopping cart applications which stored a calculated price in a parameter editable by the user. Furthermore, users who bookmark or share URLs can wreak havoc on a system dependent on URL mechanisms for storing state. Avoid using URLs to store state. Using POST data for state is not as bad, but requires every link within a user's session be a form submission. These mechanisms have noticeable flaws. While they are very good for form data and interaction, they are not very good for state. Cookies are perhaps better.
Further blurring the already fuzzy line between state and
persistence, a page using Ajax or similar techniques may indeed mean "state," in the purer sense
of internal program state for a given running program, when talking about Web access.
Ajax-based
applications use JavaScript variables to control state; these variables are then used to guide
decisions about new requests to submit. XMLHttpRequest is a
particular example of a way for JavaScript (or other scripting languages) to use internal state
to dynamically update and alter Web pages.
In practice, many users have JavaScript disabled or restricted in behavior, so relying on it for core functionality is often a bad choice. Portability problems and inconsistent implementations have also seriously compromised JavaScript as an implementation language. While this is getting better for bleeding-edge users, millions of users are on older browsers.
Security holes in JavaScript implementations may expose your users to unexpected risks, which they may blame on your Web site. More than one otherwise-secure site has been compromised through insufficiently paranoid JavaScript coding practices.
Browser plug-ins can maintain their own state, which may or may not be persistent across browser sessions. The Flash plug-in used to use cookies (see below), and still can, but has also added support for "Local Shared Objects" (LSOs), which are binary objects (not just plain text) of potentially significant size. LSOs seem to have been introduced for things like high-score lists in games, but can be (and are) used for many other purposes as well. Don't count on this: As users become aware of the security implications, plug-in vendors may start providing security features. Once the security features are there, these methods for data storage will start being disabled or cleared, just as cookies are today.
All plug-in-based mechanisms face the weakness that a user may not have a plug-in, and may be unable to get the plug-in. Perhaps it isn't available for a given platform, or the user's corporate IT department won't allow the plug-in. Linux® or UNIX® users may not thank you for building a site which depends on a browser plug-in, and if you want to talk about standards conformance, you'll have to look elsewhere no matter what your users think.
Many of the same criticisms that apply to plug-ins apply to "webOSes" which try to mimic a desktop environment on the Web. A virtual file system or similar mechanism can, of course, store state—for the users who have it.
Login credentials can also be stored on a browser. These are the familiar name/password pairs used by simple server security systems. HTTP authentication has been widely used, and does allow for a little bit of security, and possibly a little bit of state. While these can maintain a very small amount of state, they are inadequate for most stateful applications. I mention them here because they do have one crucial use: They can help control access to state that's being stored on the server.
Like cookies, HTTP authentication credentials can be used to identify a user, with actual state management occurring elsewhere. However, they are harder for users to manipulate—which is often a disadvantage, as users find it annoying. Most users would be hard-pressed to clear their HTTP authentication credentials for a site by any means other than logging out. On the other hand, using login credentials like this is not an abuse of the spec. Having authenticated once, a browser may preemptively send stored login credentials to a site without being prompted. This is nice for transparency to the user. In some browsers, a JavaScript alert can look just like the HTTP authentication password window—not a good thing at all.
The default form of HTTP authentication, "basic authentication," sends passwords in plain text. This is insecure (except maybe over SSL). One option is to use digest authentication, which uses MD5 hashes and is less vulnerable to snooping. Ideally, for security, you would want to use digest authentication over SSL.
One other weakness is that HTTP authentication makes no provision for how to establish the name and password used for a login, only for how to use them once they have been created.
All of the mechanisms available for storing state information on the client have flaws; the most fundamental is that the client can lose this state, leaving the server with no idea what happened. Furthermore, the client can alter this state. Thus, the server must validate all state content on every transaction, if the state is provided by the client. The obvious solution is to keep state on the server, instead.
To
do this, servers have the concept of a "session," which is a collection of data associated with
a single user's interactions with the site. Languages used for Web programming tend to acquire
session management features. Perl's CGI::Session module,
Ruby's
CGI::Session class (no relation, I'm told), and PHP's session_*()
functions, all reflect common design requirements.
The persistence mechanism is not the only architectural decision that needs to be made. Another very important decision is how full state is partitioned between the client and server sides. In fact, once you think about that, the details of how session data is stored on the server are surprisingly trivial for purposes of understanding session handling. All you really need to know is that data is reliably stored and can be accessed given only a single key, usually called the session key or session ID. If you can just find out the user's session ID, you have a large quantity of data available.
Think of the alternatives this way: Does your "shopping cart" inhabit the browser or the server?
In the former case, the server discards the data represented by the shopping cart as soon as it has been used to render an appropriate page. With this model, it's the browser that takes responsibility for tracking that you want one anchovy pizza and two automobile batteries, for immediate delivery. If the browser session is lost, the whole shopping cart also vanishes. In most cases, that's a disadvantage.
In the latter case, the server accumulates in its datastore a collection of your selections, indexed by your session. Your browser might crash, but, as long as you have the cookie or bookmark or login from your shopping trip, you can rejoin and finish your shopping.
This does offer significant improvements. Data can be stored in the session with no direct way for a malicious user to alter it. (Bugs in your code may still give an attacker an opening, of course.) Important data need not be transferred over the Internet, especially not as cleartext. However, it still leaves the requirement that the session key be somehow communicated.
Server-side state management seemed obvious, at first. The server has substantial resources to devote to maintaining state. What it doesn't have is a completely fool-proof way to figure out which of the maintained states it is using should be associated with a given query. This sounds like a wonderful application for client-side state mechanisms.
With a collection of ways to maintain simple state on the client, and a collection of ways to maintain detailed and complicated state on the server, what is missing is a connection between them. Enter the session ID. The client maintains only one piece of state—a session identifier of some sort. The server can then use that identifier to look up all the complicated state (some of which the client shouldn't have access to!) and provide stateful behavior.
Session IDs are managed in several ways. A cookie can store the session ID, a session ID can be embedded in a form as a hidden value, or a session ID can be appended to URLs. In fact, all three of these methods can be used, and some session management toolkits will transparently try all of them in the hopes that at least one will work. This behavior is automatic in essentially every session handler I know of.
This will get you to the point of consistency within a browsing session, or possibly for the lifetime of the cookie. What if the user clears out private browser data, or this happens automatically? Where does the session go?
This line of inquiry leads to a more advanced technique. Data is associated with user accounts of some sort, using user name and password keys. When a user logs in, session management is used primarily to retain awareness that the user is logged in; other state data is associated with the user account, allowing the user to regain that state after a disconnection or loss of state. This clearly crosses the line from "state" to "persistence."
This furthermore allows a clearer distinction between temporary settings which should be remembered only during a given series of browser interactions, and permanent data about the user.
Finding out which method or combination of methods your toolkit uses, and learning as much about each as you can, can help you make better decisions and improve the security and performance of your Web applications.
As mentioned throughout this article, state mechanisms expose a risk. What if someone other than the intended user gets access to the user's state information? There are a number of other features which may help reduce the risks of trying to keep state for HTTP. This section reviews a few of the most commonly used tools.
SSL and related technologies have two primary purposes: providing encrypted communication between two parties, and providing some assurance of the identity of at least one of the two parties.
Cookies, HTTP authentication, and the various ways of encoding session IDs in URLs or forms all face the common weakness that, by default, they make at least some amount of state data available in cleartext. Using SSL/TLS to encrypt communications reduces this risk. Furthermore, the confirmation of the identity of the Web server reduces the risk of man-in-the-middle attacks being used to obtain and abuse a user's credentials, or take advantage of information stored in a given session.
Secure sockets are usually managed, for HTTP, using the alternative "HTTPS" protocol, which is really just HTTP over a secure socket. This imposes a number of requirements. For instance, HTTPS uses a different port (although most modern firewalls let it through without complaint). However, the conflation of encryption with identification often leads to user confusion. You can't just use encryption; you must have a certificate of identification, and if the user doesn't recognize the authority that signed your certificate, dire warnings are presented.
In a world of dynamic IP addresses and proxies, an IP address can hardly be considered a completely reliable user identifier, but as a secondary data point to help verify a user's identity, or track when a new user has connected, an IP address can be a wonderful item to have in your toolkit. It does help that broadband providers often try to keep a user's IP static as long as possible, but you can't always be sure.
The biggest advantage of the IP address, of course, is that it is tracked automatically. After all, without an IP address, the server couldn't respond to the request. Proxy servers make it quite possible that the IP address visible to the server is only marginally useful, however. In particular, don't get too clever about invalidating a session when an apparently different user shows up from the same IP address; a large corporate proxy may have several users visiting the same site at the same time. In practice, IP tracking is useful in combination with other methods, or for statistics where good guesses are good enough, but is not usable as a reliable state-tracking mechanism.
The decision to spell "Referrer" as "REFERER" was, as irritations go, right up there with the flagrant grammatical error in Apache's default 404 error message. However maddening the spelling error is, information about the referring page is often useful in determining whether a given request ought to be viewed as part of an ongoing session, or the beginning of a new one. Much like the IP address, the referrer is useful as a heuristic, but is no substitute for a real state tracking mechanism. Again, like the IP address, this can be useful in accumulating statistics about user behavior.
Be aware that the referrer is completely forge-able: Some browsers will refuse to provide it, and some will lie.
In all, I would end this look at the state of statefulness on a note of reasoned optimism for Web apps. For a great many applications, they are "good enough." There are many non-technical types—executives or Venture Capitalists (VCs), for instance—that apparently believe that desktop applications, or traditional client-server applications, need never be deployed again. While this is overly optimistic, it does affirm that Web development has come a long way.
There are even technical types who will agree that Web applications are easier to deploy (see Resources). After all, they offer an installation-free environment with a well-known API—and one that works through most firewalls. The lack of state is what makes this installation-free environment possible. By forcing developers to resolve persistence problems almost entirely on the server, without relying on the client system, the lack of state has pushed developers into independence.
It was not always this easy. Web applications were originally crippled in performing long or interactive processes by their lack of state. Many pages performed a single task at a time, all at once, resulting in excessively complicated forms with poor or obnoxious error checking. The current state of the art in session management reflects years of trial and error development. However, with those problems mostly overcome, Web applications are now utterly ubiquitous—because they have no requirements, no installation, and no configuration for the user. The same weakness that required years of development and research to resolve and work around has become the "killer app" for Web development.
Projects such as NanoHUB (again, please see Resources) demonstrate the extent to which a Web application can be pushed. If you really and truly want the strengths of a desktop app to be accessible over the Web, you can have that—but not "for free" as VCs and others sometimes seem to think. Right now, it comes at a price (NanoHUB has over 12 years of programming behind it) that most organizations and companies are not prepared to pay.
All too often, your possibilities are only as good as the browser is. Browser developers need to pick up the pace: Perhaps set aside their differences and really do one standards-compliant and SECURE combo of authentication and cookies. Make "persistent HTTP" live up to the promise its name seems to imply. At least, they really should all implement digest authentication, and stop using (or warn users about) basic authentication. As a developer, you must be aware of these limitations. Work with the user, not against the user; a warning that your site "requires cookies" is not as good as a site that uses other mechanisms as a fallback.
We have a long way to go. Browser developers need to do a better job. Session management tools need to be more mature and more robust. Web frameworks could, in many cases, provide better and more reliable session management tools. Too many people are writing their own wrappers which perform the same common tasks over and over. Individual app developers and users also need to do better. As a Web developer, you can't do much about browsers, but you can do a lot about how your application uses the tools available. Understand where the security concerns are, and you can design your application so it doesn't expose private data over insecure channels.
Learn
- Read the original specification for Hypertext Transfer
Protocol (HTTP): RFC 2616
- Learn about MIT magic cookies, the
basis for the
browser's "cookie" terminology.
- Read the original specification for HTTP cookies: RFC 2965
- Cookie
Monster: "Cookies are a sometimes food." (Even the Cookie monster has to cut back on
cookies.)
- LSO Flash cookies:
definition and general usage information by Adobe
- See an example of the ease of Web application deployment with
the developerWorks article "Remotely deploy Web
applications on Apache Geronimo" (May 2006).
- Projects such as nanoHUB demonstrate the extent to which a Web application can be pushed.
- Learn session management in Perl with the CGI::Session
module tutorial.
- Lear session management in Ruby with the CGI::Session documentation.
(Note that Ruby and Perl share the same name but are in different environments.)
- Learn about built-in session management features for PHP.
Get products and technologies
- Download the latest release of Apache Geronimo.
- Download your free copy of
IBM WebSphere® Application Server Community Edition V2.0 -- a lightweight J2EE application
server built on Apache Geronimo open source technology that is designed to help you accelerate
your development and deployment efforts.
Comments (Undergoing maintenance)
Table of contents
Next steps from IBM
Take advantage of open, flexible and New IBM approaches for delivering Web 2.0 applications.
- Try: Creating Web 2.0 Social Network applications is a snap with Rational Application Developer.
- Article: This article Explains how to use Ajax and JSF together in Rational Application Developer.
- Tutorial: Develop a Facebook app using PHP, Java and Rational Application Developer - a full featured Eclipse based IDE.
- Demo: This demo shows how to use Rational Application Developer with the new Java EE 5 Java Persistence API (JPA) framework.
- Buy: Rational Application Developer
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
