Roland Barcia is a Senior Technical Staff Member (STSM) and lead Web 2.0 architect for IBM Software Services for WebSphere.

How to kill REST? Add transactions.

Why do we do this to ourselves. As my friend Keys said, we go through endless cycles as an industry solving the same problems again and again. Technology X is too complex, create technology Y cause it is much simpler. Then technology Y lacks features, so lets add it. Now technology Y is too complex, let's create technology Z, and so forth. I have gone through distributed technologies like ONC RPC, COM, CORBA, EJB, Web Services, SCA. I am a fan of REST, because it exploited existing HTTP infrastructure.

So, why the rant, cause I see people doing exactly what destroys simple technologies. Most recent is this link TX support for JAXRS and REST and transactions. Adding transactional support for JAXRS? Why would I want distributed transactions for REST. Really, don't we learn anything in this industry. Why on earth would we want to propagate transactions over REST? We already did that with SOAP? Or why would I want to expose a Tx API over multiple HTTP calls? Leaving Tx's open across multiple HTTP calls has already proven to be a bad practice? REST is stateless, and therefore makes middleware scalable.

A Business process or facade itself can be a resource (/MyBusinessProcess), but the implementation of that service can use regular Tx API's and hide those details.

REST is about "Exploitation." Web Services and SCA is about "Abstraction" Let's put a stake in the ground and use the right technologies for the job rather than destroy something as beautiful as REST.

Situational Environments, Clouds in the Enterprise

I just got back from a long week at Impact. I had a great time. I presented 10 sessions around REST, Web 2.0, and Persistence. At Impact, we announced CloudBurst. Cloudburst is a secure appliance that provides speed and repeatability for deploying WebSphere environments into a private cloud. It comes with loaded with pre-loaded WebSphere Application Server Hypervisor Edition virtual image software and pre-configured topologies. WebSphere Application Server Hypervisor Edition is special edition of WebSphere Application Server that runs on top of a hypervisor, such as VMware ESX, and supports the Open Virtualization Format (OVF).

I have been involved with many SWG development projects. Project delays are too often the norm. I would say that the top delay in most projects is the under estimation of the deployment process. In an ideal world, a strong development process involves a promotion process across several environments, much like is described in this article. However, in a realistic world where the economy is struggling, people are often looking for shortcuts, removing or reducing testing cycles. Often times these shortcuts end up costing people in the end. There are many reasons why people reduce testing.

*

• Cutting out a development cycle to meet a deliverable.
  
• Development teams lack understanding of the amount of work it takes to setup infrastructure.
  
• Administrators lack knowledge of application patterns.
  
• Infrastructure people are often sharing environments between different development projects, and people have to wait in line to use environments.
  
• Not enough hardware to support various projects going live at one time.
  
• Managing hardware between QA, UAT, Stress, Integration, and various other test environments.

All these things add up. Cloudburst in my eyes provides a great opportunity to play a huge role. In Web 2.0, there is the notion of Situational Applications. These are applications that I want to build and deploy quickly, and for a short period of time. Usually, the application has a short shelf life, fulfilling an immediate need and then thrown away.

Cloudburst provides the opportunity to build "Situational Environments." Cloudburst can be used to quickly create Environments for specific needs in the development life cycle, automating tasks that often suck the life out of a project.

Imagine standing up a Quality Assurance Test environment, and when done, throwing it away and using the hardware for perhaps Quality Assurance of another project.

To me, this is an administrator's dream. I could see a new class of Cloud based admin applications asking development teams to deploy into the Admin's Cloud, generating environments to their liking, based on proven models and best practices.

RESTing at Impact !!!

Hi guys and gals, hope all is well. IBM Impact 2009 is on its way. I will be there. Somehow, I managed to end up presenting 7 lectures and 4 labs. I am not sure why I do this to myself. I don't even plan out to do this many sessions. Last year, I took great pleasure in delegating. This year, the delegation just did not happen. Though I do have great co-presenters and lab advocates, so I may be able to escape here and there to see some other great sessions. Even though I am quite busy, I plan to take a REST!!!

I am the track lead for Application Development and Web 2.0. One of the major themes for this track is REST. We have great sessions in the technical track around REST. Here is the list of sessions (Lectures only, will Blog on Labs soon), their speakers, and their time slots:

*

• REST and SOA: Kyle Brown, Roland Barcia, and Greg Truty: Monday 3:45 – 5:00
  
• Introduction to REST: Greg Truty: Monday 2:00–3:15 and Wed 1:30–2:45
  
• Case Study on REST at LexisNexis: Bruce Maxfield from LexisNexis and Roland Barcia: Monday 5:15 – 6:30
  
• Best Practices for Building REST Web Services: Roland Barcia and Greg Truty: Tuesday 10:30 – 11:45
  
• Realizing REST Patterns with WebSphere sMash: Roland Barcia: Tuesday 4:45 – 6:00
  
• SOA, Give it a REST: Jerry Hermes from the Navy Federal Credit Union and Russ Teubner from HostBridge Technology: Web 1:30 – 2:45
  
• Using REST and WS Specifications in the Cloud: Doug Tidwell: Wed 4:45 – 6:00
  
• Secure and Scalable REST Services in Web 2.0 with IBM Mashup Center: John Boezeman and Joel A. Farrell: Thursday 1:30 – 2:45
  
• Accessing IBM WebSphere® eXtreme Scale Data Using REST: Billy Newport, Lan Vuong, and Robert Wisniewsk: Tuesday 3:15 – 4:30

A sMash in the Clouds

In case you haven't heard, IBM and Amazon have partnered to provide you with the ability to use Amazon EC2 to build and run a range of IBM platform technologies.

This relationship will enable you to bring your own IBM licenses to Amazon EC2, utilize IBM’s “Development” AMIs, or leverage the “Production” Amazon EC2 running IBM service. All of these options will enable you to get started quickly using popular IBM platform technologies in the Amazon EC2 environment.

One of the development platforms available is WebSphere sMash. This means that you can run WebSphere sMash applications inside an Amazon Cloud. You can think of a Cloud as "Infrastructure as a Service." Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.

I authored a book back in 2004 called IBM WebSphere: Deployment and Advanced Configuration. The book focused on what we (my co-authors and I) thought is the hardest part of any application development effort: The deployment. I have always been one to love the application development side of the world, but I know that where projects succeed or fail is because a lack of understanding of this process.

I will say that Clouds do not eliminate process, testing, and best practices. But it does reduce the cost, hardware planning phases, and hardware maintenance.

One Best Practice that would seem to apply heavily in a Cloud is RESTful Architecture. I have been on many engagements where scalability problems were due mainly to middleware state. sMash enforces a RESTful style of development, leveraging the browser for some of that Wizard state, and allowing you to write more stateless applications.

To get started with WebSphere sMash and Amazon's EC2, you can start here:

I needed some REST... Practices. 15 or so Practices for building RESTful Web Services.

Sorry for not blogging for a while. I have been quite busy getting some REST.

I have just published 2 new articles:

Implementing and testing server-driven content negotiation for your REST resources with WebSphere sMash

Scaling WebSphere sMash Web 2.0 applications: Part 1: Overview of WebSphere sMash topologies

But I have been spending a lot of time helping some of my customers with REST Architecture. I figured I would post some here. I will give some bullet points and will be publishing papers soon. Note, these are not in any order of importance. All are important. Note, I am just putting pointers in rough format here, and I will publish some more articles on the topics later.

1.) Understand REST. Here are the top 2 misconceptions about REST:

a.) REST is just any XML over HTTP not using SOAP?

NO !!! REST is a Pattern of exchanging Resources. RPC is Not REST

b.) REST is only useful for CRUD(Create, Read,Update, and Delete) semantics.

RESTful Principals Include:

* Identifiable resources (URIs)

http://sports.espn.go.com/oly/summer08/swimming/news/story?id=3530615

* Uniform Interface

GET, PUT, POST, DELETE

* Stateless Communication

Scalable, loose coupling

* Resource Representations

Multiple ways to represent (PDF, HMTL, XML,) – via content types

HTTP has negotiation capabilities (e.g. ACCEPT)

* Hypermedia

Server provides links to resources

Allows for evolution

2.) Define a strategy for defining resource/services:

* What are the resource identifiers?

• Is there a hierarchy of resources?
  
  E.g. a list of employees may be one resource, each employee is another
  
  /employees vs. /employee/empid1
  
• What HTTP Request/Response Headers Supported
  
  Accept, Content-Type, Range, etc….
  
  What is the representation of the data exchanged?
  
  JSON? XML? PDF?

* What are the methods supported at each resource?

Do you support POST, GET, PUT, DELETE?

* What are the status codes returned?

HTTP defines standard return codes

200 = ok

301 = moved permanently

400 = bad request , 401 = unauthorized , 404 = not found

3.) Categorize Resource types:

Example:

• Simple Resource Types (”.../" )
  
• Compound (“/compoung-entity/part”
  
• Collections (“./collection/index/key” )
  
• Algorithmic (No new verbs, only new resources types). See my previous blog entry on Algorithmic Resources.

4.) Maintain Mutability of Verbs

* GET / HEAD (No Side Affects)

• POST/PUT/DELETE (Assumed Side Affects)
  
• Middleware State (No-no, State Transfer)
  
• Atomic Operations – RESTful Resources should start and end DB Transactions.

5.) Implement Optimistic Concurrency

* Optimistic concurrency via E-Tags, Timestamps

or

* Check-in/Check-out semantics if stricter policy is needed, but never hold database locks

6.) Define a Consistent Query Syntax in your organization

REST defines no standards on query parameter names. For example, you may choose to provide standard params for filtering, Sorting, and Ranges.

/resource?filter="name startsWith b && age gt 30"

7.) Implement Filtering, Sorting, and Ranges for Data Sets

Even if you start out small, if your data grows, need to provide these.

8.) Linking over Relationships

Prefer Linking over tightly coupled data. Use ATOM as a payload for establishing relationships

9.) But, sometimes I have relationships....

Sometimes you need to do bulk loading of related data. Some Techniques:

* Special parameter

/Order?loadRelated=LineItems

Very quickly starts becoming RPC

* Headers and Schemas (Better)

Accept: application/eager+atom+xml

Accept: application/atom+xml

10.) Think about API Versioning from the start:

* Follow a Close Spec (slow moving)/Open Extension (Popular Extensions become the next wave of standards) Closed/Open

• How do I represent Versions?
  
  a.) Use URI? /v/3/myResource (better for Bookmarking, expressive)

b.) Use Header (Cleaner looking)

Accept: application/myVer+atom+xml

Custom Header

11.) Lock Down your URI's and Implement Instance Based Security

With REST, I can easily secure URI's, but make sure to implement Instance Based security for User sensitive Data. If I log in successfully as /account/roland, I have to protect against /account/mike.

The service provider should make sure to access the identity from the Subject or Context and compare to the URI requested.

12.) READ THE HTTP SPECS and stick to them

Do not add semantics to headers that are not meant. Routers and firewalls make use of many headers in strange and curious ways.

13.) Implement Content Negotiation

14.) Sometimes you need to Relax.

Partial Updates

Batch Updates

Query params (Be careful, could become RPC Very Fast)

The HTTP Spec is looking at a new verb called PATCH

15.)Documentation As a Service.

Atom can be a great way to document since you can link to the service URI Patterns.

What should REST Documentation have?

Documentation as a Service

/resource/formats

Catalog, Registry

What are my URI Patterns

What Headers do I support?

What Verbs are valid for my resource?

What Response codes do I support?

What do they mean for my service?

What formats do I support? (JSON, and ATOM)

Which header do I use? Accept, query param?

What schema describe it?

16.) Write your Unit Tests first using something like HttpUnit.

I demonstrate this here.

WebSphere sMash 1.1 is available

WebSphere sMash 1.1 is now available. Please go to the Project Zero Website to download the Developer's Edition.

The following Blog post contains an overview on 1.1 contents.

Google's Application Centric Browser, very similar to what WebSphere sMash does on the server.

A Beta Driver of the new Google Browser has been released for Windows (which leaves us MAC users waiting to try it.) There has been a lot of blogs and comments on the new browser so I am not going to discuss a laundry list of features, but focus in on the application centric approach.

I love that each tab is it's own process, and that application isolation can be achieved on the client. One thing I love about Web 2.0 and Ajax is the focus on applications. Having your desktop run different apps rather than one monolithic browser process obviously solves a lot of issues.

By the way, this is exactly the model that WebSphere sMash (Project Zero) has for applications on the server. Every application runs in its own process, and you do not deploy an app to a server, you just run the app, which is the server.

Today, I just released a new article (with Steve Ims as a co-author) on WebSphere sMash (this is actually a rewrite of last years article we did on P0, but with the latest driver and tools). We discuss application centric design of sMash in the article.

The google browser model matches what we have been doing with WebSphere sMash and Project Zero on the server. I can definitely see an architecture where applications running on the desktop call applications in the cloud that service them (or are bound to them as shown in the figure below).

RESTful Facade Pattern - turning verbs to nouns.

Certain entities within an Enterprise make a lot of sense to RESTtify. For example an account. It does not take much of a mental leap to create a RESTful service out of an account.

POST to /Account creates an account.

GET to /Account/233 retrieves details of account.

PUT to /Account/233 updates and account

DELETE to /Account/233 deletes an account.

So then there are actions that do not easily translate. People then drop to some HTTP-RPC thing. An example is when I transfer money from one account to another. A conversation with one of my collegues Brad Bouldin got me thinking about how one might go about RESTfully creating such a service. In an RPC SOA style, I would most likely have something like:

someService.transferFund (Account a, Account b, amount); transferFunds here is a verb that operates on multiple Accounts.

transferFund usually assumes a transaction, so implementing this in the client tier is something that is not recommended.

transferFunds(a,b,amont)

{

startTran();

GET a;

GET b;

a.amount = a.amount – amount;

b.amount = b.amount + amount;

PUT a;

PUT b;

commitTran();

}

This would be ok if this was a programmatic BPEL process that can propagate transactions, but not in a browser or client tier.

So the transferFund method would have to be a service which in itself is hosted on a server platform, usually on the same App Server that is exposing the account entities. So this would involve a local update.

Now, there is no transferFund verb in REST. Only HTTP methods. So you could use something like JSON-RPC, SOAP, or XML-RPC.

Or you can make a mental switch and consider a Transfer as a noun (or Resource itself). Most likely, besides transferFunds(), you have operations like viewTransfers(), getTransferDetail(), etc...

So fundamentally, the way to create this Facade for Transfer in a RESTful fashion is to make Transfer itself a resource.

POST to /Transfer executes a new Transfer.

You may have a payload with

{ accountFrom: ‘/Account/223‘,accountTo:‘/Account/333‘,amount:100.00}

a GET to /Transfer/someGeneratedId would return a history of the Transfer record.

PUT could update a Transfer that has not completed, and a DELETE could concel one that has not completed.

REST actually can force a more complete facade which makes you think of 'Auditing' right up front. Specifically for REST services that are resources that map really to some event in the back end.

This does lead to having you to think about navigation between Transfer and Account and forming relationships you might not have thought of. Like an Account Object may have a link to Transfer.

Usually, there is some fundamental namespace that they are under. For example, Customer. You may really have the following:

GET /Customer/custId/Account returns list of accounts for Customer

GET /Customer/custId/Transfer returns list of Transfers for customer

GET /Customer/custId/Account/accountId/Transfer returns a list of transfers involving this account

But the semantics can get tricky. What if I want the transfer only involving fromAccount.

I could use query parameters:

GET /Customer/custId/Transfer?fromAccount=accountId

Other discussions on this:

http://rest.blueoxen.net/cgi-bin/wiki.pl?VerbsCanAlsoBeNouns

Social Web takes Social skills, so what about private people?

I have not written a blog entry in a couple months. To tell you the truth, I struggle with blogging. I consider myself a private person in many aspects of my life. It is funny that my role at work is forcing me to be more 'social' than I would normally be. I like people, don't get me wrong, but the social web takes social skills.

Anyone who knows me knows I can talk one on one. Once I get started, it is hard to shut me up. But in crowds I tend to like to move to the back and observe.

It is funny, when I go to my parents house with my wife and kids, my wife and parents dominate the conversation. I do try to get a word in edge wise, but I find myself out quickly. Part of that is because my Father and Wife just interrupt me. So I have become used to it. I actually leave the room and play with my kids and my sister's kids like a big kid myself.

I started to think about the reason, and I found that the conversation actually does not interest me. They talk about 'so and so' over here, and that person over there, typically called gossiping, which I do not like to do (and the Bible, since I am a Christian, actually discourages it).

So relating that to the 'social web', one can very quickly find that if you are in the wrong community, you may not have much to say. Now, I have a lot to say about Web 2.0, Middleware, etc... so there are other things.

I look at social websites like Facebook. I have an account, with friends. They all post a lot of pictures, send virtual gifts, and other types of applications. I do not like these things too much. First off, I would much rather receive a real gift. Also, I don't like posting pictures of my family. I seem to think the web has a lot of security problems and the pictures wind up in some sick-o site. The status feature is interesting, except, do I want the world to know what I am doing. Though I enjoy reading what others are doing. I have found the scrabble app as cool, and I play a lot with my wife, so that is fun.

I like to share, I try to teach my kids this all the time. But do I want to share what I am doing, or where I am flying?

I think it comes down to is I like to interact with people face to face, I like being around people and listening to them more than talking, and I like to contribute what I know, but not every single thing I am doing. I also like to work in smaller groups of people.

But what about more 'private' people. Maybe they are read only members of the social web.

This is a rant I know, and I am going to post next on a technical topic. But isn't this what blogging is suppose to be about?

Anyway, it takes 'social' skills to be on the social web. This is something I have to work on I guess.

Dojo Explorer on Dojo Campus

I have found several cool tutorials on Dojo from DojoCampus.org

Specifically, the Dojo Explorer is my favorite Feature. You can navigate through various Dojo Libraries and Widgets and sample it and see the code.

Persistence in the Enterprise

I have just finished 5 straight weeks of conferences starting with IBM Impact and finishing last week at the WebSphere Services Technical Conference which is an internal conference. I am glad it finished. Last week, my book, Persistence in the Enterprise was released.

Besides myself, my coauthors are Geoff Hambrick, Kyle Brown, Robert Peterson, and Kulvir Bhogal. The book focuses on the persistence layer of an application. The first half of the book is primarily focused on things like requirements (persistence focused), design, best practices, and criteria for selecting a persistence layer. The second half of the book focuses on implementing those using 5 different technologies: JDBC, IBatis, Hibernate, OpenJPA, and pureQuery.

WebSphere sMash powered by Project Zero

Today, at IBM Impact, we announced a suite of Mashup Solutions, including WebSphere sMash. IBM WebSphere sMash is a development platform that supports dynamic scripting languages and aggregates data from various sources. It employs REST. The developer edition of sMash remains free, but a commercial platform will be licensed. WebSphere sMash is based on IBM's Project Zero. In addition to WebSphere sMash, we also announced the IBM Mashup Center.

IBM Mashup Center: The mashup center is a group of Web applications that account for IT requirements such as security and governance. Underpinning the Mashup Center is IBM’s Lotus Mashups and InfoSphere MashupHub. IBM has been experimenting with mashups internally. Among the components:

*

• Lotus Mashups allows business types to drag and drop mashup components from Web, personal and enterprise sources to launch custom Web apps.
  
• The InfoSphere MashupHub is “a lightweight environment for unlocking, transforming and mixing enterprise, departmental, Web and personal systems while enforcing enterprise-class security and governance.” In other words, it’s for folks that want to mess with a little or no code.

IBM IMPACT and Web 2.0

Hi everyone. Next week is IBM Impact. I plan to be there, God willing. (I am recovering from a bronchitis infection that had me in the hospital on Sunday). I am excited at all the Web 2.0 content. There is going to be a lot of sessions on Project Zero, the WebSphere Web 2.0 Feature Pack, Ajax, and others. Below are the list of Sessions I will be speaking at or part of:

*

• TSD-2203A Lab: Constructing mashups and integrating services with Project Zero MGM Grand: Studio 8 Tue, 8/Apr, 09:00 AM - 11:45 AM
  
• ICR-1396A SOA foundation deployment: Summary of 92 case studies MGM Grand: Vista Room 210 Wed, 9/Apr, 03:15 PM - 04:30 PM
  
• TSD-1238A Developing Web extended service-oriented architecture applications with Project Zero, Groovy, and REST MGM Grand: Room 119 Wed, 9/Apr, 04:45 PM - 06:00 PM
  
• TSD-2201A Lab: Developing RESTful services with Project Zero and PHP or Groovy MGM Grand: Studio 6 Thu, 10/Apr, 09:00 AM - 11:45 AM
  
• TSD-2203B Lab: Constructing mashups and integrating services with Project Zero MGM Grand: Studio 8 Fri, 11/Apr, 09:00 AM - 11:45 AM

I will also be participating in session: 1519: Showcase: Web 2.0 innovations: Mon, 7/Apr 03:45 PM - 05:00 PM MGM Grand: Room 118

There is also Web 2.0/App Dev Tech Zone. I will be host for the time slot below.

*

• Date: Tuesday, April 8, 2008. 3:15–4:30pm
  
• Topic: Best Practices for Web 2.0 Development
  
• Location: Web 2.0/App Dev Tech Zone, 3rd Floor of Conference Center

I will also be there during Jason and Kyle's 'What is Project Zero and how does it help you?'

Hopefully, my bronchitis goes away and I will be good to go.

You can look over the Web 2.0 track here:

Chat on Dojo Toolkit and WebSphere Web 2.0 FP

Several of my colleagues and I will be hosting a chat. We will be answering questions on new features in Dojo 1.1 as well as questions on the WAS Web 2.0 FP.

Coding Servlets is cool again

With the EJB 3 and Ajax Patterns, I am finding it much easier to code Service Endpoints in Servlets, rather than using frameworks sometimes. Traditionally, the use of MVC abstractions was favored because a lot of view logic was handled in the Container. But for Ajax apps coded in something like Dojo, I find myself just writing RESTful Services for Java EE apps in Servlets. The fact that I can inject Session Beans right into them makes it easy to hook up Ajax endpoints to business logic. Below is an example of a Servlet using the JSON4J API that is part of the WebSphere Web 2.0 Feature Pack.

public class Product extends javax.servlet.http.HttpServlet

@EJB

private ProductSearchService productSearchService;

protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {

response.setHeader(“Content-Type“, “application/json” );

if(request.getPathInfo() != null)

{

try

{

int productId = Integer.parseInt(request.getPathInfo().substring(1));

org.pwte.example.domain.Product product = productSearchService.loadProduct(productId);

JSONObject productJson = ProductToJsonHelper.serializeProductToJson(product);

productJson.serialize(response.getWriter()) ;

}

catch (ProductDoesNotExistException e)

{

response.sendError(response.SC_NOT_FOUND) ;

}

catch (Exception e)

{

response.sendError(response.SC_BAD_REQUEST) ;

}

}

else if (request.getParameter(“category” ) != null)

{

try

{

int categoryId = Integer.parseInt(request.getParameter(“category” )) ;

List products = productSearchService.loadProductsByCategory(categoryId);

JSONArray productArray = new JSONArray();

for (org.pwte.example.domain.Product product: products)

{

JSONObject jsonProduct = ProductToJsonHelper.serializeProductToJson(product);

productArray.add(jsonProduct);

}

productArray.serialize(response.getWriter()) ;

}

catch (Exception e)

{

response.sendError(response.SC_BAD_REQUEST) ;

}

}

else response.sendError(response.SC_BAD_REQUEST) ;

}

}

Of course, writing logic to move between POJO's and JSON is not as fun. Forwarding to a JSP template to do this could be easier. Or using a higher level API like that in the RPC package may be preferred as well.