As I write this, summer is drawing to a close, the school year is starting up, and Twitter's servers are continuously pumping out updates from geeks and non-geeks around the world. For many of us in North America, thoughts are turning from beach parties to football, from outdoor pursuits to projects we can do inside. In keeping with this, I think it's a good time to revisit Scitter, the Scala client library for accessing Twitter.
If you've followed Scitter's development so
far, you know that the library is able to read from the various Twitter APIs in order
to look at a user's friends, followers, and timeline, among other things. What's
missing is the ability to post status updates. In this final article about Scitter,
we'll round out the library's functionality with some fun additions (endings and
ratings) along with the essential methods update(), show(), and destroy(). Along the way,
you'll learn more about the Twitter API and how well it interacts with Scala, and you'll also get some ideas for overcoming the inevitable programming challenges between the two.
Note that by the time this article reaches your eyes, the Scitter library will reside inside of a public source-control repository. I'll also include source code with this article, of course, but be aware that changes could be made to the source base. In other words, the code in the project repository may be subtly or drastically different from what you find here.
So far in the development of Scitter, we've focused on doing HTTP
GET-powered operations, mainly because these calls are fairly easy, and I wanted to ease into the Twitter API. Adding POST and DELETE operations to the library will also be a big step in terms of visibility. Up until now, it was possible to run unit tests against your personal Twitter account without anybody knowing what you were up to. But once you start sending update messages, all the world will know that you're running Scitter unit tests.
If you continue to test Scitter, you'll want to create your own "test" account on Twitter. (The lack of any reasonable testing or mocking facility for Twitter is probably the biggest drawback to writing against the Twitter API.)
Before we launch into the library's new UPDATE
functionality, let's review what we've created thus far. (I won't include the
full source listing because Scitter is starting to get longer than is
reasonable to display inline. Instead, have a look at the code in another window while reading.)
Roughly speaking, the Scitter library is broken into four parts:
- The request and response types that are sent back and forth as part of the API (
User,Status, and so on); these are modeled as case classes. - The
OptionalParamtypes that are also part of the API in certain places; also modeled as case classes inheriting from a baseOptionalParamtype. - The
Scitterobject used for communication fundamentals and for anonymous (non-authenticated) access to Twitter. - The
Scitterclass, which holds a username and password for authenticated access to a given Twitter account.
Note that between the last article and this one, in order to keep the file sizes relatively manageable, I've broken the request/response types out into their own file.
So now we're clear on our goals. We'll start working toward them by
implementing two Twitter APIs that are "read-only": the end_session API (which closes down a user's session) and
the rate_limit_status API (which describes how many
posts a user account has left available for a particular period of time).
The end_session API is, like its cousin verify_credentials, a pretty simple API: simply call it with an authenticated request, and it will "close down" the working session. Implementing it on the Scitter class is relatively easy, as shown in Listing 1:
Listing 1. Implementing end_session on Scitter
package com.tedneward.scitter
{
import org.apache.commons.httpclient._, auth._, methods._, params._
import scala.xml._
// ...
class Scitter
{
/**
*
*/
def endSession : Boolean =
{
val (statusCode, statusBody) =
Scitter.execute("http://twitter.com/account/end_session.xml",
username, password)
statusCode == 200
}
}
}
|
Well, okay: I lied. It's not quite so easy.
Unlike the other APIs we've worked with thus far in the Twitter API, end_session wants the incoming message to be sent with HTTP POST semantics. Right now, the Scitter.execute method does everything via GET, which means it has to somehow differentiate between those APIs that want a GET and those that want a POST.
Leaving that aside for just a moment, there's another obvious change: POSTed API calls also need to pass name/value pairs in to the execute() method. (Remember, in other API calls, using GET, all the parameters can and do appear as query parameters on the URL line; using POST, they appear in the body of the HTTP request.) In Scala, any time name/value pairs are mentioned, the Scala Map type leaps to mind, so the easiest way to think about modeling the data elements sent as part of a POST is to put them into a Map[String,String] and pass that.
For example, if we were passing in a new status message to Twitter, which requires the
140-characters-or-less message to be in a name/value pair called status, it would look like Listing 2:
Listing 2. Basic map syntax
val map = Map("status" -> message)
|
Given that, we can restructure the Scitter.execute() method
to take a Map as a parameter. If the Map is empty, we can then assume that we should be using a GET instead of a POST, as shown in Listing 3:
Listing 3. Refactoring execute()
private[scitter] def execute(url : String) : (Int, String) =
execute(url, Map(), "", "")
private[scitter] def execute(url : String, username : String,
password : String) : (Int, String) =
execute(url, Map(), username, password)
private[scitter] def execute(url : String,
dataMap : Map[String,String]) : (Int, String) =
execute(url, dataMap, "", "")
private[scitter] def execute(url : String, dataMap : Map[String,String],
username : String, password : String) =
{
val client = new HttpClient()
val method =
if (dataMap.size == 0)
{
new GetMethod(url)
}
else
{
var m = new PostMethod(url)
val array = new Array[NameValuePair](dataMap.size)
var pos = 0
dataMap.elements.foreach { (pr) =>
pr match {
case (k, v) => array(pos) = new NameValuePair(k, v)
}
pos += 1
}
m.setRequestBody(array)
m
}
method.getParams().setParameter(HttpMethodParams.RETRY_HANDLER,
new DefaultHttpMethodRetryHandler(3, false))
if ((username != "") && (password != ""))
{
client.getParams().setAuthenticationPreemptive(true)
client.getState().setCredentials(
new AuthScope("twitter.com", 80, AuthScope.ANY_REALM),
new UsernamePasswordCredentials(username, password))
}
client.executeMethod(method)
(method.getStatusLine().getStatusCode(), method.getResponseBodyAsString())
}
|
The biggest change to the execute() method is the introduction of the Map[String,String] parameter, along with the "if" test based on its size. The test determines whether we're dealing with a GET or POST request. Because Apache Commons HttpClient wants the POST request body to be done in NameValuePairs, we iterate through the elements of the map using the foreach() call. We pass in the map key and value as a two-part tuple pr, and extract them into the local bound variables k and v, respectively, using those values as the constructor parameters to the NameValuePair constructor.
We could do all of this more easily by using the setParameter(name, value) API on PostMethod. I've chosen the approach in Listing 3 for pedagogical purposes: to show off the fact that Scala arrays, even though the array reference is marked as a val, are still mutable in the way that Java arrays are. Just keep in mind that it would be far better, in working code, to use the setParameter(name, value) method on PostMethod, for each (k,v) tuple.
Something else to note is that the Scala compiler doing its type inference does the right thing for the type of the "method" object returned by the if/else. Because Scala can see that if/else returns either a GetMethod or a PostMethod object, it chooses the immediate base type, HttpMethodBase, as the returned type for "method." This also means that any methods not available on HttpMethodBase are not accessible in the remainder of the execute() method. Fortunately, we don't need them, so it's unlikely that this will be a problem, at least for now.
One final issue lurks behind the implementation in Listing 3, having to do with the
fact that I've chosen to use the Map to discriminate
(whether the execute() method) is dealing with a GET or POST operation. If we ever need to use some other HTTP verb (such as PUT or DELETE), we'll have to refactor execute() once again. So far, this hasn't been a concern, but we'll want to keep it in mind as we go forward.
Before we implement this refactoring, let's run ant test to
make sure that all the original GET-based request APIs
still work — and they do. (This assumes no changes to the production Twitter
API or the availability of the Twitter servers.) Everything works (or at least it
did on my box), so implementing the new execute() method is pretty trivial:
Listing 4. Scitter v0.3: endSession
def endSession : Boolean =
{
val (statusCode, statusBody) =
Scitter.execute("http://twitter.com/account/end_session.xml",
Map("" -> ""), username, password)
statusCode == 200
}
|
Can't get much simpler than that.
The next thing we want to do is implement the rate_limit_status API, which has both an authenticated and a non-authenticated version. We'll implement the method as rateLimitStatus on both the Scitter object and the Scitter class, as shown in Listing 5:
Listing 5. Scitter v0.3: rateLimitStatus
package com.tedneward.scitter
{
object Scitter
{
// ...
def rateLimitStatus : Option[RateLimits] =
{
val url = "http://twitter.com/account/rate_limit_status.xml"
val (statusCode, statusBody) =
Scitter.execute(url)
if (statusCode == 200)
{
Some(RateLimits.fromXml(XML.loadString(statusBody)))
}
else
{
None
}
}
}
class Scitter
{
// ...
def rateLimitStatus : Option[RateLimits] =
{
val url = "http://twitter.com/account/rate_limit_status.xml"
val (statusCode, statusBody) =
Scitter.execute(url, username, password)
if (statusCode == 200)
{
Some(RateLimits.fromXml(XML.loadString(statusBody)))
}
else
{
None
}
}
}
}
|
Also pretty straightforward, I think.
Now, armed with the new POSTable version of the HTTP communication layer, we can attack what is arguably the centerpoint of the Twitter API, the update call. Not surprisingly, it requires a POST with at least one parameter, named status.
The status parameter contains the not-longer-than-140-character message to be posted to the authenticated user's Twitter feed. There's also one optional parameter: in_reply_to_status_id, which gives the id of another update that the POSTed update is replying to.
That's pretty much it for the update call, as shown in
Listing 6:
Listing 6. Scitter v0.3: update
package com.tedneward.scitter
{
class Scitter
{
// ...
def update(message : String, options : OptionalParam*) : Option[Status] =
{
def optionsToMap(options : List[OptionalParam]) : Map[String, String]=
{
options match
{
case hd :: tl =>
hd match {
case InReplyToStatusId(id) =>
Map("in_reply_to_status_id" -> id.toString) ++ optionsToMap(tl)
case _ =>
optionsToMap(tl)
}
case List() => Map()
}
}
val paramsMap = Map("status" -> message) ++ optionsToMap(options.toList)
val (statusCode, body) =
Scitter.execute("http://twitter.com/statuses/update.xml",
paramsMap, username, password)
if (statusCode == 200)
{
Some(Status.fromXml(XML.loadString(body)))
}
else
{
None
}
}
}
}
|
Probably the most "different" part of this method is the nested function defined inside of it — unlike the other Twitter API calls that use GET, Twitter expects the parameters to POST to appear inside the POSTed body, meaning we have to turn them into Map entries before calling Scitter.execute(). However, the default Map (from scala.collections.immutable) is an immutable creature, meaning we can combine Maps, but we can't add an entry into an existing Map. (Actually, we could but we'd prefer not to. See the "Mutable collections" sidebar for more about that.)
The easiest way to approach this little conundrum is to recursively attack the list (which is actually an Array[]) of OptionalParam elements passed in. We'll take each one apart and turn it into its own Map entry. Then we'll return a new Map (made up of the newly created Map and the Map returned from the recursive call) to optionsToMap.
After that, it's pretty trivial to pass the Array[] of OptionalParam to the optionsToMap nested function. We'll concatenate the returned Map to the one that we build containing the status message. Finally, we'll pass the new Map to the Scitter.execute() method, along with a username and password, for transmission to the Twitter server.
Incidentally, all of this takes longer to explain than to do in code, which indicates some fairly elegant programming.
What's happening with the optional parameters passed in to update is pretty identical, in theory, to the optional parameters passed in as part of the other, GET-based API calls; it's only the result format (name/value pairs suitable for POSTing as opposed to name/value pairs meant for a URL) that differs.
If it turns out that the Twitter API requires other HTTP verb support (PUT and/or DELETE being the two big ones that could be required), we could always make the HTTP parameter a specific parameter — perhaps another set of case classes — and have execute() take an HTTP verb, the URL, the map of name/value pairs, and (optionally) the username/password as five parameters. We could then turn the optional parameters into either a single string or a set of POSTed parameters as necessary. Just something to keep in mind for the future.
The show call displays a single Twitter status when given
the id of the Twitter status to retrieve. Like update, it's
pretty self-explanatory, shown in Listing 7:
Listing 7. Scitter v0.3: show
package com.tedneward.scitter
{
class Scitter
{
// ...
def show(id : Long) : Option[Status] =
{
val (statusCode, body) =
Scitter.execute("http://twitter.com/statuses/show/" + id + ".xml",
username, password)
if (statusCode == 200)
{
Some(Status.fromXml(XML.loadString(body)))
}
else
{
None
}
}
}
}
|
Any questions?
For those who want to experiment with pattern matching just a
bit more, Listing 8 is another way to write the show() method:
Listing 8. Scitter v0.3: show redux
package com.tedneward.scitter
{
class Scitter
{
// ...
def show(id : Long) : Option[Status] =
{
Scitter.execute("http://twitter.com/statuses/show/" + id + ".xml",
username, password) match
{
case (200, body) =>
Some(Status.fromXml(XML.loadString(body)))
case (_, _) =>
None
}
}
}
}
|
Whether this is more clear than the if/else version is more a matter of aesthetics than anything else, but it is fair to say that it is probably more terse. (Chances are, the more the code reviewer sees the "functional" part of Scala, the more attractive this version is.)
The pattern-matched version does have one advantage over its if/else cousin, though: if new conditions (such as different error conditions or response codes from HTTP) end up being returned from Twitter, then the pattern-matched version will likely be more clear in its distinction between them. For example, if Twitter decides one day to return 400 response codes and an error message in the body itself to indicate some sort of formatting error (perhaps you didn't re-Tweet correctly), then the pattern-matched version can test both the response code and the contents of the body more easily (and clearly) than the if/else approach.
Also note that we could potentially use the form in Listing 8 to create some partially applied functions, requiring only URL and parameters. But, frankly, that would be a solution looking for a problem, so I won't go down that path.
We'll probably want to give Scitter users the ability to undo an action that has just
been done. For this, we need a destroy call, which will
(what else?) delete a posted Twitter status, as shown in Listing 9:
Listing 9. Scitter v0.3: destroy
package com.tedneward.scitter
{
class Scitter
{
// ...
def destroy(id : Long) : Option[Status] =
{
val paramsMap = Map("id" -> id.toString())
val (statusCode, body) =
Scitter.execute("http://twitter.com/statuses/destroy/" + id.toString() + ".xml",
paramsMap, username, password)
if (statusCode == 200)
{
Some(Status.fromXml(XML.loadString(body)))
}
else
{
None
}
}
def destroy(id : Id) : Option[Status] =
destroy(id.id.toLong)
}
}
|
And with that, I think we can consider the Scitter client library to be "alpha," at least to implement a simple Scitter client. (Which, in the grandest tradition of authorial spirit, I leave up to you, as an "exercise for the reader.")
Writing up the Scitter client library has been an interesting exercise. While it can't be said to be completely production ready, Scitter is certainly good enough to be used to implement a simple text-based Twitter client, which means it's ready to be cast loose. Releasing code into the wild is the best way to find out what people can do with it, and which features are needed in order to make it more useful.
I've committed the code from this and previous articles about Scitter as a first revision to the Scitter project home page, hosted by Google Code. Please feel free to download the library, play around with it, and let me know your thoughts. Bug reports, fixes, and suggestions are also welcome.
You needn't feel constrained to stick with my codebase, either. Following the development of Scitter over the past three articles should have left you with a good feel for how the Twitter API works. If you also have ideas about how to approach the API differently, then do it: toss Scitter out the door and build your own Scala client library. That's part of the fun of doing these little at-home projects, after all.
For now, we'll wave farewell to Scitter and start the happy hunt for a new project to solve with Scala. Have fun with that, and let me know if you find work programming in Scala!
| Description | Name | Size | Download method |
|---|---|---|---|
| Source code for this article | j-scala10209.zip | 812KB | HTTP |
Information about download methods
Learn
-
The busy Java
developer's guide to Scala" (Ted Neward, developerWorks): Read the complete series, including the two previous articles about Scitter.
- "The
busy Java developer's guide to Scala: Don't get thrown for a loop!" (Ted
Neward, developerWorks, March 2008): Further explains the underlying theme in
Scala that developers do not need mutable state nearly as often as we think we
do.
- "The
busy Java developer's guide to Scala: Collection types" (Ted
Neward, developerWorks, June 2008): Learn more about tuples, arrays, and lists in Scala.
- "Twitter API wiki: Want to do more with Twitter? Get started right here.
- "Scala by Example" (Martin Odersky, December 2007): A short, code-driven introduction to Scala (in PDF).
- Programming in Scala (Martin Odersky, Lex Spoon, and Bill Venners; Artima, December 2007): The first book-length introduction to Scala.
- The developerWorks Java technology zone: Hundreds of articles about every aspect of Java programming.
Get products and technologies
- Download the Scitter library, now on Google Code.
-
The Jakarta (Apache) Commons HttpClient component provides an efficient, up-to-date, feature-rich package implementing the client side of the most recent HTTP standards and recommendations. You may also want the Commons Logging and Commons Codec components.
- Download Scala: Learn it with this series.
- SUnit: Part of the standard Scala distribution, in the scala.testing package.
Discuss
- Get involved in the My developerWorks community.
Comments (Undergoing maintenance)
Table of contents
Next steps from IBM
Spring, JRuby, and Ajax development is easier with WebSphere Application Server - a smart Java 5 and J2EE Web services-based application server.
- Try: The no-charge WebSphere Application Server Community Edition is a pre-integrated, lightweight Java 5 application server built on Apache Tomcat and other best-of-breed open source software such as OpenEJB, Apache Axis, and Apache Derby.
- Article: The article leverage the Spring Framework and the WebSphere Application Server to improve your J2EE project productivity.
- Tutorial: See how the free WebSphere Application Server and XML can improve the efficiency of your JRuby on Rails and Ajax development.
- Buy: WebSphere Application Server - Express
My developerWorks community
Tags
Use the slider bar to see more or fewer tags.
Popular tags shows the top tags for this particular content zone (for example, Java technology, Linux, WebSphere).
My tags shows your tags for this particular content zone (for example, Java technology, Linux, WebSphere).
Popular article tags |
My article tagsSkip to tags list
Popular article tags |
My article tags
