Kelvin Lawrence laments that systems programming seems to be a dying art. In the general computing case, he's probably right, and it's probably a good thing for the general case. Most programmers are probably best off sticking to a language which will take care of releasing unused resources, like memory, for you. Languages such as standard scripting languages or lisp.
Once place where it's alive and well is in the appliance area, which is a niche within the general embedded area.As I've mentioned before, we write "close to the iron" by using C/C++ instead of a virtual machine like Java. Since we have no disks, we have to be very careful about getting the most use out of all the physical memory we have, which means we're not only concerned about leaks and resultant out of memory conditions, but fragmentation as well.
An appliance should be a very long-running piece of hardware, so leaking as little as 4K per hour will quickly become noticeable--it's not like we can restart (like the Apache child processes can be configured) as a casual manner.
I hope to provide a real-life example of this (some useful open source C++ code) soon.
I really like Italian Ice. Those little cups you buy in the store with the wooden spoon that you scrape away at the top, getting a kind of "spoonful of slush." On a hot and muggy summer day, I still find them almost as refreshing as a cold beer. Especially if you find a tangy citrus flavor that is not too sweet; too much sugar makes you too thirsty.
So, it seemed like a good idea when our local supermarket started carrying the five-quart pails, and my wife brought one home. After the first day, I felt like Scrat trying to save and eat his acorns. Maybe if I had a gourmet ice cream "trowel" I could scrape off a serving's worth of slush, but all I ever managed to do was chop off mini-icebergs from the pail. Once you put those in your bowl, they just slide away from the spoon as you try to scrape them. I ended up eating it like finger food. Yuk.
From now on, we stick to the cups.
The Nickelodean show Avatar ended a few days ago. I hadn't watched it much in the past year, so some of the wrap-up seemed a bit sudden. I never understood how only one part of the world could have electricity. I found a great Avatar wiki, so maybe I'll read up on all I missed.
The show was really well done; it was much more than a kiddie cartoon. I'm looking forward to the movies.[Read More]
My colleague and pal Andrew Spyker just blogged that TPC (the Transaction Processing Council
, not the phone company
) is looking at a SOA benchmarking submission by IBM. His blog entry
has more details. TPC benchmarks are an industry standard, and it will be good to have this for the SOA industry. I don't know for sure, but if my guesses about the submission are correct, it will be up to the usual high quality of the other real-world TPC benchmarks.
SOA appliances are about more than just performance, and I know we've given Andrew and his group some fits over the past couple of years. Consumability, fitness for purpose, the whole concept of SOA processing in the network--those things are at least as important as raw processing speed. We had to work hard to make sure people bought into the concept and not get pigeonholed as an "offload accelerator." I think we're pretty much past that now; certainly by the time the TPC is ready. And then, once we have objective third-party benchmarks, I look forward to our participation.
There will still be challenges, of course. For example, how do we get apples and apples comparison? Unlike some of our former competitors
, we don't divulge what's inside the box. An appliance is an integrated solution, so we'll have to be careful about the various hardware and software combinations that others use. The only valid comparison seems to be by price--how will our appliances compare in a new TPC-SOA benchmark in terms of value for total solution cost? That one's easy to answer: we'll kick butt.[Read More
One of IBM's core values is "dedication to every client's success," and the company does various things to make sure that all IBMers take these values seriously. It can also show up in some non-obvious ways. For example, if you read through part of our Software Support Handbook you see that the customer specifies the severity of a bug, and further that they can change it at any time if the business circumstances change.
I recently heard a great story on this. It was a Memorial Day holiday weekend in the US, and a customer was calling with a severity one problem because sometimes the menu in a webpage pulldown was translucent. At first it was hard to see how an unattractive GUI issue was equivalent to a "production down" situtation, but then the customer explained that they sell medical items over the web and many of their customers are older with vision problems. That's all it took--a "minor" UI issue became a "sev 1" and some developer was pulled away from their picnic to go fix the menu.
Severity one is intense -- and keep in mind, the customer makes that classification. This sentence impresses the hell out of me:
If a client designated a problem as a Severity 1, IBM will work on it 7 days a week, 24 hours a day, providing the client is also available to work during those hours.
and is probably why so much of the world's IT infrastructure runs on IBM products. It also scares me.
When a customer buys an appliance, you're selling them a "magic box" that meets their needs and if something goes wrong, we have to fix it. It doesn't matter if it's a hardware bug, a kernel bug, or some arbitrary piece of open source software that you're using--there's no finger-pointing to someone else, we have to fix it. There are times, at 3am, when it would be really nice to say something like "install the latest RedHat and see if that fixes it" but you can't.
It's almost enough to make me feel sorry for our support team. Almost. :)
Thanks to Dave Orchard, I'm now on dopplr. I don't travel much--and most of them are one-day customer visits along the US Boston/Washington corridor. It's probably more useful for people coming to Boston who might want to look me up. Boy, that sounds egotistical, doesn't it?
We're still hiring. IBM revamped (outsourced, really) its job offerings website so the link I mentioned earlier is broken. Here's a link to the search page: https://jobs3.netmedia1.com/cp/search.jsp. Enter "datapower" in the keyword list. Don't be put off by the entries that say "world wide sales"--those aren't pre-sales roles, but really developer jobs.
Send me email if you have questions. Send me your resume me interested. :)[Read More]
How should you be able to configure, monitor, and manage an appliance?
- Command line; via SSH, or Telnet, or either/both?
- Web-based interface?
- Thick client?
- SOAP-based? If so, task-based or config-object-based? WSDM or WS-Man or DMTF or OVF?
How do you send out notifications of things happening?
- SNMP traps?
- File-based logging?
- SMTP alerts?
- Syslog? Syslog-ng?
- SOAP? If so, WS-Notification, WSDM or WS-MAN or DMTF, etc?
You probably need all of those. But how do you keep them in sync? Could you build a framework that makes it possible? Could you keep that framework generic so that it is independant of, or at least loosely-coupled to, the task the appliance is doing? And if you do all that, is the end result still an appliance?[Read More]
Another aspect of an appliance is that it must present a unified configuration interface. This is the biggest user-visible difference between a real appliance and a virtual, or software, appliance. It's a dissservice to the customer if they have to create administrators by using the
command, even if dressed up with a fancy Tk or browser-based interface; or if they have to edit the key-value format of an NTP configuration, or shell environment variable syntax of DHCP configuration, ... and so on.
Pictorially, it might look something like this, where each box is a subsystem or facility and the shaded boxes within it represent configuration information:
What you really want is something like this:
You want this because it should be simpler: all the gory details and flexibility of some things (what's an NTP "drift file"?) can be removed. You also want it because it shows that your "appliance" vendor is thinking about things holistically, and hopefully building a unified system that, e.g., merges the NTP "logfile" into whatever logging facilities the appliance supports.[Read More
We had an internal IBM conference on IT appliances that just ended. There was lots of dicussion about software appliances, virtual appliances, real appliances, etc. I understand the appeal of virtualization--making the most of unused CPU's makes a lot of sense--but a "virtual appliance" is an oxymoron.
(BTW, the word itself is an oxymoron; I didn't know that until looking for an online definition.)
An appliance is a combination of purpose-built hardware and software, and the pair should be inseparable. For example, the "business logic" of the appliance should know about, and properly leverage, the number of cores (or hyperthreads) in the CPU's. If running on a conventional operating system, it should be in an embedded context so it can almost completely account for every single byte of physical memory. Virtualization systems like Xen or VMWare just get in the way, at best, or make it impossible to know what's really going on at worst. Java is the worst-case here: the words "virtual machine" in JVM is the give-away. For a real appliance, the mantra should be _write-once run-once._ The Java mantra _write-once, run anywhere_ is just plain wrong.
For the DataPower appliances, the way we do this is through our compiler. We understand many XML-based languages (including XSD, WSDL, XSLT, and XACML) and we generate object code that is highly tuned for the processors and runtime in the box. We know how to multiplex thousands of connections over a finite number of cores, how to provide quality of service without spending too much time looking for higher-priority work, how to use our XML accelerator hardware, and so on. We encapsulate all of that in the compiler, and we make sure that all data manipulation work goes through the compiler. If we add more accelerators or change processors, then it's just a simple matter of changing the code generator.
A "simple matter of changing the code generator." That's another oxymoron. :)[Read More
... should be very interesting.
I'll be at the IMPACT
conference all next week. This is a major conference, the kind that has Drew Carey, the B-52's, car giveaways, etc. Of course that's only the "bling" -- the more important things are the chance to meet some of the more than 6,000 customers and business partners, hundreds of speakers, and so on. I've been big conferences before (RSA, JavaOne, and so on) including speaker and booth duty, but this is the first time I've seen it from the inside.
I'm a tiny cog in the big blue machine, but the little I've seen is fascinating:
- Getting a presentation accepted wasn't any easier just because I work for the sponsor :)
- The internal briefing book is the size of a small paperback, perfect for in-flight reading or the three-hour layover in Reagan airport.
- There's a complicated reservations system for having meetings with custmers and executives.
- There's all sorts of events within the conference that have various levels of exclusivity. Somehow I managed to end up leading one of them.
- The amount of care and detail that everyone takes over their presentations is really impressive.
If you're coming out next week (and want to talk about DataPower, or anything), get in touch (posting a comment will work).[Read More
Like many in the SOA/Web Services field, I'd periodically get "zinged" by someone mentioning Pete Lacey's blog entry, The S stands for Simple
. He wrote it almost a year and a half ago, but it was the current contretemps over office document formats that clued me in on how to respond.
The key thing to remember is that the changes Pete desribes took place over many years, and were (mostly) done in an open manner. Yes, the industry didn't get everything right the first time -- nobody does. For example, the HTTP/1.0 RFC
was published in 1996; and the HTTP/1.1 RFC
was published eight years later. Compare the differences in caching, an incredibly fundamental concept, between the two.
There's no doubt we've had a lot of mis-steps along the way, and folks who started working on "day one" have had to rip things up and start over. My own SOAP framework, ZSI
, was completely oriented toward RPC encoding, and while I still hold a little frustration, hey -- things change and we move on. Sometimes we make mistakes and sometimes they can be public embarassments. The alternative, tho, is a solution developed in closed offices on stone tablets, and I don't think anyone wants that.
Almost two and a half years ago, IBM purchased the startup I was with. The company was DataPower, and we made XML network appliances. We still do, only now they're WebSphere DataPower SOA Appliances, and we're more succesful than ever. I'll write about that later. Right now, I want to write about IBM.
It takes a fair amount of adjusting to go from a decade of startups to a company with more than 350K employees. The mailing list of "people involved with XML standards" has more members than the entire payroll of my last couple of companies! You can easily get swamped in Sametime instant messaging, bookmarking, wiki's, blogs, mashups (er sorry, I meant situational applications:), tagging, Notes teamrooms, online Learning@IBM, and so on. If there's anything I get from that, it's that IBM is really big into collaboration and innovation within developement. Like the title says, this place is cool. Of course, that's a misnomer, since "this place" is really several dozen spread across the globe, and all those technologies are just ways to get a globally integrated development effort. It works.
By the way, the DataPower team has some openings, so if you want to work on some of the coolest products at a cool place, look at the jobs pages at the IBM website
(this direct link seems to work for the US: http://careers.peopleclick.com/Client40_GLDTR/bu1/External_Pages/Careers.htm
) and search for DataPower. Or drop me a line.