Two books I have read recently made a surprising connection. The first was the 1972 science fiction novel entitled “Hellstrom’s Hive” by Frank Herbert—the author of the well know Dune series. The premise of the book was that human communities, though extreme in the story, may act collectively in ways that one might call intelligent.
Almost a year later, I found myself reading a book by Steven Johnson titled “Emergence: The connected lives of ants, brains, cities, and software.” I find these two books interesting because they consider human behavior on a macro scale and they address the key point that the collective has its own behavior, tastes, and values.
With the introduction of media, starting with the first news papers, the notion of a feedback loop was introduced to human experience. This established a sort of dialectic between a community as it is and what it might become. Feedback loops also exist in with software. The requirements process needs the feedback loop to ensure features match the market its intended for. Software that fails to have the feedback loop tends to have a short life.
From a Web 2.0 perspective, feedback loops can be used as a tool to solve a larger set of problems that scale beyond the capacity of traditional human organization. Johnson gives several great examples of well-known web sites using feedback to manage everything from fraud to the promotion of solid content. However, all of these examples from the Internet have something in common—they most certainly are using some form of rules technology behind the scenes. Imagine the decentralized behavior of millions of web users on a single site providing, in aggregate, perspective on content, tastes, and enforcement of terms of service. Rules technology provides the right kind of tooling that correlate user behavioral data with policies. In aggregate, it takes the work of the collective and directs it toward something that’s manageable.
There are still companies today that must be organized by traditional means; however, even they can take advantage of emergent behavior. Random remains the most efficient way to load an airplane and so long as the data represents human activity (a loan, a purchase, a claim) rules technology can scale enforcement of policy beyond the performance of most teams. There is still room to mature how best to achieve 100% straight-through-processing in many industries but harnessing the power of the collective remains the only way to solve some business problems at Internet scale.