Editor’s note: This blog posting was authored by Michael Factor, Distinguished Engineer and expert on Storage Systems at IBM Research – Haifa
Traditional file systems store our information in a tree structure. Although this works fine for small collections of data – like those on our local hard drive – they are not designed for the massive volumes of unstructured content most businesses are collecting, storing, and accessing on the cloud.
A new method of storing information is called object storage. This approach stores information as objects. Each object contains the data (the bits and bytes of our documents, movies, images, and so forth), together with metadata that holds user- and system-defined tags. These smart data objects include rich information – or metadata – that describes the content of the data, how the object is related to other objects, how the data should be handled, replicated, or backed up, and more.
“But what if we could turn a software-defined object store into a smart storage platform?”
Although object storage can store objects, manage them, protect them, and so on – it doesn’t by itself dramatically increase the rate at which we can extract value from objects. But what if we could turn a software-defined object store into a smart storage platform?
Imagine if every time you wanted to cook a meal, you had to bring all your ingredients to a central neighborhood depot where stoves, appliances, and cooking utensils were available to “process” the food. That’s similar to the current situation with data on the cloud. Storlets come to remedy this situation by moving the heavy lifting to where it’s needed – similar to allowing you to cook everything in your kitchen, where all your raw materials are already located.
The impact of storlets is substantial. Stored data can be processed locally, and no longer needs to be transferred over the network to a remote computer, processed, and then put back onto the storage server – all of which incurs both network transfer latencies and thus real dollar costs.
Our vision is to reduce costs, increase flexibility and improve security by turning the object store into a platform, and allowing the functionality of the object store to be extended using software.
For example, a media company could upload a movie to an object store and have it automatically generate a representative image. A physician doing rounds could have the object store send only the portion of a patient’s x-ray needed to her wireless device for immediate viewing. A lawyer could request that the object store provide a document from a previous court case in a specific format. Or pathologists could have images analyzed in the object store itself. All this sophisticated computation is moved into the storage infrastructure via the software of dynamically loaded storlets, making it faster, more flexible, and far less expensive.