Organizing unstructured data in a flat environment

Object storage is a hierarchy-free method of storing data, typically used in the cloud. Unlike other data storage methods, object-based storage does not use a directory tree. Discrete units of data (objects) exist at the same level in a storage pool. Each object has a unique, identifying name that an application uses to retrieve it. Additionally, each object may have metadata that is retrieved with it.

Key features

  • Data are stored as discrete objects.
  • Data are not placed in a hierarchy of directories and live in a flat address space.
  • Applications identify discrete data objects by their unique address.
  • Often compared to valet parking. A data object is like a car; the address is the receipt.
  • Designed for access at the application level using an API rather than at the user level.

Why object storage

Icon representing the scalability of object storage

Scalability

Object storage can accommodate virtually any quantity of data without requiring partitioning of the data set.

Icon representing the efficiency of object storage

Efficiency

A lack of hierarchy means there are no bottlenecks created by complex directory systems.

Icon representing the availability of data within object storage

Availability

Object storage systems have mechanisms to ensure data consistency, enabling automatic data replication, rolling updates, and no downtime.

Considerations for object storage

  • Limited features

    Object APIs for accessing data are often fairly simple. Applications must implement more complex data management requirements.

  • Compatibility

    File system tools such as POSIX utilities cannot interact with object-based storage systems without additional layers in between.

  • Types of data

    Object storage is ideal for unstructured data such as media and web content. It is not suitable for regularly modified data.

  • API support

    Applications must be updated to take advantage of object storage. Many vendors are releasing revisions with built-in support.

Discover hybrid cloud object storage with flexibility designed for today's demanding data needs.

 

An IBM perspective: A fresh approach to object storage

As employees create more content, enterprise IT organizations are experiencing exponential growth in data storage requirements. Multimedia asset creation is a key driver.

Demand for object storage systems will grow as a result of the need to archive more unstructured data. This growth will fuel the deployment of new, cost-effective distributed storage solutions that can scale to support hundreds of petabytes.

Market requirements

A decade ago, we anticipated global levels of data to grow beyond petabyte level for most organizations, even exabyte scale for some. Fast-forward to today – a world in which the cloud has fostered an era of explosive innovation and the Internet of Things (IoT) has empowered millions of devices to create, collect, and send data every second. That prediction has become a reality.

To manage the unprecedented volume of data generated, businesses must determine how to efficiently preserve, protect, analyze, and maximize the value of their unstructured data. Object storage is designed to do so at web-scale.

Just as the Internet is designed today, Information Dispersal technology pioneered is peer-to-peer, with nothing centralized. Each object within an object storage system is unique and identified by an object ID, which ultimately determines how that object is found.

Within a given storage system, there are a number of physical nodes that operate independently. At any point, more nodes can be added to the system. This structure enables enterprises to scale capacity and performance independently.

As part of this process, data written to the object storage system by an application are transferred through an access layer, where it is encrypted, sliced, and distributed.

A unique object ID is used to retrieve the object through the access layer by locating a threshold number of the slices stored and reconstituting the object.

Storage technology

By referencing objects with IDs rather than file names, the system can scale. This approach is not constrained by size limitations and retrieving data is easier, as extensive metadata can be associated to a given object.

An object storage system ensures IT can leverage its existing investments. It also positions the organization to capitalize on future opportunities regardless of whether data are managed on-premises, in the cloud, or both.

Market outlook

As organizations reexamine their data storage strategy to contend with rapid growth in data creation and consumption, object storage brings businesses a secure, adaptable, and cost-effective solution for managing data.

The bigger the pool of data gets, the more dramatic the increase in the long-term cost of ownership. Information is accessible at all times, and a simplified management platform streamlines maintenance and operational overhead.

As the cloud, IoT and mobile continue to power a data-rich world, an increasing number of organizations will find that investing in their tomorrow begins today. Object storage enables organizations with web-scale data needs to implement a scalable, forward-looking storage solution that positions their business for growth over the next 20 years and beyond.

Learn about IBM Cloud object storage services