What is block storage?
Block storage is a type of cloud storage that divides files and data into equally sized blocks. This storage method allows for fast retrieval of data, since it does not rely on a file system. Think of the difference between looking through a city library's digital catalog to find a book, and knowing exactly where a book is on the shelves. The former is more like file storage; the latter is more like block storage.
Developers often prefer block storage for applications that regularly need to load data from the backend. Block storage is fast and scales up extremely well. It also works well with several types of computing and networking models, including container computing, virtual machines, and storage area networks (SANs).
However, block storage is not without its downsides. File metadata has to be very basic and usually cannot be customized (imagine a library where only the title of a book is recorded). Block storage is also a more costly storage option than some other cloud storage models, like object storage.
How does block storage work?
Data write
When an application that uses block storage writes data to the block storage database, instead of storing it as one file, it divides the data into several sections — the "blocks." These blocks do not have to be stored in any particular order.
Unique identifier
Each block has a unique identifier number that enables the application to find it later.
Data lookup table
These unique identifiers are stored in a data lookup table — a format that allows the application to easily find where each block is when it is needed.
Data read
Whenever data stored in the blocks is requested, the application consults the data lookup table to find where the requested data is stored. Usually, the requested data is spread out over multiple blocks. The application uses the identifiers from the table to retrieve the data, and it merges the disparate blocks back into their original form.
What are the benefits of block storage? What are the downsides?
Benefits include:
Block storage is fast: The use of unique identifiers, rather than searching for data using metadata or a file hierarchy, means that data can be retrieved extremely quickly.
There are multiple paths to data: Block storage allows data to be reached in multiple ways, since the unique identifier is all that is needed for retrieval. By contrast, file storage involves following the file hierarchy path until arriving at the desired file.
Downsides include:
Block storage is expensive: Partially because it is optimized for fast performance, block storage costs more than object storage. Consider how a race car costs more than a large passenger van.
Block storage metadata is limited: Block storage only includes basic file attributes as metadata.
How does block storage compare to object storage?
Object storage keeps files and data in what is called a "data lake," a non-hierarchical collection of unstructured data. Any type of data or file format can go into object storage, which means that unlike block storage, metadata can be complex and customized. Media (such as video and audio), logs, and disaster recovery backups are some of the common uses for object storage, although it is extremely flexible and works with a variety of use cases.
Because object storage is not structured or hierarchical, it can quickly, and almost limitlessly, store vast quantities of data — just as tossing clothes loosely into a big bag is a faster way to pack for a vacation than carefully folding and sorting clothes into a suitcase. However, much like packing in this fashion, object storage can make data retrieval less efficient.
Object storage vs. block storage Object storage Block storage
Storage capacity Almost unlimited Depends on vendor
Data retrieval Sometimes slow Fast
Metadata Customizable Basic, limited
Blob storage is another type of object storage, used for Binary Large Objects (colloquially called "blobs"). It also works best for unstructured data that does not need to be retrieved often.
How does block storage compare to file storage?
Cloud file storage is essentially a traditional hierarchy of files and folders, hosted in the cloud. Folders of data nest within directories and subdirectories, and inside each folder, files are tagged with metadata for easy identification. File storage keeps data organized, but it does not scale up to large amounts of data very well. Going through the hierarchy adds time to data retrieval.
However, file storage may work just fine for some use cases. Individuals who do not need to store and retrieve data on an enterprise-level scale may find that file storage fits their needs.
What is Cloudflare R2?
The cost of data retrieval, also known as data egress, is a major concern for organizations today. To combat these rising costs, Cloudflare offers zero-egress-fee object storage via a service called Cloudflare R2. R2 allows for fast and free data retrieval, and when paired with Cloudflare Workers' distributed code functions, it is endlessly customizable. Cloudflare aims to help developers and organizations avoid vendor lock-in with this service.
FAQs
What is block storage?
Block storage is a cloud storage method that splits data into equally sized blocks. Block storage data retrieval is fast because each block can be accessed directly, rather than searching through a file system.
What are common use cases for block storage?
Block storage is often used for applications that need to load backend data frequently. It works well with certain cloud infrastructure models, including containers, virtual machines, and storage area networks (SANs).
How do unique identifiers work in block storage?
Each block of data gets a unique identifier number. This number lets the system quickly find and retrieve specific blocks when needed.
How does block storage compare to object storage?
While object storage is overall more flexible, scalable, and cheaper, block storage is fast and efficient. Block storage divides data into blocks and assigns each block an identifier, object storage uses a data lake and finds data by searching via metadata. Object storage is essentially unlimited, while block storage capacity depends on the vendor, and expanding it can be expensive.
Why is block storage considered high-performance?
Block storage is fast because it retrieves data using unique identifiers instead of searching through a file hierarchy or using metadata.
What is the purpose of a data lookup table in block storage?
A data lookup table keeps track of where each block is stored using the blocks' identifiers. When data is requested, the system uses this table to find the right blocks.
What are the metadata limitations of block storage?
Block storage only stores basic file information as metadata. There is usually little ability to customize or add extra details about the data. This contrasts with object storage, which typically allows a lot of leeway for customizing metadata.