Object and blob storage

Think of it like this

Analogy

Object storage is like a coat check with infinite space. You hand over a coat and get a ticket number. Later you give the ticket and get the coat back. You can't alter the coat while it's hanging there; you swap it for a new one. The coat check doesn't care whether it's a scarf or a parka, and it never runs out of hooks.

The problem it solves

Apps are full of files: profile photos, videos, PDFs, logs, backups. Putting them in the database bloats it, slows backups and costs far more per gigabyte. Putting them on a web server's disk breaks the moment you add a second server (lesson 7), and the disk fills up or dies.

A real example

Example

How a photo app handles an upload

You pick a photo and tap share. The app asks the backend for a presigned URL, a short-lived link that allows one upload to one key. Your phone then sends the 4 MB file straight to the storage bucket, so the app servers never touch the bytes.

The upload triggers an event, and a worker makes three thumbnail sizes and saves them next to the original. The database stores only the object key, the owner and the size.

Viewers load images through a CDN in front of the bucket. A lifecycle rule moves originals older than a year to a cheaper, colder storage class, because almost nobody opens them.

How it works

Three kinds of storage

Type	What it is	Used for
Block	A virtual disk attached to one server	Databases, operating systems
File	A shared network file system with folders	Lift-and-shift apps, shared content between servers
Object	Whole files addressed by key over HTTP	Media, uploads, backups, static websites, data lakes

Objects, buckets and keys

You store objects in a bucket. Each object has a key (like users/7/avatar.jpg), the bytes, and metadata (content type, custom tags). You PUT to write, GET to read and DELETE to remove, all over HTTP. Keys look like paths, but there are no real folders.

Big object stores keep each object on multiple devices across availability zones, and the major clouds design for 99.999999999% (“eleven nines”) annual durability. They scale to any number of objects without planning.

Upload directly with presigned URLs

Don't stream every upload through your app servers. Instead:

The same works for downloads of private files: a presigned URL grants temporary access to one object. Public files are usually served through a CDN (lesson 15) with the bucket as its origin.

Storage tiers and lifecycle rules

Most data is read a lot when new and rarely afterwards. Each storage tier trades access cost and speed for a lower storage price:

• Hot / Standard: frequently read.
• Cool / Infrequent access: cheaper storage, a charge per read.
• Archive: very cheap, but retrieval takes minutes to hours.

Lifecycle rules move objects automatically, for example “move to cool after 30 days, archive after a year, delete after seven”.

Things to know

• Objects are replaced, not edited. To change one byte, upload the whole object again. (Large uploads use multipart upload.)
• Versioning keeps old copies so accidental deletes or overwrites can be undone.
• Lock down buckets. Make them private by default; publicly exposed buckets are a classic data leak.
• Encrypt at rest (usually on by default) and consider customer-managed keys for sensitive data.
• Spread prefixes for extreme request rates. Request limits apply per key prefix, so very hot workloads spread keys across prefixes.

Trade-offs

What you gain

• Very cheap per gigabyte, with automatic tiering for old data.
• Extremely durable and practically unlimited.
• Works with CDNs, analytics engines and every language via HTTP.

What it costs you

• Higher latency than a local disk: milliseconds per request, not microseconds.
• No in-place edits, no file locking, and no real folders.
• Per-request charges and data-transfer-out costs add up for very chatty access.

When not to use it

• Database files, or anything needing small random reads and writes, belongs on block storage.
• Lots of tiny, frequently changing records belong in a database, not millions of tiny objects.

In the cloud

Object storageEvery major cloud sells this as a managed service, so you configure it rather than build it.

AWS1

• Amazon S3 (opens vendor docs)

  Stores any amount of files as objects in buckets, with storage classes from frequent access to deep archive.

Azure1

• Azure Blob Storage (opens vendor docs)

  Object storage with hot, cool, cold and archive tiers.

Google Cloud1

• Cloud Storage (opens vendor docs)

  Object storage with Standard, Nearline, Coldline and Archive classes.

Clients upload straight to storage with a short-lived signed URL; the app only stores the object's key.

AWSAzureGoogle Cloud

1. App issues presigned URL
2. Client uploads to S3
3. CloudFront serves downloads

Interview questions

Try answering out loud first, then open the model answer.

Design file uploads for an app where users upload videos up to 5 GB.+

Model answer

The client asks the API to start an upload; the API checks permissions and quotas, creates a database row with status “uploading”, and returns presigned multipart-upload URLs. The client uploads parts in parallel straight to object storage and can resume failed parts. When done, the client (or a storage event) notifies the backend, which marks the upload complete and puts a processing job on a queue: virus scan, thumbnails, transcoding. Processed files go to another bucket served through a CDN. Unfinished multipart uploads are cleaned up by a lifecycle rule.

Why not store images in the database?+

Model answer

Large binary values bloat the database: backups and replication get slower, the cache fills with image bytes instead of hot rows, and storage costs several times more. Databases also aren't built to stream large files to many clients. Object storage plus a CDN is cheaper, scales without effort, and the database just keeps the key and metadata.

How would you keep storage costs down for years of user uploads?+

Model answer

Use lifecycle rules to move objects to cooler tiers as they age and to archive what's rarely touched; delete what you don't need to keep (failed uploads, old temporary files, superseded versions). Store derived files such as thumbnails in a way that lets them be regenerated rather than kept forever. Serve through a CDN to cut repeated reads, and watch request and egress costs, which often outgrow storage costs.

Quiz

Pick an answer to see whether it’s right and why.

1. 01Where should a photo-sharing app store the photos themselves?

   In a database columnIn object storage, with the key stored in the databaseOn the web server's disk
2. 02What's the advantage of letting clients upload with a presigned URL?

   The file is encrypted twiceLarge uploads go straight to storage instead of through your servers, and the link expiresIt makes downloads faster
3. 03Which storage type should a database's data files live on?

   Object storageBlock storageA CDN
4. 04What should the database store for each photo?

   The object key and metadata like owner and sizeThe image bytesNothing
5. 05Originals older than a year are rarely viewed. How can you cut storage cost?

   Store them in the databaseA lifecycle rule that moves them to a colder, cheaper storage classDelete them
6. 06Why is object storage a poor fit for a file you change a few bytes of, many times a second?

   It can't store binary dataIt has no APIObjects are replaced whole, not edited in place

0 of 6 answered

Key terms

Object storage

Storage for whole files (objects) addressed by a key in a bucket, with nearly unlimited capacity.

Bucket

A named container for objects in object storage.

Presigned URL

A temporary link that lets someone upload or download one object without having your credentials.

Block storage

A virtual hard disk attached to one server, used for databases and operating systems.

Storage tier

A price/speed class for stored data. Rarely read data can move to cheaper, slower tiers.

CDN

Content delivery network. Servers around the world that keep copies of files close to users.