Storage, APIs, and Auth Basics

Storage, APIs, and auth are the contract layer of a system. They decide what data is durable, how clients interact with it, and who is allowed to do what.

SQL, NoSQL, And Object Storage

Use SQL when the product needs relationships, constraints, transactions, ad hoc queries, or clear reporting. PostgreSQL and MySQL are excellent defaults for payments, permissions, accounts, orders, and admin workflows.

Use NoSQL when the access pattern is simple, scale is high, schema changes quickly, or availability matters more than immediate consistency. DynamoDB, Cassandra, MongoDB, and Bigtable-style stores are common for events, profiles, feeds, counters, and time-series data.

Object storage such as S3 is for blobs: images, PDFs, model artifacts, exports, logs, and backups. Store metadata in a database and the large object in object storage. Do not put 20 MB documents directly in Redis or a relational row unless you have a very specific reason.

Indexes And Isolation

An index is a data structure that speeds reads by maintaining an ordered or searchable copy of selected columns. Indexes improve lookup latency but slow writes and consume storage.

Common indexes:

• Primary key: unique row identity.
• Composite index: supports queries like (tenant_id, created_at).
• Full-text index: supports keyword search.
• Vector index: supports semantic similarity search.

Transaction isolation controls what one transaction can observe from another. Read committed is a common default. Repeatable read prevents rows you already read from changing during the transaction. Serializable gives the strongest behavior but costs more coordination and may require retries.

API Styles

REST is the best default for public APIs, browser clients, and simple CRUD. It is human-readable, cacheable, and easy to debug.

gRPC is useful for internal service-to-service calls that need strict schemas, lower overhead, bidirectional streaming, or generated clients. It uses Protocol Buffers and HTTP/2.

WebSockets keep a long-lived connection open for real-time updates. They fit chat, collaborative editing, multiplayer state, live dashboards, and token streaming when the client needs bidirectional interaction. For one-way server-to-browser streams, Server-Sent Events are often simpler.

Scaling rules:

• REST scales through stateless servers, HTTP caching, pagination, and idempotency.
• gRPC scales through connection pooling, deadlines, backpressure, and load balancing that understands HTTP/2.
• WebSockets scale through sticky connection management, fanout services, heartbeats, and careful per-connection memory limits.

Auth Basics

Authentication answers “who are you?” Authorization answers “what can you do?”

JWTs are signed tokens containing claims such as user ID, issuer, expiry, and scopes. They are fast to validate but hard to revoke because services can verify them without calling a central database. Keep access tokens short-lived, store them in httpOnly cookies for browser apps when possible, and use refresh tokens carefully.

JWT revocation patterns:

• Short access token TTL plus refresh token rotation.
• Token version stored on the user record.
• Denylist for high-risk revocations.
• Introspection endpoint for sensitive operations.

OAuth 2.0 lets a user authorize an app to access resources. PKCE protects public clients by binding the authorization code exchange to a one-time verifier, reducing the risk of stolen authorization codes.

CORS is a browser control that decides which origins can call your API from frontend JavaScript. It is not a replacement for authentication.

Idempotency keys make retries safe. For payment creation, order submission, and job scheduling, the client sends a unique key; the server returns the same result if the request is retried.

POST /payments
Idempotency-Key: 7f1c4d6e-8a9b-4f1b-a87a-2c77f1df0c4a

Walkthrough: Key-Value Store

Requirements: support put, get, and delete; store small values; handle 50,000 reads per second and 10,000 writes per second; provide high availability; tolerate eventual consistency for non-critical data.

API:

PUT /kv/{key}
GET /kv/{key}
DELETE /kv/{key}

Data model: key, value bytes, version, TTL, created_at, updated_at.

Architecture: API servers route requests to storage nodes using consistent hashing. Each key has a primary replica plus two followers. Writes go to a quorum such as 2 of 3 replicas; reads can go to one replica for low latency or quorum reads for stronger consistency. A background repair process reconciles divergent versions.

Storage: keep a write-ahead log for durability, an in-memory memtable for recent writes, and immutable sorted files on disk for older data. Bloom filters avoid unnecessary disk reads for missing keys.

Trade-offs: stronger quorum settings reduce stale reads but increase latency and reduce availability during failures. TTL cleanup can be lazy on read plus periodic compaction.

Design Checklist

• Choose SQL, NoSQL, object storage, or search based on access pattern.
• Define indexes from queries, not from guesses.
• Pick REST, gRPC, WebSocket, or SSE from client needs and traffic shape.
• Add auth scopes, token lifetime, revocation, and audit requirements.
• Make retryable writes idempotent.

Interview Practice

1. When is PostgreSQL a better default than a NoSQL database?
2. What index would support querying all invoices for one tenant by creation time?
3. Explain read committed, repeatable read, and serializable in product terms.
4. When would you choose gRPC over REST?
5. How do WebSockets change load balancing and autoscaling?
6. Why are long-lived JWTs risky, and how can they be revoked?
7. How does PKCE improve OAuth security for browser or mobile apps?
8. Design idempotency for a payment API.