All About APIs - A Deep Dive into API Design

Overview

An API (Application Programming Interface) is a contract between a provider and a consumer. It defines how software components communicate, what inputs are expected, and what outputs are guaranteed. A well-designed API is the backbone of scalable, maintainable systems.

What makes a good API:

• Consistent - follows predictable patterns across all endpoints
• Intuitive - a developer can guess the behavior without reading docs
• Evolvable - can grow without breaking existing consumers
• Minimal - exposes only what's necessary, hiding internal complexity
• Well-documented - self-describing with clear contracts

API as a Contract: Once published, an API is a promise. Consumers build systems on top of it. Breaking that promise (changing response shapes, removing fields, altering semantics) cascades failures across dependent systems. Treat every public API as a legal agreement.

API-First Design: Design the API before writing implementation code. This approach forces you to think about the consumer's experience first, enables parallel development (frontend and backend teams work simultaneously), and produces better abstractions. Tools like OpenAPI let you define the contract upfront and generate server stubs and client SDKs.

The Cost of Bad API Design: A poorly designed API creates technical debt that compounds over time. Every consumer that integrates with a bad API becomes a stakeholder resisting change. Amazon's famous "Bezos mandate" (2002) required all teams to expose functionality through service interfaces - this forced API-first thinking at scale and became a foundation of AWS. The lesson: invest heavily in API design upfront because the cost of change grows exponentially with adoption.

Design Principles Summary:

• Principle of Least Surprise - behave as developers expect
• Robustness Principle (Postel's Law) - be conservative in what you send, liberal in what you accept
• Single Responsibility - each endpoint does one thing well
• Discoverability - new developers can explore without reading every doc page

---

REST Principles

REST (Representational State Transfer) is an architectural style defined by Roy Fielding in his 2000 dissertation. It models the world as resources identified by URIs, manipulated through a uniform interface.

Core Principles:

• Resources - Everything is a resource (user, order, product). Each has a unique URI: /users/42
• Uniform Interface - Use standard HTTP methods with consistent semantics
• Statelessness - Each request contains all information needed to process it. No server-side session state between requests
• Client-Server Separation - Client and server evolve independently
• Cacheability - Responses declare themselves cacheable or not
• HATEOAS (Hypermedia As The Engine Of Application State) - Responses include links to related actions, enabling clients to discover capabilities dynamically

Statelessness in Practice: Every request must carry its own authentication token, pagination cursor, and context. The server stores no session between requests. This enables horizontal scaling - any server instance can handle any request. Load balancers don't need sticky sessions. The tradeoff is slightly larger request payloads, but the scalability gains far outweigh this cost.

HATEOAS in Practice: While few APIs implement full HATEOAS, partial adoption (including self, next, prev links) dramatically improves client resilience. Clients that follow links rather than constructing URLs survive endpoint restructuring without code changes.

CRUD to HTTP Methods Mapping:

---

API Design Best Practices

Naming Conventions:

• Use plural nouns for resources: /users, /orders, /products
• Use kebab-case for multi-word resources: /order-items, /payment-methods
• Avoid verbs in URIs - the HTTP method is the verb: /getUsers → GET /users
• Use nesting for relationships: /users/42/orders (orders belonging to user 42)

Resource Hierarchy:

/organizations/{orgId}/teams/{teamId}/members/{memberId}

Keep nesting to 2-3 levels maximum. Beyond that, promote sub-resources to top-level with filters.

Filtering, Sorting, Pagination:

GET /products?category=electronics&sort=-price&page=2&limit=20

• Filtering: ?status=active&role=admin
• Sorting: ?sort=created_at (ascending), ?sort=-created_at (descending)
• Pagination: ?page=1&limit=25 or ?cursor=abc123&limit=25

Versioning Strategies Comparison:

---

Request/Response Design

Consistent Response Structure (Envelope Pattern):

{
  "status": "success",
  "data": {
    "id": 42,
    "name": "Alice",
    "email": "alice@example.com"
  },
  "meta": {
    "request_id": "req_abc123",
    "timestamp": "2026-05-31T10:00:00Z"
  }
}

Pagination Metadata:

{
  "status": "success",
  "data": [...],
  "pagination": {
    "total": 245,
    "page": 3,
    "limit": 20,
    "has_next": true,
    "next_cursor": "eyJpZCI6NjB9"
  }
}

HATEOAS Links:

{
  "data": {
    "id": 42,
    "name": "Alice"
  },
  "links": {
    "self": "/users/42",
    "orders": "/users/42/orders",
    "update": "/users/42",
    "delete": "/users/42"
  }
}

HATEOAS enables clients to navigate the API without hardcoding URLs, making the API self-discoverable and reducing coupling.

---

Error Handling in APIs

Standard Error Response Format:

{
  "status": "error",
  "error": {
    "code": "VALIDATION_FAILED",
    "message": "Request validation failed",
    "details": [
      {"field": "email", "message": "must be a valid email address"},
      {"field": "age", "message": "must be at least 18"}
    ]
  },
  "meta": {
    "request_id": "req_xyz789",
    "timestamp": "2026-05-31T10:00:00Z"
  }
}

HTTP Status Codes Reference:

Validation Error Aggregation: Always return ALL validation errors in a single response rather than failing on the first error. This reduces round-trips and improves developer experience. Group errors by field name for easy client-side mapping to form fields.

---

API Versioning

Strategies Deep Comparison:

Breaking vs Non-Breaking Changes:

---

Pagination

Offset-Based vs Cursor-Based Comparison:

Offset-Based Pseudocode:

def get_users_offset(page, limit):
    offset = (page - 1) * limit
    users = db.query("SELECT * FROM users ORDER BY id LIMIT ? OFFSET ?", limit, offset)
    total = db.query("SELECT COUNT(*) FROM users")
    return {
        "data": users,
        "pagination": {
            "page": page,
            "limit": limit,
            "total": total,
            "has_next": offset + limit < total
        }
    }

Cursor-Based Pseudocode:

def get_users_cursor(cursor, limit):
    last_id = decode_cursor(cursor) if cursor else 0
    users = db.query("SELECT * FROM users WHERE id > ? ORDER BY id LIMIT ?", last_id, limit + 1)
    has_next = len(users) > limit
    users = users[:limit]
    next_cursor = encode_cursor(users[-1].id) if has_next else None
    return {
        "data": users,
        "pagination": {
            "next_cursor": next_cursor,
            "has_next": has_next
        }
    }

---

Rate Limiting

Why Rate Limit: Protect services from abuse, ensure fair usage across consumers, prevent cascading failures, and manage infrastructure costs. Without rate limiting, a single misbehaving client can monopolize resources and degrade service for everyone.

Common Algorithms:

• Token Bucket - Tokens accumulate at a fixed rate; each request consumes one token. Allows bursts up to bucket capacity.
• Sliding Window Counter - Counts requests in a rolling time window. More accurate than fixed windows, avoids boundary burst issues.
• Leaky Bucket - Requests queue and process at a fixed rate. Smooths traffic but adds latency.

Communicating Limits via Headers:

X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 742
X-RateLimit-Reset: 1685520000
Retry-After: 60

429 Response:

{
  "status": "error",
  "error": {
    "code": "RATE_LIMIT_EXCEEDED",
    "message": "Too many requests. Limit: 1000 requests per hour.",
    "retry_after": 60
  }
}

Token Bucket Rate Limiter Pseudocode:

class TokenBucketRateLimiter:
    def __init__(self, capacity, refill_rate_per_second):
        self.capacity = capacity
        self.tokens = capacity
        self.refill_rate = refill_rate_per_second
        self.last_refill = current_time()

    def allow_request(self, client_id):
        self.refill(client_id)
        bucket = self.get_bucket(client_id)
        if bucket.tokens >= 1:
            bucket.tokens -= 1
            return True  # Allow
        return False  # Reject with 429

    def refill(self, client_id):
        bucket = self.get_bucket(client_id)
        elapsed = current_time() - bucket.last_refill
        new_tokens = elapsed * self.refill_rate
        bucket.tokens = min(self.capacity, bucket.tokens + new_tokens)
        bucket.last_refill = current_time()

---

Authentication & Authorization

API Keys: Simple token passed in header (X-API-Key: abc123). Easy to implement but offers no granular permissions, hard to rotate, and if leaked grants full access.

OAuth 2.0: Industry-standard authorization framework. Supports multiple grant types (authorization code, client credentials, PKCE). Provides scoped access tokens with expiration. Separates authentication from authorization.

JWT (JSON Web Tokens): Self-contained tokens encoding claims (user ID, roles, expiry). Stateless verification via signature. No database lookup needed per request. Risk: tokens can't be revoked until expiry without a blocklist. Structure: header.payload.signature - base64url encoded. Use short-lived access tokens (15 min) paired with longer-lived refresh tokens for security.

OAuth 2.0 Grant Types:

• Authorization Code + PKCE - For SPAs and mobile apps. Most secure for user-facing flows.
• Client Credentials - For machine-to-machine. No user context.
• Device Code - For devices without browsers (smart TVs, CLI tools).
• Refresh Token - Exchange expired access token for a new one without re-authentication.

Comparison Table:

---

Idempotency

What it means: An operation is idempotent if performing it multiple times produces the same result as performing it once. This is critical for reliability - network failures cause retries, and retries must not create duplicate side effects.

HTTP Method Idempotency:

Idempotency Keys for POST:

Since POST is not naturally idempotent, clients send a unique Idempotency-Key header. The server stores the result of the first execution and returns the cached result for subsequent requests with the same key.

POST /payments
Idempotency-Key: idem_8f14e45f
Content-Type: application/json

{"amount": 100, "currency": "USD"}

Server logic:

1. Check if Idempotency-Key exists in cache/store
2. If yes → return stored response (do not re-execute)
3. If no → execute operation, store response keyed by idempotency key, return response

Stripe uses this pattern extensively - every mutating API call accepts an Idempotency-Key header.

Implementation Considerations:

• Store idempotency keys with TTL (typically 24-48 hours) - don't keep them forever
• The stored response must include the full HTTP response (status code, headers, body)
• If a request with the same key arrives while the first is still processing, return 409 Conflict or block until completion
• Idempotency keys should be client-generated UUIDs - never server-generated
• Scope keys per API key/user to prevent cross-tenant collisions

---

API Documentation

OpenAPI/Swagger: The OpenAPI Specification (formerly Swagger) is the industry standard for describing REST APIs. It defines endpoints, parameters, request/response schemas, authentication, and examples in a machine-readable YAML/JSON format.

Contract-First vs Code-First:

Documentation must include: endpoint descriptions, request/response examples, authentication requirements, error codes, rate limits, and changelog.

Best Practices for API Documentation:

• Provide runnable examples (curl commands, language-specific snippets)
• Include a "Getting Started" guide that gets developers to their first successful call in under 5 minutes
• Document every error code with resolution steps
• Maintain a public changelog with migration guides for breaking changes
• Offer interactive API explorers (Swagger UI, Redoc, Postman collections)
• Version your documentation alongside your API - docs for v1 should remain accessible even after v2 launches

---

Real-World Examples

GitHub API:

• Uses URI versioning (Accept: application/vnd.github.v3+json)
• Hypermedia-driven - every response includes _links
• Pagination via Link header with rel="next" and rel="last"
• Rate limit: 5000 requests/hour for authenticated users
• Consistent error format with message and documentation_url

Stripe API:

• Exemplary versioning - date-based versions (Stripe-Version: 2023-10-16)
• Every API change is backwards-compatible within a version
• Idempotency keys on all POST requests
• Expandable objects - ?expand[]=customer inlines related resources
• Excellent error taxonomy: card_error, invalid_request_error, api_error

Twitter (X) API:

• OAuth 2.0 with PKCE for user context
• Cursor-based pagination for timelines
• Rate limits per endpoint with 15-minute windows
• Fields selection: ?tweet.fields=created_at,public_metrics
• Versioned via URI path (/2/tweets)

Key Takeaways from Industry APIs:

• All three use consistent envelope patterns for responses
• All implement rate limiting with clear header communication
• Stripe and GitHub invest heavily in documentation and changelogs
• All provide sandbox/test environments for development
• Stripe's approach to versioning (date-based, per-customer pinning) is considered best-in-class for APIs with many consumers
• GitHub's use of conditional requests (ETag, If-None-Match) reduces unnecessary data transfer significantly

---

Interview Follow-ups

Q1: How would you handle backward compatibility when you need to rename a field?

A: Never rename directly. Add the new field alongside the old one, mark the old field as deprecated in documentation, return both fields for a migration period, then remove the old field only in the next major version. Use sunset headers (Sunset: Sat, 31 Dec 2026 23:59:59 GMT) to communicate timelines.

Q2: How do you design an API for a resource that has multiple valid representations?

A: Use content negotiation via the Accept header. The client requests application/json or application/xml or a custom media type. Alternatively, use a ?format= query parameter for simpler implementations. For partial representations, support field selection: ?fields=id,name,email.

Q3: How would you handle long-running operations in a REST API?

A: Return 202 Accepted with a location header pointing to a status endpoint. The client polls that endpoint until the operation completes. Alternatively, provide a webhook callback URL in the original request. For real-time updates, consider Server-Sent Events or WebSockets for the status stream.

Hint-Only Questions:

Q4: Design a rate limiting strategy that handles both per-user and global limits.

• Hint: Think hierarchical token buckets - global bucket feeds into per-user buckets. Consider sliding window counters in Redis with composite keys.

Q5: How would you design an API gateway that supports canary deployments of new API versions?

• Hint: Think weighted routing at the gateway layer, version headers for sticky sessions, and percentage-based traffic splitting with automatic rollback on error rate thresholds.

---

Counter Questions to Ask Interviewer

1. Who are the API consumers? - Internal teams, third-party developers, or public? This determines versioning strictness, documentation depth, and rate limiting policies.
2. What's the expected request volume and latency SLA? - Influences pagination strategy, caching layers, and whether synchronous REST is appropriate vs async messaging.
3. Is backward compatibility a hard requirement? - Determines versioning strategy and whether additive-only changes are sufficient.
4. What's the authentication context? - Machine-to-machine vs user-facing changes the auth flow (API keys vs OAuth2 with PKCE).
5. Are there existing API conventions in the organization? - Consistency across services matters more than any single "best" practice.

---

References & Whitepapers

1. Roy Fielding - "Architectural Styles and the Design of Network-Based Software Architectures" (2000) - The original REST dissertation. Defines the constraints that make REST scalable. ics.uci.edu/~fielding/pubs/dissertation/top.htm

2. Microsoft REST API Guidelines - Comprehensive, opinionated guide covering naming, versioning, error handling, and collections. github.com/microsoft/api-guidelines

3. Google API Design Guide - Covers resource-oriented design, standard methods, custom methods, and error model. cloud.google.com/apis/design

4. Zalando RESTful API Guidelines - Practical guidelines from a large-scale microservices organization. opensource.zalando.com/restful-api-guidelines

5. RFC 7231 - HTTP/1.1 Semantics and Content - Defines method semantics and status codes.

6. RFC 6749 - The OAuth 2.0 Authorization Framework.

---

Related Topics

• Design Patterns in LLD
• Database Design & Normalization
• Caching Strategies
• Load Balancing & API Gateways
• Authentication & Security
• Microservices Communication