Problem Statement

Design a rate limiter for a large-scale backend system.

The rate limiter should restrict the number of requests a client can make within a specified time window. The system must operate correctly in a distributed environment where multiple stateless application servers handle traffic.

Example limits:

• 100 requests per minute per user
• 1000 requests per hour per API key
• Different limits per endpoint

Functional Requirements

1. Limit requests based on:
   • User ID
   • API Key
   • IP Address
2. Support configurable rate limits (e.g., 10 requests/second, 100 requests/minute).
3. Support per-endpoint rate limiting.
4. Return an appropriate response (e.g., HTTP 429) when the limit is exceeded.
5. Allow dynamic updates to rate limit configurations.

Non-Functional Requirements

1. High availability.
2. Low latency — rate check should not significantly impact request processing time.
3. Horizontally scalable.
4. Fault tolerant.
5. Minimal operational overhead.

Scale Assumptions