Quick answer: What is API rate limiting?

API rate limiting controls how quickly a consumer can call an API, rejecting or delaying requests that exceed a defined allowance.

Key controls:

• Sustained request rate: The number of requests a consumer can send per second or minute on an ongoing basis.
• Burst allowance: The extra requests a consumer can send during a short traffic spike.
• Concurrency limit: The maximum number of requests a consumer can have in flight at once.
• Usage quota: The total requests or units a consumer can use over a day, month, or billing cycle.
• Rejection behavior and retry guidance: How the API responds when a limit is reached and what guidance it provides for retrying.

API throttling enforces these controls by rejecting or delaying excess requests. A monthly quota limits total usage, but a separate rate or burst limit is needed to protect the backend from a sudden surge.

Below, we explain how to translate backend capacity and partner commitments into limits that hold across gateways and regions.

How do you set limits from backend capacity and partner workflows?

Start with sustainable backend capacity, then check whether each partner can complete its workflow within that budget. Load-test representative operations and traffic mixes, including shared dependencies and degraded infrastructure, against agreed p95/p99 latency targets and an acceptable error rate.

A cached payment-status lookup and a payment submission that calls a ledger and fraud service may need different limits because their resource costs differ. Pair request-rate limits with concurrency protection: slower requests can exhaust connections even when requests per second remain unchanged.

For broader abuse controls, see the API security checklist.

Consider a payroll provider that polls a payment-status API to check whether employee payments have completed. Its payroll engine and reconciliation service use separate credentials to access the same backend via an EU gateway with 3 nodes and a US gateway with 2 nodes. The platform team needs to protect that backend while allowing the provider to finish its checks on time.

In this illustrative example, load testing establishes sustainable throughput of 1,500 requests per second, which the platform team allocates as follows:

• Measured sustainable throughput: 1,500 requests per second.
• Reserved headroom: 450 requests per second for spikes and degraded conditions.
• Internal-channel reserve: 250 requests per second for your own applications.
• Remaining partner capacity: 800 requests per second across all partners.

The payroll provider's workflow requires sustained throughput of 200 requests per second, with bursts up to 400 for 10 seconds.

Before configuring limits, verify batch size, calls per item, completion deadline, polling, and retries. Divide the required calls by the available seconds to estimate the minimum average rate, then account for bursts and overlapping interactive traffic; check that the combined partner demand fits within the capacity budget.

How do you choose the limiting identity and scope?

The limiting identity determines which requests share an allowance; the scope determines which APIs, operations, and environments it applies to. For a partner-wide commitment, map authenticated credentials to the partner or contract that owns the allowance.

In the payroll example, both credentials should draw from the provider's 200-request-per-second sustained budget. Giving each credential an independent allowance of 200 would let the provider sustain 400 requests per second, rather than allowing 400 only as a 10-second burst. It could then consume half of the 800-request-per-second partner capacity, leaving less room for other partners' commitments.

IP-based limits can group unrelated clients behind a single shared public IP address, while application-based limits can grant one partner several independent allowances. Resolve the partner identity from validated authentication data or a trusted credential mapping.

Define the applicable limits:

• Partner-wide limit: A shared allowance across the partner's credentials and operations.
• Operation limit per partner: A tighter allowance for expensive calls, such as payment submissions.
• Backend-operation limit: A total allowance across all consumers of an operation.
• Environment limits: Separate allowances for production and sandbox traffic.

A request must satisfy every applicable limit. To share an allowance across gateways, use the same counter identifier for that allowance and coordinate its state. Including the region in the identifier creates separate regional counters; matching identifiers in independent stores do not create a shared budget.

Document whether rejected attempts, gateway retries, cached responses, and upstream failures count toward each allowance. Track backend load separately from billable usage: a retry may consume capacity without representing another billable transaction.

Which rate limiting algorithm should you choose?

Once you know whose requests share a budget, choose how to enforce it over time. Base the algorithm on the partner’s burst requirements, the backend’s tolerance for sudden traffic, and the accuracy you need. Redis’s algorithm guide explains these trade-offs.

‍

For the payroll provider, a token bucket can support a sustained rate of 200 requests per second and a burst to 400 for 10 seconds. Each request consumes 1 token; the bucket refills at 200 tokens per second and can hold up to 2,000 tokens.

When initially full, the bucket supplies the extra 200 tokens per second needed for the 10-second burst: (400 − 200) × 10 = 2,000 tokens.

However, the provider could spend the accumulated tokens more quickly. If 400 requests per second is a strict ceiling, add a separate peak-rate or pacing control and specify the interval over which it applies. Token refill rate and bucket capacity govern different parts of the allowance. (AWS documentation)

For queued enforcement, set a maximum waiting time that fits the partner’s workflow. A request admitted to a queue can still miss its payroll deadline.

Whichever algorithm you choose, check and update each counter in one indivisible operation, so simultaneous requests cannot use the same remaining allowance.

Should the rate limit budget be local, regional, or global?

Decide whether a limit applies separately to each gateway node, to all traffic within a region, or to the partner’s total traffic across regions. Local limits protect individual nodes, regional limits protect regional capacity, and a global limit keeps the partner’s combined traffic within its allowance. You can use these controls together.

For the payroll provider example, copying a 200-requests-per-second policy to every gateway does not create one shared allowance. If each node or gateway counts requests independently, the provider can receive several separate allowances:

‍

These are alternative configurations that exclude bursts and assume that enough traffic reaches every path. The gateway-level rows assume nodes within each gateway share their counts. The first configuration would let one provider exceed both the 800-request partner pool and the backend’s measured throughput of 1,500 requests per second.

If you divide the allowance between regions, base the split on expected peak traffic and available backend capacity. Suppose 60% of the provider’s peak traffic is expected in the EU and 40% in the US, and both regions have sufficient capacity. Allocating 120 requests per second to the EU and 80 to the US preserves the sustained total of 200 requests.

That fixed split becomes restrictive when traffic shifts. If the provider sends 150 requests per second to the EU and 50 to the US, the EU rejects 30 per second while 30 of the US allocation goes unused. Divide burst capacity deliberately too; giving each region the full burst allowance would duplicate it.

A global budget allows the provider to use its allowance wherever requests arrive, but coordination can introduce latency and dependence on shared infrastructure. Regional budgets keep enforcement within each region, but need a transfer mechanism if unused allowances must follow changing demand.

During failover, transfer an allocation only after ensuring the previous holder can no longer spend it and confirming that the surviving backend has enough capacity. Moving traffic to the EU does not automatically transfer the US allowance.

How do you design counter coordination and failure behavior?

Once you decide where the budget applies, choose how gateways track consumption together. Balance the delay added to each request against the amount of traffic above the allowance your backend can tolerate.

• Check shared state for every request: Each gateway checks and updates the shared allowance before admitting a request. This provides coordinated decisions but adds a network call to the request path.
• Count locally and synchronize later: Gateways admit requests using local counts and exchange updates periodically. Decisions are faster, but delayed updates can let combined traffic exceed the allowance.
• Allocate token batches: A coordinator reserves a portion of the shared budget for each gateway to spend locally. This reduces coordination calls, but batches must not overlap or be reassigned while the previous gateway can still spend them.

Synchronizing counts after admission cannot, by itself, guarantee a strict global ceiling. Token batches can preserve the budget only if allocation, expiry, and reassignment prevent the same allowance from being spent twice.

Gateways also need compatible counter identifiers, algorithms, and stored data formats. Pointing them at the same counter store does not automatically make their limits work together.

Plan for lost counter data and an unavailable store. For example, Redis documents that acknowledged writes can be lost during failover; losing consumption records can make previously spent allowance available again.

Choose what happens when a gateway cannot verify the remaining allowance:

• Fail open: Admit requests without the shared-limit check, relying on other controls to protect the backend.
• Fail closed: Reject requests whose admission cannot be verified, protecting the budget but interrupting partner workflows.
• Use fallback local limits: Continue admitting a bounded amount of traffic at each gateway node, accepting that the combined allowance may differ from the normal shared budget.

For the payroll provider’s five-node setup, a fallback limit of 40 requests per second per node totals 200. A limit of 200 per node totals 1,000, which is already above the entire 800-request partner pool. Check fallback totals across all partners and the maximum number of active nodes.

Set a timeout for coordination calls, alert when fallback begins, and define how gateways resume coordinated counting without duplicating outstanding allowances.

How should you give partners actionable limit responses?

Return a consistent, machine-readable response that explains the exhausted allowance and what the client should do next. RFC 6585 defines HTTP 429 Too Many Requests and permits a Retry-After header.

Include:

• Stable error code and limit scope
• Request ID for support
• Meaningful Retry-After guidance
• Documented remaining-capacity and reset metadata, where available

Retry-After accepts a delay in seconds or an HTTP date. Problem Details provides a standard error-body structure; extensions can identify the applicable policy without exposing internal infrastructure.

Keep names, units, and response shapes consistent across gateways. Make it clear whether the partner has used up its allowance or the gateway cannot check it because the rate-limiting service is unavailable. Return HTTP 429 for an exceeded allowance; a failure to check the allowance should be reported as a service failure — HTTP 503 with Retry-After , not as the partner exceeding its limit.

Partners should keep their workers' combined request rate within the shared allowance. When retrying, wait at least as long as the Retry-After header specifies, increase the wait after repeated failures, and add a random delay to avoid retries happening at the same time. Limit retry attempts, retry writes only when the API supports protection against duplicate processing, and save batch progress so completed work is not repeated.

‍

How do agentic consumers change rate limiting?”

Rate limits were designed around human-paced applications: traffic that follows business hours, retries written by a developer who read the documentation, and request volumes that scale with users rather than with compute.

Autonomous agents satisfy none of those assumptions. An agent treating HTTP 429 as a transient error will often retry immediately rather than honouring Retry-After, turning a single limit breach into a sustained one. Agent traffic arrives in bursts shaped by task execution rather than user behaviour, so burst allowances calibrated against historical human patterns may not fit. And a single partner credential can now front an unpredictable number of agents, which weakens the assumption that one credential represents one workload.

Three adjustments are worth considering. Treat Retry-After compliance as something to verify rather than assume, and consider progressively longer rejection windows for consumers that ignore it. Review burst allowances against observed agent traffic rather than the workflow description a partner supplied at onboarding. And where a partner exposes your API to agents, establish whether the limiting identity should remain the partner, or move to something finer-grained.

‍

How do you test rate limit rollout and regional failover?

Test admitted aggregate traffic and partner completion times across the complete topology. Passing a single-gateway test leaves distributed rate limiting unverified.

For the payroll provider example, cover:

• Both credentials across both gateways simultaneously
• Uneven routing, autoscaling, and node restarts
• Sustained demand, allowed bursts, and window boundaries
• Store latency, loss of connectivity, and lost counter state
• Regional failover, recovery, and allocation overlap
• Quota exhaustion, aggressive retries, and duplicate writes

Where supported, run the proposed limits in observe-only mode, recording requests that would be rejected without blocking them, through a representative business cycle. For the payroll provider example, include peak payment-status polling around payroll runs. Compare those potential rejections with backend utilization and whether the provider completes its checks before its deadline.

Set pass conditions before testing: aggregate admission within the chosen budget and overage tolerance; limiter-added p95/p99 latency within its budget; acceptable error rates; and the partner completing its API workflow before its deadline.

Start enforcing the new limits for a small group of partners after communicating the change. Record failure triggers, recovery conditions, and rollback thresholds; test rollback without resetting consumed quotas or issuing duplicate allocations.

Keep this evidence with the deployment review, alongside other API lifecycle release checks.

How do you review rate limit fairness and exceptions?

Review limits against both backend health and partner outcomes on a regular schedule and after capacity or contract changes.

Track admitted aggregate traffic, rejections by policy and partner, the proportion of requests that are retries, latency, saturation, time spent using fallback limits, and batch completion time. Spare backend capacity, alongside repeated partner failures, should trigger an investigation into regional allocations, competing operations, and client pacing.

Separate time-sensitive traffic from bulk work when their priorities differ, while retaining an aggregate backend limit. Every temporary increase needs an owner, a business reason, a scope, a capacity check, an expiry, and a rollback condition.

Version these changes using your policy-as-code process so an incident workaround remains reviewable.

How does APIwiz help govern rate-limit policies across gateways?

APIwiz supports the management, ownership, testing, and usage visibility needed to govern rate limits across gateways. For the payroll provider example, these capabilities help the platform team put the agreed allowance into practice and review its impact:

• API Gateway Federation: Provides a single control plane for managing APIs across multiple gateways. This gives the team a common management layer for the EU and US deployments.
• Service Registry: Records API ownership, versions, and dependencies. This helps identify the payment-status service owner and the teams that should review capacity changes or partner exceptions.
• API Test Suite: Supports automated, scenario-driven integration tests. The team can include limit-response and retry scenarios, alongside the load and failover tests needed to verify aggregate enforcement.
• Consumer Insights: Provides usage metrics by endpoint and consumer. These help the team examine payment-status polling patterns when reviewing the provider’s sustained and burst allowances.
• API monitoring and tracing: Provides latency, runtime metrics, and transaction traces. These help investigate slow payment-status checks alongside gateway rejection data and counter-service telemetry.

Confirm which gateway integrations and telemetry are available in your deployment. Centralized management does not, by itself, create a shared allowance: the gateways and coordination service must enforce it, and multi-gateway testing must verify that both credentials draw from the provider’s combined budget.

What should a distributed rate-limit policy worksheet include?

Record the agreed allowances, counting rules, and failure behavior before deployment. Complete a worksheet for each partner and protected operation group, linking it to the shared backend capacity budget.

Rollout checklist

• Backend capacity and partner workflow requirements verified
• Credential mapping and combined allowances across gateways checked
• Burst, uneven routing, scaling, and failure tests passed
• Failover and rollback tested without resetting consumed quotas or duplicating allocations
• Partner notice and retry documentation published
• Monitoring, alert thresholds, and exception expiry checks in place

Key takeaways

Set rate limits from tested backend capacity and partner workflow requirements. Keep each partner’s combined traffic within its agreed allowance across credentials, gateways, and regions. Verify burst handling, fallback behavior, and regional failover without duplicating allowances, and give partners clear rejection responses and retry guidance.

To explore APIwiz’s multi-gateway management and usage visibility for your environment, book a demo.

FAQs about API rate limiting

Why can a partner receive HTTP 429 below its contracted rate?

An operation-specific limit, regional allocation, burst allowance, or usage quota may be exhausted even when the partner’s average rate is below its contract limit. Check which policy rejected the request and the interval it measures; minute-level averages can hide short spikes. Concurrency limits may also cause requests to be rejected, depending on the API’s documented behavior. A backend or limiter failure should be reported separately from an exceeded allowance.

Can clients avoid a 429 HTTP status code by switching gateways?

Not when both gateways enforce the same shared partner allowance. Switching routes does not increase that allowance. If gateways are counted independently, switching may expose additional capacity, but that is not a valid way to enforce a shared budget. Coordinate request pacing across workers and follow the documented retry guidance.

Should regional failover reset API quotas?

No, unless the contract explicitly allows a reset. Requests or units already counted must remain counted when traffic moves to another region. Test that failover and recovery preserve consumed usage rather than making already-spent quota available again.

What if the rate-limit store stays unavailable?

Follow the documented failure policy. If local fallback limits are used, check their combined allowance across all active gateway nodes and partners against available backend capacity. If requests cannot be admitted safely, reject them as a service failure, or defer them only where a bounded queue and waiting-time limit are supported.

Related reads

• Federated API management
• Unified API usage insights
• API management beyond the gateway

‍