There’s a conversation I find myself having repeatedly across enterprise engagements. A development team asks whether they should add rate limiting to their API, and the instinct is to frame it as a performance concern, something you bolt on when things get slow. That framing is wrong, and it leads to wrong decisions.
Rate limiting is a reliability and governance control.
The architectural problem we’re actually solving
When you expose an API to external consumers (partners, ISVs, internal teams operating as separate entities) you are operating a multi-tenant platform. Multi-tenant platforms have a fundamental contract: one tenant’s behaviour must not degrade another tenant’s experience.
Most backend systems are designed around expected load profiles with room for reasonable spikes. What they’re not designed for is a single misconfigured client hammering them at ten times the expected rate because someone deployed a broken retry loop on a Friday afternoon. Without a control boundary, that client’s failure becomes a platform-wide failure.
Protecting backend systems is the obvious benefit. The less obvious one is the more interesting architectural argument: when a client receives a clear HTTP 429, it knows exactly what happened. The failure is explicit, fast, and actionable. That is categorically better than the alternative — the request hangs or silently succeeds, the client retries aggressively, the backend degrades, and your on-call engineer spends three hours diagnosing something that should have been a one-minute integration fix. Predictable rejection is safer than unpredictable system behaviour.
The Governance Principles That Follow From This
Every external API must have rate limiting, without exception. Even if you set the threshold at 10,000 requests per minute, a number no legitimate client will ever approach, that ceiling still contains catastrophic failure. A broken integration must degrade itself, not the platform. High limits aren’t about restriction; they’re about containment.
Limits must be per client, never global. This is the principle I see violated most often. A global rate limit creates shared-bucket contention: one aggressive client starves others. You’ve created the exact cross-client interference you were trying to prevent. Per-client limits enforce isolation, fairness, and predictable behaviour — and as a bonus, produce usage analytics that are actually meaningful.
Limits must be justified against real numbers. If your backend can sustain 200 requests per second and you’re onboarding five clients, you cannot allocate 200 RPS to each. Limit design requires reasoning about expected peak usage per client, total backend capacity, and active consumers simultaneously. This is a capacity planning exercise dressed as a configuration decision.
Where Azure API Management Fits
APIM sits at the boundary between consumers and backend services, at the right place to enforce a reliability control.
APIM’s rate-limit-by-key policy is the implementation primitive that matters. The standard pattern uses the subscription key as the client discriminator:
<inbound>
<base />
<rate-limit-by-key
calls="1000"
renewal-period="60"
counter-key="@(context.Subscription.Id)" />
</inbound>Each client gets their own counter. Their consumption doesn’t affect anyone else’s. When they breach the limit, they receive a 429 with a Retry-After header automatically — no custom error handling required.
In OAuth or federated identity scenarios where clients authenticate via JWT rather than subscription keys, key the counter off the Authorization header instead:
<inbound>
<base />
<rate-limit-by-key
calls="1000"
renewal-period="60"
counter-key="@(context.Request.Headers.GetValueOrDefault("Authorization", ""))" />
</inbound>The approach is flexible. The principle is fixed: the counter key must resolve to a per-client identifier.
What is explicitly prohibited:
<!-- DO NOT USE -->
<rate-limit calls="1000" renewal-period="60" />This creates a single shared global bucket across all consumers. It violates the fairness principle, introduces cross-client interference, and defeats the purpose of the control. If I see this in a production policy, it’s a finding that needs remediation.
With customized error handling
One practical advantage of enforcing this at the APIM layer is that the error handling contract is handled automatically. APIM returns HTTP 429 and includes the Retry-After header. Your backend never sees the request.
This is the right separation of concerns. Backend services shouldn’t need to know anything about rate limiting. That’s a gateway responsibility, enforced at the boundary, transparently.
If you need a custom error body:
<on-error>
<return-response>
<set-status code="429" reason="Too Many Requests" />
<set-body>{
"error": "rate_limit_exceeded",
"message": "Too many requests. Please retry later."
}</set-body>
</return-response>
</on-error>Closing thought
When teams treat rate limiting as a performance optimisation, they add it reactively, at thresholds that were guessed rather than reasoned, often globally rather than per client. The control ends up technically present but architecturally hollow.
Treat it as a governance and reliability control, and it gets applied consistently, configured deliberately, and enforced at the right layer from day one.