rate
which is the maximum number of requests that will be permitted during the interval (window)per
which is the length of the interval (window) in secondsrate
to 10 (requests) and per
to 60 (seconds).
rate
and per
are configured in multiple policies applied to the same key then the Gateway will determine the effective rate limit configured for each policy and apply the highest to the key.
Given, policy A with rate
set to 90 and per
set to 30 seconds (3rps) and policy B with rate
set to 100 and per
set to 10 seconds (10rps). If both are applied to a key, Tyk will take the rate limit from policy B as it results in a higher effective request rate (10rps).
rate
and highest per
from the policies applied to the key, so for the example above the key would have rate
set to 100 and per
set to 30 giving an effective rate limit of 3.33rps. This has now been corrected.HTTP 429
, is written to the sliding log in RedisHTTP 429
responses stop and traffic is resumed. This behavior is called spike arrest.
The complete request log is stored in Redis so resource usage when using this rate limiter is high.
This algorithm will use significant resources on Redis even when blocking requests, as it must
maintain the request log, mostly impacting CPU usage. Redis resource
usage increases with traffic therefore shorter per
values are recommended to
limit the amount of data being stored in Redis.
If you wish to avoid spike arrest behavior but the DRL is not suitable, you might use the Fixed Window Rate Limiter algorithm.
You can configure Rate Limit Smoothing to manage the traffic spike, allowing time to increase upstream capacity if required.
The Redis Sentinel Rate Limiter reduces latency for clients, however increases resource usage on Redis and Tyk Gateway.
threshold
) and the maximum configured request rate (rate
). As the request rate approaches the current
current allowance, Tyk will emit an event to notify you that smoothing has been triggered. When the event is emitted,
the current allowance will be increased by a defined increment (step
). A hold-off counter (delay
) must expire
before another event is emitted and the current allowance further increased. If the request rate exceeds the
current allowance then the rate limiter will block further requests, returning HTTP 429
as usual.
As the request rate falls following the spike, the current allowance will gradually reduce back to the lower bound (threshold
).
Events are emitted and adjustments made to the current allowance based on the following calculations:
current allowance - (step * trigger)
,
a RateLimitSmoothingUp
event is emitted and current allowance increases by step
.allowance - (step * (1 + trigger))
,
a RateLimitSmoothingDown
event is emitted and current allowance decreases by step
.smoothing
object alongside the standard rate
and per
parameters:
enabled
(boolean) to enable or disable rate limit smoothingthreshold
is the initial rate limit (current allowance) beyond which smoothing will be appliedstep
is the increment by which the current allowance will be increased or decreased each time a smoothing event is emittedtrigger
is a fraction (typically in the range 0.1-1.0) of the step
at which point a smoothing event will be emitted as the request rate approaches the current allowancedelay
is a hold-off between smoothing events and controls how frequently the current allowance will step up or down (in seconds).smoothing
object within access keys and policies. For API-level rate limiting, this configuration is within the access_rights[*].limit
object.
An example configuration would be as follows:
per
durationper
). Gateway and Redis
resource usage is increased with this option.
This option can be enabled using the following configuration option
enable_sentinel_rate_limiter.
To optimize performance, you may configure your rate limits with shorter
window duration values (per
), as that will cause Redis to hold less
data at any given moment.
Performance can be improved by enabling the enable_non_transactional_rate_limiter. This leverages Redis Pipelining to enhance the performance of the Redis operations. Please consult the Redis documentation for more information.
Please consider the Fixed Window Rate Limiter algorithm as an alternative, if Redis performance is an issue.
per
) duration.rate
in a window, the requests are processed without delay, until
the rate limit is reached and requests are blocked for the remainder of the
window duration.
When using this option, resource usage for rate limiting does not
increase with traffic. A simple counter with expiry is created for every
window and removed when the window elapses. Regardless of the traffic
received, Redis is not impacted in a negative way, resource usage remains
constant.
This algorithm can be enabled using the following configuration option enable_fixed_window_rate_limiter.
If you need spike arrest behavior, the Redis Rate Limiter should be used.
drl_threshold
configuration.
This threshold value is used to dynamically switch the rate-limiting
algorithm based on the volume of requests. This option sets a
minimum number of requests per gateway that triggers the Redis Rate
Limiter. For example, if drl_threshold
is set to 2, and there are 5
gateways, the DRL algorithm will be used if the rate limit exceeds 10
requests per second. If it is 10 or fewer, the system will fall back to
the Redis Rate Limiter.
See DRL Threshold for details on how to configure this feature.
rate
requests in any per
time period (given in seconds).
As explained in the Rate Limiting Concepts section, Tyk supports configuration of rate limits at both the API-Level and Key-Level for different use cases.
The API-Level rate limit takes precedence over Key-Level, if both are configured for a given API, since this is intended to protect your upstream service from becoming overloaded. The Key-Level rate limits provide more granular control for managing access by your API clients.
HTTP 429 Too Many Requests
error.
Check out the following video to see this being done.
rate=15 per=60
then a client using a key generated from that policy will be able to make a total of 15 requests - to any combination of those APIs - in any 60 second period before receiving the HTTP 429 Too Many Requests
error.
rate=5 per=60
then three clients using keys generated from that policy will each independently be able to make 5 requests in any 60 second period before receiving the HTTP 429 Too Many Requests
error.
429 Too Many Requests
error.
If no per-endpoint rate limit is defined, the endpoint will inherit the key-level per-API rate limit or the global rate limit, depending on what is configured.
global_rate_limit
field in the API definition you can specify the API-level rate limit in the following format: {"rate": 10, "per": 60}
.
An equivalent example using Tyk Operator is given below:
allowance
and rate
are set to the same value: this should be number of requests to be allowed in a time period, so if you wanted 100 requests every second, set this value to 100.
per
is set to the time limit. Again, as in the above example, if you wanted 100 requests per second, set this value to 1. If you wanted 100 requests per 5 seconds, set this value to 5.
disable_rate_limit
to true
in your API definition.
Alternatively, you could also set the values of Rate
and Per (Seconds)
to be 0 in the API Designer.
rate_limit
object to the extended_paths
section of your API definition.
The rate_limit
object has the following configuration:
path
: the endpoint pathmethod
: the endpoint HTTP methodenabled
: boolean to enable or disable the rate limitrate
: the maximum number of requests that will be permitted during the interval (window)per
: the length of the interval (window) in secondsrate_limit
objects.
GET
requests to the /anything
endpoint, limiting requests to 60 per
second.
rate_limit
array. For example, if you wanted to limit the rate of
POST
requests to your API allowing a higher rate to one specific
endpoint you could configure the API definition as follows:
POST
requests to /user/login
to 100 requests per second (rps). Any other POST
request matching the
regex pattern /.*
will be limited to 60 requests per second. The order
of evaluation ensures that the specific /user/login
endpoint is matched
and evaluated before the regex pattern.
The per-endpoint rate limit middleware allows you to enforce rate limits on specific endpoints. This middleware is configured in the Tyk OAS API Definition, either via the Tyk Dashboard API or in the API Designer.
If you’re using the legacy Tyk Classic APIs, then check out the Tyk Classic page.
operationId
defined in the OpenAPI Document that declares both the path
and method for which the middleware should be added. Endpoint paths
entries (and the associated operationId
) can contain wildcards in the
form of any string bracketed by curly braces, for example
/status/{code}
. These wildcards are so they are human-readable and do
not translate to variable names. Under the hood, a wildcard translates to
the “match everything” regex of: (.*)
.
The rate limit middleware (rateLimit
) can be added to the operations
section of the
Tyk OAS Extension (x-tyk-api-gateway
) in your Tyk OAS API Definition
for the appropriate operationId
(as configured in the paths
section
of your OpenAPI Document).
The rateLimit
object has the following configuration:
enabled
: enable the middleware for the endpointrate
: the maximum number of requests that will be permitted during the interval (window)per
: the length of the interval (window) in time duration notation (e.g. 10s)GET /status/200
endpoint, limiting requests to 60 per second.
The configuration above is a complete and valid Tyk OAS API Definition
that you can import into Tyk to try out the Per-endpoint Rate Limiter
middleware.
paths
section of the OpenAPI description. For example,
if you wanted to limit the rate of POST
requests to your API allowing a
higher rate to one specific endpoint you could configure the API
definition as follows:
POST /user/login
endpoint to 100 requests per second (rps). Any other POST
request to an
endpoint path that matches the regex pattern /{any}
will be limited to
60 rps. The order of evaluation ensures that the specific POST /user/login
endpoint is matched and evaluated before the regex pattern.
The configuration above is a complete and valid Tyk OAS API Definition
that you can import into Tyk to try out the Per-endpoint Rate Limiter
middleware.
rate_limit
for specifying a rate limit resource to use. For example, if we wanted to use our foobar
rate limit with a HTTP request:
int
1000
interval
The time window to limit requests by.
Type: string
"1s"