Rate Limit Speed Using REST API
Learn how to configure the rate limit speed for migration jobs using the DryvIQ REST API.
Written by Andrea Harvey
Updated at May 2nd, 2025
Table of Contents
Parmeters
Three parameters can be configured to define the minimum and maximum operations per second against a connection, and how quickly these rates increase when rate limit exceptions stop occurring. These parameters can be applied globally by platform or to individual connections.
| Options | Behavior |
|---|---|
| min_rate | This is the minimum number of operations per second that can be performed on a connection. This is the absolute floor for a rate limiter. The default is 1, which means that the minimum request rate will be one operation every second. |
| max_rate | This is the maximum number of operations per second that can be performed on a connection. This is the absolute ceiling for a rate limiter as well as the the starting rate used when the connection is first opened. The default is 60, which means that the maximum request rate will not exceed 60 operations per second. |
| recovery_scale |
Every rate limiter starts out with a maximum number of operations per second that can be performed on the connection. Every time a rate limit exception is encountered, the rate will be decreased. As time goes on without any exceptions, speed will attempt to be recovered by increasing the rate.
The longer that the rate limiter is active, the longer it takes for this recovery process based on the assumption that we are closer to zeroing in on the true rate that the connection can tolerate and shouldn't need to make many modifications. We use a scaling constant as one of the inputs that determine how fast the rate limiter will recover. The initial value of that scaling constant is 500 and will double with every rate limit exception until we hit a defined maximum.
This parameter defines the scaling maximum and effects how fast a rate limiter will recover. The lower the number, the faster that a rate limiter will recover but could also cause more rate limit exceptions. The default value is 100,000 and is in no particular units. |
Global Configuration
To configure these parameters globally, add the following to the appSettings.json file found in the directory where DryvIQ was installed. Modify the min_rate, max_rate, and recovery_scale to reflect the desired values.
{
"box": {
"rate_limit": {
"min_rate": 15,
"max_rate": 75
}
}
}Configure by Connection
To configure these parameters for a specific connection, follow these steps:
- Create the connection using the connector-specific authentication flow.
- Using Postman or another REST client and the ID of the connection created in step one, make the request GET /v1/connections/<id>?include=auth.
- Finally, make the following request using the response body from step 2 and the applicable new rate_limit parameters added to the auth section of the request body.
PATCH /v1/connections/<id>{
"auth": {
...all previous parameters
"rate_limit": {
"min_rate": 15,
"max_rate": 75
}
}
}Rate Limiting and 429 - TooManyRequests Responses
DryvIQ will respect any 429 response from an API that contains a “Retry-After” header with a time value. In this case, the application will wait for the given amount of time before retrying the request. In most cases, the subsequent request should succeed. In cases where multiple 429 responses are received, the rate-limiting algorithm will take effect to throttle the number of operations per second.