API Request Throttling

Ben Lambert · June 2017

API request throttling is a fairly standard practice throughout the web. Most SaaS companies that provide web APIs do not permit unlimited API use. Some companies throttle requests to protect their backend resources where load and throughput are important metrics of service delivery, while others throttle requests as part of a broader business model where higher throttles are available at cost. Some companies even throttle the fidelity of API requests and only provide some functionality with elevated subscriptions.

At Blackbaud, the SKY API employs two forms of API request throttling: rate-limits and quotas.

We use a rate-limit throttle to protect our backend servers from being overloaded with too many requests during sporadic moments of heavy API traffic in a short period of time (seconds). We use a quota throttle to enforce an appropriate level of sustained API traffic over a broader period of time (daily). Taken together, these throttling measures protect our backend resources from problems caused by heavy traffic, runaway code, infinite loops, inappropriate usage, etc. They allow us to ensure that we can provide a good API experience for all consumers.

The throttling thresholds are based on your SKY API subscription level. Currently, we offer two flavors of the SKY API:

A Standard Edition tier for general developers and customers
A Partner Edition tier for Partner-level developers who build complementary products that integrate with Blackbaud offerings

The use cases for the two tiers are different: One is for developers building small-scale apps (from a distribution perspective) and for general customer access to Blackbaud data, while the other is for official Blackbaud Partners building full solutions that many Blackbaud customers can use. As such, the Partner-level tier provides higher thresholds for both rate-limit and quota throttles, which makes sense because the API traffic profile of a Partner application is different than the profile of a customer or small-scale application.

You can view your API subscription level on the developer profile page of the SKY API developer portal. For example, the screenshot shows that I have a subscription to the SKY API Standard Edition tier.

For standard subscriptions, the threshold for the rate-limit throttle is currently 5 calls/second with a quota threshold of 25K calls/day. For Partner subscriptions, the thresholds are similar but are applied on a per-application, per-customer basis. This allows Partner developers to build and sell applications to as many customers as possible without each new customer lowering the threshold for existing customers.

For both standard and Partner editions of the SKY API, we offer paid upgrade tiers that provide even higher quota throttling characteristics (for example, 100K calls/day or 250K calls/day). To upgrade your subscription, you can submit an upgrade request in the developer portal: https://developer.sky.blackbaud.com/usage-increase. And to join the Partner program, you can find more details here:
https://www.blackbaud.com/partners/become-a-partner

When you reach a throttling threshold, you receive a response from the SKY API with details about how long to wait before making additional API calls. In general, you should code your app to be robust in the face of throttling. For example, if you hit the rate-limit threshold and receive a "429 Too Many Requests" response, you might implement a 1-second delay before retrying your API request. Here’s a sample response that you might receive when you encounter a rate-limit throttle:

Retry-After: 1
Date: Mon, 15 May 2017 19:47:44 GMT
Content-Length: 83
Content-Type: application/json

{

"statusCode": 429,

"message": "Rate limit is exceeded. Try again in 1 second."
}

Note the Retry-After header in the response above. This value indicates how many seconds your app should wait before making additional API requests.

As an example, we've updated our Barkbaud sample application to handle 429 error responses. We recommend wrapping your calls through a proxy that handles the possibility of getting 429 error responses. As previously mentioned, the response includes a Retry-After header when you get a 429. The value of this header is given in seconds, and we can use it to retry a request that hit a rate-limit error. Here is the code in our sample app that handles 429 responses:

1e1d490118c4a59b8ac653d6010f1bdd-huge-ba

The code wraps the call to the API with a catch, and if the status code on the error is a 429, it waits the number of seconds provided in the Retry-After header and then attempts to make another request. This allows for a seamless experience and makes it much easier to write your application knowing that 429s are all automatically handled by this proxy.

For more information about rate-limits, quotas, and subscriptions, take a look at the content on the following links:

As always, we welcome your feedback on this, and please let us know if you have any further questions. Thanks!

API Request Throttling

Categories