Introduction: Why the Client Needs Its Own Rate Limit

Imagine walking into a bank with a single open teller window and a queue of a thousand people behind you. You can shove, you can shout, but the window's throughput won't budge. Your only real options are to arrive at a sensible pace or get turned away. The server-side rate limit is that teller who eventually just stops serving you. And the client-side rate limit is your own decision not to walk up to the window more often than it can physically handle.

This article is entirely about algorithms. We deliberately won't rehash how to handle a 429 status code or set up exponential backoff, and we won't discuss how many threads to run. The focus here is different: how to figure out, before you ever send a request, whether you can send it right now — and if not, how long to wait. That math is what separates a client that "sometimes gets rejected" from one that works with the same API for years without irritating its owners.

Here's what you'll learn:

  • how the four families of algorithms work — fixed window, sliding window, token bucket, and leaky bucket — and how they differ in behavior, not just in name;
  • why token bucket and GCRA (a leaky bucket variant) became the de facto standard, and how to implement them in an evening without rounding errors;
  • how to build a limit hierarchy: a global budget, per-host, per-endpoint, per-auth-key, and per-outbound-IP budgets;
  • how to bake IP rotation into the limiter model without breaking either server-side quotas or your own telemetry;
  • which design mistakes look harmless but lead to infinite waits, "bursts" at the start of every minute, and memory leaks.

We'll speak like an engineer-mentor: with formulas but without the mumbo-jumbo, with examples but no fluff. If you write parsers, SDKs for third-party APIs, marketplace integrations, or aggregators hitting hundreds of hosts, this guide deserves a bookmark.

Fundamentals: What a Client-Side Rate Limit Is and Why the Server Alone Isn't Enough

Two Views of One Problem

A server-side rate limit protects the API owner's infrastructure: it counts your requests and refuses when the quota runs out. A client-side rate limit solves a different problem — not letting it get to a refusal. The difference is fundamental. The server reacts after the fact: the request is already gone, the channel is occupied, the log is polluted, and your client's reputation has taken a small hit. The client acts preemptively: it knows (or estimates) its budget and spends it carefully.

Why can't you just rely on server rejections and retry? Because every rejected attempt wastes a connection, wastes an auth token, adds a line to someone else's security log, and typically worsens your standing with adaptive defense systems. A client that paces itself looks to the server like a well-mannered partner. A client that "keeps trying until it gets in" looks like a source of trouble. Guess which one gets clamped harder when the policies are next reviewed?

Basic Terminology

  • Rate — the steady-state request frequency, e.g. "ten requests per second." That's what the server is willing to accept indefinitely.
  • Burst — how many requests you can fire off back-to-back without waiting for replenishment. Servers often tolerate short bursts as long as the average rate holds.
  • Window — the time span over which requests are counted: a second, a minute, an hour, a day.
  • Quota — the maximum number of requests in a window. Quota divided by window length is the average rate.
  • Limit key — the attribute by which the server groups requests: IP address, token, account, "host plus endpoint" pair. A single client almost always falls under several keys at once.
  • Cost — the weight of a request. Some APIs count a search as ten units and a card lookup as one.

The Core Principle: the Limiter Decides "When," Not "How Many"

A common misconception is confusing a limiter with a semaphore. A semaphore caps the number of concurrent operations and answers "how many are in flight right now?" A limiter caps the frequency and answers "can I start one more operation right now, and if not, how many milliseconds until I can?" These mechanisms complement each other but aren't interchangeable. You can hold a single connection and still exceed the limit if requests are short. You can hold a hundred connections and still respect the limit if requests are slow.

Three Qualities of a Good Client-Side Limiter

  1. Predictability. Same input, same answer. No "it sometimes lets a little extra through."
  2. Cheapness. The decision takes constant time and constant memory per key. With thousands of hosts, the limiter can't store the history of every request.
  3. Fairness. If several tasks are waiting for permission, they get it in queue order, not in "whoever happened to wake up luckiest" order.

Keep these three criteria in mind: they're how we'll compare algorithms in the next section.

Deep Dive: Algorithm Families and Their Math

Fixed Window — Honest but Tricky

The simplest option: split time into fixed-length windows, keep a counter per window, and when the quota is hit, wait for the next window to start. Memory: one number and one timestamp per key. Sounds ideal.

The problem is the window boundary effect. With a quota of one hundred requests per minute, the client can send a hundred requests in the last second of one window and another hundred in the first second of the next. From the client's perspective everything is legal: exactly one hundred per window. From the server's perspective two hundred requests arrived in two seconds — double any sensible reading of the limit. If the server counts on a sliding window (as most modern ones do), you'll get rejected while genuinely believing you're innocent.

The second issue is burst synchronization. All your workers wake up at the start of the minute at the same time, because that's when the counter resets. This spikes load on the network, on the proxy, and on the target server.

Sliding Window Log — Precise but Costly

Store the timestamps of every request in the last interval. Before sending, discard stale ones and count the rest. Perfectly accurate accounting, no boundary effect. But memory grows linearly with the quota: a quota of ten thousand requests per hour means storing ten thousand timestamps per key. Tolerable for one host, not for a thousand.

Sliding Window Counter — A Compromise

Keep two counters: the current window's and the previous window's. The load estimate is the current counter plus the previous one multiplied by the fraction of the previous window that still falls inside the sliding interval. If 20 percent of the current minute has passed, take 80 percent of the previous counter. It's an approximation, but it smooths the boundary nicely and needs only two variables. Many server-side implementations use exactly this, so mimicking it on the client is a good way to "think like the server."

Token Bucket — A Bucket of Tokens

Picture a bucket holding B tokens. Each second, r tokens drip in, and the excess spills over the edge. To send a request costing c, you take c tokens. If there aren't enough, you wait until they accumulate.

Two parameters give you two independent properties. The r parameter is the steady-state rate: no matter how long you wait, the long-term pace won't exceed r requests per second. The B parameter is the maximum burst: after idling, you can immediately send B requests. This maps perfectly onto how real server policies are worded: "no more than one hundred per minute, no more than ten at a time."

The key implementation detail is lazy replenishment. You don't need a background thread dripping tokens every second. At each call, just compute how much time has passed since the last call and add the corresponding amount: tokens = min(B, tokens + elapsed * r). Two numbers per key, constant-time computation. Fully satisfies the cheapness criterion.

Leaky Bucket — A Bucket with a Hole

Here the metaphor is reversed: requests pour into a bucket and drain out through a hole at the bottom at a constant rate r. If the bucket overflows, the new request is rejected or waits. The output is always uniform: no bursts, strictly one request every 1/r seconds. If the server has no tolerance for bursts, or you want the smoothest possible traffic profile, this is your choice.

Leaky bucket comes in two interpretations. The first is as a queue: requests are literally buffered and a separate process releases them on a timer. The second is as a counter: we simply check whether the imaginary bucket would overflow, and either let the request through or not. In telecom, the second interpretation is called GCRA (Generic Cell Rate Algorithm) and is mathematically equivalent to token bucket with different parameterization. That's an important observation: if you've already implemented token bucket, you know how to do leaky bucket too.

GCRA: One Timestamp Instead of a Counter

GCRA stores a single value — the TAT (theoretical arrival time): the moment before which the next "ideal" request should not arrive. Parameters: emission interval T = 1/r and tolerance τ equal to (B − 1) * T. The algorithm:

  1. Get the current time, now.
  2. If TAT is less than now, set TAT to now (the bucket has long been empty).
  3. If TAT − τ is greater than now, the request came too early; the wait time is TAT − τ − now.
  4. Otherwise, allow the request and move TAT forward by T.

One number per key, no floating-point counters that gradually drift apart. That's why GCRA is the favorite for distributed limiters: its state is easy to store in a single key-value field and update atomically.

Comparative Summary in Words

  • Fixed window: minimal memory, low accuracy, bursts at boundaries, only good as a rough safety net.
  • Sliding log: absolute accuracy, memory linear in quota, suitable for a few keys with small quotas.
  • Sliding counter: memory of two variables, approximate accuracy, good at mimicking server logic.
  • Token bucket: memory of two variables, explicit burst control, the standard for client SDKs.
  • Leaky bucket / GCRA: memory of one variable, perfectly even output, the standard for distributed systems.

A Word on Clocks: Monotonicity and Drift

All algorithms rely on "the current time." Use a monotonic clock, not wall-clock time. Wall-clock time can jump backward during synchronization, and your limiter will think a negative number of seconds has passed, reset the replenishment, or, conversely, produce a massive burst. Monotonic clocks are guaranteed never to go backward. The only exception is a distributed limiter in shared storage, where time is taken from the storage side rather than from the clients, so it doesn't depend on clock skew between machines.

Method 1: Token Bucket in Practice — Implementation Without Pitfalls

Step 1. Choosing Parameters

Translate the server policy into the pair (r, B). If the docs say "one hundred requests per minute," the base rate is r = 100/60 tokens per second. Capacity B is a separate decision. A safe choice for an unfamiliar API is B equal to five to ten percent of the window quota — five to ten tokens at a quota of one hundred. This lets you process a small batch quickly without creating spikes that a sliding-window server would read as a violation.

Even better: count with a margin, using 80–90 percent of the declared rate. Server counters, network latency, and rounding on both sides inevitably introduce error, and the margin absorbs it.

Step 2. State Structure

Per key, store: tokens (a floating-point value, the current token count), last (the monotonic timestamp of the last recalculation), rate, capacity. Protect the state with a mutex or, in async code, with a single arbiter task through which all token requests pass.

Step 3. The acquire(cost) Algorithm

  1. Acquire the lock.
  2. now = monotonic(); elapsed = now − last; last = now.
  3. tokens = min(capacity, tokens + elapsed * rate).
  4. If tokens ≥ cost: tokens −= cost; release the lock; return "zero wait."
  5. Otherwise: wait = (cost − tokens) / rate; deduct into the negative: tokens −= cost; release the lock; return wait.
  6. The caller sleeps for wait seconds and then sends the request.

Note point five. Deducting into the negative is a reservation. It guarantees fairness: the next call sees a negative balance and gets an even larger wait, i.e. it queues behind the previous one. Without reservation you get the classic bug: ten tasks simultaneously see zero tokens, all compute the same wait, all wake up at once, and produce the very burst the limiter was meant to prevent.

Step 4. Bounding Reservation Depth

Reserving into the negative can run away to infinity: if tasks arrive faster than tokens replenish, the waiting queue grows and the last one's wait approaches hours. Introduce a maximum delay (max_delay). If the computed wait exceeds the threshold, don't reserve; return a "limiter overloaded" error. Let the scheduler decide what to do: postpone, deprioritize, switch to another host. This is the point where the client limiter stops being a passive brake and becomes a load-management tool.

Step 5. Weighted Cost

Let requests have different costs. Search endpoints, bulk selects, and write operations are usually "more expensive" for the server, and many APIs reflect that in their policies. If the docs are silent, assign weight based on your own observations of response times: a request the server takes longer to process is reasonably counted as more expensive. The wait formula doesn't change — cost just factors into it.

Step 6. Warm-up and Cold Start

The initial value of tokens is a separate decision. A full bucket at startup means an instant burst of B requests the moment a process launches. That can be undesirable if dozens of processes start simultaneously after a deploy. Safer to start from zero or half capacity. And if the server is known to count the quota by a window not tied to your start time, a zero start is also more honest.

Token Bucket Checklist

  • Monotonic clocks are used.
  • Replenishment is lazy, based on time difference, with no background timers.
  • Reservation into the negative gives a fair queue.
  • There's a max_delay and an explicit rejection on overflow.
  • The rate is set with a margin over the server's.
  • Capacity is limited to a sensible fraction of the window quota.
  • Cost is supported for weighted requests.
  • The initial state doesn't create a burst on deploy.

Method 2: Leaky Bucket and GCRA — When You Need an Even Flow

When to Prefer Leaky Bucket

Token bucket allows bursts. That's convenient, but there are situations where a burst is harmful in itself: the server reacts aggressively to short spikes, the proxy channel has limited bandwidth, or you want your traffic pattern to look like that of an ordinary, unhurried client. In all these cases you need output with a strictly constant interval between requests. That's leaky bucket.

The Queue Implementation

The classic variant: a fixed-capacity buffer and a single consumer that pulls an element every T seconds and sends it. Pros — simplicity and clarity. Cons — you need a separate thread or task, and if the consumer stops, the buffer "freezes." This approach is appropriate when you already have an explicit pipeline: a task queue from which requests are pulled one at a time.

The GCRA Implementation

For most client scenarios, the counter interpretation is more convenient. We described the algorithm above; let's break down how to set the parameters. If a strict interval is required, set τ = 0: then the tolerance is zero and requests pass no more often than once every T. If you want to allow a small burst, increase τ. Thus a single implementation covers both strict leaky bucket and token bucket. That's precisely why GCRA is a good "unified engine" for a rate-limiting library.

Step-by-Step GCRA Procedure with Reservation

  1. Read TAT for the key; if it doesn't exist, set TAT = now.
  2. new_TAT = max(TAT, now) + T * cost.
  3. allow_at = new_TAT − τ − T * cost; this is the moment when a request of this cost could be sent without waiting.
  4. If allow_at ≤ now: write new_TAT, return "zero wait."
  5. Otherwise: wait = allow_at − now; if wait exceeds max_delay — return a rejection without writing; else write new_TAT and return wait.

In distributed storage, steps one through five are performed in a single atomic operation — for example, a script on the storage side. This guarantees that two clients can't read the same TAT and "duplicate" the permission.

Equivalence and Choosing by Traffic Character

It's useful to understand: token bucket with parameters (r, B) and GCRA with parameters (T = 1/r, τ = (B − 1)/r) let through exactly the same sequences of requests. The only difference is the state representation. So the question "which is better" is really: do you need a burst? Yes — set B greater than one. No — set B = 1 (τ = 0). The rest is a matter of taste and storage convenience.

Leaky Bucket / GCRA Checklist

  • You've decided whether you need a burst or a strictly even flow.
  • The state is a single TAT value per key.
  • The read-check-write operation is atomic.
  • There's a max_delay and an explicit rejection.
  • In the distributed variant, time is taken from the storage side.

Method 3: Per-Host Limits — Registry, Hierarchy, and Composition

Why "One Limiter for Everything" Doesn't Work

If your client only talks to one API, a single bucket is enough. But aggregators, parsers, and integration buses reach out to dozens or hundreds of hosts, each with its own policy. A single global limit either underutilizes fast hosts or overloads slow ones. You need a limiter registry: a map of "limit key — bucket state," where the key is built from the request's characteristics.

The Limit Key: From Host to Endpoint

The minimal key is scheme plus host plus port. Two domains of the same owner often share a quota, while subdomains may count separately; you need to learn this from the docs or by observation. A finer key is host plus path template: for example, a search endpoint and a product card endpoint have different quotas. Finer still is host plus auth token, if the server counts the quota per API key and you have several.

Rule of thumb: build the key from the same attributes the server counts by. If the server counts by IP, your outbound IP goes into the key. If by token, the token does. If by both at once, you need two limiters, and a request must pass through both.

The Limit Hierarchy

A well-designed system layers its limiters:

  1. Global — the overall budget of the process. Protects your own resources: sockets, memory, bandwidth.
  2. Per outbound IP — how many requests leave through a specific address. The key level when working through a pool of addresses.
  3. Per host — the target server's policy as a whole.
  4. Per endpoint — separate quotas for expensive methods.
  5. Per credentials — quotas per token or account.

A request must get permission at every level. That's limiter composition.

Composition Without Deadlock

The naive approach is to call acquire on each limiter sequentially and sum the waits. It's wrong: the first limiter has already deducted a token, while the second makes you wait ten seconds, during which the first limiter's token is "stale" from a fairness standpoint. Worse, two tasks acquiring limiters in different orders can block each other indefinitely.

The correct procedure:

  1. For each limiter, compute the wait without deducting (a peek operation).
  2. Take the maximum wait — that's the overall wait.
  3. If the wait exceeds max_delay — reject.
  4. Otherwise, deduct in all limiters, passing them the same arrival time, now + wait, so each reserves its resource for the same moment.
  5. Sleep for wait and send.

Always walk the limiters in the same fixed order (e.g. from global to narrowest). This prevents deadlocks under concurrent access.

Configuration and Defaults

The registry must be able to create a limiter for a host it hasn't seen before. Set a conservative default: low rate, minimal burst. For known hosts — explicit configuration. Good practice is to keep the policy in a declarative form: host or pattern, rate, burst, cost by path template, a "count by IP" flag, and a "count by token" flag. Then when the server's policy changes, you edit one file instead of the code.

Eviction and Leaks

A registry with thousands of hosts is a memory leak candidate. Hosts that haven't been hit in a long time must be evicted. Since a bucket's state after a long idle period is "full" anyway (or its TAT is in the past), eviction loses no information: on the next request the limiter is recreated in the same state. Use an LRU cache with a size cap, or periodically purge entries whose last is older than several windows.

Per-Host Limits Checklist

  • The limit key mirrors the attributes the server counts by.
  • There are levels: global, IP, host, endpoint, credentials.
  • Composition goes peek — maximum — single deduction.
  • Limiters are walked in a fixed order.
  • Conservative defaults for unknown hosts.
  • Policies are externalized into configuration.
  • The registry is size-capped and evicts inactive keys.

Method 4: The Limiter and IP Rotation — How to Account for Address Changes in the Model

Why Rotation Breaks the Naive Limiter

If the target server counts the quota by IP, then changing the address changes the limit key on the server side. A naive client that keeps one counter per host will keep throttling after the IP change, even though the server now sees a "new" visitor. The reverse mistake is more dangerous: the client resets all limits when the IP changes, including those the server counts by token or account — and gets rejected on a quota that has nothing to do with the IP. The takeaway: rotation should only affect the "per outbound IP" level of the hierarchy; the other levels go on with their own lives.

Two Modes of Address Change

In Proxeon infrastructure, the IP on a port changes either on a timer or via a link/API (as of 22.09.2026). These are two fundamentally different scenarios for the limiter. With timer-based changes, the client doesn't control the moment: the address changes "from outside," and the per-IP limiter must learn about it — via an event, by polling the current address, or on a schedule if the interval is known. With link-based changes, the client initiates the switch itself and can synchronize it with its own state: wait for in-flight requests to finish, record the moment, and create a new "host plus IP epoch" key.

How common is each mode? As of 14.11.2026, timer-based IP changes account for 51.1%, and the rest fall to link/API. So both strategies occur roughly equally, and a universal limiter must support both.

The Address Epoch as Part of the Key

Instead of storing the IP itself in the key (which you may not know in advance), use an epoch — a counter incremented on every change. The "per IP" level key becomes the pair "port plus epoch." When the epoch changes, the old key simply stops being used and is later evicted by the registry, and a new one is created in the initial state. Note that the initial state must again be conservative. A new address is a stranger to the server, and the first thing it sees is your traffic pattern.

Aligning the Limiter Window with the Rotation Timer

As of 14.11.2026, the median interval between timer-based IP changes is 120 s, and 78.1% of timer intervals are exact multiples of a minute. That's very convenient for design. If you have a per-host quota of "N requests per minute per IP" and an address lives two minutes, the budget for one address is exactly two windows. The per-IP limiter can run as a token bucket with rate N/60 and a capacity chosen so that 2N isn't exceeded over the address's lifetime. The key is not to "catch up" on missed requests: if you sent little in the first minute, you can't send twice as much in the second, because the server counts by the minute window, not by your epoch.

Pause During the Switch

An address change is not instantaneous. As of 14.11.2026, the average duration of the change itself — from command to new address — is 4.56 s, and 80.8% of changes finish within 5 seconds. What does this mean for the limiter? During the switch, requests either go through the old address or don't go at all. The right strategy is to treat the switch as a short "drain" interval: before the change command, the limiter stops issuing new permits, waits for in-flight requests to finish, performs the switch, then opens a new epoch. Since four out of five switches complete within five seconds, it's sensible to set the wait-for-new-address timeout a bit above that mark and fall into error handling if it's exceeded.

Rate-Limit the Change Command Itself

Here's an insight that often gets overlooked: the IP change command is also a request to an API, and it needs limiting too. As of 14.11.2026, the share of duplicate records for link-based changes (interval of 2 s or less, same IP) is 1.2%. These are traces of situations where the client pulled the link twice in a row: due to a race between workers, a retry on timeout, or the absence of a single decision point. The fix is a separate bucket with capacity one and a strict interval (a pure leaky bucket with τ = 0) on the rotation command, plus deduplication: if a change is already underway, the other workers wait for its result rather than starting a new one. The same principle as for target hosts, just applied to your own infrastructure.

Scale and Budget Planning

To get a sense of scale: as of 14.11.2026, the entire Proxeon network performed 2,196,242 IP changes per day, involving 7,924 proxy ports, and the median number of unique IPs per modem per day is 95. For a limiter designer, this means one port passes through dozens of address epochs per day, and the "per IP" state must be lightweight and quickly evictable: storing the history of all epochs makes no sense, but guaranteeing a constant cost for creating a new epoch is essential.

Protocols and Port Addressing

A small detail that simplifies configuration: in Proxeon, the SOCKS5 port equals the HTTP port plus one (as of 22.09.2026). If you use both protocols for the same channel, the per-IP limiter should count them as one key, not two: they share an address, so they share a budget. Write normalization into the registry: the key is built from the HTTP port, and the SOCKS5 port is mapped to it by subtracting one.

A Distributed Per-IP Limiter

When several processes or machines go through one port, the bucket must be shared. Use storage with atomic operations, store the TAT (GCRA) under the key "port plus epoch plus host," update it with a script on the storage side, and take the time from there too. Store the epoch centrally as well and change it atomically together with the rotation command — then all processes transition to the new key in a coordinated way.

Limiter Checklist Under Rotation

  • Rotation only affects the "per IP" level; the other levels aren't reset.
  • Both modes are supported: timer and link/API change.
  • The IP-level key includes the address epoch.
  • A new epoch starts in a conservative state.
  • The per-IP rate is aligned with the server window, not the epoch length.
  • Permit issuance pauses during the switch.
  • The rotation command itself is rate-limited and deduplicated.
  • HTTP and SOCKS5 ports of the same channel are normalized into one key.
  • For multi-process scenarios, state lives in shared storage.

Typical Client-Side Rate Limit Design Mistakes

Mistake 1. Sleep Instead of a Limiter

"Let's put a hundred-millisecond pause between requests" is the most common solution and the most fragile. It ignores the request's own duration, doesn't work with several workers, doesn't allow bursts where they're permitted, and doesn't protect against them where they're forbidden. A pause isn't a limiter — it's an imitation of one.

Mistake 2. Counting by Window When the Server Counts Sliding

The window boundary effect is covered above. If you're unsure how the server counts, assume the worst — a sliding window — and use token bucket with a modest capacity.

Mistake 3. Racing Without Reservation

Several tasks compute the same wait simultaneously, sleep, and wake up together. The result is exactly the burst the limiter was meant to prevent. The cure is reservation (deducting into the negative or shifting the TAT) under a lock.

Mistake 4. An Infinite Waiting Queue

Without max_delay, the queue grows unbounded and tasks wait minutes and hours while holding memory and connections. A limiter must be able to refuse.

Mistake 5. System Clocks

One time jump during synchronization — and the bucket either overflows or "freezes." Monotonic clocks on the client only; storage time only in the distributed variant.

Mistake 6. Resetting All Limits on IP Change

Quotas per token and per account don't depend on the address. Reset them and you immediately violate a policy that had nothing to do with you.

Mistake 7. A Full Bucket at Startup

Dozens of processes start with full buckets after a deploy and burst all at once. Start with an empty bucket or add a random start delay.

Mistake 8. A Registry Without Eviction

Every new host — a new permanent entry. After weeks of operation, the process holds limiters in memory for hosts it will never need again.

Mistake 9. Ignoring Request Cost

If the server counts a search as ten units and you count it as one, your limiter lets through ten times more than it should and is genuinely surprised by the rejection.

Mistake 10. An Unbounded Rotation Command

Retrying the address-change command on timeout or from several workers leads to duplicates and extra switches. The command is a resource too; give it a bucket.

Tools and Resources for Implementation

Ready-Made Libraries by Ecosystem

  • Go: the golang.org/x/time/rate package — the reference token bucket implementation with Allow, Reserve, and Wait methods. Reserve returns an object with the wait time and a cancellation option — exactly what's needed for composition.
  • Python: the limits library provides fixed window, sliding window, and GCRA with various backends; aiolimiter implements leaky bucket for asyncio; pyrate-limiter supports multiple windows at once.
  • Java/Kotlin: Bucket4j — a mature token bucket with support for distributed storage; Resilience4j RateLimiter — a variant with fixed refresh cycles.
  • JavaScript/TypeScript: bottleneck can rate-limit by groups of keys and cluster via shared storage; p-throttle — a minimalist option for simple cases.
  • Rust: governor — a GCRA implementation with and without keys, suitable for async code.
  • .NET: the built-in System.Threading.RateLimiting namespace with token bucket, sliding window, and partitioned limiters by key.

Storage for Distributed State

Redis and compatible stores with server-side scripts are the standard for shared GCRA: one atomic operation, time taken via the store's command, keys with TTL automatically evicted. For scenarios within a single application cluster, built-in mechanisms like distributed caches with atomic operations will do.

Observability Tools

A limiter without metrics is a black box. Export: the number of permits without waiting, the distribution of wait times, the number of rejections by max_delay, the current bucket fill by key, and the frequency of address epoch changes. A per-host wait histogram instantly shows where the policy is too strict and where you're on the edge.

Address Infrastructure

Scenarios with rotation will need a provider that offers both modes of address change — timer and link/API — and predictable port addressing. In Proxeon both modes are available, and the SOCKS5 port sits exactly one above the HTTP port (as of 22.09.2026), which simplifies key normalization in the limiter registry.

What to Read

  • The GCRA description in telecom specifications — the primary source for leaky bucket as a counter.
  • The x/time/rate package docs — an exemplary statement of the semantics of Reserve and Wait.
  • The rate limit sections in the docs of major public APIs — all counting models show up there, and it's the best way to learn how to translate a policy into bucket parameters.

Cases and Application Scenarios

Scenario 1. A Price Aggregator Across Hundreds of Hosts

Goal: regularly crawl product cards across many marketplaces, each with its own policy. Solution: a limiter registry keyed by "host plus path template," a conservative default for new marketplaces, and explicit policies for known ones. Composition with a global limit on the number of sockets. The result such systems aim for: even load across all hosts with no idle time on fast ones and no overload on slow ones; when a marketplace's policy changes, one config line gets edited. The main lesson — moving from a single global limit to a hierarchy sharply cuts the share of requests sent "for nothing."

Scenario 2. An SDK for a Partner API with Per-Token and Per-IP Quotas

Goal: a library for clients who have several tokens and several outbound addresses. Solution: two limiter levels — per token and per address — composed via peek and a single arrival time. The scheduler picks the "token plus address" pair with the smallest wait. Lesson — the limiter becomes a routing tool: instead of "waiting," it tells you "where to send so you don't have to wait."

Scenario 3. A Parser on Mobile Addresses with Timer-Based Rotation

Goal: crawling public pages through a port pool, with addresses changing on a timer. Solution: a per-IP limiter keyed by "port plus epoch," with the rate aligned with the target host's minute window. Given that as of 14.11.2026 the median timer interval is 120 s and 78.1% of intervals are multiples of a minute, the budget per epoch is computed as a whole number of minute windows. During the switch — and 80.8% of switches complete within 5 seconds (as of 14.11.2026) — permit issuance is paused with a timeout just above that threshold. Lesson — aligning the limiter window with the rotation rhythm eliminates "catch-up" bursts at the start of a new epoch.

Scenario 4. Workers with Link-Based Rotation and Duplicate Commands

Goal: several workers share one port and each initiates an address change as needed. The symptom — traces of repeated commands in the logs. Solution: a single rotation point with a leaky bucket (τ = 0) and deduplication: if a change is already underway, the others wait for it to finish. As of 14.11.2026, the share of duplicate records for link-based changes across the network is 1.2%; these are exactly the situations a single decision point eliminates. Lesson — you need to rate-limit not only requests to third-party APIs but also commands to your own infrastructure.

Scenario 5. A Single GCRA Engine for All Levels

Goal: simplify a codebase where fixed window, sleep pauses, and hand-rolled counters historically coexisted. Solution: one GCRA implementation with (T, τ) parameters, applied at every level of the hierarchy, with state stored in shared storage for distributed keys and in memory for local ones. Lesson — one formula replaces a zoo of solutions and makes the system's behavior predictable.

FAQ: Answers to the Tricky Questions

What fundamentally distinguishes token bucket from leaky bucket if they're equivalent?

They're equivalent as mathematical objects: with the right correspondence of parameters, they pass exactly the same sequences. The difference is in the intuition and in which parameter you control directly. Token bucket thinks in terms of "rate and burst," leaky bucket in terms of "interval and tolerance." For policies of the form "N per minute, no more than M at a time," the first is more convenient; for "exactly one request every T," the second. GCRA merges both intuitions into a single state.

How do I choose bucket capacity if the server doesn't publish an allowed burst?

Start with a small fraction of the window quota — around five to ten percent — and observe. If there are no rejections and wait times are large, carefully increase it. Remember that a sliding-window server won't tolerate bursts exceeding the quota over a short subinterval, so capacity should never approach the full quota.

Do I need a limiter if I have only one thread?

Yes. A single thread with short requests easily exceeds the quota: if the response comes back in twenty milliseconds, you'll send fifty requests per second without even thinking. A limiter is needed regardless of thread count; it solves a different problem than a semaphore.

How do I rate-limit requests with different costs if the server counts them differently?

Pass cost into acquire. Tokens are deducted proportionally, and the wait is computed by the same formula. If the cost is unknown, estimate it from response time or response size and refine it as you observe. Better to overestimate than underestimate.

What if one request must pass through five limiters, each giving a different wait?

First ask each for its wait without deducting, take the maximum, then deduct in all for the same arrival time. Always walk the limiters in the same order. If the maximum exceeds the threshold — reject without deducting, otherwise resources will be reserved "for nothing."

How should the limiter react to a timer-based IP change if I don't know the exact moment?

Use an event or poll the current address, and if the interval is known, a schedule with a margin. In Proxeon, both timer-based and link/API changes are standard modes (as of 22.09.2026), and if you need precise synchronization, the link-based mode is preferable: then you determine the moment of the change and open a new epoch in your registry exactly when the switch is confirmed.

Why can't I reset all limits after an address change?

Because the server may count the quota by several attributes at once. The address changed — the per-IP quota restarts, but the per-token, per-account, or per-session quota keeps counting. Reset everything and you'll violate the policies that don't depend on the address. Rotation should only touch the "per IP" level of the hierarchy.

What's better for a distributed limiter — token bucket or GCRA?

GCRA. It has a single state value, which is easier to update atomically and easier to store with a TTL. Token bucket needs two values and slightly more complex replenishment logic, which complicates the atomic script. Semantically both give the same result.

How do I verify the limiter works correctly before going to production?

Write a test with substitutable clocks: feed in requests at given moments and compare the decisions with a reference sequence. Check the edge cases: zero time, long idle, many requests arriving simultaneously, a cost greater than capacity (there should be an explicit rejection, not an infinite wait). Separately test composition: two limiters with different rates and a check that neither lets through more than its own quota.

Conclusion: What to Do Next

A client-side rate limit is not a brake but a control system. A well-built limiter does three things: it keeps you from getting rejected, it uses the available budget evenly, and it tells the scheduler where and when to send the next request. At its core there's just one formula, whether token bucket or GCRA, but it needs to be surrounded by discipline: monotonic clocks, reservation under a lock, a maximum delay, a key hierarchy, and eviction.

Quick summary:

  • Fixed window is simple but tricky at boundaries; prefer token bucket or GCRA.
  • Token bucket gives separate control of rate and burst; GCRA does the same in a single state number.
  • The limit key mirrors the attributes the server counts by; limiters are layered into a hierarchy and composed via peek — maximum — single deduction.
  • Address rotation changes only the "per IP" level; the rotation command deserves its own bucket.
  • Wait and rejection metrics are the only honest way to know how well your policy matches the server's.

Your next steps: pick a single engine (we recommend GCRA), describe the policies of known hosts in configuration, add a conservative default, enable metrics, and run tests with substitutable clocks. If you work through a pool of addresses, add the epoch to the key and a separate bucket for the change command. After that, your client will look to any server like the most polite guest — the one that never gets turned away.