Introduction

This guide is dedicated to one narrow but very sneaky group of problems: a request either doesn't go through or goes through partially because its headers or cookies have grown too large. On the surface, it looks like a 431 Request Header Fields Too Large error, like nginx's specific 494 Request Header Too Large code, like a vague 400 Request Header Or Cookie Too Large, and in the worst case, it doesn't look like anything at all: some intermediate node silently drops part of the headers, and your application receives a user without a session, without an auth token, or without the cookies it needs.

It's important to immediately separate this from a neighboring topic. In a separate article, we covered codes 407, 502, and 504, which the proxy layer itself returns when establishing a tunnel: proxy auth failed, the upstream didn't come up, the timeout expired. There the problem is with the connection and the proxy itself. Here, the connection is established normally, the tunnel exists, but the data you're trying to send through it doesn't fit into the buffers of one of the nodes in the chain. The cause is always the volume and composition of the headers, and it's treated with completely different tools.

What You'll Get Out of This

  • The ability to precisely measure the header and cookie size of a specific request in just a few minutes.
  • An understanding of exactly which node in the chain (browser or client, proxy, load balancer, web server, application) is rejecting or truncating the data.
  • A ready-made echo endpoint to detect silent truncation without a single error code.
  • Working limit configurations for nginx, Apache, Node.js, HAProxy, Tomcat, and Gunicorn with explanations for each parameter.
  • A cookie reduction strategy so the problem doesn't come back a month later.
  • Monitoring that will alert you before you hit the limit.

Who This Guide Is For

For web developers, sysadmins, scraping and automation specialists, and anyone who works with many accounts through proxies and regularly runs into puzzling rejections. The difficulty level is intermediate: you don't need to be an HTTP protocol expert, but basic terminal skills and an understanding of what a config file is will be needed.

What You Should Know Beforehand

  • How to open a terminal and run a command.
  • How to open developer tools in your browser (F12 key).
  • Where your web server's configs live, if you administer it.
  • A general idea that a request travels from the client through intermediate nodes to the server.

How Much Time It Will Take

Diagnostics in steps 1-4 takes 30 to 60 minutes. Fixing things on the server side (steps 5-7) will take another 1-2 hours, accounting for restarts and verification. If you're only checking someone else's service and don't have access to its settings, the first four steps and the cookie reduction section will be enough.

Preliminary Preparation

Nothing exotic is needed, but prepare your tools in advance so you don't get distracted along the way.

Required Tools

  • curl version 7.70 or newer. It's available on macOS, in all Linux distributions, and in Windows 10 and 11 out of the box. Check with curl --version.
  • A browser with developer tools: Chrome, Edge, Firefox, or Yandex Browser. You'll need the Network tab and the Application or Storage tab to view cookies.
  • A text editor with syntax highlighting: VS Code, Notepad++, or the built-in nano on the server.
  • SSH access to the server if you're going to change limits. Read access to logs and the ability to restart services are mandatory.
  • Python 3 or Node.js to run the echo endpoint. Any current version will do.
  • Optional: mitmproxy or the built-in analyzer in an antidetect browser if you want to inspect traffic between the client and the proxy.

System Requirements

Any working machine. For the echo server, 100 MB of RAM is enough. The only real requirement is network access to the service you're testing and, if needed, to the proxy you're using.

What to Download and Set Up

  1. Make sure curl works: run curl -sI https://example.com. You should see response header lines starting with HTTP/2 200 or similar.
  2. If you're working through a proxy, prepare its address, port, login, and password. Test the basic connection with curl -x http://login:password@host:port -sI https://example.com. If you get a 407 error here or the connection doesn't establish at all, go back to the article about proxy-layer codes: our topic only begins after a successful tunnel.
  3. On the server, find the main config: for nginx it's usually /etc/nginx/nginx.conf and the /etc/nginx/conf.d/ folder, for Apache /etc/apache2/apache2.conf or /etc/httpd/conf/httpd.conf.
  4. Find the logs: /var/log/nginx/error.log, /var/log/nginx/access.log, /var/log/apache2/error.log. Make sure you can read them with tail -n 50.

Backups

Before touching any configs, make a copy. It takes seconds and saves hours.

  1. Run sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak-2026.
  2. For the sites folder: sudo cp -r /etc/nginx/conf.d /etc/nginx/conf.d.bak-2026.
  3. For Apache, similarly: sudo cp /etc/apache2/apache2.conf /etc/apache2/apache2.conf.bak-2026.
  4. Write down the current limit values in a separate notes file so you know what you started from.

Warning: never edit a production config without first checking the syntax. For nginx that's sudo nginx -t, for Apache sudo apachectl configtest. A single-line error during a restart will leave the service down.

Basic Concepts

Before we chase down the problem, let's agree on terminology. There's nothing complicated here, but mixing up words often sends diagnostics down the wrong path.

What Request Headers Are

Any HTTP request consists of a start line (method, path, protocol version), a block of headers, and an optional body. Headers are Name: value pairs. The browser adds them automatically: Host, User-Agent, Accept, Accept-Language, Referer, Authorization, and of course Cookie. Applications add their own: tokens, trace IDs, experiment flags.

Why Cookies Are the Most Common Culprit

The Cookie header is assembled from all cookies matching the request's domain and path. Every analytics system, every chat widget, every ad network adds a few. A single cookie's value can be up to 4 KB, and there are dozens of such cookies on a domain. And the browser sends them with every request, including images and fonts. As a result, the Cookie header easily grows past 8-16 KB, which is exactly the default threshold on most servers.

Code 431

431 Request Header Fields Too Large is a standard code described in the HTTP spec. The server is saying: the headers are too large, either one of them or all of them together. Node.js, Go, Envoy, some CDNs, and modern frameworks return this code. A 431 response is protocol-correct and clearly visible in logs, making it the most convenient variant for diagnostics.

Code 494

494 Request Header Too Large is nginx's internal, non-standard code. Nginx uses it internally when a header doesn't fit into the buffers set by the large_client_header_buffers directive. In that case, by default the client gets a 400 with the body Request Header Or Cookie Too Large. You can see the actual number 494 in two cases: if the administrator has explicitly set error_page 494, or if you're reading nginx's internal variables and logs. So if a user complains about a blank page saying 400 and mentioning cookie, know that it's the same 494.

Silent Truncation

The most unpleasant scenario. An intermediate node doesn't reject the request, but truncates a header to the size of its buffer or drops some headers entirely. The application receives incomplete data. The symptoms vary: the user is suddenly logged out, the cart is lost, the token won't parse, the region ID header disappears. There's no error code, and the web server logs are all green. This behavior shows up in hand-rolled proxies with fixed buffers, in old corporate gateways, in some WAFs in soft-processing mode, in client libraries that split or trim long values, and even in the browser itself, which silently ignores Set-Cookie headers longer than 4096 bytes.

Where Limits Actually Live

Picture a chain: browser or script, then your outbound proxy (say, mobile), then a CDN or WAF, then a load balancer, then the web server, then the application server. Every link has its own limit and its own behavior when exceeded. The job of diagnostics is to find the narrowest point. That's why simply raising a limit in nginx often isn't enough: behind it sits Node.js with its own 16 KB threshold.

Boundary with the 502 upstream sent too big header Error

A special case at the intersection of two topics. If nginx writes upstream sent too big header while reading response header from upstream in error.log and returns 502, the issue isn't the tunnel or a crashed upstream, but the response headers from the application that don't fit into proxy_buffer_size. It's formally a 502, but it's fixed by the buffer parameters we're discussing here. Remember this marker: it will help you avoid looking for the problem in the proxy layer.

Step 1: Reproduce the Problem and Record the Symptoms

Goal of this stage: get a stable scenario in which the error occurs, and record the exact signs: code, response text, Server header, and timing.

  1. Open the problematic page in your browser. Press F12, go to the Network tab.
  2. Check Preserve log so entries don't disappear during navigation.
  3. Reload the page with F5. Find the request highlighted in red, or a request with status 400, 431, or 494. Click it.
  4. On the Headers tab, note: the status code, the value of the Server response header (nginx, cloudflare, AkamaiGHost, gunicorn), the value of Via if present, and any headers with an X- or CF- prefix.
  5. Go to the Response tab and copy the response body. The phrase Request Header Or Cookie Too Large points to nginx. The phrase Request Header Fields Too Large without the word Cookie is typical for Node.js. The phrase Bad Request - Request Too Long is characteristic of IIS.
  6. Scroll down to the Request Headers block. Estimate the length of the Cookie line: if it takes multiple screens, that's already a hint.
  7. Open an incognito window and repeat the request. If a clean profile doesn't show the error, the accumulated cookies of that specific user are the culprit.

If the problem reproduces in a script rather than the browser, record the same data from your library's response: status, body, headers. In Python requests that's response.status_code, response.text, and response.headers.

Tip: export all requests to a HAR file by right-clicking the request list and choosing Save all as HAR with content. You can open this file later or share it with a colleague, and it preserves exact header sizes.

Check: you should have a note with four items: the status code, the response body text, the Server header value, and the incognito test result. If the incognito result differs, you already know 70 percent of the cause.

Potential Issues

  • The error doesn't happen every time. That means the cookie size is hovering around the threshold: for example, visiting a specific section adds another cookie. Note which actions trigger the error.
  • There's no red request, but the page misbehaves. That's a sign of silent truncation. Go to step 4 right after measuring.

Step 2: Measure the Real Header and Cookie Size

Goal of this stage: get exact byte numbers: the total header block size, the Cookie header size, and the size of each individual cookie.

Measuring in the Browser

  1. In developer tools, open the Console tab.
  2. Type document.cookie.length and press Enter. You'll see a number: this is the length in characters of all cookies accessible to scripts, excluding HttpOnly ones. The real header will be larger because HttpOnly cookies don't appear here.
  3. Type document.cookie.split('; ').map(c => [c.split('=')[0], c.length]).sort((a, b) => b[1] - a[1]). The console will show an array of name-length pairs sorted by descending length. The heaviest cookies will be at the top.
  4. Go to the Application tab (in Firefox Storage), expand Cookies, select the domain. Here you can see all cookies, including HttpOnly ones, with a Size column. Click the column header to sort by size.
  5. Add up the sizes, or just look at the Cookie line in the request from step 1: in Chrome, hovering over the header shows its length.

Measuring via curl

Curl lets you send headers of any size and pinpoint exactly at what volume the server starts rejecting. Let's generate a cookie of controlled length.

  1. Create a string of 8000 characters. On Linux and macOS: BIG=$(head -c 8000 /dev/zero | tr '\0' 'a'). If your shell complains about escaping, use BIG=$(python3 -c "print('a'*8000)").
  2. Send the request: curl -s -o /dev/null -w '%{http_code}' -H "Cookie: test=$BIG" https://your-site.com/. Only the response code will appear.
  3. Vary the size: 4000, 8000, 12000, 16000, 32000, 64000. Note the value at which the code changes from 200 to 400 or 431.
  4. If you're working through a proxy, add -x http://login:password@host:port and repeat the series. The difference between the with-proxy and without-proxy thresholds will show if the proxy itself has a limit.

Example of a binary search for the threshold in a single bash command:

for n in 4000 8000 12000 16000 24000 32000 64000; do BIG=$(python3 -c "print('a'*$n)"); echo -n "$n: "; curl -s -o /dev/null -w '%{http_code}' -H "Cookie: t=$BIG" https://your-site.com/; echo; done

Windows PowerShell equivalent: $big = 'a' * 16000; curl.exe -s -o NUL -w '%{http_code}' -H "Cookie: t=$big" https://your-site.com/.

Interpreting the Numbers

  • Threshold around 8 KB: typical for Apache with default settings, Tomcat, Gunicorn, and for nginx when a header doesn't fit into a single one of its four 8 KB buffers.
  • Threshold around 16 KB: Node.js starting in version 13, HAProxy with a base tune.bufsize, IIS.
  • Threshold around 32 KB: the total limit of some CDNs. A single header there is often capped at 16 KB.
  • Threshold around 60-64 KB: Envoy and some cloud load balancers.
  • Threshold around 1 MB: the standard Go server.

Tip: don't just measure Cookie. Repeat the series with an Authorization: Bearer $BIG header and with several smaller headers. Some servers limit a single field's size separately, their count separately, and the total volume separately. Nginx, for example, by default won't pass a single header larger than 8 KB even with a small total size, because each line must fit into its own buffer.

Check: your notes should contain the number at which the server starts rejecting, plus a list of the three to five heaviest cookies with their sizes. If you can't find the threshold up to 64 KB but the browser shows an error, then it's not the server rejecting, but something between the browser and the server in your specific setup. Move on to the next step.

Step 3: Determine Which Layer Is Rejecting

Goal of this stage: precisely identify the chain node that returns the error or trims the data, so you change settings in the right place.

Reading the Response Signature

  1. Run curl -sv -H "Cookie: t=$BIG" https://your-site.com/ 2>&1 | grep -i -E 'server|via|cf-|x-served|x-cache|HTTP/', where BIG is known to exceed the threshold.
  2. Read the Server header. A value of nginx with the body Request Header Or Cookie Too Large means nginx rejected it. A value of cloudflare means the request never reached your server. An empty Server and a JSON-format body often point to the application.
  3. Note the protocol version in the first line. If normal requests come back over HTTP/2 but the error arrives over HTTP/1.1, that's a sign the response was formed by an intermediate node, not the endpoint.
  4. Compare the error bodies: the nginx page is terse and includes its name at the bottom; the Cloudflare page includes a Ray ID; Node.js often returns an empty body with status 431.

Bypassing Layers

  1. If you know the server's direct IP behind the CDN, repeat the request with --resolve your-site.com:443:1.2.3.4. This forces curl to connect directly while preserving the correct Host and SNI. If the request goes through directly but not through the CDN, the limit is on the CDN.
  2. If you have server access, run the request locally: curl -s -o /dev/null -w '%{http_code}' -H "Cookie: t=$BIG" -H 'Host: your-site.com' http://127.0.0.1:80/. That excludes all external nodes.
  3. Then hit the application directly, bypassing nginx: curl ... http://127.0.0.1:3000/ for Node.js or the port Gunicorn is listening on. If nginx passes it but the application returns 431, the limit is in the application.
  4. Test the impact of your outbound proxy: one series of requests with -x, one without. For HTTPS requests through a proxy, the CONNECT method is used, and the proxy doesn't see headers inside the encrypted tunnel, so it can't affect their size. But for plain HTTP requests, the proxy parses headers itself, and here its own limit and even truncation are possible.

Reading Logs on the Server

  1. Run sudo tail -f /var/log/nginx/error.log and repeat the problematic request.
  2. A line like client sent too long header line confirms a single header line exceeded nginx's buffer. A line like client sent too large request indicates the total volume overflowed.
  3. For Apache, search error.log for Request header exceeds LimitRequestFieldSize or Size of a request header field exceeds server limit.
  4. For Node.js, look in application logs for HPE_HEADER_OVERFLOW, the internal parser code that surfaces externally as 431.
  5. In nginx access.log, check the field with the request size. If you use the standard format, add the $request_length variable to log_format: it shows the full request size including headers and will be useful for monitoring.

Warning: if you see upstream sent too big header in nginx error.log, it's not the incoming request being rejected but your application's response, whose headers don't fit into nginx's buffer. This case looks like 502 but belongs to our topic. The cause is usually a gigantic Set-Cookie or dozens of headers from the framework. The fix is in step 6.

Check: you should confidently say one of: the CDN rejects it; nginx rejects it; Apache rejects it; the application rejects it; my own proxy rejects it. If the picture is ambiguous, then multiple nodes are over the limit, and you'll have to fix them all one by one, re-running the step 2 test each time.

Step 4: Detect Silent Truncation with an Echo Endpoint

Goal of this stage: confirm that headers reach the application exactly as sent and catch any node that silently modifies them.

The idea is simple. We'll spin up a tiny server that returns all received headers and their sizes. We'll send a header of known length with a checksum and compare what arrived.

Running an Echo Server in Python

  1. Create a file called echo.py with the following contents:
import hashlib, json
from http.server import BaseHTTPRequestHandler, HTTPServer

class H(BaseHTTPRequestHandler):
def do_GET(self):
out = {}
total = 0
for k, v in self.headers.items():
total += len(k) + len(v) + 4
out[k] = {"len": len(v), "sha": hashlib.sha256(v.encode()).hexdigest()[:12]}
out["_total"] = total
out["_count"] = len(self.headers)
body = json.dumps(out, indent=1).encode()
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)

HTTPServer(("0.0.0.0", 8089), H).serve_forever()
  1. Run it: python3 echo.py. The server listens on port 8089.
  2. Deploy it where your real application runs, or temporarily route a location in nginx to it: location /__echo { proxy_pass http://127.0.0.1:8089; }. Then the request will go through the whole real chain.

Note that Python's standard HTTPServer doesn't impose a strict header length limit, but it caps the header count at a hundred. That's enough for the test.

Sending the Control Request

  1. Generate a value of 12000 characters, but not a uniform one — with position markers so you can see where the cut happened: V=$(python3 -c "print(''.join(str(i%10) for i in range(12000)))").
  2. Compute the reference hash: python3 -c "import hashlib,sys; print(hashlib.sha256(sys.argv[1].encode()).hexdigest()[:12])" "$V".
  3. Send it through the whole chain: curl -s -H "Cookie: probe=$V" -H "X-Probe-Len: 12000" https://your-site.com/__echo.
  4. In the response, compare the len field for Cookie with the number 12006 (the name probe plus the equals sign plus 12000 characters) and compare the hash with the reference.
  5. Repeat with several headers: add 30 headers X-Test-1 ... X-Test-30 and check the _count field.

How to Read the Result

  • Length matches, hash matches: no truncation on this path.
  • Length is shorter, the header is cut off: someone chopped off the tail. Look at the last received character: it'll hint at the cut position, often a round boundary of 4096 or 8192 bytes.
  • The header is missing entirely: a node dropped a field that didn't fit its buffer. Some WAFs and old gateways behave this way.
  • _count is less than what you sent: the header count limit was hit. Apache and Gunicorn accept 100 fields by default, HAProxy accepts 101, and some implementations silently drop the extras.
  • Length matches but the hash doesn't: a node altered the content. This happens with whitespace normalization or re-encoding; check whether the value contains characters the proxy considers invalid.

Tip: if you're working through a mobile or other external proxy, host the echo server on a publicly accessible host and run the requests through the proxy over plain HTTP without TLS. In that mode the proxy parses the headers itself and may reveal its own limits. In HTTPS mode through CONNECT, the contents are opaque to the proxy, and truncation at that layer is physically impossible.

Check: you have a table: node, bytes sent, bytes received, hash match. If all rows match, there's no silent truncation and your symptoms are explained by the explicit rejections from steps 1-3. If there's a discrepancy, you know exactly where.

A Separate Browser Check for Set-Cookie

The browser also trims silently, but not the request — the response. If the server sends a Set-Cookie longer than roughly 4096 bytes, Chrome and Firefox simply ignore it without a single warning. Check it: on the Network tab, find the response with Set-Cookie, check the line length, then open the Application tab and confirm the cookie was saved. If it's missing, split the value into parts or move the data server-side.

Step 5: Reduce Cookie and Header Volume

Goal of this stage: eliminate the root cause so requests fit into standard limits and don't depend on the mercy of intermediate nodes.

Raising limits is tempting, but it treats the symptom. Every extra kilobyte of cookie goes out with every request from every user, including requests for images. Start by cutting back.

Cookie Audit

  1. Take the list from step 2, sorted by size.
  2. For each cookie, answer three questions: who sets it, does the server need it, and is it needed on every path of the site.
  3. Cookies from analytics systems and ad pixels are almost never needed by the server. They don't need to be deleted, but they shouldn't be sent to the API domain.
  4. Cookies with serialized state (a cart in JSON, UI settings, browsing history) are candidates for moving to localStorage or server-side storage with a short ID.

Reduction Techniques

  1. Move the API to a separate subdomain. Cookies set on www.site.com without an explicit Domain are not sent to api.site.com. Heavy marketing cookies stay on the storefront, and the API only receives the session.
  2. Set the Path attribute. A cookie with Path=/admin won't go out in requests to /static. Check that your frameworks don't set all cookies at the root unnecessarily.
  3. Store an ID in the cookie, not the data. Instead of a 3 KB JSON, put a 32-character key there and keep the data in Redis or a database. This also solves the tampering problem.
  4. Shorten JWTs. A token in a cookie or Authorization bloats because of extra payload fields: the role list, address, avatar. Keep the ID and expiry, and fetch the rest with a separate call. A typical JWT should fit in 500-800 bytes.
  5. Delete obsolete cookies explicitly. If you renamed a cookie, the old one stays with users until it expires. Send a Set-Cookie with the old name and Max-Age=0.
  6. Use a cookie-free domain for static assets. Classic technique: static.site.com or a CDN domain that never sets cookies.
  7. Check for duplication. Often the same cookie exists in two variants: for site.com and for .site.com with a leading dot. The browser sends both.

For Scripts and Automation

If you manage many accounts through proxies and antidetect browsers, watch your profiles. A profile alive for six months accumulates cookies from dozens of third-party domains, and requests to the target site start failing precisely because of a bloated header. Set up periodic cleanup of third-party domain cookies without touching the target site's auth cookies. In libraries like requests or httpx, store cookies in a separate CookieJar per account and purge expired entries once a day.

Tip: add a simple pre-flight check on the client side: if document.cookie.length exceeds 6000 characters, send a metric to your analytics system. That way you'll hear about the problem from the first affected users, not from mass complaints.

Check: repeat the measurement from step 2 on a clean profile, walk through a typical user scenario (login, catalog, cart, checkout), and confirm that the total header size doesn't exceed 4 KB on the API and 8 KB on the storefront. That's a comfortable margin against any standard limits.

Step 6: Configure Limits on the Web Server, Load Balancer, and Application

Goal of this stage: establish consistent limits on all the nodes you control, so that no node is unexpectedly narrower than its neighbors.

The golden rule: an outer node's limit should be no larger than an inner node's limit. Otherwise nginx will pass 32 KB, and Node.js behind it will reject with 431, and you'll get an error from deep in the stack that's harder to diagnose. A reasonable 2026 target: 16 KB for the whole header block and 8-16 KB per individual field. That's enough for almost everyone, while still guarding against abuse.

nginx

  1. Open /etc/nginx/nginx.conf. Find the http { ... } block.
  2. Add or modify three directives:
http {
client_header_buffer_size 4k;
large_client_header_buffers 4 16k;
proxy_buffer_size 16k;
proxy_buffers 8 16k;
}
  1. client_header_buffer_size is the small initial buffer for typical requests. The default 1k value makes nginx switch to large buffers almost always, which isn't terrible, but 4k cuts out the extra allocations.
  2. large_client_header_buffers 4 16k means: up to four 16 KB buffers. Each header line must fit entirely into a single buffer, and the whole block must fit into the sum of buffers. That means a single cookie or Authorization can't exceed 16 KB, and all headers combined can't exceed 64 KB.
  3. proxy_buffer_size handles the first portion of the upstream response, including its headers. Increasing it is what eliminates the 502 with the phrase upstream sent too big header. The proxy_buffers value must be no smaller than proxy_buffer_size.
  4. If you want the client to see a meaningful response instead of the standard 400 page, add inside server: error_page 494 =431 /431.html; and place a 431.html file with clear text and advice to clear cookies. This turns the non-standard 494 into a standard 431.
  5. Check syntax: sudo nginx -t. You should see syntax is ok and test is successful.
  6. Apply without dropping connections: sudo nginx -s reload.

Note that large_client_header_buffers can also be set in the server context, but for HTTP/2 the value from the http context is what applies, so edit there. For HTTP/2 there's also a http2_max_header_size directive in older versions; it was removed in current builds, and large_client_header_buffers is enough.

Apache

  1. Open the main config and add at the server or VirtualHost level:
LimitRequestFieldSize 16384
LimitRequestFields 150
LimitRequestLine 16384
  1. LimitRequestFieldSize limits a single field, 8190 bytes by default. LimitRequestFields limits the field count, 100 by default. LimitRequestLine limits the start line, which matters for long URLs with query strings.
  2. If Apache works as a reverse proxy via mod_proxy_http, add ProxyIOBufferSize 32768 for responses with large headers.
  3. Verify: sudo apachectl configtest, then sudo systemctl reload apache2.

Node.js

  1. The limit is set when the process starts via the --max-http-header-size flag. The value is in bytes: node --max-http-header-size=32768 server.js.
  2. If you use PM2 or systemd, set the flag in node_args or in the environment variable NODE_OPTIONS=--max-http-header-size=32768.
  3. Alternative in code: http.createServer({ maxHeaderSize: 32768 }, app). This parameter is supported in current Node versions and lets you set the limit without restarting with new flags.
  4. After restart, verify with a series of curl requests straight to the Node port: the threshold should shift.

Tomcat and Spring Boot

  1. In server.xml, add the attribute maxHttpHeaderSize="16384" to the Connector element. The default is 8192.
  2. In Spring Boot, a line in application.properties is enough: server.max-http-request-header-size=16KB. The old property name server.max-http-header-size also works in some versions but is marked deprecated.
  3. Restart the application and verify the threshold.

Gunicorn and Uvicorn

  1. For Gunicorn, add to the config or command line: --limit-request-field_size 16380 --limit-request-fields 150 --limit-request-line 8190. Note the underscore in field_size: it's a historical artifact, and with a hyphen the parameter won't work.
  2. A value of 0 for limit-request-field_size removes the limit entirely, but don't do that on public servers.
  3. Uvicorn limits header size via the h11 parser and permits fairly large values by default; the bottleneck is usually in front of it, in nginx.

HAProxy

  1. In the global section: tune.bufsize 32768 and tune.http.maxhdr 150.
  2. The whole request including all headers must fit into one buffer of size tune.bufsize minus the tune.maxrewrite reserve. With the default 16384, the actual available space is around 15 KB.
  3. Verify with haproxy -c -f /etc/haproxy/haproxy.cfg and restart. Keep in mind that increasing bufsize raises memory consumption per connection.

IIS

  1. Limits live in the registry under HTTP.sys: the MaxFieldLength and MaxRequestBytes parameters in the key HKLM SYSTEM CurrentControlSet Services HTTP Parameters. The default is 16384 bytes for both, with a maximum of 65534.
  2. After the change, you need to restart the HTTP service or the server.
  3. Additionally check maxAllowedContentLength and maxQueryString in requestFiltering if the problem involves long URLs.

CDNs and WAFs

Most cloud CDNs have fixed limits on base plans that only change on enterprise tiers. If the threshold from step 3 hits the CDN, the only universal path is cookie reduction from step 5. Additionally, check whether your WAF is in a mode where oversized headers are stripped rather than blocked: that mode creates the very silent truncation we discussed, and it's better to switch it to an explicit rejection with a code.

Warning: don't set limits in the megabytes for peace of mind. Every kilobyte of buffer is multiplied by the number of concurrent connections, and an attacker can deliberately send gigantic headers to exhaust memory. Keep a reasonable margin of two to three times the real maximum, no more.

Check: repeat the curl series from step 2 for each node separately (directly to the application, through nginx, externally through the CDN). The thresholds should either match or decrease from the inner to the outer node. Make sure that when a node is over the limit, it returns a clear code rather than dropping the connection.

Step 7: Make Rejections Explicit and Set Up Monitoring

Goal of this stage: eliminate silent truncation in the future and learn about header growth before it hits the limit.

Explicit Responses Instead of Silence

  1. Make sure each of your nodes returns 431 or at least 400 with a message when the limit is exceeded. In nginx, that's already done in step 6 via error_page 494.
  2. If you have a hand-rolled proxy or gateway with a fixed buffer, add a check: if a header doesn't fit, respond with 431 and close the connection. Never pass a truncated value on.
  3. In the application, add validation of critical headers. For example, if a JWT won't parse or a session cookie is shorter than its expected length, log it as a separate event tagged possible truncation. This is a cheap detector of problems on any external node.

Logging Sizes

  1. In nginx, extend the log format. In the http block:
log_format withsize '$remote_addr - $status "$request" req_len=$request_length cookie_len=$cookie_len';
map $http_cookie $cookie_len { default 0; ~^(?<c>.*)$ $c; }
  1. A simple and reliable option without map: just log $request_length. That variable is enough to see the growth.
  2. Once a day, run awk over access.log to find the maximum and 99th percentile of req_len. If the 99th percentile exceeds half the limit, it's time to trim cookies.
  3. For Apache, use the %I variable from mod_logio in LogFormat: it shows incoming bytes including headers.

Alerts

  1. Set up a metric for the number of 431 and 400 responses with the cookie-related body. In Prometheus with nginx-exporter, it's statuses from access.log; cloud monitoring dashboards usually have a code breakdown.
  2. Set a threshold: more than 10 such responses in 5 minutes is a signal. A mass 431 error almost always means someone rolled out a new heavy cookie.
  3. Add a synthetic check: every hour a script sends a request with a header at 90 percent of the limit and verifies it passes, then with a header at 110 percent and verifies that it returns exactly 431. That way you'll catch both accidental limit tightening and accidental limit removal.

Tip: keep the echo endpoint from step 4 permanently, locked down by IP or simple auth. It'll come in handy every time your infrastructure changes: added a WAF, switched load balancers, connected a new proxy pool — and in a minute you can confirm the headers arrive intact.

Check: send a deliberately large request and confirm it landed in the log with the correct size, that the 431 counter incremented, and that the alert fired when you pushed the threshold. Then restore your working thresholds.

Verifying the Result

Walk through the checklist. Every item should pass, otherwise go back to the corresponding step.

Checklist

  • The problematic page from step 1 loads without errors in your main browser profile.
  • The total header size of a typical request doesn't exceed 8 KB, and 4 KB on the API domain.
  • No single cookie exceeds 2 KB, and the heaviest third-party cookies aren't sent to the API.
  • The rejection threshold on each node is known and recorded; limits decrease or match from the application to the outer edge.
  • When the limit is exceeded, the client gets a 431 or 400 with a message, not a dropped connection or a silently trimmed request.
  • The echo endpoint shows matching length and hash when sending through the whole chain, including through your proxies.
  • access.log includes the request size, and an alert is configured for 431 growth.
  • Config backups are sitting nearby, and you know the rollback command.

How to Test End-to-End

  1. Clear cookies for the test domain and walk through a user scenario from login to the key action. There should be no errors.
  2. Run the curl loop from step 2 externally. Record the threshold; it should match your new settings.
  3. Send a control request to the echo endpoint through each path you use: directly, through the CDN, through the proxy pool.
  4. Deliberately send a header twice the limit and confirm you get a 431 with a clear body.
  5. Confirm that application responses with large Set-Cookie headers don't trigger 502s in nginx.

Success Indicators

The share of 431 and 400 responses with header-related bodies trends toward zero in normal traffic. The 99th percentile of request size is less than half of the narrowest limit. Unexplained logouts and lost sessions no longer show up in user reports.

Common Mistakes and Solutions

Raised the Limit in nginx but 431 Persists

Cause: Node.js or Tomcat sits behind nginx with its own 8-16 KB limit, and now it's the one rejecting. Solution: check the threshold directly on the application port per step 3 and raise it with the --max-http-header-size flag or maxHttpHeaderSize. Then make sure the nginx limit is no larger than the application's.

Set large_client_header_buffers in the server Block but Nothing Changed

Cause: for HTTP/2, nginx uses the value from the http context, and the directive doesn't inherit the way you expect if set in multiple server blocks sharing an address. Solution: move the directive to the http block and reload the config.

The Error Appears Only for Some Users

Cause: cookie size depends on each specific user's history: which sections they visited, which experiments they're in, old cookies from previous site versions. Solution: gather size stats via $request_length, find which cookies are growing, and explicitly delete stale ones via Set-Cookie with Max-Age=0.

After Clearing Cookies It Works, But Breaks Again a Week Later

Cause: the growth source isn't eliminated: for example, an analytics script appends data to one cookie on every visit. Solution: find the cookie that grows by sorting by size on different days, and move its contents to localStorage or limit it on the script vendor's side.

Nginx Returns 502 Even Though the Application Is Alive

Cause: the error.log has upstream sent too big header: the application's response headers didn't fit into proxy_buffer_size. Usually a huge Set-Cookie or a pile of debug headers is to blame. Solution: raise proxy_buffer_size to 16k and proxy_buffers to 8 16k, and cut down the response headers in the application.

Through a Proxy, an HTTP Request Loses Some Headers but HTTPS Works Fine

Cause: for HTTP, the proxy parses headers itself and hits its own buffer, while for HTTPS through CONNECT, the contents are opaque and pass through untouched. Solution: switch to HTTPS for the target resource or reduce headers. If the proxy is yours, raise its limit and configure an explicit rejection instead of truncation.

The Browser Doesn't Save the Cookie After the Server Responds

Cause: the Set-Cookie line is longer than 4096 bytes, or the total cookie count on the domain exceeded the browser's limit (about 180 in Chrome), and it silently dropped the new one or evicted an old one. Solution: store only an ID in the cookie and keep the data server-side; don't create dozens of tiny cookies, merge them.

Gunicorn Ignores the Limit Parameter

Cause: the parameter is written with a hyphen as limit-request-field-size, while the correct name has an underscore: limit-request-field_size. Solution: fix the spelling and restart the workers.

Additional Possibilities

Different Limits for Different Paths

You don't have to raise the limit for the entire site. If heavy headers are needed only on one path, say for file uploads with long metadata, in nginx you can move that path to a separate server on a separate name or port with larger large_client_header_buffers, while keeping the main site strict. That way you preserve abuse protection where it matters more.

Header Compression in HTTP/2 and HTTP/3

HTTP/2 and HTTP/3 compress headers with HPACK and QPACK algorithms. Repeated cookies travel compactly over the wire, but on the server they're decompressed to full size, and limits apply to the decompressed data. So enabling HTTP/2 saves traffic but doesn't save you from 431. The SETTINGS_MAX_HEADER_LIST_SIZE parameter that the server advertises to the client sets the upper bound of the decompressed list; in nginx it's computed automatically from large_client_header_buffers.

Chunked Cookies

Some auth libraries can split a long value into several cookies with .0, .1, .2 suffixes and reassemble it on the server. That solves the 4 KB per-cookie browser limit but doesn't reduce the total Cookie header size. Use this trick only as a temporary measure while migrating to server-side session storage.

Server-Side Sessions Instead of Cookie Containers

The most reliable architectural solution: only a random 32-64 character ID remains in the cookie, and all state lives in Redis with a TTL. The Cookie header then rarely exceeds 200 bytes, and the whole topic of limits becomes irrelevant. Migration takes from one day for a small app to a few weeks for a large one, but it pays off in reduced traffic and simplified security.

Automated Checks in CI

Add a test to your pipeline: start the application, send a request with headers at 90 percent of the declared limit and check for code 200, then at 120 percent and check for code 431. The test takes seconds and prevents accidentally narrowing the limit when a framework update changes default values.

Working with a Proxy Pool

If you use multiple proxy servers or a mobile pool with rotation, run the control request to the echo endpoint through each exit node over HTTP and HTTPS. Save the results in a table. That way you'll find exits with non-standard behavior and can either exclude them or route only HTTPS traffic through them, where truncation is impossible.

FAQ

What's the Difference Between 431 and 494, If Both Are Basically About Large Headers?

431 is a standard code from the HTTP spec, understood by all clients and monitoring tools. 494 is nginx's internal code, which by default becomes a 400 with the body Request Header Or Cookie Too Large externally. For the user there's no difference, but for diagnostics 494 in logs or in error_page unambiguously points to nginx as the source.

Can I Just Set a Huge Limit and Forget About It?

Technically yes, but that raises memory consumption per connection and makes attacks with gigantic headers easier. Besides, you don't control CDN and browser limits, and the problem just moves there. The right order: reduce cookies first, then set a moderate limit with margin.

Why Does Incognito Work but Regular Mode Shows the Error?

Incognito has no accumulated cookies. That's almost a 100 percent sign the cause is the Cookie header size for a specific user. Clearing cookies for the domain fixes it once, but step 5 fixes it forever.

How Do I Measure Header Size If I Don't Have Server Access?

Use developer tools: the Application tab shows each cookie's size, and the document.cookie.length command in the console gives the total length of script-accessible cookies. For the exact rejection threshold, send curl with an artificially large header and check the response code.

Can My Proxy Truncate Headers in HTTPS Requests?

No. With HTTPS through a proxy, the CONNECT method is used, and the proxy passes an encrypted byte stream without seeing its contents. It physically cannot truncate or modify headers inside it. But for plain HTTP requests, the proxy parses headers itself, and here both limits and truncation are possible.

How Do I Know a Header Was Truncated If There's No Error?

Spin up the echo endpoint from step 4, send a header of known length with position markers, and compare the length and hash of what arrived with what you sent. A mismatch reveals both the fact of truncation and the cut position.

Nginx Passes It, but the Node.js Application Returns 431. What Do I Do?

Raise the Node.js limit via the --max-http-header-size flag or the maxHeaderSize parameter in createServer, then bring the nginx limit to the same or a lower value so rejections happen at the outer layer with a clear page.

What's More Important to Limit: Single Header Size or Total Size?

Both. Total size protects memory, and the per-field limit protects parsers. In nginx, both limits derive from large_client_header_buffers: the buffer size is the per-line limit, and the buffer count multiplied by the size is the total limit.

Should I Intercept 431 on the Client and Auto-Clear Cookies?

For your own site, this is a great trick: on receiving a 431 or a 400 with the right body, the frontend deletes clearly optional cookies and retries the request. Don't delete auth cookies without the user's consent, and always log such events so you can fix the root cause.

How Do I Make Sure a New Framework Version Didn't Change the Default Limit?

Keep the synthetic check from step 7 and the CI test from the additional possibilities section. Both send requests at the limit boundary and will immediately show if the threshold moved in either direction.

Conclusion

Let's summarize what you've done. You learned to reproduce the error and distinguish 431, 494, and its public 400 disguise by the response signature. You measured the real header and cookie size and found the rejection threshold via binary search with curl. You identified which chain node is responsible for the rejection and separated this problem from proxy-layer codes like 407, 502, and 504, keeping the useful upstream sent too big header marker in your toolkit. With the echo endpoint, you caught or ruled out silent truncation that leaves no trace in logs. Then you reduced cookies, moved the API to a clean domain, aligned limits across nginx, Apache, Node.js, Tomcat, Gunicorn, and HAProxy, made rejections explicit, and set up monitoring.

What to do next. Over the next week, keep an eye on the request size metric and the 431 counter: they'll show whether a source of cookie growth is still lurking. Establish a team rule: every new cookie goes through review with questions about size, domain, and path. If your project uses serialized state in cookies, schedule a migration to server-side storage with a short ID.

Where to grow. The next level is a deep understanding of header compression in HTTP/2 and HTTP/3, fine-tuning buffers under load, and building transparent observability where every request's size is visible in tracing. It's also useful to study how limits behave across different types of proxies and load balancers and to build your own threshold table for the infrastructure you work with. The tools assembled in this guide, including the echo endpoint and the curl loop, will stay with you and save hours on every next puzzling rejection. You did it, and now this category of problems has stopped being a mystery.