Tuesday, May 27, 2025

Fix: Large file upload tests in a Rails app start failing after bumping rack to 3.0.16

Problem: After bumping the Rack gem in a Rails app from 3.0.14 to 3.0.16, a couple of tests which tested an HTTP POST of a large-ish (~30 MB) PDF file to a controller started failing. Other similar tests, which did not upload a large file, continued passing.

The failure symptom was the POST returning an HTTP 400 Bad Request error. The error was returned before the request made it to any of our application code (as evidenced by none of our breakpoints being hit, or debug statements appearing in the output log).  

For our particular test, the specific error message was: Minitest::Assertion: Expected response to be a <413: payload_too_large>, but was a <400: Bad Request>

Fortunately, there were no other gem dependencies that came with the bump of Rack to 3.0.16. So we knew the problem must be caused by some change in Rack 3.0.15 or 3.0.16. 

Solution: The single item in the rack 3.0.16 release notes did turn out to be the pertinent change:

Security: CVE-2025-46727 Unbounded parameter parsing in Rack::QueryParser can lead to memory exhaustion.

As that linked advisory mentions, a new check is added in rack 3.0.16 that turns away "extremely large" application/x-www-form-urlencoded bodies. From this nist.gov page on CVE-2025-46727, links were available to the fix's specific code changes. From the first linked change, we can see that a "BYTESIZE_LIMIT" of 4194304 (a little over 4 MB) is now imposed on application/x-www-form-urlencoded request bodies. 

Per the mozilla.org page on HTTP POSTapplication/x-www-form-urlencoded is only one possible form Content-Type HTTP header value -- and not the one that is supposed to be used with binary data, such as PDF file uploads! Instead, multipart/form-data should be used.

Our original unit test code was actually not specifying any Content-Type HTTP header value in our test POST request. Changing the test's POST request to additionally send the HTTP header Content-Type: multipart/form-data fixed the problem, and got our test passing once again.

Aside: When I spent a minute during the investigation to ask ChatGPT about this problem -- a POST request failure introduced as of Rack 3.0.16, possibly due to a large request payload -- ChatGPT obligingly hallucinated up a plausible-sounding (but 100% fictional) response about Rack introducing a default limit in POST request payload size, and a corresponding plausible-sounding (but 100% fictional) response about a configuration line to be added to config/applicaiton.rb to override the supposed new behavior. 😂🙄

Friday, May 23, 2025

Yet another use case for LIFO stacks (humor)

This is like what happens while I'm working on a coding project at my job on an unfortunately-often basis. Anyone else? 😂